When the algorithm enters the care relationship

Medical ethics is nothing new. The Hippocratic Oath, in its various modern versions, has for centuries laid down the foundations of a medicine practised in the exclusive interest of the patient, with respect for their autonomy, with honesty and discretion. The Code of Medical Ethics, governed in France by the National Council of the Order of Physicians (CNOM), translates these principles into concrete obligations applicable to every practising doctor.

But this code was designed for a doctor who decides, who prescribes, who assumes their responsibility. What happens when an algorithm inserts itself into this process? When an AI suggests a diagnosis, proposes a treatment, or detects an anomaly that the clinician had not noticed? Who is responsible if the algorithm is wrong, and the doctor followed it without question? These questions are not hypothetical. They arise today, concretely, in departments that deploy clinical AI tools.

The cardinal principle: medical responsibility remains human

Let us start with the most fundamental point, the one on which all professional bodies, all regulatory authorities and all ethical texts converge unambiguously: the use of an AI tool in medicine does not transfer the doctor’s responsibility to the machine.

Article 69 of the Code of Medical Ethics is explicit: the doctor cannot delegate their clinical and ethical responsibility to a third party, human or otherwise. They remain the author of the medical decision, even when that decision relies on information produced by an algorithm.

In concrete terms, this means that:

• If an AI system suggests that a pulmonary nodule is benign, and the doctor validates this suggestion without exercising their own clinical judgement, and the nodule turns out to be malignant, it is the doctor whose liability will be engaged for failing to apply critical analysis.
• If an automated prescribing algorithm proposes a dosage, and the doctor validates it without checking its consistency with the patient’s physiological parameters, it is the doctor who answers for any medication error.

AI is a tool. And like any medical tool — a stethoscope, an electrocardiograph, a scanner — it is the professional who uses it who is responsible for the use they make of it.

The duty of professional independence

Article 5 of the Code of Medical Ethics guarantees the doctor’s professional independence. They must practise their art in complete freedom, without allowing themselves to be influenced by considerations external to the patient’s interest.

This principle takes on a new dimension with AI. An algorithm integrated into consultation software that systematically suggests certain additional examinations or certain medications may, if not used with discernment, create a form of cognitive dependence or disguised commercial influence.

Automation bias, the cognitive bias that leads us to trust an automated system rather than our own judgement, is a documented phenomenon in medicine, particularly in anaesthesia and radiology. Highly experienced airline pilots have missed human alarms because an automated system indicated that everything was fine. Experienced clinicians have ignored telling clinical signs because the algorithm had concluded there was a low risk.

Ethics requires the doctor to maintain, in all circumstances, their capacity for independent judgement. Using an AI does not exempt one from thinking: it must nourish clinical reflection, not replace it.

Informed consent in the age of AI

Article 35 of the Code of Medical Ethics requires the doctor to give the patient fair, clear and appropriate information on their state of health and the treatments envisaged.

The integration of AI into the care process raises a new question: must the patient be informed that an algorithm took part in their diagnosis or in the proposed treatment?

The answer from professional bodies and health-law experts is convergent: yes, in many situations, information is owed. Not because AI must be flagged as a danger, but because respect for the patient’s autonomy implies that they understand how the decisions concerning them are made.

This obligation is all the stronger when AI plays a substantial role in the decision, when the patient could legitimately object to it if they were informed, or when the tool uses their personal data for analysis or training purposes.

Medical confidentiality and data protection

Medical confidentiality, enshrined in article 4 of the Code of Medical Ethics, is absolute. It covers “everything that has come to the doctor’s knowledge in the exercise of their profession.” It admits of exceptions only in the cases expressly provided for by law.

Now, using an AI tool whose terms and conditions authorise the processing of entered data for the purposes of improving the model, transferring it to third parties or storing it outside the European Union constitutes a possible breach of medical confidentiality, independently of GDPR considerations.

Ethics is not satisfied by formal legality. A doctor may theoretically be in compliance with the GDPR while failing their ethical obligations if the processing of their patient’s data is not strictly limited to the purpose of care. The Order may be seised independently of the data-protection authorities.

This point is particularly important for self-employed doctors who use mainstream AI tools without having checked their terms of use: professional liability before the Order is individual and cannot be transferred to the software publisher.

Non-maleficence and critical evaluation of tools

The principle of non-maleficence, primum non nocere, is one of the founding pillars of medical ethics. Applied to AI, this principle generates an active obligation of critical evaluation.

Concretely, this means asking several questions before adopting a tool:

• On what population was it validated? An algorithm validated on a young American hospital population may perform very differently on your elderly, multimorbid patients in private practice.
• What is the nature of its errors? A very sensitive algorithm will produce many false positives, leading to unnecessary additional examinations. A very specific algorithm will miss real cases.
• Has it been evaluated under real-world conditions or only under experimental conditions? Performance measured in the laboratory on well-cleaned retrospective data is systematically superior to the performance observed during a real deployment.

Justice and equity: algorithmic bias as an ethical problem

The principle of justice requires treating patients fairly, without discrimination based on origin, sex, age or social condition. AI algorithms can violate this principle invisibly, by reproducing and amplifying the biases present in their training data.

Examples documented in the international medical literature illustrate this risk:

• A chest-pain triage algorithm that was less sensitive for women, because the historical training data under-represented atypical female presentations of heart attack.
• A care-recommendation system that systematically disadvantaged African-American patients, because it used historical care costs as a proxy for the burden of illness — yet these costs reflected unequal access to care rather than real health status.
• A dermatological screening tool that was less effective on dark skin, because the training image databases lacked diversity.

Ethics requires active vigilance on this point, particularly for practitioners who care for diverse or vulnerable populations.

Continuing education: an ethical obligation in the face of AI

Article 11 of the Code of Medical Ethics requires the doctor to maintain their skills and update their knowledge throughout their career. A doctor who uses an AI tool without having received minimal training on its operation, its limits and the risks linked to its use potentially fails this obligation.

The CNOM clearly formulated this in its 2022 report on AI in medicine: doctors have the responsibility to train in the AI tools they use, and establishments have the responsibility to give them the means to do so.

What the National Council of the Order of Physicians says

The CNOM published in 2022 a reference report entitled “Doctors and patients in the world of data, algorithms and artificial intelligence.” This founding document sets out several clear positions:

• AI must remain a tool at the service of the doctor and the patient, without ever substituting for human clinical judgement.
• Transparency towards patients regarding the use of algorithms in their care.
• Rigorous and independent evaluation of AI tools before their clinical deployment.
• Involvement of doctors in the design and evaluation of these tools, to guarantee their real clinical relevance.

For other healthcare professionals: equivalent principles

The Code of Ethics for nurses (2016 decree) requires respect for the patient’s dignity, responsibility in care acts and the obligation of continuing education. The Codes of Ethics for dental surgeons, midwives, physiotherapists and pharmacists all set out principles of responsibility, professional independence and respect for the patient that apply fully to the use of AI.

No regulated healthcare professional can consider that the ethics of their practice stop where the algorithm begins.

AI does not change values, it puts them to the test

Medical ethics was not designed for the era of artificial intelligence. But its founding principles — responsibility, independence, beneficence, non-maleficence, justice, respect for the patient’s autonomy, confidentiality — are precisely those needed to govern an ethical use of AI in healthcare.

A healthcare professional who has integrated the ethical foundations of their practice already has the framework of thought needed to use AI responsibly. What they sometimes lack is the minimal technical knowledge to apply this framework to tools they do not yet understand sufficiently.

This is precisely the role of suitable training: not to replace ethics with technique, but to give healthcare professionals the keys to exercising their ethical responsibility in a rapidly changing technological environment. Discover our training courses.

Sources and references

1. Code of Medical Ethics. Articles R. 4127-1 to R. 4127-112 of the French Public Health Code.
2. National Council of the Order of Physicians (CNOM). “Doctors and patients in the world of data, algorithms and artificial intelligence.” Report, 2022.
3. Parasuraman, R. & Manzey, D. H. (2010). “Complacency and bias in human use of automation: An attentional integration.” Human Factors, 52(3), 381–410. [Automation bias]
4. Goddard, K., Roudsari, A. & Wyatt, J. C. (2012). “Automation bias: a systematic review of frequency, effect mediators, and mitigators.” Journal of the American Medical Informatics Association, 19(1), 121–127.
5. Obermeyer, Z. et al. (2019). “Dissecting racial bias in an algorithm used to manage the health of populations.” Science, 366(6464), 447–453. [Racial bias in a health algorithm]
6. Adamson, A. S. & Smith, A. (2018). “Machine learning and health care disparities in dermatology.” JAMA Dermatology, 154(11), 1247–1248.
7. Article L. 1111-2 of the French Public Health Code — Patient’s right to information.
8. Decree No. 2016-1605 of 25 November 2016 establishing the Code of Ethics for nurses.
9. Regulation (EU) 2024/1689 — Artificial Intelligence Act. Official Journal of the European Union, 2024. [Transparency and human-oversight obligations]
10. Char, D. S., Shah, N. H. & Magnus, D. (2018). “Implementing machine learning in health care — Addressing ethical challenges.” New England Journal of Medicine, 378(11), 981–983.