Healthcare diligently seeks guardrails for AI

Although exact figures are lacking, adoption rates are also on the rise. According to some estimates, 70 percent of healthcare organizations are already currently deploying some form of AI. What is clear is that the advent of general purpose AI, also referred to as generative AI or Large Language Models (LLMs), is boosting the adoption of AI in healthcare. Unlike specialized algorithms for specific tasks, LLMs are particularly broadly applicable, which obviously promotes their implementation.

Practical guidelines

This broad applicability makes it difficult to keep track of the precise application, let alone the outcomes. So there is every reason to draw up clear rules for the use of AI in health care. The Coalition for Health AI (CHAI) recently made an attempt to do so with the publication of a substantial handbook containing practical guidelines for the ethical and clinically sound application of AI. These revolve around concepts such as usability, equity, fairness, safety, reliability, transparency, accountability and privacy.

Along with industry?

The fact that these basic principles within CHAI are endorsed by both major hospital chains such as Mayo, Johns Hopkins and Kaiser Permanente and representatives of Big Tech, including Amazon, Google and Microsoft, should give the guidelines additional weight.

But critics say the close collaboration with industry also harbors a problem. Does CHAI dare to expose controversial AI practices? There have been several recent cases in the U.S. of developers collecting data improperly within healthcare organizations. Will CHAI soon choose technical standards over principled goals such as "empowering patients"?

Consumer good

The recently formed Digital Rights and Health Alliance (DRHA) can already see the storm brewing. "With the entry of large technology companies, healthcare is increasingly becoming a commercial consumer good," the alliance said in a manifesto co-signed by EuroHealthNet, European Digital Rights, the European Public Health Alliance, Global Health Alliance and Health Action International and several individual researchers including Dutch researchers Hannah van Kolfschooten (University of Amsterdam) and Jolien van de Sande (Tilburg University). "The business model of these companies is inconsistent with mainstream public values."

"Using patient data for profit places citizens in a vulnerable position and creates an unequal balance of power. Loss of individual control and ownership of data lurks." What's more, according to the signatories, AI continues and reinforces current health disparities. Current regulations fall short in this regard, according to DRHA.

AI Act

What about the AI Act, many will ask? The European Parliament approved the law in March, which contains far-reaching new obligations and requirements for developers of AI. For example, there will be a mandatory risk classification. On this basis, AI applications with higher risks will be subject to stricter requirements. Healthcare applications quickly fall into a high risk category. To enforce compliance with the law, there will be penalties and fines. Even a total ban for use within the EU is a possibility.

Consequences

The problem with AI Act is that the law was not primarily drafted with healthcare in mind, notes Professor Stephen Gilbert of the Else Kröner Fresenius Center for Digital Health in Dresden in Nature. The elaboration and consequences for healthcare are also not yet well crystallized. In particular, the interaction with the already existing Medical Device Regulation (MDR) and the In-Vitro Diagnostic Device Regulation (IVDR) raises questions.

The DRHA welcomes the new rules of the game, but sees an omission. Nowhere are issues such as public interest and public benefits explicitly stated. The alliance therefore believes there should be an evidence-based definition of public interests and benefits so that they are considered in all decisions about AI applications.

Dutch values

Echoes of this argument resound in the recent exploration of the opportunities and dangers of AI in healthcare, commissioned by TNO for the Ministry of Health, Welfare and Sport. TNO speaks of "the need to continue to secure important Dutch values, such as justice, transparency, safety, accountability and sustainability in legislation and policy."

One way to secure those Dutch values is to invest in AI innovations within the Netherlands to reduce dependence on a small number of large, foreign parties.

Up front

This was something the previous administration was keen on, according to the vision on generative AI presented in January. By collaborating with all stakeholders, stimulating knowledge development and use, and developing appropriate laws and regulations "the Netherlands can be at the forefront of the inevitable changes in our society due to generative AI.

AI ethics lab

Good to know that Dutch healthcare has already taken up the gauntlet. Last year, the "Guide to Quality AI in Healthcare" was published. Erasmus MC and TU Delft also boast the world's first healthcare AI ethics lab. In June, together with IT company SAS, they launched the Responsible and Ethical AI in Healthcare Lab (REAiHL). The goal: to create a framework defining the ethical requirements for a healthcare AI model.

"The last thing we want is for an AI model to harm a patient or disadvantage certain groups," said co-initiator and ICU physician Michel van Genderen of Erasmus MC. "As far as I'm concerned, we need to set up the validation of AI the same way we do with medicine. Fixed principles and fixed steps, so that we all develop it in the same explainable way. And with which it can be implemented scalably and responsibly in the clinic."