AI in Erasmus MC: from blackbox to transparency

Erasmus MC is a forerunner among hospitals in the development and use of artificial intelligence (AI). Many AI applications are developed and used in-house. By no means everyone knew from each other what they were doing. Rebecca Steketee developed a process that provides insight. Employees with new applications now know better what steps to take.

Even during her research as a neuroscientist, Rebecca Stekestee noticed that AI was increasingly being used in the analysis of MRI scans in the radiology department. The department already had its own research group that designed, trained and tested its own AI models on data. "I saw its value for research, but at the same time it chafed. Even though it was all so promising, no one knew exactly what AI was really doing for the clinic. It was a bit of a black box. We didn't know what was really working and what we were allowed and able to do."

AI Policy

All those questions she decided to bring together in her course in Clinical Informatics at TU/e in her year-long project. She was able to carry out that project at Erasmus MC. The Rotterdam hospital positions itself as the "first technical UMC in the Netherlands" to develop and use AI applications.

"There was actually little understanding of all the developments that were going on," Rebecca Steketee says. "All kinds of things were happening side by side. That's how people felt at the strategic level." So from the Strategic IT Consultation came the assignment to develop AI policies to ensure the quality of AI applications, as well as transparency and privacy.

As part of that assignment, she set to work developing a process for AI algorithms developed in-house and commercial external algorithms. The guiding principles were compliance and efficiency. They had to comply with the laws and regulations that were known for them at the time, as well as requirements for safe use. The broader process of development, validation and use had to ensure that innovation with AI was not inhibited and rather accelerated.

Funnel with practical steps

What she ended up delivering as a year-long project was a funnel that included all the practical steps required from the development of an AI algorithm to compliant AI -use. She used frameworks that already existed, such as the Ministry of Health's Valuable AI for Health program, which aimed to increase the value of AI for healthcare providers, patients and citizens and help implement and scale up AI in healthcare.

"Within that, such a kind of funnel had also already been developed as a tool," she says. "I further developed that funnel for Erasmus MC. What steps do you take? What do you have to think about? What expertise is already in house or needed? What law or rule is there? What ethical considerations are there?"

Benchmark survey

Steketee also conducted a benchmark survey of other healthcare institutions around the country. What are they doing with AI, do they already have a policy for it, and what might that look like in the processes and procedures that were already in place at Erasmus MC in other areas, such as medical software development? Steketee: "Those, of course, partly overlap with how you do for AI. The lawyers and the ethics department were already working on that. I actually overlaid all that. On the one hand the frameworks that were already there, the benchmark from the other houses and what was the starting situation at Erasmus MC at that time."

Clinical validation

The funnel shows the steps involved. From a broad exploration of a problem in the clinic, determining with each other that an AI model could be a possible solution for this problem to developing, testing, validating and finally using that model. Then, if the clinical validation is positive, you can actually start implementing it. "I added two more phases that were fairly underexposed in existing frameworks at the time. First of all, the monitoring of an algorithm after its implementation, in which you also consider scaling it up in-house, or toward other hospitals. And secondly, how and when to decommission an application."

MDR legislation

Steketee then started testing the process in-house. "We inventoried who is working on AI at Erasmus MC and what kind of projects they are. For example, is it for operations, healthcare or logistics? Are they algorithms that have been developed in-house or are they algorithms that have been purchased externally? What phase of the funnel are they in? We really zoomed in on some of those use cases. We did interviews to look at the funnel with those involved and to verify with them how they experienced the development of the use of their AI model. What came back from that was that in the first phase they actually already need an infrastructure where people can experiment in, but in which, for example, the necessary steps are also immediately laid down for the MDR legislation, the Medical Device Regulation."

AI Stewards

Many researchers developing AI for a clinical problem run into a wall of laws and regulations the moment they want to introduce their application into clinical practice, according to Steketee. This is where the funnel can help. Precisely at that stage, they need guidance in the preconditions they must meet for use in the clinic.

That is why Steketee included as one of the recommendations with her year-long project that Erasmus MC appoint AI Stewards. These are colleagues who have knowledge of the whole process in the funnel and can guide AI developers and users to take the right steps. "Especially in the legal and ethical aspects of research, colleagues want support. They want to know at the beginning what they need to consider when going into the clinic with their applications. An AI Steward can help with support in meeting the prerequisites, such as ethical issues and legal issues, so developers and users can really focus on the development of AI and what it does for your patient or for your business operations."

Center of Expertise

When Steketee mapped out the process, Erasmus MC was in the process of setting up an expertise center around the use of AI in healthcare. Whether her year-long project was really an accelerator in the arrival of the expertise center at the Rotterdam hospital, Steketee does not know, "I do think my year-long project provided some starting points where the AI expertise center could focus."

Want to know more? Read the final report of the annual project here