A growing number of healthcare facilities are (re)organising based on the Value-Based Healthcare (VBHC) principles. This model offers a more holistic approach to patient care than the traditional fee-for-service approach, defining patient value as a function of outcomes per treatment costs. Yet ‘value‘ remains an ambiguous concept. For my master’s research, I tried to pinpoint its meaning in relation to medical technology – specifically, artificial intelligence (AI) aiding radiology departments.
Radiology is an indispensable component of VBHC. Medical imaging is often the start of a diagnostic and treatment pathway; thus, radiologists significantly influence follow-up decisions and patient outcomes. At the same time, they are under increasing pressure due to workforce shortages and growing demand for scans. AI is expected to turn the tide if its value can be proven.
Defining and measuring the added value of any early-stage technology is challenging. While more work is needed to reach a consensus on the subject, it’s worth reflecting on the end-users’ thoughts. In this article, I zoom in on the interviews I conducted with radiologists to understand what the ‘value’ of AI is to them. (Some of the interviewees agreed to be mentioned in this article while a few did not.)
A clear need for AI in radiology
The foundation of my research was the consensus around the need for AI in radiology. All interviewed radiologists felt that AI is necessary to compensate for the increase in medical images they need to assess. Without AI, the demand for scans would outgrow the human ability to review them timely and accurately. In the words of an interviewee:
“The number of images of a CT is constantly increasing. They have already grown to almost 1,200 images per scan. Therefore, AI will help maintain the quality and assure you that you won’t miss anything.” (Dr Raymond van Os, Canisius Wilhelmina hospital)
To investigate the perceived value of AI, I approached radiologists from Santeon and mProve hospitals in the Netherlands, because they are actively trying to integrate VBHC. I discussed their experience and expectations with AI solutions, specifically Aidence’s Veye Lung Nodules.
(For a more detailed and personal insight into the impact of the high workload on radiologists, read this popular article by Lizzie, our Medical Director and a former radiologist.)
The human-AI complementarity
The physicians I spoke to explained that AI could fulfil their need for a second pair of eyes when reviewing the currently high volume of medical images. Some of them compared AI tools to an assistant or a radiology trainee.
Most respondents considered AI tools complementary to their work and particularly valuable when supporting relatively simple but high-volume repetitive tasks, e.g. lung nodule management. They appreciated the homogenous and precise measurements an AI solution can provide without being susceptible to human limitations such as fatigue or distraction. One of the physicians using Veye Lung Nodules said:
“It’s nice that someone, or in this case a computer, is concurrently assessing the scan. I can say that I have gradually become dependent on Veye Lung Nodules and use it with every chest scan.” (Dr. H.W. Wouter van Es, St Antonius hospital)
Going into the experience of an AI system in radiology, all respondents agreed on the importance of ease of use. For AI to improve their workflow, it must be fully integrated into their current PACS infrastructure and not require them to open a new application to view the results. In straightforward terms, a clinician said:
“It [AI] must be easy to use. If it works pleasantly, you tend to use it more often.” (Dr Marieke Vermaat, Canisius Wilhelmina hospital)
Improved decision-making
The radiologists using Veye Lung Nodules or other AI tools indicated that the value also lies in the support for diagnostic or follow-up decisions. For instance, Veye Lung Nodules can detect small nodules they may have missed if unaided or provide reassurance when no nodules are found. One of the radiologists commented:
“It gives me a more relaxed feeling when Veye has analysed the scan with no nodules. If no nodules are found, you start assessing with a whole different mindset. Of course, you always check the scan for yourself, but when Veye says ‘no nodules’, then I think ‘, ah, nice, no nodules, so let’s have a quick look.” (Dr M. F. Boomsma, Isala hospital)
Most respondents felt that the AI tool they were using could improve patient outcomes:
“Of course, it can affect treatment. If the primary tumour is on the left and Veye Lung Nodules sees a metastasis on the right that I missed, you start talking about a whole different stage of surgery.” (Dr. H.W. Wouter van Es, St Antonius hospital)
The impact of AI on radiology decision-making is highly relevant yet barely studied. We will soon begin research into the topic as part of our AI in Health and Care Award. For more on this research, have a look at this news item.
The influence of the environment
Another key finding was the influence of radiology colleagues and the hospital culture on the radiologists’ attitudes towards AI solutions. Adopting AI is a group rather than an individual decision, and radiologists emphasise that they always discuss a product before considering it. For fellows and residents, their supervisors mentioning an AI tool was an incentive to use it.
One of the non-thoracic radiologists mentioned that they did not use an available AI solution because colleagues who had tried it out did not praise it. On the other hand, hearing of colleagues’ positive experiences aroused interest in the tools.
“If a colleague thoracic radiologist does not see the added value of AI right now, then it’s very clear for me. You always discuss everything within your department and with the radiologists of that specific field.”
A hospital’s culture also influences the deployment of AI tools. Radiologists working in hospitals that aim to be on the frontline of innovation were more eager to adopt AI earlier than others. However, AI adoption never happened for the sake of innovation, but after evaluating the benefits of a specific product.
Finally, as paying actors, the government and health insurers also play a part in the implementation of AI in hospitals. They will consider investing in a solution if presented with evidence of a positive business case. But defining return on investment is no easy feat; our co-founder Jeroen discussed it in a well-received article last year.
Sources of reluctance
Despite an overall positive inclination, interviewees were aware that AI is imperfect and will sometimes make mistakes, e.g. detect false positives (lesions that are not actionable lung nodules). Good to note on this matter is that the negative predictive value (i.e. AI indicating that no pathologies are found) was perceived as a positive, reassuring feature:
“It’s not always a matter of detecting cancer. I also think it’s very important to rule things out with more confidence.” (Dr M. Lobbes, Zuyderland hospital)
A second possible source of reluctance was the so-called ‘black box problem’: AI tools do not explain their reasoning. However, some researchers argue that explainability is a ‘false hope’: asking AI for a human-understandable explanation is impossible. The debate is ongoing, and my colleagues will delve into the topic in a future article.
The third and possibly the main barrier to trust in imaging AI was bias – the concern that the result of the AI will be less reliable for patient populations that are underrepresented in the training data. A radiologist explained:
“First, you must gain experience with the tool on your patient population. AI is often based on a particular database, and the limitations are that there are regional differences around the whole world.” (Dr R. A. Niezen, Maasstad hospital)
To better understand how bias in AI works and what we as manufacturers can do about it, I recommend this recent blog post combining clinical and scientific insights.
Conclusions
Although medical imaging AI is still in its infancy, most interviewed radiologists did not foresee any problems using commercially available solutions. Their overall expectation was that AI solutions will ultimately allow them to focus more on tasks requiring human interaction.
Interestingly, radiologists with more than ten years of experience trusted AI more than their less experienced colleagues. They also suggested that medical students may be intimidated by the incorrect perception that AI systems will replace radiologists. Educating the younger generation on AI is crucial for a well-prepared workforce.
Of course, there’s more than one way to assess the value of AI in radiology. Kicky van Leeuwen from Radboud University looked at cost-effectiveness in a robust recent study. She found that AI aiding the detection of intracranial large vessel occlusions can improve healthcare outcomes and save costs.
My journey at Aidence began with the research I outlined in this article, but my learning about AI in radiology continues. So, I hope to add more to this conversation soon.
About Atena
Atena Mahboubian is Clinical Programme Specialist at AidenceAtena always had a heart for healthcare. While finishing her master's in Policy Analysis and Entrepreneurship in Health and Life Sciences, she worked as a radiographer. During her time in the hospital, she understood the value of technological innovation in helping patients receive better care and wanted to be a part of it. Having gained an understanding of the processes within a hospital, she is now applying her practical knowledge to the theoretical issues at stake in her current role. Her hands-on attitude further makes her a good fit for Aidence's scale-up environment.
