The five-step guide to AI adoption in clinical practice

The adoption of medical imaging AI is about getting your hospital or screening programme ready to implement the right solution for a clinical need. Running into speed bumps along the way is common for early adopters. How do you define the needs, budget, and outcomes? Which boxes should you check when selecting vendors? How do you manage internal stakeholders? The adoption curve is steep. Luckily, you don’t have to climb it alone.

Drawing from our experience deploying AI in clinical practice and lung cancer screening, I’ve designed a five-step guide to streamlined adoption. If you’re looking to adopt artificial intelligence but don’t know where to start, these actionable tips and advice will see you through. For the video breakdown of the steps, watch this presentation from ECR 2020.

1. Consider

Where do you start working with AI? First, look away from all the solutions out there, and focus on your organisation. Bring together all the stakeholders into a project team that includes the sponsor, if applicable, IT and legal representatives. Involving them from the beginning will expedite the process.

Start with defining the challenge you are looking to solve, or the specific clinical question that is relevant to your workflow. Some hospitals are looking to experiment with the technology, while others aim to solve a particular issue. Over the past years, I have seen the latter getting more out of AI, which is why my advice is to start from a clinical challenge.

When considering this challenge, make sure to already determine your expected outcomes. When is the adoption a success? Are you aiming to have an AI solution in use? Should it apply to a certain patient population, or yield specific results like time or cost savings?

Also, although it may seem early, this is also the stage to organise a budget dedicated to the AI solution. The size of this budget should relate to the cost or time savings a solution is expected to bring. Both the amount secured and its source will impact the next steps. For example, it will guide you to look for PhD researchers versus seeking a vendor that offers a mature solution.

2. Evaluate

The AI in healthcare space is widely populated; a Google search or a look at the list of vendors at the RSNA can confirm that. To weigh the existing options for your scope, do your (desk) research using this high-level checklist for each solution:

• How was the AI solution validated?

It is important that the claim that the AI solution has validated covers the use case you identified in the previous step. Take the time to understand if the manufacturer has done studies confirming this claim.

• How does it integrate into the workflow? 

Try to get a feel of the amount of effort needed to add an AI system into your workflow. A good practice is to start with an AI solution that is easy to integrate with the current workflow and IT infrastructure. Workflow integration is of utmost importance for the radiologist; in this article, we explained why that is and how it works.

• What regulations does the solution fulfil for use in clinical practice? 

Commercializing medical devices requires a CE Mark in the EU and an FDA clearance in the US. Note that local regulations may apply to different countries. Again, pay attention to which claim is covered by the acquired certification.

3. Choose

By this stage, you should have narrowed your search down to a few vendors. This is the moment to go in-depth into the workflow and test if a specific solution is a good fit from both a clinical and a technical standpoint. A well-integrated AI system should not create hurdles for physicians, such as requiring them to leave their workstations to upload studies. It should further blend within the existing IT infrastructure.

There are two checks that are vital to make the right choice:

• Validate the accuracy 

Legitimate vendors would have done a study and can provide a clinical background for accuracy. To know if the solution is good at performing the defined task for your organisation, ask questions about the datasets used to develop and test the AI solution.

There are three datasets required to build an AI model: a training dataset, a validation dataset, and a test dataset.

The test dataset is the most relevant to look at because it is what the accuracy is based on. The performance on this dataset should be applicable to your hospital, with its specific protocols, type or number of scanners, and patients. To achieve this, the test set must cover the patient population your organisation serves (e.g. types of patients, comorbidities distribution, etc.). Thus, inquire about the specifics of the test set and the performance of the AI model on this dataset.

Secondly, you may want to know what the size of the training dataset is and how it was labelled. Both quantity and quality are important to train an accurate AI model. Labelling the data should be done by experienced radiologists, preferably with multiple readers per study.

• Check the regulatory compliance

In Europe, medical device classification is divided up between risk class I, IIa/b, or III. If looking for a solution for clinical practice, be wary of class I medical devices. The new Medical Device Regulation, which will come into force in May 2021, will require many AI products currently classified as class I devices to update their classification. For instance, software that supports diagnostic decisions should fall under class II at a minimum. For more guidance on the new regulation, read our recent expert piece.

Apart from the regulatory approval, check if the vendor also has a quality management certification (e.g. ISO 13485). Reviewing the data processing policy and the cybersecurity measures in place will further help you understand if the AI company is going the extra mile in regard to safety.

A bonus tip for the choosing stage: do a reference check. Ask other organizations how they are working with the AI solution you have chosen. You may get the insights you need to make the final decision.

4. Approve

Approving the chosen solution internally requires the involvement of and coordination between IT and PACS administrators, procurement officers, physicians, often also privacy departments and legal officers. If you have a project team in place since the first step, you should be well on track.

To move forward and avoid delays, assign an internal AI champion responsible for driving the project. This may be an executive sponsor, a budget holder, or a department manager. One of my learnings from past deployments is that the risk of failure is high without a person fulfilling this role. What we have further learned as a vendor is the importance of empowering the AI champion, by providing the necessary information and documentation in a timely manner.

Furthermore, make sure end users are trained to use the new medical device. If they don’t benefit from it, the impact of the AI solution will be limited. Additionally, setting up a feedback mechanism with the AI vendor from the get-go will help improve the AI product.

5. Deploy (& evaluate)

All the paperwork is signed – well done! To make the deployment work, create a clear project plan, including actions, timelines, and owners. Depending on the type of deployment – on-premise or cloud-based – different actions will be needed. As outcomes, set the deployment and acceptance dates, make agreements on the service levels, fixes, and upgrades, and discuss post-market surveillance.

The initial or trial phase of using the AI solution should show if it answers the problem you were trying to solve. It is a good moment to revisit step one and start evaluating the results to decide if you will continue using the solution.

A common question I get at this stage is: “Do I need to do a full clinical study?” The answer fully depends on the purpose of using the product. It is necessary for research, but not for other use cases. What matters is validating that the AI solution is adding value to your clinicians and their patients.

Make it better

AI adoption does not end with deployment. Service and maintenance are essential, and their quality is often a differentiating factor between AI vendors. The implementation process usually acts as a good test for the AI companies fulfilling their promises and being prompt when handling requests.

Beyond these five steps, you and your organisation play a role in improving the chosen AI solution through valuable feedback and feature suggestions. The collaboration between humans and software allows us to achieve much more than humans would on their own. If done right, it can be transformative for patients.

Are you ready to start the AI journey? Get in touch!