We established AI guardrails for our health system. Why you should, too. | Viewpoint

As AI has grown more powerful, it has become the foundation for several promising healthcare applications, and this trend seems poised only to accelerate.

Joseph Evans, MD

A recent study shows three out of four healthcare executives expect artificial intelligence-related technologies to be “widespread” within three years. Still, only about 40% said their organizations reviewed or planned to review their AI regulatory guidance.

While many can afford to experiment with AI, healthcare organizations cannot ignore the risks. As medical professionals who recognize both the potential and dangers of AI, we have chosen to embrace what AI could make possible by establishing clear guardrails.

Embracing the potential

One recent study estimates that AI could save the healthcare industry up to $360 billion per year if adopted more widely.

Yet, organizations and patients are wisely hesitant. Failures and mistakes involving AI in healthcare could further erode public trust in healthcare and undermine the potential benefits of AI for the industry. Possible risks include data privacy and algorithm bias, or even applications of AI that result in patient harm. There are also concerns from healthcare workers that AI is coming for their jobs.

AI poses clear risks in the medical field. For example, an AI-enabled clinical tool could make a dangerous and costly error absent human oversight, such as misdiagnosing a condition or recommending an inappropriate treatment.

A recent study shows the hill we also need to climb to achieve AI acceptance among patients—some 60% would be uncomfortable with their provider relying on AI in their healthcare. However, in the same survey, a larger share of respondents thinks using AI would reduce rather than increase the number of mistakes healthcare providers make.

While we don’t want to rush headlong into something risky, we also don't want to ignore tools that have such potential. How can we navigate a course through these treacherous waters?

No process will be foolproof, but to reap the benefits of AI while doing everything possible to avoid its potential risks, Sentara Health created an AI Oversight Program staffed by senior leaders from our organization.

The committee oversees use cases and development of AI tools across Sentara’s integrated delivery network, which includes 12 hospitals, five free-standing emergency departments, more than 1 million health plan members, and a medical group that completes more than 2.8 million patient visits annually.

Establishing guardrails

David Torgerson, our chief analytics officer, and I chair the AI Oversight Committee. It comprises our chief nursing officer, chief quality officer, chief data officer, ethics and legal representatives, and other highly qualified experts.

Our executive team, who ultimately decide whether to move forward or not with a solution, include our chief clinical officer, chief consumer officer, chief information officer, and chief legal officer.

Our first act as a group was to devise principles reflecting our commitment to innovation, patient safety, and high-quality care and ensuring we develop and use AI solutions safely, responsibly, and in a trustworthy framework. They include:

1. Human oversight: There will always be a “human in the loop” to ensure proper human oversight of AI tools.

2. Technical robustness and safety: AI tools will not negatively impact Sentara’s information technology infrastructure or cybersecurity.

3. Privacy and data governance: AI tools will adhere to Sentara’s rigorous standards for maintaining privacy and protecting health information.

4. Transparency: AI solutions will be as transparent as possible with inputs and the algorithm used to derive the outputs.

5. Non-discrimination and fairness: AI tools will not create or reinforce bias and will not provide variable outputs based on race, ethnicity, etc.

6. Environmental and societal well-being: AI tools will promote environmental and societal well-being. The committee will consider various factors to ensure that we develop broadly beneficial solutions.

7. Accountability: Humans will regularly monitor AI tools to ensure they produce the desired results and do not change or “drift” over time.

8. Benefit: The benefits of AI tools will outweigh the risks.

Unless these principles are met, we do not advance an AI initiative in our organization. By adhering to them in every circumstance, we will ensure any AI tools used in our system comply with legal, regulatory, and ethical considerations while aligning with Sentara’s focus on promoting our consumers’ overall health and well-being.

Lest one assume all this is theoretical, we are moving forward with several AI initiatives that have met this high standard. One recent example is the rollout of a solution that uses AI to help draft clinical notes.

These necessary notes document patient visits but add administrative burden to provider workloads and can detract from one-on-one patient interactions. According to the American Medical Association, physicians spend nearly two hours on documentation and administrative duties for every hour of patient care. They spend an additional hour or two nightly doing computer or clerical work.

With patient permission, our physicians and providers can now use an AI tool to reduce this administrative burden while improving patient-doctor visits. The provider reviews the note, makes edits, and approves.

One physician says the tool helps him be a better physician by making his interactions with patients more productive and satisfying. He adds that his mental energy is saved for more important aspects of his work.

Advances like this highlight AI's promise. By adhering to our principles, we intend to deliver on that promise.