Almost all health organizations don’t know what AI they’re using

As more health systems and healthcare organizations are using AI and encouraging their employees to adopt new solutions, they’re running into a problem.

Most hospitals, insurers and technology companies don’t know what AI tools are actually being used in their system.

In fact, 91% of healthcare organizations can’t verify what AI is running in their organization, according to a new survey by Clearwater Security, a cybersecurity firm that works with healthcare organizations. Clearwater surveyed 150 health systems, health plans and tech firms in its 2026 Healthcare AI Benchmark Study.

Baxter Lee, president of Clearwater, tells Chief Healthcare Executive® that he didn’t expect so many respondents to be uncertain about the AI tools being used inside their organizations.

“I was a little surprised by the numbers, but the 91%, at least within our sample set, indicates there's a big gap in terms of really understanding how AI is being deployed within these environments,” Lee says.

Lee says that’s why Clearwater wanted to survey healthcare organizations because they are all dealing with similar challenges.

“AI adoption is happening very rapidly,” Lee says. “Everyone's trying to figure out how to get a handle on it, and they're rushing to implement governance structures, inventories, requirements, policies, etc. So we thought it'd be helpful to really take stock of where everyone really is in that journey and how effective their efforts have been so far, and then set a baseline from which we can then monitor going forward.”

• Read more: What ‘The Pitt’ got right on a hospital ransomware attack

Verifying vendors

Many respondents in the survey (46%) recognize that cyberattacks aimed at third parties represent a threat to their organizations. Cybersecurity experts have warned that many ransomware groups are targeting the vendors used by hospitals and insurers to penetrate their systems. Analysts warn health systems and providers to guard against attacks aimed at third parties.

Yet the survey also found that only 14% of respondents say they are verifying vendors’ safeguards in their use of AI.

“We know third-party risk has been a challenge for the industry for a long time now, and that continues to grow,” Lee says. “And now, with the adoption of AI among those third parties, that only creates a greater concern around third-party risk and how data is being managed within those vendors.”

Health systems face a daunting task in managing how third parties are using AI, since hospitals are dealing with hundreds of vendors.

“I just think it comes back to some of the fundamentals, of having a really good understanding of who your vendor partners are, having a strong vendor risk management program in place that then tiers those vendors based on their criticality,” Lee says.

Health systems need more than assurance from their vendors that they are using AI safely and appropriately, he stresses.

“You really need evidence that they have the controls in place to manage now with this increasing threat from AI,” Lee says, adding, “We really need to understand how vendors are using that, protecting data within their environment. So I think it's really having a strong fundamental program in place that thinks about risk from the top down and then manages it through those vendor relationships.”

• Read more: Cost of breaches: Healthcare still leads all sectors, and AI is used in more attacks

‘Governance has to catch up’

Two-thirds of the survey’s participants said security and privacy concerns represent their biggest hurdle in AI adoption.

Even as organizations are racing to incorporate AI in their systems, Lee says some are seeing the need to slow down and develop sound guardrails and governance over the use of those tools.

“Everybody wants these organizations to benefit from those opportunities,” Lee says. “But I think increasingly, you know, the organizations are recognizing they need to have good security and privacy and governance around their AI adoption.”

About half of the survey’s respondents say their organizations don’t have clear enforcement behind AI policies. Lee says that didn’t surprise him.

“We know that leaders, executives are really pushing the organizations to adopt AI to recognize the benefits of what this advanced technology could bring to their organizations, and now governance has to catch up,” Lee says.

Unsure of spending

Eighty percent of healthcare organizations said they spend less than 5% of their security budget on AI security, or they are unsure how much they spend.

Lee says that result from the survey was unexpected.

“Only 5% of our respondents actually could demonstrate how much they are actually spending, and many of them didn't know how much they're spending at all on security of AI, so I think that was a surprise,” Lee says.

Lee sees value in gauging perspectives from across different facets of the healthcare industry, including hospitals and vendors. He hopes it helps provide insights on similar challenges organizations are seeing as well as the benefits.

“We think it's important to get that broad perspective across all those different participants to really understand,” Lee says. “You know, AI is being deployed across the industry writ large, and so I think it was really important to understand not just from a hospital landscape or ambulatory provider, but how vendors and others are using AI.”