What AI hesitancy says about healthcare’s real priorities | Viewpoint

At a time of scarce dollars and staffing, the healthcare industry is turning to artificial intelligence as a cure for its many ills. For an industry known for avoiding risks, rushing an AI rollout introduces many: patient safety, PHI exposure, accelerating clinician burnout, and misallocated capital, to name a few.

In 2025, the healthcare sector deployed AI at more than twice the rate of the broader economy, according to a recent report from Menlo Ventures. It notes that the adoption of AI tools, for everything from clinical documentation and decision support to automating key revenue cycle functions, has taken place rapidly, over just the past two years.

That’s despite an incredibly high failure rate; a report from MIT found that 95% of generative AI pilot projects across sectors generated no return on investment.

And perhaps unsurprisingly, a backlash to AI is brewing in healthcare. Citing concerns about patient safety and staffing levels, nurses' unions have organized protests across the country demanding that hospitals slow the pace of AI adoption. Surveys also indicate that patients have concerns, even if they also see potential benefits.

Understanding the risks

While AI undoubtedly has a role in modern healthcare operations, with much unrealized potential, there are both right and wrong ways to introduce it into clinical environments. A well-functioning healthcare system hinges on human expertise and empathy, not simply AI automation.

Meaningful adoption of AI in healthcare begins with understanding the reasons for resistance and addressing them directly.

It also involves understanding the limitations for providers and especially patients, who may find themselves interacting with AI chatbots that trap them in a feedback loop, with no clear way to connect with a real, live human being.

Here are some key considerations when evaluating AI for clinical use.

Clinical staff are shut out of the purchasing process

In an American Medical Association survey, two-thirds of physicians said they used AI in 2024, a 78% increase from the previous year, with many saying the technology’s biggest promise lay in alleviating their administrative burden. But another survey found that 65% of primary care physicians had limited or no involvement in selecting AI tools for their own practices.

Decisions about technology purchases and implementations often fall to executives or other non-clinical staff who prioritize cost and other factors, not clinical or workflow needs. If clinicians are involved at all, they’re brought in only at the final stages, after a vendor has been selected and an operational strategy established.

When clinicians have a voice early and often, they’re more likely to trust AI tools, use them appropriately, and integrate them into everyday workflows. Otherwise, efforts to streamline workflows may mean the bots supplant the physician-patient relationship

Shallow integration and fragmented data

AI models are only as effective as the data they can access. Healthcare’s foundation was weak in terms of integration maturity and data proliferation at the dawn of the AI age.

Superficial EHR integrations, unstructured and poorly governed data, and fragmented data environments can lead to incomplete or misleading outputs. That reinforces skepticism when implementations feel rushed or disconnected from real clinical workflows.

Before you consider integrating AI into clinical operations, make sure your data environment is robust. Many existing AI applications aren’t truly ready for the market. You must ensure you aren't building a smart house on a broken foundation.

Safety, hallucinations, and regulatory risk

By now, we’ve all heard cautionary tales about generative AI. One high-profile example involved Google’s healthcare AI, Med-Gemini, misidentifying a part of the brain in a research paper and blog post.

AI models are trained to appear confident in answering questions, even when they’re wrong. Probabilistic AI carries an inherent, non-zero-percent chance of hallucinating or inventing clinical data. That poses considerable risks to patient safety and regulatory compliance, especially when considering the data-quality issues outlined above.

In an era of strict malpractice and regulatory risk, we must look beyond the hype and demand transparency: a distinct audit trail that explains recommendations and protects against any clinical errors made by the autonomous bot.

AI as a marketing label rather than a proven solution

For all their professed intelligence and capabilities, AI models are built on historical data, with a limited ability to offer predictive answers for things not reflected in that data. As a result, many tools marketed as “AI-powered” lack meaningful intelligence or evidence of real-world impact. Some are merely existing automation tools dressed up in fancier language, with no case studies or ROI benchmarks to support their claimed capabilities.

Technologies introduced into healthcare environments should demonstrate measurable effectiveness within their intended workflows—whether in scheduling, billing, clinical documentation, or patient follow-up—before being scaled.

Patient-facing AI and unclear human oversight

With many EHR-based AI applications, such as ambient clinical documentation or background AI scribes, it’s easy to connect the dots with human oversight; clinicians review and validate outputs, and they make adjustments as needed.

Without transparent oversight models, these tools can raise concerns among both patients and clinicians. Did the AI voice chatbot get triage correct, accurately summarize symptoms, or dispense the right advice about the patient’s next steps? Is it accurately routing all patient identification and intake forms, and syncing them with your system? Did it double-book appointments?

And does the AI chatbot soothe patients with an empathetic tone, or frustrate them as lacking nuanced human understanding?

The human element

In the rush to embrace AI, we often forget that this is a new technology. While it has the potential to help solve some of healthcare’s biggest problems, it remains imperfect and prone to error. Human experts should always verify AI outputs, investigate the sources of their conclusions, request references, and cross-check information.

In practice, that might take the form of a methodical approach, such as establishing a clinical-focused governance board, identifying a small group of clinician early champions to pilot tools before rolling them out more broadly, and mandating ROI benchmarks to ensure the technology is delivering on its promises.

As we look for ways to incorporate AI into healthcare, we need to be pragmatic innovators. We must be clear-eyed about the ultimate goals and design strategies that improve the day-to-day working conditions of clinical and office staff and support their decision-making, rather than compete with it.

Beyond saving money or boosting revenues, any attempts to fix our highly flawed system must ultimately rest on the real priorities: increasing trust, ensuring data integrity, preserving clinician autonomy, and improving the patient experience.

John Deutsch is the CEO of Bridge.