How states are taking the lead on AI in health care

Congress hasn’t passed much legislation covering the use of artificial intelligence in the healthcare industry, but some states are not waiting on Washington, D.C.

Some state legislatures have crafted bills designed to offer some regulations of the use of AI in health care, analysts say.

Jared Augenstein, senior managing director at Manatt Health, said in a recent American Medical Association webinar that more than 250 AI-related bills have been introduced in 34 states this year.

That’s a significant uptick already over 2024, when about 100 bills on AI were introduced at the state level, according to Manatt Health.

“There really hasn't been that much activity, especially from a legislative perspective at the federal level,” Augenstein said. “The states have really started to step in.”

It’s an open question whether states will be able to impose their own regulations. The House spending and tax plan approved last month includes a provision that would bar states and local governments from regulating AI for the next 10 years. The Senate is now reviewing the legislation.

This isn’t to say there’s no federal activity regarding AI in the healthcare industry. The Food and Drug Administration has approved more than 1,000 AI-enabled medical devices to date, and President Trump’s administration “is very positive on the potential for AI,” he said.

Trump’s administration has expressed strong support for the use of AI in general. Trump issued an executive order just after returning to the White House in January aimed at ensuring America is the leader in artificial intelligence, and the White House has directed federal agencies to use AI in government operations.

But states are looking at legislation regarding privacy, consumer protections, and even the clinical use of AI in patient care, analysts say. State lawmakers are looking at legislation that includes protections to ensure AI tools aren’t used to discriminate against certain groups.

Shannon Curtis, assistant director of federal affairs for the AMA, said states are increasingly willing to take up the use of AI.

“States have really started picking up kind of the slack here in the lack of a lot of federal action,” Curtis said.

What states are doing

California enacted a law at the beginning of the year that requires doctors to review decisions made by a health insurance company’s AI tools or algorithms. California Gov. Gavin Newsom signed the bill, dubbed the Physicians Make Decisions Act, last fall.

Colorado has passed legislation requiring companies to inform consumers if AI technologies are making decisions that could affect them, and gives consumers the ability to appeal. The law is slated to take effect next year, although business groups and some lawmakers have moved to revise the legislation, or push back the implementation date.

Utah has also passed legislation that requires companies to disclose if consumers are interacting with AI technologies. Utah lawmakers have also moved to add provisions regarding protections for consumers interacting with chatbots for mental health services, and other states are looking at regulations regarding mental health chatbots.

“We're starting to see states begin to regulate the use of those tools as well,” Augenstein said.

Some of the state legislative efforts have been focused on establishing task forces and working groups to look at AI in health care, said Kim Horvath, a senior attorney with the AMA’s advocacy resource center.

“Last year really was the first year that we saw a lot of state level activity on AI,” Horvath said.

Analysts expect states to take a look at how health plans are using AI and if it’s being used to make it harder for patients to get treatment. An AMA survey found a majority of doctors (61%) say they are worried that insurance companies are using AI, or will use the technology, to increase the rate of denials.

Even with many doctors seeing a need for safeguards, some are worried that states may impose regulations that could slow down innovation, Horvath said.

“There's a lot of excitement about what AI can do and how it can benefit patients and help ease the burden on physicians … but there is still this fear of over-regulating,” Horvath said.

States such as California and Washington are aiming to increase the transparency of how health insurance companies are using AI and require public disclosure of how AI is being used to manage claims and coverage, said Emily Carroll, a senior attorney with the AMA’s Advocacy Resource Center.

Some states are looking to give their insurance commissioners the authority to regulate how health plans are using AI, she said.

‘Thoughtful and measured’

Carroll said she’s encouraged by some of the legislation being taken at the state level.

“I think that while we are recognizing how the efficiencies that AI can bring to physician and health plans’ interactions, it's really important that the states are considering what role AI should play when it comes to such consequential decisions as determining whether a patient can access care,” Carroll said. “And to me, so far, it seems the states are being really generally thoughtful and measured in their approaches.”

Carroll also said that she’s encouraged that states are looking at measures to ensure AI tools aren’t used to discriminate against certain patients. And she is encouraged that some states aren’t looking to slow down advances in AI.

“The states are jumping in and figuring out what might work best in terms of balancing that regulation with innovation,” she said.

Two out of three doctors (66%) say they are using AI in their practice, compared to 38% in 2023, according to an American Medical Association survey released in February.

Hospitals and health systems are increasingly utilizing AI tools to reduce the workload on their staff, including documentation tools to summarize patient appointments. But more hospitals are using AI in clinical uses as well.

Nurses have protested the labeling or marketing of AI tools as a “nurse” or “AI nurse.” In Oregon, a bill has been introduced that would prohibit companies from using “nurse” in the title of any AI tools.