Healthcare industry navigates AI regulatory challenges on its own in absence of federal guidance

In the absence of federal standards, medical companies are struggling to adopt artificial intelligence, and the Trump administration is unlikely to provide any help, leaving the burden of implementing AI responsibly entirely on the industry itself.

Experts said at the HIMSS healthcare conference in Las Vegas that this task is becoming increasingly difficult as the technology grows more complex.

"One thing is clear: this administration will not regulate AI. For better or worse, figure it out on your own," said Tanay Tandon, CEO of vendor automation company Commure, during a panel discussion.

President Trump's hands-off approach to AI governance means hospitals hoping to use AI to save costs and provide relief to overworked clinicians may operate in a regulatory gray area for at least the next four years.

The president has said his goal is to free American developers to spur innovation. But this also has downsides for technology developers and medical companies eager for guardrails, because AI is prone to errors, degrades over time, and exacerbates existing biases.

Not to mention, some experts believe that the lack of national standards could actually hinder AI development and adoption.

"When there is no federal framework, it can absolutely create all sorts of issues," said Leigh Burchell, president of the Electronic Health Record Association. "We all want to know what the rules are, and then we can comply."

Biden and Trump's differing stances on healthcare AI

So far, a handful of federal agencies, including the HHS technology office, CMS, and the Food and Drug Administration, have issued targeted rules on the use and quality of AI in healthcare. But neither Congress nor the executive branch has focused on creating a comprehensive regulatory framework—although some progress was made during the Biden administration, when an HHS working group was working to build a unified regulatory structure.

That working group released a strategic plan in January—just 10 days before Trump took office. However, Trump struck down the blueprint in one of his first executive orders.

Meanwhile, federal employees working on AI regulation, including those at the FDA, have been swept up in the Trump administration's personnel purge. Amid this turmoil, the future of the HHS office responsible for AI policy remains unclear.

As a result, the little momentum Washington had toward developing a concrete healthcare AI regulatory strategy appears to have stalled, at least for now. Instead, Trump has announced "Project Stargate," a $500 billion investment deal with private companies aimed at prioritizing AI development and maintaining America's edge in the field—a high-stakes bet immediately complicated by the release of China's high-performance, low-cost open-source model DeepSeek.

"This administration—the brakes are off, the pedal is down."

—Brian Spisak, Director of AI and Leadership Programs at Harvard's National Preparedness Leadership Initiative

The Trump administration issued a request for information in early February seeking public input on a potential national AI action plan. However, the plan's wording made clear the administration's priorities: "sustaining and enhancing America's AI dominance and ensuring that unnecessary burdensome requirements do not hinder private-sector AI innovation."

Reversing Biden-era AI plans was largely symbolic, as agencies had not yet imposed any requirements on developers or users.

But with "this administration—the brakes are off, the pedal is down," said Brian Spisak, Director of AI and Leadership Programs at Harvard's National Preparedness Leadership Initiative, at HIMSS. "Healthcare system leadership has a significant responsibility to find the best balance between innovation and speed, safety and tradition."

A technological sea change

This responsibility—also falling on AI developers creating the models, software vendors integrating AI into health records, and clinicians using them—is not trivial.

Currently, even the most cutting-edge AI in healthcare institutions is mainly used for administrative automation, touching patient care only marginally. But this appears to be changing: according to a survey conducted by HIMSS in the fall, healthcare organizations are showing growing interest in more clinical use cases for AI, such as tailoring treatment plans or helping clinicians make diagnoses.

Many use cases involve generative AI, which can create original text and images. But these models are known for "hallucinations," providing factually incorrect or irrelevant answers. AI may miss important information, an error called omission. Models can also drift, meaning AI performance changes or degrades over time.

Experts say that given AI's growing prevalence in extracting data from EHR systems, transcribing doctor-patient conversations, and more, such errors could interfere with clinicians' ability to care for patients.

Meanwhile, the technology is advancing at a staggering pace. Last year, the healthcare industry was just beginning to grapple with governance issues around generative AI. But now, discussions have shifted to AI agents, which can complete complex tasks with little to no human oversight.

At the HIMSS health IT conference in Las Vegas in March, multiple companies showcased AI agents.

Commure's Tanay compared the current moment to the late 19th century when the U.S. shifted from kerosene to electricity. "The way we did things six months ago is completely irrelevant," he said.

Experts say that because of the rapid pace of development, any federal standards from the government would need to remain flexible.

The Biden administration's HHS working group suggested the government could develop guidelines around testing and pilot tools and provide some adoption support. However, it shied away from prescriptive approaches.

This aligns with the wish list of many stakeholders. Several executives from tech companies and hospital systems said any federal standards should be tiered by the level of risk posed by the AI model—for example, stricter oversight for algorithms that help doctors diagnose diseases, and lighter oversight for algorithms that help hospital staff allocate beds.

"We have to weigh the balance between under-regulation, which may increase risk, and over-regulation, which will stifle innovation," said Anthony Chang, Chief Intelligence and Innovation Officer at Children's Hospital of Orange County, California, during a panel. "This administration is more likely to lean toward under-regulation. So as professionals, we must be careful not to allow that to happen."

States and industry groups fill the void

The lack of guidance from Washington has left hospitals and medical groups scrambling to build their own internal controls amid a patchwork of state laws and voluntary standards issued by industry organizations.

States including Colorado, Utah, and California have enacted laws imposing disclaimer requirements on AI systems. More states are considering similar laws: according to Burchell, the Electronic Health Record Association is tracking 150 state bills related to healthcare AI.

"The number of bills has exploded," Burchell said.

But differing standards could prevent healthcare AI developers and software companies from launching products in certain states, potentially putting patients at a disadvantage based on where they live. She added that more risk-averse software companies might avoid AI or certain states altogether.

"State-level legislation of all shapes and sizes poses a risk to us because it means we have to do all sorts of different development. We would prefer to develop a system that can be widely used and accepted across the country," Burchell said.

Healthcare AI standards organizations are also filling the void left by the federal government. These groups are typically composed of leading hospitals, digital health companies, and tech giants, including the Health AI Partnership, an industry AI learning network, and the Coalition for Health AI, which recently launched a hospital AI registry.

"I think we may see more non-governmental organizations like the Health AI Partnership become our North Star today because they provide leadership in this space," said Rachel Wilkes, enterprise lead for generative AI initiatives at EHR vendor Meditech.

But experts say that without federal backing, standards from industry coalitions carry little weight. Historically, voluntary standards have not been particularly effective.

"Without a federal framework, people have room to act in their own best interests, whatever that may be," Wilkes said.

"We still don't quite know how to respond"

EHR vendors and hospital operators say they are building rigorous internal standards for AI tools, including validation and frequent audits.

"Government oversight has its place, but I do think the evolution of clinical practice is often driven more by what happens inside healthcare systems," said Seth Howard, executive vice president of R&D at Epic, the largest U.S. EHR company.

In interviews, executives from Epic, Oracle, Meditech, and eClinicalWorks said they are providing AI to physicians with rigorous oversight, including backend accuracy checks and continuous monitoring.

However, technology leaders emphasized that hospitals and clinicians also have a responsibility to ensure everything runs as planned.

"We are working in an industry that involves human lives. It cannot be trivial. What guardrails, checks and balances, and discussions are needed cannot be underestimated," said Girish Navani, CEO of eClinicalWorks.

Tech giants making big bets on AI echo this sentiment. For example, Google has partnered with for-profit hospital giant HCA to develop evaluation frameworks to catch any errors produced by its AI models and ensure reliability, according to Aashima Gupta, head of healthcare at Google Cloud.

"We provide these tools for evaluation frameworks, and all of this has people involved in the feedback loop that makes the models more effective," Gupta said. "That gives me peace of mind."

"We are working in an industry that involves human lives. It cannot be trivial. What guardrails, checks and balances, and discussions are needed cannot be underestimated."

—Girish Navani, CEO of eClinicalWorks

Although some in the private sector say they have governance handled, AI engineers say modern AI is extremely difficult to oversee. The main strength of generative AI—its creativity—also introduces subjectivity, complicating the scoring of its outputs.

For example, if two clinicians are asked to summarize a patient's medical history based on clinical notes, their results could be quite different, yet still accurate. The same is true for generative AI, experts say: when there is that level of variability, how do you measure quality in a standardized way?

"AI governance is still a very immature process," Harvard's Spisak said.

Hospital executives say they are handling oversight carefully. But some research suggests that governance systems for simpler predictive AI models are already insufficiently rigorous. According to a study published last year in the New England Journal of Medicine, hospitals with clear AI tool usage and evaluation procedures still struggle to identify and mitigate problems.

A patient undergoes an exam with ambient listening AI technology to transcribe the interaction.

Even some of the most well-resourced and technologically advanced systems are struggling.

Cleveland Clinic has an AI governance body that includes stakeholders from across the academic medical center, according to Rohit Chandra, the clinic's chief digital officer. The working group oversees AI's impact on the organization and patients while ensuring clinical safety and tackling thorny issues of privacy, legality, and bias.

But "I don't think we've fully figured it out," Chandra said during a panel. "The word 'hallucination' has only emerged in the past two or three years. We still don't quite know how to respond."

Hospitals should try to hold specific individuals accountable for tool performance as part of a larger governance body that includes executives, lawyers, physicians, and nurses, said Brenton Hill, operations lead at standards organization CHAI.

Hospitals need to decide how to effectively monitor AI and report that information, depending on the products they already have. They also need to consider what resources AI will use and establish appropriate data use agreements with AI vendors, Hill said during the panel.

But "there is no silver bullet governance structure that solves everything," Hill said.

A "pipe dream"

Although a roadmap from federal regulators would help, stakeholders working to integrate AI tools into healthcare say they are not holding out much hope.

"While self-regulation is good, we don't think it's enough. We think AI is too important not to be regulated," Google's Gupta said.

But when asked about her expectations for the Trump administration, Gupta was noncommittal. "It's hard to say right now. We're trying to figure out the best way to work with them, share our best practices... It's too early to tell. I think the entire healthcare community is waiting."

Other experts say the Trump administration is a wake-up call for hospital executives who had hoped Washington would take on AI oversight.

Instead, responsibility should fall on everyone who touches the technology to ensure AI algorithms—given their variability and inherent opacity—operate as designed, especially in an industry where any error can affect patient health.

"The simple answer is, if the regulator says it's safe, then I can trust it. I think people had hoped that would be the case with AI too," said Aaron Neinstein, chief medical officer at agentic AI company Notable. "I think that was a pipe dream."