Healthcare Dive Trendline on Artificial intelligence

Health systems are hustling to implement artificial intelligence across their organizations, but how to measure returns on those investments isn’t always clear, experts said at the HLTH 2025 conference.

AI technology, which includes ambient documentation scribes, summarization tools and revenue cycle management products, costs money and time to put in place. And it’s not always obvious how AI produces hard-dollar ROI, which is important to justify expenses and measure success, experts say.

“The use of AI to simplify administration, the use of AI to assist with productivity is absolutely out there,” said Thom Bales, principal and health services advisory leader at PwC. “There still are questions around the ROI on it and hitting the bottom line.”

At the same time, health systems are also facing increased financial pressures. This summer, President Donald Trump signed a massive tax and policy bill that makes historic cuts to Medicaid, likely culling millions of people from the safety-net insurance program.

Meanwhile, more generous financial assistance for people who buy coverage on the Affordable Care Act exchanges — the issue at the center of a weeks-long government shutdown — is set to expire at the end of the year, also pushing more people off coverage.

More uninsured Americans means less revenue for providers and an increase in uncompensated care.

Still, the healthcare sector is hopeful that AI will move the needle on some of its biggest challenges, including workforce shortages, provider burnout and lengthy administrative tasks. Amid the challenging economic environment, healthcare companies are prioritizing AI investments most likely to improve profit margins and demonstrate clear returns, according to a survey published by Klas Research and Bain & Company.

But given those returns can be hard to show, hospitals may have to get creative in measuring the value of AI, including by looking at time savings for providers, stronger employee retention or improved patient satisfaction, experts say. Those softer measures could then be linked to financial improvement over time.

“I do think ultimately we can trace the majority of these things back to a bottom line, but you may have to connect more dots,” said Jennifer Goldsack, CEO of  industry association the Digital Medicine Society.

Piecing together ROI

Measuring financial returns is always tricky, given the complex interplay of factors that complicate attributing eventual savings to a specific product. This is also true when it comes to AI, where the direct financial impact of a product depends on the tool, the costs needed to adopt it and each provider’s specific priorities, experts said at HLTH.

“In healthcare, particularly in the healthcare delivery side, it’s really hard to apply line of sight to what we might think of as ROI,” Micky Tripathi, chief AI implementation officer at the Mayo Clinic, said during a Monday panel. “I don’t think that any of us should really expect you’re going to see a cost reduction that a CFO could look at and say, ‘Oh, I see it there.’”

Measuring financial returns is easier with certain AI products. For example, revenue cycle management tools might allow health systems to see metrics like time to collections improve, according to Sandra Johnson, senior vice president of client services at EHR vendor CliniComp.

But the impact of other AI tools is less obvious. There’s limited evidence that AI documentation assistants, which record physicians’ conversations with patients and create a clinical note, are affecting productivity and financial performance, according to a report published by the Peterson Health Technology Institute. 

However, the scribes are likely reducing clinician burnout, the report found — a significant concern that could be exacerbated by the amount of time and effort needed to document care and do other work in EHRs.

If AI scribes prevent a clinician from leaving the health system, that could be a major cost saver, experts say. The expense of replacing a physician can be two to three times the physician’s annual salary, studies suggest. 

And there are other potential financial and efficiency gains from ambient scribes, said Dr. Nele Jessel, the chief medical officer at health technology firm and EHR vendor Athenahealth.

For example, providers could complete notes more quickly and possibly see more patients. They also might be able to document services more accurately and bill for a higher level of services.

Patient satisfaction is another important metric for health systems. Patients tend to like ambient scribes in part because their doctors can be more present during appointments, instead of staring at a computer screen while they take notes. Documentation assistants might also help patients understand their care plans better, since physicians have to speak out loud during an exam to ensure the scribe picks up the information, Jessel said. 

“Providers are getting much better, for example, [at] vocalizing the physical exam, talking about the plan, right?” she said. “So it increases the transparency and also makes it a much more enjoyable experience for patients.”

How health systems are measuring AI’s impact

To implement AI successfully, health systems need to think critically about the biggest problems they want to solve and their main priorities, experts say. That plan should include how they’re going to evaluate the AI tool, which can help them determine if the project was a win — whether financially or by other metrics.

“To me, step zero is being very, very clear about, what is it that we’re doing? Why are we doing it? And what does success look like for that?” Mouneer Odeh, chief data and AI officer at Cedars-Sinai, said during a Sunday panel. 

Cleveland Clinic evaluates AI using two types of metrics, according to Sonja O’Malley, the general manager of business development and licensing at Cleveland Clinic Innovations, the system’s commercialization and innovation arm. 

Those include quantitative measures — like reducing no-show rates or cutting documentation time, compared to a baseline metric — as well as more qualitative criteria, like feedback on patient or clinician experience. 

Some tools will never be revenue generators. But they might do other important things, like allowing a clinician to spend time on tasks that require their full training and expertise, O’Malley said.

When an AI tool is brought before Ardent Health, metrics of success have to be clearly defined and measurable alongside a baseline, according to Anika Gardenhire, the health system’s chief digital and transformation officer. 

Financial return on investment is almost always important, but it might be correlative rather than direct causation, she said.

If a tool only breaks even, the system needs to consider other associated costs, such as implementation expenses or the time clinicians spent on the project when they could have been working elsewhere.

“If you get to a place where you’re just like, ‘Hey, this thing didn’t even break even. It doesn’t even pay for itself.’ Unless it’s doing something absolutely miraculous for patients, it’s out,” Gardenhire said.