A Healthcare AI Due Diligence Framework: The Criteria That Separate Products From Demos

1. Why Healthcare AI Diligence Is Different

A healthcare AI product can be clinically plausible and commercially dead at the same time. Most software cannot fail that way, which is why frameworks built for ordinary SaaS diligence keep missing it.

The reason is structural. In most software, the user, the buyer, and the payer are the same party. In healthcare AI they are usually four different parties: the clinician who uses it, the health system that buys it, the payer who may or may not reimburse anything connected to it, and the regulator who decides what may be sold and with what claims. A product can delight the first, get purchased by the second, and still die because the third never pays and the fourth constrains the roadmap.

One scope note before the framework. This guide is for diligencing companies that sell AI into healthcare: the target is an AI business. If your question is the reverse, deploying AI inside a healthcare portfolio company you already own, that is a different job with a different playbook, covered in AI agents for healthcare private equity.

And a boundary worth stating plainly: everything here is a set of investment diligence questions. It is not clinical guidance and it is not legal or regulatory advice. Specialist counsel and clinical advisors belong in confirmatory work. This framework tells you what to make them prove.

2. The Framework at a Glance

The framework is seven dimensions, scored separately, because healthcare AI deals fail one dimension at a time.

Clinical validation: is the evidence real, and does it travel. Regulatory pathway: does the clearance match the claims and the roadmap. Data rights: does the company own what its models are built on, and does that ownership survive the deal. Reimbursement: does anyone pay, today, from a named budget. Model risk: who watches performance after deployment. Integration: what it costs to install, and what it costs a customer to leave. Team: whether clinical, machine learning, and regulatory literacy sit in the same room.

A thesis usually survives a weak score on one dimension, priced accordingly. It rarely survives two. The discipline is scoring each on evidence rather than on the pitch, which is the same discipline that separates AI businesses from AI stories in deal screening.

If your firm has used an AI vendor evaluation scorecard for its own tools, the shape will feel familiar. The healthcare dimensions are what get added, and they are the ones that kill deals.

3. Clinical Validation: Whose Patients, Whose Benchmark

Every healthcare AI pitch includes an accuracy number. The first diligence question is never how high it is. It is: validated on whose patients?

Evidence quality sorts on three axes. Internal benchmark versus peer review. A test set curated by the team that built the model is the weakest form of evidence; peer-reviewed publication with external authors is meaningfully stronger. Retrospective versus prospective. Retrospective studies test the model against historical data, where nothing was at stake. Prospective evidence, where the model ran in live clinical use, is rarer and worth far more. Single-site versus multi-site. A model trained and tested at one academic center often degrades at a community hospital with different equipment, different populations, and different documentation habits.

The subtler check is claim drift. Read the validation study and the sales deck side by side. The study validated one narrow task on one population; the deck sells a broader promise. The gap between the validated claim and the marketed claim is a diligence finding in itself, and sometimes a regulatory one.

What good looks like: external, multi-site, peer-reviewed evidence, ideally prospective, that matches the claim on the website. What a demo looks like: an internal benchmark, one site, and a deck that has outrun the data.

4. The Regulatory Pathway, in Plain Language

Software that performs a medical function on its own, rather than as part of a physical machine, is generally treated as software as a medical device, and it needs a regulatory pathway before it can make clinical claims. Plenty of healthcare AI is not a device at all: administrative tools, back-office automation, and some forms of decision support sit outside that perimeter. The first question is simply which side of the line the product is on, and whether the company's answer matches what the product actually does.

For products that are devices, three pathways cover most of the map. A 510(k) clearance rests on showing the device is substantially equivalent to one already on the market, and it is the most common route. De Novo is for genuinely novel devices of low to moderate risk with no predecessor to point at. Premarket approval, PMA, is the most demanding route, reserved for the highest-risk devices and built on full clinical evidence.

The FDA maintains a public list of AI-enabled medical devices it has authorized, and that list has grown past a thousand entries. It has two uses in diligence. Verify the target actually appears on it, with the clearance covering what sales says it covers. And notice what the list's size means: authorization is table stakes in a crowded field, not a moat by itself.

Then ask the question that dates fastest: what happens when the model changes? A cleared product that cannot ship improvements without regulatory friction is frozen at its clearance date. How the company manages updates within its regulatory obligations is a roadmap question, a cost question, and occasionally the whole thesis. These are questions to hand regulatory counsel in confirmatory work, not conclusions to reach from a data room alone.

5. Data Rights and Provenance

An AI company's moat is usually described as its data. In healthcare, the sharper question is whether the company has the right to use that data, and whether the right survives the transaction.

Walk the provenance chain. Where did the training data come from: named health system partnerships, purchased datasets, public research data, or "our customers"? Under what agreements, including business associate agreements where patient data was involved, and with what permitted uses? Was the data de-identified, by whom, under a recognized standard, with documentation, or is there just an assurance? A vague answer at any link in that chain is not a detail. It is a valuation input.

Then read the agreements for the clause that matters to you specifically: change of control. Data rights negotiated by a friendly founder with an academic partner do not automatically transfer to a PE-owned platform, and some agreements terminate or reopen exactly when your deal closes. A model you cannot legally retrain on the data that built it is an asset with a decay curve.

What good looks like: a documented chain from source to model, agreements that survive the transaction, and de-identification someone will stand behind in writing. The red flags are the shrugs: "we scraped it," "the hospitals have never complained," or rights that die at closing.

6. Reimbursement: Does Anyone Actually Pay

Now the question that kills more healthcare AI theses than any regulatory issue: does anyone pay?

Three commercial patterns cover most of the market. A minority of products are paid through reimbursement: CPT pathways exist for some AI services, and where they apply they are powerful, because revenue rides the billing system instead of a discretionary budget. Most healthcare AI is sold as software: subscriptions to health systems, priced against staff time saved or capacity gained, competing for a finite IT and operations budget. And a growing slice sells into value-based arrangements, where the product earns its keep through cost avoidance that someone has contractually agreed to share.

Each pattern has a different diligence test. If the model depends on reimbursement, verify that the code exists today, that the target's product is actually billable under it, and that the payment is large enough to carry the revenue plan. Projections built on codes or coverage decisions that do not yet exist are hope wearing a spreadsheet. If it is software, find the named budget owner and ask what got cut to pay for it, because "the chief medical officer loves it" is not a line item. If it is value-based, read the contracts and check who bears the measurement burden.

The pattern to price most carefully is the company that has been selling pilots for three years. Pilot revenue is real money and weak evidence: it proves curiosity, not a budget line.

7. Model Risk and Post-Deployment Monitoring

A deployed model is not a finished product. It is a process that decays, and diligence should price who is watching the decay.

Three exposures matter most. Drift. Clinical practice, patient populations, coding habits, and upstream systems change, and a model tuned to the data of three years ago quietly loses accuracy against this year's. Site variation. The same product performs differently across hospitals, because equipment, workflows, and documentation differ, so an average accuracy number can hide sites where it is materially worse. Bias. Performance can differ across demographic groups, and buyers, health systems, and regulators increasingly ask for evidence that someone has looked.

The diligence question is not whether these risks exist. They exist for every model. The question is whether the company treats monitoring as a product function: performance tracked in production, per site; thresholds that trigger review; a retraining process with an owner; bias audits it can produce rather than promise. A target that can show you a live monitoring dashboard is in a different risk class from one that answers with the accuracy number from the launch study.

An unmonitored model in production is an unpriced liability. Someone will eventually measure its real-world performance. Better if it was the company, before the customer, before the regulator, and before you owned it.

8. Integration and Switching Costs

EHR integration is the most double-edged line in the healthcare AI pitch deck. It is the moat and the tar pit at once, and diligence has to price both.

The moat is real. A product wired into the electronic health record, inside the clinician's actual workflow, is painful to replace: switching means another integration project, more retraining, and change management no health system wants twice. Retention follows.

The tar pit is the same fact facing the other way. Every new customer is an integration project: months from contract to live, engineering time that belongs in an honest cost of revenue, and a sales cycle stretched by the customer's IT queue rather than the buyer's enthusiasm. Growth becomes rate-limited by implementation capacity, not demand.

The diligence questions are mechanical. How long from signature to live, by customer cohort, and trending which way? Who pays for integration, and do the gross margins survive an honest allocation of it? How many EHR platforms are supported, and what share of revenue sits on a single one? And the adoption question that determines everything downstream: does the product live inside the clinical workflow, or is it a separate screen with a separate login? Separate screens produce impressive pilot metrics and quiet non-renewals.

9. The Team Question

Healthcare AI requires three literacies at decision-making altitude: clinical, machine learning, and regulatory. Most founding teams have two, and the missing one usually becomes visible in the product within a year.

The pattern shows up predictably. Strong ML with thin clinical grounding builds accurate answers to questions clinicians do not ask. Strong clinical with thin ML rents its differentiation from whoever it managed to hire. Strong product with regulatory treated as outside counsel's problem discovers, late, that a roadmap feature changes the product's regulatory posture.

In diligence, test where each literacy actually sits. Is there a clinician with recent practice experience and real authority over the roadmap, or a well-known advisory board that meets twice a year? Is ML leadership publishing, retaining people, and honest about limitations? Is regulatory a function with a seat in product decisions, or a bill that arrives after them?

Team is the dimension most fixable after close, and the fix belongs in the value-creation plan with a name and a cost attached, not an assumption. But price the fix. Senior clinical and regulatory operators who also understand ML are scarce, and scarce is expensive.

10. The Scorecard

The framework compresses to one table. Score each dimension from evidence in the data room and reference calls, not from the pitch.

A target does not need seven green rows. The point is knowing which rows are red and what the fix costs. One red row is a price negotiation. Two is a rebuilt thesis. Three is a pass, however good the demo felt.

11. Where to Start

Run the framework in two passes that match the deal clock.

The screening pass takes days, not weeks, and asks the kill questions from the data room and the public record: what does the validation evidence actually show, is the product on the FDA's public list if it claims to be, does anyone pay today, and do the data rights survive a change of control. Most healthcare AI targets that will die in diligence die on one of those four, and it is far cheaper to find out before the specialists are engaged. This is the same screening discipline as any AI-heavy deal, covered in AI due diligence for private equity, with the healthcare dimensions added.

The confirmatory pass brings in the people this guide keeps deferring to: regulatory counsel on the pathway and the model-update question, clinical advisors on the evidence, technical diligence with code and data access on the monitoring claims. The scorecard becomes the work plan: every red or unknown row gets an owner and a deliverable.

If you want this run rather than staffed, that is the shape of our AI Diligence service: a $15,000 screening pass built to kill bad deals cheaply, and a $25,000 confirmatory pass when the deal is real. And if healthcare AI keeps appearing in your pipeline, the durable fix is a deal team fluent in these questions, which is what our training programs build.