Has Detailing AI’s Reasoning Become Confused With The Tech’s Safety in a PV Context?

Contributed commentary by Beena Wood, Chief Product Officer, Qinecsa Solutions 

November 6, 2026 | For any life sciences organization keen to establish appropriate guardrails around their AI use, recent debate about preventing AI’s potential harms has helped frame some of the critical parameters. Previously, too much of the “trusted AI” discussion has centered on the ability to explain how AI has arrived at its conclusions. But this quality alone does not make for robust governance, especially where patient safety is at stake.  

2026 has seen major regulators internationally, including the UK MHRA, the US FDA and the EU’s EMA, hone their focus on AI governance, and conclude that trust measures should be risk-based, proportionate, outcome-focused, and calibrated to the context of use. Final recommendations published 10 September 2026, following the largest engagement exercise of its kind in the UK, gathering evidence from more than 12,000 people over roughly a year. The headline recommendation is that AI-enabled health technologies should be subject to continuous, real-world monitoring throughout their working life, not approved once and left unmonitored.  

Although the ability to explain AI’s rationale remains important, such a facility is only useful after what may have been a damaging event—as with the black-box recorder on an aircraft. To maximize safety ahead of time, other gauges of potential risk are required too (equivalent to an aircraft’s altitude alarms). 

Regulators have already identified the dangers of automation bias (of blind trust in AI to the point that human checks are bypassed or subconsciously downgraded). In April 2026, the FDA issued a landmark warning letter that explicitly cited overreliance on AI as a violation, after a company (Purolea) used AI to generate compliance documents without human review and then attributed a missed regulatory requirement to the AI not having flagged it. 

Drug safety leads also need to be sure that an AI-enhanced system will fail “safely,” rather than reach a dubious conclusion without flagging any possible concerns. Demonstrating due diligence isn’t simply a matter of documentation, meanwhile. Inspectors can reasonably expect to see operational evidence of what is happening in practice (whether the AI system has adhered to SOPs), so companies need to have ready access to this at all times. 

Horses for Courses 

None of this is about being heavy-handed, hence regulators’ emphasis on an appropriate, risk-based approach to AI governance. To understand how risk profile and controls should vary (e.g. according to what the use case touches, and where it comes in the PV lifecycle), consider two contrasting use cases: case intake, and signal detection and evidence building.  

In the first scenario, the primary risk is source and context, (e.g. does the incoming data accurately reflect a single reported event, correctly triaged for seriousness and causality?). The second, on the other hand, collates data across multiple sources and contexts (e.g. building a disproportionality trend or cumulative evidence across cases, potentially for onward review by a safety review committee). Where getting one case wrong is still a serious error, incomplete scoping or issues with the provenance of the underlying data could mean missing an entire signal. Consider also the additional risk of what is feeding into the “escalate or close” decision. Have the statistics, trends and supporting cases shown to a reviewer been reliably calculated from real data, or is the source actually an AI-generated summary which, although reading as plausible, doesn’t stand up to scrutiny? 

Maintaining Vigilance Across the AI System’s Lifecycle 

AI system governance needs to flex with the risk profile, which extends to how much the system is allowed to do without human review. For the most consequential decisions, an extreme option may be to remove AI “judgement” completely. That could be by requiring a human to sign off before anything proceeds (so that the AI capability can never act on its own initiative), or by handling the decision using a fixed rule (rather than allowing AI to supply and explain its own reasoning).  

To guard against “automation complacency”, companies may need to reinforce personal accountability or by setting alerts if human reviews appear to be completing high volumes of AI output reviews unnaturally quickly. 

Other useful forms of continuous monitoring would include detecting when a normally active data source appears to have been muted (e.g. a distributor that would usually report every fortnight producing nothing for 40 days) and flagging this anomaly proactively; and highlighting possible contradictions between sources—e.g. a patient reporting no concomitant medication against an HCP report vs. contradictory lab data (a discrepancy that could materially change a causality assessment)—to avoid scenarios where the AI system might make and take action based on wrong assumptions that otherwise might not be spotted. 

Across the board, evidence generation should become continuous and built into the way that an AI-based system operates day to day. In the AI age, everything should be expected and nothing should be assumed. 

Beena Wood has spent more than 20 years at the sharp end of PV technology. She is a regular keynote speaker at global safety congresses, including the World Patient Safety Congress and, most recently, the Pharmacovigilance & Drug Safety Conference in Madrid. She is chief product officer at Qinecsa and can be reached at beena.wood@qinecsa.com