Artificial intelligence (AI) undeniably has the potential to transform the pharmaceutical industry, speeding up drug discovery, enhancing patient insights, and improving data analysis. Yet, amidst rapid adoption in specific areas, a crucial question persists: How do we build and maintain trust in AI outputs, especially when biases can lurk beneath the surface?
This article series explores strategies for fostering trust in everyday AI use. Part one focuses on general principles that help us use AI outputs with confidence and preserve the high standards expected by our stakeholders, while future parts will address identifying and mitigating biases in AI systems.
Why Trust Matters
Medical affairs professionals are entrusted with decisions that impact both patient outcomes and corporate reputations. As we increase our use of AI, whether for literature reviews, insight generation, material approval or strategic planning, it is essential that we and our stakeholders can place ongoing trust in the technology’s outputs. Ultimately, by adhering to the principles of transparency, explainability and reliability, AI based technologies become trustworthy and earn stakeholder confidence.
Focusing on these general principles
In clinical trials, practices like double-blinding reduce biases, and pre-planned statistical testing ensures robust study design and adequate power. These measures help build trust by increasing confidence that positive results are due to the intervention under study rather than external factors.
In contrast, as users, we may not exert the same degree of influence over AI system design or output. When using large language models (LLMs), we often lack visibility into their “black box” workings, such as data collection methods, model weightings, training processes, and outcome reporting. While we may not fully influence these underlying mechanisms, we can apply certain principles to build confidence and trust in AI outputs.
- Transparency & Traceability: AI systems should ideally ensure transparency through explainable decisions, auditable models, and clear accountability. Additionally, tracing AI-generated outputs back to their original sources is critical. For instance, literature search platforms like Elicit or Consensus link section summaries directly to the underlying articles. Similarly, our product, Beacon (Circuit Medical AI approval assistant), provides full traceability for AI-generated comments. Users can see where a comment/insight came from, which reassures them that the content isn’t conjured from thin air.
- Explainability: When an AI insight appears inconsistent with known data, you want a clear explanation. This builds confidence that the system’s logic aligns with clinical facts/relevance. If a user spots an anomaly, they can investigate the original sources and verify the chain of reasoning.
- Reliability: How consistent are AI outputs when asked to perform the same task under identical conditions? For example, LLMs are powerful but non-deterministic, meaning they might produce different responses to the same query at different times. Imagine a med info chatbot providing three different answers to the same question about the stability of a MAb when stored out of the fridge? How confident would you feel in its reliability? To address this, some products/models use specific techniques to enhance reproducibility and consistency (as well as accuracy) in their outputs.
Practical Steps to Build Confidence
No human, algorithm, or process is perfect. However, there are concrete steps to maintain confidence:
- Seek to Understand Your Model: Understand the capabilities, limitations, and ideal use cases of the AI tool/product you’re using. Just as we do with non-AI sources, understand and manage the limitations.
- Keep a “Human in the Loop”: Especially in high-stakes environments, human oversight ensures that AI outputs are cross-checked before final decisions are made. This is standard practice with Beacon.
- Provide Feedback: When you encounter odd or potential biased AI outputs, report them to the product owner or development team to drive iterative improvements. Equally important is positive feedback so development teams know what is working well.
The rapid pace of change with AI has seen bodies like the FDA playing catch-up. Recent guidance on AI in drug development underscores trustworthiness as a core principle. This suggests a growing consensus on the importance of transparency, explainability, and reliability.
AI holds tremendous promise for accelerating progress in the pharmaceutical industry. By focusing on these core principles, we can harness this technology while upholding the trust of our stakeholders, and peers. It is for this reason we prioritised traceability, explainability and reliability from the start of Beacon’s development. How are you using these principles to choose the AI products you work with?
References
