Optimizing Your Endpoint Adjudication Process Today and Beyond

Medidata would like to thank attendees who participated in the 3rd annual Optimizing Your Endpoint Adjudication Process Today and Beyond virtual experience. This event brought together industry leaders and colleagues to discuss key aspects of endpoint adjudication in clinical trials. 

This year, two expert panels shared their knowledge of common pitfalls and best practices in endpoint adjudication, along with setting a vision for the future of artificial intelligence/machine learning (AI/ML) integration. 

Read on for insights on how to enhance the endpoint adjudication process.

PANEL 1: Lessons Learned from Challenging Adjudication Trials

Speakers:

• Jill Wnukowski – Solution Product Specialist, Medidata
• Sadaf Sharfaei, MD – Medical Director, Baim Institute for Clinical Research
• Shea Hogan, MD – Clinical Scientist, CPC Clinical Research

What Challenges Have You Been Facing When Navigating Adjudication Trials?

Insufficient and poor-quality source documentation presents one of the most significant challenges in the adjudication process. This arises from inconsistencies between electronic data capture (EDC) and source documents, or from ambiguously defined endpoints in the charter. 

When required documents are missing, adjudicators are tasked with requesting additional data from sites, leading to time-consuming back-and-forth communications. This not only causes delays in event processing but also creates frustration and reduces efficiency for all parties

involved. In extreme cases, more information is needed to let adjudicators assess more conservatively, leading to skewed study endpoint results. 

Poorly designed adjudication forms can cause discrepancies between EDC and source documents. As the number of fields on EDC forms increases, so does the likelihood of discrepancies on event calls. This issue is particularly pronounced for subjective data, which may be interpreted differently by adjudicators. Additionally, ambiguously defined endpoints in the charter further exacerbate the potential for inconsistencies.

How Are These Challenges Being Addressed/Overcome?

Ensuring adequate documentation requires early and frequent communication between all involved parties. The charter should ideally include a comprehensive list of required documents for each endpoint. Site investigators should receive thorough initial and ongoing training to understand the specific information needed for adjudication and why the data is important. 

Proactive communication from sites regarding documents that will be unavailable (e.g., due to a death event) can significantly save time for both sides. This upfront investment in clear communication and preparation can prevent downstream problems and inefficiencies. 

To minimize discrepancies between source documents and EDC, form design should prioritize conciseness while avoiding unnecessary fields. Additionally, industry standards from published studies and census documents should be referenced to develop clear definitions of endpoints. Ensuring complete alignment with the Clinical Endpoint Committee (CEC) on these definitions is important to minimize discordance—particularly for potentially problematic endpoints that may fall into gray areas for event calls. Ultimately, effective leadership, proactive engagement among all parties, proper training, and a well-defined charter are key elements in addressing the challenges inherent to the adjudication process.

Audience Q&A:

What Are the Key Lessons Learned You Want to Share?

All parties involved need to be thoughtful and clear in terms of the organizational structure, form design, and drafting of the charter. While this process may involve some iteration, anticipating logistical challenges early on allows for greater focus on trial enrollment, adjudication, and achieving overall study objectives. Investing in a robust and dynamic adjudication platform with automated features can yield long-term benefits through easier data visualization and analysis.

PANEL 2: The Future State of Endpoint Adjudication in Clinical Trials

Speakers:

• Andrea Falkoff – Vice President, Product Management, Medidata 
• David Saunders – Senior Director, Integrated Evidence, Medidata
• Ben Shim – Data Strategist in Excellence and Quality, Eli Lilly
• Diego Bartolome – Head of Research, ai.now, TransPerfect

What Changes Have You Seen in Endpoint Adjudication in the Last 3 Years?

There has been significant growth in terms of the requirements for endpoint adjudication across many therapeutic areas. Regulatory agencies are also looking for consistent evaluation assessments for the relevant endpoints of interest across trials. 

AI is becoming increasingly integrated into the adjudication process. This technology is being applied to a variety of areas, including the removal of personal identifiable information (PII), and protected health information (PHI), generating insights from large datasets, extracting sentiments, and summarizing and translating information. 

The adoption of patient privacy regulations represents another area of growth for AI. With the EU's General Data Protection Regulation (GDPR), a delicate balance must be navigated between patient privacy/informed consent and retrieval of medical source documents. Consequently, it has become increasingly important for EDCs to develop new capabilities in document collection and personal information redaction.

Where Do You See the Industry Going in the Next 3–5 Years?

Over the next 3-5 years, AI and machine learning (ML) will become more deeply integrated throughout the adjudication process. This automation will address key challenges highlighted by the first panel of speakers, including improving data capture, redacting protected health information (PHI), flagging quality issues, and enhancing quality control for endpoint adjudication. Looking further ahead, AI may potentially serve as a substitute for one of the clinician reviewers. The overarching aim is to identify AI applications that can significantly boost workflow efficiency for all stakeholders involved in the adjudication process.

While AI/ML integration offers many potential benefits, it’s important to recognize that these technologies are not simply plug-and-play solutions. Substantial effort is required to test and implement algorithms, many of which are still in the piloting phase. Once a model's effectiveness is established, it can then proceed to validation and then scaling and integration across trials.

In general, the industry has shown reluctance to fully embrace AI for adjudication trials due to uncertainties surrounding regulatory positions and the level of validation required for the algorithms. But as the number of successful use cases increases, the adoption of the technology is expected to accelerate across the sector.

Audience Q&A:

What Role Will AI Play in Adjudication?

AI has become a top priority for translations in adjudication processes. Many CROs have adopted neural machine translation (NMT), which provides cost-effective, high-quality, and accurate translations with faster turnaround times. While human review remains necessary for cases requiring 100% accuracy, AI can still help get quicker insights from the desired language. But the integration of AI/ML does not mean that human involvement becomes obsolete; rather, it enhances their efficiency and quality of work. 

AI also shows promise in identifying clinical outcomes such as major adverse cardiac events (MACE). A recent proof of concept study used historic trials split into test and training sets to evaluate AI performance. The model was trained on all of the information captured during the clinical trial, with adjudication panel events serving as the “truth” to determine how well the model performed. Results demonstrated that the algorithm could identify MACE events comparably to the adjudication panel.

Audience Q&A:

What Regulatory Considerations Should We Be Thinking of Moving to in the Future? 

Regulations that are coming—particularly the EU's AI Act and Responsible AI framework—will be very important for adjudication. High-risk patient data will have to be handled securely to ensure privacy. Patient safety is another area of consideration since regulators want investigators and adjudicators to submit reliable data. To meet these requirements, full transparency regarding the algorithms used and how the solutions behave will be essential. Lastly, AI does not have to be perceived as a "black box." AI should instead be viewed as a tool that enhances work being conducted by humans by enhancing efficiency, speed, and accuracy.

Summary

The event panels discussed a variety of challenges in the endpoint adjudication process, with an emphasis on issues related to source documentation quality, availability, and accuracy. Key takeaways include the importance of developing a robust charter that incorporates feedback along the way and investing in a dynamic adjudication platform with automated features that can yield long-term benefits. Discussions also covered the current and future roles of AI/ML in the adjudication process, while addressing the integration of these technologies with changing patient privacy regulations, applications in translations, and potential use in clinical outcomes prediction.

Watch the event recording on-demand:

Ready to optimize your endpoint adjudication process? Learn more about Medidata Adjudicate here.