The New Era of Audit Trail Review in Clinical Research

Adopting a modern approach to the audit trail review (ATR) process is more important than ever. Continue reading to learn why audit trails are important, explore several use cases, and discover how generative AI can be instrumental in overcoming the challenges of conducting ATRs within today’s complex clinical trial landscape.

The Purpose of Audit Trail Data Review 

Audit trails are a chronological record that logs changes or actions related to clinical trial data, capturing:

• Who has access to the system to identify possible unauthorized access and unblinding 
• Who and when entered data points to ensure compliance with authorized access, location, entry expectations, etc. 
• When, by who, and why was any data changed to identify possible problems with frequent changes, data manipulation, etc. 
• What other action is taken on data points, e.g., query, review, approval, etc. to ensure protocol compliance 
• Frequency of user log ins to ensure proper oversight 
• Data timestamps in case of integrated systems to ensure no connection failures, etc.

In other words, audit trails provide valuable insights into the life cycle of each data point by tracking who and how we interact with the data, and systems to ensure holistic trial oversight and data integrity. 

Regulatory Requirements

Regulatory agencies worldwide have established guidelines and regulations for the implementation of audit trails pertaining to electronic data systems—within the clinical research industry and beyond. 

One of the key regulations in the U.S. is 21 CFR Part 11, which outlines the FDA’s guidelines for using electronic records and signatures. This regulation requires the generation of audit trails that record the date and time of entries and actions that create, modify, or delete electronic records to ensure traceability. It also states that audit trails must be securely stored to prevent alteration or deletion and retained for the required regulatory period to support data integrity. 

Audit trail records must also be available for inspection in a human-readable format and be regularly reviewed to ensure compliance and data reliability. The FDA's guidance on Electronic Source Data in Clinical Investigations further emphasizes that organizations using electronic records and signatures must comply with FDA guidelines to ensure data security, traceability, and authenticity.

Similarly, the International Council for Harmonisation's ICH E6(R3) Good Clinical Practice (GCP) guidelines emphasize the necessity of audit trails within electronic data systems to ensure data integrity. The European Medicines Agency (EMA) has also issued guidance on computerized systems and electronic data in clinical trials. Their directive stresses the importance of robust audit trails to demonstrate the authenticity and traceability of data. Finally, the Medicines & Healthcare products Regulatory Agency (MHRA) issued their GxP guide on data integrity in 2018, which emphasizes the importance of audit trails in demonstrating the integrity of electronic data. 

Collectively, these guidelines and regulations underscore the importance of ensuring that robust audit trails are maintained in clinical trials, ultimately ensuring trial data integrity.

Challenges in Audit Trail Review

There are several challenges in implementing effective audit trail review in modern clinical trials. These include:

• Data Overload: A massive volume of data generated in modern trials can make it difficult to monitor and review effectively
• Lack of Standardization: Inconsistent data formats can hinder the generation of a uniform audit trail
• Time-consuming Manual Reviews: The manual review process for audit trails is often labor-intensive and slow, making it impractical for handling large volumes of data
• Lack of Expertise in Interpreting Audit Trail Data: ATRs require personnel with specialized knowledge and skills
• Inconsistent Documentation: Variability in how data is documented can lead to gaps or errors in the audit trail, undermining its reliability
• Cross-platform Integration Issues: Integrating audit trails from multiple systems can be technically challenging, inefficient, and prone to errors

Generative AI for Audit Trail Reviews

Generative AI offers the ability to scale processes and expedite data insights. An audit trail, being a massive data source, where there is a need to extract a manageable subset of data to answer a particular question is a perfect use case for the implementation of AI to help extract insights from audit data.

Large language models (LLMs), such as ChatGPT and similar tools, let users ask targeted questions about data entries, changes within a specific timeframe, modifications to critical data variables, query resolutions, and more. AI can also expedite anomaly detection, flagging unusual access patterns and potential security breaches.

Additionally, AI-powered tools can help generate refined datasets based on quick user queries, which can then be integrated with advanced analytics. These analytics can be structured into repeatable templates to continuously monitor trends, such as unexpected data modifications, site-level discrepancies, or unusual patient data entries.

What would this look like? Let’s use a few examples 

1. Tracking critical data changes 
   1. AI subsets audit trail data to focus only on changes to critical data variables
   2. This refined dataset is then used to set up ongoing monitoring via site-level key risk indicators (KRIs)
   3. The system tracks data change rates and flags sites with significantly higher modification frequencies
   4. Follow-up actions: site re-education, process review, staff training, and further investigation into potential risks
2. Detecting unusual patterns in patient-reported data 
   1. AI subsets eCOA audit data from specific sites flagged due to prior risk signals
   2. Machine learning anomaly detection is applied to identify irregular data entry patterns
   3. Example: A site enters all patient data within an unusually short timeframe, whereas other sites show a more natural spread of data entry
   4. Follow-up actions: investigation into potential protocol non-compliance or fraudulent data entry
3. Identifying after-hours data entry for site investigation
   1. AI identifies audit log entries occurring outside regular working hours
   2. Example: A user queries for all data entries after 7 PM EST, revealing a site where multiple patient records were updated at 11 PM
   3. Follow-up actions: site-level review to determine if data entries were conducted in a clinical setting or externally, assessing compliance risks

The Future of Audit Trail Review

In today’s clinical trial landscape, the need for enhanced data oversight tools is more critical than ever. Generative AI has emerged as a powerful solution for optimizing audit trail review. Not all audit trail data holds equal importance, and even with AI assistance, comprehensive forensic analysis is neither required nor expected by regulators. Instead, companies should align their audit review strategy with broader risk-based principles focusing on:

• Critical data that directly impacts patient safety or study integrity
• Potential failure points where data inconsistencies may arise
• Key risk areas that warrant deeper investigation

As the role of audit data expands, its scope extends beyond EDC and eCOA systems, including other platforms and system integration checks. This evolution underscores the growing need for:

• Comprehensive metadata that captures not just user actions but also the rationale behind changes
• Full accessibility to audit metadata for seamless interrogation and analysis
• User education and training to ensure teams can effectively navigate this increasingly complex landscape

Contact us to learn more about how Medidata Clinical Data Studio can help with your Audit Trail Review.