Foundations of AI for Early Detection of Lung Cancer: Balancing Cost, Efficacy, and Patient Engagement

Lucem and EarlySign recently hosted an insightful roundtable discussion called “Foundations of AI for Early Detection of Lung Cancer.” This roundtable brought together a diverse group of experts, including physicians, data scientists, and clinical researchers. These senior leaders from Lucem Health and Medial EarlySign joined forces to explore crucial aspects of developing AI models for early lung cancer detection. In the first part of this series, our panel of experts examined:

• Data modalities and feature engineering for machine learning model training
• Complexities and constraints in developing robust models with heterogeneous real-world datasets
• Rigorous validation protocols and performance metrics for clinical predictive models in early disease detection

In this subsequent part of the blog series our panel examines the fiscal impact and clinical efficacy of AI-augmented screening protocols, sheds light on the concept of model drift and its implications on longitudinal model performance and explores strategies for optimizing patient engagement and adherence to lung cancer screening programs.

Our panelists for the roundtable include:

• Eran Choman, Chief Clinical Research and Compliance Officer, Medial EarlySign
• Coby Metzger, MD, Data Scientist, Medial EarlySign
• Ben Glicksberg, PhD, Chief Data Science Advisor, Lucem Health
• Andrei Petrus, Senior Manager of AI Solutions, Lucem Health
• Moderator: Marilyn Agbeko, Product, Lucem Health

You can access the recording of the discussion, here.

 

Financial and Clinical Benefits of Improved Screening Compliance

One of the key advantages of AI-enabled technologies in lung cancer screening is its potential to improve compliance rates for screening, which has both financial and clinical benefits. This aspect of AI implementation represents a significant stride towards more effective preventive care and resource allocation in healthcare systems.

Eran Choman provides valuable insights into the financial implications of improved screening compliance: “We see the improvement in compliance… which is also associated with the budget impact from both a revenue perspective as well long-term cost savings.” This statement underscores the dual benefit of AI-driven screening programs – not only do they enhance patient care, but they also offer substantial economic advantages to healthcare organizations.

Elaborating on the financial aspect, Choman explains, “Our research shows that with prioritized screenings, health systems are breaking even between the expenses and the cost savings associated. After the third year, they are realizing significant savings and avoiding unnecessary costs for the organization.”

Seeing that initial investments in AI technology can yield substantial returns over time, the cost savings associated with improved screening compliance stem from several factors. Primarily, early detection of lung cancer through prioritization and engaging patients in the screening process allows for intervention at earlier stages of the disease. As Choman notes, “This improvement in compliance not only reduces healthcare costs associated with late-stage lung cancer treatment but also contributes to earlier detection and better patient outcomes.”

Andrei Petrus also notes, “A majority of cancer is diagnosed at stage 3 and stage 4 where the chances of five-year survival range between 10 and 30%. Whereas stage 1 and stage 2 cancer, the chance of survival is between 50 and 80%. So, by shifting the diagnosis into stage one and stage two when the disease is more addressable, when the quality of life can be drastically improved and when you can save so many lives, providers create a huge impact on the quality of life, on the productivity of that individual in their safe return into the workforce. That is something that goes way beyond the clinical and financial aspects that we model for the provider and create downstream benefits to society at large.”

 

Navigating Model Drift and Ensuring Long-term Accuracy

The panelists also address the critical issue of model drift, emphasizing the need for continuous evaluation and adaptation of AI models in lung cancer screening. This topic is crucial for maintaining the long-term effectiveness, accuracy, and reliability of AI-driven healthcare solutions.

Andrei Petrus provides a comprehensive explanation of the phenomenon: “Models don’t drift on their own… The data itself and the population changes over time.” This statement underscores the dynamic nature of both the data used to train AI models and the populations they serve. Petrus goes on to outline three main types of drift that can affect AI models in healthcare:

1. Data drift: This occurs when the distribution of input data changes over time. For example, if smoking habits in a population change significantly, the input data for lung cancer risk models would shift.
2. Prior probability shift: This refers to changes in the prevalence of the target variable, such as the overall incidence of lung cancer in a population.
3. Concept drift: This happens when the relationship between input features and the target variable changes. For instance, new research might reveal different risk factors for lung cancer, altering the conceptual framework of the model.

He additionally touches on the practical implications of these drift types: “Consider an example where in the next few years we have an update of the coding system. At that point, we can retrain the model and see how it performs for various pathologies of cancer.” This example illustrates the need for regular model updates to account for changes in medical coding practices, diagnostic criteria, or treatment protocols.

The dialogue pivots to the technical challenges of model maintenance, specifically addressing methodologies to combat drift-induced performance attenuation. The panel agrees that implementing a robust monitoring system is essential. This system should track key performance indicators of the AI model, such as its predictive accuracy, sensitivity, and specificity, over time. Any significant deviations from expected performance should trigger a review and potential retraining of the model.

Building on this foundation of continuous monitoring, there are key considerations for data in maintaining model performance – highlighting the need for diverse, representative datasets that capture the full spectrum of patient demographics and risk factors. This diversity helps ensure that models remain accurate across different population subgroups and can adapt to changing demographics over time.

As the discussion progresses from data considerations to broader implications, the conversation naturally turns to the regulatory landscape surrounding AI in healthcare. The panel explores the increasing need for clear guidelines on how often models should be evaluated, updated, and potentially recertified for clinical use. A well-constructed regulatory framework would need to balance the need for model stability with the imperative to incorporate new medical knowledge and adapt to changing population characteristics.

 

Cultural Sensitivity and Patient Engagement

Shifting from the technical considerations of model drift and performance, the panel turns its attention to equally critical aspects of AI-driven lung cancer screening programs: cultural sensitivity and patient engagement. This part of the discussion underscores that the success of these advanced technologies hinges not just on their technical capabilities, but also on their patient-facing ability to resonate with diverse patient populations and motivate action.

Coby Metzger, MD, succinctly captures the essence of this challenge: “You could have the best model in the world to identify those patients at highest risk. But, if the patient doesn’t come for the pre-screening, then there is no benefit.” This statement underscores a fundamental truth in healthcare: technological advancements, no matter how sophisticated, are only as effective as their implementation and patient uptake.

He further addresses the complexity of patient engagement: “Invoking the cooperation of the patient is an art form itself…Clinical leaders and care managers have studied for years how people will respond to various kinds of outreach.” This insight highlights the need for health systems to create a nuanced, culturally informed approach to patient communication and outreach. It’s not enough to simply notify patients of their risk status; healthcare providers must consider how to frame this information in a way that influences and resonates with diverse cultural backgrounds and motivates positive action.

Expanding on this notion of tailored patient engagement requires an exploration into the potential of AI-enabled personalized outreach strategies. By analyzing not just medical data but also socioeconomic factors, cultural indicators, and past healthcare interactions, AI systems could potentially tailor outreach methods and language to individual patients. For example, some patients might respond better to text messages, while others might prefer phone calls or in-person conversations with community health workers.

Having touched on the nuances of personalized patient outreach Eran Choman examines an essential element of AI adoption in medical practice: assessing outcomes and defining measurable goals.

 

Measuring Impact and Setting Benchmarks

Choman kicks off the discussion into measuring the impact of AI in lung cancer screening –highlighting the critical importance of establishing clear, quantifiable metrics to assess the effectiveness of these advanced technologies. “The higher compliance that we get, the better results there are,” underscores a fundamental principle (and challenge) in preventive healthcare: increased participation in screening programs leads to improved outcomes.

Elaborating on this point of providing specific benchmarks: “We are expecting that a new quality index will be introduced in 2025 associated with compliance rate of screening for lung cancer.” This anticipated quality metric reflects the growing recognition of screening compliance as a crucial factor in lung cancer care and prevention. By setting concrete targets for screening rates, healthcare systems can more effectively measure the impact of AI-driven interventions.

While compliance rates offer one measure of success, Choman further expands on the multifaceted approach to evaluating AI’s impact. He emphasizes the importance of demonstrating increased yield compared to standard criteria, aiming for “somewhere between six to eight times higher elevated risk within the USPSTF criteria for the flagged population by using the model.” This ambitious goal highlights the potential of AI to significantly enhance the precision of risk stratification in lung cancer screening, moving beyond mere participation to focus on the accuracy and efficiency of the screening process itself.

Andrei Petrus chimes in by adding an important perspective on the dynamic nature of these benchmarks: “As models evolve and adapt to changing populations, our benchmarks need to be regularly reassessed and updated.” This insight demonstrates the need for flexible, adaptive measurement frameworks that can keep pace with technological advancements and shifting healthcare landscapes.

Petrus’s insight into dynamic benchmarking sets the stage for a related, yet broader consideration – the crucial role of standardization in assessing AI’s effectiveness. The panel agrees that developing industry-wide standards for evaluating AI in lung cancer screening would facilitate more meaningful comparisons across different healthcare systems and AI models.

 

Conclusion

In conclusion, the application of AI in lung cancer detection presents both immense opportunities and complex challenges. From ensuring model clarity and utility to addressing compliance and engagement issues, the integration of AI in healthcare demands a comprehensive approach. As AI continues to evolve as a vital tool in the fight against lung cancer, the insights from this discussion illuminate the path forward, combining technological innovation with patient-centered care. While challenges remain, the potential for improved outcomes, cost savings, and ultimately, lives saved, makes this a promising avenue for future improvements in healthcare.

Stay tuned for part three of this blog series where panelists will share insights on:

• Equity considerations and algorithmic fairness in the context of population-based screening initiatives
• The evolving dynamics between artificial intelligence and clinical practitioners in diagnostic processes
• Quantifiable operational efficiencies and resource optimization achieved through the integration of AI-driven screening methodologies