The first study to compare how well different lung cancer risk models select ever-smokers for screening shows that even the best-performing models require refinements to improve their predictive value. The study was conducted by researchers from the National Cancer Institute (NCI) and the American Cancer Society. An analysis of nine risk prediction models in a representative sample of the US population showed there was no consensus on which ever-smokers should be screened or how many, the authors say in a report published online on May 14 in Annals of Internal Medicine. At a 5-year risk threshold of 2.0%, the screening populations chosen by the different models ranged from 7.6 million to 26 million ever-smokers, say Hormuzd A. Katki, PhD, of the Division of Cancer Epidemiology and Genetics, at the NCI in Bethesda, Maryland, and colleagues. Four risk models were more accurate than the others for predicting risk and for selecting ever-smokers for screening. For these models, there was closer agreement on the size of the screening population (7.6 million to 10.9 million), and there was also agreememt on 73% of the ever-smokers chosen. These four models were the Bach model, the Ovarian Cancer Screening Trial Model 2012 (PLCO-M2012), the Lung Cancer Risk Assessment Tool (LCRAT), and the Lung Cancer Death Risk Assessment Tool (LCDRAT). They "had the highest discrimination overall, the highest sensitivity at a fixed specificity, and vice versa, and similar discrimination at a fixed risk threshold," the researchers write. "These observations indicate that any of these models could be used to select US smokers who are at the greatest risk for lung cancer incidence or death. Each of these models has been validated in external cohorts," they add. They note that their findings may be used to inform future guidelines. Although the results confirm findings from previous studies demonstrating superior performance of the PLCO-M2012 and the Bach models, the study authors emphasize that "the models should be further refined to improve their performance in certain subpopulations." For instance, the Bach model did not account for race/ethnicity, family history of lung cancer, or presence of chronic obstructive pulmonary disease. As a result, it either underestimated or overestimated who should be screened. The PLCO-M2012 model underestimated lung cancer risk in people of Hispanic descent by a factor of 2 to 3, and the LCRAT and LCDRAT models both underestimated risk in the "Asian/other" subgroup. There are currently no risk thresholds for lung cancer screening, the researchers point out. The 2018 lung cancer screening guidelines from the National Comprehensive Cancer Network confirm that individualized risk models can be used to select ever-smokers for screening. "There is growing recognition that, rather than selecting smokers for screening by using simple dichotomized risk factors, individualized risk calculations that account for certain demographic, clinical, and smoking characteristics could substantially enhance the effectiveness and efficiency of CT screening programs," the study authors say. Issue Is "Rapidly Evolving" In an accompanying editorial, Martin C. Tammemägi, PhD, of Brock University in St. Catharines, Ontario, Canada, points out that lung cancer screening "is rapidly evolving" and "likely to improve" as a number of problems are addressed. "First, we must convince policymakers to accept the use of models to identify screening-eligible persons," he writes. He points out that the US Preventive Services Task Force (USPSTF) and the Centers for Medicare & Medicaid Services do not recommend using model-estimated risk. "The 2018 guidelines from the National Comprehensive Cancer Network are the first to move in this direction and have approved selection based on the PLCO-M2012," he writes. Looking ahead, Tammemägi suggests that policy makers might be convinced by results from prospective studies, such as the International Lung Screen Trial, which will compare screening based on the PLCO-M2012 model of 1.5% or greater with screening using USPSTF risk criteria. "In Canada, Cancer Care Ontario has gone one step further," Tammemägi adds. "The Lung Cancer Screening Pilot for People at High Risk (HR_LCSP) is evaluating how best to implement a province-wide lung cancer screening program and is enrolling persons for screening on the basis of PLCO-M2012 risk of 2% or greater." Screening Smokers Can Save Lives Lung cancer screening is critically important and "has been shown to have the potential to save lives when applied to smokers," said Lecia V. Sequist, MD, MPH, director of the Center for Innovation in Early Cancer Detection, Massachusetts General Hospital, in Boston, when asked to comment. However, she told Medscape Medical News, "the US is doing a suboptimal job implementing lung cancer screening in practice for a complicated host of reasons including but not limited to cost, lack of infrastructure, a complex health system, and nihilism and stigma about lung cancer." Studies such as the current one are important to determine whether "there is a more defined subpopulation on which we should focus intense screening effort," said Sequist, who is also associate professor of medicine at Harvard University, Boston. "Interpreting the results of modeling studies designed with built-in assumptions can be difficult, she added. "Small changes in model design can result in big differences in the output, as these authors show." Bottom line? "This study does not change the current clinical recommendations for lung cancer screening," said Sequist. "At the current time, our best evidence for lung cancer screening is derived from prospective clinical trials, which compel us to screen those over 55 years of age with at least a 30 pack-year smoking history." When approached for comment, Rohit Kumar, MD, associate professor in the Department of Medicine, Fox Chase Cancer Center in Philadelphia, Pennsylvania, said that using risk prediction models "seems to be the better way to enrich the screening population." He noted that the best-performing models were developed in the United States and that the current study had some useful findings, including the fact that the PLCO-2012 model underestimated risk in patients of Hispanic descent. "It is valuable to note that 17% of those qualifying by USPSTF criteria were chosen by none of the models," he pointed out. "Thus, validating models in large national samples, as done in this study, is important and proves that further improvements in the way we assess risk is needed," Kumar told Medscape Medical News. Study Details For the study, the investigators focused on risk prediction models that used 2010-2012 data from the National Health Interview Survey to estimate the number of US persons eligible for screening. To evaluate performance in each prediction model, they used data from two large US cohorts. The first cohort was made up of 337,388 ever-smokers in the National Institutes of Health-AARP (formally the American Association of Retired Persons) Diet and Health Study. The second cohort included 72,338 ever-smokers in the CPS-II (Cancer Prevention Study II) Nutrition Survey cohort. The researchers measured the ratio of model-predicted cases to observed cases to determine model calibration. They used area under the curve (AUC) to determine discrimination. The Bach model, the PLCO-M2012, the LCRAT, and the LCDRAT models had the highest sensitivity at a fixed specificity, and vice versa. Discrimination was similar at a fixed risk threshold. The expected-observed ratio ranged from 0.92 to 1.12, and the AUCs ranged from 0.75 to 0.79. This was higher than the Spitz model, the Liverpool Lung Project (LLP) model, the LLP Incidence Risk Model, the Hoggart model, and the Pittsburg Predictor. These five models overestimated risk; the expected-observed ratio range was from 0.83 to 3.69. For these models, the AUCs were also lower, ranging from 0.62 to 0.75. Source
