Summary

Objective: The use of molecular pathology is critical in diagnostics and theranostics. Today, cytological smears are utilized for molecular testing more often than ever. Accurate tumor cell percentage estimation is essential for reliable molecular testing, but its consistency remains uncertain. This study evaluates the reliability of tumor cell percentage estimations among an expert cytopathologist, a molecular cytopathologist, and a molecular pathologist.

Material and Methods: Digital images from May-Grünwald-Giemsa (MGG)-stained smear slides of ten EBUS-guided mediastinal lymph node samples were selected. Five regions per slide were evaluated (50 areas from 10 patients). Three pathologists independently estimated tumor cell percentages using predefined categories (0–10%, 11–20%, 21–50%, etc.). Cells were also counted manually as the gold standard.

Results: The molecular cytopathologist (Observer 1 -) showed the highest consistency (Kappa = 0.69), followed by the expert cytopathologist (Observer 3 -, Kappa = 0.64), both demonstrating substantial agreement with the gold standard. The molecular pathologist (Observer 2 -) displayed moderate consistency (Kappa = 0.52). Agreement was most significant in the 71–100% category, aligning in over 95% of cases. The lowest value occurred in the 11–20% category. In this category, tumor proportions were frequently overestimated compared to the gold standard.

Conclusion: Variability in tumor percentage estimations shows the need for standardized protocols and training. Substantial agreement was reached in specific categories. However, discrepancies in borderline cases highlight the importance of accurate assessments. More research is needed to improve estimation methods.

Introduction

Precision medicine has changed how we approach cancer patients. Molecular pathology is central in identifying genetic alterations that can be targeted therapeutically. Next- Generation Sequencing (NGS) has made it much easier to analyze tumors` genetic and transcriptomic features. Still, its utility largely depends on factors like tumor cell ratio and specimen quality. Detecting actionable mutations in non-small cell lung cancer (NSCLC) is a classical example[1]. Introducing tyrosine kinase inhibitors has changed how we evaluate and treat lung cancer patients. However, the effectiveness of these targeted therapies depends on correctly identifying these alterations. This can be challenging especially in specimens with low tumor content[2,3].

Most metastatic NSCLC cases are diagnosed based on endoscopic ultrasound (EBUS) -guided mediastinal lymph node sampling[4]. Because of the technique`s inherent difficulties, cytological material usually harbors lower tumor content and ratio than surgical resection specimens. Despite these disadvantages, cytological samples are used in NGS-based studies, and guidelines recommend cytological material with sufficient tumor cell content[5-7].

Although the importance of tumor content in molecular pathology tests is well recognized, there is still limited data on the consistency and accuracy of these estimations in cytologic samples. Our study aims to evaluate the consistency of tumor cell percentage estimations among cytopathologists and surgical pathologists and assess the impact of these estimations on molecular testing. We aim to contribute to methodologies that enhance the reliability of molecular diagnostics and improve patient care.

Methods

Digital Imaging
The study included ten smear slides from NSCLC mediastinal lymph node EBUS samples. Slides were stained with May-Grünwald-Giemsa (MGG) and digitized at 20x magnification using a Philips IntelliSite UltraFast scanner (Koninklijke Philips, Amsterdam, Netherlands). Five distinct areas of each slide (n=50 in total) were captured for analysis. Each captured area measured 700 x 1,000 pixels on average, with slight variations depending on the field of interest.

Pathologist Assessment
Three pathologists independently estimated the tumor cell percentage in each captured area using predefined interval categories: 0–10%, 11–20%, 21–50%, 51–70%, and 71–100%. The first pathologist (observer 1), experienced in molecular pathology and cytopathology, had ten years of experience. The second pathologist (observer 2) specialized in neuropathology and molecular pathology, with over ten years of experience. The third (observer 3) was a senior consultant in cytopathology with over 20 years of experience. All three observers made their initial estimations by visual inspection (`eyeballing`) of digital images, typically spending less than 30 seconds per field.

After these rapid assessments, experienced cytopathologists (observer 3) printed the images and performed detailed manual counts to establish the gold standard. For each field, all intact and evaluable nucleated cells were included. Degenerated, crushed, or morphologically unclassifiable nuclei were excluded. Non-tumor elements such as lymphocytes and background cells were counted in the denominator to maintain consistency.

Tumor proportion was calculated by dividing the number of tumor cells by the total number of intact nucleated cells. On average, 84 cells (range: 18–439) were counted per field, with a median of 71 cells. Tumor percentages ranged from 0.9% to 98.4%, with a mean of 47.1%. These detailed counts served as the reference standard for validating observer estimations.

Statistical Analysis
Cohen`s Kappa (κ) coefficient was used to measure the level of agreement between each observer and the gold standard across categorical tumor proportion estimates. Kappa values were interpreted as follows: <0.20 poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and >0.80 almost perfect agreement. In addition to the Kappa coefficients, corresponding 95% confidence intervals and pvalues were calculated to assess the statistical significance and precision of the agreement. All statistical analyses were performed using IBM SPSS Statistics for Windows, version 28.0 (IBM Corp., Armonk, NY, USA), and a p-value of <0.05 was considered statistically significant. The Kappa results, including confidence intervals and significance values, are presented in detail in the Results section.

Results

Cohen`s Kappa analysis revealed varying levels of agreement between each observer and the gold standard. Observer 1 demonstrated the highest agreement, with a Kappa value of 0.69 (95% CI: 0.532–0.842, p < 0.001). Observer 3 also showed substantial agreement, with a Kappa of 0.64 (95% CI: 0.479–0.797, p < 0.001). Observer 2, a molecular pathologist without cytopathology training, had a moderate level of agreement, with a Kappa of 0.52 (95% CI: 0.328– 0.704, p < 0.001). These results emphasize the impact of cytopathology expertise on tumor proportion estimation in smear slides and support the need for standardized training and consensus-based evaluation in molecular pathology workflows. Among the categories, the highest agreement was observed in the 71–100% category, where all observers (observer 1, observer 2, observer 3) and the gold standard aligned in over 95% of the cases. In contrast, the lowest agreement was noted in the 11–20% category, where differences in tumor proportion estimations were higher than the gold standard, particularly in borderline cases.

Category Analysis
Below are the detailed agreement results between observers after splitting tumor cell ratios (defined by the gold standard) into categories.

0–10% Category
Agreement was high in this category. Observer 1 and Observer 2 matched the gold standard in 5 out of 6 fields (83%), while Observer 3 achieved perfect concordance in all 6 fields (100%). This consistency highlights the ease of classification for low tumor proportions (Figure 1).

Figure 1: Tumor proportion by gold standard: 0–10%. (Observer evaluations: Observer 1: 0–10%, Observer 2: 0–10%, Observer 3: 0–10%).

11–20% Category
Agreement in this category was moderate. Observer 1 and Observer 2 each aligned with the gold standard in 4 out of 6 fields (67%), while Observer 3 matched in 3 out of 6 (50%) (Figure 2A,B).

Figure 2: Tumor proportion by gold standard: 11–20%. (2a Observer evaluations: Observer 1: 21-50%, Observer 2: 11–20%, Observer 3: 11-20%) (2b Observer evaluations: Observer 1: 11–20%, Observer 2: 21–50%, Observer 3: 11–20%).

21–50% Category
Observer 3 showed the highest agreement in this category, matching the gold standard in 13 of 16 fields (81%). Observer 1 and Observer 2 each correctly classified 11 of 16 fields (69%) (Figure 3A,B).

Figure 3: Tumor proportion by gold standard: 21–50%. (3a Observer evaluations: Observer 1: 21–50%, Observer 2: 21–50%, Observer 3: 21–50%) (3b Observer evaluations: Observer 1: 51–70%, Observer 2: 51–70%, Observer 3: 21–50%).

51–70% Category
Agreement was more variable. Observer 1 correctly identified 11 of 15 fields (73%), while Observer 2 and Observer 3 matched the gold standard in 8 (53%) and 7 (47%) fields, respectively (Figure 4A,B).

Figure 4: Tumor proportion by gold standard: 51–70%. (4a Observer evaluations: Observer 1: 71–100%, Observer 2: 51–70%, Observer 3: 71–100%) (4b Observer evaluations: Observer 1: 51–70%, Observer 2: 21–50%, Observer 3: 71–100%).

71–100% Category
This category demonstrated the highest consistency for Observers 1 and 3, both of whom correctly classified all 7 fields (100%). Observer 2 aligned with the gold standard in 4 of 7 fields (57%) (Figure 5A,B).

Figure 5: Tumor proportion by gold standard: 71–100%. (5a Observer evaluations: Observer 1: 71–100%, Observer 2: 71–100%, Observer 3: 71–100%) (4b Observer evaluations: Observer 1: 71–100%, Observer 2: 51–70%, Observer 3: 71–100%).

Lowest Agreement
The lowest agreement was observed in fields with mixed cell types and borderline tumor proportions. An example is Figure 6, where the gold standard categorized the tumor proportion as 11–20%, but Observer 1 assessed it as 0–10%, and Observer 2 and Observer 3 as 21–50%. This variability emphasizes the challenges in ambiguous fields (Figure 6).

Figure 6: Lowest agreement, tumor proportion by gold standard: 11–20%. (Observer 1 evaluations: 0–10%, Observer 2 and Observer 3:21–50%).

Discussion

Estimating tumor cell percentage in samples for molecular testing is a crucial step that directly impacts the reliability of test results. The variation in these assessments across different pathologists highlights the need for standardized methods. This becomes particularly important when the estimated tumor percentage approaches the threshold required for reliable mutation detection, and an overestimation due to insufficient tumor DNA and too much normal DNA contamination from non-tumor cells can lead to false-negative results[2,8,9].

Studies have shown that estimation of tumor ratio may vary between observers[3,10]. This may potentially cause serious problems. Overestimating tumor ratio above 20% may cause a false negativity while underestimation may result in unnecessary re-biopsy[11,12]. These findings suggest that even experienced pathologists could benefit from trainings focused on tumor estimation.

The highest observer agreement was in the 71–100% category, probably because of the ease of recognizing high tumor proportions. On the other hand, the 11–20% category demonstrated the lowest agreement, emphasizing the difficulty in borderline cases. Overestimation in the 11-20% category is another problem as it is the borderline category and has the potential of false negative results.

The observers in our study came from different backgrounds; observer 1, a cytopathologist with experience in molecular pathology; observer 2, a surgical pathologist with experience in molecular pathology; observer 3, a consultant cytopathologist. Observer 1, no surprise, had the highest concordance whereas observer 2 had the lowest. This shows that cytopathology expertise is critical in tumor proportion assessment in cytology samples. Molecular pathology laboratories should include cytopathologists in their workflow in order to achieve better results. Direct communication between molecular pathologists and cytopathologists would also improve tumor ratio estimation, and thus test accuracy.

Pathologists should consider taking a more conservative approach in their estimations, using tissue macrodissection or microdissection techniques when needed[13-15], and staying informed about the sensitivity of the assays performed in their labs. Standardized trainings that focus on cytological sample evaluation, including workshops, could significantly improve interobserver agreement.

The tumor percentage in the existing material is crucial in selecting the appropriate test for molecular analysis. If the tumor percentage is too low for NGS, other options like immunohistochemistry, FISH, or single-gene analysis can be considered. The integration of NGS and digital PCR, holds the potential to overcome some of these issues by detecting mutations in very small fractions of tumor cells[16]. In cases where the number of tumor cells is not adequate, ctDNA analysis using peripheral blood might be an alternative. AI-driven image analysis tools could also help improve consistency.

While this study provides valuable insights, there are some limitations. The gold standard was established through manual cell counting, but distinguishing between certain cells (e.g., small lymphocytes vs. tumor cells) was not always straightforward. This could introduce minor inaccuracies in the gold standard, potentially affecting its reliability as a reference. Notably, the expert cytopathologist who determined the gold standard had a Kappa value of 0.64 in her estimations, showing substantial but not perfect agreement with her own manual counts. This underscores the challenge of achieving absolute precision, even for highly experienced pathologists.

Additionally, the study was limited to 50 fields from 10 cytological slides. While this was enough to capture interobserver variability and agreement trends, a larger dataset with more diverse specimen types would improve the generalizability of these findings. Lastly, although the observers worked independently, their familiarity with everyday challenges in cytological evaluation might have introduced some subtle biases in their estimations.

In fact, for whole-slide assessment or for defining the precise area to be scraped for molecular studies, performing manual counting on a printed version of the selected area would theoretically yield the most accurate results. However, since this is not feasible in routine practice, an alternative approach could be recommended: selecting and printing the area that most closely matches the visually estimated tumor-rich region, and performing manual counting on this printed area similar to the approach used in Ki- 67 evaluation[17]. This practical strategy may help bridge the gap between ideal and real-world workflows.

Future studies should include larger case numbers and a more diverse set of observers to confirm the generalizability of our findings and support the development of standardized evaluation protocols.

Approaches such as multiplexed imaging and AI-based counting systems have the potential to improve the accuracy of tumor proportion assessments significantly. Incorporating these technologies into routine workflows could also help standardize evaluations across different pathology laboratories. Recent advances in artificial intelligence and intense learning-based tools have demonstrated promising results in tumor cell detection and quantification in nongynecologic cytology specimens, offering reproducibility and scalability in diagnostic workflows[18-21]. As these systems become more accessible and better integrated, they may help overcome current challenges related to interobserver variability and subjective estimation.

Acknowledgment
This study was presented as an oral presentation at the 45th European Congress of Cytology in Leipzig, Germany, June 23-26, 2024.

This manuscript was prepared with the assistance of natural language processing tools driven by artificial intelligence (AI) for language refinement and formatting. These tools were used solely to enhance clarity and structure and were not involved in the generation or analysis of data.

Ethical Approval
This study was performed in accordance with the Declaration of Helsinki.

Conflict of Interest
The authors declare that they have no conflict of interest to disclose.

Funding
No funding received.

Authorship Contributions
Concept: CAM, PF, Design: CAM, PF, Data collection or processing: CAM, IK, PF, Analysis or Interpretation: CAM, Literature search: CAM, Drafting: CAM, Critical Review: CAM, IK, PF.

Reference

1) Hendriks LE, Kerr KM, Menis J, Mok TS, Nestle U, Passaro A, Peters S, Planchard D, Smit EF, Solomon BJ, Veronesi G, Reck M; ESMO Guidelines Committee. Electronic address: clinicalguidelines@ esmo.org. Oncogene-addicted metastatic non-smallcell lung cancer: ESMO Clinical Practice Guideline for diagnosis, treatment and follow-up. Ann Oncol. 2023;34(4):339-57. PMID: 36872130. DOI: 10.1016/j.annonc.2022.12.009.

2) Jennings LJ, Arcila ME, Corless C, Kamel-Reid S, Lubin IM, Pfeifer J, Temple-Smolkin RL, Voelkerding KV, Nikiforova MN. Guidelines for Validation of Next-Generation Sequencing-Based Oncology Panels: A Joint Consensus Recommendation of the Association for Molecular Pathology and College of American Pathologists. J Mol Diagn. 2017;19(3):341-65. PMID: 28341590. DOI: 10.1016/j.jmoldx.2017.01.011.

3) Luquain A, Arbez-Gindre F, Bedgedjian I, Felix S, Harimenshi JM, Mihai IM, Monnien F, Singeorzan C, Valmary-Degano S. Habilitation des pathologistes pour l`évaluation du pourcentage de cellules tumorales lors du contrôle morphologique avant extraction d`acides nucléiques [Authorization of pathologists for the estimation of the tumor cell percentage on tissue sample for molecular analysis purpose]. Ann Pathol. 2016;36(4):268-74. PMID: 27474532. DOI: 10.1016/j.annpat.2016.06.004.

4) Sehgal IS, Agarwal R, Dhooria S, Prasad KT, Aggarwal AN. Role of EBUS TBNA in Staging of Lung Cancer: A Clinician`s Perspective. J Cytol. 2019;36(1):61-4. PMID: 30745743. DOI: 10.4103/ JOC.JOC_172_18.

5) Martin-Deleon R, Teixido C, Lucena CM, Martinez D, Fontana A, Reyes R, García M, Viñolas N, Vollmer I, Sanchez M, Jares P, Pérez FM, Vega N, Marin E, Marrades RM, Agustí C, Reguart N. EBUS-TBNA Cytological Samples for Comprehensive Molecular Testing in Non-Small Cell Lung Cancer. Cancers (Basel). 2021;13(9):2084. PMID: 33923116. DOI: 10.3390/cancers13092084.

6) Muriana P, Rossetti F. The role of EBUS-TBNA in lung cancer restaging and mutation analysis. Mediastinum. 2020;4:23. PMID: 35118291. DOI: 10.21037/med-20-24.

7) Stoy SP, Segal JP, Mueller J, Furtado LV, Vokes EE, Patel JD, Murgu S. Feasibility of Endobronchial Ultrasound-guided Transbronchial Needle Aspiration Cytology Specimens for Next Generation Sequencing in Non-small-cell Lung Cancer. Clin Lung Cancer. 2018;19(3):230-8.e2. PMID: 29277564. DOI: 10.1016/j. cllc.2017.11.010.

8) Aziz N, Zhao Q, Bry L, Driscoll DK, Funke B, Gibson JS, Grody WW, Hegde MR, Hoeltge GA, Leonard DG, Merker JD, Nagarajan R, Palicki LA, Robetorye RS, Schrijver I, Weck KE, Voelkerding KV. College of American Pathologists` laboratory standards for next-generation sequencing clinical tests. Arch Pathol Lab Med. 2015;139(4):481-93. PMID: 25152313. DOI: 10.5858/ arpa.2014-0250-CP.

9) Roy S, Coldren C, Karunamurthy A, Kip NS, Klee EW, Lincoln SE, Leon A, Pullambhatla M, Temple-Smolkin RL, Voelkerding KV, Wang C, Carter AB. Standards and Guidelines for Validating Next-Generation Sequencing Bioinformatics Pipelines: A Joint Recommendation of the Association for Molecular Pathology and the College of American Pathologists. J Mol Diagn. 2018;20(1):4-27. PMID: 29154853. DOI: 10.1016/j. jmoldx.2017.11.003.

10) Lhermitte B, Egele C, Weingertner N, Ambrosetti D, Dadone B, Kubiniek V, Burel-Vandenbos F, Coyne J, Michiels JF, Chenard MP, Rouleau E, Sabourin JC, Bellocq JP. Adequately defining tumor cell proportion in tissue samples for molecular testing improves interobserver reproducibility of its assessment. Virchows Arch. 2017;470(1):21-7. PMID: 27853865. DOI: 10.1007/s00428- 016-2042-6.

11) Betz BL, Post CS, Bergendahl J, Lefebvre N, Weigelin H, Brown NA. Optimization of Tumor Dissection Procedures Leads to Measurable Improvement in the Quality of Molecular Testing. J Mol Diagn. 2024;26(10):876-87. PMID: 39067571. DOI: 10.1016/j.jmoldx.2024.06.009.

12) Viray H, Li K, Long TA, Vasalos P, Bridge JA, Jennings LJ, Halling KC, Hameed M, Rimm DL. A prospective, multi-institutional diagnostic trial to determine pathologist accuracy in estimation of percentage of malignant cells. Arch Pathol Lab Med. 2013;137(11):1545-9. PMID: 24168492. DOI: 10.5858/arpa.2012- 0561-CP.

13) Geiersbach K, Adey N, Welker N, Elsberry D, Malmberg E, Edwards S, Downs-Kelly E, Salama M, Bronner M. Digitally guided microdissection aids somatic mutation detection in difficult to dissect tumors. Cancer Genet. 2016;209(1-2):42-9. PMID: 26767919. DOI: 10.1016/j.cancergen.2015.12.004.

14) Kumar G, Timmaraju VA, Song-Yang JW, Repko B, Narick C, Mireskandari A, Finkelstein S. Utility of microdissected cytology smears for molecular analysis of thyroid malignancy. Diagn Cytopathol. 2019;47(4):289-96. PMID: 30548138. DOI: 10.1002/ dc.24100.

15) Marchetti I, Iervasi G, Mazzanti CM, Lessi F, Tomei S, Naccarato AG, Aretini P, Alberti B, Di Coscio G, Bevilacqua G. Detection of the BRAF(V600E) mutation in fine needle aspiration cytology of thyroid papillary microcarcinoma cells selected by manual macrodissection: an easy tool to improve the preoperative diagnosis. Thyroid. 2012;22(3):292-8. PMID: 22181337. DOI: 10.1089/ thy.2011.0107.

16) Altimari A, de Biase D, De Maglio G, Gruppioni E, Capizzi E, Degiovanni A, D`Errico A, Pession A, Pizzolitto S, Fiorentino M, Tallini G. 454 next generation-sequencing outperforms allelespecific PCR, Sanger sequencing, and pyrosequencing for routine KRAS mutation analysis of formalin-fixed, paraffin-embedded samples. Onco Targets Ther. 2013;6:1057-64. PMID: 23950653. DOI: 10.2147/OTT.S42369.

17) Reid MD, Bagci P, Ohike N, Saka B, Erbarut Seven I, Dursun N, Balci S, Gucer H, Jang KT, Tajiri T, Basturk O, Kong SY, Goodman M, Akkas G, Adsay V. Calculation of the Ki67 index in pancreatic neuroendocrine tumors: a comparative analysis of four counting methodologies. Mod Pathol. 2015;28(5):686-94. PMID: 25412850. DOI: 10.1038/modpathol.2014.156.

18) Kim D, Sundling KE, Virk R, Thrall MJ, Alperstein S, Bui MM, Chen-Yost H, Donnelly AD, Lin O, Liu X, Madrigal E, Michelow P, Schmitt FC, Vielh PR, Zakowski MF, Parwani AV, Jenkins E, Siddiqui MT, Pantanowitz L, Li Z. Digital cytology part 2: artificial intelligence in cytology: a concept paper with review and recommendations from the American Society of Cytopathology Digital Cytology Task Force. J Am Soc Cytopathol. 2024;13(2):97- 110. PMID: 38158317. DOI: 10.1016/j.jasc.2023.11.005.

19) Kim D, Sundling KE, Virk R, Thrall MJ, Alperstein S, Bui MM, Chen-Yost H, Donnelly AD, Lin O, Liu X, Madrigal E, Michelow P, Schmitt FC, Vielh PR, Zakowski MF, Parwani AV, Jenkins E, Siddiqui MT, Pantanowitz L, Li Z. Digital cytology part 1: digital cytology implementation for practice: a concept paper with review and recommendations from the American Society of Cytopathology Digital Cytology Task Force. J Am Soc Cytopathol. 2024;13(2):86-96. PMID: 38158316. DOI: 10.1016/j. jasc.2023.11.006.

20) Hays P. Artificial intelligence in cytopathological applications for cancer: a review of accuracy and analytic validity. Eur J Med Res. 2024;29(1):553. PMID: 39558397. DOI: 10.1186/s40001-024- 02138-2.

21) Giarnieri E, Scardapane S. Towards Artificial Intelligence Applications in Next Generation Cytopathology. Biomedicines. 2023;11(8):2225. PMID: 37626721. DOI: 10.3390/biomedicines11082225.