Material and Method: A total of 1.500 randomized ThinPrep Pap tests that were screened manually and archived in 2015 were reviewed by a blinded researcher manually and by the ThinPrep automatic method.
Results: There was a 83.3% increase in the detection of ASCUS (Atypical squamous cells of undetermined significance) with the TPAM compared to the TPMM, and with respect to the reference results, the accuracy was higher for the TPAM than for the TPMM. We also noted a 33.3% increase in the rate of LSIL (Low grade squamous intraepithelial lesion) and 20% increase in the rate of HSIL (High grade squamous intraepithelial lesion) by the TPAM. Concordance was best between the TPAM and reference cytologic diagnoses. The sensitivity was higher for the TPAM and the specificity was similar for both methods. The false positive rate was higher for the TPAM than the TPMM but the false negative rate was higher for the TPMM. We determined a 30% gain in screening time per smear by the TPAM. However, rejection of many samples by the system, especially because of air bubbles, was a limitation of the TPAM.
Conclusion: The TPAM has advantages over the TPMM as well as disadvantages such as limiting features and a high false positive rate. The TPAM should be supported by the manual method to decrease the false positive rate.
The Food and Drug Administration has approved many liquid-based cytology techniques in recent decades [3]. These methods differ from classical Papanicolaou smears in many aspects. ThinPrep (HOLOGIC) is one such method, and the ThinPrep Imaging System (TIS) is an automated imaging and review system that selects 22 fields of view from the samples presented to it. The manufacturers of the TIS particularly note that the system aims to help cytopathology specialists by highlighting areas of a slide for further manual review and that the product is not intended as a replacement for manual review [4]. When compared with conventional cytology, the relative true positive and false positive rates are 1.13 and 1.12, respectively. This indicates that the ThinPrep test has a high sensitivity but low specificity [5]. This technique is more expensive than the classical Pap smear test; however, since it is a more sensitive test, the requirement of longer between-screening intervals balances its higher cost. Liquid-based cytology can decrease the rate of false negative results arising from errors made during screening tests and from interpretation of cytological test results [6].
In the present study, we aimed to compare 1500 randomized ThinPrep slides by using the TPMM and the TPAM. Our study differs from other similar studies by using the same ThinPrep Pap smear samples to compare the two methods and the same samples for reference screening.
Study Design
A total of 1500 ThinPrep slides that were randomly
selected and were then screened and diagnosed manually
by 7 different researchers and archived in 2015 at the
Gaziosmanpasa Taksim Education and Research Hospital
were included in the study. The Pap tests were rescreened
manually (TPMM) then pre-screened on a ThinPrep 2000
device (Cytyc Corporation, Boxborough, MA, USA) and
rescreened again only for 22 fields of view on a TIS
microscope at x10 magnification (TPAM) by the same
blinded, inexperienced researcher (assistant doctor) to
compare the diagnostic results and the positive/negative
sides of the two methods.
ThinPrep Imaging System
A barcode that includes the protocol number and the year
data was placed on each ThinPrep slide and prescreened
by the same inexperienced researcher from the above
section with the TIS. Eventually, data from 22 fields of
view for each slide were obtained by and stored on the
system. During that process, the system refused to screen
some of the slides for a variety of reasons, such as bubble
artifacts. Approximately 300 pap smears that were not
recognized or refused to be read by the device were kept in
a xylene solution to remove the probable artifacts following
careful replacement of their cover slips. Subsequently,
approximately 70 pap smears were refused by the device
again and therefore excluded from the study. Another 70
new pap smears were randomly selected from the database
that was archived in 2015 and included in the study (this
preserved the original number of slides as n=1500).
ThinPrep Manual Method
Pap smears were rescreened on a TIS microscope by
the TPMM by the same inexperienced researcher. Each
day, an average of 100 slides were reviewed. Screening
was performed without knowledge of the individual
histopathologic results. While screening, cases were grouped
into reproductive, premenopausal or perimenopausal and menopausal categories. Slides were classified according to
the Bethesda 2014 classification.
ThinPrep Automated Method
Pap smears were rescreened on a TIS microscope at
x10 magnification over 22 fields of view by the same
inexperienced researcher. Each day, an average of 130
slides were reviewed. Screening was performed without
knowledge of the first pathologic results or the results
obtained by the TPMM. Slides were classified according to
the Bethesda 2014 classification.
Reference Diagnosis
A total of 91 cases that were diagnosed as Epithelial cell
anomaly on the original cytopathology reports from
2015 or by the TPMM or the TPAM were screened again
by an experienced consultant doctor (reference), and the
new diagnoses were accepted as the gold standard. For the
cases that were diagnosed with No intraepithelial lesion or
malignancy (NILM) on the original cytopathology reports
from 2015 or by the TPMM or the TPAM, the original
cytopathology diagnoses from 2015 were accepted as the
gold standard.
Statistical Analysis
Data were reported as percent values as appropriate.
Group comparisons were performed using Fleiss kappa
and Krippendorffs alpha tests. If the same results were
obtained, Cohens kappa was calculated to determine the
concordance between two tests. A two-sided p-value <0.05
was considered statistically significant. The reference values
for the concordance between all three tests were as follows:
Unimportant (0.20), Low (=0.20 - 0.40), Median (=0.40 -
0.60), Important (=0.60 - 0.80), and Very important (=0.80
- 1.00). For optimal statistical dual and triple concordance
analyses between the results of the screening methods
and the references, we excluded two cases that were
diagnosed as Atypical glandular cell (AGC) only by the
TPMM and three cases that were diagnosed as Atypical
squamous cells, cannot exclude HSIL (ASC-H) only by
the TPAM. Additionally, four cases that were diagnosed
as unsatisfactory for evaluation by the two methods were
not included in any comparison analysis.
The breakdown diagnoses was 1455 (97%) NILM, 14 (0.9%) LSIL, 7 (0.5%) HSIL, 18 (1.2%) ASCUS and 2 (0.1%) ASC-H by the TPMM; 1432 (95.5%) NILM, 19 (1.2%) LSIL, 9 (0.6%) HSIL, 33 (2.2%) ASCUS and 3 (0.2%) ASC-H by the TPAM; and 1440 (96%) NILM; 31 (2.1%) LSIL, 6 (0.5%) HSIL and 19 (1.3%) ASCUS by the reference method.
The ASC/SIL ratio was < 2 for both methods (TPMM=0.9 vs. TPAM=1.2).
Diagnostic analysis showed that 26 cases diagnosed with ASCUS, 6 with LSIL and 1 with HSIL by the TPAM were all diagnosed with NILM by the TPMM (Table IA,B). There was no distinct difference between the LSIL diagnoses for the two methods. However, 7 cases were diagnosed with NILM, 3 with LSIL and 1 with HSIL by the TPAM were all diagnosed with ASCUS by the TPMM. There was an 83.3% increase in the detection of ASCUS (n=33 vs. 18) and a 33.3% increase in the detection of LSIL (n=19 vs. 14) and a 20% increase in the detection of HSIL (n=9 vs. 7) by the TPAM versus TPMM (Figure 1). With respect to the reference results, the accuracy was higher for the TPAM than the TPMM (Table IIA,B, Table IIIA,B).
Table IA: Diagnostic comparison between the TPMM and the TPAM.
Table IB: Symmetric measures of the cross-tabulation analysis for Table IA.
Table IIA: Diagnostic comparison between the TPMM and reference.
Table IIB: Symmetric measures of the cross-tabulation analysis for Table IIA.
Table IIIA: Diagnostic comparison between the TPAM and reference.
Table IIIB: Symmetric measures of the cross-tabulation analysis for Table IIIA.
Concordance was worst between the TPAM and TPMM diagnoses (p<0.05 and kappa value=0.495) and best between the TPAM and reference cytologic diagnoses (p<0.05 and kappa value=0.631) (Table IV-VI).
Table IV: Diagnostic concordance between the TPAM and the TPMM.
Table V: Diagnostic concordance between the TPMM and reference.
Table VI: Diagnostic concordance between the TPAM and reference.
The reference diagnoses accepted as the gold standard were used to calculate sensitivity, specificity, and the false positive and false negative rates. The sensitivity was higher for the TPAM (for TPMM=62.5% vs. for TPAM=80%), and the specificity was similar for both methods (for TPMM=99.6% vs. for TPAM=98.7%). The false positive rate was higher for the TPAM than the TPMM (for TPMM=0.4% vs. for TPAM=1.3%), but the false negative rate was higher for the TPMM (for TPMM=37.5% vs. for TPAM=20%) (Table VII,VIII).
Table VII: Rate-calculating template for the TPMM.
Table VIII: Rate-calculating template for the TPAM.
We determined a 30% decrease in screening time per smear by the TPAM. However, the rejection of many samples by the system, especially because of air bubbles, is a limitation of the TPAM.
It has been shown that the number of atypical smear diagnoses increase when using TIS [5,6]. In studies that compared the TPMM and the TPAM, similar results for ASCUS, LSIL and HSIL diagnoses were noted [7-9]. In the present study, we noted a 33.3% increase in the rate of LSIL and a 20% increase in HSIL by the TPAM compared to the TPMM, but with respect to the reference results, the false positive rate was higher for the TPAM than the TPMM. As a result, the TPMM seems to be more valuable than the TPAM in this respect. Additionally, there was an 83.3% increase in the detection of ASCUS with the TPAM compared to the TPMM; however, with respect to the reference results, accuracy was higher for the TPAM than the TPMM. The concordance was best between the TPAM and reference cytologic diagnoses. The low concordance between the TPMM and reference diagnoses may be related to the evaluations being performed by an inexperienced and an experienced pathologist, respectively. In the present study, in two cases diagnosed as ASCUS by the TPAM, candida was detected by TPMM. Therefore, screening only with 22 fields of view can be a reason for the increased false positive rate for TPAM. The ASC/SIL ratio is a quality control method for gynecologic cytology results and is typically between 2-3. The CAP (College of American Pathologists) identifies the median of the ASC/SIL ratio as 1.7. In our study, the ratio was < 2 for both methods. Similar studies have reported a range of values between 0.74 and 2.25 [7,8,10,11]. Satisfactoriness is the only quality control method of the Bethesda System. The unsatisfactory for evaluation rate was 0.3% for both methods in our study. In the literature, varying results of this rate have been noted (between 0.3% and 3%) [7,8,12-16). Studies in the literature mainly consist of a great number of cases but used Pap smears screened by different cytotechnologists at different times and which were rescreened by different cytotechnologists to compare the different methods [7,8,17]. Our study differentiates from the others by using the same ThinPrep Pap smear samples to compare the two methods and to perform reference screening.
Renshaw and Elsheikh investigated the correlation between the sensitivity for HSIL in the TIS and the epithelial cell abnormality (ECA)-adjusted workload and showed that the performance of the TIS at the threshold for HSIL and above was negatively correlated with the ECA-adjusted workload [9]. Kitchener et al. argued that monotony could have been a contributing factor in reduced vigilance while screening by the TPAM [13]. Lozano et al. mentioned that automated screening causes cytotechnology operators to be more fastidious in their analyses [8]. In the present study, we reviewed 12 smears per hour with the TPMM and 16 smears per hour with the TPAM.
The most significant methods of clinical efficiency in screening are specificity and sensitivity. In the literature, some of the studies accepted follow-up biopsies or TPMM results performed in different years as a reference and reported differing results regarding sensitivity and specificity [12,13,18]. In the present study, sensitivity was higher for the TPAM than the TPMM, and specificity was similar for both methods. Despite of screening with only by 22 fields of view with the TPAM, the sensitivity and specificity values were high for the TPMM. Additionally, the high false positive rate for the TPMM compared to the TPAM and the similar results for accuracy and negative predictive value between the two tests favor the use of the TPAM.
Colakkadioglu and Erkilic stated that smear slides were rejected while screening with the automated system because of the presence of blood, a small number of cells and air bubbles [19]. In the present study, we excluded a total of 70 slides from the study mainly because of the presence of air bubbles. It must be emphasized that smear rejection by the device is an important limitation of TIS.
We are aware that there are clear limitations in the case series presented here. The main limitation of our study was its retrospective design. The second limitation is that the results were restricted to the outcomes from a single institution. Third, some details of the patients history and factors that may influence the outcome may not have been completely documented. Due to these restrictions, the associations presented here should be interpreted with caution.
In conclusion, the TPAM has advantages over the TPMM as well as disadvantages such as limiting features and a high false positive rate. The TPAM should be supported by the manual method to decrease the false positive rate.
CONFLICT of INTEREST
The authors declare no conflict of interest.
Copyright © 2020 The Author(s). This is an open-access article published by Federation of Turkish Pathology Societies under the terms of the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.
1) World Health Organization. Comprehensive Cervical Cancer
Control: A guide to essential practice 2nd ed. (Electronic
Version). 2014. 27.
2) Demir F, Kimiloglu E, Igdem AA, Ayanoglu YT, Erdogan
N. High risk HPV in situ hybridization, p16 INK 4A, and
survivin expressions in cervical carcinomas and intraepithelial
neoplasms: Evaluation of prognostic factors. Eur J Gynaecol
Oncol. 2014;35:708-17.
3) Rozemeijer K, Penning C, Siebers AG, Naber SK, Matthijsse SM,
van Ballegooijen M, van Kemenade FJ, de Kok IM. Comparing
SurePath, ThinPrep, and conventional cytology as primary test
method: SurePath is associated with increased CIN II+ detection
rates. Cancer Causes Control. 2016;27:15-25.
4) ThinPrep® Imaging System, Review Scope Operators Manual,
HOLOGIC®, MAN-03352-001 rev.003
5) Tripathy K, Misra A, Ghosh JK. Efficacy of liquid-based cytology
versus conventional smears in FNA samples. J Cytol. 2015;32:17-20.
6) Kımıloglu E, Gundogar O, Demir F, Ozcan Z, Cin M, Erdogan
N. Evaluation of HPV and p16 expression in the cervical
intraepithelial neoplasia. J Curr Pathol. 2017;3:53-9.
7) Chivukula M, Saad RS, Elishaev E, White S, Mauser N, Dabbs DJ.
Introduction of the Thin Prep Imaging System (TIS): Experience
in a high volume academic practice. Cytojournal. 2007;4:6.
8) Lozano R. Comparison of computer-assisted and manual
screening of cervical cytology. Gynecol Oncol. 2007;104:134-8.
9) Renshaw AA, Elsheikh TM. HSIL, epithelial cell abnormalityadjusted
workload, and the Thinprep Imaging System. Diagn
Cytopathol. 2012;40:201-3.
10) Biscotti CV, OBrien DL, Gero MA, Gramlich TL, Kennedy
AW, Easley KA. Thin-layer Pap test vs. conventional Pap smear.
Analysis of 400 split samples. J Reprod Med. 2002;47:9-13.
11) Catteau X, Simon P, Noël JC. Evaluation of the oncogenic human
papillomavirus DNA Test with Liquid-Based Cytology in primary
cervical cancer screening and the importance of the ASC/SIL
ratio: A Belgian study. ISRN Obstet Gynecol. 2014;2014:536495.
12) Halford JA, Batty T, Boost T, Duhig J, Hall J, Lee C, Walker K.
Comparison of the sensitivity of conventional cytology and the
ThinPrep Imaging System for 1,083 biopsy confirmed highgrade
squamous lesions. Diagn Cytopathol. 2010;38:318-26.
13) Kitchener HC, Blanks R, Dunn G, Gunn L, Desai M, Albrow
R, Mather J, Rana DN, Cubie H, Moore C, Legood R, Gray A,
Moss S. Automation-assisted versus manual reading of cervical
cytology (MAVARIC): A randomised controlled trial. Lancet
Oncol. 2011;12:56-64.
14) Castle PE, Bulten J, Confortini M, Klinkhamer P, Pellegrini
A, Siebers AG, Ronco G, Arbyn M. Age-specific patterns of
unsatisfactory results for conventional Pap smears and liquidbased
cytology: Data from two randomised clinical trials. BJOG.
2010;117:1067-73.
15) Strander B, Andersson-Ellström A, Milsom I, Rådberg T, Ryd W.
Liquid-based cytology versus conventional Papanicolaou smear
in an organized screening program: A prospective randomized
study. Cancer. 2007;111:285-91.
16) Bak M, Séberné Éll M, Bóka M, Veleczki Z, Nyári T, Pete I,
Szentirmay Z. Liquid-based cervical cytology ThinPrep screening
in Hungary. [Article in Hungarian] Orv Hetil. 2014;155:708-14.
17) Miller FS, Nagel LE, Kenny-Moynihan MB. Implementation of
the ThinPrep Imaging System in a high-volume metropolitan
laboratory. Diagn Cytopathol. 2007;35:213-7.
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}