Reproducibility of cytological diagnoses in evaluating liquid cervical smears and immunocytochemical co-expression of p16/Ki-67 using manual and automatic methods

Cover Page

Cite item

Full Text

Abstract

Aim. To assess the reproducibility of cytological diagnoses in evaluating liquid cervical smears and immunocytochemical co-expression of p16/Ki-67 using manual and automatic methods.

Materials and methods. Cytological smears prepared using the liquid cytology method on the Becton Dickinson device (SurePath technology) were studied. An immunocytochemical study was carried out using a Ventana BenchMark Ultra automatic immunostainer with a commercial CINtec kit (determination of p16/Ki-67 co-expression). In total, 100 cytological slides (50 pairs of Pap-smears and immunocytochemical slides) were studied. The diagnostic kit was reviewed by five cytologists independently, and the cytologic slides were evaluated using four categories according to the Bethesda system (2014) and according to the categories of normal/abnormal. The co-expression of p16/Ki-67 was assessed per the manufacturer's recommendations (Roche) using the manual method (light microscope) and the automatic Vision Cyto Pap ICC system. Statistical processing of the results was performed using the SPSS software package version 26.0.0.0 with the calculation of the reproducibility indices of Cohen's kappa and Fleiss' kappa.

Results. When assessing the reproducibility of four categories of cytological diagnoses according to the Bethesda system (2014), Cohen's kappa was 0.048–0.265. The overall Fleiss' kappa between all cytologists was 0.103. When only two categories (normal/abnormal) were used, the reproducibility ranged from 0.058 to 0.377. When assessing the co-expression of p16 and Ki-67, Cohen's kappa reproducibility was from 0.196 to 0.574, while the overall Fleiss' kappa was 0.407. When comparing the evaluation results of each of the cytologists with the neural network, Cohen's kappa reproducibility ranged from 0.103 to 0.436.

Conclusion. The reproducibility of cytological diagnoses according to the Bethesda system (2014) and two categories (normal/abnormal) based on the Pap smear study was low. Such results are primarily due to a large number of abnormal smears in the study. The immunocytochemical method has diagnosis reproducibility three times higher, indicating the need to measure the co-expression of p16 and Ki-67 to increase the sensitivity and specificity of the cytological method. Similar reproducibility when comparing the manual and automatic evaluation of the "double label" suggests that the neural network algorithm can currently help in decision support rather than replace the cytologist at the diagnostic stage.

About the authors

Anna V. Tregubova

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: a.asaturova@gmail.com
ORCID iD: 0000-0003-4601-1330

Res. Assist., Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Nadezda S. Tevrukova

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: tevrukova@mail.ru
ORCID iD: 0000-0003-3305-8543

Cand. Sci. (Biol.), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Larisa S. Ezhova

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: larserezhova@yandex.ru
ORCID iD: 0000-0002-9804-8349

Cand. Sci. (Med), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Marina V. Shamarakova

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: a.asaturova@gmail.com
ORCID iD: 0000-0002-0972-4350

Cand. Sci. (Med), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Alina S. Badlaeva

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: a.asaturova@gmail.com
ORCID iD: 0000-0001-5223-9767

Res. Assist., Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Darya A. Dobrovolskaya

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: dashaGRI@yandex.ru
ORCID iD: 0000-0002-1409-9959

Graduate Student, Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Giuldana R. Bayramova

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: v_prilepskaya@oparina4.ru
ORCID iD: 0000-0003-4826-661X

D. Sci. (Med), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Niso M. Nazarova

Kulakov National Medical Research Centre for Obstetrics, Gynaecology and Perinatology

Email: grab2@yandex.ru
ORCID iD: 0000-0001-9499-7654

D. Sci. (Med), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Alexey Yu. Shilyaev

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Email: 89265507667@mail.ru
ORCID iD: 0000-0001-7200-2708

Cand. Sci. (Med), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

Aleksandra V. Asaturova

Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Author for correspondence.
Email: a.asaturova@gmail.com
ORCID iD: 0000-0001-8739-5209

D. Sci. (Med), Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology

Russian Federation, Moscow

References

  1. Siegel RL, Miller KD, Fuchs HE. Cancer statistics, 2022. CA Cancer J Clin. 2022;72(1):7-33.
  2. Boring CC, Squires TS, Tong T. Cancer statistics, 1999. CA Cancer J Clin. 1999;49(1):8-31.
  3. Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285(11):1500-5.
  4. Selvaggi SM. Implications of low diagnostic reproducibility of cervical cytologic and histologic diagnoses. JAMA. 2001;285(11):1506-8.
  5. Hwang H, Follen M, Guillaud M, et al. Cervical cytology reproducibility and associated clinical and demographic factors. Diagn Cytopathol. 2020;48(1):35-42.
  6. Kloboves Prevodnik V, Jerman T, Nolde N, et al. Interobserver variability and accuracy of p16/Ki-67 dual immunocytochemical staining on conventional cervical smears. Diagn Cytopathol. 2019;14(1):48.
  7. Wentzensen N, Lahrmann B, Clarke MA, et al. Accuracy and Efficiency of Deep-Learning–Based Automation of Dual Stain Cytology in Cervical Cancer Screening. J Natl Cancer Inst. 2021;113(1):72-9.
  8. Dey P. Artificial neural network in diagnostic cytology. CytoJournal. 2022;19:146.
  9. Sanyal P, Barui S, Deb P, Sharma HC. Performance of A Convolutional Neural Network in Screening Liquid Based Cervical Cytology Smears. J Cytol. 2019;36(3):146.
  10. Mohammed MA, Abdurahman F, Ayalew YA. Single-cell conventional pap smear image classification using pre-trained deep neural network architectures. BMC Biomed Eng. 2021;3(1):1-8.
  11. Zhang L, Lu L, Member S, et al. DeepPap: Deep Convolutional Networks for Cervical Cell Classification. IEEE J Biomed Health Inform. 2017;21(6):1633-43.
  12. Фириченко С.В., Манухин И.Б., Роговская С.И., Манухина Е.И. «Подводные камни» цервикального скрининга. Доктор.Ру. 2018;2(146):26-34 [Firichenko SV, Manukhin IB, Rogovskaya SI, Manukhina ЕI. Pitfalls in Cervical Screening. Doctor.Ru. 2018;2(146):26-34 (in Russian)].
  13. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977;33(1):159.
  14. Swailes AL, Hossler CE, Kesterson JP. Pathway to the Papanicolaou smear: The development of cervical cytology in twentieth-century America and implications in the present day. Gynecol Oncol. 2019;154(1):3-7.
  15. Salehiniya H, Momenimovahed Z, Allahqoli L, et al. Factors related to cervical cancer screening among Asian women. Riv Eur Sci Med Farmacol. 2021;25(19):6109-22.
  16. Cudjoe J, Nkimbeng M, Turkson-Ocran RA, et al. Understanding the Pap Testing Behaviors of African Immigrant Women in Developed Countries: A Systematic Review. J Immigr Minor Health. 2021;23(4):840-56.
  17. Chin SS, Jamonek Jamhuri NAB, Hussin N, et al. Factors influencing pap smear screening uptake among women visiting outpatient clinics in Johor. Malays Fam Physician. 2022;17(2):46-55.
  18. Settakorn J, Rangdaeng S, Preechapornkul N, et al. Interobserver reproducibility with LiquiPrep liquid-based cervical cytology screening in a developing country. Asian Pac J Cancer Prev. 2008;9(1):92-6.
  19. Strander B, Andersson-Ellström A, Milsom I, et al. Liquid-based cytology versus conventional Papanicolaou smear in an organized screening program: a prospective randomized study. Cancer. 2007;111(5):285-91.
  20. Nishio H, Iwata T, Nomura H, et al. Liquid-based cytology versus conventional cytology for detection of uterine cervical lesions: a prospective observational study. Jpn J Clin Oncol. 2018;48(6):522-8.
  21. Sharma P, Gupta P, Gupta N, et al. Evaluation of the Performance of CinTec® PLUS in SurePathTM Liquid-Based Cervico-Vaginal Samples. Turk Patoloji Dergisi. 2021;37(1):32-8.
  22. Сухих Г.Т, Прилепская В.Н., Асатурова А.В., и др. Диагностика, лечение и профилактика цервикальных интраэпительных неоплазий. М., 2020 [Sukhikh GT, Prilepskaia VN, Asaturova AV, et al. Diagnostika, lecheniie i profilaktika tservikal'nykh intraepitel'nykh neoplazii. Moscow, 2020 (in Russian)].
  23. Sharma J, Toi P, Siddaraju N, et al. A comparative analysis of conventional and SurePath liquid-based cervicovaginal cytology: A study of 140 cases. J Cytol. 2016;33(2):80-4.
  24. Confortini M, Bondi A, Cariaggi MP, et al. Interlaboratory reproducibility of liquid-based equivocal cervical cytology within a randomized controlled trial framework. Diagn Cytopathol. 2007;35(9):541-4.
  25. Sriamporn S, Kritpetcharat O, Nieminen P, et al. Kritpetcharat O. Consistency of cytology diagnosis for cervical cancer between two laboratories. Asian Pac J Cancer Prev. 2005;6(2):208-12.
  26. Tjalma WAA. Diagnostic performance of dual-staining cytology for cervical cancer screening: A systematic literature review. Eur J Obstet Gynecol Reprod Biol. 2017;210:275-80.
  27. Bergeron C, Ikenberg H, Sideri M, et al. Prospective evaluation of p16/Ki-67 dual-stained cytology for managing women with abnormal Papanicolaou cytology: PALMS study results. Cancer Cytopathol. 2015;123(6):373-81.
  28. Li Y, Fu Y, Cheng B, et al. A Comparative Study on the Accuracy and Efficacy Between Dalton and CINtec® PLUS p16/Ki-67 Dual Stain in Triaging HPV-Positive Women. Front Oncol. 2022;11:815213.
  29. Han Q, Guo H, Geng L, Wang Y. p16/Ki-67 dual-stained cytology used for triage in cervical cancer opportunistic screening. Chin J Cancer. 2020;32(2):208.
  30. McMenamin M, McKenna M, McDowell A, et al. Intra- and inter-observer reproducibility of CINtec® PLUS in ThinPrep® cytology preparations. Cytopathology. 2017;28(4):284-90.
  31. Wentzensen N, Fetterman B, Tokugawa D, et al. Interobserver reproducibility and accuracy of p16/Ki-67 dual-stain cytology in cervical cancer screening. Cancer Cytopathol. 2014;122(12):914-20.
  32. Goh ST, TayKah T, Lim L. Inter-observer Variabilty of CINtec PLUS Dual Staining for p16/ki67. J Am Soc Cytopathol. 2017;6(5):S30.
  33. Hammer A, Gustafson LW, Christensen PN, et al. Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience. Cancer Med. 2020;9(21):8235-42.
  34. Benevolo M, Allia E, Gustinucci D, Montaguti A. Interobserver reproducibility of cytologic p16INK4a /Ki-67 dual immunostaining in human papillomavirus-positive women. Cancer Cytopathol. 2017;125(3):212-20.
  35. Benevolo M, Mancuso P, Allia E, et al. Interlaboratory concordance of p16/Ki-67 dual-staining interpretation in HPV-positive women in a screening population. Cancer Cytopathol. 2020;128(5):323-32.
  36. Sornapudi S, Addanki R, Stanley RJ, et al. Automated Cervical Digitized Histology Whole-Slide Image Analysis Toolbox. J Pathol Informatics. 2021;12(1):26.
  37. Kanavati F, Hirose N, Ishii T, et al. A Deep Learning Model for Cervical Cancer Screening on Liquid-Based Cytology Specimens in Whole Slide Images. Cancers. 2022;14(5):1159.
  38. Özcan Z, Kimiloğlu E, Akyildiz İğdem A, Erdoğan N. Comparison of the Diagnostic Utility of Manual Screening and the ThinPrep Imaging System in Liquid-Based Cervical Cytology. Turk Patoloji Dergisi. 2020;36(2):135-41.
  39. Nuttall DS, Hillier S, Clayton HR, et al. A retrospective validation of the FocalPoint GS slide profiler NFR technology by analysis of interval disease outcomes compared with manual cytology. Cancer Cytopathol. 2019;127(4):240-6.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Examples of automatic evaluation of p16 and Ki-67 expression. Top row: false positive interpretation; there are cells with positive expression of Ki-67 (red staining) and positive expression of p16 (brown staining) but no co-expression of markers. Bottom row: correct interpretation, there are cells with simultaneous red and brown staining, positive "double label."

Download (147KB)
Indexing metadata

Copyright (c) 2023 Consilium Medicum

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Согласие на обработку персональных данных с помощью сервиса «Яндекс.Метрика»

1. Я (далее – «Пользователь» или «Субъект персональных данных»), осуществляя использование сайта https://journals.rcsi.science/ (далее – «Сайт»), подтверждая свою полную дееспособность даю согласие на обработку персональных данных с использованием средств автоматизации Оператору - федеральному государственному бюджетному учреждению «Российский центр научной информации» (РЦНИ), далее – «Оператор», расположенному по адресу: 119991, г. Москва, Ленинский просп., д.32А, со следующими условиями.

2. Категории обрабатываемых данных: файлы «cookies» (куки-файлы). Файлы «cookie» – это небольшой текстовый файл, который веб-сервер может хранить в браузере Пользователя. Данные файлы веб-сервер загружает на устройство Пользователя при посещении им Сайта. При каждом следующем посещении Пользователем Сайта «cookie» файлы отправляются на Сайт Оператора. Данные файлы позволяют Сайту распознавать устройство Пользователя. Содержимое такого файла может как относиться, так и не относиться к персональным данным, в зависимости от того, содержит ли такой файл персональные данные или содержит обезличенные технические данные.

3. Цель обработки персональных данных: анализ пользовательской активности с помощью сервиса «Яндекс.Метрика».

4. Категории субъектов персональных данных: все Пользователи Сайта, которые дали согласие на обработку файлов «cookie».

5. Способы обработки: сбор, запись, систематизация, накопление, хранение, уточнение (обновление, изменение), извлечение, использование, передача (доступ, предоставление), блокирование, удаление, уничтожение персональных данных.

6. Срок обработки и хранения: до получения от Субъекта персональных данных требования о прекращении обработки/отзыва согласия.

7. Способ отзыва: заявление об отзыве в письменном виде путём его направления на адрес электронной почты Оператора: info@rcsi.science или путем письменного обращения по юридическому адресу: 119991, г. Москва, Ленинский просп., д.32А

8. Субъект персональных данных вправе запретить своему оборудованию прием этих данных или ограничить прием этих данных. При отказе от получения таких данных или при ограничении приема данных некоторые функции Сайта могут работать некорректно. Субъект персональных данных обязуется сам настроить свое оборудование таким способом, чтобы оно обеспечивало адекватный его желаниям режим работы и уровень защиты данных файлов «cookie», Оператор не предоставляет технологических и правовых консультаций на темы подобного характера.

9. Порядок уничтожения персональных данных при достижении цели их обработки или при наступлении иных законных оснований определяется Оператором в соответствии с законодательством Российской Федерации.

10. Я согласен/согласна квалифицировать в качестве своей простой электронной подписи под настоящим Согласием и под Политикой обработки персональных данных выполнение мною следующего действия на сайте: https://journals.rcsi.science/ нажатие мною на интерфейсе с текстом: «Сайт использует сервис «Яндекс.Метрика» (который использует файлы «cookie») на элемент с текстом «Принять и продолжить».