From visual diagnostics to deep learning: automatic mineral identification in polished section images

Cover Page

Cite item

Full Text

Abstract

Studying mineralogical composition of ores is a fundamental step in the exploration of new deposits, as it allows determining the forms in which useful components are found, the processes of ore formation, and the potential recoverability of valuable elements. The mineral associations, textures, and structures of ores not only provide key information about the geology of a deposit, but also determine the choice of beneficiation methods. Despite the development of modern analytical tools and existing solutions for automatic mineral diagnosis, such as those based on the SEM-EDS method, optical microscopy remains the most accessible means of quantitative mineralogical analysis. However, it remains labor-intensive and requires highly skilled specialists. In addition, its visual nature limits the accuracy and reproducibility of results, creating a need for more effective approaches. One promising area is the automation of ore mineral identification based on images of polished sections. The aim of the work was to develop and validate a universal segmentation model based on deep learning. In the course of the research, related problems were also solved, including the creation of an open LumenStone dataset, the development of color adaptation methods, joint analysis of PPL and XPL images, panorama construction, and the development of a fast annotation method. The work applied convolutional neural network architectures, color correction and joint image processing algorithms, as well as an original sampling method that compensates for class imbalance. The proposed segmentation model demonstrated high accuracy (IoU up to 0.88, PA up to 0.96) for nine minerals. The obtained results confirmed the effectiveness of integrating deep learning and modern image processing algorithms in mineralogical analysis systems and laid the foundation for further development of digital methods in automated petrography.

About the authors

D. M. Korshunov

Geological Institute of the Russian Academy of Sciences (GIN RAS)

Email: dmit0korsh@gmail.com
ORCID iD: 0000-0002-8500-7193
SPIN-code: 3118-6460

A. V. Khvostikov

Lomonosov Moscow State University

Email: khvostikov@cs.msu.ru
ORCID iD: 0000-0002-4217-7141
SPIN-code: 1972-4606

G. V. Nikolaev

Lomonosov Moscow State University

Email: nickolaev.gleb03@gmail.com
ORCID iD: 0009-0003-0814-3997

D. V. Sorokin

Lomonosov Moscow State University

Email: dsorokin@cs.msu.ru
ORCID iD: 0000-0003-3299-2545
SPIN-code: 4390-8842

O. I. Indychko

Lomonosov Moscow State University

Email: olesyaindychko@gmail.com
ORCID iD: 0009-0007-0936-4088

M. A. Boguslavskii

Lomonosov Moscow State University

Email: mboguslavskiy@yandex.ru
ORCID iD: 0000-0003-0133-7185
SPIN-code: 2474-1564

A. S. Krylov

Lomonosov Moscow State University

Email: kryl@cs.msu.ru
ORCID iD: 0000-0001-9910-4501
SPIN-code: 4218-6894

References

  1. De Castro B., Benzaazoua M., Chopard A., et al. Automated mineralogical characterization using optical microscopy: Review and recommendations. Minerals Engineering. 2022;189:107896. https://doi.org/10.1016/j.mineng.2022.107896
  2. Duncan P., Gavyn K. R. Unlocking the applications of automated mineral analysis. Geology Today. 2011;27(6):226–235. https://doi.org/10.1111/j.1365-2451.2011.00818.x
  3. Юшко С. А. Методы лабораторных исследований руд. М.: Недра; 1984. 389 с.
  4. Craig J. R., Vaughan D. J. Ore microscopy and ore petrography. Manchester: A Wiley-interscience Publication; 1994. 446 p.
  5. Bonifazi G. Digital multispectral techniques and automated image analysis procedures for industrial ore modelling. Minerals Engineering. 1995;8(7):779–794. https://doi.org/10.1016/0892-6875(95)00039-S
  6. Marschallinger R. Automatic mineral classification in the macroscopic scale. Computers & Geosciences. 1997;23(1):119–126. https://doi.org/10.1016/S0098-3004(96)00074-X
  7. Berry R., Walters S.G., McMahon C. Automated mineral identification by optical microscopy. In: Ninth International Congress for Applied Mineralogy. Brisbane, Australia, 8–10 September 2008. Brisbane: QLD; 2008. Pp. 91–94.
  8. Shoji T., Keneda H. An interactive system to assist mineral identification in ore microscopy. Mathematical Geology. 1994;26:961–972. https://doi.org/10.1007/BF02083424
  9. López-Benito A., Catalina J. C., Alarcón D., et al. Automated ore microscopy based on multispectral measurements of specular reflectance. I–A comparative study of some supervised classification techniques. Minerals Engineering. 2020;146:106136. https://doi.org/10.1016/j.mineng.2019.106136
  10. Berrezueta E., Ordóñez-Casado B., Bonilla W., Banda R., Castroviejo R., Carrión P., Puglla S. Ore petrography using optical image analysis: application to Zaruma-Portovelo deposit (Ecuador). Geosciences. 2016;6(2):30. https://doi.org/10.3390/geosciences6020030
  11. Köse C., Alp I., İkibaş C. Statistical methods for segmentation and quantification of minerals in ore microscopy. Minerals Engineering. 2012;30:19–32. https://doi.org/10.1016/j.mineng.2012.01.008
  12. Krawczykowska A., Trybalski K., Krawczykowski D. The application of modern techniques and measurement devices for identification of copper ore types and their properties. Archives of Mining Sciences. 2013;58(2):433–448. https://doi.org/10.2478/amsc-2013-0029
  13. Iglesias J. C. A, Augusto K. S, Gomes O. D. F. M., et al. Automatic characterization of iron ore by digital microscopy and image analysis. Journal of Materials Research and Technology. 2018;7(3):376–380. https://doi.org/10.1016/j.jmrt.2018.06.014
  14. Tang K., Chen J., Zhou H., Liu J. Deep convolutional neural network for 3D mineral identification and liberation analysis. Minerals Engineering. 2022;183:107592. https://doi.org/10.1016/j.mineng.2022.107592
  15. Zhou Z., Yuan H., Cai X. Rock Thin section image identification based on convolutional neural networks of adaptive and second-order pooling methods. Mathematics. 2023;11(5):1245. https://doi.org/10.3390/math11051245
  16. Kirillov A., Mintun E., Ravi N., et al. Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023;4015–4026. https://doi.org/10.48550/arXiv.2304.02643
  17. Hatamizadeh A., Kautz J. Mambavision: A hybrid mamba-transformer vision backbone. arXiv preprint arXiv: 2407.08083. 2024. https://doi.org/10.48550/arXiv.2407.08083
  18. Liu M. W., Lin Y. H., Lo Y. C., et al. Defect Detection of grinded and polished workpieces using faster R-CNN. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2021. Pp. 1290–1296. https://doi.org/10.1109/AIM46487.2021.9517664
  19. Zhongliang L. V., Zhenyu Lu., Kewen Xia., et al. LAACNet: Lightweight adaptive activation convolution network-based defect detection on polished metal surfaces. Engineering Applications of Artificial Intelligence. 2024;133(E):108482. https://doi.org/10.1016/j.engappai.2024.108482
  20. Sivkova T., Gusev A., Syropyatov A. Technology for cast iron microstructure analysis in SIAMS software using neural networks. In: Proceedings of the 31th International Conference on Computer Graphics and Vision. September 27–30, 2021, Nizhny Novgorod, Russia. 2021;2:772–780.
  21. Amaral B., Soares A.K., Iglesias J.C.Á., Caldas T.D.P., Santos R.B.M., Paciornik S. Instance segmentation of quartz in iron ore optical microscopy images by deep learning. Minerals Engineering. 2024;211:108681. https://doi.org/10.1016/j.mineng.2024.108681
  22. Maitre J., Bouchard K., Bédard L.P. Mineral grains recognition using computer vision and machine learning. Computers & Geosciences. 2019;130:84–93. https://doi.org/10.1016/j.cageo.2019.05.009
  23. Song Y., Huang Z., Shen C., et al. Deep learning-based automated image segmentation for concrete petrographic analysis. Cement and Concrete Research. 2020;135:106118. https://doi.org/10.1016/j.cemconres.2020.106118
  24. Donskoi E., Hapugoda S., Manuel J. R., et al. Automated optical image analysis of iron ore sinter. Minerals. 2021;11(6):562. https://doi.org/10.3390/min11060562
  25. Donskoi E., Poliakov A. Advances in optical image analysis textural segmentation in ironmaking. Applied Sciences. 2020;10(18):6242. https://doi.org/10.3390/app10186242
  26. Santoro L., Lezzerini M., Aquino A., et al. A novel method for evaluation of ore minerals based on optical microscopy and image analysis: preliminary results. Minerals. 2022;12(11):1348. https://doi.org/10.3390/min12111348
  27. Su C., Wang Y., Zhu J., Zhang X. C. Rock classification in petrographic thin section images based on concatenated convolutional neural networks. Earth Science Informatics. 2020;13:1477–1484. https://doi.org/10.1007/s12145-020-00505-1
  28. Tang H., Wang H., Wang L., et al. An improved mineral image recognition method based on deep learning. JOM. 2023;75:2590–2602. https://doi.org/10.1007/s11837-023-05792-9
  29. Khvostikov A. V., Korshunov D. M., Krylov A. S., Boguslavskii M. A. Automatic identification of minerals in images of polished sections. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2021;XLIV-2/W1-2021:113–118. https://doi.org/10.5194/isprs-archives-XLIV-2-W1-2021-113-2021
  30. Chen H., Zhang H., Yang G. A., Zhang L. Mutual information domain adaptation network for remotely sensed semantic segmentation. In: IEEE Transactions on Geoscience and Remote Sensing. 2022;60:1–16. https://doi.org/10.1109/TGRS.2022.3203910
  31. Nasim M. K., Tannistha M., Shrivastava A., Singh T. Seismic facies analysis: a deep domain adaptation approach. In: IEEE Transactions on Geoscience and Remote Sensing. 2020;60:1–16. https://doi.org/10.1109/TGRS.2022.3151883
  32. Lin D., Dai J., Jia J., et al. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA; 2016. Pp. 3159–3167. https://doi.org/10.1109/CVPR.2016.344
  33. Chen X., Cheung Y. S. J., Lim S. N., Zhao H. ScribbleSeg: Scribble-based interactive image segmentation. arXiv preprint arXiv: 2303.11320. 2023. https://doi.org/10.48550/arXiv.2303.11320
  34. Cheng B., Parkhi O., Kirillov A. Pointly-supervised instance segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA; 2022. Pp. 2617–2626. https://doi.org/10.1109/CVPR52688.2022.00264
  35. Tang H., He. L., Huang B, et al. Segmentation and labeling of polished section images based on deep learning. Mining, Metallurgy & Exploration. 2025;42:1053–1063. https://doi.org/10.1007/s42461-025-01205-4
  36. Tabani H., Balasubramaniam A., Marzbanet S., et al. Improving the efficiency of transformers for resource-constrained devices. In: 24th Euromicro Conference on Digital System Design (DSD). Palermo, Italy; 2021. Pp. 449–456. https://doi.org/10.1109/DSD53832.2021.00074
  37. Bressan P. O., Junior J. M. Martins J. A. C., et al. Semantic segmentation with labeling uncertainty and class imbalance. International Journal of Applied Earth Observation and Geoinformation. 2022;108:102690. https://doi.org/10.1016/j.jag.2022.102690
  38. Li Z., Kamnitsas K., Glocker B. Analyzing overfitting underclass imbalance in neural networks for image segmentation. In: IEEE Transactions on Medical Imaging. 2021;40(3):1065–1077. https://doi.org/10.1109/TMI.2020.3046692
  39. Kochkarev A., Khvosticov A., Korshunov D., Boguslavskii M. Data balancing method for training segmentation neural networks. In: Proceedings of the 30th International Conference on Computer Graphics and Machine Vision (GraphiCon 2020). Saint Petersburg, Russia, 22–25 September. Saint Petersburg: Ceur Workshop Proceedings; 2020.
  40. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional networks for biomedical image segmentation. B: Navab N. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer; 2015. Pp. 234–41. https://doi.org/10.1007/978-3-319-24574-4_28
  41. Zhao H., Shi J., Qi X., et al. Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA; 2017. Pp. 2881–2890. https://doi.org/10.1109/CVPR.2017.660
  42. Xiao T., Liu Y., Zhou B., et al. Unified perceptual parsing for scene understanding. In: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds.) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science. Vol 11209. Springer, Cham; 2018. Pp. 418–434. https://doi.org/10.1007/978-3-030-01228-1_26
  43. Rezatofighi H., Tsoi N., Gwak J., et al. Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the IEEE. 2019. Pp. 658–666. https://doi.org/10.48550/arXiv.1902.09630
  44. Reinhard E., Adhikhmin M., Gooch B., Shirley P. Color transfer between images. In: IEEE Computer Graphics and Applications. 2001;21(5):34-41. https://doi.org/10.1109/38.946629
  45. Indychko O. I., Khvostikov A. V., Korshunov D. M., Boguslavskii M. A. Color adaptation in images of polished sections of geological specimens. Computational Mathematics and Modeling. 2022;33:487–500. https://doi.org/10.1007/s10598-023-09588-z
  46. Wolf S. Color correction matrix for digital still and video imaging systems. Washington, D.C.: National Telecommunications and Information Administration; 2003. 28 p.
  47. Sharma G., Wu W., Dalal E. N. The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research & Application. 2005;30(1):21–30. https://doi.org/10.1002/col.20070
  48. Razzhivina D. I., Korshunov D. M., Boguslavskiy M. A., et al. Registration and segmentation of PPL and XPL images of geological polished sections containing anisotropic minerals. Computational Mathematics and Modeling. 2024;34:16–26. https://doi.org/10.1007/s10598-024-09592-x
  49. Lowe D. G. Distinctive image features from scale invariant keypoints. International Journal of Computer Vision. 2004;60:91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  50. Fischler M. A., Bolles R. C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM. 1981;24(6):381–395.
  51. Ro S.-H., Kim S.-H. An image stitching algorithm for the mineralogical analysis. Minerals Engineering. 2021:169;106968. https://doi.org/10.1016/j.mineng.2021.106968
  52. Nikolaev G., Korshunov D., Khvostikov A. Automatic stitching of panoramas for geological images of polished sections. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2024;X-2/W1-2024:39–46. https://doi.org/10.5194/isprs-annals-X-2-W1-2024-39-2024
  53. Brown D. C. Decentering distortion of lenses. Photogrammetric Engineering. 1966:32(3);444–462.
  54. Seibert J. A., Boone J. M., Lindfors K. K. Flat-field correction technique for digital detectors. In: Proceedings of SPIE, Medical Imaging 1998: Physics of Medical Imaging. 1998:3336;348–354. https://doi.org/10.1117/12.317034
  55. Sun J., Shen Z., Wang Y., et al. LoFTR: Detector-free local feature matching with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. Pp. 8922–8931. https://doi.org/10.48550/arXiv.2104.00680
  56. Boykov Y., Kolmogorov V. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. 2004;26(9):1124–1137. https://doi.org/10.1109/TPAMI.2004.60
  57. Achanta R., Shaji A., Smith K., et al. SLIC superpixels compared to state-of-the-art superpixel methods. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. 2012;34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120
  58. Felzenszwalb P. F., Huttenlocher D. P. Efficient graph-based image segmentation. International Journal of Computer Vision. 2004:59;167–181. https://doi.org/10.1023/B:VISI.0000022288.19776.77
  59. Indychko O., Korshunov D., Khvostikov A. Using uncertainty to expand training sets for mineral segmentation in geological images. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2025. In press.
  60. Franco L., Mandica P., Kallidromitis K., et al. Hyperbolic Active Learning for Semantic Segmentation under Domain Shift. In: Proceedings of the 41st International Conference on Machine Learning. 2024. https://doi.org/10.48550/arXiv.2306.11180
  61. Khvostikov A., Ippolitov V., Krylov A., et al. PathScribe: new software to work with whole slide histological images for education and research. In: Proceedings of the 2023 8th International Conference on Biomedical Imaging, Signal Processing. Singapore: ACM; 2023. Pp. 63–70. https://doi.org/10.1145/3634875.3634884

Supplementary files

Supplementary Files
Action
1. JATS XML
Download


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Согласие на обработку персональных данных с помощью сервиса «Яндекс.Метрика»

1. Я (далее – «Пользователь» или «Субъект персональных данных»), осуществляя использование сайта https://journals.rcsi.science/ (далее – «Сайт»), подтверждая свою полную дееспособность даю согласие на обработку персональных данных с использованием средств автоматизации Оператору - федеральному государственному бюджетному учреждению «Российский центр научной информации» (РЦНИ), далее – «Оператор», расположенному по адресу: 119991, г. Москва, Ленинский просп., д.32А, со следующими условиями.

2. Категории обрабатываемых данных: файлы «cookies» (куки-файлы). Файлы «cookie» – это небольшой текстовый файл, который веб-сервер может хранить в браузере Пользователя. Данные файлы веб-сервер загружает на устройство Пользователя при посещении им Сайта. При каждом следующем посещении Пользователем Сайта «cookie» файлы отправляются на Сайт Оператора. Данные файлы позволяют Сайту распознавать устройство Пользователя. Содержимое такого файла может как относиться, так и не относиться к персональным данным, в зависимости от того, содержит ли такой файл персональные данные или содержит обезличенные технические данные.

3. Цель обработки персональных данных: анализ пользовательской активности с помощью сервиса «Яндекс.Метрика».

4. Категории субъектов персональных данных: все Пользователи Сайта, которые дали согласие на обработку файлов «cookie».

5. Способы обработки: сбор, запись, систематизация, накопление, хранение, уточнение (обновление, изменение), извлечение, использование, передача (доступ, предоставление), блокирование, удаление, уничтожение персональных данных.

6. Срок обработки и хранения: до получения от Субъекта персональных данных требования о прекращении обработки/отзыва согласия.

7. Способ отзыва: заявление об отзыве в письменном виде путём его направления на адрес электронной почты Оператора: info@rcsi.science или путем письменного обращения по юридическому адресу: 119991, г. Москва, Ленинский просп., д.32А

8. Субъект персональных данных вправе запретить своему оборудованию прием этих данных или ограничить прием этих данных. При отказе от получения таких данных или при ограничении приема данных некоторые функции Сайта могут работать некорректно. Субъект персональных данных обязуется сам настроить свое оборудование таким способом, чтобы оно обеспечивало адекватный его желаниям режим работы и уровень защиты данных файлов «cookie», Оператор не предоставляет технологических и правовых консультаций на темы подобного характера.

9. Порядок уничтожения персональных данных при достижении цели их обработки или при наступлении иных законных оснований определяется Оператором в соответствии с законодательством Российской Федерации.

10. Я согласен/согласна квалифицировать в качестве своей простой электронной подписи под настоящим Согласием и под Политикой обработки персональных данных выполнение мною следующего действия на сайте: https://journals.rcsi.science/ нажатие мною на интерфейсе с текстом: «Сайт использует сервис «Яндекс.Метрика» (который использует файлы «cookie») на элемент с текстом «Принять и продолжить».