title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¼Ð½Ð¾Ð³Ð¾Ð²Ð°ÑÐ¸Ð°Ð½ÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑÐ° Ð´Ð»Ñ Ð´Ð¸Ð°Ð³Ð½Ð¾ÑÑÐ¸ÑÐµÑÐºÐ¾Ð³Ð¾ Ð°Ð½Ð°Ð»Ð¸Ð·Ð° Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð±ÑÐ°Ð·ÑÐ¾Ð²
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¼Ð°Ð³Ð¸ÑÑÑÐ°
titleType: Subtitle
title: Development of Multiple Instance Learning model with Visual Transformer for diagnostic analysis of histopathological images
Creators: name: Ð¡Ð¼Ð¸ÑÐ½Ð¾Ð²Ð°, ÐÐ°ÑÑÑ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2023
Subjects: Ð¼Ð½Ð¾Ð³Ð¾Ð²Ð°ÑÐ¸Ð°Ð½ÑÐ½Ð¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑ
Ð¾Ð±ÑÑÑÐ½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð¸Ð½ÑÐµÐ»Ð»ÐµÐºÑ
LIME
Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð²Ð½Ð¸Ð¼Ð°Ð½Ð¸Ñ
ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ñ Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹
Ð°Ð½Ð°Ð»Ð¸Ð· Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð±ÑÐ°Ð·ÑÐ¾Ð²
WSI
multiple instance learning
visual transformer
explainable AI
attention mechanism
histopathological images classification
histopathological images analysis

description: Ð¢ÐµÐ¼Ð° Ð²ÑÐ¿ÑÑÐºÐ½Ð¾Ð¹ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ: Â«Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¼Ð½Ð¾Ð³Ð¾Ð²Ð°ÑÐ¸Ð°Ð½ÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑÐ° Ð´Ð»Ñ Ð´Ð¸Ð°Ð³Ð½Ð¾ÑÑÐ¸ÑÐµÑÐºÐ¾Ð³Ð¾ Ð°Ð½Ð°Ð»Ð¸Ð·Ð° Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð±ÑÐ°Ð·ÑÐ¾Ð²Â». ÐÐ°Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐµ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¼Ð½Ð¾Ð³Ð¾Ð²Ð°ÑÐ¸Ð°Ð½ÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑÐ° Ð´Ð»Ñ Ð°Ð½Ð°Ð»Ð¸Ð·Ð° (ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¸ Ð¸ ÑÐ°Ð·Ð¼ÐµÑÐºÐ¸) Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ ÑÐ½Ð¸Ð¼ÐºÐ¾Ð² ÐºÐ¾Ð»Ð¾ÑÐµÐºÑÐ°Ð»ÑÐ½Ð¾Ð¹ Ð°Ð´ÐµÐ½Ð¾Ð¼Ñ. Ð ÑÐ¾Ð´Ðµ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ¸ ÑÐµÑÐ°Ð»Ð¸ÑÑ ÑÐ»ÐµÐ´ÑÑÑÐ¸Ðµ Ð·Ð°Ð´Ð°ÑÐ¸: 1. ÐÐ±Ð·Ð¾Ñ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¸ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑÐ° Ð´Ð»Ñ Ð°Ð½Ð°Ð»Ð¸Ð·Ð° Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹. ÐÐ±Ð·Ð¾Ñ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¸Ð½ÑÐµÑÐ¿ÑÐµÑÐ°ÑÐ¸Ð¸ Ð¿ÑÐµÐ´ÑÐºÐ°Ð·Ð°Ð½Ð¸Ð¹ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹. 2. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑÐ° Ð´Ð»Ñ ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¸ Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð±ÑÐ°Ð·ÑÐ¾Ð² Ð² ÑÐ°Ð¼ÐºÐ°Ñ Ð¼ÐµÑÐ¾Ð´Ð° Ð¼Ð½Ð¾Ð³Ð¾Ð²Ð°ÑÐ¸Ð°Ð½ÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ. 3. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð°Ð²ÑÐ¾Ð¼Ð°ÑÐ¸Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð·Ð¼ÐµÑÐºÐ¸ Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð±ÑÐ°Ð·ÑÐ¾Ð² Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¸Ð½ÑÐµÑÐ¿ÑÐµÑÐ°ÑÐ¸Ð¸ Ð¿ÑÐµÐ´ÑÐºÐ°Ð·Ð°Ð½Ð¸Ð¹ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ ÑÐµÐ¿Ð»Ð¾Ð²ÑÑ ÐºÐ°ÑÑ Ð¸ ÑÐµÐ³Ð¼ÐµÐ½ÑÐ°ÑÐ¸Ð¾Ð½Ð½ÑÑ Ð¼Ð°ÑÐ¾Ðº. 4. ÐÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð½Ð°Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð¸ Ð¿Ð¾ÑÑÐ°Ð½Ð¾Ð²ÐºÐ° Ð²ÑÑÐ¸ÑÐ»Ð¸ÑÐµÐ»ÑÐ½Ð¾Ð³Ð¾ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½ÑÐ° Ð¿Ð¾ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð½ÑÐ¼ Ð¼Ð¾Ð´ÐµÐ»ÑÐ¼: â¢ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ðµ ÑÐ¾ÑÐ½Ð¾ÑÑÐ¸ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¿ÑÐ¸ ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¸ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹; â¢ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ðµ ÑÐ¾ÑÐ½Ð¾ÑÑÐ¸ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² ÑÐ°Ð·Ð¼ÐµÑÐºÐ¸ Ð³Ð¸ÑÑÐ¾Ð¿Ð°ÑÐ¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð±ÑÐ°Ð·ÑÐ¾Ð² (ÑÐµÐ³Ð¼ÐµÐ½ÑÐ°ÑÐ¸Ð¸); â¢ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½ÑÑ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð² Ñ Ð¼ÐµÑÑÐ¸ÐºÐ°Ð¼Ð¸ Ð´ÑÑÐ³Ð¸Ñ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹, ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð½ÑÑ Ð´Ð»Ñ ÑÐ°ÑÑÐ¼Ð°ÑÑÐ¸Ð²Ð°ÐµÐ¼ÑÑ Ð½Ð°Ð±Ð¾ÑÐ¾Ð² Ð´Ð°Ð½Ð½ÑÑ. Ð ÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð¼ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ ÑÐ°Ð±Ð¾ÑÑ ÑÐ²Ð»ÑÐµÑÑÑ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð°, ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½Ð½Ð°Ñ Ð½Ð° ÑÐ·ÑÐºÐµ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Python Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð±Ð¸Ð±Ð»Ð¸Ð¾ÑÐµÐºÐ¸ Pytorch. ÐÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð° ÑÐ¾ÑÑÐ¾Ð¸Ñ Ð¸Ð· 3 Ð¼Ð¾Ð´ÑÐ»ÐµÐ¹, ÐºÐ¾ÑÐ¾ÑÑÐµ Ð¼Ð¾Ð³ÑÑ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°ÑÑÑÑ Ð½ÐµÐ·Ð°Ð²Ð¸ÑÐ¸Ð¼Ð¾: Ð¼Ð¾Ð´ÑÐ»Ñ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ, Ð¿ÑÐµÐ´ÑÐºÐ°Ð·Ð°Ð½Ð¸Ñ Ð¸ Ð¾Ð±ÑÑÑÐ½ÐµÐ½Ð¸Ñ (ÑÐ°Ð·Ð¼ÐµÑÐºÐ¸). ÐÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ðµ ÐºÐ°ÑÐµÑÑÐ²Ð° ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¸ Ð¸ ÑÐ°Ð·Ð¼ÐµÑÐºÐ¸ Ð±ÑÐ»Ð¾ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ Ð½Ð° Ð´Ð²ÑÑ Ð½Ð°Ð±Ð¾ÑÐ°Ñ Ð´Ð°Ð½Ð½ÑÑ, ÑÐ¾Ð´ÐµÑÐ¶Ð°ÑÐ¸Ñ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ñ ÐºÐ¾Ð»Ð¾ÑÐµÐºÑÐ°Ð»ÑÐ½Ð¾Ð¹ Ð°Ð´ÐµÐ½Ð¾Ð¼Ñ. ÐÑÐ°ÐºÑÐ¸ÑÐµÑÐºÐ°Ñ Ð·Ð½Ð°ÑÐ¸Ð¼Ð¾ÑÑÑ ÑÐ°Ð±Ð¾ÑÑ ÑÐ¾ÑÑÐ¾Ð¸Ñ Ð² Ð²Ð¾Ð·Ð¼Ð¾Ð¶Ð½Ð¾ÑÑÐ¸ Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸Ñ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð² Ð¼ÐµÐ´Ð¸ÑÐ¸Ð½ÑÐºÐ¸Ñ ÑÐ¸ÑÑÐµÐ¼Ð°Ñ, Ð°ÑÑÐ¸ÑÑÐ¸ÑÑÑÑÐ¸Ñ Ð²ÑÐ°ÑÐ°Ð¼ Ð² Ð¿ÑÐ¸Ð½ÑÑÐ¸Ð¸ Ð´Ð¸Ð°Ð³Ð½Ð¾ÑÑÐ¸ÑÐµÑÐºÐ¸Ñ ÑÐµÑÐµÐ½Ð¸Ð¹.
description: The subject of the graduate qualification work is âDevelopment of Multiple Instance Learning model with Visual Transformer for diagnostic analysis of histopathological imagesâ. The given work is devoted to the development of Multiple Instance Learning model with Visual Transformer for diagnostic analysis (classification and segmentation) of histopathological images of colorectal adenoma. The research set the following goals: 1. Review of methods and models of machine learning using Multiple Instance Learning and Visual Transformer for the analysis of histopathological images. Review of methods for interpreting model predictions. 2. Development of a machine learning model based on the Visual Transformer for classification of histopathological images within the Multiple Instance Learning method. 3. Development of methods for automated segmentation of histopathological samples using heat maps and segmentation masks. 4. Program implementation and setting up a computational experiment based on the developed models: â¢ determining the accuracy of the model in image classification; â¢ determining the accuracy of segmentation methods; â¢ comparison of the obtained results with the metrics of other models developed for the considered datasets. The result of the work is a program implemented in the Python programming language using the Pytorch library. The program consists of 3 modules that can be used independently: learning module, prediction and explanation (segmentation). Classification and segmemtation quality study was conducted on on two datasets containing images of colorectal adenoma. The practical significance of the work lies in the possibility of using the model in medical systems, assisting doctors in making diagnostic decisions.