What is the current role of Artificial Intelligence in digital pathology? Part 2 – state of art.

This article explains the pathology informatics terminology (part 1) and summarizes the current state of AI in pathology (part 2)

Current status of artificial intelligence in pathology

Even though there is a lot of hype about AI, and it feels like this is the next big thing to take over pathology, currently the applications in this discipline are limited to research. As of today, there is no  FDA approved AI solution for pathology on the market, but there are initiatives dedicated to supporting the development of such applications. FDA Digital Health Innovation Action Plan is designed to issue guidance on medical software and together with their Digital Health Software Precertification Program (Pre-Cert), accelerate the approval of potential future applications. AI officially (with an FDA clearance or approval) made it to the diabetes research, cardiovascular and brain disease treatment and radiology, but not pathology yet… Many papers about AI in pathology are published, coming from the R&D departments of commercial entities and academic institutions. Hopefully, the path from there to an approved clinical application will not be too long.

In order to promote and contribute to the AI development researchers are strongly encouraged to build new solutions and take part or host challenges in biomedical image analysis. Challenges in biomedical image analysis are competitions hosted by different organizations aiming to compare new and existing solutions and algorithms in biomedical image analysis. The participants try to solve a stated problem on a common data set with a solution of choice and are required to publish their results in a peer-reviewed journal as a part of the challenge. Many relevant challenges, like the “Camelyon challenge” organized in 2016 and 2017, are gathered on the platform of the Consortium for Open Medical Image Computing (COMIC) called “Grand Challenges”, but there are also other platforms developed by different organizations which serve the same purpose such as:

• Codalab
• Covalic developed by Kitware
• Virtual Skeleton developed by Sicas
• And Kaggle

One of the most famous challenges in biomedical image analysis was the Camelyon challenge hosted by the Diagnostic Image Analysis Group (DIAG) and Department of Pathology of the Radboud University Medical Center (Radboudumc) in Nijmegen, The Netherlands. It had two editions: Camelyon 16 and Camelyon 17 organized in 2016 and 2017 respectively. The challenges received almost 100 submissions from both academic institutions and companies to solve the problem of detection of breast cancer metastasis in whole slide images of lymph nodes. The participants were working with the same training and test data set (1399 H&E stained lymph node sections) and were free to use any image analysis method to best solve this problem.

The Camelyon challenge similar to other challenges hosted by COMIC required

• Defining a meaningful task →  in this case it was the detection and classification of breast cancer metastasis in histological lymph node sections. Note that this was a binary classification problem (metastases present or absent) in just one disease entity (breast cancer).
• Gathering representative data →  whole slide images of lymph nodes labeled by a pathologist as containing metastasis or metastasis-free as well as a subset of images with pathologist’s annotations of metastasis
• Defining the reference standard → pathologist’s diagnosis and annotations
• Defining a discriminative evaluation metric → Cohen’s kappa as a measure of consensus between the algorithm and the pathologist
• Writing a peer-reviewed scientific publication

For breast cancer patients detecting the metastases in lymph nodes is an indispensable part of the diagnostic workup. The manual evaluation is monotonous and time-consuming as often numerous sections of the lymph nodes are submitted. As this task is a binary problem, and the detection of metastatic cells takes place within a relatively homogenous, morphologically similar population of lymphocytes in the lymph node, it is a perfect candidate for computer-aided diagnosis. In this case, an algorithm would suggest to the pathologist the suspicious regions for review. This seems to be a straightforward task, ready to be implemented in the clinics –  since the challenge winner (jaylee from Lunit Inc. ) achieved a Cohen’s kappa of 0.8993, which is pretty impressive!  Unfortunately, as enthusiastic as one can (and wants to) be, about the advancements of AI in pathology, we need to keep in mind that the participants of the challenge were advanced programmers equipped with all necessary computational resources, which is not a common setting in pathology departments, most of which did not even go digital yet. The algorithms used in the challenge were all independently designed for the sole purpose of detecting breast cancer in lymph nodes, which is the case with most research AI applications in pathology.

So far there is no commercially available FDA approved AI solution for pathology, so even if the early adopting pathology departments would like to take AI to the next level, they can’t due to a lack of available tools.

For research use, image analysis software companies are starting to incorporate AI modules into their solutions, where pathologists can perform annotations of target structures to train a model that later detects them automatically.

Such solutions are currently available from :

• Indica Labs in their Halo AI
• Visiopharm in the Oncotopix®/Biotopix™ AI image analysis platform
• Aiforia in the AIforia create

These tools incorporate random forest or deep learning modules for image analysis but are for research use only.

Lack of validated tools on the market is just one of the limitations in AI implementation in pathology. There are several other obstacles to overcome before it becomes the daily bread for pathologists. Some of these challenges are:

Most of the deep learning algorithms applied so far to pathology problem solution were supervised learning methods, meaning that they required a large amount of labeled data (annotations) for the training of the model. This task is often a bottleneck and limits the scalability of AI projects in pathology. The annotations need to be very detailed to avoid introducing noise into the system. This limits crowdsourcing as a method to obtain them. They also need to be provided in high quantity – the more labeled training data, the better the model. This is not a part of daily pathologist’s routine, which makes it a significant hurdle, as the pathologist’s resources are limited and their focus lies elsewhere. The unsupervised learning methods and data augmentation techniques may help to overcome this challenge in the future.

Even though the human body only consists of a limited number of tissues (nervous tissue, epithelial tissue, connective tissue, muscles), each of them appears different in different organs (the epithelium of the intestine looks different from the epithelial cells of which the liver is built). If on top of this we add the different morphologies of cancerous cells (different types for different cancers, highly variable in appearance due to tumor heterogeneity), changes caused by inflammation, necrosis, and preparation artifacts on histology slides, the number of possible combinations grows exponentially and constitutes a challenge even for a trained human observer (pathologist) which makes it a so far an insurmountable obstacle for computer vision.

The conducted pathology challenges, as well as many research papers, reduce the dimensionality of diagnostic tasks to a binary problem (benign vs. malignant, presence vs. absence of metastasis), however, in daily pathology practice the tasks are much more complex, include reading through biological and processing artifacts and integrating information from other disciplines. If they were to be broken down into binary decisions (which may not always be possible nor desirable), there would be thousands of them per case so designing an AI algorithm for each of them is clearly an impossibility.

Pathologists diagnosis, descriptions of lesions, annotations or scores are considered the gold standard and serve as a reference for the AI algorithms. However, even though the pathologists are highly trained professionals, they do not always agree  Ideally, a consolidated ground truth from multiple pathologists should be used to train deep learning models (eg. only the objects annotated by all pathologists, when multiple pathologists are annotating, would be used), which very rarely is the case as there are not enough pathologists to perform these tasks. Recognizing and categorizing tissue structures is a task the pathologists are very good at, so the consolidation of ground truth could actually solve the problem of inter-pathologist variability, however when it comes to quantitative or semiquantitative scoring tasks, eg. where pathologists are counting (like for Ki67) or estimating the percentage (like for PDL1) of positive cells in IHC stainings, they are not (nor is any other human observer) a good reference for a machine, which can count pixels of a particular color with far greater precision (if only they knew which pixels to count…but this exactly where pathologists expertise should be used). To solve this problem an independent metric to compare to, like e.g. survival data or treatment outcome data should be used, and they are not always available or accessible.

AI algorithms not only require special software, which is not accessible to pathologists yet, but they also need a much more powerful hardware to process the images. They are dependent on GPU equipped computers, which together with the storage space for the gigapixel histology slides significantly increases the cost of an already expensive process of going digital (WSI scanner acquisition and maintenance, image management software etc.).

Even though those challenges still seem big, already available and potential benefits of digital pathology will drive the advancement in the field. The technology will advance and pathologists will need to do the same. They will follow the direction of innovation. Some sooner others later, but in the end, all will benefit, and so will the patients.

AI is affecting our lives every time we touch any of our mobile devices.  We use it when taking pictures in a portrait mode with our smartphone or using google maps for navigation. It has already officially entered various medical fields, but in pathology, the current applications still remain restricted to research use only. It will probably officially enter the pathology domain soon, but there are still many challenges to overcome, which are pathology specific, like the enormous size of pathology images or the complexity of morphological patterns, which are not present in other disciplines. Thank the lower bar of AI implementation in other medical areas, regulatory pathways have been already paved and FDA officially supports the digital health initiatives with their Digital Health Innovation Action Plan and the Digital Health Software Precertification Program.

But what about pathologists? Shall we fear the AI era or will it boost our performance and increase our capacities? The feelings and opinions are mixed, but as the process seems to be inevitable, the only right thing to do is prepare. Prepare in terms of recognizing, where AI applications could contribute to streamlining the work and in terms of educating oneself. Gradually more and more educational opportunities and resources are available spanning from digital pathology and pathology informatics fellowships to online based courses like the DPA’s Digital Pathology Certificate Program. Taking a proactive approach towards the AI incorporation in pathology will translate into helping patients earlier and quicker in the future. Through screening algorithms pathologists would be able to save time by focusing on preselected regions of interests in such mundane tasks like searching for micrometastases in lymph nodes, image analysis would accelerate and increase consistency in pathology scorings and the time freed through these applications would allow pathologists attend more cases, not only within their institutions, but also worldwide, through digitalization and telepathology.

PART 1