Aleksandra: [00:01:18] Today I have Regan Baird from Visiopharm as my guest again, and this is the third and last part of our multiplex series. Today, we’re going to be talking about what to consider when choosing an image analysis solution for phenotyping. Hi, Regan. Great to talk to you again. How are you?
Regan: [00:01:38] I’m doing great, Aleks. Thanks for having me.
Aleksandra: [00:01:40] This is the third episode of our multiplex series. So for those who didn’t listen to the first and second part, let us start with a few words about yourself.
Regan: [00:01:50] Sure. Hi everybody. My name is Regan Baird. I am the regional manager of the Eastern North America for Visiopharm and Visiopharm has probably one of the world’s premier image analysis software for professional digital pathology image analysis.
Aleksandra: [00:02:07] And we started talking about how image analysis and computational pathology can help us with phenotyping. In the last episode, let’s [00:02:18] start now with take a little step back and talk about what computer vision tasks are involved in phenotyping.
Regan: [00:02:26] Sure. So the biggest challenge really in phenotyping and in doing multiplex, if you’re trying to identify every single cell in a piece of tissue, it’s what defines the rules for what is a cell and cell segmentation.
[00:02:42] So how do we draw borders around every single cell? So we know which biomarkers are associated with that cell. And it can be a challenge to elucidate those boundaries because we’re dealing with tissue cells and tissue are interacting, as we would expect, they can be of all different shapes and sizes and it also depends on the plane of sectioning of where cells are going to be captured. And so it can be a challenge to elucidate those edges of cells that are interacting in complex tissues with a lot of heterogeneity and also gets [00:03:18] worse as disease states get worse.
Aleksandra: [00:03:21] So we have the cell segmentation and we already mentioned a little bit the classification into different Can this be tackled with classical computer vision?
[00:03:32] We mentioned machine learning. It can be both, either classical or artificial intelligence-powered. there still a place for classical computer vision or do we need to implement AI for phenotyping?
Regan: [00:03:44] So two places where you’re going to deal with this. The first is in cell segmentation, as we just mentioned, and yes, you can create rules for segmenting cells and that has been done for a long time. It can be a challenge with rule-based assessments because you have to make concessions when you start making rules. And so you’re going to get close to cells, good cell segmentation, but it’s never going to be as perfect as people are hoping it to be. Most people when they think about cells in general, they think of a nice, [00:04:18] beautiful, thin confocal image where the boundaries of the cells are perfectly highlighted. And that’s what you want to see, but you just can’t get to that in tissue. It’s just not possible. And so when you’re making rules, you have to make those concessions also. And we talked about this in the last episode, when you’re making rules for phenotyping.
[00:04:38] And you have a high Plex, you could be making a lot of rules because there’s a lot of possible combinations when you have a lot of biomarkers to look at. And so starting to leverage machine learning and deep learning artificial intelligence can really help with both of these tasks. The first being cell segmentation, where if you can train a deep learning classifier to understand what nuclei look like or what cells look like, then you have a nice, powerful tool to be able to go in and just do that segmentation for you with the best approximation.
[00:05:13] And remember machine learning and AI is not magic. It’s all about [00:05:18] And does this look like a cell this is probably where the edge of the cell is to some high fidelity based on what we’ve trained it on. Now. Cell boundaries are pretty challenging to find a nice panel that has biomarkers for cell segmentation. But what we’re doing now at Visiopharm is we’ve got a tool that does know what nuclei look like in tissue and so it doesn’t matter the shape or the size of the nuclei. You can go in and find nuclei, and then we can expand the size of that nucleus to encompass the entire cell either by doing some simple growth of that nuclear signal or by adding signals from other biomarkers in the panel.
[00:06:00] So if you have a big macrophage or something like that, you can say, all right, I want to take my nuclear signal, but I want to expand nuclear signal to that of the entire CD68 signal from all of the macrophages. And you can start combining more and more biomarkers to get the expansion to [00:06:18] fit that of the entire cell.
[00:06:19] And again, when it comes to actually phenotyping, you can use machine learning, artificial intelligence to do auto clustering. Which is a technique that will automatically phenotype all of the cells in the tissue based on their biomarker expression patterns.
Aleksandra: [00:06:35] There are segmentation biomarkers. I didn’t know that they were, it makes sense.
But I guess the multiplex capabilities were not. So you couldn’t multiplex so many markers that you could afford in your experiments to add a segmentation biomarker.
Regan: [00:06:53] Yeah, exactly. And So it’s often overlooked in experiments, right? But now with multiplex, we have a virtually unlimited number of biomarkers that we can apply. Why not spend some of those channels on segmentation markers? And so if you can get a single or a couple of biomarkers that are ubiquitously expressed in all cells to help you delineate the edges of every single [00:07:18] cell, then you’re going to be much better off than trying to guesstimate from other signals in the panel.
Aleksandra: [00:07:25] If we are relying on image analysis, which we are.
[00:07:28] So this brings us to the next question. When we are considering software for phenotyping what software capabilities are we looking for?
Regan: [00:07:38] So as we said before, we’re probably going to be looking for segmentation assistance. we want something to help us delineate the edges of every single cell. So we know which biomarkers are inside of those cells are upregulated in those cells. And then we’re going to want some AI to help us automatically phenotype because as our Plex gets higher and higher, we don’t want to set the rules for.
[00:08:02]the phenotyping. But you don’t want to rely strictly on the computer to make all of those decisions. It still requires a good deal of data to train the AI and the machine learning models. And it also requires your [00:08:18] expertise as the person that designed the panel for which biomarkers you do want to see, and you do want to go after because the AI is only going to be as good as the image that we presented.
[00:08:29] There’s a lot of heterogeneity and there’s going to be mistakes that the computers are going to be making. Obviously, they’re all probabilistic models and so, it’s really up to the expert to determine in the end what is true and what is artifact.
[00:08:45] Other tools that are really useful from software is we’re dealing with high dimensional data. And so, being able to look at the image and turn on and turn off channels, even turn on and turn off biomarker channels rather than the imaging signals can be challenging because there are so many things going on at the same time. And really what you’d like to do is look at graphs, look at plots, look at two-dimensional reduction plots and data visualizations to help you interpret what’s going on not only in the one image that you’re looking at on the screen but also all [00:09:18] the images that you’ve probably multiplexed as population studies. And so I think that data visualizations are really an important tool to be able to evaluate how good the phenotyping has performed as well as to get some really exciting results out.
Aleksandra: [00:09:33] And what about the hardware? What do we need in terms of hardware? Do we need very powerful computers or can we do it on our laptops? How does that look
Regan: [00:09:43] Sure. Each software is going to have its own requirements, but from a Visiopharm perspective, really there’s not much more than is already being used in digital pathology. So I do it regularly on my laptop. There isn’t too much that’s needed from a computational standpoint. As you start to get into deep learning, you might have to upgrade your graphics card or the GPUs that you’re using.
Aleksandra: [00:10:06] So basically a gaming laptop would be something that we could use.
Regan: [00:10:11] Yeah, exactly. something that you used to mine cryptocurrency-
[00:10:14] Okay.
[00:10:16] … On the side.
Aleksandra: [00:10:17] Yeah. If we already have a powerful computer, we might as well use it to its full potential. And regarding image analysis skills, we already mentioned a couple of steps that maybe were not necessary when the multiplex was not so high plex-like data visualization. Is the barrier to entry for a person starting with phenotyping now higher or how do we approach this?
[00:10:48] Regan: [00:10:48] Sure. Really, the challenge is to be able to get a good set of images that you want to analyze and generate the rules that you want. So you have to understand the limitations of actually working with tissue, working with the Pre-analytical settings so that you are getting those good images. And once you have good images, then it’s knowing when to trust your eyes or when to not trust your eyes, because you can be confused by looking at some of multiplexes. It’s also knowing when to [00:11:18] trust the computer and when to trust your knowledge of the biology.
[00:11:22] And so together, the science and the AI really work well to try to get those answers. And once you finished the image analysis, you’re going to have a lot of multiplexing data. And so the next challenge is doing the data analysis to understand what’s going on once you’ve extracted all the information out of the phenotype images.
Aleksandra: [00:11:43] Mm-hmm [affirmative]. So it’s not only software or hardware but also your team. Me as a pathologist, I would definitely reach out to a data scientist to help me with sorting through the data because I would probably be the person interpreting it, but to sort through it and to extract the meaning, I would need somebody to help me with that.
Regan: [00:12:04] Yeah, data scientists are a phenotypers best friend.
Aleksandra: [00:12:08] Definitely. Thank you so much for letting us know what to consider when we choose software for phenotyping and for the whole series. I invite [00:12:18] all the listeners to listen to all the episodes. This is our last episode. So thanks again and have a great day.
Regan: [00:12:26] Thanks, Aleks. And if anybody out there has any phenotyping or multiplexing challenges, please feel free to reach out to us at Visiopharm.
Aleksandra: [00:12:33] I will leave the link in the description in our show notes. Thanks, Regan.
[00:12:38] Bye-bye.
Regan: [00:12:39] Great, Aleks. It was fun. Thank you.
