[00:01:42] Aleksandra Zuraw: Today my guest is Tuomas Ropponen, the Chief Technology Officer at Aiforia. Hi, Tuomas, how are you today?
[00:01:49] Tuomas Ropponen: Hi, Aleksandra, thanks for the invitation, and nice to talk with you.
[00:01:52] Aleksandra: Thanks for joining me today. Let’s start with you. You’re the Chief Technology Officer at Aiforia, but tell us about your background and how you joined the company and what is exactly your role in the company. What does a Chief Technology Officer at Aiforia do?
[00:02:10] Tuomas: So first, for the background, so I started in the Technical University of Helsinki, computer science, machine vision, software development, robotics, autonomous cars. Actually, from my master thesis i did it from machine vision. We started the first company, doing machine vision system for the heavy industry, like automotive and paper manufacturers. I have 20 years history in software, machine vision, creating products not research. And actually, the University that I studied the invention of backpropagation, the algorithm that is used for neural network teaching. The invention was done 51 years ago. It’s the first known implementation. So there’s a long history in that university for the neural networks, much before the hype.
[00:03:04] Aleksandra: Which the hype is just the recent, what, five, seven years? Not even 10 years.
[00:03:10] Tuomas: Five years is maybe the hype, or seven. There has been couple of hypes before already, but this is the maybe the second hype or third hype of the neural networks.
[00:03:23] So every technology has this hype curve, of course. And now, we are may be in the upper corner. Let’s see.
[00:03:31] Aleksandra: So you said that you worked in product development in different companies. How did you join Aiforia? Because what you said before, it had nothing to do with pathology. And then what motivated you to join a company that is providing services for pathology actually?
[00:03:50] Tuomas: Actually, I met one of the current workers that were in the starting of the Aiforia. They work at this medical device robotics company that was doing automatic pipetting. I was leading the software team there, so I have changed company. They’re asking that, “Do you know anybody that would like to come to us?” And I said, “I know myself.” I joined the company.
[00:04:21] Because I knew that it is five years ago when I started this company, so I knew then that the deep learning is just coming and it’s ready for the commercial use, not anymore academic research. It was good time for me to join the company
[00:04:37] Aleksandra: You joined as the chief technical officer, what are your responsible for as chief technology officer at Aiforia?
[00:04:48] Tuomas: Basically, I’m leading the software R&D that is tightly related to AI R&D, so also testing the neuron networks, doing a lot of practical things together with pathologist trying to build the platform for them. CTO has many roles in a small company, but main job is leading the AI platform development.
[00:05:13] Not just the models or the software.
[00:05:16] Aiforia has a cloud-based artificial intelligence for image analysis platform for pathology. It was founded in 2013. First, let’s start with what drove the decision for the platform to be cloud-based? Because I don’t think there was anything cloud-based in 2013, and now, even though it is a great technology, not everything is cloud-based. Why did you decide it should be cloud-based? How did you know?
[00:05:49] Actually, the history of the product is much longer than the 2013, so it was under University oof Helsinki maybe beginning of 2000, and the platform, this kind of image sharing platform started with teaching the medical student of the pathologist, and this kind of seminars or example, I just found that 2006, the ECP, European Pathology Conference was hosted by the former name of Aiforia web microscope, so that’s actually almost 20 years history of this platform.
[00:06:29] But it was under the University of Helsinki or the Finnish Molecular Institute where basically this company’s spinoff for that platform. It basically has been run always in the cloud or server-based, web-based system.
[00:06:50] Aleksandra: Basically, the origin was telepathology was sharing images remotely, and then …
[00:06:58] Tuomas: Yes.
[00:06:59] Aleksandra: … on top of that, the image analysis capabilities were built?
[00:07:03] Tuomas: Yes. The origin is this image sharing telepathology type of platform for research use.
[00:07:11] I was hired five years ago to build the AI platform on top of that, so doing the AI, functionalities are needed for AI development.
[00:07:21] Because I think that the cloud-based systems, if you think about AI, it’s about teaching. Who is the teacher of AI in pathology? So pathologists. You need to have easy access to see what the AI has learned and how to do annotation. The teaching is done by annotating, so if you have some server that is in the basement of some company, how to get easy access for the pathologists? Having web-based system that runs in the cloud solves the many practical problems, how to teach AI, how to get pathologists easy access to AI teaching.
[00:07:56] Aleksandra: How did you know that it’s going to be supervised deep learning? Because again, okay, you said that the origin of the company and origin of the product is even before 2013, the official foundation of the company. It was only in 2012 that deep convolutional neural networks started outperforming the classical image analysis method, classical computer vision method in the natural image processing challenges, this image net challenge, one of the famous ones. How did you know in 2013 or five years ago when you joined, that it’s going to be supervised deep learning?
[00:08:43] Tuomas: If you talk about unsupervised, everybody wants to have unsupervised AI. Basically, you draw just data, and it tells you what to do. I wouldn’t really recommend that with cancer. Do you let your kid to internet and self-learn what is the world. We don’t let humans to be in unsupervised learning. It’s strictly supervised whole school system. There’s many practical problems with unsupervised learning. Everybody wants to do it, because it seems that easy. It’s just raw data and it self-learns the stuff. There’s really few practical application that can be done with unsupervised learning.
[00:09:22] You will get … if you can create ground truth, what is cancer, what is benign, then the supervised learning will beat that always. Of course, the practical problem is with the supervised learning. It’s that how do you get the ground truth? How you can get the access from the pathologist knowledge? How can a pathologist annotate what is cancer or benign? What is in the border? Those are the challenges in the supervised learning. Because pathologist’s time is really limited, that can try to always minimized.
[00:09:55] Why the convolutional neural network? I think it’s the competition wins both. There’s couple of independent … One German that actually, they did mitosis counting 2012 with convolution , it was image net, it wat the same time. Actually, the invention of convolutional … I think it’s convolutional deep neural network was the region. Also, creating practical tools that it can be taught with GPUs. You need a lot of computational power to teach convolutional neural networks. Also, there was a lot of libraries coming out that you don’t need to hard code it. Basically, you don’t need to have a lot of coders to do the … how to teach AI in GPUs.
[00:10:42] There’s a lot of these small things that happened in 2012, 2013, 2014. It was ready for this kind of commercial application I think that then … It doesn’t matter what is the system that you want to teach AI human, so you need to have the good study book. Basically, what we are doing, they’re creating the … not rules, but the book for the AI what needs to be studied. Examples of the cancer, examples of benign, example of mitosis. What is stroma? What is epithelium?
[00:11:15] What is not related? What is basically different type of tissue like lung tissue or prostate or … there’s a lot of sub domains even in the pathology world. You need to basically draw it. That’s the best way to teach it.
[00:11:31] Aleksandra: Yes. You are on the technology side of things, responsible for leading the product development, all aspects of product development. First, was this always meant to be for pathology? Or was it for some other medical imaging at the beginning? You said that the founders were at the company that was using machine learning for different purposes for pipetting. Was pathology from the beginning?
[00:12:04] Tuomas: Pathology, for this company, Aiforia, it has been pathology for the beginning. The Molecular Institute of … so it’s research institute in Helsinki University. They have been huge digital pathology was the main platform, so for this company, it has been always the pathology first. Because we can do pathology. This technology actually fits really well also for the satellite imaging that’s really close technically to pathology. Basically, what is forest, what is tumor area? How many rivers? What is the stroma?
[00:12:37] There’s a lot of analogy in … and same technical problems. The darn large images that doesn’t fit any computer memory, or really few memory can handle those. Similar, you cannot see the data in one class. You need to zoom in and pan it. Technically, it’s same than satellite imaging, pathology tech side.
[00:13:02] Aleksandra: How was the initial process of matching this technology, the Aiforfia technology to the needs of pathology? What was your experience in this, coming from a non-medical background, non-pathology background?
[00:13:19] Tuomas: Maybe the key text that we match it, so that you need easy access for the pathology. No need for any in software installation or go to some basement to some dedicated computer. Web browser is basically the technology side, so user interface has been always been in the website. It’s difficult to make it, working as efficiently on the desktop computer. But the easy access, and the easy access is really crucial when you’re teaching the AI models. If you have multiple example, one prostate pathology from U.S. and one from Europe, and they need to create consensus or check what will work.
[00:14:01] How do you create that easy access for AI and pathology’s actually doing … at least did in digital pathology, the annotation is part of the work. Not maybe for AI but marking the tumor area for the diagnosis. Annotation is the communication method with the AI. Because you’re communicating already with annotations, then you need the supervised technology, and the state of the art is at the moment than was five years ago, the deep learning convolutional neural networks.
[00:14:35] Easy access state of the art technology is the key crucial choosing points.
[00:14:42] Aleksandra: When you started and you gave this tool to the pathologists, how were they working with you? What were the problems that you encountered? Both on the technology side and on the pathology side.
[00:14:57] Tuomas: Actually, there’s some commonness in between pathologist and computer guys, or computer nerd scientist. We like details much more than maybe marketing persons. The similiar mind and maybe more technology than humans. That’s average. Radiologists, pathologists, computer guys are maybe a little bit different than average persons. There’s much more common I think. But of course, the language and ways of working are quite different. Language is the main thing. I was in the first meeting with maybe eight pathologist trying to explain, “This is stroma and this is epithelium,” and then I said, “This is a forest and these are the agriculture of fields.” I needed to have a common language.
[00:15:45] I can recognize the patterns in the image, but the terms of the pathology was totally new for me. The pathologist laughed at me, but they were really happy that they found common language with me.
[00:15:56] Aleksandra: This is great.
[00:15:58] Tuomas: So, there’s a lot of analogy in the satellite world maybe. That was the easy way to start. Of course, I know now basic terms after five years, but still there’s 90% of the pathology terms is not familiar. Having common language is important.
[00:16:16] Aleksandra: Definitely.
[00:16:18] You found so both the computer scientists on your side and the pathologist, you found a common language that helped you work together. You kind of figured out how to translate the pathology pattern names, or the patterns into something that you knew. You said satellite images of forests, of rivers, and geographical formations. The communication crucial. I never looked at it from that point of view, that actually regarding maybe character and personality, there are a lot of similarities between computer scientists and pathologists.
[00:17:02] Once you figure out a good way to communicate then it’s actually not as difficult as you would imagine.
[00:17:11] Tuomas: That’s actually I think the personalities are much more similiar than the previous so-called customers that … or partners where we are making the software.
[00:17:22] That’s quite unique in the … When you’re talking to communication, because of course, computer scientist and coders need to understand where they are developing the platform, and also this technology needs to be taught also to pathologist. There’s a lot of hype in the 2012, 2013, 2014. Because a couple of major companies maybe over marketed the AI. Everybody thought that it’s AI is this magic box that you throw data and images and it tells you what is the cancer and not cancer. That was totally untrue for my side.
[00:18:07] There are many people still think that you buy AI, and you don’t need to communicate. It just does the work. But actually, it’s a lot of … in Aiforfia, our pathologist are teaching the AI, not the computer scientist. 90% of our work or AI model teaching is done totally without computer scientist or machine learning engineer or machine learning scientist.
[00:18:36] The idea was to teach that, we created the platform for pathologist to teach the AI. Of course, there’s a cultural difference that now in our company, the pathologists are actually R&D developers. They are teaching the AI, so they are developing the AI model. That’s different culture than clinical pathology, or even research. All the what is agile and Scrum and kanban, this kind of computer … where the computer nerds are familiar is ways of working and R&D. And also a little bit teaching that what are the basic principles of AI or deep learning? But that was the easy part. The communication is the most crucial, and still I think the pathologist basically if I understand correctly, the last 400 years that it has been same viewing from the microscope.
[00:19:38] Aleksandra: Yes
[00:19:39] Tuomas: Detailed pathologist even knew. Maybe not all the pathologist are so familiar with computers, and computer stuff. Actually, it’s darn large revolution for the pathology, changing in couple of years or 10 years.
[00:19:59] Aleksandra: But you say okay, so from the pathology side, it was more or less the same process of they were building another part of the bridge. They had to learn what is scrum, what are the steps of software development I guess. Also, a little bit about deep learning. It’s interesting to hear that, so there’s a little part from each side regarding the basic knowledge, but then the main part of this bridge is the communication.
[00:20:32] And actually, even the FDA is recommending that in the AI pipeline, you need to have this kind of quality review. Who is the person that can do in pathology the quality review? It’s the pathologist. Only person that can check that. Does it recognize cancer or not, who is qualified.
[00:20:54] Tuomas: Actually, in that process, that good machine learning process or pathologist is still needs to be in the loop, even the development of the products and continuous improvement. Like Tesla’s improving all the time the self-driving car. I don’t believe that when you buy now AI that it will be the best possible, because everybody knows that if you give more teaching material, high quality material from the pathologist, it will be better. If you can save lives creating better more AI model or AI product, I think that it needs to be continuously improved.
[00:21:32] If something changes in the pathology or medical domain, you recognize some biomarker, or you … Actually, I think that because the AI makes the possibility to read slide in every pixel, so it can give also a lot of information for the pathologist that was impossible to procedure before. There’s not enough pathologists do it manually.
[00:21:59] Aleksandra: That’s true.
[00:22:01] Tuomas: It will change the research in many, many medical domain. Or not medical, but I mean pharma research, medical practice, and also this kind of outcomes. What is the best treatment for different type of tumors? There’s a lot of this research that it’s not yet have been done that can be done with the AI.
[00:22:19] So there’s a lot of learning from the pathologist to utilize this technology. You don’t need to be super expert in deep learning. It’s basic principle of teaching something. Even it’s similiar to teaching humans. During this company, my boy has been grown from one year old to six year old, that is actually a bit similiar than AI when he’s learning. At least in the beginning.
[00:22:48] Aleksandra: In the daily business at Aiforia, what are the ways or processes when in your daily work to stay aligned with the pathologist working on projects or on the product itself? Because you say they’re using it continuously, so I assume they communicate what the user requirements are, what the improvements should be. What are the processes for communication? Like when you sit together, how do you do it?
[00:23:21] Tuomas: Actually, when we started this office and hired first pathologist to be … they were basically next room from the coders. Actually, there’s one wall between, so creating small team that doesn’t have this kind of barriers. That was really important in the beginning. This COVID situation has a little bit changed the situation, so we have transferred the digital Slack messages, but still, there’s this open channels that the pathologist or microbiologist and medical students that are using our platform daily. That they have easy access to there what is not working, what should be working? And now, of course, we have the UX designers are crucial in addition to that team.
[00:24:09] Coders are not maybe best always saying that what is the user interface to use AI or teach it? We have between the coders and pathologist user experience designers that are familiar with talking different so-called domains and interpreting the needs. But still, the direct conversation with the UI developer and pathologist, or the AI developer or AI scientist and pathologists, I think is crucial. Before that, of course, you need to have the common language.
[00:24:44] Aleksandra: Definitely. How often do you release new version more or less?
[00:24:51] Tuomas: I think the new version for the internal users, maybe weekly.
[00:24:58] Aleksandra: Oh, weekly.
[00:25:00] Tuomas: Yeah, we are rapid
[00:25:03] Aleksandra: Okay. That’s frequent! That’s very agile development.
[00:25:08] Tuomas: That’s for the internal users. But of course end customers are getting … so pathologist in our house also our so-called testers, testing the AI models, testing the user interface. They’re using not this kind of polished product always. It’s better to get the feedback as fast as possible, because we are in competition against many, many … not many, but some companies. We want to be agile, and if we have an in-house pathologist team that is using daily the product, and you are not afraid of updating the product. Of course, we will test everything, but still, updating … so they will get the new features to be tested.
[00:25:52] If it doesn’t solve the problem, we get the feedback immediately and can cancel the feature or improve it. If it’s good, then we can push it to the official production releases that are done maybe four times a year.
[00:26:05] Aleksandra: This is really fast and a great process. I love to hear that you’re working with pathologists like that. Because this is immediate feedback from the users. This is really amazing.
[00:26:22] Tuomas: It can be done also only this kind of research environment that not clinical workflow, this kind of factory like that we’re giving 1,000 patients a day. But in research use, that our pathologists are using for medical research, publications, teaching the AI models for the research use. Then we are getting this option to get the feedback much faster. I think that it’s crucial when you’re developing.
[00:26:46] Aleksandra: To get thefeedback, do you have daily meetings, weekly meetings? Do you don’t have meetings? You just chat on Slack, or how does this happen?
[00:27:00] Tuomas: Before the COVID, there’s a lot of this kind of coffee room discussions, because there was basically one meter or two meter between the pathologist and coders. It was really easy. Of course, there’s official weekly meetings between the teams or team leaders. Also, keeping all the time the open channel.
[00:27:21] We are using Slack as a messaging software, and we have open channels for the AI developments of development, in each analysis clinical development. Having this kind of direct conversations grouped with these kind of channels is I think COVID way of working. Also, because we are not anymore all the people are not from Helsinki or Finland, so the electronic comunication, Slack messaging and those are much easier with this kind of growing company.
[00:27:52] Aleksandra: Definitely. In this process, can you tell me about a misunderstanding between the pathologist and computer scientist that you had, something that you had to solve? How did you solve it? Something that caused trouble.
[00:28:08] Tuomas: Actually, one of the first … there was this one of the first AI models that I did, I was personally doing, the annotation actually, even was fat liver. It’s basically white holes in the liver tissue, so it’s not technically.
[00:28:21] Aleksandra: Yes, but I mean this is how you have to translate those patterns to people who are not pathologists. You basically describe what it visually is, so exactly white holes. The fatty vacuoles in the liver look like white holes.
[00:28:37] Tuomas: Actually, one analog for the white holes is that actually in concrete industry, there’s some bridges falling down, and they are staining the concrete with this kind of same color than hematoxyline so this kind of pink. The white holes are when the water comes out of the concrete, so more you have holes, the more brittle, easier the bridges will collapse. They’re actually doing same measurement in concrete industry, so white holes in the image. But in pathology, you have then more fat in the … more brittle liver.
[00:29:13] But the maybe miscommunication is that I worked couple of days, because unnecessary maybe, because I was afraid of asking pathologist. I got a list that this has now 20%, 70%, 50%, 60%, the pathologist was scoring the liver based on percentage. Then I had an AI model that can calculate exactly the pixel areas of the fat cells, compared to liver tissue. I got percentage from 0.5 to maybe 16%. None of the numbers for 100 or 80%. I was, “Why my AI model doesn’t work at all according to ground truth?”
[00:29:59] It was basically hard lessons that pathologists are using normalizing sometimes the scales, so most severe fat liver can have 90% fat. But actually, if you calculate the pixel areas, it’s 60%. Because [inaudible] in the fat cells, there’s a lot of tissues. Maybe the misunderstanding is [inaudible] some of the scoring systems are this kind of in pathology, they are not actual pixels. Because no pathologist will draw every area or count pixels.
[00:30:34] Sometimes the scoring systems are differently normalized. Most severe cases the highest score and less severe is the lowest score. But what is the scoring system in pixels? That’s either quite easy to make a misunderstanding. Actually, some publication, they actually prove that. Pathologist, always when we solve the results, then the pathologist, those medical experts say that, “It can’t be 16%.” They don’t believe the results of the pixels, because they have always normalized the tissue.
[00:31:09] We needed to do this kind of blinding that we draw the pixel area in this kind of manual tool of the fat cell, and people will ask you that, “What is the score?” They’ll said 80%. When we draw the exact pixel areas, and they can see when we measure it, and it actually 20% or 16%. I think that many of the grading system in pathologists are normalized somehow in the medical domain, so they’re not exact. Not exact this kind of numbers of cells or areas.
[00:31:43] Aleksandra: Yes. This is very important thing that you’re mentioning, the translation of pathology scoring into even before the scoring as output from an AI model, into building a model, this is I think an art of its own, and this is where communication between pathologist and computer scientist is even more important than for all the other things to avoid the misunderstandings like you said. Very, very interesting.
[00:32:17] Tuomas: That’s crucial for the … I have met a couple of mistakes that were the same after that… Because I’ve read many of the scoring systems, and still the liver fat was the easy explanation, because I thought that it’s the pixel area, but when computer guy talks, it’s area of pixels, or area of micrometers or count of objects. Some of the scoring systems are not exactly that. If you think about Gleason grading, you have different patterns or melanoma grading is much more difficult even. And computer guy thinks that example if you have five grade systems, you have benign is maybe zero and one is the starting malign or the first cancers to example in melanoma.
[00:33:08] The agreement with the pathologist … it’s difficult problems to estimate things, and computer guys are calculating exactly, and translate that to the … I think many of the scoring system cannot be tought directly to the AI.
[00:33:25] One thing, that in real world, when we talk that the scoring systems are this kind of grading systems, they are normally this kind of continuous. When the cells or tumors start to mutate, there’s no exact orders. There’s always something between, so it’s a continuous curve. Actually, it should be a number like Gleason 3.5, but no human can put that number. There’s still a lot of things that needs to be investigated to get all that can be done with AI.
[00:33:59] Aleksandra: What would be your recommendation? Let’s take this problem of translating scoring into an AI model. What would be your recommendation for the best collaboration between computer scientist and pathologist? How do pathologists explain this scoring best to the computer scientist and how should computer scientist … what do they need to know? What are the requirements from both sides to solve this problem?
[00:34:27] Tuomas: Of course, you can teach this kind of classification. But it’s always better that it can be converted to area of something or count of something. This kind of numeric exact things that you calculate area of tumor area, inside of stroma, or whole tissue, this kind of tumor burden. If it needs a discussion between the pathologist and computer scientist. Can it be converted this kind of exact pixel number, so area of micrometers in pathology or number of cells compared the biomarkers like Ki-67 are much easier. But actually when you need to detect also the epithelium or cancer epithelium, then it’s what is the border between cancer epithelium and benign? There’s always this kind of … they’re not easy to anybody.
[00:35:23] Aleksandra: No.
[00:35:23] Tuomas: How to convert? Because current pathology systems are designed or the scoring system are designed for humans. You can fastly do good enough diagnosis. It’s a Gleason 5, Gleason 5, you have three minutes or five minutes time to score it. There’s no possibility count one million lymphocytes in cancer.
[00:35:43] Aleksandra: No.
[00:35:44] Tuomas: And also the discussion is important because then the AI can do the counting of the area, or counting of the cells. They can do the whole slide. Is it one million, 151 lymphocytes or positive Ki-67 cells? And count area of the tumors and then it’s that how the current system of creating can be converted, so it needs a long discussion. Example Gleason grading, there’s a different opinions on how it can be converted.
[00:36:16] Like opinion that Gleason grading, do you estimate in gland level? Or sub-gland, because some part of the gland can be Gleason 5 and we make patterns, so patterns are not … or do you need this kind of 500 micrometer area that is sane. What is exact way basically this kind of fast visual diagnosis that is done in multiple level of the tumor, and multiple areas? How it can be converted to this kind of common rules? I think there’s a lot of discussion to be done.
[00:36:49] Aleksandra: Yes. I think there is not always one answer that is correct. It may be that you decide for something, and then other team is going to decide for something else. Because like you said, the current scoring is designed for humans to do it fast under the microscope. It’s not always the best. It’s not always objective. This is basically the best we could come up with before we had the tools that we have now. Now translating this into our tools … it’s a totally different challenge, so thank you so much for presenting this to us, and for pointing this out to the listeners. I think this is really important. I think this is something that’s underestimated, the thing that you mentioned that pathologists visually normalize the scores.
[00:37:46] Often they are higher than the pixel count. I don’t think we’re always aware of that, and then like you said, there can be misunderstanding with, okay, the model is not matching the ground truth. Our ground truth is annotations, it’s a different story. But if our ground root is scoring, that’s another problem.
[00:38:10] Tuomas: There’s a lot of things that needs to be … and there’s a couple of dangers even that you change the scoring that the hospital has used, the oncologist used. There’s a lot of practical problems coming up when the AI is utilized in clinical. There’s no global ground truth for many of the scoring systems. At least for the grading systems.
[00:38:33] Aleksandra: Yes. I think that means that computer scientist and pathologist are bound together for a long term if we are to use AI for our … to power pathologists. We just keep improving our communication. Thank you so much for joining me today Tuomas.
[00:38:55] Tuomas: Thank you for invitation. It was nice to talk with you.
[00:38:59] Aleksandra: Have a great day.
[00:39:01] Tuomas: Thank you.
