It is well known today that AI models can predict molecular information from images, microscopic images of the tumor, meaning from H&E slide images. Many studies have already been done on this topic. In fact, we already started developing such models 10 years ago. And two months ago, one of our studies was published in Lancet Oncology, where we developed an AI model to predict the oncotype score from H&E images and validated it on the Taylor X randomized trial...
It is well known today that AI models can predict molecular information from images, microscopic images of the tumor, meaning from H&E slide images. Many studies have already been done on this topic. In fact, we already started developing such models 10 years ago. And two months ago, one of our studies was published in Lancet Oncology, where we developed an AI model to predict the oncotype score from H&E images and validated it on the Taylor X randomized trial. Now, although these studies showed better and better performance over the years, this performance reached a limit, meaning that there are always cases where the AI model is wrong surrogates of the gold standard molecular assays, which highly limits them clinically. What we show in our new study here at ASCO is that the mistakes of the AI are not actually random errors. We show that AI learns new information that it was not directly trained to predict. And what happens is that the model doesn’t really see the molecular information in the image. It doesn’t see the receptors or the gene expression. Instead, it sees the downstream effect of this molecular status reflected in the image. For example, a tumor can be ER positive by immunohistochemistry, meaning that it has estrogen receptors, but still unresponsive to estrogen because its ER pathway is inactivated. For example, a tumor can be ER positive by immunohistochemistry, which means it has estrogen receptors, but still unresponsive to estrogen because its ER pathway is inactivated. This means that the tumor may act like an ER negative tumor, which means that it may look like an ER negative tumor. And this is what the AI eventually sees in the H&E image. So the AI predicts it as ER-negative, meaning that the AI doesn’t see the receptors. It sees if the tumor is responsive to estrogen. We also showed that these discordances between the AI predictions from H&E images and the molecular assays are biologically meaningful and can be even useful for better guiding clinical decisions.
This transcript is AI-generated. While we strive for accuracy, please verify this copy with the video.
