New AI tool maps cancer cells with built-in doubt detection for safer diagnoses

An AI that knows what it doesn't know
HISTO-UNet flags its own uncertainties, telling doctors exactly where to look twice.

In the quiet labor of cancer diagnosis, where a pathologist's gaze must hold steady across hundreds of tissue images, a new artificial intelligence system called HISTO-UNet has emerged from a collaboration between Indian and British researchers — one that not only maps the boundaries of tumors and glands with unusual precision, but also confesses its own doubts. In a field where silence about uncertainty can cost lives, this tool introduces something rare in medicine: a machine that knows what it does not know, and says so.

  • Pathologists face an exhausting and high-stakes task — scanning hundreds of microscopic tissue images where a single missed boundary between healthy and cancerous cells can alter a patient's fate.
  • Existing AI diagnostic tools deliver verdicts without hesitation, offering no signal of how reliable those verdicts actually are — a dangerous silence in a domain where errors carry mortal consequences.
  • HISTO-UNet runs each image twenty-five times, measuring both the fuzziness inherent in the tissue sample and the gaps in the AI's own confidence, then flags the regions most likely to mislead even a seasoned eye.
  • Topology-preserving mathematical constraints force the system to honor the true shape of biological structures, preventing the fragmented outlines and phantom holes that have plagued earlier computational models.
  • Tested against three major medical imaging datasets, HISTO-UNet outperformed standard models — and while its repeated processing takes longer, it narrows a pathologist's workload to only the genuinely ambiguous cases, potentially accelerating treatment for cancer patients.

A pathologist scanning tissue samples for cancer faces a task that demands total attention across hundreds of images, each one potentially shaping a patient's future. A team of researchers from institutions across India — including IISER Bhopal, AIIMS Bhopal, and Jawaharlal Nehru Cancer Hospital — alongside the University of Oxford, has built an AI system designed to serve as a second set of eyes in that room. They named it HISTO-UNet, and what distinguishes it is not only what it sees, but what it acknowledges it might be getting wrong.

The system maps the precise boundaries of glands and tumors in microscopic tissue images using two interlocking innovations. The first embeds topology-preserving constraints into the neural network — mathematical rules that force the AI to recognize the true skeleton and center of biological structures, preventing it from drawing fragmented shapes, fabricating connections, or inventing holes that do not exist. The second teaches the system to express uncertainty: by processing each image twenty-five times and comparing how its answers vary, the tool measures both the fuzziness inherent in differently stained laboratory samples and the gaps in its own training and confidence.

This dual-layered uncertainty detection is something no previous diagnostic AI had successfully combined into a single framework. Older tools, including the standard UNet algorithm still widely used in hospitals, deliver a single diagnosis with no indication of how trustworthy it is — a silence that, in medicine, can be dangerous. When tested against three major medical imaging datasets, HISTO-UNet consistently outperformed these standard models in both accuracy and reliability.

The system's repeated processing does take more time than simpler approaches. But in a pathology lab where fatigue accumulates across long shifts, that trade-off may be worthwhile. By automatically surfacing only the most ambiguous regions — the ones most likely to challenge even experienced judgment — HISTO-UNet transforms an exhausting hunt through complex imagery into a focused review of genuinely difficult cases. For a disease where early intervention shapes survival, a tool that knows its own limits may prove as valuable as one that simply sees more.

A pathologist sits at a microscope, scanning tissue samples for the telltale signs of cancer. It is painstaking work—hundreds of images, each one demanding complete attention, each one a potential diagnosis that will shape a patient's future. Now, a team of researchers from institutions across India and the United Kingdom has built an artificial intelligence system designed to be a second set of eyes in that room, one that not only identifies tumors and glands but also admits when it is unsure.

The system is called HISTO-UNet. It analyzes microscopic images of human tissue, mapping the precise boundaries of glands and tumors with a level of accuracy that previous computational tools could not match. What sets it apart is not just what it sees, but what it knows it might be getting wrong. The researchers programmed the AI to flag regions where it feels uncertain, creating a kind of built-in doubt detector that tells doctors exactly where to look twice.

The team behind HISTO-UNet includes scientists from the Indian Institute of Science Education and Research in Bhopal, Maulana Azad Medical College, Jawaharlal Nehru Cancer Hospital and Research Centre Bhopal, All India Institute of Medical Sciences Bhopal, and the University of Oxford. Their approach rested on two key innovations. First, they embedded mathematical rules called topology-preserving constraints into the neural network—the type of artificial intelligence that learns by mimicking human cognition. These rules force the computer to recognize the true skeleton and center of a tumor or gland, preventing it from drawing fragmented shapes, making false connections, or inventing holes that do not exist.

The second innovation was teaching the AI to express uncertainty. The researchers programmed the system to process each image twenty-five separate times. By comparing how much the AI's answers varied across these runs, they could measure two distinct types of doubt. One captures the inherent fuzziness in the medical image itself—the result of different staining chemicals used in laboratories. The other measures the AI's own confidence gaps, the moments when its training leaves it genuinely unsure. This dual-layered uncertainty quantification is something no previous system had successfully combined into a single tool.

Older models, including the standard UNet algorithm that many hospitals still use, would deliver a single diagnosis with no indication of how reliable that diagnosis actually was. In medicine, where a missed diagnosis can cost a life, that silence is dangerous. Some newer experimental systems attempted to address shape errors, others to calculate uncertainty, but none had married both capabilities together. When the researchers tested HISTO-UNet against three major medical image datasets, it consistently outperformed the standard models, delivering greater accuracy and reliability.

There is a trade-off. Because the system must run each image through its process twenty-five times to calculate uncertainty, it takes longer to produce a result than simpler models. But in a busy hospital pathology lab, where fatigue can blur judgment after hours of staring at slides, that extra time may be a bargain. By automatically highlighting the most ambiguous regions—the ones most likely to trip up even an experienced eye—HISTO-UNet transforms the pathologist's job from an exhausting hunt through hundreds of complex images into a focused review of only the genuinely difficult cases. The system could accelerate diagnosis, reduce the chance of error, and ultimately get patients into treatment faster. For a disease like cancer, where time is measured in weeks and outcomes depend on early intervention, that matters.

By automatically highlighting the most ambiguous or difficult regions, HISTO-UNet could change the doctor's job from an exhausting, exhaustive search into an efficient review of only the most challenging spots.
— Research team
The Hearth Conversation Another angle on the story
Inventor

Why does an AI system need to admit when it's uncertain? Wouldn't that just slow doctors down?

Model

The opposite, actually. A doctor looking at a tissue sample needs to know which regions are genuinely ambiguous—where the image quality is poor, or where the pattern could mean two different things. If the AI just gives a single answer without flagging doubt, the doctor has to second-guess everything. By highlighting only the truly uncertain spots, the system tells the doctor where to focus their expertise.

Inventor

So the AI runs the same image twenty-five times and gets different answers each time?

Model

Yes, but that variation is the point. It's not a bug—it's the signal. When the AI's answers are all over the place, that tells you the image itself is fuzzy or the AI wasn't trained well enough to be confident. When they're consistent, you can trust the result.

Inventor

What happens if a pathologist ignores the uncertainty flags?

Model

That's the human element. The system can't force a doctor to do anything. But in a busy lab where someone is exhausted after looking at hundreds of slides, having the AI say "pay close attention here" is like having a colleague tap you on the shoulder. It's a safety net.

Inventor

Does this replace pathologists?

Model

Not at all. It makes them better at their job. It takes the exhausting, repetitive scanning work and automates it, freeing them to use their judgment where it actually matters—on the hard cases the AI flagged.

Contact Us FAQ