New Method Reduces AI Bias in Medical Imaging Without Retraining Models

A model trained to spot melanoma in one country can fail badly when it moves to another. Researchers at PLOS Digital Health have now shown why—and offered a fix that doesn't require retraining from scratch.

The problem is old and stubborn. When a hospital in Seoul builds an AI system using only its own patient images, the model learns not just what melanoma looks like, but also the particular lighting in that hospital's dermatology clinic, the camera equipment they use, the skin tones most common in their patient population. Move that same model to Australia, where the ISIC dataset was collected, and accuracy plummets. A model trained only on Korean data achieved 95.6 percent accuracy on Korean test images but just 76.5 percent on Australian ones. The reverse was equally stark: a model trained only on Australian data hit 89.2 percent accuracy at home but dropped to 78.8 percent in Korea.

This is not mere statistical noise. It is a form of bias baked into the learned features themselves—the model has internalized dataset-specific cues that have nothing to do with disease. The researchers call these "dataset-associated features," and they are the invisible scaffolding holding up unfair performance.

The team, working with skin lesion images, developed what they call the fair I-score method. The name comes from the influence score, a statistical measure that quantifies how strongly a subset of features predicts an outcome. The method works in five steps. First, train a model normally on combined data from both regions. Second, freeze that model and retrain only its output layer to predict which dataset each image came from—a task it accomplishes with 99.1 percent accuracy, proving that strong geographic signals exist in the learned representations. Third, use the influence score and a backward-dropping algorithm to identify which of the 512 features in the model's pooling layer most strongly predict dataset membership. Fourth, set a threshold based on the model's performance on a validation set. Fifth, remove the identified dataset-associated features—in this case, 126 out of 512—and keep the rest.

The result is striking. The modified model maintained strong diagnostic performance: on a combined test set, it achieved 90.5 percent accuracy, 85 percent specificity, and a 0.93 area under the ROC curve. More importantly, the performance gap between the two populations narrowed dramatically. The model fairness indicator—a measure of how equal the sensitivity (true positive rate) is across the two groups—improved from 0.915 to 0.947. In plain terms, the model now performs more equally across populations without sacrificing its ability to detect disease.

The researchers tested external generalization on a third dataset from Brazil, the PAD-UFES-20 collection. The fair I-score model achieved comparable accuracy and higher AUC than the baseline, suggesting the approach generalizes beyond the two training populations. Visualization using Grad-CAM, a technique that highlights which regions of an image the model attends to, showed the difference visually: the original model sometimes focused on background cues and image borders associated with dataset membership, while the fair I-score model shifted its attention toward the actual lesion.

The method has clear practical advantages. It requires no new data collection, no retraining of the full model, and no modification of the original datasets—a major constraint in healthcare, where data governance and privacy rules are strict. It is also interpretable: the researchers can point to exactly which features were removed and why. Unlike adversarial debiasing approaches, which add complexity and computational cost, or reweighting strategies, which require access to and modification of training data, the fair I-score method operates at the model level after training is complete.

The authors acknowledge limitations. The method currently works best for binary classification tasks and relies on the presence of strong dataset-membership signals in the learned features. Extending it to multi-class problems—diagnosing among many skin conditions, not just melanoma versus benign nevus—will require further work. The approach has been tested on VGG16, a relatively older architecture; validation on modern transformers and other deep networks remains ahead. And while the method reduces reliance on dataset-associated cues, it does not address all sources of unfairness, including those rooted in true differences in disease prevalence or presentation across populations.

Still, the work points toward a practical path forward for medical AI. As these systems move into real clinical use across hospitals and countries, ensuring they perform equitably is not optional. The fair I-score method offers a way to audit and correct for hidden biases without starting over.