AI model identifies 44 exoplanet candidates, but 99% precision applies only to simulations

A machine-learning model trained on simulated planetary systems has produced a shortlist of 44 real stars worth observing more carefully—but the headline number hiding inside this work requires careful unpacking. The classifier achieved 99% precision, a figure that sounds like a triumph until you learn where it was measured: not in the actual sky, but only within the synthetic universe where the model learned its rules.

The researchers, led by Jeanne Davoult, Romain Eltschinger and Yann Alibert, built a random-forest classifier—a voting system of 500 decision trees—trained on 53,882 simulated planetary systems generated by the Bern global model of planet formation. The model's job was to predict whether a real planetary system harbored an unseen planet in a specific mass and temperature range: between 0.5 and 3 Earth masses, with equilibrium temperatures between 160 and 510 kelvin (roughly minus 113 to 237 degrees Celsius). The researchers called this target an "Earth-like planet," though the label describes only a narrow mass-and-temperature class, not another Earth. It says nothing about whether a world is rocky, water-rich, or capable of supporting life.

The synthetic advantage is that the Bern model knows every planet it creates, including worlds too faint for current instruments to detect. The researchers masked these hidden planets using detection thresholds based on the radial-velocity wobble they would produce—essentially asking the classifier to infer an unseen planet from the visible architecture of a system. When they divided their synthetic data into training and test sets, the model reached 83% precision at a standard threshold. But when they demanded much stricter consensus—requiring 90% of the 500 trees to vote yes—the precision jumped to 99% on the held-out synthetic test data. Five false positives among 715 positive classifications. That is where the 99% figure originated.

But the same test revealed a hidden cost. At that strict threshold, the model missed 861 systems that actually contained a qualifying planet. The recall—the fraction of true positives it caught—was only 45%. The classifier was designed to concentrate confidence, not to find every candidate. Telescope time is scarce, so a short list with few false alarms, even if incomplete, can be strategically valuable.

When the researchers applied this model to 1,567 real planetary systems assembled from observations, 51 systems scored above the 90% threshold. They then removed seven binary-star systems, since the Bern populations used for training contained only single stars. The final list contained 44 systems: HD 103949, HD 42618, HD 85390, HIP 41378, Kepler-22, Kepler-538, and KMT-2021-BLG-0171L among them. These are already known systems whose visible planetary architectures resemble synthetic systems that often contained an additional Earth-mass planet. The classifier did not detect a new transit, isolate a new radial-velocity signal, or image a new world. It identified places where follow-up observations might be unusually productive.

The central limitation is what machine-learning researchers call domain shift. The Bern model is a detailed physical framework that reproduces several broad patterns in observed systems, but it also contains systematic mismatches. Its synthetic populations contain at least 1.7 times too many planets per system. The planets tend to orbit closer to their stars than observed planets do. Too many occupy mean-motion resonances. The simulated association between inner super-Earths and outer cold giants is weaker than what observations suggest. These are not cosmetic details when the model's inputs are system architecture and the innermost visible planet. A classifier can be excellent within the statistical world that generated its training data, then perform differently when deployed in nature.

The 99% precision was measured only within the synthetic test set, which was generated by the same Bern framework as the training systems. They shared its formation rules, population assumptions, and simplified observation filter. The 44 real systems are different. There is no answer key showing which ones contain an undiscovered planet in the chosen mass and temperature range. The study includes no observational campaign capable of measuring real-sky precision. A 90% voting rate does not automatically mean there is a calibrated 90% probability that a real star hosts an Earth-mass planet. Calibration would require checking model scores against many real systems whose full planetary inventories are known—precisely what exoplanet surveys do not yet possess.

The authors performed a second check after producing the shortlist. They inserted a hypothetical Earth-mass planet at locations within the target zone and used a mutual-Hill separation criterion to ask whether it could fit between the known planets without making the spacing obviously unstable. Forty-two of the 44 systems could accommodate at least one inserted planet. This makes the candidates more plausible in a limited geometrical sense. It does not mean a planet formed there, survived there, or currently occupies the available orbit. The calculation is not a full long-term integration across unknown masses, inclinations, and eccentricities. "There is room" is valuable when deciding where not to look. It remains a long way from "there is a planet."

The real experiment is whether the model's predictions transfer from simulation to nature. Positive results would support the idea that system architecture contains recoverable information about unseen small planets. Negative results would be informative too, exposing where the Bern population or the simplified detectability thresholds diverge from actual surveys. The authors point toward missions like PLATO, designed to find and characterise terrestrial planets around nearby stars, as a path to expanding the evidence. For now, the 44 systems are a map of where the model thinks observers should look. Only repeated measurements can reveal whether the same map works under a real sky.