Simpson's paradox discussed, then, an AI comparison to 21gene test Oncotype.
##
Simpson’s paradox is when a relationship looks strong in the pooled data, but weakens, disappears, or even reverses when you split the data into meaningful subgroups.
In this breast-cancer AI setting, the idea is:
A model appears to predict ER status from H&E very well. But maybe it is not seeing ER-specific morphology. Instead, it is seeing grade/subtype morphology.
For example:
| Group | ER-positive tumors | ER-negative tumors | Model performance |
|---|---|---|---|
| Low-grade / luminal-looking tumors | many | few | model says “ER+” |
| High-grade / basal-looking tumors | few | many | model says “ER−” |
In the overall dataset, the model looks good because ER status is strongly correlated with grade/subtype. But if you ask a harder question — “Can it distinguish ER+ from ER− within high-grade tumors?” or “within Luminal B?” — the performance may collapse.
That is the Dawood concern. The AI may not have learned:
“This morphology means ER-positive.”
It may have learned:
“This morphology looks low-grade/luminal, and in this dataset low-grade/luminal tumors are usually ER-positive.”
So when I wrote “Simpson’s-paradox-like inflation,” I meant that the global AUC/accuracy may be inflated by hidden subgroup structure. Strictly speaking, classic Simpson’s paradox often involves a reversal of direction after stratification. In this setting it may be softer: not necessarily reversal, but an impressive pooled result that becomes much less impressive after stratifying by grade, subtype, receptor status, stage, or co-mutations.
For the blog, I might phrase it this way:
Dawood’s concern is essentially a Simpson’s-paradox problem: an H&E model may appear to predict a molecular biomarker in the overall dataset because the biomarker is unevenly distributed across grade, subtype, and other clinicopathologic strata. Once performance is tested within those strata, the apparent molecular prediction may weaken sharply, revealing that the model had learned a correlated phenotype rather than biomarker-specific morphology.
The direct comparison is in Witowski et al., “Multi-modal AI for comprehensive breast cancer prognostication,” Nature Communications, 2026; 17:5879. DOI: 10.1038/s41467-026-73088-y.
The comparison was against Oncotype DX, the standard 21-gene recurrence score assay, in 858 HR-positive/HER2-negative patients from three external cohorts: Karmanos, Basel, and UChicago. The AI test was not a pure H&E model. It combined digital pathology features from routine H&E slides, extracted with the Kestrel pathology foundation model, plus routine clinical variables including T stage, N stage, age, ER, PR, HER2, and ductal/lobular histology. In the whole study, the model was developed and tested across 8,161 patients, with 3,502 held out for external evaluation; the Oncotype comparison was a subset where Oncotype scores were available.
The headline result was that the AI test had a numerically higher pooled C-index for disease-free interval than Oncotype DX: 0.67 [0.61–0.74] for the AI test versus 0.61 [0.49–0.73] for Oncotype DX. Cohort by cohort, the AI test looked better in Karmanos and Basel, while Oncotype looked slightly better in UChicago. Karmanos: AI C-index 0.62 and HR 3.82 versus Oncotype C-index 0.54 and HR 1.36. Basel: AI C-index 0.70 and HR 3.98 versus Oncotype C-index 0.55 and HR 1.76. UChicago: AI C-index 0.67 and HR 3.26 versus Oncotype C-index 0.71 and HR 2.78. So the paper does not show a clean across-the-board win in every cohort, but it does show a favorable pooled comparison and a strong signal that the AI model carries prognostic information.
The more interesting analysis is the intermediate Oncotype group. Oncotype’s clinical problem is not really the obvious low and high extremes; it is the middle, where treatment decisions are hardest.
Witowski reports that among the 858 Oncotype-tested patients, the AI model would reclassify 666 patients, or 77.6%, into different risk categories.
All 526 intermediate-risk Oncotype patients were moved into either AI-low or AI-high risk groups: 423 of 526, or 80.4%, became AI-low, and 103 of 526, or 19.6%, became AI-high. Within the intermediate Oncotype group, the continuous AI score was significantly associated with recurrence, with HR 3.45 [1.85–6.42], p < 0.001, adjusted for dataset; the figure legend also reports a dichotomized HR of 2.84 [1.47–5.47], p = 0.002.
They also ran multivariable models. After adjustment for Oncotype DX score, Nottingham grade, dataset, and race, the AI test remained significant, with adjusted HR 2.95 [1.82–4.79], p < 0.001. In the same model, Oncotype DX had adjusted HR 1.43 [0.91–2.27], p = 0.12, not statistically significant. That is an important claim: not merely that the AI model correlates with Oncotype, but that it appears to add independent prognostic information beyond Oncotype and grade. They also state that the pathology component of the AI score remained informative after adjusting for clinical score and Oncotype, suggesting the model is not simply repackaging T stage, N stage, or receptor variables.
The authors also used decision-curve analysis at 5 and 10 years. They report that both tests beat the “treat all” and “treat none” baselines, but that the AI test provided greater clinical utility than Oncotype DX across threshold probabilities and both time points. That matters because a small C-index difference can sound abstract; decision-curve analysis asks whether the model would actually improve decisions at clinically plausible risk thresholds.
The implication, if validated, is substantial. Oncotype DX is expensive, consumes tissue, requires a molecular workflow, and is mainly used in HR-positive/HER2-negative disease. Witowski’s AI test uses routine H&E plus ordinary clinical variables, and the full paper reports prognostic performance across major subtypes, including TNBC, where Oncotype is not the standard tool. So the commercial and clinical vision is clear: an AI pathology test could become either an adjunct to Oncotype, a triage tool for who really needs genomic testing, or eventually a lower-cost competitor if it can show equivalent or superior clinical utility.
But this is where the caution matters. Oncotype DX is not just a prognostic test in the abstract; it is supported by major prospective clinical decision trials, especially TAILORx and RxPONDER, that linked recurrence score categories to chemotherapy decision-making.
Witowski is explicit that their study used observational data and did not directly test predictive value, meaning it did not prove that AI-high patients benefit from chemotherapy or that AI-low patients can safely avoid it. The authors hypothesize that a strong prognostic test may later prove predictive in the same way genomic assays did, but that is not yet shown.
So my bottom line would be: this is one of the strongest AI-pathology breast cancer prognostication papers because it directly benchmarks against a real clinical incumbent rather than only reporting internal model metrics. The AI test appears to outperform or at least rival Oncotype DX in recurrence risk discrimination, especially by resolving the Oncotype intermediate group. However, it remains a prognostic result, not yet a chemotherapy-benefit result. For clinical adoption, the decisive next step would be prospective-retrospective validation in trial cohorts, ideally asking whether the AI score predicts chemotherapy benefit or safely supports chemotherapy de-escalation.
