Experiments 19

Lost in Translation

Cancer drugs are tried on cells in a dish long before they reach people. Each square is one prediction from the dish: its upper half is what the cells did, its lower half is what 25,000 patients did. Drag from promise to patients and watch what survives.

Python · PyMC · DuckDB · DepMap · MSK-CHORDView source 

26 sure calls in the dish2 held up1 reversed23 can’t tell

Genes shown

drug works better with it drug works worse with it paler is less sure named on the drug’s label
Promiseif every dish result carried overPatientswhat 25,000 records showed

All five cancers: 73 sure dish calls, 5 held up, 2 reversed.

Notes on how it works

Notes

The question

Before a cancer drug reaches people, it is tested on cancer cells grown in dishes. When cells carrying some mutation die faster on a drug, the mutation becomes a candidate biomarker: a hint that patients with it should get that drug. Thousands of these hints exist. Very few are ever tested in patients. This page tests all of them that public data allows, at once, against the treatment records of 24,950 patients.

The dish

Mutations and copy number come from DepMap’s 1,968 cell lines, and drug response from the PRISM screen, which dosed about 480 lines with each drug at eight strengths. For every gene and drug class, a Bayesian model asks whether lines with the gene altered respond to that class differently from how they respond to other drugs. It looks within each cancer type, borrows from other cancers when a type has few lines, and corrects for how many mutations a line carries, so that a gene marking fast-mutating cells does not borrow their response.

The bedside

MSK-CHORD follows patients with lung, breast, colorectal, pancreatic and prostate cancer through every drug they were given. Each course of a drug class counts until it was stopped, and only courses that began after the tumor was sequenced count. The model compares patients with and without the alteration on one class against the same comparison on their other classes, adjusting for earlier treatment and tumor mutation burden. A mutation that simply marks a harder cancer moves every class together, and that cancels out.

Sealed before it met the patients

Every dish prediction was written to one file, fingerprinted and time stamped at 2026-10-08 20:26:12 UTC, before the patient model saw any of those genes. The fingerprint begins 1099935f8fba. Nothing about the dish side was changed after that.

Checking the instrument first

The patient model had to pass a known-answer test before it judged anything. It was given every drug and mutation pair that an FDA label names, such as EGFR pills for EGFR-mutant lung cancer, and 389 pairs of randomly chosen genes that should do nothing. It recovered 12 of the 19 label pairs it had enough patients for, never pointed one the wrong way, and flagged 2 of the 389 random pairs.

What it found

Across the five cancers the dish made 73 sure calls. 5 held up in patients, 2 went the other way, and the rest the records cannot settle. The genes on drug labels carry over cleanly. Beyond them, how strongly a mutation moved cells in the dish says nothing about how patients did: the fitted slope is -0.00, with a 95% range of -0.03 to 0.02, against about 0.17 for the label genes. The two reversals are TP53 with taxanes, in lung and in breast cancer: cell lines with TP53 altered were more sensitive, patients came off the drug sooner.

What this cannot say

It is one hospital’s patients, observed, not randomized. Time on a drug is not survival. The patient model finds large effects reliably and misses modest ones, so “can’t tell” often means too small to see, not absent. Five-day cell assays read slow drugs poorly. And these are development results: a second hospital’s records (AACR GENIE) are the planned confirmation. None of this is medical advice.

Sources