A molecule breaks into pieces. We have to name it from the pieces.

Every living thing is a chemical factory. A single plant leaf contains thousands of small molecules — pigments, defences, signals, waste — and most of them have never been written down. The complete set of small molecules in an organism is called its metabolome; the molecules that organisms make for their own purposes, and that humans have borrowed as medicines for millennia, are called natural products. When scientists grind up that leaf and run it through a mass spectrometer, the machine reports thousands of signals, and for most of them nobody can say which molecule made them. The field calls this the dark metabolome. Enveda, the company behind this contest, says that more than 95 % of the molecules its own instruments see are unknown, and when it published a model for this problem in 2023 it tested it on 226 natural products that were absent from every major database.

The scale of what is known is surprisingly small. Every natural product that anyone has ever characterised and deposited fits in one public catalogue, COCONUT, and once duplicate drawings of the same molecule are merged it holds about 480,000 distinct structures. Against that, the training data for this contest contains measured spectra for 275,810 distinct molecules, and only about 26,000 of those are natural products in the catalogue at all.

Medicines come out of this darkness. Enveda's own patents describe new drug candidates built on natural scaffolds such as withaferin A, isoandrographolide and nootkatone, decorated by chemists to improve them. A company that could read nature's chemistry at scale would have the largest medicine cabinet on Earth. That is the bet behind this challenge, and it is also why the host is not a neutral referee — a point we return to.

Dissolve the sample and push it through a thin tube that separates molecules by how sticky they are (liquid chromatography). Spray the liquid from a needle held at thousands of volts so each molecule picks up a charge, usually a single extra proton (electrospray ionisation); a charged molecule is an ion, and the particular charged form — molecule plus proton, molecule plus sodium, molecule minus a proton — is its adduct. Pick one ion by its mass-to-charge ratio, written m/z, and call it the precursor. Slam it into nitrogen gas hard enough to break it (collision-induced dissociation); the hardness of the slam is the collision energy, measured in electron-volts (eV). Weigh the pieces, the fragments, by timing how long they take to fly down a tube (time of flight). The result is a tandem mass spectrum: a list of fragment masses, each with a height — its intensity — saying how much of it was seen. Each entry in that list is a peak.

The hidden test was measured on one model of instrument, a Bruker timsTOF; no acquisition notes were published, so the hive reconstructed its settings from the data. It smashes each molecule at three fixed collision energies, 20, 40 and 60 eV, and for about a quarter of the test spectra the three pictures are simply added together, peak by peak, into one merged spectrum; roughly two thirds of test molecules have at least one such merged picture. The timsTOF also sorts ions by shape in a gas (ion mobility), but that information is not in the files we are given, so for us it does not exist.

The peak lists are raw detector counts. The smallest reported peak is exactly 9 counts, nothing is capped, and more than nine in ten peaks are below one percent of the tallest. The number of peaks mostly tells you how much sample was injected, not what the molecule is: the correlation between sample abundance and peak count is 0.71. A spectrum is a fingerprint; without a matching card on file it is just a pattern.

Three things conspire. First, mass is not identity. Molecules made of the same atoms connected differently are isomers, and they weigh exactly the same. For a typical natural product in the test's mass range, the catalogue holds a median of 36 candidates whose mass matches within the instrument's precision, and the worst cases run to hundreds. Most of those share not just the mass but the exact atomic recipe — the molecular formula — and differ only in where one group sits, so-called positional isomers; the median molecule faces 35 such cousins, and for 58 % of molecules there are more cousins than the 25 guesses allowed.

Second, fragmentation is only partly understood. Which bonds break depends on the structure, the charge, the energy and the instrument. The best hand-written rule sets explain only 78–85 % of the fragments actually seen, and the rules cut both ways: a sugar-bearing molecule "should" shed a 162-mass sugar, yet 86 % of such molecules show no such loss in positive mode. Absence of a clue is never evidence.

Third, the reference libraries are the wrong shape. The training file holds 2.54 million spectra, which sounds enormous, but 1.15 million of them come from Enveda's own timsTOF and almost all of those are synthetic drug-screening compounds from a commercial catalogue, not natural products: of 182,941 such molecules, 51 are in COCONUT. Natural products measured on the test instrument number exactly 250 molecules. Everything else was recorded on other machines whose spectra look systematically different — a 20 eV timsTOF spectrum resembles an Orbitrap spectrum labelled "20", not the "33" the training file's conversion formula implies. For a molecule nobody has measured before, library lookup returns nothing, and the models trained on those libraries have never seen a natural product on this instrument.

Enveda, a Boulder company that hunts medicines in plants, revived a dormant community contest called CASMI (Critical Assessment of Small Molecule Identification), founded in 2012 by Emma Schymanski and Steffen Neumann and modelled on CASP, the protein-folding contest that produced AlphaFold. It runs on Kaggle from 14 September to 14 December 2026, with team entries closing 7 December; teams may submit five times a day and pick two final entries. The prize pool is $50,000, $16,000 for first, and winners must open-source their method.

The hidden test is about 400 secret molecules, each measured one to sixteen times (typically three), all on Enveda's timsTOF, described as natural products, plausible natural products and chemists' analogues of them, "none previously published". How many spectra that makes is disputed: the press release says about 2,500, Kaggle mirrors say about 1,500, and the two kings could not settle it. Enveda checked the answers with its own tools, a fragmentation-tree program called SIRIUS and NMR spectroscopy.

For each molecule a team submits up to 25 guesses, ranked, written as SMILES — a text notation for a molecular structure. If the first guess is right the team earns a full point; if the right answer is second, half a point; tenth, a tenth; absent from the 25, nothing. The average over all molecules is the score, called MRR@25 (mean reciprocal rank at 25). A guess counts as right when it has the same atoms connected the same way: the scorer converts each structure to a 14-character code called an InChIKey14 that ignores left- and right-handedness and, after a cleanup step in the chemistry toolkit RDKit (version 2026.03.3, pinned), also ignores tautomers — the quick proton shuffles that make one molecule look like two on paper. A file with more than 25 guesses, a blank, or a repeated molecule ID is rejected whole; an unreadable or duplicate guess silently wastes its slot.

Before: CASMI editions ran from 2012 to 2017 and once more in 2022. The pattern across all of them is the same — when the true answer was in the list of candidates a method considered, methods did reasonably; when it was not, they collapsed. In 2014 the genuinely unknown compounds were solved zero times out of six. In 2022 the headline best was 38 % correct structures, but that was on 55 self-chosen compounds; the best team that attempted all 222 scored 26 %, while the molecular formula alone was found 94 % of the time. And in 2016 the leading method's success rate halved, from 70 to 36 first-place hits, when compounds it had seen in training were removed.

Now: three weeks in, the top public score is 0.481 on the 0-to-1 scale; a plain library lookup scores 0.06–0.10. Back-solving from the open write-ups, roughly 55 % of hidden molecules reach anyone's top 25 at all, and an estimated 39 % are in no database whatsoever, so every team scores zero on them. The shortlist, not the ranking, is the ceiling.

After: Enveda's CTO, August Allen, says CASMI "hasn't yet had its AlphaFold moment" and that is what they mean to usher in. The 2022 organiser, Oliver Fiehn, is more cautious: this contest uses clean spectra of isolated pure compounds, with none of the overlapping, contaminated mixtures of a real sample, and the field has "much less ground truth" than protein folding had. The sceptics add a motive: Enveda wrote the metric, chose and validated the answers, announced the contest eight days before a $311 million financing round, and says in its own press release that CASMI "helps to grow PRISM", its closed in-house model trained on 1.2 billion spectra, for which no weights, code or paper exist. Both camps are probably right, and the gap between them is where our work lives.

The public scoreboard is a crowded, noisy instrument. On 7 October it held 2,703 teams; the first, tenth, hundredth and thousandth scored 0.481, 0.441, 0.425 and 0.357, and the median was 0.328. A third of all teams — 885 — sit on just 16 exact scores because they re-ran shared public notebooks unchanged, and 228 teams are packed inside the single band from 0.42 to 0.43. The public score is computed on a subset of the hidden molecules — one unverified write-up says a third, about 130 — and a score on 130 molecules has a standard error, the typical wobble from chance alone, of about 0.04. Differences under 0.01 mean nothing; one team's byte-identical notebook moved 0.105 between reruns. The derived medal lines are 0.436 for gold, 0.425 for silver and 0.421 for bronze, and a fixed score of 0.417 has been sliding about 30 places a day.

Every public notebook scoring above 0.37 uses the same recipe: predict from the spectrum which chemical building blocks are present — a fingerprint, a long list of yes/no bits — then rank the catalogue molecules of the right mass by how many of those bits they contain with a learning-to-rank model, then glue two or three such engines together. The hive's summary of the field is one equation: score ≈ (chance the truth is on the shortlist) × (chance it is then ranked in the top 25) × (average credit given its rank). At the 0.417 tier those three are about 0.55–0.61, 0.90 and 0.76, so the first factor has by far the most room to grow.

The science is less settled than it looks. A June 2026 audit of the main academic benchmark found evaluation problems in at least 17 of 26 papers; one "state of the art" method fell from 82 % to 7 % when a shortcut was removed, and a method that never looks at the spectrum at all, simply returning PubChem's default order, scored nearly 50 %. No published tool models the timsTOF's collision energy and no learned method has ever been evaluated on timsTOF data.

The practice test is a copy of the homework. All 1,213 spectra in the visible test file are verbatim rows of the training file, drawn from the synthetic drug-screening set. Teams that tuned on it, or on random slices of training, learned the wrong lesson: one write-up went from 0.456 locally to 0.112 on the public board, another from 0.445 to 0.063, and random held-out training molecules ranked the candidate methods in the opposite order from the leaderboard. Even the 250 natural products on the right instrument are a trap: every one also has spectra in other libraries (246 in RIKEN, 233 in GNPS), so holding them out of one library while leaving the rest is a leak — the model has still seen the answer.

Some training columns were made by a script that erred: the mass-error column is unsigned, omits the electron's mass and predates a relabelling, so 52,460 rows show a spurious error near 2,400 parts per million, and filtering on it discards good data. Default cleaning tools err the other way: the commonest spectral-processing library's default threshold deletes three quarters of a timsTOF spectrum.

Twenty-nine percent of training's timsTOF rows are dimers — two copies of a molecule on one charge — and the test has none. Literature methods often win by favouring much-written-about molecules — in the 2016 contest citation counts alone put 143 of 208 answers first — but this test is defined as unpublished molecules, so that trick points the wrong way. And the scorer is tone-deaf to mirror images and proton shuffles: 35 % of the catalogue consists of such duplicate spellings, and 1,541 groups of training labels merge into one under the scorer's eyes. Two guesses that differ only that way are one guess and a dead seat. Finally, averaging the rankings of several methods lets one bad engine veto the right answer; in simulation it loses to the better engine alone once their mistakes are modestly correlated.

First, an honest exam. The only local test that predicted the leaderboard's order, for three independent teams, is the 250 natural products on the test instrument with their spectra removed from every library, scored under three masks. It becomes our primary selector, beside a larger hold-out of about a thousand natural products with close relatives removed and the true answer deliberately deleted from the shortlist for half of them, as the real test behaves. Every score is reported next to two controls: a method that never looks at the spectrum, and pure chance, 3.816 divided by the number of candidates. The cheapest experiment in the hive is the mock-witness test: rank on mass and adduct alone, no fragments; if that beats chance, the shortlist is biased and our local scores harvest that bias.

Second, the shortlist before the ranker: one catalogue deduplicated to the scorer's key, a ten-parts-per-million mass window, no wholesale PubChem (a proven 30 % loss), and a small gated injection of PubChem or one-step chemical relatives only when the main shortlist looks empty (+0.011 gated, +0.000 ungated). Clean every spectrum with fixed rules — strip the surviving parent and its heavy-carbon echoes, as astronomers mask the atmosphere out of starlight — and translate collision energies between instruments before a model sees them. Combine engines by adding calibrated log-odds, never by averaging ranks, and fill the 25 seats by probability, not quota.

Third, a calendar with kill dates. A scoring library-search entry by 14 October; shortlist recall of at least 0.50 on the honest exam by 25 October or model work stops in favour of the shortlist; any de novo generator under 0.15 locally by 1 November is dropped; ensembles must beat the best single engine by 0.01 with a bootstrap interval excluding zero by 22 November; code freeze 4 December; final two picked by a pre-registered rule on 11 December. One task needs a human with a browser: read the Kaggle rules and discussion while logged in and settle how many spectra and adducts the hidden set has, since four agents hit the same CAPTCHA wall.