CASMI 2026 hard rules: timeline, prizes, code requirements, official metric mechanics, submission edge cases

TL;DR. CASMI 2026 is a Kaggle code competition (Kaggle API field isKernelsSubmissionsOnly: true): an offline notebook (internet off, ≤ 9 h CPU or GPU per mirrored rules) re-runs on a hidden ~400-molecule timsTOF set and must write submission.csv with up to 25 semicolon-separated SMILES per molecule_id. The scorer is public — Kaggle notebook metric/casmi-mean-reciprocal-rank — and does exactly Chem.MolFromSmiles → rdMolStandardize.TautomerEnumerator().Canonicalize → Chem.MolToInchiKey(...)[:14] with RDKit pinned to 2026.03.3 installed from a bundled wheel; the answer key is canonicalised at scoring time with the same function. Hard numbers: 5 submissions/day, max team 5, entry + merge deadline 2026-12-07 23:59 UTC, close 2026-12-14 23:59 UTC, $50,000 across the top five ($16,000 first), data licence CC BY-NC 4.0; the visible test.parquet is a host-acknowledged "dummy" drawn from train and is not what the public leaderboard scores.

Web search budget was exhausted when this worker started; every fact below comes from direct fetches (Kaggle API v1 via the authenticated CLI, pulled Kaggle notebooks, raw GitHub READMEs, the C&EN and Business Wire texts, casmi-contest.org) and from local RDKit 2026.03.6 experiments. Kaggle HTML pages are 6 KB JS shells and were not readable; the Kaggle discussion forum is not reachable through the public API, so forum facts are second-hand and labelled so.

1. Rules table

Status key: V = verified from a primary source I opened; M = mirrored by third parties (quoted rule text in community READMEs/notebooks, ≥ 2 independent, consistent with API fields); I = inferred; U = unknown.

2. Code-competition mechanics (Q1)

• The Kaggle API is unambiguous: isKernelsSubmissionsOnly: true. The notebook that produced the output is re-executed with the hidden test.parquet mounted at /kaggle/input/competitions/enveda-CASMI26-molecule-id-mass-spectra/; the host tutorial reads TEST_PATH from there and warns that "test.parquet carries no labels" and that molecule_id must ride along because scoring is per molecule.
• Runtime/internet: the 9 h / internet-off pair is the standard Kaggle code-competition text and is quoted by three independent mirrors; I could not open the Rules page itself. The scoring notebook runs with internet off and installs RDKit from a wheel, which is strong circumstantial evidence that participant notebooks are in the same regime.
• RDKit is not in the Kaggle Python image (# RDKit is not in docker-python, so the wheel ships as a competition input). The host wheel dataset metric/rdkit-2026-3-3-wheel carries rdkit-2026.3.3-cp310/cp311/cp312/cp313-manylinux_2_28_x86_64.whl (≈ 37 MB each, uploaded 2026-09-08) — so scoring runs on CPython 3.10–3.13, Linux x86_64. The tutorial's first cell is # pip install rdkit==2026.3.3 "Use the Dependency Manager".
• Budget reality (from the tutorial): a T4 trains the host's de novo transformer at ~840 ms/step, batch 128; 200 k spectra ≈ 22 min; all 1.39 M non-enveda-180 spectra ≈ 2.5 h. The 66 GB-activation OOM warning in the same notebook shows the session has far less than that.
• GPU quota: yasu's notes record "Kaggle batch GPU max 2 concurrent" and switching to TPU; q7 cites a 30 h/week quota (not re-verified here).

3. Admissible external data and models (Q2)

Verified: the Data page invites "any freely and publicly available external data — see the Rules for what qualifies", names PubChem and COCONUT as the Class 2 universe, and points to MIST-CF, SIRIUS, matchms, GNPS suspect annotations ("excluded from train.parquet because the labels are inferred rather than measured"), MassIVE, GeMS/DreaMS and FragHub. The mirrored Rule 2.6 adds "at minimal cost" and the standard Kaggle "Reasonableness Standard". Nothing I opened bans commercial tools; SIRIUS's own licence terms (academic vs commercial) were not checked in this session and must be before any SIRIUS-derived artefact goes into a winning (MIT-licensed) submission. Practical constraint, not a rule: everything must be attachable as a Kaggle Dataset and must finish inside 9 h offline, so PubChem-scale pools (community dataset dmitriigluzdov/casmi26-pubchem-popularity-prior, 1.29 GB) and model weights (e.g. hengck23/hengck23-dreams-enveda-casmi26, 5.7 GB) are the norm.

4. The decoy test (Q3)

Verbatim, Data page: "Note: the file you see is comprised of examples from the training dataset; it will be replaced by the hidden test set during re-run, and will be approximately the same size." Community audits (mxm-1x, Clyde0513, AmanDubal, josefreitasalvesneto item 10, kitopl §2) all map 1,213/1,213 visible spectra to enveda-180 rows by exact peak-array match; mxm-1x additionally links a "host clarification" at .../discussion/741404 calling it a dummy file (thread not opened; status M). There is no rule against using it and no benefit: q7 measured a public LB (2,673 teams, max 0.480) that library search cannot saturate, so the public LB is scored on the hidden set, not on the visible file. Use the visible file only for schema/runtime smoke tests; the hosts' phrase "approximately the same size" is the only official statement about hidden-set size that agrees with the 1,213-row file.

5. Official metric mechanics (Q5) — metric/casmi-mean-reciprocal-rank

Pulled with kaggle kernels pull metric/casmi-mean-reciprocal-rank (author "Kaggle Competition Metrics", last run 2026-09-14 12:59 UTC, docker image gcr.io/kaggle-images/python@sha256:dafd4ce5…). Key lines, quoted:

# Solution key canonicalized with RDKit 2026.03.3. RDKit's TautomerEnumerator output has changed
# between releases, so this pin decides which submissions match; it is not a convenience.
EXPECTED_RDKIT_VERSION = '2026.03.3'
RDKIT_REQUIREMENT = 'rdkit==2026.3.3'
RDKIT_WHEEL_DIR = '/kaggle/input/datasets/metric/rdkit-2026-3-3-wheel'
...
    _tautomer_canonicalizer = rdMolStandardize.TautomerEnumerator()

def canonicalize_smiles_to_ik14(smiles: str) -> str:
    """Canonicalize SMILES to InChIKey14, or None if parsing fails."""
    if not isinstance(smiles, str):
        return None
    try:
        mol = Chem.MolFromSmiles(smiles)
        if mol is None:
            return None
        canonical_mol = _tautomer_canonicalizer.Canonicalize(mol)
        inchikey = Chem.MolToInchiKey(canonical_mol)
        return inchikey[:14]
    except (ValueError, RuntimeError, TypeError):
        return None

Answers to the sub-questions:

Local checks on RDKit 2026.03.6 (closest build on this machine; defaults maxTautomers=1000, maxTransforms=1000, removeSp3Stereo=True, removeBondStereo=True, reassignStereo=True): keto/enol pairs that differ in raw InChIKey14 (acetylacetone YRKCREAYFQTBPV vs POILWHVDKZOXJZ; warfarin PJVWKTKQMONHTI vs FVSFCRPKSVCTBA) collapse to one key after Canonicalize; 2-hydroxypyridine/2-pyridone, guanine, quercetin already share a raw key (standard InChI mobile-H handling); quercetin costs 335 ms per call (q7 saw MaxTransformsReached on polyphenols at this cap). Stereo (C[C@H](O)CC vs enantiomer vs flat) and E/Z share InChIKey14 regardless. Isotopes ([13CH3]CO, [2H]C([2H])([2H])O) do not change block 1. Charge states CC(=O)[O-] vs CC(=O)O share a key, but salts do not: [Na+].CC(=O)[O-] gives VMHLLURERBWHNL, acetic acid QTBSBXVTEAMEQO. Community measurement (kitopl, 2026-09-21): 4,370 of 275,810 train structures (1.58 %) change key under canonicalisation and 3,167 structures (1.15 %) collapse into shared keys; over COCONUT's 738,827 structures 10.4 % re-key, 2.30 % can matter, simulated MRR cost 0.48 % (dariushafshar, cited by kitopl; not opened).

6. Submission format edge cases (Q6)

7. What is known about the hidden test and host FAQ (Q7)

Verified official statements (Data page): ~1,500 spectra / ~400 molecules, 1–16 spectra per molecule (median 3), all timsTOF, 157–1,159 Da (median 348), three novelty classes with hidden proportions, minimal cleaning (three listed steps), collision_energy_orig_units always eV, ten adducts. Press release (Business Wire 2026-09-15, via Yahoo Finance): "roughly 2,500 mass spectra", "All unpublished data", 90-day contest, lead organiser David Healey (Founding Data Science Fellow). C&EN (Oldach, 2026-09-17, DOI 10.47287/cen.612552.newsarticle): test "curated … from molecules that have not been characterized in public datasets"; Fiehn: "They did not try to have real datasets with … overlapping coeluting entities, chimeric spectra, weird adducts … They focused on mass spectra of isolated compounds"; Allen: "CASMI hasn't yet had its AlphaFold moment". casmi-contest.org/2026 (last modified 2026-10-02) only redirects to Kaggle and names Enveda as organiser. Host discussion threads could not be opened; the only thread id surfaced is 741404 (decoy clarification, via mxm-1x).

8. Forced analogy: law of evidence — rules of procedure and admissibility

The analogy, precisely. The Rules page is a code of evidence. (a) Admissibility is the "freely and publicly available … at minimal cost" standard (Rule 2.6): an exhibit (external DB, pretrained weights) is admissible only if the opposing parties could have obtained it too; Enveda's internal 1.2 B-spectrum PRISM corpus is held by the court, not admissible to litigants. (b) Burden and standard of proof sit entirely on the submitter: 25 ordered assertions per molecule, judged against a single binary identity test (canonical InChIKey14). There is no preponderance standard — a Tanimoto-0.95 analog is as worthless as CCO. (c) Discovery: the visible test.parquet is a disclosed but non-probative exhibit — the court told everyone it is a copy of training evidence and will be swapped at trial. (d) Hearsay: the hosts excluded GNPS propagated ("suspect") annotations from train "because the labels are inferred rather than measured" — the hearsay rule applied by the court to its own record. (e) Court-appointed expert: the metric notebook is the published jury instruction; the judge even canonicalises its own answer key live rather than trusting a stored column.

Derived rule — what may be pre-computed and what must run live. In court, documentary exhibits prepared before trial are admissible if authenticated and disclosed; testimony about the facts at issue must be given live. Translated: anything that is a function of admissible, pre-existing material may be computed offline and shipped as a Kaggle Dataset — candidate pools (COCONUT/PubChem/LOTUS/NPAtlas) already deduplicated on the scorer's key (RDKit 2026.03.3 TautomerEnumerator().Canonicalize → InChIKey14) and indexed by heavy-atom composition and neutral monoisotopic mass; fingerprints of every candidate; trained model weights; library-spectrum indices built from train. Only functions of the hidden exhibit (the test spectra) must run inside the 9 h offline notebook: spectrum embedding, formula/mass windows, candidate scoring, aggregation across a molecule's spectra, writing 25 SMILES. Concrete, testable idea: because the scorer re-keys both sides with the same function, the only thing that can hurt is a disagreement about identity between our pool key and the scorer's. So (1) build the pool key with the exact metric/rdkit-2026-3-3-wheel wheel (cp312 is in the dataset; it is public, attachable), never with 2026.03.6; (2) store, per candidate, the key and a flag tautomer_shifted (raw IK14 ≠ canonical IK14); (3) at submission time emit one SMILES per canonical key and emit the RDKit-canonical tautomer's SMILES, not the database spelling, so that whatever RDKit the scorer runs, the guess is already in the canonical form. Prediction to test on a held-out enveda-np-examples split keyed both ways: the measured MRR difference between raw-key dedup and canonical-key dedup should be ≤ 0.005 (kitopl's 1.15 %-collapse upper bound times the fraction of answers that rank high) — if it is larger, our dedup is merging things the scorer separates. A second testable derivation from the "no standard short of identity" point: slots 20–25 should hold tautomer/protomer-distinct spellings of nothing — i.e. there is zero value in hedging with near-isomers that share a key; every slot must carry a distinct canonical key, which is cheap to assert in the notebook (len(set(keys)) == len(keys)).

Where the analogy holds, partially holds, breaks. Holds: admissibility by public availability; burden on the submitter; disclosed-but-non-probative discovery; the hearsay exclusion the hosts applied; published jury instructions. Partially holds: "best evidence rule" (submit the parent structure, not the salt — my local test shows the sodium salt gets a different key) maps well, but the court here never weighs credibility: frivolous guesses cost only a slot, there are no sanctions, and no cross-examination. Breaks: there is no adversary (other teams do not contest our evidence), no appeal (a ParticipantVisibleError just burns one of five daily filings), and the judge is a lookup, not a fact-finder — the metric cannot be persuaded, only matched. The analogy also says nothing about the public/private split, which is the one procedural fact (what fraction of the hidden set the public LB sees) that we still cannot verify.

9. Gotchas — what could silently fool us

1. The Kaggle API labels the metric "Mean Average Precision". It is MRR@25; the scorer's variable is still named ap. Anyone building local eval from the API label will compute the wrong thing.
2. train.inchikey14 is not the scoring key. It is derived from normalized_smiles without tautomer canonicalisation; 1.58 % of train structures shift and 1.15 % collapse (kitopl). Deduplicate and validate on the canonical key or local MRR will undercount.
3. RDKit version drift is a one-sided risk. The scorer warns but continues on mismatch. If our notebook keys a pool with 2026.03.6 and the scorer uses 2026.03.3, a correct candidate can be merged away before it is emitted. Pin the same wheel.
4. Salts and multi-fragment SMILES are not stripped by the scorer. [Na+].CC(=O)[O-] ≠ acetic acid. Any pool that keeps COCONUT/PubChem salt forms must be parent-stripped by us (the hosts' normalized_smiles is "RDKit-standardised", so train labels are presumably parents — not verified for every library).
5. Polyphenol canonicalisation hits maxTransforms=1000 (q7: MaxTransformsReached on quercetin/rutin/tetracycline; my timing 335 ms for quercetin). A truncated enumeration is the configuration most likely to differ between builds; it is also why canonical-keying a 740 k pool costs ~ 1 CPU-hour (4.7 ms/structure median per kitopl; polyphenols 50–70× worse).
6. Hidden-set size is stated three ways: ~1,500 spectra (Data page), "approximately the same size" as 1,213 (Data page), ~2,500 (press release). Seven vs ten adducts is also inconsistent in the same official text. Treat the adduct list as ten and the per-molecule median as 3.
7. Public LB is a sample of the hidden set with an unknown fraction. With ~400 molecules and per-molecule reciprocal-rank SD ≈ 0.45, the public score's standard error is ≈ 0.045 if the public share is 100 molecules, 0.032 at 200, 0.022 at 400 (inferred; see Open questions). q7's observed 228-team pile-up in 0.42–0.43 reflects shared public notebooks, not precision.
8. Missing molecule_ids are not rejected by the scorer — they score 0 silently. Always start from sample_submission.csv (the tutorial does this) so every id is present.
9. Invalid SMILES consume slots silently. A generator that emits 30 % unparsable strings wastes 7–8 of 25 ranks unless filtered through MolFromSmiles first.
10. Rule 3.6 and CC BY-NC 4.0: a public GitHub repo containing competition-derived code or weights during the competition is a rules exposure (kabir0774's reading); keep repos private until the close.
11. Entry status: as of this run the Kaggle API returns userHasEntered: true for the local account (q7 reported 403 earlier the same day) — someone accepted the rules; the Rules page text is now readable from a logged-in browser and should be captured verbatim to close the "M" rows above.

Implications for the challenge

1. Attach metric/rdkit-2026-3-3-wheel (or a copy) to every notebook and pip install --no-index --find-links … rdkit==2026.3.3; key every candidate and every validation answer with the scorer's exact three-line function. Never key with 2026.03.6.
2. Pre-compute offline (Explorer cluster) and ship as Kaggle Datasets: the canonical-keyed, salt-stripped, heavy-atom-composition-indexed candidate pool; per-candidate fingerprints; model weights; train library index. Reserve the 9 h window for inference only; target ≤ 3 h to leave headroom for Kaggle's slower CPUs.
3. Enforce in the notebook before writing: 400 rows from sample_submission.csv, 1–25 non-empty tokens, all parseable, all distinct canonical keys, no ; or whitespace inside tokens, no NaN. A rejected file burns one of five daily submissions.
4. Emit RDKit-canonical tautomer SMILES (post-Canonicalize) rather than database spellings; emit parent structures only.
5. Build the local validation set as "novel molecules on timsTOF" (hold out by canonical key, assert zero exact spectral matches remain, per kitopl §6) and report MRR with ±1 SE for n = 250 (≈ ±0.03); do not react to public-LB deltas < 0.01 (q7).
6. Budget submissions: 68 days × 5 = 340 slots remain before 2026-12-14; the leaders average ~2.3/day. Keep two clearly different final selections (e.g. retrieval-heavy vs generation-heavy).
7. Licence hygiene for the MIT open-source obligation: COCONUT CC0/CC BY, LOTUS CC BY 4.0, NPAtlas CC BY-NC 4.0 (q7's table), competition data CC BY-NC 4.0 — record attribution now; check SIRIUS/CSI:FingerID terms before depending on them.
8. Somebody with the accepted account should copy the Rules, Overview/Evaluation and Timeline pages verbatim into research/q4_challenge_intelligence/ to convert every "M" row to "V" and to settle rows 5, 12, 26, 29.

Open questions (and how to settle each)

Sources

• Kaggle API v1, GET /api/v1/competitions/list?search=CASMI (authenticated, 2026-10-07): fields isKernelsSubmissionsOnly, maxDailySubmissions, maxTeamSize, mergerDeadline, newEntrantDeadline, deadline, enabledDate, dateCreated, reward, licenseName, evaluationMetric, awardsPoints, teamCount (2,672).
• Official metric notebook: https://www.kaggle.com/code/metric/casmi-mean-reciprocal-rank (pulled via kaggle kernels pull, kernel id 133620699, dataset source metric/rdkit-2026-3-3-wheel).
• Wheel dataset listing: kaggle datasets files metric/rdkit-2026-3-3-wheel (cp310–cp313 manylinux_2_28 wheels, 2026-09-08).
• Host tutorial: https://www.kaggle.com/code/inversion/casmi-denovo-tutorial-notebook (pulled; 33 cells).
• sample_submission.csv downloaded via kaggle competitions download -f sample_submission.csv (400 rows, 25 × CCO).
• Data page text, mirrored verbatim in scratch file dallo_desc.md (from https://github.com/dalloliogm/kaggle_competitions) — consistent with the Kaggle API and the metric notebook.
• Overview/Rules mirror: https://github.com/NoName7n11/Enveda_CASMI_2026 README (9 h, internet off, Rule 2.6, MIT, prize breakdown, 2 final selections).
• https://raw.githubusercontent.com/corentinlaudicina/enveda-casmi-2026/main/README.md ("code competition … ≤ 9 h", "official metric notebook is the reference").
• https://raw.githubusercontent.com/kabir0774/Enveda-Casmi-2026/main/README.md (rule 3.6, CC BY-NC 4.0).
• https://raw.githubusercontent.com/mxm-1x/Enveda-CASMI-2026/main/README.md (links host clarification discussion/741404).
• https://raw.githubusercontent.com/ibnmarzuk/casmi26-molecule-id/main/README.md; https://raw.githubusercontent.com/gnshx/Enveda-CASMI-2026---Molecule-ID-From-Mass-Spectra/main/README.md; https://raw.githubusercontent.com/AmanDubal/Enveda-CASMI-2026---Molecule-ID-From-Mass-Spectra/main/README.md; https://raw.githubusercontent.com/Clyde0513/CASMI-2026/master/README.md (submission mechanics, decoy audits).
• Community notebooks pulled: https://www.kaggle.com/code/kitopl/casmi-2026-data-pitfalls-that-break-your-cv (key-shift statistics, forum Class-1 estimate, offline wheel install); https://www.kaggle.com/code/josefreitasalvesneto/casmi-2026-a-complete-statistical-tour (item 10, decoy).
• Business Wire press release 2026-09-15 via https://finance.yahoo.com/healthcare/articles/enveda-launching-next-alphafold-contest-120700676.html.
• Oldach L., "Metabolomics has a data problem. Could this competition fix it?", C&EN 2026-09-17, DOI 10.47287/cen.612552.newsarticle, https://cen.acs.org/analytical-chemistry/mass-spectrometry/competition-identify-small-molecules-capture/104/web/2026/09 (fetched HTML).
• http://www.casmi-contest.org/2026/ (last modified 2026-10-02).
• Local RDKit 2026.03.6 experiments (scratchpad/rd/taut.py), and q7's channel note on MaxTransformsReached (research/_channels/rules_x_execution.md).