Critiques of CASMI 2026 benchmark validity: what the test does not measure

TL;DR. CASMI 2026 scores one thing: whether a ranked list of 25 planar structures contains the right 2D skeleton (InChIKey first block, after RDKit 2026.03.3 tautomer canonicalisation) for ~400 pure compounds measured on one instrument (Bruker timsTOF). It does not measure stereochemistry, coelution/chimeric spectra, adduct or formula inference (the adduct is handed to you), retention time, ion mobility, MS1 isotope patterns, or whether a model knows when it is wrong. Fiehn's on-record critique (C&EN, 2026-09-17) is that it is "mass spectra of isolated compounds, which is difficult enough" with "much less ground truth" than CASP had; the published CASMI organisers (Schymanski, Neumann) and the ML-benchmark literature (MassSpecGym, Böcker lab) add that honest splits cut headline numbers by roughly half (58.5% → 34.4% top-1 in Dührkop 2015; 70 → 36 Top-1 ranks for CSI:FingerID leave-out in CASMI 2016). For us the operational consequences are: validate on structure-disjoint (ideally MCES-disjoint) splits, deduplicate candidates on canonical-tautomer InChIKey14 so stereo- and tautomer-duplicates never waste one of the 25 slots, and treat leaderboard differences below ~0.04 MRR as noise.

1. The C&EN article (verified, full text read)

Verbatim quotes, with attribution:

• Oliver Fiehn (UC Davis; ran CASMI 2022): "They did not try to have real datasets with . . . overlapping coeluting entities, chimeric spectra, weird adducts," ... "They focused on mass spectra of isolated compounds, which is difficult enough." He "doesn't expect a breakthrough this time around. 'AlphaFold took CASP many years,' he says, adding that in metabolomics there is 'limited information and much less ground truth, in a way.'" Article framing: "the dataset that entries will be tested on is less complex than a genuine metabolome."
• August Allen (CTO, Enveda): "CASMI was inspired by CASP ... The difference is CASMI hasn't yet had its Alphafold moment . . . and that's what we're trying to usher in." In an email, Allen "counters that these are exactly the problems CASMI is designed to solve" (referring to the two cited failure modes below).
• Viswa Colluru (CEO, Enveda): "We think there's a line of sight to be able to characterize the world's chemistry at scale, essentially akin to what sequencing did to DNA."
• Not quoted in the article: Schymanski, Neumann, Dorrestein, Böcker, Wang. The article's only other critical voices are two hyperlinked papers: "models struggle to recognize the same molecule captured under different experimental conditions and to extrapolate beyond the structures in their training datasets" links to Khoo & Barzilay, Nature Metabolism 8:1247–1249 (June 2026), "Why machine learning fails at mass spectrometry for small molecules", DOI 10.1038/s42255-026-01544-6 (paywalled; only bibliographic record opened); "the available training data, according to one analysis, do not offer broad coverage of known metabolites" links to Kretschmer et al., Nat Commun 2025, DOI 10.1038/s41467-024-55462-w (opened, see §3).
• Article facts: Enveda "is providing a training dataset of MS/MS spectra from its own collection of purified natural products" plus "standardized ... open-source metabolomics datasets"; test set "spectra from molecules that have not been characterized in public datasets"; "$50,000 prize ... split between the top competitors"; open "until mid-December". The press release (Yahoo Finance mirror, 2026-09-15) says ~400 molecules / ~2,500 spectra and quotes David Healey (Enveda Founding Data Science Fellow): "There are thousands of AI and machine learning researchers who have never heard of mass spectrometry." The GitHub mirror of the data card says ~1,500 test spectra — the two figures disagree; the data card number is the operative one.

2. What the CASMI founders said the contest does not measure

Schymanski & Neumann, "CASMI: And the Winner is...", Metabolites 3(2):412–439, 24 May 2013, DOI 10.3390/metabo3020412 (PMC3901266). CASMI 2012: 14 LC-HRMS/MS challenges (Cat. 1–2) and 16 GC-MS challenges (Cat. 3–4); only LC received submissions ("five submissions (three external, two internal)" in Cat. 1, "six submissions (three external, three internal)" in Cat. 2). Scoring was absolute rank with the worst-case rank for tied scores ("the devil is in the details, e.g., if several candidates have an identical score"). The correctness criterion that CASMI 2026 inherits was set here: "a structural candidate [is] correct for CASMI if the first block of the InChI Key [is] identical". Key limitation sentence: "In real identification efforts, one needs to know how many 'incorrect' solutions are placed higher." They already proposed "a new contest category ... where a list of candidates is provided along with spectral data for the participants to rank."

Schymanski et al., "Critical Assessment of Small Molecule Identification 2016: automated methods", J Cheminform 9:22, 27 March 2017, DOI 10.1186/s13321-017-0207-1 (PMC5368104). 208 challenges from 188 substances (127 positive, 81 negative), authentic standards on a Q Exactive Plus with stepped 20/35/50 HCD. Winners by "gold medals" (lowest rank among contestants per challenge): Cat. 2 (in-silico only) Team Brouard CSI:IOKR 86 gold / 62 Top-1; Team Dührkop CSI:FingerID 82 gold / 70 Top-1 (34%); MS-FINDER 70 gold / 46 Top-1. Cat. 3 (full metadata) MS-FINDER+MD 159 gold / 146 Top-1 (70%); CFM_retrain+DB 156 gold / 120 Top-1. The critiques, verbatim:

• "Category 2 was a 'structure dereplication' contest, finding the best structure within a pre-defined list of structures, not a completely open in silico test on all exhaustive structures in the chemosphere."
• "Stereoisomerism is another aspect of identification that was not covered in this contest." "None of the current approaches are able to distinguish stereoisomers (even cis/trans isomers) using only MS/MS information". "Due to inconsistencies with how participants dealt with multiple stereoisomers ... submissions were filtered post-submission to remove duplicate stereoisomers using the first block of the InChIKey."
• "The ability to distinguish stereoisomers using MS/MS alone also remains a challenge for the future that is not yet ripe enough for a CASMI contest; distinguishing (positional) isomers is likely sufficient challenge for the next few years."
• "The incorporation of metadata in automated methods is essential to guide users to the identification for known substances—but misleading when assessing the performance of computational methods." "the vast majority of substances were either ranked first or second based purely on the reference count."
• Leakage, quantified: "Team Dührkop noted that there was a large overlap between the challenges and their training set"; the CSI:FID_leaveout entry fell to "36 vs. 70 Top 1 ranks in positive mode"; for another retrained entry "the percentage of maximum Top 1 ranks dropped from 34 to 18%".
• "Challenges for future contests remain true unknowns, i.e. substances that are not present in compound databases." "The true 'unknown unknowns' remain to be evaluated in future CASMI contests."
• Recommendation adopted by CASMI 2026: "future CASMIs could consider providing an extensive, open training dataset ... and ensure all CASMI challenges are absent from this set"; alternative: "require participants to submit InChIKey lists of their training sets."

Schymanski et al., Environ. Sci. Technol. 48(4):2097–2098, 2014, DOI 10.1021/es5002105 (bibliographic record verified via Europe PMC; full text paywalled). The five-level scheme: Level 1 confirmed structure (reference standard, RT, MS, MS²); Level 2 probable structure (2a library match, 2b diagnostic evidence); Level 3 tentative candidate(s); Level 4 unequivocal molecular formula; Level 5 exact mass of interest (wording paraphrased from a vendor summary, Thermo AnalyteGuru blog, which mis-cites the year; exact ES&T phrasing unverified). A CASMI-correct rank-1 hit, by this scheme, is Level 3 evidence at best — no standard, no library spectrum.

Sumner et al., Metabolomics 3:211–221, Sept 2007, DOI 10.1007/s11306-007-0082-2 (PMC3772505). Verbatim: Level 1 "Identified compounds"; Level 2 "Putatively annotated compounds (e.g. without chemical reference standards, based upon physicochemical properties and/or spectral similarity with public/commercial spectral libraries)"; Level 3 "Putatively characterized compound classes"; Level 4 "Unknown compounds". Requirement: "A minimum of two independent and orthogonal data relative to an authentic compound analyzed under identical experimental conditions are proposed as necessary to validate non-novel metabolite identifications (e.g. retention time/index and mass spectrum ... accurate mass and tandem MS ...)". CASMI 2026 gives exactly one of the two orthogonal data (MS/MS + precursor m/z), i.e. the benchmark can at most certify Level 2–3 evidence.

Theodoridis, Fiehn, et al., "What's in a name? Metabolite identification: challenges and pitfalls in untargeted metabolomics", Metabolomics 22(2):22, 2026, DOI 10.1007/s11306-025-02387-0 (PMC12886295). Fiehn as co-author: "an alarming increase in the frequency of erroneous metabolite identifications"; errors "are not always recognised during the reviewing process"; "Level 1 identifications can sometimes be ambiguous, e.g., if tryptophan is reported one cannot be sure whether it is the D or L enantiomer"; "The easy and convenient use of fast computational and AI tools is highly tempting." No proposal for rank-k reporting was found in the 2023–2026 MSI/mQACC documents opened (Mosley et al. 2024 QA/QC framework, DOI 10.1007/s11306-023-02080-0 — body text could not be extracted). The "identification probability" preprint (bioRxiv 10.1101/2024.07.30.605945) could not be opened (HTTP 429) — unverified.

3. ML-evaluation critiques: how much honest splits cost

Note on the MassSpecGym bonus task: giving the formula lowers MIST's hit@1 (14.64 → 9.57) because formula-matched decoys are harder than mass-matched ones. The CASMI 2026 hidden set behaves like the bonus task for any model that uses a tight mass window: all surviving candidates are isomers.

4. What InChIKey14 + tautomer canonicalisation cannot see (verified with RDKit 2026.03.6)

Tautomers: standard InChI already folds mobile-H tautomers into the first block (2-hydroxypyridine/2-pyridone, 4-hydroxypyrimidine/pyrimidinone, imidazole N-H, guanidine all share raw InChIKey14), but not keto-enol (acetylacetone keto vs enol: YRKCREAYFQTBPV vs POILWHVDKZOXJZ; phenol vs its keto form; 2-methyl-1,3-cyclohexanedione). RDKit's TautomerEnumerator.Canonicalize folds those too (all pairs collapse). It does not fold ring-chain tautomers that change connectivity (warfarin open-chain keto vs cyclic hemiketal: PJVWKTKQMONHTI vs PIBBOXWKSPNJFI stay distinct) — so if Enveda's answer and your candidate database store a different ring-chain form of a hemiketal/hemiacetal natural product, no amount of ranking will hit. Enumeration limits: default MaxTautomers = MaxTransforms = 1000; isoquercitrin hits MaxTransformsReached after 162 tautomers (0.33 s), procyanidin B2 after 422 (0.62 s). Robustness of the canonical key when the enumeration is truncated is tested in §4a.

Hash safety is not the issue: first-block collision probability is 1.3% per database of 10⁹ compounds (Pletnev et al., J Cheminform 4:39, 2012, DOI 10.1186/1758-2946-4-39), i.e. zero expected collisions in a 400-molecule test.

Consequence: every hexose epimer, every enantiomer, every E/Z pair in a candidate pool is the same answer. A pool that is not collapsed to canonical-tautomer InChIKey14 before ranking spends 25-slot budget on duplicates; a pool that is collapsed gets, e.g., all of glucoside/galactoside/mannoside for one slot. Conversely, regioisomers of glycosides (3-O vs 7-O), which MS/MS discriminates only weakly, remain distinct answers — exactly the "positional isomers" Schymanski et al. 2017 named as the frontier.

4a. Does the canonical tautomer depend on the input tautomer when enumeration is truncated?

Negative result (good for the benchmark, RDKit 2026.03.6, default limits): for six natural products we enumerated every tautomer RDKit can reach, re-parsed each from SMILES as a submission would be, and canonicalised independently.

Two readings. (1) The scorer's canonicalisation is robust: input drawing does not change the key, truncation does not change the key, and the result is idempotent — so a "tautomer error rate" of the scorer is, on this evidence, ~0 for drawings that differ only by H/double-bond placement. (2) The raw first block is not: 162 of 162 enumerated isoquercitrin tautomers have different raw InChIKey14s, because standard InChI's mobile-H layer does not cover the keto-enol class that dominates polyphenols. A candidate pool, a training label, or a dedup step keyed on raw InChIKey14 (what train.parquet's inchikey14 column most likely is — unverified) can carry many "different" entries for one scorer-identical molecule. Caveat: this is 2026.03.6; the scorer runs 2026.03.3.

5. Is MRR@25 with any-hit a lower standard than CASMI 2016 Top-1?

• CASMI 2012 scored absolute worst-case rank; CASMI 2016 scored "gold medals" (best rank among contestants per challenge) and reported Top-1 counts; CASMI 2022 (Fiehn lab page) judged 2D structure by "first block of the InChIKey" and also scored adduct, formula and compound-class categories.
• MRR@25 gives 1.0 / 0.5 / 0.333 / … / 0.04 for ranks 1 … 25 and 0 beyond. Relative to Top-1 it is more lenient (partial credit), relative to the 2016 medal system it is absolute (your rank, not your rank versus other teams). Because the credit curve is 1/r, roughly 80% of any realistic score comes from ranks 1–3; "any-hit within 25" is worth at most 0.04 per molecule, so the 25-slot list is a recall safety net, not a scoring lever. Back-solved from a GitHub write-up (recall@25 ≈ 0.55 at MRR ≈ 0.417), the median hit is at rank ~1–2 when it is in the list at all.
• "Any-hit" also means per molecule, not per spectrum: 1–16 spectra (median 3) per molecule_id collapse to one unit of 400. Molecules with 16 spectra carry the same weight as molecules with one.
• Community view: de Jonge et al. 2022 recommend Top-1 as "the most informative approach" with top-10 as a complement; MSI levels (Sumner 2007; Schymanski 2014) are confidence tiers, not ranks — a ranked list of 25 is Level 3 "tentative candidate(s)". Nothing in the MSI/mQACC documents opened endorses rank-k as a reporting standard. So yes: MRR@25 is a lower evidentiary standard than Top-1 and much lower than MSI Level 1/2, but it is a reasonable benchmark metric because it is continuous and rewards calibrated ordering.

6. What a "good" benchmark would additionally carry, and what the test actually provides

Forced analogy: law of evidence (Daubert / Frye)

Statement. Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (decided June 28, 1993) replaced Frye's single question — is the technique "sufficiently established to have general acceptance in the particular field in which it belongs" — with a "flexible" reliability inquiry under FRE 702: (1) "whether the theory or technique in question can be (and has been) tested"; (2) "whether the theory or technique has been subjected to peer review and publication"; (3) "the known or potential rate of error"; (4) "the existence and maintenance of standards controlling the technique's operation"; (5) "general acceptance". Since 1 Dec 2023, FRE 702 requires the proponent to show these "more likely than not". Put CASMI 2026 — the measurement instrument that will decide $50,000 and the field's "persistent benchmark" — in the witness box.

Concrete, testable idea from the failing prong (error rate). Build the error budget Enveda did not publish, and let it govern model selection: (i) bootstrap MRR@25 over molecules (not spectra) on a 400-molecule, MCES-disjoint (threshold 10, MassSpecGym rule) local hold-out of natural-product-like timsTOF spectra from enveda-180; report the 95% interval (expect ≈ ±0.04); refuse to prefer model A over B unless the paired-bootstrap difference clears it. (ii) Measure slot waste: for every submission, count top-25 entries that collapse to an already-present canonical-tautomer InChIKey14 — those slots are literally zeros. §4 shows raw-key dedup leaves keto-enol duplicates and stereo duplicates intact. (iii) Measure unhittable answers: fraction of hold-out truths whose candidate pool contains no entry with the same canonical key (ring-chain forms, salts, charged species) — this is the label-side error rate we can estimate for ourselves; back-solved recall@25 ≈ 0.55 at MRR 0.417 says the ceiling is coverage, so this number, not the ranker, is the first thing to fix.

Where the analogy holds, partially holds, breaks. It holds for prongs 1–3: a benchmark is a measuring instrument and Daubert's questions (tested? reviewed? error rate?) are exactly the metrology questions MassSpecGym and Dührkop 2015 ask. It partially holds for prong 4 — Daubert's "standards" concern the technique's operation, and here the scorer is standardised while the sample is not, which the law would treat as a chain-of-custody problem rather than a method problem. It breaks on gatekeeping: a court admits or excludes one method on a preponderance showing before the trial; Kaggle admits every method, and the "trial" (the hidden set) is the only showing anyone gets. It also breaks on adversarial posture — Daubert assumes cross-examination, but the benchmark's proponent (Enveda) is also judge, sample collector and label author, with no discovery of the labels until the contest ends. Finally, Frye's "particular field" is ambiguous here: the MS community (Fiehn) and the ML community (Kaggle) accept different things, and the benchmark was explicitly built for the latter.

Gotchas / what could fool us

1. Visible test is a decoy (rows of train, enveda-180). Any local score on it is a training-set score. Verified by multiple GitHub write-ups ("every spectrum in it is also in train").
2. Dedup on the wrong key. Raw InChIKey14 leaves keto-enol tautomers as separate entries (acetylacetone: two raw keys, one canonical); canonical-tautomer InChIKey14 is the scorer's key. Dedup on anything else wastes slots. Conversely, ring-chain tautomers (hemiketals) are not collapsed — if the database form differs from Enveda's drawn form, the molecule is unhittable.
3. RDKit version drift. Scorer uses 2026.03.3; our box has 2026.03.6. The tautomer rule set has been rewritten before (the 2020 releases); we did not verify equality between .3 and .6 — pin 2026.03.3 in the submission pipeline and diff canonical keys for the full candidate pool.
4. Truncated tautomer enumeration (MaxTransformsReached at 1000) on routine polyphenol glycosides does not make the canonical key input-dependent in our tests (§4a) — but the raw inchikey14 column in train may split one scorer-identical molecule across up to hundreds of raw keys; count unique canonical keys before quoting "275,810 structures".
5. Stereo collapse is a gift and a trap. Gift: hexose epimers, enantiomers, E/Z all merge — one candidate covers the family. Trap: candidate pools (COCONUT, PubChem) hold many stereo-variants of the same skeleton; un-collapsed pools shrink the effective 25.
6. Regioisomers remain distinct (3-O vs 7-O glycosides, methoxy positions). These are the realistic misses, and MS/MS evidence for them is weak (Schymanski 2017: "sufficient challenge for the next few years"). Expect reciprocal-rank credit of 0.5–0.33 on families, not 1.0.
7. Metric noise ≈ ±0.02 SE (400 units). Public LB (if computed on a subset) is noisier still. Don't chase deltas < 0.04; use 2 final submissions as a hedge, not a hope.
8. Metadata/popularity priors can mislead — CASMI 2016 showed reference counts alone ranked most substances 1st or 2nd when the answers were well-known compounds; CASMI 2026's answers are "none previously published", so a database-popularity prior may be anti-informative on the novel class and informative only on the known class.
9. Near-duplicate leakage in local validation. InChIKey14-disjoint splits leak Tanimoto > 0.85 neighbours (MassSpecGym); scaffold splits still leave "almost all" test structures with a close train neighbour (Kretschmer 2025). Use MCES-distance splits; expect local scores to drop by roughly half versus random splits (58.5 → 34.4; 70 → 36).
10. Single-instrument test vs 82-instrument train. Models that learn instrument signatures will mis-weight Orbitrap-style fragment intensities; the C&EN-cited Nature Metabolism piece says ML "struggle[s] to recognize the same molecule captured under different experimental conditions".
11. Merged multi-CE spectra are not any single library spectrum; cosine against single-CE references is systematically biased. Merge references the same way before comparing.
12. Adduct is given, so errors in Enveda's adduct assignment are label errors you cannot detect — the only comparable rate is 0.8% (CASMI 2022 precursor corrections). Budget ~1% unhittable molecules.
13. Per-molecule aggregation: 16 spectra for one molecule do not earn 16× credit; a molecule with a single noisy spectrum weighs the same. Aggregate predictions across spectra (max or rank-fusion), do not vote per spectrum.
14. Mirror-sourced facts. Hidden-test adduct list, MW range, preprocessing, "three novelty classes" and ~1,500 spectra come from a GitHub mirror of the Kaggle data card, not the Kaggle page itself (JS shell); the press release says ~2,500 spectra.

Implications for the challenge

1. Collapse every candidate pool to canonical-tautomer InChIKey14 (RDKit 2026.03.3, TautomerEnumerator.Canonicalize → MolToInchiKey[:14]) before ranking and before writing the 25 slots. Free recall; zero risk.
2. Audit unhittable truths: on the local hold-out, measure the fraction of truths with no canonical-key match in the pool; if > 10%, pool construction (ring-chain forms, salt stripping, charge neutralisation, COCONUT + PubChem-NP union) is the highest-leverage work, not the ranker.
3. Validate MCES-disjoint, molecule-level, with bootstrap CIs; publish our own "error rate" table (SE, slot waste, unhittable %) so model decisions are made on paired-bootstrap differences, not point estimates.
4. Treat the problem as the MassSpecGym "bonus" task: after a ≤10 ppm neutral-mass window all candidates are isomers; invest in isomer-discriminating features (fragment connectivity, neutral-loss sequences across the merged CE list), not in mass filters.
5. Mimic the hidden measurement in validation: timsTOF-only enveda-180 spectra, multi-CE merged exactly as the test is, both polarities, the 10 adducts, MW 157–1,159 Da.
6. Spend the 25 slots on distinct skeletons: after the top ~5, prefer diversity (different regioisomer families) over near-duplicates — credit beyond rank 5 is ≤ 0.2, but a zero is a zero.
7. Do not trust popularity priors on the novel class; gate them by a "known-vs-novel" classifier (library-search score as the gate), per the three-class structure.
8. Keep the physical limitation list as a product feature: anything we build is Level 3 evidence at best (no RT, no standard); the dashboard should say so.

Open questions

• What is Enveda's label-error rate (structure assignment, adduct assignment) on the hidden set, and will it be published with the "persistent benchmark"?
• Were TIMS mobilities recorded for the test compounds, and will CCS ever be released (it would convert the benchmark from one to two orthogonal data per Sumner 2007)?
• Does RDKit 2026.03.3 vs 2026.03.6 give identical canonical tautomers over the full candidate pool? (Testable locally once 2026.03.3 is installed.)
• How many hidden molecules fall in each of the three novelty classes, and is the public LB a subset of the hidden set or a separate sample?
• Is the ~1,500 (data-card mirror) or ~2,500 (press release) spectra figure correct?

Sources (all opened unless marked)

• Oldach L. "Metabolomics has a data problem. Could this competition fix it?" C&EN, 17 Sept 2026. https://cen.acs.org/analytical-chemistry/mass-spectrometry/competition-identify-small-molecules-capture/104/web/2026/09 ; DOI 10.47287/cen.612552.newsarticle
• Enveda press release (Yahoo Finance mirror), 15 Sept 2026. https://finance.yahoo.com/healthcare/articles/enveda-launching-next-alphafold-contest-120700676.html
• Schymanski EL, Neumann S. "CASMI: And the Winner is..." Metabolites 2013;3:412–439. DOI 10.3390/metabo3020412. https://pmc.ncbi.nlm.nih.gov/articles/PMC3901266/
• Schymanski EL et al. "Critical Assessment of Small Molecule Identification 2016: automated methods." J Cheminform 2017;9:22. DOI 10.1186/s13321-017-0207-1. https://pmc.ncbi.nlm.nih.gov/articles/PMC5368104/ (and Europe PMC full-text XML)
• Schymanski EL et al. "Identifying small molecules via high resolution mass spectrometry: communicating confidence." Environ Sci Technol 2014;48:2097–2098. DOI 10.1021/es5002105 (bibliographic record via Europe PMC; levels paraphrased from https://www.thermofisher.com/blog/analyteguru/all-about-the-new-annotation-confidence-scoring-node-in-compound-discoverer-software/)
• Sumner LW et al. "Proposed minimum reporting standards for chemical analysis." Metabolomics 2007;3:211–221. DOI 10.1007/s11306-007-0082-2. https://pmc.ncbi.nlm.nih.gov/articles/PMC3772505/
• Theodoridis G, Fiehn O, et al. "What's in a name? Metabolite identification: challenges and pitfalls in untargeted metabolomics." Metabolomics 2026;22:22. DOI 10.1007/s11306-025-02387-0. https://pmc.ncbi.nlm.nih.gov/articles/PMC12886295/
• Bushuiev R et al. "MassSpecGym: A benchmark for the discovery and identification of molecules." NeurIPS 2024; arXiv:2410.23326 (v2, 14 Feb 2025). https://arxiv.org/html/2410.23326
• Dührkop K et al. "Searching molecular structure databases with tandem mass spectra using CSI:FingerID." PNAS 2015. https://pmc.ncbi.nlm.nih.gov/articles/PMC4611636/
• Kretschmer F et al. "Coverage bias in small molecule machine learning." Nat Commun 2025. DOI 10.1038/s41467-024-55462-w. https://pmc.ncbi.nlm.nih.gov/articles/PMC11718084/
• Hoffmann MA et al. "High-confidence structural annotation of metabolites absent from spectral libraries." Nat Biotechnol 2022. DOI 10.1038/s41587-021-01045-9. https://pmc.ncbi.nlm.nih.gov/articles/PMC8926923/
• de Jonge NF et al. "Good practices and recommendations for using and benchmarking computational metabolomics metabolite annotation tools." Metabolomics 2022. DOI 10.1007/s11306-022-01963-y. https://pmc.ncbi.nlm.nih.gov/articles/PMC9722809/
• Bittremieux W, Wang M, Dorrestein PC. "The critical role that spectral libraries play in capturing the metabolomics community knowledge." Metabolomics 2022. DOI 10.1007/s11306-022-01947-y. https://pmc.ncbi.nlm.nih.gov/articles/PMC10284100/
• Pletnev I et al. "InChIKey collision resistance: an experimental testing." J Cheminform 2012;4:39. DOI 10.1186/1758-2946-4-39. https://pmc.ncbi.nlm.nih.gov/articles/PMC3558395/
• Fiehn lab CASMI 2022 page. https://fiehnlab.ucdavis.edu/casmi
• Khoo LMS, Barzilay R. "Why machine learning fails at mass spectrometry for small molecules." Nat Metab 2026;8:1247–1249. DOI 10.1038/s42255-026-01544-6 — not opened (paywall); cited via C&EN paraphrase and Europe PMC record.
• Mosley JD et al. Metabolomics 2024, DOI 10.1007/s11306-023-02080-0 — PDF fetched, body not extractable; not relied on.
• bioRxiv 10.1101/2024.07.30.605945 ("identification probability") — not opened (HTTP 429).
• Kaggle data-card mirror: https://raw.githubusercontent.com/OfficialBhattacharya/kaggle_CASMI_2026/main/docs/COMPETITION.md ; https://raw.githubusercontent.com/corentinlaudicina/enveda-casmi-2026/main/README.md (Kaggle page itself is a JS shell and was not read directly)
• Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993). https://www.law.cornell.edu/supremecourt/text/509/579 ; FRE 702 (as amended 1 Dec 2023). https://www.law.cornell.edu/rules/fre/rule_702
• Local experiment: RDKit 2026.03.6, scripts ik14_test.py and taut_robust.py (scratchpad), outputs reproduced in §4 / §4a.