TL;DR. Until 2025, de novo generators scored 0.00% exact matches on the only structure-disjoint public benchmark (MassSpecGym). Between Feb 2025 and Jul 2026 the exact-match rate on MassSpecGym jumped from 2.3% (DiffMS) to 18.3% (FRIGID), 23.9% (DualLGD) and 23.9%/28.7% top-1/top-10 (MS-GPT) — given the true molecular formula, [M+H]+ only, and with a leakage audit ("MassSpecGym in the Wild") showing that 17/26 papers using the benchmark had evaluation problems. Open MIT/Apache code with 1-3 GB checkpoints and ~1-7 s/spectrum inference exists today (FRIGID, DualLGD, MS-GPT, DiffMS, MADGEN), so a de novo filler for the ~45% of hidden molecules that never reach the top-25 of a COCONUT retrieval is feasible inside a Kaggle notebook; the realistic gain is a few hundredths of MRR, not a leap, and nobody has published a working "is the answer in the database?" detector.
1. Method table
Per-method numbers are exact-match (InChIKey14-style connectivity) top-1 / top-10 on the two de facto benchmarks: NPLIB1 (= CANOPUS GNPS subset, ~800 test spectra, random-ish split with many test molecules at Tanimoto >> 0.85 to training molecules — DiffMS and MBGen both say so) and MassSpecGym (231k spectra / 29k structures, test molecules at MCES edit distance >= 10 from any training molecule; [M+H]+; QTOF + Orbitrap; 0.01 Da bins). Numbers are copied from the papers' own tables as fetched on 2026-10-07; the JSON twin of this table carries sources per row.
| Method (year, venue) | Open code / weights today | Formula needed? | Needs SIRIUS / frag tree? | NPLIB1 top-1 / top-10 | MassSpecGym top-1 / top-10 | GPU / speed |
|---|---|---|---|---|---|---|
| MassGenie (2021, Biomolecules) | No (code "in due course"; only FragGenie) | yes (formula filter on predicted ion) | no | n/a | n/a | A100, 2.6 s/prediction; own test: 49/93 exact on CASMI-2017 <500 Da |
| MSNovelist (2022, Nat Methods) | Code CC BY 4.0, github.com/meowcat/MSNovelist marked obsolete; live only inside SIRIUS 6 (account/licence) | yes | yes (CSI:FingerID fingerprint + SIRIUS formula) | 5.40 / 11.04 (as MIST+MSNovelist in DiffMS) | 0.00 / 0.00 | CPU, <5 min/spectrum; own GNPS test (n=3,863): 25% rank-1, 45% in top-128 |
| Spec2Mol (2023, Commun Chem) | Code github.com/KavrakiLab/Spec2Mol, no licence/weights shown | no formula; MW assumed known; wants 4 NIST-style spectra (±, 35/130% NCE) | no | 0.00 / 0.00 | 0.00 / 0.00 | unspecified; own NIST20 random split: 7.0% exact |
| MS2Mol (2023, Enveda, ChemRxiv) | Closed (no code, no weights) | no (BART spectrum tokens -> SMILES; 25 beams + ranker) | no | n/a | n/a | n/a; EnvedaDark (226 NPs absent from DBs): 21% "close match", 62% "meaningful"; exact matches "very few" |
| MADGEN (ICLR 2025) | MIT, github.com/HassounLab/MADGEN; ckpts 40-67 MB | yes (picks scaffold candidates) | no | 2.10 / 2.39 (oracle scaffold 18.7 / 22.2) | 1.31 / 1.54 (oracle 10.5 / 12.4) | CUDA GPU; scaffold retrieval accuracy only 13.2% on MSG |
| DiffMS (ICML 2025) | MIT, github.com/coleygroup/DiffMS; 940 MB ckpt (Zenodo 15122968) | yes (MIST-CF OK; Table 3 ablation small loss) | no (MIST encoder) | 8.34 / 15.44 | 2.30 / 4.25 | 2080Ti 12 GB; ~4 min per 100 samples (131-160 s/spectrum) |
| OMG (Anal Chem 2025, Hassoun) | unverified | yes (REINVENT4 fine-tuned on PubChem hits of that formula) | no | 10.51 / — | 2.42 / — | unspecified |
| MS-BART (NeurIPS 2025) | CC BY 4.0 paper; github.com/OpenDFM/MS-BART; weights Figshare | formula only for re-ranking | no (MIST fingerprint) | 7.45 / 10.99 | 1.07 / 1.11 | ~3 s/spectrum consumer GPU |
| TTT-LM, IBM/ETH (ICLR 2026) | MIT, rxn4chemistry/MultimodalAnalytical@ttt-msms | yes | no | 12.88 / 31.32 | 3.16 / 6.07 | ~150 M params; time unspecified |
| MBGen (arXiv 2602.01643) | github.com/biomed-AI/MBGen (no licence/ckpt in README) | yes (SIRIUS formula) | no | 12.20 / 22.29 | 7.58 / 12.54 | unspecified |
| FlowMS (arXiv 2603.18397) | unverified | yes | no | 9.15 / — | — | unspecified |
| FRIGID (arXiv 2604.16648, Coley) | Apache-2.0 code (README) but paper says CC BY-NC-SA; weights CC-BY-NC-4.0, 2.8 GB (Zenodo 19685145) | yes (formula + MIST fp); formula-free re-run by MARLIN: 13.95 / 23.29 | no (ICEBERG forward model for refinement) | 25.03 / 33.37 (base 19.80) | 18.29 / 22.00 (base 16.09) | 6.58 s/spectrum on 1x RTX 6000 Ada; 182 M params |
| DualLGD (arXiv 2605.07048, UNC Charlotte) | MIT code; weights CC BY 4.0, 3.2 GB (Zenodo 20281201) | yes (atoms fixed, only bonds denoised) | no | 34.37 / 44.96 | 23.89 / 33.08 | H200 used (46 s/epoch NPLIB1 fine-tune); 67.6 M params; T=500 steps |
| MARLIN (arXiv 2607.04774, UNC Charlotte) | not yet ("upon publication") | no (mass-shell constrained decoding; recovers correct formula for 76.7% of spectra) | no (DreaMS or MIST encoder) | 19.18 / 26.65 (MIST), 16.94 / 23.54 (DreaMS) | not reported | 34.1 s/spectrum, 384 candidates |
| MS-GPT (arXiv 2607.23607, SJTU/ByteDance) | Apache 2.0, github.com/VIKI623/MS-GPT, ckpts on SJTU cloud | yes | no (dataset-specific MIST encoder) | 29.76 / 41.07 (B=1500: 30.95 / 46.88) | 23.91 / 28.65 | 0.93 s/spectrum at B=100, RTX 6000 Ada |
| MassSpecGym baselines (NeurIPS 2024) | github.com/pluskal-lab/MassSpecGym | with and without | no | — | 0.00 / 0.00 (SMILES, SELFIES transformers) | — |
Similarity metrics where reported (top-1 Tanimoto, Morgan): DiffMS 0.35 / 0.28, MS-BART 0.44 / 0.23, MBGen 0.41 / 0.38, FRIGID 0.58 / —, DualLGD 0.60 / 0.52, MS-GPT 0.61 / 0.52, TTT-LM 0.62 / 0.46 (NPLIB1 / MassSpecGym). MCES@1: DiffMS 11.95 / 18.45, DualLGD 5.44 / 10.11, MS-GPT top-10 MCES 5.51 on NPLIB1.
Random vs structure-disjoint. The same model loses roughly half or more of its exact-match rate going from NPLIB1 (near-duplicate-rich) to MassSpecGym (MCES >= 10): DiffMS 8.3 -> 2.3, MBGen 12.2 -> 7.6, FRIGID 25.0 -> 18.3, DualLGD 34.4 -> 23.9, MS-GPT 29.8 -> 23.9. The TTT-LM paper shows the split definition inside NPLIB1 matters less (NPLIB1-Full 12.88 vs NPLIB1-DiffMS 13.95). Older own-test numbers (MSNovelist 25% rank-1 on GNPS, Spec2Mol 7% on NIST20, MassGenie 53% on CASMI-2017) are random/temporal splits and are not comparable — when re-run under DiffMS's protocol MSNovelist and Spec2Mol give 0.00% on MassSpecGym.
2. What is open today (and what it costs to run)
- Ready-to-run with public weights, permissive licence: DiffMS (MIT, 940 MB), MADGEN (MIT, 40-67 MB), DualLGD (MIT code, CC BY 4.0 weights, 3.2 GB), MS-GPT (Apache 2.0; weights behind SJTU cloud link), MS-BART (Figshare weights), TTT-LM (MIT).
- Open but non-commercial weights: FRIGID weights are CC-BY-NC-4.0 (Zenodo) and the paper states CC BY-NC-SA 4.0 for the method although the README says Apache-2.0 for code. Whether a prize competition with an open-sourcing clause counts as non-commercial use is a legal question we have not resolved (flagged, unverified).
- Not open: MS2Mol (Enveda), MassGenie, MARLIN (promised), OMG (unverified), FlowMS (unverified).
- Common dependency: every top performer (DiffMS, MS-BART, FRIGID, MS-GPT, MBGen, DualLGD) conditions on a MIST spectrum-to-fingerprint encoder trained on the benchmark's own training split, and on a known formula (ground truth in the headline tables). None needs a SIRIUS fragmentation tree; MSNovelist is the only one that needs CSI:FingerID. DiffMS Table 7: MIST-CF formula accuracy is 84% top-1 on NPLIB1 but only 48% top-1 / 69% top-5 on MassSpecGym — the formula step, not the generator, is the first bottleneck in the realistic setting. MARLIN's mass-shell decoding is the only published formula-free route (76.7% implicit formula recovery) and it is not released.
- VRAM: DiffMS was trained/run on 12 GB 2080Ti; MS-GPT and FRIGID report an RTX 6000 Ada (48 GB) but inference is single-pass/short, so a Kaggle T4/P100 (16 GB) should suffice at reduced batch (unverified for DualLGD, whose line-graph attention is O(M) with FAVOR+ and whose authors note softmax attention is impractical beyond ~60 heavy atoms).
- Wall-clock for the hidden test (~1,500 spectra / ~400 molecules): MS-GPT ~0.9 s x 1,500 = ~25 min; FRIGID 6.6 s x 1,500 = ~2.8 h; MARLIN 34 s -> ~14 h; DiffMS 131-160 s -> 55-65 h (not viable unbatched). Running per molecule on a merged spectrum cuts this ~4x.
3. Does any de novo method hit InChIKey14 at a rate useful for MRR@25?
Yes on paper, with heavy caveats. Threshold in the brief: >5% top-25 on novel molecules. On the MCES-disjoint MassSpecGym test, top-10 exact-match is 22.0% (FRIGID), 28.65% (MS-GPT), 33.08% (DualLGD), 12.54% (MBGen); top-25 is not reported but MS-GPT's candidate-pool scaling (top-10 rising from 41.07% to 46.88% on NPLIB1 as B goes 100 -> 1500) shows that extra slots keep paying. Even DiffMS at 4.25% top-10 falls below the bar.
Discount factors to apply before believing these for the hidden test:
- Formula given. Multiply by your formula accuracy. MIST-CF on MassSpecGym is 48% top-1; a timsTOF with isotope pattern and ±5 ppm precursor should do better (SIRIUS-class formula assignment on isolated pure compounds is typically far higher, but that is another worker's topic). MARLIN's formula-free protocol re-ran FRIGID at 13.95% top-1 on NPLIB1 (down from 25.03%).
- [M+H]+ only, single spectrum. MassSpecGym training/eval is restricted to [M+H]+ on QTOF/Orbitrap; the hidden test has 7 adducts and both polarities. No published de novo number exists for [M+Na]+ or negative mode.
- Leakage audit. "MassSpecGym in the Wild" (arXiv 2606.19624, Liu, Coley, Pluskal et al.) found evaluation problems in at least 17 of 26 papers; decoder pretraining corpora filtered only by exact InChIKey let MolForge top-1 exceed 30% vs ~11% under strict MCES >= 10 separation, and the original MIST encoder had a padding-mask bug (
attn += attn_maskwithout -inf conversion) that inflated MIST+MolForge de novo top-1 by ~17 pp (31.75% -> 10.73% after fix). The audit's corrected de novo leaderboard (its Table 9) still lists FRIGID at 18.29 / 22.00 and DiffMS at 2.30 / 4.25, i.e. those two survive; DualLGD and MS-GPT post-date the audit and are unaudited (MS-GPT's SAFE-GPT backbone saw ~100 M structures; DualLGD's pretraining corpus filtering vs test is not stated in what I read). - NP-likeness. NPLIB1 is the NP benchmark and every method scores higher there (DualLGD 34.4%, MS-GPT 29.8%), but part of that is near-duplicate leakage (Tanimoto >> 0.85 train/test neighbours), so "better on NP-like molecules" is confounded with "easier split". MS2Mol's EnvedaDark numbers (21% "close match" on 226 NPs absent from databases) are the only NP-novel numbers from the host, and they are similarity thresholds, not exact matches.
Back-of-envelope for the challenge: public write-ups put recall@25 of the COCONUT-retrieval pipeline at ~0.55 at MRR 0.417, so ~45% of hidden molecules are currently scored 0. If a de novo filler achieved, on those molecules, an MRR of ~0.19 (FRIGID-like: 0.183 top-1 + a little from ranks 2-10) x 0.6 formula correctness x ~0.8 adduct/instrument penalty ≈ 0.09, the overall gain is 0.45 x 0.09 ≈ +0.04 MRR — about the gap between rank ~110 and the top-20 on the public board. With DualLGD/MS-GPT-level numbers the ceiling is ~+0.08. Those are assumptions, not measurements; item 2 in Implications says how to measure them.
4. Hybrid use: generated structures as pool augmentation and as similarity anchors
| Hybrid idea | Who tried it | What was measured | Verdict |
|---|---|---|---|
| (a) Add generated structures to the candidate pool | MSNovelist (2022): de novo candidates scored with the same CSI:FingerID/ModPlatt score as DB hits; on 224 bryophyte spectra the de novo candidate outscored the DB best for 169 (75%), with 7 spectra showing ΔModPlatt > 50 | Score, not correctness — a generator that over-fits the predicted fingerprint will always outscore the DB; the paper's GNPS test says DB search (CSI:FingerID) gets 39% rank-1 vs MSNovelist 25% with identical fingerprints | Augmentation helps only where the DB has no member; never let the fingerprint-matching score alone re-rank DB vs de novo |
| (a') Generate, then rank with an independent ranker | OMG (Anal Chem 2025): REINVENT4 fine-tuned on PubChem molecules of the target formula, generated candidates ranked by JESTR/ESP; 10.51% (CANOPUS), 2.42% (MSG) top-1 | Exact match | Independent ranker on generated pool works but does not beat pure generators |
| (b) Generated molecule as similarity anchor to re-rank DB candidates | GLMR (AAAI 2026, arXiv 2511.06259, Zhang et al.): contrastive retrieval picks top candidates as context, a generator produces a refined molecule, DB candidates re-ranked by similarity to it; ">40% improvement in top-1" on MassSpecGym retrieval and MassRET-20k | Retrieval hit@1 (relative improvement; absolute numbers not in abstract) | The only paper quantifying anchor-style re-ranking; code URL not found in what I read; the "MassSpecGym in the Wild" candidate-set artefacts (non-canonical ground-truth SMILES, PubChem ordering giving 49.98% hit@1 spectrum-blind) make any MassSpecGym retrieval gain suspect until re-checked |
| (c) Retrieve a scaffold, generate the rest | MADGEN: scaffold-retrieval accuracy 13.2% (MSG) / 17.7% (CANOPUS) / 40.3% (NIST23); with oracle scaffold top-1 jumps 1.31 -> 10.5% (MSG) | Exact match | The retriever is the bottleneck; a COCONUT-derived scaffold pool for NPs is an obvious untested variant |
| (d) Inference-time cycle consistency with a forward model | FRIGID: ICEBERG re-scores generated candidates against the measured spectrum and re-masks spectrum-inconsistent fragments; base 19.80 -> 25.03 (NPLIB1), 16.09 -> 18.29 (MSG) | Exact match | A forward simulator is a legitimate re-ranker for both DB and de novo candidates — but the audit shows ICEBERG trained on test molecules inflates hit rates 9.57% -> 67.75%, so train it strictly split |
| (e) Kaggle-side gating between pools | Isaackjoshua/casmi-2026 write-up: per-molecule gate between curated and PubChem pools, accuracy 42.8% vs 59.4% base rate — "no signal and it points the wrong way"; chose tiering (curated first, PubChem fills empty slots) instead | MRR@25 on local validation | Negative result; slot-filling beat switching |
No paper quantifies the exact combination we need (COCONUT retrieval + de novo filler scored by MRR@25 on novel NPs); the closest evidence is MS-GPT's B-scaling and the Kaggle tiering result.
5. MS2Mol / PRISM (Enveda): what is public, and the hidden-test design
- MS2Mol (Butler et al., ChemRxiv June 2023; press release 2023-06-21): BART encoder-decoder from tokenized spectrum to SMILES, 25-beam output, validity filter, secondary ranker. Evaluated on EnvedaDark (226 NP spectra for molecules absent from major databases), EnvedaLight (454 known NPs) and CASMI-2022. Reported: 21% within "close match" and 62% within "meaningful similarity" on EnvedaDark; on the top-10% most confident predictions 63% / 98%. Exact matches were "very few" (centuryofbio summary). The Tanimoto thresholds behind "close"/"meaningful" are in the PDF, which returned 403 on both ChemRxiv and ResearchGate; treat thresholds as unverified. No code, weights or EnvedaDark release found (Enveda GitHub org shows no such repo among its visible 10 of 37).
- PRISM (Enveda blog 2024-05-28; with Microsoft): BERT-style masked-peak model on 1.2 B spectra (~600 M public, ~600 M internal), 85 B tokens; downstream: 7-16% relative R² gain on 12 properties, 23% relative gain in spectral-library retrieval. Nothing public on PRISM-conditioned de novo generation. Enveda's own 2021-2025 review (PMC13373924) says de novo tools are "reliable only for lower-molecular-weight compounds and representative chemical classes".
- Hidden-test design implications. The C&EN/Yahoo coverage describes ~400 molecules / ~2,500 spectra (Kaggle page says ~1,500), "none previously published", real or potential natural molecules and analogs from Enveda's isolation platform; enveda-180 synthetic molecules overlap COCONUT by only 0.03%. That is exactly the EnvedaDark recipe: the host has already built a benchmark on which its own generator scores 21% close-match and few exact matches. Expect a hidden set where a sizeable minority is outside COCONUT by design, which is why the public back-solved recall@25 is ~0.55. Whether Enveda enters its own models is not stated anywhere I found (unverified).
Forced analogy: medical differential diagnosis (diagnosis of exclusion)
Statement. Database retrieval is the textbook differential: rank the known diseases by likelihood given the findings. De novo generation is the novel-pathogen work-up: expensive, low-yield, justified only when the pre-test probability that the answer is in the textbook, P(in-DB), is low. A good clinician does not run the exotic panel on everyone; they estimate P(in-DB) from how well the textbook answers fit (score, margin to runner-up) and from red flags (presentation unlike anything catalogued).
Mapping to our measurement. Pre-test probability features that exist in the literature:
- Score and margin: COSMIC (Hoffmann et al., Nat Biotech 2022) builds exactly this — calibrated CSI:FingerID score, difference to runner-up, fraction of intensity explained by the fragmentation tree, fingerprint cardinality, linear SVM with enforced feature directionality, Platt-scaled. In cross-validation 22.70% of queries had the correct answer missing from the DB; among its nine highest-confidence wrong annotations, seven were cases where the true structure was not in the database. The authors state COSMIC cannot inherently tell whether the correct structure exists in the DB ("non-trivial to generate decoys").
- Conformal sets: Rakhshaninejad et al. (bioRxiv 2026.03.12.711424) build spectrum-specific prediction sets with guaranteed coverage; under distribution shift the sets expand, and confidence-based grouping improves subgroup reliability. Set size is a usable "how sure is retrieval" statistic, but it says nothing about the true molecule being absent.
- Formula-level novelty: trivially computable — if no COCONUT/train structure has the assigned formula (within ppm), P(in-DB)=0 and the slots are free anyway.
- Explicit spectral novelty detector: none found in the de novo literature (MARLIN, FRIGID, DiffMS state none). The only direct test is the Kaggle write-up's gate (42.8% vs 59.4% base rate: no signal).
Concrete, testable gating protocol (slot-filling, not switching):
- Build a validation set that mimics the hidden test: hold out ~400 train molecules, and for a "class-3" subset remove their structures from the candidate DB (the OfficialBhattacharya write-up does this).
- For each molecule compute: retrieval top-1 score s1, margin s1 - s2, conformal set size, pool size at ±5 ppm, formula-in-COCONUT flag, nearest-library cosine.
- Fit a logistic model for P(true structure in top-25 of retrieval) on the validation set. Report AUC; if AUC < 0.6 the analogy breaks for us as it did for that Kaggle team.
- Regardless of AUC, fill slots: top-k retrieval candidates then de novo candidates (deduplicated by InChIKey14 after RDKit tautomer canonicalisation), with k = 25 when P is high and k decreasing to ~10 when P is low; sweep k and measure MRR@25. Expected: the gain comes almost entirely from molecules with empty or tiny pools, where no gate is needed.
Does the analogy hold? Partially. It holds in structure (retrieval first, generation as exclusion work-up) and in the existence of pre-test features (COSMIC, conformal). It breaks on two points: (i) in medicine the exotic panel is costly, whereas de novo at 1-7 s/spectrum is cheap enough to run on everyone, so the right policy is "always run, decide only ordering"; (ii) the pre-test probability has no proven discriminative signal in the only field test, because retrieval scores saturate (fingerprint models produce a plausible top-1 for every spectrum, correct or not — model confidence is not accuracy, the sibling-project lesson).
Gotchas / what could fool us
- Headline numbers assume the ground-truth formula. Every >10% number in the table is formula-given. MIST-CF gives 48% top-1 formulas on MassSpecGym; MARLIN's formula-free re-run of FRIGID lost 11 points on NPLIB1.
- NPLIB1 is leaky; both DiffMS and MBGen say test molecules sit at Tanimoto >> 0.85 to training molecules. Quote MassSpecGym numbers, not NPLIB1, when estimating novel-molecule performance.
- Decoder pretraining leakage. Pretraining corpora (DiffMS 2.8 M fp2mol incl. COCONUT; FRIGID up to 1 B; MS-GPT SAFE-GPT ~100 M; DualLGD unstated) filtered only by exact InChIKey can contain near-neighbours of test molecules; the audit shows ~3x inflation (30% vs 11%) from Tanimoto >= 0.7 neighbours. For CASMI this cuts both ways: pretraining on COCONUT is good if hidden molecules are COCONUT-like, but our local validation must exclude neighbours of held-out molecules from the decoder's pretraining set or it will lie.
- MIST padding-mask bug. Any generator conditioned on a MIST encoder built before the v1.5 fix may carry inflated numbers (~17 pp top-1 inflation for MIST+MolForge; Tanimoto 0.37 -> 0.52). FRIGID and DiffMS numbers survived the audit's corrected Table 9; MS-GPT, DualLGD, MBGen, MS-BART are unaudited.
- Metric drift. The audit found InChIKey matching without the 14-char connectivity standardisation, different Morgan radii/bit-lengths, and MCES solver parameters each reorder leaderboards on identical predictions. CASMI's metric adds RDKit 2026.03.3 tautomer canonicalisation — de novo outputs with unusual tautomers (enols, keto-imines, 2-hydroxypyridines) must be canonicalised with the same RDKit before dedup, or two generated tautomers waste two slots.
- Synthetic-spectrum contamination. Simulating spectra for test-like molecules (ICEBERG/forward model trained on them) inflated MassSpecGym retrieval from 9.57% to 67.75%; the same trick on our held-out validation would make a forward-model re-ranker look miraculous.
- [M+H]+ only. No de novo model has been evaluated on sodium adducts or negative mode; the hidden test has 7 adducts and both polarities. Expect near-zero for [M+Na]+ unless the model is retrained with adduct tokens.
- Frequency ranking of samples (DiffMS, K=100) is not a probability. The generator's top-1 is its mode; for MRR the ordering among the 25 slots should come from an independent scorer (forward model, fingerprint cosine), not sample frequency alone.
- Multi-spectra per molecule. Benchmarks are single-spectrum; the hidden test has 1-16 spectra per molecule (median 3). Pooling generations across spectra (vote by InChIKey14) is free accuracy that no paper has measured — but also a trap if one spectrum is a noisy low-CE scan that generates garbage with high frequency.
- Licence trap. FRIGID weights are CC-BY-NC-4.0; the competition pays prizes and requires open-sourcing winners. Check rules before shipping FRIGID weights in a submission.
- Closed-host advantage. Enveda has PRISM (1.2 B spectra) and MS2Mol; any public write-up claiming "Enveda-level" de novo performance is unverifiable, and EnvedaDark (226 NPs) is not public, so the host's own novel-NP numbers (21% close match) cannot be reproduced.
- "Better on NP-like molecules" is confounded with the easier NPLIB1 split; there is no clean public NP-novel exact-match number for any generator.
Implications for the challenge
- Ship a de novo filler, not a de novo primary. Candidates: MS-GPT (Apache 2.0, 0.93 s/spectrum) and DualLGD (MIT/CC BY, 3.2 GB) first; FRIGID only after the licence question; DiffMS only as a fallback (too slow). Package weights as a Kaggle dataset; budget <1 h GPU for ~1,500 spectra with MS-GPT.
- Measure before trusting: build the class-3 validation (structures removed from the candidate DB) and report, per generator, exact InChIKey14 top-1/10/25 on our held-out NPs with (i) true formula, (ii) our own formula pipeline, (iii) per-adduct breakdown. If the formula-pipeline numbers on class-3 are below ~8% top-25, the filler is not worth the risk of displacing DB candidates — restrict it to empty slots.
- Slot policy: always fill all 25 slots. DB candidates first (ordered by the retrieval ranker), de novo candidates deduplicated against them, canonicalised with RDKit 2026.03.3 tautomer rules, InChIKey14-deduplicated. Sweep the DB/de novo split k on validation; expect the optimum near k=20-25 unless the per-molecule gate shows AUC > 0.65.
- Try the anchor re-rank (GLMR-style) on our own ranker: compute Tanimoto between each DB candidate and the top-3 generated molecules and add it as a feature to the re-ranker; it costs nothing once generation runs and is the one hybrid with a published positive result.
- Vote across spectra per molecule_id: generate per spectrum (or per merged multi-CE spectrum), aggregate by InChIKey14 with weights from a forward-model re-score; measure the gain over single-spectrum generation.
- Retrain or fine-tune the generator's encoder on the 2.5 M-spectrum train set with adduct and polarity tokens; no public checkpoint has seen [M+Na]+ or negative mode. MIST-style encoders fine-tune in ~1 h/epoch on an H200 (DualLGD: 22 min/epoch on MassSpecGym).
- Use COCONUT scaffolds as the MADGEN scaffold pool if the oracle-scaffold ceiling (10.5% -> 18.7%) looks attractive; measure scaffold-retrieval accuracy on class-3 first — it was the bottleneck (13.2%).
- Do not use retrieval score alone as a novelty gate; if a gate is wanted, train it on class-3 validation with COSMIC-style features and demand AUC > 0.65 before using it to drop DB candidates.
- Audit our pipeline the way the MassSpecGym audit did: single-spectrum inference check for MIST-type encoders, uniform RDKit canonicalisation of DB candidates and generated SMILES, pinned metric implementation, 99.9% bootstrap CIs on validation MRR.
Open questions
- What are MS2Mol's "close match" / "meaningful similarity" Tanimoto thresholds and its exact-match rate on EnvedaDark? (PDF blocked on ChemRxiv/ResearchGate; needs a manual download.)
- Does the MS-GPT SAFE-GPT backbone or the DualLGD pretraining corpus contain Tanimoto >= 0.7 neighbours of MassSpecGym test molecules? (Unaudited; would change the 24% claims.)
- What is any generator's exact-match rate under [M+Na]+ / negative mode? (No data anywhere.)
- Is FRIGID's weight licence (CC-BY-NC-4.0) compatible with the competition's open-sourcing/prize rules? (Rules not fetched; Kaggle page is JS-rendered.)
- Can the per-molecule P(in-DB) gate be rescued with conformal set size + formula-presence features, or does it stay at chance as in the one public attempt?
- Will MARLIN (formula-free, 19% NPLIB1) be released before 2026-12-14?
Sources
- MSNovelist: Stravs et al., Nat Methods 2022, doi:10.1038/s41592-022-01486-3 (read via https://pmc.ncbi.nlm.nih.gov/articles/PMC9262714/); repo README https://raw.githubusercontent.com/meowcat/MSNovelist/master/README.md; SIRIUS 6 docs https://v6.docs.sirius-ms.io/
- Spec2Mol: Litsa et al., Commun Chem 2023, doi:10.1038/s42004-023-00932-3 (https://pmc.ncbi.nlm.nih.gov/articles/PMC10290119/); https://raw.githubusercontent.com/KavrakiLab/Spec2Mol/master/README.md
- MassGenie: Shrivastava et al., Biomolecules 2021 (https://pmc.ncbi.nlm.nih.gov/articles/PMC8699281/)
- MS2Mol: ChemRxiv item 648dead54f8b1884b77082da (PDF 403; numbers from search snippets), secondary: https://centuryofbio.com/p/enveda-biosciences ; Enveda PRISM blog https://enveda.com/prism-a-foundation-model-for-lifes-chemistry/ ; Enveda GitHub org https://github.com/enveda ; Enveda review https://pmc.ncbi.nlm.nih.gov/articles/PMC13373924/ . Note: the brief's bioRxiv 2023.07.06.547963 is Mass2SMILES (volvox292), not MS2Mol.
- MassSpecGym: Bushuiev et al., NeurIPS 2024, https://arxiv.org/html/2410.23326v2
- "MassSpecGym in the Wild": Liu, Coley, Pluskal et al., https://arxiv.org/abs/2606.19624 and https://arxiv.org/html/2606.19624
- DiffMS: Bohde et al., ICML 2025, https://arxiv.org/html/2502.09571v2 ; README https://raw.githubusercontent.com/coleygroup/DiffMS/master/README.md ; weights https://zenodo.org/records/15122968
- MADGEN: Wang, Chen, Liu, Hassoun, ICLR 2025, https://arxiv.org/html/2501.01950v4 ; README https://raw.githubusercontent.com/HassounLab/MADGEN/main/README.md ; https://zenodo.org/records/17115068
- OMG: Martin & Hassoun, Anal Chem 2025, doi:10.1021/acs.analchem.5c01770 (abstract via Europe PMC REST, PMID 40955540)
- MS-BART: Han et al., NeurIPS 2025, https://arxiv.org/html/2510.20615 ; README https://raw.githubusercontent.com/OpenDFM/MS-BART/main/README.md
- TTT-LM: Mismetti, Alberts, Krause, Graziani, https://arxiv.org/html/2510.23746v2
- MBGen: Sun et al., https://arxiv.org/html/2602.01643 ; README https://raw.githubusercontent.com/biomed-AI/MBGen/main/README.md
- FlowMS: Nie & Gao, https://arxiv.org/abs/2603.18397
- FRIGID: Bohde et al., https://arxiv.org/html/2604.16648v1 ; README https://raw.githubusercontent.com/coleygroup/FRIGID/main/README.md ; https://zenodo.org/records/19685145
- DualLGD: Che, Du, Xu, https://arxiv.org/html/2605.07048 ; README https://raw.githubusercontent.com/du-lab-data-science/DualLGD/main/README.md ; https://zenodo.org/records/20281201
- MARLIN: Che, Du, Xu, https://arxiv.org/html/2607.04774v1
- MS-GPT: Zhao et al., https://arxiv.org/html/2607.23607v1 ; README https://raw.githubusercontent.com/VIKI623/MS-GPT/main/README.md
- GLMR (generative re-ranking): Zhang et al., AAAI 2026, https://arxiv.org/abs/2511.06259
- MSAlign: Krzakala et al., https://arxiv.org/abs/2605.19752
- COSMIC: Hoffmann et al., Nat Biotech 2022, doi:10.1038/s41587-021-01045-9 (https://pmc.ncbi.nlm.nih.gov/articles/PMC8926923/)
- Conformal retrieval: Rakhshaninejad et al., bioRxiv 10.64898/2026.03.12.711424 (abstract page)
- Challenge coverage: https://finance.yahoo.com/healthcare/articles/enveda-launching-next-alphafold-contest-120700676.html ; Kaggle write-ups (GitHub mirrors, Kaggle itself is JS-rendered): https://raw.githubusercontent.com/Isaackjoshua/casmi-2026/main/README.md ; https://raw.githubusercontent.com/corentinlaudicina/enveda-casmi-2026/main/README.md ; https://raw.githubusercontent.com/OfficialBhattacharya/kaggle_CASMI_2026/main/README.md
- Not reachable this session (noted, not cited for facts): nature.com full texts (redirect wall), ChemRxiv/ResearchGate PDFs (403), PubMed (cookie wall), C&EN article (406). Web-search budget was exhausted after ~15 searches; remaining facts came from direct fetches.