In-silico fragmenters and forward spectrum predictors as candidate re-rankers (MetFrag, MS-FINDER, CFM-ID, MAGMa, ICEBERG, FraGNNet, SCARF, MassFormer, 3DMolMS, GrAFF-MS, GLACIER)

TL;DR. Bond-breaking fragmenters (MetFrag, MAGMa, MS-FINDER) cost ~0.02–0.1 CPU-s per candidate and alone put the true structure first only 10–20 % of the time on ~800-candidate pools, but they are complementary: in CASMI 2016 an equal-weight six-method consensus beat every single entry (Top-10 166/208 vs 131/208), and on the 87 challenges whose answers were in nobody's training set, fragmenters matched the fingerprint ML methods. Learned forward predictors (ICEBERG 2, GLACIER, FraGNNet) now beat MetFrag 3–5× in Top-1 on NIST20 isomer-decoy retrieval at 0.003–0.4 s per molecule on GPU and are the right "confirmatory test" for the retrieval top-50. CFM-ID 4 (5–11 CPU-s per molecule) cannot cover 400 k pairs in a 9-hour Kaggle notebook; it must be precomputed off-platform and shipped as a dataset.

1. Tool table

Costs are single-CPU-core seconds per molecule (one predicted spectrum) unless marked GPU. "Unverified" = no primary source opened states it.

2. Feasibility: 400 molecules × 1,000 candidates in a Kaggle notebook

Platform (confidence medium — Kaggle pages are JS shells): a search snippet of the competition's Code Requirements says "CPU Notebook runtime is limited to 9 hours, GPU Notebook runtime is limited to 9 hours", internet disabled, output submission.csv, public external data and pretrained models allowed. GitHub write-ups (mxm-1x/Enveda-CASMI-2026, kabir0774/Enveda-Casmi-2026) confirm internet is disabled and weights must ship as Kaggle datasets. Generic Kaggle docs (snippet): 12 h interactive sessions, 30 GPU-h/week, 16 GB GPU memory.

Budget: 9 h = 32,400 s; assume 4 vCPU (unverified) ≈ 130 k core-s, or one T4-class GPU. The expensive count is ≈ 400 × 1,000 = 400 k predictions, one per candidate (per CE if CE-conditioned); matching against the 1–16 spectra per molecule_id is negligible.

Escape hatch: precompute. COCONUT-scale (~0.7 M structures) CFM-ID 4 at 5 s = 3.5 M core-s ≈ 40 core-days — ~5 h on 200 Explorer cores — then ship ≤ 30 peaks × 3 energies per molecule. GrAFF-MS estimated ChEMBL (2.26 M structures < 1,000 Da) at 2 h on one GPU vs 4 days on 64 CFM-ID instances. The ICEBERG atlas already serves PubChem-wide predictions; check COCONUT InChIKey coverage first.

3. Does fragmenter re-ranking on top of fingerprint retrieval help?

CASMI 2016 (Schymanski 2017; 208 challenges = 127 pos + 81 neg; Q Exactive Plus; ChemSpider candidates ±5 ppm, mean 816; stereoisomers collapsed by InChIKey block 1). Table 3/5 of the paper:

Readings: the consensus added only +4 Top-1 over the best single entry but +35 Top-10 (166 vs 131) — the tail MRR@25 still pays for. MetFrag+CFM lifted MetFrag's Top-1 24→41; in negative mode 9 (MetFrag) and 12 (CFM) → 20. A crude MRR proxy from the Top-1/3/10 bins (weights 1, 0.417, 0.2; ranks > 10 ignored; my derivation): MetFrag 0.23, CFM 0.31, MS-FINDER 0.31, MetFrag+CFM 0.33, CSI:IOKR_AR 0.45, Combined 0.52 — consensus ≈ +0.07 over the best single ML ranker on ~800-candidate pools (lower bound).

Novel molecules (the condition that matches our hidden test). Table 7 — the 87 challenges whose answer was in nobody's training set: CSI:IOKR_AR 16 Top-1, CSI:IOKR_A 15, CFM_retrain 15, CSI:FID_leaveout 13, MetFrag+CFM 13, MS-FINDER 10, MetFrag 5. The fingerprint lead evaporates once memorised compounds are excluded (CSI:FID 70 → 36). Ruttkies 2019 then reached 21 Top-1 / 55 Top-10 / median 5 on those 87 spectra with MetFrag 2.4.5's Bayesian annotation terms vs CSI:IOKR 16/46/10 and CSI:FID 13/32/3 — post hoc, tuned on 5,622 MoNA/GNPS spectra with ≤ 500 random candidates per spectrum during optimisation.

CASMI 2017 (Jena blog snippet; page 503 on fetch): CSI:FingerID won categories 1, 2, 4; category 4: 66/198 (33.3 %), "more than 6-fold of … the best non-CSI:FingerID contestant" — the opposite picture, on a benchmark without isotope patterns.

Forward predictors vs MetFrag on identical candidate sets. NIST20 [M+H]+, 49 Tanimoto-nearest PubChem decoys, Top-1 random / scaffold split (GLACIER Table 18): MetFrag 0.107 / 0.127; GrAFF-MS 0.211 / 0.143; FraGNNet 0.238 / —; MassFormer 0.252 / 0.184; ICEBERG 1.0 0.251 / 0.229; ICEBERG 2.0 0.400 / 0.335; GLACIER 0.533 / 0.452; Top-10 MetFrag 0.567 vs ICEBERG 2.0 0.923. FraGNNet Table 2: FraGNNet-D4 36.0 / 30.3, ICEBERG 36.0 / 27.9, MassFormer 24.2 / 20.4. SCARF: 18.7 % on NIST20 but 13.5 % on NPLIB1, where a fingerprint-FFN predictor retrieved better despite worse cosine. CASMI 2022 positive mode, PubChem isomers (mean 1,784 per formula, formula assumed known; ICEBERG Table S1): ICEBERG 12.9 %, NEIMS-FFN 8.6 %, MassFormer 7.6 %, FixedVocab 7.3 %, random 1.3 %; CFM-ID's contest entry 26/303 = 8.6 %. ICEBERG's cosine fell 0.727 (NIST20) → 0.417 on those natural products.

MassSpecGym (arXiv 2410.23326 v3; 231 k spectra, 29 k molecules, 98 % Orbitrap/QTOF, [M+H]+/[M+Na]+, MCES ≥ 10 split, ≤ 256 candidates): retrieval Hit@1/@5/@20 — Random 0.37/2.01/8.22; Fingerprint FFN 2.54/7.59/20.00; DeepSets+Fourier 5.24/12.58/28.21; MIST 14.64/34.87/59.15 (mass-based), MIST 9.57/22.11/41.12 (formula-based: isomer-only decoys are harder). Simulation challenge (subset with CE + instrument, same 256-candidate protocol): FraGNNet cosine 0.52, Hit@1 46.64 / @5 72.56 / @20 83.58 (mass) and 31.93/63.20/82.70 (formula) vs GNN 8.44/21.43/38.57. Subsets differ, so 46.6 % vs 14.6 % is suggestive, not controlled; GLACIER reports 69.95 % (mass) / 49.88 % (formula) with contrastive fine-tuning. A "MIST + ICEBERG" ensemble number does not exist: ICEBERG's protocol assumes a MIST/CSI pre-filter to 50 isomers, i.e. sequential testing is baked into the evaluation, not measured as a gain.

4. Where fragmenters systematically fail

• Rearrangements. CASMI 2016: "combinatorial fragmenters performed poorly for Challenge 41 (monobenzyl phthalate), where the main peak is a well-known rearrangement". MetFrag has 5 neutral-loss rules, MS-FINDER 9 H-rearrangement rules; MAGMa's CASMI 2013 post-mortem lists "chemical rearrangements not represented by substructures" first; SCARF's motivation: rearrangements "cannot be approximated by bond breaking alone"; ICEBERG 2 covers ±6 H shifts and concedes "revisions to the architecture might yield a more general solution".
• Unexplained peaks. GrAFF-MS: CFM-ID "fails to assign a formula annotation within 10 ppm to 42 % of monoisotopic peaks in the NIST-20 test set". Coverage of true-spectrum peaks by the prediction for the true molecule (ICEBERG Table S2/S3): CFM-ID 0.278, ICEBERG 0.754, SCARF 0.807, MassFormer 0.790 on NIST20; 0.235 / 0.549 / 0.552 / 0.573 on NPLIB1. FraGNNet: residual peaks "may arise from complex chemistry, such as cyclizations" beyond its depth-4 recursion.
• Ring systems / NP scaffolds. Cosine drops NIST20 → NPLIB1 for all (ICEBERG 0.727 → 0.627, MassFormer 0.721 → 0.568); MAGMa needed 5 bond breaks (default 3) for fused-ring ofloxacin; MetFrag 2.2 re-engineered fragment storage "to generate reliable fragments for molecules with complex ring structures" at higher tree depths.
• Glycosides. CFM-ID 4.0 added hand-written rules for flavonoid glycosides (and lipids, acylcarnitines, acylcholines, flavonols, flavones, flavanones) because the learned model under-performed; MassSpecGym finds "carbohydrates and carbohydrate conjugates" the class with the highest cross-method variability.
• Stereo/positional isomers. "None of the current approaches are able to distinguish stereoisomers (even cis/trans)"; formula-only predictors (GrAFF-MS, SCARF, FraGNNet) are blind to positional isomers by construction; structure-based ones separate them only through bond-energy-driven intensity differences.
• Pool size. MetFrag 2010 on the same 102 compounds: median rank 8 with 338 candidates vs 31.5 with 2,508. MAGMa+ "decreases with increasing candidate numbers (… 1116 on average)". The worst CASMI 2016 challenge cluster had 1,603 mean candidates vs 816 overall. COSMIC: "the number of candidates in the structure database had a strong effect", mass none. CFM-ID 4 web server: Top-1 48.1 % on HMDB (~250 k) vs 7.0 % on PubChem (~2.1 M). This is the base-rate effect of §6 and matches the challenge lesson that PubChem-scale pools hurt.
• Calibration. MetFrag raw scores "depend on both the query spectra and the compound classes of the candidates" (LipidFrag) — hence per-class foreground/background models.

5. Merged multi-CE spectra and QTOF/timsTOF CE

• No tool has timsTOF-specific CE scaling. MassFormer: "FT instruments with HCD" only (NCE); 3DMolMS: numeric CE with QTOF and Orbitrap checkpoints; CFM-ID 4: QTOF 10/20/40 eV; ICEBERG 2/GLACIER: absolute eV positional encoding; MassSpecGym converts NCE → eV (its Eq. 3) and otherwise assumes absolute. None model a Bruker ramp over m/z or mobility.
• Merged spectra are the native input of most models. ICEBERG 1 and SCARF "merge all scans at various collision energies, combine peaks within 1e-4 m/z, … square-root, … top 50 peaks > 0.003" and take no CE. FraGNNet trains on NIST20 merged across CEs with the CE set embedded as the mean of Fourier embeddings; its authors list "unmerged spectra … different instrument platforms and precursor adducts" as untested. ICEBERG 2 handles stepped CE by predicting per energy and summing intensities "if their m/z values are the same to the 4th decimal point".
• CFM-ID's three energies vs one merged spectrum. cfm-id scores "the Jaccard score averaged over collision energies" (documentation snippet; Anal Chem full text inaccessible — medium confidence); the NAR 2022 paper allows identification "at one or more collision energies". CASMI 2016 organisers noted challenge spectra "are merged spectra collected in the stepped mode, where CFM-ID is predicting … individual CID levels"; the practical fix is to sum/max-merge the 10/20/40 eV predictions into one spectrum, or match the merged spectrum to each energy and take max/mean.
• MetFrag/MAGMa/MS-FINDER are CE-agnostic (m/z-only matching), so merging costs nothing — but more peaks inflate accidental matches for every candidate (gotcha 3).

6. Forced analogy: medical differential diagnosis

Statement. Fingerprint retrieval is the cheap screening test over the whole differential (mass-window pool, prevalence 1/N); a fragmenter or forward predictor is the costly confirmatory test ordered on the shortlist. Clinicians combine tests on log-odds — posterior odds = prior odds × LR₁ × LR₂ — and order them by cost and specificity.

What already exists in this form. LipidFrag fits gamma densities to MetFrag scores for correct (foreground) and deliberately wrong (background, other lipid subclasses within ±150 ppm) candidates and outputs a posterior "foreground class probability" — a textbook LR; class AUCs 0.90–1.0; false positives 91 % → 57 % (positive) and 93 % → 27 % (negative) at FCP ≥ 0.6. MetFrag 2.4.5's Bayesian peak/loss terms are per-peak LRs. COSMIC's confidence SVM (CSI score, score gap to runner-up, candidate count, explained intensity) found the fragmentation-tree score "received weights close to zero" while candidate count was "highly important" — clinically: once the imaging is in, a second lab test adds little, but prevalence matters a lot.

The LR the brief asks for — P(MetFrag explains ≥ X % intensity | true) / P(… | decoy) — is not reported in any source I opened (MetFrag 2016/2019, CASMI 2016, LipidFrag, COSMIC, ICEBERG, FraGNNet, MassSpecGym; MS-FINDER's 78.4 %/84.8 % explained ions are for true structures only). Sensitivity-side proxies: ICEBERG coverage 0.75 (NIST20) / 0.55 (NPLIB1) for the true molecule; MassSpecGym discarded 135,190 spectra (926 structures) explaining < 50 % of sqrt-intensity even with the true formula's subformulae — the test's ceiling is data quality.

Testable protocol (one week; Explorer). (1) Hold out 2,000 molecules from enveda-180 and enveda-np-examples (timsTOF, NPs), unseen by the fingerprint model. (2) Pools: COCONUT + train structures within 10 ppm per adduct (~1,000 each). (3) MetFrag CL (depth 2, 5 ppm/0.001 Da) on all candidates → explained-intensity fraction E (sqrt-weighted) and FragmenterScore; ICEBERG 2.1 (MassSpecGym weights) on the fingerprint top-100 → entropy/cosine similarity S. (4) Fit foreground/background densities of E and S conditional on peak count and precursor-mass bin; tabulate LR(E ≥ x), LR(S ≥ s); rank by log retrieval posterior + log LR_E + log LR_S; compare MRR@25 against retrieval-only, rank-averaging and score-product; compare "confirm top-50" vs "confirm all" to quantify the recall cost of sequential testing (back-solved recall@25 ≈ 0.55 at LB 0.417 says this binds). (5) Expected effect from CASMI 2016: ≈ +0.07 MRR proxy over the best single ranker, concentrated at ranks 2–25; on novel molecules fragmenters were at parity with fingerprint ML, so the LR is informative exactly where our test lives.

Does it hold? Partially. Cost ordering and base-rate logic (pool size = prevalence) transfer cleanly. It breaks on conditional independence: a rearrangement-dominated spectrum fools the fingerprint model and every bond-breaker at once (Challenge 41), so LR products over-count; the CASMI consensus worked because the engines had different failure clusters (the paper's clustering put MS-FINDER in a cluster of its own). It also breaks on fixed specificity: a fragmenter's false-positive rate rises with peak count and pool size, hence the conditional densities in step 4.

7. Gotchas / what could fool us

1. Training leakage inflated every published fingerprint number (CSI:FingerID 70 → 36 Top-1 without its training compounds; MassSpecGym's MCES ≥ 10 split exists for this reason). Validate on held-out InChIKey14/scaffolds; the hidden test is "none previously published".
2. "MetFrag explains the peaks" is weak per se: CASMI 2013 Challenge 2 — three most intense peaks explained, answer ranked 44th of 12,702. Calibrate against decoys, per peak count and mass bin.
3. Merged spectra inflate random matches (more peaks → more accidental formula hits for every candidate). Fit densities on merged spectra, or score per CE and combine.
4. Pool size confounds every score (median 8 → 31.5 when 338 → 2,508 candidates). Always include a pool-size covariate.
5. Stereo/tautomer collapse changes ranks: CASMI kept "the highest scoring isomer"; tautomers can differ in InChIKey block 1. Canonicalise candidates (RDKit 2026.03.3 tautomers) before scoring so duplicates do not eat the 25 slots.
6. Instrument domain shift: public weights are Orbitrap-HCD (NIST20, MassFormer) or mixed QTOF (NPLIB1, CFM-ID); timsTOF eV ramps are unmodelled; CASMI 2022 NPs cut ICEBERG's cosine to 0.417 and all Top-1s below 13 %.
7. Cosine ≠ retrieval: SCARF found them "uncorrelated" on NPLIB1; FraGNNet's ranking-method choice moves Top-1 25.3 → 36.0 with the same model. Pick the similarity (entropy, Hungarian 10 ppm, sqrt-intensity, precursor in/out) by held-out MRR.
8. Formula assumed known in ICEBERG-CASMI22 and FraGNNet retrieval; mass-window pools contain other formulae that subformula coverage rejects trivially — do not credit the fragmenter for that.
9. Negative mode is the orphan: no CSI negative model in 2016 (816 vs 7,300 spectra); FraGNNet [M+H]+ only; MassSpecGym positive only; Che et al. 2026 (arXiv 2601.00941) find [M+H-2H2O]+ and [M-H+2Na]+ near zero for all methods. Gate per adduct.
10. Licence traps for an open-sourced winner: 3DMolMS CC BY-NC-SA; NIST-trained weights (MassFormer, FraGNNet, ICEBERG-NIST) non-redistributable; LGPL/MIT tools fine.
11. Timings are not cross-comparable: CFM-ID 4.9 s (GrAFF-MS) vs 11.1 s (ICEBERG, with model load) vs 8,979 s per candidate list (MetFrag 2016, CFM-ID 2.0); GrAFF-MS 2.8 ms (own batch-512) vs 57 ms (FraGNNet's sparse-batch reimplementation). Budget with 3× margin.
12. Intensity use differs: MS-FINDER ignores intensities in scoring; MetFrag/MAGMa use them; forward predictors expect sqrt-normalised top-50 peaks. Match ms2_normalized_intensities preprocessing to each tool.

Implications for the challenge

1. Run MetFrag CL (depth 2) on the full ~1,000-candidate pools (< 3 h CPU); use explained-intensity fraction and FragmenterScore as calibrated LR features, never as a standalone ranker (24/208 Top-1).
2. Use ICEBERG 2.1 (public MassSpecGym weights, MIT) as the confirmatory test on the retrieval top-50/100 (≈ 40 k predictions, ≤ 1 h GPU); switch to GLACIER if its weights load (8× faster). Precompute on Explorer for the union of pools once the pipeline is fixed and ship as a dataset.
3. Never run CFM-ID 4 inside the notebook; if wanted, precompute 10/20/40 eV spectra on the cluster (~40 core-days for 0.7 M structures) and sum the three energies before matching merged spectra.
4. Combine on log-odds with pool-size and peak-count covariates; compare rank-averaging, LR product and a learned stacker on a structure-disjoint timsTOF/NP hold-out; expect gains mostly at ranks 2–25.
5. Measure the true-vs-decoy explained-intensity densities on train.parquet (protocol §6) and publish the LR table to the hive — nobody has.
6. Check iceberg-ms.mit.edu for precomputed predictions covering COCONUT before spending GPU time.
7. Ensemble orthogonal failure modes: one bond-breaker (MetFrag), one learned DAG (ICEBERG 2), one formula-vocabulary model (SCARF/GrAFF-MS-style); two bond-breakers add little.
8. Candidate coverage stays first-order: no confirmatory test rescues a true structure outside the top-50; widen retrieval recall before deepening confirmation.
9. Gate by adduct: fragmenter/forward scores only where the adduct model exists ([M+H]+, [M-H]-, [M+Na]+); retrieval-only elsewhere.

Open questions

• Actual Kaggle hardware for this competition (vCPU count, GPU type); the 9 h limits came from a snippet.
• Does the public ICEBERG 2.1 MassSpecGym checkpoint cover negative-mode adducts, and what is its T4 throughput?
• Is there a Linux/mono-usable MS-FINDER console build?
• Has Enveda released GrAFF-MS code/weights anywhere? Otherwise FixedVocab reimplementation is the only route.
• CFM-ID 4's exact merged-spectrum scoring rule (documentation says Jaccard averaged over energies; paper unverified).
• The size of the explained-intensity LR on timsTOF NP data — the central measurement above.

Sources (opened)

• Ruttkies C et al. MetFrag relaunched. J Cheminform 2016;8:3. doi:10.1186/s13321-016-0115-9 (Europe PMC PMC4732001 full text: Table 1, runtime paragraph, Availability).
• Ruttkies C, Neumann S, Posch S. Improving MetFrag with statistical learning of fragment annotations. BMC Bioinformatics 2019;20:376. doi:10.1186/s12859-019-2954-7 (PMC6612146: Table 2).
• Schymanski EL et al. CASMI 2016: automated methods. J Cheminform 2017;9:22. doi:10.1186/s13321-017-0207-1 (PMC5368104: Tables 3, 4, 5, 7, text).
• Schymanski EL et al. Solving CASMI 2013 with MetFrag, MetFusion and MOLGEN-MS/MS. Mass Spectrom (Tokyo) 2014. PMC4321341.
• Ridder L et al. Automatic compound annotation using MAGMa (CASMI 2013). Mass Spectrom (Tokyo) 2014. PMC4321337.
• Tsugawa H et al. Hydrogen rearrangement rules … MS-FINDER. Anal Chem 2016;88:7946. doi:10.1021/acs.analchem.6b00770 (abstract via Europe PMC).
• Wang F et al. CFM-ID 4.0. Anal Chem 2021;93:11692. doi:10.1021/acs.analchem.1c01465 (abstract only; full text inaccessible). Wang F et al. CFM-ID 4.0 web server. Nucleic Acids Res 2022. doi:10.1093/nar/gkac383 (PMC9252813 full text). CFM-ID Bitbucket README; Docker Hub wishartlab/cfmid.
• Witting M et al. LipidFrag. PLoS ONE 2017;12:e0172311.
• Hoffmann MA et al. COSMIC. Nat Biotechnol 2022. doi:10.1038/s41587-021-01045-9 (PMC8926923).
• Goldman S, Li J, Coley CW. ICEBERG. Anal Chem 2024;96:3419. doi:10.1021/acs.analchem.3c04654; arXiv 2304.13136v2 (Tables 1, S1–S3).
• Goldman S et al. SCARF. NeurIPS 2023; arXiv 2303.06470 (Table 2, A1–A3).
• Wang R et al. Neural spectral prediction for structure elucidation (ICEBERG 2). bioRxiv 2025, doi:10.1101/2025.05.28.656653 (PMC12154671; partial).
• Wang, Wang, Coley. GLACIER. arXiv 2606.29161v1 (Tables 18–19, throughput). coleygroup/ms-pred README + LICENSE (MIT).
• Young A et al. FraGNNet. TMLR 08/2025; arXiv 2404.02360 (Tables 1, 2, 9, 10, 12; §A.5.4, A.19). GitHub FraGNNet/fragnnet (BSD-2).
• Young A, Wang B, Röst H. MassFormer. arXiv 2111.04824; GitHub Roestlab/massformer (BSD-2; NIST weights withheld).
• Hong Y et al. 3DMolMS. Bioinformatics 2023;39:btad354; GitHub JosieHong/3DMolMS (CC BY-NC-SA 4.0).
• Murphy M et al. GrAFF-MS. arXiv 2301.11419 (§5.1.1, §6.3).
• Bushuiev R et al. MassSpecGym. arXiv 2410.23326v3 (Tables 3, 4; data cleaning; App. 3.4); pluskal-lab/MassSpecGym README.
• Che X, Du X, Xu D. arXiv 2601.00941 (2026).
• MetFrag site ipb-halle.github.io/MetFrag (LGPL 2.1+; CL parameters); GitHub ipb-halle/MetFragRelaunched README (Java 21); GitHub API NLeSC/MAGMa (Apache-2.0); systemsomicslab/MsdialWorkbench (LGPL v3 / CC-BY 4.0).
• Kaggle code requirements (9 h CPU/GPU, internet off, submission.csv): search snippet of the Kaggle page; corroborated by GitHub write-ups mxm-1x/Enveda-CASMI-2026 and kabir0774/Enveda-Casmi-2026. CASMI 2017: bio.informatik.uni-jena.de post (snippet; page 503).