Borrowed ideas

TL;DR (5 bullets a chief of staff would read)

• Five lenses (cryptanalysis, forensics, art attribution, MT, diagnosis/astronomy/ecology) converge on three moves: score candidates in additive log-odds and stack calibrated channels (never average ranks/cosines); train with candidate-set softmax cross-entropy over the real ±ppm pool (published 19.5 % → 47.5 % R@1 vs in-batch InfoNCE); treat pool coverage as the ceiling (~45 % of hidden molecules never reach top-25) and build an explicit truth-absent branch.
• Most dangerous import: the popularity prior. MetFrag's headline gain (top-1 6 % → 89 %) comes from literature reference counts — inverted on a test set defined as "never previously published". Rankers trained on GNPS/MassBank learn it implicitly; audit for it.
• Morelli test (minor peaks beat the base peak for isomers): every optimised intensity exponent since 1994 is 0.25–0.6 and Li 2021 states the rationale — but no ablation removes the base peak, and Li's rule switches OFF above spectral entropy 3 (ion-rich spectra, which merged multi-CE timsTOF spectra may be). We must run it.
• De novo SMILES is 0.00 % exact on MassSpecGym; use only as MBR-style consensus filler behind DB candidates, routed by an A/B/C triage with a measured truth-absent detector.
• Calibration is solved in principle: COSMIC-style Platt classifier on description-matched fillers with n_f as input; only first-pass confidence is diagnostic; the public LB (228 teams in a 0.01 band) is not.

New concrete facts (file; source count; confidence)

1. Candidate-set softmax CE ≫ in-batch InfoNCE: MSAlign 2026, same encoder, 47.5 % vs 19.5 % R@1; fusion weights tuned on dev MRR → 58.2 %. Hard negatives give no gain (De Waele 11.64 vs 12.29; JESTR 0 on MSG). No paper optimises MRR directly. — linguistics, 24 sources, medium (unaudited preprints).
2. De novo not competitive: all SMILES/SELFIES decoders 0.00 % exact on MSG structure-disjoint split; SMILES transformer MCES 53.8 vs random 28.6; best with formula given MS-GPT 23.91/28.65, DualLGD 23.89/33.08 (top-1/top-10); DiffMS 2.30 → 1.86 % when MIST-CF supplies the formula. — linguistics, high/medium.
3. Fragmentation grammar: SIRIUS tree = max-weight colourful subtree, MAP-scored with ~40 common-loss priors + log-normal loss-mass prior; Böcker & Dührkop 2016 formula top-1 31.1 → 76.0 %. Breaks at H-rearrangements (ICEBERG ±6 H); parse is per formula, not per isomer. — linguistics, high.
4. Open-set in eDNA (Murali 2018): true genus deleted → forced wrong assignment IDTAXA 0.01 %, RDP 3.83 %, SINTAX 8.76 %, BLAST 17.3 %, SPINGO 26.7 %; spurious genera BLAST 65 vs IDTAXA 2; coverage price 22.5 % → 19.9 %. — healthcare, 32 sources, high.
5. Priors: MetFrag relaunched (473 spectra) top-1 30 → 336 (PubChem refs) → 420 (ChemSpider refs); biological-source prior lifts rank-1 NP annotation 10.2 % → ~72 %. — healthcare, high; inversion on hidden set is inference, medium.
6. Transferable tricks: CSLS hubness +2.1–6.5 P@1 (Conneau 2018), never tried on spectra; energy score FPR95 51.04 → 33.01 % (Liu 2020); BirdCLEF 2024 "nocall" class dropped at inference, private 0.655 → 0.671; OpenMax F 0.595 vs 0.568. — healthcare, numbers high, transfer low–medium.
7. Intensity weighting: Stein & Scott 1994 rank-1 75 % with sqrt; Li 2021 entropy similarity best of 43 (AUC 0.958), FDR < 10 % at 0.75 vs dot product ≥ 15 % at 0.95, rule I^w, w = 0.25+0.25·S for S < 3, w = 1 above; Hart 2024 I^0.25·freq^0.25 lifts networking F1 0.094 → 0.14 but library P@1 only marginally; some weightings fall below unweighted cosine. No ablation removes the base peak. — art + healthcare, high.
8. Posterior / pool size: P(H_top|S) = π_in·w·LR_top / [π_in·Σ w_j LR_j + (1−π_in)]; ordering invariant to pool size (Balding & Donnelly 1996), but PubChem multiplies n_f by 10²–10³ (MetFrag Top-1 105 ChemSpider vs 30 PubChem); break-even needs > 50–70 % relative coverage gain. — forensics, medium.
9. Calibration: COSMIC AUC 0.82 vs 0.40–0.55 for raw scores; 57/123 correct at 5 % FDR (CASMI 2016); passatutto +139 % annotations at 1 % FDR; Wixted & Wells 2017: only pristine first-pass confidence is reliable. — forensics, high.
10. Unicity: ±10 ppm COCONUT window median 36, p90 204 isobars → 5–8 bits needed; per-peak bits (Ma 2014): H2O 0.57, NH3 2.3, hexose 3.9, H3PO4 4.4, pool-relative; SIRIUS loss ratios H2O 64.9, CH3 69.5, CO 23.9. — cryptanalysis, medium.
11. Slot arithmetic: at LB 0.417 with recall@25 ≈ 0.55, mean 1/rank among hits ≈ 0.76; slots 1–5 carry ≥ 93 % of EV (q4). Slots 6–25 are for diversity, not score depth. — cryptanalysis + channel, medium.

Forced analogies — what the lens produced (duds too)

• Morelli → partial pay-off: field already flattens intensities for Morelli's reason; the ablation (base peak removed / peaks < 10 % only / sqrt / log / entropy-adaptive; isomer-swap rate as readout; < 1 GPU-day) is ours to run. Boundary: Li says it reverses above entropy 3.
• Mosteller–Wallace / Burrows Delta → partial: common neutral losses are function words; SIRIUS loss priors are a rate model; nobody z-scores a per-candidate loss profile against the corpus norm. One orthogonal reranker feature; keep only if CI excludes 0. Breaks: losses are compositional (H2O count ≈ OH count) and tree-dependent.
• IDF vs Hart contradiction → resolved: Hart is Burrows' logic (frequent tokens judged against their norm); IDF survives only as candidate-conditional IDF (weight a peak by how few window candidates explain it).
• Rembrandt Research Project A/B/C → paid off as design: hard evidence (mass, formula, adduct consistency across 1–16 spectra) vs holistic similarity; six-feature triage, quantile thresholds (~30 % C, ~25 % B, then tuned), negative control that routing must not lower MRR on the truth-present half. Breaks: the RRP could stop at "C"; we cannot — C is a routing decision. Committee consensus "washing away doubts" = naive-Bayes over correlated channels.
• Known-plaintext cribs → paid off: ~40–150 confident library hits in the hidden set estimate the shared timsTOF "day key" (ppm offset, CE response, score temperature). Caveat: cribs are library-like, easy-tail overfit.
• Enigma's flaw (deterministic elimination) → dud: MS-FINDER rules explain only 78–85 % of fragment ions; hard structural filters discard 15–22 % of true answers. Eliminate only on mass/adduct/formula.
• Sequential line-ups → cascade → dud (q5, q7 concur): collapses under ROC; use one log-odds stacker.
• Beam-search curse → untested in MS; moot since de novo is filler.

Contradictions within my subdomain

• IDF (rare peaks) vs Hart (common m/z helps): reconciled as pool-relative, unmeasured.
• Morelli vs Li's entropy cut-off: which regime merged timsTOF spectra occupy is unknown.
• Context priors: biological-source prior helps hugely, popularity prior is inverted; NP-likeness is useless on an all-NP pool (apply to generated candidates only).
• Calibrate on description-matched fillers (forensics) vs hard training negatives give no gain (MT): calibration ≠ training, but easy to conflate.
• eDNA rewards abstention; MRR@25 never does — a detector without a generative branch is strictly harmful.

Gotchas for the modeler

1. ms2_normalized_intensities makes the base peak always 1.0; noise floors differ by ingest_lib, so minor-peak features can leak library of origin.
2. Implicit popularity prior via training-spectrum counts; audit score-vs-train-count correlation on hold-out.
3. Spectrum-blind baseline (q5): PubChem default order = 49.98 % hit@1 on MSG; report it beside every local score.
4. Score gap to runner-up measures pool weakness unless normalised by n_f.
5. LR 10³ with n_f 10⁴ → posterior 0.09; never threshold raw scores.
6. Confidence after test-time adaptation or LB steering is contaminated; freeze first-pass scores for final-submission choice.
7. Tautomer canonicalisation is a no-op for DB SMILES but 180–360 ms and maxTransforms-capped on polyphenols (q7); de-novo tautomers are the exposure.
8. Validity ≠ correctness: SELFIES decoders 100 % valid, 0 % exact. Never rank seq2seq output above DB candidates without a forward-model check.
9. Paired bootstrap over molecules; 400 hidden molecules → 0.01 MRR ≈ noise.
10. Open-set detectors degrade under shift; keep top pool candidates even in tier C.

Questions for other queens

• q1_instrument_physics: noise floor and typical spectral entropy of merged multi-CE timsTOF spectra (decides Morelli regime); ppm offset and CE-response spread within one campaign.
• q2_data_forensics / q6_candidate_space_retrieval: InChIKey14 collapse of COCONUT 2.0; per-library intensity floors; n_f distribution for COCONUT vs PubChem on NP formulas.
• q7_compute_execution: will you own the truth-absent hold-out (~1,000 scaffold-disjoint NPs, truth deleted from pool for a random half)?
• q5_methods_lineage: can any fragmenter emit per-candidate neutral-loss profiles cheaply enough for a Delta feature (~100 candidates × 1,500 spectra)?
• q4_challenge_intelligence: is the hidden set's COCONUT coverage nearer enveda-np-examples (99.6 %) or enveda-180 (0.03 %)? That number sets π_in.

Layman sentences

1. A mass spectrum is like a painting with a famous signature everyone copies and tiny brushstrokes nobody bothers to — the tiny strokes tell near-identical molecules apart, so stop staring at the biggest peak.
2. Almost half the answers are not in our catalogue at all, so the biggest win is knowing when to say "none of these" and look elsewhere — ecologists already measure exactly that.
3. Every reliable identification system, from DNA databases to supernova classifiers, adds up calibrated log-odds and distrusts confidence formed after peeking at the answer key.

Gaps I could not fill

• No published ablation removing the base peak; no measured bits-per-peak among COCONUT isomers on timsTOF; no CSLS/energy-score result on spectrum→structure.
• Not opened (cited via secondary sources, flagged in-file): MS2Mol ChemRxiv, SIRIUS 4 Nat Methods, Mosteller & Wallace 1963, Stein & Scott 1994 full text, SINTAX preprint, NRC 1996.
• Isabel "96 % top-10" is a vendor claim; SIRIUS 4 39.4 %/74.0 % came via a search summary (low confidence).
• MSAlign / GLMR / SpecBridge are unaudited preprints; the June-2026 MSG audit (arXiv 2606.19624) shows how such numbers collapse.
• No published Tanimoto-expected-utility MBR; the "smoothed reciprocal-rank at 25" fine-tune is untested anywhere.