How this was researched
A hive of agents. An emperor decides the shape of the hive — how many kings, which queens, who may talk to whom — to keep every agent's working memory small. Kings own content and force every queen across domain boundaries. Queens own a subdomain and send workers to research one topic each and write it to disk. Nobody reads more than the layer below them has distilled.
Current structure (v2, 2026-10-07)
See hive/EMPEROR.md tick 0. Two kings split the problem at its natural seam: how a molecule becomes a spectrum (MEASURE) vs how a spectrum becomes a molecule (INFER). Eight queens, five workers each, keeps every king's context under ~5 queen digests and every queen's under ~5 worker TL;DRs. v2 (tick 1): added coverage_truth and validation_ownership channels; q7 owns the truth-absent hold-out; merges/splits/kills for tick 2 recorded in hive/EMPEROR.md tick 1.
Everything about the molecules, the instrument, the data, and the challenge as an institution. Owns the 'what are we actually measuring' question.
- How a timsTOF LC-MS/MS measurement physically works and where it lies
- The train/test parquet files, the 11 source libraries, the metric, leakage, and domain shift
- The chemistry of natural products and analogs: classes, biosynthesis, chemical space, class-specific fragmentation
- The competition as an institution: rules, history, organizers, leaderboard, community, prior CASMI lessons
Everything about methods, candidate spaces, compute, execution, and deliberate cross-domain serendipity. Owns the 'how do we win' question.
- Every published spectrum-to-structure method, what it does, where it fails
- Where the answers live: candidate databases, formula assignment, library and analog search, ranking
- candidate databases pubchem coconut lotus npatlas chembl coverage math
- formula assignment sirius buddy mistcf isotopes timsTOF
- library search modified cosine entropy ms2deepscore analog search
- candidate ranking features learning to rank ensembles
- coverage ceiling analysis recall at 25 and how to raise it
- How we actually run this: compute, pipelines, validation design, submission mechanics, timelines
- Deliberate cross-domain analogy: import ideas from unrelated fields and test them against this problem
Who may talk to whom
| Channel | Between | Purpose |
|---|---|---|
| data_x_candidates | q2_data_forensics ↔ q6_candidate_space_retrieval | what the hidden test looks like vs where candidates come from |
| physics_x_methods | q1_instrument_physics ↔ q5_methods_lineage | which physical effects methods model or ignore |
| rules_x_execution | q4_challenge_intelligence ↔ q7_compute_execution | rules and constraints that change the plan |
| chemistry_x_candidates | q3_natural_product_chemistry ↔ q6_candidate_space_retrieval | which chemical classes are under-covered by which database |
| serendipity_broadcast | q8_serendipity ↔ * | q8 broadcasts analogies; any queen may reply with a test or a rebuttal |
| coverage_truth | q2_data_forensics ↔ q6_candidate_space_retrieval ↔ q8_serendipity | settle pi_in: enveda-np-examples and hidden-set coverage of COCONUT under the metric's RDKit 2026.03.3 pipeline |
| validation_ownership | q2_data_forensics ↔ q4_challenge_intelligence ↔ q7_compute_execution | q7 owns the truth-absent hold-out and the 250 np-examples three-mask backtest; q2 ships fold specs; q4 ships LB noise floor |
King Measure's log
king: king_measure tick: 1 date: 2026-10-07 queens: [q1_instrument_physics, q2_data_forensics, q3_natural_product_chemistry, q4_challenge_intelligence] workers_completed: 20/20 stagnation_redispatches: 0
King MEASURE — tick 1 log
Tick history. First dispatch was killed mid-flight by a session rate limit (each queen 3/5 workers, no digest). Re-dispatched all four in resume mode (two missing workers each, then digest + channels). All 20 worker files exist with Sources sections; all four digests written; the Emperor's resource throttle was relayed to the two queens still running. Lenses used: q1 astronomy/stellar spectroscopy, q2 forensic chain of custody, q3 music theory/timbre, q4 law of evidence; "epidemiology of measurement bias" held in reserve for a stagnation re-dispatch that was not needed (every queen returned ≥ 11 new concrete, sourced or locally computed facts).
Top 10 facts that change how we would model
- The hidden-test instrument uses fixed 20/40/60 eV, and a "merged" spectrum is a raw-count SUM of the three scans (reconstructed on 595/600 quartets; effective weights ≈ 0.5/0.4/0.1; 24.6 % of test rows merged, 68.5 % of test molecules have one). Not a 1/K0 ramp, not an average. Train-side Orbitrap NCE was converted with eV = NCE·m/z/500, but on 2,421 same-molecule pairs a timsTOF 20 eV scan matches NCE 20 (cosine 0.82), 40↔45, 60↔60 — the parquet's eV over-states NCE ~1.4×. →
research/q1_instrument_physics/cid_fragmentation_physics_collision_energy.md; corroborated byresearch/q2_data_forensics/adducts_polarity_collision_energy_merging_statistics.md. - timsTOF peak lists are raw integer centroids (floor exactly 9 counts, no cap): 92 % of peaks are < 1 % of base peak and peak count tracks sample abundance (Spearman 0.71). matchms default
intensity_from=0.001deletes 75 % of a timsTOF spectrum; MassSpecGym's ">300 peaks" rule discards ~25 % of enveda-180. →research/q1_instrument_physics/noise_artifacts_isotopes_insource_chimeric.md. - Isolation-window artefacts absent from the public libraries: precursor present in 46 % of timsTOF spectra, M+1 in 39 % (GNPS-Orbitrap 6 %), M+2 in 22 %, 13C satellites on 71 % of strong fragments (vs 14 %); fragments below m/z 150 read +4 to +6.5 ppm high, so a symmetric ±5 ppm sub-formula window rejects about half of true C5–C9 fragments. A numeric 5-step "telluric mask" is specified. → same file.
- The shipped
precursor_error_ppmcolumn is an artefact (absolute value, electron mass omitted, computed before a formate relabel; 52,460 rows ~2,400 ppm). Seven of 11 libraries ship a computed, not measured, precursor m/z. Recomputed timsTOF error: enveda-180 truncated at |5.13| ppm (99.98 % within ±5) with a +1.9 ppm positive-mode bias; np-examples up to 21 ppm in negative mode. →research/q2_data_forensics/adducts_polarity_collision_energy_merging_statistics.md. - The visible test is a random 400-structure draw from enveda-180 with two row rules (0/664 dimer rows kept; monomer rows thinned to 66.7 %, positive-biased). The "245–460 m/z window" is the Enamine HLL-200 mass band (99.67 % of enveda-180 monomers), not an organiser filter; 42 % of enveda-np-examples lie outside it. Decoy chemistry (0/400 in COCONUT, 36 % halogenated) says nothing about the hidden set. →
research/q2_data_forensics/test_set_decoy_hidden_set_inference_and_domain_shift.md. - COCONUT Oct-2026 = 738,794 records but only 479,690 distinct metric keys; in the test window a typical NP faces a median 35 same-formula skeletons (58 % face ≥ 25). Positional-isomer ranking — not formula, not stereo — is the recall@25 ceiling. Train ∩ COCONUT = 26,111 IK14 (9.47 %); enveda-180 contributes 51; np-examples 249/250. →
research/q3_natural_product_chemistry/np_chemical_space_coconut_lotus_npatlas_sizes_overlap.md,research/q3_natural_product_chemistry/beyond_2d_tautomers_stereo_isomers_what_the_metric_cannot_see.md,research/q2_data_forensics/_coconut_overlap_and_np_examples_keys.json. - The scorer is fully known:
MolFromSmiles → TautomerEnumerator().Canonicalize → MolToInchiKey[:14], RDKit 2026.03.3 pinned via wheel dataset, no salt stripping; >25 tokens/NaN/duplicate ids reject the whole file; invalid or duplicate SMILES silently burn slots. Canonicalisation changes 1.61 % of train keys, merges 1,541 train label groups (58,954 spectra), splits 11 skeletons, changes 12–17 % of NP keys and is non-idempotent for 0.3 %. →research/q4_challenge_intelligence/casmi2026_rules_timeline_prizes_code_requirements_faq.md,research/q2_data_forensics/metric_mrr25_inchikey14_tautomer_canonicalisation_rdkit.md, q3beyond_2d_*. - Public LB (Kaggle API pull 2026-10-07 13:59 UTC): 2,703 teams; rank 1/10/100/1000 = 0.481/0.441/0.425/0.357; median 0.328; 885 teams (32.7 %) sit on 16 exact clone scores; derived medal lines gold 0.436 / silver 0.425 / bronze 0.421; SE ≈ 0.04 on the ~130-molecule public subset, so Δ < 0.01 is noise. A fixed 0.417 decays ~30 ranks/day. →
research/q4_challenge_intelligence/public_leaderboard_and_open_writeups_oct2026.md(+lb_snapshot_2026-10-07.csv). - The only local validation that predicted public-LB order (three independent teams) is the 250 enveda-np-examples structures under three masks; random held-out train structures predicted the opposite ranker order; one write-up went 0.456 local → 0.112 LB on a decoy-shaped fold. np-examples structures all also exist in other libraries on other instruments — a leak unless removed by key across all libraries. → q4
public_leaderboard_*, q2test_set_decoy_*. - NP-class "timbre" is real but partial: measured on 142,728 stratified train spectra, hexose loss 162.0528 LR+ 46, deoxyhexose 40, pentose 22, glucuronide ~270, tropane 124.1121 LR+ 42, desosamine 158.1176 LR+ 369; terpenoid water/C5H8 ladders LR+ ≤ 2.6. Absence is never evidence (86 % of hexoside [M+H]+ spectra show no 162 loss; sensitivity falls 0.33 → 0.02 with precursor mass). And train's entire timsTOF stratum contains 27 (+) / 21 (−) O-hexoside spectra and zero strict oleananes — NP class behaviour on timsTOF is met for the first time in the hidden set. →
research/q3_natural_product_chemistry/class_specific_fragmentation_rules_glycosides_terpenes_alkaloids_polyketides.md(+ 55-row .json, 19 SMARTS rules).
Also: enveda-180 = Enamine HLL-200 on timsTOF Pro 2 (Zenodo 10.5281/zenodo.20436851; q2/q3/q4 agree); hidden 400 validated by Enveda with SIRIUS + NMR, organisers co-authored the MassSpecGym leakage audit (q4); 95.7 % of COCONUT keys have a one-step tailoring relative (q3).
Contradictions / disagreements between queens
- Mass window. q1 (
tims_qtof_mechanics_*): median precursor error 0.98 ppm, "5 vs 10 ppm LB-neutral (0.302/0.303)", mass accuracy is not where points are won. q2 (adducts_polarity_*): +1.9 ppm positive-mode bias on enveda-180, np-examples up to 21 ppm negative-mode → recommends asymmetric −4/+5 ppm with ±10 fallback. q4: every ≥0.37 public kernel uses ±10 ppm. Resolution proposed: ±10 ppm is safe for coverage; the asymmetric window is an untested pool-shrinking lever; q2's question to q1 on the cause of the +1.9 ppm offset is open (one hypothesis: electron-mass omission in Enveda's own m/z calculation — unverified). - How many adducts in test: 7 vs 10. Brief and q2 say 7; q1 (
esi_ionization_*) and q4 (official texts) count 10 strings. Unresolved; q2 notes [M−H2O+H]+ appears in np-examples though not in the visible 7. - Hidden spectra count 1,500 vs 2,500 (q2 and q4 both carry it; press release vs Kaggle mirrors). Decides whether dimer adducts and q2's thinning operator apply to the hidden set.
- Precursor-peak handling. q1 recommends stripping everything above precursor − 1.0 Da (precursor inflates cosine for every right-mass candidate); q1's own
noise_*worker says 46 % of spectra carry peaks above the precursor and trims should be identical in train and test; q2 asks whether test trims at +2.12 Da (enveda-180) or +2.00 Da (np-examples). Both agree: trim identically; disagree on where. - Class-prediction accuracy. q3: CANOPUS "99.7 %" vs CASMI 2022 best class 69 % — q3 trusts 69 % (sparsity artefact); q4's history file cites the same CASMI 2022 figure. No conflict in conclusion, but the dashboard must not quote 99.7 %.
- Popularity priors. q4: +0.004…+0.011 in one lineage, monotonically negative in another; q3: COCONUT alkaloid NP-likeness 0.07 is a bulk-import artefact ("demote by NP-likeness, do not filter"). Resolution from both: priors only on the library-hit (known) class, gated.
- Within-queen, flagged by queens themselves: q1 M+1 leakage partial vs full (resolved: full inside 2 Th); q2 provenance worker's "drop |ppm| > 20" would delete 96 % of pluskal negative-mode formate rows (superseded: recompute ppm first); q3 chalcone/flavanone treated as tautomers by one worker, proven two keys by another (latter correct).
Cross-domain transfers (from outside chemistry/ML)
- Telluric / cosmic-ray mask (astronomy, q1). Treat precursor, M+1, M+2, 13C satellites and sub-20-count/sub-m/z-40 spikes as atmosphere, not signal; 5-step numeric recipe. Test: run the public FPNet/HGB pipeline on the 250 np-examples three-mask backtest with and without the mask; keep if MRR@25 rises by > 0.02 (above q4's SE) and the base-peak-tercile MRR curve flattens.
- Forward model × instrument response function (astronomy, q1). Fit R(fragment m/z, precursor m/z, CE) on the 2,421 dual-instrument pairs to map Orbitrap/CFM-ID/ICEBERG intensities into timsTOF space before scoring. Test: scaffold-split MRR@25 on a COCONUT ±10 ppm pool, ICEBERG scores with vs without R; the 2,538 dual-instrument structures are the calibration set. Kill if the gain is < 0.01.
- Custody log of a number (forensics, q2). Every column has an author; the ppm artefact and the seven computed-m/z libraries fell out of asking "who computed this, when". Test: recompute signed ppm with electron mass for all 2.54 M rows, re-filter at 20 ppm, and check that the 62,834 "beyond 20 ppm" rows shrink to the true outliers (q2 predicts most are pluskal formate rows); also check whether the +1.9 ppm bias disappears under one specific m/z convention.
- Standard of proof → seat-filling (law of evidence, q4). Within-list hit curve ∝ k^−2.07 (64 % of hits at rank 1); Bayes-optimal seating sorts all hypotheses' candidates by posterior; block seating ("13–15 primary, then hedges") loses 0.01–0.03 EV; first alternative-hypothesis candidate belongs at seat 2 unless P(primary) ≥ 0.90. Test: on the np-examples backtest, compare block vs posterior-sorted seating for the same candidate scores; keep if posterior-sorted gains ≥ 0.01.
- Daubert "known error rate" (law, q4). Never promote a model on an unpaired LB delta < 0.04 (SE on ~130 public molecules). Test: re-submit a byte-identical notebook (q4 saw moves up to 0.105 from reruns) — if the variance is that large, all model selection must be local.
- Timbre / overtone ratios (music, q3). Neutral-loss LRs as pitch-invariant priors, precursor-mass-dependent, promote-only. Test: add the 19 SMARTS log-LR rules to the reranker on np-examples; expect ≤ +0.01 when a fingerprint model is already present (double counting at ρ ≈ 0.6) and more on enumerated/de-novo candidates. Kill the rule set if MRR falls on glycosides.
- Enharmonic spelling (music, q3) — already decisive. One slot per metric key; 35 % of COCONUT and 1,541 train label groups are spellings. Test: count duplicate canonical keys in any public top submission; each is a dead seat.
- Duds kept honestly: sky subtraction (recurring m/z bins are chemistry; Hart 2024 says up-weight them); Saha equilibrium for adducts (ESI is kinetic); "burden shifts / double jeopardy" (wrong guesses cost nothing); decoy halogen fraction as a hidden-set hint (killed by the random-draw test).
Coverage gaps — what nobody researched and should have
- Enveda's acquisition/export pipeline (isolation width, resolution mode, calibration cadence,
ms_prepper/extract.pyparameters, the +2.12 vs +2.00 Da trim). Everything on the hidden instrument is inferred from data; the enveda-180 preprint DOI did not resolve and TIMS-Bench returned 429. - Mechanism of the +1.9 ppm positive-mode precursor bias and the +0.4 mDa low-mass fragment bias. Nobody owns "timsTOF calibration physics at low m/z".
- Replicate reproducibility of timsTOF fragment intensities (no published CV); the proposed response map assumes intensities are stable.
- The 1,500 vs 2,500 / 7 vs 10 adduct / public-fraction questions need a logged-in human on Kaggle (discussion 741404, Rules 2.6, notebook RAM, 9 h limit). Every queen hit the JS/reCAPTCHA wall.
- enveda-np-examples itself: no public description; nobody characterised its 250 structures by class, mass or CE pattern (35/50/80 eV, ";"-lists) even though it is the only instrument-matched NP validation set and the one that predicts the LB.
- Negative-mode chemistry: q2 shows negative mode is ~1.3× under-represented if the thinning applied; q3's LRs are mostly positive-mode; nobody studied [M−H]− fragmentation of NPs on timsTOF.
- Ion mobility / CCS is not exported, treated as a tie by q1 — but whether Enveda's pipeline merged mobility-separated isomers into one spectrum (chimeric by construction) was not asked.
- Per-class accuracy does not exist in the literature (q3 negative result); it must be measured on our own stratified hold-out — nobody has specified that hold-out at class level.
- RDKit 2026.03.3 overlap recomputation (~50 CPU-h) is queued, not done; all overlap numbers are on standard keys.
Recommendations to the Emperor (structure)
- Merge q1 and q2's instrument-data seam into one shared "timsTOF ground-truth" worker next tick (q1
noise_*+ q2adducts_*both computed on the parquet and overlapped on merge semantics, ppm, trimming). One owner for "the physics of the shipped numbers" would have resolved the window and trim contradictions in-tick. - New channel
measure_x_infer_validationbetween q2/q4 and q7: three queens independently converged on "the 250 np-examples three-mask backtest is the only LB-predictive validation"; it belongs to nobody formally. Make q7 the owner and have q2 ship the fold definition (heldout_A 1,977 structures / 13,556 spectra; heldout_B 293). - Kill or shrink
q3/np_classes_biosynthesis_npclassifier_classyfireandq4/casmi_history_*for future ticks — both are complete and largely textbook from here; redirect their worker budget to (a) an enveda-np-examples characterisation worker and (b) a negative-mode NP fragmentation worker. - Add a human-in-the-loop task (not an agent): open Kaggle logged-in once and copy Rules, Overview, discussion 741404, the public/private fraction, RAM and rerun limits verbatim into
research/_meta/. Four queens burned effort on the same wall. - Resource throttle is binding: q3's workers saw C: hit 0 bytes free mid-run (one CSV write failed and was recovered). Any further parquet computation should be serialised across the hive or moved to Explorer; the 50 CPU-h canonicalisation job must not run on this box.
- Fold the "epidemiology of measurement bias" lens into q2 next tick (unused this tick): the library-barcode problem (intensity floors, base-peak null patterns, CE null patterns, computed m/z) is exactly a confounding-by-source problem and deserves a formal treatment.
Recommendations to the Scribe — five things a layman most needs from the MEASURE half
- The practice test is a copy of the training data, and from a drug-screening library, not from nature. It tells us the file format and nothing about the real molecules; anyone "validating" on it is validating on train.
- The hidden spectra come from one Bruker instrument that smashes molecules at three fixed hardness settings and sometimes adds the three pictures together. The public libraries were recorded differently, so the same molecule looks different; we must translate (energy map) and clean (mask) before a model sees it.
- Nine of ten peaks are tiny and the number of peaks mostly tells you how much sample there was, not what it is. Default cleaning tools either delete three quarters of a spectrum or let a model learn concentration instead of structure.
- The scorer is tone-deaf to mirror images and some proton shuffles; two drawings that differ only that way count as one answer, and a third of the candidate database is such duplicate spellings. Never spend two of 25 guesses on them — and the hard part is the dozens of same-formula cousins differing only in where one group sits.
- The public scoreboard is crowded with hundreds of copies of the same shared notebook and is measured on ~130 molecules, so small differences mean nothing. The host made the answer key with its own tools (SIRIUS + NMR) on new natural products; the only honest local exam is one the host could have set — 250 natural products on the same instrument, never seen in training.
King Infer's log
king: king_infer tick: 1 (interrupted by a session rate limit mid-tick; q5, q6, q8 resumed with fresh queen agents; q7 completed on the first pass) date: 2026-10-07 queens: q5_methods_lineage 5/5, q6_candidate_space_retrieval 5/5, q7_compute_execution 5/5, q8_serendipity 5/5 stagnation_rule: not triggered (every queen returned ≥ 10 new concrete sourced facts; none generic) re_dispatches: 0 (the three "resume" agents replaced dead queens; they did not repeat finished workers)
King INFER — tick 1 log
Read only the four _QUEEN_DIGEST.md files, per protocol. Worker bodies unread; every file path below is the queen's attribution.
Top 10 facts that change how we would model
- The score is bounded by pool coverage, not by the ranker. MRR@25 = c·r·e; the LB-0.417 regime back-solves to c ≈ 0.55–0.61 (truth in the candidate pool), r ≈ 0.90 (ranked in top-25 given in pool), e ≈ 0.76 (mean 1/rank given ranked). Headroom c +0.27, e +0.13, r +0.05. ~39 % of hidden molecules are in no database at all ("Class 3") and score zero for everyone. —
research/q6_candidate_space_retrieval/coverage_ceiling_analysis_recall_at_25_and_how_to_raise_it.md(medium confidence: back-solved from public write-ups, not measured). - There is one natural-product map and it is small: COCONUT Oct-2026 = 479,690 InChIKey14 (738,794 rows collapse). LOTUS adds +254, NPAtlas ≤ +138, the 29-source THEOBROMA union only reaches 486,215. "Union of NP databases" is not a lever. ±10 ppm window in IK14-dedup COCONUT: median 36 candidates, p90 204. —
q6/candidate_databases_pubchem_coconut_lotus_npatlas_chembl_coverage_math.md(high; q3 independently reproduced the 479,690). - PubChem wholesale is a proven loss; gated injection is a small win. One public write-up: train+COCONUT LB 0.112 → train+PubChem 0.079 (−30 %); another: PubChem top-50 injected only when library confidence < 0.7 gives +0.011; ungated one-step analog enumeration gave +0.000. Prior-density ratio COCONUT : PubChem-only ≈ 1,500–2,000×, i.e. a PubChem candidate needs ~7 nats of spectral evidence to earn a slot. — same q6 files (medium: LB numbers carry ±0.01 noise).
- Formula-first hard filtering is a net loss on MRR@25. Inside a ±10 ppm COCONUT window the true formula already covers 62–90 % of structures (median 2–4 formulas per window), while MS2-only formula top-1 is 50–77 % on honest benchmarks (MIST-CF 50.5 % on MassSpecGym MCES split; SIRIUS single-spectrum error > 50 % toward m/z 800). Break-even P(formula) ≈ 0.75–0.9 → use the formula posterior as a soft multiplicative feature; hard-gate only on mass and adduct. —
q6/formula_assignment_sirius_buddy_mistcf_isotopes_timsTOF.md,q5/failure_modes_generalisation_instrument_shift_evaluation_pitfalls.md(high). - Published benchmark numbers are largely inflated. The June-2026 MassSpecGym audit (arXiv 2606.19624) found problems in ≥ 17 of 26 papers; one retrieval method fell 82.41 % → 6.70 % hit@1 when a canonicalisation shortcut was removed; a spectrum-blind baseline (PubChem default order) scores 49.98 % hit@1; honest formula-free retrieval tops out ~27 %. —
q5/foundation_models_dreams_prism_massspecgym_benchmarks.md,q5/failure_modes_*.md(medium-high). - Our natural local validation is a library-hit test and lies. All 250 enveda-np-examples structures are also in public libraries (RIKEN 246, GNPS 233; 0 exclusive); the visible test is verbatim train (1213/1213); 29 % of enveda-180 rows are dimer adducts ([2M+Na]+ 16 %, [2M+H]+ 8 %) the test never has (test: 79 % [M+H]+, 16 % [M−H]−, no dimers). One public write-up scored 0.445 locally vs 0.063 on LB. There is no instrument-matched NP hold-out; only 2,538 train structures are measured on both timsTOF and other instruments. —
q5/failure_modes_*.md,q7/validation_design_that_mimics_hidden_test_novel_timsTOF_np.md(high; own audit of train.parquet). - Fuse by summing calibrated log-odds; never rank-average, never cascade by rank. Simulation (400 × 1,000 candidates): rank-averaging two scorers loses to the better one alone once error correlation ρ ≥ 0.3; independence-assuming combiners stop helping at ρ ≈ 0.6 — then only a jointly fitted stacker (LightGBM lambdarank over log-LR features) helps. CASMI 2016 six-method consensus beat every single entry (Top-10 166/208 vs 131/208) because those engines were orthogonal. RRF k=60 is the no-training control. —
q5/foundation_models_*_lr_sim.py,q6/candidate_ranking_features_learning_to_rank_ensembles.md,q7/sibling_project_lessons_consensus_confidence_poolsize.md(ordering high, magnitudes low). - The popularity prior is inverted on this test. MetFrag's largest documented gain (top-1 30 → 336 → 420 of 473 with PubChem/ChemSpider reference counts; 143/208 CASMI 2016 challenges at rank 1 from reference counts alone) encodes "already published" — the hidden set is defined as never published. Any ranker trained on GNPS/MassBank learns it implicitly via training-spectrum counts. Exclude the features, audit score-vs-train-count correlation on hold-out. —
q8/healthcare_diagnosis_astronomy_ecology_other_rank_25_problems.md,q6/candidate_ranking_*.md(high for the numbers, medium for the inversion inference). - The training objective closest to MRR@25 is softmax cross-entropy over the real mass-matched candidate set, not in-batch InfoNCE (MSAlign 2026: 47.5 % vs 19.5 % R@1, same encoder); hard negatives give no gain in two papers; no paper optimises MRR directly. De novo SMILES decoders score 0.00 % exact on the structure-disjoint MassSpecGym split; best with true formula + [M+H]+ is 18–24 % (FRIGID, DualLGD, MS-GPT) — slot-filler value only. —
q8/linguistics_translation_and_grammar_of_fragmentation.md,q5/denovo_generation_ms2mol_diffms_madgen_spec2mol_msnovelist.md(medium: unaudited preprints). - No published tool models timsTOF collision energy and no learned method has a timsTOF evaluation. MetFrag/MAGMa/MS-FINDER/SCARF/ICEBERG-1 take no CE; CFM-ID fixed 10/20/40 eV; MassFormer NCE only. Library-search sensitivity drops to 88.0/91.5 % cross-instrument vs 99.5/99.7 % self. CSI:FingerID/SIRIUS is a licensed web service unusable offline; MIST and DreaMS (116 M params, MIT) are the open options; PRISM has no weights, code or preprint. Kaggle image has no RDKit (cp313 wheel 37.2 MB must be vendored);
Canonicalize()silently truncates atmaxTransforms=1000on polyphenols but changed zero IK14s when raised to 20,000. —q5/insilico_fragmentation_metfrag_msfinder_cfmid_iceberg.md,q5/failure_modes_*.md,q7/kaggle_code_competition_constraints_offline_notebook_rdkit_pin.md(high).
Honourable mentions: public LB is 2,673 teams, max 0.480, median 0.328, 228 teams inside the 0.42–0.43 band — deltas < 0.01 are noise (q7, high). Random ranker MRR@25 = H₂₅/n = 3.816/n, so pool size is a first-order multiplier (q7). A 400-molecule MRR has s.e. ≈ 0.02 (q5). Modal A100-40GB is $2.10/GPU-h, not the brief's $1.40; gyorilab's 4×H200 are fully occupied by 11–26-day jobs (q7, high). 1,612 train InChIKey14 have multiple SMILES; a corrupt precursor_mz of 2,430,333 exists (q5, q7).
Contradictions / disagreements between queens
- enveda-np-examples ∩ COCONUT: 99.6 % vs "8 of 250". q6's coverage ceiling (
coverage_ceiling_*.md) assumes 99.6 %; q6 reports q2 has seen write-ups swinging to 8/250 depending on keying. Every c-estimate and the π_in prior in q8's posterior formula hang on this. Unresolved; q2 asked to recompute under RDKit 2026.03.3. - Pool-size penalty: monotone vs non-monotone. q6 (MetFrag 338 → 2,508 candidates: median rank 8 → 31.5; Laudicina −30 %) and q7's CSI:FingerID figure (68.5 % → 34.4 %) say monotone decay; q7's CASMI 2016 citation (PMC5368104) says performance is worst at 200–1,000 candidates and better both above and below, confounded by formula. Resolution both accept: any pool-size experiment must stratify by molecular formula.
- Consensus magnitude. q5 and q8 cite CASMI 2016 consensus (+35 Top-10 hits) as a win; q7's sibling evidence says the gain halved (+0.038 → +0.018) when a weaker engine was added and that consensus is a catastrophe detector, not a fine ranker. Not a contradiction in direction, but q7's admission-gate requirement should bind.
- Analog reachability. q6: 82.8 % of the 250 np-examples vs 38.7 % of random COCONUT have a one-step neighbour in COCONUT; q3 (via q6) reports 95.7 % at formula level. Same direction, different definitions; q6 treats 82.8 % as a ~2× upper bound for the hidden set.
- Morelli vs Li 2021. q8's art lens says low-intensity peaks discriminate isomers; Li 2021's entropy-adaptive weighting switches the minor-peak boost off above spectral entropy 3. Whether merged multi-CE timsTOF spectra sit above or below that threshold is unknown (q8 asked q1).
- Sequential cascade: q5 (simulation), q7 (eyewitness ROC literature) and q8 independently killed it. Three-way agreement, recorded as such.
Cross-domain transfers (from outside chemistry/ML)
- Bayesian search theory — Stone's optimal-search theorem (q6). Allocate the 25 slots by sorting the union of channels by calibrated posterior P(candidate = truth), never by ranker quota; widen the map only when the dense cell comes up empty. Test: on the three-mask np-examples split + masked-COCONUT set (MCES ≥ 10 from train), submit union-of-channels ordered by isotonic-calibrated posterior vs a fixed 15/7/3 quota vs COCONUT ranker alone; prediction +0.01–0.03 over quota, +0.03–0.05 over single pool. Kill if the posterior ordering is not sign-consistent in ≥ 70 % of class × adduct cells.
- Eyewitness line-ups — the "mock-witness" test (q7). A line-up is biased when fillers do not match the description. Test: train a ranker on precursor m/z + formula + adduct only, no MS2; if it beats H₂₅/n on our pool, our local score is harvesting pool bias, not spectral evidence. Cheapest experiment produced this tick (CPU-hours).
- Art attribution — Morelli's minor details (q8). Test: intensity ablation on an isomer-rich hold-out: base peak removed / peaks < 10 % only / sqrt / log / entropy-adaptive weighting; readout = isomer-swap rate and MRR@25; < 1 GPU-day. Stratify by spectral entropy to resolve the Li 2021 boundary.
- Stylometry — Mosteller–Wallace function words (q8). Common neutral losses (H2O, CO, CH3) are "function words"; z-score each candidate's explained-loss profile against the corpus norm as one orthogonal reranker feature. Test: add as a single LightGBM feature; keep only if the paired-bootstrap CI excludes 0.
- Cryptanalysis — known-plaintext cribs (q8). The ~40–150 confident library hits in the hidden set estimate the shared timsTOF "day key" (ppm offset, CE response, score temperature). Test: fit the calibration on crib hits, apply to the rest, measure Δ on the analog-excluded hold-out; kill if the gain is confined to the library-like easy tail.
- eDNA taxonomy — open-set assignment (q8). When the true genus is deleted, IDTAXA forces a wrong assignment 0.01 % of the time vs BLAST 17.3 %, but abstention costs coverage and MRR@25 never rewards abstaining. Test: an "absence detector" (q7 gate: AUROC ≥ 0.8, ECE ≤ 0.05) routes slots 11–25 to a gated analog/PubChem generator; negative control = routing must not lower MRR on the truth-present half.
- Differential diagnosis — class-conditional likelihood ratios (q5). CSI:FingerID already does per-bit naive Bayes; nobody conditions on NPClassifier class. Test: class-conditional LR rescoring; success threshold ≥ 0.02 Class-2 MRR in glycosides.
Duds, kept: Enigma-style deterministic elimination (MS-FINDER rules explain only 78–85 % of fragment ions → would discard 15–22 % of true answers); sequential line-up cascade (collapses under ROC); double-blind administration (we can resubmit; only pre-registration survived as q7/prereg_template.md); beam-search curse (moot since de novo is filler); "diagnosis of exclusion" hard switching (the in-DB gate showed no signal: 42.8 % vs 59.4 % base rate).
Coverage gaps — what nobody researched and should have
- The hidden set's actual COCONUT coverage (π_in). Everything downstream depends on it and it is only back-solved. The 250 np-examples are the wrong proxy (dense-region bias). Needs q2 + q6 jointly, with the metric's exact RDKit pipeline.
- The multi-CE merge function. The hold-out design needs to reproduce how Enveda merged 1–16 single-CE spectra into one row; q7 asked q2, no answer yet.
- A real timsTOF evaluation of any learned method ("P-inst": 2,538 train structures measured on timsTOF and another instrument). Measurable in hours; nobody has run it.
- Learning-to-rank in MS: no paper with a LambdaMART/XGBoost ablation could be opened; the LTR recommendation rests on IR evidence.
- Nearest-analog → candidate-structure MRR on a structure-disjoint benchmark: no publication; MassSpecGym ran no library-similarity baseline.
- Negative-mode coverage (16 % of test is [M−H]−): de novo and fragmenter numbers are all [M+H]+.
- Kaggle hard limits (9 h re-run, GPU type, dataset size) rest on one GitHub mirror; q7 asked q4 for a second source.
- Licence audit for a Kaggle-open-sourced solution: data CC BY-NC 4.0, NPAtlas CC BY-NC, 3DMolMS CC BY-NC-SA, FRIGID unresolved — nobody owns this.
Recommendations to the Emperor (structure)
- Create a cross-king channel
coverage_truthbetween q2_data_forensics, q6 and q8 (or widendata_x_candidatesto admit q8). The single most load-bearing number in the whole hive — np-examples/COCONUT overlap under the metric's canonicalisation, hence π_in — is being estimated three different ways by three queens. - Give q7 explicit ownership of the shared hold-out ("truth-absent hold-out": ~1,000 scaffold-disjoint NPs, structure-disjoint by IK14, analogs removed at MCES < 10 / Tanimoto ≥ 0.7, truth deleted from the pool for a random half, test adduct × n-CE mix). q5, q6 and q8 all asked for it independently; without it none of the seven transfers above is measurable. Consider renaming q7's validation worker to that deliverable.
- Merge q5's de-novo worker into q6's coverage-ceiling worker next tick. De novo is now a Class-3 slot-filler and belongs with analog enumeration and gated PubChem as "ways to raise c"; keeping it under methods-lineage over-weights it.
- Split q5's failure-modes worker into two: (a) evaluation leakage/benchmark audit (it is doing the hive's quality control and has the most actionable findings), (b) physical shift (CE, instrument, adduct) which belongs with the
physics_x_methodschannel. - Kill or demote: PRISM research (no weights, code or preprint — nothing more to learn), SuperNatural/NPASS/KNApSAcK (+~0 IK14, NC licences), and the "bigger NP union" question (closed).
- Open a
licences_and_releaseworker somewhere (q4 or q7): winning methods must be open-sourced and several inputs are NC-licensed. - Rate-limit resilience: this tick lost three queens mid-flight. Recommend queens write
_QUEEN_DIGEST.mdas a stub immediately after dispatch and overwrite at the end, so a dead queen leaves a visible marker.
Recommendations to the Scribe — five things a layman most needs from INFER's half
- The shortlist, not the ranking, is the ceiling. Roughly four in ten test molecules are in no catalogue at all; for those, every team scores zero. The biggest win is getting the right answer onto the list of suspects, and widening the list carelessly buries it.
- All known natural products fit in one list of ~480,000, and for any measured molecule only a few dozen have the right mass. The hard part is choosing among look-alikes; searching the 124-million-entry PubChem makes things worse, not better.
- Our practice exam is rigged. The visible test is copied from training; the natural hold-out molecules all have public spectra; 228 teams are tied within one hundredth on a public scoreboard that cannot tell them apart. We trust only scores on molecules whose relatives the model never saw.
- Combine evidence like a doctor, not a committee. Add up calibrated log-odds from engines that fail differently; averaging rankings or voting lets one bad engine veto the right answer; a "which engine goes first" cascade buys speed, not accuracy.
- "Famous" is a trap. Methods that score well in the literature often win by preferring molecules that have been written about a lot. This test is defined as molecules nobody has published, so that trick points the wrong way — and so does staring only at the biggest peak.
Channel traffic
chemistry_x_candidates
2026-10-07 01:20 [emperor] — see research/_meta/train_eda.md
Element coverage and carbon-count histograms of the 275,810 train structures are in dashboard/src/data/eda_train.json (keys element_coverage_structures, carbon_count_hist_structures). enveda-np-examples = 250 NP structures on timsTOF, all also in public libraries.
2026-10-07 — [q6_candidate_space_retrieval] -> q3_natural_product_chemistry: which databases under-cover which classes
- There is effectively one open NP map, not several. COCONUT Oct-2026 = 479,690 unique InChIKey14 (computed from the download); LOTUS v11 adds 254; the 29-source THEOBROMA union reaches only 486,215 IK14 families. NPAtlas 3.0 (36,545 microbial NPs, v2024_09) and SuperNatural 3.0 / NPASS 3.0 add zero IK14 coverage over COCONUT and are CC BY-NC, so they are not worth shipping. Practical consequence: class-level gaps in COCONUT are gaps in every pool we can build.
- Mass-window density is class-relevant: +/-10 ppm in COCONUT gives median 36 candidates, but p90 204 and a worst case of 1,761 (a curation artefact at C6H5N3O). Classes with many isomeric variants (glycosides, terpenoid scaffolds with the same formula) will sit in the dense tail.
- A public write-up reports 70.8 % of in-window candidates already share the true formula, so isomer discrimination - not formula - is the chemistry problem that decides our rank.
Ask for q3: two concrete questions. (a) Which NP superclasses are systematically absent or thin in COCONUT — microbial post-PKS tailoring products, marine halogenated metabolites, plant specialised metabolites from under-sampled taxa? I need a class list to decide where analog enumeration must cover for the ~40 % of hidden molecules likely outside every database. (b) For "one-step analogs", which biosynthetic transformations are the realistic ones to enumerate (hydroxylation, O-methylation, glycosylation, prenylation, halogenation) and what mass shifts do they imply? I want an enumeration operator grounded in biosynthesis, not in generic RDKit substituent swaps. — [q6_candidate_space_retrieval]
2026-10-07 (tick 2) — [q6_candidate_space_retrieval] -> q3_natural_product_chemistry: our numbers agree; two refinements; one ask
- Agreement. Your worker and mine both recomputed COCONUT Oct-2026 to 479,690 IK14 from 738,794 rows. Minor difference: you find NPAtlas adds 138 new IK14, my worker reported ~0 — immaterial (Δc < 0.001). Your "27 median competitors at ±5 ppm, p90 193" and my "36 median at ±10 ppm, p90 204" are consistent.
- Your 95.7 % formula-level parent / 247-of-250 figure vs my structure-level 82.8 %. Different definitions:
mine requires the one-step product to be an exact COCONUT IK14 (15 RDKit operators), yours requires a
formula-level relative. The gap (97 % → 83 %) is the regiochemistry combinatorics you flagged. On a random
COCONUT sample (not the 250 well-studied NPs) my reachability falls to 38.7 % — so the 250 overstate
analog reach by ~2×. Operators ranked by how many of the 250 they reach: dehydroxylate 113, sp3-hydroxylate
106, aromatic-hydroxylate 83, O-methylate 83, O-hexosylate 79, O-acetylate 61 (
coverage_ceiling_...md§4). Cost: 33 products/parent mean → ±10 ppm pool inflates ~28× (median 35 → ~1,050). - Ask (refines my tick-1 question). You say sesquiterpene/diterpene formulas carry 600–1,300 isomers. Which NPClassifier superclasses/classes populate the dense tail (p90+ windows) in the 245–460 m/z test band? I want to restrict analog enumeration to classes where (a) one-step relatives are plausible and (b) the window is not already saturated with isomers, and to build a per-class "sensor sweep width" for the ranker. — [q6_candidate_space_retrieval]
2026-10-07 — [q3_natural_product_chemistry] -> q6_candidate_space_retrieval: answers to (a) thin classes and (b) biosynthetic enumeration operator
(a) Which classes are thin or absent in COCONUT — and the one that matters is absent by policy, not by sampling.
- COCONUT deletes fluorinated and sulfonamide-bearing entries at import (COCONUT 2.0 paper; confirmed by our enveda_platform worker). Enveda's own analog patents (withaferin A, isoandrographolide, nootkatone families) put F at C-4, on side-chain CH2F and as aryl-F, plus heteroaryls/NMe2/nitriles. So the "synthetics plausibly in nature / analogs" slice of the hidden 400 (our estimate 15–25 %) is unreachable in COCONUT at any pool size. Train shows the contrast: enveda-np-examples 0.4 % F / 3.6 % S vs enveda-180 23.0 % F / 28.3 % S. Mitigation: a small side pool of enumerated F/Cl/Br/heteroaryl children of top-k COCONUT scaffolds, or scaffold-restricted PubChem (same Murcko scaffold as a COCONUT hit) — not full PubChem. File:
research/q3_natural_product_chemistry/enveda_platform_isolation_profiling_what_analogs_means.md§6. - Kingdom skew, not class absence, is the second gap. Terpenoids are 46.5 % of plant structures but 6.2 % of bacterial; peptides 1.5 % vs 27.5 %. NPAtlas (microbial) adds only 138 new IK14 over COCONUT, so microbial post-PKS/NRPS products are present but the COCONUT-wide class prior is plant-shaped; 10.8 % of NPAtlas entries are pathway hybrids (PKS-NRPS 1,997; alkaloid-peptide 891). Marine halogenated: CMNPD is inside COCONUT (30,700 rows / 26,538 IK14) — we found no evidence it is systematically thin, but note train has only 8,522 Br-containing and 369 I-containing structures, so the spectral model, not the pool, is the weak link for marine halogenated NPs. Files:
np_classes_biosynthesis_npclassifier_classyfire.md§2–3,np_chemical_space_coconut_lotus_npatlas_sizes_overlap.md§1. - COCONUT's alkaloid pathway (23 % of COCONUT vs 11–13 % of LOTUS/NPAtlas) has mean NP-likeness 0.07 — it is diluted with synthetic-looking N-heterocycles from bulk imports (Super Natural II is the largest contributor: 311,885 rows, 110,432 sole-sourced). Treat alkaloid-window candidates with NP-likeness < 0 as filler; your 1,761-candidate C6H5N3O bucket is the same artefact.
- Lipids/fatty acids: 44.6 k enumerated triacylglycerols inflate COCONUT's fatty-acid pathway (MW median 807 Da). Over-represented in the pool, implausible in a 245–460 m/z timsTOF test window — these are q7's "wrong fillers".
(b) The enumeration operator, grounded in measured COCONUT density. For every COCONUT IK14 we asked whether another COCONUT formula sits exactly one tailoring delta away (fraction with such a relative): +CH2 (O/N-methylation, 14.01565) 84.1 %; −H2 (desaturation, 2.01565) 80.2 %; +O (hydroxylation/epoxidation, 15.99491) 79.8 %; +C2H2O (acetylation, 42.01057) 78.0 %; +CH2O (methoxylation, 30.01056) 77.7 %; +C5H8 (prenylation, 68.06260) 68.9 %; +C3H2O3 (malonylation, 86.00039) 58.0 %; +C5H8O4 (pentosylation, 132.04226) 46.6 %; +C6H10O5 (hexosylation, 162.05282) 33.7 %; +C6H8O6 (glucuronidation, 176.03209) 25.5 %; +SO3 (sulfation, 79.95682) 4.1 %. Any of the eleven: 95.7 %. On the 250 enveda-np-examples molecules, 247/250 have ≥1 such relative in COCONUT (+O 237, −O 234, +CH2 233, +C2H2O 229, +H2 229, +C6H10O5 200). Add halogenation (+Cl−H 33.96103, +Br−H 77.91051) for the analog slice only. Expected gain if gated (fire only when in-window IK14 pool < ~15 or best in-pool score is low): +0.01…+0.04 MRR@25; ungated it dilutes the window 10–100× (a 6-OH parent yields 6 O-methyl regio-isomers per delta) and will cost score, same mechanism as PubChem. File: np_chemical_space_coconut_lotus_npatlas_sizes_overlap.md §Forced analogy.
(c) Fact you did not ask for but need: the neutral-loss "timbre" that would break regio-isomer ties is nearly absent from the timsTOF stratum of train. Our fragmentation worker found only 27 ([M+H]+) and 21 ([M−H]−) timsTOF spectra of terminal O-hexosides in 1.16 M timsTOF rows, and zero oleanane triterpenes matching a strict SMARTS — the timsTOF rows are enveda-180 synthetics. Class rules (hexose 162.0528 LR+ 46, glucuronide 176.0321 LR+ ~270, tropane 124.1121 LR+ 42, desosamine 158.1176 LR+ 369, C-glycoside cross-ring 120.0423 LR+ 66–76; water/terpene losses LR+ ≤ 2.6 = useless) were measured on Orbitrap/QTOF rows and meet timsTOF for the first time in the hidden set. 19-rule soft log-LR reranking table with SMARTS: class_specific_fragmentation_rules_glycosides_terpenes_alkaloids_polyketides.md §8 (+ .json, table reranking_rules). Use as promote-only evidence inside the formula window; never as a hard filter (86 % of hexoside [M+H]+ spectra show no 162 loss at ≥5 %).
Question for q6: when you build the window, can you emit per-candidate NPClassifier pathway + is_glycoside + NP-likeness columns from the COCONUT CSV so the reranker can apply the rules without re-running SMARTS on a million structures?
— [q3_natural_product_chemistry]
2026-10-07 — [q3_natural_product_chemistry] -> q6: the metric-key numbers for your pool (addendum)
- Re-key the pool with the grader's own function, not plain InChIKey14.
K(s) = InChIKey(TautomerEnumerator().Canonicalize(flat, largest-fragment, proton-neutral mol))[:14], rdkit pinned 2026.03.3 (our 60 NP pair tests were bit-identical on 2026.03.3 and 2026.03.6; 2026.09.x changes the tie-break). On train it merges 1,541 groups of distinct stored inchikey14 (3,167 skeletons, 58,954 spectra) and splits 11 skeletons (large peptides, bilins, thiopeptides, rifamycins — enumeration hits maxTransforms). Key changes for 1.61 % of train SMILES overall but 6.9 % of COCONUT-member structures (ansa-macrolides 79 %, dihydroflavonols 52 %, pregnane steroids 41 %, naphthoquinones 41 %). Cost: 17.5 ms mean, >0.4 s p99 → ~3.6 CPU-h for COCONUT; do it once offline on Explorer, shipkey → representative flat SMILES. - Your 479,690 is the right denominator; raw entries are 35 % spellings (steroids 54 %, macrolides 59 %, glycosides 47 %; one skeleton has 72 stereo entries). Flavonoid hexoside formulas are ~68 % enharmonic (C21H20O11: 449 entries → 142 keys; rutin C27H30O16: 495 → 157), triterpene acids C30H48O3: 890 → 385, whereas aglycone formulas are 2–5 % (quercetin C15H10O7: 61 → 59). A top-25 drawn from raw entries carries only 8–17 distinct keys in those classes.
- Positional-isomer density inside the test window (244–459 Da neutral mass): median 35 same-formula skeletons per structure, mean 76, p90 198; 58 % of skeletons have ≥25 same-formula neighbours. This is the recall@25 ceiling you back-solved — not stereo, not formula.
- Train SMILES carry zero stereo and zero multi-fragment SMILES; COCONUT holds 4,247 multi-fragment and ~2,400 metal-containing entries, which change the key. Strip to largest organic fragment; leave quaternary cations charged (berberine is keyed as the cation).
Files:
research/q3_natural_product_chemistry/beyond_2d_tautomers_stereo_isomers_what_the_metric_cannot_see.md(+ .json: 60-pair table, per-formula density, redundancy by superclass). — [q3_natural_product_chemistry]
data_x_candidates
2026-10-07 01:20 [emperor] — official train.parquet EDA now on disk
Read research/_meta/train_eda.md (numbers from the official Kaggle download, not a mirror). Headlines: 2,539,608 spectra / 275,810 InChIKey14 / 51,291 formulas; 11 libraries; timsTOF rows = 1,154,969 and ALL of them are enveda-180 (1,153,785) + enveda-np-examples (1,184); enveda-np-examples has 250 structures, 0 exclusive (all also present in public libs) — it is the only NP-on-timsTOF proxy for the hidden test; visible test.parquet rows match enveda-180 rows 1213/1213 on (precursor_mz, num_peaks, base_peak) — decoy confirmed. 64.6 % of train precursors fall in the test's 245–460 m/z window.
2026-10-07 — [q6_candidate_space_retrieval] -> q2_data_forensics: three pool numbers to anchor on
- At the metric's resolution the natural-product universe is small. My worker downloaded the Oct-2026
COCONUT CSV: 738,794 rows collapse to 479,690 unique InChIKey14. LOTUS v11 (151,885 IK14) is
99.83 % inside COCONUT and contributes only 254 new connectivities; THEOBROMA (29-source union,
bioRxiv 2026.06.12.731585) dedups to 486,215 IK14 families. Unioning NP databases does NOT enlarge
the map. File:
research/q6_candidate_space_retrieval/candidate_databases_pubchem_coconut_lotus_npatlas_chembl_coverage_math.md. - Window sizes, computed on IK14-deduplicated COCONUT: a +/-10 ppm neutral-mass window yields median 36 candidates per molecule, mean 87, p90 204, max 1,761. In the 300-400 Da band the median is 64; 400-500 Da it is 49; 500-600 Da it is 20. Since your EDA says 64.6 % of train precursors sit in the test's 245-460 m/z window, the hidden test lands in the densest part of the window-size distribution, not the sparsest.
- Formula is a weak filter here: in one public write-up's pool, 70.8 % of in-window candidates
already carry the true molecular formula, so perfect formula prediction buys under 30 % pool
reduction. See
research/q6_candidate_space_retrieval/formula_assignment_sirius_buddy_mistcf_isotopes_timsTOF.md.
Ask for q2: enveda-np-examples is 250 NP structures on timsTOF with 0 exclusive structures — can you
publish (a) their InChIKey14 list and (b) the precursor_error_ppm distribution restricted to
instrument_type == timsTOF, split by adduct? I need the real ppm spread to set the retrieval window;
vendor specs are the wrong source. Also: how many of the 275,810 train IK14 are inside COCONUT? That
number sets P(true structure in pool) for any COCONUT-only strategy.
— [q6_candidate_space_retrieval]
2026-10-07 (tick 2) — [q6_candidate_space_retrieval] -> q2_data_forensics: one contradiction to settle, two facts
- Contradiction that decides my coverage estimate. Your
enveda180_and_np_examples_anatomy.mdsays the enveda-np-examples/COCONUT overlap "swings from 8 of 250 to 99.6 % across write-ups depending on how structures were keyed". My coverage-ceiling worker assumed 99.6 % (and measured 82.8 % one-step reachability on those 250). If the true IK14 overlap after RDKit tautomer canonicalisation is anywhere near 8/250, every c-estimate in my digest shifts and the analog-enumeration case gets stronger. Request: compute, with the metric's own pipeline (RDKitTautomerEnumerator().Canonicalize()→ InChIKey14), how many of the 250 np-example IK14s and how many of all 275,810 train IK14s fall in COCONUT Oct-2026 (C:\Temp\casmi26\npdb\coconut\). q3's worker independently got 51/182,941 enveda-180 in COCONUT; that part is settled. - Fact: train structures are mostly pool dilution, not coverage. enveda-180 is Enamine HLL-200 chemistry (0.03 % in COCONUT). Adding the 275,810 train IK14s to COCONUT raises the ±10 ppm window median from 36 to ~56 (q1's write-up number) with ~zero coverage gain for a novel-NP test. Keep train structures in the pool only for the ~16 % Class-1 library-hit channel, and give them a per-source prior, not equal footing.
- Fact: cleaning asymmetry. Your finding that 62,834 train rows exceed 20 ppm while the test was cleaned on this criterion means the retrieval window must be set from the cleaned timsTOF slice (|ppm| ≤ 10), not from the raw column. q1 reports (unverified, one notebook) 99 % of enveda-180 errors < 4.4 ppm — if you can confirm the p99 on timsTOF rows I will set the window at ±5 ppm with a ±10 ppm fallback. — [q6_candidate_space_retrieval]
2026-10-07 — [q2_data_forensics] -> q6_candidate_space_retrieval: answers to your three asks (part 1 of 2)
Computed from the official local train.parquet joined to your COCONUT CSV (coconut_csv_lite-10-2026, 738,794 rows -> 479,690 unique IK14, same number you got). Keys are the shipped inchikey14 / COCONUT standard_inchi_key[:14], NOT re-canonicalised with the RDKit 2026.03.3 tautomer protocol (expect a few % drift; my metric worker measured 12-17 % of NP keys change under that protocol, so recompute before trusting any single structure).
- Train ∩ COCONUT = 26,111 of 275,810 IK14 (9.47 %). Per library: enveda-180 51/182,941 (0.03 %); pluskal_ms2 8,128/46,821 (17 %); gnps 16,912/45,750 (37 %); riken 12,552/15,892 (79 %); mona 9,450/11,681 (81 %); massbank 5,795/9,180 (63 %); msdial 7,326/9,127 (80 %); spectraverse 2,092/9,631 (22 %); drug_plus 1,949/2,539 (77 %); masaryk 353/416 (85 %); enveda-np-examples 249/250 (99.6 %). The 9 non-Enveda libraries together hold 95,157 IK14 of which 26,068 are COCONUT -> that is the whole "NP with a training spectrum" universe, and none of it is on timsTOF. So P(hidden true structure in COCONUT | organiser says "none previously published") is NOT estimable from train; the only empirical anchor is np-examples' 99.6 %, which was chosen to be common NPs.
- enveda-np-examples InChIKey14 list (250 keys) is at
research/q2_data_forensics/_coconut_overlap_and_np_examples_keys.json(fieldnp_examples_inchikey14), with the per-library table above as JSON. - timsTOF
precursor_error_ppmby adduct: my adducts worker is computing it now (with an independent recomputation of ppm from SMILES + adduct mass to catch mis-assigned adducts); will append as part 2.
Reply to your point 2 (window density): agreed, and note that the 64.6 % figure is for ALL train; for the decoy test itself the m/z range is 245-460 by construction, so the hidden set's window-size distribution should be taken from your COCONUT 250-460 band numbers, not from the whole-COCONUT median 36. — [q2_data_forensics]
2026-10-07 — [q2_data_forensics] -> q6_candidate_space_retrieval: part 2 — the real timsTOF ppm spread, and two corrections to the window logic
All computed from the official local train.parquet by my adducts worker (file: research/q2_data_forensics/adducts_polarity_collision_energy_merging_statistics.md, tables in the .json).
- Do not use the shipped
precursor_error_ppmto set the window. It is an absolute value computed with a theoretical m/z that omits the electron mass (inflates every singly-charged row by ~0.55 mDa = 1.57 ppm at m/z 350), it is uncorrelated (r = -0.02) with the true signed error, and it is 1 Da wrong (median 2,403 ppm) on 52,460 formate rows whose adduct_orig was[M+CH2O2]-. Only enveda-np-examples ships a signed, electron-corrected value. The worker recomputed signed ppm fromnormalized_smiles+ electron-corrected adduct mass. - timsTOF, recomputed signed ppm, per adduct (enveda-180; n / median / p90|abs| / max|abs|): [M+H]+ 588,507 / +2.21 / 3.32 / 5.13; [M-H]- 184,747 / +0.80 / 1.91 / 5.08; [M+CH2O2-H]- 24,556 / +0.31 / 1.68 / 5.08; [M+Na]+ 15,957 / +0.96 / 2.85 / 5.13; [M+NH4]+ 4,702 / +1.42 / 4.02 / 5.02; [M+Cl]- 2,407 / -0.14 / 2.44 / 4.88; [M+K]+ 1,084 / +0.21 / 4.03 / 5.04. enveda-180 is hard-truncated at |ppm| <= 5.13 by Enveda's extraction pipeline (99.98 % within +-5, 98.1 % within +-4) and biased +1.9 ppm in positive mode (~+0.7 mDa on [M+H]+; cause unverified). enveda-np-examples (a different batch, un-truncated) is the better hidden-set proxy: [M+H]+ n=569 median +2.00, p90 3.38, max 8.7, 2.5 % beyond 5 ppm; [M-H]- max 21.1 ppm; [M+K]+ 12.5 % beyond 5 ppm. -> Recommendation: primary window -4/+5 ppm (or +-5 centred at +1.5 ppm in positive mode) on the electron-corrected adduct m/z; a +-10 ppm fallback tier for molecules whose top-25 is otherwise empty. Do not go below +-4 ppm.
- Correction to "64.6 % of train in 245-460": my test-set worker found that 99.67 % of all enveda-180 monomer rows already lie in 245-460 — it is the Enamine HLL-200 mass band, not an organiser filter. The hidden set is described as 157-1,159 Da (unverified verbatim) and 42 % of np-examples structures fall OUTSIDE 245-460. Build the pool over the full NP mass range; do not prune it to 245-460.
- The 7 test adducts and offsets to use: [M+H]+ 1.007276, [M-H]- -1.007276, [M+CH2O2-H]- +44.998201, [M+Na]+ +22.989218, [M+NH4]+ +18.033823, [M+K]+ +38.963158, [M+Cl]- +34.969402. The 7-adduct list is a sample, not a whitelist: np-examples also carry [M-H2O+H]+ (2 %) — worth a low-prior hypothesis for ranks 20-25.
Question back to q6: with the window moved from +-10 to -4/+5 ppm, what do the COCONUT per-molecule window sizes become in the 250-460 and 460-1,160 bands? — [q2_data_forensics]
physics_x_methods
Channel: physics_x_methods (q1_instrument_physics <-> q5_methods_lineage; append-only, dated, signed with queen_id)
Purpose: which physical effects methods model or ignore.
2026-10-07 — [q5_methods_lineage] → q1_instrument_physics: what the methods model, what they ignore, and the three physical numbers I need from you
Facts you most need (what the method lineage actually models):
- Collision energy is mostly ignored or merged away. MetFrag, MAGMa, MS-FINDER, SCARF and ICEBERG 1 take no CE at all; ICEBERG 1/SCARF/FraGNNet train on spectra merged across all CEs (sqrt intensity, top-50 peaks > 0.003); CFM-ID 4 predicts fixed QTOF 10/20/40 eV; MassFormer accepts NCE but only for FT/HCD (Orbitrap) instruments; only ICEBERG 2/GLACIER (absolute eV sinusoidal encoding) and 3DMolMS (numeric CE, QTOF or Orbitrap checkpoints) use CE as a number. No tool has timsTOF-specific CE scaling — SIRIUS docs never mention timsTOF, DreaMS pretraining is 97 % Orbitrap, MassSpecGym is 98 % Orbitrap+QTOF. (files: insilico_fragmentation_.md §tool table; failure_modes_.md rows 8–12.)
- Measured cost of CE and instrument mismatch (library search, not ML): sensitivity 95.1–98.4 % inside a 20–50 eV window but < 81 % for QqTOF 5–10 eV and 21–61 % for Orbitrap NCE > 90 (PMC6359582); cross-instrument library sensitivity 88.0 % / 91.5 % vs 99.5 % / 99.7 % self, with a 4.7 eV mean CE shift QTOF→Orbitrap Elite (PMC6359582; PMC9400100). No published cross-instrument number exists for any learned model, and none for timsTOF.
- Intensity is treated inconsistently: MS-FINDER's score ignores intensities entirely; MetFrag/MAGMa weight by intensity; CFM-ID leaves ~42 % of NIST-20 peaks unannotated; MS-FINDER's H-rearrangement rules explain only 78.4 %/84.8 % of fragment ions — so any hard physical filter would discard 15–22 % of true answers. Adducts: forward/fragmenter models exist only for [M+H]+, [M-H]-, [M+Na]+ (ICEBERG 2: 10 adducts incl. [M+Cl]-); [M+H-2H2O]+ and [M-H+2Na]+ are near zero for all methods (arXiv 2601.00941). Train has 29 % dimer-adduct rows ([2M+Na]+ 188,158; [2M+H]+ 89,487; [2M-H]- 52,933) that never occur in the test.
- The one instrument-shift probe that exists in our data: 2,538 train structures are measured on both timsTOF (enveda-180/np-examples) and non-timsTOF instruments. My failure-modes worker's "P-inst" diagnostic scores their timsTOF spectra with a model trained without them and reads the drop vs their own Orbitrap/QTOF spectra as the pure instrument penalty. Nobody has run it yet.
Questions for q1 (answers change what we build):
- Q1. For a Bruker timsTOF with merged multi-CE spectra (test
collision_energy_evlists), what is the effective CE distribution per precursor m/z (stepped/ramped? which eV values?) and how does it map onto QTOF 10/20/40 eV (CFM-ID) and Orbitrap NCE (MassFormer)? If a linear eV↔NCE map at given m/z exists, ICEBERG 2's eV encoding becomes usable with a per-spectrum offset. - Q2. What relative-intensity noise floor should a forward-model matcher assume on timsTOF (q8 asked the same): below what fraction of base peak are peaks chemical noise rather than discriminating signal? Every forward model keeps top-50 peaks > 0.003 — is that physically sensible on timsTOF?
- Q3. Any physics that distinguishes sugar-linkage positional isomers (rutinoside vs neohesperidoside give literally identical MS/MS, PMC2722651; a dedicated classifier reaches 84.7 %, PMC11644359)? E.g. does TIMS mobility leak into the merged spectrum in any way, or are these a hard physical tie we should treat as a prior-ordered tie in the 25 slots? — [q5_methods_lineage]
2026-10-07 — [q1_instrument_physics] → q5_methods_lineage: answers to Q1–Q3 plus the physics facts your methods ignore
All numbers below are measured on the official train/test parquet (enveda-180 = 1,153,785 timsTOF rows, the same pipeline as the hidden test) unless a source is named. Files: research/q1_instrument_physics/cid_fragmentation_physics_collision_energy.md (CID) and noise_artifacts_isotopes_insource_chimeric.md (NOISE).
Q1 — effective CE on the test timsTOF and the map to QTOF eV / Orbitrap NCE.
- enveda-180 and test do NOT use the 1/K0-ramped dda-PASEF CE. They use fixed lab-frame 20, 40, 60 eV (negative mode stored as −20/−40/−60). Test patterns: [60] 363, [40] 362, [20,40,60] 298 (24.6 %), [20] 190. Train enveda-180: [40] 327,468 / [20,40,60] 304,780 / [60] 299,465 / [20] 220,130; 186,392 (InChIKey14, adduct) groups carry all four. (CID §2)
- A "merged" spectrum is the raw-ion-count sum of the three scans, re-centroided (enveda-180
extract.py/utils.py; reconstructed numerically on 595/600 groups). Because 20/40 eV scans carry more counts, the merged object is count-weighted ≈ 0.5/0.4/0.1 (NNLS-fittable on the 186 k quartets) — NOT the equal-thirds Thermo stepped-NCE object and NOT a max-merge. Merged median num_peaks 258 vs 88–132 for singles. - Train converts Orbitrap NCE with
eV = NCE × (m/z)/500(ratio median 0.9998 over 966,330 values, charge factor 1); QTOF volts pass through; ion-trap NCE leftNone(10,318 rows); all 220,849 GNPS rows have no energy at all. - Empirical map (2,421 same-molecule timsTOF/Orbitrap pairs): timsTOF 20 eV ↔ Orbitrap NCE 20 (median sqrt-cosine 0.82), 40 eV ↔ NCE 45 (0.70), 60 eV ↔ NCE 60 (0.61); merged ↔ NCE 30–45 (0.68–0.69). Same-instrument 20 vs 60 eV cosine is only 0.285. Rule of thumb: NCE_equiv ≈ 0.65–0.75 × eV × 500/(m/z), i.e. the parquet's own conversion over-states NCE by ~1.4×. For CFM-ID use the 10–20 / 20–40 / 40 eV models for the 20 / 40 / 60 eV scans. So yes: ICEBERG 2's eV encoding is usable, with this per-m/z offset rather than a per-spectrum one. Confidence medium (own measurement; no published small-molecule timsTOF-vs-Orbitrap study exists).
- Centre-of-mass energy with N2: m/z 300 at 20/40/60 eV → 1.71/3.42/5.12 eV; precursor survival for [M+H]+ 250–450 Da: ~11–13 % of TIC at 20 eV, absent in 63–72 % of spectra at 40 eV, 99 % at 60 eV.
Q2 — noise floor on timsTOF. There is no relative floor; the lists are raw integer centroids with a hard floor of exactly 9 counts and no peak cap (p95 = 823 peaks). 91.8 % of enveda-180 peaks are < 1 % of base peak, 75.1 % < 0.1 %, 39.8 % < 0.01 % (GNPS-Orbitrap: 60 / 16 / 5 %). Peak count tracks ion abundance (Spearman(bpi, n_peaks) = +0.71; weakest tercile median 30 peaks, strongest 332). So "top-50 > 0.003" throws away ~75 % of a timsTOF list and keeps a count that depends on concentration, not chemistry. Physically sensible rule: cut on absolute counts (I_rel × base_peak_intensity < 20, 7.1 % of peaks; bpi is in test.parquet), not on relative intensity above 1e-4; then cap to top-K ≈ 100–150 so abundance does not leak through peak density. (NOISE §1)
Q3 — sugar-linkage positional isomers. Nothing in the exported peak list carries mobility: the merge sums microscans across the TIMS ramp and the parquet has no 1/K0 or CCS column, so the ion-mobility axis is lost before you see it. The only energy-explicit physics that survives is precursor survival / breakdown-curve shape (Ecom50 scales with degrees of freedom; Kertesz 2009, Memboeuf 2010), and it is preserved in the 20/40/60 eV singles but blurred in the merged object — a weak, untested lever for linkage isomers. Treat rutinoside/neohesperidoside-type pairs as a physical tie and order them by prior. Confidence low (not studied by a dedicated worker).
Three physical effects every public model ignores, now quantified: (i) precursor family inside the 2 Th isolation window — precursor peak in 45.9 % of enveda-180 spectra, M+1 in 38.7 % (6.1 % on GNPS-Orbitrap), M+2 in 21.7 %; (ii) 13C satellites at +1.0034 on 71.1 % of peaks ≥ 5 % (14.0 % on GNPS-Orbitrap); (iii) low-mass fragments (< m/z 150) read +0.3–0.6 mDa (+4 to +6.5 ppm) high on enveda-180, ≈ 0 on Orbitrap — a symmetric ±5 ppm subformula window rejects ~half of true C5–C9 fragments. A 5-step mask recipe is in NOISE "Forced analogy".
Question back to q5: your P-inst diagnostic (2,538 dual-instrument structures) — please run it with and without our telluric mask on the timsTOF side; if the instrument penalty collapses after masking, the gap is artefact, not chemistry, and no timsTOF fine-tuning is needed. Also: does any fingerprint model's training recipe ever strip +1.0034 satellites? — [q1_instrument_physics]
rules_x_execution
2026-10-07 — [q7_compute_execution] → q4_challenge_intelligence
Measured today with the locally installed, authenticated Kaggle CLI 2.2.4 (the API works even though the
web pages are JS shells — q4, you can use this too: kaggle competitions list -s CASMI,
kaggle competitions files <ref>, kaggle competitions leaderboard <ref> -d).
- The public leaderboard is NOT scored on the visible decoy test. Full LB pulled 2026-10-07: 2,673 teams,
max 0.480, median 0.328. If the scored set were the visible
test.parquet(verbatim enveda-180 train rows), plain library search would score near 1.0 and the top would be saturated. It is not. So the public LB is a sample of the real hidden set. Correction to BRIEF.md: top-1 is now 0.480 (team "pikachu"), not 0.471; team count is 2,673, not "800+". - The LB cannot discriminate in the medal zone. Teams at or above: 0.48=1, 0.46=2, 0.45=3, 0.44=12, 0.43=35, 0.42=263, 0.40=727. That is a 228-team pile-up inside the single 0.42–0.43 band — ~0.01 MRR separates rank 35 from rank 263. Any "improvement" below ~0.01 on the public LB is noise.
- We have not entered.
kaggle competitions downloadreturns 403 Forbidden — the competition rules have not been accepted on this account, so no data can be downloaded and no submission can be made. A human must accept them. Entry/team-merge deadline is 2026-12-07. This is the single hardest blocker in the project. - Effort profile: median submissions used is 54 for teams ≥0.44 (n=12), 22 for the 0.42–0.43 band (n=228), 3 for teams <0.30. Leaders average ~2.3 submissions/day, not the permitted 5. 2,541 of 2,673 teams are solo.
Question for q4: the competition is listed as Featured, $50,000, deadline 2026-12-14 23:59. Can you confirm
from write-ups/mirrors (a) whether the notebook rerun is on the hidden set only, (b) the rerun time limit, and
(c) what fraction of the hidden set the public LB is computed on? Our submission-slot allocation and all of our
kill criteria depend on the standard error of the public score, which depends on (c).
— [q7_compute_execution]
2026-10-07 — [q4_challenge_intelligence] → q7_compute_execution (interim; rules worker still running, more to follow)
- Metric noise floor. With ~400 hidden molecules and per-molecule reciprocal ranks of SD ≈ 0.4, the SE of MRR@25 is ≈ 0.02; a 95% bootstrap interval on a 400-molecule local hold-out is ≈ ±0.04. Do not promote model A over B on a delta < ~0.04 unless a paired bootstrap clears it. (critiques_fiehn_benchmark_validity..., sourced to de Jonge 2022 + own arithmetic.)
- Slot economics (queen's EV model, pure-Python, scratchpad
mrr_ev.py). A single-formula ranked list captures ~93% of its expected MRR in 3 slots and ~98% in 5 (geometric within-hit decay q=0.6). Slots 6–25 are nearly free: spend them on alternate formula / adduct / regioisomer hypotheses, never on same-InChIKey14 near-duplicates. A duplicate canonical-tautomer InChIKey14 at slot 2 costs ~10% of that molecule's EV; at slot 25 ~0. Dedup onTautomerEnumerator.Canonicalize→MolToInchiKey[:14], not raw keys (raw InChIKey14 differs across 162/162 enumerated isoquercitrin tautomers; canonicalisation collapses them to one). - Back-solve. (recall@25 ≈ 0.55, MRR ≈ 0.417) ⇒ ~59% of hits are at rank 1 ⇒ top-1 ≈ 0.32. The ceiling is pool coverage; every prior CASMI edition without a candidate list containing the answer collapsed (2014 unknowns: 0/6 at rank 1; 2022: best automated 26–29% top-1).
- Validation design the precedents demand: leave-out by InChIKey14 AND scaffold/MCES-disjoint (MassSpecGym threshold 10); expect top-1 to roughly halve vs spectrum-disjoint (CSI:FingerID 70 → 36 Top-1 in CASMI 2016 leave-out; 58.5% → 34.4% in Dührkop 2015). The test provides multi-CE lists, both polarities, adduct given; it provides NO RT, NO CCS/mobility, NO MS1 isotopes.
- Question for q7: what is the Kaggle notebook runtime/GPU/internet rule you have verified? Our rules worker is checking the same thing; whichever side finds the official metric script or the RDKit call first, please post the URL here.
2026-10-07 — [q4_challenge_intelligence] → q7_compute_execution — VERIFIED RULES (rules worker done)
Source: research/q4_challenge_intelligence/casmi2026_rules_timeline_prizes_code_requirements_faq.md (+ .json, 48 rows with verified/mirrored/unknown status). Status letters: V = verified from Kaggle API v1 or official text; M = mirrored in GitHub write-ups only; U = unknown.
- Code competition (V): Kaggle API
isKernelsSubmissionsOnly: true;maxDailySubmissions: 5;maxTeamSize: 5; entry+merge deadline 2026-12-07T23:59Z; close 2026-12-14T23:59Z; enabled 2026-09-14; data licence CC BY-NC 4.0;awardsPoints: true. API'sevaluationMetriclabel says "Mean Average Precision" — stale; the scorer is MRR. - Official metric script (V): Kaggle notebook
metric/casmi-mean-reciprocal-rank(Kaggle Competition Metrics, 2026-09-14). Exact pipeline:Chem.MolFromSmiles→rdMolStandardize.TautomerEnumerator().Canonicalize(mol)(default params; NO salt stripping, NO charge neutralisation) →Chem.MolToInchiKey(mol)[:14]. InChIKey is computed AFTER canonicalisation. Answer key is canonicalised live at scoring and supports comma-separated multiple acceptable SMILES. A heavy-atom-composition gate skips canonicalisation for impossible candidates. RDKit pinned to 2026.03.3 via datasetmetric/rdkit-2026-3-3-wheel(cp310–cp313 manylinux); version mismatch only warns. - Submission parsing (V, from code): split on
;, tokens stripped, empty tokens dropped; >25 non-empty tokens, NaN, empty row, duplicate molecule_id, or missing column → WHOLE FILE rejected. Duplicates within a row and invalid SMILES silently consume ranks (score as zeros). Unknown ids ignored; missing ids score 0. - Runtime (M): 9 h CPU/GPU, internet OFF, MIT licence required for winners, 2 final selections, prize split $16k/$12k/$9k/$7k/$6k — mirrored in three write-up READMEs, not opened on Kaggle. Press release verifies $16k first / top five paid / "winning methods will be open-sourced".
- Decoy (V): Data page verbatim: "the file you see is comprised of examples from the training dataset; it will be replaced by the hidden test set during re-run, and will be approximately the same size". No penalty, no benefit from using it. Host clarification thread
discussion/741404cited by a mirror. - Public/private split (U): the scorer's solution file has a
UsagePublic/Private column so a split exists; no source states the fraction. Only a host post settles it. - Salt gotcha (V, local RDKit 2026.03.6):
[Na+].CC(=O)[O-]gets a different InChIKey14 than acetic acid under the official scorer — candidate pools must be parent-stripped BY US before submission. Quercetin canonicalisation ≈ 335 ms → budget canonicalisation of large pools offline, ship canonical keys as a Kaggle Dataset. - Official-text contradictions: hidden set ~1,500 spectra (Data page) vs ~2,500 (press release); ten adducts listed vs "one of the seven above" in the column table.
- Kaggle API now reports
userHasEntered: true(your 403 earlier today is gone) — the Rules/Overview pages can now be copied verbatim from a logged-in browser to close the M/U rows (notebook RAM, Rule 2.6 external-data text, eligibility, public/private fraction). Please do that and post here.
2026-10-07 — [q4_challenge_intelligence] → q7_compute_execution — LEADERBOARD + ENVEDA workers done; answers to your three questions
Files: research/q4_challenge_intelligence/public_leaderboard_and_open_writeups_oct2026.md (+ .json, lb_snapshot_2026-10-07.csv 2,703 rows, kernels_2026-10-07.csv 200 rows) and enveda_company_prism_people_motives.md (+ .json).
Your questions. (a) Rerun on hidden set only: YES by construction — the Data page says the visible file "will be replaced by the hidden test set during re-run"; corroborated by graded-output size (530,716 B hidden vs 393,908 B for the 400-row decoy, from a write-up), i.e. the hidden file has different and more molecule_ids. (b) Rerun limit: 9 h CPU/GPU, internet off — still only MIRRORED (3 READMEs + yasu notes), not read on Kaggle. (c) Public fraction: ≈ 33 % of the hidden set (~130 molecules) — SECOND-HAND, from the kitopl "data pitfalls" kernel, unattributed to a host post; nobody quotes host text. Consequence if true: one molecule ≈ 0.0076 public MRR; SE of the public score ≈ 0.04. Observed: byte-identical reruns moved 0.335→0.328 (prvsiyan) and landed 0.011 and 0.105 below advertised (nghianguyen7171). Treat the public LB as a ±0.01 instrument at best.
LB anatomy, 13:59 UTC pull (your 05:20 numbers are the same file 8.6 h older, not wrong): 2,703 teams; rank 1/5/10/50/100/250/500/1000 = 0.481/0.448/0.441/0.429/0.425/0.421/0.413/0.357; p50 0.328, p90 0.421, p99 0.432; ≥0.45/0.44/0.43/0.42/0.40 = 3/12/49/317/762. Derived medal lines (gold top-10+0.2 %, silver 5 %, bronze 10 %; rule applied from memory, verify at kaggle.com/progression): gold rank 15 = 0.436, silver rank 135 = 0.425, bronze rank 270 = 0.421. 885 teams (32.7 %) sit on 16 exact clone scores: 149 at exactly 0.425 (bobthebot369 "V18 Sovereign Zenith" replay — the silver line runs through this block, so medal vs none there is decided by submission timestamp), 74 at 0.417 (imranarif536 V44 PairTail), 106 at 0.328 (prvsiyan rerun). Top moved 0.425 (09-26) → 0.471 (10-05) → 0.481 (10-07); bronze 0.342 → 0.401 → 0.421 in 10 days; a fixed 0.417 decays ~30 ranks/day. The public pile-up is clone replay, not convergence — your "do not act on <0.01" rule stands and should be ~0.01–0.04 given the 130-molecule sample.
What the top public tree shares (14 kernels pulled, 20 READMEs): pool = train ∪ COCONUT 2.0 (~710–775k), ±10 ppm (±8.5 ppm with a claimed +1.4 ppm systematic timsTOF offset in haideptry V17; ±5 vs ±10 ppm made no difference for yasumorishima), FPNet 6,930-bit / Morgan r2+r3 4096 + MACCS, HGB or 4×LightGBM LambdaRank rerankers on 31–160 features, RRF of 2–3 engines, ICEBERG/GLACIER forward models in every ≥0.37 kernel (fixed wall-clock budgets — the 0.421 notebook uses 5,400 s; at 300 s ICE scored only 18 % of molecules), library lock at lib_max ≥ 0.9, PubChem only as ≤50-candidate gated tail (+0.011; as a pool it is catastrophic: class-2 sim 0.732→0.379). Every retrieval kernel indexes enveda-180. DreaMS appears only as tail filler with two published negatives; MIST one title; NO SIRIUS, MS2DeepScore or PRISM anywhere. The decoy is exploited only as a commit-vs-rerun detector (SHA256/MD5 of test.parquet) — a landmine if Kaggle ever swaps the placeholder.
Validation that predicted LB order (3 independent reports): only the 250 enveda-np-examples structures under three masks (shiferaxa; leesirleesirlee-star "Tier 1", paired SE ±0.002). Random held-out train structures predicted the OPPOSITE ranker order; LGB CV 0.65 → LB 0.279 (worse than no LGB). Please make the 250-NP three-mask backtest your primary selector; the LB is a sanity check only.
Enveda facts you need: enveda-180 = Enamine HLL-200 library on timsTOF Pro 2 at 20/40/60 eV (Zenodo, MassIVE MSV000102122, ChemRxiv 2026-06-04) — an instrument twin and a chemistry decoy; hidden 400 "generated on Enveda's isolation and profiling platforms", validated (only documented stack) by SIRIUS fragmentation-tree curation + NMR; three organisers (Lightheart, Taylor, Healey) co-authored the June-2026 MassSpecGym leakage audit (arXiv 2606.19624) — assume the answer key defeats canonicalisation and PubChem-order shortcuts. PRISM has no preprint, weights, or absolute numbers; use DreaMS as the open foundation-model baseline. Internal tools named in Enveda papers/job ads: MZmine, XCMS, MS-DIAL, MetaboScape, SIRIUS, CSI:FingerID + MS2Mol, DreaMS. GPU quota (30 h/week/account) is the real bottleneck per a 0.417 team ("~6 variants per week") — team merge pools it.
Still open for you (U): host wording of discussion 741404, public/private fraction, notebook RAM, Rule 2.6 text — copy from a logged-in browser. — [q4_challenge_intelligence]
serendipity_broadcast
2026-10-07 — [q7_compute_execution] → q8_serendipity and all queens
King INFER forced me the lens "jurisprudence of eyewitness line-ups". Two concrete results worth reusing:
- The public leaderboard is a biased line-up and I can now prove it is non-discriminating. Measured from the live Kaggle API today: 263 teams score ≥0.42 but only 35 score ≥0.43. 228 teams sit inside one 0.01-wide band. In line-up terms the fillers are so easy that almost every witness picks the same face. Practical rule for every queen: do not let a public-LB delta below ~0.01 change any decision.
- "Chain of custody" is a real, measurable risk here, but smaller than feared. The metric's identification
procedure is RDKit 2026.03.3 tautomer canonicalisation; our local build is 2026.03.6. I measured on
2026.03.6 that
TautomerEnumerator().Canonicalize()silently hits its defaultmaxTransforms=1000cap on polyphenols — quercetin (363 tautomers, 179 ms), rutin (357 ms), tetracycline (287 ms) all return statusMaxTransformsReached, i.e. a truncated search — while caffeine/morphine/erythromycin finish in 0.2–6 ms. A truncated search is the thing most likely to differ between builds. Good news: within 2026.03.6 the result is input-order stable (12 randomized SMILES → 1 InChIKey14 for all 8 test molecules) and equals the standard database InChIKey14, so canonicalisation is a no-op for database-derived SMILES. The exposure is therefore concentrated in de-novo-generated SMILES written in unusual tautomeric forms, not in retrieval candidates. Also note the cost: 180–360 ms/molecule for flavonoids makes tautomer-aware dedup of a million-candidate pool infeasible — dedup pools on plain InChIKey14 and reserve canonicalisation for the final 25. - Null/dud from this lens, recorded honestly: "sequential vs simultaneous line-ups" looked like a clean argument for a cascaded ranker. It is not — in the eyewitness literature the sequential-superiority effect was overturned once Gronlund/Wixted/Mickes applied ROC analysis instead of diagnosticity ratios. The lens therefore argues against the cascade, which is the opposite of what I expected. My worker is testing this properly.
Open invitation: q1/q3 — "filler selection" says a line-up is biased when the fillers do not match the witness's description. Our description is "natural product, timsTOF, one of 7 adducts". Which chemical classes would make the wrong fillers, i.e. are over-represented in training but implausible in the hidden set? — [q7_compute_execution]
Channel: serendipity_broadcast (q8_serendipity -> all queens; append-only, sign with queen_id)
2026-10-07 [q8_serendipity] Broadcast 1 — the Morelli test (low-intensity peaks vs base peak)
- FACT: every field-standard similarity flattens intensities (Stein & Scott 1994 sqrt·m/z^2-3; Li 2021 entropy-adaptive power w=0.25+0.25·S; MS2DeepScore sqrt + 0.1 % floor; Spec2Vec sqrt word weights; Hart 2024 I^0.25·freq^0.25). Li et al. 2021 Nat Methods states the rationale outright: abundant fragment ions are shared between structurally similar molecules; low-abundance ions separate them (FDR <10 % at entropy sim 0.75 vs dot product ≥15 % even at 0.95). Source file: research/q8_serendipity/art_music_philosophy_what_is_identity_of_a_molecule.md
- NEGATIVE FINDING: no published ablation removes the base peak or keeps only peaks <10 % relative intensity and reports identification change. The decisive test is ours to run (protocol in the file above: isomer-rich novel-molecule timsTOF-like hold-out, formula-exact COCONUT pool, MRR@25 + isomer-swap rate).
- SURPRISE: the one frequency weighting in use (Hart 2024) UP-weights common m/z (anti-IDF) and improved correct network edges ~30 % (2,773 -> 3,568). This is Burrows' Delta logic (judge frequent tokens against their population norm), not IDF.
- GOTCHA (for q2_data_forensics, q5, q6):
ms2_normalized_intensitiesmakes the base peak always 1.0 (information-free as a feature); noise floors differ by ingest_lib, so low-intensity features can leak library of origin -> a "Morelli" model may learn the library, not the molecule. Please confirm per-library intensity floors. - QUESTION to q2_data_forensics / q6: has anyone counted the InChIKey14 collapse of COCONUT 2.0 (695,133 isomeric structures)? No published number found; needed to dedupe the pool.
- QUESTION to q1_instrument_physics: on timsTOF with merged multi-CE spectra, what is the realistic noise floor (relative intensity) below which "minor peaks" are chemical noise rather than Morellian signal?
2026-10-07 — [q4_challenge_intelligence] reply/test for q8 (forensics_law_and_evidence_for_annotation_confidence)
King MEASURE forced the law-of-evidence lens on my subdomain. Results you can build on or rebut:
- Standard of proof by edition: CASMI 2012–2016 scored "best rank among participants" (a rank-100 answer could win); 2017 used Formula-1 points over ranks 1–10; 2022 used single-answer exact match (top-1 only). MRR@25 is stricter than 2012–2016, more lenient than 2017 at ranks 11–25, far more lenient than 2022. (casmi_history worker, casmi-contest.org results pages.)
- Line-up EV rule (tested numerically): sort by calibrated posterior; the first 3–5 slots carry ≥93% of a hypothesis's EV; hedge formula/adduct hypotheses in slots 6–25; optimal split ~13–15 slots top-formula vs 10–12 alternate when P(top formula right) = 0.5–0.8, gain +0.01 to +0.04 MRR, vanishing above 0.95. Analogy breaks: no "acquittal" cost for wrong guesses below the hit, and tautomer/stereo collapse means the same suspect can appear twice (slot waste).
- Daubert verdict on the benchmark (critiques worker): testable (narrow pass), peer review (fail), known error rate (fails worst — no published label-error rate; MRR SE ≈ 0.02), standards (partial), general acceptance (pass). Fiehn verbatim (C&EN 2026-09-17, DOI 10.47287/cen.612552.newsarticle): "They focused on mass spectra of isolated compounds, which is difficult enough."
- Question for q8: does your forensics worker have a calibration method (e.g. Wixted & Wells eyewitness confidence–accuracy) that maps to per-candidate posteriors we can sort by?
2026-10-07 — [q5_methods_lineage] reply to q8 Broadcast 1 and to q7's "line-up" note
- Reply to q7 ("the line-up lens argues against the cascade"): my simulation worker reaches the same place from a different direction. Calibrated to MIST/JESTR hit curves (400 molecules × 1,000 candidates, two scorers with shared-noise correlation ρ), rank-averaging two scorers is worse than the better scorer alone once ρ ≥ 0.3, and every independence-assuming combiner (rank, z, naive-Bayes product) stops beating the single best scorer at ρ ≈ 0.6. What does survive is summing calibrated per-scorer log-odds (+0.03–0.05 MRR@25 over rank-averaging) with per-class calibration (+0.02–0.03). So: no cascade by ranks; one jointly-fitted log-odds stacker. File: research/q5_methods_lineage/foundation_models_dreams_prism_massspecgym_benchmarks.md (§Part B, code in
*_lr_sim.py). Caveat: Gaussian equal-variance toy; magnitudes are not predictions, the ordering is robust. - The one empirical ensemble number that exists: CASMI 2016 (208 challenges, ~816 candidates/challenge): equal-weight six-method consensus Top-10 166/208 vs best single 131/208; on the 87 challenges with no training-set overlap, bond-breaking fragmenters (MetFrag+CFM 13 Top-1) matched fingerprint ML (CSI:IOKR 16, CSI:FID 13). Consensus helps when the engines are orthogonal (fragmenter vs fingerprint), not when they are two fingerprint models. File: insilico_fragmentation_metfrag_msfinder_cfmid_iceberg.md.
- Reply to q8's intensity gotcha: confirmed from the method side — MS-FINDER's score ignores intensities entirely; CFM-ID leaves ~42 % of NIST-20 peaks unannotated; ICEBERG 1/SCARF/FraGNNet train on CE-merged spectra. No published ablation of "drop the base peak" exists in the method literature either. Your Morelli test is open.
- Gotcha for everyone (from the June-2026 MassSpecGym audit, arXiv 2606.19624): a spectrum-blind baseline — PubChem default candidate order — scores 49.98 % hit@1 on MassSpecGym; a SMILES-canonicalisation mismatch between truth and decoys let a published retrieval method report 82 % that fell to 7 % when fixed. Report a spectrum-blind baseline next to every local score; if the model is within a few points of it, it learned the pool. — [q5_methods_lineage]
2026-10-07 [q8_serendipity] Broadcast 2 — cryptanalysis: cribs, unicity distance, conditional IDF
- FACT (for q6_candidate_space_retrieval, q5_methods_lineage): Shannon unicity estimate — a ±10 ppm COCONUT window (median 36, p90 204 isobars) needs ~5–8 bits from the spectrum; per-peak bits from Ma 2014 NIST11 neutral-loss likelihood ratios: H2O 0.57, NH3 2.3, hexose 3.9, H3PO4 4.4, H2S 5.0 — but informativeness is POOL-RELATIVE (hexose loss ≈ 0 bits inside a glycoside-rich window). Source: research/q8_serendipity/cryptanalysis_and_codebreaking_as_spectrum_decoding.md
- FACT: CSI:FingerID's per-bit ML score and SIRIUS's MAP tree score are already Turing-style additive log-likelihood-ratio sums (SIRIUS common-loss ratios: H2O 64.9, CH3 69.5, CO 23.9, CO2 1.41; implausible losses 1e-3). Cosine is not. Blend engines in log-odds space, not by score averaging.
- IDEA (for q6, q7_compute_execution): the "crib" pipeline — use hidden spectra with a confident library hit (score ≥ 0.9, top-2 margin ≥ 0.15; expect ~40–150 molecules) transductively to estimate the shared timsTOF "day key": ppm offset (test lacks precursor_error_ppm), CE response, and a temperature/Platt recalibration of fingerprint scores. Label-free, cheap.
- IDEA: candidate-conditional IDF, w_p = log((N+1)/(n_explain+1)) where n_explain = number of window candidates whose depth-≤2 fragments explain peak p. Reconciles Hart 2024 (global common-m/z up-weighting helps analog networking) with IDF intuition (rare-within-window peaks discriminate isomers). Expected +0.01–0.03 MRR, CI half-width ≈ 0.015 on a ~2,000-molecule scaffold-disjoint timsTOF fold.
- NEGATIVE (for q5): no deterministic structure-level eliminator with published precision/recall exists; MS-FINDER H-rearrangement rules explain only 78.4 %/84.8 % of fragment ions, so hard filters would discard 15–22 % of true answers. Hard-eliminate only on mass/adduct/formula.
- ARITHMETIC (for q7 validation design): at LB 0.417 with recall@25 ≈ 0.55, mean reciprocal rank among hits ≈ 0.76 — slots 6–25 barely move MRR; recall@25 (candidate coverage) is the lever.
- QUESTION to q1_instrument_physics: what systematic ppm offset and CE-response variation should we expect across a single timsTOF campaign (so we know whether the "day key" is worth estimating)?
2026-10-07 — [q6_candidate_space_retrieval] replies to q8 Broadcasts 1 and 2
- Answer to q8's question (IK14 collapse of COCONUT): measured on the Oct-2026 CSV-lite, 738,794 rows → 728,390 full InChIKeys → 479,690 InChIKey14; q3's worker independently got the same number. The 695,133 headline overcounts the scored pool by 1.54×. LOTUS v11 adds 254 IK14; THEOBROMA (29 sources) reaches only 486,215 IK14 families. There is one NP map, not several.
- Your unicity arithmetic checks out against my windows: ±10 ppm IK14-COCONUT windows are median 36 (5.2 bits), p90 204 (7.7 bits), max 1,761 (10.8 bits, a curation artefact at C6H5N3O). Inside a window the true formula already covers 62–90 % of structures, so formula bits are mostly spent; the remaining bits must discriminate isomers — your candidate-conditional IDF is aimed at exactly the right term.
- Test I will run for the "crib" idea: the gnshx write-up already gates PubChem injection on library confidence (< 0.7) and gained +0.011 on LB; that is a posterior update after an unsuccessful search, i.e. your crib pipeline in reverse. The clean experiment is: three-mask enveda-np-examples split, report (c, r, e) with and without a ppm-offset/Platt recalibration estimated from confident library hits only.
- Rebuttal-in-part to "slots 6–25 barely move MRR": correct for insertion (appending a foreign candidate at slots 11–25 costs ≈ 0.0001–0.0003 MRR each) but the headroom argument says the ~0.39 of mass outside every pool (Class 3) is worth up to +0.27, and those molecules can only enter through slots we are willing to spend on analogs. Cheap slots are the budget for the map extension, not a reason to ignore them. — [q6_candidate_space_retrieval]
2026-10-07 — [q4_challenge_intelligence] reply 2 to q8 / q7 / q5 — seat-filling rule from the law-of-evidence lens, now checked against ten public write-ups
- RULE (tested numerically,
public_leaderboard_and_open_writeups_oct2026.md§Forced analogy): fit the within-list hit curve f(k) ∝ k^−s to the one measured ranking-only MRR (0.765, leesirleesirlee-star) → s ≈ 2.07 (64 % of hits at rank 1, 92 % in top-5). Bayes-optimal seating of the 25 slots is simply sort every candidate from every hypothesis by posterior; block seating ("13–15 primary then hedges", my earlier rule) loses 0.01–0.03 EV. The first hedge (alternative formula/adduct/PubChem isomer) belongs at seat 2 unless P(primary) ≥ 0.90; strict alternation at P ≈ 0.5–0.7. This corrects my earlier "~13–15 primary slots" line: that was the block-optimal m, not the optimum. - CHECK: across 10 write-ups with both MRR and recall@25, within-hit MRR 0.75–0.95 is consistent; the LB rank-1 share back-solved from 0.481 ≈ 0.35 matches an independent 0.30–0.35. So the missing half of every score is coverage (recall@25 ≈ 0.545 at 0.417), agreeing with q8's arithmetic — slots 6–25 are nearly free, but seat 2 is not.
- FOR q5 (log-odds stacker): public tree uses RRF (α 0.6, k 3) of 2–3 engines + LambdaRank; nobody stacks calibrated log-odds; your +0.03–0.05 is untested in the wild. For q7's line-up lens: the public LB is 32.7 % clone replays on 16 exact scores — a line-up where a third of witnesses are reading the same cue card.
- DUD: "burden shifts to the defence" has no analogue — wrong guesses cost nothing; the only cost is a dead seat from a duplicate canonical InChIKey14 (2.3 % of COCONUT keys change under the scorer's canonicalisation; one team measured −0.105 on that subset).
- QUESTION for q8 (forensics calibration): we now have a within-hit curve shape; what we lack is a per-hypothesis P(formula correct) calibrator with no MS1 — any analogue to confidence–accuracy curves for a formula claim? — [q4_challenge_intelligence]
2026-10-07 — [q5_methods_lineage] reply to q8 Broadcast 2 (cryptanalysis) — one test, one caution
- Your "crib" day-key idea has a free validation set: 2,538 train structures were measured on both timsTOF and non-timsTOF instruments (my failure-modes worker's local audit of train.parquet). Fit the ppm offset / CE-response / Platt-temperature recalibration on their timsTOF spectra and check that it transfers — it is the only labelled timsTOF-vs-other pairing on disk. File: research/q5_methods_lineage/failure_modes_generalisation_instrument_shift_evaluation_pitfalls.md §8 ("P-inst").
- Caution on candidate-conditional IDF: the lineage's own numbers say the inside-window entropy is lower than your Shannon estimate assumes. On the ±10 ppm COCONUT window 70.8 % of candidates already share the molecular formula, so formula-level cribs (hexose loss, H2O) are ~0 bits exactly where the ranking is decided; and formula-free retrieval tops out at ~27 % hit@1 even on the audited MassSpecGym (arXiv 2606.19624). Your IDF has to be computed on structure-level fragment explanations (MetFrag depth-2, ICEBERG DAG), not on neutral-loss formulas — which costs the CPU budget my fragmenter worker priced (~0.02–0.1 s per candidate for MetFrag, 0.003–0.4 s/molecule GPU for ICEBERG 2/GLACIER).
- Agree on log-odds fusion, with one measured limit: in my simulation every independence-assuming combiner (including the Turing-style LR sum) stops beating the best single scorer once scorer errors are ≥ 0.6 correlated; two fingerprint models are typically in that regime, a fragmenter + a fingerprint is not. — [q5_methods_lineage]
2026-10-07 — [q2_data_forensics] replies to q8 Broadcast 1 and to q7's "wrong fillers" question
- q8 intensity-floor gotcha: CONFIRMED, computed from the official train.parquet. Minimum non-zero
ms2_normalized_intensitiesis a library barcode: MassBank 1/999, MS-DIAL 0.01, RIKEN 0.01/0.005, pluskal_ms2 ~7e-5, timsTOF (enveda-180/np-examples) no floor down to 3e-8.base_peak_intensityis null in exactly the 7 public libraries and present in the Enveda slices. A tree model finds either in one split. Also: enveda-180 merged 20/40/60 eV rows are per-m/z sums of raw intensities (merged base peak = sum of the three singles' base peaks, median ratio 1.00), median 258 vs 116 peaks, floor 3.5x lower than singles — so "minor peaks" in a merged test row are partly a merging artefact, not Morellian signal. 46 % of enveda-180 spectra have peaks above precursor m/z + 0.5 Da. File: research/q2_data_forensics/adducts_polarity_collision_energy_merging_statistics.md. - q8's COCONUT IK14 collapse question: 738,794 rows -> 479,690 unique InChIKey14 (standard key, not tautomer-canonicalised); q6 got the same number independently. Train ∩ COCONUT = 26,111 IK14 (9.47 %); enveda-180 contributes 51. File: research/q2_data_forensics/_coconut_overlap_and_np_examples_keys.json.
- q7's "wrong fillers" question (which chemistry is over-represented in train but implausible in the hidden set): the 400 visible-test decoys are a statistically random draw of enveda-180 (NP_Score mean -1.47 vs -1.45 for random draws, 0/400 in COCONUT, 36 % halogenated, Enamine amide/triazole space), whereas np-examples (the only NP-on-timsTOF slice) has NP_Score +1.31 and is 52.8 % nitrogen-free. So the wrong fillers are: halogenated (F/Cl/Br: 51,421/29,848/8,522 train structures), sulfonamide/amide-rich, N-rich drug-like compounds in the 245-460 Da band. Only 1.2 % of random enveda-180 structures have NP_Score > 0. File: research/q2_data_forensics/test_set_decoy_hidden_set_inference_and_domain_shift.md.
- Rebuttal to the "day key" crib idea (q8 Broadcast 2): the shipped
precursor_error_ppmcannot calibrate it (absolute, electron-mass-free, r = -0.02 with true error), but the recomputed enveda-180 error shows a +1.9 ppm positive-mode offset and a hard 5.13 ppm truncation from Enveda's extraction pipeline, while np-examples (another batch) reaches 21 ppm. A per-campaign offset is real and worth estimating from confident library hits; the truncation is not something the hidden set will share unless the same extraction code was used. — [q2_data_forensics]
2026-10-07 [q8_serendipity] Broadcast 3 — replies to q4/q5/q7 + open-set, objectives, popularity-prior inversion
Reply to q4_challenge_intelligence (calibration -> per-candidate posteriors). Yes, three pieces, all in research/q8_serendipity/forensics_law_and_evidence_for_annotation_confidence.md and healthcare_diagnosis_astronomy_ecology_other_rank_25_problems.md:
- The posterior to sort by is P(H_i|S) = π_in·w_i·LR_i / [π_in·Σ_j w_j·LR_j + (1−π_in)], where π_in = probability the truth is in the pool (~0.55 at LB 0.417), w_i = source prior (COCONUT/LOTUS membership, NP-likeness), LR_i = calibrated per-candidate likelihood ratio. Ordering is invariant to pool size (Balding & Donnelly 1996); the posterior magnitude is not — a "strong" LR of 10^3 against n_f = 10^4 same-formula isomers is posterior ≈ 0.09. Never threshold a raw score without n_f.
- How to get calibrated LRs: COSMIC-style (Hoffmann 2022 Nat Biotechnol 40:411; AUC 0.82 vs 0.40–0.55 for raw MetFrag/CFM-ID/CSI scores): Platt-scaled classifier on (score, score gap to runner-up, log n_f, explained intensity) trained on correct-vs-wrong candidate pairs — and the line-up rule is that the "fillers" used to calibrate must match the description (same formula, same adduct, NP-like), or the score gap just measures a weak pool.
- Wixted & Wells 2017's "pristine conditions" rule transfers literally: only FIRST-PASS confidence (untouched spectrum, no test-time adaptation, no LB-steered reranking) is diagnostic. Use it to pick the two final submissions.
- Two model-agnostic confidence features from other fields: RDP-style bootstrap (resample 1/8 of peaks ×100, fraction of resamples agreeing on top-1; Wang 2007 / Murali 2018) and the energy score — unnormalised log-sum-exp of top-25 candidate scores instead of a softmax probability (Liu 2020: FPR95 51.04 % -> 33.01 %).
Reply to q5_methods_lineage. Agree: log-odds stacker, not rank cascade. Two new numbers that support it from the MT side (linguistics file): (i) the training objective closest to MRR@25 is softmax cross-entropy over the formula/mass-matched candidate set (candidate InfoNCE = ListNet with one relevant item): MSAlign 2026 reports 47.5 % R@1 vs 19.5 % for in-batch InfoNCE on the SAME encoder; (ii) harder-than-pool negatives do NOT help (De Waele 11.64 vs 12.29; JESTR regularisation 0 gain on MassSpecGym) — spend effort making the training pool look like the test pool instead. (iii) MERT-style tuning of fusion weights directly on dev MRR: MSAlign fusion 47.5 -> 58.2.
Reply to q7_compute_execution ("which fillers are wrong"). One filler class I can name with a number: WELL-STUDIED molecules. MetFrag relaunched (Ruttkies 2016, 473 Eawag spectra) gets its headline gain from literature popularity — top-1 30 -> 336 with PubChem reference counts -> 420/473 (89 %) with ChemSpider references. The hidden set is "none previously published", so that prior is INVERTED here; a ranker trained on GNPS/MassBank learns it implicitly (training-spectrum count correlates with how well-studied a scaffold is). Audit: correlate held-out score vs. number of training spectra of structurally similar compounds.
New facts for everyone (confidence medium unless stated):
- De novo SMILES decoders score 0.00 % exact match on MassSpecGym (structure-disjoint); a SMILES transformer is WORSE than random on MCES (53.8 vs 28.6). Best de novo with formula given: MS-GPT 23.91 % / DualLGD 23.89 % top-1 — still < half of retrieval. Use de novo only as MBR-style consensus filler (sample, group by InChIKey14, rank by frequency) for slots behind DB candidates, and only where the pool is empty or triaged "C".
- Open-set numbers exist in eDNA taxonomy (Murali 2018 Microbiome 6:140): with the true genus deleted from the reference, fraction still forced into a (wrong) genus = IDTAXA 0.01 %, RDP 3.83 %, SINTAX 8.76 %, BLAST top-hit 17.3 %; BLAST invented 65 spurious genera, IDTAXA 2. Naive library search = our BLAST top hit. Abstention has a coverage price (IDTAXA 19.9 % vs RDP 22.5 % of reads assigned) — abstaining WITHOUT a generative/analog branch is strictly worse than not abstaining under MRR@25.
- The one validation artefact nobody in the public write-ups has: a scaffold-disjoint NP hold-out (~1,000 molecules) where the true structure is DELETED from the pool for a random half. It is the only way to measure a truth-absent detector and to report conditional vs unconditional MRR@25. q7 — please consider owning this fold.
- CSLS hubness correction (Conneau 2018): +2.1 to +6.5 P@1 on cross-modal retrieval; never applied to spectrum->structure embeddings; a few lines to implement; expect small gain, test inside the largest isomer families.
- Morelli test update: Li 2021's rule is I^w, w = 0.25+0.25·S for entropy S < 3, w = 1 ABOVE — they report the transform REVERSES on ion-rich spectra. Merged multi-CE timsTOF spectra are ion-rich. So test entropy-adaptive weighting on our data; do not assume the NIST20 result transfers. (Question to q1_instrument_physics stands: noise floor on merged timsTOF spectra.) — [q8_serendipity]
2026-10-07 — [q1_instrument_physics] replies to q8 (Broadcasts 1 & 2) and to q7's filler question
q8 B1 — "realistic noise floor on merged multi-CE timsTOF spectra": measured on enveda-180 (same pipeline as the hidden test). The lists are raw integer-count centroids: I_rel × base_peak_intensity is an integer for 100 % of 51 M peaks, hard floor exactly 9 counts, no relative threshold, no peak cap. Consequences: 91.8 % of peaks are < 1 % of base peak, 75.1 % < 0.1 %, 39.8 % < 0.01 %; peak count follows abundance (Spearman +0.71 with bpi; bpi p5/p50/p95 = 2,543 / 201,708 / 14.3 M counts). So there is no fixed relative floor: 20 counts is 1e-4 on a median spectrum but 0.8 % on a weak one. For the Morelli test, define "minor peak" in absolute counts (≥ 20–50 counts, recoverable because base_peak_intensity is in test.parquet) and first exclude the deterministic non-Morellian peaks: precursor (45.9 % of spectra), M+1 at +1.0034 (38.7 %), M+2 (21.7 %), and 13C satellites on 71 % of strong fragments — these are instrument, not identity. Merged spectra (24.6 % of test) have ~2× the peaks of singles (median 258 vs 88–132) and are count-weighted toward the 20/40 eV scans (~0.5/0.4/0.1). Your "sky" concern is confirmed: recurring m/z in enveda-180 are library chemistry (pyridinium 80.05 in Enamine-like synthetics vs furanyl 81.03 in NPs), so frequency weighting learned on train is partly a library fingerprint. File: research/q1_instrument_physics/noise_artifacts_isotopes_insource_chimeric.md.
q8 B2 — "systematic ppm offset and CE-response variation across a timsTOF campaign (is the day key worth estimating?)": CE response is not a nuisance to estimate — it is fixed 20/40/60 eV lab-frame on enveda-180 and test (not the 1/K0 ramp), and the merge rule is a deterministic count sum; the one learnable thing is the eV→NCE map for Orbitrap-trained models (timsTOF 20 eV ≈ NCE 20, 40 eV ≈ NCE 45, 60 eV ≈ NCE 60, from 2,421 same-molecule pairs). ppm: precursor |error| median 0.98 ppm on enveda-180 (train EDA), 99 % < 4.4 ppm, errors correlated within a run (consensus over the 1–16 spectra of a molecule does not shrink as 1/sqrt(n)). The one systematic offset worth a key is on fragments: below m/z 150 enveda-180 fragments read +0.3–0.6 mDa (+4–6.5 ppm) high vs ≈ 0 on Orbitrap, vanishing by m/z 180. Verdict: estimate a mass-dependent fragment offset (cheap, deterministic); do not spend crib effort on a CE key. File: cid_fragmentation_physics_collision_energy.md.
q7 — "which fillers mismatch the description": from the physics side, the wrong fillers are molecules whose ion type cannot appear: train has 29 % dimer rows ([2M+Na]+ 190,033, [2M+H]+ 92,041, [2M-H]- 56,172) that never occur in the 7/10-string test vocabulary, plus multiply charged rows ([M+2H]2+ 1,717). Chemistry-side fillers are q3's call; note only that all 1,154,969 timsTOF train rows are Enamine-like synthetics (enveda-180) plus 1,184 NP rows, so "timsTOF" is itself a chemistry label in train. — [q1_instrument_physics]
2026-10-07 — [q3_natural_product_chemistry] reply to q8 Broadcast 1 question, q7's "wrong fillers" invitation, and King MEASURE's music lens
- Answer to q8's question "has anyone counted the InChIKey14 collapse of COCONUT?" — yes, now, on the local Oct-2026 dump: 738,794 isomeric entries → 479,690 metric keys (35 % spellings). By superclass: macrolides 59 %, steroids 54 %, glycosides 47 %, flavonoid hexoside formulas ~68 %, aglycone formulas 2–5 %. Train is stereo-flat (0 of 277,566 SMILES carry stereo), has 0 label noise, and the grader's canonicaliser merges 1,541 groups of train inchikey14 (58,954 spectra) and splits 11 skeletons. File:
research/q3_natural_product_chemistry/beyond_2d_tautomers_stereo_isomers_what_the_metric_cannot_see.md. - Answer to q7's "which classes are the wrong fillers": (i) COCONUT's alkaloid pathway (23 % of COCONUT vs 11–13 % of LOTUS/NPAtlas) with mean NP-likeness 0.07 — bulk-imported N-heterocycles; (ii) 44.6 k enumerated triacylglycerols (fatty-acid pathway MW median 807 Da, outside the 245–460 window anyway); (iii) in train, the whole timsTOF stratum is enveda-180 Enamine synthetics (23 % F, 28 % S) — the instrument-matched fillers are chemically the least plausible. Filler-matching the description means: window ∩ NP-likeness > 0 ∩ no F/sulfonamide, then sample by NPClassifier pathway at LOTUS plant proportions.
- Music lens, tested (King MEASURE forced "timbre as fragmentation pattern"). Measured on 142,728 stratified train spectra (±5 mDa, ≥5 %): neutral losses behave like pitch-invariant overtone ratios for sugars (hexose 162.0528 LR+ 46, deoxyhexose 40, pentose 22, glucuronide 176.0321 LR+ ~270, C-glycoside cross-ring 120.0423 LR+ 66–76) and diagnostic ions like formants for a few alkaloids (tropane 124.1121 LR+ 42, desosamine 158.1176 LR+ 369). Duds: water/C5H8 ladders for terpenoids and steroids (LR+ ≤ 2.6), coumarin/lignan ions (LR+ 2–9), peptide immonium ions in this train (LR+ 4–10; 120.0808 collides with the pyrrolizidine necine ion); the brief's 152.0106 radical RDA ion did not reproduce (LR+ 1–2). Where the analogy breaks: "chords" are not additive — attaching a sugar roughly halves the flavonoid RDA ion rate (0.13 → 0.08); the 162 loss fades with precursor mass (0.33 at 400–500 Da → 0.02 at 800–1,200 Da) and with CE (>50 eV: 0.036); the sugar's own note is silent in positive mode (lost as a neutral); 86 % of hexoside [M+H]+ spectra show no 162 loss at ≥5 %, so absence is never evidence (LR− 0.6–0.9). Use rules as promote-only log-LR priors inside the formula window; 19-rule SMARTS table in
class_specific_fragmentation_rules_glycosides_terpenes_alkaloids_polyketides.md§8. - Enharmonic rule for the 25 slots: one slot per metric key, never two spellings; positional isomers ("semitones": 3-O vs 7-O, rutinoside vs neohesperidoside, oleanane vs ursane) are the right use of slots 2–10 — MS2 hears them only by intensity ratios (ViT on 17,615 GNPS flavonoid-glycoside spectra: 84.7 % positional accuracy, F1 8-C 0.72), and the test's multi-CE merge erases the energy-breakdown dimension that the best isomer papers needed.
- Gotcha for q1/q2/q5: the sugar/flavonoid/saponin timbre is essentially absent from train's timsTOF rows (27 [M+H]+ and 21 [M−H]− terminal O-hexoside spectra in 1.16 M; zero strict-SMARTS oleananes). Class-level fragmentation behaviour is learned on Orbitrap/QTOF CE calibration and met on timsTOF for the first time in the hidden test — a candidate for q8's "crib" transductive recalibration. — [q3_natural_product_chemistry]
The emperor's charter and decision log
EMPEROR — charter and decision log
The Emperor does not research and does not manage content. The Emperor manages the shape of the hive: how many kings, which queens exist, how many workers each gets, which queens may talk to each other, and how much context each level is allowed to carry. The Emperor optimises for (a) context-window hygiene — nobody reads more than they need, (b) coverage — no blind spots, (c) serendipity pressure — every level is forced across domain boundaries, (d) finishing — the dashboard ships.
Standing rules of the structure
- Fan-in discipline. A worker writes one topic file. A queen reads only her workers' TL;DRs +
frontmatter (not bodies) and writes
research/<queen_id>/_QUEEN_DIGEST.md(≤ 1,500 words). A king reads only queen digests and writesresearch/_meta/<king_id>_KINGLOG.md. The Emperor reads only king logs andresearch/_meta/coverage.json. Nobody re-pastes the brief. - Serendipity is mandatory, not optional. Every worker frontmatter has a
serendipity:line. Every queen digest has a "Forced analogies" section. Every king log has a "Cross-domain transfers" section with at least one idea that came from outside chemistry/ML. Kings rotate the lens list instructure.json. - Stagnation rule (king-level). If a queen's second round yields < 2 new concrete, sourced facts, the king reassigns her to a different subdomain or to a random field (art, law, philosophy, healthcare, ecology, astronomy, cryptography, music, sports analytics) with the instruction "find what this field knows about ranking a true answer among look-alikes".
- Channels are explicit. Queens may only write to channels listed in
structure.json. The Emperor opens or closes channels based on observed need (a queen asking a question in her digest that another queen can answer ⇒ open a channel; a channel with no traffic after a round ⇒ close it). - Restructure triggers (Emperor-level). Split a king when its log exceeds ~3,000 words or it manages > 5 queens. Merge queens whose digests overlap > 50 % in tags. Add a queen when a king log names an uncovered subdomain twice. Kill a worker topic when two rounds yield nothing new.
- Every tick ends with artifacts on disk. No silent ticks.
Tick 0 — 2026-10-07 (initial structure)
Observation. Fresh project. The user's template referenced a protease / docking; the actual challenge is small-molecule identification from LC-MS/MS (Enveda CASMI 2026, Kaggle). The "protein" slot of the template is therefore replaced by two subjects: the molecules (natural products and analogs) and the measurement (timsTOF ESI-CID MS/MS). The user's requested depth ("down to the subatomic level") maps onto: ion physics → fragmentation chemistry → spectra → libraries → candidate structures → metric.
Decision. Two kings, split at the problem's natural seam:
king_measure— molecule → spectrum (physics, chemistry, data, institution). 4 queens × 5 workers.king_infer— spectrum → molecule (methods, candidate space, compute/execution, serendipity). 4 queens × 5 workers.
Why two and not one: a single king holding 8 queen digests (~12k words) plus the brief plus its own cross-domain duties would spend its context reading, not deciding. Why not three: the third natural seam (the challenge as an institution) is thin enough to be one queen (q4) under MEASURE.
Why a dedicated serendipity queen (q8) and a serendipity mandate on everyone: the mandate guarantees breadth; the queen guarantees depth — someone has to actually read the cryptanalysis / translation / evidence-law literature rather than gesture at it.
Channels opened: data×candidates, physics×methods, rules×execution, chemistry×candidates, serendipity broadcast. Rationale: these are the four seams where one queen's facts change another queen's conclusions.
Context budget per level (target): worker ≤ 1 topic; queen ≤ 5 TL;DRs (~2.5k words in); king ≤ 4 digests (~6k words in); emperor ≤ 2 king logs (~5k words in).
Post-research roles: scribe_narrative (synthesis + dashboard content), predator_citations (audit).
Next Emperor tick happens when both king logs exist. Checks: coverage gaps named by kings; overlap between q5/q6 (likely); whether q8 produced anything a modeler would actually try; whether q4 could reach the Kaggle leaderboard/discussion (browser offline) — if not, open a user request.
Tick 1 — 2026-10-07 (both king logs received)
What happened. 40/40 worker files (≈150 k words, 7–32 sources each), 8/8 queen digests, 2/2 king logs. A session rate limit killed both kings and six queens mid-flight; resume-mode re-dispatch finished only the missing workers. The box ran out of disk twice (C: 0 bytes, D: 1.1 GB) and CPU hit 100 % from other sessions' jobs; a resource rule was added to queen/worker prompts and concurrency capped at 3 workers per queen. Stagnation rule: not triggered anywhere (every queen ≥ 10 new concrete facts, most computed on the real parquet or pulled from the Kaggle API).
Convergent findings (both kings, independently).
- MRR@25 ≈ c·r·e with c (truth in pool) ≈ 0.55–0.61 the only large lever; ~39 % of hidden molecules are in no database. COCONUT collapses to 479,690 metric keys; NP unions add nothing; PubChem wholesale costs −30 %.
- The only validation that predicted public-LB order is the 250 enveda-np-examples structures under three masks (train-key removed across all libraries). Random train splits predicted the opposite ranker order.
- The hidden instrument: fixed 20/40/60 eV, merged = raw-count sum; train's eV over-states Orbitrap NCE ~1.4×; peak lists are raw integer centroids (floor 9 counts) with precursor/M+1/13C artefacts absent from public libs.
- Fuse by summed calibrated log-odds; rank-averaging and rank cascades lose (q5, q7, q8 agree). Popularity priors invert on a never-published test. Formula-first hard filtering is a net loss.
- Public LB: ~2,700 teams, 228 within 0.42–0.43, SE ≈ 0.04 on ~130 public molecules → Δ < 0.01 is noise.
Structure decisions (→ structure.json v2).
- OPEN channel
coverage_truth(q2, q6, q8): π_in — np-examples ∩ COCONUT under RDKit 2026.03.3 — is the single most load-bearing number and three queens estimate it three ways. - OPEN channel
validation_ownership(q2, q4 → q7): q7 owns the truth-absent hold-out (≈1,000 scaffold- and IK14-disjoint NPs, analogs removed at MCES < 10 / Tanimoto ≥ 0.7, truth deleted from the pool for a random half, test adduct × n-CE mix; plus the 250 np-examples three-mask set as the primary selector). q2 ships the fold spec (heldout_A 1,977 structures / 13,556 spectra; heldout_B 293). - MERGE q1
noise_artifacts_*+ q2adducts_polarity_*→ one next-tick workertimsTOF_ground_truth_of_the_shipped_numbersunder q2 (merge semantics, ppm, trimming, intensity floor) with the "epidemiology of measurement bias" lens. - MERGE q5
denovo_*into q6coverage_ceiling_*("ways to raise c": gated PubChem, analog enumeration, de novo as Class-3 slot filler). - SPLIT q5
failure_modes_*into (a)benchmark_leakage_audit(hive QC; stays in q5) and (b)physical_shift_ce_instrument_adduct(moves to the physics×methods seam, owner q1). - KILL for future ticks: q3
np_classes_biosynthesis_*, q4casmi_history_*(complete, textbook from here), PRISM research (no weights/code/preprint), "bigger NP union" (closed), SuperNatural/NPASS/KNApSAcK. - NEW workers next tick: q2
enveda_np_examples_characterisation(250 structures by class/mass/CE; the one instrument-matched NP set and nobody described it); q3negative_mode_np_fragmentation_timsTOF(16 % of test is [M−H]−, all our LRs are positive-mode); q4licences_and_release(data CC BY-NC, NPAtlas NC, 3DMolMS NC-SA — the winning method must be open-sourced); q7p_inst_timsTOF_evaluation(2,538 dual-instrument structures: the first real timsTOF evaluation of any learned method; runs on Explorer, not this box). - RESILIENCE: queens write a
_QUEEN_DIGEST.mdstub immediately after dispatch and overwrite at the end. - RESOURCE: all parquet computation serialised (one job at a time on this box); the ~50 CPU-h RDKit 2026.03.3 canonicalisation job goes to Explorer. Prompts updated.
- HUMAN TASK (not an agent): a logged-in read of Kaggle Rules §2.6, Overview, discussion 741404, public/private
fraction, notebook RAM/GPU/9-h limits, copied verbatim into
research/_meta/kaggle_verbatim.md. Four queens hit the reCAPTCHA wall. → ask the user (or connect the Chrome extension).
Open contradictions carried forward. 7 vs 10 test adducts; 1,500 vs 2,500 hidden spectra; ±10 ppm vs asymmetric −4/+5 ppm window; +1.9 ppm positive-mode bias mechanism; np-examples ∩ COCONUT 99.6 % vs 8/250; where to trim above the precursor (+2.12 vs +2.00 Da); pool-size penalty monotone vs CASMI-2016 non-monotone (stratify by formula).
Post-research launched this tick: scribe_narrative (hive/prompts/scribe.md) and predator_citations
(hive/prompts/predator.md), in parallel — two agents only, because the box is shared.
Context accounting. Emperor read: 2 king logs (~5 k words) + 1 EDA file. Did not read queen digests or worker bodies. Kings read 4 digests each; queens read ≤ 40 lines per worker. Fan-in discipline held.
Next tick after Scribe + Predator return: quarantine any file the Predator flags; redeploy; then decide whether tick 2 research (the five new workers + two merges) runs before or alongside the modeling phase. Recommendation: modeling starts now on the two cheapest, highest-value experiments the hive specified — the mock-witness test (ranker on precursor/formula/adduct only, no MS2; if it beats H₂₅/n the pool is biased) and the np-examples three-mask backtest harness — while tick-2 workers fill the gaps.
Tick 2 — 2026-10-07 11:15 (research phase closed)
Scribe delivered research/_synthesis/{narrative.md, expert_synthesis.md, glossary.json (72), facts.json (60)}; eight
chapters, every number traced to a hive file, fourteen contradictions carried openly in synthesis §8. Corrections to the
Emperor's draft accepted (dark-metabolome figure, 2022 "38 %" → 26 % on all 222, top LB 0.481, "thousands share a mass"
→ median 36 candidates at ±10 ppm). Predator audited 33 claims: 29 verified / 4 imprecise / 0 unsupported / 0
fabricated; no quarantine; four one-sentence fixes queued for tick-3 workers (Enveda-180 DOI resolves at /v1; LOTUS
Zenodo version 10; Kaggle RAM/vCPU figures need a primary source; "RDKit absent from Kaggle image" needs a primary source).
Decision. Research phase v1 is complete and deployed. Modeling phase opens with the two cheapest decisive experiments
the hive specified: (1) the mock-witness test (ranker on precursor m/z + formula + adduct only, no MS2, vs H₂₅/n) and (2)
the np-examples three-mask backtest harness as the single primary selector. Tick-3 research (structure.json
tick2_plan: five new workers, two merges, one split) runs alongside, serialised, ≤ 3 workers at a time, and only when
C: and D: each have > 3 GB free. Compute for anything heavier than a CPU-hour goes to Explorer.