Measure/The data

Provenance and custody of the 11 train.parquet source libraries (GNPS, MassBank, MoNA, RIKEN, MSnLib, Spectraverse, MS-DIAL, drug_plus, masaryk, enveda-180, enveda-np-examples)

expertconfidence mediumrelevance 4/538 sources4.7k words
Forced analogyForensic chain of custody: treat each ingest_lib as an exemplar collection that must be vetted before a match is admissible. Partially pays off — it yields a per-source custody score and a 'source-disagreement' label-noise test, but breaks on the point that spectral libraries fail by honest mislabeling at deposition (one hop), not by tampering along a chain, and custody says nothing about instrument shift, which is the bigger problem here.

Provenance and custody of the 11 ingest_lib sources in train.parquet

TL;DR. train.parquet is 2,539,608 spectra from 11 sources, but 45 % of it (enveda-180) is instrument-matched synthetic chemistry, ~21 % (MSnLib/pluskal_ms2) is Orbitrap data from commercial screening libraries, and the remaining ~34 % is a layered set of public/community libraries (RIKEN, GNPS, MassBank, MoNA, Spectraverse, MS-DIAL) that contain copies of one another. Published audits put GNPS-style community libraries at roughly 1–2 % wrong 2D structure, ~6–7 % precursor/adduct inconsistency and, by spectrum count, >30 % "not explainable by the labelled formula" — but those bad spectra concentrate in a few hundred structures. The train file ships precursor_error_ppm uncleaned (62,834 rows beyond 20 ppm, 4,286 null) while the hidden test had precursor-inconsistent spectra dropped, so the cheapest custody filter is one the host already applied to the test and did not apply to train.

Environment note: no local train.parquet (Kaggle API 403). Every per-library count below comes from GitHub-mirrored audits of the actual file (chiparon/enveda-casmi26 artifacts/data_audit.json; mxm-1x/Enveda-CASMI-2026 DATA_AUDIT.md, 29 Sep 2026; Clyde0513/CASMI-2026 reports/data_profile.json, 16 Sep 2026) and from a mirror of the official Kaggle data page (dalloliogm/kaggle_competitions references/data_description.md; matsuken92.github.io/casmi.html). Three independent file audits agree with each other on all 11 counts; the official page differs from the file on exactly one row (enveda-np-examples, see Gotcha 1).

1. The 11 sources: counts and what each one is

ingest_lib Spectra (file audit) Unique InChIKey14 Spectra / structure Share of train Official one-line description (Kaggle data page, mirrored) Count status
enveda-180 1,153,785 182,941 6.3 45.4 % Enveda's published dataset, same Bruker timsTOF as test; synthetic drug-like screening compounds verified (3 audits + page)
pluskal_ms2 527,581 46,821 11.3 20.8 % MSnLib (Pluskal lab): Orbitrap ID-X, commercial screening/bioactive libraries, one protocol, multiple NCEs verified
riken 347,171 15,892 21.8 13.7 % RIKEN public libraries, plant specialised metabolites verified
gnps 220,849 45,750 4.8 8.7 % GNPS community libraries, largest NP collection, most heterogeneous verified
massbank 101,727 9,180 11.1 4.0 % MassBank (EU/JP consortium), curated, many labs/instruments verified
mona 92,416 11,681 7.9 3.6 % MassBank of North America, Fiehn lab, community-hosted verified
spectraverse 50,933 9,631 5.3 2.0 % Recent harmonised aggregation of public libraries incl. obscure ones verified
msdial 40,765 9,127 4.5 1.6 % Public collections distributed with MS-DIAL, multi-instrument verified
drug_plus 2,545 2,539 1.0 0.1 % Pharmaceutical LC-MS/MS database, ~1 spectrum per drug, no CE metadata verified count; identity of the upstream database unverified
enveda-np-examples 1,184 (page says 1,151) 250 4.7 0.05 % 250 common natural products on the test instrument and pipeline file 1,184 verified by 3 audits; page 1,151
masaryk 652 416 1.6 0.03 % Small library of chemical standards from RECETOX (Masaryk University) verified

Sum of unique structures = 346,428 vs 275,810 distinct in train, so 70,618 structure-library pairs are repeats of a structure seen in another source (20 % redundancy at structure level). Ionisation: 1,899,640 positive / 639,968 negative. Nulls: base_peak_intensity 1,163,138; collision_energy_orig 320,464; instrument_type 33,053; precursor_error_ppm 4,286 (Clyde0513 profile).

Enveda's own internal catalogue (enveda/enveda-180 repo, notebooks/create_figures/01_mk_prepdata.ipynb) lists its Databricks sources as casmi22, drug_plus, gnps, gnps_drug, masaryk, massbank, mona, ometa, pluskal_ms2, riken — i.e. the Kaggle file omits casmi22, gnps_drug and ometa and adds spectraverse, msdial and the two enveda sets. That casmi22 exists internally but not in train is itself a hint about what the organisers consider test-adjacent.

2. Per-source deposition process, standards, and audited error rates

enveda-180 (Krettler, Kind, ..., Healey 2026; Zenodo 10.5281/zenodo.20436851, v1.0.0, 29 May 2026)

  • Zenodo record: 1,158,293 curated MS/MS spectra, 184,330 unique compounds, median 6 spectra/compound, 14 adducts, Bruker timsTOF Pro 2 with heated ESI, both polarities, "precursors fragmented at three collision energies". QC statement: "multi-criteria curation" combining fragmentation scoring and integrity validation; "approximately 91.4 % exceed 70 % explained-intensity thresholds", median spectral entropy 0.44.
  • Repo code (data_processing/curation/constants.py): TOP_N = 20, TOLERANCE_DA = 20 for library matching; exported QC columns include sirius_score, explained_intensity, median_mass_error_fragment_peaks_ppm, mass_error_precursor_ppm. SIRIUS is run externally and merged in. Compounds are Enamine REAL-space purchases pooled in wells ("14 enamine batches"); the pipeline does targeted extraction per compound across 17 adduct hypotheses, i.e. the label is assigned by expected mass, not by an orthogonal identity check.
  • Train has 4,508 fewer spectra and 1,389 fewer structures than the Zenodo release (dedup/standardisation; unverified).
  • Structure verification: purchased compounds, vendor-declared identity; no NMR; purity not reported in the sources I could open (the ChemRxiv preprint 10.26434/chemrxiv.15004319 returned 403 both via DOI and API, so acquisition details beyond Zenodo are unverified here; q2 sibling enveda180_and_np_examples_anatomy owns the deep dive).
  • Chemistry overlap with COCONUT ~0.03 % (BRIEF).

pluskal_ms2 = MSnLib (Brungs, Schmid, ..., Pluskal, Nat Methods 2025, doi 10.1038/s41592-025-02813-0; PMC12510872; Zenodo 10.5281/zenodo.21105617 v8, 1 Jul 2026)

  • Orbitrap ID-X, flow-injection DDA, robotic pooling of 10 compounds per well, ~3,000 substances/day. Paper: 7 commercial libraries, 30,008 unique compounds, ">2.3 million MSn and >357,000 MS2 spectra", "87 % coverage" of ordered compounds; ~14,300 detected in both polarities, >12,200 positive-only, >3,400 negative-only. Stepped HCD 20/60 (MS2) and 20/40/60 (MSn). Zenodo v8 now lists 9 libraries incl. MCEDIV_50k_Sub (20,000 cpds) and TargetMolHTSNP (2,175 NP-like cpds) — consistent with train's 46,821 structures exceeding the paper's 30,008.
  • QC in mzmine: precursor-purity score, chimeric spectra flagged, noise removal below 2.5× lowest signal, minimum intensity 6e5 (pos) / 2e5 (neg). Merged spectra exist in the release ("pseudo MS2", SAME_ENERGY, ALL_ENERGIES); MassSpecGym explicitly removed all MSnLib merged spectra before use.
  • Structure verification: vendor identity (MCE, Enamine, OTAVA, NIH NPAC); no orthogonal confirmation; 13 % of ordered compounds never detected, so misassignment within a 10-compound pool is possible but unquantified in the paper.
  • CE recorded as NCE (confirmed by its authors per the Kaggle page); the eV conversion in collision_energy_ev is a best-effort transform.

riken

  • Content = RIKEN PlaSMA (Plant Specialized Metabolome Annotation) authentic-standard library plus related RIKEN collections. A MoNA PlaSMA record (RIKENPlaSMA002015) shows: Waters Xevo G2 Q-Tof, LC-ESI-QTOF, 30 eV, [M+H]+, submitter Ryo Nakabayashi, library "RIKEN PlaSMA Authentic Standard Library", mass accuracy 1.165 ppm. MS-DIAL's distribution lists PlaSMA authentic standards at 4,439 (pos) / 4,216 (neg) records and PlaSMA bio-MS/MS from plant tissues at 2,384 / 1,732 — the latter are annotated from tissue, not standards. MassBank-data has RIKEN, RIKEN_IMS, RIKEN_NPDepo, RIKEN_ReSpect contributor folders (MoNA ReSpect export: 6,374 spectra).
  • 347,171 spectra for 15,892 structures (21.8/structure) cannot be explained by the ~9k-record PlaSMA MSP alone; the ingest is most likely raw multi-CE/multi-adduct RIKEN exports. Which sub-libraries and whether tissue-derived (non-standard) annotations are included is unverified — the single most important open question for this source, because plant NPs are the test chemistry.
  • Instrument: QTOF (Waters) for PlaSMA; closest public analogue to the timsTOF's beam-type CID among the NP sources.

gnps (community libraries; Wang 2016; Bittremieux, Wang, Dorrestein 2022; docs)

  • Deposition: any registered user may add Bronze spectra; Silver/Gold require approval from the Wang lab. Quality tiers (GNPS docs): Gold = synthetic compound characterised by MS plus NMR/crystallography; Silver = isolated/crude sample with a publication; Bronze = putative/partial annotation, default. Batch upload requires ADDUCT and LIBQUALITY fields but the docs describe no structure-vs-precursor validation at upload.
  • Size: 586,647 non-commercial spectra (Sep 2022); GNPS2 export 2,091,754 spectra across 98 libraries (Aug 2026, Rovai audit).
  • Audited error rates:
    • matchms cleaning (de Jonge et al., J Cheminf 2024, 10.1186/s13321-024-00878-1; ALL_GNPS_NO_PROPOGATED 21-08-2023, 500,569 spectra): on 413,314 annotated spectra, SMILES could not be derived from the compound name for 27.6 %; of those where it could, 1.62 % had a different 2D structure than the deposited one; 0.024 % wrong adduct among the 99.98 % with a derivable adduct. Without repair 83,843 spectra would be removed; with repair 31,758 removed and 52,084 repaired; final 448,485.
    • MassSpecGym (Bushuiev et al., NeurIPS 2024, arXiv 2410.23326, Table 2; GNPS+MoNA+MassBank+MSnLib): require_matching_adduct_precursor_mz_parent_mass removed 41,049; require_parent_mass_match_smiles 3,726; repair_inchi_inchikey_smiles changed 33,319 and removed 628,478 (largely entries without usable structure); repair_adduct_based_on_smiles changed 305,728; after metadata filters 414,049 spectra remained; SIRIUS explained-intensity < 50 % removed 135,190 spectra but only 926 structures; zero peaks shared with NIST20 for 14,600 spectra (38 structures); cosine < 0.2 vs NIST for 9,290 more (126 structures); within-dataset outliers 11,740. Final 231,104. "A manual examination of a small random subset of the removed spectra found all of them at least 'suspicious', with many clearly being wrongly annotated."
    • Rovai / Tesseract audit (Aug 2026, not peer-reviewed, repo github.com/fabio-rovai/spectral-library-ontology): of 1,485,480 GNPS2 records with both SMILES and InChI, 0.883 % disagree on stereo, 0.045 % (662; 229 after RDKit recomputation) on connectivity; 7.21 % precursor m/z mismatches (113,458 of 1,573,871, 0.05 Da tolerance); 9.81 % InChI strings lacking the InChI=1S/ prefix. Per-library structure-divergence rates: BILELIB19 59.44 %, GNPS-NUTRI-METAB-FEM-POS 37.91 %, MassBank 8.53 %, GNPS-LIBRARY 6.90 %, MoNA 1.41 % (these are mostly stereo, irrelevant to InChIKey14).

massbank (Neumann et al., NAR 2025, 10.1093/nar/gkaf1193; MassBank-data GitHub)

  • Release 2025.05.1: 119,845 unique spectra (by SPLASH), 18,529 compounds (by InChI), 53 contributors. 82.3 % MS2; 71.2 % positive. Instrument mix: LC-ESI-QTOF 39.6 %, LC-ESI-ITFT 16.2 %, LC-ESI-QFT 14.8 %, EI-B 9.5 %, LC-ESI-QQ 7.8 %.
  • Deposition: contributors generate records (RMassBank or own pipeline), fork MassBank-data, open a PR against dev; GitHub Actions runs the MassBank-web Validator; "a validation system is in place to check new records and retrospectively screen existing records for compatibility with the record format and consistency of chemical information". Records from RMassBank carry peak formula annotations with mass error and ppm error. No published count of records failing validation.
  • Standards: "most of MassBank is spectra recorded on reference standards or isolated compounds"; 32,372 spectra tagged as from isolated/synthesised compounds and 31,977 as MSI level 1; only 251/644/31 records claim levels 2/3/4 and 238 are "biological origin" (tentative structures are allowed and marked). The 101,727 train rows are ~85 % of MassBank's whole release, so GC-EI and QQQ records may have been filtered or may be present — unverified.

mona (MassBank of North America, Fiehn lab)

  • Self-described "auto-curated" repository mixing experimental and in-silico libraries; predefined downloads include In-Silico Spectra 3,190,996, LipidBlast 2022 1,346,798, MassBank 72,439, GNPS 23,801, BMDMS-NP 227,307, Vaniya/Fiehn NP library 44,936 (in silico per MS-DIAL docs), MSnLib 87,948, Enveda-180 1,158,293, ReSpect 6,374, HMDB 7,415, VF-NPL QTOF 9,997, VF-NPL QExactive 28,260.
  • The Kaggle page does not say whether in-silico entries were excluded from the 92,416 mona rows. 11,681 structures at 7.9 spectra/structure is consistent with experimental sub-libraries, but this is unverified. MoNA's own documentation page returned no content; I found no Fiehn-lab paper quantifying MoNA errors (brief's "Fiehn lab on MoNA errors" could not be located; unverified).
  • MoNA re-hosts MassBank, GNPS, MSnLib and Enveda-180 copies, so cross-source duplication with massbank, gnps, pluskal_ms2 and enveda-180 is structural, not incidental.

spectraverse (Gupta, Qiang, Chung, Herbst, Skinnider, Anal Chem 2026, 10.1021/acs.analchem.5c06256; PMC12903054; Zenodo 17870921)

  • 1,672,217 input spectra from GNPS, MoNA, MS-DIAL, HMDB, MSnLib plus literature/MetaboLights/Zenodo/GitHub sets; filters: uniform intensities removed; precursor m/z > 1000 removed; invalid SMILES removed; ≤ 2 decimal m/z (low-res) removed; parent mass must match monoisotopic mass within 0.1 Da; near-duplicates (cosine > 0.99) collapsed; theoretical-vs-experimental precursor within 10 ppm; nine adducts kept; fragments < 0.1 % removed; QQQ excluded. Output 488,630 spectra / 44,237 molecules. Documented pitfalls: identical intensities, zero-intensity fragments, precursor m/z equal to neutral monoisotopic mass.
  • Train takes only 50,933 of these (the "obscure" remainder after dedup against the primary sources, presumably). Cleanest metadata of the public sources, but by construction nearly every molecule also lives in another source.

msdial (MS-DIAL public MSP kit, Tsugawa lab; VS19, 8 Aug 2024)

  • MSMS-Public-Pos-VS19: 16,232 compounds / 324,191 records "from authentic standards"; Neg: 8,887 / 44,669. The "all" variants add bio + in-silico (Fiehn/Vaniya NP 28,347 pos / 11,590 neg in silico; BMDMS-NP 227,307 in silico). The kit also bundles GNPS (8,782/2,351), MassBank (8,068/4,782) and PlaSMA subsets. Which variant was ingested is unverified; 40,765 rows is far below the standards-only size, so the ingest is probably the residual after dedup against gnps/massbank/riken.

drug_plus

  • Official description only: ~2,500 drug substances, ~1 spectrum each, no CE metadata. I could not identify the upstream database; treat provenance as unverified (confidence low). One spectrum per structure means no internal consistency check is possible.

masaryk (RECETOX, Masaryk University)

  • RECETOX publishes HR-MS libraries "collected from authentic compounds" (CC-BY-NC): Exposome HR-[ESI]-MS v1.0.0 (Zenodo 6325572, 3 Mar 2022; pesticides from Restek and pharmaceuticals from the Prestwick library) and GC-Orbitrap EI libraries. 652 spectra / 416 structures, likely Orbitrap ESI. Highest standards provenance per spectrum, negligible size.

enveda-np-examples

  • 250 common NPs, 99.6 % in COCONUT, uniformly timsTOF, "same instruments and same pipeline as the test". 1,184 spectra in the file. Deliberately overlaps other libraries. Multiple write-ups report held-out retrieval MRR@25 ≈ 0.93 when the structure is still present in other libraries (chiparon 0.934; mxm-1x 0.9261) — i.e. it is a class-1 proxy only.

3. GNPS consistency-failure numbers, consolidated

Check Rate Denominator / tolerance Source
Deposited SMILES differs (2D) from name-derived SMILES 1.62 % 72.4 % of 413,314 annotated spectra where name resolved de Jonge 2024
Name cannot be resolved to a structure 27.6 % 413,314 de Jonge 2024
Wrong adduct (SMILES-mass-based repair) 0.024 % 99.98 % of annotated de Jonge 2024
Spectra removed by cleaning (with repair) 6.3 % (31,758 / 500,569) ALL_GNPS 2023-08-21 de Jonge 2024
Precursor m/z vs adduct+parent mass mismatch 41,049 removed MassSpecGym input (GNPS+MoNA+MassBank+MSnLib) Bushuiev 2024 Tab. 2
Precursor m/z inconsistent with neutral mass + adduct 7.21 % 1,573,871 GNPS2 records, 0.05 Da Rovai 2026 (not peer-reviewed)
Connectivity (InChIKey14-level) SMILES/InChI disagreement 0.045 % (0.015 % after RDKit) 1,485,480 Rovai 2026
< 50 % intensity explained by labelled formula 135,190 spectra / 926 structures of ~414k after metadata cleaning Bushuiev 2024
No peak shared with NIST20 for same structure 14,600 spectra / 38 structures 1/3 of structures overlap NIST Bushuiev 2024

Takeaway: by structure, 2D-label errors in public libraries are ~1–2 %; by spectrum, >30 % of community spectra can fail a formula-explainability test, because bad depositions come in large batches for a few compounds.

4. Multi-CE merging and timsTOF/Bruker content

Source CE representation Merged multi-CE rows? Bruker/timsTOF spectra
enveda-180 eV, 3 energies Zenodo says 3 CEs per precursor; train rows carry collision_energy_ev lists where merged (test decoys are verbatim enveda-180 rows and show [20, 40, 60, 80]-style lists) all 1,153,785 (timsTOF Pro 2)
enveda-np-examples eV same pipeline as test all 1,184
pluskal_ms2 NCE (stepped 20/60) MSnLib ships merged "pseudo-MS2"/ALL_ENERGIES; whether train kept them is unverified 0 (Orbitrap ID-X)
riken eV (QTOF ramps/fixed) mostly single-CE records; raw exports may include per-CE rows unknown; PlaSMA is Waters Xevo G2 QTOF
gnps / mona / massbank / msdial / spectraverse free text, often missing (320,464 null collision_energy_orig train-wide) mixed; MassBank has "ramp" records some Bruker maXis/impact QTOF records exist in MassBank/GNPS (contributors such as Eawag, UFZ) but no write-up gives a count; timsTOF content outside the two enveda sets is unverified and likely near zero
drug_plus none no unknown
masaryk Orbitrap no 0

No write-up I could open gives per-library instrument_type counts; bobownYAO's audit states only that pluskal_ms2 is Orbitrap and enveda-np-examples timsTOF, "other sources contain multiple platforms, nulls, or free text". The 82 instrument strings need to be bucketed by hand (sibling worker adducts_polarity_collision_energy_merging_statistics).

5. Custody score

Formula (weights sum to 1): C = 0.30·V + 0.25·P + 0.15·M + 0.15·I + 0.15·D, where V = structure verification (1 = authentic standard with documented identity; 0.5 = community putative), P = precursor-mass consistency (1 − known fail rate; values without a published audit are estimates, flagged), M = metadata completeness (CE units, instrument, adduct), I = instrument similarity to timsTOF beam-CID QTOF (1 timsTOF; 0.6 other QTOF; 0.3 Orbitrap HCD/unknown), D = independence (1 − expected fraction of rows duplicated in another source).

Source V P M I D Custody C Basis
enveda-np-examples 1.0 1.0 1.0 1.0 0.9 0.99 same pipeline as test; precursor-inconsistent rows dropped
enveda-180 0.9 1.0 1.0 1.0 1.0 0.97 vendor identity, no orthogonal check; 91.4 % ≥ 70 % explained intensity
masaryk 1.0 0.95 0.8 0.3 0.9 0.84 authentic standards; Orbitrap
massbank 0.8 0.97 0.9 0.6 0.8 0.83 validator-gated PRs; 40 % QTOF; tentative records < 1 %
pluskal_ms2 0.9 0.98 0.9 0.3 0.8 0.82 vendor identity, pooled wells, mzmine purity flags; NCE only
riken 0.8 0.9 (est.) 0.8 0.6 0.7 0.78 PlaSMA standards + possible tissue-derived annotations
spectraverse 0.7 1.0 0.8 0.5 0.3 0.70 10 ppm enforced upstream; nearly all duplicated elsewhere
msdial 0.7 0.9 (est.) 0.7 0.5 0.5 0.69 standards kit, but bundles GNPS/MassBank/in-silico variants
gnps 0.5 0.93 0.6 0.5 0.8 0.67 Bronze default; 1.62 % wrong 2D; ~7 % precursor mismatch
drug_plus 0.7 (est.) 0.85 (est.) 0.3 0.3 0.9 0.65 unknown upstream; no CE
mona 0.5 0.9 (est.) 0.7 0.5 0.5 0.63 user + in-silico host; re-hosts other sources

Custody is not usefulness: enveda-180 scores 0.97 and is the wrong chemistry; riken scores 0.78 and is the right chemistry on a QTOF. Use C to weight trust in the label, and a separate domain weight for relevance to test. Because the team has the file, replace the P estimates with the exact per-source value mean(|precursor_error_ppm| <= 10) — the field exists precisely for this.

6. Are Enveda's internal spectra cleaner or dirtier than public ones?

Evidence that they are cleaner on the custody axes: (a) one instrument, one pipeline, eV units, zero null instrument/CE; (b) the Zenodo QC statement (91.4 % of spectra ≥ 70 % explained intensity by SIRIUS) is stricter than MassSpecGym's 50 % cut, which removed ~33 % of public spectra; (c) the same precursor-consistency filter applied to the test was applied to enveda-180's extraction (targeted by expected adduct mass). Evidence of a different kind of dirt: labels are assigned by expected mass inside 10-compound (MSnLib) or multi-compound (enveda-180) pools, so an isobaric pool-mate or an in-source fragment can be credited to the wrong compound without any ppm violation; no orthogonal identity check exists; 13 % of MSnLib compounds were never detected and the enveda-180 detection rate is not stated in sources I could open. One community data point (bobownYAO docs/validation.md): applying the test's raw base-peak ≥ 1,000-count rule to GNPS molecules deleted 58 molecules' spectra and lifted a retrieval MRR from 0.4162 to 0.5999 once lifted — intensity scales are not comparable across sources, so "cleaner" must be judged per criterion. Net: cleaner metadata, equal-or-unknown structure error, and no public audit of either enveda set; confidence medium.

Forced analogy: forensic chain of custody

Statement. A forensic match (fingerprint, DNA, toolmark) is admissible only if the exemplar it was compared with has an intact chain of custody: documented origin (who collected it), verified identity (is the reference really from the claimed person), a tamper-evident log (every handler, every transformation), and a known laboratory error rate (a Daubert factor). Map: library source → exemplar collection; depositor → collecting officer; Gold/Silver/Bronze, MSI level, MassBank CONFIDENCE tag → identity verification; adduct_orig, collision_energy_orig, instrument_type, precursor_error_ppm → the custody log; audits (de Jonge, Bushuiev, Rovai) → published error rates.

Concrete, testable ideas.

  1. Admissibility gate for class-1 library search. Only rows with |precursor_error_ppm| ≤ 10, non-null adduct in the test's 10-adduct set, and C ≥ 0.65 are eligible exemplars; everything else is used for representation learning but not for the final match. Test: on the enveda-np-examples holdout with the structure retained in other libraries, compare MRR@25 with and without the gate; the hypothesis is a small gain in MRR and a larger drop in confident wrong top-1s.
  2. Source-disagreement as a mislabel detector (the "two witnesses" rule). For the 70,618 structure-source repeats, compute the max cosine between a source's spectra and any other source's spectra for the same InChIKey14 (same polarity, adduct-aware). MassSpecGym's version of this (within-dataset cosine < 0.2, structures with ≥ 4 spectra) flagged 11,740 spectra. Rows whose source is the lone dissenter are down-weighted (label noise weight ∝ C). Testable: train the fingerprint model with and without the down-weighting; evaluate on the np-examples holdout and on a riken-only holdout.
  3. Error-rate disclosure per prediction. Report, for each hidden molecule, which custody tier the supporting evidence came from (matched enveda row vs. GNPS Bronze). This does not change MRR but makes ensemble disagreement interpretable, which the sibling-project lesson ("confidence ≠ accuracy") says matters.

Does it hold? Partially. It holds for deposition provenance and metadata trustworthiness, and it correctly predicts the empirical structure of errors (batch-wise, by depositor, like a single contaminated evidence locker: 135,190 bad spectra from 926 structures). It breaks in three places: (i) there is no chain — a library spectrum is one hop from depositor to file, and errors are honest mislabels, not tampering, so tamper-evidence tools (hashes, SPLASH) catch format drift but not wrong structures; (ii) custody says nothing about instrument shift, which here is the dominant failure mode (a perfectly-custodied Orbitrap exemplar is still the wrong instrument); (iii) in court a broken chain excludes the item outright, whereas ML tolerates 1–7 % label noise gracefully and benefits from the noisy rows for representation learning — exclusion is the wrong remedy, weighting is right.

Gotchas / what could fool us

  1. Data-version drift. The official table says enveda-np-examples = 1,151 spectra; three independent audits of the file say 1,184. Freeze the train.parquet hash (bobownYAO reports SHA256 9423f90ec08b941a9646b535bf0606183e86ff0c5a297978f5a70693d41bcd98, 20 Sep 2026) and re-audit if it changes.
  2. precursor_error_ppm ships uncleaned in train but the test was cleaned on exactly this criterion. 62,834 train rows exceed 20 ppm; 4,286 are null. Training on them teaches adduct/charge confusion the test cannot contain.
  3. base_peak_intensity is null for 46 % of train (pre-normalised sources), so the test's "base peak ≥ 1,000 counts" filter cannot be replicated; and raw counts are not comparable across platforms (GNPS example above).
  4. Near-duplicates survive "exact duplicate removal". spectraverse, mona and msdial are aggregations that re-host gnps, massbank, pluskal_ms2 and even enveda-180; re-normalisation or m/z rounding defeats exact hashing. Random row splits leak; split by InChIKey14 at minimum, by MCES/scaffold for honest class-2/3 estimates.
  5. Label conflicts exist inside train: 7,240 spectra in 118 structure groups carry conflicting structure labels (bobownYAO audit) — identical spectra credited to different InChIKey14s.
  6. Spectrum-count vs structure-count error rates differ by 100×. Weight losses per structure, not per spectrum, or a few badly deposited compounds with hundreds of spectra dominate.
  7. Stereo divergence numbers (0.88 %, BILELIB19 59 %) are irrelevant to InChIKey14 but will appear in audits as "wrong structure". Only connectivity divergence (0.015–0.045 %) matters for the metric.
  8. In-silico spectra may be present in mona and msdial (MoNA hosts 3.19 M in-silico; MS-DIAL "all" kits include Vaniya/BMDMS-NP in-silico). Not stated either way on the data page. Screen for tell-tales: uniform intensities, all-integer or 2-decimal m/z, precursor m/z equal to neutral mass.
  9. MSnLib merged "pseudo-MS2" rows (full MSn tree flattened into one spectrum) are not MS2 spectra; MassSpecGym removed them. Check collision_energy_orig strings in pluskal_ms2 for multi-value lists.
  10. NCE ≠ eV. pluskal_ms2 CE is NCE; collision_energy_ev for it is a model, not a measurement.
  11. riken's 21.8 spectra/structure is redundancy, not diversity; PlaSMA bio-MS/MS (tissue-derived annotations) may be included and are not standards.
  12. Rovai audit is not peer-reviewed and runs on GNPS2's 2.09 M-record export, which includes propagated/suspect libraries; its 7.21 % precursor-mismatch rate is an upper bound for the curated subset Enveda likely ingested.
  13. Enveda-180's own chain has a gap: labels come from expected pool masses; a Zenodo-vs-train discrepancy (4,508 spectra) is unexplained; the preprint could not be opened to verify acquisition details.

Implications for the challenge

  1. Compute per-source P exactly: groupby(ingest_lib)[precursor_error_ppm].apply(lambda s: (s.abs()<=10).mean()), and (s.abs()>20).mean(); replace the estimated P values above and publish the table.
  2. Drop (or zero-weight) train rows with |ppm| > 20 or null ppm for any supervised model; keep them only for self-supervised pre-training. This mirrors the host's own test cleaning.
  3. Weight the fingerprint-model loss per structure (1/√n_spectra) and per source by C; evaluate with/without on a riken-only InChIKey14 holdout (right chemistry, wrong instrument) and the np-examples holdout.
  4. Build the "two-witness" disagreement table for the 70,618 repeated structure-source pairs; flag lone-dissenter spectra (max cross-source cosine < 0.2); expect ~1–3 % of public rows.
  5. For the library-search channel (class 1 ≈ 10 % of hidden molecules), restrict exemplars to the 10 test adducts and C ≥ 0.65 sources; measure top-1 precision, not just MRR.
  6. Dedupe near-duplicates across sources with a 0.99-cosine + same InChIKey14 + same adduct rule (Spectraverse's rule) before any split.
  7. Bucket the 82 instrument_type strings into {timsTOF, other QTOF, Orbitrap/FT, ion trap, QQQ, unknown}; train with instrument as a conditioning token or, at minimum, report per-bucket validation.
  8. Treat spectraverse/mona/msdial rows as low-independence: never let them be the only source that puts a structure in validation.
  9. Screen mona/msdial for in-silico tell-tales (Gotcha 8) and remove.
  10. Ask on the Kaggle forum (or infer from adduct_orig/collision_energy_orig patterns) which RIKEN sub-libraries and which MS-DIAL kit variant were ingested; this changes riken's V from 0.8 to either 0.95 or 0.6.

Open questions

  • Exact composition of riken (PlaSMA standards vs bio-MS/MS vs ReSpect vs IMS) and of msdial (standards kit vs "all" kit).
  • Identity of drug_plus.
  • Whether in-silico spectra entered via mona/msdial.
  • Per-source instrument_type histogram and count of Bruker QTOF (non-tims) spectra in public sources.
  • Enveda-180 detection rate per batch and the 4,508-spectrum Zenodo-vs-train gap.
  • Whether pluskal_ms2 includes MSnLib merged pseudo-MS2 rows.
  • The Kaggle discussion thread 741471 cited by one mirror as the source of the official table (JS-only; could not be read).

Sources

Opened and used:

Not opened / unverified: Kaggle discussion 741471; Fiehn-lab MoNA error audit (none found); Enveda-180 ChemRxiv preprint body; Tsugawa 2019 Nat Methods PlaSMA details (search snippets only).