Provenance and custody of the 11 ingest_lib sources in train.parquet
TL;DR. train.parquet is 2,539,608 spectra from 11 sources, but 45 % of it (enveda-180) is instrument-matched synthetic chemistry, ~21 % (MSnLib/pluskal_ms2) is Orbitrap data from commercial screening libraries, and the remaining ~34 % is a layered set of public/community libraries (RIKEN, GNPS, MassBank, MoNA, Spectraverse, MS-DIAL) that contain copies of one another. Published audits put GNPS-style community libraries at roughly 1–2 % wrong 2D structure, ~6–7 % precursor/adduct inconsistency and, by spectrum count, >30 % "not explainable by the labelled formula" — but those bad spectra concentrate in a few hundred structures. The train file ships precursor_error_ppm uncleaned (62,834 rows beyond 20 ppm, 4,286 null) while the hidden test had precursor-inconsistent spectra dropped, so the cheapest custody filter is one the host already applied to the test and did not apply to train.
Environment note: no local train.parquet (Kaggle API 403). Every per-library count below comes from GitHub-mirrored audits of the actual file (chiparon/enveda-casmi26 artifacts/data_audit.json; mxm-1x/Enveda-CASMI-2026 DATA_AUDIT.md, 29 Sep 2026; Clyde0513/CASMI-2026 reports/data_profile.json, 16 Sep 2026) and from a mirror of the official Kaggle data page (dalloliogm/kaggle_competitions references/data_description.md; matsuken92.github.io/casmi.html). Three independent file audits agree with each other on all 11 counts; the official page differs from the file on exactly one row (enveda-np-examples, see Gotcha 1).
1. The 11 sources: counts and what each one is
ingest_lib |
Spectra (file audit) | Unique InChIKey14 | Spectra / structure | Share of train | Official one-line description (Kaggle data page, mirrored) | Count status |
|---|---|---|---|---|---|---|
| enveda-180 | 1,153,785 | 182,941 | 6.3 | 45.4 % | Enveda's published dataset, same Bruker timsTOF as test; synthetic drug-like screening compounds | verified (3 audits + page) |
| pluskal_ms2 | 527,581 | 46,821 | 11.3 | 20.8 % | MSnLib (Pluskal lab): Orbitrap ID-X, commercial screening/bioactive libraries, one protocol, multiple NCEs | verified |
| riken | 347,171 | 15,892 | 21.8 | 13.7 % | RIKEN public libraries, plant specialised metabolites | verified |
| gnps | 220,849 | 45,750 | 4.8 | 8.7 % | GNPS community libraries, largest NP collection, most heterogeneous | verified |
| massbank | 101,727 | 9,180 | 11.1 | 4.0 % | MassBank (EU/JP consortium), curated, many labs/instruments | verified |
| mona | 92,416 | 11,681 | 7.9 | 3.6 % | MassBank of North America, Fiehn lab, community-hosted | verified |
| spectraverse | 50,933 | 9,631 | 5.3 | 2.0 % | Recent harmonised aggregation of public libraries incl. obscure ones | verified |
| msdial | 40,765 | 9,127 | 4.5 | 1.6 % | Public collections distributed with MS-DIAL, multi-instrument | verified |
| drug_plus | 2,545 | 2,539 | 1.0 | 0.1 % | Pharmaceutical LC-MS/MS database, ~1 spectrum per drug, no CE metadata | verified count; identity of the upstream database unverified |
| enveda-np-examples | 1,184 (page says 1,151) | 250 | 4.7 | 0.05 % | 250 common natural products on the test instrument and pipeline | file 1,184 verified by 3 audits; page 1,151 |
| masaryk | 652 | 416 | 1.6 | 0.03 % | Small library of chemical standards from RECETOX (Masaryk University) | verified |
Sum of unique structures = 346,428 vs 275,810 distinct in train, so 70,618 structure-library pairs are repeats of a structure seen in another source (20 % redundancy at structure level). Ionisation: 1,899,640 positive / 639,968 negative. Nulls: base_peak_intensity 1,163,138; collision_energy_orig 320,464; instrument_type 33,053; precursor_error_ppm 4,286 (Clyde0513 profile).
Enveda's own internal catalogue (enveda/enveda-180 repo, notebooks/create_figures/01_mk_prepdata.ipynb) lists its Databricks sources as casmi22, drug_plus, gnps, gnps_drug, masaryk, massbank, mona, ometa, pluskal_ms2, riken — i.e. the Kaggle file omits casmi22, gnps_drug and ometa and adds spectraverse, msdial and the two enveda sets. That casmi22 exists internally but not in train is itself a hint about what the organisers consider test-adjacent.
2. Per-source deposition process, standards, and audited error rates
enveda-180 (Krettler, Kind, ..., Healey 2026; Zenodo 10.5281/zenodo.20436851, v1.0.0, 29 May 2026)
- Zenodo record: 1,158,293 curated MS/MS spectra, 184,330 unique compounds, median 6 spectra/compound, 14 adducts, Bruker timsTOF Pro 2 with heated ESI, both polarities, "precursors fragmented at three collision energies". QC statement: "multi-criteria curation" combining fragmentation scoring and integrity validation; "approximately 91.4 % exceed 70 % explained-intensity thresholds", median spectral entropy 0.44.
- Repo code (data_processing/curation/constants.py):
TOP_N = 20,TOLERANCE_DA = 20for library matching; exported QC columns includesirius_score,explained_intensity,median_mass_error_fragment_peaks_ppm,mass_error_precursor_ppm. SIRIUS is run externally and merged in. Compounds are Enamine REAL-space purchases pooled in wells ("14 enamine batches"); the pipeline does targeted extraction per compound across 17 adduct hypotheses, i.e. the label is assigned by expected mass, not by an orthogonal identity check. - Train has 4,508 fewer spectra and 1,389 fewer structures than the Zenodo release (dedup/standardisation; unverified).
- Structure verification: purchased compounds, vendor-declared identity; no NMR; purity not reported in the sources I could open (the ChemRxiv preprint 10.26434/chemrxiv.15004319 returned 403 both via DOI and API, so acquisition details beyond Zenodo are unverified here; q2 sibling
enveda180_and_np_examples_anatomyowns the deep dive). - Chemistry overlap with COCONUT ~0.03 % (BRIEF).
pluskal_ms2 = MSnLib (Brungs, Schmid, ..., Pluskal, Nat Methods 2025, doi 10.1038/s41592-025-02813-0; PMC12510872; Zenodo 10.5281/zenodo.21105617 v8, 1 Jul 2026)
- Orbitrap ID-X, flow-injection DDA, robotic pooling of 10 compounds per well, ~3,000 substances/day. Paper: 7 commercial libraries, 30,008 unique compounds, ">2.3 million MSn and >357,000 MS2 spectra", "87 % coverage" of ordered compounds; ~14,300 detected in both polarities, >12,200 positive-only, >3,400 negative-only. Stepped HCD 20/60 (MS2) and 20/40/60 (MSn). Zenodo v8 now lists 9 libraries incl. MCEDIV_50k_Sub (20,000 cpds) and TargetMolHTSNP (2,175 NP-like cpds) — consistent with train's 46,821 structures exceeding the paper's 30,008.
- QC in mzmine: precursor-purity score, chimeric spectra flagged, noise removal below 2.5× lowest signal, minimum intensity 6e5 (pos) / 2e5 (neg). Merged spectra exist in the release ("pseudo MS2", SAME_ENERGY, ALL_ENERGIES); MassSpecGym explicitly removed all MSnLib merged spectra before use.
- Structure verification: vendor identity (MCE, Enamine, OTAVA, NIH NPAC); no orthogonal confirmation; 13 % of ordered compounds never detected, so misassignment within a 10-compound pool is possible but unquantified in the paper.
- CE recorded as NCE (confirmed by its authors per the Kaggle page); the eV conversion in
collision_energy_evis a best-effort transform.
riken
- Content = RIKEN PlaSMA (Plant Specialized Metabolome Annotation) authentic-standard library plus related RIKEN collections. A MoNA PlaSMA record (RIKENPlaSMA002015) shows: Waters Xevo G2 Q-Tof, LC-ESI-QTOF, 30 eV, [M+H]+, submitter Ryo Nakabayashi, library "RIKEN PlaSMA Authentic Standard Library", mass accuracy 1.165 ppm. MS-DIAL's distribution lists PlaSMA authentic standards at 4,439 (pos) / 4,216 (neg) records and PlaSMA bio-MS/MS from plant tissues at 2,384 / 1,732 — the latter are annotated from tissue, not standards. MassBank-data has
RIKEN,RIKEN_IMS,RIKEN_NPDepo,RIKEN_ReSpectcontributor folders (MoNA ReSpect export: 6,374 spectra). - 347,171 spectra for 15,892 structures (21.8/structure) cannot be explained by the ~9k-record PlaSMA MSP alone; the ingest is most likely raw multi-CE/multi-adduct RIKEN exports. Which sub-libraries and whether tissue-derived (non-standard) annotations are included is unverified — the single most important open question for this source, because plant NPs are the test chemistry.
- Instrument: QTOF (Waters) for PlaSMA; closest public analogue to the timsTOF's beam-type CID among the NP sources.
gnps (community libraries; Wang 2016; Bittremieux, Wang, Dorrestein 2022; docs)
- Deposition: any registered user may add Bronze spectra; Silver/Gold require approval from the Wang lab. Quality tiers (GNPS docs): Gold = synthetic compound characterised by MS plus NMR/crystallography; Silver = isolated/crude sample with a publication; Bronze = putative/partial annotation, default. Batch upload requires ADDUCT and LIBQUALITY fields but the docs describe no structure-vs-precursor validation at upload.
- Size: 586,647 non-commercial spectra (Sep 2022); GNPS2 export 2,091,754 spectra across 98 libraries (Aug 2026, Rovai audit).
- Audited error rates:
- matchms cleaning (de Jonge et al., J Cheminf 2024, 10.1186/s13321-024-00878-1; ALL_GNPS_NO_PROPOGATED 21-08-2023, 500,569 spectra): on 413,314 annotated spectra, SMILES could not be derived from the compound name for 27.6 %; of those where it could, 1.62 % had a different 2D structure than the deposited one; 0.024 % wrong adduct among the 99.98 % with a derivable adduct. Without repair 83,843 spectra would be removed; with repair 31,758 removed and 52,084 repaired; final 448,485.
- MassSpecGym (Bushuiev et al., NeurIPS 2024, arXiv 2410.23326, Table 2; GNPS+MoNA+MassBank+MSnLib):
require_matching_adduct_precursor_mz_parent_massremoved 41,049;require_parent_mass_match_smiles3,726;repair_inchi_inchikey_smileschanged 33,319 and removed 628,478 (largely entries without usable structure);repair_adduct_based_on_smileschanged 305,728; after metadata filters 414,049 spectra remained; SIRIUS explained-intensity < 50 % removed 135,190 spectra but only 926 structures; zero peaks shared with NIST20 for 14,600 spectra (38 structures); cosine < 0.2 vs NIST for 9,290 more (126 structures); within-dataset outliers 11,740. Final 231,104. "A manual examination of a small random subset of the removed spectra found all of them at least 'suspicious', with many clearly being wrongly annotated." - Rovai / Tesseract audit (Aug 2026, not peer-reviewed, repo github.com/fabio-rovai/spectral-library-ontology): of 1,485,480 GNPS2 records with both SMILES and InChI, 0.883 % disagree on stereo, 0.045 % (662; 229 after RDKit recomputation) on connectivity; 7.21 % precursor m/z mismatches (113,458 of 1,573,871, 0.05 Da tolerance); 9.81 % InChI strings lacking the
InChI=1S/prefix. Per-library structure-divergence rates: BILELIB19 59.44 %, GNPS-NUTRI-METAB-FEM-POS 37.91 %, MassBank 8.53 %, GNPS-LIBRARY 6.90 %, MoNA 1.41 % (these are mostly stereo, irrelevant to InChIKey14).
massbank (Neumann et al., NAR 2025, 10.1093/nar/gkaf1193; MassBank-data GitHub)
- Release 2025.05.1: 119,845 unique spectra (by SPLASH), 18,529 compounds (by InChI), 53 contributors. 82.3 % MS2; 71.2 % positive. Instrument mix: LC-ESI-QTOF 39.6 %, LC-ESI-ITFT 16.2 %, LC-ESI-QFT 14.8 %, EI-B 9.5 %, LC-ESI-QQ 7.8 %.
- Deposition: contributors generate records (RMassBank or own pipeline), fork MassBank-data, open a PR against
dev; GitHub Actions runs the MassBank-web Validator; "a validation system is in place to check new records and retrospectively screen existing records for compatibility with the record format and consistency of chemical information". Records from RMassBank carry peak formula annotations with mass error and ppm error. No published count of records failing validation. - Standards: "most of MassBank is spectra recorded on reference standards or isolated compounds"; 32,372 spectra tagged as from isolated/synthesised compounds and 31,977 as MSI level 1; only 251/644/31 records claim levels 2/3/4 and 238 are "biological origin" (tentative structures are allowed and marked). The 101,727 train rows are ~85 % of MassBank's whole release, so GC-EI and QQQ records may have been filtered or may be present — unverified.
mona (MassBank of North America, Fiehn lab)
- Self-described "auto-curated" repository mixing experimental and in-silico libraries; predefined downloads include In-Silico Spectra 3,190,996, LipidBlast 2022 1,346,798, MassBank 72,439, GNPS 23,801, BMDMS-NP 227,307, Vaniya/Fiehn NP library 44,936 (in silico per MS-DIAL docs), MSnLib 87,948, Enveda-180 1,158,293, ReSpect 6,374, HMDB 7,415, VF-NPL QTOF 9,997, VF-NPL QExactive 28,260.
- The Kaggle page does not say whether in-silico entries were excluded from the 92,416
monarows. 11,681 structures at 7.9 spectra/structure is consistent with experimental sub-libraries, but this is unverified. MoNA's own documentation page returned no content; I found no Fiehn-lab paper quantifying MoNA errors (brief's "Fiehn lab on MoNA errors" could not be located; unverified). - MoNA re-hosts MassBank, GNPS, MSnLib and Enveda-180 copies, so cross-source duplication with
massbank,gnps,pluskal_ms2andenveda-180is structural, not incidental.
spectraverse (Gupta, Qiang, Chung, Herbst, Skinnider, Anal Chem 2026, 10.1021/acs.analchem.5c06256; PMC12903054; Zenodo 17870921)
- 1,672,217 input spectra from GNPS, MoNA, MS-DIAL, HMDB, MSnLib plus literature/MetaboLights/Zenodo/GitHub sets; filters: uniform intensities removed; precursor m/z > 1000 removed; invalid SMILES removed; ≤ 2 decimal m/z (low-res) removed; parent mass must match monoisotopic mass within 0.1 Da; near-duplicates (cosine > 0.99) collapsed; theoretical-vs-experimental precursor within 10 ppm; nine adducts kept; fragments < 0.1 % removed; QQQ excluded. Output 488,630 spectra / 44,237 molecules. Documented pitfalls: identical intensities, zero-intensity fragments, precursor m/z equal to neutral monoisotopic mass.
- Train takes only 50,933 of these (the "obscure" remainder after dedup against the primary sources, presumably). Cleanest metadata of the public sources, but by construction nearly every molecule also lives in another source.
msdial (MS-DIAL public MSP kit, Tsugawa lab; VS19, 8 Aug 2024)
- MSMS-Public-Pos-VS19: 16,232 compounds / 324,191 records "from authentic standards"; Neg: 8,887 / 44,669. The "all" variants add bio + in-silico (Fiehn/Vaniya NP 28,347 pos / 11,590 neg in silico; BMDMS-NP 227,307 in silico). The kit also bundles GNPS (8,782/2,351), MassBank (8,068/4,782) and PlaSMA subsets. Which variant was ingested is unverified; 40,765 rows is far below the standards-only size, so the ingest is probably the residual after dedup against
gnps/massbank/riken.
drug_plus
- Official description only: ~2,500 drug substances, ~1 spectrum each, no CE metadata. I could not identify the upstream database; treat provenance as unverified (confidence low). One spectrum per structure means no internal consistency check is possible.
masaryk (RECETOX, Masaryk University)
- RECETOX publishes HR-MS libraries "collected from authentic compounds" (CC-BY-NC): Exposome HR-[ESI]-MS v1.0.0 (Zenodo 6325572, 3 Mar 2022; pesticides from Restek and pharmaceuticals from the Prestwick library) and GC-Orbitrap EI libraries. 652 spectra / 416 structures, likely Orbitrap ESI. Highest standards provenance per spectrum, negligible size.
enveda-np-examples
- 250 common NPs, 99.6 % in COCONUT, uniformly timsTOF, "same instruments and same pipeline as the test". 1,184 spectra in the file. Deliberately overlaps other libraries. Multiple write-ups report held-out retrieval MRR@25 ≈ 0.93 when the structure is still present in other libraries (chiparon 0.934; mxm-1x 0.9261) — i.e. it is a class-1 proxy only.
3. GNPS consistency-failure numbers, consolidated
| Check | Rate | Denominator / tolerance | Source |
|---|---|---|---|
| Deposited SMILES differs (2D) from name-derived SMILES | 1.62 % | 72.4 % of 413,314 annotated spectra where name resolved | de Jonge 2024 |
| Name cannot be resolved to a structure | 27.6 % | 413,314 | de Jonge 2024 |
| Wrong adduct (SMILES-mass-based repair) | 0.024 % | 99.98 % of annotated | de Jonge 2024 |
| Spectra removed by cleaning (with repair) | 6.3 % (31,758 / 500,569) | ALL_GNPS 2023-08-21 | de Jonge 2024 |
| Precursor m/z vs adduct+parent mass mismatch | 41,049 removed | MassSpecGym input (GNPS+MoNA+MassBank+MSnLib) | Bushuiev 2024 Tab. 2 |
| Precursor m/z inconsistent with neutral mass + adduct | 7.21 % | 1,573,871 GNPS2 records, 0.05 Da | Rovai 2026 (not peer-reviewed) |
| Connectivity (InChIKey14-level) SMILES/InChI disagreement | 0.045 % (0.015 % after RDKit) | 1,485,480 | Rovai 2026 |
| < 50 % intensity explained by labelled formula | 135,190 spectra / 926 structures of ~414k | after metadata cleaning | Bushuiev 2024 |
| No peak shared with NIST20 for same structure | 14,600 spectra / 38 structures | 1/3 of structures overlap NIST | Bushuiev 2024 |
Takeaway: by structure, 2D-label errors in public libraries are ~1–2 %; by spectrum, >30 % of community spectra can fail a formula-explainability test, because bad depositions come in large batches for a few compounds.
4. Multi-CE merging and timsTOF/Bruker content
| Source | CE representation | Merged multi-CE rows? | Bruker/timsTOF spectra |
|---|---|---|---|
| enveda-180 | eV, 3 energies | Zenodo says 3 CEs per precursor; train rows carry collision_energy_ev lists where merged (test decoys are verbatim enveda-180 rows and show [20, 40, 60, 80]-style lists) |
all 1,153,785 (timsTOF Pro 2) |
| enveda-np-examples | eV | same pipeline as test | all 1,184 |
| pluskal_ms2 | NCE (stepped 20/60) | MSnLib ships merged "pseudo-MS2"/ALL_ENERGIES; whether train kept them is unverified | 0 (Orbitrap ID-X) |
| riken | eV (QTOF ramps/fixed) | mostly single-CE records; raw exports may include per-CE rows | unknown; PlaSMA is Waters Xevo G2 QTOF |
| gnps / mona / massbank / msdial / spectraverse | free text, often missing (320,464 null collision_energy_orig train-wide) |
mixed; MassBank has "ramp" records | some Bruker maXis/impact QTOF records exist in MassBank/GNPS (contributors such as Eawag, UFZ) but no write-up gives a count; timsTOF content outside the two enveda sets is unverified and likely near zero |
| drug_plus | none | no | unknown |
| masaryk | Orbitrap | no | 0 |
No write-up I could open gives per-library instrument_type counts; bobownYAO's audit states only that pluskal_ms2 is Orbitrap and enveda-np-examples timsTOF, "other sources contain multiple platforms, nulls, or free text". The 82 instrument strings need to be bucketed by hand (sibling worker adducts_polarity_collision_energy_merging_statistics).
5. Custody score
Formula (weights sum to 1): C = 0.30·V + 0.25·P + 0.15·M + 0.15·I + 0.15·D, where V = structure verification (1 = authentic standard with documented identity; 0.5 = community putative), P = precursor-mass consistency (1 − known fail rate; values without a published audit are estimates, flagged), M = metadata completeness (CE units, instrument, adduct), I = instrument similarity to timsTOF beam-CID QTOF (1 timsTOF; 0.6 other QTOF; 0.3 Orbitrap HCD/unknown), D = independence (1 − expected fraction of rows duplicated in another source).
| Source | V | P | M | I | D | Custody C | Basis |
|---|---|---|---|---|---|---|---|
| enveda-np-examples | 1.0 | 1.0 | 1.0 | 1.0 | 0.9 | 0.99 | same pipeline as test; precursor-inconsistent rows dropped |
| enveda-180 | 0.9 | 1.0 | 1.0 | 1.0 | 1.0 | 0.97 | vendor identity, no orthogonal check; 91.4 % ≥ 70 % explained intensity |
| masaryk | 1.0 | 0.95 | 0.8 | 0.3 | 0.9 | 0.84 | authentic standards; Orbitrap |
| massbank | 0.8 | 0.97 | 0.9 | 0.6 | 0.8 | 0.83 | validator-gated PRs; 40 % QTOF; tentative records < 1 % |
| pluskal_ms2 | 0.9 | 0.98 | 0.9 | 0.3 | 0.8 | 0.82 | vendor identity, pooled wells, mzmine purity flags; NCE only |
| riken | 0.8 | 0.9 (est.) | 0.8 | 0.6 | 0.7 | 0.78 | PlaSMA standards + possible tissue-derived annotations |
| spectraverse | 0.7 | 1.0 | 0.8 | 0.5 | 0.3 | 0.70 | 10 ppm enforced upstream; nearly all duplicated elsewhere |
| msdial | 0.7 | 0.9 (est.) | 0.7 | 0.5 | 0.5 | 0.69 | standards kit, but bundles GNPS/MassBank/in-silico variants |
| gnps | 0.5 | 0.93 | 0.6 | 0.5 | 0.8 | 0.67 | Bronze default; 1.62 % wrong 2D; ~7 % precursor mismatch |
| drug_plus | 0.7 (est.) | 0.85 (est.) | 0.3 | 0.3 | 0.9 | 0.65 | unknown upstream; no CE |
| mona | 0.5 | 0.9 (est.) | 0.7 | 0.5 | 0.5 | 0.63 | user + in-silico host; re-hosts other sources |
Custody is not usefulness: enveda-180 scores 0.97 and is the wrong chemistry; riken scores 0.78 and is the right chemistry on a QTOF. Use C to weight trust in the label, and a separate domain weight for relevance to test. Because the team has the file, replace the P estimates with the exact per-source value mean(|precursor_error_ppm| <= 10) — the field exists precisely for this.
6. Are Enveda's internal spectra cleaner or dirtier than public ones?
Evidence that they are cleaner on the custody axes: (a) one instrument, one pipeline, eV units, zero null instrument/CE; (b) the Zenodo QC statement (91.4 % of spectra ≥ 70 % explained intensity by SIRIUS) is stricter than MassSpecGym's 50 % cut, which removed ~33 % of public spectra; (c) the same precursor-consistency filter applied to the test was applied to enveda-180's extraction (targeted by expected adduct mass). Evidence of a different kind of dirt: labels are assigned by expected mass inside 10-compound (MSnLib) or multi-compound (enveda-180) pools, so an isobaric pool-mate or an in-source fragment can be credited to the wrong compound without any ppm violation; no orthogonal identity check exists; 13 % of MSnLib compounds were never detected and the enveda-180 detection rate is not stated in sources I could open. One community data point (bobownYAO docs/validation.md): applying the test's raw base-peak ≥ 1,000-count rule to GNPS molecules deleted 58 molecules' spectra and lifted a retrieval MRR from 0.4162 to 0.5999 once lifted — intensity scales are not comparable across sources, so "cleaner" must be judged per criterion. Net: cleaner metadata, equal-or-unknown structure error, and no public audit of either enveda set; confidence medium.
Forced analogy: forensic chain of custody
Statement. A forensic match (fingerprint, DNA, toolmark) is admissible only if the exemplar it was compared with has an intact chain of custody: documented origin (who collected it), verified identity (is the reference really from the claimed person), a tamper-evident log (every handler, every transformation), and a known laboratory error rate (a Daubert factor). Map: library source → exemplar collection; depositor → collecting officer; Gold/Silver/Bronze, MSI level, MassBank CONFIDENCE tag → identity verification; adduct_orig, collision_energy_orig, instrument_type, precursor_error_ppm → the custody log; audits (de Jonge, Bushuiev, Rovai) → published error rates.
Concrete, testable ideas.
- Admissibility gate for class-1 library search. Only rows with |
precursor_error_ppm| ≤ 10, non-null adduct in the test's 10-adduct set, and C ≥ 0.65 are eligible exemplars; everything else is used for representation learning but not for the final match. Test: on the enveda-np-examples holdout with the structure retained in other libraries, compare MRR@25 with and without the gate; the hypothesis is a small gain in MRR and a larger drop in confident wrong top-1s. - Source-disagreement as a mislabel detector (the "two witnesses" rule). For the 70,618 structure-source repeats, compute the max cosine between a source's spectra and any other source's spectra for the same InChIKey14 (same polarity, adduct-aware). MassSpecGym's version of this (within-dataset cosine < 0.2, structures with ≥ 4 spectra) flagged 11,740 spectra. Rows whose source is the lone dissenter are down-weighted (label noise weight ∝ C). Testable: train the fingerprint model with and without the down-weighting; evaluate on the np-examples holdout and on a riken-only holdout.
- Error-rate disclosure per prediction. Report, for each hidden molecule, which custody tier the supporting evidence came from (matched enveda row vs. GNPS Bronze). This does not change MRR but makes ensemble disagreement interpretable, which the sibling-project lesson ("confidence ≠ accuracy") says matters.
Does it hold? Partially. It holds for deposition provenance and metadata trustworthiness, and it correctly predicts the empirical structure of errors (batch-wise, by depositor, like a single contaminated evidence locker: 135,190 bad spectra from 926 structures). It breaks in three places: (i) there is no chain — a library spectrum is one hop from depositor to file, and errors are honest mislabels, not tampering, so tamper-evidence tools (hashes, SPLASH) catch format drift but not wrong structures; (ii) custody says nothing about instrument shift, which here is the dominant failure mode (a perfectly-custodied Orbitrap exemplar is still the wrong instrument); (iii) in court a broken chain excludes the item outright, whereas ML tolerates 1–7 % label noise gracefully and benefits from the noisy rows for representation learning — exclusion is the wrong remedy, weighting is right.
Gotchas / what could fool us
- Data-version drift. The official table says enveda-np-examples = 1,151 spectra; three independent audits of the file say 1,184. Freeze the train.parquet hash (bobownYAO reports SHA256
9423f90ec08b941a9646b535bf0606183e86ff0c5a297978f5a70693d41bcd98, 20 Sep 2026) and re-audit if it changes. precursor_error_ppmships uncleaned in train but the test was cleaned on exactly this criterion. 62,834 train rows exceed 20 ppm; 4,286 are null. Training on them teaches adduct/charge confusion the test cannot contain.base_peak_intensityis null for 46 % of train (pre-normalised sources), so the test's "base peak ≥ 1,000 counts" filter cannot be replicated; and raw counts are not comparable across platforms (GNPS example above).- Near-duplicates survive "exact duplicate removal". spectraverse, mona and msdial are aggregations that re-host gnps, massbank, pluskal_ms2 and even enveda-180; re-normalisation or m/z rounding defeats exact hashing. Random row splits leak; split by InChIKey14 at minimum, by MCES/scaffold for honest class-2/3 estimates.
- Label conflicts exist inside train: 7,240 spectra in 118 structure groups carry conflicting structure labels (bobownYAO audit) — identical spectra credited to different InChIKey14s.
- Spectrum-count vs structure-count error rates differ by 100×. Weight losses per structure, not per spectrum, or a few badly deposited compounds with hundreds of spectra dominate.
- Stereo divergence numbers (0.88 %, BILELIB19 59 %) are irrelevant to InChIKey14 but will appear in audits as "wrong structure". Only connectivity divergence (0.015–0.045 %) matters for the metric.
- In-silico spectra may be present in
monaandmsdial(MoNA hosts 3.19 M in-silico; MS-DIAL "all" kits include Vaniya/BMDMS-NP in-silico). Not stated either way on the data page. Screen for tell-tales: uniform intensities, all-integer or 2-decimal m/z, precursor m/z equal to neutral mass. - MSnLib merged "pseudo-MS2" rows (full MSn tree flattened into one spectrum) are not MS2 spectra; MassSpecGym removed them. Check
collision_energy_origstrings in pluskal_ms2 for multi-value lists. - NCE ≠ eV. pluskal_ms2 CE is NCE;
collision_energy_evfor it is a model, not a measurement. - riken's 21.8 spectra/structure is redundancy, not diversity; PlaSMA bio-MS/MS (tissue-derived annotations) may be included and are not standards.
- Rovai audit is not peer-reviewed and runs on GNPS2's 2.09 M-record export, which includes propagated/suspect libraries; its 7.21 % precursor-mismatch rate is an upper bound for the curated subset Enveda likely ingested.
- Enveda-180's own chain has a gap: labels come from expected pool masses; a Zenodo-vs-train discrepancy (4,508 spectra) is unexplained; the preprint could not be opened to verify acquisition details.
Implications for the challenge
- Compute per-source P exactly:
groupby(ingest_lib)[precursor_error_ppm].apply(lambda s: (s.abs()<=10).mean()), and(s.abs()>20).mean(); replace the estimated P values above and publish the table. - Drop (or zero-weight) train rows with |ppm| > 20 or null ppm for any supervised model; keep them only for self-supervised pre-training. This mirrors the host's own test cleaning.
- Weight the fingerprint-model loss per structure (1/√n_spectra) and per source by C; evaluate with/without on a riken-only InChIKey14 holdout (right chemistry, wrong instrument) and the np-examples holdout.
- Build the "two-witness" disagreement table for the 70,618 repeated structure-source pairs; flag lone-dissenter spectra (max cross-source cosine < 0.2); expect ~1–3 % of public rows.
- For the library-search channel (class 1 ≈ 10 % of hidden molecules), restrict exemplars to the 10 test adducts and C ≥ 0.65 sources; measure top-1 precision, not just MRR.
- Dedupe near-duplicates across sources with a 0.99-cosine + same InChIKey14 + same adduct rule (Spectraverse's rule) before any split.
- Bucket the 82
instrument_typestrings into {timsTOF, other QTOF, Orbitrap/FT, ion trap, QQQ, unknown}; train with instrument as a conditioning token or, at minimum, report per-bucket validation. - Treat spectraverse/mona/msdial rows as low-independence: never let them be the only source that puts a structure in validation.
- Screen mona/msdial for in-silico tell-tales (Gotcha 8) and remove.
- Ask on the Kaggle forum (or infer from
adduct_orig/collision_energy_origpatterns) which RIKEN sub-libraries and which MS-DIAL kit variant were ingested; this changes riken's V from 0.8 to either 0.95 or 0.6.
Open questions
- Exact composition of
riken(PlaSMA standards vs bio-MS/MS vs ReSpect vs IMS) and ofmsdial(standards kit vs "all" kit). - Identity of
drug_plus. - Whether in-silico spectra entered via
mona/msdial. - Per-source
instrument_typehistogram and count of Bruker QTOF (non-tims) spectra in public sources. - Enveda-180 detection rate per batch and the 4,508-spectrum Zenodo-vs-train gap.
- Whether
pluskal_ms2includes MSnLib merged pseudo-MS2 rows. - The Kaggle discussion thread 741471 cited by one mirror as the source of the official table (JS-only; could not be read).
Sources
Opened and used:
- Official data page content, mirrored: https://github.com/dalloliogm/kaggle_competitions (competitions/enveda-casmi26-molecule-id-mass-spectra/references/data_description.md); https://github.com/matsuken92/matsuken92.github.io (casmi.html). Original: https://www.kaggle.com/competitions/enveda-CASMI26-molecule-id-mass-spectra/data (JS shell, not readable here).
- File audits (GitHub): https://github.com/chiparon/enveda-casmi26 (artifacts/data_audit.json); https://github.com/mxm-1x/Enveda-CASMI-2026 (DATA_AUDIT.md); https://github.com/Clyde0513/CASMI-2026 (reports/data_notes.md, reports/data_profile.json); https://github.com/bobownYAO/Enveda-CASMI-2026-Molecular_Formula (docs/validation.md, scripts/report_external_sources.py); https://github.com/OfficialBhattacharya/kaggle_CASMI_2026 (docs/COMPETITION.md); https://github.com/NoName7n11/Enveda_CASMI_2026 (OVERVIEW.md); https://github.com/shiferaxa/casmi26; https://github.com/yasumorishima/kaggle-competitions/pull/29.
- Enveda-180: https://github.com/enveda/enveda-180 (README, notebooks/create_figures/01_mk_prepdata.ipynb, data_processing/curation/constants.py); https://zenodo.org/records/20436851; preprint DOI 10.26434/chemrxiv.15004319 (403, not read).
- MSnLib: Brungs et al., Nat Methods 2025, doi 10.1038/s41592-025-02813-0, https://pmc.ncbi.nlm.nih.gov/articles/PMC12510872/; https://zenodo.org/records/11163380 (resolves to 10.5281/zenodo.21105617 v8).
- matchms cleaning: de Jonge et al., J Cheminform 2024, doi 10.1186/s13321-024-00878-1, https://pmc.ncbi.nlm.nih.gov/articles/PMC11285329/.
- MassSpecGym: Bushuiev et al., NeurIPS 2024, https://arxiv.org/abs/2410.23326 (PDF, Sec. 2.3 and Table 2); https://arxiv.org/html/2410.23326; https://huggingface.co/datasets/roman-bushuiev/MassSpecGym.
- Spectraverse: Gupta et al., Anal Chem 2026, doi 10.1021/acs.analchem.5c06256, https://pmc.ncbi.nlm.nih.gov/articles/PMC12903054/.
- GNPS: https://ccms-ucsd.github.io/GNPSDocumentation/gnpslibraries/, .../batchupload/, .../spectrumcuration/; Bittremieux, Wang, Dorrestein, Metabolomics 2022, https://pmc.ncbi.nlm.nih.gov/articles/PMC10284100/.
- GNPS/MassBank/MoNA integrity audit (not peer-reviewed): Rovai, Tesseract Academy, Aug 2026, https://tesseractsemantics.com/research/spectral-library-ontology; https://github.com/fabio-rovai/spectral-library-ontology.
- MassBank: Neumann et al., NAR 2025, doi 10.1093/nar/gkaf1193 (PDF via https://orbilu.uni.lu/handle/10993/66329); https://github.com/MassBank/MassBank-data (README, contributor directory listing); https://massbank.github.io/MassBank-documentation/contributor_documentation.html.
- MoNA: https://mona.fiehnlab.ucdavis.edu/rest/downloads/predefined; https://mona.fiehnlab.ucdavis.edu/rest/spectra/RIKENPlaSMA002015.
- MS-DIAL libraries: https://systemsomicslab.github.io/compms/msdial/main.html.
- RECETOX: https://zenodo.org/records/6325572; https://www.recetox.muni.cz/en/services/recetox-central-laboratories/biomarker-analysis-laboratories/recetox-mass-spectrum-reference-library (search snippet only).
Not opened / unverified: Kaggle discussion 741471; Fiehn-lab MoNA error audit (none found); Enveda-180 ChemRxiv preprint body; Tsugawa 2019 Nat Methods PlaSMA details (search snippets only).