Measure/The data

Custody log of adduct, polarity, collision energy and merged-spectrum fields in train.parquet (computed from the official file)

expertconfidence highrelevance 4/511 sources8.6k words
Forced analogyForensic chain of custody applied to every number in the adduct/CE/ppm columns. Paid off: the shipped precursor_error_ppm turns out to be a post-custody artefact (absolute value, electron mass omitted, computed before the formate relabel) and 7 of 11 libraries carry a computed rather than measured precursor m/z; the analogy breaks where a custody break is mass-neutral (string cosmetics) versus mass-changing (one proton in [M+CH2O2]- -> [M+CH2O2-H]-).

TL;DR. Every adduct string in train.parquet already arrives pre-normalised (adduct_orig is a Hill-formula rewrite such as [M+H4N]+, never a submitter's raw M+H), the mapping to adduct is 1:1 except one mass-changing fix ([M+CH2O2]- -> [M+CH2O2-H]-, 52,460 rows), and polarity contradicts the adduct sign in 6,087 rows (0.24 %). The shipped precursor_error_ppm is not what it looks like: it is an absolute value computed with a theoretical m/z that omits the electron mass, it was computed before the formate relabel (so 63 % of formate rows show ~2,400 ppm), and for 7 of 11 libraries precursor_mz is a calculated, not measured, number. Collision energies were converted NCE->eV with the Thermo rule eV = NCE x m/z / 500 x f(z) (f = 1, 0.9, 0.85 for z = 1, 2, 3), enveda-180 stores negative-mode CEs as negative numbers and merges 20/40/60 eV spectra by summing intensities, and each public library leaves a distinct intensity-floor fingerprint (MassBank 1/999, MS-DIAL 0.01, RIKEN 0.01/0.005) that the timsTOF rows do not have. All numbers below are computed from the local official train.parquet (2,539,608 rows) and test.parquet (1,213 rows) unless marked otherwise.

1. Custody log of the adduct

1.1 Adduct x library (top 25 adducts; counts computed from local train.parquet)

adduct drug_plus enveda-180 enveda-np gnps masaryk massbank mona msdial pluskal_ms2 riken spectraverse rows with contradicting ionization_mode
[M+H]+ 2,012 588,507 569 134,815 652 64,370 46,268 15,335 288,671 169,287 14,099 675 (gnps 441, massbank 198, mona 36)
[M-H]- 0 184,747 276 32,385 0 30,231 22,192 14,494 133,911 34,755 12,697 5,258 (riken 4,870, mona 298, gnps 81, massbank 9)
[2M+Na]+ 0 188,158 0 866 0 20 158 240 0 591 0 0
[M+Na]+ 225 15,957 80 20,039 0 876 8,922 3,401 5,332 121,677 1,583 18 (mona)
[2M+H]+ 0 89,487 0 1,664 0 49 128 230 0 483 0 0
[M+CH2O2-H]- 0 24,556 105 3,321 0 1,095 700 257 50,785 1,556 813 0
[2M-H]- 0 52,933 0 2,339 0 51 91 620 0 138 0 0
[M+NH4]+ 71 4,702 98 6,898 0 317 6,576 2,052 8,645 7,283 16,810 10 (mona)
[M+Cl]- 0 2,407 7 1,416 0 42 102 169 23,021 221 126 18
[M-H2O+H]+ 0 0 24 6,105 0 332 1,007 1,115 11,406 1,693 0 70 (riken 67)
[M+K]+ 164 1,084 16 6,334 0 19 4,976 194 0 5,439 729 0
[M]+ 65 0 0 1,025 0 2,378 185 135 3,598 1,134 298 26
[M+C2H4O2-H]- 0 376 0 232 0 992 367 861 0 830 3,778 0
[M-2H2O+H]+ 0 0 8 1,468 0 70 87 62 1,367 377 0 0
[M+2H]2+ 8 0 0 616 0 54 138 716 0 185 0 0
[2M+Na-2H]- 0 0 0 667 0 1 84 535 0 144 0 0
[2M+CH2O2-H]- 0 659 0 0 0 2 2 0 0 2 0 0
[M-H2O]+ 0 0 0 16 0 0 0 0 608 0 0 0
[M-CH3]- 0 0 0 0 0 186 19 0 0 414 0 0
[M-2H]- 0 0 0 0 0 314 13 0 0 127 0 0
[M]- 0 0 0 32 0 46 6 0 237 125 0 0
[2M+NH4]+ 0 0 0 19 0 6 125 97 0 131 0 0
[M-H2O-H]- 0 0 1 39 0 8 41 110 0 86 0 5
[M-H5O3]+ 0 0 0 215 0 0 2 0 0 13 0 0
[2M+C2H4O2-H]- 0 187 0 0 0 0 0 0 0 0 0 0

ionization_mode has exactly two values (positive 1,899,640; negative 639,968); adduct and adduct_orig have no nulls. Per-library distinct adduct strings: massbank 82, riken 74, mona 57, gnps 40 (42 orig), msdial 25, enveda-180 14, pluskal_ms2 11, enveda-np-examples 10, spectraverse 9, drug_plus 6, masaryk 1. The 14 enveda-180 adducts equal the 14 the Zenodo record advertises and are a subset of the 17 COMMON_ADDUCTS + 1 CATION_ADDUCTS ([M]+) hypotheses in the enveda-180 repo README (8 positive, 9 negative, 1 cation); [M-H4O2+H]+, [M+Br]- (25 rows survive), [M-H2O-H]- and [M-H2O+H]+ are the hypotheses that produced (almost) nothing in enveda-180.

1.2 adduct_orig -> adduct: what the "original" actually is

  • 121 distinct adduct strings, 123 distinct adduct_orig strings. The mapping is injective except two cases: [M+CH2O2-H]- <- {[M+CH2O2]- 52,460; [M+CHO2]- 30,728} and [M]+ <- {[Cat]+ 8,770; [M]+ 48}. No adduct has >= 3 origin strings.
  • adduct_orig is not the submitter's string. Every one of the 123 strings is already in bracketed Hill-formula form ([M+H4N]+, [M-HO]+, [M-H3O2]+, [Cat]+, [Anion]-). Raw GNPS/MassBank spellings (M+H, [M+H], M-H2O+H, [M+FA-H]-, [M+HCOO]-, M+ACN+H) never occur. The custody log therefore starts one hop downstream: whatever cleaned the raw strings (matchms-style clean_adduct plus a formula rewrite) ran before the file was cut, and its decisions are not auditable from the file.
  • Charge, multimer count and sign agree between adduct_orig and adduct in 100 % of rows (0 mismatches on each of the three parses).
  • The 18 string pairs where adduct_orig != adduct (183,785 rows, 7.2 % of train) are all cosmetic rewrites of the same ion except one:
adduct adduct_orig rows semantics
[M+NH4]+ [M+H4N]+ 53,452 same ion, Hill order
[M+CH2O2-H]- [M+CH2O2]- 52,460 mass-changing: orig literally = M + neutral formic acid as an anion (1.00783 Da heavier than formate). Rows: pluskal_ms2 50,785, gnps 1,675. Measured m/z fits formate (my recomputed ppm median 0.009), so the relabel is right, but the shipped ppm was computed from the orig (median 2,403 ppm, see 2.3)
[M+CH2O2-H]- [M+CHO2]- 30,728 same ion (formate)
[M-H2O+H]+ [M-HO]+ 21,682 same ion
[M]+ [Cat]+ 8,770 same ion (natively charged)
[M+C2H4O2-H]- [M+C2H3O2]- 7,436 same ion (acetate)
[M-2H2O+H]+ [M-H3O2]+ 3,439 same
[M+2H]2+ [M+H2]2+ 1,717 same
[2M+Na-2H]- [2M+Na-H2]- 1,431 same
[2M+CH2O2-H]- [2M+CHO2]- 665 same
[M-2H]- [M-H2]- 454 same string, physically dubious ion (z = 1 after loss of two H); matchms would rewrite [M-2H] to [M-2H]2-
[M]- [Anion]- 446 same
[2M+NH4]+ / [M-H2O-H]- / [2M+C2H4O2-H]- / [M+3H]3+ / [M+2Na-H]+ / [M-CO2-H]- [2M+H4N]+ / [M-H3O]- / [2M+C2H3O2]- / [M+H3]3+ / [M+Na2-H]+ / [M-CHO2]- 378 / 285 / 187 / 138 / 110 / 7 same

Fraction of rows with adduct_orig == adduct per library: masaryk 1.000, enveda-180 0.974, riken 0.961, massbank 0.945, drug_plus 0.943, gnps 0.907, mona 0.899, pluskal_ms2 0.856, msdial 0.852, enveda-np-examples 0.801, spectraverse 0.574 (its 16,810 [M+NH4]+ are all written [M+H4N]+).

1.3 Polarity contradictions (adduct sign vs ionization_mode): 6,087 rows

ingest_lib adduct ionization_mode rows instrument diagnosis (precursor_mz - exact mass, Da)
riken [M-H]- positive 4,870 Linear Ion Trap (4,872 of all riken contradictions) -0.94 to -1.01 Da: the ion IS [M-H]-; ionization_mode is wrong; low-resolution rows anyway
gnps [M+H]+ negative 441 Orbitrap 276, qTof 219 275 rows at -1.01 Da: the ion is really [M-H]- and the adduct is wrong; 59 rows at +1.01 Da: adduct right, mode wrong
mona [M-H]- positive 298 Linear Ion Trap 287 mixed: 38.95 Da (21 rows, really [M+K]+), 22.99 Da (19, really [M+Na]+), -1.01 (17)
massbank [M+H]+ negative 198 LC-ESI-QQ 107, LC-ESI-ITFT 89 mostly -1.01 Da (37 of the top bins): adduct wrong
gnps [M-H]- positive 81
riken [M-H2O+H]+ (orig [M-HO]+) negative 67
others ([M]+/[Cat]+ 26, [M+Na]+ 18, [M+Cl]- 18, [M+NH4]+ 10, [M-H2O-H]- 5, [M+H3O]+ 3, [M+Na-H2]- 3, [M-H2]2- 1) 104

Only 4.3 % of the contradicting rows have |recomputed ppm| < 10 under their labelled adduct, i.e. the label and the mass disagree in 96 % of them; matchms' derive_ionmode would log these as warnings and leave them (see section 5), which is exactly what the file shows. Dropping all 6,087 costs nothing (none are timsTOF).

2. Precursor-mass forensics

Method: RDKit CalcExactMolWt on normalized_smiles (277,566 unique SMILES, 0 parse failures), adduct mass from my own parser of the adduct string with monoisotopic masses (H 1.00782503, Na 22.98976928, K 38.96370649, Cl 34.96885268, Br 78.9183376, C 12, N 14.00307400, O 15.99491462, Ca 39.96259086) and electron mass 0.000548580 subtracted per positive charge (added per negative charge): proton = 1.007276, [M+Na]+ offset 22.989218, NH4+ 18.033823, Cl- 34.969402, formate (CH2O2-H + e) 44.998201. 120 of 121 adduct strings parse; [Cat]2+ (15 rows) does not. my_ppm = (precursor_mz - theo)/theo x 1e6, signed.

2.1 What the shipped precursor_error_ppm is (and is not)

test result (computed)
Sign shipped column is >= 0 in 2,535,125 of 2,535,322 non-null rows (197 negative values, min -21.06; q25 > 0 in every library). It is an absolute error.
Correlation with my signed ppm (rows with abs(my) < 50) r = -0.022
Hypothesis A: abs(precursor_mz - theo_no_electron) / theo x 1e6 where theo_no_electron = (nM x M + sum of neutral atom masses)/z, i.e. electron mass ignored reproduces the shipped value within 0.2 ppm for enveda-180 100.0 %, spectraverse 99.2 %, masaryk 99.1 %, drug_plus 95.3 %, riken 94.0 %, gnps 93.3 %, msdial 91.9 %, mona 91.5 %, massbank 89.3 %, pluskal_ms2 88.0 %; within 0.05 ppm for 54 % overall (per-library 31-99 %, so the host's atomic-mass table differs from mine at the 0.05-0.15 ppm level)
Hypothesis B: abs(my signed ppm) (electron included) matches 0.7 % overall, but 83 % of enveda-np-examples; in fact for enveda-np-examples the shipped value equals my signed ppm within 0.01 ppm in 100 % of rows (median 1.541 both). The 1,184 np-examples rows went through a different, correct code path.
Hypothesis D: host used the adduct_orig formula literally (no electron) identical to A everywhere except the formate rows: for [M+CH2O2]- it explains 0 %, for [M+CHO2]- 49 % - the host's number for the 52,460 [M+CH2O2]- rows is not reproducible from either string with modern masses; it is 1 Da wrong (median 2,403 ppm).

Consequence: a 1-charge ion at m/z 350 picks up a spurious 0.549 mDa = 1.57 ppm in the shipped column, positive in both polarities because of the absolute value. The train-EDA statement "median |ppm| = 1.51, 90 % within 5 ppm" is largely this artefact.

2.2 Is precursor_mz even a measurement? Fraction of rows with |my signed ppm| < 0.3 (i.e. m/z equals the computed ion mass)

pluskal_ms2 0.993, msdial 0.965, drug_plus 0.929, massbank 0.758, mona 0.690, spectraverse 0.603, riken 0.232, masaryk 0.224, gnps 0.159, enveda-180 0.085, enveda-np-examples 0.084. By instrument string: pluskal Orbitrap 0.993, riken ESI-QFT 0.974, mona ESI-QFT 0.974, massbank LC-ESI-QFT 0.897, massbank LC-ESI-QTOF 0.816, riken QTOF 0.031, riken Orbitrap 0.022, enveda-180 timsTOF 0.085. So for MSnLib, MS-DIAL, drug_plus and most of MassBank/MoNA the exported "precursor m/z" is the theoretical adduct mass written back by the library software; only enveda-180, enveda-np-examples, GNPS and RIKEN's QTOF/Orbitrap sub-libraries carry a measured value. Any model that uses precursor_error_ppm or precursor_mz - theoretical as a feature learns library identity, not mass accuracy.

2.3 Distribution per adduct (top 20) - shipped column vs recomputed signed ppm

adduct n ship median ship p90 ship >10 ppm my median (signed) my IQR my p90 of abs my >10 ppm rows where abs(ship - my) > 2 ppm note
[M+H]+ 1,324,585 1.370 3.76 4.8 % 0.818 -0.02..2.32 3.74 4.7 % 23.3 %
[M-H]- 465,688 1.875 3.65 3.3 % 0.070 -0.07..0.77 1.93 3.5 % 20.6 %
[2M+Na]+ 190,033 0.871 2.66 0.01 % 0.760 -0.13..1.63 2.85 0.01 % 17.2 % 99 % enveda-180
[M+Na]+ 178,092 2.715 15.38 20.5 % -0.038 -4.07..1.59 14.86 19.0 % 50.2 % riken QTOF median -2.5 ppm, 28 % > 10 ppm; riken Orbitrap 31 % > 10 ppm
[2M+H]+ 92,041 1.044 3.26 0.06 % 0.652 -0.43..1.67 3.35 0.06 % 22.4 %
[M+CH2O2-H]- 83,188 1,990 2,797 63.2 % 0.018 -0.07..0.11 1.03 0.08 % 64.4 % shipped value 1 Da wrong for the 52,460 [M+CH2O2]--origin rows; the m/z itself is fine
[2M-H]- 56,172 1.098 2.64 0.06 % -0.009 -0.82..0.76 2.38 0.06 % 13.8 %
[M+NH4]+ 53,452 0.915 2.84 3.6 % 0.015 -0.07..0.21 2.44 3.5 % 17.6 %
[M+Cl]- 27,511 1.396 1.77 0.15 % 0.004 -0.07..0.07 0.58 0.15 % 4.0 % 84 % pluskal (computed m/z)
[M-H2O+H]+ 21,682 2.069 6.60 8.3 % -0.024 -0.52..0.11 4.62 8.0 % 53.7 %
[M+K]+ 18,955 1.324 3.17 5.8 % -0.009 -0.11..0.15 2.62 5.9 % 23.9 %
[M]+ 8,818 1.675 4.57 6.6 % 1.426 0.09..2.07 5.18 7.5 % 24.9 %
[M+C2H4O2-H]- 7,436 0.687 4.49 2.3 % 0.002 -0.05..0.05 4.19 2.4 % 15.0 %
[M-2H2O+H]+ 3,439 1.723 2,048 19.1 % -0.013 -0.63..0.19 2,020 18.4 % 47.4 % ~18 % are 1 Da off (wrong water count)
[M+2H]2+ 1,717 1.198 506 15.1 % 0.012 -0.09..1.23 504 15.0 % 35.9 %
[2M+Na-2H]- 1,431 1.222 2.45 0 % 0.016 -0.49..0.13 1.49 0 % 10.6 %
[2M+CH2O2-H]- 665 1.855 4.06 0 % -0.298 -1.79..1.81 3.94 0 % 31.3 %
[M-H2O]+ 624 1.786 2.86 0 % 0.019 -0.07..0.14 0.23 0 % 27.4 %
[M-CH3]- 619 490 596 88 % -486 -549..-386 591 91 % 96.5 % MassBank/RIKEN phosphatidylcholines on LC-ESI-ITFT with 2-decimal precursor m/z (e.g. 743.29 vs exact [M-CH3]- 743.61): low-res + odd; 172 rows have null shipped ppm
[M-2H]- 454 0.000 25.2 10.0 % -0.016 -0.08..0.04 6.30 9.9 % 14.3 % shipped exactly 0 for half: precursor computed as M - 2 x H atom

2.4 Per library

ingest_lib n ship median ship p90 ship >10 ppm ship null my median (signed) my IQR my p90 abs my >10 ppm abs(ship - my) > 2 ppm
drug_plus 2,545 1.734 3.88 3.5 % 0 -0.002 -0.08..0.08 0.23 3.5 % 38.5 %
enveda-180 1,153,785 0.981 2.77 0.00 % 0 1.567 0.59..2.40 3.14 0.00 % 10.7 %
enveda-np-examples 1,184 1.541 3.19 0.17 % 0 1.541 0.49..2.20 3.19 0.17 % 0.0 %
gnps 220,849 1.673 6.62 8.3 % 91 0.098 -0.90..1.08 4.41 7.6 % 43.8 %
masaryk 652 1.479 2.82 0.15 % 0 0.580 0.22..1.05 1.57 0.15 % 18.4 %
massbank 101,727 1.949 4.90 7.9 % 3,170 0.002 -0.11..0.11 11.46 10.2 % 48.7 %
mona 92,416 1.799 4.69 6.4 % 746 -0.016 -0.14..0.11 3.75 6.6 % 44.5 %
msdial 40,765 1.470 3.41 0.24 % 0 0.000 -0.06..0.06 0.19 0.16 % 30.1 %
pluskal_ms2 527,581 1.664 3.92 9.6 % 215 0.010 -0.06..0.08 0.14 0.00 % 22.4 %
riken 347,171 3.753 22.49 26.5 % 64 -0.067 -5.23..1.84 21.89 25.3 % 56.8 %
spectraverse 50,933 0.650 1.64 0.05 % 0 0.032 -0.04..0.32 1.15 0.01 % 7.6 %

The ">2 ppm disagreement" column is dominated by the abs/electron convention, not by adduct errors; the adduct-level anomalies are the formate, [M-2H2O+H]+, [M+2H]2+, [M-CH3]- rows above and RIKEN's 25 % of rows beyond 10 ppm (riken QTOF 29.5 %, riken "Orbitrap" 32.6 %, riken Linear Ion Trap 97.7 %; riken ESI-QFT 2.1 %). Nothing in massbank/mona/pluskal at the >2 ppm level is a real adduct mis-assignment beyond the formate case.

2.5 timsTOF rows (enveda-180 + enveda-np-examples): recomputed signed ppm per adduct (for q6's retrieval window)

ingest_lib adduct n median q25 q75 p90 abs p99 abs max abs > 5 ppm > 10 ppm
enveda-180 [M+H]+ 588,507 2.214 1.622 2.791 3.321 4.114 5.13 0.01 % 0
enveda-180 [2M+Na]+ 188,158 0.771 -0.125 1.631 2.847 4.556 5.06 0.03 % 0
enveda-180 [M-H]- 184,747 0.804 0.215 1.385 1.907 2.762 5.08 0.00 % 0
enveda-180 [2M+H]+ 89,487 0.686 -0.419 1.690 3.350 4.759 5.07 0.04 % 0
enveda-180 [2M-H]- 52,933 0.006 -0.848 0.789 2.390 4.466 5.06 0.03 % 0
enveda-180 [M+CH2O2-H]- 24,556 0.311 -0.278 0.895 1.676 3.793 5.08 0.06 % 0
enveda-180 [M+Na]+ 15,957 0.961 0.026 1.759 2.853 4.519 5.13 0.03 % 0
enveda-180 [M+NH4]+ 4,702 1.420 -0.650 2.462 4.021 4.904 5.02 0.30 % 0
enveda-180 [M+Cl]- 2,407 -0.142 -0.901 0.570 2.440 4.364 4.88 0 0
enveda-180 [M+K]+ 1,084 0.210 -2.027 1.967 4.031 4.957 5.04 0.55 % 0
enveda-180 [2M+CH2O2-H]- 659 -0.275 -1.788 1.809 3.944 4.889 4.94 0 0
enveda-180 [M+C2H4O2-H]- 376 0.319 -1.620 1.487 3.966 4.996 5.01 0.53 % 0
enveda-180 [2M+C2H4O2-H]- 187 0.185 -1.209 2.085 3.785 4.562 4.63 0 0
enveda-180 [M+Br]- 25 0.576 -0.293 1.383 2.833 3.054 3.06 0 0
enveda-np-examples [M+H]+ 569 2.003 1.469 2.583 3.383 6.565 8.71 2.5 % 0
enveda-np-examples [M-H]- 276 0.557 -0.217 1.256 2.012 6.110 21.06 2.5 % 0.4 %
enveda-np-examples [M+CH2O2-H]- 105 0.192 -0.319 1.053 2.361 3.477 7.73 1.0 % 0
enveda-np-examples [M+NH4]+ 98 1.220 0.530 2.033 2.985 3.690 4.01 0 0
enveda-np-examples [M+Na]+ 80 1.574 1.045 2.121 3.744 6.149 6.87 5.0 % 0
enveda-np-examples [M-H2O+H]+ 24 2.125 1.563 3.124 4.250 6.634 7.30 4.2 % 0
enveda-np-examples [M+K]+ 16 1.134 0.380 2.287 5.701 10.878 11.51 12.5 % 6.3 %
enveda-np-examples [M-2H2O+H]+ / [M+Cl]- / [M-H2O-H]- 8 / 7 / 1 1.688 / -0.066 / -5.012 4.18 / 2.38 / 5.01

Three facts for the window: (i) enveda-180 is hard-truncated at |true ppm| <= 5.13 (99.984 % within 5, 100 % within 6; by polarity the 0.1-99.9 % range is -4.74..+4.84 ppm positive, -4.61..+4.63 negative), consistent with the query_ppm_window(..., ppm=5) default in the enveda-180 extraction code applied to the correct (electron-corrected) ion mass; enveda-np-examples, built by a different path, reaches 21 ppm. (ii) The distribution is not centred on zero: positive mode median +1.88 ppm (median offset +0.72 mDa for [M+H]+, +0.53 mDa for [2M+Na]+, +0.47 for [2M+H]+), negative mode +0.62 ppm (+0.26 mDa for [M-H]-, +0.12 for formate, -0.06 for Cl). A +0.5 mDa constant offset in positive mode is numerically the electron mass; whether Enveda's calibrant list omits it is unverified (hypothesis, confidence low). (iii) Coverage of symmetric windows on enveda-180: +-2 ppm 60.5 %, +-3 ppm 87.7 %, +-4 ppm 98.1 %, +-5 ppm 99.98 %. A window of -4/+5 ppm, or +-5 ppm centred on +1.5 ppm (positive mode), is the tightest safe choice; the ~0.3 % of np-examples rows beyond 5 ppm say the hidden set (different acquisition batch) may not be truncated.

3. Collision-energy custody

3.1 Units and nulls per library (rows; computed)

ingest_lib eV NCE V unknown null collision_energy_orig empty collision_energy_ev list
drug_plus 0 0 0 2,545 100 % 100 %
enveda-180 1,153,785 0 0 0 0 % 0 %
enveda-np-examples 1,184 0 0 0 0 % 0 %
gnps 0 0 0 220,849 100 % 100 % (even the 2,165 rows whose instrument_type says Quattro_QQQ:10eV/25eV/40eV)
masaryk 0 0 0 652 100 % 100 %
massbank 41,726 39,660 18,672 1,669 0.3 % 1.9 %
mona 16,501 51,234 9,821 14,860 15.0 % 17.0 %
msdial 7,354 9,165 11,025 13,221 31.2 % 41.7 %
pluskal_ms2 0 527,581 0 0 0 % 0 %
riken 147,110 158,807 17,038 24,216 6.6 % 9.1 %
spectraverse 1,248 2,619 0 47,066 91.8 % 92.4 %

"V" rows are Waters/Agilent/Shimadzu QTOF and QqQ records (LC-ESI-QTOF 37,780, LC-ESI-QQ 16,401) and were copied 1:1 into eV ("6V" -> 6, "30 V" -> 30), which is correct for z = 1 lab-frame energies. 5,213 rows have a non-null collision_energy_orig but unit "unknown" and an empty eV list, i.e. the value was discarded: 30.0,50.0,70.0 (304 rows), 15, 30, 45, 60, 70 or 90 (nominal) (416), 0.6458333 (547), 3 kV (175), 0eV (239), plain 10.0/30.0/40.0 (505). Multi-value strings were kept as lists in the eV column where the unit was recognised: RIKEN/MS-DIAL Ramp 5-60 V -> [5, 60]; RIKEN 30-50 eV -> [30, 50]; MassBank -10, -35, -60V (189 rows) and 25,60,100% (stepped) (31) exist in orig.

3.2 NCE -> eV: converted, not copied, with the Thermo rule

Only 0.15 % of the 789,066 NCE-unit rows have ev[0] == NCE (copy). Instead ev = round(NCE x precursor_mz / 500 x f(z), 1) with f(1) = 1.000, f(2) = 0.900, f(3) = 0.850 (median ratios computed over 787,840 / 739 / 111 rows; |difference| < 0.15 eV for 90.6 % / 98.4 % / 100 %; for massbank, mona, riken, spectraverse 95-100 %, residual |diff| <= 0.049 eV = rounding). This is Equation 3 of MassSpecGym (CE = m x NCE / 500) extended with the charge factors of Thermo's HCD implementation. The 12 % "mismatches" in pluskal_ms2 (62,245 rows) are an ordering artefact, not a conversion error: collision_energy_orig keeps acquisition order ([20.0, 15.0, 60.0, 40.0], only 30.6 % of multi-value strings are sorted) while collision_energy_ev is sorted ascending in 100 % of 174,722 multi-value rows ([14.0, 18.7, 37.5, 56.2] at m/z 468.14), with list lengths matching in 99.999 %. The remaining msdial mismatches (49 % of its 9,165 NCE rows) are rows with null instrument_type, where the same formula still reproduces the median ratio 1.0003. MassBank free-text NCE strings were truncated to their first stepped value: 42% (nominal) with stepped collision energies 30% and 55% -> [13.7] at m/z 228.10 (= 30 x 228.1/500), Ramp 20%-70% (nominal) -> [10.3] at m/z 286.23, z = 2 (= 20 x 286.23/500 x 0.9), 15,30,45 (NCE) -> single value [14.9] at m/z 553.30, z = 2. So for MassBank the ramp/stepped information survives only in collision_energy_orig.

Caveat on what "NCE" means for pluskal_ms2: the MSnLib paper text says "For MS2, the energies were set to 20 eV and 60 eV" and "three fixed collision energies of 20 eV, 40 eV and 60 eV" for MS3, yet the file labels all 527,581 rows NCE and the orig strings are 60.0/20.0/[60.0]/[20.0, 30.0, 60.0, 40.0] (74,710 / 72,490 / 46,171 / 20,978 rows). The host converted them as NCE (20 NCE at m/z 343 -> 13.7 eV). If the paper's "eV" were literal, every pluskal eV value is ~1.5x too low at m/z 350; the brief's note that the authors confirmed NCE is the only thing settling it (not independently verified here).

3.3 List lengths and enveda-180 patterns

ingest_lib len 0 len 1 len 2 len 3 len 4 len 5 len 6
enveda-180 0 847,063 1,942 304,780 0 0 0
enveda-np-examples 0 773 213 82 101 15 0
pluskal_ms2 0 352,859 3,548 82,949 83,611 4,598 16
massbank 1,912 95,273 3,983 559 0 0 0
mona 15,745 72,472 722 3,477 0 0 0
msdial 16,985 22,490 505 785 0 0 0
riken 31,680 311,875 1,649 1,967 0 0 0
spectraverse 47,075 3,858 0 0 0 0 0
drug_plus / gnps / masaryk all 0 0 0 0 0 0

enveda-180 collision_energy_orig -> collision_energy_ev: 40 -> 40 (244,197), 20,40,60 -> 20,40,60 (240,127), 60 (229,217), 20 (173,189), -40 -> 40 (83,271), -60 -> 60 (70,248), -60,-40,-20 -> 20,40,60 (64,653), -20 -> 20 (46,941), 20,40 (1,037), -40,-20 (739), 20,60 (86), 40,60 (42), -60,-20 (30), -60,-40 (8). The sign is a perfect polarity flag: all 265,890 negative-mode rows have a negative orig CE, all 887,895 positive rows a positive one (the extraction code applies abs() when it writes collision_energies=[abs(ix[1])], extract.py line 308). Net eV patterns: 40 (327,468), 20,40,60 (304,780), 60 (299,465), 20 (220,130), 20,40 (1,776), 20,60 (116), 40,60 (50). No 4-step lists in enveda-180; enveda-np-examples uses a different scheme (20;50 209 rows, 20;40;60;80 100, 0;20;40;60;80 15, 35 124, 80 110, 16 rows whose merged list contains 0 eV).

4. Merged multi-CE spectra and intensity floors

4.1 Merged vs single in enveda-180 (computed from the peak lists)

rows num_peaks median / mean / p90 base_peak_intensity median (raw counts) min non-zero normalised intensity: median / min peaks < 1 % of base, median fraction
single CE (len 1) 847,063 116 / 189 / 461 149,245 1.8e-4 / 4.9e-8 -
merged (len 3: 20,40,60) 304,780 258 / 406 / 966 459,725 5.0e-5 / 3.0e-8 -
merged (len 2: 20,40 / 20,60 / 40,60) 1,942 8.5 / 6 / 5 (medians per pattern) 3,871 / 2,425 / 1,795 0.014-0.086 -
all enveda-180 1,153,785 138 200,233 1.3e-4 / 3.0e-8 0.89

For the 297,183 (structure, adduct) keys that have both a merged row and single-CE rows (1,067,456 rows): merged num_peaks / sum of single num_peaks has median 0.78 (<= sum in 80.8 %, equal in 0.7 %); merged / largest single = 1.73; merged >= largest single in 99.7 %. Merged base_peak_intensity / sum of the singles' base-peak intensities has median 1.00 (vs 1.80 against the largest single): the merge sums raw intensities per m/z across the three energies (sum_intensity_per_msms_mz in extract.py) and then re-normalises to the new maximum, which is why the merged floor sits ~3.5x lower. The 2-CE merged rows are degenerate (5-9 peaks, base peak ~2-4 x 10^3 counts), i.e. precursors seen at only two energies with weak signal. Set patterns per key: 20 | 20,40,60 | 40 | 60 185,449 keys; 20,40,60 | 40 | 60 73,512 (20 eV single missing but inside the merge); 40 | 60 31,497; 20 | 20,40,60 | 40 20,026; 40 alone 8,472; 20 | 20,40,60 8,208; 20,40,60 alone 7,744.

Visible test (1,213 rows, verbatim enveda-180): 298 rows (24.6 %) are merged 20,40,60, 915 single (60: 363, 40: 362, 20: 190); 68.5 % of the 400 molecules have at least one merged row; per-molecule patterns mirror train (20|20,40,60|40|60 83 molecules, 20,40,60|40|60 59, 40|60 32, 60 25). Test collision_energy_orig keeps the negative sign (-40 81, -60 74, -60,-40,-20 41, -20 30 = 226 rows = exactly the 226 negative-mode rows). Test adducts: [M+H]+ 959, [M-H]- 193, [M+CH2O2-H]- 31, [M+Na]+ 22, [M+NH4]+ 4, [M+K]+ 2, [M+Cl]- 2. Test min non-zero intensity: min 2.3e-7, median 6.0e-5 (between enveda-180 single and merged medians, as expected for a 25 % merged mix); base_peak_intensity non-null in 100 %.

4.2 Per-library intensity floors (min non-zero ms2_normalized_intensities per spectrum; all libraries have max = 1.0 in 100 % of rows)

ingest_lib floor min p01 median most frequent per-spectrum minima base_peak_intensity null peaks above precursor + 0.5 Da (fraction of spectra)
enveda-180 3.0e-8 8.5e-7 1.3e-4 continuous (1.3e-6..1.9e-6 each ~2,300 rows) 0 % 46.0 %
enveda-np-examples 8.2e-7 2.7e-6 4.1e-4 continuous 0 % 50.0 %
gnps 7.1e-13 1.6e-5 2.1e-3 1.0 (1,111 single-peak rows), 0.005 (1,105), 0.02, 0.006 0 % 40.2 %
masaryk 1.0e-3 1.0e-3 2.9e-3 continuous above 0.001 0 % 0 %
massbank 1.001e-3 1.001e-3 1.1e-2 0.001001 = 1/999 (16,354 rows), 0.005005, 0.01001, 0.002002 100 % 9.6 %
mona 1.0e-8 1.1e-5 1.5e-3 0.005 (2,691), 0.006, 0.007, 0.01 100 % 46.8 %
msdial 5.0e-3 1.0e-2 1.6e-2 0.01 (11,097), 0.011, 0.012, 1.0 100 % 14.6 %
pluskal_ms2 7.0e-5 3.3e-4 5.1e-3 continuous, 5-digit (0.00138, 0.00125 ...) 100 % 3.9 %
riken 5.0e-3 5.0e-3 1.5e-2 0.01 (59,260), 1.0 (41,370 single-peak rows), 0.005 (39,891) 100 % 37.4 %
spectraverse 1.0e-3 1.0e-3 5.1e-3 0.05, 0.01, 0.001, 0.1667 (=1/6), 0.2174 (=5/23) 100 % 28.4 %
drug_plus 1.0e-4 1.0e-4 6.6e-4 continuous just above 1e-4 100 % 26.2 %

q8's suspicion is confirmed: the floor is a library barcode. MassBank's 1/999 (integer 1-999 relative scale), MS-DIAL's and RIKEN's 0.01/0.005 (integer-percent exports), spectraverse's rational minima (1/6, 5/23 from tiny integer spectra), drug_plus's 1e-4 cut, pluskal's 7e-5, and the timsTOF rows' absence of any floor (minima to 3e-8, 89 % of peaks below 1 % of the base peak) are all mutually exclusive. base_peak_intensity being null in exactly the seven public libraries and present in the four others is a second barcode. The test has the timsTOF signature (no floor, raw base-peak counts present), so a model trained to exploit "no floor => enveda" is learning the right domain for the wrong reason.

5. What the sources say about harmonisation

  • matchms (matchms.filtering): clean_adduct "transform[s] adduct strings of type 'M+H+' to '[M+H]+'" by stripping whitespace/asterisks, replacing ++/-- with 2+/2-, adding brackets and moving the charge outside, then applying known_adduct_conversions.csv (47 rules, e.g. [M+H-H2O] -> [M-H2O+H]+, [M+CH3COOH-H] -> [M+CH3COO]-, [M-H1] -> [M-H]-, [M-2H] -> [M-2H]2-, [M-H-/M-Ser]- -> [M-H]-). derive_adduct_from_name: "Find adduct in compound name and add to metadata (if not present yet)". derive_ionmode: "Derive missing ionmode based on adduct"; when the charge-derived ionmode disagrees with the adduct-derived one the code "logs a warning ... but does not update ionmode". require_matching_adduct_precursor_mz_parent_mass default tolerance 0.1 (Da). repair_adduct_based_on_parent_mass / repair_adduct_and_parent_mass_based_on_smiles pick the adduct whose implied parent mass matches the SMILES mass within a tolerance. None of these create the Hill-formula strings seen in adduct_orig; that rewrite is Enveda's.
  • MassSpecGym (arXiv 2410.23326, Supplementary Table 2, read from the PDF): sequential matchms report - clean_adduct changed 8,102; derive_adduct_from_name 376,859; correct_charge 322,993; repair_adduct_based_on_smiles 305,728; require_adduct_in_list removed 41,049; require_correct_ionmode removed 153,321; require_matching_adduct_precursor_mz_parent_mass removed 3,726; require_parent_mass_match_smiles 8,694; remove_not_ms2_spectra 628,478; then "< 50 % explained intensity" 135,190, "No peaks shared with NIST" 14,699, "Cosine similarity against NIST < 0.2" 9,290; final 231,104 spectra. Text: "First all merged spectra in the MS/MS library from Brungs et al. were removed"; "We further only retained spectra of two adducts, [M+H]+ or [M+Na]+"; positive mode only; CE: "CE(p) = m(p) x NCE(p) / 500 ... applied to all collision energies that were identified as normalized. In cases of ambiguity, we assumed that the provided collision energy was not normalized"; candidate retrieval uses "10 parts per million (ppm)" on mass. The train file does the opposite on every count: merged MSnLib-style spectra kept (pluskal len 3-6 lists: 171,174 rows), 121 adducts, both polarities, ambiguous CE strings converted anyway.
  • enveda-180 (github.com/enveda/enveda-180; Zenodo 10.5281/zenodo.20436851 v1.0.0, 29 May 2026): README lists 8 positive + 9 negative COMMON_ADDUCTS and [M]+ as CATION_ADDUCTS; load.py compute_adduct_metadata applies offset -= charge * ELECTRON_MASS (0.0005485799); utils.py query_ppm_window(mz, df, key, ppm=5); extract.py per-CE branch writes collision_energies=[abs(ix[1])], collision_energies_merged=False, the merged branch requires collision_energies.nunique() >= 2, sets collision_energies_merged=True, num_merged_microscans, num_merged_precursor_ions, and intensities are combined by sum_intensity_per_msms_mz. Zenodo: "Bruker timsTOF Pro 2 ... PASEF mode, under heated electrospray ionization (300 degC, 4500 V)", three nominal CEs "20, 40, 60 eV", 14 adducts, "[M+H]+ and [M-H]- together account for roughly 67 % of all spectra", curation "SIRIUS rank-1 fragmentation-tree scoring, explained-intensity and minimum-peak thresholds, precursor-integrity checks, and round-trip InChIKey validation". Train's collision_energies_merged flag is not shipped; it is recoverable exactly as len(collision_energy_ev) > 1.
  • MSnLib (Brungs et al., Nat Methods 2025, PMC12510872): MS2 "energies were set to 20 eV and 60 eV", assisted CE "in 15 eV steps, starting at 15 eV ... 75 eV", MS3/MS4 "20 eV, 40 eV and 60 eV", MS5 "40 eV and 60 eV"; spectrum types SINGLE_BEST_SCAN, SAME_ENERGY, ALL_ENERGIES ("all fragmentation energies for individual precursors (three energies in our method)"), ALL_MSN_TO_PSEUDO_MS2; noise "factor of 2.5", MS1 minimum intensity "6 x 10^5 a.u. in positive and 2 x 10^5 a.u. in negative". pluskal_ms2's len-3/4/5 lists are these ALL_ENERGIES merges; its 15.0 singles (22,722 rows) are the assisted-CE scans.
  • Thermo PSB 104 (Normalized Collision Energy Technology): CE "set using a simple scale of 0-100 %"; the optimum "follow[s] a linear correlation with m/z"; the bulletin gives no equation and no charge factor - the m/z / 500 and f(z) factors are instrument-software conventions reproduced empirically above.
  • PASEF CE on timsTOF (Meier et al. 2018, MCP, PMC6283298): "the collision energy was ramped stepwise as a function of increasing ion mobility: 52 eV for 0-19 % of the ramp time; 47 eV from 19-38 %; 42 eV from 38-57 %; 37 eV from 57-76 %; and 32 eV for the remainder" - i.e. on a timsTOF the "eV" a method requests can be a 1/K0-dependent ramp, not one number. Whether Enveda's 20/40/60 are fixed energies or ramp anchors is not stated in any source I could open (Bruker method documentation not retrieved; unverified).

Forced analogy: forensic chain of custody

Statement. Treat each metadata value as evidence: it was collected by vendor software (CE setpoint, measured m/z), labelled by a submitter (adduct, polarity), transferred by an aggregator (MoNA/GNPS/MassBank exports, matchms cleaning), and re-labelled by Enveda's harmoniser (adduct_orig -> adduct, collision_energy_orig -> collision_energy_ev, precursor_error_ppm). A custody break is any hop where the value could change without a record. Evidence is admissible only if every hop is either recorded or reproducible.

Testable idea 1 (implemented above): reproduce every harmoniser hop and classify breaks as mass-neutral or mass-changing. 17 of 18 adduct rewrites are mass-neutral; 1 ([M+CH2O2]-) changes the neutral mass by 1.00783 Da for 52,460 rows and the shipped ppm still carries the pre-relabel value. The CE hop is reproducible to 0.05 eV (Thermo rule + ascending sort) for 99.8 % of NCE rows, so it is a recorded, harmless break; the MassBank ramp/stepped strings are an unrecorded lossy break (ramp bounds collapsed to the first value). The ppm hop is a lossy break with two code paths (absolute/no-electron for 10 libraries, signed/electron-corrected for np-examples). Prediction: a candidate pool built from precursor_mz and the shipped adduct with the correct electron-corrected mass will contain the true molecule for 99.98 % of enveda-180 rows inside +-5 ppm; a pool built from the shipped ppm or from adduct_orig will systematically miss the formate rows and mis-centre positive-mode windows by ~1.5 ppm. This is directly measurable on the decoy test (should be 100 %) and on a held-out enveda-180 slice.

Testable idea 2: use custody completeness as a sample weight. Rows with (a) measured (not computed) precursor m/z, (b) recorded CE in eV or reproducible NCE, (c) adduct sign consistent with polarity, (d) |recomputed ppm| <= 5 are "admissible"; the rest are hearsay. By this rule 100 % of enveda-180 passes, 100 % of gnps fails on CE, 25 % of riken fails on ppm and 9 % on CE. Train a spectrum->fingerprint model with and without the hearsay rows and compare on a timsTOF-only, structure-disjoint validation set.

Where it holds. The analogy holds for the harmoniser hops: they are deterministic transforms and I could reconstruct them from the file alone (Thermo rule, sort, abs(), electron omission). It holds for polarity: the sign of collision_energy_orig in enveda-180 is an independent witness that confirms ionization_mode in 100 % of rows, a true two-source corroboration.

Where it breaks. (1) Custody assumes the original evidence existed: for 7 of 11 libraries precursor_mz was never a measurement, so "mass accuracy" has no custody chain to audit - the question is moot, not falsified. (2) A perfect chain can carry a wrong label from hop zero: GNPS [M+H]+/negative rows whose m/z fits [M-H]- are a submitter error that no downstream check touched; custody is silent on origin errors. (3) Custody treats every break as equally serious; here the only breaks that matter for MRR@25 are the mass-changing ones (formate relabel, water-loss count in [M-2H2O+H]+, polarity-adduct pairs), because they move the neutral mass and therefore the candidate pool, whereas cosmetic string breaks and CE rounding do nothing to the ranking. (4) The analogy says nothing about the physics: a CE value can be custody-perfect and still mean different things on a timsTOF (mobility ramp) versus an Orbitrap (NCE), which is the bigger confounder for transfer learning.

Gotchas / what could fool us

  1. precursor_error_ppm is an absolute value computed without the electron mass: it inflates every singly-charged row by ~0.55 mDa (1.57 ppm at m/z 350), it is positive in both polarities, and it is uncorrelated (r = -0.02) with the true signed error. Never use it as a feature or a filter; recompute.
  2. Two conventions in one column: enveda-np-examples ppm is signed and electron-corrected; everything else is absolute and uncorrected. A per-library ppm plot will show np-examples as "different chemistry" when it is only a different script.
  3. 52,460 formate rows carry a 1-Da-wrong shipped ppm (median 2,403) although precursor_mz and adduct are correct; a filter "drop |ppm| > 20" silently deletes 96 % of pluskal_ms2 negative-mode formate adducts and 50 % of gnps's.
  4. Computed precursor m/z: pluskal_ms2 (99.3 %), msdial (96.5 %), drug_plus (92.9 %), massbank (75.8 %), mona (69 %), spectraverse (60 %) have precursor_mz equal to theory. Any model fed precursor_mz - theoretical learns library identity. A validation split that mixes these with measured-m/z libraries will overstate mass-window recall.
  5. enveda-180 precursor errors are truncated at 5 ppm by the extraction pipeline, and biased +1.9 ppm in positive mode. A +-3 ppm window built from the (artefactual) shipped distribution loses 12 % of enveda-180 rows; a window centred on 0 loses the right tail first.
  6. RIKEN: 25 % of rows > 10 ppm by recomputation (QTOF 29.5 %, "Orbitrap" 32.6 %, Linear Ion Trap 97.7 %) and 4,870 Linear-Ion-Trap rows with polarity/adduct contradiction. RIKEN's [M+Na]+ (121,677 rows, 68 % of all sodium adducts in train) has median -2.5 ppm on QTOF; a sodium-adduct model trained on train is a RIKEN-QTOF model.
  7. adduct_orig is not original; the raw spellings and the first cleaning step are not in the file. Do not count on it to recover lost information (e.g. [M+FA-H]- vs [M+HCOO]- provenance).
  8. CE lists are sorted ascending, orig is not; aligning collision_energy_ev[i] with the i-th token of collision_energy_orig is wrong for 69 % of pluskal multi-CE rows.
  9. MassBank ramp/stepped CEs are collapsed to the first value (Ramp 20%-70% -> 20 NCE; 30% and 55% -> 30). Treat MassBank eV values from free-text strings as lower bounds.
  10. GNPS has no CE at all (220,849 rows), including rows whose instrument_type string encodes the energy (Quattro_QQQ:25eV). Any "CE-conditioned" model is trained on 87 % of rows without GNPS and on 0 % of GNPS.
  11. Merged spectra are intensity sums, not averages or unions: peaks that appear only at 60 eV are down-weighted by the 20/40 eV base peak; merged num_peaks is 0.78x the sum of singles because peaks coincide within the m/z merge tolerance. 25 % of visible-test rows (and by construction of the hidden set presumably a similar fraction) are of this kind; a model that only saw single-CE spectra sees a different intensity distribution at test time.
  12. Intensity floors and base_peak_intensity nulls are library barcodes (MassBank 1/999, MS-DIAL/RIKEN 0.01, pluskal 7e-5, timsTOF none). Two cheap features that a tree model will find in one split. The test has the timsTOF barcode, so this leak helps on the decoy and is neutral-to-harmful on the hidden set (no public-library rows there).
  13. 46 % of enveda-180 spectra contain peaks above the precursor m/z + 0.5 Da (50 % of np-examples, vs 3.9 % pluskal, 9.6 % massbank). Whatever these are (isotopologues of the precursor, co-isolated ions, multimer fragments), they are a timsTOF-pipeline feature that in-silico fragmenters and formula-explained-intensity scorers will penalise unless the spectrum is trimmed at the precursor.
  14. 2-CE merged rows in enveda-180 (1,942) are junk (5-9 peaks, base peak ~2e3 counts); they exist because the merge branch only needs two distinct energies.
  15. [M-2H]-, [M-CH3]-, [Cat]2+, [M-H5O3]+ and the ~70 singleton adduct strings (one to ten rows each) are curiosities that cannot be learned; 33 of 121 adducts have < 5 rows.

Implications for the challenge

  1. Recompute precursor error from normalized_smiles + electron-corrected adduct mass for every train row; drop the shipped precursor_error_ppm column entirely (it is absolute, electron-free, two-convention and 1 Da wrong on 52k rows).
  2. Retrieval window for timsTOF (q6): +-5 ppm around the electron-corrected adduct m/z covers 99.98 % of enveda-180 and 99.7 % of np-examples; use an asymmetric -4/+5 ppm or shift the centre by +1.5 ppm in positive mode if the window must be tighter. Do not go below +-4 ppm (98.1 % coverage). Expect the hidden set (a different acquisition batch) to be un-truncated like np-examples (0.3 % beyond 5 ppm, max 21 ppm), so keep a +-10 ppm fallback tier for molecules whose top-25 is otherwise empty.
  3. Adduct-conditioned mass hypotheses for the hidden test should cover exactly the 7 test adducts ([M+H]+, [M-H]-, [M+CH2O2-H]-, [M+Na]+, [M+NH4]+, [M+K]+, [M+Cl]-) with the offsets 1.007276, -1.007276, 44.998201, 22.989218, 18.033823, 38.963158, 34.969402; [2M+X] and water-loss adducts are absent from the visible test and would double the candidate pool for nothing.
  4. Filter train: drop the 6,087 polarity contradictions, the 1,942 two-CE enveda-180 merges, rows with |recomputed ppm| > 10 outside enveda-180 (riken loses 25 %), and consider dropping or down-weighting the 7 computed-m/z libraries for any mass-window experiment.
  5. CE as a model input: use collision_energy_ev (converted, sorted) but add an indicator for "list length > 1" (merged) and for the source unit (eV vs NCE-converted vs V vs none); treat GNPS (no CE) and MassBank free-text ramps as "unknown", not as 0.
  6. Train with merged spectra explicitly: 26.6 % of enveda-180 rows and 24.6 % of the visible test are 20/40/60 sums; build augmentation that sums the raw-intensity single-CE spectra of the same (structure, adduct) (the 185,449 keys with all three singles plus the merge give exact supervision for this operation) and re-normalises, so the model sees the test-time intensity distribution.
  7. Validation: any local metric computed on a mix of libraries is contaminated by the floor/base-peak/CE barcodes; validate on enveda-180 + np-examples only, structure-disjoint, with the 25 % merged ratio preserved.
  8. Peaks above the precursor: decide once (trim at precursor + 0.5 Da or keep) and apply identically to train and test; 46 % of timsTOF spectra are affected, in-silico scorers (SIRIUS/CFM-ID/MetFrag) will otherwise lose explained intensity on nearly half the test.
  9. Formate rows are fine to keep (adduct and m/z correct); only their shipped ppm is wrong.

Open questions

  • What are the peaks above the precursor m/z in 46 % of enveda-180 spectra (13C isotopologues of the precursor, co-isolated multimers, or the uncleaned isolation window)? Needs a per-spectrum look at (peak - precursor) offsets; q8 territory.
  • Does Enveda's timsTOF calibration omit the electron mass (the +0.5 mDa positive-mode offset), or is it a genuine instrument bias? A plot of offset vs m/z on np-examples (different batch) would discriminate: constant mDa offset = calibrant list, proportional = calibration slope.
  • Are the enveda-180 20/40/60 eV values fixed energies or 1/K0-ramp anchors (Bruker method files not retrieved)? If ramped, "40 eV" on a heavy, slow ion is a different energy than on a light one, and CE should be modelled jointly with m/z (or CCS).
  • Is pluskal_ms2 truly NCE (host's assertion) when the Nat Methods text says eV? If the paper is literal, 527k rows have CE values 1.4-1.7x too low.
  • Why do 73,512 enveda-180 (structure, adduct) keys have a 20/40/60 merge but no 20 eV single? If the 20 eV single was dropped by curation (low explained intensity at low CE?) the merge inherits uncurated peaks.
  • Who produced the Hill-formula adduct_orig strings and what did the raw submitter strings look like? The GNPS/MassBank originals could still be re-joined via SPLASH or peak hashing if the pre-cleaning file were available.

Sources