TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks
TabPrep exposes a structural blind spot in tabular ML evaluation: modern benchmarks measure model architecture sophistication while ignoring feature engineering, which dominates real-world pipelines. The work demonstrates that carefully targeted preprocessing can outperform architectural innovation on standard benchmarks, suggesting the field has optimized the wrong variable. This reframes the tabular ML research agenda and implies that published model comparisons may systematically undervalue engineering-first approaches, affecting how practitioners prioritize investment in modeling infrastructure versus algorithm development.62
