Science requires reproducibility. However, the replication crisis in medical, psychological, biological, and social sciences has revealed that satisfactory reproducibility is only achieved when it is measured, investigated, and critically discussed. Despite the inherent sensitivity and complexity of heterogeneous electrocatalysis, little is known about the reproducibility of its experimental protocols. Herein, we present a global interlaboratory study of nickel-iron-based oxygen evolution electrocatalysts, revealing substantial reproducibility challenges. Using a process characterization tool of the pharmaceutical industry, we identify that these reproducibility challenges originate from undescribed but critical process parameters. Finally, we discuss solution approaches to address electrocatalysis’ inherent experimental complexity without affecting academia’s explorative nature. This report sets a data-based foundation for a critical discussion of reproducibility and a following credibility revolution in heterogeneous catalysis research.