Robbin Bouwmeester (Zwijnaarde / BE), Holda A. Anagho (Copenhagen / DK), Sven Degroeve (Zwijnaarde / BE), Nadezhda T Doncheva (Copenhagen / DK), Viktoria Dorfer (Hagenberg / AT), Klemens Fröhlich (Basel / CH), Ralf Gabriels (Zwijnaarde / BE), Vedran Kasalica (Amsterdam / NL), Caroline Lennartsson (Copenhagen / DK), Matthias Mattanovich (Copenhagen / DK), Emmanuelle Mouton-Barbosa (Toulouse / FR), Martin Rykaer (Copenhagen / DK), Veit Schwämmle (Odense / DK), Maximilian T. Strauss (Copenhagen / DK), Julian Uszkoreit (Bochum / DE), Bart Van Puyvelde (Ghent / BE), Tim Van Den Bossche (Zwijnaarde / BE), Jakub Vašíček (Bergen / NO), Henry Webel (Copenhagen / DK), Witold Wolski (Copenhagen / DK; Zurich / CH), Marie Locard-Paulet (Toulouse / FR)
Mass spectrometry (MS)-based proteomics is a crucial method for analyzing complex biological mixtures. With the abundance of dedicated data analysis pipelines, continuously updated and developed, the community would benefit from a platform to compare their performance in an objective and unbiased manner.
Here we propose ProteoBench (proteobench.cubimed.rub.de/), a comprehensive and open web platform that allows for easy and controlled comparison of proteomics tools developed or used by the participants to other state-of-the-art pipelines. ProteoBench originated as a community project by the European Bioinformatics Community for Mass Spectrometry (EuBIC-MS) and its design and development are open to all interested researchers. Its goal is to provide a centralized resource, which is easy to use for non-coders.
ProteoBench will comprise several benchmarking modules dedicated to the comparison of specific analysis tools developed for identification, quantification (DDA or DIA), and statistical analysis. Users shall be able to provide their analysis results for comparison, with the option of making these results publicly available.
Currently, we have five modules in preparation, of which two are in active development and one is already available as a prototype. For this module you need to download a set of six publicly-available runs containing 3 species with known quantities. Any user can analyse these data with any workflow, and then upload analysis outputs back to ProteoBench, which then evaluates sensitivity and quantification accuracy (mean difference between expected and measured ratio) for precursor ions. It allows the quick visualisation of the corresponding results in the cloud of already-computed analysis outputs.
The submitted data will continuously grow and should remain up-to-date with the latest developments. It will create a frame of reference for evaluating the performance of new and/or custom tools when publishing results and should increase transparency and reproducibility between data analysis pipelines developed in the field. It will allow end-users to select a well-performing workflow to meet their needs; developers to discern the strengths and limitations of their workflow, thereby directing further development; publishers and reviewers to easily position workflows in the context of existing state-of-the-art workflows.