Overview of ProteoBench#

ProteoBench is an open and collaborative platform for community-curated benchmarks for proteomics data analysis pipelines. Our goal is to allow a continuous, easy, and controlled comparison of proteomics data analysis workflows.

ProteoBench provides a centralized web platform for developers and end-users to compare proteomics data analysis pipelines. This community-curated effort will allow for an easy and controlled comparison of tools developed or used by the participants to other state-of-the-art pipelines for specific applications. The goal is not to select a single best one-fits-all data analysis workflow, but to allow:

  1. end-users to identify a good workflow to fulfill their specific needs

  2. developers to identify the specific strengths and weaknesses in their workflows, guiding the development process

  3. the field to easily position a newly published workflow within the context of the existing state-of-the-art

Description#

Participants (both end-users and developers) can download a set of input files (e.g., raw MS files, search databases, or spectral libraries) tailored to specific benchmark metrics. They can then analyze the data with any workflow, and upload the results in a homogenized format. A set of metrics will be retrieved or calculated from these results and can be visualized alongside all the other metrics calculated from the results of other participants (or of curated benchmark runs).

Goals and non-goals#

ProteoBench:

  1. Allows for an easy and controlled comparison of existing data analysis workflows

  2. Provides a frame of reference for newly developed workflows

  3. Documents and implements benchmarks that each highlight strengths or weaknesses of data analysis workflow (steps)

  4. Evolves continuously, according to the needs of the field

  5. DOES NOT point to a single best one-fits-all data analysis workflow

  6. SHOULD NOT be used as evidence for generalized statements about a workflow’s performance

  7. SHOULD NOT be used by developers as single performance measure of their workflow

Organization#

The github repository for ProteoBench can be found here.

ProteoBench documentation is divided into two main parts:

The different modules/index that exist, how they evolve and how to used them.

The Developer guide provides information on the structure of the project and how to contribute.

Contact us#

email us here.


Contribution#

People who contributed to ProteoBench (in alphabetical order)

Robbin Bouwmeester
VIB-UGent Center for Medical Biotechnology, VIB, Ghent, Belgium
Department of Biomolecular Medicine, UGent, Ghent, Belgium
Nadezhda T. Doncheva
Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Viktoria Dorfer
Bioinformatics Research Group, University of Applied Sciences Upper Austria, Hagenberg, Austria
Ralf Gabriels
VIB-UGent Center for Medical Biotechnology, VIB, Ghent, Belgium
Department of Biomolecular Medicine, UGent, Ghent, Belgium
Vedran Kasalica
Netherlands eScience Center, Science Park 402, 1098 XH, Amsterdam, The Netherlands
Caroline Lennartsson
Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Marie Locard-Paulet
Institut de Pharmacologie et de Biologie Structurale (IPBS), Université de Toulouse, CNRS, Université Toulouse III - Paul Sabatier (UT3), Toulouse, France
Infrastructure nationale de protéomique, ProFI, FR 2048, Toulouse, France
Matthias Mattanovich
Novo Nordisk Foundation Center for Basic Metabolic Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Emmanuelle Mouton-Barbosa
Institut de Pharmacologie et de Biologie Structurale (IPBS), Université de Toulouse, CNRS, Université Toulouse III - Paul Sabatier (UT3), Toulouse, France
Infrastructure nationale de protéomique, ProFI, FR 2048, Toulouse, France
Martin Rykær
Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Julian Uszkoreit
Ruhr University Bochum, Medical Faculty, Medical Bioinformatics, Bochum, Germany
Tim Van Den Bossche
Department of Biomolecular Medicine, Ghent University, Ghent, Belgium
VIB - UGent Center for Medical Biotechnology, VIB, Ghent, Belgium
Bart Van Puyvelde
ProGenTomics, Laboratory of Pharmaceutical Biotechnology, Ghent University, Belgium
Veit Schwämmle
Department of Biochemistry and Molecular Biology, University of Southern Denmark, Odense, Denmark
Maximilian T. Strauss
Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Henry Webel
Novo Nordisk Foundation Center for Basic Metabolic Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Witold E. Wolski
Functional Genomics Center Zurich (FGCZ)−University of Zurich/ETH Zurich, Winterthurerstrasse 190, CH-8057 Zurich, Switzerland
Swiss Institute of Bioinformatics (SIB) Quartier Sorge−Batiment Amphipole, 1015 Lausanne, Switzerland