About ProteoBench#

Introduction#

ProteoBench <https://proteobench.cubimed.rub.de/>_ is an open and collaborative platform for community-curated benchmarks for proteomics data analysis pipelines. Our goal is to allow a continuous, easy, and controlled comparison of proteomics data analysis workflows.

ProteoBench provides a centralized web platform for developers and end-users to compare proteomics data analysis pipelines. This community-curated effort will allow for an easy and controlled comparison of tools developed or used by the participants to other state-of-the-art pipelines for specific applications. The goal is not to select a single best one-fits-all data analysis workflow, but to allow:

  1. end-users to identify a good workflow to fulfill their specific needs

  2. developers to identify the specific strengths and weaknesses in their workflows, guiding the development process

  3. the field to easily position a newly published workflow within the context of the existing state-of-the-art

Workflow#

Participants (both end-users and developers) can download a set of input files (e.g., raw MS files, search databases, or spectral libraries) tailored to specific benchmark metrics. They can then analyze the data with any workflow, and upload the results in a homogenized format. A set of metrics will be retrieved or calculated from these results and can be visualized alongside all the other metrics calculated from the results of other participants (or of curated benchmark runs).

Goals and non-goals#

ProteoBench:

  1. Allows for an easy and controlled comparison of existing data analysis workflows

  2. Provides a frame of reference for newly developed workflows

  3. Documents and implements benchmarks that each highlight strengths or weaknesses of data analysis workflow (steps)

  4. Evolves continuously, according to the needs of the field

  5. DOES NOT point to a single best one-fits-all data analysis workflow

  6. SHOULD NOT be used as evidence for generalized statements about a workflow’s performance

  7. SHOULD NOT be used by developers as single performance measure of their workflow