Species translation challenge

The Species Translation Challenge aimed at verifying that changes in phosphorylation status and gene set activation induced by cellular response to 52 different perturbations in human cells can be predicted to a certain extent given responses generated in rat cells.

The Species Translation Challenge consisted of four sub-challenges, each addressing a different aspect of translatability:

  • Sub-Challenge 1: Intra-Species Protein Phosphorylation Prediction Ranking
  • Sub-Challenge 2: Inter-Species Protein Phosphorylation Prediction Ranking
  • Sub-Challenge 3: Inter-Species Pathway Perturbation Prediction Ranking
  • Sub-Challenge 4: Species Specific Network Inference Ranking
Background
Challenge detail
Participants
Scoring and ranking
sbv IMPROVER events
Media library
Testimonials

BACKGROUND

Rodent studies have proved indispensable as models of human diseases and have undoubtedly helped to unravel molecular mechanisms. The biomedical field has generally worked under the assumption that biological processes in mice or rats can correspond to biological processes in humans under analogous conditions. Yet few studies have addressed the limitation in which biological events observed in rodents can be translated to humans. Providing some answers to this fundamental question will be invaluable to the biomedical research community as rodent models will continue to be a central tool in biomedical research.

Intra-Species Protein Phosphorylation Prediction

While the main aim of the set of sub-challenges was focused on translation between species, the intra-species protein phosphorylation sub-challenge focused on translation between gene expression data and the corresponding phosphoprotein data in rat.

The gene expression data were collected using the Affymetrix® microarray platform and were a standard data type in the field with mature analysis techniques. On the other hand, transcriptomics data have known limitations. The systems biology field is therefore, moving to proteomics in order to provide a more comprehensive view of cellular dynamics. The expression level of protein phosphorylation is one type of proteomic data that can augment gene expression data in order to provide a more complete view of cellular signaling.

Inter-Species Protein Phosphorylation Prediction

For more than two decades, transcriptomics microarray gene expression profiles have been widely used in research to investigate genome-wide perturbations of the biological systems under various conditions, e.g. chemical, disease, environmental, mutation. However, the observed gene expression regulation is already the result of a cascade of upstream signalling events going from the cell membrane to the nucleus. Therefore, acquiring phosphoproteomic data can provide a comprehensive view of cellular dynamics. In terms of comparability and translatability between species, the field has acknowledged that there can be difficulties in discerning clear relationships between the genes of two species, sometimes even for genes that appear to be orthologous. Although very similar in sequence, genes may diverge in function or take on new roles. Such features can limit simple translation of the functional roles of similar genes from one species to another. Therefore the inter-species protein phosphorylation prediction sub-challenge was posed at the protein level to assess the translatability of phosphorylation status of different proteins that cover a range of diverse cellular processes.

Inter-Species Pathway Perturbation Prediction

The biomedical field has generally worked under the assumption that biological processes in mice or rats will correspond to biological processes in humans under analogous conditions. However, comparisons of transcriptomic data have pointed to the difficulties of performing a translation based uniquely on the expression level of orthologous genes. The difficulty could lie at several levels. For example, orthologous genes may have diverged in function and play different roles in the cell. One potential method to improve translation is to take a pathway-centric view and consider whether similar pathways in rat and human are activated under analogous conditions. The inter-species pathway perturbation sub-challenge aimed at aiming the translatability at the pathway level.

Species Specific Network Inference

Systems biology emphasizes the study of relationships and connectivity between the components of a complex system. As such, pathway diagrams are the primary representation of complex biological systems, and the construction of accurate and complete pathway maps is an on-going challenge in the field.

The two main approaches that have been taken to build pathway maps are knowledge-driven and data-driven. The knowledge-driven approach uses a priori data, often curated from the literature, to define entities (nodes) and connections (edges) that can be assembled into network diagrams. In contrast, the data-driven approach seeks to infer the connection based on inference from large dataset using methods such as regression analysis and Bayesian probabilistic models.

Combining disparate data types in pathway maps is a useful way of synthesizing such diverse knowledge into a consistent and unified view of a complex biological system. In addition, knowledge-driven approaches are often used to construct the scaffold network that can be augmented and refined using data-driven approaches.

The challenge

Aim

The aim of the Species Translation challenge was threefold:

  1. to identify rules which map measurements derived from systematic perturbations in one species to another species;
  2. to quantify the translatability between species;
  3. to understand the limitation of species translatability.

 

Challenge overview

STC1

Sub-challenge 1: Intra-Species Protein Phosphorylation Prediction

The Intra-Species Protein Phosphorylation Prediction sub-challenge aimed to evaluate if gene expression data are sufficiently informative to infer the protein phosphorylation status using a reverse inference process. Participants were asked to predict the protein phosphorylation status, given the gene expression data, in the same species (rat) and under the same perturbations.

SC1A

Intra-Species Protein Phosphorylation Prediction sub-challengeThe objective of sub-challenge 1 was the prediction of the activation status of phosphoproteins based on gene expression data in Subset B for 26 stimuli. Data in Subset A, collected with 26 different stimuli, was provided for training.

Participants were provided with Gene Expression (GEx) and Protein Phosphorylation (P) from Subset A for training. For testing, the participants were asked to predict which proteins show changes in their phosphorylation status (up or down regulation) for each stimulus in Subset B. These predictions were reported as confidence values between 0 and 1, where 1 indicates the highest confidence of a phosphorylation change as measured by the Luminex xMAP technology in normal human and rat bronchial epithelial cells. These measurements were performed in triplicate, at both 5 and 25 minutes, after the cells' growing conditions were modified by adding one of the 52 stimuli. Luminex xMAP is a bead based assay where microspheres are coated with antibodies designed to bind specifically to phosphorylated proteins. The signal from individual beads was then measured by a flow cytometry detection device as a distribution of fluorescent intensities. Finally, the median value from each distribution was calculated as the signal intensity value for each protein.

STC2

Sub-challenge 2: Inter-Species Protein Phosphorylation Prediction

The Inter-Species Protein Phosphorylation Prediction sub-challenge aimed to evaluate whether phosphorylation data in one species is sufficiently informative to infer the phosphorylation status in another species. Participants were asked to predict the human protein phosphorylation status using the rat protein phosphorylation status when cells were perturbed with the same stimuli.

SC2A

Inter-Species Protein Phosphorylation Prediction sub-challengeSub-challenge 2 required the prediction of the activation status of human phosphoprotein based on analogous phosphoproteins data in rat. Rat phosphoprotein data in Subset B, collected with 26 different stimuli, was provided for training.

STC3

Sub-challenge 3: Inter-Species Pathway Perturbation Prediction

The Inter-Species Pathway Perturbation Prediction sub-challenge aimed to determine whether activation of regulatory processes can be predicted/translated from rat to human. Participants were asked to predict gene sets (collection of genes representative of pathways/regulatory processes) that range from the most to least enriched among differentially expressed genes, with respect to control for each stimulus in human based on the corresponding data in rat.

SC3A

Inter-Species Pathway Perturbation Prediction sub-challengeSub-challenge 3 required the prediction of gene sets that were the most to least enriched among differentially expressed genes, with respect to control for each stimulus in Subset B in human based on the corresponding data in rat. The gene sets were provided and were chosen to allow the assessment of a range of functional processes and pathways in the cell.

STC4

Sub-challenge 4: Species Specific Network Inference

The Species Specific Network Inference sub-challenge aimed to address the building of species-specific biological networks that leverage diverse ‘omics’ data, and assess the commonalities and differences between the species. Participants were asked to build phosphoprotein, gene, and cytokine networks that are specific to rat and human. The Reference Network should be used a starting point.

SC4A

Species Specific Network Inference sub-challengeIn sub-challenge 4, a Reference Network was provided to participants. Participants were asked to construct human- and rat-specific networks given the omics data that was provided. Participants used network inference to add or remove edges from the Reference Network based on Phosphoprotein (P), Gene Expression (GEx) and Cytokine (Cy) data in training sets for rat and human. Only human and rat data from subset A had to be used to allow for proper comparability between the respective networks.

Data

To learn more about the data set used during the Species Translation Challenge the following article has been published in Scientific Data:

Poussin, C. et al. The species translation challenge-a systems biology perspective on human and rat bronchial epithelial cells. Scientific data 1, 140009 (2014).

Rules and awards

Description

Challenge participants

sbv participation map

Scoring and ranking

Scoring

Description

Ranking

The Scoring Review Panel reviewed and approved the scoring methodology and procedures before the challenge closure as well as the below results of the scoring and final ranking:

Ranking
STC1

Sub-challenge 1: Intra-Species Protein Phosphorylation Prediction

  1. Rank 1
    • Team PRB: Adi L. Tarca, Roberto Romero (Wayne State University & Perinatology Research Branch, NICHD, NIH)
    • Team AMG: Gyan Bhanot (Rutgers University, Dept. of Molecular Biology and Biochemistry and Dept. of Physics), Adel Dayarian (Kavli Institute for Theoretical Physics, University of California Santa Barbara), Michael Biehl (University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science), Sahand Hormoz (Kavli Institute for Theoretical Physics, University of California Santa Barbara)
    • Team Clemson: Feng Luo (Clemson University), Zhiming Wang (Clemson University, Hunan Agricultural University)
STC2

Sub-challenge 2: Inter-Species Protein Phosphorylation Prediction

  1. Rank 1
    • Team AMG: Gyan Bhanot (Rutgers University, Dept. of Molecular Biology and Biochemistry and Dept. of Physics), Adel Dayarian (Kavli Institute for Theoretical Physics, University of California Santa Barbara), Michael Biehl (University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science), Sahand Hormoz (Kavli Institute for Theoretical Physics, University of California Santa Barbara)
  2. Rank 2
    • Team IGB: Peter Sadowski, Michael Zeller (University of California Irvine)
STC3

Sub-challenge 3: Inter-Species Pathway Perturbation Prediction

  1. Rank 1
    • Team AMG: Gyan Bhanot (Rutgers University, Dept. of Molecular Biology and Biochemistry and Dept. of Physics), Adel Dayarian (Kavli Institute for Theoretical Physics, University of California Santa Barbara), Michael Biehl (University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science), Sahand Hormoz (Kavli Institute for Theoretical Physics, University of California Santa Barbara)
  2. Rank 2
    • Team PRB: Adi L. Tarca, Roberto Romero (Wayne State University & Perinatology Research Branch, NICHD, NIH)
    • Team Edith: Christoph Hafemeister (New York University)
STC4

Sub-challenge 4: Species Specific Network Inference

  1. Rank 1
    • Team Reconstructors: Steffen Klamt, Robert Johann Flassig, Sandra Heise, Regina Samaga (Max Planck Institute for Dynamics of Complex Technical Systems)
    • Team PITT.DBMI.DREAM: Lujia Chen, Xinghua Lu (University of Pittsburgh)
  2. Rank 3
    • Team UPITT.Trans.Med: Chunhui Cai (University of Pittsburgh)
  3. Rank 4
    • Team Vital-IT: Anastasia Chasapi, Leonore Wigger, Julien Dorier, Ioannis Xenarios, Mark Ibberson, Nicolas Guex (SIB Swiss Institute of Bioinformatics, Center of Integrative Genomics, UNIL, Switzerland)
    • Team PNNL: Hugh Mitchell, Susan Tilton, Jason McDermott, Joel G. Pounds (Pacific Northwest National Laboratory)

The best performing teams were announced at the sbv IMPROVER Symposium 2013 in Greece and the identity of the best performing teams for each challenge was published in Nature.


winners_announcement_STC

Best performers announcement as published in Nature (Volume 503 Number 7476, 21 November 2013, Naturejobs page 12).

Challenge symposium

The Species Translation Challenge: Understanding the Limits of Rodent Models for Human Biology Symposium was successfully conducted at the Grand Resort Lagonissi Athens, Greece, on 29 – 31 October 2013. The event included lectures, presentations by the best performers in the challenge, a workshop on network verification and network inference, a poster session, and social events and networking.

Further details regarding the sbv IMPROVER Symposium 2012 can be found in the following links:

Media library

The challenge in the news

Asia BiotechDec 2013Challenged TO ImproveINTERVALS_Icons-wwwINTERVALS_Icons-pdf
Genome webNov 2013IMPROVER Species Translation Challenge Results ReleasedINTERVALS_Icons-wwwINTERVALS_Icons-pdf
BioITWorldNov 2013Sometimes You Can Trust a RatINTERVALS_Icons-wwwINTERVALS_Icons-pdf
American LaboratoryNov 2013Results are in for the Second sbv IMPROVER Challenge on Species TranslationINTERVALS_Icons-wwwINTERVALS_Icons-pdf
Science ViewNov 2013sbv IMPROVER Symposium 2013 in Athens INTERVALS_Icons-pdf
TechonomyApr 2013Do We Get Sick Like Rats? A New Philip Morris Prize Asks the CrowdINTERVALS_Icons-wwwINTERVALS_Icons-pdf
American LaboratoryJun 2013sbv IMPROVER: Species Translation Challenge Open to the Scientific Community for SubmissionsINTERVALS_Icons-wwwINTERVALS_Icons-pdf
GENFeb 2013Crunching Complex DataINTERVALS_Icons-wwwINTERVALS_Icons-pdf
American LaboratoryFeb 2013R&D Informatics at Molecular Med Tri-Con 2013INTERVALS_Icons-wwwINTERVALS_Icons-pdf
BioITWorldFeb 2013sbv IMPROVER Launches Species Translation ChallengeINTERVALS_Icons-wwwINTERVALS_Icons-pdf

Tutorials and webinars

Flyers and posters

Testimonials

What they say about the challenge