
The Species Translation Challenge aimed at verifying that changes in phosphorylation status and gene set activation induced by cellular response to 52 different perturbations in human cells can be predicted to a certain extent given responses generated in rat cells.
The Species Translation Challenge consisted of four sub-challenges, each addressing a different aspect of translatability:
- Sub-Challenge 1: Intra-Species Protein Phosphorylation Prediction Ranking
- Sub-Challenge 2: Inter-Species Protein Phosphorylation Prediction Ranking
- Sub-Challenge 3: Inter-Species Pathway Perturbation Prediction Ranking
- Sub-Challenge 4: Species Specific Network Inference Ranking
BACKGROUND ↑
Rodent studies have proved indispensable as models of human diseases and have undoubtedly helped to unravel molecular mechanisms. The biomedical field has generally worked under the assumption that biological processes in mice or rats can correspond to biological processes in humans under analogous conditions. Yet few studies have addressed the limitation in which biological events observed in rodents can be translated to humans. Providing some answers to this fundamental question will be invaluable to the biomedical research community as rodent models will continue to be a central tool in biomedical research.
Intra-Species Protein Phosphorylation Prediction
While the main aim of the set of sub-challenges was focused on translation between species, the intra-species protein phosphorylation sub-challenge focused on translation between gene expression data and the corresponding phosphoprotein data in rat.
The gene expression data were collected using the Affymetrix® microarray platform and were a standard data type in the field with mature analysis techniques. On the other hand, transcriptomics data have known limitations. The systems biology field is therefore, moving to proteomics in order to provide a more comprehensive view of cellular dynamics. The expression level of protein phosphorylation is one type of proteomic data that can augment gene expression data in order to provide a more complete view of cellular signaling.
Inter-Species Protein Phosphorylation Prediction
For more than two decades, transcriptomics microarray gene expression profiles have been widely used in research to investigate genome-wide perturbations of the biological systems under various conditions, e.g. chemical, disease, environmental, mutation. However, the observed gene expression regulation is already the result of a cascade of upstream signalling events going from the cell membrane to the nucleus. Therefore, acquiring phosphoproteomic data can provide a comprehensive view of cellular dynamics. In terms of comparability and translatability between species, the field has acknowledged that there can be difficulties in discerning clear relationships between the genes of two species, sometimes even for genes that appear to be orthologous. Although very similar in sequence, genes may diverge in function or take on new roles. Such features can limit simple translation of the functional roles of similar genes from one species to another. Therefore the inter-species protein phosphorylation prediction sub-challenge was posed at the protein level to assess the translatability of phosphorylation status of different proteins that cover a range of diverse cellular processes.
Inter-Species Pathway Perturbation Prediction
The biomedical field has generally worked under the assumption that biological processes in mice or rats will correspond to biological processes in humans under analogous conditions. However, comparisons of transcriptomic data have pointed to the difficulties of performing a translation based uniquely on the expression level of orthologous genes. The difficulty could lie at several levels. For example, orthologous genes may have diverged in function and play different roles in the cell. One potential method to improve translation is to take a pathway-centric view and consider whether similar pathways in rat and human are activated under analogous conditions. The inter-species pathway perturbation sub-challenge aimed at aiming the translatability at the pathway level.
Species Specific Network Inference
Systems biology emphasizes the study of relationships and connectivity between the components of a complex system. As such, pathway diagrams are the primary representation of complex biological systems, and the construction of accurate and complete pathway maps is an on-going challenge in the field.
The two main approaches that have been taken to build pathway maps are knowledge-driven and data-driven. The knowledge-driven approach uses a priori data, often curated from the literature, to define entities (nodes) and connections (edges) that can be assembled into network diagrams. In contrast, the data-driven approach seeks to infer the connection based on inference from large dataset using methods such as regression analysis and Bayesian probabilistic models.
Combining disparate data types in pathway maps is a useful way of synthesizing such diverse knowledge into a consistent and unified view of a complex biological system. In addition, knowledge-driven approaches are often used to construct the scaffold network that can be augmented and refined using data-driven approaches.
The challenge ↑
Aim
The aim of the Species Translation challenge was threefold:
- to identify rules which map measurements derived from systematic perturbations in one species to another species;
- to quantify the translatability between species;
- to understand the limitation of species translatability.
Challenge overview

Sub-challenge 1: Intra-Species Protein Phosphorylation Prediction
The Intra-Species Protein Phosphorylation Prediction sub-challenge aimed to evaluate if gene expression data are sufficiently informative to infer the protein phosphorylation status using a reverse inference process. Participants were asked to predict the protein phosphorylation status, given the gene expression data, in the same species (rat) and under the same perturbations.

Intra-Species Protein Phosphorylation Prediction sub-challengeThe objective of sub-challenge 1 was the prediction of the activation status of phosphoproteins based on gene expression data in Subset B for 26 stimuli. Data in Subset A, collected with 26 different stimuli, was provided for training.
Participants were provided with Gene Expression (GEx) and Protein Phosphorylation (P) from Subset A for training. For testing, the participants were asked to predict which proteins show changes in their phosphorylation status (up or down regulation) for each stimulus in Subset B. These predictions were reported as confidence values between 0 and 1, where 1 indicates the highest confidence of a phosphorylation change as measured by the Luminex xMAP technology in normal human and rat bronchial epithelial cells. These measurements were performed in triplicate, at both 5 and 25 minutes, after the cells' growing conditions were modified by adding one of the 52 stimuli. Luminex xMAP is a bead based assay where microspheres are coated with antibodies designed to bind specifically to phosphorylated proteins. The signal from individual beads was then measured by a flow cytometry detection device as a distribution of fluorescent intensities. Finally, the median value from each distribution was calculated as the signal intensity value for each protein.

Sub-challenge 2: Inter-Species Protein Phosphorylation Prediction
The Inter-Species Protein Phosphorylation Prediction sub-challenge aimed to evaluate whether phosphorylation data in one species is sufficiently informative to infer the phosphorylation status in another species. Participants were asked to predict the human protein phosphorylation status using the rat protein phosphorylation status when cells were perturbed with the same stimuli.

Inter-Species Protein Phosphorylation Prediction sub-challengeSub-challenge 2 required the prediction of the activation status of human phosphoprotein based on analogous phosphoproteins data in rat. Rat phosphoprotein data in Subset B, collected with 26 different stimuli, was provided for training.

Sub-challenge 3: Inter-Species Pathway Perturbation Prediction
The Inter-Species Pathway Perturbation Prediction sub-challenge aimed to determine whether activation of regulatory processes can be predicted/translated from rat to human. Participants were asked to predict gene sets (collection of genes representative of pathways/regulatory processes) that range from the most to least enriched among differentially expressed genes, with respect to control for each stimulus in human based on the corresponding data in rat.

Inter-Species Pathway Perturbation Prediction sub-challengeSub-challenge 3 required the prediction of gene sets that were the most to least enriched among differentially expressed genes, with respect to control for each stimulus in Subset B in human based on the corresponding data in rat. The gene sets were provided and were chosen to allow the assessment of a range of functional processes and pathways in the cell.

Sub-challenge 4: Species Specific Network Inference
The Species Specific Network Inference sub-challenge aimed to address the building of species-specific biological networks that leverage diverse ‘omics’ data, and assess the commonalities and differences between the species. Participants were asked to build phosphoprotein, gene, and cytokine networks that are specific to rat and human. The Reference Network should be used a starting point.

Species Specific Network Inference sub-challengeIn sub-challenge 4, a Reference Network was provided to participants. Participants were asked to construct human- and rat-specific networks given the omics data that was provided. Participants used network inference to add or remove edges from the Reference Network based on Phosphoprotein (P), Gene Expression (GEx) and Cytokine (Cy) data in training sets for rat and human. Only human and rat data from subset A had to be used to allow for proper comparability between the respective networks.
Data
To learn more about the data set used during the Species Translation Challenge the following article has been published in Scientific Data:
Poussin, C. et al. The species translation challenge-a systems biology perspective on human and rat bronchial epithelial cells. Scientific data 1, 140009 (2014).
Rules and awards
Description
Challenge participants ↑

Scoring and ranking ↑
Scoring
Description
Ranking
The Scoring Review Panel reviewed and approved the scoring methodology and procedures before the challenge closure as well as the below results of the scoring and final ranking:


Sub-challenge 1: Intra-Species Protein Phosphorylation Prediction
- Rank 1
- Team PRB: Adi L. Tarca, Roberto Romero (Wayne State University & Perinatology Research Branch, NICHD, NIH)
- Team AMG: Gyan Bhanot (Rutgers University, Dept. of Molecular Biology and Biochemistry and Dept. of Physics), Adel Dayarian (Kavli Institute for Theoretical Physics, University of California Santa Barbara), Michael Biehl (University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science), Sahand Hormoz (Kavli Institute for Theoretical Physics, University of California Santa Barbara)
- Team Clemson: Feng Luo (Clemson University), Zhiming Wang (Clemson University, Hunan Agricultural University)

Sub-challenge 2: Inter-Species Protein Phosphorylation Prediction
- Rank 1
- Team AMG: Gyan Bhanot (Rutgers University, Dept. of Molecular Biology and Biochemistry and Dept. of Physics), Adel Dayarian (Kavli Institute for Theoretical Physics, University of California Santa Barbara), Michael Biehl (University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science), Sahand Hormoz (Kavli Institute for Theoretical Physics, University of California Santa Barbara)
- Rank 2
- Team IGB: Peter Sadowski, Michael Zeller (University of California Irvine)

Sub-challenge 3: Inter-Species Pathway Perturbation Prediction
- Rank 1
- Team AMG: Gyan Bhanot (Rutgers University, Dept. of Molecular Biology and Biochemistry and Dept. of Physics), Adel Dayarian (Kavli Institute for Theoretical Physics, University of California Santa Barbara), Michael Biehl (University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science), Sahand Hormoz (Kavli Institute for Theoretical Physics, University of California Santa Barbara)
- Rank 2
- Team PRB: Adi L. Tarca, Roberto Romero (Wayne State University & Perinatology Research Branch, NICHD, NIH)
- Team Edith: Christoph Hafemeister (New York University)

Sub-challenge 4: Species Specific Network Inference
- Rank 1
- Team Reconstructors: Steffen Klamt, Robert Johann Flassig, Sandra Heise, Regina Samaga (Max Planck Institute for Dynamics of Complex Technical Systems)
- Team PITT.DBMI.DREAM: Lujia Chen, Xinghua Lu (University of Pittsburgh)
- Rank 3
- Team UPITT.Trans.Med: Chunhui Cai (University of Pittsburgh)
- Rank 4
- Team Vital-IT: Anastasia Chasapi, Leonore Wigger, Julien Dorier, Ioannis Xenarios, Mark Ibberson, Nicolas Guex (SIB Swiss Institute of Bioinformatics, Center of Integrative Genomics, UNIL, Switzerland)
- Team PNNL: Hugh Mitchell, Susan Tilton, Jason McDermott, Joel G. Pounds (Pacific Northwest National Laboratory)
The best performing teams were announced at the sbv IMPROVER Symposium 2013 in Greece and the identity of the best performing teams for each challenge was published in Nature.

Best performers announcement as published in Nature (Volume 503 Number 7476, 21 November 2013, Naturejobs page 12).
Challenge symposium ↑
The Species Translation Challenge: Understanding the Limits of Rodent Models for Human Biology Symposium was successfully conducted at the Grand Resort Lagonissi Athens, Greece, on 29 – 31 October 2013. The event included lectures, presentations by the best performers in the challenge, a workshop on network verification and network inference, a poster session, and social events and networking.
Further details regarding the sbv IMPROVER Symposium 2012 can be found in the following links:
- Symposium Participant Program
- Symposium Agenda Overall
- Symposium Agenda Day 3
- Symposium Keynote Speakers
- Symposium Workshop Speakers and Presenters
- Symposium Hosts
- Some photos of the symposium
- Symposium presentations:
Keynote talk: Dr Adi Laurentiu Tarca
Machine learning and pathway analysis as basic tools in systems biology- Raquel Norel Diagnostic Signature Challenge: Perennial Benchmarking
Species Translation Challenge: Introduction, Data and Lessons Learned
- Jean Binder & Jeremy Rice The Species Translation Challenge (management perspective)
- Pablo Meyer Lessons Learned from the Species Translation Challenge
- Julia Hoeng & Leonidas Alexopoulos Species Translation Challenge. From Rat to Human: Understanding the Limits of Animal Models
- Raquel Norel Strategy for Scoring Prediction Performance
SC1: Intra-Species Protein Phosphorylation Prediction - Challenge Formulation, Scoring, and Best Performers Presentations
- Carine Poussin Sub-Challenge 1: Intra-Species Protein Phosphorylation Prediction
- Gayan Bhanot, Michael Biehl, Sahand Hormoz, Adel Dayarian sbv IMPROVER Understanding the Limits of Rodent Models for Human Biology (Sub-Challenge 1)
- Feng Luo sbv IMPROVER Species Translation Challenge Report
Keynote talk: Dr George Kollias
Mapping Human Disease Pathogenesis via Alignments with Animal Models
SC2: Inter-Species Protein Phosphorylation Prediction - Challenge Formulation, Scoring, and Best Performers Presentations
- Vincenzo Belcastro, Erhan Bilal, Raquel Norel Sub-Challenge 2: Inter-Species Protein Phosphorylation Prediction
- Gayan Bhanot, Michael Biehl, Adel Dayarian, Sahand Hormoz Inter Species Prediction of Protein Phosphorylation
- Michael Zeller sbv IMPROVER 2013
Keynote talk: Dr Joaquín Dopazo
Using mechanism-based biomarkers to predict complex traits- Pablo Meyer Computational lessons
SC3: Inter-Species Pathway Perturbation Prediction - Challenge Formulation, Scoring, and Best Performers Presentations
- Pablo Meyer Sub-Challenge 3: Computational Lessons
- Christoph Hafemeister sbv Improver Sub-Challenge 3: Inter-Species Pathway Perturbation Prediction
- Sahand Hormoz Sub-challenge 3: Inter-Species Pathway Perturbation Prediction
- Carine Poussin, Kahn Rhrissorrakrai Sub-Challenge 3: Computational Lessons
SC4: Species Specific Network Inference Challenge Formulation, Scoring and Review
- Carine Poussin, Theodore Sakellaropoulos, Erhan Bilal Sub-Challenge 3: Computational Lessons
Keynote talk: Dr Carolyn Cho
Translational Modeling: Impact on Critical Decision-Making in Drug DevelopmentWorkshop session 1: Network verification challenge
- Yukiko Matsuoka Weaving Knowledge of Biological Pathways in Collaboration
- Joanna Taylor Network Verification Challenge Communication activities
- Monica Munoz-Torres Web Apollo: lessons learned from community-based biocuration efforts
- Kirill Tarasov Challenges and rewards of curation of lipidomic information
- Alexander Klenner How can Text Mining improve the Improver Challenge for you?
Workshop session 2: Beyond network inference
- Xinghua Lu, Chunhui Cai, Lujia Chen, Songjan Lu, Xia Jiang, Gregory Cooper A Bayesian Network Framework for Learning Species Specific Signaling Pathways
- Jie Cheng Our analysis for sub-challenge 4: species specific network inference
- Heinz Koeppl Bayesian network reconstruction from dynamic signaling data
- Gayan Bhanot, Michael Biehl, Adel Dayarian, Sahand Hormoz Sub-Challenge 4: Species Specific Network Inference
- Anastasia Chasapi, the Vital-IT group, Swiss Institute of Bioinformatics Pathway Based Networks for Species Specific Network Interference
Media library ↑
Scientific publications
- Rhrissorrakrai, K. et al. Understanding the limits of animal models as predictors of human biology: lessons learned from the sbv IMPROVER Species Translation Challenge. Bioinformatics 31 (4), 471–83 (2015).
- Dayarian, A. et al. Predicting protein phosphorylation from gene expression: top methods from the IMPROVER Species Translation Challenge. Bioinformatics 31 (4), 462–470 (2015).
- Hafemeister, C. et al. Inter-species pathway perturbation prediction via data-driven detection of functional homology. Bioinformatics 31 (4), 501–508 (2015).
- Biehl, M. et al. Inter-species prediction of protein phosphorylation in the sbv IMPROVER species translation challenge. Bioinformatics 31 (4), 453–461 (2015).
- Poussin, C. et al. The species translation challenge-a systems biology perspective on human and rat bronchial epithelial cells. Scientific data 1, 140009 (2014).
The challenge in the news
Tutorials and webinars
Flyers and posters
Testimonials ↑
What they say about the challenge






