Main uses of biological databases
- Search data
- Download data
- Data analysis using integrated tools
- Submit data
FAIR principles applied to genome assembly and annotation
Module 3: data management
2024-01-17
Definition of biological databases
- Collection of structured biological data that can be accessed, searched and analyzed
- They play a crucial role in bioinformatics and computational biology : scientists can access to a wide variety of biological relevant data
- Organized in a collection of biological information : DNA, proteins sequences, molecular structure, gene expression data …
- Designed to facilitate efficient data storage, retrieval and analysis, allowing researchers to explore and interpret biological data on a large scale
=> FAIR (Findable Accessible Interoperable Reusable) context!
image from https://digitalworldbiology.com/sites/default/files/DWB%20Product/teaser-images/exploring_databases.png
Main uses of biological databases
Search data
Main uses of biological databases
Download data
Main uses of biological databases
Data analysis using integrated tools (BLAST, genome browser, variation viewer, taxonomy browser…)
Main uses of biological databases
Submit data
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
EBI website contains 165 databases
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of DNA & RNA database : ENA (European Nucleotide Archive)
ENA (European Nucleotide Archive) : nucleotide sequencing information, covering raw sequencing data, sequence assembly information and functional annotation
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of DNA & RNA database : ENA (European Nucleotide Archive)
[ENA (European Nucleotide Archive)]
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of DNA & RNA database : Ensembl
Ensembl : genome browser for animal genomes that supports research in comparative genomics, evolution, sequence variation and transcriptional regulation.
Ensembl annotates genes, computes multiple alignments, predicts regulatory function and collects disease data.
Ensembl tools include BLAST, BLAT, BioMart and the Variant Effect Predictor (VEP) for all supported species.
Release 111 (01/2024) : 317 genomes!
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of DNA & RNA database : Rfam
Rfam : collection of RNA families, each represented by multiple sequence alignments, consensus secondary structures and covariance models
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of DNA & RNA database : Rfam
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : Uniprot
Uniprot : high-quality, comprehensive and freely accessible resource of protein sequence and functional information.
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : Uniprot
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : AlphaFold
AlphaFold : database for protein structure predictions for numerous species
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : AlphaFold
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : InterProscan
InterProscan : functional analysis of proteins by classifying them into families and predicting domains and important sites
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : InterProscan
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : Pfam
Pfam : A database of conserved protein families and domains. Pfam is a member database of InterPro. The Pfam database is a large collection of protein families, each represented by multiple sequence alignments and hidden Markov models (HMMs)
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : HMMER
HMMER : Fast sensitive protein homology searches using profile hidden Markov models (HMMs) for querying against both sequence and HMM target databases.
EMBL-EBI (European Molecular Biology Laboratory - European Bioinformatics Institute)
Example of protein database : HMMER
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Biomedical and genomic information
NCBI website contains 59 Databases
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : Nucleotide Database
Nucleotide Database : A collection of nucleotide sequences from several sources, including :
- GenBank,
- RefSeq,
- The Third Party Annotation (TPA) database,
- Protein Database.
Searching the Nucleotide Database will yield available results from each of its component databases.
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : Nucleotide Database
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : GenBank
GenBank : an annotated collection of all publicly available DNA sequences, is part of the International Nucleotide Sequence Database Collaboration , which comprises :
- the DNA DataBank of Japan (DDBJ),
- the European Molecular Biology Laboratory (EMBL),
- GenBank at NCBI.
GenBank consists of several divisions, most of which can be accessed through the Nucleotide database.
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : SRA (Sequence Read Archive)
SRA (Sequence Read Archive) : stores sequencing data from the next generation of sequencing platforms including Roche 454 GS System®, Illumina Genome Analyzer®, Life Technologies AB SOLiD System®, Helicos Biosciences Heliscope®, Complete Genomics®, and Pacific Biosciences SMRT®.
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : SRA (Sequence Read Archive)
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : Assembly
Assembly : structure of assembled genomes, assembly names and other meta-data, statistical reports, and links to genomic sequence data.
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : Assembly
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : Genome
Genome : Contains sequence and map data from the whole genomes of over 1000 organisms
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA database : Genome
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA AND protein database : RefSeq
RefSeq : A comprehensive, integrated, non-redundant, well-annotated set of reference sequences including genomic, transcript, and protein.
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of DNA & RNA AND protein database : RefSeq
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of protein database : Protein Database
Protein Database : includes protein sequence records from a variety of sources, including GenPept, RefSeq, Swiss-Prot, PIR, PRF, and PDB.
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of protein database : Protein Database
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of publication database : PubMed
PubMed : collection of biomedical books that can be searched directly or from linked data in other NCBI databases
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Example of publication database : PubMed
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Tool for sequence analysis : Blast
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Tool for downloading genome data : NCBI Datasets
NCBI National Center for Biotechnology Information (NIH national institutes of health, USA)
Tool for downloading genome data : command line NCBI Genome
A catalog of worldwide biological databases
Database Commons is a curated catalog of worldwide biological databases, with the aim to provide a full landscape of biological databases throughout the world and enable easy retrieval and access to a specific collection of databases of interest
FAIR: general principles
FAIR: guidelines proposed by the scientific community 1.
Objectives: promote best-practice tools to help produce and publish reproducible analyses throughout the data lifecycle.
FAIR: why do it?
Over 70% of researchers have tried to reproduce another scientist’s experiments without success, and over half have failed to reproduce their own experiments!
FAIR: an obligation!
- The FAIR principles are a key element of a more general concept: the data management plan (DMP).
- In France, there is an obligation (e.g. for ANR and European projects) to make public the data and software of scientific projects funded with at least 50% public money.
DMP: Data Management Plan
- The Data Management Plan (DMP) or Plan de gestion de données (PGD) is a synthetic, evolving document that helps to organize and anticipate every stage in the data life cycle. For each data set, it explains how project data will be managed, from creation or collection to sharing and archiving.
- The data management plan is based on the data life cycle, which describes the various stages of data processing during a research project.
Figure from: https://www.universite-paris-saclay.fr/recherche/science-ouverte/les-donnees-de-la-recherche/introduction-aux-plans-de-gestion-de-donnees
DMP: its content
- The Data Management Plan (DMP) is a synoptic and editable text document that aims to organize and anticipate all stages of the data life cycle.
- How to?
- Use templates provided by ANR or institutes;
- Use online tools such as DMP opidor 1 or Data Stewardship Wizard 2.
A long, not very quiet river
A long, not very quiet river
Very useful tools exist to collect, manage and share data and metadata over the time.
Data is stored in a structured and secure manner
- Use a dedicated directory to store all data for all projects
- Assign admin permissions to data managers only
- Limit number of data managers
- Assign read-only permissions to users
- Setup a separate data archive system
- “separate” = a different storage in a different place
Data is stored in a structured and secure manner
- Setup a same organization for all sub folders
- “users” DO NOT control naming
- only “data managers” DO that
- Use controlled vocabulary for folder naming
- i.e. EDAM bioinformatics 1 ontology
- Ensure to always store in such a safe place raw data AND metadata
Note: ‘operation’ folder many contain particular final results, e.g. a sequence assembly.
Data is stored with metadata
What is metadata?
Metadata 1 describes the data (main characteristics), such as author, creation date, content, format, associated project, associated sample, etc.
Given types of collected data, its description should be achieved using metadata standards, e.g. controlled vocaularies, ontologies, etc. Do not invent your own terms!
Data licensing
Data has to be associated with a license that describes how data can be used by anyone.
As a French public research institute, we should use the ETALAB 1 public license from the Direction Interministérielle du Numérique (DINUM).
- There are several types of CC licenses 2: CC0, CC-BY, CC-BY-NC, etc.
However, in practice: we should use the CC-BY license to be in an international legal framework. Not using CC-BY license prevents ENA submission.
Making data accessible to the community
Using the services of a certified public research institute
Ifremer Sextant Catalog of Oceanographic Data - CoreTrustSeal Certified
Making data accessible to the community
Using the services of country-based initiatives
Recherche Data Gouv 1 has been designed to support research teams in their work of structuring data to make it easy to find, accessible, interoperable and reusable - in other words, in line with the “FAIR” principles. These services are operated by INRAe.
Making data accessible to the community
Using one of the “Big Three”: NCBI, EBI, DDBJ
Publish sequencing data to international databases for international referencing.
- The International Nucleotide Sequence Database Collaboration (INSDC) collects and disseminates international databases containing DNA and RNA sequences
- All data is available for free and unrestricted access, for any purpose, with no restrictions on analysis, redistribution, or re-publication of the data (CC-BY license)
ENA metadata model description
Sequencing data will be described using 5 objects
https://ena-docs.readthedocs.io/en/latest/submit/study.html
ENA metadata model description
About these 5 objects
ENA metadata model description
Metadata are described using ENA checklists
These are controlled vocabularies organized as a collection of scientific themes.
- There is a minimum amount of information required during ENA sample registration and all samples must conform to a defined checklist of expected metadata values.
- These sample checklists have been developed to meet the needs of different research communities. Different communities have different requirements on the minimum metadata expected to describe biological samples.
ENA checklists
As of November 2023, there are about 40 ENA sample checklists:
Do not invent your terms to describe your metadata. Instead, use above one!
Collecting metadata for ENA submission
Here is an example of a spreadsheet-based document designed to collect metadata following ENA requirements 1:
ENA submission
Time to publish data and metadata?
- Time-consuming
- Difficult to handle
- Several tools available
ENA submission done
Example of a catalog entry programmatically submitted to ENA:
Metadata related to ENA Study object
Are you FAIR?
Tool to check FAIRness of your materials: FAIR-Checker
Example for the Nextflow-based pipeline SAMBA published on Elixir Workflowhub:
FAIRness evaluation