BioWarehouse – Database Integration for Bioinformatics

[ Download ]  [ PublicHouse ]  [ Documentation ]  [ License ] [ Contact ]

News

• Schema extensions to the Support table
• Extension to the BioCyc loader to load regulation data
• Bug fixes

• Schema extensions to accomodate transcription units.
• Extensions to the BioCyc loader to load transcription units and associated features.
• Extensions to the BioCyc loader to load asociations to GO and MetaCyc Multifun ontology terms.
• Changes to MAGE loader to fix Description loading and in OntologyEntry loading.

Overview

BioWarehouse is a component of the Bio-SPICE project. BioWarehouse is an open-source software environment for integrating a set of biological databases into a single physical database management system for data management, mining, and exploration.

Key features of BioWarehouse:

• A relational database schema that models important bioinformatics datatypes
• BioWarehouse instances can be implemented using either the Oracle or MySQL database management systems
• A collection of loader programs that populate the warehouse with data from public biological databases
• The loader programs transform the syntax of the source databases into relational form, and transform the diverse semantics of the source database into the common semantics of the BioWarehouse schema.

BioWarehouse Loaders

The BioWarehouse is populated using loader programs that translate the flat file representation of a source database into the warehouse schema. A loader is provided for each source database supported by BioWarehouse. Once loaded within a BioWarehouse instance running on e.g. MySQL, a set of source DBs can now be queried together.

Some loaders are specific to a data format rather than to a single source database. For example, the BioPAX and MAGE-ML loaders can load any database that is in BioPAX or MAGE-ML format, respectively.

BioWarehouse loaders:

BioWarehouse Schema

The BioWarehouse schema (schema documentation) is designed to capture as much of the data of each component DB as possible within a uniform representation.

For example, in encoding data from a set of source DBs pertaining to proteins, BioWarehouse uses a single set of schema definitions that spans all attributes of proteins found across this set of DBs. This approach eliminates the semantic heterogeneity present in these DBs, allowing users to query all protein sequence DBs using the same schema. Such sharing of tables is applied wherever practical. The translation from the component DB to the warehouse is achieved by the DB loaders, which convert the conceptualization used in each component DB into the conceptualization used by the warehouse schema.

The major biological objects of the BioWarehouse and their interrelationships are depicted below. Arrows indicate that the objects in those tables can refer to entries in the same table.

Documentation

For each loader, there are two pieces of documentation: how to build and run the loader, and a "manual" for developers describing the details of the loader implementation and schema mappings.

Overall documentation:

• Release notes
• Quick Start guide
• Environment Setup documentation
• Schema description and database setup instructions
• A description of the integration with the Dashboard for the February 2004 Bio-SPICE demonstration
• Descriptions of the Perl utilities and Perl demo scripts
  

All of the overall documentation is listed in a table of contents at doc/index.html.  The TOC also has a table listing some statistics about all the loaders (latest supported version of the data, input size, #Objects loader, load time, etc.)

Usage and Obtaining the BioWarehouse 

BioWarehouse can be used in two ways:

1. Users can query the public BioWarehouse server, PublicHouse maintained by SRI International via Internet SQL query.
     [Click here for Information on PublicHouse.]

2. Users can also download the BioWarehouse software distribution to create their own BioWarehouse instance containing the subset of supported BioWarehouse DBs that are of interest. This approach allows access to DBs that SRI cannot redistribute, and lets each user control when new DB versions are loaded. Users can also apply large hardware configurations to their BioWarehouse instance, and add proprietary data to their BioWarehouse instance.

The Open Source release of the BioWarehouse is distributed as a zip file (BioWarehouse-<version>.zip) and is available for download here.

Release notes for the current version may be viewed here.

BioWarehouse Applications and Publications

Projects using BioWarehouse include:

Bio-SPICE use cases
SRI’s Enzyme Genomics project
SRI’s Pathway Hole Filling project

Poster presented at 2004 Bio-SPICE PI meeting

BioWarehouse paper in BMC Bioinformatics.

Credits

The BioWarehouse team:

Peter D. Karp, Principal Investigator
Thomas J Lee
Yannick Pouliot 
Valerie Wagner
Nan Guo
David Dunkley

Support

BioWarehouse is supported by DARPA contract F30602-01-C-0153.

For support and inquiries please contact support@biowarehouse.org