CosmoSim Database for cosmological simulations released

23 June 2014. The CosmoSim database ( has now been released after an intensive testing period. This service to the scientific community is the successor of the MultiDark database ( and it is used as a platform for publishing and sharing data products from cosmological simulations.
CosmoSim Database for cosmological simulations released

Website of the CosmoSim database.

It provides access to currently six cosmological simulations - including a highresolution resimulation of selected regions with hydrodynamics and star formation - which originate from different international projects in collaboration with the Leibniz Institute for Astrophysics Potsdam (AIP).


Outputs of cosmological simulations are typically stored at supercomputing centres with restricted access and encompass terabytes of data - too much to be downloaded by everyone. By providing the data via CosmoSim, scientists from all over the world are able to access the data, filter or combine the results directly on the server and use them for their own research.

The available data products include catalogues of dark matter halos, their inner properties, merging histories, information about the cosmic web and for selected snapshots even the raw particle distributions allowing for much deeper studies of dark matter halos and their environment. All the simulations and database tables are made available through a modern web interface. Additional features include an extensive documentation, along with some background on the database structure and selected images and movies for outreach purposes.

The increase in resolution for cosmological simulations has led to larger data volumes, with individual tables reaching sizes in the terabyte range. By exploring a new database technology, it is now possible to store and analyse snapshots from simulations with nearly 60 billion particles directly in the CosmoSim database.

The new database technology, the Spider engine for MariaDB/MySQL, allows to spread the data over many server nodes, and one head node, resulting in a distribution of the computational task over many server nodes. Several additional software components were developed by AIP’s E-Science team to enable transparent handling of parallel queries. Further, a job queue was implemented as a direct plugin for MariaDB/MySQL, so that even long running queries are permitted without stalling the servers.

CosmoSim utilizes the modern web framework Daiquiri, which was developed by the E-Science team during the recent years. It provides direct data access via an SQL query form, a database browser and SQL validation before sending the query to the server. An interface to Virtual Observatory tools like TopCat allows further quick investigations and processing of the retrieved results.

CosmoSim is based fully on open source technology. The modules developed by the E-Science team are available on GitHub at and

Demo movie:             


The CosmoSim-Team:
- Kristin Riebe, (data management and support)
- Jochen Klar, (web interface - backend and frontend)
- Harry Enke, (management)
- Stefan Gottloeber, (simulation data)
- Adrian Partl, (backend and database development)


The key topics of the Leibniz Institute for Astrophysics Potsdam (AIP) are cosmic magnetic fields and extragalactic astrophysics. A considerable part of the institute's efforts aim at the development of research technology in the fields of spectroscopy, robotic telescopes, and e-science. The AIP is the successor of the Berlin Observatory founded in 1700 and of the Astrophysical Observatory of Potsdam founded in 1874. The latter was the world's first observatory to emphasize explicitly the research area of astrophysics. Since 1992 the AIP is a member of the Leibniz Association.