Research Data Publication at Large Scale

Abstract

The MPCDF is the high performance computing center of the Max Planck Society. Beside hosting compute clusters, around 300 petabytes of research data are stored at the MPCDF. Many of these datasets have a size of several terabytes up to petabytes and are stored over a heterogeneous landscape of storage systems. The datasets are covering a wide variety of scientific disciplines, many different data formats and access restrictions. For these reasons, it is not possible to offer one centralized data publishing solution for the datasets. Instead, the MPCDF developed a flexible data publishing concept.