Diffraction data deposition and photon / neutron facilities

[HeinzWeyer]

Dear Colleagues,

- first a few words about PaNdata (from the PaNdata website): PaNdata brings together major
multidisciplinary Research Infrastructures to construct and operate a shared data infrastructure for
Neutron and Photon laboratories. Such an infrastructure will enhance all research done in this
community, by making data accessible, preserving the data, allowing experiments to be carried out
jointly in several laboratories and by providing powerful tools for scientists to remotely interact with
the data.
The participating neutron and photon facilities are major scientific data producers, serving an
expanding user community of 25,000 to 30,000 scientists across Europe. The experiments in these
facilities are of increasing complexity, they are increasingly done by international research groups
and many of them will be done in more than one laboratory. The resulting data needs to be
accessible over the Internet and remain on-line until the results are published and in many cases
much longer to allow re-processing and to allow for the preservation of knowledge.
PaNdata will provide our user communities with data repositories and data management tools to:
deal with large sets and large data rates from the experiments, enable easy and standardized
annotation of data, allow transparent and secure remote access to data, establish sustainable and
compatible data catalogs, allow long-term preservation of data, and provide compatible open
source data analysis software.
This will have a major impact on our scientific user community because it will offer cross facility and
cross discipline data analysis, secure access to large data sets over the network instead of using
portable media, maintaining the records of science by having properly annotated data, linking
publications to data, allowing efficient software developments, and efficient scientific collaborations
across Europe by providing compatible data formats and analysis software.
At the heart of the vision is a series of federated data catalogs which allow scientists to perform
cross-facility, cross-discipline interaction with experimental and derived data, with near real-time
access to the data – a ‘Google Earth’, at the scale of atoms and molecules. Along with this
PaNdata will thrive for a unification of data management policies in order for the common technology
to be successfully adopted.

- the Diffraction Data Deposition Working Group initiative has been presented and discussed at the
RAL kick-off meeting (November 2011) of the PaN-data ODI. From the description above it is obvious
that the topic of the Diffraction Data Deposition Working Group launched by IUCr is very much in
line with the goals of PaNdata and this IUCr initiative has been very positively received. This is
especially important, as IUCr is emphasizing the viewpoint of the users of the large facilities. The
PaNdata consortium offers any help for reaching the goal of this initiative.

- furthermore, PaN-data proposes that IUCr one one side and the PaN-data consortium on the other
side cooperate closely together to reach an efficient data infrastructure by simultaneously
addressing the important issues from the user and facility point of view, respectively. A first step in
this direction could be mutual information about IT-related interests and goals, e.g. within a common
seminar or workshop or session in an IUCr meeting). Here the main topics of PaNdata ODI as
present project could be presented and vice versa PaN-data would be very much interested in getting
the feedback on these concepts from one of the largest user communities at our photon / neutron
facilities.

Best regards,
Heinz J Weyer
for the PaNdata consortium