First a little about myself
In [2]:
locations=(
(52.945530, 12.389213, 'Kyritz', folium.Icon(color='lightblue', prefix='fa', icon='fa-universal-access')),
(49.941938, 12.083177, 'Hard', folium.Icon(color='orange', prefix='fa', icon='fa-user')),
(52.457739, 13.310591, 'FU Berlin', folium.Icon(color='lightgreen', prefix='fa', icon='fa-university')),
(-37.909199, 147.131103, 'Monash Uni', folium.Icon(color='blue', prefix='fa', icon='fa-university')),
(48.464822, -123.314190, 'Uni of Victoria', folium.Icon(color='orange', prefix='fa', icon='hand-o-up')),
(50.727208, -3.474619, 'UK Met Office', folium.Icon(color='black', prefix='fa', icon='hand-o-up')),
(-37.797191, 142.965023, 'Uni of Melbourne', folium.Icon(color='brown', prefix='fa', icon='fa-university')),
(53.588599, 9.829234, 'Eu XFEL', folium.Icon(color='brown', prefix='fa', icon='database')),
(53.488599, 9.939234, 'MPI', folium.Icon(color='darkgreen', prefix='fa', icon='diamond')))
m = folium.Map(location=(48,20), zoom_start=4.9)
for nn, loc in enumerate(locations):
folium.Marker(loc[0:2], popup=loc[2], tooltip=, icon=loc[-1]).add_to(m)
display(m)
#folium.Icon?
There are 6 "Instruments" receiving the laser beam:
One common scientific theme is:
What was I doing there?
Part of the Control and Analysis Software Engineering Team (CAS) that:
- Develop the distributed control system, also for real time data analysis
- Consult external research group on software deployment strategies and software development
Adaptive Gain Adaptive Gain Integrating Pixel Detector (AGIPD)
- Multiple Panels Detecting Signals Independently
- Hole - Variable Geometry
- Records at 5 MHz = 2.6 GB/s
Many different data sources are orchestrated by the control system
What about Data Management?¶
- During the Experiment:
- Rapid feedback by the live scenes and user tools coming from the bridge.
- Data (run) can be (temporarily) recorded on a cluster at EuXFEL
- After Experiment:
- After each run, scientist decides on quality of the data: good, unclear, not interesting
- Good or unclear data get migrated to the cluster at Desy
- Analyse files on Maxwell cluster (Desy):
- During and after the experiment
- 180 nodes with 40 cores and ~512GB each
File base analysis:¶
- Problem: Data from bid detectors or cameras is recorded separately many different files:
bergem@max-exfl014 in ~ %-
ls –lh /gpfs/exfel/exp/SPB/201701/p002012/raw/r0359
total 92G
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD00-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD01-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD02-S00000.h5
-rw-r--r-- 1 xdata xdata 241K Jul 6 11:03 RAW-R0359-AGIPD03-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD04-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD05-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD06-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD07-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD08-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD09-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD10-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD11-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD12-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD13-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD14-S00000.h5
-rw-r--r-- 1 xdata xdata 6.1G Jul 6 11:03 RAW-R0359-AGIPD15-S00000.h5
-rw-r--r-- 1 xdata xdata 788M Jul 6 11:03 RAW-R0359-DA01-S00000.h5
-rw-r--r-- 1 xdata xdata 38M Jul 6 11:03 RAW-R0359-DA02-S00000.h5
import extra_data as xd
run_folder = '/gpfs/exfel/exp/XMPL/201750/p700000/raw/r0273'
run_dir = xd.RunDirectory(run_folder)
run_dir.info()
# of trains: 156
Duration: 0:00:15.500000
First train ID: 198425241
Last train ID: 198425396
16 detector modules (SPB_DET_AGIPD1M-1)
e.g. module SPB_DET_AGIPD1M-1 0 : 512 x 128 pixels
176 frames per train, 27456 total frames
2 instrument sources (excluding detectors):
- SA1_XTD2_XGM/XGM/DOOCS:output
- SPB_XTD9_XGM/XGM/DOOCS:output
13 control sources:
- ACC_SYS_DOOCS/CTRL/BEAMCONDITIONS
- SA1_XTD2_XGM/XGM/DOOCS
- SPB_IRU_AGIPD1M/PSC/HV
- SPB_IRU_AGIPD1M/TSENS/H1_T_EXTHOUS
- SPB_IRU_AGIPD1M/TSENS/H2_T_EXTHOUS
- SPB_IRU_AGIPD1M/TSENS/Q1_T_BLOCK
- SPB_IRU_AGIPD1M/TSENS/Q2_T_BLOCK
- SPB_IRU_AGIPD1M/TSENS/Q3_T_BLOCK
- SPB_IRU_AGIPD1M/TSENS/Q4_T_BLOCK
- SPB_IRU_AGIPD1M1/CTRL/MC1
- SPB_IRU_AGIPD1M1/CTRL/MC2
- SPB_IRU_VAC/GAUGE/GAUGE_FR_6
- SPB_XTD9_XGM/XGM/DOOCS
Software Deployment and Documentation¶
Central IT infrastructure maintained and organized by Desy-IT:
- Deployed central Xfel anconda distribution:
- Once a new proposal (/work Project) gets created a certain software stack (mini-conda) is deployed
- User can ask for proposal specific software stack, usually before beamtime.
- Release and update circle independent from Desy IT
- Proposal based jupyter-kernel gets auto installed via modules and use the kernel in jupyter-hub instance from Desy-IT
- Jupyter Notebook as basis for user documentation
- All software projects using sphinx for auto-building documentation
- GitLab CIs to constantly test notebooks and keep documentation up to date.
Non-Python based software and Reproducibility¶
- Containerizing scientific software via
singularity:- Singularity because of better integration in HPC systems (than Docker)
- Containers are also made a part of the proposal software stack
- Effort to design and define fully automated FaaS pipelines via slurm, kubernetes - warppipes.
Whats it lit to work at a place like EuXfel
- Plenty of home-made things: Machines, Software etc
- Lots of requirements: from Instrument groups
- On Call Duty: Night time sessions
- Control Software still not mature
- Agile Software Engineering Paradigms
- Lots of Unit Testing and Documentation
In [ ]: