IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - W E P 0 4 6

PROGRESS ON DISTRIBUTED IMAGE ANALYSIS FROM DIGITALCAMERAS AT ELSA USING THE RABBITMQ MESSAGE BROKER
M. Switka∗ , K. Desch, T. Gereons, S. Kronenberg, D. Proft, A. Spreitzer
Physikalisches Institut, University of Bonn, Germany
Abstract
In the course of modernization of camera based imaging and image analysis for accelerator hardware and beam
control at the ELSA facility, a distributed image processing
approach was implemented, called FGrabbit. We utilize
the RabbitMQ message broker to share the high data throughput from image acquisition, processing, analysis (e.g. profilefit), display and storage between different work stations toachieve an optimum efficacy of the involved hardware. Recalibration of already deployed beam profile monitors using
machine vision algorithms allow us to perform qualitative
beam photometry measurements to obtain beam sizes and dynamics with good precision. We describe the robustness ofthe calibration, image acquisition and processing and present
the architecture and applications, such as the programmingand web-interface for machine operators and developers.

INTRODUCTION
The ELSA facility [1] deploys around 50 optical beam
monitors based on luminescence screens (Chromox or OTR)
and synchrotron radiation. The previous installation of cameras with analogue video signal output provided mediocre
beam analysis capability due to limited signal quality and thelack of comprehensive camera control. To obtain qualitative
beam data from photometry we modernize hardware andsoftware of the optical monitors, primarily utilizing Ethernetcapable digital cameras controlled with non-commercial
camera drivers and open source machine vision libraries
for image calibration. TheFGrabbit framework was created [2] to provide resource-optimized camera control, data
handling and computing power based on the RabbitMQ message broker for distributed analysis. A Gigabit Ethernet(GigE) network between PoE-capable switches links processing computers and image generating devices, such as
GenIcam [3] compatible digital cameras, other digitizers
or image generating programs. A graphical user interface
allows access to the FGrabbit framework from the accelerator control system or the facility website and provides
comprehensive device and image control.

drivers. We favor the Aravis [4] driver in combination withGenICam [3] compatible digital cameras. Streams fromother drivers (e.g. Video4linux 1 ), VNC2 servers or image generating programs may also be used as input for theFGrabbit framework. Comprehensive image calibration
can be applied via the OpenCV computer vision library [5]to account for beam profile distortions resulting from tilted
screens (extrinsic properties) or lens imperfections of theimaging system (intrinsic properties). Calibrated images
are typically cropped to a useful area of interest (AOI) andideally visualize an undistorted transverse cross section ofan observed beam with correct pixel to millimeter scaling.
The calibrated images are typically reduced in size, allowing to economically stream images from multiple cameras
through the GigE network. The data load from imaging
and image analysis is handled by the RabbitMQ message
broker. Computing power (e.g. for computational expensive
2D profile fits) is distributed to multiple standard computers
within the network. A QT-based [6] graphical user interface
1 https://www.kernel.org/doc/html/v4.8/media/v4l-drivers/
2 virtual network computing

camera features (config.)

calibration data

AOI
definition
fit function
ﬁt worker
1∗ switka@physik.uni-bonn.de

calibra�oncalibrated
image

raw
image

ﬁt worker
nimage streamﬁtdispatcher...
ﬁtresults

ARCHITECTURE
The architecture of the FGrabbit framework is depictedin Fig. 1. All peripheral devices (cameras, computers) participating in the FGrabbit framework communicate througha GigE network, separated from the ELSA control system, but accessible through a gateway computer. As camera control and readout can be achieved through various

cameradrivercameraﬁt writerGUI

image writer

FGwebbitanalog, ethernet, etc.
RabbitMQ
Configuration / RPC(via RabbitMQ)

Figure 1: Architecture of the FGrabbit framework.
WEP04609 Data Acquisition and Processing Platforms

449

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

IBIC2023, Saskatoon, CanadaJACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - W E P 0 4 6(GUI) allows full-scale user interaction. A web interface
allows down-scaled interaction from anywhere via the ELSA
website3 .

a)

b)

c)

d)

Performance
A RabbitMQ message is limited to 128 MB, defining thelimit for the streamable image size. While the cumulative
streaming data rate is limited by the network bandwidth, thecomputationally expensive fit calculations are outsourced
to an arbitrary amount of computers participating in thenet- and framework. Computation and network load is conveniently monitored via the RabbitMQ analysis interface.
Using standard desktop computers we achieve repetition
rates up to 10 Hz of display and beam analysis on a sufficiently large AOI (2D data fit on a hundreds by hundreds
large pixel grid).

MEASUREMENT PRECISION
Image Calibration
Beam images taken from e.g. Chromox screens sufferfrom projection and optical imaging errors which can beclassified as intrinsic and extrinsic properties4 . To obtain aprojection-free image without optical errors, a 3 × 3 projection matrix 𝑃 can be applied to the uncalibrated image
𝑃 = 𝐾 ⋅ 𝑀, 𝑀 = 𝐻 ⋅ 𝑡 ⃗ ,Figure 2: Image calibration using a printed checkerboard
pattern on a tilted screen and OpenCV [5] transformation
functions: a) original image, b) recognized pattern, c) image
after homography transformation including precision errors(red circles), d) scaled and calibrated image.

raw image(1)

where 𝐾 denotes to the intrinsic and 𝑀 to the extrinsic properties of the individual setup. 𝑀 is calculated from a rotation
matrix 𝐻 and a translation vector 𝑡.⃗ The main influence tothe extrinsic uncertainties are image distortions due to theobservation perspective, e.g. a screen under a tilt angle oftypically 45∘ . This can be resolved by using a homographymatrix as rotation matrix. The matrix entries can be computed from an image taken with a regular pattern of known
dimensions (checkerboard or circular patterns consisting of𝑚 × 𝑛 corners or circles, respectively). Such a calibration image can be temporally installed on the surface of a screen, asdemonstrated in Fig. 2. A calibration pattern printed on cardboard with a regular laser printer usually suffices the required
image quality. For the calibration we use an OpenCV [5]approach as depicted in Fig. 3. Therewith, a raw image iscaptured on which an OpenCV detector function finds theposition of the corners of the checkerboard (or centers ofcircles, resp., compare with circular calibration patterns inAppendix). The obtained coordinates are then optimized
by performing an additional search for the position using
sub-pixel accuracy. Then a coordinate grid that matches theshape of the used pattern is created with a predefined step
range. By mapping the found image coordinates into thereal world grid the homography matrix can be calculated.

RabbitMQ
Analysiscircle-/cornerdetection

center-/cornersubpixel

calibration
errorshomographytransformation

Figure 3: Schematic sketch of the calibration chain. The
chain is divided into RabbitMQ framework (red) and
OpenCV tasks (orange). The chain can be applied to circular patterns (dark-orange) and checkerboard patterns (lightorange).
minimize statistical fluctuations a sequence of 𝑛 images isrecorded to obtain an averaged radial error, as visualized inFig. 2 c). Intrinsic properties, such as radial or tangential
distortions introduced by the camera lens may be representedby
𝑥dist,rad = 𝑥(1 + 𝑘1 𝑟 2 + 𝑘2 𝑟 4 + 𝑘3 𝑟 6 )𝑦dist,rad = 𝑦(1 + 𝑘1 𝑟 2 + 𝑘2 𝑟 4 + 𝑘3 𝑟 6 )𝑥dist,tan = 𝑥 + (2𝑝1 𝑥𝑧 + 𝑝2 (𝑟 2 + 2𝑥 2 ))𝑦dist,tan = 𝑦 + (𝑝1 (𝑟 2 + 2𝑦2 ) + 2𝑝2 𝑥𝑦) ,The radial displacement of the transformed positions to
the expected grid is classified as the calibration error. To

with distortion coefficients (𝑘1 , 𝑘2 , 𝑘3 , 𝑝1 , 𝑝2 ). OpenCV
functions allow to determine these coefficients and to undistort the respective image. The distortion coefficients are
obtained from taking multiple pictures of a regular patternfrom multiple angles5 . We find that intrinsic distortions playa neglectable role with usage of industry standard photographic objectives (e.g. 𝑓 = 16 mm) for our monitor setups.

3 https://www-elsa.physik.uni-bonn.de

5 For details see https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibra-Measurement Error

4 compare e.g. with https://docs.opencv.org/4.x/tion.html

WEP04645009 Data Acquisition and Processing Platforms

IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0
d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - W E P 0 4 6The calibration error determined from the deviation fromthe keypoint grid can be interpolated to every pixel, forexample using radial basis functions6 , as shown in Fig. 4.
Thereon, non-linear weighted fit algorithms can be applied
to take local uncertainties into account. For this purpose theGNU scientific library (GSL) [7] back-end of the in-housedeveloped fit library is used. For the unweighted case thecommon function to be minimized is
Φ(𝑥) =numerous available camera settings are listed and manipulatable, as shown in Fig. 6. Favorite settings can be marked
and made available for the main menu. A detail view of theimage analysis gives the operator fast and comprehensive
feedback of fit parameters and fit quality.

11 𝑛2∥𝑓 (𝑥)∥ = ∑ 𝑓𝑖 (𝑥1 , … , 𝑥𝑝 )2 ,2
2 𝑖=1

where 𝑓 is the residual function, 𝑛 the number of data points(e.g. the number of pixels of the captured image) and 𝑝 is thenumber of parameters or the dimension of the fit function.
In the weighted case this formula is expanded toΦ(𝑥) =1 𝑛∑ 𝑤 ⋅ 𝑓 (𝑥 , … , 𝑥𝑝 )2 ,2 𝑖=1 𝑖 𝑖 1(2)where an individual weight 𝑤𝑖 is added for each individual data point. These weights can be utilized for various
purposes or represent different effects. Using the obtained
inverse calibration variances as weights allows us to suppress potential errors introduced by the calibration process.
Furthermore, this allows to mask areas on the screens thatexhibit altered behavior to achieve a better fit result. InFig. 4 a profile captured on a calibrated screen as well as
the weighted fit to this data is shown. The inverse varianceof the calibration was used as weight, resulting in a profile
parameter deviation of up to 2 % comparing weighted andunweighted case.

Figure 5: FGrabbit graphical user interface with detailview on profile analysis.

Figure 6: FGrabbit configuration explorer listing all properties of the digital cameras obtained by the Aravis [4] driver.
Figure 4: The variance of an exemplary screen calibration
with circled nodes and in-between interpolation (left) and abeam profile image on a calibrated screen analyzed with aweighted fit algorithm (right).

FGRABBIT INTERFACES
While RPC7 commands allow for automated settings andapplication of the FGrabbit framework, the graphical userinterfaces provide practical user interaction and monitoring
capability.

Operator GUI
A QT-based [6] graphical user interface allows for convenient control of camera and imaging settings (Fig. 5). The
6 e.g. using Python SciPy RBFInterpolator
7 remote procedure call

FGwebbit
In addition to the Operator GUI a slimmed down Interface
is accessible via an in-house developed web application,
called FGwebbit. It provides platform independent accessto FGrabbit combined with flexible availability via internet
and intranet. The responsive architecture (Fig. 7) enables
control and views optimized for handheld devices whenworking on experiments or maintenance tasks in confinedspaces. The simplified user interface provides access to live
views of multiple camera streams simultaneously. The userinterface is complemented by a modular card for each camera.
Within a camera card, favorite settings of the attached device
are directly available. An expert menu contains all available
device parameters for detailed adjustments, accompanied
by tooltips for the available settings. Each client can adjust
the compression, frame rate, color model and preference
for calibrated or uncalibrated data. These settings are saved
WEP04609 Data Acquisition and Processing Platforms

451

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0
d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - W E P 0 4 6

and applied only for that particular client, allowing for anoptimized application e.g. for mobile use with limited dataavailability. The web application is prepared to host a global
overview of the state of the FGrabbit systems, summarizing
the status of all associated nodes and devices. Furthermore,
the application provides space for the documentation of theFGrabbit framework and interfaces.
Figure 8: Screen with confined illumination and recognizedpattern (left). Calibrated image using circular patterns including precision errors (right).

Virtual Calibration Pattern
Screens with sufficient markings for reference can be calibrated with a virtual pattern, e.g. created with a drawing
tool and fed to the calibration algorithm, as shown in Fig. 9.
Figure 7: Responsive behavior of the FGwebbit interfaceand active session settings menu.
The FGwebbit backend extends the FGrabbit Pythoninterface [2] and uses the lightweight Python web framework Flask. To reduce network traffic and data usage inmobile or low bandwidth environments, the live camera
feeds are served as compressed JPEG streams. The OpenCV
library [5] is used for compression and color model application. Further streamlining is achieved by smart handling of
TCP connections to the underlying FGrabbit framework.
Multiple clients and simultaneous streams from or to thesame camera device are detected and clustered into a single
open TCP connection. This alone significantly improvesconnection times for additional clients and reduces serverload. Unused TCP connections are closed after a predefined
timeout, freeing up system resources. For modern designand easy responsive implementation the popular Bootstrap
frontend toolkit is used, resulting in familiar design elements
and usability concepts for FGwebbit.

CONCLUSION
Screen calibration for extrinsic properties is the key advantage for beam photometry with monitors installed inenvironments typical for accelerator beamlines. Calibration
errors may be used to improve measurement accuracy using weighted data fit algorithms. The performance of theFGrabbit framework is continuously tested and has shown
to be robust and scalable. Minor software development is
yet carried out to improve usability for long-term accelerator
operations.

APPENDIX
Circular Calibration Pattern
Circular calibration patterns are generally more robust(successful) than checkerboard patterns, e.g. when illumination is non-ideal, as shown in Fig. 8.

Figure 9: a) Sufficient visible markings may be used to createa b) virtual pattern for c) & d) image calibration.

REFERENCES[1] W. Hillert et al., “Beam and spin dynamics in the fast ramping
storage ring ELSA”, EPJ Web Conf., vol. 134, p. 05002, Jan.
2017. doi:10.1051/epjconf/201713405002[2] D. Proft et al., “Beam profile monitoring and distributed analyis using the RabbitMQ message broker”, in Proc. IBIC’22,
Kraków, Poland, Sept. 2022, pp. 140–143.
doi:10.18429/JACoW-IBIC2022-MOP38[3] European Machine Vision Association, “GenICam Standard”,
https://www.emva.org/standards-technology/genicam/[4] A vision library for genicam based cameras,
https://github.com/AravisProject/aravis[5] G. Bradski,“The OpenCV Library”, Dr. Dobb’s Journal of
Software Tools, 2000.
[6] The Qt Company, https://www.qt.io/[7] M. Galassi et al., GNU Scientific Library Reference Manual,
3rd ed., ISBN 0954612078,
https://www.gnu.org/software/gsl/

WEP046452

09 Data Acquisition and Processing Platforms