Content from this work may be used under the terms of the CC BY 4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

19th Int. Conf. Accel. Large Exp. Phys. Control Syst.
ICALEPCS2023, Cape Town, South Africa
JACoW Publishing
ISBN: 978-3-95450-238-7
ISSN: 2226-0358
doi:10.18429/JACoW-ICALEPCS2023-THPDP041

THE RF PROTECTION INTERLOCK SYSTEM
PROTOTYPE VERIFICATION∗
W. Cichalewski† , W. Jalmuzna, P. Amrozik, G. Jablonski, R. Kielbik, K.Klys, R. Kotas,
P. Marciniak, B. Pekoslawski, W. Tylman
Department of Microelectronics and Computer Science, Lodz University of Technology, Poland
B. Chase, N. Patel, P. Varghese, E. Harms, P. Prieto
Fermi National Accelerator Laboratory, Batavia IL, USA
Abstract
The Radio Frequency Protection Interlock system plays
a vital role in the LLRF-related/dependent accelerator sections Protection. Its main role is to collect information froma number of different sensors and indicators around the nearest cavities and cryomodules and provide instant RF signal
termination in case of safety threshold violation. This submission describes a newly designed RFPI system tailoredto the requirements of Proton Improvement Plan II (PIP-II).
The proof of concept prototype of this system has been built.
The paper includes also the PIP2IT (PIP-II integrated test
stand) environment evaluation test results and findings as
input to the next full-scope prototype design.

INTRODUCTION
The Low Level RF (LLRF) control systems are widelyused in normal and superconducting accelerators to provide
optimal conditions for energy transfer from accelerating
structures to the accelerated proton or electron beam. The
performance of this system can determine overall beam parameters hence influence conditions of laser light or neutron
beam generation or others - depending on the given infrastructure type.
The LLRF system has to ensure instantaneous correctionof the amplitude and phase parameters of electromagnetic accelerating waves. In the case of superconducting resonators
working as standing wave systems its main focus is the execution of the fast feedback algorithms responsible for suchactions. To perform its role efficiently LLRF system has tobe provided with strictly defined operating conditions forthe whole accelerating structure. A system that is responsible for such conditions monitoring and system hardware
protection exists in many accelerator implementations. In
the case of the newly designed Proton Improvement Plan
II (PIP-II) H- accelerator in Fermilab [1], such system has
been named Radio Frequency Protection Interlock (RFPI)The RFPI system’s main role is to monitor various cryomodule components’ status values and drop the permit signals in case any predefined safety thresholds are exceeded [2].
To gather comprehensive knowledge about subsystems’ current state the RFPI has to observe several various signals
from vacuum, cryogenics, temperature sensors, RF signals∗ Work supported, in part, by the U.S. Department of Energy, Office of

Science, Office of High Energy Physics, under U.S. DOE Contract No.
DE-AC02-07CH11359.
† wcichal@dmcs.pl

THPDP041
1406

leak sensors, electron pick-up, input fundamental couplerhigh voltage power supply, solid state amplifier status, LLRF
system status and others.
For many years the RFPI system developed and implemented earlier in Fermilab provided constant protection forsystems operated in various locations of local accelerator
infrastructure. Together with the new design of the LLRF
system, it has been also decided that the PIP-II dedicated
RFPI system needs to be adapted to the specific project needs.
The requirement concerning system redesign according tomodern electronic standards was not the only one. A newly
designed RFPI system needs to be modular and provide configurable sub-sets of input signals depending on the deployment place. Because of that not only the flexibility in system
hardware configuration has to be delivered. The protection
function logic needs to be also susceptible to reconfigurationand scaling. What is more, a single PIP-II RFPI system has
to protect from one to four LLRF systems (and resonators)
gathered in a single (or different) cryomodule.
The proposed new design fulfills the requirements concerning modularity, scalability, and reconfiguration challenges. The new solution incorporates dedicated, customdesigned signal-conditioning modules connected to the carrier board via custom FMC modules. This component contains an FPGA chip that holds and executes protection function logic. Hence configuration can be modified by removing
or introducing a given conditioning submodule and adjusting the protection function when needed. This approach
foreseen also the System On-Chip (SOC) chip configuration
usage for better integration of the EPICS control system
dedicated IOC with the RFPI infrastructure.
The first version of the system has been designed and produced in the form of a Proof of Concept (PoC) prototype [3].
This version of the system consists of custom-designed signal conditioning and dedicated interface FMC modules aswell as COTS components integrated into a single RFPI
instance. Although the PoC does not possess enough input
channels to cover the full four cavity set-up its main goal
was the verification of each input channel type conditioning/processing quality and performance.

PROOF OF CONCEPT PROTOTYPE
HARDWARE STRUCTURE
The RFPI system PoC hardware structure is illustrated inFig. 1.
The PoC system consists of the following modules:
HardwareHardware Technology

Figure 2: The FMC GPIO module block diagram.

Figure 1: The block diagram of the PoC structure.
• Main logic realization unit
• FPGA Mezzanine Card (FMC) interface modules- FMC General Purpose IO (GPIO) board- FMC ADC board• Input/output signals conditioning modules- Contact Switches board- Field emission probe (FEP) module- RTD and coupler bias voltage board- Non-ionising Radiation Probe module- Solid State Amplifier (SSA) and LLRF inhibit
submodule• Management and Diagnostics

Main Logic Unit
The ZCU106 (by Xilinx) development board was the main
carrier of the PoC design. This module contains Zynq UltraScale+™ FPGA chip and a quad-core Arm Cortex A53
application processor as well as two FMC slots. The FPGA
has enough resources to execute full protection function
logic for the RFPI system. The Arm processor delivers therequired resources for the EPICS IOC that manages and monitors main logic functionality. The available FMC slots havebeen used to interface in/out signals conditioning modules
via dedicated analog and digital FMC boards.

FMC Modules
To exchange signals between the main FPGA and conditioning boards two FMC modules have been designed andproduced. The FMC GPIO module provides 40 GPIO (8 foreach board) and 2 power connections. To effectively workwith conditioning boards each GPIO can cope with signals
up to 50 MHz (see Fig. 2).
In addition, the FMC ADC module has been developed(see Fig. 3). It provides 8 ADC channels (14-bits, 25 MS/s)
to interface FEP and NIRP conditioning modules. As NIRP
requires RF signals detection the FMC ADC has been
HardwareHardware Technology

Figure 3: The FMC ADC module block diagram.
equipped optionally with RF detectors. Each channel ofthis board can be configured to use or bypass the RF detector
stage.

Input and Output Signals Conditioning Modules
To cover the RFPI input and output interface requirements
five different signal conditioning (SC) modules have been
proposed, designed, and produced in-house. There wereeight different groups of signals identified for the RFPI setup.
That is why single SC congregate one or more signal type.
As the PoC main goal was each type of channel implementation verification the SC modules contain a limited quantityof channels that fulfill requirements for single cavity andcryomodule RFPI protection.
The Contact Switches SC The RFPI-critical signals in
the form of open/close PLC-based switches were gatheredon the Contact Switch module (see Fig. 4). This device generates different voltages depending on the monitored signal
type. The design of this part allows for different HW typesettings (NC or NO) and provides cable diagnostics capabilities as well as signal latching on each input and masking
THPDP041
1407

Content from this work may be used under the terms of the CC BY 4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

19th Int. Conf. Accel. Large Exp. Phys. Control Syst.
ICALEPCS2023, Cape Town, South Africa
JACoW Publishing
ISBN: 978-3-95450-238-7
ISSN: 2226-0358
doi:10.18429/JACoW-ICALEPCS2023-THPDP041

Content from this work may be used under the terms of the CC BY 4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

19th Int. Conf. Accel. Large Exp. Phys. Control Syst.
ICALEPCS2023, Cape Town, South Africa
JACoW Publishing
ISBN: 978-3-95450-238-7
ISSN: 2226-0358
doi:10.18429/JACoW-ICALEPCS2023-THPDP041

Figure 6: The RTD/HV PS SC diagram.

Figure 4: The contact switches signal conditioning module
block diagram.

Figure 7: The RF detector characterization for different passband filters used in the design.

functionalities. The helium pressure and level, window cooling airflow permit, coupler vacuum permit, beam vacuum
permit are examples of signals monitored by this module.

The Non-Ionising Radiation Probe The RF leakage
detection was covered separately, outside of the design SC
modules. Three different RF passband filters were examined
for PIP-II typical frequencies (162.5, 325, 650 MHz) (seeFig. 7). In case of the PoC an extensive leakage was reported
thanks to the mentioned filter and RF detector placed at the
input of chosen FMC-ADC channels. The digitized versionof the NIRP signal was delivered later to the main logic unit
and included in the protection function estimation.

The Field Emission Probe (FEP) SC The electron
pick-up channel monitoring is covered by the field emission
probe conditioning module (see Fig. 5). To detect microand nano-ampere level current dedicated highly sensitive
circuit has been adapted from previous RFPI realization. To
increase and improve the detection level the biasing voltage
was provided too.

The Solid State Amplifier (SSA) Permit SC The RFPI
system communicates certain system failures via four output
signals. Some or all of these permits are dropped in case of
system fault detection. Then the RFPI system drops the 5 V
signal and disables LLRF output or Solid State Amplifier
output or its DC voltage and finally also it sends information
to the Machine Protection System (MPS).

Figure 5: The FEP SC diagram.

The Resistance Temperature Detector (RTD) and Coupler Bias High Voltage Power Supply SC The PT-103
unit-based temperature read-outs from two different locations next to the cavity need to be monitored. According to
the requirements, the four-wire measurement configuration
has been provided. Other channels on this module (spareanalog) were dedicated for coupler bias HV power supply
current and voltage diagnostics as depicted in Fig. 6.
THPDP041
1408

Management and Diagnostics
The prototype sections’ working conditions are supervisedby the dedicated Management and Diagnostics sub-circuit.
Its main role is to gather information from voltage, current
and temperature sensors from individual SC modules andother locations. Moreover, via relay array, it is capable ofswitching on/off power to a given part of the RFPI. Thanks
to that it protects the SC modules (and main unit) from
HardwareHardware Technology

Figure 9: The management and diagnostics SW diagram.

Figure 8: The main logic unit software diagram.
long-term over-voltage or over-current issues. As management and diagnostics is also responsible for fans control it
is regulating airflow in the chassis in case of overheating.
Finally, this part of the system implements also the main
logic unit watchdog functionality which is crucial for thewhole RFPI system disabling while the main logic unit response is not reliable.

SOFTWARE STRUCTURE
The main logic unit software structure is presented inFig. 8. The lowest layer is the FPGA firmware part. On thislevel, the pre-processed digital input signals are gathered
and the main protection function logic is executed. The
output signals state is determined on this level too. The
firmware interfaces to the higher operating system (OS) level
via AXI, GPIO, SPI channels. The ARM processor (locatedon the same SoC) uses Ubuntu distribution OS drivers toestablish communication through these channels. The user
space library not only provides access to the FW registers.
It provides already the functionality needed for the RFPi
configuration (like acceptable signal thresholds set/get orindividual input channels masking on the protection functionlevel).
The upper SW layer EPICS IOC ASYN module actively
uses the user space library to communicate with the hardware. The top layer here is the OPI part that provides GUI
functionalities and access for the system operator and experts.
HardwareHardware Technology

Different software layers have been identified and implemented for the management and diagnostics sub-system. The
lowest layer is the firmware of the STM32 microcontroller.
As depicted in Fig. 9 this part is responsible (among theothers) for diagnostics sensors readouts, health status signal
preparation as well as watchdog signal for main logic unit
FPGA.
The next layer is already the operating system part. As in
the case of the main unit Ubuntu 22.04 distribution was used.
The OS drivers provide sufficient functionality to interface
between STM32 firmware and the user space driver library.
On top of that We have developed the dedicated EPICS IOC
together with supporting OPI’s. The XVC (Xilinx Virtual
Cable) driver is a part of this software and provides a debugging and programming interface for the main unit FPGA.

PROTOTYPE EVALUATIONPoC Evaluation on the Test Bench
Evaluation of each channel type happened during thesignal conditioning modules post-production testing. The
hardware designers provided also dedicated sub-circuits toprovide the required input signals parameters. These setups
allowed the determination of the particular channel performance and limits. Then the complete prototype was built
from all the described earlier components. We have prepared the PoC in the form of 3U, 19-inch box with most
input/output interfaces at the back-panel (see Fig. 10). For
simplicity either BNC or RJ45 connectors was used for mostsignals.
Post-integration testing included individual channels evaluation together with protection function logic implementation checks. The verification on the electrical level was
possible thanks to developed software modules. They provide configuration and management capabilities to the expert
THPDP041
1409

Content from this work may be used under the terms of the CC BY 4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

19th Int. Conf. Accel. Large Exp. Phys. Control Syst.
ICALEPCS2023, Cape Town, South Africa
JACoW Publishing
ISBN: 978-3-95450-238-7
ISSN: 2226-0358
doi:10.18429/JACoW-ICALEPCS2023-THPDP041

Content from this work may be used under the terms of the CC BY 4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

19th Int. Conf. Accel. Large Exp. Phys. Control Syst.
ICALEPCS2023, Cape Town, South Africa
JACoW Publishing
ISBN: 978-3-95450-238-7
ISSN: 2226-0358
doi:10.18429/JACoW-ICALEPCS2023-THPDP041

Figure 10: The RFPI PoC box picture (ref. to IPAC paper).
Figure 12: The forward RF power blocking after cryo permit
signal activation.

Figure 13: The forward RF power blocking after NIRP signal
introduction.
Figure 11: The PoC main logic unit operation OPI window.
during test runs. Performed tests proved parameters expectedfrom the system modules. Some room for improvement wasrecognized especially for the FEP SC module. It needs to beemphasized that the work on the LLRF and other subsystems
design (within PIP-II) is still ongoing. This influence also
requirements list for the RFPI systems.

Prototype Evaluation at Cryomodule Test Facility(Fermilab)
The final evaluation of the PoC device took place at the
PIP-II integrated test stand in Fermilab. Due to the limited
quantity of the prototype inputs it has been agreed that the
PoC test will include single cavity protection. After initialinstallation, each RFPI input was verified by an artificial
test signal induced by real subsections of the cryomodule
systems. An example of the individual channel testing procedure step is presented in Fig. 11. The active signals thatparticipate in the protection signal generation are indicatedby green LEDs, and other masked signals - by yellow LED.
Examples of the PoC system performance for induced
signals are presented in Figs. 12 and 13.
Achieved protection system response, when LLRF permits have been dropped by RFPI were 9 µs and 2 µs for “cryopermit” and NIRP signals respectively. These results are
positive and much better than the allowed system reaction
time. The FPGA based system not only provided protection function response within given limits but also delivered
post-mortem information about signals chronology.
THPDP041
1410

The final test of the PoC included single high beta (HB650)
cavity protection during several hours of operation in different conditions. Also in this case the test result was positive.
The PoC behaved as expected and did not introduce any false
protection action.

CONCLUSION
The PIP-II accelerator is in the construction phase. Since
it is a good opportunity to push some linac systems to the newtechnologies. An example can be the FPGA-based protection
RFPI system. The initial prototype design and producedby the FNAL-DMCS/LUT effective collaboration fulfills
the majority of the requirements defined for such a system.
Although not equipped with the final channel quantity, the
PoC proved its reliability and performance in test-bench andreal accelerator environment studies. Technical conclusions
drawn from this device are already part of the ongoing fullscale prototype design.

REFERENCES[1] V. Lebedev, “The PIP-II Reference Design Report”, FNAL,
Batavia, IL, USA, Rep. FERMILAB-DESIGN-2015-01, 2015.
doi:10.2172/1365571[2] E. Pozdeyev et al., “Beam Commissioning and Integrated Testof the PIP-II Injector Test Facility”, in Proc. LINAC’22, Liverpool, UK, Aug.-Sep. 2022, pp. 13–17.
doi:10.18429/JACoW-LINAC2022-MO1PA01[3] W. Cichalewski et al., “Evaluation of the first version of the
new RFPI system dedicated to PIP-II project accelerator”, in
Proc. IPAC’23, Venice, Italy, May 2023, pp. 4226-4228.
doi:10.18429/JACoW-IPAC2023-THPA114

HardwareHardware Technology