# Real-Time Volumetric Ultrasound Research Platform with 1024 Parallel Transmit and Receive Channels
## 2 Materials and Methods
### Software  
The software architecture that controls the electronics and processes the measured data is based on our latest generation of DiPhAS software tools written in C# and C++. All user interface components, file handling and general logic were developed in C#; hardware device control with the PCI express hardware interface and the GPU-based programming were developed using C++. The hardware operation mode is configured by the software with a set of parameters that are downloaded to the firmware before measurements are performed continuously. To minimize the motion artefacts by tissue or material movement that will lead to incoherent summation over subsequent pulse-echo measurements, sequences including multiple transmit-receive events can be grouped and measured with the fastest pulse repetition possible. The digitized data of each receive phase is collected in the front-end memory and can be transferred as a large block after the sequence is completed. This is crucial for ultrafast ultrasound imaging. Even though the system consists of four synchronized electronics units, the software is implemented to control a \"synchronized multi DiPhAS\" unit that transparently handles the programming of all 1024 channels for the user. The software internally manages which delays to send to which system and combines the measurement data stored in multiple DMA memory blocks (Figure 7). The ultrasound electronics firmware does not compute any preprocessing or beamforming reconstruction using the digitized data. The data are transferred to the PC for all processing in terms of a fully \"software-defined\" ultrasound system. This approach provides all flexibility needed to develop and test any new beamforming and other signal  
Figure 7: Software layers for user-transparent programming of four synchronized units. processing technique in interchangeable software libraries. Depending on the target application and transducer used, this versatile research unit can easily be used by multiple researchers without requiring custom firmware changes to adapt the internal processing pipeline in-between. As the system is designed to be used for real-time measurements and imaging, the data acquisition of the new digitized ultrafast sequence data and processing of previously transferred data is performed in parallel. Depending on the real-time processing used, the data acquisition needs to wait before performing the next measurements. This ensures a continuous streaming operation mode without recording limit while optimizing the resulting imaging frame rate. For performant computations, we developed OpenCL-based kernels for signal conditioning, receive beamforming computation, signal post-processing and scan conversion. The implemented beamforming reconstruction includes classical delay and sum techniques and adaptive beamforming techniques, such as sign coherence beamforming [19] as well as standard deviation calculations [20]. Apart from these rather computationally fast adaptive beamforming implementations to be used in real time, we enabled more complex approaches that cannot be computed online to be reconstructed after the measurements are stored to disc. An open source file format called \"open research binary (ORB)\" was developed and implemented for storing high-frequency ultrasound data from research devices. This is a modular and extendable file format, including all device-specific hardware parameters, transmit parameters for all sequences and the received single-element raw channel data. If any processing, including the receive beamforming reconstruction or volumetric scan conversion for image generation, is applied to the measured data during real-time operation, these data are also added to the file container to be saved. Based on existing import routines for C#, C++, MATLAB and Python, the processing pipeline can be applied again on the stored data sets and optimized processing and analysis steps can be performed in the future as well with new implementations.