OADAT: Experimental and Synthetic Clinical Optoacoustic Data for Standardized Image Processing
Authors: Firat Ozdemir
Berkan Lafci
Xose Luis Dean-Ben
Daniel Razansky
Fernando Perez-Cruz
Venue: Accepted by TMLR ()

Abstract: Optoacoustic (OA) imaging is based on excitation of biological tissues with nanosecond-duration laser pulses followed by subsequent detection of ultrasound waves generated via light-absorption-mediated thermoelastic expansion. OA imaging features a powerful combination between rich optical contrast and high resolution in deep tissues. This enabled the exploration of a number of attractive new applications both in clinical and laboratory settings. However, no standardized datasets generated with different types of experimental set-up and associated processing methods are available to facilitate advances in broader applications of OA in clinical settings. This complicates an objective comparison between new and established data processing methods, often leading to qualitative results and arbitrary interpretations of the data. In this paper, we provide both experimental and synthetic OA raw signals and reconstructed image domain datasets rendered with different experimental parameters and tomographic acquisition geometries. We further provide trained neural networks to tackle three important challenges related to OA image processing, namely accurate reconstruction under limited view tomographic conditions, removal of spatial undersampling artifacts and anatomical segmentation for improved image reconstruction. Specifically, we define 44 experiments corresponding to the aforementioned challenges as benchmarks to be used as a reference for the development of more advanced processing methods.

Published in Transactions on Machine Learning Research (02/2023)
OADAT: Experimental and Synthetic Clinical Optoacoustic
Data for Standardized Image Processing
Firat Ozdemir Berkan 3
Xosé Luís Deán-Ben
Daniel Razansky²..
Fernando Perez-Cruz!i
Swiss Data Science Center, ETH Zurich and EPFL, Zurich, Swilzerland Institute of Pharmacology and Toricology and Institute for Biomedical Engineering, Facully of Medicine, University of Zurrich, Switzerland Instätate for Biomedionl Engineering, Department of Information Technology and Electrical Engineering, ETH Zurich, Switzerland
Instttute for Machine Learning, Deportment of Computer Science, ETH Zurich, Switzerland
Reviewed on OpenReview: https: // openreview. net/ pet/forumm id=BV96/tK00G
Abstract
Optoacoustic (OA) imaging is based on excitation of biological tissues with nanosecond- duration laser pulses followed by subsoquent detection of ultrasound waves generated via light- absorption-mediatedd thermoelastic expansion. OA imaging features powerful combination between rich optical contrast and high resolution in deep tissues. This enabled the exploration of number of attractive new applications both in clinical and laboratory settings. However, no standardized datasets generated with different types of experimental set-up and associated processing methods are available to facilitate advances in broader applications of QA in clinical settings. This complicates an objective comparison between new and established data processing methods, often leading qualitative results and arbitrary interpretations of the data. In this paper, we provide both experimental and synthetic OA raw signals and reconstructed image domain datasets rendered with different experimental parameters and tomographic acquisition geometries. We further provide trained neural networks to tackle three important challenges related to OA image processing, namely accurate reconstruction under limited view tomographic conditions, removal of spatial undersampling artifacts and anatomical sogmentation for improved image reconstruction. Specifically, we define 44 experiments corresponding to the aforementioned challenges as benchmarks to be used as a reference for the development of more advanced processing methods.
1 Introduction
Optoacoustic (OA) imaging is being established as a powerful method with increasing application areas in
clinical (Steinberg et al.) 2019 Su al. 2010) and preclinical settings (Lafci al. 2020 Merčep al. 2015).
Using nanosecond-duration pulsed lasers operating in the visible and near-infrared (NIR) optical wavelength range, biological tissues are thermoelastically excited. This excitation yields ultrassunnd (US) waves, from which OA images are tomographically reconstructed (Fig. [A). The rich optical contrast from endogenous tissue chromophores such as blood, melanin, lipids and others are combined with high US resolution, i.c., tens of micrometers. This unique feature makes OA particularly suitable for molecular and functional imaging. Other important advantages such as the fcasibility of hand-held operation, the fast acquisition performance (real time feedback) and the non-invasive safe contrast. (i.e., non-ionizing radiation) further foster the wide
Equual contribution. Author ordering determined by coin flip. Correspondence: www..oodemii@ddatsaeieeeee
Markan.lafciuuhh..

---

Published in Transactions on Machine Learning Research (02/2023)
use of OA in multiple biomedical studies. OA imaging has been shown to provide unique capabilities in
Diot et al. 2017 Butler et al. 2017). atherosclerotic carotid As the range of applications of OA imaging gets broader, the need for different data processing pipelines increases in parallel. Also, new methods are continuously being developed to provide an enhanced OA performance. Specífic examples include increased temporal resolution with compressed/speee /sparse data acquisitions (Özbek al. 2018), accurate image reconstruction algorithms (Deán-Ben et al.) 2012), light fluence correction by segmenting the tissue boundaries (Lafci et al. 2021 or enhanced spectral unmixing algorithms from multispectral data (Touumas
studies with disease models e.g,, of breast cancer MMnnhaar & Dantuma 2019
2018), as well as for the clinical assessment of Crohn's 's disease (Knieling et al.
plaques (Karlas et al. 2 or skin cancer (Deán-Ben Ben & Razansky [021].
& Ntziachristos 2017].
Three major challenges suitable for data-driven approaches in clinical OA imaging are summarized below: Sparse acquisition: OA imaging provides a unique potential to monitor fast-changing events such as cardiac arrhythmias (Cagla Özsoy et al., 2021). neuronal activity (Robin et al. 2021) or indocyanine green clearance (Grünherz et. a 2022) in vivo. For this, ultra-fast. imaging systems capable of capturing changes in living organisms occurring at up to millisecond temporal scales are required. The main limiting factor affecting the achievable frame rate is the data transfer capacity. This limitation can be climinated by reducing the number of acquired channels (signals). Therefore, sparse or compressed sensing methods have been proposed both using conventional methods (Ozbek et al., 2018) and deep learning algorithms Davoudi et al. 2019
Limited view reconstruction: OA inherently tomographic imaging modality. Acquisition of pressure signals from different angles is essential to capture the information encoded in US waves traveling in a 3D medium in order to render accurate tomographic reconstructions. This further inereases the image contrast, resolution and quantitativeness. However, tomographic coverage of the samples is often hindered by physical restrictions. Thereby, new image processing pipelines have been suggested to improve challenges in OA imaging by using data-driven algorithms in the image domain (Guan et al., 2020). signal domain (Klinovskaia et al., 2022) and combination of both domains (Davoudi et al. 2021 Lan et al., 2019)
Segmentation: Optimal OA reconstruction algorithms need to account for different optical and acoustic properties in biological tissues and in the coupling medium (water). For example, the speed of sound (SoS)
depends on the clastic properties of the medium. Proper assignment of SoS values in tissues and in water requires accurate delineation of the lissue boundaries. Thereby, segmentation of structures (Lafci et al. 2021 Schellenberg et al. 2022) in OA images has been shown to enhance the image reconstruction performance.
Additionally, the optical fluence (intensity) also varies with depth across different tissues. This issue remains as one of the main factors affecting quantification in OA images (Brochu et al. 2017 and can also be corrected with tissue segmentation (Mandal et al., 2016).
As an emerging method, OA imaging requires standardization, open source code publication and data sharing practices to expedite the development of new application areas and data processing pipelines. Particularly, the aforementioned challenges associated to high data throughput, limited angular coverage, SoS assignment and fluence corrections require coordinated efforts between experimental and data science communities. Initial efforts to standardize data storage formals and image reconstruction algorithms have been undertaken (Gröhl lt al. 2022). However, Gröhl et al. (2022) focus on standard reconstruction algorithms and propose data storage formats for acquisition related metadata. Data-driven image and signal processing methods require additional initiatives on fast access to large bulks of OA image and signal data and benchmarks for learning- based methods.
Here, we provide experimental data and simulations of forcarm datasets as well as benchmark networks aiming at facilitating the development of new image processing algorithms and benchmarking. These "Experimental and Synthetic Clinical Optoacoustic Data (OADAT)" include, (i) large and varied clinical and simulated forearm datasets with paired subsampled or limited view image reconstruction counterparts, (ii) raw signal acquisition data of each such image reconstruction, (iii) definition of 44 experiments with gold standards focusing on the aforementioned OA challenges, (iv) pretrained model weights of the networks used for each task, and (v) user-friendly scripts to load and evaluate the networks on our datascts. The presented datascts and algorithms will expedite the research in OA image processing.
2

---

Published in Transactions on Machine Learning Research (02/2023)
(a)
(b)
Semi Circle Array
(c) Multisegment Array
Arm
Transducer Array
Imaging
Medium
(d)
SC Sparse 128
SC Sparse 64
SC Sparse 32
SC Limited 128
WER TLL
ARUNE LIRRR DRUVE
(e)
MS Sparse 128
MS Sparse 64
MS Sparse 32
MS Linear 128
MINM KND NEN
Figure 1: Experimental data acquisition, transducer arrays and resulting images. (a) Experimental setup for optoacoustic forearm imaging. (b) Semi circle array along with an example of acquired images. (c) Multisegment array along with an example of acquired images. (d) Uniform subsampling for the semi circle array (128, 64 and elements) and limited view acquisition for the semi circle array with reduced angular coverage (128 elements). (e) Uniform subsampling for the multisegment array (128, 64 and 32 elements), and lincar array acquisition for the multisegment array (128 clements). Transducer elements are shown actively receiving (red) off (white).
3

---

Published in Transactions on Machine Learning Research (02/2023)
2
Background
For OA imaging, the objects are excited with the nanosecond-duraas laser pulses in visible or NIR light wavelengths which result in thermoelastic expansion of the structures. This expansion generates pressure waves (US signals) that are detected by transducer arrays. Corresponding images are reconstructed by solving the OA inverse problem on the acquired signals. Below, we explain the transducer arrays used for data acquisition, the reconstruction algorithm used generate images from acquired signals and the sampling/ acquisition techniques. Detailed explanation about OA imaging and used tools can be found in Appendix A.
2.1 Transducer arrays
Semi circle array contains 256 transducers elements distributed equidistantly over a semi circle (concave surface, Fig. [b). Multisegment array is a combination of linear array in the center and concave parts on the right and left sides, designed to increase angular coverage, as shown in Fig. Dc. The linear part contains 128 elements and each of the concave parts consist of 64 elements, totaling to 256. Linear array is the central part of the multisegment array with 128 elements (Fig. [+>- The array geometry is optimized for US data acquisitions with planar waves. Hence, it produces OA images with limited view artifacts due to reduced angular coverage which is limiting factor for OA image acquisitions. Virtual circle array is generated to simulate images with 360 degree angular coverage and yields artifact. free reconstructions (Fig. 2h). It. contains 1,024 transducer elements distributed over a full circle with equal distance. We also have a virtual multisegment array that correspond to its physical counterpart. Additional geometric and technical details are listed in Appendix A.2.
2.2 Reconstruction method
We use backprojection algorithm in this study to generate OA images from the acquired signal This algorithm is based delay and sum beamforming approach (Ozbek et al.) 2013) (see Appendix A.3 for details). First, a mesh grid is created to represent the imaged field of view. Then, the distance between the points of the mesh grid and array elements are calculated based on the known locations of the transducers. Time of flight is obtained through dividing distance by the SoS values that are assigned based on the temperature of the imaging medium and tissue properties. The clinical and simulated data are reconstructed with SoS of 1,510 m/s in this study as the simulations and the experiments were done at the corresponding imaging medium temperature. Unlike more sophisticated model-based reconstruction approaches Deán- Ben et al. 2012), backprojection is parameterized using only SoS, making it stable choice across all imaged scenes.
2.3 Sparse sampling
Sparse sampling yields streak artifacts on reconstructed images due to large inter-element pitch size. For a given angular coverage, i.e., transducer array geometry, using less transducer elements for reconstruction causes stronger artifacts due to increased inter-element pitch size. We define sparse sampled semi circle array acquisitions semi circle sparse 128, semi circle sparse 64 and semi circle sparse 32 when using 128, 64 and 32 elements out of the 256 of semi circle array (Fig, [d first three columns), respectively. Similarly, define sparse sampled virtual circle array acquisitions virtual circle sparsc 128, virtual circle sparse 64 and virtual circle sparse 32 when using 128, 64 and 32 elements out of 1,024 of virtual circle array (Fig. 2 first three rows), respectively. In addition, we also define sparse sampled multisegment array acquisitions multisegment sparse 128, multisegment sparse 64 and multisegment sparse 32 when using 128, 64 and 32 elements out of the 256 elements of the multisegment array (Figs. D and 2 first three columns), respectively. All items correspond to uniform and hence equidistant subsampling of the corresponding transducer array signals.
Python module for OA reconstruction: stthub.com//beraanafffe
4

---

Published in Transactions on Machine Learning Research (02/2023)
Full Sampling
VC Sparse 128
VC Sparse 64
Sparse
VC Limited 12B
Multisegment
MS Sparse 128
MS Sparse 64
MS Sparse 32
MS Linear 128
35.0
0.08
0.6
0.5
30.0
Probability
0.06
Dendry
0.4
BSNA
0.3
25.0
22.5
0.02
20.0
0.00
0.0
17.5
1000
2000
3000
4000
5000
1
2
3
5
6
7
8
9
ms
finer
vc.lv18
ms.so ss12
ms 3584
2532
re.s5226
vo.s532
#pixels per vessel
#vessels per Image
Figure 2: Overview of the simulated data. (a) Virtual circle array and an image reconstructed using 1,024 transducer elements. (b) Multisegment array and the corresponding image reconstructed using combined linear and concave parts of the transducer array. (c) Uniform subsampling of virtual circle array with 128, and 32 elements and limited view acquisition with reduced angular coverage (128 elements). (d) Uniform subsampling of multisegment array with 128, 64 and elements and linear array acquisition (128 clements). (e) Vessel size distribution (pixels per vessel), number of vessels per image, and peak signal-to-noiss ratio of full sampling compared to other reconstructions (x axis naming conventions explained in Sec. 3.3 Transduccr elements are shown as actively recciving (red) or off (white).

---

Published in Transactions on Machine Learning Research (02/2023)
Table Publicly available OA datasets, supported tasks, provided data format(s), size, and content. Davoudi ct al. (2019) contains 274 mice and 469 phantom slices. [Huang (2021) has 10 mice with 10 frames (100
slices)
27 different wavelengths and 20 phantom slices.
tasks
image reconstruction
raw signal
size 5k instances)
clinical data
Dataset
limited view
sparse sampling
pixel annotations
Davoudi et
(2199)
x
x
x
Huang et al. 2021
x
x
x
/
x
MSFD (ours)
X
/
SWFD (ours)
X
V
/
/
SCD (ours)
/
OADAT-mini (ours)
V
/
/
X FUV
2.4
Limited View
Limited view acquisitions lead to distorted geometry (e.g., elongated vessels) due to the reduced angular coverage (Figs. I & :d last column, limited 128 and linear 128). To mimic commonly occurring limited view settings, we use a continuous subset elements for a given transducer. This corresponds to retaining inter-element pitch size while reducing the angular coverage. We define a limited view acquisition for each transducer array as follows: (i) Lincar array is the common practice in clinical scttings for US data acquisition [Jensen] 2007 Luca al. 2018). Typically, the same linear geomelry is combined with OA imaging to provide complementary information (Merčep et al. 2017 To model this clinically realistic scenario, we use the lincar part of the multisegment array for OA image reconstruction. (ii) Semi circle limited 128 uses half of the semi circle array; 128 transducer elements, yielding a quarter circle. The differences between linear and semi circle limited view array acquisitions are the inter-element pitch size, focusing and geometry the active area. (iii) Virtual circle limited 128 uses 128 consecutive elements (one eighth of circle) out of 1,024.
3 Datasets
We present four datasets (two experimental, one simulated, one fully annotated subset) where cach has several subcategories for the purpose of tackling different challenges present in the domain. Raw signal acquisition data that is used reconstruct all images are also provided with the datasets. Experimental datasets also include details about the volunteer Fitzpatrick skin phototype (Gupta & Sharma 2019). which relates to the amount of melanin pigment in the skin (see Appendix B for distribution and further details). We also display comparative overview of publicly available and our proposed OA datasets in Table Please refer to Tables 5, 6, 7, 8, and 9 for summaries of the file contents of the datasets in the Appendix.
3.1 Multispectral forearm dataset
Multispectral forearm dataset (MSFD) is collected using multisegment array (Sec. 2.1 from nine volunteers at six different wavelengths (700, 730, 760, 780, 800, 850 nm) for both arms. Sclected wavelengths are particularly aimed for spectral decomposition aiming to separate oxy- and deoxy-hemoglobin (Perekatoval et al. 2016). All wavelengths are acquired consecutively, yielding almost identical scene being captured for a given slice across different wavelengths with slight displacement errors. For each of the mentioned category 1,400 slices are captured, creating a sum of x 6
400 1511,200 unique signal matrices.
From this data, using backprojection algorithm, multisegment array images MSFD ma. (iii) multisegment sparse 128 images (Sec MSFD ms.ss 28
reconstruct (i) linear array images MSFD liearr (ii)
(iv)
multisegment sparse 64 images (Sec 2.3 AISFDDss.see and (v) multisegment sparse 32 images (Sec 2.3
2Link datasets: hdd.handle.net//22 005.118800555111 /511512
3 Repository for accessing and reading dalasels: github.com /berkanlafct/ooddk

---

Published in Transactions on Machine Learning Research (02/2023)
MSFD a3, where each dataset has 151,200 images of 256 x 256 pixel resolution; totaling to 756,000 image instances.
3.2 Single wavelength forearm dataset
Single wavelength forearm dataset (SWFD) is collected using both multisegment. and semi circle arrays (Sec. 2.1] from 14 volunteers at single wavelength (1,064 nm) for both arms. The choice of the wavelength is based on maximizing penctration depth for excitation light source (laser) (Sharma et al., 2019). Out of the 14 volunteers, eight of them have also participated in the MSFD experiment. and their unique identifiers match across the dataset files. For each array, voluntoer, and arm, we acquired 1,400 slices, creating sum of x 14 2 x 1, 400 = 78, 400 unique signal matrices. It important note that despite the data being acquired from the same volunteers, signals between multisegment array and semi circle array are not paired due to physical constraints.
From this data, using backprojection algorithm, we reconstruct (i) linear array images, SWFD (ii) multisegment array images, SWFDD ms: (iii) semi circle array images, SWFD sc. (iv) semi circle array limited
128 images (Sec 2.4p. SWFD sc.128 (v) semi circle sparse 128 images (Sec 2.3) SWFDge.ss125 (vi) semi
circle sparse 64 images (Sec 2.3). SWFDsc,sdee (vii) semi circle sparse 32 images (Scc SWFD 5883,
(viii) multisegment sparse 128 images (Sec 2.3). ms,s1m, (ix) multisegment sparse 64 images (Sec 2.3
SWFD ma, se64; and (x) multisegment sparse 32 images (Sec 2.3). SWFD $3,3,, where each dataset has 39,200
images of 256 x 256 pixel resolution; totaling to 392,000 image instances.
3.3 Simulated cylinders dataset
Simulated cylinders dataset (SCD) is a group of synthetically generated 20,000 forcarm acoustic pressure maps that we heuristically produced based on a range of criteria we observed in experimental images. The acoustic pressure maps are generated with 256 x 256 pixel resolution where skin curves and afterwards a random amount of ellipses with different intensity profiles are generated iteratively for a given image (see Fig. 2). We explain details for the simulation algorithm for generating acoustic pressure map in Appendix E.
Based on the acoustic pressure map, we generate its annotation map with three labels, corresponding to background, vessels, and skin curve. For each acoustic pressure map, generate signal matrices for the geometries of linear, multisegment and virtual circle arrays. Using linear and multisegment array signals, we use backprojection algorithm to reconstruct. (i) linear array images, SCD and (ii) multisegment array images, SCD ms, (iii) multisegment sparse 128 images (Sce 2.3
(iv) multisegment sparse 64 images 2.3) SCDms, and (v) multisegment sparse 32 images (Sec SCDms, From virtual circle array signals, we use backprojection algorithm reconstruct (vi) virtual circle images, SCDve, (vii) virtual circle limited 128 images (24), SCD (vii) virtual circle sparse 128 images (Sec SCD vc.ss18. (ix) virtual circle sparse 64 images (.3). SCDve, ve,se61, and (x) virtual circle sparse 32 images (Sec 2.3 SCD SCD ve,332, where each dataset has 20,000 images of 256 x 256 pixel resolution; totaling to 200,000 image instances. All ten image reconstruction variations of SCD have corresponding pairs for each of the 20k image; i.c., produced from the same acoustic pressure map.
3.4 OADAT-mini dataset
Using a subset of 100 signal and corresponding reconstruction instances from each of the previously mentioned datasets, we also present OADAT-mini, which a fragment of OADAT that significantly smaller yet. comprehensive for understanding the contents of OADAT. In addition, OADAT-mini contains manual annotation maps for vessels in the reconstructed images.
4 Tasks
we define a list of experiments related to image translation to overcome (i) sparse sampling and (ii) limited view artifacts, and semantic segmentation of images.
Based on the datasots presented in Sec. 3
4Python module for acoustic map simulation: fenkulaa..o//ghlab///feee mmblualbw/iiaa///d...eee/aaaaaar
7

---

Published in Transactions on Machine Learning Research (02/2023)
Table 2:
List of tasks and experiments we define on MSFD, SWFD, and SCD, Experiment names are built of (i) dataset name for translation tasks or seg for segmentation task, (ii) input data and corresponding number of active array elements; sparse sampling (ss), limited view (Iv), virtual circle (ve), and (iii) input array type; semi circle (sc), virtual circle (vc), linear (li), multisegment (ms). Image data and annotation maps are represented with v and y, while predicted image and annotations are shown as x* and y².
Limited view correction
Semantic segmentation
[MSPD_Ivi2ee lv_8, 2 MSF linear + x² MSFD ma
fe,__, 2 ~ SCD,c - 9" ~ labels
SWWFD_iv12s./ 2 ~ SWFDiinea + x² SWFDma
fsog Iv128,8 2 SCDDneaa + y labels fang Iv128,i x SCD + -² ~ labels
SSWFD_iv128.0 a ~ SWTDas,lv12 -x SWFDsc
JSCD_ BBCD_Iv126,11 x ~ SCDtinear xx SSCDv
fsog_ss128, se1 x ~ ScDves > y" ~ labels
JsCD_Ivi28 x ~ vc. > xx" ~ SCDve
f ve x ~ ve. ve.5 > yy ~ labcls
3v2D_iv256.m= ~ SCDDm 7 x²' ~ SCDve
feog ve: x ~ ve. + y² ~ labels
Jsog_ ss128 msi a ~ ms.ee2s + y" labels
Sparse sampling correction
Jsog_ ms:4, x ~~ SCDms. 2664 y² ~ labels
SSWFD_=125,4 a ~ SWFD 22 + x²" SWFDsc
feeg_ m=32mms x SCD m3, 32 + y² ~ labels
JSWPD_ s6t,s: x ~ SWFD No 00 + x² ~ .WFD
SSWPD_ =dd,ac x ~ SWFD s0 32
+ x² SWFD
Jwy MSFD ms I. ~ MSFD MSFDinnar linear + y ~ labels
JSCD_ _et a ~ SCD ss.118 2218 + xx" ~CCDe
fwy MSFD mai X ~ MSFD ams. + y" ~ labels
fwe MSFD ms84,msi I ~ MSFD s081 > 4 ~ labcls
JsCD_ [scD_ad64,el 2 ~ SCD + x" ~ SSCDve
JSCD ss32.vel 2 ~ SCDvr, + r" ~ SCDvv
faog MSFD ass x. ~ MSFD
+ Dy ~ labels
JSWPD_ m12 : ~ SWFD 1a 28 xx ~ SWFD
fang SWED Iv128,ma: x. ~ SWFD Dinour + yy ~ labels
JSWPD ms SWWD__stt.ms
SWFD a64 - x- ~ SWFDms
fang. SWFD 1v128,56 & ~ SWFD 1v.128
y" ~ labels
JSWFD_ -02__322mmm__ me ~ SWFD m32 x² ~ SWFD
fweg_SW SWFD masma!
SWFD + y ~ labels
x² SCDve
fsog SWED sc: 2 SWFD,, + 4 labels
JSCD s I SCD 5064 x² SCDve
Jsog SWFD a=128.se 2 ~ SWFD se,
labels
JSCD_ 6=3,__<32,ms* SCD x² SCDve
Jang SWFD a0t64,ac x SWFD sc auti4 + 42 ~ labels
MSFFD_=125,m ~ MSFD mss as128 x² MSFDms ms
f Jog SWED SWFD_ss32sce _s32,scc I SWFD aa, ax32 - y² labels
MMSFD_=61,m= I MSFMMSPDD.... sa, xx MSFD
f Jog SWED WWFD_as128.nn ms1 x SWFD 5x1 +y yy labels
[MSFD_==32mes me: x MSF2
MSFDas
f Jog SWFD SWFD_sst4,ms" x SWFD me 2464
8² labels
Joog SWFD as32,ms i SWFD ms as32
- y² labels
4.1 Image translation
Through a list of permutations of our datasets, we can define several pairs of varying difficulty of image translation experiments where the larget images are also available (see Table 2 We present. sparse sampling and limited view reconstructions of SWFD, MSFD and SCD for all transducer arrays. Sparse sampling correction experiments learn mapping functions listed in Table 2 where the function notations denote the dataset used, the task of sparse sampling (ss) correction from the given number of elements used for image reconstruction and the array that is used to generate the input. Limited view correction experiments learn mapping functions, listed in Table 2 where the function notations denote the dataset used, the task of limited view (Iv) correction from the given number of elements used for image reconstruction and the array that is used to generate the input.
4.2 Semantic segmentation
SCD includes pixel annotations for skin curve, vessels and background. In addition to segmentation of these structures on the ideal reconstructions SCDve, we define this task on sparse sampling and limited view reconstructions that contain the relevant artifacts encountered in experimental data. Accordingly, we compose the nine segmentation experiments listed in Table 2 where the function notations denote the task segmentation (sog), type of the reconstructod input boing used (virtual circle (vc), limited view (Iv), and sparse sampling (ss)) and the array that is used to generate the input. All data is generated from SCD and the objective is to match the ground truth annotations of the acoustic pressure map. Different than SCD, experimental datasets under OADAT-mini include pixel annotations for vessels and consist of 14 segmentation
experiments, also listed in Table 2
8

---

Published in Transactions on Machine Learning Research (02/2023)
5 Experiments and results
c3-j
BN
relu
c3-J
BN
relu
22
c1-j
For all experiments we standardize the architecture that we based on UNet (Ronneberger et al.) 2015). Specifically, we adopt the five spatial feature abstraction levels and use skip connections to concalenate with features of matching spatial dimension along the upsampling path. However, we make several distinct design choices that vary from vanilla UNet. First, we use attention gates (Oktay al,, 2018) at the end of each skip connection. Second, opt for residual convolutional blocks with batch normalization (Toffe & Szegedy 2015) at each level, shown in Fig. 3 Third, we use 2D bilinear upsampling instead of deconvolutions. Finally, we use half the number of convolutional kernels at each layer; e.g,, start with 32 convolutional filters as opposed to 64. Full schematic as well as other implementation details are discussed in Appendix C. We refer to our modified UNet architecture as modUNet hereon.
Figure 3: Residual convolutional block with batch normalization (BN). ci-j cony. layer have x kernels and j filters.
5.1 Data split and preprocessing
We standardize how split each dataset into training and test sets regardless of the task in order to ensure consistency in our and urt experiments. Out of the nine volunteers in MSFD, we use five for training (IDs: 2, 5, 6, 7, 9), one for validation (ID: 10) and three for testing (IDs: 11, 14, 15). Out of the 14 volunteers in SWFD, we use eight for training (IDs: 1, 2, 3, 4, 5, 6, 7, 8), one for validation (ID: 9) and five for testing (IDs: 10, 11, 12, 13, 14). Out of the 20k slices in SCD, we use the first 14k for training, following 1k for validation, and the last 5k for testing. For each experiment conducted on OADAT-mini, we use the first 75 samples for training, next 5 for validation and the last 20 for quantitative evaluation. This translates to six times the numbers for MSFD-mini, where there 100 samples for each of the six wavelengths. As before, we conduct the MSFD-mini segmentation experiments as single modUNet attempting segment any of the given six wavelength samples. As a preprocessing step, all data instances (except for annotation maps) are independently scaled by their maximum and then clipped at a minimum value of -0.2 (Ding et al. 2015
5.2 Results
We evaluate modUNet performance the test sets using standard metrics. Namely, we report mean absolute error (MAE), root mean squared error (RMSE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) for image translation experiments between modUNet predictions and targets. Segmentation task performance is reported using Dice coefficient (Fl-score), intersection over union (IoU, i.e., Jaccard index) and 95-percentile Hausdorff distance (HD95) metrics between modUNet predictions and annotation maps for vessels and skin curves. HD95 is calculated in several steps: First, the set of pixels along the contour for each predicted (set A) and annotated (set B) target structures are found. For each set point, the closest point from the other set is determined based on l2 distance. Different from the standard Hausdorff distance, the 95-percentile distance value is taken the directional distance from set A to B (and vice versa) instead of the maximum distance. Then the maximum of these two values is calculated as the symmetric HD95 for a given image. In OADAT-mini experiments, only vessel annotation maps are available. Since OADAT -mii
500 vc
SWTD SWID,s512 32.3 ms
SCD vv
SWFD 1118.8
SWFD_ssb4.m
5CD m2
ScD_ss64,v
s532.s2
MSFD MSFD,ss32.mg ss300ss32mmm
SWFD_5s128.0
SCD mo128mms
SWFD 2554.5c
MSFD_Mv120.1
sCD_
SWFD, SWFD_M1228 (v18..cc
MSFD_2064.m5
2cD_ss100vc
SWFD_ss120.s:
MSFD_s6128.ms
0.92
0.04
0.96
0.98
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0
50
0.75
1.00
SSIM
SSIM
Figure 4: Distribution of modUNet. SSIM performance (CD(),ft), SWFD (middle) and MSFD (right) image translation experiments, sorted in ascending median test sample performance.

---

Published in Transactions on Machine Learning Research (02/2023)
Table 3: Image translation results of the proposed modUNet model reported as mean #std. Each row corresponds to the results of the experiment where input data identified through (i) the input data and corresponding number of active transducer elements; sparse sampling (ss), limited view (Iv) and (ii) the array type used for input; semi cirele (sc), virtual circle (vc), linear (li), multisegment (ms).
MAE
RMSE
SSTM
PSNR
SCD
1v128,
0.005
+1.3e-3
0.012 33.3e-3
0.978
#7.2e-3
39.909
22.15
Limited view
Iv128, vc
0.005 11.3e-3
0.013 133.e-3
0.978
16.2e-3
39.853
12.1
ly256, ms
0.005 11.2e-3
0.009 13.0e-3
0.984 16.1e-3
42.459
12.52
ss128,vc
0.004 11.1e-3
0.008 1:2.0e-3
0.985 16.1e-3
44.068
12.1
ss64,vc
0.005 11.3e-3
0.011 12.9e-3
0.981 16.9--3
41.402
-225
8s32,vc vc
0.006 +1.5e-3
0.013 +3.7e-3
0.973 +8.7e-3
39.309
+2.23
Sparse view
ss128,ms ms
0.005 +1.1e-3
0.010 +3.0e-3
0.984 +6.1e-3
42.345
+2.39
ss
0.005 #1.2e-3
0.010 #3.0e-3
0.982 #6.7e-3
41.525
22.19
ss32,ms
0.005 +1.3e-3
0.011 #3.0e-3
0.979 #7.3e33
40.678 #2.10
SWFD
Iv128, li
0.028 +1.6e-2
0.089 +2.0e-2
0.613 =e
29.397
#4.75
Limited view
lv128,sc
0.016 +1.2e-2
0.021 #1.5e-2
0.811 =1.3e1
36.791 #5.30
8s128, sc
0.015 +1.2e-2
0.019 +1.5e-2
0.863 +1.0e1
38.233
+5.35
ss64,sc
0.019 +1.5e-2
0.024 +1.8e-2
0.769
+1.6e1
35.605 +5.22
ss32,sc
0.021 =1.6e-2
0.028 #2.0e-2
0.693 #2.0e-1
33.852 =5.37
Sparse view
ss128,ms ms
0.023 =1.5e-2
0.028 #1.9e-2
0.784 #9.7e-2
33.764 #4.53
ss64,mss
0.029 11.0e-2
0.037 12.3e-2
0.636
11.5e-1
31.311 14.80
ms
0.033 12.1e-2
0.042 12.5e-2
0.521 11.8e1
29.813 15.11
MSFD limited view
1v128,
0.023 11.le-2
0.035 11.4e-2
0.640
11.4e1
29.731 13.97
ss128,ms ms
0.017 4-9.ee-
0.022 +1.2e-2
0.839
18.
35.798 +3.84
Sparse view
ss64,ms
0.022
11.2e-2
0.029
11.5e2
0.719 #1.4-1
33. #4.02
ss32,ms
0.026 #1.4e22
0.036 11.7e22 0.608 =1.6-1
30.873 #4.22
consist of subsets of the other three datasets, we do not repeat image translation tasks. In Tables 3 we report modUNet results for mean and standard deviations aggregated over the corresponding test set images.
Using SSTM, show performance across all datasets in Fig. 4 Upon exploring the reason behind the long tails, we notice that most of the lower scores occur when acquisition noise and/or artifacts are more pronounced. Depending on the sample, this can imply either modUNet reduced the noise present in the target, or both input/output pair in the test set had low signal-to-noise ratio(SNR). Nevertheless, mod succossfully corrects geometric distortions for limited view experiments. Given the low mcan and standard deviations in MAE and RMSE, we can comment that modUNet can generalize well to previously unseen volunteer data. This is further corroborated with the narrow SSIM interquartile range Fig. violin plots.
Similarly, we plot the segmentation performance for IoU across different experiments SCD in Fig. 5 for vessel and skin curve labels. While skin curve segmentation performance almost never drops below loU of 0.80 for SCD, one can see that IoU can be drastically lower for vessel segmentation. We observed that this only happens when the size of vessel is small. For example, there are examples with ground truth vessels having as low as four pixels while the prediction has six, leading to an loU of 0.6. As for experiments with OADAT-mini, vessel segmentation performance in experimental datasets are worse. Specifically, the worst scores are observed for MSFD-min D-mini experiments. This due the movement artifact between different wavelengths of given slice. All MSFD-mini instances have expert annotations for 800 wavelength. Slight movement across different wavelengths can yield poor quantitative metrics, particularly exacerbated when the observed vessels are small. Given the limited training and test sizes of OADAT-mini experiments, we believe that the quantilative results should be taken as reference. The qualitative results in the Appendix can be
10

---

Published in Transactions on Machine Learning Research (02/2023)
seg seg_ss32,vc
seg_ss32.vc
seg_ vc
seg_ss64,vC
seg eeg_iv128,i
seg. eeg_lv128,vc
vc
reg_ss32.ms
seg_ss33.ms
seg seg_lv178.1i
seg_ss178,vc
seg_ss128,vc
seg_ss64,ms
seg_ss64,ms
seg_ ms_
seg seg_ss128,ms
seg_ vC,v
Vc,_v
0.0
0.2
0.4
0.6
0.8
1.0
0.7
0.8
0.9
1.0
loU
loU
Figure 5: Distribution of modUNet IoU performance on SCD semantic segmentation experiments for vessel (left) and skin curve (right)) labels, sorted in ascending median test sample performance.
Table 4: Segmentation results of our proposed modUNet model reported as mean Estd. Each row corresponds to the results of the experiment where input data is identified through (i) the input data and corresponding number of active transducer elements; sparse sampling (ss), limited view (lv), virtual circle (vc), semi circle (sc), multisegment (ms) and (ii) the array type used for input; virtual circle (vc), semi circle (sc), multisegment. (ms), and linear (li). SWFD- and MSFD-miní correspond to experiment conducted OADAT-mini dataset.
Dice
ToU
HD93
vessels
skin curve
vessels
skin curve
vessels
skin curve
SCD Full view
vc,vc
119.80. 3o-3
18.7704 76-4
0.993 11.86 86-8
0.999 11.76-3
0.792 11.0c
0.164 1:2.cc
1v128,1i
0.963
1:3.4 46-4
0.988
17.3c-3
0.931
15.5 15.5e-2
11.4-22 0.976
2.734 12.90 I0
2.196 17.4c
Limited view
1v128,ve
0.933
5.-e22
0.978
11.1e-2
0.878
18.668 18.66-2
0.957
12.1
4.310 +8.96+00
7.372 11.4+11
ss1 vc
0.980 +-2.3e-2
0.990
+
0.961 +
0.980 +9.1e-3
2.499 +5.2e+00
3.897 +1.0e+1
ve
0.967 2e-8
0.965 +1.5e-2
0.937
24.6e-2
0.933 #2.8e-2
2.738 +3.0e+0
9.634 +1.5e+1
ve
0.938 #3.9e-2
0.946 +2.3e-2
0.886 #6.00-
0.899 #4.1e-2
3.074 #3.1e+0
12.220 +1.6++1
Sparse view
as 1288
0.983 +
0.994 +2.9e-3
0.968 +4.1e-2
0.989 +5.6e-3
2.289 +2.4e+0
5.311 +1.3++1
ss64,ms
0.984 12.46-2 4e-2
0.993
13.1c-3
0.969
14.1e-2
0.985 16.1e-3
2.772 18.20
2.668 18.5c
ss32,ms
0.971 12.88-2
0.985 14.86-
0.945 14.5e-2
0.971 19.2e-3
3.026 +7.00+0
6.795 11.6e+1
SWFD-mini
Full view
SC,Sc
0.857 15.7e-2
N/A
0.755 1..56
N/A
17.225 +
N/A
0.842 #5.ee-2
N/A
0.732 #8.66-)
N/A
17.218 +2.6e+1
N/A
1v128,li
0.794
16.40-2
N/A
0.662 18.3-2
N/A
28.685 13.9c
N/A
Limited view
1v128,sc
0.843 15.6e2
N/A
0.732 87.8 N/A
24.444 +2.60+1
N/A
ss
0.864 14.3e-2
N/A
0.763 46.5e-2
N/A
23.146 13.0e+1
N/A
ss64,sc
#1.le1
N/A
0.737 11.3e1
N/A
20,216 +2.8e+1
N/A
ss32,sc
0.864 14.1e-2
N/A
0.762 16336
N/A
26.832 13.0et1
N/A
Sparse view
ss ms
0.836 15.8e-2
N/A
0.722 18.362 N/A
17,909 12.8c
N/A
ss
0.837
15.6e-2
N/A
0.723 18.002
N/A
15.811 12.3c
N/A
ss
0.809 +-6.e-2
N/A
0.684 +8.2+2
N/A
20.603 +3.1e+1 N/A
MSFD-mini Limited view
1v128,1i
0.474 +1.3e1 N/A
0.320 +1.26-
N/A
20.134 +3.3e+1 N/A
as128,ms
0.563
110el
N/A
0.400 +1.le1
N/A
14.312 #1.9+11
N/A
Sparse view
8s64,ms
0.572
+ 1
N/A
0.411 1.33e1
N/A
19.716 12.1e+1
N/A
ss32,ms
0.639 +1.4e1
N/A
0.485 +1.ee1
N/A
11.499 +1.7+11
N/A
more informative for gaining insight for modUNet performance when trained with a very limited amount of data. We provide qualitative results well conduct. further analysis for all tasks in Appendix D.
11

---

Published in Transactions on Machine Learning Research (02/2023)
6 Discussion
Major differences exist between simulated and experimental datasets. Even if the content is different, training and test samples of the simulated dataset are inherently sampled from the same distribution. On the other hand, experimental datasets feature shifts due to different volunteers being imaged, inherent noise from data acquisition system, and difference in directional sensitivity resulting from transducer aligument and positioning of the hand-held probe. Furthermore, despite the efforts to avoid corrupted acquisitions during the data collection, experimental datasets still contain samples with relatively low signal-to-noise ratio. Such samples are expected to yield reduced performance metrics for image translation tasks due to significant. mismatch between the predicted and noisy target images. In a clinical setting, medical expert typically repeats acquisition if they deem the signal quality is significantly lower than expected. However, beyond this subjective filtering step, one needs to make sure that even the worst results are either sufficiently good or their poor performance can be attributed to a cause. Accordingly, we further analyze some of the worst samples the Appendix and believe that this should be a standard for future work. Provided datase is limited to one body part of volunteers without any known health issues. The image reconstruction methods for OA imaging can be applied on any other body part or imaging setup as they solve the same physical inverse problem. The image translation algorithms for sparse acquisition and limited view problems can be adapted for different devices and acquisitions by another clinician clinician/technicia technician at another center, as the streak artifacts originating from sparse acquisition and limited view follow the same pattern.
We envision that future research will tackle additional challenges such as unsupervised or weakly supervised domain adaptation across the datasets provided this work. There are initial studies to correct limited view artifacts in OA using transfer lcarning between simulated and experimental datascts after domain adaptation (Klimovskaia et. al. 2022). Similarly, transfer learning wet simulated and experimental domains can enhance segmentation performance of the vessels and skin curve in clinical images. Using propertics of the detected tissues for fluence correction and heterogencous SoS image reconstructions would then yield more accurate and quantitative images (Deán-Ben el al.) 2014). We anticipate additional contributions in the field of representation ion learning using OADAT, e.g. through self-supervised learning, could allow overcoming bottlenecks for specialized downstream tasks with limited amount of task-specific available data. The multispectral datasets with paired images across multiple wavelength acquisitions are expected to facilitate the investigation of generative modeling of multispectral signals from a given wavelength. We also anticipate future work to explore novel multispectral unmixing approaches using MSFD, enabling more accurate quantification of oxygenation, melanin and lipid content of the tissues. Finally, the provided raw signal data, not available in commercial devices, has a high value for data- drive research. For example, it can serve to benchmark methodologies in image reconstruction e.g., based on variational networks with loop unrolling (Vishnevskiy et al. 2019 as well as ultra-fast imaging through adaptive channel sampling.
7 Conclusion
In this work, we provide experimental and synthetic clinical OA data covering a large variety of examples to be used as common ground to compare established and new data processing methods. The datasets correspond to samples from volunteers of varying skin types, different OA array geometrics, and several illumination wavelengths. A subset of this experimental data is annotated by an expert. The dataset is supplemented with simulated samples containing ground truth acoustic pressure maps, annotations, and combine pairs of samples reconstructed with different OA array geometries. We define a set of 44 experiments lackling major challenges in the OA field and provide reconstructions of the images under these scenarios along with their corresponding ground truths. We propose and release 44 5 trained neural networks that achieve a good performance for all these examples which can be used as baselines for future improvements. Additional problems can further be defined with the data provided, such as the effects of random sparse sampling or the presence of noise in the signal matrices prior to reconstruction. We believe that these datasets and benchmarks will play a significant role in fostering coordinated efforts solve major challenges in OA imaging.
SPretrained model weights and various scripts to train and evaluate modUNet. evatee
available at
12

---

Published in Transactions on Machine Learning Research (02/2023)
Broader Impact Statement
The dataset is anonymized by randomly assigning identity numbers for each volunteer. The true identities will not be accessible by any third party now or the future. The dataset should not be used draw any medical conclusions. The purpose of the dataset is help researchers to develop new image processing methods and provide benchmark scores.
Author Contributions
B.L. setup and acquired raw datasets, B.T., and F.O. analyzed and parsed datasets, F.O. conducted numerical experiments basod on the defined tasks. All authors reviewed the manuscript.
Acknowledgments
This work was supported by Swiss Data Science Center (grant C19-04).
References
Frederic M. Brochu, Joanna Brunker, James Joseph, Michal R. Tomaszewski, Stefan Morscher, and Sarah E. Bohndiek. Towards quantitative evaluation of tissue absorption coefficients using light fluence correction in optoacoustic tomography. IEEE Transactions on Medical Imaging. :6(2))322-3) -322-331, 2017. doi: 10.1109/TMI. 2607799
Reni Butler, Philip T. Lavin, F. Lee Tucker, Lora D. Barke, Marcela Böhm- Vélez, Stamatia Destounis, Stephen R. Grobmyer, Janine Katzen, Kenneth A. Kist, Erini V. Makariou, Kathy J. Schilling, Catherine A. Young, Basak E. Dogan, and Erin I. Neuschler. Optoacoustic breast imaging: imaging-pathologs correlation of oploacoustic features benign and malignant breast masses. American Journal of Roenigenology, 211 (5):1155 1170, 2018. doi: 10.2214/AJRR. AJR. 18435. URL 30106610.
www..o///...22//...11111 18435
PMID:
Neda Davoudi, Xosé Luís Deán-Ben, and Daniel Razansky. Deep learning optoacoustic tomography with sparse data. Nature Machine Intelligence, 1(10):453 460, Oct 2019. ISSN 2522-5839. doi: 10.1038/ s42256-019 0095-3. URL https:///oi. org/ 1038/ 01110//45522000000000-0
Neda Davoudi, Berkan Lafci, Ali Özbek, Xosé Luis Deán-Ben, and Daniel Razansky. Deep learning of image- and time-domain data enhances the visibility of structures in optoacoustic tomography. Opt. Lett., 46(13): 3029 3032, Jul 2021. doi: 10.1364/0L.424571. URL http://opg /
http:://og.ootica.ore... optica. wtt.icaa.rg/////aatttto
cfm?URI=
01-46-13-3023
Xosé Luis Deán- Ben, Andreas Buehler, Vasilis Ntziachristos, and Daniel Razansky. Accurate model-based reccton algorithm for three-dimensional optoacoustic tomography. IEEE Transactions on Medical Imaging, 31:1922-1922 2012.
X. Luís Deán-Ben Ben, Vasilis Nuziachristos, and Daniel Razansky. Elfects of small variations of speed of sound in optoacoustic tomographic imaging. Medical Physics, 41(7)0073301, 2014. doi: https://doi.org/// wtt..o/////..///....... 4875691. URL https:://aaappp onwemellnnnnnnnnel htt.eeeebbrrra..leeeo.eooo////// http:////.../aaaa.1.....111/1/60
Xosé Luis Deán-Ben and Daniel Razansky. Optoacoustic imaging of the skin. Experimental Dermatology, 30(11):1598 1609, 2021. doi: htlps://doi.org ftt..or//1..11//xx.... URL httpp:///omlinelibbaayy wiley
con//oo/abs//0.111110 1438
Lu Ding, Luis Deán- Ben, Christian Lutzweiler, Daniel Razansky, and Vasilis Ntziachristos, Efficient non-negative constrained model-based inversion in optoacoustic tomography. Physics in Medicine and Biology, 60(17):6733 6750, aug 2015. doi: 10.0085/0033-99155///////// URL https:://doi.org/10 1085/0081--1155/66/1//6
Gael Diot, Stephan Metz, Aurelia Noske, Evangelos Liapis, Barbara Schroeder, Saak V. Ovsepian, Reinhard Meier, Ernst Rummeny, and Vasilis Ntxiachristos. Multispectral Optoacoustic Tomography (MSOT)
13

---

Published in Transactions on Machine Learning Research (02/2023)
of Human Breast Cancer. Clinical Cancer Research, (((2)):6912-6222 11 2017. ISSN 1078-0432. doi: 10. 0000//078-0422.CRRC
URL
http///i..o.///.o//// 10/ /tt.://///...1111000---000000000 0057788-4422.000 CCR-16-3200
Janek Gröhl, Lina Hacker, Ben T. Cox, Kris K. Dreher, Stefan Morscher, Avotra Rakotondrainibe, François Varray, Lawrence C.M. Yip, William C. Vogt, and Sarah E. Bohndiek. The ipasc data format: A consensus data format for photoacoustic imaging. Photoacoustics, 26:100339, 2022. ISSN 2213-5979. doi: patp://......./...........2..... pis/S22136577222000166
E. Grüneisen. Theorie des festen zustandes einatomiger elemente, January 1912. URL https://doi.orgg//
0002/andp.19123441220
Lisanne Grünherz, Epameinondas Gousopoulos, Carlotta Barbon, Semra Uyulmaz, Berkan Lafci, Daniel Razansky, Andreas Boss, Pietro Giovanoli, and Nicole Lindenblatt. Preoperative mapping of lymphatic vessels by multispectral optoacoustic tomography. Lymphatic Research and Biology. 0(0), 2022. doi:
10.1059/1tb.2221.000 URL https:///oi. 10..0000000/101.11/111bb www.:100911bbbb 2021. 2067
PMID: 35230197.
Steven Guan, Amir A. Khan, Siddhartha Sikdar, and Parag V. Chitnis. Limited-view and sparse photoacoustic tomography for ncuroimaging with deep learning. Scientific Reports, 1:510, May 2020. ISSN 2045-2322. doi: (0008///11598000--0111
URL http:::/bb...rr///..100//4444553
65235-22
Vishal Gupta and Vinod Kumar Sharma. Skin typing: Fitzpatriikk grading grading and others. Clinics in Dermatology, 37(5):430-436 2019. ISSN 0738-081X. doi: http:///bbiiong///00000///ddddde
2019.0000000 URL The Color of Skin.
Shan Huang, Andreas Blutke, Annette Feuchtinger, Uwe Klemm, Robby Zachariah Tom, Susamna M Hofmann, Andre C Stiel, and Vasilis Niziachristos. Functional multispectral optoacoustic tomography imaging of hepatic steatosis development in mice. EMBO Molecular Medicine, 13(9)x113490, 2021. doi: https:/ hoops://doi.rrg.. 10.15252 (0.15222/emmm.22222 (mmmm.202013400000
URL httph:htts/wwtt httpps:/www.eebborrrsssssss
ewbopress http:pppws...rg.oo/o/a///a/////s 15252/ e0055mmmmmmmmmmmmmmm 202013490
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. PMLR, 2015.
Jørgen Arendt Jensen. Medical ultrasound imaging. Progress in Biophysicc and Molecular Biology, 93(1): 153 165, 2007. ISSN 0079-6107. doi: http:://doooooo//100000//ooooooo wattnolbio.2006.000 20..20000000000 07.05. URL hittps:/wwat com relevant to human health.
Effects of ultrasound and infrasound
Angelos Karlas, Michael Kallmayer, Michael Bariotakis, Nikolina-Alexia Fasoula, Evangelos Liapis, Fabien Hyafil, Jaroslav Pelisck, Moritz Wildgruber, Hans-Honning Eckstein, and Vasilis Ntziachristos. Multispectral optoacoustic lomography of lipid and hemoglobin contrast in human carotid atherosclerosis. Pholoucouslics, 23: 23:100283, 2021. ISSN 2213-5979. doi: http:://bbi.ong//00000/// pacp:////jpp....a.ee pacs.2021 100283.00.... URL https:: //w/ww
sciencedirect. com http:///ww....aa...........cccta
Anna Klimovskaia, Berkan Lafci, Firat Ozdemir, Neda Davoudi, Xose Luis Dean-Ben, Fernando Perez-Cruv and Daniel Razansky. Signal domain learning approach for optoacoustic image reconstruction from limited view data. In Medical Imaging with Deep Learning, 2022. URL https:://penreevie:: wtt.://wwww.eeteooomormoiiii.
9NOyr
Ferdinand Knieling, Clemens Neufert, Arndt Hartmann, Jing Claussen, Alexander Urich, Cornelia Egger, Marcel Vetter, Sarah Fischer, Lukas Pfeifer, Alexander Hagel, Christian Kielisch, Rüdiger S. Görtz, Dane Wildner, Matthias Engel, Jens Röther, Wolfgang Uter, Jürgen Siebler, Raja Atreya, Wolfgang Rascher, Deike Strobel, Markus F. Neurath, and Maximilian J. Waldner. Multispectral optoacoustic tomography for assessment of crohn's 's discasc activity. New England Journal of Medicine, 376(13):1292-120) 2017. doi: D0.156/NNEJMc161993
URL https::////ee //doi. owggoog//// 1etpg/....106//111
PMID: 28355498.
Berkan Lafci, Elena Merěep, Joaquin L. Herraiz, Xosé Luis Deán- Ben, and Daniel Razansky, Noninvasive multiparametric characterization of mammary tumors with transmission-reflectii optoacoustic ultrasound.
14

---

Published in Transactions on Machine Learning Research (02/2023)
Neoplasia, 22(12):770-777 2020. ISSN 1476-5586. doi: http:://dbooorg//1.0006//// 202.2000..00 URL
http://www.ciinnedireeec.. com http:///sssaaaaaa.aaaaaaa///////
Berkan Lafci, Elena Merčep, Stefan Morscher, Xosé Luis Deán- Ben, and Daniel Razansky, Deep learning for automatic segmentation of hybrid optroacoustic ultrasound (opus) images. IEEE Transactions on Ultrusonics, Ferroelectries, and Prequency Control, 68(3):688 696, 2021. doi: 10. 10.108/TUFFTC.20000002222
Hengrong Lan, Daohuai Jiang, Changchun Yang, and Fei Gao. Y-net: A hybrid deep learning reconstruction http://www/..////////////////000
framework for photoacoustic imaging in vivo, 2019. URL
Ramona De Luca, Tommaso Daltoma, Leonardo Forzoni, Jeffrey Bamber, Paolo Palchetti, and Alessandro
Gubbini. Diagnostic ultrasound probes: a typology and overview of technologies. Current Directions in Biomedical Engineering, 4 4(1):49-53, 2018. doi:
URL bttpss//ddo..rg/ org/10
4615/cdbmet 2018-00033
Subhamoy Mandal, Xosé Luís Deán-Ben, and Daniel Razansky. Visual quality enhancement in optoacoustic tomography using active contour segmentation priors. IEEE Transactions on Medical Imaging, 35(10): 2209-2217, 2016. doi: 10.1109/TMM 20t....11..2553111
Srirang Manohar and Maura Dantuma. Current and future trends in photoacoustic breast imaging. Pho- toacoustics, 16:100134, 2019. ISSN 2213-5979. doi: httpss//dooiorr///0000/// ptt.:/.................... URL
https://www.cr http://wwwssciincedereee wwwpsciicc////eaaeeeaeeeeeaeeeee wtt.://cccc.......o..o...////aaa
Elena Merčep, Neal C. Burton, Jing Claussen, and Daniel Razansky. Whole-body live mouse imaging by
hybrid reflection-mode ultrasound and optoacoustic tomography. Opt. Lett., 40(20):4643 4646, Oct 2015.
doi: 10.1364/ 00.4/000440.004443 URL
Elena Merčep, Xosé Luis Deán-Ben, and Daniel Razansky. Combined pulse-echo ultrasound and multispectral optoacoustic tomography with a multi-segment detector array. IEEE Transactions on Medical Imaging, 36 (10):2129 2137, 2017. doi: 10.1009/TM..2017.27700
Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven MeDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: Learning where to look for the pancreas. Xiv preprint arXiv:180; 08999, 2018.
Ali Ozbek, X. L. Deán-Ben, and Daniel Razansky. Realtime parallel back-projection algorithm for three- dimensional optoacoustic imaging devices. In Vasilis Ntziachristos and Charles P. Lin (eds.), Opto-Acoustic Methods and Applications, volume 8800, pp. 90 95. International Society for Optics and Photonics, SPIE, 2013. doi: 10.1117/12.20888
URL sattps://dot http::////ooo...11.11111101
2033376
Ali Özbek, Xosé Luis Deán-Ben, and Daniel Razansky. Oploacoustic imaging at kilohertz volumetric frame rates. Optica, 5(7) 863, Jul 2018. doi: 101164//PTTIAA55000055 URL http://opg http://ppppppp.....ooo optica. org//
Valeriya Perekatova, Pavel Subochev, Mikhail Kleshnin, and Πlya Turchin. Optimal wavelengths for op- toacoustic measurements of blood oxygen saturation in biological tissues. Biomed. Opt. Express, 7(10): 3979-3995, Oct 2016. doi: 10.1364/B0E...0033 URL http:://pg..ptteeee
www.bo///ga/ttaaaeeeeeeo cfm?
URI=boe-7-10-3977
Justine Robin, Richard Rau, Berkan Lafci, Aileen Schroeter, Michael Reiss, Xosé Luis Deán-Ben, Oreun Goksel, and Daniel Razansky. Hemodynamic response to sensory stimulation mice: Comparison between functional ultrasound and optoacoustic imaging. Neurolmage, 237:118111, 2021. ISSN 1053-8119. doi:
http:://do..ongg//00000ojjonennn 10t atticle/ppi/S10038111100888
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234-241 Springer, 2015.
15

---

Published in Transactions on Machine Learning Research (02/2023)
Melanie Schellenberg, Kris K. Dreher, Niklas Holzwarth, Fabian Isensee, Annika Reinke, Nicholas Schreck, Alexander Seitel, Minu D. Tizabi, Lena Maier-Hein, and Janek Gröhl. Semantic segmentation of multi- spectral photoacoustic images using deep learning. Photoncoustics, 26:100341, 2022. ISSN 2213-5979. doi:
https::/dbbi.org/10..000 wttp::/o...///.p..00000 20ccc.2022..000444 URL sttpg:://ww..ddcinddddre....... http:///w....cocccccc//c/arrrrae p:1/SS21359792200018
Arunima Sharma, Srishti Srishti, Vijitha Periyasamy, and Manojil Pramanik. Photoacoustic imaging depth comparison at 532-, 800-, and 1064-nm wavelengths: Monte Carlo simulation and experimental validation. Journal of Biomedical Optics, 24(12):1 10, 2019. doi: 01.117/1.B8.22.122222111 URL
https://doi..rr http///oo.org//// 1117/1 JBO. 24. 12. 121904
Idan Steinberg, David M. Huland, Ophir Vermesh, Hadas E. Frostig, Willemieke S. Tummers, and Sanjiv S. ISSN 2213-5979, doi: 31 31293884[pmid]
Gambhir. Photoacoustic clinical imaging. Photoacoustics, 14:77-98, Jun 2019.
10.1016/1.pacc.20111 2at.s.0011.00000 URL htttps:/puubmed ncbi nlm mih. go1/39999119
Jimmy L. Su, Bo Wang. Katheryne E. Wilson, Carolyn L. Bayer, Yun-Sheng Chen, Seungsoo Kim, Kimberly A. Homan, and Stanislav Y. Emelianov. Advances in clinical and biomedical applications of photoacoustic
imaging. Expert opinion on medical diagnostics, 4(6):497-510, Nov 2010. ISSN 1753-0067. doi: 10.1517/
09330059.001.52911 URL httpps:/pubbed: ncbi nlm. mih. gov/2134402 21 21344060(pmid]
Stratis Tzoumas and Vasilis Ntziachristos. Spectral unmixing techniques for optoacoustic imaging of tissue pathophysiology. Philosophical transactions. Series A, Mathematical, physical, and engineering
sciences, 375(2107):20170222, Nov 2017. ISSN 1471-2962. doi: 10.1098/ssta.2201.0222. URL https 29038385(pmid];
//pubmed.ncbl ncbi. www.ciii.......mmmmmmmm.m...m..m www.mmmmm.m...mmmmmmmmmm.......m www.nii..go.... wtt......o//200333333333333
Valery Vishnovskiy, Richard Rau, and Oreun Goksel. Deep variational networks with exponential weighting for learning computed tomography. In Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essort, Sean Zhou, Pew-Thiam Yap, and Ali Khan (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2019, pp. 310 318, Cham, 2019. Springer International Publishing. ISBN
Çağla Özsoy, Ali Özbek, Michael Reiss, Xosé Luis Deán-Bec and Daniel Razansky. Ultrafast four-dimensional imaging of cardiac mechanical wave propagation with sparse optoacoustic sensing. Proceedings of the National Academy of Sciences, 11039991,,,,, 2021. doi: 10.1073/pnas 2103979118. 20..73//92a22210379111111 URL https //www. .nas. http//00//as/11..00///paaa 2t1.9/99.....099911111 2103979118
16


Reviews:
[{"review_id": "1lcrXUAlRB", "reviewer": "Reviewer_LBfi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Well written manuscript.\n- Good introduction to optoacoustic data acquisition.\n- The discussed applications are motivated clinically.\n- The idea of introducing standardization practices in the optoacoustic data model comparison is sound and well motivated.\n\nWeaknesses (for details see below -- requested changes section)\n- Unclear what are the key learnings of the performed analysis.\n- The construction of the benchmark has missing motivation, and the benchmark is lacking comparisons to prior art.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a set of standardized datasets and tasks for machine learning processing of optoacoustic data. The paper introduces three tasks -- limited view correction, sparse sampling correction, and semantic segmentation – as well as four simulated and experimental datasets -- Multispectral forearm dataset (MSFD), Single wavelength forearm dataset (SWFD), Simulated cylinders dataset (SCD) and OADAT-mini a subset of 100 signal and corresponding reconstruction instances from each of the previously mentioned datasets. The paper concludes by reporting UNet-like model results for all three tasks ablating the type of input data, number of active transducer elements as well as the type of array used to acquire the signal.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "The paper is well written and provides a good overview of optoacoustic imaging. The details about data acquisition are interesting as well as the potential of the use of optoacoustic imaging in the clinic. Below the reviewer addresses the two acceptance criteria of TMLR journal. \n\n**Are the claims made in the submission supported by accurate, convincing and clear evidence?**\nIn general, the paper does not make unsupported claims and the reviewer appreciates the standardization efforts in the OA field. However, there are some aspects of the submission that might require clarifications/corrections.\n- From the text it is unclear to the reviewer why the three introduced tasks are truly the main challenges of OA. Further, the manuscript does not say why these three tasks are considered. Please clarify.\n- To truly refer to the paper as a benchmark the reviewer would expect to see a comparison to simple methods in each task. In the introduction, the authors cover a variety of approaches for which the results could and should be reported in the manuscript. For example, sparse acquisition – “sparse or compressed sensing methods have been proposed both using conventional methods (Özbek et al., 2018) and deep learning algorithms (Davoudi et al., 2019).”--, limited view reconstruction – “new image processing pipelines have been suggested to improve limited-view-associated challenges in OA imaging by using data-driven algorithms in the image domain (Guan et al., 2020), signal domain (Klimovskaia et al., 2022) and combination of both domains (Davoudi et al., 2021; Lan et al., 2019).” – and Segmentation -- “segmentation of structures (Lafci et al., 2021; Schellenberg et al., 2022) in OA images has been shown to enhance the image reconstruction performance. Additionally, the optical fluence (intensity) also varies with depth across different tissues. This issue remains as one of the main factors affecting quantification in OA images (Brochu et al., 2017) and can also be corrected with tissue segmentation (Mandal et al., 2016).” --. Any reason why the introduced model is not compared to prior art?\n- Any reason why the benchmark dataset is composed of relatively small datasets, e.g., “nine volunteers at six different wavelengths”, “14 volunteers at a single wavelength”? The datasets are large in the number of slices but small in the number of volumes. \n\n\n\n**Would some individuals in TMLR's audience be interested in the findings of this paper?**\nAlthough the applications of machine learning tools to the OA data could be of interest for the ML community and the introduction of datasets simplifies the application of ML tools to the OA data, the reviewer is not convinced whether the submission in question would generate interest in the TMLR community. \n- First, please see the reviewer’s comments w.r.t. the choice of the tasks and the lack of comparisons with prior art. Please clarify this in your rebuttal.\n- Second, the discussion of the reported results is rather short making it unclear to the reviewer what the added value to the community is, what the learnings of the performed data analysis are what the new generalizable learnings are. Please clarify this in your rebuttal.\n- Finally, the OA data analysis might be too niche topic for the TMLR community and as such might not generate enough interest. From this point of view, the manuscript could be better suited for more OA oriented venue or medical imaging journal. To address this comment the authors could work on the presentation of the paper so that it becomes more accessible and as a result also more attractive to the broader ML/CV audience.\n", "broader_impact_concerns": "The Broader Impact is included in the manuscript. The reviewer has no comments here.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tZKTID1Zk4", "reviewer": "Reviewer_zakN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: large-scale, annotated dataset, as presented in this work, is always welcomed by the research community, especially as there exist few public datasets for optoacoustic imaging data. \n\nWeakness: Table 2 is confusing to read; consider a more intuitive way of presenting the tasks. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presented a set of optoacoustic datasets, both real and synthetic, with various image acquisition parameters and reconstruction methods that could be used for the development and evaluation of image analysis methods of optoacoustic imaging. The paper also provided three “model cards” to illustrate how the proposed data could be used in different tasks (i.e., reconstruction under limited view tomographic conditions, removal of spatial under-sampling artifacts, and anatomical segmentation for improved image reconstruction).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1, Layout of table 2 should be revised\n\n2, It is recommended that an extra table should be added to summarize the sample size, data constitution, and unique characteristics of the datasets presented in this work.\n", "broader_impact_concerns": "The Broader Impact Statement in this work is sufficient.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "URKpgOah0u", "reviewer": "Reviewer_WrPA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n- The motivation and the importance of Optoacoustic (OA) imaging are well demonstrated.\n- As an emerging method, the provided experimental data and simulations of forearm datasets are important and promising.\n- The paper is well organized and the writing logic is clear.\n\nCons:\n- The backprojection algorithm is used to generate OA images from the acquired signal. Are there any other algorithms that can be used? What are the differences?\n- For sparse sampling, the sampled elements are fixed. Is it possible to make the sampling adaptive? i.e., using the different number of elements for different circle arrays.\n- For the segmentation task, it would be better to also take the HD distance as one evaluation metric.\n- For samples with a relatively low signal-to-noise ratio, besides repeating acquisition, are there any other technical solutions?\n- On SWFD-mini, some results from the sparse view even outperform the full view. What could be the reason? More explanations are appreciated here.\n- The UNet has already achieved good performance on different experiment tasks, the space for further improvements might be limited.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to study the topic of  OA tomographic imaging. The authors provide both experimental and synthetic OA raw signals and reconstructed image domain datasets with different experimental parameters and tomographic acquisition geometries. And use trained neural networks to tackle three important challenges related to OA image processing.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors need to make changes accordingly to address the concerns in the above section.", "broader_impact_concerns": "The Broader Impact Statement has been clearly stated, I have no extra concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The main open comment in the final assessment is: "However, at the same time the authors fall short in providing a proper benchmarking of current and past approaches to the OA tasks -- they only provide one model and as they recognize in the rebuttal the results section provide little discussion, making it unclear what are the interesting observations for the TMLR community from this analysis...."

This is phrased as a matter of interest to the community, as the content of the experiments and benchmarking matches the claims of the paper.  As such, no additional changes are strictly required.  I would suggest to the authors that actively enabling benchmarking through the software release, e.g. by also hosting a leaderboard, might be a useful way of increasing potential impact.
Decision: Accept (poster)