1

Deep Learning in RF Sub-sampled B-mode
Ultrasound Imaging

arXiv:1712.06096v2 [] 21 Dec 2017

Yeo Hun Yoon, Shujaat Khan, Jaeyoung Huh, and Jong Chul Ye, Senior Member, IEEE

Abstract—In portable, three dimensional, and ultra-fast ultrasound (US) imaging systems, there is an increasing need to
reconstruct high quality images from a limited number of RF
data from receiver (Rx) or scan-line (SC) sub-sampling. However,
due to the severe side lobe artifacts from RF sub-sampling, the
standard beamformer often produces blurry images with less
contrast that are not suitable for diagnostic purpose. To address
this problem, some researchers have studied compressed sensing
(CS) to exploit the sparsity of the image or RF data in some
domains. However, the existing CS approaches require either
hardware changes or computationally expensive algorithms. To
overcome these limitations, here we propose a novel deep learning
approach that directly interpolates the missing RF data by
utilizing redundancy in the Rx-SC plane. In particular, the
network design principle derives from a novel interpretation of
the deep neural network as a cascaded convolution framelets that
learns the data-driven bases for Hankel matrix decomposition.
Our extensive experimental results from sub-sampled RF data
from a real US system confirmed that the proposed method can
effectively reduce the data rate without sacrificing the image
quality.
Index Terms—Deep learning, ultrasound imaging, low-rank
Hankel matrix, convolutional framelets

I. I NTRODUCTION
UE to the excellent temporal resolution with reasonable image quality and minimal invasiveness, ultrasound
imaging has been adopted as a golden-standard for many
disease diagnosis in heart, liver, etc. Accordingly, there have
been many research efforts to extend the US imaging to new
applications such as portable imaging in emergency care [1],
3-D imaging [2], ultra-fast imaging [3], [4], etc. However, the
common technical hurdle in these applications is the lack of
techniques to reconstruct high resolution images from significantly reduced radio-frequency (RF) data under acquisition
physics constraint.
Specifically, according to the Nyquist sampling theorem, the
sampling rate for US transducer should be at least four times
the central frequency of the emitted pulse [5]. Consequently, to
achieve better spatial resolution, high speed analog-to-digital
converter (ADC) should be used for receiver (Rx) portion
of the US transducer, which is the major part of the power
consumption of the transducer. Accordingly, in portable US
systems, a small number of Rx elements are used to reduce
power consumption, which also reduces the size of the receiver
aperture, resulting in deterioration of image quality.
In B-mode imaging, the whole imaging area should be
swept using focused RF scan-lines. The acquisition time of an

D

The authors are with the Department of Bio and Brain Engineering, Korea
Advanced Institute of Science and Technology (KAIST), Daejeon 34141,
Republic of Korea (e-mail:{caffemocha,jong.ye}@kaist.ac.kr).

RF line is determined by the speed of sound, which limits the
temporal resolution of B-mode imaging. Therefore, to achieve
a higher frame rate in ultra-fast imaging or 3-D US imaging,
the number of scan lines should be significantly reduced. This
in turn leads to severe downsampling artifacts.
To address these problems, compressed sensing approaches
have been investigated [6], [5], [7], [8], However, computationally expensive optimization algorithms are usually required.
Recently, Wagner et al [7] models a scan line profile as
the finite rate of innovation (FRI), and proposed a speciallydesigned hardware architecture that enables high resolution
scan line reconstruction [7]. Another recent proposal is direct
RF interpolation using the low rank Hankel matrix completion
approach [9]. In particular, thanks to the strong correlation
between RF data across scan lines and adjacent temporal
frames, Jin et al [10] showed that a rank deficient Hankel
structured matrix can be obtained from the reordered RF data.
Accordingly, the missing RF data can be estimated using an
annihilating filter-based low rank Hankel matrix completion
approach (ALOHA) without ever changing hardware structure
[9]. However, the algorithm is computationally very expensive,
which is not suitable for routine clinical applications.
Therefore, one of the most important contributions of this
paper is to show that the Hankel matrix-based low-rank
interpolation of RF data can be quickly solved using deep
convolutional neural network (CNN). In particular, inspired
by recent discovery that a CNN can be interpreted as a deep
convolutional framelets obtained from a decomposition of the
Hankel matrix [11], we construct a CNN that performs direct
RF data interpolation. Thus, the final reconstruction can be
done using the standard delay-and-sum (DAS) beamformer
without changing any hardware/software beamformer structures. Compared to an image domain CNNs that attempts
to learn acquisition geometry-dependent artifacts, one of the
most important advantages of the proposed RF domain CNN
is its generalization power. Specifically, although an image
domain deep learning requires many sets of data from different
acquisition geometries and body areas [12], our CNN can be
trained using RF data that is measured by a specific transducer
for a particular organ, but it can be also used for other type of
transducers and/or different organs. Therefore, the proposed
system is very practical in real applications.
Extensive experimental results using RF data from a real
US system confirmed that the proposed method has significant
potential for RF sub-sampled B-mode US systems.

2

II. T HEORY
A. B-mode ultrasound imaging
A B-mode ultrasound imaging, which is most widely used
in practice, scans the body using focused beams and displays
the 2-D image as shown in Fig. 1. Here, SC, Rx and DAS
denote the scan line for each ultrasound beam, receivers of the
transducer, and delay-and-sum beamformer, respectively. More
specifically, after a focused ultrasound beam is illuminated
along the scan line as shown in Fig. 1(a), the ultrasound beam
is reflected at some tissue boundaries and the reflected US data
are recorded by Rx as RF data (see Fig. 1(b)). Thus, DepthRx coordinate RF data is obtained for each scan line (SC),
and this is repeated for each scan line to obtain a 3-D cube of
RF data in Depth-Rx-SC coordinates. In classical US systems,
a transducer has a hardware DAS beamformer that uses the
3-D stacked data to generate a time frame which is then
sent to the image station (see Fig. 1(c)). On the other hand,
software-based US systems transfer RF data directly via highspeed cable, where a software-based beamformer reconstructs
images.
B. RF sub-sampled US systems
Here, we consider two types of proof-of-concept RF subsampled US systems. The first one is a randomly sub-sampled
dynamic aperture US system [10] that acquires only partial
RF data as shown in Fig. 2(a). Because the hardware does not
need to be changed, this method can be easily used in any US
Fig. 2. (Top) Data acquisitions of the randomly sub-sampled dynamic aperture
US system. (Bottom) data acquisitions of the SC sub-sampled US system,
when downsampling ratio is 2.

(a) Beam emission

(b) RF data recording

system with software beamformer to reduce data transmission
bandwidth and power-consumption by minimizing the active
Rx number. Moreover, this may be also good for a portable
US system which has a smaller number of Rx channels due to
the limited transmission bandwidth. The second one is the SC
sub-sampled US system as shown in Fig. 2(b). By skipping
some scan lines, the frame rate limit can be reduced and a
high frame rate can be achieved with 2D or 3D scans. In fact,
the SC sub-sampling can be combined with Rx sub-sampling
to further reduce the data band-width.
In both scenarios, the main objective is to obtain high
quality images using sub-sampled RF data due to limited Rx
or SC lines.

C. Redundancy of RF data and low-rank Hankel matrix

(c) DAS beamforming
Fig. 1. Imaging flow of the standard B-mode ultrasound imaging. PRF : pulse
repetition frequency, FPS : frame per second.

Recall that B-mode ultrasound measurements are obtained
by point-wise sampling for each scan line. Since the scan line
only changes incrementally, the acquired Rx data along the
detectors does not change rapidly for each scan line. This
implies that some degree of skew redundancy exists in the
Rx-SC coordinate data as shown in Fig. 3.
This type of redundancy can be easily seen as a sparsity in
the Fourier domain as demonstrated in Fig. 3. Therefore, when
the Fourier spectrum of the Rx-SC image is denoted by Fb(ω),

3

neural network.

Fig. 3. Rx-SC coordinate data from depth-Rx-SC data cube and its Fourier
spectrum.

D. Deep Convolutional Framelets
Recently, it was shown that CNN is closely related to
Hankel matrix decomposition [11]. Here, we examine this by
assuming 1-D signal model for simplicity.
Specifically, let f = [f [1], · · · , f [n]] ∈ Rn denotes the
signal, and d refers to the annihilating filter length. Then, for
a given Hankel matrix Hd (f ) ∈ Rn×d , let Φ = [φ1 · · · φn ]
and Φ̃ = [φ̃1 · · · φ̃n ] ∈ Rn×n (resp. Ψ = [ψ1 · · · ψq ] and
Ψ̃ = [ψ̃1 · · · ψ̃q ] ∈ Rd×q ) denote the frame and its dual frame,
respectively, satisfying the frame condition:
Φ̃Φ>

∀ω

.

F ~ K = 0,

.

Hd1 ,d2 (F )V EC(K) = 0,

(3)

where V EC(K) denotes the vectorization operation by stacking the column vectors of the 2-D matrix K and v is a flipped
version of the vector v. Here, Hd1 ,d2 (F ) ∈ Rn1 n2 ×d1 d2 is
the block Hankel matrix for the image F = [f1 , · · · , fn2 ] ∈
Rn1 ×n2 , which is defined under the periodic boundary condition as follows:
Hd1 ,d2 (F ) =






Hd1 (f2 ) · · ·
Hd1 (f3 ) · · ·
..
..
.
.
Hd1 (fn2 ) Hd1 (f1 ) · · ·
Hd1 (f1 )
Hd1 (f2 )
..
.

Hd1 (fd2 )
Hd1 (fd2 +1 )
..
.




,


ψi ψ̃i> = I.

(6)

Then, Yin et al [14] derived the following convolution framelet
representation:
n

f

=

q

1 XX
hf, φi ~ ψj iφ̃i ~ ψ̃j ,
d i=1 j=1

(7)

where {φi ~ ψj } and {φ̃i ~ ψ̃j } constitute the frame bases
and its dual, respectively. They further showed that if Hd (f )
is low-ranked, then for given Φ and Φ̃, the bases Ψ and Ψ̃
can be optimally learned from the data such that the framelet
coefficients hf, φi ~ ψj i can be sparsified. This means that if
a nonlinearity, such as rectified linear unit (ReLU), is added
to the framelet coefficients as a shrinkage constraint, then
the data driven basis Ψ and Ψ̃ will be learned to have the
maximum energy compaction of the framelet coefficients if
the underlying Hankel matrix is low-ranked.
Another important clue that links the CNN to the low-rank
Hankel matrix completion is given by the following matrix
equality:
Hd (f )

(4)

Hd1 (fd2 −1 )

and Hd1 (fi ) ∈ Rn1 ×d1 is a Hankel matrix:

fi [1] fi [2] · · ·
fi [d1 ]
 fi [2] fi [3] · · · fi [d1 + 1]

Hd (fi ) =  .
..
..
..
 ..
.
.
.
fi [n] fi [1] · · · fi [d1 − 1]

=

i=1

(2)

where F ∈ Rn1 ×n2 denotes the discrete 2-D image and
K ∈ Rd1 ×d2 is often referred to as the annihilating filter [13].
Furthermore, this can be represented in a matrix form :



ΨΨ̃>

q
X

(1)

This is equivalent to the convolutional relationship in the RxSC domain:

φ̃i φ>
i = I,

i=1

b
then we can find a function K(ω)
in the spectral domain such
that their multiplication becomes zero:
b
Fb(ω)K(ω)
= 0,

n
X

=

= Φ̃Φ> Hd (F )ΨΨ̃> ,
= Φ̃C Ψ̃> ,

(8)

where
C := Φ> Hd (F )Ψ.

(9)




.


(5)

Eq. (3) implies that the block Hankel matrix constructed
from the RF data in Rx-SC domain is rank-deficient. Furthermore, its rank is determined by sparsity level in the spectral
domain as theoretically proven in [9]. In fact, Jin et al [10] utilized this as well as temporal domain redundancy to interpolate
missing RF data using low-rank Hankel matrix completion.
However, the main limitation of [10] is its computational
complexity, which is prohibitive in real applications. In the
following, we will show that the low rank properties of block
Hankel matrix can be exploited using the deep convolutional

Specifically, one of the most important contributions in [11]
was to reveal that (8) and (9) can be equivalently represented
by the decoder and encoder convolution, i.e:
f

=

(Φ̃C) ~ τ (Ψ̃),

(10)

C

=

Φ> (f ~ Ψ),

(11)

where

Ψ := ψ 1

···


ψ q ∈ Rd×q

,

 
ψ̃1
1 . 
τ (Ψ̃) :=  ..  ∈ Rdq .
d
ψ̃q

The convolution in (11) is the single-input multi-output
(SIMO) convolution, while the convolution in (10) is the multiinput single-output (MISO) convolution. They corresponds to

4

Fig. 4. (a) A single layer encoder-decoder network, and (b) multi-layer
encoder-decoder network.

the convolution operations in CNN [11]. More specifically,
if Φ = I, then the structure in (11) and (10) is in fact
equivalent to one layer of encoder-decoder network as shown
in Fig. 4(a). Moreover, a similar decomposition holds for an
extended Hankel matrix Hd|p (Z) constructed from an input
matrix Z = [z1 · · · zp ] ∈ Rn×p :


Hd|p (Z) = Hd (z1 ) · · · Hd (zp ) ,
(12)

frequencies were 40MHz. The linear transducer is composed
of 64 receivers and 384 scanlines, and the convex array
transducer is composed of 64 receivers and 256 scanlines. We
first acquired RF data from carotid area of nine subjects using
a linear array transducer. The size of each Rx-SC planes is
64 × 384 (# of Rx × # of SC). 15750 Rx-SC planes of the
seven people data sets are randomly selected for generating
training data set, and 2250 Rx-SC planes of the another person
data sets are randomly selected for generating validation data
set. The remaining one person data sets are used as test set.
In addition, we acquired RF data from liver area from one
subject using a convex array transducer. This data set was used
to verify the generalization power of the algorithm. The size of
each Rx-SC planes is 64×256 (# of Rx × # of SC). In order
to use the CNN trained from linear array, we converted the
size of Rx-SC planes of the convex array transducer (64×256)
to those of the linear array (64×384) as shown in Fig. 5. Here,
the SC data around the boundary, i.e. SC 3, 4, 7, 8, · · · , 255,
256, are duplicated.

such that
Hd|p (Z) = Φ̃C Ψ̃>

=⇒ Z = (Φ̃) ~ τ (Ψ̃),

(13)

C = Φ> Hd|p (Z)Ψ

=⇒ C = Φ> (Z ~ Ψ),

(14)

where

and the encoder and decoder filters are defined by

 1
1
ψ1 · · · ψq
 .
dp×q
.. 
..

Ψ := 
.
 ..
. ,∈ R
p
ψ1

τ (Ψ̃)

:=

 1
ψ̃1
1 .
 .
d .
ψ̃q1

···

···
..
.
···

(15)

p
ψq


ψ̃1p
..  . ∈ Rdq×p
. 

Fig. 5. Reshaping of the convex array transducer (64 × 256) to the linear
array transducer (64 × 384) using boundary data duplication.

(16)

ψ̃qp

The convolutions in (13) and (14) correspond to the multiinput multi-output (MIMO) convolution used in CNN [11].
The simple signal expansion using (11), (10), (14) and (13) is
so powerful that a CNN with the encoder-decoder architecture
in Fig. 4(b) emerges from them by inserting the pair (14) and
(13) between the pair (11) and (10).
In short, because of the redundancy in Rx-SC domain data,
the associated Hankel matrix is low-ranked, which allows
convolutional filter learning so that resulting convolutional
framelet coefficients becomes sparse. This implies that a
directly interpolation of Rx-SC domain RF data is feasible
using a deep CNN.
III. M ETHOD
A. Data set
US data in the RF domain were acquired by E-CUBE
12R US system (Alpinion Co., Korea). Real RF data were
acquired by a linear array transducer (L3-12H) with the center
frequency of 8.48MHz and a convex array transducer (SC14H) with the center frequency of 3.2MHz. The sampling

B. RF sub-sampling scheme
As input data, we used randomly sub-sampled data at the
downsampling ratio of 4, 8, or 12. In our proposed method that
assumes a dynamic aperture US system, the RF data are subsampled along the receivers. Since the receiver at the center
always obtains the RF data from direct reflection as shown
in Fig. 1(c), the receiver has important data. So, we alway
include RF data from the center receiver.
In the second experiment, the RF data are sub-sampled
along the scanlines. Specifically, we consider the sub-sampling
in SC-Rx plane. Here, the input data was sub-sampled in two
steps. In the first step, the RF data was uniformly sub-sampled
along the scanlines with the downsampling ratio of 4 and then
in the next step, it was further sub-sampled randomly along
the receivers with the downsampling ratio of 4. The overall
downsampling ratio for this scheme is 4x4=16.
C. Network Architecture
The proposed CNN is composed of convolution layers,
batch normalization layers, ReLU layers and contracting path

5

Fig. 6. Our network architecture for RF-interpolation.

with concatenation as shown in Fig. 6. Specifically, the network consists of 28 convolution layers composed of a batch
normalization and a ReLU except for the last convolution
layer. The first 27 convolution layers use 3×3 kernel, and
the last convolution layer uses 1×1 kernel. Four contracting
path with concatenation exist. The role of contracting pass
in the context of deep convolutional framelets can be found
in [11]. As input data, sub-sampled Rx-SC data were used,
whereas the full data of Rx-SC plane was used as the label
data. If sub-sampling ratio is bigger than 8, additional 3×3
kernel convolution layer, batch normalization layer, and ReLU
layer are insereted to each module for enlarge the receptive
field size.
The network was implemented with the MatConvNet [15]
in MATLAB 2015b environment. For network training, the parameters were estimated by minimizing l2 norm loss function.
And the regularization parameter was 10−4 . The network was
trained by the stochastic gradient descent. The learning rate
started from 10−7 and gradually decreased down to 10−9 . The
weights were initialized using Gaussian random distribution
with Xavier method [16]. The number of epoch was 400 for
all downsampling rates.
D. Baseline algorithms
For comparative studies, our CNN based interpolation results are compared with the linear interpolation and ALOHA
[10]. Specifically, due to the irregular downsampling pattern,
the standard linear interpolation algorithm does not work, so
we use the grid-based interpolation function griddata() in
MATLAB. At high sub-sampling ratio (eg. x12), the Rx-SC
plane interpolation using griddata() is very inferior, so in
this case we use multiple temporal frames together for 3-D
interpolation to improve the performance.
ALOHA-based RF interpolation proposed by Jin et al
[10] further exploits the correlation in temporal direction in
addition to the Rx-SC domain redundancy. Specifically, due
to strong correlation in the temporal direction, the RF data
in Rx-SC domain from adjacent temporal frames shows some
level of redundancy and similar patterns as shown in Fig. 7.
So, they can be explored by ALOHA [17]. Specifically, we
can find the multi-frame annihilating filter relationship [10]:
Fi ~ Kj − Fi ~ Ki = 0,

i 6= j,

(17)

Fig. 7. Spatio-temporal redundancy of Rx-SC coordinate planes. These RxSC data are extracted at the same depth.

where Fi denotes the RF data in the Rx-SC data at the ith frame, and Ki are associated filters. Then, (17) can be
represented in a matrix form :



 V EC(Ki )
Hd1 ,d2 (Fi ) Hd1 ,d2 (Fj )
= 0.
(18)
−V EC(Kj )


so that the extended Hankel matrix Hd1 ,d2 (Fi ) Hd1 ,d2 (Fj )
is rank-deficient. Similarly, we can construct the extended
Hankel matrix using RF data from N -time frames:



Hd1 ,d2 |N {Fi }N
i=1 := Hd1 ,d2 (F1 ) · · · Hd1 ,d2 (FN ) ,
and due to the strong spatio-temporal correlation, the extended
Hankel matrix still has low rank. Accordingly, the RF interpolation problem can be solved using the following low-rank
Hankel matrix completion problem [10]:

min
kHd1 ,d2 |N {Xi }N
(19)
i=1 k∗
{Xi }N
i=1

subject to

PΛi Xi = Mi

where Λi denotes the indices of the measured Rx-SC data Mi ,
PΛi denotes the projection to the index Λi . The optimization
problem (19) can be solved using alternating direction method
of multiplier (ADMM) after the matrix factorization [10].
IV. E XPERIMENTAL R ESULTS
A. Rx sub-sampling experiments
Fig. 8 compare the results of interpolation, ALOHA and
our deep learning method from the Rx sub-sampled data. Our
method significantly outperform the other method by eliminating line and blurring artifacts. Then, the trained CNN from

6

Fig. 8. Linear transducer DAS beamformer reconstruction results from (first column) full RF data, (second column) x12 downsampled RF data, interpolated RF
data using (third column) griddata() in MATLAB, (fourth column) ALOHA, and (last column) the proposed CNN-based interpolation, from x12 sub-sampled
RF data.

Fig. 9. Convex transducer DAS beamformer reconstruction results from (first column) full RF data, (second column) x12 downsampled RF data, interpolated
RF data using (third column) griddata() in MATLAB, (fourth column) ALOHA, and (last column) the proposed CNN-based interpolation, from x12 sub-sampled
RF data.

the linear array transducer was applied to the circular array
transducer, and the DAS beamformer output images are shown
in Fig. 9. Again, excellent image qualities were observed,
which outperformed the existing algorithm significantly. This
confirms the generalization capability of our CNN based RF
interpolation network.
To quantitatively show the advantages of the proposed deep

learning method, we compared with ALOHA and interpolation
methods in terms of the peak-signal-to-noise (PSNR) value
and reconstruction time. Fig. 10 and Fig. 11 show PSNR
and reconstruction time, respectively. The PSNR value of
the proposed deep learning method show 0.7dB improvement
compared to ALOHA. For computational time, ALOHA and
interpolation reconstructed ten frames simultaneously, so we

7

compared reconstruction time for ten frames. Fig. 11 confirmed that the reconstruction time of the proposed deep
learning method was significantly lower than the ALOHA and
interpolation.

7.7% of RF data such that the beamformer removes the
blurring artifacts and generates near artifact-free images even
in complicated structures. The average of PSNR for the
reconstruction B-mode image (x12) is around 26.65dB, which
is about 7dB improvement compared to the B-mode images
from sub-sampled RF data.

Fig. 10. PSNR with respect to each downsampling ratio. Blue line : our
method, Red line : ALOHA-based RF interpolation, Yellow line : MATLAB
griddata() RF interpolation.

Fig. 13. (first column) full RF data, (second column) x12 sub-sampled RF
data, (third column) proposed CNN-based interpolation from x12 sub-sampled
RF data.

Fig. 11. Reconstruction time with respect to each downsampling ratio. Blue
line : our method, Red line : ALOHA-based RF interpolation, Yellow line :
MATLAB griddata() RF interpolation.

Fig. 12 showed the validation curve with respect to each
training epoch. The algorithm stably converges regardless of
the different sub-sampling factor, which proves the robustness
of the algorithm.

Fig. 12. PSNR with respect to each epoch. Blue line : PSNR of learning at
4x down-sampling ratio, Red line : PSNR of learning at 8x down-sampling
ratio, Yellow line : PSNR of learning at 12x down-sampling ratio.

Fig. 13 illustrate representative examples of Rx-SC coordinate data from the linear transducer, and Fig. 14 shows the
corresponding B-mode ultrasound images, when Rx downsampling ratio is 12. The proposed CNN-based interpolation
successfully reconstructed the missing RF data from only

Fig. 14. Linear transducer DAS beamformer images from (first column) full
RF data and (second column) the proposed CNN-based RF interpolation from
x12 sub-sampled RF data.

Additional reconstruction results from the convex array
transducer at x12 Rx sub-sampling are shown in Fig. 15.
While the network was trained using linear array transducers,
the Rx-SC data of linear array transducer and convex array
transducer are similar, so the DAS beamformer provided very
accurate reconstruction results without any line artifacts or
blurring. It is also remarkable that the accurate reconstruction

8

was obtained for the liver region, which was never seen by
the network trained using the linear array transducer data. The
results confirmed the generalization power of the algorithm.

data. It is noteworthy that from only 6.25% RF-data, the
proposed method can efficaciously interpolates the SC-Rx subsampled data. Our CNN-based interpolation method shows
impressive results in the reconstruction of B-mode images and
has the potential to be used for ultrafast or 3-D ultrasound
imaging systems.

Fig. 15. Convex array transducer DAS beamformer images from (first column)
full RF data and (second column) the proposed CNN-based RF interpolation
from x12 sub-sampled RF data.

B. SC line sub-sampling experiments
The B-mode images of the linear transducer, reconstructed
from the actual, SC-Rx sub-sampled, and CNN-based interpolated RF data are compared in Fig. 16. Here, the SC-Rx
sub-sampling factors were 4 and 4, respectively, resulting in
net x16 sub-sampling. Due to the SC sub-sampling, the direct
beamforming results suffered from severe line artifacts. Using
the proposed CNN-based method, the sub-sampled RF-data is
drastically improved. This improvement can easily be noticed
in the reconstruction results. The CNN-based interpolation
eliminates the line artifacts and remarkably recovers the missing information of the image. For the linear array, the average
PSNR value of the CNN-based reconstruction B-mode images
is 20.20dB, which is about 12.73dB improved compared to the
B-mode images from the sub-sampled RF data. To show the
generalization power, the training was performed for the SCRx sub-sampled linear array RF data, and the same network
was used for the interpolation of convex array transducer
RF data. For the convex array transducer data, the DAS
beamformer output images, reconstructed from the actual, subsampled, and CNN-based interpolated RF data are also shown
in Fig. 16. Here, again the excellent results are obtained
for the unseen convex array data. For the convex array, the
average PSNR value of the CNN-based reconstruction Bmode images is 22.82dB, which is about 11.37dB improved
compared to the B-mode images from the sub-sampled RF

Fig. 16. Reconstruction from SC-Rx sub-sampling RF data (4x4 subsampling) for linear and convex array transducers. DAS beamformer reconstruction results from (first row) full data, and (second row) sub-sampled data,
and (third row) the proposed CNN results from sub-sampled data.

V. C ONCLUSIONS
In this paper, we proposed a novel deep learning approach
for accelerated B-mode ultrasound imaging. To exploit the
redundancy in the RF domain, the proposed CNN was applied
to Rx-SC domain. In contrast to the existing approaches
that require hardware changes or computational expensive
algorithms, the proposed method does not need any hardware
change and can be applied to any B-mode ultrasound system
or any transducer. In addition, compared with conventional
reconstruction method, PSNR and reconstruction times were
significantly improved. Therefore, this method can be an
important platform for RF sub-sampled US imaging.
VI. ACKNOWLEDGEMENT
This work is supported by National Research Foundation of
Korea, Grant number NRF-2016R1A2B3008104.
R EFERENCES
[1] B. P. Nelson and K. Chason, “Use of ultrasound by emergency medical
services: a review,” International journal of emergency medicine, vol. 1,
no. 4, pp. 253–259, 2008.

9

[2] G. R. Lockwood, J. R. Talman, and S. S. Brunke, “Real-time 3-D
ultrasound imaging using sparse synthetic aperture beamforming,” IEEE
Trans. Ultrason., Ferroelect., Freq. Control, vol. 45, no. 4, pp. 980–988,
1998.
[3] E. Macé, G. Montaldo, I. Cohen, M. Baulac, M. Fink, and M. Tanter,
“Functional ultrasound imaging of the brain,” Nature methods, vol. 8,
no. 8, pp. 662–664, 2011.
[4] M. Tanter and M. Fink, “Ultrafast imaging in biomedical ultrasound,”
IEEE transactions on ultrasonics, ferroelectrics, and frequency control,
vol. 61, no. 1, pp. 102–119, 2014.
[5] O. Lorintiu, H. Liebgott, M. Alessandrini, O. Bernard, and D. Friboulet,
“Compressed sensing reconstruction of 3d ultrasound data using dictionary learning and line-wise subsampling,” IEEE transactions on medical
imaging, vol. 34, no. 12, pp. 2467–2477, 2015.
[6] H. Liebgott, R. Prost, and D. Friboulet, “Pre-beamformed RF signal
reconstruction in medical ultrasound using compressive sensing,” Ultrasonics, 2012.
[7] N. Wagner, Y. C. Eldar, and Z. Friedman, “Compressed beamforming
in ultrasound imaging,” IEEE Trans. Signal Process., vol. 60, no. 9, pp.
4643–4657, 2012.
[8] C. Quinsac, A. Basarab, D. Kouamé, and J.-M. Grégoire, “3D compressed sensing ultrasound imaging,” in Ultrasonics Symposium (IUS),
2010 IEEE. IEEE, 2010, pp. 363–366.
[9] J. C. Ye, J. M. Kim, K. H. Jin, and K. Lee, “Compressive sampling using
annihilating filter-based low-rank interpolation,” IEEE Transactions on
Information Theory, vol. 63, no. 2, pp. 777–801, Feb. 2017.
[10] K. H. Jin, Y. S. Han, and J. C. Ye, “Compressive dynamic aperture
b-mode ultrasound imaging using annihilating filter-based low-rank interpolation,” in 2016 IEEE 13th International Symposium on Biomedical
Imaging (ISBI). IEEE, 2016, pp. 1009–1012.
[11] J. C. Ye and Y. S. Han, “Deep convolutional framelets: A general deep learning framework for inverse problems,” arXiv preprint
arXiv:1707.00372, 2017.
[12] E. Kang, J. Min, and J. C. Ye, “A deep convolutional neural network
using directional wavelets for low-dose x-ray ct reconstruction,” Medical
Physics, vol. 44, no. 10, 2017.
[13] M. Vetterli, P. Marziliano, and T. Blu, “Sampling signals with finite
rate of innovation,” IEEE Trans. Signal Process., vol. 50, no. 6, pp.
1417–1428, 2002.
[14] R. Yin, T. Gao, Y. M. Lu, and I. Daubechies, “A tale of two bases: Localnonlocal regularization on image patches with convolution framelets,”
SIAM Journal on Imaging Sciences, vol. 10, no. 2, pp. 711–750, 2017.
[15] A. Vedaldi and K. Lenc, “Matconvnet: Convolutional neural networks
for matlab,” in Proceedings of the 23rd ACM international conference
on Multimedia. ACM, 2015, pp. 689–692.
[16] X. Glorot and Y. Bengio, “Understanding the difficulty of training
deep feedforward neural networks,” in Proceedings of the Thirteenth
International Conference on Artificial Intelligence and Statistics, 2010,
pp. 249–256.
[17] K. H. Jin, D. Lee, and J. C. Ye, “A general framework for compressed
sensing and parallel mri using annihilating filter based low-rank hankel
matrix,” IEEE Transactions on Computational Imaging, vol. 2, no. 4,
pp. 480–495, 2016.

