J. Vis. Commun. Image R. 40 (2016) 502–515

Contents lists available at ScienceDirect

J. Vis. Commun. Image R.
journal homepage: www.elsevier.com/locate/jvci

Multi-layer authentication scheme for HEVC video based on embedded
statistics q
Yiqi Tew a, KokSheik Wong a,⇑, Raphael C.-W. Phan b, King Ngi Ngan caDepartment of Computer System and Technology, Faculty of Computer Science and Information Technology, University of Malaya, 50603 Kuala Lumpur, Malaysia
University Multimedia, Malaysiac
The Chinese University of Hong Kong, Hong Kongb

a r t i c l e

i n f oArticle history:
Received 4 January 2016
Revised 5 July 2016
Accepted 21 July 2016
Available online 26 July 2016
Keywords:
Authentication
Multi-layer
Information hiding
Compressed video
HEVC
Statistical propertiesa b s t r a c t
A multi-layer authentication scheme for HEVC compressed video is proposed. The combination of CU
sizes, which is unique to HEVC and sensitive to video manipulation, is considered along with other elements in the HEVC coding standard to generate the authentication tag. Temporal dependency was
enforced, where the authentication tag generated in one slice is embedded into its subsequent slice.
By design, the authentication tag is repeatedly but selectively embedded into various elements in a
HEVC video, including nonzero DCT coefficients, QP parameter values, and prediction modes, dependingon the bit segment in the generated tag. Our scheme offers three layers of authentication to detect andlocalize the tampered regions in a HEVC video, as well as verifying the source/sender of the video using ashared secret key. Video sequences from various classes (resolutions) are considered to verify the performance of the proposed multi-layer authentication scheme. Results show that, at the expense of slightdegradation in perceptual quality, the proposed scheme is robust against several common attacks. A
functional comparison is performed between the proposed multi-layer authentication scheme and theconventional schemes.
Ó 2016 Elsevier Inc. All rights reserved.

1. Introduction
Digital video has become an important part of our daily life
thanks to the widely accepted standardization of video coding formats and their successful deployments in various applications. We
watch movies over the Internet, record video using car DVR (digital
video recorder), establish video conference across heterogeneous
network environments, etc. However, these videos can be easily
manipulated (e.g., trimmed, cropped, re-compressed) due to theavailability of high performance personal computer at affordable
prices and user-friendly yet powerful video editing software [1].
As a result, the integrity of digital video and its origin become
implausible. Hence, a video needs to be authenticated so that its
source can be confirmed to be someone trustworthy and its

qThis paper has been recommended for acceptance by Zicheng Liu.

⇑ Corresponding author.

E-mailaddresses:
koksheik@um.edu.my,(K. Wong).
URL: http://mspih.fsktm.um.edu.my (K. Wong).
http://dx.doi.org/10.1016/j.jvcir.2016.07.017
1047-3203/Ó 2016 Elsevier Inc. All rights reserved.

KokSheik.Wong.MY@ieee.org

content can be verified to be genuine prior to consumption or
broadcasting [2].
Unlike its success in providing entertainment [3], the viabilityof digital video as evidence in the judicial process has been largely
unprecedented. And yet we see an increasing number of videos
from personal cameras or mobile phones being released in social
media corresponding to incidents, e.g., road bullying. Digital evidence is often ruled inadmissible by courts because it is owned
without authentication or its authenticity cannot be verified [4].
According to the guideline stipulated in [5], it is necessary todemonstrate how evidence is authenticated and to show the integrity of each process through which the evidence was obtained. The
evidence should be preserved from any third party who is able torepeat the same process and attain the same result as that presented to the court. Therefore, implementing a secure authentication scheme to confirm the authenticity of viable video evidence isimperative. It is also important to prevent any digital video tailoredfor causing hatred or benefiting a certain party.
When handling compressed video, authentication is commonly
achieved in two ways, namely, video labeling and data embedding[6]. Specifically, in video labeling, the authentication code isappended to the video for integrity verification. This code can be

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

generated by a hash function using a set of features, such as those
extracted or derived from the video, as the input. On the other
hand, for data embedding based approach, the authentication codeis imperceptibly embedded into the video stream rather thanappended to it.
Specifically, Tartary et al. proposed an labeling based authentication scheme for any digital content by utilizing the ReedSolomon code [7]. They demonstrated the design based on the listof recoverable codes in network stream distribution. An cryptographic function is introduced along with digital signature andhash function to ensure the robustness of the designed scheme.
Later, Ren et al. introduced a video labeling based authentication
scheme with loss-tolerant feature [8,9]. They combined a cryptographic fingerprint and video to achieve video authentication.
However, this authentication scheme compromises on minor
latency (i.e., access to the fingerprint) and video file size incrementin the processed video. Baek et al. then designed a labeling based
authentication framework through public key infrastructure forvideo broadcasting network [10]. They introduced an identitybased signature to authenticate online and offline broadcasted
videos, and improved the performance of [11]. With the same
spirit, Song et al. put forward an interactive content based authentication scheme using labeling for video streaming [12]. Their
design generates levels of signature in the chosen video slices, multiple authentication paths as well as authenticating information on
network packets. It is reported that their scheme is of high tolerance against packet loss.
While offering attractive performances, video labeling based
authentication can hardly provide the security feature to authenticate video due to the nature of the code appending process, whichfails to prevent the code from being copied, manipulated or counterfeited. This shortcoming can be overcome by means of data
embedding. Roy et al. realized a video authentication approach inhardware by using field programmable get arrays [13], where theauthentication information is embedded to resist against coverup and cropping segment attacks. An authenticated video under
Roy et al.’s scheme can be easily adapted in common video standards with minor quality degradation, but may not be viable forvideo of higher resolution due to the high computational complexity. On the other hand, Wei et al. proposed an authentication
scheme in the scalable video code streams, where the authentication codes are encapsulated in the network abstraction layer unit
[14]. Their proposed scheme is efficient in detecting contentpreserving manipulation attack (e.g., recompression), but vulnerable to content-changing manipulation (e.g., color or luminance)
attack.
Along with the similar direction, some researchers utilize semifragile watermarking scheme to randomly embed authentication
code into coefficients in spatial domain [15]. These schemes detectand localize tampering (e.g., cropping, insertion) region, but notrealized in temporal domain (i.e., video). Later, Qi et al. presented
a similar scheme by utilizing the singular-value-based andprivate-key-based sequences to generate content-dependentauthentication codes [16]. These codes are embedded through an
adaptive quantization method under discrete wavelet transformation in spacial domain, and it can be easily extended to color
domain. Furthermore, researchers then proposed data embedding
based authentication by utilizing histogram shifting technique inthe spatial domain [17], or manipulating motion vector [18], coefficients [19] and macroblock [20] in the compressed domain (e.g.,
MPEG2, H.264/AVC) [21]. However, these schemes are not implemented in the latest video standard, i.e., HEVC (High Efficiency
Video Coding), which is anticipated to replace H.264/AVC standard
especially when more high resolution (e.g., 4 K) cameras, display
devices and video contents are available.

503

To the best of our knowledge, the conventional schemes (e.g.,[8–11]) authenticate one video slice at a time, which is essentiallythe same as handling sequence of still images. The temporaldependency, on which all video compression techniques exploit,
is not considered. Therefore, in this work, we propose an authentication scheme that exploits the dependency between two consecutive slices, where statistical information in the previous slice isutilized to guide the embedding process in the current slice. We
focus on multi-layer data embedding based authentication for
detecting and localizing tampered regions, as well as verifying
the origin of the HEVC video. By considering the capability of current HEVC decoder, the proposed first and second layers of authentication can be conveniently implemented to verify genuineness ofa video since the operations involved are of low complexity. On theother hand, the third layer of authentication, in which case thesecret key is required, can be deployed to verify the legitimacy ofa video, e.g., relating to the evidence in court case.
In the proposed scheme, we emphasize on the utilization ofvideo content statistics, where all the coding elements in each
video slice are considered to generate the authentication information. During encoding, the video content (i.e., visual information) isvery sensitive to the compression mechanism under the HEVC
standard. Specifically, encoding at different bitrate yields significantly different set of statistics. In particular, the act of manipulating or modifying any video element changes the video statistics
drastically, and hence the sensitivity of these statistics is exploitedfor tamper detection in this work, which is the significant part ofour proposed authentication scheme.
The remainder of this paper is organized as follows: Section 2.1
briefly reviews the coding structure of HEVC. Section 3 puts forward our multi-layer authentication scheme. Section 4 discusses
the performance of the proposed multi-layer authentication
scheme in terms of video quality, robustness and sensitivity when
considering various classes of test video and presents the functional comparison against conventional schemes. Section 5 concludes this paper.

2. Preliminaries
2.1. HEVC standard
HEVC is the latest video coding standard published by ITU-T
VCEG and ISO/IEC MPEG [22]. The main achievement of the HEVC
standard is its significant improvement in compression performance when compared to the previous state-of-the-art standard,i.e., H.264/AVC [23], with at least 50% reduction in bitrate for
producing video of similar perceptual quality [22]. HEVC standard is designed to address essentially all existing applicationsof H.264/AVC. It achieves two additional major achievements,
namely: (a) handle video of higher resolution by introducing larger coding unit sizes, and (b) capitalize on parallel processing
architecture in the video encoder design for suppressing encoding time.
HEVC introduced several novel features to achieve higher
video compression performance, including various coding unit
sizes, more intra prediction modes, residual quad tree, sample
adaptive offset, tiles and wavefront processing [24]. Among thesenovel features, the implementation of variable coding unit sizes,
quantization parameters and prediction modes are exploited inthis paper for authentication purposes by means of data
embedding.
In HEVC encoder, a video is treated as a sequence of pictures,
where these pictures are labeled as I-(intra), P-(predicted) and B(bi-directionally predicted) slices, depending on the order in which

504

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

they appear. Each slice consists of certain number of CTU (coding
tree units), while each CTU consists of some number of CU (coding
units) with size of 64  64; 32  32; 16  16 or 8  8 pixels. Each8  8 CU can be further split into 4  4 in the prediction process.
The availability of CU in various sizes allows the video encoder
to encode each part of the video slice based on its local characteristic (i.e., spatial activity). The encoder decides the CU size and thequantization value in each CTU based on the desired bitrate. In particular, due to the quantization process, a region with high spatialactivity (e.g., water waves) requires smaller CU size to precisely
capture the variation in pixel intensity values. On the other hand,a smooth region (e.g., background or cloudless sky) can be encodedby using larger CU size. In the case of low bitrate (e.g., 10 kbps), a
large quantization value (e.g., QP = 40) is utilized to encode every
CTU with larger CU sizes, which leads to quality degradation andsmaller video file size. On the other hand, for high bitrate (e.g.,
100 Mbps), small quantization value (e.g., QP = 12) is utilized andmost CTUs are coded in smaller CU sizes for representing theregion without compromising on perceptual video quality, but at
the expense of larger video file size.
The prediction and transformation processes utilize the CU
structure to perform intra/inter prediction, DCT (Discrete Cosine
Transformation) and quantization. The CU utilized in the prediction and transformation processes are called PU (prediction unit)
and TU (transform unit), respectively. Specifically, in I-slices, CU
can only be coded by using squares, which include
64  64; 32  32; . . . ; 4  4 pixels. On the other hand, in P and Bslices, CU can be encoded by using all possible arrangements,
including 2N  2N , 2N  N , N  2N , N  N for N 2 4; 8; 16; 32
and AMP (Asymmetry Motion Partition), which can assume thedimension of 2N  nU; 2N  nD; 2N  nL, or 2N  nR. The implementation of AMP in HEVC provides better prediction reference
and less bitstream size overhead for PU that contains slight movement at either the upper, lower, left or right part of a CU in P- or Bslice. Here, each CU is encoded with a depth value d, to indicate the
N value in CU size definition. The depth value, d 2 f0; 1; 2; 3g signifies that N ¼ 64=2d (e.g., CU of size ð64=2d Þ  ð64=2d Þ ¼ 32  32 areconsidered for d ¼ 1), except for d ¼ 3 where both 8  8 and 4  4blocks are included.
Each PU defines a slice region that shares the same prediction
mode (i.e., intra, inter, skip and merge) [25]. Intra prediction
defined in HEVC allows 33 angular prediction directions (i.e.,modes) and two non-angular modes, namely DC and planar. Current PU’s intra prediction is obtained through the extrapolationof values derived from the reference pixels of the neighboring
PU’s, a process which requires numerous arithmetic operations
per predicted pixel value. On the other hand, inter prediction
encodes PU by storing the motion vector, which points to the position of the matching PU in the reference slice, as well as the residual values, which are the differences (prediction errors) between
the reference PU and current PU.

3. Authentication scheme
Our proposed scheme aims to detect and localize the tampered
region(s) in a HEVC compressed video by means of data embedding
and exploiting the dependency in the temporal axis. Specifically,
the authentication code (hereinafter referred to as tag) is generated
and embedded into the video. Fig. 1 shows the process flow of the
proposed multi-layer authentication scheme, which consists of thefollowing four steps: Tag Generation, Tag Implantation, Tag Alteration and Tag Verification. In tag generation, the extracted featuresfrom the video and the secret key are combined then fed into ahash function as detailed in Section 3.1. The tag implantation

Fig. 1. Proposed multi-layer authentication scheme.

Fig. 2. Tag generation.

process using data embedding technology is detailed in Section 3.2.
Tag alteration by means of masking or skipping, as well as theembedding schemes is described in Section 3.3. To validate a video,
the tag is verified in three layers of authentication as detailed in
Section 3.4 (see Fig. 2).
3.1. Tag generation
In video authentication, a generated tag must be unique as well
as sensitive to its input, and its genuineness must be verifiable byanyone who has the secret key. To fulfill these requirements, we
exploit the unique statistical features of the video content and ahash function (e.g., SHA2) to generate the tag, which is in turnembedded into the video.
3.1.1. Feature extraction
Several video features are considered to serve as the input for
tag generation. These features, including the size types, depths
and modes in every CU, as well as non-zero DCT coefficient values,
are extracted from each CTU in every video slice. Recall from Section 2.1 that in every video slice, HEVC divides each CTU into somecombination of CUs in different sizes. To facilitate the discussion,let
cm 2 f2N  2N; 2N  N; N  2N; N  N; 2N  nU; 2N  nD; nL
2N; nR  2Ng refer the category of CU size, dm 2 f0; 1; 2; 3g refers
to the depth of quad tree decomposition, and pm 2 {intra, inter}
refers to the prediction mode in the m-th CTU, wherem 2 f1; 2; . . . ; Mg for

M ¼ dðwidth=64Þe  dðheight=64Þe:

ð1Þ

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

505

Fig. 3. Illustration of feature extraction.
Fig. 4. Mapping rules for data embedding using CU size.

The frequency of occurrences for cm ; dm and pm in the mth CTU arecomputed and referred to as Fðcm Þ; Fðdm Þ and Fðpm Þ, respectively.
In this work, the number of 4  4 pixel blocks is considered, i.e.,the block dimension divided by 16. Suppose the 3-rd CTU is being
processed (i.e., m ¼ 3). Given one CU of size 32  32, when d3 ¼ 1,
thecorresponding
frequenciesofoccurrencesareFðc3 ¼ 2N  2NÞ ¼ Fðd3 ¼ 1Þ ¼ 64 since there are exactly 64 unitsof 4  4 pixel block within it. Similarly, for a CU of size 16  16with intra mode, the frequencies of occurrence Fðp3 ¼ intraÞ ¼ 16
since there are exactly 16 units of 4  4 pixels within it. For further
illustration, frequencies of occurrences for Fðcm Þ; Fðdm Þ and Fðpm Þare calculated based on the example given in Fig. 3. Here,Fðcm ¼ 2N  2NÞ ¼ 216 since there are 1 CU of size 32  32 (i.e.,
64 units of 4  4), 7 CUs of size 16  16 (i.e., 112 units of 4  4)
and 10 CUs of size 8  8 (i.e., 40 units of 4  4). On the other hand,Fðpm ¼ intraÞ ¼ 104 since there are 104 units of 4  4 block coded
in intra mode while Fðpm ¼ interÞ ¼ 152 because there are 152
units of 4  4 block coded in inter mode.
1
2
For features extraction, let cmax
and cmax
be the two most fremmquently occurring CU categories in the mth CTU. The difference infrequency of occurrences between them, denoted by CðmÞ, is com1
2
1
2
Þ  Fðcmax
Þ. Similarly, let dmax
and dmaxbeputed as CðmÞ ¼ Fðcmax
mm
m
m
the two most frequently occurring depths in the m-th CTU, and thedifference in frequency, denoted by DðmÞ, is computed as11DðmÞ ¼ Fðdmax
Þ  Fðdmax
Þ. Similarly, for prediction mode, the difmmference between frequency of using intra and inter in the m-thPðmÞ,iscomputedasPðmÞ ¼
CTU,
denotedbyjFðp ¼ intra;mÞ  Fðp ¼ inter;mÞj. In addition, in each CTU, thecount of non-zero DCT coefficient cnzðmÞ, the sum of absolutevalue of non-zero DCT coefficient sav ðmÞ, and the difference
between the frequency of occurrences for positive and negative
signs sðmÞ are computed. Note that these entities highly sensitive
to re-compression and only available in the HEVC standard (i.e.,
cm and dm ), which will change drastically when encoded in different bitrate or when different content is encoded.
3.1.2. Secret key
The (secret) key K with a specific length is required to verify theorigin (i.e., sender) of a video. This key must be owned by both parties (i.e., sender and receiver) to generate the same tag for verification purpose. Note that the secret key is not revealed during
verification because only the generated tag is compared against
the embedded tag. In case the origin of the video need not be verified, the secret key can be conveniently replaced by any value such
as DCT coefficient values and motion vectors.
3.1.3. Sensitive function
A sensitive function (e.g., hash function, pseudo-random number generator) is required to generate a unique tag from theextracted features and shared secret key K. The tag generated bythis function should differ significantly even when the inputs(e.g., statistics of video) are similar but not identical. It should bepractically impossible to analyze this tag for inverting the mapping

process, that is, to obtain the input value from the tag. In this work,a cryptographic hash function H, namely, SHA2 [26], is utilized tomeet the aforementioned requirements. For each video slice,
the extracted feature values, viz., CðmÞ; DðmÞ; PðmÞ; cnzðmÞ;sav ðmÞ; sðmÞ for all CTUs (M in total) as well as the shared secret
key K are concatenated to form the input for the hash function H
for generating the tag w. The hash function H, input and output
w are related as expressed in Eq. (2), where hjjf concatenates hand f together. Note that the length of the tag x depends on theapplied hash function and in our case, the output tag is 32 bytes
since SHA256 [26] is considered.

M
MM
w ¼ H ðCðmÞÞM
m¼1 jjðDðmÞÞm¼1 jjðPðmÞÞm¼1 jjðcnzðmÞÞm¼1 jj

M ðsav ðmÞÞM
m¼1 jjðsðmÞÞm¼1 jjK

ð2Þ

3.2. Tag implantation
Four data embedding techniques are deployed to achieve highimperceptibility for embedding tag into the HEVC compressedvideo. These techniques utilize different elements in the HEVC
encoded video, including the CU type, non-zero DCT coefficient,quantization parameter, prediction type and motion vector, toselectively and repeatedly embed the generated tag.
3.2.1. Coding unit size
In HEVC encoder, the RDO (rate distortion optimizer) decides
the CU sizes to achieve the best compression ratio based on thedesired bitrate. In our proposed multi-layer authentication
scheme, instead of using the size determined by RDO, we force
the size of CUs in slice Snþ1 to embed the tag w, which is computedfrom the previous slice Sn based on a predefined mapping rule. An
example of the mapping rule is shown in Fig. 4 [27].
The CU sizes are divided into two categories, where one encodes‘0’ and the other encodes ‘1’. In particular, category ‘0’ includes
2N  N; 2N  nU; nL  2N, and N  N pixels, while category ‘1’
includes N  2N; 2N  nD; nR  2N, and 2N  2N. In other words,
the CU size in Snþ1 can be N  2N; 2N  nD; nR  2N, or 2N  2N
for wl ¼ 1 where l ¼ 1; 2; . . . ; 256, and vice versa, as depicted inFig. 4. For instance, if the CU size decided by RDO is 16  8 andwl ¼ 1, then our proposed scheme will force the RDO to recalculate
the required bitrate (i.e., cost) for 8  16; 16  16, and two AMP’s(i.e., 2N  nD; nR  2N), then choose the CU size that results inthe lowest cost. For CU with larger size (e.g., 32  32), it is justifiable to encode it by using some combination of blocks with smaller
sizes (e.g., two 32  16, four 16  16, etc.), because in this case, asmooth block is merely decomposed into combination of smaller
blocks, which are conventionally considered for encoding regionof higher spatial activity. The tag w is sequentially and repeatedly
embedded into all CUs, following the order from top-left tobottom-right (i.e., Z-scanning) as stipulated in the HEVC standard[22]. This approach maintains the video quality at the expense ofslight increment in bitstream size. It should be noted that, in this506

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

Fig. 5. Modified LSB of the non-zero DCT coefficient.

work, smaller blocks are not combined into a larger block to maintain the video quality at the expense of slight file size increment.
3.2.2. Non-zero DCT coefficient
Here, we utilize LSB of non-zero DCT coefficients to embed the
tag without causing significant quality degradation. To minimize
distortion, we choose the last non-zero DCT coefficient (withrespect to the scanning order in use) of each CU in every CTU, asshown in Fig. 5. The selected DCT coefficient c is modified to aneven integer for embedding wl ¼ 0, and vice versa.
3.2.3. QP (Quantization Parameter)
During encoding, RDO utilizes the QP value to achieve thedesired bitrate. In order words, it determines the quality of video,
where smaller QP value leads to higher video quality, and viceversa. HEVC encodes each CTU with different QP value based on
the predefined QP value range as stipulated in the configuration
file. Here, we force the QP of each CTU to embed wl by modifying
the offset range during the encoding process. The RDO calculates
the cost of each CTU (i.e., total bit requires to code the CTU) basedon the QP value with the selected offset. Odd QP values will be utilized in the calculation when embedding wl ¼ 1, and vice versa.
Fig. 6 shows the possible QP values for embedding
w ¼ 11111000   .
3.2.4. Prediction type
Video compression is closely tied with the implementation ofvarious prediction methods, which can be coarsely divided into
two approaches: prediction within the video slice itself (intra)
and among few neighboring slices (inter). We exploit these two

Fig. 7. Intra and inter prediction mode decision in a slice.

Fig. 8. Tag implantation and alteration process.

approaches of prediction to represent the tag wl 2 f1; 0g. Again,
RDO is set to consider only the CTU cost for all 34 types of intra
prediction (see Section 2.1) while ignoring those for inter prediction when wl ¼ 0. On the other hand, only the costs for inter prediction are considered when wl ¼ 1. Fig. 7 shows the selected CU
to embed wl 2 f0; 1g in inter prediction mode withMV ¼ ð10; 3Þ using RedIdx (i.e., reference slice index) = 1, andintra prediction mode using mode 23 and 24.
3.3. Tag alteration

Fig. 6. Quantization value for each CTU in a slice.

In our proposed multi-layer authentication scheme, the firstslice S1 of the video is utilized for generating the tag w. This tag
is conveyed to the next slice S2 via two embedding steps. The firststep utilizes the CU size embedding technique detailed in Section 3.2.1 to embed the tag by modifying the CU size in Snþ1 . The
second step embeds the tag by using three other embedding techniques by considering the bit segment in each byte of the tag. Fig. 8

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

507

Algorithm 1. Pseudo-code for tag generation, implantation andalteration

Fig. 9. Bit segment of every byte in tag.

Table 1
Syntax of bit pattern in every byte of tag.
Bits

a1 : embedding techniquea2 : mode

a3 : value

00
01
10
11

No embedding
Coefficient
QP
Prediction mode

Skipping
Adding
–
–

0
1
2
3

shows the aforementioned embedding steps. The purposes of
having two embedding processes are to: (a) enable a quick
way to check the authenticity of a given video, and (b) localize
the tampered regions, with precision up to the CU size. Notethat these processes can be performed without using the secret
key K.
The tag is divided into non-overlapping segments where eachsegment is processed and embedded one at a time. As an illustration, Fig. 9 shows an 8-bit segment of the tag, which will be processed by the second embedding process. Specifically, the second
embedding process determines the technique to be applied (forembedding), the skipping of positions, and the manipulation on(i.e., masking) the tag itself. These processes are included to complicate the act of mimicking.

3.3.1. Selection
Given a segment, the first two bits (denoted by a1 ) determine
the embedding technique to deploy. All four possible combinations
are listed in Table 1. Specifically, when a1 ¼ 01; a4 is embedded
into the last non-zero DCT coefficient of the next three CUs in
the next CTU(s). In the case of a1 ¼ 10; a4 is embedded into thequantization parameter of the next three CTUs. For a1 ¼ 11,
prediction mode for the next three CUs are utilized to encode a4 .
For a1 ¼ 00, no embedding takes place.

3.3.2. Manipulation
We complicate the embedding process to discourage mimicking of the tag by skipping selected embedding locations (synchronization) or adding the value a3 to the bit segment a4 (masking)prior to actual data embedding. For instance, once the type of
embedding technique (venue) has been decided (signaled by a1 ),when a2 ¼ 1; a3 stipulates how many of the eligible embedding
venues (of the decided type) will be skipped before a4 is embedded. For example, when ða1 ; a2 ; a3 Þ ¼ ð01; 0; 10Þ, the following 2
non-zero coefficients will be skipped, and a4 will be embeddedby using the last nonzero coefficients in the 3rd, 4th, and 5th
non-zero coefficients. On the other hand, when a2 ¼ 1; a4 is added
to a3 before being embedded into the selected location. Due to the
problem of overflow, modða4 þ a3 ; 2Þ will be embedded. The
embedding process continues until all bit segments in the tag w
are processed.
The pseudo-code of our proposed multi-layer authentication
scheme is presented as Algorithm 1, which includes the tag generation, implantation and alteration processes in the HEVC encoder.
The tag is generated in the n-th slice, i.e., Sn , and embedded intoSnþ1 .

3.4. Tag verification
The embedded tag is verified during video decoding. Threelayers of authentication are achieved in the proposed multi-layer
authentication scheme, namely: the conveniently applicable layer
without the need of the secret key (first layer); the dedicatedlayer to localize tampered region (second layer), and; the sophisticated layer which extracts the video features for computation of
the hash value (third layer). Algorithm 2 shows the extraction
and verification of tag during decoding. Here, v 1 ; v 2 and v 3
show the first, second and third layer authentication statuses,

508

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

respectively. Status 1 indicates the video is authenticated, and status 0 indicates a failed authentication in that particular slice.
3.4.1. First layer of authentication
Recall that the tag is sequentially embedded by using the size of
all CUs in each slice. This tag is repeatedly embedded until all CUs
within the same video slice are exhausted. Therefore, the first layerof authentication checks for uniformity of the embedded tag
throughout the slice under consideration. The embedded tag canbe extracted during the decoding process by examining the CU size
based on Fig. 4. The tags are extracted following the Z-scanningorder in every CTU in a slice [22]. The first instance of the tag(32 bytes in length) is extracted and stored as w0 , while the following instances within the same slice are stored as wi for i ¼ 1; 2; . . ..
at any bit location l, then the video is terIf 9i such that wil – wiþ1l
med tampered. Specifically, the group of CTUs encoding the
instance of the tag that differs from the majority are marked as
for all i
the tampered group. On the other hand, when wil ¼ wiþ1land all l, the video is authenticated with respect to the first layer.
3.4.2. Second layer authentication
Since the first layer of authentication depends only on the CU
sizes, it is possible that some elements such as coefficients and
QP are tampered, while maintaining the CU sizes. Therefore, thesecond layer is invoked to further verify the video at the bytelevel of the extracted tag. Specifically, the last non-zero DCT coefficient in a CU, the CU prediction mode or the quantization parameter in the previous slice is considered, depending on the value a1
extracted from the tag. Then, a2 ; a3 and a4 are also obtained fromthe extracted byte segment. Next, the derived a4 from the previous
slice is compared with the embedded a04 based on the embedding
technique as stipulated by a1 . When a04 – a4 , it implies that tampering occurs at the region(s) under investigation. Note that whena1 ¼ 0, no verification is performed because the tag is not embedded into any coefficient, QP or prediction mode, for that particularbyte segment of the tag.
3.4.3. Third layer authentication
When a video passes the first and second layers of authentication, the video is merely verified to be neither modified nor tampered, but its source (i.e., sender) is not verified. To verify thevideo source, the same secret key K (supplied during the encoding
process) is required (see Section 3.1.2). Specifically, the values
CðmÞ; DðmÞ; PðmÞ; cnzðmÞ; sav ðmÞ and smðmÞ in Sn are computed
to generate w (see Eq. (2)). Next, the source of the video can be verified by comparing the generated tag w against the embedded tag
wi . In case the tags match (i.e., w ¼ wi ), the video is authenticatedto be originating from a known (reliable) source, otherwise thesource cannot be verified and hence the video cannot be trusted.

Table 2
Results of embedding tags in various classes of standard test video.
Class (Test Video)

QP

Random accessOriginal

Low delay P
Processed

Low delay B

Original

Processed

Original

Processed

PSNR

SSIM

PSNR

SSIM

PSNR

SSIM

PSNR

SSIM

PSNR

SSIM

PSNR

SSIM

Class A (PeopleOnStreet)

8
16
24
32
40
48

50.526444.081338.942734.2105
29.7969
25.3706

0.99990.99970.99850.99410.97990.939950.390744.0174
38.8355
34.072129.6221
25.2184

0.99990.99970.99850.99400.97950.938851.703145.0852
39.3133
32.2255
29.7152
25.05220.99990.99980.9986
0.99360.9786
0.933451.601744.994239.241734.1066
29.5799
24.93150.99990.99980.9986
0.99360.9786
0.934551.624645.122339.462234.3092
29.769425.0255

0.99990.99980.9987
0.99360.97830.931751.503945.0277
39.3945
34.2095
29.660924.89760.99990.99980.9987
0.99350.97830.9318Class B (Tennis)

8
16
24
32
40
48

49.8782
44.2705
41.731738.2451
34.521230.38750.9996
0.99790.99390.97990.94540.868949.781344.2204
41.640438.1230
34.4460
30.48800.9996
0.99780.99370.9791
0.94380.870551.274144.5771
41.8567
38.3561
34.6043
30.54350.99970.9982
0.99420.98040.96710.870751.1930
44.532541.7730
38.231636.368130.68790.99970.9982
0.99400.97980.96600.872051.2695
44.6120
41.991138.5466
34.8057
30.63130.99970.9982
0.99440.98130.94890.873351.181644.5604
41.9232
38.461234.780530.80770.99970.9982
0.99430.98080.94840.8766Class C (PartyScene)

8
16
24
32
40
48

50.1873
42.420936.869331.6608
27.0375
22.97070.9977
0.99250.97990.94950.88980.761950.0968
42.3561
36.761031.489626.8283
22.8111

0.9977
0.9924
0.97950.94830.88600.754951.703743.2778
36.619631.0041
26.0257
22.2403

0.99830.99290.9781
0.94280.87280.740251.6155
43.223636.5374
30.9262
25.9443
22.2189

0.99830.99280.97780.94200.87030.738551.640343.280736.784231.051326.0159
22.1968

0.99830.99300.9785
0.94310.87140.737151.5349
43.223636.7004
30.9694
25.9227
22.2086

0.99830.99300.9782
0.94220.86910.7364

Class D (BasketballPass)

8
16
24
32
40
48

50.3374
44.9885
39.1638
33.588629.2736
25.56220.99540.9876
0.96290.8986
0.79430.681650.2384
44.883939.0220
33.4273
29.1272
25.5205

0.99520.98730.96160.89540.78890.678950.811745.0952
39.230233.4390
28.901225.0662

0.9956
0.98680.96000.88940.77850.664250.738045.029739.1576
33.3368
28.8067
25.0752

0.99550.9866
0.95940.88740.77650.663950.9240
45.2152
39.3147
33.494328.9669
25.0087

0.99580.98720.96040.88990.7804
0.666650.8318
45.1418
39.2333
33.3970
28.8853
25.1259

0.99570.98700.95970.88810.77810.6683

Class E (FourPeople)

8
16
24
32
40
48

50.0288
44.6663
41.9733
38.312133.6861
28.81210.9994
0.9974
0.9956
0.9896
0.96790.898949.9057
44.5989
41.880738.1473
33.4551
28.5426

0.9994
0.9974
0.99550.98900.96550.891451.210245.0461
41.5085
37.5055
32.7283
27.93250.9996
0.99770.99510.98730.95970.877251.102344.9736
41.450337.464232.701027.9864

0.9996
0.99770.99510.98730.95990.878851.149845.1308
41.6532
37.570432.743627.95770.9996
0.99770.99510.98730.95940.877451.019145.0451
41.5962
37.543932.749728.03150.9996
0.99770.99510.98730.95960.8801

Class F (ChinaSpeed)

8
16
24
32
40
48

52.5190
47.1133
41.122235.074829.9592
25.63140.9996
0.99830.99250.97170.93620.869052.3205
46.9471
40.861134.7306
29.6215
25.17120.9996
0.99830.99220.97080.93390.862553.084147.5657
41.4770
35.3295
29.8973
25.12900.9996
0.9986
0.99380.9736
0.93220.847852.9650
47.4487
41.354235.1068
29.729425.1002

0.9996
0.99850.9936
0.97290.9306
0.848153.1060
47.6250
41.5332
35.3495
29.926425.13770.9996
0.9986
0.99380.97390.93350.850952.971647.4983
41.409035.1260
29.761325.1218

0.9996
0.99850.9937
0.97320.93280.8524

509

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

Algorithm 2. Pseudo-code for tag verification, wi

41.0 dB, respectively). In other words, the processed video
requires extra 5 kbps (e.g., at 41.5 dB, the original and processed
bitrates are 40 and 45 kbps, respectively) to achieve the same
quality as the original (compressed) video. Similar performances
are observed in Class B (drop by 0.25 dB), Class C (drop by1.5 dB), Class D (drop by 1.0 dB), Class E (drop by 0.25 dB)
and Class F (drop by 1.2 dB).
Fig. 12 shows the SSIM results for all classes of video sequence
under the RA, LDP and LDB configurations. Note that only the SSIM
difference between the original and processed videos at lower
bitrates (i.e., <20 kbps in Class A, <6 kbps in Class B, <50 kbps inClass C, <8 kbps in Class D, <5 kbps in Class E and <10 kbps in ClassF) are reported here. It is because the processed video achieves
similar perceptual quality (i.e., similar SSIM value) as that of theoriginal video when encoding at higher bitrate, where the SSIM
values exceed the boundary of the graphs shown in Fig. 12. Basedon the numerical results recorded in Table 2, it is observed that the
SSIM values of the original and processed videos for small QP values (e.g., QP < 16) are similar. This conclusion can also be drawn
from Fig. 12 where the graphs of SSIM for both the original andprocessed videos converge as bitrate increases.
Overall, the quality of the processed videos degrade by <1% interms of both SSIM and PSNR when compared to their originalcompressed counterparts. Perceptually, both original and processed videos appear to be identical by visual inspection. As a representative example, the 8-th slice from the original and processedvideo of Class D are shown in Fig. 10, which appear to be identical.
4.2. Robustness against forgery
In this section, the robustness of the proposed multi-layer
authentication scheme is verified by considering the following
attacks: slice tampering, CTU replacement, generic, VQ and slice
re-compression attack.

4. Analysis
The HM10.0 reference software model is modified to implement
the proposed multi-layer authentication scheme. Video in class A(2560  1600), B (1920  1080), C (832  480), D (416  240), E(1280  720) and F (1024  768) are utilized as our test video
sequences. Three configurations, namely, RA (random access),
LDP (low delay P) and LDB (low delay B), consisting of P/B slices
are selected to collect results using QP in the range of ½8; 48. The
results are recorded in Table 2.

4.1. Video quality
The results in Table 2 indicate that both the original and processed videos exhibit similar and steady growth in image quality
when QP decreases. For video encoded with small QP (such as
those in the range of ½8; 48), the degradation in quality withrespect to SSIM (structural similarity) index [28] is hardly noticeable in all video classes. However, in terms of PSNR, the video quality drops, on average, <1 dB for QP in the range of ½8; 48 for all
video classes considered.
To further examine the results, Fig. 11 shows the rate distortion
curve for video sequences in Class A, B, C, D, E and F under RA, LDP
and LDB configurations. Each graph is featured with a magnified
region to show the detailed PSNR vs Bitrate performance between
the original and processed video. In Class A, quality of the processed video drops 0.5 dB when considering the same bitrate(e.g., at 45 kbps, original and processed videos yield 41.5 and4.2.1. Slice tampering attack
During video transmission, video content are transmitted sliceby slice. If any slice (e.g., Sn ) is accidentally dropped or intentionally removed, then the following slice (i.e., Snþ1 ) will be authenticated in first and second layers, but not the third layer due to thedependency between two consecutive slices, where features fromSn are required to generate the tag for verification in Snþ1 .
For tampering across slices such as slice shuffling (reordering),
insertion (see Fig. 13) and removal (see Fig. 14), the tampered slice
can be detected due to the dependency between adjacent slices aspart of our authentication design (see Section 8). For instance, if nslices are removed and inserted at any other position, the positionsof the removed slice as well as the starting and ending of theinserted slices can be detected. More precisely, as detailed in Section 8, the tag generated by using the features of Sn is embedded in
the subsequent slice Snþ1 . Therefore, by checking the tag in the
non-tampered slice immediately after the attacked slice, the actof removal or insertion of slice can be detected. For further investigation, we tamper the processed video by removing one video

Original (compressed) video

Processed video

Fig. 10. Illustration of the 8th slice of the test video - BasketballPass.

510

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515(a) Class A(b) Class B(c) Class C(d) Class D(e) Class E(f) Class F

Fig. 11. PSNR vs Bitrate performance for original and processed video for various test video sequences.

slice. Then, we verify the authenticity of the manipulated video bychecking the embedded tag. Fig. 15 shows the tag generated in thepervious slice (denoted by wig ðn  1Þ) as well as the tag extracted
from the current slice (denoted by wie ðnÞ) in the processed video.
The illustration shown in Fig. 15 is based on the video BasketballPass in Class D as the representative example. In the case ofno tampering, the generated tag wig ðn  1Þ in the previous sliceSn1 is always identical to the extracted tag wie ðnÞ from the current
slice Sn for i ¼ 1; 2; . . . as illustrated in Fig. 15(a). Due to space limitation, only part of the tag for i ¼ 0 (i.e., the first copy of the generated and extracted tags) are shown. In case of tampering, Fig. 15(b) illustrates an example of this attacking scenario, where the slice
Snþ2 is removed. Since wig ðn þ 1Þ completely disagrees withwie ðn þ 2Þ (extracted from S0nþ2 , which is Snþ3 of the original video),
the act of tampering is detected in S0nþ2 of the video under consideration. The same verification process is viable regardless of thenumber of slices being copied, moved, inserted, or removed. Similar results are attained when other classes of processed video aretampered, and we omit the presentation here.

4.2.2. CTU replacement attack
In this section, we consider the attacking scenario where the CU
structure is tampered. Fig. 15(c) illustrates an example where Snþ2
is tampered by means of swapping the CU structures. Specifically,
the CU structure of the 3rd CTU is swapped with the 4th CTU,
which causes part of the extracted tag wie ðn þ 2Þ to disagree withthe generated tag wig ðn þ 1Þ, namely, 20, 50, . . ., and F9 as illustrated in Fig. 15(c). Since the CU structure in Snþ2 is tampered,
the tag wie ðn þ 3Þ extracted from Snþ3 disagrees with the tag
wig ðn þ 2Þ generated in Snþ2 . This mismatch will propagate due tothe temporal dependency introduced in the design of the proposed
authentication scheme. Note that this type of tampering can also
be detected by checking the equality condition wie ðnÞ ¼ wej ðnÞ,where any mismatch for any value of i and j renders the video as
tampered. Similar results are attained when other classes of processed video are tampered, and we omit the presentation here.
On the other hand, by replacing one of the CTU contents by thatof any other CTU with the same CU structure, the tampered slice
will still be authenticated by the first layer. It is possible to change

511

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515(a) Class A(b) Class B(c) Class C(d) Class D(e) Class E(f) Class F

Fig. 12. SSIM vs Bitrate performance for the original and processed videos for various test video sequences.

ing on a1 . If a04 – a4 , the mismatched CTU can be identified and utilized to pinpoint the modified CTU.

Fig. 13. Tampering detected when a slice is removed.

Fig. 14. Tampering detected when a slice is inserted.

the content of a CTU (e.g., coefficient value) while maintaining its
size, which represents one bit of the embedded tag. However, inthe second layer of authentication, the replaced CTU content will
be examined by extracting the embedded information (viz., a04 )from the last non-zero coefficient, QP or prediction mode, depend-

4.2.3. Generic attack
Lo et al. detailed a generic attack on tagged video stream by
exploiting the coefficients of CU [29]. According to Lo et al., data
embedding in LSB of coefficients, sign of coefficients and count ofzero/non-zero coefficients are potentially attacked by changing
the coefficients that are not involved in the authentication process
so that the modified/tampered video will be authenticated at the
decoder. However, this modification is infeasible under our proposed authentication scheme due to the unpredictable location ofcoefficients utilized for tag embedding. Recall that the tag isrepeatedly embedded into selected nonzero coefficients, where
nonzero coefficients are skipped in a non-regular manner as illustrated in Fig. 9 and Table 1. Hence, any discrepancies among multiple copies of the embedded information will be detected by thesecond layer of authentication in the proposed scheme (seeSection 3.4.2).
In addition, due to the large number of possible combinations of
CUs as well as other considered entities in HEVC video (including
nonzero coefficients count and sign), although in theory othervideo content may have the same coarse features but it will be
unlikely that these visually different contents (but producing thesame authentication code) would be perceptually meaningful or
having unnoticeable visual distortion. In other words, the distortion caused by tampering would be obvious to the naked eyes
and the distortion may further propagate to other future slices
due to motion compensation. Therefore, generic attack [29] isinfeasible in attacking the proposed authentication scheme.
4.2.4. VQ attack
VQ (Vector Quantization) is a technique designed to retrieve theembedded information based on a constructed codebook obtained
from a learning process using a huge quantity of authenticated512

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515(a) Processed video(b) Tampered processed video 1(c) Tampered video 2
Fig. 15. Slice tampering detection based on tag verification.

videos with the same embedded tag. When a VQ style attack [30] is
attempted, the proposed multi-layer authentication scheme is able
to localize the attacked area. It is because our scheme requires theexact sequence of CU sizes to generate and match the tag wi .
Specifically, the tag wi of length 32 bytes may be copied fromone part of the slice and pasted onto another part in the same slice(similar to copy-move attack in image forgery). Considering theHEVC encoding structure, the large number of possible combinations of CU sizes in any CTU (i.e., > 2256 ) suggests that this attack
is practically infeasible. In other words, it may be possible to fabricate a perceptually meaningless (i.e., noise-like) video to deceive
the proposed multi-layer authentication scheme, but the fabri-cated video can be easily identified by visual inspection or nonreference image quality assessment [31]. Hence, the proposed
scheme is robust against VQ style attack.
4.2.5. Slice re-compression attack
The proposed multi-layer authentication scheme is sensitive
against modification. For instance, any modification in the video
content (e.g., pixel values, DCT coefficients) causes cnzðmÞ; sav ðmÞand sðmÞ to change, where the third layer authentication will fail(see Section 3.4) since the correct tag w cannot be reproduced. During (re-) compression, the video encoder decides the CU structureof each video slice to achieve the desired bitrate. In the nth video

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515

513

Fig. 16. Graph of CðmÞ vs m (CTU index) in slice 1 and 2 for Class B compressed with QP = 12 and 24.

QP 12, 145 CUs

QP 32, 106 CUs

Fig. 17. Re-compression result of 4 CTUs with QP = 12 and QP = 32.

slice, cm ; dm and dm are subjected to the CU structure decision when
encoding at a targeted bitrate. The tag generated from Sn is embedded into Snþ1 , which causes the CU structure in Snþ1 (and hence thestatistics) to change. Therefore, the embedded tags in the processed video are sensitive against re-compression (re-encoding)at different bitrates or different QP values. For example, Fig. 17(a) and (b) shows the same slice compressed with QP = 12 and32, respectively. It is apparent that the CTU sizes are different,and hence the same tag cannot be regenerated for authentication
purpose. For further illustration, Fig. 16 shows CðmÞ, which represents the difference in number of occurrences for two most frequently occurring CU categories in each CTU. Here, CðmÞ forvideo compressed with QP = 12 and 24 are shown. The x-axis is
the CTU index throughout the entire test video sequence while
the y-axis represents the value of CðmÞ. Results suggest thatCðmÞ for QP = 12 and 24 are significantly different, which confirm
that the same tag cannot be regenerated. Furthermore, for slice
re-sizing, cropping or rotation, the encoding process is required
to generate the format compliant HEVC video. These modifications
will further eradicate the embedded tags, hence failing the authentication process and hence indicating the sign of tampering.
4.3. Computational cost
Fig. 18 shows the graphs of total time needed for the original
and modified encoders versus bitrate for various classes of test
video sequence. It is observed that the modified encoder requires
lower computational time when encoding at higher bitrates (e.g.,for LDB configuration, >10 Mbps in Class A, and >5 Mbps in ClassB). In particular, by utilizing the proposed tag embedding tech-nique [27], RDO of the modified encoder is restricted to choose
one of the 4 types of CU to embed the authentication tag as
described in Section 3.2.1, in contrast to the original encoder thatconsiders all 8 cases. Note that the time spent on computing thecryptographic function is less than the time saved by restricting
the choices of CU type due to tag embedding. However, the opposite situation is observed at lower bitrates (e.g., for LDB case,<10 Mbps in Class A, and <5 Mbps in Class B), where the modified encoder needs longer time when compared to the originalvideo in all classes of video. It is because, at lower bitrate, the video
sequence is encoded with CUs of larger sizes (e.g., mostly 32  32
and larger). In other words, the number of CU is reduced, and thechances to embed tag (i.e., time saving) are also reduced at lower
bitrate.
Table 3 shows the percentage of increment in computationaltime, where the maximum and minimum increment among all
considered QP values are recorded. To facilitate the presentation,
let O denote the time needed to decode the original video using
the original decoder, OA denote the time needed to decode the processed video (i.e., video with tag) using the original decoder, and A
denote the time needed to decode and authenticate the processedvideo using the modified decoder. Positive percentage indicates anincrement of computational time, and vice versa. Results for OA vs.
O (i.e., Column 2 to 5) suggest that some of the test video
sequences yield negative percentage of time increment. That is,
the time needed to decode the processed video is shorter than thatof the original video. Here, the decoding time is reduced due to thedifferences in the encoded CU structure in the original and processed videos. Specifically, this happens when a more complex
CU structure is reduced to a simpler one due to tag embedding.
One of the many possible scenarios is as follows: a CTU originallyencoded with four 32  32 blocks is modified to be encoded by justone 64  64 block to embed the tag.
The negative percentages recorded in Column 6 to 9 of Table 3
suggest that the time needed for decoding and authenticating theprocessed video using the modified decoder is shorter than thetime needed to decode the original video using the original decoder. In contrast, the opposite situations are captured by the positive percentages in Table 3. All in all, the proposed
authentication scheme takes an additional computational time of9.08% and +12.69% for decoding as well as authenticating the processed video in the best and worse scenarios, respectively.
4.4. Comparison
The conventional schemes may be applicable to all video coding
standards (e.g., MPEG-2, H.264/AVC), but they are not specifically514

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515(a) Class A(b) Class B(c) Class C(d) Class D(e) Class E(f) Class F

Fig. 18. Graphs of encoding time vs bitrate for original and processed video for various classes of test sequence.

Table 3
Percentage (%) of decoding time increment for original decoder and decoder withauthentication.
Video
Class

OA vs. O

A vs. O

LDP

LDB

LDP

LDB

min.

max.

min.

max.

min.

max.

min.

max.

A
B
C

1.44
1.06
1.30

6.08
7.73
3.44

1.55
0.60
1.48

2.21
4.51
6.20

4.27
0.03
6.72

4.82
10.77
2.65

3.20
1.30
5.92

4.27
9,23
6.71

D
E

2.39
6.18

9.27
4.24

4.17
3.67

5.11
4.95

6.23
9.08

6.74
5.05

7.88
6.91

5.11
12.69

F

4.65

1.57

0.46

5.14

8.04

3.18

3.65

10.91

O = Original decoder on original video.
OA = Original decoder on processed video.
A = Modified decoder on processed video.

implemented on or experimented with the HEVC coding standard,
and hence it is not clear whether they are suitable for deployment
on HEVC. To the best of our knowledge, there is no authentication
scheme specifically designed to exploit/adapt to the coding structure of HEVC. As such, we compare our proposed multi-layer
authentication scheme to four conventional schemes under different video standard. The first scheme is proposed by Roy et al. [13],where the authentication tags are embedded in the mid-frequency
range of the non-zero DCT coefficients through hardware imple-

mentation. However, this process is only performed for I-frame
under the H.264/AVC standard. The second scheme is proposedby Wei et al. [14], where tags are embedded into the Supplement
Enhancement Information in Network Abstract Layer Unit for boththe base- and enhancement-layers in SVC (Scalable Video Coding)of H.264. The third scheme is proposed by Upadhyay and Singh[32]. They utilize a non-linear classifier to compute the statistical
local information (i.e., absolute difference) between every two consecutive slices and exploit this feature to determine whether a slice
is tampered or genuine. The fourth scheme is proposed by Ren andGorman [8], where the digital signature architecture is considered.
Local video features were calculated from slices to form a concise
fingerprint sequence, which is in turns appended to the video signal for authentication purpose.
Table 4 functionally compares the proposed and conventional
authentication schemes for compressed video using a scale from0 to 2. Here, ‘‘2” implies that the scheme is completely in line withthe function, ‘‘1” indicates that the scheme achieves part of thefunction and ‘‘0” signifies that the scheme does not have the function. All schemes are robust to video slices dropping but only our
proposed scheme exploits the dependency of all video slices inthe temporal axis, i.e., a tampered content in current slice will be
detected by the following slice. Also, our scheme extracts and utilizes the video features to verify the integrity of every slice without
the need of the secret key K, which is only required to verify theorigin of the video.

Y. Tew et al. / J. Vis. Commun. Image R. 40 (2016) 502–515
Table 4Comparison among authentication scheme.
Function

[8]

[13]

[14]

[32]

z

Require feature extraction
Apply on all slices
Robust to slice dropping
Require key for verification
Localize tampered region
Source identification
Exploit temporal axis dependency

2
1
1
2
1
2
0

1
1
1
2
0
2
1

1
2
2
0
0
2
0

2
1
2
2
1
2
1

2
2
2
1
1
2
2

2: fully functional, 1: partially functional, 0: no function.
z: Proposed authentication scheme.

5. Conclusion
A multi-layer authentication scheme for HEVC compressedvideo is put forward. The temporal dependency was enforced andexploited, where authentication tag generated based on the statistics of the current slice was embedded into the subsequent slice.
The video slices were verified by three layers of authentication:
first layer provides an surface verification without utilizing theshared secret key; second layer localizes tampered region, if any,
and; third layer verifies the source/sender by comparing the hashvalue of the combination of the shared secret key as well as thestatistics from the video against the extracted tag. Results suggestthat proposed multi-layer authentication scheme generates output
video with high perceptual quality. Robustness of the proposed
scheme against common attacks (e.g., CTU replacement, VQ attack)as well as its sensitivity against slice tampering and recompression are analyzed and justified. The proposed scheme
was also compared to the conventional video authentication
schemes.
For further work, we aim to enhance our multi-layer authentication scheme for real-time applications such as live video streaming and video conferencing as well as handling the transcoding
scenario.
Acknowledgement
This research is supported by the Fundamental Research Grant
Scheme (FRGS) MoHE Grant under project - Background/foreground separation in compressed domain (Grant No. FP0632014A). R. Phan is supported by Telekom Malaysia (TM) under project - 2beAware (Grant No. RDTC/150879).
References[1] R. Waddilove, Best free video editing software 2015 uk <http://www.
pcadvisor.co.uk/test-centre/photo-video/14-best-free-video-editing-software2015-uk-3512375/> 2015 (accessed 1-Jan-2016).
[2] P.K. Atrey, A.E. Saddik, M. Kankanhalli, Digital Video Authentication, IGI Global,
Hershey, United State of America, 2009.
[3] J. Maillard, Second emmy award for iso/iec mpeg-4 avc standard <http://www.
iso.org/iso/home/news_index/news_archive/news.htm?refid=Ref1213> 2009(accessed 1-Dec-2016).
[4] E. Casey (Ed.), Digital Evidence and Computer Crime: Forensic Sciences,
Computers and The Internet, third ed., Elsevier Inc., United State of America,
2011.
[5] D.J.W. QPM, ACPO good practice guide for digital evidence, Tech. rep.,
Association of Chief Police Officers, England, Wales and Northern Ireland, 2012.
[6] B. Furht, D. Kirovski (Eds.), Multimedia Encryption and Authentication
Techniques and Applications, Auerbach Publications, United State ofAmerica, 2006.

515[7] C. Tartary, H. Wang, S. Ling, Authentication of digital streams, IEEE Trans. Inf.
Theory 57 (9) (2011) 6285–6303.
[8] Y.J. Ren, L. O’Gorman, Accuracy of a high-level, loss-tolerant video fingerprint
for surveillance authentication, in: Int. Conference on Pattern Recognition(ICPR), 2012, pp. 1088–1091.
[9] Y.J. Ren, L. O’Gorman, L.J. Wu, F. Chang, T.L. Wood, J.R. Zhang, Authenticating
lossy surveillance video, IEEE Trans. Inf. Forensics Secur. 8 (10) (2013) 1678–
1687.
[10] J. Baek, Y. ji Byon, E. Hableel, M. Al-Qutayri, An authentication framework forautomatic dependent surveillance-broadcast based on online/offline identitybased signature, in: Int. Conference on P2P, Parallel, Grid, Cloud and Internet
Computing, 2013, pp. 358–363.
[11] J. Liu, J. Baek, J. Zhou, Y. Yang, J. Wong, Efficient online/offline identity-based
signature for wireless sensor network, Int. J. Inform. Secur. 9 (4) (2010) 287–
296.
[12] L. Song, J. Wang, L. Wang, J. Chen, A low-cost authenticating mechanism forinteractive video streaming, in: Int. Conference on Software Engineering andService Science, 2013, pp. 422–425.
[13] S. Roy, X. Li, Y. Shoshan, A. Fish, O. Yadid-Pecht, Hardware implementation of adigital watermarking system for video authentication, IEEE Trans. Circuits
Syst. Video Technol. 23 (2) (2013) 289–301.
[14] Z. Wei, Y. Wu, R.H. Deng, X. Ding, A hybrid scheme for authenticating scalable
video codestreams, IEEE Trans. Inf. Forensics Secur. 9 (4) (2014) 543–553.
[15] H.M. Al-Otum, Semi-fragile watermarking for grayscale image authenticationand tamper detection based on an adjusted expanded-bit multiscalequantization-based technique, J. Vis. Commun. Image Represent. 25 (2014)
1064–1081.
[16] X. Qi, X. Xin, A singular-value-based semi-fragile watermarking scheme forimage content authentication with tamper localization, J. Vis. Commun. Image
Represent. 30 (2015) 312–327.
[17] I. Caciula, D. Coltuc, Improved control for low bit-rate reversible
watermarking, in: IEEE Int. Conference on Acoust., Speech, and Signal
Process. (ICASSP), 2014, pp. 7425–7429.
[18] A. Sharp, J. Devaney, A. Steiner, Digital video authentication with motion
vector watermarking, in: 4th Int. Conference on Signal Process. and Commun.
Syst. (ICSPCS), 2010, pp. 1–4.
[19] S. Ababneh, R. Ansari, A. Khokhar, Iterative compensation schemes for
multimedia content authentication, J. Vis. Commun. Image Represent. 20(2009) 303–311.
[20] J. Zhang, A.T.S. Ho, Efficient video authentication for H.264/AVC, First Int.
Conference on Innovative Computing, Inform. and Control (ICICIC), vol. 3,
2006, pp. 46–49.
[21] Y. Tew, K. Wong, An overview of information hiding in H.264/AVC compressedvideo, IEEE Trans. Circuits Syst. Video Technol. 24 (2) (2014) 305–319.
[22] ISO, Information technology – high efficiency coding and media delivery inheterogeneous environments – part 2: High efficiency video coding* ISO/IEC
23008-2:2013, Int. Organization for Standardization, Geneva, Switzerland,
2013.
[23] ISO, Information technology – coding of audio-visual objects – Part 10:
Advanced video coding* ISO/IEC 14496-10:2003, Int. Organization forStandardization, Geneva, Switzerland, 2003.
[24] G.J. Sullivan, J.-R. Ohm, W.-J. Han, T. Wiegand, Overview of the high efficiency
video coding (HEVC) standard, IEEE Trans. Circuits Syst. Video Technol. 22 (12)(2012) 1649–1668.
[25] J. Vanne, M. Vitanen, T.D. Hamalainen, Efficient mode decision schemes for
HEVC inter prediction, IEEE Trans. Circuits Syst. Video Technol. 24 (9) (2014)
1579–1593.
[26] NIST, Secure Hash Standard* FIPS PUB 180-2, National Institute of Standards
and Technology, Gaithersburg, Maryland, United States, 2002.
[27] Y. Tew, K. Wong, Information hiding in HEVC standard using adaptive coding
block size decision, in: 21th IEEE Int. Conference on Image Process. (ICIP),
2014, pp. 5502–5506.
[28] Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment:from error visibility to structural similarity, IEEE Trans. Image Process. 13 (4)(2004) 600–612.
[29] S.-W. Lo, Z. Wei, X. Ding, R.H. Deng, Generic attacks on content-based video
stream authentication, in: IEEE Int. Conference on Multimedia and Expo
Workshops, 2014, pp. 1–6.
[30] J.-C. Chuang, Y.-C. Hu, An adaptive image authentication scheme for vector
quantization compressed image, J. Vis. Commun. Image Represent. 22 (2011)
440–449.
[31] A.K. Moorthy, A.C. Bovik, Blind image quality assessment: from natural scene
statistics to perceptual quality, IEEE Trans. Image Process. 20 (12) (2011)
3350–3364.
[32] S. Upadhyay, S.K. Singh, Learning based video authentication using statistical
local information, in: Int. Conference on Image Inform. Process. (ICIIP), 2011,
pp. 1–6.