# Secure Telemedicine: Biometrics for Remote and Continuous Patient Verification

Authors: Foteini Agrafioti; Francis M. Bui; Dimitrios Hatzinakos
Journal: Journal of Computer Networks and Communications
 (2012)

---

## Abstract

The technological advancements in the field of remote sensing have resulted in substantial
growth of the telemedicine industry. While health care practitioners may now monitor their
patients’ well-being from a distance and deliver their services remotely, the lack of physical
presence introduces security risks, primarily with regard to the identity of the involved parties.
The sensing apparatus, that a patient may employ at home, collects and transmits vital signals
to medical centres which respond with treatment decisions despite the lack of solid authentication
of the transmitter’s identity. In essence, remote monitoring increases the risks of identity fraud in
health care. This paper proposes a biometric identification solution suitable for continuous monitoring
environments. The system uses the electrocardiogram (ECG) signal in order to extract unique
characteristics which allow to discriminate users. In security, ECG falls under the category ofmedical biometrics, a relatively young but promising field of biometric security solutions. In this
work, the authors investigate the idiosyncratic properties of home telemonitoring that may affect
the ECG signal and compromise security. The effects of psychological changes on the ECG
waveform are taken into consideration for the design of a robust biometric system that can identify
users based on cardiac signals despite physical or emotional variations.


---

## Body

## 1. Introduction

For a number of severe diseases such as diabetes, hypertension, or respiratory disorders, where hospitalization might not always be justified or needed, home care is traditionally preferred. Moreover, the visit of a medical practitioner to the patient’s home is not necessarily an efficient solution either. This is because (i) the high health care costs for this service are undesirable; (ii) it may be infeasible for the personnel to reach highly rural areas; and (iii) monitoring is usually required on a continuous basis, rather than per visit. Home telemonitoring is now a reality, addressing the above problem very effectively, that is, it is not only cost-efficient but can also reach isolated communities and allow for 24 hr reporting on the patient’s status.Nevertheless, the widespread utilization of telemonitoring increases the risk of identity fraud in health care. Due to the lack of physical presence at the time of collection of the medical information (e.g., vital signals), the identity of the user that transmits the respective information is uncertain. Typically, every monitoring device is assigned with a unique ID (e.g., a serial code) in order to identify the transmitter. However, this only partially solves the problem since such a strategy identifies the device but not the person that utilizes it. What is needed is a means for authentication that is directly linked to the user and which can identify that person continuously.Traditional approaches to identity authentication include something that the userremembers (e.g., password, PIN number) or possesses (e.g., ID card, token). Neither of these two approaches can solve efficiently the problem of remote authentication, since they cannot be performed in a continuous manner. It is not practical to ask a user to be password authenticated during monitoring sessions. Furthermore, the above entities can be easily stolen or forgotten. Biometric security was introduced as an alternative to this problem. Linking identities with bodily characteristics that cannot be forgotten or easily stolen provides airtight security.In this work, the ECG signal is proposed for identity authentication in remote monitoring settings. The rationale behind this choice is that this signal is very likely to be collected routinely in such environments as it is typically used for medical diagnosis of several cardiac and noncardiac conditions. In addition, as opposed to “static" face or iris images, the ECG signal is time dependent which suits continuous authentication since a fresh reading can be acquired every couple of seconds and be used to reauthenticate a particular user.From a security point of view, ECG falls under the category ofmedical biometrics, that is, physiological characteristics that are typically used within health care environments for clinical diagnoses. However, there is evidence that some of these vital signals such as the ECG, phonocardiogram (PCG), photoplethysmogram (PPG), and blood volume pressure (BVP) carry information which is unique for every individual [1–5]. With the advances in the sensing technology, the potential of using these signals for biometric recognition is great. Although this work deals with the problem of human authentication using the ECG signal, similar concepts are valid for most medical biometric modalities.The ECG biometric provides inherent liveness detection which has computational advantages, since most of the existing modalities would require additional mechanisms to validate the liveness of the sensor’s reading. In addition, ECG is naturally immune to falsification or replay attacks, as it is extremely difficult to steal and inject someone’s ECG into a biometric system. Heart signals are universal, stable over a large period of time, and sufficiently unique. The intersubject variability comes from the fact that ECG pictures the electrophysiological variations of the myocardium and is affected by factors such as the heart mass orientation, conductivity of cardiac muscles and activation order [6, 7]. This variability has been extensively investigated in the medical research, for the establishment of universal diagnostic standards [8].Factors that affect the ECG waveform can be classified as physiological or psychological. While a heart rate increase due to exercise decreases the ECG “period” (in reality this signal is quasiperiodic), emotions may also change the ECG waveform. While the interaction between physiological and psychological factors on the ECG signal is obscure, there is evidence that emotional activity directly impacts the ECG waveform [9–11]. This aspect of the ECG is very important when deploying it for biometric recognition, because it can significantly affect the overall accuracy. In welfare monitoring environments, physiological and psychological variations are expected and one needs to account for such effects.The objective of this work is to first demonstrate that changes in the psychological status of a subject can potentially affect the ECG biometric template and second to provide a solution to this problem for welfare monitoring applications. While emotional activity can compromise the stability and robustness of a biometric template, prior works have concentrated on the effects of just physical activity. In this work, a template updating methodology is proposed to automatically adjust to the psychological status of a user in order to treat false rejections.


## 2. The Electrocardiogram (ECG)

The ECG is one of the most widely used signals in health care. Recorded at the surface of the body, with electrodes attached in various configurations, the ECG signal is studied for diagnostics even at the very early stage of a disease. In essence, this signal describes the electrical activity of the heart over time and pictures the sequential depolarization and repolarization of the different muscles that form the myocardium.The ECG is a quasiperiodic signal, composed of successive heart beats. Every heart beat has three essential components as follows: theP wave, the QRS complex, and the T wave. The P wave describes the depolarization of the right and left atria. The amplitude of this wave is relatively small, because the atrial muscle mass is limited. The QRS complex is the largest of three since it represents the depolarization of the right and left ventricles, which are the chambers with the most substantial mass in the heart. Finally, the T has a relatively small amplitude and it depicts the ventricular repolarization.Under normal physiological changes, that is, when the heart rate increases due to physical activity (tachycardia), or decreases due to meditative tasks (bradycardia), the relative position of the T wave may vary. Although it is usually observed about 300 ms after the QRS complex, it may appear closer to the QRS complex at rapid rates and further away at slow rates [12]. This variation is not so noticeable among the P wave and the QRS complex, which are more stable components of the heart cycle.While the effects of physical activity on the ECG signal are well established, the reactivity to psychological changes is more obscure. The autonomic nervous system (ANS) has nerve endings within the cardiac muscle which play a major role in the cardiac output because they affect the rhythm at which the muscle pumps blood. The fibers of the sympathetic system run along the atria and the ventricles and when activated stimulate the cardiac muscle to increase the heart rate. On the other hand, the parasympathetic system reduces the cardiac workload. In the presence of a mental stressor specifically, the sympathetic system dominates the parasympathetic, resulting in the following reactivity effects [13]. (1)
Automaticity: the intrinsic impulse firing (automaticity) of the pacemaker cells increases, which translates directly to an increased heart rate.(2)
Contractility: during every contraction the fibers of the heart shorten more, compared to the case during homeostasis, thereby increasing the force of contraction.(3)
Conduction rate: the natural pacemaker, the SA node, is forced to conduct faster.(4)
Excitability: during sympathetic stimulation, the person has increased perceptiveness to internal and external stimuli, which increases the irritability of the cardiac muscle and possibly lead to ectopic beats.(5)
Dilation of coronary blood vessels: the diameter of the coronary blood vessels increases, followed by increased blood flow to the cardiac muscle.While there are several open questions on the exact effects on the ECG for every experienced emotion, the ability of detecting emotion from this waveform has been demonstrated in [9]. This work demonstrates that the effects of psychological activity on the ECG waveform are significant enough to endanger biometric accuracy.


## 3. ECG in Biometric Recognition

Prior works in the ECG biometric recognition field can be categorized as either fiducial points dependent or independent. Fiducials are specific points of interest on an ECG heart beat such as the onset and the offset of the heart beat waves. Fiducial-based approaches rely on local features of the heart beats for biometric template design, such as the temporal or amplitude difference between consecutive fiducial points. On the other hand, fiducial points independent approaches treat the ECG signal or isolated heart beats holistically and extract features statistically based on the overall morphology of the waveform.Both approaches have advantages and disadvantages. While fiducial-oriented features risk to miss identifying information hidden behind the overall morphology of the biometric modality, holistic approaches deal with a large amount of redundant information that needs to be eliminated. The challenge in the latter case is to remove this information in a way that the intrasubject variability is minimized and the intersubject is maximized. For the ECG case, detecting fiducial points is a very obscure process due to the high variability of the signal. In fact, there is no universally acknowledged rule that can guide this detection [7].


### 3.1. Fiducial-Dependent Approaches

Among the earliest works in the area is Biel et al.’s [14] proposal, in 2001, for a fiducial feature extraction algorithm, which demonstrated the feasibility of using ECG signals for human identification. The standard 12 lead system was used to record signals from 20 subjects of various ages. Kyoso and Uchiyama [15] proposed four fiducial-based features for ECG biometric recognition, that is, the P wave duration, PQinterval, QRS complex, and QT durations and achieved 94.2% biometric accuracy.In 2002, Shen et al. [16] reported an ECG-based recognition method with seven fiducial-based features that relate to the QRS complex. The underlying idea was that this wave is less affected by varying heart rates and thus is appropriate for biometric recognition. More complete biometric recognition tests were reported in 2004, by Israel et al. [1]. This work presented the three clear stages of ECG biometric recognition, that is, preprocessing, feature extraction and classification. In addition, a variety of experimental settings are described in [1] such as examination of variations due to electrode placement and physical stress.Similarly, Palaniappan and Krishnan [17] used a form factor, which is a measure of signal complexity, and tested using a neural network classifier. An identification rate of 97.6% was achieved over recordings of 10 individuals. Kim et al. [18] proposed a method to normalize time domain features by upsampling the heart beats. In a similar manner, Saechia et al., [19] normalized to healthy durations and then divided into three subsequences: P wave, QRS complex, and T wave. The Fourier transform was applied on a heart beat itself and all three subsequences.Zhang and Wei [20] suggested 14 commonly used features from ECG heart beats on which a PCA was applied to reduce dimensionality. A classification method based on Bayes’ Theorem was proposed to maximize the posterior probability given prior probabilities and class-conditional densities. Singh and Gupta [21] proposed a way to delineate the P and T waveforms for accurate feature extraction. Boumbarov et al. [22] investigated different models such as HMM-GMM (hidden markov model with gaussian mixture model), HMM-SGM (Hidden Markov model with single Gaussian model) and CRF (Conditional Random Field), to determine different fiducial points in an ECG segment, followed by PCA and LDA for dimensionality reduction. Ting and Salleh [23] described in 2010 a nonlinear dynamical model to represent the ECG in a state space form with the posterior states inferred by an extended Kalman filter.Another fiducial-based method was proposed by Tawfik et al. [24]. In this work, the ECG segment between the QRS complex and the T wave was first extracted and normalized in the time domain by using Framingham correction formula or by assuming constant QT interval. The DCT was then applied, and the coefficients were fed into a neural network for classification. In summary, although a number of fiducial-based approaches have been reported for ECG-based biometrics, accurate localization of fiducial points remains a big challenge. This ambiguity risks the accuracy of the respective recognizers which require the precise location of such points. In the likely event of failing to adequately determine the locations of these points, fiducial approaches would rather reject the heart beat and require an extra reading, rather than risking the accuracy of their decision. This, however, results in increased rejection rates.


### 3.2. Fiducial-Independent Approaches

The majority of the nonfiducial approaches were reported after 2006. Among the earliest is Plataniotis et al.’s [25] proposal for an autocorrelation- (AC-) based feature extractor. With the objective of capturing the repetitive pattern of ECG, the authors suggested the AC of an ECG segment as a way to avoid fiducial points detection. Wübbeler et al. [4] have also reported an ECG-based human recognizer by extracting biometric features from a combination of leads I, II, and III, that is, a two-dimensional heart vector also known as the characteristic of the ECG. A methodology for ECG synthesis was proposed by Molina et al. [26]. A heart beat was normalized and compared with its estimate, which was previously constructed from itself and the templates from a claimed identity. Chan et al. [27] reported ECG signal collection from the fingers by asking the participants to hold two electrode pads with their thumb and index finger. The wavelet distance was used as the similarity measure with a classification accuracy of 89.1%, which outperformed other methods such as the percent residual distance and the correlation coefficient. Chiu et al. [28] proposed the use of DWT on heuristically isolated pulses. More precisely, every heart beat was determined on the ECG signal, as 43 samples backward and 84 samples forward from the R peaks. The DWT was used for feature extraction and the Euclidean distance as the similarity measure.Fatemian and Hatzinakos [29] also suggested the wavelet transform to denoise and delineate the ECG signals, followed by a process wherein every heart beat was resampled, normalized, aligned, and averaged to create one strong template per subject. A correlation analysis was directly applied to test heart beats and the template since the gallery size was greatly reduced.The spectrogram was employed in [30] to transform the ECG into a set of time-frequency bins which were modeled by independent normal distributions. Dimensionality reduction was based on Kullback-Leibler divergence where a feature is selected only if the relative entropy between itself and the nominal model (which is the spectrogram of all subjects in database) is larger than a certain threshold. Ye et al. [5] applied the discrete wavelet transform (DWT) and independent component analysis (ICA) on ECG heart beat segments to obtain 118 and 18 features, respectively. The feature vectors were concatenated. The dimensionality of the feature space was subsequently reduced from 136 to 26 using PCA which retained 99% of the data’s variance. Coutinho et al. [31] isolated the heart beats and performed an 8-bit uniform quantization to map the ECG samples to strings from a 256-symbol alphabet. Autoregressive modeling was used in [32], and the cepstral domain was investigated in [33].It is clear from the above that a large variety of fiducial-independent techniques have been proposed for ECG biometric analysis. While some approaches are more computationally intensive than others, or they operate on heart beats rather than finite ECG segments, there are practically a number of open issues in the literature with regard to ECG biometrics. Among the most prominent ones is the question of signal stability, or permanence, with time. The majority of prior works did not examine the evolution of the ECG signal with time. To some extent, the sources of intrasubject variability of the ECG signal have been ignored. We advocate that factors affecting the ECG waveform, which may render the biometric template less accurate, should be carefully studied and considered to enable real-life deployment of this technology.


## 3.1. Fiducial-Dependent Approaches

Among the earliest works in the area is Biel et al.’s [14] proposal, in 2001, for a fiducial feature extraction algorithm, which demonstrated the feasibility of using ECG signals for human identification. The standard 12 lead system was used to record signals from 20 subjects of various ages. Kyoso and Uchiyama [15] proposed four fiducial-based features for ECG biometric recognition, that is, the P wave duration, PQinterval, QRS complex, and QT durations and achieved 94.2% biometric accuracy.In 2002, Shen et al. [16] reported an ECG-based recognition method with seven fiducial-based features that relate to the QRS complex. The underlying idea was that this wave is less affected by varying heart rates and thus is appropriate for biometric recognition. More complete biometric recognition tests were reported in 2004, by Israel et al. [1]. This work presented the three clear stages of ECG biometric recognition, that is, preprocessing, feature extraction and classification. In addition, a variety of experimental settings are described in [1] such as examination of variations due to electrode placement and physical stress.Similarly, Palaniappan and Krishnan [17] used a form factor, which is a measure of signal complexity, and tested using a neural network classifier. An identification rate of 97.6% was achieved over recordings of 10 individuals. Kim et al. [18] proposed a method to normalize time domain features by upsampling the heart beats. In a similar manner, Saechia et al., [19] normalized to healthy durations and then divided into three subsequences: P wave, QRS complex, and T wave. The Fourier transform was applied on a heart beat itself and all three subsequences.Zhang and Wei [20] suggested 14 commonly used features from ECG heart beats on which a PCA was applied to reduce dimensionality. A classification method based on Bayes’ Theorem was proposed to maximize the posterior probability given prior probabilities and class-conditional densities. Singh and Gupta [21] proposed a way to delineate the P and T waveforms for accurate feature extraction. Boumbarov et al. [22] investigated different models such as HMM-GMM (hidden markov model with gaussian mixture model), HMM-SGM (Hidden Markov model with single Gaussian model) and CRF (Conditional Random Field), to determine different fiducial points in an ECG segment, followed by PCA and LDA for dimensionality reduction. Ting and Salleh [23] described in 2010 a nonlinear dynamical model to represent the ECG in a state space form with the posterior states inferred by an extended Kalman filter.Another fiducial-based method was proposed by Tawfik et al. [24]. In this work, the ECG segment between the QRS complex and the T wave was first extracted and normalized in the time domain by using Framingham correction formula or by assuming constant QT interval. The DCT was then applied, and the coefficients were fed into a neural network for classification. In summary, although a number of fiducial-based approaches have been reported for ECG-based biometrics, accurate localization of fiducial points remains a big challenge. This ambiguity risks the accuracy of the respective recognizers which require the precise location of such points. In the likely event of failing to adequately determine the locations of these points, fiducial approaches would rather reject the heart beat and require an extra reading, rather than risking the accuracy of their decision. This, however, results in increased rejection rates.


## 3.2. Fiducial-Independent Approaches

The majority of the nonfiducial approaches were reported after 2006. Among the earliest is Plataniotis et al.’s [25] proposal for an autocorrelation- (AC-) based feature extractor. With the objective of capturing the repetitive pattern of ECG, the authors suggested the AC of an ECG segment as a way to avoid fiducial points detection. Wübbeler et al. [4] have also reported an ECG-based human recognizer by extracting biometric features from a combination of leads I, II, and III, that is, a two-dimensional heart vector also known as the characteristic of the ECG. A methodology for ECG synthesis was proposed by Molina et al. [26]. A heart beat was normalized and compared with its estimate, which was previously constructed from itself and the templates from a claimed identity. Chan et al. [27] reported ECG signal collection from the fingers by asking the participants to hold two electrode pads with their thumb and index finger. The wavelet distance was used as the similarity measure with a classification accuracy of 89.1%, which outperformed other methods such as the percent residual distance and the correlation coefficient. Chiu et al. [28] proposed the use of DWT on heuristically isolated pulses. More precisely, every heart beat was determined on the ECG signal, as 43 samples backward and 84 samples forward from the R peaks. The DWT was used for feature extraction and the Euclidean distance as the similarity measure.Fatemian and Hatzinakos [29] also suggested the wavelet transform to denoise and delineate the ECG signals, followed by a process wherein every heart beat was resampled, normalized, aligned, and averaged to create one strong template per subject. A correlation analysis was directly applied to test heart beats and the template since the gallery size was greatly reduced.The spectrogram was employed in [30] to transform the ECG into a set of time-frequency bins which were modeled by independent normal distributions. Dimensionality reduction was based on Kullback-Leibler divergence where a feature is selected only if the relative entropy between itself and the nominal model (which is the spectrogram of all subjects in database) is larger than a certain threshold. Ye et al. [5] applied the discrete wavelet transform (DWT) and independent component analysis (ICA) on ECG heart beat segments to obtain 118 and 18 features, respectively. The feature vectors were concatenated. The dimensionality of the feature space was subsequently reduced from 136 to 26 using PCA which retained 99% of the data’s variance. Coutinho et al. [31] isolated the heart beats and performed an 8-bit uniform quantization to map the ECG samples to strings from a 256-symbol alphabet. Autoregressive modeling was used in [32], and the cepstral domain was investigated in [33].It is clear from the above that a large variety of fiducial-independent techniques have been proposed for ECG biometric analysis. While some approaches are more computationally intensive than others, or they operate on heart beats rather than finite ECG segments, there are practically a number of open issues in the literature with regard to ECG biometrics. Among the most prominent ones is the question of signal stability, or permanence, with time. The majority of prior works did not examine the evolution of the ECG signal with time. To some extent, the sources of intrasubject variability of the ECG signal have been ignored. We advocate that factors affecting the ECG waveform, which may render the biometric template less accurate, should be carefully studied and considered to enable real-life deployment of this technology.


## 4. The Autocorrelation/Linear Discriminant Analysis Method

ECG biometrics is essentially a pattern recognition problem, comprised of three distinct steps, that is,preprocessing, feature extraction, and classification.Preprocessing
The ECG data in raw format contain both high (powerline interference) and low frequency noise (baseline wander) that needs to be eliminated. Baseline wander is caused by low frequency components that force the signal to extend away from the isoelectric line. The source of this kind of artifacts is respiration, body movement, or inadequate electrode attachment. Powerline interference is generated by poor grounding or conflicts with nearby devices.To reduce the effects of noise, a butterworth bandpass filter of order 4 is used. Based on the spectral properties of each wave in the heart beat, the cut-off frequencies of the filter are 1 Hz and 40 Hz. The order of the filter and the passband frequencies are selected based on empirical results [34, 35].Feature Extraction
As mentioned before, the core of the proposed feature extraction method is the autocorrelation (AC) of ECG signals. The rationale for AC is that it captures the repetitive property of the ECG signal in a way that only significant, iterative components contribute to the waveform, that is, theP wave, the QRS complex, and T wave. By analyzing the AC, incidental patterns of low discriminative power are attenuated, while persistent components of discriminative power are brought to light.The syllogism behind AC with respect to fiducial points detection is that it blends, into a sequence of sums of products, ECG samples that would otherwise need to be subjected to fiducial detection. Furthermore, the AC allows a shift invariant representation of similarity features over multiple cycles. The AC can be computed as(1)R^xx[m]=∑i=0N-|m|-1‍x[i]x[i+m],
where x[i] is the windowed ECG for i=0,1…(N-|m|-1), and x[i+m] is the time-shifted version of the windowed ECG with a time lag of m=0,1,…(M-1);M≪N. Even though the major contributors to the AC are the three characteristic waves, normalization is required because large variations in amplitudes appear, even among the windows of the same subject. In addition, only a segment of the AC vector propagates to LDA, as defined between the zero lag instance and up to approximately the length of the P wave and the QRS complex. This is because these components are the least affected by the heart rate variability [7].An AC vector can be used directly for classification. However, it is important to further reduce the intrasubject variability in order to control false rejection. In addition, depending on the sampling frequency of the ECG signal, the dimensionality of an AC window can be considerably high. For these reasons the linear discriminant analysis (LDA) is recruited for dimensionality reduction.The LDA is a well-known machine learning method for feature extraction. Supervised learning is performed in a transform domain so that the AC vector’s dimensionality is reduced and the classes are better separable. The remaining discussion is based on the following definitions.(i)
LetU be the number of classes, that is, the number of subjects registered in the system.(ii)
LetUi be the number of AC windows for a subject (class) i, where i=1,…,U.(iii)
We define aszij an AC window j, where i=1,…,Ui and j=1,…,U.(iv)
Let𝒵i be the set of AC windows for a subject (class) i, defined as 𝒵i={zij}j=1Ui.(v)
Let𝒵 be a training set consisting of all AC windows of all subjects, that is, 𝒵={𝒵i}j=1U.Then a set ofK feature basis vectors {ψm}m=1K can be estimated by maximizing the Fisher ratio which is equivalent to solving the following eigenvalue problem:
(2)argmaxψ|ψTSbψ||ψTSwψ|,
where ψ=[ψ1,…,ψK] and Sb and Sw are the interclass and intraclass scatter matrices respectively, computed as follows:
(3)Sb=1N∑i=1U‍(z¯i-z¯)(z¯i-z¯)T,Sw=1N∑i=1U‍∑j=1Ui‍(zij-z¯i)(zij-z¯i)T,
where z¯i=(1/Ui)∑j=1Ui‍zij is the mean of class 𝒵i and N is the total number of windows and N=∑i=1UUi.The maximization of Fisher’s ratio is equivalent to forcing large separation between projected ECG windows of different subjects and small variance between windows of the same subject. The LDA findsψ as the K most significant eigenvectors of (Sw)-1Sb which correspond to the first K largest eigenvalues. A test input window z is subjected to the linear projection y=ψTz, prior to classification. For the purposes of this work, y is referred to as the biometric template.It is important to note that ECG biometrics benefit from supervised machine learning approaches more than other biometric modalities. This is because of this signal’s dynamic and time-dependent nature which leads the biometric to exhibit higher intraclass variability than traditional biometric modalities. With the LDA one can essentially control false acceptance and false rejection.Classification
For biometric matching, input and gallery templates are associated using the Euclidean distance as a measure of dissimilarity, while the final decision is made upon voting ofk-nearest neighbors. The normalized Euclidean distance is computed as follows:
(4)D(y1,y2)=1K(y1-y2)T(y1-y2),
where K is the dimensionality of the feature vectors. For a U class problem, LDA can reduce the dimensionality of the feature space to U-1 due to the fact that the rank of the between class scatter matrix cannot go beyond U-1.


## 5. Template Destabilization due to Emotions

Central to the design of a robust biometric solution is the study of factors that may affect the biometric signal. In the ECG case, such factors may be physiological, psychological, or environmental. Muscle contraction and movement, body fluids, and powerline interference are examples of environmental factors, that is, whose effect on the signal is added after its generation. Typical noise filters, such as the Butterworth suffice in addressing such artifacts.A healthy physiological change is usually expressed by a change in the heart rate. From a medical point of view this is, a well-studied effect. From a biometric security perspective this variation may be addressed by exploring aspects of the signal that remain unaltered, that is, theP wave and the QRS complex [7]. For instance, in the AC/LDA approach, only a segment of the AC vector corresponding to these waves is used for LDA projection.However, the effects of psychology on the ECG signal are not as well defined. While this signal has been employed in affective computing [10, 11, 36–42] in most of these cases the heart rate was used in order to determine the arousal levels. The ECG waveform (not just the heart rate) was also examined for valence classification in [9]. It was demonstrated that it is possible to detect specific emotions from the ECG signal as long as the emotional stimuli are active. (Active stimuli engage the subject in the emotion induction process (e.g., video gaming, reading, singing). Passive stimuli are presented to the subject without his/her direct engagement (e.g., film watching, listening to music)).In home telemonitoring environments active arousal is unavoidable as emotional stimulation is present in everyday activities. It is therefore important to investigate whether the emotional changes on the ECG can compromise the respective biometric template or not. In [2], it was shown that over long recording periods, while subjects are performing every-day working activities, the ECG biometric template may destabilize, that is, change to a degree that endangers security. While the experimental procedures reported in [2] were not emotion specific, it was concluded that psychology is the underlying cause of variation on the ECG signal.The purpose of the present work is twofold:(1)
to associate, within a monitoring session, instances of template destabilization with emotion changes;(2)
to provide a solution to the above problem based on biometric template updating.The reader should note that the above two objectives do not directly encompass methods to detect or classify specific emotions. While experimental procedures have been designed in order to trigger specific emotional response, determining which emotion is experienced is beyond the scope of the current work and has been addressed separately [9].


## 6. Emotion Elicitation and ECG Signals

The following experiment was conducted at the Affect and Cognition Laboratory of the University of Toronto. The purpose was to elicit active mental arousal using a commercial video game.In practice the experiment attempted to have the player gradually immersed, by increasingly concentrating in order to meet the game requirements. A research-platform video game was used, namely, the Cube 2: Sauerbraten (http://sauerbraten.org/). A pilot game was built to assist the needs of the experiment. The subjects got motivated with deception, by letting them know that the purpose of the experiment is to measure game completion time.All participants were seated in front of a computer screen and presented with a short introduction to the video game. A five-minute pilot game was played, for the participant to learn and be adjusted to the game. During that time, no physiological response was monitored. When the subject felt comfortable with the process, the ECG sensor was placed and the main game was initiated. In total 43 volunteers participated in this study. Data from one person were discarded due to excessive noise.Depending on the familiarity of the subject with game playing, the duration of the experiment varied between 20 and 45 minutes. During the game, ECG was monitored using Hidalgo’s equivital sensor, which is portable and wireless. Unobtrusiveness was very important for the subjects to be naturally immersed in the game. ECG was recorded from the chest and digitized at 256 Hz.Because of the stochastic nature of the game and the unforeseeable order of events that can take place, arousal annotation was self-determined. For this reason, a video of the player’s facial expressions was captured during the game (synchronous to ECG). Upon game completion, the subjects were asked to watch a playback video of the game and their facial expressions while continuously reporting arousal using FEELTRACE [43]. Figure 1 shows an example of such a self-assessment video, while Figure 2 illustrates the data labeling scheme.Figure 1
Game and face video playback, used for self-assessment of arousal.Figure 2
Data labeling for the active arousal experiment. The FEELTRACE is a continuous arousal indication.


## 7. Template Updating

The underlying idea of the proposed approach is to update the ECG biometric template at instances of destabilization due to psychological changes. An alternative to this solution would be to preenrol a subject with a number of different templates along the spectrum of emotional responses. However, this is not a practical solution since it is not feasible to induce such conditions on demand or ensure that one indeed succeeded in doing so. Furthermore, such an approach would be very inconvenient for real-world environments.In the proposed system, the user can be enrolled once, by submitting his/her ECG signal irrespective of the psychological status. In return, during normal operation, when the system detects that the existing biometric template has been destabilized, that is, results in low-quality matching with newly captured ECG segments, a new biometric template is created to replace the first. Therefore, the objective of the subsequent analysis is to update the biometric template at instances corresponding to the destabilization ordecoherence of the correlation scores among consecutive ECG readings and the biometric template.The evaluation of the coherence of ECG readings has been discussed in [2, 3]. The same approach is adopted in the current analysis; however the objective of the present work is to associate instances of destabilization with psychological changes.The proposed system constructs variable-length accumulated durations of ECG segments based on somefundamental time duration. The accumulated segments need to be coherent, that is, exhibit high correlation with each other (intraburst correlation). When decoherence is observed, template update is initiated based on the latest ECG segments. In practice, there is a tradeoff between frequency of template updates and computational complexity. Frequent template updating implies accurate tracking of events but increases the computational effort. On the other hand, infrequent template updating may cause inadequate system performance in terms of increased false rejection. To efficiently address this problem, a minimum time duration is forced over which the system makes a decision about whether the template needs to be updated. In an extreme case one can define this minimum duration to be equal to the smallest time resolution in the system, that is, the fundamental duration (5 seconds for the AC/LDA algorithm). However, this is computationally inefficient, and therefore the updating instances need to be strategically chosen.A variable-length accumulated duration (orburst) is constructed by accumulating various fundamental durations (for the proposed AC/LDA algorithm, these fundamental durations correspond to ECG segments of 5-second length). The following iterative description can be made.Consider the following fundamental durations{d1,d2,…}, where each di corresponds to time duration of 5 seconds. This duration is chosen to acceptably accommodate the time resolution requirement of the AC/LDA algorithm. Now suppose that, at the current iteration, the current burst, Dcurrent, contains μ fundamental durations, that is, Dcurrent={dk,…,dk+μ-1}. For the subsequent segment, dk+μ, the two choices are as follows.(C1)
Adddk+μ to the current burst, forming Dpotential={dk,…,dk+μ}. Continue the operation with dk+μ+1 as the next candidate.(C2)
Rejectdk+μ, and terminate Dcurrent. Reinitialize with dk+μ as the start of a new burst.The correlation coefficient is used for the estimation of the intraburst coherence and allows the system to decide between (C1) or (C2) as follows.(1)
Compute the correlation profile forDpotential relative to starting point dk.(2)
Find the minimum correlation valuecmin over Dpotential.(3)
Compare to a thresholdcth for decision:
(5)cmin-cth⋛C2C10.In order to account for buffering problems, along with the minimum necessary burst duration, the system also uses amaximum duration (e.g., dsizemax) which, when reached, forces the system to update the biometric template. With this treatment the algorithm can reset.For the purposes of the current analysis, the subsequent procedure aims to validate that every coherent burst describes a true emotional state. To do that, a measureqi is introduced to describe the system’s confidence that burst i represents consistently one emotional state. As explained earlier in the signal collection section, every ECG window x(t) (in the burst i) is labeled as high (H) or low (L) arousal. Let NHi and NLi be the number of windows marked as high and low arousal, respectively. Then qi can be calculated as
(6)qi=max(NHi,NLi)NHi+NLi.If template updating succeeds, every burst that is identified by the updating algorithm should have a highqi, which means that with high confidence each burst corresponds to either high or low arousal.The reader should note that the purpose of this work is not to automatically detect emotional states. Emotional confidence is calculated as ana posteriori validation to demonstrate the coincidence of instances of emotional change and biometric template destabilization.


## 8. Experimental Results

The performance of the proposed algorithm for template updating was evaluated on emotion-annotated ECG signals. ECG readings are available for the duration of the video game, as previously described in Section6. The first few seconds of every reading were used for initialization, that is, the design of an initial biometric template. Then in the sequel, the correlation of this template with subsequent ECG readings was determined and, if not sufficient, the template was updated.The number of template updates within the 30 min gaming session varies for different individuals. Subjects that were very emotionally consistent required only one template update while others updated up to eight times.The results are shown in Figure3 which lists the q values (in percentage) for all detected bursts of all subjects in the database. Even though this measure relies on the subjects’ self-reports and discrepancies are expected, the average state confidence for the bursts is 96.47%. This performance is illustrative of the accuracy of detecting homogeneous emotional states, which leads to successful template updating. Now, consider the original problem of biometric recognition when a burst is terminated due to decoherence the system performs a template update. While the above results show that each burst corresponds to either high arousal or low arousal, the question that naturally arises is the following: do two consecutive bursts correspond to opposite arousal labels? In other words, is change in arousal the only factor that is responsible for burst termination?Figure 3
Detected bursts and respective emotion consistency (qi) for all subjects in the database (N.B.: the above bursts are not necessarily of the same duration, i.e., variable-length durations may be detected). The average state confidence is 96.47%.In practice, the template updating algorithm is not only affected by emotional states. A burst may be interrupted due to one of the following three reasons.(1)
A state change, that is, a transition from one psychological state to another.(2)
Buffer overflow, that is, when dsizemax is reached. dsizemax can be adjusted according to the requirements of the application environment. For the present simulation dsizemax was set to 10 minutes.(3)
Noise artifacts (e.g., due to a sudden movement), that are not sufficiently treated by the filter.Nevertheless, a template update is necessary in all cases.For every individual in the database, a template is updated every time a new burst is detected. To quantify the verification accuracy after template updating, the false acceptance (FA) and rejection (FR) rates are estimated for every individual separately. Table1 lists the equal error rates (EER depicts the error rate at which the probability of false acceptance equal that of false rejection) that were achieved with this treatment, for all subjects in the active arousal dataset. The average equal error rate in this case is 3.96%. It should be noted that the baseline system performance that is without template updating results in 15% EER. Figure 4 demonstrates the FA and FR tradeoffs for nine randomly picked individuals. In each simulation FA was computed with comparisons of the updated template of one individual against the remaining subjects in the database.Table 1
Equal error rate for each individual in the active arousal database, after template updating. Mean equal error rate is 3.96%.
Subject
EER
Subject
EER
Subject
EER
Subject
EER
Subject
EER
1
6%
11
4.3%
21
2.65%
31
5%
41
0.2%
2
4.2%
12
3.6%
22
3%
32
5.16%
42
0%
3
8%
13
1.1%
23
5.65%
33
2.65%
43
1.35%
4
4.4%
14
4.3%
24
3.95%
34
9.2%
5
4%
15
3.2%
25
3.38%
35
8.9%
6
0%
16
2.5%
26
4.04%
36
1.85%
7
0.5%
17
2.75%
27
3.35%
37
7.65%
8
5.5%
18
4.63%
28
1.46%
38
2%
9
0.05%
19
4.2%
29
2.75%
39
2.95%
10
3.8%
20
2.3%
30
0.5%
40
13.34%Verification performance with template updating for 9 individuals of the active arousal database.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)A parameter that controls the above analysis iscth, that is, the threshold on the correlation coefficient that is used to determine burst coherency. As this threshold increases, stronger coherence is imposed on the bursts, leading to smaller EER, as shown in Figure 5. However, this is achieved at the expense of more frequent template updating needed (i.e., higher cost and complexity).Figure 5
Equal error rates of the template updating algorithm for various coherence thresholdscth.


## 9. Conclusion

In this work, a novel identity recognition system based on the ECG signal has been presented. The proposed biometric solution is suitable for welfare monitoring environments which require remote, efficient, and continuous authentication of the involved parties. In such settings, the ECG is typically collected among other vital signals and used for diagnosis and treatment decisions. Relying authentication on the very same modality not only increases security and convenience but also enhances user privacy since no other credentials are required to perform this task.This technology has significant advantages, most of which arise from the fact that ECG is a body-internal signal. For instance, ECG-enabled biometric systems can automatically and inherently assess the “liveness" of the biometric reading. This is not the case with traditional biometric modalities (e.g., iris or fingerprint) which require additional computational effort to assess the liveness of the sensor reading. In addition, ECG is a continuous signal that can be conveniently used to authenticate users several times, each time using a fresh reading.This paper addressed the problem of intrasubject variability of the ECG signal due to psychological changes. It was demonstrated that a biometric signature designed at a particular time may not allow for robust matching at a later time, under different emotional conditions (template destabilization). A solution based on template updating was proposed for welfare monitoring environments. The performance was evaluated over ECG signals from 43 volunteers, under various arousal conditions induced with a video game. On average, an equal error rate of 3.96% was achieved, which represents a dramatic reduction from the EER of 15% for the particular dataset in the absence of template updating.



---

*Source: 924791-2012-11-26.xml*