prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What types of sensors are now capable of estimating physical activity levels and physiological outcomes of older adults?

Read the following text and answer briefly.

Context 1: \section{Introduction}
Cognitive deficit of older adults is one of the biggest global public health challenges in elderly care  Approximately 5 2 million people of 65 and older are suffered with any form of cognitive impairments in United States in 2012 \cite{stat12}  Dementia is one of the major causes of the cognitive impairments which is more acute among 85 and older population (50\%) \cite{stat12}  However, the costs (financial and time) of health care and long-term care for individuals with Alzheimer's (special form of dementia) or other dementias are substantial  For example, during 2016, about 15 9 million family and friends in United States provided 18 2 billion hours of unpaid assistance to those with cognitive impairments which is a contribution to the nation valued at \$230 1 billion  One the other hand, total payments for all individuals with all form of cognitive impairments are estimated at \$259 billion  Total annual payments for health care, long-term care and hospice care for people with Alzheimer's or other dementias are projected to increase from \$259 billion in 2017 to more than \$1 1 trillion in 2050  Among the above costs, a significant amount are relevant to clinical and diagnostic tests \cite{stat17}  Although clinical and diagnostic tests have become more precise in identifying dementia, studies have shown that there is a high degree of underrecognition especially in early detection  However, there are many advantages to obtaining an early and accurate diagnosis when cognitive symptoms are first noticed as the root cause findings of impairment always lessen the progress of impairment status and sometimes symptoms can be reversible and cured 

With the proliferation of emerging ubiquitous computing technologies, many mobile and wearable devices have been available to capture continuous functional and physiological behavior of older adults  Wearable sensors are now capable of estimating number of steps being taken, physical activity levels, sleep patterns and physiological outcomes (heart rate, skin conductance) of older adults \cite{sano15}  Ambient sensors also help capture the movement patterns of objects and humans for activity and behavior recognition \cite{dawadi14,dawadi15}  Researchers also proved the existence of correlations between cognitive impairment and everyday task performance \cite{dawadi14, akl15,alam16} as well as physiological symptoms \cite{alam16,sano15}  Although current studies showed some successes in IoT-assisted cognitive health assessment in different domains individually, there are several existing challenges in developing and validating a fully automated multi-modal assessment model 

\begin{enumerate}
\item \emph{Real-time IoT System}: A real-time IoT system must include a continuous and fault tolerant data streaming capability among central hub, wearable sensors and ambient sensors regardless of network communication protocol (WiFi, Ethernet, Bluetooth etc ) which are not available in existing researches 
\item \emph{Multi-modal Context Fusion}: Though several offline clinically validated cognitive health assessment tools exist \cite{wai03, starling99, krapp07, yesavage82, zung71}, there is no universally accepted method for IoT-assisted automatic cognitive health assessment in smart home environment that can fuse multi-modal sensor contexts altogether  For example, some researchers showed ambient sensors based Activities of Daily Livigin (ADLs) sequence pattern can signify the cognitive health status of older adults \cite{akl15, dawadi15}  Researchers also showed wearable Electrodermal Activity pattern analysis may carry the significance of cognitive status \cite{sano15}  However, for validation of IoT based cognitive health assessment, self-reported surveys, clinical diagnosis and observation based tools are used individually by prior researchers \cite{akl15, dawadi15, sano15, alam16} 
\end{enumerate}

Regarding aforementioned challenges for the automation of cognitive health assessment, \emph{AutoCogniSys} considers (i) reproducibility of our model in any smart home system consists of ambient motion sensors, wearable accelerometer (ACC) sensors, wearable Electrodermal Activity (EDA) and Photoplethysmography (PPG) sensors individually or combined streams; (ii) context awareness based on ambient motion sensors and wearable ACC sensors in any types of activities such as hand gestural, postural and complex ADLs; and (iii) high accuracy, i e , a recall rate of over 90\% with less than 5\% false positive rate  More specifically, \emph{AutoCogniSys} extends our existing work \cite{alam16} in three dimensions,

\emph{(1) True Automation:} We first investigate the correlations of cognitive impairment with human activities and stress where we manually labeled activities, extract the corresponding physiological sensor (EDA and PPG) features of each activity, and use statistical method to find correlations  Then, we propose automatic complex activity recognition based on a Hierarchical Dynamic Bayesian Network (HDBN) model, fine-grained extraction of physiological sensor features and finally machine learning classification of cognitive impairment 

\emph{(2) Noises Elimination:} We define different types of noises on ACC, EDA and PPG sensors, propose extensive signal processing techniques to remove noises and show significant improvement can be achieved in cognitive impairment classification 

\emph{(3) Implementation and Evaluation:} Finally, we design and implement IoT system and analytic methods and minimize the human involvement to automate our proposed cognitive health assessment approach by considering effective smart home sensor customization and deployment, data collection, screening, cleaning and filtering, feature computation, normalization and classification, and activity  model training 

\textbf{Research Questions:} \emph{AutoCogniSys} consequently tackles the following key research questions 

$\bullet$ Can we detect simultaneously the periodic rhythms of  both hand gestures and postural activities from wrist-worn ACC sensor signal for diverse population (population with same activity but diverse ways such as walking with walker, stretcher or normally)? If so, how can we incorporate the hand gesture, posture and ambient sensor data streams to help improve the ADLs recognition models?

$\bullet$ How can we exploit and relate the micro-activity features into noise free physiological sensor signals processing to automate cognitive health assessment process? What are the critical roles of clinical survey and technology guided assessment methodologies and their inter-relationships for automating the different intermediate steps of cognitive health assessment process?

To tackle these, we make the following \textbf{key contributions}:

$\bullet$ We employ an extensive signal deconvolution technique that in conjunction with machine learning technique helps facilitate a wrist-worn ACC-based multi-label (hand gestural and postural) activity recognition for diverse population  We then leverage multi-label context sets with ambient and object sensor signals for complex activity recognition based on HDBN model 

$\bullet$ We propose a novel collaborative filter for EDA signal processing by postulating signal as a mixture of three components: \emph{tonic phase, phasic phase} and \emph{motion artifacts}, and employ convex optimization technique for filtering out the motion artifacts  We also propose a novel PPG signal processing technique to filter out the inherent motion artifacts and noises using improved Periodic Moving Average Filtering (PMAF) technique 

$\bullet$  We design and prototype an IoT system consisting of multiple devices (wearable wrist band, IP camera, object and ambient sensors) connected with central hub via WiFi, Ethernet and Bluetooth communication protocols  We collected data from 22 older adults living in a continuing care retirement community center in a very natural setting (IRB \#HP-00064387) 

$\bullet$ Finally, we employ statistical and machine learning techniques to jointly correlate the activity performance metrics and stress (EDA and PPG) features that helps achieve max  93\% of cognitive impairment status detection accuracy  We evaluate \emph{AutoCogniSys} on 5 clinically validated offline assessment tools as ground truth 
\section{Related Works}
\emph{AutoCogniSys} builds on previous works on wearable devices based low-level (postural and hand gestural) activity recognition and their integration with ambient sensors to recognize complex ADLs, the underlying signal processing and applications on cognitive health assessment automation 
\subsection{Wearable Sensor Signal Processing}
Wearable sensors can be two types: physical and physiological  Physical sensors (accelerometer, gyroscope etc ) signal values change over the movements of the sensor devices  Physiological sensors change over physiological condition of body such as EDA changes over stress and PPG changes over heart rate  However, physical movements also impose noises on physiological sensor signals which is called \emph{motion artifacts} 
\subsubsection{Physiological Signal Processing}
A continuous and descrete decomposition of EDA, and time and frequency domain analytics of PPG signal have been investigated before to extract relevant physiological features which were contaminated with noises and motion artifacts \cite{alam16}  \cite{setz10} denoised and classified EDA from cognitive load and stress with accuracy higher than 80\%  Though motion artifacts removal techniques such as exponential smoothing \cite{hern11} and low-pass filters \cite{poh10, hernandez14} provide significant improvement in filtering EDA signals, wavelet transforms offer more sophisticated refinement for any kind of physiological sensors such as electroencephalogram \cite{krish06, zikov02}, electrocardiogram \cite{erc06,alfa08}, and PPG \cite{lee03}  \cite{chen15} proposed a stationary wavelet transform (SWT) based motion artifacts removal technique  `cvxEDA' proposed  a convex optimization technique considering EDA as a mixture of white gaussian noise, tonic and phasic components where white gaussian noise includes motion artifacts and external noises \cite{greco16}  \emph{AutoCogniSys} intelligently combines SWT and `cvxEDA' together to remove noises and motion artifacts from EDA signal  On the other hand, it is more difficult to remove motion artifacts from PPG signal due to its periodicity of nature \cite{wang13}  Researchers proposed different methods such as frequency analytics \cite{garde13,wang13}, statistical analytics \cite{peng14} and digital filter \cite{lee10} to reduce noises and motion artifacts from PPG  \emph{AutoCogniSys} used Periodic Moving Average Filter (PMAF) in this regard \cite{lee07} 
\subsubsection{Physical Sensor Signal Processing}
ACC based hand gesture recognition has been explored by several researchers in past such as discrete hidden markov model \cite{liu10}, artificial neural network \cite{arce11}, weighted naive bayes and dynamic time warping \cite{mace13}  Akl et  al  proposed 18 gesture dictionary based Support Vector Machine (SVM) classifier \cite{akl11}  Wrist-worn ACC based postural activity recognition approach has been proposed using Decision Tree, Random Forest, Support Vector Machines, K-Nearest Neighbors, Naive Bayes and deep neural networks \cite{gj14, wang16}, the accuracy stagnates at 85\% using SVM method \cite{martin16}  However, neither of past works proposed any technique that can provide single body worn ACC sensor-based multiple body contexts recognition nor works efficiently for diverse posture say walking normally, with walker, with double walker or wheel chair  Our proposed 8-hand gesture recognition technique assisted sparse-deconvolution method improves classification performances on both normal and diverse postures  However, we incorporated hand gestures and postures in conjunction with ambient sensors into single-inhabitant HDBN model \cite{alam16b} that provides significant improvement in complex activity recognition 
\subsection{Cognitive Health Assessment}
Smart home environment has been used for providing automated health monitoring and assessment in the ageing population before \cite{dawadi14, gong15, akl15, dawadi15}  `SmartFABER' proposed a non-intrusive sensor network based continuous smart home environmental sensor data acquisition and a novel hybrid statistical and knowledge-based technique to analyz the data to estimate behavioral anomalies for early detection of mild-cognitively impairment \cite{riboni16}  \cite{skubic15} presented an example of unobtrusive, continuous monitoring system for the purpose of assessing early health changes to alert caregivers about the potential signs of health hazards  Though, prior researches proposed a sequence of ambient motion sensor streams as complex activity components in activity based health assessment \cite{dawadi14, gong15, akl15, dawadi15}, we consider inclusion of an wearable wrist-band with in-built ACC sensor to detect hand gesture and posture, augmenting with the ambient sensor readings to help recognize complex activities as well as cognitive health assessment of older adults  Additionally, we propose intelligent use of physiological features of skin through different physiological sensor signals (EDA, PPG) processing in daily activity tasks and incorporate context-awareness for automation of cognitive health assessment that have not been explored before 
\begin{figure}[!htb]
\begin{center}
   \epsfig{file=flowchart pdf,height=1 6in, width=3 5in}
\caption{Overall flow of \emph{AutoCogniSys} pipeline }
   \label{fig:overview}
\end{center}
\end{figure}
\section{Overall Architecture}
We first investigate existing IoT-based cognitive health care frameworks that covers every aspects of wearable (physical, physiological) and ambient (passive infrared and object) sensor signals computing  \emph{AutoCogniSys} is comprised of three component modules: (i)~sensing, (ii)~processing, and (iii)~analysis  The `sensing' module consists of clinical assessment tools (surveys, observation and clinical backgrounds) and sensing signals (ambient and wearable sensors)  `Sensor processing' module is comprised of three sub-modules: a)~clinical assessment feature extraction from assessment tools; b)~ambient sensor feature extraction; and c)~wearable sensor processing (noise removal, segmentation, feature extraction, classification etc )  `Analysis' module is comprised of machine learning and statistical analytics-based score prediction of cognitive impairment  Automation of each module's functionality and inter-intra modular transactions without human interference can be called {\it true automation} of cognitive health assessment  Fig ~\ref{fig:overview} shows the overall flow of \emph{AutoCogniSys} which is discussed in details in the following sections 
\subsection{Demographic Ground Truth Data Collection}
Currently in standard clinical practice and research, the most accurate evaluations of cognitive health assessment are one-to-one observation and supervision tasks/questionnaires for monitoring an individual's functional abilities and behavior \cite{resnick15}  In the first stage of this pilot study, we have investigated current literatures and carefully chosen the clinically proven functional and behavioral health assessment survey tools \cite{resnick15}  On the other hand, to cross check with the survey based evaluations, we have also chosen clinically justified observation based behavioral assessment methods  First, following the resident consent, our clinical research evaluator collects demographic and descriptive data (age, gender, race, ethnicity, marital status, education and medical commodities)  She has performed two types of clinical assessments: (1) \emph{Observation based} where the resident's cognition is assessed using the Saint Louis University Mental Status (SLUMS) scale \cite{wai03}  (2) \emph{Survey based} where five widely used and clinically well validated surveys are taken into account: (a) \emph{Yale Physical Activity Survey} \cite{starling99}; (b) \emph{Lawton Instrumental Activities of Daily Living}; (c) \emph{Barthel Index of Activities of Daily Living} \cite{krapp07}; (d) \emph{Geriatric Depression Rating scale} \cite{yesavage82}; and (e) \emph{Zung Self-Rating Anxiety scale} \cite{zung71} 
\subsection{Smart Environment Creation}
For an ideal IoT-based system, instrumenting and deploying it at each participant's natural living environment warrants for assembling a flexible set of hardware and software interfaces to ease the system configuration, setup, and network discovery processes  The sensor system placed in the residences of volunteers needs to meet several specific physiological signals and activity monitoring needs  However, we must confirm that the devices are reliable with potential for re-deployment as well as appear unintimidating to the participants  Inspired by the above requirements, we developed a real testbed IoT system, {\it SenseBox}, by customizing Cloud Engine PogoPlug Mobile base station firmware to integrate with WiFi (connect ambient and object sensors) and Bluetooth (connect wristband) protocol  The smart home components are as follows: (i) PogoPlug base server with a continuous power supply, (ii) 3 binary Passive Infrared sensors in three different rooms (kitchen, livingroom and bedroom) to capture room level occupancy, (iii) 7 binary object sensors attached with closet door, entry door, telephone, broom, laundry basket, trash can and trash box, (iv) three IP cameras in the appropriate positions to collect the ground truth data and (v) an Empatica E4 \cite{empatica} wrist-band (integrated sensors: PPG at 64 Hz, EDA at 4 Hz, Body temperature at 1 Hz and a triaxial ACC at 32 Hz) on the participant's dominating hand 
\section{Activity Recognition}
We aim to detect single wrist-worn ACC sensor based hand gesture and postural activities and insert these into an HDBN graphical model in conjunction with ambient and object sensor values for complex activity recognition  We consider the recognition problem asan activity tupple of $\langle gesture,posture,ambient,object \rangle$  Though, Alam et  al  provides significant performance improvement for single wrist-worn ACC sensor aided 18-hand gesture based postural activity recognition in lab environment \cite{alam17}, it faces some practical challenges in real-time smart environment with older adults due to the diversity of their postures  For example, some older adults use walker, double walking sticks or wheel chair for walking in which cases collecting 18 hand gestures and corresponding postural activities for training requires endless efforts and carefulness  To reduce the complexity of ground truth labeling and later state space explosion for graphical model (HDBN), we propose to use rotational normalization method that can merge some hand-gestures subject to directional differences and forms an 8-hand gesture model  However, our proposed Feature Weight Naive Bayes (FWNB) classifier adds significant improvement on Alam et  al  proposed sparse-deconvolution method as well as recognition in diverse postural environment 
\begin{figure}[!htb]
\begin{center}
   \epsfig{file=hand_gestures pdf,height=0 5in, width=3in}
   \vspace{- 2in}
\caption{8 hand gesture dictionary with direction}
   \label{fig:hand_gestures}
   \vspace{- 2in}
\end{center}
\end{figure}
\subsection{Hand Gesture Recognition}
\label{sec:hand_gesture}
\emph{AutoCogniSys} proposes an 8-gesture dictionary (as shown in Fig  \ref{fig:hand_gestures}) and a Feature Weighted Naive Bayesian (FWNB) framework for building, modeling and recognizing hand gestures  The method comprises of the following steps: (i) \emph{Preprocessing:} wrist-worn ACC sensor provided 3-axis data are passed through 0 4Hz low-pass filter to remove the data drift  (ii) \emph{Rotation normalization:} Normalizing the rotation of hand gestures provides greater accuracy and allows for more realistic, orientation-independent motion  At first, we find the best fit plane of the acceleration vectors thus if the motion lies in a single plane, then the acceleration vectors of a closed shape should on average lie in that main plane  Then, we take all acceleration segments between points of inflection to form one single vector called reference vector that provides us the general direction of user's motion  After that, each vector is normalized relative to the reference vector  This normalization helps remove a lot of hand gestures from prior considered 18 hand gestures resulting a reduced dictionary of 8 gestures  (iii) \emph{Feature Weighted Naive Bayesian model:} Naive Bayes classifier is light-weight and efficient technique for hand gesture recognition  We extract 12 ACC features \cite{alam17} and calculate weight for each feature type based on the similarity of feature measures of the trained gestures (0$<$weight$<$1)  While recognizing gestures, the proximity of each feature measure to the average trained feature measure of each gesture type is calculated by a normal distribution  Then, the proximity value is multiplied by the feature weight that was calculated in the training phase   All of these multiplied values are added together and the system predicts the gesture type with the greatest value as the user gesture  In the learning data points, there should be static postural activities (such as sitting, lying etc ) to avoid unexpected noises over wrist-worn ACC sensors  In the final hand gesture dictionary, we save the reference vector as our signal dictionary 
\subsection{Postural Activity Recognition}
In normal lab environment, wrist-worn ACC sensor signal is a mixture (convolution) of actual hand gesture and postural activity relevant signals \cite{alam17}  \emph{AutoCogniSys} improves the idea by reducing the number of hand gestures and postural activities to 8 (as shown in Fig \ref{fig:hand_gestures}) using rotation normalization and 4 (walking, sitting, standing and lying)  Then, we use sparse-deconvolution method (with 31\% signal reconstruction error) to get Approximately Sparse Factor  The summary of the entire process is stated bellow:

{\it Building Deconvolution Method:} We first consider the wrist-worn ACC sensor signals (3-axis values) as a convolution of hand gesture and postural activity effects and build a deconvolution framework  The deconvolution framework takes a known signal (hand gesture effects) and a equalizer parameter ($\lambda$) as input and provides an Approximately Sparse Factor signal (postural activity effects) as output  For 3-axis ACC signals, we need to learn associated 3 equalizer parameters for each hand gesture  Moreover, each equalizer parameter is involved with 4 postural activities that results a total 96 ($8\times 3\times 4$) equalizer parameters to learn  

{\it Learning Classification Model:} We use the Approximately Sparse Factor signal to extract 12 statistical features and SVM with sequential machine optimization (SMO) \cite{cao06} for postural activity recognition 

{\it Prediction Model:} After recognizing the hand gestures following the method explained in Sec ~\ref{sec:hand_gesture}, we take the corresponding reference vector as known signal and extract the Approximately Sparse Factor signals incorporating corresponding 3 equalizer parameters ($\lambda$) for the sparse-deconvolution method  Then, we apply feature extraction and prior learned SMO based SVM classifier \cite{cao06} to classify final postural activity  Fig ~\ref{fig:deconvolution} illustrates a single axis example of the deconvolution 

\begin{figure}[!htb]
\begin{center}

   \epsfig{file=deconvolution pdf,height=1 6in, width=3in}
   \vspace{- 15in}
\caption{Sample deconvolution example of X-axis  The raw x-axis of accelerometer signal, reference vector of the sample gesture and the extracted corresponding ASF signal of walking }
   \label{fig:deconvolution}
\end{center}
\vspace{- 15in}
\end{figure}

\subsection{Complex Activity Recognition}
We build a HDBN based complex activity recognition framework for single inhabitant scenario smart home environment \cite{alam16b} taking the advantage of detected hand gestural and postural activities along with the ambient and object sensor streams  At first, we obtain instant hand gestural and postural activities from our above proposed models, and additionally motion sensor and object sensor readings from our IoT-system for every time instant generating a 4-hierarchy of HDBN model  Considering the context set $\langle gestural, postural, ambient,object\rangle$ as a hierarchical activity structure (extending two 2-hierarchical HDBN \cite{alam16b}), we build complex activity recognition model for single inhabitant scenario  Finally, we infer the most-likely sequence of complex activities (and their time boundaries), utilizing the well-known Expectation Maximization (EM) algorithm \cite{dempster77} for training and the Viterbi algorithm \cite{forney73} for run-time inference 
\section{Automatic Activity Features Estimation}
The effects of cognitive ability on daily activity performance have been studied before \cite{dawadi14,akl15}  They experimentally and clinically validated that cognitive impairment highly reduces the daily activity performances and this activity performance can be computed as an indicator of cognitive ability status of older adults  The standard activity features refer to completeness of task (TC), sequential task ability (SEQ), interruption avoidance capabilities (INT) etc  In current behavioral science literature, the above activity features carry specific definition based on the sub-tasks involved with a complex activity \cite{dawadi14,akl15}  Completeness of task refers to how many sub-tasks are missed by the participants  Sequential task ability refers to how many sequences of sub-tasks are missed referring the gerontologist defined standard sequences of the sub-task for the particular complex activity  Interruption avoidance capability refers to how many times the participants stop or interleave while doing any sub-task  The final goal of activity features estimation is to provide overall task score  The task score is proportional to the functional ability of participants in performance daily activities  Our behavioral scientist team, comprises with Nursing professor, gerontologist and retirement community caregivers, carefully discus, optimize and choose 87 sub-tasks in total for 13 complex activities 

Each of the sub-task comprises with sequential occurrences of hand gesture and postural activities  However, no researchers ever considered hand gesture for activity features estimation due to complexity of multi-modal wearable and ambient sensors synchronization and multi-label activity classification \cite{dawadi14,akl15}  \emph{AutoCogniSys} exploited single wrist-worn sensor based hand gesture and postural activity recognition, and proposed an activity features (TC, SEQ and INT) estimation method including these two parameters in conjunction with object and ambient sensor features that provide significant improvement of cognitive health assessment of older adults 
\subsection{Machine Learning Based Complex Activity Features Estimation}
In current cognitive health assessment literature, complex activity features can be defined as $\langle TC,SEQ,INT,TS\rangle$  We used supervised method to estimate TC, SEQ and INT, and unsupervised method to estimate TS  We first, formulate the automated scoring as a supervised  machine learning problem in which machine learning algorithms learn a function that maps $\langle${\it hand gesture, posture, object, ambient sensor}$\rangle$ feature set to the direct observation scores  We use bagging ensemble method to learn the mapping function and SMO based SVM \cite{cao06} as base classifier  The learner averages by boostrapping individual numeric predictions to combine the base classifier predictions and generates an output for each data point that corresponds to the highest-probability label  We train three classifiers considering observation as ground truth for TC, SEQ and INT scores and test on the testing dataset  We derive unsupervised scores using dimensionality reduction technique for each feature set  First, we take all features of each activity, apply optimal discriminant analysis technique as a dimensionality reduction process \cite{zhang09} and reduce the feature sets into single dimensional value which represents the automated task completeness scores of the particular user activity  A min-max normalization is applied that provides us a uniform range of the variables using $
z_i=\frac{x_i-min(x)}{max(x)-min(x)}$ equation where $x=\{x1,\ldots,x_n\}$ and $z_i$ is $i^{th}$ normalized data  The final single dimensional score represents machine learning based TS score 
\section{Physiological Sensor Signals Processing}
The autonomic nervous system (ANS) restrains the body's physiological activities including the heart rate, skin gland secretion, blood pressure, and respiration  The ANS is divided into sympathetic (SNS) and parasympathetic (PNS) branches  While SNS actuates the body's resources for action under arousal conditions, PNS attenuates the body to help regain the steady state  Mental arousal (say stress, anxiety etc ) activates the sweat gland causing the increment and reduction of  Skin Conductance on SNS and PNS physiological conditions respectively  However, Instant Heart Rate also has similar effect on SNS and PNS physiological condtions i e , a higher value of heart rate is the effect of SNS and lower value is the outcome of PNS  EDA and PPG sensors are widely used to estimate the instant value of skin conductance and heart rate respectively \cite{alam16} 
\subsection{EDA Sensor Signal Processing}
EDA is the property of the human body that causes continuous variation in the electrical characteristics of the skin which varies with the state of sweat glands in the skin  There are three types of arousal: \emph{cognitive, affective and physical}  \emph{Cognitive} arousal occurs when a person tries to solve any problem using her cognitive ability  \emph{Affective} arousal occurs when a person is worried, frightened or angry either doing daily activities or in resting position  On the other hand, \emph{physical} arousal is related to the brain command to move bodily parts which is imposed on the total arousal as an artifact, called \emph{motion artifact}  However, there are always some noises due to the weather conditions (temperature, humidity etc ) and device motion  This \emph{motion artifact} can be the prime cause of signal contamination of physiological outcomes while performing daily activities which must be removed  \emph{AutoCogniSys} proposes an EDA sensor signal processing method consists of three steps: (i) noise and motion artifacts removal, (ii) separation of tonic component and phasic component (explained later) from contamination free EDA signal and (iii) feature extraction on the response window 
\subsubsection{Motion Artifacts Removal}
There are many types of motion artifacts but the unsual steep rise is the mostly occured ones associated with EDA signal while performing daily activities \cite{edel67}  We use well-known steep rising noises reduction technique, SWT \cite{chen15}  We first consider EDA signal as a mixture of a slow variant tonic and fast variant phasic component, i e , SWT coefficient is modeled as a mixture of two Gaussian components, phasic (close to zero valued signal) and tonic (high rising signal)  After expanding EDA signal into multiple levels of scaling and wavelet coefficients, we choose adaptively a threshold limit at each level based on the statistical estimation of the wavelet coefficients' distribution, and employ that on the wavelet coefficients of all levels  Finally, an inverse wavelet transform technique is applied to the thresholded wavelet coefficients to obtain the artifacts free EDA signal  Fig~ \ref{fig:eda_artifact_removal} shows a sample of raw and motion artifacts free EDA signal 

\begin{figure}[!htb]
\begin{center}
\vspace{- 1in}
   \epsfig{file=eda_signal_artifact pdf,height=1 6in, width=3 5in}
\caption{Dashed line represents noisy EDA signal and solid red line represents \emph{AutoCogniSys} proposed motion artifact free EDA signal}
   \label{fig:eda_artifact_removal}
\end{center}
\end{figure}
\subsubsection{Convex Optimization Technique to EDA Deconvolution}
After the motion artifact removal, we consider EDA as the sum of three components for $N$ sample: a slow tonic driver ($t$), fast (compact, bursty) non-negative sparse phasic driver ($r$) and a reminder error term ($\epsilon_r$) 
\begin{equation}
\label{eq:eda_signal}
y = t + r + \epsilon_r
\end{equation}
This additive error $\epsilon_r$ is a White Gaussian Noise  The central problem associated with the deconvolution method is to get tonic $t$ component from the above equation  \cite{greco16} showed that EDA signal deconvolution (separation of tonic, phasic and noise terms from EDA signal) is a quadratic optimization problem and defined tonic component as follows:
\begin{equation}
\label{eq:tonic}
t = Bl + Cd,
\end{equation}
where $B$ is a tall matrix whose columns are cubic $B$-spline basis functions, $l$ is the vector of spline coefficients, $C$ is a $N\times 2$ matrix, $d$ is a $2\times 1$ vector with the offset and slope coefficients for the linear trend  The above equation is subject to the following optimization problem,
\begin{eqnarray}
minimize \frac{1}{2} {||Mq + Bl + Cd- y||}^2_2 +\alpha {||Aq||}_1 + \frac{\lambda}{2} {||l||}^2_2\\
subject\;to\; Aq \geq 0\nonumber
\end{eqnarray}
where $M$ and $A$ are tridiagonal matrices and $q$ is an auxiliary variable  After solving the above equation, we can get the optimal values for $\{q,l,d\}$ that can be used to obtain tonic component from the equation~\ref{eq:tonic}  The reminder of the equation~\ref{eq:eda_signal} ($r+\epsilon_r$) is considered as a mixture of White Gaussian Noise ($\epsilon_r$) and a fast variant phasic component ($r$)  We employ butterworth low-pass filter (5Hz) and hanning smoothing with window size 4 (optimal) to remove $\epsilon_r$ from phasic component ($r$) 
\subsection{PPG Signal Processing}
PPG is used mainly for measuring the oxygen saturation in the blood and blood volume changes in skin  An ideal PPG signal processing must contain the following steps: noise and motion artifacts removal, heart rate detection, heart rate variability estimation and feature extraction 
\subsubsection{PPG Signal Noise and Motion Artifacts Removal}
Similar to EDA signal, PPG signal is also contaminated with motion artifacts and noises  However, unlike EDA signal, PPG produce quasiperiodicity in a time series spectrum \cite{mete30}  We use Periodic Moving Average Filter (PMAF) to remove motion artifacts and noises \cite{lee07}  We first segment the PPG signal on periodic boundaries and then average the $m^{th}$ samples of each period  After filtering the input PPG signal with a 5-Hz $8^{th}$-order Butterworth low-pass filter, we estimate the maximum and minimum value of each period  The mean of each period are obtained from the maximum and minimum values applying the zero crossing method  These points of the means help determine the boundaries of each period  Then, interpolation or decimation is performed to ensure that each period had the same number of samples \cite{lee07}  
\subsubsection{Heart Rate and Heart Rate Variability Estimation}
We first apply PMAF on PPG signal to remove noises and motion artifacts, refine PPG by smoothing the signal using 1-dimensional Gaussian Filter and Convolution, calculate first derivative of the convoluted signal and finally find the differences between two consecutive peak values which is called HRV \cite{sel08}  The occurrences of total peak values (R-peak or beat) in each minute is called Heart Rate (HR) with an unit of Beat Per Minute  The signal value property of HRV and HR are inversely proportional which means the mental arousal that increases HR should decrease HRV in the time segment window  Fig~\ref{fig:ppg_artifact_removal} shows a sample of the noisy and filtered PPG signal and their corresponding Instant Heart Rate 
\begin{figure}[!htb]
\vspace{- 1in}
\begin{center}
   \epsfig{file=ppg_artifact_removal pdf,height=1 4in, width=3 5in}
   \vspace{- 15in}
\caption{Top figure illustrates the noisy signal (dotted line) and filtered signal from PPG sensor based on our filtering method  Bottom figure illustrates instant heart rate calculated from noisy signal (dotted line) and filtered signal}
   \label{fig:ppg_artifact_removal}
\end{center}
\vspace{- 15in}
\end{figure}
\subsection{Physiological Sensor Signal Feature Extraction}
Using the above mentioned methods, we removed the noises and motion artifacts from EDA and PPG signals and generated two time series signal from EDA (tonic and phasic components) and one time series signal from PPG (HRV)  Then, we segment each of the time series signal based on our prior detected complex activities such that each response window starts and ends with the starting and ending points of each complex activity  We extract 7 statistical time-series features for EDA (as shown in Table~\ref{tab:eda_features}) and 8 features for HRV (Table~\ref{tab:hrv_features}) within the response window) 

\begin{table}[!t]
\begin{center}

\renewcommand{\arraystretch}{1}
\caption{EDA Features Within The Response Window}
\begin{scriptsize}


\label{tab:eda_features}
\begin{tabular}{|c|l|}
\hline
\bfseries Features& \bfseries Description\\
\hline
nSCR & Number of SCRs within response window (wrw)\\
\hline
Latency & Response latency of first significant SCR wrw\\
\hline
AmpSum & Sum of SCR-amplitudes of significant SCRs wrw\\
\hline
SCR & Average phasic driver wrw\\
\hline
ISCR & Area (i e  time integral) of phasic driver wrw\\
\hline
PhasicMax & Maximum value of phasic activity wrw\\
\hline
Tonic & Mean tonic activity wrw\\
\hline
\end{tabular}
\end{scriptsize}
\end{center}
\end{table}



\begin{table}[!t]
  \begin{center}
\renewcommand{\arraystretch}{1}
\vspace{- 3in}
\caption{Heart Rate Variability features}
\label{tab:hrv_features}
\begin{scriptsize}
\begin{tabular}{|c|l|}

\hline
\bfseries Feature& \bfseries Description\\
\hline
$\overline{RR}$&Mean RR intervals\\
\hline
SDNN&Standard deviation of RR intervals\\
\hline
SDSD&Std of successive RR interval differences\\
\hline
RMSSD&Root mean square of successive differences\\
\hline
NN50&\#successive intervals differing more than 50 ms\\
\hline
pNN50&relative amount of NN50\\
\hline
HRVTI&Total number of RR intervals/height of the histogram\\
\hline
TINN&Width of RR histogram through triangular interpolation\\
\hline
\end{tabular}
\end{scriptsize}
  \end{center}
\end{table}
\section{Experimental Evaluation}
In this section, we explain our data collection, available benchmark dataset, baseline methods and evaluation 
\subsection{Datasets and Baseline Methods}
We validate and compare \emph{AutoCogniSys} with baseline methods on both publicly available and our collected datasets 
\subsubsection{RCC Dataset: Collection and Ground Truth Annotation}
For collecting Retirement Community Center Dataset (RCC Dataset), we recruited 22 participants (19 females and 3 males) with age range from 77-93 (mean 85 5, std 3 92) in a continuing care retirement community with the appropriate institutional IRB approval and signed consent  The gender diversity in the recruited participants reflects the gender distribution (85\% female and 15\% male) in the retirement community facility  A trained gerontology graduate student evaluator completes surveys with participants to fill out the surveys  Participants are given a wrist band to wear on their dominant hand, and concurrently another trained IT graduate student have the IoT system setup in participants' own living environment (setup time 15-30 minutes)  The participants are instructed to perform 13 \emph{complex ADLs}  Another project member remotely monitors the sensor readings, videos and system failure status  The entire session lasts from 2-4 hours of time depending on participants' physical and cognitive ability 

We follow the standard protocol to annotate demographics and activities mentioned in the IRB  Two graduate students are engaged to annotate activities (postural, gestural and complex activity) whereas the observed activity performances are computed by the evaluator  Two more graduate students are engaged to validate the annotations on the videos  In overall, we are able to annotate 13 complex activities (total 291 samples) labeling for each participant; 8 hand gestures (total 43561 samples) and 4 postural activities (total 43561 samples) labeling  Annotation of postural and complex activities outcomes no difficulties from recorded videos  However, annotation of hand-gestures is extremely difficult in our scenario  We used video based hand tracker that can track and sketch wrist movements from a video episode \cite{hugo14}  This sketching can help us significantly to identify which particular hand gesture is being performed in the time segment 
\subsubsection{EES Datasets: EDA and PPG Sensor Datasets}
We used Eight-Emotion Sentics (EES) dataset to validate \emph{AutoCogniSys} proposed physiological signal processing approaches \cite{picard01}  The dataset consists of measurements of four physiological signals (PPG/Blood Volume Pulse, electromyogram, respiration and Skin Conductance/EDA) and eight affective states (neutral, anger, hate, grief, love, romantic love, joy, and reverence)  The study was taken once a day in a session lasting around 25 minutes for 20 days of recordings from an individual participant  We consider only PPG and EDA for all of the affective states in our study 
\subsubsection{Baseline Methods}
Though no frameworks ever combined all modalities together into real-time automated cognitive health assessment, we evaluate \emph{AutoCogniSys} performance by comparing the performances of its components individually with upto date relevant works  For hand gesture and postural activity recognition, we consider \cite{alam17} proposed method as baseline  For complex activity recognition, we compare our hand gesture and postural activity classifiers aided HDBN model with three-level Dynamic Bayesian Network \cite{zhu12} framework  For activity performance estimation, activity performance based cognitive health assessment; and EDA and PPG based cognitive health assessment, we have considered \cite{alam16} proposed method as baseline 
\subsection{Activity Recognition Evaluation}
The standard definition for \emph{accuracy} in any classification problem is $\frac{TP+TN}{TP+TN+FP+FN}$ where $TP,TN,FP$ and $FN$ are defined as true positive, true negative, false positive and false negative  For complex activity recognition evaluation, we additionally consider \emph{start/end duration error} as performance metric that can be explained as follows: consider that the true duration of ``cooking'' is 30 minutes (10:05 AM - 10:35 AM) and our algorithm predicts 29 minutes (10 10 - to 10 39 AM)  Then, the start/end duration error is 9 minutes ($|$5 minutes delayed start$|$ + $|$4 minutes hastened end$|$), in an overall error of e g , 30\% (9/30=0 3)  We measure cross-participant accuracy using leave-two-participants-out method for performance metrics, i e , we take out two of the participants' data points from the entire dataset, train our proposed classification models, test the model accuracy on the two left-out participants relevant data points, and continue the process for entire dataset 

\begin{figure*}[!htb]
\begin{minipage}{0 45\textwidth}
\begin{center}
   \epsfig{file=hand_gesture_accuracy pdf,height=1 6in, width=3in}
\caption{Feature Weighted Naive Bayes (FWNB) classification accuracy comparisons with baseline approaches (graphical signatures of all hand gestures are shown) }
   \label{fig:hand_gesture_accuracy}
\end{center}
\end{minipage}
\begin{minipage}{0 29\textwidth}
\begin{center}
\vspace{- 12in}
   \epsfig{file=posture_accuracy_normal pdf,height=1 6in, width=2 1in}
\caption{4-class postural level activity recognition performance and comparisons with baseline method}
   \label{fig:posture_accuracy_normal}
\end{center}
\end{minipage}
\begin{minipage}{0 25\textwidth}
 \begin{center}
 \vspace{- 12in}
   \epsfig{file=posture_accuracy_extended pdf,height=1 6in, width=2 1in}
\caption{6-class diverse postural activity recognition framework accuracy comparisons with the baseline approach }
   \label{fig:posture_accuracy_extended}
\end{center}
 \end{minipage}
\end{figure*}

Fig~\ref{fig:hand_gesture_accuracy} displays Feature Weighted Naive Bayes (FWNB) based the 8-hand gestural activity recognition accuracies comparisons with the baseline methods which clearly depicts the outperformance of our method (5\% improvement) with an overall accuracy of 92\% (FP rate 6 7\%) in RCC dataset  For postural activity recognition, dataset achieving 91\% postural activity recognition accuracy (FP rate 9 5\%) which outperforms the baseline approach significantly (8\% improvement)  Now, we expand the postural activities for RCC datasets into 3 diverse `walking' postures: `normal walking', `walking with walker', `walking with single stick' and the accuracy goes down to 88\% (FP 7 9\%)  Fig ~\ref{fig:posture_accuracy_normal} and Fig ~\ref{fig:posture_accuracy_extended} illustrate 4-class postural and extended 6-class postural classifier accuracies respectively which clearly posit that \emph{AutoCogniSys} outperforms in each case of postural activities as well as overall performances (8\% and 7\% improvement respectively) 

For complex activity classification, we choose RCC dataset to train our HDBN model  Our leave-two-participants out method results an accuracy of 85\% (FP Rate 3 6\%, precision 84 2\%, recall 84 5\%, ROC Area 98 2\%) with a start/end duration error of 9 7\%  We run the entire evaluation for baseline complex activity recognition algorithm too achieving an overall accuracy of 78\% (FP Rate 5 2\%, precision 79 6\%, recall 78 5\%, ROC Area 82 7\%) which is clearly lower performed method than our approach  Fig  \ref{fig:complex_activity_roc} and Fig~\ref{fig:complex_activity_accuracy} illustrate the ROC curve and each complex activity recognition accuracy comparisons with baseline method which depict the outperformance of our framework over baseline methods (7\% improvement)  Fig~\ref{fig:complex_activity_accuracy} also shows that inclusion of postural activity improves the final complex activity recognition (4\% improvement) 
 \begin{figure}  [!htb]
  \begin{minipage}{0 15\textwidth}
 \begin{center}
   \epsfig{file=complex_activity_roc pdf,height=1 4in, width=1 1in}
\caption{ROC curve for complex activity recognition}
   \label{fig:complex_activity_roc}
\end{center}
\end{minipage}
\begin{minipage}{0 33\textwidth}
\begin{center}

   \epsfig{file=complex_activity_accuracy pdf,height=1 4in, width=2 3in}
\caption{Complex ADLs recognition accuracy improvement and comparison with baseline \cite{zhu12} and HMM based method}
   \label{fig:complex_activity_accuracy}
\end{center}

\end{minipage}
\end{figure}

\subsection{Quantification of Performance Score}
To characterize both the qualitative and quantitative health assessment performance scores, we start with four different feature groups ranging from both functional and physiological health measures: (i) observation based activity features, (ii) automatic activity performance features, (iii) EDA features and (iv) PPG features 

In \emph{observation based activity features}, we design a complex activity set comprised of multiple subtasks which are involved with task {\it interruption, completion and sequencing}  Participants are instructed to perform the complex activities while the trained evaluator observed the aforementioned functional activity performance measures  Each incorrect attempt of performance measure will be assigned one point thus higher score reflects lower performance of functional activities \cite{dawadi14}  We first detect hand gesture and postural activities  Then, we feed the low-level activity contexts (gestural and postural) combined with ambient contexts (object and ambient motion sensor readings) into HDBN for single inhabitant model \cite{alam16b} to recognize complex activities  The complex activity recognition framework provides both activity labels and activity window (start-end points)  Then, we extract features of object sensor, ambient sensor, gestural activity and postural activity events for each activity window  The features are number of occurrences, mean number of occurrences, consecutive 1, 2, 3, $\ldots$ 20 occurrences, top 10, 20, 30, $\ldots$, 90 percentile etc (29 features in total)  In \emph{physiological features} we first detect 13 complex activities using HDBN algorithm which provides activity labels and activity window (start-end points), apply noise reduction, motion artifacts removal, extract 7 EDA features and 8 HRV features for each activity and take the mean of them over time (minutes) to get 15 (7+8) complex activity physiological features set for each participant  In summary, we extract 3 observation based activity features, 29 automatic activity performance features, 7 EDA features and 8 HRV features 
\subsection{Physiological Signal Processing Performance Evaluation}
Standard evaluation technique should use both experimental and publicly available datasets to confirm the outperformance of the novel approaches  We first evaluate our physiological signal processing techniques using a publicly available dataset (EES Dataset \cite{picard01}) to detect 8 human emotions  Then, in next section, we evaluate our methods in assessing cognitive health status of older adults using RCC dataset 

For EDA, we first apply SWT method to remove motion artifacts and noises  Then, we use cvxEDA method to separate tonic and phasic components of EDA  Then, we extract 7 EDA features on a sliding window of 4 seconds  Finally, we feed the 7 EDA features into a SMO based SVM algorithm \cite{cao06}  We use 10-fold cross validation to classify eight emotions achieving 87\% of overall accuracy (FP rate 6\%)  For PPG, we first apply our proposed PMAF based noises and motion artifacts removal technique  Then, we calculate HRV and perform time-domain feature extraction to extract 8 HRV features on a sliding window of 4 seconds  We feed these features into a SMO based SVM algorithm \cite{cao06}  Our 10-fold cross validation shows accuracy of 79\% (FP rate 11 5\%) of detecting 8 emotions on EES Dataset  Fig  \ref{fig:ees_eda} and Fig  \ref{fig:ees_ppg} clearly depict that \emph{AutoCogniSys} proposed EDA and PPG signal processing techniques significantly improve the accuracy over the baseline \cite{alam16} method (10\% and 12\% improvement) 

\begin{figure}[!htb]
\begin{minipage}{0 24\textwidth}
\begin{center}
   \epsfig{file=ees_eda pdf,height=1 2in, width=1 8in}
\caption{(EES Databaset) EDA features based Eight Emotion classification accuracy comparisons with baseline method}
   \label{fig:ees_eda}
\end{center}
\end{minipage}
\begin{minipage}{0 23\textwidth}
\begin{center}
   \epsfig{file=ees_ppg pdf,height=1 2in, width=1 7in}
\caption{(EES Dataset) PPG features based 8-Emotion classification accuracy comparisons with baseline method}
   \label{fig:ees_ppg}
\end{center}
\end{minipage}

\end{figure}
\subsection{Evaluation of Performance Scores}
The feature subsets used in the experimentation for observation and survey based clinical assessments and technology guided physiological and activity initiated health assessments are depicted in Table~\ref{tab:feature_subset}  From our 6 demographics surveys, we find significant distributions in terms of cognition only for SLUMS Score (S-Score)  Based on that, we divide our participants pool into three groups: \emph{Not Cognitively Impaired (NCI), Mild Cognitively Impaired (MCI) and Cognitively Impaired (CI)} where the number of participants are $5$, $7$ and $10$ respectively 
\begin{table}[!t]
\begin{scriptsize}


{\centering 
\renewcommand{\arraystretch}{ 6}
\caption{Feature Subsets}
\label{tab:feature_subset}
\begin{tabular}{|l|L{5 5cm}|}
\hline
\bfseries Feature& \bfseries Description\\
\hline
Observation & Task Completeness (TC), Sequencing (SEQ), Interruptions (INT)\\
\hline
Survey & SLUMS Score (S-Score), ZUNG Score (Z-Score), IADL Score (I-Score), Yale Score (YPAS), Barthel Score (B-Score), GDS Score (G-Score)\\
\hline
EDA  and HRV & 7 and 8 Features\\
\hline
Activity Performance& Supervised (TC, SEQ, INT), Unsupervised\\
\hline
Arousal& EDA and HRV features of each complex activity window\\
\hline

\end{tabular}
}
\end{scriptsize}
\end{table}


\begin{figure}[!htb]
\begin{center}
   \epsfig{file=group_correlation pdf,height=1in, width=3 3in}
\caption{\emph{AutoCogniSys} Proposed Method Based Group Correlation analysis ( $r-value$) NCI, MCI and CI represent not cognitive, mild cognitive and cognitively impaired group of population  TC, INT, SEQ, EDA and HRV represent task completeness, interruption scores, sequencing scores, electrodermal activity features and heart rate variability features}
   \label{fig:group_correlation}
\end{center}
\vspace{- 2in}
\end{figure}
\begin{figure}[!htb]
\begin{center}
   \epsfig{file=group_correlation_baseline pdf,height=1in, width=3 3in}
\caption{Baseline \cite{alam16} method based Group Correlation analysis ( $r-value$)}
   \label{fig:group_correlation_baseline}
   \vspace{- 25in}
\end{center}
\end{figure}

\subsection{Statistical Correlation Analysis of Cognitive Health}
We used Pearson correlation coefficients with significance on $p<0 05$* for individual feature and partial correlation coefficients with significance on $p<0 005$** for group of features correlation analysis  Fig  \ref{fig:group_correlation} and Fig  \ref{fig:group_correlation_baseline} show the group correlation analysis results based on \emph{AutoCogniSys} proposed framework and baseline \cite{alam16} framework respectively  It can be clearly depicted that our proposed framework improves the correlation with the ground truths 
\subsection{Machine Learning Classification of Cognitive Health}
We evaluate using machine learning classifiers to predict cognitive status of older adults using both individual modalities and combined features  We use leave-two-participants out method to train and test classification accuracy 

We first choose the individual activity features (machine learning method based interruption scores, sequencing scores, unsupervised scores) and their combined features to train and test cognitive impairment status classification for SMO based SVM algorithm \cite{cao06}  The classification accuracies are 72\%, 69\%, 76\% and 83\% respectively  Then we consider 7 EDA-activity features and 8 HRV-activity features individually in training and testing phase of SMO based SVM algorithm \cite{cao06} resulting 85\% and 80\% accuracy respectively 

\begin{figure}[!htb]
\begin{minipage}{0 24\textwidth}
\begin{center}
   \epsfig{file=combined_classification pdf,height=1 2in, width=1 7in}
   \vspace{- 15in}
\caption{Individual and combined classification accuracies comparison with baseline method for cognitive impairment status detection}
   \label{fig:combined_classification}
\end{center}
\end{minipage}
\begin{minipage}{0 23\textwidth}
\begin{center}
   \epsfig{file=each_activity_cognitive_assessment pdf,height=1 2in, width=1 7in}

\caption{Machine learning based cognitive health assessment accuracy for each complex activity in terms of activity, EDA and HRV features }
   \label{fig:each_activity_cognitive_assessment}
\end{center}
\end{minipage}
\end{figure}

For combined classifier, we first applied sequential forward feature selection to find the best combinations of 1- 3 features for cognitive impairment classification group MCI, NCI and CI in terms of combined activity features (29 features), EDA-activity features (7 features) and HRV-activity features (8) features  Our final combined classifier (SMO based SVM algorithm \cite{cao06}) provides an accuracy of {\bf 93\%} in detecting the cognitive impairment status of older adults  Fig  \ref{fig:combined_classification} shows our proposed individual and combined methods outperform the baseline \cite{alam16} significantly (13\% improvement)  Fig  \ref{fig:each_activity_cognitive_assessment} shows the cognitive impairment status prediction accuracy for each modality (activity feature, EDA and HRV) per individual complex activity 
\subsection{Discussion}
If we exclude the postural activities from automated activity performance scoring, we find reduced statistical correlation with original task completeness performance for \{NCI, MCI, CI\} participant group (INT 0 53*, SEQ 0 21' and unsupervised 0 49')  However, if we skip our proposed motion artifact removal stage, we find reduced statistical correlation with \{NCI, MCI\} and \{MCI, CI\} groups of participants (EDA and HRV correlations respectively \{0 51*, -0 51*\} and \{-0 53*,0 46\})  To test our proposed motion artifacts removal impact on EDA signals more rigorously, we choose 5 random participants, engage one expert motion artifact annotator to annotate motion artifacts segment on each participant's first 30 minutes of complex dataset using recorded video and apply both baseline and our methods to detect motion artifact segments  While baseline method achieves 75 5\% (FP rate 20 3\%) accuracy in detecting motion artifact segments, \emph{AutoCogniSys} outperforms achieving 89 9\% (FP rate 8 9\%) accuracy  In terms of experience, we have seen 100\% acceptance of wearing wrist-band,  71\% of acceptance for signing consent on using cameras and 0\% failure rate of collecting continuous data 
\section{Conclusion}
We propose, \emph{AutoCogniSys}, an IoT inspired design approach combining wearable and ambient sensors embedded smart home design, extensive signal processing, machine learning algorithms and statistical analytics to automate cognitive health assessment in terms of complex activity performances and physiological responses of daily events  Additionally, our postural activity detection approach in diverse population cum improved activity performance measurement and fundamental physiological sensor artifacts removal from physiological sensors help facilitate the automated cross-sectional cognitive health assessment of the older adults  Our efficient evaluation on each modality (physical, physiological and ambient) and each activity mode proves that any of the mode (say single activity and single sensor) also can provide significant improved cognitive health assessment measure 




Context 2: Paper Info

Title: Compressed quantum error mitigation
Publish Date: 10 May 2023
Author List: Maurits Tepaske (from Physikalisches Institut, Universität Bonn), David Luitz (from Physikalisches Institut, Universität Bonn)

Figure

FIG 3 The out-of-time-ordered correlator C otoc i=L/2,j (t) as a function of the operator position j and time t, for the infinite temperature initial state, for a denoised second-order Trotter supercircuit with Trotter depth Mtrot = 32 and denoiser depth M = 2 We consider evolution times t = 0 5, 1,    , 5, for the periodic L = 14 Heisenberg chain that is affected by two-qubit depolarizing noise with p = 0 01 
FIG  4  The complex eigenvalues λ of the noisy second-order Trotter supercircuit with Mtrot = 16 at time t = 1 (left), the corresponding optimized denoiser with M = 4 (center), and the denoised Trotter supercircuit (right) The Trotter circuit is for a L = 6 Heisenberg model with PBC, and all twoqubit channels are affected by depolarizing noise with p = 0 0046 The unit circle, on which unitary eigenvalues must lie, is shown in black, and the noiseless eigenvalues are shown as blue bars It is evident that the denoiser recovers all the noiseless eigenvalues from the noisy circuit 
FIG  2  The complex eigenvalues λ of the noisy second-order Trotter supercircuit with Mtrot = 16 at time t = 1 (left), the corresponding optimized denoiser with M = 4 (center), and the denoised Trotter supercircuit (right) The Trotter circuit is for a L = 6 Heisenberg model with PBC, and all twoqubit channels are affected by depolarizing noise with p = 0 036 The unit circle, on which unitary eigenvalues must lie, is shown in black, and the noiseless eigenvalues are shown as blue bars It is clear that the denoiser recovers with high accuracy the noiseless eigenvalues from the noisy circuit 
FIG  3  The half-chain channel entanglement entropy S at different two-qubit depolarizing noise strengths p, for a secondorder Trotter supercircuit with Mtrot = 16 and t = 2, for a M = 4 denoiser The Trotter circuit is for a Heisenberg model with PBC of size L = 6 The different curves correspond to the different supercircuits, i e  the noisy supercircuit, the denoiser, the corresponding denoised supercircuit, and the noiseless variant 
FIG  4  The out-of-time-ordered correlator C otoc i=L/2,j (t) as a function of the operator position j and stacked time t, for the infinite temperature initial state, for a denoised secondorder Trotter supercircuit with Trotter depth Mtrot = 32 and denoiser depth M = 2 It is optimized at t = 2 and stacked up to ten times The calculations are for the periodic L = 14 Heisenberg chain that is affected by two-qubit depolarization with p = 0 01 The denoiser is affected by the same noise 
FIG 6 The distribution of the ZZ angle α of M = 2 denoisers (top panels) and M = 8 denoisers (bottom panels), with the lightest color corresponding to the denoiser for the Trotter supercircuit with t = 0 5, and the darkest color with t = 5 As usual, we consider the Heisenberg model on a periodic chain, and second-order Trotter supercircuits with depths Mtrot = 8, 16, 32, 64, which together with the denoiser is affected by a two-qubit depolarizing noise with p = 0 01 The panels are arranged as Mtrot = 8, 16, 32, 64 for top left, top right, bottom left, bottom right, respectively 
FIG  7  The sampling overhead γ of the optimized denoisers from Fig  2 of the main text, with denoiser depths M = 1, 2, 4, 6, 8 and Trotter depths Mtrot = 8, 16, 32, 64 at times t = 0 5, 1,    , 5, for the Heisenberg model on a chain with PBC affected by two-qubit depolarizing noise with p = 0 01 The panels are arranged as Mtrot = 8, 16, 32, 64 for top left, top right, bottom left, bottom right, respectively 
FIG 8 The domain wall magnetization Z dw after evolving a periodic density wall |dw |dw * with the denoised second-order Trotter supercircuits D C from Fig 2of the main text These supercircuits have various Trotter depths Mtrot = 8, 16, 32, 64, denoiser depths M = 1, 2, 4, 6, 8, and evolution times t = 0 5, 1,    , 5, for the periodic L = 14 Heisenberg chain that is affected by two-qubit depolarizing noise of strength p = 0 01 The denoiser is affected by the same noise The non-denoised results are labelled with M = 0 and the noiseless results with p = 0 The panels are arranged as Mtrot = 8, 16, 32, 64 for top left, top right, bottom left, bottom right, respectively We see that the denoiser allows us to recover the noiseless behavior 

abstract

We introduce a quantum error mitigation technique based on probabilistic error cancellation to eliminate errors which have accumulated during the application of a quantum circuit  Our approach is based on applying an optimal "denoiser" after the action of a noisy circuit and can be performed with an arbitrary number of extra gates 
The denoiser is given by an ensemble of circuits distributed with a quasiprobability distribution  For a simple noise model, we show that efficient, local denoisers can be found, and we demonstrate their effectiveness for the digital quantum simulation of the time evolution of simple spin chains  Introduction 
-Quantum information processing has been theoretically shown to hold great promises, and quantum algorithms were developed which can in principle achieve an exponential speed-up over their classical counterparts, both for general purpose computing and quantum simulation   However, present day quantum computing prototypes still suffer from significant noise processes which hinder the execution of many potentially groundbreaking quantum algorithms  
Nontrivial quantum algorithms typically require large sequences of quantum gates, each of which introduces dissipation and hence an overall loss of coherence, eventually rendering the results useless  Until quantum error correction becomes practical, quantum error mitigation seems to be more feasible to increase the accuracy of expectation values 
Here the goal is to induce the (partial) cancellation of errors that stem from noisy quantum gates by extending the circuit corresponding to the desired algorithm with an ensemble of gates , sampled from a quasiprobability distribution  The traditional way to accomplish this is with the gatewise method from , where noise is mitigated by inverting the noise channel of each gate separately, i e  the cancellation of errors is performed for each gate on its own 
Here the local noise channel is approximated in a way such that it can be easily inverted analytically, e g  using Pauli twirling   Gates are then sampled from the inverted noise channel by interpreting it as a quasiprobability distribution  Because in this gate-wise approach every noisy gate has to be modified separately, the sign problem is exponentially large in the number of gates, limiting the practicality of the mitigation 
The success of the gate-wise approach resulted in a large body of work concerning these methods , including extensions for simultaneous mitigation of multiple gates by Pauli-twirling entire layers or variationally constructing a mitigating matrix product operator   In principle, errors during the execution of a circuit can propagate and accumulate 
These propagated errors * david luitz@uni-bonn de ≈ C

C

FIG  1  An example of the quantum error mitigation procedure used in this work for the time evolution of the wave function of a spin chain  The ideal second-order Trotter supercircuit C of depth Mtrot = 1 (light blue) is approximated by applying a denoiser D of depth M = 1 (red) to the noisy Trotter supercircuit C (dark blue) 
Because the denoiser is applied after fully executing the noisy Trotter supercircuit, it represents an approximate inverse of the global noise channel with a precision tunable by the depth of the denoiser  can potentially blow up and lead to large errors for the circuit as a whole   Here we introduce a mitigation technique that takes into account the propagation of errors, can be performed with a tunable number of extra gates, and works for non-Clifford local noise channels since the inversion of the accumulated global noise channel is implicit 
We first execute the targeted noisy circuit completely, letting the noise propagate and accumulate, and only afterwards we apply an extra random circuit sampled from a quasiprobability distribution  We call the corresponding ensemble of random circuits a denoiser, and we construct it such that upon averaging the accumulated errors cancel 
Essentially, the denoiser inverts a global noise channel  Since we will construct it as a local brickwall circuit, following the classical preprocessing approach from , we call this compressed quantum error mitigation  Method  -Due to the inevitable coupling of a quantum processor to its environment, every qubit operation is affected by noise 
Therefore, the simplest technique to minimize the impact of the resulting noise is to minimize the number of operations when performing a quantum algorithm  In we showed that many-body time evolution operators can be efficiently compressed into brick-wall circuits with high fidelity per gate  In this Letter, we consider the noise explicitly by treating quantum operations as (generally non-unitary) quantum channels, corresponding to completely positive and trace preserving (CPTP) maps  
For example, instead of a noiseless two-qubit gate G, which acts on a quantum state |ρ in superoperator form as G|ρ = G⊗G * |ρ , we get the noisy channel G = N G, where the noise channel N implements the two-qubit noise   These channels are used to construct a "supercircuit" C = N G i=1 Gi , consisting of N G channels, which is affected by multi-qubit accumulated noise 
This supercircuit encodes an ensemble of circuits   For simplicity, we assume that the noisy channels Gi in each half brickwall layer are lattice inversion and translation invariant, such that we can construct a denoiser with these properties, limiting the number of variational parameters  The purpose of quantum error mitigation is to modify the ensemble of circuits described by C in a way that we can use it to obtain the noiseless expectation values 
In superoperator language, we do this by following the supercircuit C with a denoiser supercircuit D, such that D C is as close to the noiseless supercircuit C = C ⊗ C * as possible  Here C is the target unitary circuit  Because the noise channel N is non-unitary, hence making the supercircuit C non-unitary, we need to use a non-unitary denoiser to retrieve the unitary C 
We illustrate the mitigation procedure in Fig  , where a denoiser with one layer is used to mitigate errors for a second-order Trotter supercircuit with one layer  This circuit architecture is commonly used to simulate the time evolution of a quantum many-body system, until some time t, with controllable precision , and we will use it to benchmark the denoiser 
In practice, we cannot directly implement a supercircuit, and so we have to utilize its interpretation as an ensemble of circuits  Essentially, after executing a shot of the noisy circuit we sample the denoiser and apply it  The goal is to construct the denoiser in a way that averaging over many of its samples cancels the accumulated errors and gives us a good approximation of the noiseless expectation values 
It should be noted that our approach requires more gate applications on the quantum processor than with the gate-wise scheme, since there each sample from the mitigation quasiprobability distribution can be absorbed into the original circuit, whereas our approach increases the circuit depth  We take this into account by imposing the same noise on the denoiser 
Furthermore, within our scheme, the dimensionality of the quasiprobabilistic mitigating ensemble can be controlled, in contrast to the gate-wise approach where it is equal to the gate count  To facilitate the stochastic interpretation we parameterize each two-qubit denoiser channel G i as a sum of CPTP maps, such that we can sample the terms in this sum and execute the sampled gate on the quantum processor 
Concretely, we use a trace preserv-ing sum of a unitary and a non-unitary channel  For the unitary part we take a two-qubit unitary channel U( φ i ) = U ( φ i ) ⊗ U * ( φ i ), with U ( φ i ) a two-qubit unitary gate parameterized by φ i   For this we take the two-qubit ZZ rotation exp(−iα(σ z ⊗ σ z )) with angle α, which can be obtained from native gates on current hardware , and dress it with four general one-qubit unitaries, only two of which are independent if we want a circuit that is space inversion symmetric around every bond 
The resulting gate has 7 real parameters φ i   For the non-unitary part, which is essential because D has to cancel the non-unitary accumulated noise to obtain the noiseless unitary circuit, we use a general onequbit measurement followed by conditional preparation channel M( , with V a general one-qubit unitary and each κ i a 3-dimensional vector, resulting in a real 9-dimensional ζ i  
This yields the two-qubit correlated measurement M( With these parts we construct the parameterization with coefficients η i ∈ R that satisfy η 0 + η 1 = 1 because G i is trace preserving  Note that here the tensor product symbol corresponds to combining two one-qubit channels to make a two-qubit channel, whereas in most of the paper it is used to link the column and row indices of a density matrix 
We construct the denoiser from the noisy channels Gi = N G i   With this parameterization one denoiser channel has 17 independent real parameters, such that a denoiser of depth M , i e  consisting of M brickwall layers, has 34M real parameters (we use one unique channel per half brickwall layer)  For reference, a general channel has 544M parameters 
To determine the mitigated expectation values we use the full expression where |ρ 0 is the initial state and |1 is the vectorized identity operator on the full Hilbert space  To evaluate this on a quantum processor, we use the stochastic interpretation of (1) to resample   In particular, from each channel (1) we get a unitary with probability p 0 = |η 0 |/γ and a measurement followed by conditional preparation with probability p 1 = |η 1 |/γ 
Here γ = |η 0 | + |η 1 | is the sampling overhead, which characterizes the magnitude of the sign problem from negative η i   For quasiprobability distributions, i e  with γ > 1, every denoiser sample has an extra sign sgn(η) = N G g=1 sgn(η g ), 2  The normalized distance between the denoised Trotter supercircuit D C and the noiseless Trotter supercircuit C (top panels), at evolution times t = 0 5, 1,    , 5, and the twopoint z-spin correlator C zz i=L/2,j=L/2 (t) of a spin on the middle site at times 0 and t (bottom panels), for the infinite temperature initial state 
We consider denoisers with depths M = 1, 2, 4, 6, 8 and second-order Trotter circuits with depths Mtrot = 16, 32, 64  In the top panels we use a Heisenberg chain with L = 8, and in the bottom panels with L = 14, both with periodic boundary conditions  All gates are affected by two-qubit depolarizing noise with p = 0 01



ground_truth: Wearable sensors.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: Can the denoiser be applied to circuits with non-Clifford noise?

Read the following text and answer briefly.

Context 2: 
The non-denoised results are labelled with M = 0, and the noiseless values with p = 0  where sgn(η g ) is the sign of the sampled coefficient of the gth channel  γ = 1 means that all signs are positive  Observables Ô p=0 for the noiseless circuit are then approximated by resampling the observables from the denoiser ensemble
where γ = N G g=1 γ g is the overall sampling overhead, with γ g the overhead of the gth gate  Clearly, a large γ implies a large variance of Ô p=0 for a given number of samples, with accurate estimation requiring the cancellation of large signed terms  The number of samples required to resolve this cancellation of signs is bounded by Hoeffding's inequality, which states that a sufficient number of samples to estimate Ô p=0 with error δ at probability 1 − ω is bounded by (2γ 2 /δ 2 ) ln(2/ω)  
Since γ scales exponentially in γ g , it is clear that a denoiser with large M and γ 1 will require many samples  We observed that decompositions with γ > 1 are crucial for an accurate denoiser  Restricting to γ = 1 leads to large infidelity and no improvement upon increasing the number of terms in or the depth M of the denoiser 
Simply put, probabilistic error cancellation of gate noise introduces a sign problem and it is crucial to find optimal parameterizations (1) which minimize γ to make the approach scalable  This issue arises in all high performance error mitigation schemes , because the inverse of a physical noise channel is unphysical and cannot be represented as a positive sum over CPTP maps 
This is clearly visible in the spectra of the denoiser, which lies outside the unit circle (cf  Fig  )  This makes the tunability of the number of gates in each denoiser sample a crucial ingredient, which allows control over the sign problem, because we can freely choose the η i in   For the parametrization (1) of denoiser channels, we try to find a set of parameters for error mitigation by minimizing the normalized Frobenius distance between the noiseless and denoised supercircuits
which bounds the distance of output density matrices and becomes zero for perfect denoising  We carry out the minimization of on a classical processor, using gradient descent with the differential programming algorithm from   Instead of explicitly calculating the accumulated global noise channel and subsequently inverting it, we approximate the noiseless supercircuit C with the denoised supercircuit D C, effectively yielding a circuit representation D of the inverse noise channel 
Results  -To benchmark the denoiser we apply it to the second-order Trotter circuits of the spin-1/2 Heisenberg chain with periodic boundary conditions (PBC) where is the Pauli algebra acting on the local Hilbert space of site i  A second-order Trotter circuit for evolution time t with depth M trot consists of M trot − 1 half brickwall layers with time step t/M trot and two layers with half time step  
We consider circuits that are affected by uniform depolarizing noise with probability p for simplicity, but our approach can be used for any non-Clifford noise  The two-qubit noise channel is which acts on neighboring qubits i and i + 1 and is applied to each Trotter and denoiser gate, and p = 0 01 unless stated otherwise 
We study circuits with depths M trot = 16, 32, 64 for evolution times t = 0 5, 1,    , 5, and denoisers D with depths M = 1, 2, 4, 6, 8  In the top panels of Fig  we show (4) for a chain of size L = 8 as a function of time t  Here it can be seen that even for M trot = 32 a denoiser with M = 1 already improves by roughly an order of magnitude at all considered t 
Depending on M trot and t, further increasing M lowers , with the biggest improvements occurring for high precision Trotter circuits with large depth M trot = 64 and short time t = 0 5, where the Trotter gates are closer to the identity than in the other cases  At the other extreme, for M trot = 16 the improvements are relatively small upon increasing M > 2  In all cases the denoiser works better at early times than at late times, again indicating that it is easier to denoise Trotter gates that are relatively close to the identity 
To probe the accuracy of the denoiser on quantities that do not enter the optimization, as a first test we consider the two-point correlator between spins at different times where we have chosen the infinite temperature initial state, and C(t) is the Trotter supercircuit for time t  In the bottom panels of Fig  we show C zz i=L/2,j=L/2 (t) for the supercircuits from the upper panels, now for a L = 14 chain 
Here we see that at M trot = 16 we can retrieve the noiseless values already with M = 1, but that increasing M trot makes this more difficult  At M trot = 64 we see larger deviations, and improvement upon increasing M is less stable, but nonetheless we are able to mitigate errors to a large extent  As a further test, we compute the out-of-time-ordered correlator (OTOC) ]
In Fig  we show the results for i = L/2, for a Trotter circuit with depth M trot = 32 and a denoiser with depth M = 2  Here we see that a denoiser with M M trot is able to recover the light-cone of correlations, which are otherwise buried by the noise  In the Supplementary Material we consider how the denoiser performs at different noise levels p, and how the denoised supercircuits perform under stacking 
There we also calculate domain wall magnetization dynamics, and show the distribution of the optimized denoiser parameters and the sampling overhead associated to the denoiser as a whole  In Fig  we show the eigenvalues of the noisy supercircuits for a noisy second-order Trotter supercircuit with M trot = 16 at t = 1 (left), the corresponding optimized denoiser with M = 4 (center), and the denoised supercircuit (right) 
The eigenvalues λ of a unitary supercircuit lie on the unit circle, and in the presence of dissipation they are pushed to the center  We see that the spectrum of the denoiser lies outside the unit circle, making it an unphysical channel which cures the effect of the noise on the circuit, such that the spectrum of the denoised circuit is pushed back to the unit circle 
The noiseless eigenvalues are shown as blue bars, making it clear that the denoiser is able to recover the noiseless eigenvalues from the noisy circuit  In the Supplementary Material we show the spectra for a p = 0 036 denoiser, where we observe a clustering of eigenvalues reminiscent of Refs    There we also investigate the channel entropy of the various supercircuits  
Conclusion  -We have introduced a probabilistic error cancellation scheme, where a classically determined denoiser mitigates the accumulated noise of a (generally non-Clifford) local noise channel  The required number of mitigation gates, i e  the dimensionality of the corresponding quasiprobability distribution, is tunable and the parameterization of the corresponding channels provides control over the sign problem that is inherent to probabilistic error cancellation 
We have shown that a denoiser with one layer can already significantly mitigate errors for second-order Trotter circuits with up to 64 layers  This effectiveness of low-depth compressed circuits for denoising, in contrast with the noiseless time evolution operator compression from , can be understood from the non-unitarity of the denoiser channels 
In particu-lar, measurements can have non-local effects, since the measurement of a single qubit can reduce some highly entangled state (e g  a GHZ state) to a product state, whereas in unitary circuits the spreading of correlations forms a light-cone  To optimize a denoiser with convenience at L > 8, the optimization can be formulated in terms of matrix product operators or channels , which is convenient because the circuit calculations leading to the normalized distance and its gradient are easily formulated in terms of tensor contractions and singular value decompositions  
This provides one route to a practical denoiser, which is relevant because the targeted noiseless circuit and the accompanying noisy variant in (4) need to be simulated classically, confining the optimization procedure to limited system sizes with an exact treatment or limited entanglement with tensor networks 
Nonetheless, we can use e g  matrix product operators to calculate (4) for some relatively small t, such that the noiseless and denoised supercircuits in (4) have relatively small entanglement, and then stack the final denoised supercircuit on a quantum processor to generate classically intractable states 
Analogously, we can optimize the channels exactly at some classically tractable size and then execute them on a quantum processor with larger size  Both approaches are limited by the light-cone of many-body correlations, as visualized in Fig  , because finite-size effects appear when the light-cone width becomes comparable with system size 
1  The normalized distance (left) and z spin correlator C zz i=L/2,j=L/2 (right), for a second-order Trotter supercircuit of depth Mtrot = 16 for time t = 1, affected by various twoqubit depolarizing errors p  We compare the values obtained with and without a denoiser, i e  M > 0 and M = 0, to the noiseless values (p = 0) 
The denoiser is affected by the same noise as the Trotter circuit  We consider denoisers with depths M = 1, 2, 4, 6, 8, and we use a L = 8 Heisenberg chain with PBC for the normalized distance, while for the correlator we use L = 14  * david luitz@uni-bonn de to observe that even for larger noise strength p, the local observable C zz improves significantly even with denoisers of depth M = 1 
For large noise strengths, we generally see that the optimization of the denoiser becomes difficult, leading to nonmonotonic behavior as a function of p, presumably because we do not find the global optimum of the denoiser  It is interesting to analyze the spectra of the supercircuits considered in this work 
As mentioned in the main text, the spectrum of the ideal, unitary supercircuit C lies on the unit circle  The comparison to this case is therefore instructive  In the main text, we showed an example of the spectra in Fig  for moderate noise strength  Here, we show additional data for stronger noise p = 0 036 in Fig  for a denoiser with M = 4 layers, optimized to mitigate errors for a second-order Trotter supercircuit with M trot = 16 layers at time t = 1 
The eigenvalues λ of the noisy supercircuit C are clustered close to zero, far away from the unit circle (except for λ = 1), showing that the circuit is strongly affected by the noise  To mitigate the impact of the noise, the denoiser consequently has to renormalize the spectrum strongly  If it accurately represents the inverse of the global noise channel, its spectrum has to lie far outside the unit circle, which is the case 
Interestingly, we observe a clustering of eigenvalues which is reminiscent to the spectra found in   By comparison to these works, we suspect that this is due to the local nature of the denoiser, and warrants further investigation  The right panel of Fig  shows the result of the denoiser, pushing the eigenvalues back to the unit circle, nearly with the exact same distribution along the circle as the noiseless eigenvalues (blue bars) 
Due to the strong noise, this is not achieved perfectly, and it is clear that this cannot work in principle if the global noise channel has a zero eigenvalue  The complexity of an operator can be quantified by its operator entanglement entropy   Here we calculate the half-chain channel entanglement entropy S of the noiseless C, noisy C, denoiser D, and denoised D C supercircuits 
We define S as the entanglement entropy of the state that is related to a supercircuit C via the Choi-Jamio lkowski isomorphism, i e  ψ C = χ C /N , where the process matrix χ ab,cd C = C ac,bd is simply a reshaped supercircuit and N ensures normalization  Then we have S = −Tr [ψ C ln ψ C ]  This entropy measure is a particular instance of the "exchange entropy", which characterizes the information exchange between a quantum system and its environment  
In Fig  we plot the various S for a second-order Trotter circuit with M trot = 16 at t = 2, for a denoiser with M = 4, both affected by two-qubit depolarizing noise with p ∈ [10 −3 , 10 −1 ]  The Trotter circuit is for a Heisenberg model with L = 6 and PBC  We see that at large p, the noise destroys entanglement in the noisy supercircuit, and that the denoiser S increases to correct for this, such that the denoised supercircuit recovers the noiseless S 
Here we investigate how denoised supercircuits perform upon repeated application  We optimize the denoiser for a Trotter supercircuit for a fixed evolution time t  Then, to reach later times, we stack the denoised supercircuit n times to approximate the evolution up to time nt: In Fig  we stack a denoised t = 1 supercircuit up to n = 20 times and calculate the correlation function, defined in the main text, for the middle site 
We consider Trotter depths M trot = 8, 16, 32, 64 and denoiser depths M = 1, 2, 4, 6, 8, for a L = 14 Heisenberg chain with p = 0 01 depolarizing two-qubit noise  The noisy results correspond to M = 0 and the noiseless results to p = 0  In Fig  we calculate the OTOC, defined in the main text, with stacked time evolution for a denoised t = 2 supercircuit with M trot = 32 and M = 2, stacked up to ten times 
We see that the stacked supercircuit performs very well, and the additional precision obtained by using deep denoisers (M = 8) pays off for long evolution times, where we see convergence to the exact result (black dashed lines in Fig  ) as a function of M   FIG    The two-point z-spin correlator C zz i=L/2,j=L/2 (t) of a spin on the middle site at times 0 and t, for the infinite temperature initial state, for denoised second-order Trotter supercircuits that are optimized at evolution time t = 1 and then stacked up to twenty times 
We use Trotter depths Mtrot = 8, 16, 32, 64 and denoiser depths M = 1, 2, 4, 6, 8  The calculations were performed for a periodic Heisenberg model with L = 14 and PBC, affected by two-qubit depolarizing noise with strength p = 0 01, which also affects the denoiser  The non-denoised results are labelled with M = 0, and the noiseless results with p = 0 
The panels are arranged as Mtrot = 8, 16, 32, 64 for top left, top right, bottom left, bottom right, respectively  The costliest and most noise-susceptible operation is the two-qubit ZZ rotation with angle α, which is the foundation of the unitary piece in our channel parameterization, defined in the main text 
For completeness, we here present the α angles of the optimized denoisers  The results are shown in Fig  , which contains histograms for the channel count N G versus α  The histograms are stacked, with the lightest color corresponding to the angles of the denoiser at t = 0 5 and the darkest at t = 5  The top four panels are for a denoiser with M = 2 and the bottom four with M = 8 
We consider M trot = 8, 16, 32, 64  We see that in both cases the distribution widens upon increasing M trot , indicating that the unitary channels start deviating more from the identity  Moreover, while the M = 2 denoisers in all cases except M trot = 64 have ZZ contributions close to the identity, this is clearly not the case for M = 8 
For simplicity, we did not focus on obtaining denoisers with the smallest sampling overhead γ, which is required to minimize the sign problem and hence ease the sampling of mitigated quantities  Instead, we let the optimization freely choose the η i in the denoiser parameterization, as defined in the main text 
In Fig  we show the sampling overhead of the denoisers from Fig  of the main text  We see that for M = 1 and M = 2 the sampling overhead is relatively small and uniform across the different t, whereas for M > 2 the optimization sometimes yields a denoiser with large γ and other times with small γ  This could be related to the difference in α distributions from Fig   
The large fluctuations of γ appears to stem from the difficulty in finding optimal deep denoisers, and our optimization procedure likely only finds a local minimum in these cases  Here C(t) is the Trotter supercircuit for time t  In Fig  we show Z dw for the circuits from Fig 



ground_truth: Yes, the denoiser works for non-Clifford local noise channels.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: How does the scoring engine generate a stream of content for the channel?

Read the following text and answer briefly.

Context 3: A system and method for generating a stream of content for a channel  The channel application includes a content categorizer, a scoring engine and a channel engine  The content categorizer categorizes new content items received from heterogeneous data sources  The channel engine identifies a channel category for a user based at least in part on at least one of a historical trend and a user activity  The scoring engine queries the new content items based on the channel category and at least one other channel attribute  The scoring engine retrieves candidate content items that include the channel category and the other channel attribute  The scoring engine then generates a stream of content from the candidate content items for the channel 
This application claims priority under 35 USC §120 to U S  application Ser  No  13/225,209, entitled, “Generating a Stream of Content for a Channel,” filed on Sep  2, 2011, and claims priority under 35 USC §119(e) to U S  Application No  61/424,636, entitled “Scoring Stream Items with Models Based on User Interests” filed Dec  18, 2010, the entireties of which are herein incorporated by reference 
The specification relates to a system and method for generating a stream of content for a channel  In particular, the specification relates to generating a stream of content for a channel based on user interests and historical trends 
Many consumers of digital media have two somewhat contradictory goals: keep apprised of information in the areas they already find interesting and discover new content that is also enjoyable  Keeping apprised of information can become burdensome in the digital age because there is so much information  Hence, there is a need to present the best and most relevant information, without overwhelming the consumer  Furthermore, consumers have varied interests depending on the time of a year or a day  As a result, there is also a need to cater to the time dependent changes in the consumer's interests while presenting information  Similarly, discovering new content is difficult when the consumer is overburdened with existing content 
Prior attempts to solve these problems allow consumers to create personalized sections in feed aggregation websites that are defined by keywords  Often, these personalized sections present any item that includes the keywords even though the item is not of interest to the consumer, per se  In another method, consumers are allowed to manually subscribe to Really Simple Syndication (RSS) feeds from multiple websites  This method often leads to the consumer viewing multiple items which contain redundant information 
In some examples, the specification describes a system and method for generating a stream of content for a channel using a channel application  The channel application includes a processing unit, a model generation engine, a scoring engine, a collaborative filtering engine, a content categorizer, a channel engine, and a user interface engine  The model generation engine generates a model that is used to determine suggestions for channels  The content categorizer categorizes new content items received from heterogeneous data sources  The channel engine identifies a channel category for a user based on at least one of a historical trend and a user activity  The historical trend is at least one of an increase in a number of new content items for a content category, an increase in a number of times one of the new content items is accessed and an event  A scoring engine queries the new content items based on the channel category and at least one other channel attribute  The scoring engine receives candidate content items that include the channel category and the at least one other channel attribute  The scoring engine then generates a stream of content from the candidate content items for the channel  The scoring engine transmits the stream of content to the channel engine, which generates a channel 
In one embodiment, the user interface engine generates a user interface for the user to define the channel category and the channel attribute  The scoring engine queries the new content items based on the user defined channel category and channel attribute and then generates the stream of content  In another embodiment, the channel engine enables the user to subscribe to an existing channel 
In one embodiment, the channel engine enables the user to share the channel with at least one of a friend of the user, a community, a group, and an internet user 
The specification is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals are used to refer to similar elements 
FIG  1A is a high-level block diagram illustrating one embodiment of a system for generating a stream of content for a channel 
FIG  1B is a block diagram illustrating one embodiment of a channel application 
FIG  2 is a high-level block diagram illustrating another embodiment of a system for generating a stream of content for a channel 
FIG  3A is a block diagram of one embodiment of the channel engine in more detail 
FIG  3B is a block diagram of one embodiment of the scoring engine in more detail 
FIG  4 is a graphic representation of a user interface that displays the stream of content of a channel 
FIG  5 is a graphic representation of a user interface that allows a user to define or customize a channel 
FIG  6 is a flow diagram of one embodiment of a method for generating a stream of content for a channel 
FIG  7 is a flow diagram of another embodiment of a method for generating a stream of content for a channel 
A system and method for generating a stream of content for a channel is described below  In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the specification  It will be apparent, however, to one skilled in the art that the embodiments can be practiced without these specific details  In other instances, structures and devices are shown in block diagram form in order to avoid obscuring the specification  For example, the specification is described in one embodiment below with reference to user interfaces and particular hardware  However, the description applies to any type of computing device that can receive data and commands, and any peripheral devices providing services 
Some portions of the detailed descriptions that follow are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory  These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art  An algorithm is here, and generally, conceived to be a self consistent sequence of steps leading to a desired result  The steps are those requiring physical manipulations of physical quantities  Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated  It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like 
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities  Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices 
The specification also relates to an apparatus for performing the operations herein  This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer  Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, flash memories including USB keys with non-volatile memory, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus 
An embodiment can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements  A preferred embodiment is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc 
Furthermore, an embodiment can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system  For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device 
FIG  1A illustrates a block diagram of a system 100 for generating a stream of content for a channel according to one embodiment  The system 100 includes user devices 115 a, 115 n that are accessed by users 125 a, 125 n, a social network server 101, a third party server 107, a ratings server 139, an email server 141, an entertainment server 137, and a search server 135  The ratings server 139 includes websites for rating places, people or objects (e g  Google Hotpot)  The entertainment server 137 includes websites with entertaining information, such as news articles  In FIG  1A and the remaining figures, a letter after a reference number, such as “115 a” is a reference to the element having that particular reference number  A reference number in the text without a following letter, such as “115,” is a general reference to any or all instances of the element bearing that reference number  In the illustrated embodiment, these entities are communicatively coupled via a network 105 
In one embodiment, the channel application 103 a is operable on the social network server 101, which is coupled to the network via signal line 104  The social network server 101 also contains a social network application 109 and a social graph 179  Although only one social network server 101 is shown, persons of ordinary skill in the art will recognize that multiple social network servers 101 may be present  A social network is any type of social structure where the users are connected by a common feature, for example, Google+  The common feature includes friendship, family, work, an interest, etc  The common features are provided by one or more social networking systems, such as those included in the system 100, including explicitly-defined relationships and relationships implied by social connections with other online users, where the relationships form a social graph 179  In some examples, the social graph 179 reflects a mapping of these users and how they are related 
In another embodiment, the channel application 103 b is stored on a third-party server 107, which is connected to the network via signal line 106  The third-party server 107 includes software for generating a website (not shown)  In one embodiment, the notifying application generates a user interface that is incorporated into the website  Although only one third-party server 107 is shown, persons of ordinary skill in the art will recognize that multiple third-party servers 107 may be present 
In yet another embodiment, the channel application 103 c is stored on a user device 115 a, which is connected to the network via signal line 108  The user device 115 a is any computing device that includes a memory and a processor, such as a personal computer, a laptop, a smartphone, a cellular phone, a personal digital assistant (PDA), etc  The user 125 a interacts with the user device 115 a via signal line 110  Although only two user devices 115 a, 115 n are illustrated, persons of ordinary skill in the art will recognize that any number of user devices 115 n are available to any number of users 125 n 
The network 105 is a conventional type, wired or wireless, and may have any number of configurations such as a star configuration, token ring configuration or other configurations known to those skilled in the art  Furthermore, the network 105 may comprise a local area network (LAN), a wide area network (WAN) (e g , the Internet), and/or any other interconnected data path across which multiple devices may communicate  In yet another embodiment, the network 105 may be a peer-to-peer network  The network 105 may also be coupled to or includes portions of a telecommunications network for sending data in a variety of different communication protocols  In yet another embodiment, the network 105 includes Bluetooth communication networks or a cellular communications network for sending and receiving data such as via short messaging service (SMS), multimedia messaging service (MMS), hypertext transfer protocol (HTTP), direct data connection, WAP, email, etc  While only one network 105 is coupled to the user devices 115 a, 115 n, the social network server 101, and the third party server 107, in practice any number of networks 105 can be connected to the entities 
The channel application 103 receives data for generating a stream of content for a channel from heterogeneous data sources  In one embodiment, the channel application 103 receives data from a third-party server 107, a social network server 101, user devices 115 a, 115 n, a search server 135 that is coupled to the network 105 via signal line 136, an entertainment server 137 that is coupled to the network 105 via signal line 138, a ratings server 139 that is coupled to the network 105 via signal line 140 and an email server 141 that is coupled to the network 105 via signal line 142  In one embodiment, the search server 135 includes a search engine 143 for retrieving results that match search terms from the Internet  In one embodiment, the search engine 143 is powered by Google®  In one embodiment, the channel application 103 generates a model based on the data from the heterogeneous data sources, identifies a channel category based on a user's activities and historical trends, receives candidate content items that include the channel category from heterogeneous data sources, scores the candidate content items by comparing them to the model, and generates a stream of content for the channel 
Referring now to FIG  1B, the channel application 103 is shown in detail  FIG  1B is a block diagram of a computing device 200 that includes the channel application 103, a memory 237 and a processor 235  In one embodiment, the computing 200 device is a social network server 101  In another embodiment, the computing device 200 is a third party server 107  In yet another embodiment, the computing device 200 is a user device 115 a 
The processor 235 comprises an arithmetic logic unit, a microprocessor, a general purpose controller, or some other processor array to perform computations and provide electronic display signals to a display device  The processor 235 is coupled to the bus 220 for communication with the other components via signal line 236  Processor 235 processes data signals and may comprise various computing architectures including a complex instruction set computer (CISC) architecture, a reduced instruction set computer (RISC) architecture, or an architecture implementing a combination of instruction sets  Although only a single processor is shown in FIG  1B, multiple processors may be included  The processing capability may be limited to supporting the display of images and the capture and transmission of images  The processing capability might be enough to perform more complex tasks, including various types of feature extraction and sampling  It will be obvious to one skilled in the art that other processors, operating systems, sensors, displays, and physical configurations are possible 
The memory 237 stores instructions and/or data that may be executed by processor 235  The memory 237 is coupled to the bus 220 for communication with the other components via signal line 238  The instructions and/or data may comprise code for performing any and/or all of the techniques described herein  The memory 237 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, flash memory, or some other memory device known in the art  In one embodiment, the memory 237 also includes a non-volatile memory or similar permanent storage device and media such as a hard disk drive, a floppy disk drive, a CD-ROM device, a DVD-ROM device, a DVD-RAM device, a DVD-RW device, a flash memory device, or some other mass storage device known in the art for storing information on a more permanent basis 
In one embodiment, the channel application 103 comprises a processing unit 202, a model generation engine 207, a scoring engine 211, a collaborative filtering engine 217, a content categorizer 250, a channel engine 240, and a user interface engine 260 that are coupled to a bus 220 
The processing unit 202 is software including routines for receiving information about a user's interests, activities and social connections and for storing the information in the memory 237  In one embodiment, the processing unit 202 is a set of instructions executable by the processor 235 to provide the functionality described below for processing the information  In another embodiment, the processing unit 202 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the processing unit 202 is adapted for cooperation and communication with the processor 235, the model generation engine 207, and other components of the computing device 200 via signal line 222 
The processing unit 202 obtains information about users from user input and/or prior actions of a user across a range of heterogeneous data sources including search (such as web, video, news, maps, alerts), entertainment (such as news, video, a personalized homepage, blogs, a reader, gadget subscriptions), social activity (such as interactions through email, profile information, text messaging such as short message service (SMS), microblogs, geographical locations, comments on photos, a social graph and other social networking information), and activity on third-party sites (such as websites that provide ratings, reviews and social networks where users indicate that they approve of content)  This information is obtained, for example, from a user's search history, browsing history and other interactions with the Internet  The processing unit 202 stores the information with a designation of the source of the information 
In one embodiment, there are multiple processing units 202 that each receive data from a different heterogeneous data source  In another embodiment, the user information is received by the same processing unit 202  The processing unit 202 transmits the user information to memory 237 for storage  In one embodiment, the memory 237 partitions the user information from each heterogeneous data source in a separate data storage location  In another embodiment, the user information from heterogeneous data sources is stored in the same location in the memory 237  In yet another embodiment, the memory 237 partitions the model and the stream of content into separate storage locations as well 
The model generation engine 207 is software including routines for retrieving the user information from the memory 237 and generating a model based on the user information  In one embodiment, the model generation engine 207 is a set of instructions executable by the processor 235 to provide the functionality described below for generating the model  In another embodiment, the model generation engine 207 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the model generation engine 207 is adapted for cooperation and communication with the processor 235, the processing unit 202, the scoring engine 211, the channel engine 240 and other components of the computing device 200 via signal line 224 
The model generation engine 207 receives user information from a variety of sources including, for example, queries, clicks, news clicks, gadgets, email interactions, etc , extracts features from the information and generates a model based on the extracted features  The model determines the relevance of items to users, along with floating point values to indicate the extent to which the relevance holds  Examples include liking a source, a primary location and a list of interests  The interests are generated from explicit information and inferred information  Explicit information is derived, for example, from a user's list of interests on a social network or indicating that they liked a particular content item  Inferred information takes into account a user's activities 
The model generation engine 207 will infer that a user is interested in a particular subject, for example, if the subject matter appears in search terms  For example, the model generation engine 207 infers that a user who searches for information about different types of butterflies is interested in butterflies  The model generation engine 207 can even infer information based on the user's friends' activities  For example, content items that interest the user's friends might also interest the user  As a result, in one embodiment, the model includes the user's friends' interests 
In one embodiment, the model generation engine 207 also generates a model that contains several pieces of global meta-information about the user's consumption patterns including how frequently the user consumes the stream of content of a channel and global statistics on how likely the user is to reshare various types of items  Lastly, the model includes a sequence of weights and multipliers that are used to make predictions about the user's likelihood of clicking on, sharing or otherwise engaging with stream items 
The model generation engine 207 generates the model from the user information across the heterogeneous data sources  In one embodiment, the model generation engine 207 builds extensions to the model that employ the patterns of behavior of other users  For example, the model predicts the user's behavior based on the reaction of similar users  All the data that is derived from other users is anonymized before it is incorporated into the model 
In one embodiment, the model generation engine 207 generates a model based on user information, for example, based on the user's search history or third-party accounts  Alternatively, the model generation engine 207 receives periodic updates (one hour, one day, one week, etc ) from the heterogeneous data sources and in turn updates the model 
In yet another embodiment, the model generation engine 207 generates a model each time it receives a request for generating a stream of content for a channel  The advantage of this method is that the newest updates are included and the model is current  The disadvantage is that generating the model and then comparing the candidate content items to the model to generate the stream of content takes more time than comparing the candidate content items to a pre-existing model  The model generation engine 207 transmits the model to memory 237 for storage 
The content categorizer 250 is software including routines for receiving and categorizing new content items from heterogeneous sources according to at least one category and other features  In one embodiment, the content categorizer 250 is a set of instructions executable by the processor 235 to provide the functionality described below for receiving and categorizing new content items  In another embodiment, the content categorizer 250 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the content categorizer 250 is adapted for cooperation and communication with the processor 235, the scoring engine 211 and other components of the computing device 200 via signal line 227 
The content categorizer 250 receives new content items from heterogeneous data sources and annotates them with specific tags, such as features, global scores, etc  In this embodiment, the heterogeneous data sources include a search engine 143, an entertainment server 137, an email server 141, a ratings server 139, a social network server 101, and a third-party server 107  Once the items are annotated, the content categorizer 250 indexes each new content item based on the features and stores the content items in the memory 237  The new content items, in one embodiment, are indexed according to an identification format (MediaType#UniqueItemID, for example, “YOUTUBE#video_id” and “NEWS#doc_id”), an item static feature column that holds an item's static features (title, content, content classification, context, etc ), an item dynamic feature column that holds an item's dynamic features (global_score, number of clicks, number of following, etc ), a source (src) static feature column where the source is a publisher of an item (magazine in news, video uploading in YouTube, etc ) and a src dynamic feature column that holds the source's dynamic features  The content categorizer 250 categorizes the new content items to make their retrieval more efficient and fast 
The channel engine 240 is software including routines for generating a channel for a user  In one embodiment, the channel engine 240 is a set of instructions executable by the processor 235 to provide the functionality described below for generating a channel for a user  In another embodiment, the channel engine 240 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the channel engine 240 is adapted for cooperation and communication with the processor 235, the scoring engine 211, the model generation engine 207, the user interface engine 240, and other components of the computing device 200 via signal line 230 
In one embodiment, the channel engine 240 identifies a channel category for a user based on historical trends and the user's activities, interests and social connections  The channel engine 240 submits a request for a stream of content that includes the channel category and channel attributes to the scoring engine 211  The channel engine 240 then receives a stream of content from the scoring engine 211 and generates the channel  The generated channel is either public or private depending on the user's settings  The channel engine 240 is explained in greater detail below with regard to FIG  3A 
The scoring engine 211 is software including routines for generating a stream of content for a channel  In one embodiment, the scoring engine 211 is a set of instructions executable by the processor 235 to provide the functionality described below for globally scoring content items and for generating a stream of content for a channel  In another embodiment, the scoring engine 211 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the scoring engine 211 is adapted for cooperation and communication with the processor 235, the processing unit 202, the collaborative filtering engine 217, the model generation engine 207, the channel engine 240 and other components of the computing device 200 via signal line 228 
In one embodiment, the scoring engine 211 receives the request from the channel engine 240 and queries the new content items stored in memory 237  In another embodiment, the scoring engine 211 directly queries the heterogeneous data sources  The scoring engine 211 receives candidate content items that include the channel category and the channel attributes  The scoring engine 211 then compares the candidate content items to the model to determine whether the user would find the candidate content items interesting 
In one embodiment, the scoring engine 211 first performs the query and then compares the results to the model to determine whether the user would find them interesting  In another embodiment, these steps are performed simultaneously  In yet another embodiment, the scoring engine 211 compares candidate content items to the model and then filters the results according to the subject matter of the queries  The scoring engine 211 is explained in greater detail below with regard to FIG  3B 
The collaborative filtering engine 217 is software including routines for generating additional candidate content items for the channel through collaborative filtering and transmitting the additional candidate content items to the scoring engine 211 that were derived from collaborative filtering  In one embodiment, the collaborative filtering engine 217 is a set of instructions executable by the processor 235 to provide the functionality described below for generating additional candidate content items for the channel  In another embodiment, the collaborative filtering engine 217 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the collaborative filtering engine 217 is adapted for cooperation and communication with the processor 235, the scoring engine 211 and other components of the computing device via signal line 226 
The collaborative filtering engine 217 obtains additional candidate content items that are socially relevant from a stream of content derived from people with whom the user has a relationship and transmits them to the scoring engine 211  For example, the stream of content is derived from friends in a social network such as the social network application 109 or people that the user frequently emails  The more important that the person appears to be to the user, the more likely that the user will be interested in the candidate content item  Thus, in one embodiment, the collaborative filtering engine 217 applies a weight to candidate content items based on the social relationship of the user to the friend  For example, users that are friends receive higher weights than candidate content items from second generation friends of the user (i e , a friend of a friend)  In one embodiment, the collaborative filtering engine 217 receives information about relationships between users from the social graph 179 
The collaborative filtering engine 217 increases the weights applied to candidate content items from friends when the user positively responds to the items  For example, if the user comments on the item or indicates that the user found the item interesting, the collaborative filtering engine 217 increase the weight so that more candidate content items from the friend become part of the stream of content 
The user interface engine 260 is software including routines for generating a user interface that, when rendered on a browser, displays a channel generated for a user and enables the user to customize the channel  In one embodiment, the user interface engine 260 is a set of instructions executable by the processor 235 to provide the functionality described below for generating a user interface  In another embodiment, the user interface engine 260 is stored in the memory 237 of the computing device 200 and is accessible and executable by the processor 235  In either embodiment, the user interface engine 260 is adapted for cooperation and communication with the processor 235, the channel engine 240 and other components of the computing device 200 via signal line 232 
The user interface engine 260 receives instructions from the channel engine 240 for generating a display  The user interface includes options for viewing a channel, requesting a new channel, modifying the user interests, and following suggested channels 
FIG  2 is a high-level block diagram illustrating another embodiment of a system for generating a stream of content for a channel  In this embodiment, the components of the channel application 103 are divided among various servers so that the information is efficiently processed  The system includes a search server 135, an entertainment server 137, a ratings server 139, an email server 141, a content categorizer 250, a data storage server 265, a model server 255, a scoring server 262, a social network server 101, a user device 115, and a channel application 103 
A content categorizer 250 crawls the heterogeneous data sources (search server 135, entertainment server 137, ratings server 139, and email server 141) are crawled for new content items by the content categorizer 250 or the new content items are directly transmitted to the content categorizer 250 
The content categorizer 250 categorizes the new content items as mentioned above with regards to FIG  1B and stores them in the database 267 of the data storage server 265  The content categorizer 240 also includes a processing unit 202 for processing user information (activities, interests and social connections)  In one embodiment, the processing unit 202 stores the database 267 
In one embodiment, the data storage server 265 dynamically phases out the old content items  For example, news items expire after 24 hours, videos expire after 48 hours and feeds are kept for 24 hours or only the 10 most recent items, whichever is larger, etc 
The content categorizer 250 also transmits the new content items to the scoring server 262 for a global user ranking  The global scores are transmitted from the scoring server 262 to the data storage server 265, which stores the global scores in association with the new content items  The global scores are helpful for organizing the new content items in the data storage server 265 according to the more popular items 
Turning now to the model server 255, the model server 255 receives the user's activity, interests and social connections from the processing unit 202 or the data storage server 265  The model generation engine 207 generates a model based on user input and/or prior actions  The model server 255 transmits a model to the scoring server 262 and the channel application 103 periodically or upon request 
The channel application 103 includes a channel engine 240 and a user interface engine 260  In one embodiment, the channel engine 240 requests the model from the model server 255 and identifies a channel category that a user would find interesting  The channel engine 240 then transmits a request for a stream of content to the scoring server 262  The channel engine 240 receives the stream of content from the scoring server 262 and generates the channel  The user interface engine 260 generates a user interface for displaying a user interface that includes the channel and transmits it to the user device 115  In addition, the user interface engine 260 generates a user interface to allow the user to customize the channel or define a new channel  These user interfaces are explained in greater detail below with regard to FIGS  4-5 
In one embodiment, the channel engine 240 transmits a query based on the channel category to the scoring server 262  The scoring server 262 queries and receives candidate content items from the data storage server 265  The scoring server 262 also queries and receives candidate content items from the social network server 101  The candidate content items from the social network server 101 are pre-scored by the collaborative filtering engine 217 and, in one embodiment, the unread candidate content items are saved to a cache on the social network server 101  These items are saved to a cache because the quantity of social updates can be large enough that performing the scoring during write time enables faster reads 
In one embodiment, the scoring engine 211 requests the model from the model server 255  The scoring server 262 then compares the candidate content items to the model and scores the candidate content items  The scoring engine 211 compares the candidate content items received from the social network server 101 to the model and rescores them according to the model  In another embodiment, the scoring engine 211 scores the candidate content items according to the category and any keywords associated with a channel  In either embodiment, the scoring engine 211 generates a stream of content based on the scored candidate content items and transmits the stream of content to the channel application 103 
Referring now to FIG  3A, one embodiment of a channel engine 240 is shown in more detail  The channel engine 240 includes a historical analyzer 372, a category identifier 374, a subscription module 376 and a channel generator 378 that are each coupled to signal line 230 
The historical analyzer 372 is used to identify when a user will be interested in a particular category  The historical analyzer 372 identifies, for example, a time of the day or a year that a user will be interested in a category by analyzing historical trends associated with the category  In one embodiment, the historical analyzer 372 performs such analyses by measuring the increase or decrease in the number of new content items that are categorized under a content category or by measuring an increase or decrease in the number of times a new content item is accessed  For example, the number of times a tutorial on filing taxes is accessed would be very high during February-April  In another embodiment, the historical analyzer 372 also keeps track of events such as holidays, festivals, etc  Tracking such events is advantageous as, for example, many users might be interested in costume rentals during Halloween or camping during the Memorial Day and July 4th weekends 
The category identifier 374 identifies a channel category for a user based on the user's interests, activities and social connections  In one embodiment, the category identifier 374 requests the model generated by the model generation engine 207 to identify the channel category  For example, the category identifier 374 identifies sports cars as a channel category because it is an explicit interest of the user  The category identifier 374 suggests channels including a source, a category, keywords, a media type, a size of a content item, and a location for a channel  For example, for a user that is interested in foreign politics, especially relations between the United States and China, the category identifier 374 suggests the category of U S  and Chinese relations (e g , entity=“us_china_relations”), keywords such as trade and deficit because the user is particularly interested in the economic aspect of the relationship between China and the United States, a source such as The Economist (source=“economist com”) because the user prefers The Economist over U S  media outlets and the media being news articles because the user does not enjoy viewing videos 
In one embodiment, the category identifier 374 uses the analyses of the historical analyzer 374 for identifying a channel category for the user  This is advantageous as a user who has searched for US taxes might not be interested in knowing about it throughout the year, but it is beneficial for the user to have a separate channel for US taxes during the tax filing season  In yet another embodiment, the category identifier 374 uses contextual cues of the user for identifying channel categories  For example, the category identifier 374 identifies skiing in Switzerland as a channel category because winter sports is listed as an interest of the user and the user's current IP address is in Switzerland 
The subscription module 376 enables a user to subscribe to existing channels that are public  In one embodiment, the subscription module 376 enables a user to subscribe to a pre-defined channel (such as breaking news, most popular videos, updates from a social group, etc )  The channel application 103 generates the stream of content for pre-defined channels based on global scores of the new content items  Subscribing to pre-defined channels such as breaking news is advantageous as it helps the user to keep apprised of current information and discover new interests  Furthermore, because in one embodiment the breaking news channel is personalized since the content items are compared to a model for the user, the breaking news channel is more relevant than simply a list of popular or recent news items 
In another embodiment, the subscription module 376 enables a user to subscribe to another user's channel (a friend, a famous person, etc ) that is public  Subscribing to another user's channel is advantageous because, for example, a user who is interested in the stock market will benefit by viewing the stream of content that is viewed by a famous stock market analyst  In yet another embodiment, the subscription module 376 enables the user to search for channels that are public using the search engine 143  The subscription module 376, suggests such channels that are viewed by other users based on the interests of the user  In another embodiment, the subscription module 376 communicates with the collaborative filtering engine 217 to suggest channels viewed by other users with whom the user has a relationship 
The channel generator 378 submits a request for a stream of content for a channel to the scoring engine 211  The request includes the channel category identified by the category identifier 374 and channel attributes  The channel attributes include any attribute known to a person with ordinary skill in the art such as a source, presence of keywords, absence of keywords, a media type, a location, a time, a size of a content item, a date, etc  In one embodiment, the channel category and the channel attributes are defined by the user  In another embodiment, channel generator 378 defines the channel attributes for the channel category based on the user's preferences and activities  For example, if a user always reads news articles and seldom watches news videos, the channel generator 378 would define the media type for the channel as text based articles  At any point in time, the user can customize both the channel category and the channel attributes  The channel generator 378 then resubmits the request based on the changes made by the user 
In response to the request, the channel generator 378 receives a stream of content from the scoring engine 211 and generates the channel for the user  The generated channel is either public or private depending upon the user's preferences  In one embodiment, the user shares the channel to a community, a group of people or any internet user  The channel is then displayed to the user with an interface generated by the user interface engine 260 
Referring now to FIG  3B, one embodiment of a scoring engine 211 is shown in more detail  The scoring engine 211 includes a query generator 301, a global scorer 302 and a content stream generator 304 that are each coupled to signal line 228 
The global scorer 302 is used to rank new content items that are stored in the data storage server 265 or memory 237 (depending upon the embodiment)  The global scorer 302 uses signals from the different verticals to compute a global user-independent score for each item to approximate its popularity or importance within the stream that produced it  The global scorer 302 normalizes the score across streams so that items from various streams are comparable to aid in generating a quick yet reasonable ranking of items  The global score is a combination of its quality specific to the source stream (depending on the rank of the source, number of known followers of a source, etc ) and its global popularity (trigger rate on universal search, relevance to trending queries, number of clicks, long clicks received, etc ) 
The global scorer 302 transmits the global score to storage where it is associated with the item  The global score helps rank the items for faster retrieval  For example, if the query generated by the query generator 301 includes a request for the top ten items about skiing, those items are already organized in the data storage server 265 or memory 237 according to the global score 
The query generator 301 receives a request for a stream of content for a channel from the channel engine 240  The query generator 301 generates a query based on the channel attributes that are included in the request  The query generator 301 queries the data storage server 265 or memory 237 depending upon the embodiment  The following is an example query generated by the query generator 301: ((Category: Politics) AND (global_score>80) AND (source: NewsWebsite) AND (media type: Text)) 
The content stream generator 304 receives candidate content items that include the channel attributes  The content stream generator 304, for the above mentioned query, receives text based articles that include the channel category politics and have a global score greater than 80  Additionally, the text based articles are from the source NewsWebsite  In one embodiment, the content stream generator 304 generates the stream by ordering the content items in order of their scores  In another embodiment, the content stream generator 304 determines an interestingness of each candidate content item to the user  The content stream generator 304 determines the interestingness by comparing the candidate content items with a model generated for the user by the model generation engine 207 and scoring them 
where p is a property, that is, a setting A=a of the attributes  The latter quantity, Pr(p|user) is approximated from the user's history of interactions with content items as well as the user's search history and other opt-in data  Similarly, the former quantity, Pr(item|p) is approximated by the (suitably weighted) reciprocal of the number of items with property p (e g , if it is expected that p=((Politics) AND (global_score>80) AND (source: NewsWebsite) AND (media type: Text)) to generate 300 items, take Pr(item|p) to be 1/300) 
where the properties p are summed over single-attribute properties (as opposed to all possible settings of an entire collection of attributes), and G is an exponential function of the form G(x)=2(100 x), so that when applied in this form, if there are several values of p for which Pr(item|p) Pr(p|user) is large, the sum of their G-values slowly increases 
Once the scores are calculated, the content stream generator 304 generates a stream of content for the channel that is ordered according to the candidate content item scores  In one embodiment, only the candidate content items that exceed a certain threshold are included in the stream of content for the channel 
Turning now to the user interface engine 260, FIG  4 is a graphic representation 400 of a user interface generated by the user interface engine 260 for displaying the stream of content of a channel  In this example, the user interface 400 also includes channels 405 that are pre-defined, channels 410 that are suggested for the user and channels 415 that are subscribed to by the user  The user can also define new channels and attributes by clicking the link 420 
The example includes the stream of content for the user's soccer channel 425  The stream of content includes news items 445, videos 450 and social network news feeds 455 from the content sources 440 defined by the user  The candidate content items are listed in decreasing order of their scores  The user interface engine 260 lists five candidate content items with the highest scores in the hot items section 430  The remaining candidate content items are listed in the other items section 435  In another embodiment, the entire stream of content is listed in a single section 
FIG  5 is a graphic representation 500 of a user interface that is generated by the user interface engine 260 for a user to define a new channel or customize an existing channel  In this example, the user interface includes all the channel categories 505 that have been either pre-defined, suggested to the user, or subscribed by the user, and the content sources 510 for each channel category  The user customizes a channel by adding or removing content sources for the channel  In one embodiment, the user edits more advanced channel attributes such as media type, size of the content items, etc  by clicking on the link 515  The user makes the channel public, private or restricts it to a group of people by clicking on link 520  Additionally, the user can also define a new channel by adding a new channel category 
Referring now to FIGS  6-7, various embodiments of the method of the specification will be described  FIG  6 is a flow diagram 600 of one embodiment of a method for generating a stream of content for a channel  The channel engine 240 defines 602 a channel category and submits a request for a stream of content  The request includes channel attributes including any of a category, a source, keywords, a media type, a location, a size of a content item, and a date  The channel category is defined based on a model for a user that is generated by the model generation engine 207 or the channel is defined by a user  The scoring engine 211 receives 604 the request including the channel category and generates 606 a stream of content based on the channel category  The channel engine 240 generates 608 a channel with the stream of content and transmits it to the user 
FIG  7 is a flow diagram 700 of another embodiment of a method for generating a stream of content for a channel  The content categorizer 250 categorizes 702 new content items that are received from heterogeneous data sources  The new content items that are received from heterogeneous data sources include, for example, news articles, microblogs, blogs, videos, photos, etc  The content categorizer 250 categorizes the content according to a category and other features  The content categorizer 250 also stores 704 the new content items in a data storage server 265 or a memory 237, depending upon the embodiment  The global scorer 302 generates 706 a global score for each new content item  The category identifier 374 identifies 708 a channel category for a user based on the user's activities and a historical trend identified by the historical analyzer 372  The user's activity includes a search (such as web, video, news, maps, alerts), entertainment (such as news, video, a personalized homepage, blogs, a reader, gadget subscriptions), social activity (such as interactions through email, profile information, text messaging such as short message service (SMS), microblog, comments on photos, a social graph, and other social networking information), and activity on third-party sites (such as websites that provide ratings, reviews and social networks where users indicate that they approve of content) In one embodiment, the category identifier 374 also uses contextual information of the user to identify the channel category 
The query generator 301 generates a query based on the channel category and the channel attributes and queries 710 the new content items stored on the data storage server 265  The content stream generator 304 receives 712 candidate content items that include the channel category and channel attributes  In one embodiment, the content stream generator 304 receives additional candidate content items from the collaborative filtering engine 217 
The content stream generator 304 scores 714 each candidate content item by comparing it to a model generated by the model generation engine 207  The score is calculated by determining an interestingness of the candidate content item to the user  The content stream generator 304 then generates 716 the stream of content based on the scores for each candidate content item  The channel engine 240 then generates 718 a channel with the stream of content and transmits it to the user 
The foregoing description of the embodiments has been presented for the purposes of illustration and description  It is not intended to be exhaustive or to limit the specification to the precise form disclosed  Many modifications and variations are possible in light of the above teaching  It is intended that the scope of the embodiments be limited not by this detailed description, but rather by the claims of this application  As will be understood by those familiar with the art, the examples may be embodied in other specific forms without departing from the spirit or essential characteristics thereof  Likewise, the particular naming and division of the modules, routines, features, attributes, methodologies, and other aspects are not mandatory or significant, and the mechanisms that implement the description or its features may have different names, divisions and/or formats  Furthermore, as will be apparent to one of ordinary skill in the relevant art, the modules, routines, features, attributes, methodologies, and other aspects of the specification can be implemented as software, hardware, firmware, or any combination of the three  Also, wherever a component, an example of which is a module, of the specification is implemented as software, the component can be implemented as a standalone program, as part of a larger program, as a plurality of separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and/or in every and any other way known now or in the future to those of ordinary skill in the art of computer programming  Additionally, the specification is in no way limited to implementation in any specific programming language, or for any specific operating system or environment  Accordingly, the disclosure is intended to be illustrative, but not limiting, of the scope of the specification, which is set forth in the following claims 
providing, with the one or more processors, the customized stream of content 
2  The computer-implemented method of claim 1 comprising removing pre-existing content items included in the customized stream of content for the channel 
3  The computer-implemented method of claim 1 wherein the historical trend is one of an increase in a number of the new content items for a content category and an increase in a number of times one of the new content items is accessed 
categorizing the new content items 
5  The computer-implemented method of claim 3 wherein the heterogeneous data sources include at least two from the group of a news article post, a news feed, a social feed, a blog post, a micro-blog post, a photo, a video, an audio, an email message, and a text based message 
6  The computer-implemented method of claim 1 comprising receiving a request from the user to subscribe to an existing channel 
7  The computer-implemented method of claim 1 wherein the channel category is also based on an interest of the user and a connection of the user 
8  The computer-implemented method of claim 1 wherein the user activity is an interaction of the user with an application, wherein the interaction of the user with the application includes providing at least one of a user preference, a user interest, a comment, a tag, and a search 
provide the customized stream of content 
10  The computer program product of claim 9, wherein the computer readable program when executed on the computer also causes the computer to remove pre-existing content items included in the customized stream of content for the channel 
11  The computer program product of claim 9, wherein the historical trend is one of an increase in a number of the new content items for a content category and an increase in a number of times one of the new content items is accessed 
categorize the new content items 
13  The computer program product of claim 12, wherein the heterogeneous data sources include at least two from the group of a news article post, a news feed, a social feed, a blog post, a micro-blog post, a photo, a video, an audio, an email message, and a text based message 
14  The computer program product of claim 9, wherein the computer readable program when executed on the computer also causes the computer to receive a request from the user to subscribe to an existing channel 
15  The computer program product of claim 9, wherein the channel category is also based on an interest of the user and a connection of the user 
16  The computer program product of claim 9, wherein the user activity is an interaction of the user with an application, wherein the interaction of the user with the application includes providing at least one of a user preference, a user interest, a comment, a tag, and a search 
18  The system of claim 17 wherein the system is further configured to remove pre-existing content items included in the customized stream of content for the channel 
19  The system of claim 17 wherein the historical trend is one of an increase in a number of the new content items for a content category and an increase in a number of times one of the new content items is accessed 
21  The system of claim 20 wherein the heterogeneous data sources include at least two from the group of a news article post, a news feed, a social feed, a blog post, a micro-blog post, a photo, a video, an audio, an email message, and a text based message 
22  The system of claim 17 wherein the system is further configured to receive a request from the user to subscribe to an existing channel 
23  The system of claim 17 wherein the channel category is also based on an interest of the user and a connection of the user 
24  The system of claim 17 wherein the user activity is an interaction of the user with an application, wherein the interaction of the user with the application includes providing at least one of a user preference, a user interest, a comment, a tag, and a search 
Adamic et al , "Search in power-law networks," Physical Review E, 2001, vol  64, HP Labs/Stanford University, The American Physical Society 
Boyd, et al , "Social Network Sites: Definition, History, and Scholarship," Journal of Computer-Mediated Communication, International Communication Association, 2008, pp  210-230  8 
MediaSift Ltd , DataSift: Realtime Social Data Mining Platform, Curate and Data Mine the Real Time Web with DataSift, Dedipower, Managed Hosting, May 13, 2011, 1 pg 
Ring Central, Inc , Internet, retrieved at http://www ringcentral com, Apr  19, 2007, 1 pg  28 
Singh et al , "CINEMA: Columbia InterNet Extensible Multimedia Architecture," Department of Computer Science, Columbia University, May 2002 pp  1-83 
Yu et al , "It Takes Variety to Make a World: Diversification in Recommender Systems," 2009, pp  1-11, downloaded from https://openproceedings org/2009/conf/edbt/YuLA09 pdf 



ground_truth: By comparing candidate content items to a model and scoring them.