Acoustic model adaptation using splines

Described is a technology by which a speech recognizer is adapted to perform in noisy environments using linear spline interpolation to approximate the nonlinear relationship between clean speech, noise, and noisy speech. Linear spline parameters that minimize the error the between predicted noisy features and actual noisy features are learned from training data, along with variance data that reflect regression errors. Also described is compensating for linear channel distortion and updating noise and channel parameters during speech recognition decoding.

BACKGROUND

Contemporary speech recognizers operate by having a large number of Gaussian distributions or the like. When audio corresponding to an utterance is input, the recognizer finds the best matching distributions based on training data, and uses those distributions to determine the words of the utterance.

As is well known, speech recognition systems tend to perform poorly in a noisy environment. One reason that speech recognizers fail in noisy environments is that the environmental conditions present in deployment differ from those seen in the training data.

Various compensation techniques have been attempted to reduce the mismatch between training and testing conditions and thus improve recognition accuracy. Generally there are two types of techniques, namely feature compensation techniques and model compensation techniques.

In feature compensation, the captured signal or the features extracted from the signal are processed prior to recognition to mitigate the effect of noise. These techniques are computationally efficient and do not require changes to the recognizer itself. However, they have the drawback that they make point estimates of the enhanced speech features, and errors in this estimation can cause further mismatch to the recognizer's acoustic models, further degrading performance.

Model compensation techniques avoid this problem by directly adapting the distributions inside the recognizer to better match the current environmental conditions. Such techniques may operate in a data driven fashion, although faster performance is typically achieved by methods that exploit the known relationship between clean speech, noise, and the resulting noisy speech.

However, model compensation is a challenging problem because the features that characterize these three quantities (clean speech, noise and noisy speech) are related nonlinearly. One option is to digitally mix the noise with the clean speech to produce noisy speech, and retrain the recognizer from scratch with the noisy speech. This results in improved accuracy, but is slow in computation time, and thus approximations have been attempted that are much faster to compute

Several different approximation methods for handling this nonlinearity have been proposed. For example, Monte Carlo sampling has been used to generate samples from the constituent speech and noise distributions, which are then used to estimate the parameters of the resulting distribution of noisy speech. In Vector Taylor Series (VTS) adaptation, the nonlinear function that describes noisy speech features as a function of the clean speech and noise features is linearized around expansion points defined by the speech and noise models. Other model compensation techniques have been used; however regardless of which technique is used, there is still room for improvement.

SUMMARY

Briefly, various aspects of the subject matter described herein are directed towards a technology by which a speech recognizer's speech parameters are adapted to a noisy environment based upon adaptation parameters of a spline model. The spline model, which may be trained from training data including clean speech and noise, approximates the nonlinear relationship between speech, noise and noisy speech parameters via a (e.g., linear) spline function. A variance parameter may be used to model spline regression error, and used in adapting the clean speech parameters.

In one implementation, the adapted speech recognizer parameters are computed by interpolating the parameters of the segments of the spline function. The interpolation weights are computed based upon a probability distribution representing the clean speech and the probability distribution representing the noise. The variance parameters of the spline functions can also be used to adapt the speech recognizer parameters.

DETAILED DESCRIPTION

Various aspects of the technology described herein are generally directed towards improving speech recognition via a data-driven technique for performing acoustic model adaptation to noisy environments. The technique uses linear spline (or another type of spline such as nonlinear, polynomial and so forth) regression to model the nonlinear function that describes noisy speech features as a function of the clean speech and noise features. As described below, the set of spline parameters that minimizes the error between the predicted and actual noisy speech features is learned from training data, and used at runtime to adapt cleanly trained acoustic model parameters to the current noise conditions.

It should be understood that any of the examples described herein are non-limiting examples. As such, the present invention is not limited to any particular embodiments, aspects, concepts, structures, functionalities or examples described herein. Rather, any of the embodiments, aspects, concepts, structures, functionalities or examples described herein are non-limiting, and the present invention may be used various ways that provide benefits and advantages in computing and speech recognition in general.

FIG. 1shows various aspects related to training a noise robust model for adaptation using linear spline interpolation. In general, training102data is used in a known manner by a training mechanism104to train a clean speech recognizer106(e.g., the Gaussian distributions of a Hidden Markov Model, or HMM type recognizer) in a clean speech environment.

The training data102is also mixed with condition examples108, e.g., noise, which is then used by a spline interpolation parameter learning mechanism110to learn the adaptation parameters112(also referred to as the spline model), as described below. In one implementation, the parameters correspond to linear splines, however other spline functions may be alternatively used.

Once the adaptation parameters112are known, they may be used in an online, runtime environment as generally represented inFIG. 2to help in recognizing an unknown utterance222in an environment with some amount of noise224that is arbitrary at any given time. In general and as described below, the appropriate parameters for the noise sample are selected from the available adaptation parameters112based upon one or more noise measurements that are input when the utterance222is not concurrent, e.g., sampled just before the utterance. Those selected parameters are then used by an adaptation mechanism226to adapt the distributions of the clean speech recognizer106to provide a noise adapted speech recognizer228. The recognition result230is thus noise adapted distributions.

FIG. 3shows additional details of how linear spline interpolation is integrated into speech recognition technology. InFIG. 3, y represents the noisy speech, which is converted into a sequence of digital values that are grouped into frames in a known manner by a frame constructor302and passed through a fast Fourier transform (FFT)304. The FFT304computes the phase and magnitude of a set of frequencies found in the frame. The magnitude or the square of the magnitude of each FFT is then processed by block306.

In one implementation, the magnitude values are applied to a Mel-frequency filter bank308, which applies perceptual weighting to the frequency distribution and reduces the number frequency bins that are associated with the frame; other frequency-based transforms may be used. When a Mel-frequency filter bank308is used, the observation vector is referred to as a Mel-Frequency Cepstral Coefficient (MFCC) vector.

A log function310may be applied to the values to compute the logarithm of-each frequency magnitude, and the logarithms of each frequency may be applied to a discrete cosine transform312. As described below, the spline operations313are performed at this stage, (on a per-filter basis such that there is a subset of the adaptation parameters for each filter), essentially processing the DCT into another data form, performing spline-related processing, and then re-constructing the DCT. Further recognition processing314is then performed in a known manner.

Turning to details of the spline-related operations, in the presence of additive noise, the relationship between log mel spectra of speech, noise and noisy speech is nonlinear. The technology described herein models this nonlinear relationship using spline regression, which in one implementation is linear spline regression. In the offline training phase (FIG. 1), the set of spline parameters112that minimizes the error between the predicted and actual noisy speech features is learned from training data102in the presence of (e.g., noise) conditions108. In the online runtime phase (FIG. 2), these parameters112are used to adapt clean acoustic model parameters of the clean speech recognizer106to the current noise224conditions.

More particularly, an HMM adaptation scheme interprets the nonlinear relationship between speech, noise and noisy speech as a nonlinear function between the prior SNR (signal to noise ratio) and the posterior SNR in the log mel filterbank domain. As generally represented inFIG. 4and described below, this nonlinear function is then modeled as a series of linear segments using linear spline regression. Each segment is characterized with a set of spline parameters (slope and y-intercept) which characterize the line, and a variance that characterizes the uncertainty of the linear model in that segment. Adaptation is performed by applying a linear transform to the prior SNR to obtain the statistics of the posterior distribution. This transformation is determined by interpolating the linear spline parameters, as also described below. In this way, the parameters of the noisy speech distribution are determined from the distribution of the posterior SNR.

Note that the spline parameters are learned from the training data. Unlike other techniques, the lines are not restricted to be tangent to the theoretical line that defines the nonlinear relationship. Rather, the spline learning mechanism (algorithm) finds the set of parameters that minimizes the error between the predicted noisy speech features (determined by the model) and the actual noisy speech features.

Further, in one implementation, the spline model includes a variance for each spline segment that explicitly models the uncertainty that arises when transforming clean speech models to noisy speech models. This uncertainty comes from several sources, including the phase asynchrony between the clean speech and the additive noise. A second source of additional variance is the conversion from the cepstral domain to the log mel spectral domain that is typically performed using a pseudo-inverse of the Discrete Cosine Transform (DCT) matrix rather than an actual inverse DCT. Incorporating these two features into the spline model more accurately captures the inexact nature of the relationship between clean speech and noisy speech.

Let X, N, and Y be the spectra of clean speech, noise and noisy speech, respectively. The power spectrum of the noisy speech can be expressed as a function of the clean speech and the noise as shown in the equation below:
|Y|2=|X|2+|N|2+2 cos(θ)|X∥N|(1)
where θ is the relative phase between X and N. Most state-of-the-art speech recognizers use mel-frequency cepstral coefficients (MFCCs) or features derived from MFCCs. The cepstra are extracted from the power spectrum by first computing the energy at the output of each filter in the mel filterbank, and then applying a truncated DCT to the vector of log filterbank energies. If x, n, and y represent the cepstral coefficients extracted from the corresponding spectra of speech, noise, and noisy speech, the cepstra of the noisy speech can be expressed as:
y=n+Clog(1+eD(x−n)+2αeD(x−n)/2)  (2)
where C is the truncated DCT matrix, D is the pseudo-inverse of C and α represents the contribution of the relative phase term in Equation (1) in the feature domain. Thus, the MFCC feature vector for noisy speech y is a non-linear function of the cepstra of clean speech x and the cepstra for the noise n and the relative phase between the clean speech and the noise.

If u=x−n is defined to be the prior SNR and v=y−n to be the a posterior SNR, Equation (2) can be rewritten as
y−n=Clog(1+eD(x−n)+2αeD(x−n)/2)  (3)
v=Clog(1+eDu+2αeDu/2)  (4)

Most model adaptation or feature enhancement algorithms utilize the distortion model that assumes α=0 which further simplifies Equation (4) as
v=Clog(1+eDu)  (5)

FIG. 4shows a plot of u versus v for the fifteenth log mel spectral coefficient based on actual example training data. The plot shows the scatter of the true data (the gray dots), the mode of the nonlinear relationship (v=ln(1+exp(u))), (the black line) and the fitted spline with segments between the segment change points (the circles), referred to as knots. Because both the clean and multi-condition (noisy) training data are available, training can compute the features for clean speech, noise, and noisy speech, for all utterances in the training set.

Given samples of x, n, and y, u and v are computed for each input audio frame. The cloud of points in the scatter plot shows the influence of the relative phase term. The solid black line shows the mode of the data, which coincides with the α=0 condition given by Equation (5). As the figure shows, ignoring the effect of the relative phase term restricts other adaptation algorithms to the mode of the nonlinear relationship between the prior SNR and the posterior SNR, thereby ignoring the variance in the data.

In contrast, in one implementation of the spline interpolation method described herein, this variance is explicitly modeled, which leads to a more accurate model adaptation strategy. Details of modeling the variance are described below.

In linear spline regression, ordered (x,y) pairs of data are modeled by a series of non-overlapping segments in which the data in each segment is modeled by a linear regression. The regression parameters for each segment are computed under the constraint that adjacent regression lines intersect at the knots. In the linear spline regression method described herein, the relationship between the prior SNR u and the posterior SNR v expressed in Equation (4) is modeled using a linear spline regression composed of K line segments:

v=a1⁢u+b1+ε1∀u≤U1v=a2⁢u+b2+ε2∀U1<u≤U2⋮=⋮⋮v=ak⁢u+bk+εk∀Uk-1<u≤Uk⋮=⋮⋮v=aK⁢u+bK+εK∀u>UK-1(6)
where ak, bkand εkare the slope, y-intercept and regression error for the kth line segment, and U1, . . . UK-1are the K−1 knots that define the change points of the model. The regression error for each segment is modeled as a zero mean Gaussian with variance σεk2and is assumed to be uncorrelated with x, n and y.

Note that there is generally an optimal number of segments with respect to providing the best adaptation performance for a given system. The adaptation performance does not necessarily improve by increasing the number of line segments of the spline, and indeed, the performance of the system starts deteriorating once the optimum is exceeded. The parameters of the spline may be estimated from the speech observations, without regard to the recognizer, however it is feasible to estimate the spline parameters within the recognizer training framework.

The choice of knot location may be significant, in that the localized variance of the data may be considered when selecting knots. In one implementation, the mode is modeled in the prior SNR interval of −5 to 5 dB with relatively more line segments; the variance of u versus v changes rapidly in this SNR range. Having more points in this range allows the algorithm to track these more rapid variance changes and improve the estimate of the posterior SNR.

Given the set of knot locations, the regression parameters are then computed. Note that while the regression error εkis assumed to be Gaussian, it readily apparent from the scatter plot shown inFIG. 4that the data within a given segment is not Gaussian. Instead the regression is asymmetric, with more data lying below than above the mode. As a result, the regression parameters learned from the data may not accurately track the mode of the data.

To ensure that the linear spline regression tracks the mode, the method finds the linear regression parameters {ak,bk} for each segment that minimize the mean squared error between the linear segment and the mode of the data. That is, in each segment, the following squared error is minimized:

∑n=1Nk⁢{ak⁢un+bk-ln⁡(1+ⅇun)}2,⁢Uk-1<un≤Uk(7)
where Nkis the number of samples that lie in the kth spline segment. The K sets of regression parameters that minimize the error shown in Equation (7) for all segments and satisfy the constraints may be found by simultaneously solving a system of linear equations, in a known manner.

Once the parameters for the linear spline regression are estimated, they can be used to construct an MMSE (minimum mean square error) estimate of the noisy speech y from the clean speech x and the noise n. The MMSE estimator is based upon the computation of the conditional probability of v, which can be computed using Equation (9) (derived from equation (8)):

p⁡(v,u)=∑k=1K⁢p⁡(v❘u,k)⁢p⁡(u)⁢wk(8)⇒p⁡(v❘u)=∑k=1K⁢p⁡(v❘u,k)⁢wk(9)
where wkis the probability that the prior SNR u lies between the knots Uk-1<u≦Uk. If it is assumed that the clean speech x and the noise n are independent Gaussian random variables, then the distribution of u=x−n is also Gaussian with mean μu=μx−μnand variance σu2=σx2+σn2. Given the distribution of u, the probability wkis computed from the continuous density function (cdf) of u using Equation (10)

wk=∫Uk-1Uk⁢p⁡(u)⁢ⅆu=Φ⁡(Uk;μu,σu2)-Φ⁡(Uk-1;μu,σu2)(10)
where Uk-1and Ukare the knots that bound the kth segment and Φ is the cdf of a Gaussian distribution. The conditional expectation of v can then be written using Equation (9):

E⁡[v❘u]=∫v⁢⁢p⁡(v❘u)⁢ⅆv=∑k=1K⁢wk⁢∫v⁢⁢p⁡(v❘u,k)⁢ⅆv(11)
Note that one particular implementation sets K=1 and w1=1, which results in faster computation.

Under the linear spline model, the posterior SNR v is a linear function of the prior SNR u for a given segment k, i.e., v=aku+bk. This linear transform preserves the Gaussian nature of the random variable, which implies that both the joint distribution p(v,u) and the conditional distribution p(v|u) are Gaussian. With algebraic manipulation, the segment-conditional distribution of v in equation (11) can be written as:
p(v|u,k)=N(v;aku+bk,σεk2).  (12)

The MMSE estimate of the a posterior SNR v may then be written as:

Because v=y−n, the MMSE estimate of y can be computed from equation (13) as:

Equation (15) shows that the MMSE estimate of y under the linear spline interpolation model is a linear combination of x and n. As described below, this MMSE estimate of the noisy speech features y may be used to adapt the acoustic models of a speech recognizer trained on clean speech.

Turning to acoustic model adaptation using linear spline interpolation, as described above the MMSE estimate of y is computed based on knowledge of clean speech x and the noise n. In the model domain, the actual x and n are unknown, and thus modeled as Gaussian distributions p(x) and p(n). From these, an estimate of the parameters of the noisy speech distribution p(y) is desired, which is also Gaussian under the linear form of the estimate of y in equation (15).

The mean of y is computed by applying an expectation operator to both sides of Equation (14). The variance can then be computed from the second moment E[y2] and the estimate of the mean of y. This gives the following estimates of the mean and variance of p(y):

These adaptation equations are valid for a single log mel filter bank component. If x, n and y are M-dimensional log mel spectral vectors modeled as Gaussians with means μxl, μnland μyland covariances Σxl, Σnland Σylrespectively, then the adaptation formulae for the mean and covariance of p(y) can be written as:

Most speech recognition systems operate on cepstra, not log Mel spectra. To perform model adaptation on acoustic models trained from L-dimensional cepstra derived from M-dimensional log Mel spectra (L<M), splines need to be trained for all M mel filters. Transformation of the parameters between cepstra and mel spectra is performed using a truncated DCT CεRL×Mand pseudo-inverse DCT DεRM×L. The final update equations for cepstral domain parameters can then be expressed as
μy=Āμx+(I−Ā)μn+Cb(21)
Σy=ŪΣxŪT+VΣnVT+CΣεCT(22)
where Ā=CAD and similar notation is adopted for U and V. Even though Σx,Σnare diagonal covariances, Σyis full covariance. However, Σymay be assumed to be diagonal, in order to be able to use a decoder that is optimized for diagonal covariances.

With respect to adaptation of the dynamic model parameters, as a consequence of representing the relationship between clean and noisy speech using a linear spline, the adaptation relationships among the dynamic components of x, n and y are similar to the statics components. First order difference coefficients are computed according to Δy=y(t+τ)−y(t−τ). Assuming that the wk's do not change over a short time interval, this can be written as:

Δ⁢⁢y=∑k⁢wk⁢Δ⁢⁢yk(23)Δ⁢⁢y=Δ⁢⁢n⁢∑k⁢wk⁡(1-ak)+Δ⁢⁢x⁢∑k⁢wk⁢ak+∑k⁢wk⁢ε.k(24)
where the εkis a Gaussian with zero mean and variance σεk2−2σεk2. The update equations for the Gaussian means and variances for the delta parameters are:
μΔy=ĀμΔx+(I−Ā)μΔn(25)
ΣΔy=ŪΣΔxŪT+VΣΔnVT+CΣėCT(26)

The update equations for the second order differences ΔΔy are similar to Δy. The only difference is that the variance of {umlaut over (ε)}kis σ{umlaut over (ε)}k2=4σεk2. The update equations for the means and variances of the Gaussians of delta-delta parameters are:
μΔΔy=ĀμΔΔx+(I−Ā)μΔΔn(27)
ΣΔΔy=ŪΣΔΔxŪT+VΣΔΔnVT+CΣ{umlaut over (ε)}CT(28)

The improvement in estimating the variance of v is a direct result of incorporating the regression error ε as one of the parameters for the spline. This variance parameter not only accounts for the deviation from the mode, but also models the smearing that occurs due to the phase term present in the spectrum generated using the Fourier transform.

Turning to extending the linear spline interpolation algorithm, the adaptation scheme may be extended to compensate for the presence of linear channel distortion. Further, the noise and channel parameters can be updated during decoding in an unsupervised manner within the linear spline interpolation framework.

Linear spline interpolation was used to perform HMM adaptation for a fixed estimate of the noise parameters and no channel distortion μh=0. The noise and channel parameters can be re-estimated in an unsupervised manner using a generalized EM (expectation maximization) approach, starting with the following auxiliary function:
Q(λ,{circumflex over (λ)})=Σt,sγtslog(yt|s,λ))(29)
where γtsis the posterior probability of Gaussian component s occurring at frame t given the observation sequence and p(yt|s,λ)=(yt,μy,s,Σy,s) is the likelihood of the observation under the adapted linear spline interpolation model.

For the re-estimation of the noise and channel means, the re-estimation formulae for the noise mean can be determined by taking the derivative of equation (29) with respect to μnand setting the result equal to zero. Solving for μnobtains:

Note that the transformation parameters have a subscript s to indicate they are a function of the Gaussian component. The update equation for the channel mean μhcan be similarly computed as:

The re-estimation formulae for the means of the dynamic noise and channel parameters can be similarly computed. However, in one implementation it is assumed that the channel is fixed and the noise is stationary. As a result, the means of the dynamic noise and channel parameters are set equal to zero.

Turning to re-estimation of the noise variances, because there is no closed-form solution for the noise variance update, the variance is updated iteratively using Newton's method. The new estimate of the variance is computed as:
Σnnew=Σn[H(Σn)]−1[∇Q(Σn)]  (32)
where(Σn) is the Hessian matrix with elements defined as:

ℋij⁡(Σn)=∂2⁢Q∂σn2⁡(i)⁢∂σn2⁡(j)(33)
Because the variances are not guaranteed to remain non-negative, {tilde over (Σ)}n=log(Σn) is optimized in practice. The expressions for the terms of the gradient and Hessian are set forth below:

This approach is also used to update the variances of the dynamic noise parameters ΣΔn,ΣΔΔn. The static, delta, and delta-delta components are assumed independent, so the Hessian matrices for each set of parameters can be computed independently.

FIG. 5summarizes an example sequence of steps involved in performing model adaptation using linear spline interpolation, beginning at step502which reads in the noisy utterance. Step504initializes the channel mean μhto zero and computes sample estimates of {μn,Σn,ΣΔn,ΣΔΔn} from the first and last N frames of the utterance.

As represented by step506, for each Gaussian, the spline weights {wk} and transformation parameters {F,G,e,Σε} are computed. Step508adapts the HMM parameters and decodes the utterance. At this point, the decoded utterance may be output, or further updating of the noise and channel parameters from the initial estimate may be performed. Step510represents making this decision, and thus continues to step512as described below, or branches to step520to output the utterance based upon the initial estimate.

If updating is desired, using the hypothesized transcription, step512represents computing the posterior probabilities γstand re-estimating the channel and noise parameters. As represented by step514, for each Gaussian, the spline weights and transformation parameters are recomputed, and the HMM parameters adapted. Step516decodes the utterance following the updating.

The above set of sequence of steps may comprise a single iteration of generalized EM for updating the noise and channel parameters. However, multiple iterations of steps512,514and516may be performed as desired, as represented via step518; (note that in one implementation, three iterations of Newton's method were performed to update the noise variances). After the single or multiple iterations, step520outputs the transcription.

Note that it is feasible to share a given transformation for a number of Gaussians. This is another approximation that if used, may result in faster computation.

As can be seen, there is described aspects of adapting a speech recognizer trained on clean speech to a noisy environment. The mechanism/algorithm approximates the nonlinear mapping between speech, noise and noisy speech parameters using an interpolated linear spline regression learned from training data. The model for the spline further tries to accommodate the Uncertainty present in the true data with a variance parameter that also models the spline regression error.

Exemplary Operating Environment

FIG. 6illustrates an example of a suitable computing and networking environment600on which the examples ofFIGS. 1-5may be implemented. The computing system environment600is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing environment600be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment600.

With reference toFIG. 6, an exemplary system for implementing various aspects of the invention may include a general purpose computing device in the form of a computer610. Components of the computer610may include, but are not limited to, a processing unit620, a system memory630, and a system bus621that couples various system components including the system memory to the processing unit620. The system bus621may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.

The system memory630includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM)631and random access memory (RAM)632. A basic input/output system633(BIOS), containing the basic routines that help to transfer information between elements within computer610, such as during start-up, is typically stored in ROM631. RAM632typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit620. By way of example, and not limitation,FIG. 6illustrates operating system634, application programs635, other program modules636and program data637.

The computer610may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,FIG. 6illustrates a hard disk drive641that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive651that reads from or writes to a removable, nonvolatile magnetic disk652, and an optical disk drive655that reads from or writes to a removable, nonvolatile optical disk656such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive641is typically connected to the system bus621through a non-removable memory interface such as interface640, and magnetic disk drive651and optical disk drive655are typically connected to the system bus621by a removable memory interface, such as interface650.

The drives and their associated computer storage media, described above and illustrated inFIG. 6, provide storage of computer-readable instructions, data structures, program modules and other data for the computer610. InFIG. 6, for example, hard disk drive641is illustrated as storing operating system644, application programs645, other program modules646and program data647. Note that these components can either be the same as or different from operating system634, application programs635, other program modules636, and program data637. Operating system644, application programs645, other program modules646, and program data647are given different numbers herein to illustrate that, at a minimum, they are different copies. A user may enter commands and information into the computer610through input devices such as a tablet, or electronic digitizer,664, a microphone663, a keyboard662and pointing device661, commonly referred to as mouse, trackball or touch pad. Other input devices not shown inFIG. 6may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit620through a user input interface660that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor691or other type of display device is also connected to the system bus621via an interface, such as a video interface690. The monitor691may also be integrated with a touch-screen panel or the like. Note that the monitor and/or touch screen panel can be physically coupled to a housing in which the computing device610is incorporated, such as in a tablet-type personal computer. In addition, computers such as the computing device610may also include other peripheral output devices such as speakers695and printer696, which may be connected through an output peripheral interface694or the like.

The computer610may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer680. The remote computer680may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer610, although only a memory storage device681has been illustrated inFIG. 6. The logical connections depicted inFIG. 6include one or more local area networks (LAN)671and one or more wide area networks (WAN)673, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN networking environment, the computer610is connected to the LAN671through a network interface or adapter670. When used in a WAN networking environment, the computer610typically includes a modem672or other means for establishing communications over the WAN673, such as the Internet. The modem672, which may be internal or external, may be connected to the system bus621via the user input interface660or other appropriate mechanism. A wireless networking component such as comprising an interface and antenna may be coupled through a suitable device such as an access point or peer computer to a WAN or LAN. In a networked environment, program modules depicted relative to the computer610, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,FIG. 6illustrates remote application programs685as residing on memory device681. It may be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

An auxiliary subsystem699(e.g., for auxiliary display of content) may be connected via the user interface660to allow data such as program content, system status and event notifications to be provided to the user, even if the main portions of the computer system are in a low power state. The auxiliary subsystem699may be connected to the modem672and/or network interface670to allow communication between these systems while the main processing unit620is in a low power state.

CONCLUSION