Source: http://www.google.co.uk/patents/US9100685
Timestamp: 2018-01-17 05:36:48
Document Index: 557571061

Matched Legal Cases: ['Application No. 12195349', 'Application No. 2775700', 'Application No. 201110159923', 'Application No. 2775700', 'Application No. 2775814', 'Application No. 2775814', 'Application No. 201110159923', 'Application No. 201110159923', 'Application No. 12194891', 'Application No. 12195349', 'Application No. 12194891', 'Application No. 12195349', 'art 2', 'art 2', 'art-2', 'art0009']

Patent US9100685 - Determining audience state or interest using passive sensor data - Google Patents
This document describes techniques and apparatuses for determining an audience's state or interest using passive sensor data. The techniques receive sensor data that measures an audience during presentation of a media program. The techniques, based on this sensor data, then determine a state or interest...http://www.google.co.uk/patents/US9100685?utm_source=gb-gplus-sharePatent US9100685 - Determining audience state or interest using passive sensor data
Publication number US9100685 B2
Application number US 13/316,351
Also published as US9628844, US20130152113, US20150341692, US20170188079
Publication number 13316351, 316351, US 9100685 B2, US 9100685B2, US-B2-9100685, US9100685 B2, US9100685B2
Inventors Michael J. Conrad, Geoffrey J Hulten, Kyle J. Krum, Umaimah A. Mendhro, Darren B. Remington
Patent Citations (575), Non-Patent Citations (279), Referenced by (4), Classifications (10), Legal Events (2)
US 9100685 B2
receiving a media type describing a particular portion of a media program;
determining, based on the sensor data, a state of the person during the particular portion of the media program, the state being a sad, a related talking, an unrelated talking, a disgusted, an afraid, a smiling, a scowling, a placid, a surprised, an angry, a laughing, a screaming, a clapping, a waving, a cheering, a looking-away, a looking-toward, a leaning-away, a leaning-toward, an asleep, or a departed state;
adjusting the determined state based, at least in part, on historical sensor data;
responsive to the adjusting, deriving an interest level graph based, at least in part, on the determined state of the person during the particular portion of the media program; and
providing the interest level graph and the media type describing the particular portion of the media program effective to enable measurement of a value of the media program, assessment of a potential value of a similar media program or portion thereof, control of presentation of the media program, or automatic rating, for the person, of the media program.
2. A system as described in claim 1, wherein the sensor data includes a head orientation of the person and the operation of determining the state determines the state to be the looking-away or the looking-toward state.
3. A system as described in claim 1, wherein the operation of determining the state of the person determines a numerical probability for the state or multiple numerical probabilities for multiple states, respectively.
the sensor data includes infrared data frames and the operation of determining the state determines the state based on the infrared data frames;
the sensor data includes skeletal movement of the person and the operation of determining the state determines the state to be the clapping state;
the sensor data includes skeletal movement of the person and no or low-amplitude audio of the person and the operation of determining the state determines the state to be the waving state;
the sensor data includes skeletal orientation or movement of the person and high-amplitude audio from the person and the operation of determining the state determines the state to be the cheering state;
the sensor data includes head movement of the person, facial movement or orientation of the person, and high-amplitude audio from the person and the operation of determining the state determines the state to be the laughing state or the screaming state;
the sensor data includes facial orientation of the person and the operation of determining the state determines the state to be the sad, disgusted, afraid, smiling, placid, scowling, surprised, or angry state;
the sensor data includes skeletal movement or a thermal reading of the person and the operation of determining the state determines the state to be the departed state;
the sensor data includes audio and the operation of determining the state determines, using speech recognition, the state to be the related talking state or the unrelated talking state; or the sensor data includes facial orientation or respiration rate of the person and the operation of determining determines the state to be the asleep state.
5. A system as described in claim 1, wherein the sensor data is received for multiple periods during the presentation of additional portions of the media program, the operation of determining determines states for each of the multiple periods, and the operations further comprising providing the states for each of the multiple periods.
6. A system as described in claim 1, the operations further comprising determining, based on the sensor data, an identity of the person and providing the identity of the person.
7. A system as described in claim 1, the operations further comprising determining, based on the sensor data, a demographic of the person and providing the demographic of the person.
8. A system as described in claim 1, the operations further comprising determining, based on the media type and the state, an interest level of the person relative to the particular portion of the media program.
receiving sensor data for an audience, the sensor data passively sensed during presentation of a media program to the audience;
determining, based on a media type describing a particular portion of the media program and the sensor data for the audience passively sensed during the presentation of the particular portion of the media program to the audience, an interest level of the audience to the particular portion of the media program;
normalizing the determined interest level of the audience based on historical interest levels for a person in the audience;
producing an interest level graph that indicates the determined and normalized interest level of the audience to the particular portion of the media program; and
providing the interest level graph of the audience, the interest level graph indicating a value of the media program, usable to assess a potential value of a similar media program or portion thereof, usable to control presentation of the media program, or indicating a rating for the media program.
10. A computer-implemented method as described in claim 9, wherein the normalizing the determined interest level of the audience is further based on historical interest levels for other persons not in the audience.
11. A computer-implemented method as described in claim 9, further comprising determining, based on the sensor data, an engagement of the audience to the particular portion of the media program, and where determining the interest level of the audience is based on the engagement of the audience and the media type.
12. A computer-implemented method as described in claim 9, further comprising determining, based on the sensor data, states of one or more persons in the audience during presentation of the particular portion of the media program, and wherein the determining the interest level of the audience is based on the states of the one or more persons and the media type.
13. A computer-implemented method as described in claim 9, wherein the sensor data for the audience includes multiple sensor readings over a time in which the media program is presented to the audience and determining the interest level of the audience determines multiple interest levels over the time.
14. A computer-implemented method as described in claim 9, wherein the sensor data is received from an audience-sensing device local to the audience.
15. A computer-implemented method as described in claim 14, wherein the audience-sensing device is a forward-facing camera integrated within a display on which the media program is rendered.
16. A computer-implemented method as described in claim 15, wherein the forward-facing camera is capable of sensing facial orientation and skeletal movement of the audience, and the sensor data includes facial orientation data and skeletal movement data for a person in the audience, the skeletal movement including movement of an arm of the person.
17. A computer-implemented method as described in claim 14, wherein the audience-sensing device includes a microphone capable of sensing audio generated by the audience and a location of the audio relative to persons in the audience.
18. A computer-implemented method as described in claim 9, wherein the audience includes two or more persons and the sensor data is received with or includes information indicating portions of the sensor data attributable separately to each of the two or more persons.
19. A computer-implemented method as described in claim 9, the method performed by a System-on-Chip (SoC) integrated within a media presentation device that presents the media program to the audience, the media presentation device including an audience-sensing device from which the sensor data is received.
passively sensing or receiving sensor data for an audience, the sensor data passively sensed at time periods during which a media program is presented to the audience and from an audience-sensing device capable of sensing head or skeletal orientation or movement, the sensor data including the head or skeletal orientation or movement;
determining, based on the sensor data, multiple states of at least one person in the audience for the time periods during which the media program is presented to the audience;
normalizing one of the determined multiple states based, at least in part, on historical sensor data for the at least one person in the audience;
receiving one or more media types describing respective portions included in the media program;
determining, based on the one or more media types describing respective portions included in the media program and the multiple states of the person in the audience, multiple interest levels of the person for the time periods;
generating an interest graph that includes the determined multiple interest levels of the person for the time periods; and
providing the interest graph to an advertising entity effective to enable the advertising entity to measure a value of the media program, a media creator effective to enable the media creator to assess a potential value of a similar media program, a controller effective to enable the controller to control presentation of the media program, or a rating entity effective to enable the rating entity to automatically rate the media program for the person.
The talking state can be a general state indicating that a person is talking, though it may also include subcategories based on the content of the speech, such as talking about the media program (related talking) or talking that is unrelated to the media program (unrelated talking). State module 106 can determine which talking category through speech recognition.
Further, this information may be provided to other entities as well. Providing this information to a rating entity, for example, can be effective to enable the rating entity to automatically rate the media program for the person (e.g., four stars out of five or a “thumbs up”). Providing this information to a media controller, for example, may enable the media controller to improve media control and presentation, such as by pausing the media program responsive to all of the persons in the audience departing the MOM.
US20070150916 * 28 Dec 2005 28 Jun 2007 James Begole Using sensors to provide feedback on the access of digital content
US20120124603 * 29 Jun 2011 17 May 2012 Tadashi Amada Audio Signal Processor, Television Set and Computer Readable Medium
US20130145384 * 2 Dec 2011 6 Jun 2013 Microsoft Corporation User interface presenting an animated avatar performing a media reaction
1 "Advisory Action", U.S. Appl. No 10/396,653, (May 23, 2008),3 pages.
2 "Advisory Action", U.S. Appl. No. 10/396,653, (May 2, 2007),3 pages.
4 "Application Titled "Controlling Electronic Devices in a Multimedia System Through a Natural User Interface"", U.S. Appl. No. 13/038,024, filed Mar. 2, 2011, pp. 1-46.
5 "Application Titled "Interaction with Networked Screen Content Via Motion Sensing Device in Retail Setting"", U.S. Appl. No. 13/025,180, filed Feb. 11, 2011, pp. 1-23.
6 "Commanding Overview", MSDN, retrieved from <http://msdn.microsoft.com/en-us/library/ms752308.aspx> on Sep. 27, 2011, 11 pages.
7 "Commanding Overview", MSDN, retrieved from on Sep. 27, 2011, 11 pages.
8 "Corrected Notice of Allowance", U.S. Appl. No. 13/309,859, (Oct. 29, 2013), 3 pages.
9 "Definition of "Subscribe"", Retrieved from <http://dictionary.reference.com/browse/subscribe?s=t> on Jan. 7, 2015, 5 pages.
10 "Definition of "Synchronize"", Retrieved from <http://dictionary.reference.com/browse/synchronize?s=t> on Jan. 7, 2015, 4 pages.
11 "Designing CEC into your next HDMI Product", Quantum Data White Paper, Retrieved from Internet:. Quantum Data, Inc., Elgin, IL, USA, (May 13, 2006),12 pages.
12 "Designing CEC into your next HDMI Product", Quantum Data White Paper, Retrieved from Internet:<URL:http://www.quantumdata.com/pdf/CEC-white-paper.pdf>. Quantum Data, Inc., Elgin, IL, USA, (May 13, 2006),12 pages.
13 "European Search Report", European Patent Application No. 12195349.1, (Apr. 22, 2013), 3 pages.
15 "Final Office Action", U.S. Appl. No. 10/396,653, (Feb. 20, 2009), 12 pages.
16 "Final Office Action", U.S. Appl. No. 10/396,653, (Feb. 25, 2008),20 pages.
17 "Final Office Action", U.S. Appl. No. 10/396,653, (Feb. 26, 2007), 18 pages.
18 "Final Office Action", U.S. Appl. No. 11/626,794, (Jun. 11, 2009), 14 pages.
19 "Final Office Action", U.S. Appl. No. 12/474,453, (May 10, 2012), 14 pages.
20 "Final Office Action", U.S. Appl. No. 12/794,406, (Apr. 22, 2013),14 pages.
25 "Final Office Action", U.S. Appl. No. 13/114,359, Mar. 23, 2015, 12 pages.
26 "Final Office Action", U.S. Appl. No. 13/309,859, (May 15, 2013),13 pages.
27 "Final Office Action", U.S. Appl. No. 13/331,886, Jan. 7, 2015, 19 pages.
28 "Final Office Action", U.S. Appl. No. 13/363,689, Apr. 24, 2015, 24 pages.
29 "Final Office Action", U.S. Appl. No. 13/363,689, Feb. 11, 2014, 18 pages.
30 "Final Office Action", U.S. Appl. No. 13/411,859, Aug. 8, 2014, 16 pages.
31 "Final Office Action", U.S. Appl. No. 13/411,859, Jun. 3, 2015, 17 pages.
32 "Final Office Action", U.S. Appl. No. 13/439,284, (Jun. 3, 2013), 27 pages.
33 "Final Office Action", U.S. Appl. No. 13/439,284, Feb. 10, 2014, 15 pages.
34 "Final Office Action", U.S. Appl. No. 13/441,228, (Sep. 11, 2013),15 pages.
35 "Final Office Action", U.S. Appl. No. 13/441,228, May 21, 2015, 21 pages.
36 "Final Office Action", U.S. Appl. No. 13/482,867, Feb. 21, 2014, 15 pages.
37 "Final Office Action", U.S. Appl. No. 13/488,046, Dec. 10, 2013, 12 pages.
38 "Final Office Action", U.S. Appl. No. 13/488,046, Jan. 27, 2015, 13 pages.
39 "Final Office Action", U.S. Appl. No. 14/108,008, Feb. 3, 2015, 9 pages.
40 "Final Office Action", U.S. Appl. No.. 13/488,046, May 1, 2014, 12 pages.
41 "Foreign Notice of Allowance", Canadian Application No. 2775700, (Jan. 3, 2013), 1 page.
42 "Foreign Notice of Allowance", CN Application No. 201110159923.8, Dec. 2, 2014, 4 pages.
43 "Foreign Office Action", Canadian Application No. 2775700, (Aug. 24, 2012), 2 pages.
44 "Foreign Office Action", Canadian Application No. 2775814, (Aug. 24, 2012), 3 pages.
45 "Foreign Office Action", Canadian Application No. 2775814, (Dec. 14, 2012), 3 pages.
46 "Foreign Office Action", CN Application No. 201110159923.8, May 22, 2014, 10 pages.
47 "Foreign Office Action", CN Application No. 201110159923.8, Sep. 2, 2013, 13 pages.
48 "Foreign Office Action", European Patent Application No. 12194891.3, (Apr. 24, 2013),5 pages.
49 "Foreign Office Action", European Patent Application No. 12195349.1, (May 10, 2013), 5 pages.
50 "Future Media Internet Research Challenges and the Road Ahead", European Commission Information Society and Media, Available at ,(Apr. 2010),31 pages.
51 "Future Media Internet Research Challenges and the Road Ahead", European Commission Information Society and Media, Available at <http://www.gatv.ssr.upm.es/nextmedia/images/fmi-tf-white-paper-042010.pdf>,(Apr. 2010),31 pages.
52 "GWindows: Light-Weight Stereo Vision for Interaction", http://research.microsoft.com/~nuria/gwindows/htm, (Jul. 8, 2005),2 pages.
53 "GWindows: Light-Weight Stereo Vision for Interaction", http://research.microsoft.com/˜nuria/gwindows/htm, (Jul. 8, 2005),2 pages.
54 "International Search Report and Written Opinion", Application No. PCT/US2012/034641, (Nov. 30, 2012), 9 pages.
55 "International Search Report and Written Opinion", Application No. PCT/US2013/035348, Sep. 25, 2013, 16 pages.
56 "International Search Report and Written Opinion", Application No. PCT/US2013/038710, Jan. 8, 2014, 18 pages.
57 "International Search Report and Written Opinion", Application No. PCT/US2013/039591, Aug. 1, 2014, 10 pages.
58 "International Search Report", PCT Application No. PCT/US2010/036005, (Dec. 24, 2010),3 pages.
59 "KinEmote uses Kinect to translate key strokes for Windows applications", techshout.com [online], Retrieved from the Internet:<URL:http://www.techshout.com/gaming/2010/28/kinemote-uses-kinect-to-translate-key-strokes-for-windows-applications/>,(Dec. 28, 2010),2 pages.
60 "Measuring emotion over the web", Retrieved at <<http://www.affectiva.com/affdex/>>, Retrieved Date: Nov. 4, 2011, pp. 3.
61 "Measuring emotion over the web", Retrieved at >, Retrieved Date: Nov. 4, 2011, pp. 3.
62 "Non-Final Office Action", U.S. Appl. No. 10/396,653, (Sep. 19, 2006),24 pages.
63 "Non-Final Office Action", U.S. Appl. No. 10/396,653, (Sep. 6, 2007), 17 pages.
64 "Non-Final Office Action", U.S. Appl. No. 10/396,653, (Sep. 8, 2008), 13 pages.
65 "Non-Final Office Action", U.S. Appl. No. 11/626,794, (Dec. 23, 2008), 18 pages.
66 "Non-Final Office Action", U.S. Appl. No. 11/626,794, (Oct. 27, 2009), 15 pages.
67 "Non-Final Office Action", U.S. Appl. No. 12/474,453, (Sep. 6, 2011), 10 pages.
68 "Non-Final Office Action", U.S. Appl. No. 12/794,406, (Sep. 14, 2012),17 pages.
69 "Non-Final Office Action", U.S. Appl. No. 12/794,406, Sep. 6, 2013, 13 pages.
70 "Non-Final Office Action", U.S. Appl. No. 12/972,837, (Jun. 26, 2013),10 pages.
71 "Non-Final Office Action", U.S. Appl. No. 13/025,180, (Apr. 5, 2013),17 pages.
72 "Non-Final Office Action", U.S. Appl. No. 13/025,180, Jan. 15, 2015, 17 pages.
73 "Non-Final Office Action", U.S. Appl. No. 13/039,024, Apr. 7, 2014, 12 pages.
74 "Non-Final Office Action", U.S. Appl. No. 13/093,621, (Jun. 20, 2013), 7 pages.
75 "Non-Final Office Action", U.S. Appl. No. 13/114,359, Oct. 20, 2014, 7 pages.
76 "Non-Final Office Action", U.S. Appl. No. 13/118,884, Dec. 3, 2013, 10 pages.
77 "Non-Final Office Action", U.S. Appl. No. 13/309,589, (Dec. 18, 2012),10 pages.
78 "Non-Final Office Action", U.S. Appl. No. 13/331,886, Jun. 19, 2014, 18 pages.
79 "Non-Final Office Action", U.S. Appl. No. 13/363,689, (Jul. 26, 2013),18 pages.
80 "Non-Final Office Action", U.S. Appl. No. 13/363,689, Sep. 15, 2014, 21 pages.
81 "Non-Final Office Action", U.S. Appl. No. 13/411,859, Mar. 11, 2014, 13 pages.
82 "Non-Final Office Action", U.S. Appl. No. 13/411,859, Nov. 5, 2014, 16 pages.
83 "Non-Final Office Action", U.S. Appl. No. 13/439,284, (Feb. 25, 2013),31 pages.
84 "Non-Final Office Action", U.S. Appl. No. 13/439,284, (Nov. 8, 2013),14 pages.
85 "Non-Final Office Action", U.S. Appl. No. 13/441,228, (Mar. 20, 2013),12 pages.
86 "Non-Final Office Action", U.S. Appl. No. 13/441,228, Oct. 2, 2014, 18 pages.
87 "Non-Final Office Action", U.S. Appl. No. 13/482,867, (Nov. 5, 2013),13 pages.
88 "Non-Final Office Action", U.S. Appl. No. 13/488,046, (Jun. 13, 2013), 8 pages.
89 "Non-Final Office Action", U.S. Appl. No. 13/488,046, Jul. 23, 2014, 12 pages.
90 "Non-Final Office Action", U.S. Appl. No. 13/488,046, Mar. 14, 2014, 11 pages.
91 "Non-Final Office Action", U.S. Appl. No. 13/866,699, Feb. 7, 2014, 15 pages.
92 "Non-Final Office Action", U.S. Appl. No. 14/108,008, Aug. 14, 2014, 16 pages.
93 "Notice of Allowance", U.S. Appl. No. 10/396,653, (Nov. 19, 2009),7 pages.
94 "Notice of Allowance", U.S. Appl. No. 11/626,794, (May 13, 2010),4 pages.
95 "Notice of Allowance", U.S. Appl. No. 12/474,453, (Dec. 12, 2012), 8 pages.
96 "Notice of Allowance", U.S. Appl. No. 12/794,406, Jan. 21, 2015, 11 pages.
97 "Notice of Allowance", U.S. Appl. No. 12/972,837, (Oct. 11, 2013), 10 pages.
98 "Notice of Allowance", U.S. Appl. No. 13/093,621, (Aug. 21, 2013), 7 pages.
99 "Notice of Allowance", U.S. Appl. No. 13/118,884, Feb. 4, 2014, 7 pages.
100 "Notice of Allowance", U.S. Appl. No. 13/309,859, (Sep. 4, 2013), 7 pages.
101 "Notice of Allowance", U.S. Appl. No. 13/439,284, Jul. 21, 2014, 6 pages.
102 "Notice of Allowance", U.S. Appl. No. 13/482,867, Sep. 30, 2014, 17 pages.
103 "Notice of Allowance", U.S. Appl. No. 13/866,699, Sep. 17, 2014, 6 pages.
104 "Notice of Allowance", U.S. Appl. No. 14/108,008, May 11, 2015, 6 pages.
105 "PCT Search Report and Written Opinion", Application No. PCT/US2013/035047, (Jul. 5, 2013),10 pages.
106 "Recognizing Visual Focus of Attention from Head Pose in Natural Meetings", IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics-Special Issue on Human Computing, vol. 39, Issue 1, (Feb. 2009), 36 pages.
107 "Response to Non-Final Office Action", U.S. Appl. No. 12/794,406, (Feb. 14, 2013), 12 pages.
108 "Response to Office Action", U.S. Appl. No. 12/794,406, (Jul. 22, 2013), 9 pages.
109 "Restriction Requirement", U.S. Appl. No. 13/039,024, (Oct. 1, 2013), 5 pages.
110 "Restriction Requirement", U.S. Appl. No. 13/114,359, (Sep. 10, 2013), 6 pages.
111 "Restriction Requirement", U.S. Appl. No. 13/114,359, Dec. 18, 2013, 6 pages.
112 "Restriction Requirement", U.S. Appl. No. 13/482,867, (Sep. 6, 2013), 6 pages.
113 "Restriction Requirement", U.S. Appl. No. 13/488,046, (May 2, 2013), 5 pages.
114 "Signal Processing Institute", http://Itswww.epfl.ch/˜alahi/student-projects/proposals.shtml#4, Downloaded Feb. 2, 2009, 4 pages.
115 "Simulation and Training", Division Incorporated,(1994),6 Pages.
116 "Summons to Attend Oral Proceedings", EP Application No. 12194891.3, Sep. 17, 2014, 7 Pages.
117 "Summons to Attend Oral Proceedings", EP Application No. 12195349.1, Sep. 17, 2014, 7 Pages.
118 "Supplementary European Search Report", European Patent Application Number 12194891.3, (Apr. 4, 2013),3 pages.
119 "The Case for Kinect", Eurogamer [online] Retrieved from the Internet on Aug. 20, 2010: URL:., (Aug. 7, 2010),pp. 1-7.
120 "The Case for Kinect", Eurogamer [online] Retrieved from the Internet on Aug. 20, 2010: URL:<http://www.eurogamer.net/articles/digitalfoundry-the-case-for-kinect-article?page=2>., (Aug. 7, 2010),pp. 1-7.
121 "Virtual High Anxiety", Tech update, (Aug. 1995),1 Page.
122 Agarwal, Ankur et al., "High Precision Multi-touch Sensing on Surfaces using Overhead Cameras", Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer System, available at <<http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4384130>>,(Nov. 19, 2007),4 pages.
123 Agarwal, Ankur et al., "High Precision Multi-touch Sensing on Surfaces using Overhead Cameras", Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer System, available at >,(Nov. 19, 2007),4 pages.
124 Aggarwal, et al., "Human Motion Analysis: A Review", IEEE Nonrigid and Articulated motion Workshop, University of Texas at Austin, Austin, TX,(1997),pp. 90-102.
125 Ali, Azarbayejani et al., "Real-Time Self-Calibrating Stereo Person Tracking Using 3-D Shape Estimation from Blob Features", Proceedings of ICPR, Vienna, Austria, (Aug. 1996),pp. 627-632.
126 Althoff, Frank et al., "Using Multimodal Interaction to Navigate in Arbitrary Virtual VRML Worlds", PUI 2001 Orlando, FL USA, available at ,(2001),8 pages.
127 Althoff, Frank et al., "Using Multimodal Interaction to Navigate in Arbitrary Virtual VRML Worlds", PUI 2001 Orlando, FL USA, available at <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.16.8034&rep=rep1&type=pdf>,(2001),8 pages.
128 Argyros, et al., "Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse", Retrieved from: <http://www.ics.forth.gr/˜argyros/mypapers/2006-05-hci-virtualmouse.pdf> on Oct. 31, 2007,(2006),pp. 40-51.
129 Argyros, et al., "Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse", Retrieved from: on Oct. 31, 2007,(2006),pp. 40-51.
130 Asteriadis, Stylianos et al., "Estimation of Behavioral User State based on Eye Gaze and Head Pose-Application in an e-Learning Environment", Journal of Multimedia Tools and Applications, vol. 41, Issue 3, (Feb. 2009), 25 pages.
131 Azarbayejani, et al., "Visually Controlled Graphics", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, No. 6, (Jun. 1993),pp. 602-605.
132 Azoz, Yusuf et al., "Reliable Tracking of Human Arm Dynamics by Multiple Cue Integration and Constraint Fusion", IEEE Conference on Computer Vision and Pattern Recognition, (1998),6 pages.
133 Ba, Sileye O., et al., "Head Pose Tracking and Focus of Attention Recognition Algorithms in Meeting Rooms", Proceedings of the 1st International Evaluation Conference on Classification of Events, Activities and Relationships, (Apr. 6, 2006),12 pages.
134 Baudel, Thomas et al., "Charade: Remote Control of Objects using Free-Hand Gestures", Communications of the ACM, vol. 36. No. 7, (Jul. 1993),10 pages.
135 Becker, David A., "Sensei: A Real-Time Recognition, Feedback and Training System for T'ai Chi Gestures", http://citeseer.ist.psu.edu/cache/papers/cs/405/ftp:zSzzSzwhitechapel.media.mit.eduzSzpubzSztech-reporterzsSzTR-426pdf/becker97sensei.pdf, (Jun. 1993),50 pages.
136 Berard, Francois "The Perceptual Window-Head Motion as a New Input Stream", Proceedings of the Seventh IFIP Conference of Human-Computer Interaction, (1999),238-244.
137 Bhuiyan, Moniruzzaman et al., "Gesture-controlled user interfaces, what have we done and what's next?", Retrieved at <<http://www.newi.ac.uk/computing/research/pubs/SEIN-BP.pdf>>, (Nov. 27, 2009),10 pages.
138 Bhuiyan, Moniruzzaman et al., "Gesture-controlled user interfaces, what have we done and what's next?", Retrieved at >, (Nov. 27, 2009),10 pages.
139 Bobic, Nick "Rotating Objects Using Quaternions", Retrieved from the Internet on Aug. 20, 2010: URL http://www.gamasutra.com/view/feature/3278/rotating-objects-quarternions.php.?page=2>., (Jul. 5, 1998),14 pages.
140 Boser, Bernhard E., et al., "A Training Algorithm for Optimal Margin Classifiers", Proceedings of the Fifth Annual Workshop on Computational Learning Theory, (Jul. 27, 1992), 9 pages.
141 Boverie, S. et al., "Comparison of Structured Light and Stereovision Sensors for New Airbag Generations", Control Engineering Practice vol. 11, Issue 12 (2003), available at <<http://homepages.laas.fr/lerasle/pdf/cep03.pdf>>,(Dec. 2003),pp. 1413-1421.
142 Boverie, S. et al., "Comparison of Structured Light and Stereovision Sensors for New Airbag Generations", Control Engineering Practice vol. 11, Issue 12 (2003), available at >,(Dec. 2003),pp. 1413-1421.
143 Bowman, Doug A., et al., "New Directions in 3D User Interfaces", The International Journal of Virtual Reality, retrieved from <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.99.1121&rep=rep1&type=pdf> on Nov. 15, 2011,(2006),pp. 3-14.
144 Bowman, Doug A., et al., "New Directions in 3D User Interfaces", The International Journal of Virtual Reality, retrieved from on Nov. 15, 2011,(2006),pp. 3-14.
145 Bradley, Margaret M., et al., "Measuring Emotion: The Self-Assessment Manikin and the Semantic Differential"In Journal of Behavior Therapy and Experimental Psychiatry, vol. 25, Issue 1, (Mar. 1994),11 pages.
146 Breen, David et al., "Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality", Technical report ECRC-95-02 European Computer-Industry Research Centre GmbH, Munich, Germany, (1995),22 Pages.
147 Brogan, David et al., "Dynamically Simulated Characters in Virtual Environments", vol. 18, Issue 5, IEEE Computer Graphics and Applications, (Sep./Oct. 1998),pp. 58-69.
148 Buxton, William et al., "A Study of Two-Handed Input", Proceedings of CHI'86, (1986),pp. 321-326.
149 Cedras, Claudette et al., "Motion-based Recognition: A Survey", IEEE Proceedings, Image and Vision Computing, vol. 13, No. 2, (Mar. 1995),pp. 129-155.
150 Chang, Chih-Chung et al., "LIBSVM: A Library for Support Vector Machines", retrieved from <http://www.csie.ntu.edu.tw/˜cjlin/libsvm/> on Apr. 1, 2013, 4 pages.
151 Crawford, Stephanie "How Microsoft Kinect Works", Howstuffworks[online] Retrieved from the Internet on Aug. 19, 2010: URL:., pp. 1-5.
152 Crawford, Stephanie "How Microsoft Kinect Works", Howstuffworks[online] Retrieved from the Internet on Aug. 19, 2010: URL:<http://electronics.howstuffworks.com/microsoft-kinect.htm/printable>., pp. 1-5.
153 Dalton, Angela B., et al., "Sensing User Intention and Context for Energy Management", Duke University, Department of Computer Science, Retrieved from the Internet:, (Feb. 23, 2003),5 pages.
154 Dalton, Angela B., et al., "Sensing User Intention and Context for Energy Management", Duke University, Department of Computer Science, Retrieved from the Internet:<URL:http://www.cs.duke/edu/ari/millywatt/faceoff.pdf>, (Feb. 23, 2003),5 pages.
155 Darrell, T et al., "Integrated Person Tracking Using Stereo, Color and Pattern Detection", Proceedings of the Conference on Computer Vision and Pattern Recognition, (1998),pp. 601-609.
156 El Kaliouby, Rana et al., "Real Time Inference of Complex Mental States from Facial Expressions and Head Gestures", Proceedings of Conference on Computer Vision and Pattern Recognition Workshop, (Jun. 27, 2004), 20 pages.
157 Fisher, et al., "Virtual Environment Display System", ACM Workshop on Interactive 3D Graphics, Chapel Hill, NC, (Oct. 1986),12 Pages.
158 Fitzgerald, et al., "Integration of Kinematic Analysis into Computer Games for Exercise", Proceedings of CGames 2006-9th International Conference on Computer Games: Al, Animation, Mobile, Educational and Serious Games, Dublin Ireland, (Nov. 2006),pp. 24-28.
159 Fitzgerald, Will et al., "Multimodal Event Parsing for Intelligent User Interfaces", IUI Conference, (Jan. 2003),8 pages.
160 Freed, Natalie "Toys Keeping in Touch: Technologies for Distance Play", Retrieved from <<http://people.ischool.berkeley.edu/˜daniela/tei2010/gsc09e-freed.pdf>>, (Jan. 24, 2010),2 pages.
161 Freed, Natalie "Toys Keeping in Touch: Technologies for Distance Play", Retrieved from >, (Jan. 24, 2010),2 pages.
162 Freeman, William et al., "Television Control by Hand Gestures", International Workshop on Automatic Face and Gesture Recognition, (1995),pp. 179-183.
163 Gonzalez, Barb "HDMI CEC", Home Theater University [online] Retrieved from the Internet:, (Mar. 24, 2008),3 pages.
164 Gonzalez, Barb "HDMI CEC", Home Theater University [online] Retrieved from the Internet:<URL:http://www.hometheatre.com/hookmeup/208hook>, (Mar. 24, 2008),3 pages.
165 Grace, Richard et al., "A Drowsy Driver Detection System for Heavy Vehicles", Proceedings of the 17th Digital Avionics Systems Conference, vol. 2, (Oct. 31, 1998), 8 pages.
166 Granieri, John P., et al., "Simulating Humans in VR", The British Computer Society, Academic Press, (Oct. 1994),15 Pages.
167 Grunder, Alexander "Updated: Xbox 360 Kinect Hand Gesture Media Controls, Voice Control, TV Video Chat.", eHomeUpgrade [online] retrieved from the internet:<URL:http://www.ehomeupgrade.com/2010/06/14/updated-xbox-360-kinect-hand-gesture-media-controls-voice-control-tv-video-chat/>, (Jun. 14, 2010),8 pages.
168 Guiard, Yves "Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model", Journal of Motor Behavior, vol. 19 Issue 4, (1987),486-517.
169 Guler, Sadiye Z., "Split and Merge Behavior Analysis and Understanding Using Hidden Markov Models", (Oct. 8, 2002),21 pages.
170 Guyon, Isabelle et al., "An Introduction to Variable and Feature Selection", In Journal of Machine Learning Research, vol. 3, (Mar. 2003), pp. 1157-1182.
171 Hardin, Winn "Machine Vision Makes the Leap to Consumer Gaming", Machine Vision Online, retrieved from <<http://www.machinevisiononline.org/vision-resources-details.cfm?content-id=2398>> on Mar. 14, 2011,(Dec. 8, 2010),3 pages.
172 Hardin, Winn "Machine Vision Makes the Leap to Consumer Gaming", Machine Vision Online, retrieved from > on Mar. 14, 2011,(Dec. 8, 2010),3 pages.
173 Hasegawa, Shoichi et al., "Human-Scale Haptic Interaction with a Reactive Virtual Human in a Real-Time Physics Simulator", ACM Computers in Entertainment, vol. 4, No. 3, (Jul., 2006),12 Pages.
174 He, Lei "Generation of Human Body Models", University of Auckland, New Zealand (Apr. 2005),111 Pages.
175 Hongo, Hitoshi et al., "Focus of Attention for Face and Hand Gesture Recognition Using Multiple Cameras", 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, (Mar. 2000),pp. 156-161.
176 Horvitz, Eric "Principles of Mixed-Initiative User Interfaces", Proceedings of CHI, (1999),8 pages.
177 Horvitz, Eric et al., "A Computational Architecture for Conversation", Proceedings of the Seventh International Conference on User Modeling, (1999),pp. 201-210.
178 Hourcade, Juan P., "Architecture and Implementation of Java Package for Multiple Input Devices (MID)", HCIL Technical Report No. 99-08 (May 1999); http://www.cs.umd.edu/hcil, (May 1999),7 pages.
179 Isard, Michael et al., "CONDENSATION-Conditional Density Propagation for Visual Tracking", International Journal of Computer Vision 29(1), Netherlands, (1998),pp. 5-28.
180 Jacko, "HDI Dune Prime 3.0 Part 2.", Retrieved from the internet: , (Jun. 19, 2010),15 pages.
181 Jacko, "HDI Dune Prime 3.0 Part 2.", Retrieved from the internet: <URL:http://www.jacko.my/2010/06/hdi-dune-prime-30-part-2.html>, (Jun. 19, 2010),15 pages.
182 Jojic, Nebojsa et al., "Detection and Estimation of Pointing Gestures in Dense Disparity Maps", Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, (2000),pp. 1000-1007.
183 Kabbash, P et al., "The "Prince" Technique: Fitts' Law and Selection Using Area Cursors", Proceedings of CHI'95, http://www.billbuxton.com/prince.html, (1995),pp. 273-279.
184 Kanade, et al., "Development of Video-Rate Stereo Machine", Proceedings of 94 ARPA Image Understanding Workshop, (1994),pp. 549-558.
185 Kanade, Takeo et al., "A Stereo Machine for Video-rate Dense Depth Mapping and Its New Applications", IEEE Computer Society Conference on Computer Vision and Pattern Recognition, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA,(1996),pp. 196-202.
186 Kapoor, Ashish et al., "Multimodal Affect Recognition in Learning Environments", Proceedings of the 13th Annual ACM International Conference on Multimedia, (Nov. 6, 2005), 6 pages.
187 Kim, Song-Gook et al., "Multi-Touch Tabletop Interface Technique for HCI", retrieved from <<http://210.119.33.7/apis6/paper/data/63-multi-touch%20tabl.pdf>> on Mar. 16, 2011,4 pages.
188 Kim, Song-Gook et al., "Multi-Touch Tabletop Interface Technique for HCI", retrieved from > on Mar. 16, 2011,4 pages.
189 Kjeldsen, Frederik "Visual Interpretation of Hand Gestures as Practical Interface Modality", Ph.D. Dissertation, Columbia University Department of Computer Science, (1997),168 pages.
190 Klompmaker, Florian "D5.1-State of the art analysis and recommendations on 'Context Awareness', 'Human Computer Interaction' and 'Mobile Users Interfaces'", Information Technology for European Advancement (ITEA), Local Mobile Services, Retrieved from the Internet:, (Jul. 2, 2007),55 pages.
191 Klompmaker, Florian "D5.1-State of the art analysis and recommendations on 'Context Awareness', 'Human Computer Interaction' and 'Mobile Users Interfaces'", Information Technology for European Advancement (ITEA), Local Mobile Services, Retrieved from the Internet:<URL:http://www.loms-itea.org/deliverables/LOMS-D5.1-v1.0.pdy>, (Jul. 2, 2007),55 pages.
192 Kohler, Marcus "Technical Details and Ergonomical Aspects of Gesture Recognition applied in Intelligent Home Environments", Germany, (1997),35 Pages.
193 Kohler, Markus "Special Topics of Gesture Recognition Applied in Intelligent Home Environments", In Proceedings of the Gesture Workshop, Germany, (1998),12 Pages.
194 Kohler, Markus "Vision Based Remote Control in Intelligent Home Environments", University of Erlangen-Nuremberg, Germany, (1996),8 Pages.
195 Kolsch, Mathias et al., "Vision-Based Interfaces for Mobility", Retrieved from <<http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1331713>>, (Aug. 22, 2004),9 pages.
196 Kolsch, Mathias et al., "Vision-Based Interfaces for Mobility", Retrieved from >, (Aug. 22, 2004),9 pages.
197 Kwon, et al., "Combining Body Sensors and Visual Sensors for Motion Training", Computer Graphics Laboratory, http://graphics.ethz.ch/~dkwon/downloads/publications/ace05-ace.pdf, Downloaded 2009,(2005),pp. 1-8.
198 Kwon, et al., "Combining Body Sensors and Visual Sensors for Motion Training", Computer Graphics Laboratory, http://graphics.ethz.ch/˜dkwon/downloads/publications/ace05-ace.pdf, Downloaded 2009,(2005),pp. 1-8.
199 Latoschik, Marc E., "A User Interface Framework for Multimodal VR Interactions", ICMI'05, Trento, Italy, Oct. 4-6, 2005, available at ,(Oct. 4, 2005),8 pages.
200 Latoschik, Marc E., "A User Interface Framework for Multimodal VR Interactions", ICMI'05, Trento, Italy, Oct. 4-6, 2005, available at <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.2941&rep=rep1&type=pdf>,(Oct. 4, 2005),8 pages.
201 Le, Nguyen T., "EmuPlayer: Music Recommendation System Based on User Emotion Using Vital-sensor", Thesis, Keio University, Available at ,(2010),85 pages.
202 Le, Nguyen T., "EmuPlayer: Music Recommendation System Based on User Emotion Using Vital-sensor", Thesis, Keio University, Available at <http://www.sfc.wide.ad.jp/thesis/2011/files/sunny-publish-thesis.pdf>,(2010),85 pages.
203 Leal, Anamary et al., "Initial Explorations into the User Experience of 3D File Browsing", Proceedings of HCI 2009, retrieved from <http://www.eecs.ucf.edu/isuelab/publications/pubs/p339-leal-3dfiles.pdf> on Nov. 15, 2011,(Sep. 2009),pp. 339-344.
204 Leal, Anamary et al., "Initial Explorations into the User Experience of 3D File Browsing", Proceedings of HCI 2009, retrieved from on Nov. 15, 2011,(Sep. 2009),pp. 339-344.
205 Li, Stan Z., et al., "A Near-Infrared Image Based Face Recognition System", available at <<http://www.cbsr.ia.ac.cn/Li%20Group/papers/IR-Face-FG06.pdf>>,(Apr. 2006),6 pages.
206 Li, Stan Z., et al., "A Near-Infrared Image Based Face Recognition System", available at >,(Apr. 2006),6 pages.
207 Liang, Lin et al., "Face Alignment via Component-Based Discriminative Search", Computer Vision, ECCV 2008, Lecture Notes in Computer Science, vol. 5303, (2008),14 pages.
208 Livingston, Mark A., "Vision-based Tracking with Dynamic Structured Light for Video See-through Augmented Reality", TheUniversity of NorthCarolina at ChapelHill, North Carolina, USA, (1998),145 Pages.
209 Long, Jr., Allan C., et al., "Implications for a Gesture Design Tool", Proceedings of Chi'99, (1999),pp. 40-47.
210 Maes, Pattie et al., "The ALIVE System: Wireless, Full-body, Interaction with Autonomous Agents", ACM Multimedia Systems, Special Issue on Multimedia and Multisensory Virtual Worlds, (Nov. 1995),17 pages.
211 Maltby, John R., "Using Perspective in 3D File Management: Rotating Windows and Billboarded Icons", Proceedings of the International Conference on Computer Graphics, Imaging and Visualisation (CGIV'06 ), available at ,(Jul. 28, 2006),8 pages.
212 Maltby, John R., "Using Perspective in 3D File Management: Rotating Windows and Billboarded Icons", Proceedings of the International Conference on Computer Graphics, Imaging and Visualisation (CGIV'06 ), available at <http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1663764>,(Jul. 28, 2006),8 pages.
213 Martin, Benoit "VirHKey: A VIRtual Hyperbolic KEYboard with Gesture Interaction and Visual Feedback for Mobile Devices", http://delivery.acm.org/10.1145/1090000/1085794/p99-martin.pdf?key1=1085794&key2=4890534611&coll=portal&dl=ACM&CFID=11111111&CFTOKEN=2222222, (Sep. 2005),8 pages.
214 McCrae, James et al., "Exploring the Design Space of Multiscale 3D Orientation", AVI '10, retrieved from <http://www.autodeskresearch.com/pdf/avi2010-final.pdf> on Nov. 15, 2011,(May 29, 2010),8 pages.
215 McCrae, James et al., "Exploring the Design Space of Multiscale 3D Orientation", AVI '10, retrieved from on Nov. 15, 2011,(May 29, 2010),8 pages.
216 McDuff, Daniel "Affective Storytelling: Automatic Measurement of Story Effectiveness from Emotional Responses Collected over the Internet", PhD Thesis, retrieved from <http://web.media.mil.edu/˜djmcduff/documents/McDuff-Thesis-Proposal.pdf> pdf>>,(Jun. 6, 2012),16 pages.
217 McDuff, Daniel et al., "Crowdsourcing Facial Responses to Online Videos", Proceedings of the IEEE Transactions on Affective Computing, vol. 3, Issue 4, (Oct. 2012), pp. 456-468.
218 McDuff, et al., "AffectAura: An Intelligent System for Emotional Memory", In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Retrieved from <http://www.affectiva.com/assets/Q-Sensor-Microsoft-Publication.pdf>,(May 5, 2012),10 pages.
219 Mignot, Christopher et al., "An Experimental Study of Future 'Natural' Multimodal Human-Computer Interaction", Proceedings of INTERCHI93, (1993),pp. 67-68.
220 Millan, Maria S., et al., "Unsupervised Defect Segmentation of Patterned Materials under NIR Illumination", Journal of Physics: Conference Series 274 (2011) 012044, available at <<http://iopscience.iop.org/1742-6596/274/1/012044/pdf/1742-6596-274-1-012044.pdf>>,(2011),9 pages.
221 Millan, Maria S., et al., "Unsupervised Defect Segmentation of Patterned Materials under NIR Illumination", Journal of Physics: Conference Series 274 (2011) 012044, available at >,(2011),9 pages.
222 Minge, Michael "Dynamics of User Experience", Workshop on Research Goals and Strategies for Studying User Experience and Emotion, Available at ,(2008),pp. 1-5.
223 Minge, Michael "Dynamics of User Experience", Workshop on Research Goals and Strategies for Studying User Experience and Emotion, Available at <http://www.cs.uta.fi/˜ux-emotion/submissions/Minge.pdf>,(2008),pp. 1-5.
224 Miyagawa, Ryohei et al., "CCD-Based Range-Finding Sensor", IEEE Transactions on Electron Devices, vol. 44, No. 10, (Oct. 1997),pp. 1648-1652.
225 Moeslund, Thomas B., et al., "A Survey of Computer Vision-Based Human Motion Capture", Computer Vision and Image Understanding: CVIU, vol. 81, No. 3, (2001),pp. 231-269.
226 Morency, Louis-Philippe et al., "Contextual Recognition of Head Gestures", Trento, Italy http://delivery.acm.org/10.1145/1090000/1088470/p18-morency.pdf?key1=1088470&key2=8870534611&coll=portal&dl=ACM&CFID=11111111&CFTOKEN=2222222, 7 pages.
227 Morrison, Gerald D., "A Camera-Based Touch Interface for Pervasive Displays", Retrieved from <<http://ubicomp.algoritmi.uminho.pt/perdisplay/docs/Morrison-Camera%20Touch-SV-Rev1.pdf>> on Mar. 16, 2011,7 pages.
228 Morrison, Gerald D., "A Camera-Based Touch Interface for Pervasive Displays", Retrieved from <> on Mar. 16, 2011,7 pages.
229 Moscovich, Tomer "Multi-touch Interaction", Brown University, CHI 2006, Apr. 22-27, 2006, Montreal, Quebec, Canada, (Apr. 22, 2006),4 pages.
230 Moyle, et al., "Gesture Navigation: An Alternative 'Back' for the Future", Proceedings of CHI'02, (2002),pp. 882-823.
231 Nielsen, Michael et al., "A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for Man-Machine Interaction", Technical Report CVMT 03-01, ISSN 1601-3646. CVMT, Aalborg University, (Mar. 2003),12 pages.
232 Oh, Alice et al., "Evaluating Look-to-talk: A Gaze-Aware Interface in a Collaborative Environment", CHI'02, (2002),650-651.
233 Op Den Akker, Rieks et al., "Supporting Engagement and Floor Control in Hybrid Meetings", In Cross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions, (Jul. 2009),15 pages.
234 Oviatt, Sharon "Ten Myths of Multimodal Interaction", Communications of the ACM. vol. 42, No. 11, (Nov. 1999),8 pages.
235 Paquit, Vincent et al., "Near-Infrared Imaging and Structured Light Ranging for Automatic Catheter Insertion", Proceedings of SPIE vol. 6141, 61411T, (2006), available at <<http://www.cs.rpi.edu/˜chakrn2/work/catheter-plan/paquit-06.pdf>>,(2006),9 pages.
236 Parrish, Kevin "Microsoft Does Want Core Games, FPS for Kinect", Tom's Guide: Tech for Real Life [online], Retrieved from the Internet on Aug. 20, 2010: URL: <http://www.tomsguide.com/us/Core-Gamers-Kinect-FPS-Action,news-7195.html>., (Jun. 23, 2010),1 page.
237 Pavlou, Paul A., et al., "Measuring the Effects and Effectiveness of Interactive Advertising: A Research Agenda", Journal of Interactive Advertising, vol. 1, No. 1 (Fall 2000), Available at <http://scholargoogle.co.in/scholar-url?hl=en&q=http://jiad.org/download%3Fp%3D6&sa=X&scisig=AAGBfm3He5PA4sgMGDXTyQuqaVQn4Q3nZw&oi=scholarr>,(Oct. 2000),pp. 62-78.
238 Pavlovic, Vladimir et al., "Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, No. 7, (Jul. 1997),pp. 677-695.
239 Peacock, James et al., "Which Broadcast Medium Better Drives Engagement? Measuring the Powers of Radio and Television with Electromyography and Skin-Conductance Measurements", In Journal of Advertising Research, vol. 51, Issue 4, (Dec. 2011), 8 pages.
240 Poels, Karolien et al., "How to Capture the Heart? Reviewing 20 Years of Emotion Measurement in Advertising", In the Journal of Advertising Research, vol. 46, Issue 1, (Mar. 2006), 48 pages.
241 Qian, et al., "A Gesture-Driven Multimodal Interactive Dance System", IEEE International Conference on Multimedia and Expo, Taipei, (Jun. 2004),pp. 1579-1582.
242 Raymer, A "Gestures and Words: Facilitating Recovery in Aphasia", The ASHA Leader, http://www.asha.org/about/publications/leader-online/archives/2007/070619/f070619a.htm, (Jun. 19, 2007),6 pages.
243 Rigoll, Gerhard et al., "High Performance Real-Time Gesture Recognition Using Hidden Markov Models", Gesture and Sign Language in Human-Computer Interaction, vol. LNAI 1371, Frohlich, ed., (1997),pp. 69-80.
244 Rosenhahn, Bodo et al., "Automatic Human Model Generation", University of Auckland (CITR), New Zealand, (2005),pp. 41-48.
245 Sakir, Samit "Kinect is your personal trainer in EA Sports Active 2", Gamerss [online] Retrieved from the Internet on Aug. 20, 2010: URL: <http://www.gamerss.co.uk/kinect-is-your-personal-trainer-in-ea-sports-active-2>., (Jul. 26, 2010),4 pages.
246 Schick, Alexander et al., "Extending Touch: Towards Interaction with Large-Scale Surfaces", ITS '09, Nov. 23-25, 2009, Banff, Alberta, Canada, available at <<http://www.iosb.fraunhofer.de/servlet/is/33404/urn-nbn-de-0011-n-1159494.pdf>>,(Nov. 23, 2009),8 pages.
247 Schielel, Seth "A Home System Leaves Hand Controls in the Dust, Kinect by Microsoft Keeps You Entertained Hands Free", The New York Times [online]Retrieved from the Internet:<URL:http://www.nytimes.com/2010/11/04/arts/television/04kinect.html>, (Nov. 4, 2010),3 pages.
248 Shao, Jiang et al., "An Open System Architecture for a Multimedia and Multimodal User Interface", Japanese Society for Rehabilitation of Persons with Disabilities (JSRDP), Japan, (Aug. 24, 1998),8 Pages.
249 Sharma, et al., "Method of Visual and Acoustic Signal Co-Analysis for Co-Verbal Gesture Recognition", U.S. Appl. No. 60/413,998, (Sep. 19, 2002),16 pages.
250 Sharma, Rajeev M., et al., "Speech-Gesture Driven Multimodal Interfaces for Crisis Management", Proceedings of IEEE Special Issue on Multimodal Human-Computer Interface, (2003),28 pages.
251 Shen, Guobin et al., "Dita: Enabling Gesture-Based Human-Device Interaction using Mobile Phone", Retrieved at <<:http://research.microsoft.com/en-us/people/jackysh/dita.pdf>>, (Oct. 1, 2010),pp. 1-14.
252 Sheridan, Thomas et al., "Virtual Reality Check", Technology Review, vol. 96, No. 7, (Oct. 1993),9 Pages.
253 Shivappa, et al., "Person Tracking with Audio-Visual Cues Using the Iterative Decoding Framework", IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance, AVSS 08, Santa Fe, NM, (Sep. 2008),260-267.
254 Simeone, Luca et al., "Toys++ AR Embodied Agents as Tools to Learn by Building", Retrieved from <<http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=05572598>>, (Jul. 5, 2010),2 pages.
255 Stevens, Jane "Flights into Virtual Reality Treating Real World Disorders", The Washington Post, Science Psychology, (Mar. 27, 1995),2 Pages.
256 Tep, et al., "Web Site Quality Evaluation Combining Eyetracking and Physiologicial Measures to Self-Reported Emotions: An Exploratory Research", Retrieved at <<http://www.noldus.com/mb2008/individual-papers/FPS-eye -tracking/FPS-eye-tracking-Prom-Tep.pdf>>, Proceedings of Measuring Behavior, Aug. 26-29, 2008, pp. 242-225.
257 Tilley, Steve "E3 09: Project Natal exposed", Load This [online] Retrieved from the Internet:<URL:http://blogs.canoe.ca/loadthis/general/e3-09-project-natal-exposed/>, (Jun. 1, 2009),4 pages.
258 Todd, Paul "Google Campaign Insights: Better Measurement for Display Advertising", Retrieved from: <http://adwordsagency.blogspot.com/2009/10/campaign-insights-better-measurement.html> on Nov. 14, 2011,(Oct. 19, 2009),3 pages.
259 Toyama, Kentaro et al., "Probabilistic Tracking in a Metric Space", Eighth International Conference on Computer Vision, Vancouver Canada, vol. 2, (Jul. 2001),8 pages.
260 Tresadern, Philip A., et al., "Visual Analysis of Articulated Motion", DPhil Thesis, University of Oxford, Oxford, U.K., (Oct. 12, 2006),1-171.
261 U.S. Appl. No. 12/794,406, filed Jun. 4, 2010,37 pages.
262 Vaucelle, Cati et al., "Picture This! Film Assembly Using Toy Gestures", Retrieved from <<http://web.media.mit.edu/˜cati/PictureThis-Ubicomp.pdf>>, (2008),10 pages.
263 Viola, Paul et al., "Robust Real-Time Face Detection", In International Journal of Computer Vision, vol. 57, Issue 2, (May 2004),18 pages.
264 Voit, Michael et al., "Deducing the Visual Focus of Attention from Head Pose Estimation in Dynamic Multi-View Meeting Scenarios", Proceedings of the 1oth International Conference on Multimodal Interfaces, (Oct. 20, 2008), 8 pages.
265 Walker, et al., "Age Related Differences in Movement Control: Adjusting Submovement Structure to Optimize Performance", Journals of Gerontology, (Jan. 1997), pp. 40-52.
266 Wedel, Michel et al., "Eye Fixations on Advertisements and Memory for Brands: A Model and Findings", Journal of Marketing Science, vol. 19, Issue 4, (Oct. 2000), pp. 297-312.
267 Welford, Alan T., "Signal, Noise, Performance, and Age.", Human Factors, vol. 23, Issue 1, http://www.ingentaconnect.com/content/hfes/hf/1981/00000023/00000001/art0009, (1981), pp. 97-109.
268 Wilson, Andrew et al., "GWindows: Towards Robust Perception-Based UI", Microsoft Research, (2003),pp. 1-8.
269 Wilson, et al., "Hidden Markov Models for Modeling and Recognizing Gesture Under Variation", Hidden Markov Model: Applications in Computer Vision., T. Caelli, ed. World Scientific, (2001),36 pages.
270 Wood, Orlando "Using Faces: Measuring Emotional Engagement for Early Stage Creative", In ESOMAR, Best Methodology, Annual Congress, (Sep. 19, 2007), 29 pages.
271 Worden, Aileen et al., "Making Computers Easier for Older Adults to Use: Area Cursors and Sticky Icons", CHI 97, Atlanta Georgia, USA, (1997),pp. 266-271.
272 Wren, Christopher et al., "Pfinder: Real-Time Tracking of the Human Body", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, No. 7, (Jul. 1997),pp. 780-785.
273 Yakut, Isil D., et al., "User and Task Analysis of Multi-Level 3D File Browser", Dept. of Computer Engineering, Bilkent University, Ankara, Turkey, retrieved from <http://www.cs.bilkent.edu.tr/˜cansin/projects/cs560-3dui/multi-level-3d-file-browser/3dui-report.pdf> on Nov. 15, 2011,4 pages.
274 Yoda, Ikushi et al., "Utilization of Stereo Disparity and Optical Flow Information for Human Interaction", Proceedings of the Sixth International Conference on Computer Vision, IEEE Computer Society, Washington D.C., USA, (1998),5 pages.
275 Zhai, Shumin et al., "The "Silk Cursor": Investigating Transparency for 3D Target Acquisition", CHI 94, (1994),pp. 273-279.
276 Zhang, Zhengyou "A Flexible New Technique for Camera Calibration", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, No. 11, (Nov. 2000),pp. 1330-1334.
277 Zhang, Zhengyou "Flexible Camera Calibration by Viewing a Plane from Unknown Orientations", Microsoft Research, (1999),8 pages.
278 Zhang, Zhenqiu et al., "Head Pose Estimation in Seminar Room Using Multi View Face Detectors", Proceedings of the 1st International Evaluation Conference on Classification of Events, Activities and Relationships, (Mar. 30, 2006), 7 pages.
279 Zhao, Liang "Dressed Human Modeling, Detection, and Parts Localization", The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, (2001),121 Pages.
International Classification H04N21/422, H04N21/442, H04N21/4223, H04N21/466
Cooperative Classification H04N21/44213, H04N21/4223, H04N21/4667, H04N21/44218, H04N21/42203, H04N21/42201
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CONRAD, MICHAEL J.;HULTEN, GEOFFREY J;KRUM, KYLE J.;AND OTHERS;SIGNING DATES FROM 20111202 TO 20111205;REEL/FRAME:027374/0449