Patent ID: 12212887

DETAILED DESCRIPTION

In the following description of various illustrative embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown, by way of illustration, various embodiments in which aspects of the disclosure may be practiced. It is to be understood that other embodiments may be utilized, and structural and functional modifications may be made, without departing from the scope of the present disclosure.

FIG.1illustrates an example communication network100on which many of the various features described herein may be implemented. Network100may be any type of information distribution network, such as satellite, telephone, cellular, wireless, etc. One example may be an optical fiber network, a coaxial cable network, or a hybrid fiber/coax distribution network. Such networks100use a series of interconnected communication links101(e.g., coaxial cables, optical fibers, wireless, etc.) to connect multiple premises102(e.g., businesses, homes, consumer dwellings, etc.) to a local office or headend103. The local office103may transmit downstream information signals onto the links101, and each premises102may have a receiver used to receive and process those signals.

There may be one link101originating from the local office103, and it may be split a number of times to distribute the signal to various premises102in the vicinity (which may be many miles) of the local office103. The links101may include components not illustrated, such as splitters, filters, amplifiers, etc. to help convey the signal clearly. Portions of the links101may also be implemented with fiber-optic cable, while other portions may be implemented with coaxial cable, other lines, or wireless communication paths.

The local office103may include an interface, such as a termination system (TS)104. More specifically, the interface104may be a cable modem termination system (CMTS), which may be a computing device configured to manage communications between devices on the network of links101and backend devices such as servers105-107(to be discussed further below). The interface104may be as specified in a standard, such as the Data Over Cable Service Interface Specification (DOCSIS) standard, published by Cable Television Laboratories, Inc. (a.k.a. CableLabs), or it may be a similar or modified device instead. The interface104may be configured to place data on one or more downstream frequencies to be received by modems at the various premises102, and to receive upstream communications from those modems on one or more upstream frequencies.

The local office103may also include one or more network interfaces108, which can permit the local office103to communicate with various other external networks109. These networks109may include, for example, networks of Internet devices, telephone networks, cellular telephone networks, fiber optic networks, local wireless networks (e.g., WiMAX), satellite networks, and any other desired network, and the network interface108may include the corresponding circuitry needed to communicate on the external networks109, and to other devices on the network such as a cellular telephone network and its corresponding cell phones.

As noted above, the local office103may include a variety of servers105-107that may be configured to perform various functions. For example, the local office103may include a push notification server105. The push notification server105may generate push notifications to deliver data and/or commands to the various premises102in the network (or more specifically, to the devices in the premises102that are configured to detect such notifications). The local office103may also include a content server106. The content server106may be one or more computing devices that are configured to provide content to users at their premises. This content may be, for example, video on demand movies, television programs, songs, text listings, etc. The content server106may include software to validate user identities and entitlements, to locate and retrieve requested content, to encrypt the content, and to initiate delivery (e.g., streaming) of the content to the requesting user(s) and/or device(s).

The local office103may also include one or more application servers107. An application server107may be a computing device configured to offer any desired service, and may run various languages and operating systems (e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD, Ubuntu, Redhat, HTML5, JavaScript, AJAX and COMET). For example, an application server may be responsible for collecting television program listings information and generating a data download for electronic program guide listings. Another application server may be responsible for monitoring user viewing habits and collecting that information for use in selecting advertisements. Yet another application server may be responsible for formatting and inserting advertisements in a video stream being transmitted to the premises102. Although shown separately, one of ordinary skill in the art will appreciate that the push server105, content server106, and application server107may be combined. Further, here the push server105, content server106, and application server107are shown generally, and it will be understood that they may each contain memory storing computer executable instructions to cause a processor to perform steps described herein and/or memory for storing data.

An example premises102a, such as a home, may include an interface120. The interface120can include any communication circuitry needed to allow a device to communicate on one or more links101with other devices in the network. For example, the interface120may include a modem110, which may include transmitters and receivers used to communicate on the links101and with the local office103. The modem110may be, for example, a coaxial cable modem (for coaxial cable lines101), a fiber interface node (for fiber optic lines101), twisted-pair telephone modem, cellular telephone transceiver, satellite transceiver, local wi-fi router or access point, or any other desired modem device. Also, although only one modem is shown inFIG.1, a plurality of modems operating in parallel may be implemented within the interface120. Further, the interface120may include a gateway111. The modem110may be connected to, or be a part of, gateway111. Gateway111may be a computing device that communicates with the modem(s)110to allow one or more other devices in the premises102a, to communicate with the local office103and other devices beyond the local office103. The gateway111may be a set-top box (STB), digital video recorder (DVR), computer server, or any other desired computing device. The gateway111may also include (not shown) local network interfaces to provide communication signals to requesting entities/devices in the premises102a, such as display devices112(e.g., televisions), additional STBs113, personal computers114, laptop computers115, wireless devices116(e.g., wireless routers, wireless laptops, notebooks, tablets and netbooks, cordless phones (e.g., Digital Enhanced Cordless Telephone—DECT phones), mobile phones, mobile televisions, personal digital assistants (PDA), etc.), landline phones117(e.g., Voice over Internet Protocol—VoIP phones), and any other desired devices. Examples of the local network interfaces include Multimedia Over Coax Alliance (MoCA) interfaces, Ethernet interfaces, universal serial bus (USB) interfaces, wireless interfaces (e.g., IEEE 802.11, IEEE 802.15), analog twisted pair interfaces, Bluetooth interfaces, and others.

The various devices in the system may be configured to determine and/or adjust an image resource allocation for displaying an image based on dynamic tracking of a user's movements. For example, gateway111may receive and process data from an image sensing or capturing device, such as a camera, to determine an image resource allocation for displaying content (e.g., video images) within a display area (e.g., display device112) in accordance with a user's distance from the display area.

FIG.2illustrates general hardware elements that can be used to implement any of the various computing devices discussed herein. The computing device200may include one or more processors201, which may execute instructions of a computer program to perform any of the features described herein. The instructions may be stored in any type of computer-readable medium or memory, to configure the operation of the processor201. For example, instructions may be stored in a read-only memory (ROM)202, random access memory (RAM)203, removable media204, such as a Universal Serial Bus (USB) drive, compact disk (CD) or digital versatile disk (DVD), floppy disk drive, or any other desired storage medium. Instructions may also be stored in an attached (or internal) hard drive205. The computing device200may include one or more output devices, such as a display206(e.g., an external television), and may include one or more output device controllers207, such as a video processor. There may also be one or more user input devices208, such as a remote control, keyboard, mouse, touch screen, microphone, etc. The computing device200may also include one or more network interfaces, such as a network input/output (I/O) circuit209(e.g., a network card) to communicate with an external network210. The network input/output circuit209may be a wired interface, wireless interface, or a combination of the two. In some embodiments, the network input/output circuit209may include a modem (e.g., a cable modem), and the external network210may include the communication links101discussed above, the external network109, an in-home network, a provider's wireless, coaxial, fiber, or hybrid fiber/coaxial distribution system (e.g., a DOCSIS network), or any other desired network. Additionally, the device may include a location-detecting device, such as a global positioning system (GPS) microprocessor211, which can be configured to receive and process global positioning signals and determine, with possible assistance from an external server and antenna, a geographic position of the device.FIG.2illustrates an image resource allocator212component, which may be a dedicated processor configured to perform the various image resource allocation functions described herein, or it may be implemented by the device's main processor201.

TheFIG.2example is a hardware configuration, although the illustrated components may be implemented as software as well. Modifications may be made to add, remove, combine, divide, etc. components of the computing device200as desired. Additionally, the components illustrated may be implemented using basic computing devices and components, and the same components (e.g., processor201, ROM storage202, display206, etc.) may be used to implement any of the other computing devices and components described herein. For example, the various components herein may be implemented using computing devices having components such as a processor executing computer-executable instructions stored on a computer-readable medium, as illustrated inFIG.2. Some or all of the entities described herein may be software based, and may co-exist in a common physical platform (e.g., a requesting entity can be a separate software process and program from a dependent entity, both of which may be executed as software on a common computing device).

One or more aspects of the disclosure may be embodied in a computer-usable data and/or computer-executable instructions, such as in one or more program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by a processor in a computer or other data processing device. The computer executable instructions may be stored on one or more computer readable media such as a hard disk, optical disk, removable storage media, solid state memory, RAM, etc. As will be appreciated by one of skill in the art, the functionality of the program modules may be combined or distributed as desired in various embodiments. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents such as integrated circuits, field programmable gate arrays (FPGA), and the like. Particular data structures may be used to more effectively implement one or more aspects of the disclosure, and such data structures are contemplated within the scope of computer executable instructions and computer-usable data described herein.

FIG.3illustrates an example environment in which one embodiment of the disclosure may be employed. This example environment comprises a room300which may be, e.g., a room in a user's home. In other embodiments, the room300may be any space where the user may access or consume content. For example, the room300may be, e.g., the user's place of employment, a train, an airplane, a school, a church, etc. The room300includes a first user306and a second user307accessing content via a display device302. For example, the first user306may be watching a television program on the display device302. The display device302may be any device in the art suitable for displaying content as discussed above in reference toFIG.1including, but not limited to, a television, a video display, a computer monitor, a desktop computer, a laptop computer, a tablet computer, a smartphone, and the like. The term “display area” may be used herein to refer to an area of the display device or combination of display devices, and/or any other suitable surface where an image may be outputted, transmitted, or displayed. Additionally the term “display device” may be used herein to refer to a computing device (e.g., television) and/or the display screen associated with the display device (e.g., television screen). Such usage is merely for ease in discussing one or more of the embodiments disclosed herein, and should not be interpreted as limiting the exact embodiment in which the method described further below may be performed. As will be appreciated, the display area is not limited to the display screen of the display device. For example, a projector or other computing device may display a video image on a separate display screen or a wall within the room300. In another example, a projector or other computing device may output a first portion of a video image on a display device or combination of display devices, and a second portion of the image on a surface (e.g., display screen or wall) surrounding and/or adjacent to the display device or combination of display devices. AlthoughFIG.3depicts the display area as a single display device (i.e., the display device302), the display area may span multiple display devices. For example, an image or video content may be displayed across a plurality display devices. The exact configuration of the display area can vary significantly without departing from the present disclosure.

InFIG.3, the display device302is located across from the first user306sitting on a sofa314, and the second user307sitting on a sofa315. The room300also includes further exemplary items that may be found in and around the first user306and the second user307accessing content, including a table316, and an image such as a picture frame308. Again, each item is merely representative of objects which may be located around one or more users accessing content and is not intended to limit or define a typical environment for use of the present disclosure. The range and number of objects which may be around the first user306or the second user307in other settings is virtually limitless.

An image sensing or capturing device, such as a camera304may be associated with the display device302. InFIG.3, the camera304is depicted as sitting to the side of the display device302, but the camera304may be located at any suitable location throughout the room300such that one or more users are in the field of view of the camera304. For example, the camera304may be mounted on top of the display device302. Further, in some embodiments, the camera304may be integral to the display device302. For example, televisions, personal computers, laptop computers, tablet computers, smartphones, and others may contain cameras integrally formed within the display device. The exact location, configuration, and association of the camera304with the display device302can vary significantly without departing from the present disclosure. The camera304may be configured to capture a three-dimensional (“3D”) image or 3D information of the users in the room300. For example, the camera may be used to capture the depth of objects in the room to determine 3D structures or objects in space such as bodies or body parts. In other embodiments, the camera304may be configured to detect different wavelengths of light or electromagnetic radiation, for example, an infrared (IR) signal transmitted by a remote control device.

Broken lines318represent an exemplary field of view of the camera304. In the illustrated example, the camera304is positioned such that the table316, the sofas314and315, the first user306, the second user307, and the frame308are all in its field of view. In various other embodiments, the camera304may be positioned in such a way that less than all of the items in the room300are in its field of view. For example, in one embodiment the camera304may be focused on the sofa314, and thus the first user306and/or items situated on or around the sofa314may be in the camera's304field of view. In another embodiment the camera304may be focused on the sofa315, and thus the second user307and/or items situated on or around the sofa315may be in the camera's304field of view. In yet another embodiment the camera304may be focused on the first user306, and thus the first user306and/or items situated near the first the first user306may be in the camera's304field of view. In some embodiments, the camera304may be focused on multiple users within the room300. For example, the camera304may be focused on the first user306and the second user307, and thus the first user306, the second user307, and/or items situated near the first user306and the second user307may be in the camera's field of view. The image capture device (e.g., camera)304may be dynamically controllable by one or more computing devices. For example, one or more computing devices in this system may be configured to adjust the field of view of the camera304. In other examples, the camera304may be programmed and/or configured to continuously scan the room300. In some embodiments, the first user306or the second user307may physically change the location of the camera304within the room300. In other embodiments, the first user306or the second user307may adjust the field of view of the camera304. The location and configuration of the camera304may be further varied without departing from the scope of this disclosure.

Broken lines320represent an exemplary field of view of the first user306, illustrating the direction in which the first user306is looking. While accessing content, the first user306may alter his or her field of view depending on, e.g., an object of interest appearing on the display device302. As depicted by broken lines320, in this example the first user306is directing his/her eye focus towards the left side of the display device302. As will be appreciated with discussion of the following figures, the first user's306field of view is also focused towards the lower part of the display device. That is, in this example, the first user306is directing his/her eye focus towards the lower, left side of the display device302. Furthermore, broken lines322represent an exemplary field of view of the second user307. While viewing content on the display device302, the second user307may also alter his or her field of view. As depicted by broken lines322, in this example the second user307is also directing his/her eye focus towards the left side of the display device302. In this example, although the first user306and the second user307are directing their respective eye focus to the same area of display device302, as will be appreciated with discussion of the following figures, the second user's307field of view may differ from the first user's306field of view. For example, the first user306may direct his/her eye focus the lower right side of the display device302, while the second user307may direct his or her eye focus towards the upper left side of the display device302.

FIG.4illustrates an exemplary captured image400by the camera304according to one embodiment of the disclosure. Specifically, the captured image400is representative of an image that the camera304may capture having a field of view illustrated by broken lines318. In this embodiment, the captured image400is a perspective view of the room300and each of its components that appear within broken lines318. Specifically, the captured image400includes a perspective view of the first user306and the second user307. In some embodiments, captured image400may include one or more objects. As used herein, “object” refers broadly to any identifiable item, article, piece of furniture, and the like residing within the room. For example, in this embodiment, captured image contains a table object416(i.e., the table residing in the room300); a sofa object414(i.e., the sofa where the first user306is sitting); a sofa object415(i.e., the sofa where user307is sitting); and a frame object408(i.e., the frame hanging from the wall in the room300). An image capture device (e.g., the camera304) or other data capture device may also recognize the first user's306eye focus, and more specifically, what is being viewed within the display area (e.g., the display device302) by the eyes412of the first user306as will be discussed more fully below. Moreover, the camera304or other data capture device may also recognize user's307eye focus, and more specifically, what is being viewed on the display device302by the eyes413of the second user307.

As discussed above, the first user's306eye focus is generally fixed down and towards the left side of the display device302. That is, as seen in the perspective view of the first user306, the first user's eyes412appear to be gazing down and towards the right side of captured image400(corresponding to the left side of the display device302). As will become evident with respect to the discussion of other figures below, the first user's eyes412are thus an indication that the first user's306eye focus is directed towards the lower, left side of the display device302. Likewise, as seen in the perspective view of the second user307, the second user's eyes413appear to be gazing down and towards the right side of the captured image400(corresponding to the left side of the display device302). Thus, an indication that the second user's307eye focus is directed towards the lower, left side of the display device302.

Moving toFIG.5, similar toFIG.3, this example environment comprises the room300, that includes the first user306, the second user307, the sofas314and315, the table316and the frame308. The field of view for the first user306inFIG.5differs from that inFIG.3. As depicted by broken lines520, in this example, the first user306is directing his/her eye focus towards the right side of the display area (e.g., the display device302). As will be appreciated with discussion of the following figures, user's306field of view is also focused towards the center part of the display area. That is, in this example, the first user306is directing his/her eye focus towards the center, right side of the display device302. Furthermore, broken lines522represent an exemplary field of view of the second user307. Like in the previous example (i.e.,FIG.3), the second user307is still focusing his or her eye focus towards the lower, left side of the display device302.

FIG.6illustrates an exemplary captured image600by the camera304according to one embodiment of the disclosure. Specifically, the captured image600is representative of an image that the camera304may capture having a field of view illustrated by broken lines318. In this embodiment, the captured image600is a perspective view of the room300and each of its components which appear within broken lines318. Specifically, the captured image600includes a perspective view of the first user306and the second user307. In some embodiments, the captured image600may include one or more objects. Again, each object presented inFIG.6is for illustrative purposes only. In other embodiments, users accessing content on a display device may have any number of objects on or around them that may appear in a captured image, such as the captured image600. Objects may be recognized by comparing the detected image with predefined images of known objects, articles, pieces of furniture, etc. using any desired image-matching technique. Additionally, objects may be recognized as fitting the basic shape of a known type of object. In this example, broken lines616depict the recognition by image sensing device (e.g., the camera304) of the table316in the captured image600. In other embodiments, the camera304may detect different wavelengths of light or electromagnetic radiation. In this example, broken lines620depict the recognition by image sensing device (e.g., the camera304) of a remote control signal618that is illustrated as a flash of light inFIG.6.

Broken lines608,612,613,615, and616are representations of one embodiment of the present disclosure recognizing objects and/or users in the captured image600. Specifically, broken lines608represent recognition of the frame308, broken lines615represent recognition of the sofa315, and broken lines616represent recognition of the table316. Moreover, in this example, broken lines612represent recognition of the first user's eyes412, and broken lines613represent recognition of the second user's307face. Any desired image-matching and recognition technique may be used to recognize the objects in the room or the users' eyes. In one embodiment, such techniques may be used to, e.g., recognize an object or other item in the captured image600(e.g., recognize a table). In other embodiments, these techniques may be used to, e.g., recognize features of the users (e.g., facial recognition and/or recognize the first user's eyes412or the second user's eyes413).

Merely by way of example, recognition may be performed by a processor operatively coupled to memory containing application software well known in the art as suitable for object recognition. A processor and/or memory for performing such a task may be integral to the display device302and/or the camera304or may be remote from each component (at, e.g., a centralized server). This application software may perform various recognition tasks to, e.g., the captured image400and/or the captured image600in order to recognize objects therein. In one embodiment, the application software may comprise a database of exemplary objects and a processor may thus compare, e.g., the captured image600to the database of exemplary objects. If the captured image600, or any aspect of the captured image600(e.g., a specific item, article or piece of furniture, etc.) is similar to an exemplary image, the processor running the application software may recognize the captured image600and/or specific aspects of the captured image600as being the same as the exemplary object.

In other embodiments, the application software may comprise a database of images previously captured by the camera304and the processor may thus compare, e.g., the captured image600to a recently captured image in the database of images. If the captured image600, or any aspect of the captured image600(e.g., the location of the first user306, the location of the second user's eyes413, etc.) differs from a recently captured image, the processor running the application software may recognize that specific aspects of the composition of the room have changed. For example, in the captured image600, the first user's306eye focus is generally centered and fixed towards the right side of the display area (e.g., the display device302). That is, as seen in the perspective view of the first user306, the first user's eyes412appear to be gazing towards the left side of the captured image600(corresponding to the right side of the display device302) Likewise, as seen in the perspective view of the second user307, the second user's eyes413appear to be gazing down and towards the right side of captured image400(corresponding to the left side of the display device302). Thus an indication that the second user's307eye focus is directed towards the lower, left side of the display device302.

By comparing the captured image400to the captured image600, the processor may determine that the first user's eyes412have shifted from gazing down and to the right in the captured image400, to gazing slightly left and toward the center in the captured image600. Accordingly, the processor may be configured to determine that user's306eye focus has changed during the time period that the captured image400and the captured image600were taken by the camera304. In some embodiments, a computing device, such as the processor, or any other computing devices operatively coupled to a processor804(e.g., the camera304) may be configured to determine whether one of the user's eyes is focused on the display area (e.g., display device) when processing data relating to a user's eyes focus. This may occur for several reasons, such as the user only has one eye, the user has a medical condition (e.g., Strabismus), etc.

As another example, by way of comparing multiple captured images, the application software may recognize that a user has entered and/or is exited a door within the room. This method of object recognition is merely illustrative of one method of recognizing objects in a captured image, and is not intended to define or limit the object recognition process of the present disclosure. Of course, those skilled in the art will appreciate various other methods and hardware/software combinations that are suitable for recognizing objects within a captured image. Any number of these suitable methods for object recognition may be employed herein without departing from the present disclosure.

As is well known in the art, the native resolution of a display device represents the number of distinct pixels in each dimension that can be displayed on the display device. For example, a “1024×768” pixel screen of a display device is capable of displaying 1024 distinct pixels horizontally across the width of the display device and 768 pixels vertically along the height of the display device, or approximately 786,000 pixels total. Consequently, in this example, an image displayed on this display device will comprise a mosaic of 786,000 pixels. The human visual system has a fixed capacity to detect or perceive image quality. Several factors can affect a user's capacity to recognize detail or visually perceive an image displayed on a display device, including, but not limited to, the size of the display device, viewing distance, the transmission rate of the image(s), and resolution of the image(s) or content being displayed.

FIGS.7a-7ddepict illustrated examples of how these various factors may affect a user's perception of image quality based on the images outputted for display within a display area (e.g., display device). With regards to the size of a display area, keeping viewing distance and native resolution constant, as the size of the display area increases, the overall quality of the image being displayed to the user may decrease as the image is resized and appears increasingly pixilated, as the user can more clearly perceive the individual pixels of the image displayed within the display area. Conversely, keeping viewing distance and native resolution constant, as the size of the display area decreases, the overall quality of the image being displayed to the user may increase as the image appears less pixilated, and the likelihood of the user visually perceiving the individual pixels of the image displayed within the display area decreases.

By way of example, this principle is illustrated inFIGS.7a-7b. InFIG.7a, a display device702is displaying a content710(in this example, an image of two trees) to a user701. InFIG.7b, the content710is being displayed to the user701via a display device705. In this example, the size of the display device702is smaller than the size of the display device705illustrated inFIG.7b; however, the native resolution for display devices702and705are the same. When the content710is displayed on the display device702, the visual appearance of the content710as displayed on the display device705is larger, more pixilated, and likely to be visually perceived by the user701as having a lower image quality due to the resizing that must occur.

Similarly, with regards to the factor of viewing distance, when viewing content within a display area from an appropriate distance, the pixels may blend together to create an image that is visually perceived by the user as complete and smooth. However, keeping the size and native resolution of the display area (e.g., display device) constant, as the distance between the user and the display area decreases, a point may be reached where pixilation occurs and the appearance of individual pixels can be perceived by the user. When this occurs, the perceived quality of the displayed image decreases and any advantages of moving closer to the display area produce less desirable effects for the user consuming content displayed within the display area. Conversely, as the distance between the user and the display area increases, a point may be reached where limitations in the human visual system may prevent the user from recognizing detailed aspects of the displayed image. When this occurs, the perceived quality of the displayed image may decrease and any advantages of moving further away from the display area produce less desirable effects for the user consuming content displayed within the display area.

By way of example, this principle is illustrated inFIGS.7c-7d. InFIG.7c, the display device702is displaying a content712(in this example, an image of a building) to the user701. Specifically,FIG.7cdepicts the user701viewing the display device702at a close viewing distance. InFIG.7d, the user701is viewing the content712on the display device702from a farther viewing distance. As discussed above,FIG.7cillustrates the principle that as the user701views the display device702at a close viewing distance the appearance of individual pixels are more perceptible by the user701, as illustrated by an image714. Thus the advantages of moving closer to the display device702produce less desirable effects as visually perceived by the user701. By contrast, inFIG.7d, the user701is viewing the display device702from a farther viewing distance (a viewing distance greater than the example illustrated inFIG.7c), thus lowering the probability that the content712displayed on the display device702appears pixilated or distorted to the user701(as illustrated by an image716) and increasing the perceived image quality of the content712to the user701.

FIGS.8a-8cillustrate embodiments of determining and adjusting an image resource allocation for displaying content within a display area. Specifically,FIG.8adepicts example content that may be displayed to the first user306within a display area (e.g., the display device302) associated with the camera304. By way of example, inFIG.8a, content consisting of a video presentation of a sporting event is displayed on the display device302. In this embodiment the display device302may be, e.g., a television set, desktop computer, laptop computer, tablet computer, or smartphone or any other desired display area (e.g., the display device702), displaying media content via fiber, coaxial cable, wireless, or other transmission media. Further, the camera304may be integral to the display device302, or may be located externally to the display device302and operatively coupled to the display device by any method well known in the art.

FIGS.8a-8billustrate a first embodiment of adjusting image resource allocations within a display area (e.g., the display device302) according to one aspect of the disclosure. InFIG.8a, the display device302is displaying content (in this example, a football game). Specifically, the viewing distance between the first user306and the display device302is processed by the processor804, as illustrated by element808in a processing cloud810. The processing clouds used throughout the disclosure are for illustrative purposes only and are not intended to limit or define the manner in which information is processed.

The processor804may be a processor integral to the camera304and/or the display device302. Alternatively, the processor804may be a processor located in another of a user's devices which are operatively coupled to or otherwise communicatively linked to the camera304and the display device302. For example, the processor804may be located in a desktop computer, laptop computer, tablet computer, smartphone, video projector and the like operatively coupled to the camera304. Alternatively, the processor804may be located remotely from the display device302and the camera304. For example, the processor804may be located at a centralized server. Referring back toFIG.1, in one embodiment the processor804may be located at a server in the local office103. In this embodiment, the display device302and the camera304may be communicatively linked to the local office103through, e.g., links101. In another embodiment, the processor804may be located at the push notification server105, the content server106, and/or the application server107. In other embodiments, the processor804may be located at the gateway111. The processor804may take a variety of forms in a variety of locations without departing from the disclosure. For example, referring toFIG.2, in one embodiment, the processor804may be located at the image resource allocator212.

Returning toFIG.8a, in one embodiment, the camera304may be configured to capture an image of the room where the first user306is viewing content within a display area (e.g., the display device302), for example, the captured image400and the captured image600illustrated inFIGS.4and6. As discussed above, the processor804may be configured to process data relating to an image captured by the camera304to determine a user's viewing distance and detect structures or objects such as bodies, faces, body parts, furniture, etc. The processor804may also be configured to recognize the depth of objects in a room (e.g., the user's distance from the display area). A computing device, such as the processor804, may also be configured to recognize the distance between a user and/or object and the display area. For example, the processor804may be configured to determine the distance between a user and a display area (e.g., display device) by processing information relating to the viewing distance of one or more objects in the room300, such as the sofa314, where the user was previously located. There are a variety of ways in which the processor804may determine a user's location within a room or the user's viewing distance from the display area. In one embodiment, the processor804may be configured to receive information relating to the viewing distance of a user from the camera304, or one or more computing devices operatively coupled to the camera304. In one embodiment, a computing device, such as the processor804, may be configured to determine a user's viewing distance from the display device302based on the distance of one or more objects to the display device302. In another embodiment, the processor804may be configured to determine a user's viewing distance by identifying the distance of an object within a predetermined distance from the first user. For example, the processor804may be configured to determine a user's viewing distance by identifying the distance of the object closest to the user and the display device302.

In another embodiment, a computing device, such as the processor804, may be configured to determine a user's viewing distance by processing data relating to the inter-pupillary distance of a user viewing the display device. In some embodiments a user may calibrate the system to ensure a more accurate calculation based on inter-pupillary distance. In one embodiment, the processor804may be configured to store in memory data relating to one or more users' viewing distance in a user profile.

As will be appreciated by one of skill in the art, the functionality of the system may be combined with various technologies known in the art (e.g., Bluetooth, Wi-Fi, RF devices) so as to determine a user's distance from the display area (e.g., display device) in various embodiments. For example, the processor804may be configured to determine a user's viewing distance by processing data relating to a remote control signal transmitted from a remote control device. In other embodiments, the camera304may be configured to detect a remote control signal618from a remote control device. In one of these embodiments, the camera may be configured to transmit information relating to the detection of the remote control signal618to one or more computing devices, such as the processor804. In another of these embodiments, one or more computing devices, such as the gateway111, may be configured to detect the remote control signal618and subsequently transmit data to the processor804relating to the detection of the remote control signal618.

In some embodiments, a user may have the option of calibrating a determined viewing distance based on the detection of a remote control signal. For example, the user may be prompted to transmit a remote control signal from various distances within the room300in order to assist the calibration of the correct viewing distance of the user. In another embodiment, the processor804may be configured to receive data from one or more computer devices relating to a user's viewing distance from the display device. For example, one or more Bluetooth enabled computing devices, such as a cell-phone or laptop, may be configured to communicate with the display device and determine the user's viewing distance from the display device. In this example, the one or more computer devices, (e.g., cell-phone or laptop), may be configured to transmit data relating to the user's viewing distance to the processor804, or any other computing devices operatively coupled to the processor804.

As discussed above, the camera304may be configured to detect the depth of objects in the room, including the viewing distance of one or more users or objects in a room. In one embodiment, the camera304may be configured to locate a user within the room300by processing data relating to previous user locations within the room300. For example, the camera304may be configured to locate a user within the room300by processing data stored in one or more user profiles. In some embodiments, the camera304may be configured to transmit to one or more computing devices data relating to a user's viewing distance from the display area (e.g., the display device). For example, the camera304may be configured to transmit to the processor804data relating to the user's viewing distance. The processor804may also be configured to store in memory data relating to the viewing distance of one or more users viewing the display device. In other embodiments, the camera304may be configured to transmit to one or more computing devices data relating to the distance of a recognized object in the room300. For example, the camera304may be configured to transmit to the processor804data relating to the distance of the table316to the display device302. As another example, the processor804may be configured to store in memory data relating to the distance of the detected the table316to the display device302.

In some embodiments, the display device302may be configured to calibrate one or more previously stored viewing distances associated with a detected object. For example, a user may be prompted to transmit a remote control signal from one or more locations in the room300, such as the sofa315. In this example, the display device302may be configured to process data relating to the distance of the remote control device to adjust a previously saved viewing distance associated with detected the sofa315(illustrated as broken lines615). In another example, the display device302may be configured to transmit data relating to the distance of the remote control device to the processor804. In this example, the processor804may be configured to adjust a previously saved viewing distance associated with detected the sofa315based on information relating to the distance of the remote control device when a user is located at the sofa315.

To determine which users are in the viewing area of an image capturing device, such as the camera304, facial recognition of users in a captured image may be performed using a computing device. In some embodiments, a computing device may be configured to process data relating to one or more images of a user's face to identify whether the user is in the room during a content session. For example, the processor804may be configured to process data relating to the second user's307face (illustrated inFIG.6as broken lines613), to determine that the second user307is located in the room300. In one embodiment, a computing device may be configured to process data relating to one or more images of a user's face to identify whether a user is a child or an adult. For example, the processor804may be configured to process data relating to the second user's307face (illustrated inFIG.6as broken lines613), to determine whether the second user307is a child or an adult. In some embodiments, the processor804may prompt a user to provide demographic information (e.g., sex, age, etc.) relating to the one or more users whose face has been recognized by the processor804or a computing device operatively coupled to the processor804. Such demographic information may be stored in a user profile or account associated with the user whose face has been recognized. In other embodiments, to determine which users are viewing a display device, a computing device may be configured to process data relating to a user's eyes focus. In one of these embodiments, a computing device may be configured to process data relating to one or more captured images of a user's eyes to determine whether the user is viewing the display device. For example, the processor804may be configured to detect and/or monitor the first user's eyes412(illustrated inFIG.6as broken lines612), to determine whether the first user306is viewing the display device302.

To determine which users are viewing a display device, recognition of a transmitted remote control signal in one or more captured images may be performed using a computing device. In one embodiment, one or more computing devices may be configured to determine which user in a room initiated a remote control signal, transmitted from the remote control device, to determine whether that user is viewing the display device. In another embodiment, one or more computing devices may be configured to adjust one or more settings associated with the display device in accordance with the preferences of the user that initiated transmission of the remote control signal. For example, the processor804may be configured to determine that the first user306initiated remote control signal618(as illustrated inFIG.6), and further configured to adjust one or more settings of the display device302in accordance with user preferences and/or a user profile associated with the first user306.

There are a variety of ways to determine which user initiated a remote control signal. In one embodiment, the processor804may be configured to process data relating to one or more images captured by the camera304to determine proximity or the distance between the location of remote control signal618and a user. In another embodiment, the processor804may be further configured to associate initiation of the remote control signal618with the closest user to the signal upon transmission. For example, the processor804may be configured to associate the initiation of the remote control signal618with the first user306in view of the first user's306proximity to the remote control signal618. In another embodiment, the processor804may be configured to process data relating to one or more images captured by the camera304to determine which user is holding or in possession of the remote control device. In other embodiments, the processor804may be configured to calculate the likelihood that a particular user is in control of the remote control device based on varying distances from the user in various directions.

In some embodiments, where a computing device is unable to recognize a user in the room, a computing device may be configured to store data associated with the new user. For example, the processor804may be configured to process data relating to one or more images captured by the camera304to determine a new user in the room300. The processor804may be further configured to prompt the new user to select or more user and/or viewing preferences. The processor804may be further configured to store in memory data relating to the user preferences a detected user. In other embodiments, the processor804may be configured to adjust or customize one or more settings or user preferences in accordance with a detected user. In one of these embodiments, the processor804may be configured to adjust one or more settings of a computing device in accordance with the preferences of a specified user. For example, after determining that the first user306is in the room300, the processor804may be configured to adjust one or more settings of the display device302in accordance with the stored user preferences and/or user profile of the first user306. In another example, after determining that the second user307is in the room300, the processor804may be configured to adjust one or more settings of the display device302in accordance with the stored user preferences and/or user profile of the second user307. In yet another example, the processor804may be configured to determine which user profile is active during the present content session when adjusting the one or more settings of the display device302.

In some embodiments, the camera304may be configured to detect the movements of one or more users in the room300. In one embodiment, the camera304may be configured to constantly scan the room300to detect user movement. In other embodiments, the camera304may be configured to transmit information relating to a user's movement to one or more computing devices. For example, the camera304may be configured to transmit information relating to a user's movement to the processor804. In some of these embodiments, the camera304may be configured to detect whether a user's viewing distance from the display device302has changed over time. In some embodiments, where the camera304has detected user movement, the camera304may be configured to communicate data to the processor804relating to the movement of one or more users. In one of these embodiments, the camera304may be configured to transmit to the processor804information relating to an updated viewing distance for one or more user.

In other embodiments, where the camera304has detected user movement, the processor804may be configured to process data relating to the movement of one or more users captured by the camera304. In one of these embodiments, the processor804may be configured to determine an updated viewing distance based on the change in location of one or more users within the room. In another of these embodiments, the processor804may be configured to save in memory data relating to the updated location of one or more users. For example, the processor804may be configured to store in memory an updated viewing distance for one or more users viewing the display device302. In some embodiments, the camera304may be configured to continuously monitor user movement and location within the room300. In one of these embodiments, the processor804may be configured to continuously determine an updated viewing distance for one or more users in the room300.

In one embodiment, the processor804may be configured to dynamically determine and/or adjust an image resource allocation for displaying content on the display device302based on user movement. In some embodiments, where an image capturing device has detected user movement, a computing device may be configured to determine and/or adjust an allocation of image resources for displaying content on a display device after the user movement has ceased. For example, referring toFIG.6, if the first user306moves from the sofa314to the frame308, the processor804may be configured to determine and/or adjust an allocation of image resources for displaying content on the display device302after user movement has ceased (e.g., when the first user306arrives at the frame308). In another embodiment, the processor804may be configured to determine whether where a user has moved a predetermined distance within the viewing area. For example, the processor804may be configured to determine whether a user has moved a distance of 1 foot within the viewing area. In some of these embodiments, the processor804may be configured to adjust an allocation of image resources for displaying content if a user moves a threshold distance within the viewing area.

With respect to the eye focus of a user viewing a display device, in some embodiments, an image capturing device may be configured to detect a change in eye focus. For example, the camera304may be configured to determine that a user's field of view changes from a lower display region within the display area (e.g., the display device302) to an upper region within the display area. In another example, the camera304may be configured to determine that a user's field of view changes from a center region within the display area to a left region within the display area. In some embodiments, the camera304may be configured to continuously determine the eye focus of a user by detecting and monitoring the user's eye movement. Thus, a computing device, such as the processor804, may be configured to dynamically determine and/or adjust a resource allocation for displaying content on the display device302based on the user's eye movement. In other embodiments, a computing device, such as the processor804may be configured to determine the movement of one of the user's eyes when detecting a change in eye focus. In one of these embodiments, a user may have the option of determining which eye the system may monitor when detecting a change in eye focus. For example, the user may modify their user profile to establish that only their left eye should be monitored by the image capturing device (e.g., the camera304). In this example, a computing device, such as the processor804may not monitor the user's right eye when processing data relating to a user's eye focus or movement. Similarly, in this example, the user's right eye may not be monitored to determine whether the user is viewing the display area or focusing on a particular region of the display area.

In some embodiments, the camera304may be configured to process information relating to a user's eye movement to determine what display region within the display area the user is viewing. For example, the camera304may be configured to process information relating to the position of a viewer's eyes and/or iris. In other embodiments, the camera304may be configured transmit information relating to a user's eye movement to one or more computing devices. For example, the camera304may be configured transmit information relating to a user's eye movement to the processor804or other computing devices operatively coupled to the processor804. In some of these embodiments, the camera304may be configured to store in memory information relating to a user's eye movement. In other embodiments, the processor804may be configured to receive information relating to the size and position of a display region within the display area (e.g., the display device302) where the user is focusing. In another embodiment the camera304may be configured to store in memory information relating to the size and location of a display region within the display area (e.g., the display device302) where the user is focusing. In another embodiment, a computing device such as the processor804may be configured to determine the location of a user's eye focus based on information readily available to the processor, such as user activity displayed within the display area (e.g., the display device). For example, the processor804may determine the position of a user's eye focus by detecting the position of an input device, such as a mouse cursor or other pointer on the display device.

Content transmitted to the display device302may include one or more triggers embedded within the content to assist in identifying one or more display regions within the display area to visually emphasize to the user. These triggers may contain or have access to information identifying the subject matter of the transmitted content (e.g., video image). A plurality of triggers may be placed throughout the content to emphasize one or more images in the plurality of images comprising the transmitted content. For example, triggers may be embedded in the transmitted content to identify one or more regions within the display area to emphasize to the user while the content is being presented. In another example, different triggers may be placed throughout a television program. In this example, a director or content provider may embed triggers in the television program (e.g., displayed content), to identify certain images in the television program to be visually emphasized to the user. Various formats for the triggers may be used without departing from the scope of the present disclosure. In one embodiment, the trigger format may be the Enhanced TV Binary Interchange Format (EBIF).

In some embodiments, the user may have the option to calibrate an image capturing device's detection of eye movement and the one or more display regions within a display area that the user directs his or her eye focus. For example, the user may be prompted to direct their eye focus towards a series of images located on various regions within the display area. The user may also be prompted to direct their eye focus towards the display area from one or more locations within the room. In this example, the camera304may be configured to monitor a user's eye movement in response to a series of displayed images, and transmit to a computing device, such as the processor804, information relating to the change in position of the user's pupil and/or iris (e.g., user eye movement) in response to each displayed image. In one of these embodiments, the processor804may be configured to receive data from an image capturing device, such as the camera304, relating to a user's eye movement in response to one or more displayed images. In another of these embodiments, the processor804may be configured to process data relating to the user's eye movement to determine what region of the display area the user is viewing. One of skill in the art will appreciate that various other embodiments may be utilized to calibrate an image capturing device to detect user eye movement and what region of the display area a user is viewing, without departing from the scope of the present disclosure.

Returning toFIG.8a, in one embodiment, one or more computing devices may be configured to adjust an image resource allocation for displaying content within a display area (e.g., display device(s)). For example, the processor804may be configured to adjust an image resource allocation for displaying content on the display device302. The allocation of image resource for displaying content may encompass a variety of image resources or characteristics, including, but not limited to, image resolution, processing power per pixel, degree of image compression, image transmission rate, color depth, polygon count, texture resolution, etc. One of skill in the art will appreciate that various other image resources and combinations thereof may comprise an image resource allocation for displaying content within a display area.

As illustrated inFIG.8a, the first user306is positioned in front of a display area (e.g., the display device302) having four display regions, such as a first display region806, a second display region,807, a third display region,808, and a fourth display region809. AlthoughFIG.8aillustrates a display having four display regions, one of ordinary skill in the art will recognize that the display area (e.g., the display device302) may include any number of display regions. In some embodiments, a computing device such as the processor804may be configured to rank one or more display regions within a display area based on a viewer's eye focus. For example, referring toFIG.8a, the processor804may be configured to rank the display regions for the display device302based on proximity to one or more display regions where user's306eyes are focused. In this example, the eye focus of the first user306is directed toward the display region807, and thus, the processor804may rank the display region for the display device302, from most to least important, as follows: the second display region807is ranked first, the first display region806and the third display region808are ranked second, and the fourth display region is ranked last. As will be described further below, display region rank may be used to determine and/or adjust an image resource allocation for displaying content on the display device.

In one of these embodiments, a user's eye focus may span multiple regions. For example, the eye focus of the first user306may span the display regions806and807. In this example, the processor804may rank the display regions806and807with the same rank. Alternatively, the processor804may select one of the display regions806and807as having higher priority over the other. For instance, the processor804may be configured to prioritize the display region based on the location of the eye focus for one or more other users in a room. In another of these embodiments, display regions adjacent to the display regions where a user's eyes are focused may receive the same rank. For example, referring toFIG.8a, the display regions806and808may receive the same rank.

In one embodiment, the processor804may be configured to determine that the eye focus of multiple viewers is in one or more display regions. For example, referring to the display area (e.g., the display device302) inFIG.8a, the first user306may focus his or her eye focus within the display region807, while a second user in the room may be focusing his or her eye focus within the display region809. In this example, the processor804may be configured to determine that the eye focus of the first user306and the second user is within the display regions807and809, respectively. The processor804may be configured to assign multiple display regions the same rank. In this example, the display regions807and809may be assigned a ranking of first, while adjacent display regions (e.g., the regions806and808) may be assigned a lower ranking. As another example, referring toFIG.8a, the first user306may focus his or her eye focus within the display region807, while a second user in the room may also direct his or her eye focus within the display region807. In this example, the processor804may be configured to determine that the eye focus of the first user306and the second user is within the display region807.

In other embodiments, the processor804may be configured to re-rank display regions in accordance with user eye movement. For example, referring toFIG.8a, the first user306may shift his or her eye focus from the display region807to the display region808, and the processor804may detect the change in user eye movement. In some embodiments, the processor804may determine whether the change in the position of a user's eye focus exceeds a predetermined movement threshold. In one of these embodiments, the predetermined threshold may depend on the location of one or more display regions. For example, referring toFIG.8a, the predetermined threshold may be a virtual boundary between two display regions, such as the display regions806and807. In this example, if the user eye focus changes from within the display region807to somewhere within the display region806, the processor804may determine that the change in user eye focus exceeds the predetermined movement threshold.

As another example, the predetermined threshold may be a predetermined distance from the focal point of the user's eye focus. The processor804may be configured to detect the position (e.g., coordinates) of a user's eye focus within the display area. In this example, if the system detects user eye movement, the processor804may be configured to identify the new position of the user's eye focus and determine whether that updated position of the user's eye focus exceeds a predetermined distance (e.g., 10 pixels, 20 pixels, 50 pixels) from the previous position of the user's eye focus within the display area. In some embodiments where the change in user eye focus exceeds a predetermined movement threshold, the processor804may be configured to re-rank one or more display regions within the display area. For example, the processor804may be configured to re-rank a plurality of display regions based on a detected or updated position of a user's eye focus within the display area.

Referring toFIG.8a, in some embodiments, the positions and/or size of display regions may be predetermined, e.g., the position and/or size of display regions do not change in response to user eye movement. In other embodiments, the position and/or size of display regions may be determined by the processor804in response to detecting a change in the position of a user's eye focus within the display area. For example, referring toFIG.8a, the processor804may determine that the display region807extends 100 pixels above, below, to the right, and to the left of the focal point of the user's eye focus. In this example, the display region associated with the user's eye focus may include a square region 200 pixels in height and 200 pixels in width. In another example, the processor804may determine that the display region807extends 100 pixels in a radial direction from the focal point of the user's eye focus. In this example, the display region associated with the user's eye focus may include a circular region having a radius of 100 pixels.

In one embodiment, the size of one or more display regions may be determined by the processor804based on the availability of data transmission resources, such as bandwidth. For example, if the bandwidth of the network (e.g., network100) decreases due to any number of reasons, such as increased network traffic, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect this decrease in bandwidth and adjust the image resource allocation for displaying content on the display device. In this example, the processor804may be configured to decrease the size of the display region within the display area associated with the user's eye focus, which is generally transmitted using more image resources (e.g., higher resolution or transmission rate) than other regions within the display area. Similarly, if the bandwidth of the network (e.g., network100increases) due to any number of reasons, such as decreased network traffic, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect this increase in bandwidth and adjust the image resource allocation for displaying content within the display area. In this example, the processor804may be configured to increase the size of the display region associated with a user's eye focus.

In other embodiments, a user may have the option of determining the amount of data transmission resources (e.g., bandwidth) associated with the transmission and displaying of content (e.g., video images) to a display device. For example, the user may set a minimum bandwidth level that the system must maintain while transmitting content to the display device. In this example, a computing device, such as the processor804, may adjust the image resource allocation for displaying content on a display device in accordance with the bandwidth restrictions set by the user. In one of these embodiments, the processor804may be configured to request from one or more computing devices an amount of available data transmission resources (e.g., bandwidth) on the network.

In some embodiments a computing device, such as the processor804, may be configured to rank one or more display regions within a display area based on image compression data included within the transmitted content. For example, referring toFIG.8a, the processor804may be configured to rank the display regions for the display device302by analyzing image data relating to one or more frames (e.g., I-Frames, P-Frames, and/or B-frames) in the transmitted content. In this example, the processor804may analyze image data relating to the plurality of frames (e.g., I-Frames, P-Frames, and/or B-frames) in content being transmitted to the display device302to detect which portions of the display area are associated with the highest rate of image adjustment (e.g., rate of change or movement of an image between frames) during the transmission. In one of these embodiments, the processor804may rank one or more display regions for the display device302based on which portions of the display area are associated with the highest rate of image adjustment. For example, the processor804may assign a higher rank to display regions of the display device302corresponding to the portions of the display device302having higher rates of image adjustment. In another of these embodiments, the processor804may be configured to devote more image resources to those portions of the display area associated with a higher rate of image adjustment, and conversely, reduce the amount of image resources devoted to those portions of the display area associated with a lower rate of image adjustment.

In another embodiment, the display device302may be configured to rank one or more display regions within a display area in accordance with image compression data included within transmitted content. For example, the display device302may be configured to analyze image data relating to a plurality of frames (e.g., I-Frames, P-Frames, and/or B-frames) in the transmitted content, and detect which portions of the display area are associated with the highest rate of image adjustment during the transmission. In one of these embodiments, the display device302may be configured to devote more image resources to those portions of the display area associated with a higher rate of image adjustment, and reduce the amount of image resources devoted to those portions of the display area associated with a lower rate of image adjustment. In another of these embodiments, the display device302may rank one or more display regions based on those portions of the display area are associated with the highest rate of image adjustment. In another embodiment, a computing device, such as the processor804or other suitable computing device, may be configured to embed one or more data markers (e.g., triggers) in the transmitted content to identify one or more portions of the display area associated with a higher or lower rate of image adjustment. In this embodiment, the display device302may be configured to process the one or more data markers embedded within the content to determine which portions of the display area associated with a higher or lower rate of image adjustment. The display device302may be further configured to process the one or more data markers embedded within the content to determine a prioritization of image resources for one or more display regions.

In some embodiments, a computing device, such as the processor804, may be configured to determine and/or adjust an image resource allocation for displaying content in one or more display regions within a display area. In one of these embodiments, a computing device, such as the processor804, may be configured to adjust an image resource allocation for displaying content in accordance with the ranking of the one or more display regions within the display area. For example, referring toFIG.8a, the display region associated with a user's eye focus (e.g., the display region807) may be given a primary rank (e.g., ranked first) and assigned and/or allocated a high level of image resources (e.g., high image resolution, high frame rate, etc. . . . ). Furthermore, display regions adjacent to the region associated with the user's eye focus (e.g., the display regions806and808) may be given a secondary rank (e.g., ranked second) and assigned and/or allocated a medium level of image resources. Lastly, display regions not adjacent to the region associated with the user's eye focus (e.g., the display region809) may be given a tertiary rank (e.g., ranked third) and assigned and/or allocated a low level of image resources. In some embodiments, a plurality of ranks may be assigned to the one or more display regions within the display area. The processor804may be configured to assign a level of image resources to each display region in accordance with the ranking of that region. For example, higher-ranked display regions may be assigned and/or allocated higher levels of image resources, while lower-ranked display regions may be assigned and/or allocated lower levels of image resources.

In other embodiments, a computing device, such as the processor804, may be configured to adjust an image resource allocation for displaying content based on the proximity of one or more pixels from a user's eye focus. For example, the processor804may be configured to determine the location (e.g., X and Y pixel coordinates) within the display area associated with the focal point of a user's eye focus. In one of these embodiments, a computing device, such as the processor804, may be configured to adjust an image resource allocation for displaying content based on the distance of one or more pixels from the location of the focal point of a user's eye focus. For instance, the plurality of pixels within a 100 pixel radius of the location (e.g., X and Y pixel coordinates) within the display area associated with the focal point of a user's eye focus may be assigned and/or allocated a high level of image resources (e.g., high image resolution, high frame rate, etc. . . . ).

Similarly, the plurality of pixels between a 100 pixel and 250 pixel radius of the focal point of the user's eye focus may be assigned and/or allocated a lower (e.g., reduced) level of image resources than the plurality of pixels within the 100 pixel radius of the focal point of the user's eye focus. Likewise any pixels outside of the 250 radius of the focal point of the user's eye focus may be assigned and/or allocated a lower level of image resources than the plurality of pixels between the 100 pixel and 250 pixel radius of the focal point of the user's eye focus. Although the example above uses a 100 pixel radius to determine the various boundaries for allocating differing levels of image resources, one of ordinary skill in the art will appreciate that any number of pixels may be used to determine the size of the pixel radius (e.g., 25 pixels, 60 pixels, 100 pixels, etc.). In some embodiments, the level of image resources assigned and/or allocated to pixels within the display area may decrease moving outwardly from the focal point of a user's eye focus in a gradient fashion.

There may be a variety of ways in which image resources may be adjusted. For example, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to adjust image resolution. The processor804may be configured to transmit higher-ranked display regions at a higher image resolution. For instance, the processor804or other computing devices operatively coupled to the processor804may be configured to transmit content within higher-ranked display regions (e.g., video images) at a first image resolution, and transmit content within lower-ranked display regions at a second image resolution, wherein the first resolution is higher than the second resolution. In some embodiments, the processor804may be configured to adjust the second resolution (e.g., the resolution of a lower-ranked display region) as a percentage of the first resolution. For instance, the higher-ranked display region may be transmitted at the first image resolution, while the lower-ranked display regions may be transmitted at 90% of the first image resolution. The processor804may also be configured to adjust image resolution based on the availability of data transmission resources on the network (e.g., network100).

As another example, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to adjust an image transmission rate. The processor804may be configured to transmit higher-ranked display regions at a higher image transmission rate (e.g., frame rate). For instance, the processor804or other computing devices operatively coupled to the processor804may be configured to transmit content within higher-ranked display regions at a first frame rate, and transmit content within lower-ranked display regions at a second frame rate, wherein the first frame rate is higher than the second frame rate. In some embodiments, the processor804may be configured to adjust the second frame rate (e.g., the frame rate of a lower-ranked display region) as a percentage of the first frame rate. For instance, the higher-ranked display region may be transmitted at 60 frames per second, while the lower-ranked display regions may be transmitted at 90% of the first frame rate (e.g.,12fewer frames per second). In other embodiments, when determining and/or adjusting an image resource allocation for displaying content, the processor804may be configured to adjust the image transmission rate in accordance with the type of network connection (e.g., wide area network, wireless local area network, local area network, etc.). The processor804may also be configured to adjust image transmission rate based on the availability of data transmission resources on the network (e.g., network100).

As yet another example, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to adjust the processing power per pixel for one or more pixels within the display area (e.g., the display device302). The processor804may be configured to transmit higher-ranked display regions at a higher processing power per pixel. For instance, the processor804or other computing devices operatively coupled to the processor804may be configured to transmit content within higher-ranked display regions at a first processing power per pixel ratio (e.g., floating point operations per second (“FLOPS”)-per-pixel), and transmit content within lower-ranked display regions at a second processing power per pixel ratio, wherein the first processing power per pixel ratio is higher than the second processing power per pixel ratio. In some embodiments, the processor804may be configured to adjust the second processing power per pixel ratio as a percentage of the first processing power per pixel ratio. For instance, the higher-ranked display region may be transmitted at400FLOPS/pixel, while the lower-ranked display regions may be transmitted at 90% of the first processing power per pixel ratio (e.g.,360FLOPS/pixel). In some embodiments, the processor804may be configured to transmit instructions to a computing device, such as a graphics processing unit associated with the display device302, to adjust the processing power per pixel for one or more display regions within the display area. The processor804may also be configured to adjust processing power per pixel ratios based on the availability of data transmission resources on the network (e.g., network100).

As another example, in cases where content transmitted to a display device is rendered in 3D, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to adjust the polygon count of images displayed on the display device. The processor804may be configured to transmit 3D rendered images within higher-ranked display regions at a higher polygon count, and transmit 3D rendered images within lower-ranked display regions at a lower polygon count. In still other examples, a computing device, such as the processor804, may be further configured to adjust texture resolution. The processor804may be configured to transmit higher-ranked display regions at a higher texture resolution. For instance, the processor804or other computing devices operatively coupled to the processor804may be configured to transmit content (e.g., 3D rendered images) within higher-ranked display regions at a first texture resolution, and transmit content within lower-ranked display regions at a second texture resolution, wherein the first texture resolution is higher than the second texture resolution. In some embodiments, the processor804may be configured to adjust shading/lighting models associated with rendering 3D content displayed on a display device by modifying one or more algorithms defined in a software application program, such as a pixel shader. For example, the processor804or other computing devices operatively coupled to the processor804may be configured to transmit content (e.g., 3D rendered images) within one or more higher-ranked display regions using a first shading algorithm and transmit content within one or more lower-ranked display regions using a second shading algorithm. As will be appreciated, various shading algorithms may be utilized by a suitable software application program to render 3D images, such as flat shading, Gouraud shading, Phong shading, etc.

As still another example, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to adjust the bit depth of the content transmitted to the display device(s). As yet another example, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to adjust a degree of image compression for content transmitted to the display device(s). The processor804may be configured to transmit higher-ranked display regions at a higher degree of compression (e.g., higher bit rate). For instance, the processor804or other computing devices operatively coupled to the processor804may be configured to transmit content within higher-ranked display regions at a first bit rate, and transmit content within lower-ranked display regions at a second bit rate, wherein the first bit rate is higher than the second bit rate. In some embodiments, the processor804may be configured to adjust the second bit rate (e.g., the bit rate of a lower-ranked display region) as a percentage of the first bit rate. For instance, the higher-ranked display region may be transmitted at 8 Mbps, while the lower-ranked display regions may be transmitted at 90% of the first bit rate (e.g., 7.2 Mbps). The processor804may also be configured to adjust bit rates based on the availability of data transmission resources on the network (e.g., network100).

An allocation of image resources may be determined and/or adjusted based on any various combination of the image resources (e.g., characteristics) discussed above. For example, in some embodiments, a computing device, such as the processor804, may be configured to simultaneously adjust one or more image characteristics such as image resolution, degree of image compression, bit rate, and/or processing power per pixel. In other embodiments, when determining and/or adjusting an image resource allocation for displaying content, a computing device, such as the processor804, may be configured to prioritize the order in which the one or more image characteristics may be adjusted. In one of these embodiments, the processor804may be configured to prioritize the order in which one or more image characteristics are adjusted based on the viewing distance of one or more users in a room. For example, if a user is within a first threshold distance to the display device302, the processor804may be configured to first adjust the image resolution of content transmitted to the display device302, and then subsequently adjust one or more other image characteristics. In this example, if a user is within a second threshold distance, wherein the second threshold distance is a further viewing distance than the first threshold distance, the processor804may be configured to first adjust the degree of image compression for the content transmitted to the display device302.

In some embodiments, the processor804may be configured to adjust one or more image characteristics within a threshold image resource range. In one of these embodiments, the processor804may be configured to adjust a first image resolution of content transmitted to the display device302between a minimum and maximum level of image resolution. For example, the processor804may be configured to adjust the image resolution of content transmitted to the display device302between a minimum image resolution (e.g., 1,000 pixels-per-inch) and a maximum image resolution (e.g., 10,000 pixels-per-inch) within a first display region. In this example, the minimum and maximum levels of image resolution (e.g., threshold image resource range) may be modified by one or more viewers viewing the display device302. In another example, the minimum and maximum levels of image resolution may be modified by an administrator for the content provider.

In another of these embodiments, the processor804may be configured to adjust the threshold image resource range based on the viewing distance of one or more users viewing the display device302. For example, if a user moves a certain distance from the display device302such that their current viewing distance exceeds a threshold viewing distance, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect the user's current viewing distance and adjust the maximum level of image resolution downward. In this example, the processor804may be configured to decrease the maximum level of image resolution for transmitting content because the user may no longer visually perceive the benefits of a higher image resolution given their current viewing distance (e.g., the user is too far from a display device to distinguish content transmitted at higher levels of image resolution). Similarly, if the user's viewing distance decreased, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect this decrease in viewing distance and adjust the maximum level of image resolution upward.

In still another of these embodiments, the processor804may be configured to adjust the threshold image resource range based on the availability of data transmission resources, such as bandwidth. For example, if the bandwidth of the network (e.g., network100) decreases due to any number of reasons, such as increased network traffic, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect this decrease in bandwidth and adjust the maximum level of image resolution downward. In this example, the processor804may be configured to decrease the maximum level of image resolution for transmitting content given that content transmitted at higher levels of image resolution consumes more bandwidth than content transmitted at a lower level of image resolution. Similarly, if the bandwidth of the network (e.g., network100) increases due to any number of reasons, such as decreased network traffic, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect this increase in bandwidth and adjust the maximum level of image resolution upward.

In yet another of these embodiments, the processor804may be configured to adjust a threshold image resource range based on one or more technical limitations of the display device302, such as display screen resolution or display size. For example, for display devices having smaller display screens, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect the size of the display screen and adjust a default maximum level of image resolution downward. In this example, the processor804may be configured to decrease the default maximum level of image resolution for transmitting content on a display device because the user may no longer visually perceive the benefits of a higher image resolution given the size of the display area (e.g., the display screen of a display device is too small for a user to distinguish content transmitted at higher levels of image resolution). As another example, if the display area increases for any number of reasons, such as content is transmitted on a larger portion of the display device, or to a different, larger display screen, a computing device, such as the processor804or other computing device(s) operatively coupled to the processor804, may detect the increase in the display area and adjust the maximum level of image resolution upward.

In some embodiments, when adjusting an allocation of image resources for a first image characteristic, the processor804may be configured to begin adjusting a second image characteristic when a threshold image resource range for the first image characteristic has been met and/or exceeded. For example, where the maximum image resolution has been set to 10,000 pixels-per-inch and the processor804adjusts image resolution (e.g., the first image characteristic) by increasing the image resolution of a first display region, the processor804may be configured to stop adjusting the image resolution of content transmitted to the first display region when the image resolution of that region reaches and/or exceeds 10,000 pixels-per-inch. In this example, when the maximum image resolution of the first display region has been met and/or exceeded, the processor804may be configured to begin adjusting the degree of image compression or other image characteristic (e.g., second image characteristic) for content transmitted to the first display region.

In another embodiment, when adjusting an allocation of image resources for a first image characteristic, the processor804may be configured to begin adjusting a second image characteristic based on an availability of data transmission resources, such as bandwidth. For example, where the image resolution for a first display region has been increased, the processor804may be configured to stop adjusting the image resolution of content transmitted to the first display region when the amount of bandwidth (e.g., transmission resources) consumed by the system meets or exceeds a maximum threshold limit. In this example, when the maximum bandwidth threshold limit has been met and/or exceeded, the processor804may be configured to begin adjusting the degree of image compression or other image characteristic (e.g., second image characteristic) for content transmitted to the first display region. As will be appreciated, there a variety of ways in which the transmission resources threshold limit may be set and/or modified. For example, one or more users operating the display device302may have the option of setting and/or modifying the transmission resources threshold limit. As another example, the transmission resources threshold limit may be set and/or modified by an administrator for the content or network provider. As still another example, a computing device, such as the processor804, may be configured to adjust the transmission resources threshold limit based on a level of data transmission resource usage and/or data transmission resource demand for the system.

In some embodiments, where an image resource allocation for displaying content on a display device is adjusted, one or more computer devices may be configured to store in memory the content transmitted to the display device. In one of these embodiments, a computing device, such as the processor804, may be configured to store in memory content presented to the user having an adjusted resource allocation for displaying said content. In another of these embodiments, a computing device, such as the processor804, may be configured to store in memory the original content (i.e., content displayed utilizing an initial, non-adjusted image resource allocation) transmitted to the display device. In this example, although a user may be presented with content on a display device having an adjusted image resource allocation, the processor804may be configured to store in memory the original content that is transmitted to the user from the content provider.

In one embodiment, one or more computing devices associated with the camera304and the display device302may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302. In some embodiments, one or more computing devices operatively coupled to the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302. For example, the gateway111may be configured to adjust an image resource allocation for displaying content on the display device302. In this example, the gateway111may be further configured to transmit information to one or more computing devices, such as the processor804, regarding the adjustment of an image resource allocation for displaying content on the display device302. In other embodiments, the processor804may be configured to transmit information to one or more computer devices, such as the gateway111, regarding the adjustment of an image resource allocation for displaying content on the display device302.

In some embodiments, one or more computing devices may be configured to determine and/or adjust an image resource allocation for displaying content on a display device in accordance with one or more triggers embedded in said content. For example, a trigger may be embedded in content transmitted to the display device302identifying one or more portions of the image on the display device302where a character's face may be displayed. In this example, the processor804may be configured to process information associated with one or more embedded triggers to adjust the allocation of image resources (e.g., image resolution) for displaying content within those portions of the display device302where the character's face may be displayed. In other embodiments, a trigger may be embedded in content transmitted to the display device302identifying one or more portions of the image on the display device302to be visually emphasized to the user. The embedded triggers may contain information identifying different characteristics of the content being transmitted, such as content type or subject matter, content duration, image resource allocations, etc.

In one of these embodiments, the processor804may be configured to process information associated with one or more embedded triggers identifying the particular position on a display device where image resources should be allocated. For example, the one or more triggers detected by the system may contain information identifying one or more display regions within the display area and their respective allocation of image resources. In this example, the processor804may be configured to increase or decrease the image resolution, degree of image compression, and/or other image characteristic for one or more display regions within the display area in accordance with information obtained from the one or more embedded triggers. Information obtained from the one or more triggers may also include a priority ranking for one or more display regions within the display area (to be discussed further below).

In another of these embodiments, the one or more triggers detected by the system may contain coordinate information identifying one or more locations (e.g., coordinates) within the display area to visually emphasize to the viewer. For example, the coordinate information obtained from the one or more triggers may indicate the specific horizontal and vertical coordinates (e.g., X and Y pixel coordinates) within the display area where a viewer's eyes should focus (e.g., virtual focal point of user's eye focus). The one or more display regions associated with the identified coordinates may receive a higher rank and/or level of image resources, while display regions moving away from the identified coordinates may receive a lower rank and/or level of image resources. In some instances, display regions moving away from the identified coordinates (or region associated with the identified coordinates), such as in a radial fashion, may be assigned or allocated lower levels of image resources in a gradient fashion. Similarly, display regions moving away from the identified coordinates (or region associated with the identified coordinates), such as in a radial fashion, may be assigned lower rankings (e.g., priority) in a gradient fashion.

In some embodiments, one or more computing devices may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302in accordance with a user's viewing distance from the display device302. For example, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302in accordance with the first user's306viewing distance from the display device302. In this example, the viewing distance of the first user306(as illustrated by a viewing distance808) is processed by the processor804, as illustrated by the processing cloud810. The processing clouds used throughout the disclosure are for illustrative purposes only and are not intended to limit or define the manner in which certain information is processed.

In one embodiment, a computing device, such as the processor804, may be configured to adjust an image resource allocation for displaying content on the display device302in accordance with the viewing distance of the user closest to the display device302(e.g., shortest viewing distance). In another embodiment, a computing device, such as the processor804, may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302in accordance with the viewing distance of the user closest to a transmitted remote control signal, as illustrated by element620in the processing cloud810. In some embodiments, a computing device, such as the processor804, may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302in accordance with one or more viewing distances stored in a user profile. In one embodiment, a computing device, such as the processor804, may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302when the user exceeds a threshold distance from the display device302. For example, the processor804may be configured to reduce the image resolution of the content displayed on the display device302to a predetermined level of resolution when the user is more than 10 feet from the display device. In another example, the processor804may be configured to reduce the image resolution of the content displayed on the display device302to a predetermined level of resolution when the user is more than 15 feet from the display device.

In one of these embodiments, the processor804may be configured to identify an active user profile for the purpose of determining and/or adjusting an image resource allocation for displaying content on the display device302in accordance with the distance of the user associated with the identified user profile. In yet another embodiment, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302based on the aggregate viewing distance of multiple users viewing the display device302within the room300. For example, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302based on a weighted average viewing distance of multiple users within the room300. In another example, referring toFIG.5, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302based on the average distance of the first user306and the second user307from the display device302.

In other embodiments, the processor804may be configured to determine and/or adjust an image resource allocation based on a weighted average of one or more distances of users in the room. The processor804may be configured to weight (e.g., prioritize) the viewing distance of specific users in a room over other users. For example, the processor804may be configured to apply one or more weights to the viewing distance of the user(s) associated with a user profile or account. For example, the processor804may be configured to detect a user within the room300associated with a stored user profile, and apply one or more weights to the viewing distance of that user when determining an average user viewing distance. In some embodiments, the processor804may be configured to weight (e.g., prioritize) the viewing distance of one or more users in a room based on the type or subject matter of content being transmitted. For example, if children's video programming is being transmitted, the processor804may be configured to apply one or more weights to the viewing distance of any user(s) under a predetermined age that are viewing the display device.

In some embodiments, a computing device, such as the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302in based on a predetermined viewing distance or range of distances for a user viewing the display device. A fixed viewing distance from a display device or range of distances from a display device (e.g., maximum and minimum distance) may be calculated to optimize the visual experience of a user. Optimal viewing distances for a user are based on the size of the display device. For example, manufacturers or retailers of a display device may provide a recommended viewing distance for viewing the display device based on the size of the display area for that display device. In one of these embodiments, the processor804may be configured to retrieve from memory one or more predetermined viewing distances for the display device302.

In another of these embodiments, the processor804may determine and/or adjust an image resource allocation for displaying content on the display device302based on a comparison of the predetermined viewing distance with the actual viewing distance of one or more users viewing the display device302. For example, the processor804may determine a first allocation of image resources based on the predetermined viewing distance for the display device302in accordance with one or more embodiments of the present disclosure. In this example, the processor804may be configured to adjust the first allocation of image resources in accordance with the actual (e.g., determined) viewing distances of one or more user viewing the display device302. As will be appreciated, the processor804may utilize the predetermined viewing distance for the display device302to determine a baseline allocation of image resources, and may subsequently adjust said allocation of resources based on the movement of one or more users within the viewing area.

In other embodiments, a computing device, such as the processor804, may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302based on the size of the display area. For example, the processor804may be configured to determine a predetermined viewing distance in accordance with the size of the display area for the display device302. In this example, the processor804may be configured to determine and/or adjust an allocation of image resources for transmitting content to the display device302in accordance with the predetermined viewing distance. In one of these embodiments, as discussed in more detail further below, the size of a display region associated with a user's eye focus may be determined in accordance with the size of the display area.

In other embodiments, when determining and/or adjusting an image resource allocation for displaying content on a display device, the processor804may be configured to exclude the viewing distances of one or more users. In one of these embodiments, the processor804may be configured to determine whether a user is exiting or entering a room, and exclude that user's viewing distance when determining and/or adjusting image resource allocations. The processor804may determine that a user is entering or exiting a room by recognizing that the user is within a predetermined distance from and/or moving towards an exit (e.g., door) within the room. In another of these embodiments, the processor804may be configured to determine whether one or more users are within a predetermined distance relative to the display area, and exclude those users' viewing distance when determining and/or adjusting image resource allocations. For example, any users having a viewing distance greater than 15 feet from a display device may be excluded. As another example, any users having a viewing distance greater than and/or less than 3 feet from the predetermined viewing distance for the display device302may be excluded. In yet another of these embodiments, the processor804may be configured to exclude one or more users' viewing distance based on the subject matter or type of content being transmitted to the display device. For example, if children's video programming is being transmitted, the processor804may be configured to exclude the viewing distances of any viewers over a predetermined age. In still another of these embodiments, the processor804may be configured to exclude one or more user's viewing distance based on whether the user is focusing his or eye upon the display device. As will be appreciated, in accordance with the examples of excluding one or more user viewing distances provided above, the processor804may be configured to exclude the viewing distance of one or more users when determining the average viewing distance of multiple users within a viewing area.

In some embodiments, one or more computing devices may be configured to determine and/or adjust an image resource allocation for displaying content on a display device in accordance with data relating to the eye focus a user. For example, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content based on data relating to the location of a user's eye focus within the display area (e.g., the display device302). As another example, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302in accordance with data relating to the change in position (e.g., movement) of the first user's eyes412, as illustrated inFIG.8bas broken lines612within the processing cloud810. In another embodiment, when user eye movement is detected and the user is also moving within the room300, the processor804may be configured to determine and/or adjust an image resource allocation for displaying content on the display device302after user movement has ceased. For example, if user movement is detected, the processor804may be configured to reduce the image resolution of content displayed on one or more regions of the display device302while the user is moving. In yet another embodiment, when user eye movement is detected and the user is also moving within the room300, the processor804may be configured to determine and/or adjust an allocation of image resources a predetermined period of time after user movement has ceased.

Referring toFIG.8b, the image resource allocation for displaying content on the display device302may be adjusted by a computing device, such as the processor804, in accordance with data relating to the user's eye focus. In some embodiments, the image resource allocation for displaying content on the display device302may be adjusted such that the allocation of image resources is higher at a central region of the display area where the user's eyes are focused. In other embodiments, the image resource allocation for displaying content on the display device302may be adjusted such that the image resolution of the displayed content may be reduced in one or more regions of the display device302where a user's eyes are not focused. For example, the processor804may be configured to transmit content at a higher image resolution for the region of the display area where the user's eyes are focused, as illustrated by a display region814. In this example, the processor804may transmit content at a lower image resolution for those portions of the display area outside of the display region814.

As will be appreciated, there are a variety of ways in which to determine the size of the central region of the display area associated with a user's eye focus. In one embodiment, a user may have the option of establishing the size of the central region by using a slider tool that adjusts the size of the central region. In another embodiment, a computing device, such as the processor804, may determine a size of the central region based on the size of the display area. For example, the processor804may determine a size of the central region, wherein the geometric area of the central region is a percentage of the geometric area of the display screen for the display device. For instance, where the geometric area of the central region is a percentage (%) of the geometric area of the display screen, and where the display screen is a rectangle consisting of (l) pixels in length and (w) pixels in width, the geometric area (e.g., size) of the central region may be calculated as follows:
Geometric Area of Central Region (in pixels)=√(((l*w)*%)/π)

In some embodiments, one or more computing devices may be configured to adjust the image resource allocation for displaying content such that the image resolution of the displayed content may be highest near a central region of the display area associated with the user's eye focus, and gradually decrease in a radial gradient from the central region of where the user's eyes are focused within the display area to the periphery of where the user's eyes are focused within the display area. In one of these embodiments, the processor804may be configured to reduce image resources, by a predetermined factor, for a first outer region of the display area, wherein said outer region is a predetermined distance from the central region of where the user's eyes are focused, as illustrated by a display region815inFIG.8b. For example, the processor804may be configured to reduce image resources by one-half for a first outer region, wherein said first outer region consists of the plurality of pixels outside the central region but within a predetermined distance from the central region (e.g., the display region815). As will be appreciated, the processor804may be configured to further reduce image resources for a plurality of outer regions, wherein each outer region is a predetermined distance from the central region of where the user's eyes are focused.

Users viewing content on the display device302may have varying limits of visual capacity. In some embodiments, a user may modify the default size of the central region of eye focus (e.g., the display region814) utilized by the system to determine and/or adjust a resource allocation for displaying content on a display device. In one of these embodiments, a computing device, such as the processor804, may be configured to calibrate the size of the central region of eye focus within the display area based on the user's visual capacity. The term “visual capacity” encompasses various metrics to determine a user's ability to view and comprehend visual images, including, but not limited to a user's visual acuity, peripheral vision, and other visual metrics known to one of ordinary skill in the art. For example, the processor804may be configured to identify a user's visual capacity, such as visual acuity, and adjust the size of the central region of eye focus within the display area in accordance with the determined visual capacity. One of skill in the art will appreciate that various other embodiments may be utilized to calibrate the size of the central area of eye focus, without departing from the scope of the present disclosure. For example, the display device302may display various images to a user, and based on information provided by the user, the one or more computing devices may be configured to process the user feedback to determine the limits of the user's visual capacity (e.g., visual acuity, peripheral vision, etc.). In another example, the user may be provided with a slider option to adjust the size of the central region of focus within the display area, thus allowing the user to prioritize the allocation of image resources within the central region of the display area associated with the user's eye focus, as well as other regions, such as those regions associated with the periphery of the user's eye focus.

In another embodiment, a user may adjust the default radial image resolution gradient utilized by the system based on the user's visual acuity and/or peripheral vision. In other embodiments, a user may adjust the default image resolution utilized by the system. In one of these embodiments, the user may have the option of calibrating the default image resolution of the content displayed on the display device302based on the user's visual acuity. For example, the display device302may display various images or content to a user, and based on feedback provided by the user, one or more computing devices may be configured to process user input to determine an optimal image resolution for displaying content within the display area (e.g., the display device302) based on the user's visual acuity. In another example, the user may be provided with a slider option that adjusts the image resolution of content displayed within the display area, thus allowing the user to provide the system with data relating to the user's display preferences.

FIG.9depicts a flowchart for an exemplary method according to one embodiment of the disclosure, which can be performed by one or more computing devices such as the gateway111, the display device302, the camera304, the server107, the processor804, or any other desired computing device. When describing the steps of the method below, the term “system” may be used when referring to each component, either singularly or collectively, employed at each step of the method. Such usage is merely for ease in discussing the method, and should not be interpreted as limiting the exact embodiment in which the following method may be performed.

Referring toFIG.9a, at step902, an initial configuration of the system may be performed such as a calibration of a camera and/or a display device, or loading of previously saved user profile information and other associated data. Other system configuration may include identifying one or more display regions within the display area (e.g., the display device), calibration of a default image resource allocation, calibration of a user's central eye focus or preferred image resolution gradient, and various other user preferences. At step904, the system may determine whether a content session is currently active or in progress. A content session may occur when the system is in the process of providing content for a user's consumption, such as displaying a television program for a user to view. If there is presently no active content session, the device may repeat step904until the device determines that there is an active content session.

In this example, if the system determines that a content session is currently active or in progress, the method may proceed to step906, where the system may capture one or more images of the room or viewing area using an image capturing device (e.g., the camera304), which may be integrated into a display device.FIGS.4and6are illustrations of examples of captured images of the viewing area. An image capturing device may capture an image or a plurality of images. One or more image capturing devices may also be used to capture the plurality of images which may be used to compose a captured image for processing by the system. For example, different cameras which are configured to capture light in different electromagnetic spectrums may be used to capture the images. A composed captured image for processing may be formed from a combination of visible light and light outside the visible spectrum such as infrared radiation. One or more image capturing devices may be used to capture or collect 3D information of objects within the viewing area to recognize and/or determine one or more users, their bodies, user eye movement, the remote control device or signal, and various other objects.

In this example, after the viewing area image has been captured by an image capturing device, such as a camera, the captured image may be analyzed for facial recognition and user location determination at step908. During step908, the location of one or more users within the viewing area may be stored by the system, as well as information identifying one or more users in the captured image(s). The system may also be configured to store information relating to the viewing distance of one or more users in the captured image(s).

At step910, the system may determine whether one or more users in the captured image(s) are viewing the content displayed on the display device. If there is (are) presently no user(s) viewing the display device, the method may proceed to step912, where the system may transmit content (e.g., output one or more images) for display within the display area (e.g., the display device) in accordance with a default image resource allocation. In some embodiments, if there is (are) presently no user(s) viewing the display device, the system may continue to transmit content for display within the display area (e.g., the display device) in accordance with the current (e.g., most recent) allocation of image resources. In this example, after the system transmits the content for display within the display area (e.g., the display device) in accordance with the appropriate image resource allocation, the method may return to step906, where the system may capture additional image(s) of the viewing area using an image capturing device.

Returning to step910, after a determination has been made that one or more users are viewing content on the display device, the method may proceed to step914, where the system may determine whether one or more users in the viewing area have not been previously recognized by the system. In this example, if the system detects an unknown user in the viewing area, the method may proceed to step916. As will be appreciated, in some embodiments, the system may be configured to not perform steps914,916, and918. For example, a user may be provided with a preference option permitting the user to identify whether the system may determine and/or adjust image resource allocations based on the location of unknown users within the viewing area.

At step916, the system may identify the unknown user detected during step914, and determine the unknown user's viewing location and viewing distance from the display device. During step916, the system may prompt the user to input user data and/or other preferences. At step918, the system may store the information determined and/or collected during step916. In this example, after the system stores this information, the method can return to step914and the system identifies whether additional users in the viewing area have not been previously detected. In some embodiments, the system may simultaneously perform steps916and918for each of the unknown users detected during step914, and the method may proceed to step920after step918. In other embodiments, the system may be configured to skip steps914,916, and918where at least one or more users in the viewing area have been previously detected or recognized by the system.

After detecting one or more unknown users in the viewing area, the method may proceed to step920, where the system may determine whether any user movement in the viewing area has been detected. During step920, the system may identify one or more users in the viewing area associated with the detected movement. In this example, if the system detects user movement in the viewing area, the method may proceed to step922. At step922, the system may determine the location of one or more users in the viewing area associated with the movement detected during step920. During step922, the system may determine the viewing distance(s) of one or more users in the viewing area associated with the movement detected during step920. At step924, the system may store in memory the location, viewing distance(s), and other data determined and collected during step922. In some embodiments, the system may perform steps922and924for each user associated with the movement detected in step920.

In some embodiments, the system may be configured to perform steps920,922, and924for a subset of the users in the viewing area. For example, the system may be configured to perform steps920,922, and924for the user(s) in the viewing area associated with an active user profile. In another example, the system may be configured to perform steps920,922, and924for the user(s) in the viewing area based on stored user preferences. In other embodiments, the system may be configured to skip one or more of steps920,922, and924.

At step926, the system may determine whether one or more users in the viewing area have directed their eye focus to a different location of the display device. During step926, the system may determine whether one or more users have directed their eye focus on a different display region on the display device. In this example, if the system detects a change in eye focus for one or more users in the viewing area, the method may proceed to step928, where the system may store in memory the location of the detected user eye focus, along with other data for the one or more users associated with the change in eye focus detected in step926. As will be appreciated, in some embodiments, the system may be configured to skip steps926and928. In one of these embodiments, the system may be configured not to monitor or detect a change in one or more user's eye focus when the one or more users in the viewing area are outside of a threshold viewing distance. For example, if all of the users in the viewing area are more than 15 feet from the display device, the system may not detect or monitor the change in a user's eye focus since the user(s) are sufficiently far away to not warrant this form of tracking.

Referring toFIG.9b, at step930, the system may retrieve information relating to the current allocation of image resources for the content (e.g., video image) being displayed to one or more users. During step930the system may also retrieve information relating to the current priority (e.g., rankings) of the one or more display regions within the display area (e.g., display device). After retrieving information relating to the current display region priority and image resource allocation, the method may move to step931. At step931, the system may retrieve movement data for the one or more users within the viewing area. Such movement data may include location and distance data collected during step922, and user eye focus data collected and stored during steps926and928, respectively.

At step932, the system may determine whether the detected change in location and/or eye focus for one or more users exceeds a predetermined threshold. In this example, if the system determines that a user's location or change in eye focus exceeds the predetermined threshold, the method may proceed to step933, where the system may identify one or more display regions associated with the change in user eye focus exceeding the predetermined threshold during step932. In some embodiments, a user may be provided with a preference option permitting the user to establish the one or more predetermined thresholds utilized by the system. In other embodiments, during step933, the system may be configured to identify one or more display regions in accordance with triggers embedded within the transmitted content.

If the system determines that the user's location or change in eye focus does not exceed the predetermined threshold, the method may proceed to938, where the system may transmit the content (e.g., output one or more images) for display within the display area (e.g., display device) in accordance with the adjusted allocation of image resources. In this example, since the user's location or change in eye focus did not exceed the predetermined threshold, the system may not adjust the allocation of image resources for the content being displayed within the display area (e.g., display device), and thus the system may display the content in accordance with the image resource allocation retrieved during step930.

At step934, the system may begin a loop that is performed for one or more of the display regions identified in step933. In some embodiments the system may not perform step933, and may begin a loop that is performed for the one or more display regions identified during step902. In another embodiment, during step934, the system may be configured to begin a loop that is performed for each display region identified during step902.

In step935, the system may retrieve updated image resource allocation and priority (e.g., ranking) data for the display region being analyzed. In step936, the system may begin a loop for one or more image resources (e.g., image resolution, image transmission rate, etc.) identified at step935. In step937, the system may begin to adjust (e.g., increase or decrease) the image resource being analyzed at step936in accordance with the updated image resource allocation and priority data retrieved during step935.

After one or more resources have been analyzed for a display region, the method may return to step934to continue the loop until all of the display regions identified at step933have been analyzed. After the identified regions have been analyzed, the method may proceed to step938, where the system may transmit the content (e.g., output the one or more images) for display within the display area (e.g., the display device) in accordance with the adjusted image resource allocation. Referring toFIG.9a, after step938, the method may proceed to step904, where the system may determine whether a content session is currently active or in progress. If yes, the method may proceed through each step as indicated above. If no, the method ends.

Although example embodiments are described above, the various features and steps may be combined, divided, omitted, and/or augmented in any desired manner, depending on the specific outcome and/or application. Various alterations, modifications, and improvements will readily occur to those skilled in art. Such alterations, modifications, and improvements as are made obvious by this disclosure are intended to be part of this description though not expressly stated herein, and are intended to be within the spirit and scope of the disclosure. Accordingly, the foregoing description is by way of example only, and not limiting. This patent is limited only as defined in the following claims and equivalents thereto.