POSITIONAL TRACKING SYSTEM WITH HOLOGRAPHIC ENCODED POSITIONS

An emitter laser illuminates multiple reflectors each of which produces a unique robust code, with the laser being sequentially moved between illuminating successive reflectors. The reflectors send light to a holographic film, and the laser poses can be associated with the respective codes. Subsequently, an illuminator such as another laser can illuminate the film, and a sensor positioned near the film records what code is produced, so that the code can be correlated to a laser pose with respect to the film and sensor.

FIELD

The application relates generally to positional tracking systems with holographic encoded positions.

BACKGROUND

Many applications benefit from knowing the relative location of an object such as a virtual reality (VR) or augmented realty (AR) headset relative to an object such as a display. Computer games for instance, can benefit from knowing such locations.

SUMMARY

A system records a hologram of an array of position encodings onto photographic film (holographic film) to be read by a digital or analog sensor (charge coupled device (CCD), complementary metal-oxide semiconductor (CMOS), photodiodes, etc.). The position of the laser source used for the recording of the hologram is encoded into the holographic film as a reflection of its light onto a series of objects used to encode the physical position of the laser source relative to the film. The encoding onto the hologram can be a simple series of bars, splotches, lines, bar codes, QR codes, or any form of noise robust image based encoding scheme to encoding one or more values representing the relative Azimuth angle A, Bearing angle B, Roll angle R, X position, Y position, Z position, or any combination of angles and positions to form one, two or three dimensions of positions and angles of the laser source. These encodings represent the relative pose of the laser source to the holographic film.

A mechanized system can move a laser source and a series of position encoding object reflectors such that for each position of the laser emitter, a different encoding object reflection is recorded into one or more areas on the holographic film. The object reflectors are positioned to reflect an image based encoding of one or more pose values. The pose value(s) are a ground truth measurement of the laser source relative to the holographic film. The mechanized system moves the laser light source and the pose encoding reflector objects over a 1D, 2D or 3D array of positions to record the positional spacing and/or relative angles of the laser source into the holographic film.

A real-time positional tracking system is achieved by placing the previously recorded holographic film over a light to electronic sensor like a CCD, CMOS, Photodiode array. The sensors measure the light from areas of the holographic film that relay an encoding of the pose of a remote laser source. The laser source can be the fixed reference point in this positional tracking system. By decoding the light patterns from the holographic film onto the sensor(s) the 1D, 2D or 3D pose of the remote laser source can be determined. In addition, polarization can be used to improve the robustness of the pose tracking or add a single axis of orientation (Azimuth, Bearing or Roll) tracking. A static polarizer on the laser source and on the holographic film can change the encoding patterns to portray additional information to aid the positional tracking.

Using the techniques herein, a positional tracking system can be constructed from a coherent laser source and holographic film placed over a light sensor, with software or hardware to decode the holographic patterns as positional and/or orientation tracking information. The laser source can operate in the infrared range and be invisible to the naked eye, as well as being modulated at a very high carrier frequency to be robust to noise from sunlight.

Accordingly, a method for recording a hologram of an array of position and/or orientation encodings (all examples of pose encodings) onto holographic film includes moving light from an encoding laser across plural position encoding object reflectors. At least some of the reflectors emit patterns of the light differently from each other to establish respective coded emissions. The method includes receiving each of the coded emissions from the reflectors on respective regions of the film, and correlating the coded emissions to respective positions of a laser.

In some implementations, the method can include illuminating the film using at least one indicator laser, juxtaposing the film with at least one sensor to sense light from areas of the film illuminated by the indicator laser and representing at least one of the coded emissions, and decoding signals from the sensor representing the at least one coded emission to return a respective position of a laser. In such embodiments, the position (and if desired other pose information) of a laser returned from decoding the signals is a position of the indicator laser, which may be an IR laser. Light from the indicator laser can be modulated at a carrier frequency of at least one megahertz.

In some embodiments the position encoding object reflectors establish plural different splotches, plural different lines, plural different bar codes, and plural different quick response (QR) codes.

In another aspect, an apparatus includes at least one indicator laser and at least one holographically recorded film having plural coded regions, with each coded region representing a code different from other coded regions on the film. At least one sensor is provided to sense light from at least one coded region of the film illuminated by the indicator laser. Also, at least one decoder is configured for decoding signals from the sensor representing the at least one coded region to return a respective position, orientation, or other pose information of the indicator laser.

In another aspect, an apparatus includes at least one holographically recorded film having plural coded regions. Each coded region represents a code different from other coded regions on the film. At least one data storage medium correlates the coded regions to respective positions of a laser. Alternatively or in addition, a circuit such as but not limited to an application specific integrated circuit (ASIC) may be provided for decoding information in the coded regions to render an output representing the pose information of the laser.

The details of the present application, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which:

DETAILED DESCRIPTION

This disclosure relates generally to computer ecosystems including aspects of consumer electronics (CE) device networks such as but not limited to computer game networks. A system herein may include server and client components, connected over a network such that data may be exchanged between the client and server components. The client components may include one or more computing devices including game consoles such as Sony PlayStation® or a game console made by Microsoft or Nintendo or other manufacturer virtual reality (VR) headsets, augmented reality (AR) headsets, portable televisions (e.g. smart TVs, Internet-enabled TVs), portable computers such as laptops and tablet computers, and other mobile devices including smart phones and additional examples discussed below. These client devices may operate with a variety of operating environments. For example, some of the client computers may employ, as examples, Linux operating systems, operating systems from Microsoft, or a Unix operating system, or operating systems produced by Apple Computer or Google. These operating environments may be used to execute one or more browsing programs, such as a browser made by Microsoft or Google or Mozilla or other browser program that can access websites hosted by the Internet servers discussed below. Also, an operating environment according to present principles may be used to execute one or more computer game programs.

Information may be exchanged over a network between the clients and servers. To this end and for security, servers and/or clients can include firewalls, load balancers, temporary storages, and proxies, and other network infrastructure for reliability and security. One or more servers may form an apparatus that implement methods of providing a secure community such as an online social website to network members.FIG. 16described below provides example components that may be used herein in the appropriate combinations.

A processor may be any conventional general purpose single- or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, and control lines and registers and shift registers.

Software modules described by way of the flow charts and user interfaces herein can include various sub-routines, procedures, etc. Without limiting the disclosure, logic stated to be executed by a particular module can be redistributed to other software modules and/or combined together in a single module and/ or made available in a shareable library.

Further to what has been alluded to above, logical blocks, modules, and circuits described below can be implemented or performed with a general purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device such as an application specific integrated circuit (ASIC), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor can be implemented by a controller or state machine or a combination of computing devices.

FIG. 1illustrates a system10that includes one or more encoding lasers12that may emit light through one or more adjustable polarizers14such as polarization filters onto a selected reflector A in an array16of reflectors. As shown inFIGS. 1 and 6, in the example transmissive system the laser is on the same side of the holographic film discussed below as are the reflectors, Each reflector may deflect light including by way of internal reflection or refraction. As discussed more fully below, each reflector (or a group of reflectors when simultaneously illuminated by the laser) may establish a pattern of light deflection that creates a robust code.

In the example shown the array16is a two dimensional array but could be a three dimensional array as shown by the x-y-z axes18. Light20from the encoding laser12that does not impinge on a reflector can interfere with light22that passes through a reflector, with the resulting interference pattern being encoded in a region24of a holographic film26. Once illumination of a first reflector “A” is encoded onto the region24of the film26, a motor28that is coupled to the encoding laser12by a mechanism30(such as a gimbal, servo, rail, rack-and-pinion, etc.) can be activated to move the encoding laser12to illuminate another one of the reflectors in the array, with each reflector (or groups of reflectors when simultaneously illuminated) establishing its own unique code. If desired, reflectors that are not to be illuminated for a particular location of the encoding laser12can be masked by, for example, a movable physical mask (not shown for clarity) with a single opening placed over the reflector sought to be illuminated and with a mask substrate that blocks light from other reflectors. A similar movable mask32can also be formed with an opening34and positioned over the region24during encoding to mask other nearby regions, reducing cross-talk. A polarization filter35may be disposed in the opening34if desired. Motors and mechanisms similar to those used to move the laser can be used to move the mask(s). Various other mechanisms can be utilized for masking the exposure for the areas outside of region24, including but not limited to a LCD polarizing screen and other forms of dynamic light blocking schemes.

Note that the polarization filters herein may be altered spatially for the hologram recording to reduce cross-talk with neighboring encoding areas on the holographic film. The polarization can be dynamic by using an electronically controlled spatial light modulator in addition to or in lieu of the polarizer14.

Prior to further explanation of present techniques, reference is directed toFIGS. 2-5, which illustrate various non-limiting examples of robust unique codes that can be established by each reflector in the array and encoded in its own respective region of the film26.FIG. 2shows a single “splotch”200with a unique configuration. Each reflector in the array16may encode its own respective splotch.FIG. 3shows a series of unique linear codes, e.g., first and second codes300,302that can be established by respective reflectors.FIG. 4illustrates that each unique code may be a quick response (QR) code400, whileFIG. 5shows that each unique code may be a bar code500. Combinations of the codes inFIGS. 2-5may be used. Thus, each reflector may be configured with its own unique code printed, etched, or otherwise formed on it.

InFIGS. 1 and 6, the location (also referred to herein as “position”) the encoding laser12is in with respect to the film26when irradiating the reflector “A” to encode the interference pattern in the region24is recorded, along with the unique code established by the reflector. The location can include the location in one, two, or three dimensions, and may also include the orientation (line-of-sight angle) of the encoding laser12with respect to the film26as discussed above (all examples of “pose” information), and the polarization used to encode the region24.

Once the region24has encoded the unique pattern from the reflector A, the encoding laser12is moved one increment to a next nearest location as shown inFIG. 6(with the mask32being correspondingly moved as shown) and activated to irradiate a second reflector “B” in the array16. The angle of irradiation/orientation of reflector B may be established, if desired, such that the resulting interference pattern from the direct beam600and deflected beam602is encoded in a second region604that is significantly distanced from the first region24. In the example shown, while the first and second regions encode respective unique codes that are associated with respective laser locations that are only a single increment of location recording apart, they may be physically separated from each other by regions of the film26that encode other codes associated with other laser locations. Also, successive reflectors in the array16may be irradiated with respective different polarizations. In this way, when the film26subsequently is used to determine the location of an illuminating laser with respect to a detector as described more fully below, discrimination of the precise location of the laser is made more robust by reducing the possibility of cross-talk. The reflector B is illuminated and its code recorded along with the location of the laser during illumination. It may now be appreciated that for each of multiple laser locations with respect to the film, an individual (or individual group of) reflectors in the array is illuminated to encode its unique code on a region of the film, the laser is moved, another reflector illuminated to encode another unique code in a different region of the film, and so on, with each laser location (pose information) being recorded if desired against the code of the reflector that was used for that laser location.

FIG. 7shows an example data structure that may be recorded on, e.g., disk-based or solid state memory in which laser locations in a first column700are correlated with respective robust codes in a column702, for purposes to be shortly disclosed.

WhileFIGS. 1 and 6show a transmissive system,FIG. 8shows that a laser800may be used to sequentially irradiate each of a series of reflectors802in a reflective arrangement to encode the respective codes onto the film26. In the arrangement ofFIG. 8, the laser is on the opposite side of the film from the reflectors. Note that the sensor inFIG. 9described below may be positioned on either side of the holographic film regardless of whether the hologram was encoded using transmissive or reflective principles.

It may now be appreciated that once the film26has been encoded as described above, when another laser (referred to herein as an “indicator” laser) subsequently illuminates the film, the indicator laser will illuminate the region of film that was encoded by the encoding laser12when the encoding laser12was in the same relative location to the film26as the subsequent indicator laser is in.FIG. 9illustrates an indicator laser900(with adjustable polarizer not shown) illuminating one of plural encoded regions902on the film26, with each region902encoding a unique robust code that is correlated to a respective location of the laser as described above. A sensor904such as but not limited to a charge-coupled device (CCD), complementary metal-oxide semiconductor (CMOS) detector, or photodiode array detector senses laser light emitted from the film26(and, hence, the unique code of the region902that is illuminated) and sends a signal representative thereof to one or more decoders such as one or more processors906. The processor906can execute image recognition to determine which unique code is received, and access the data structure shown inFIG. 7to correlate the code to a location of the indicator laser900with respect to the film26. The processor906may execute a software-based computer game908and output demanded images from the game908onto a display910, with game execution (and, hence, the demanded images) using, if desired, the laser location to alter the game images. This is amplified on further below.

The indicator laser900may be an infrared (IR) laser, although other wavelengths including visible and ultraviolet are contemplated. In some embodiments the wavelength of the light emitted by the indicator laser900may be greater than 1,000 nanometers, e.g., 1,440 nm to ensure that a game player does not see the laser light. The laser may be pulsed using a pulse repetition rate (PRR) that uniquely identifies the laser from other nearby indicator lasers. The laser may be modulated at a very high carrier frequency, e.g., in excess of thirty kilohertz, more preferably in excess of fifty kilohertz, and more preferably still at least one megahertz to be robust to noise from sunlight.

If desired, light from the indicator laser900can be polarized and changed over time using a polarizer920to improve signal to noise ratio of encoding. In this way, a plurality of signals can be decoded for each temporal polarization and the encoding with the highest signal to noise ratio may be chosen.

FIG. 10shows the logic of the encoding technique described above whileFIG. 11shows the logic of the subsequent location determination technique, with some or all of the logic steps being controlled by any of the processors described herein. Commencing at block1000, the encoding laser12is moved to a first location relative to the film26and activated to illuminate a first reflector A at block1002. The code is captured or encoded at block1004on the holographic film26in a first region A of the film (region24inFIGS. 1 and 6).

Proceeding to block1006the encoding laser12is moved to the next location relative to the film12, and if desired its polarization is changed at block1008for reasons explained above. A second reflector B is illuminated by the laser at block1010and its code captured (encoded) in the film26at block1012. The described process of moving the encoding laser, changing polarization if desired, and successively illuminating reflectors continues at block1014for subsequent locations3, . . . N to encode subsequent respective unique reflector codes C, . . . N onto the film26, with each code being recorded and correlated to the respective location information of the laser12at block1016.

Recalling the subsequent location determination system ofFIG. 9and turning now to relatedFIG. 11, at block1100, if desired the indicator laser900is calibrated to a location relative to the film26to approximate that of the encoding laser12in the earlier encoding process. This may be done, e.g., by instructing a user or installer to mount the indicator laser at a certain point or location, e.g., on the top middle of a display on which a computer game is to be displayed. This reference location may be supplied to a computer game so that subsequent locations of game objects as described below, for example, can be known relative to the reference location. That is, a game designer can assume, for instance, that any locations dynamically received during game play can be assumed to be referenced to a particular physical location on the game display, as but one example.

Proceeding to block1102, the film26is illuminated with the indicator laser900. The sensor904senses the resultant unique robust code pattern of light emitted from the film and its signal representative thereof is received at block1104. Image recognition is applied to the signal to recognize the code at block1106. In one example, the recognized code can be used at block1108as entering argument to, e.g., the data structure ofFIG. 7to return the corresponding location of the laser900with respect to the film. In another implementation, the correlation from coded region to pose information can be undertaken algorithmically/mathematically without a data storage medium holding a database of values. For example, a binary encoding scheme could directly store the binary representation of the pose information values, e.g., if X Code blocks:1101=X=13. In an example implementation, a circuit such as but not limited to a simple ASIC attached to the sensor904decodes the binary encoding of a recognized code efficiently and outputs it out as an analog (voltage) or digital value (SPI or other interface) for the pose value. No additional host processing is required.

In any case, however derived from the code, the pose information including location of the laser may be output at block1110to an AR or VR computer game console for reasons to be shortly illuminated.

More specifically and turning toFIG. 12for a first example arrangement for applying the system ofFIGS. 9 and 11, an architecture is shown in which a fixed illuminator1200such as the indicator laser900can illuminate one or more assemblies1202that are movable. Each assembly1202may include a film such as the holographic film26and a sensor such as the sensor904. Because the laser locations correlated to the robust codes on the film are relative locations between the laser and film, the codes that are illuminated when an assembly1202moves relative the fixed illuminator1200indicate the respective relative locations of the assemblies1202with respect to the fixed illuminator1200. Thus, as the assembly1202moves from location1inFIG. 12to location2, the regions of the film in the assembly that are illuminated change from a first region to a second region, meaning that the sensor in the assembly senses one robust code at location1, which is correlated to the first location, and another robust code at location2, which is correlated to a second location.

Note that plural fixed illuminators1200may be used in a system, each using a unique PRR as indicated above. If desired, each fixed illuminator may be associated with a respective fixed sensor904with film26assembly, and each illuminator with its sensor and film, in a fixed assembly, can be aware of other fixed illuminator assemblies. This allows multiple illuminators to self-calibrate, enabling a single tracking space.

Alternatively and again because the recorded locations inFIG. 7are relative between the film and laser, if desired an architecture as inFIG. 13may be used, in which an assembly1300with the film26and sensor904is fixed and an illuminator such as the indicator laser900can move from a first location1302to a second location1304, with the locations1302,1304being derived from the codes on the illuminated film in the fixed assembly1300.

Assuming the architecture ofFIG. 12, the movable film/sensor assembly1202may be implemented by a VR or AR headset such as the one shown inFIG. 16and described further below. A single headset may include multiple assemblies1202that can be illuminated by one fixed indicator laser900or by plural respective indicator lasers. Since the arrangement of plural film/sensor assemblies on a headset is known, their relative locations with respect to each other also are known.

Or, the movable film/sensor assembly1202may be implemented by a game controller such as the controller1400shown inFIG. 14. Yet again, the movable film/sensor assembly1202may be implemented by an eyeglasses-type frame1500(FIG. 15). A laser1502may be mounted in the frame and a light pipe1504may be used to direct laser light onto glasses-type displays1506.

In any case, it may now be appreciated that the locations of objects such as but not limited to the movable game-related objects described herein can be ascertained with respect to a reference point that can be tied to a computer game. Each movable film/sensor assembly1202can determine its location as described above and wirelessly report the location to the game processor. Or, the assembly can simply send a signal representing the unique code being illuminated to the game processor for derivation of the location by the game processor. Regardless, the game processor may then know, for example, the location of a VR/AR headset relative to the display on which the game is presented, and/or the location of the game controller1400, etc. and tailor VR/AR presentation accordingly.

In some implementations, the encoded regions of the film26(e.g., the regions24,604) can be exposed to the encoding laser light multiple times to be reused. For example, the region24can be exposed for laser position X=0, Y=0 and again for laser position X=0, Y=1. This will allow fewer code blocks to be used for encoding. An example can use 2's complement unsigned binary encoding, where 2 code blocks=3 codes, 3 code blocks=7 codes, 4 code blocks=15 codes, etc.

As mentioned previously, temporally changing polarization of the encoding laser12(and subsequent decoding by the indicator laser900) can be used to improve code block robustness. During the encoding phase, adjoining code blocks on the holographic film26may be recorded from differently polarized laser light to reduce cross-talk of laser light into adjoining code blocks. During the sensing phase, the laser light from the indicator laser900can be temporally polarized with differing polarizations. The successive polarizations over a short duration and subsequent lit code blocks will facilitate the code-to-position determination as one polarization on a specific code block (that was recorded with that polarization) will have a significantly higher signal-to-noise ratio (SNR) than other polarizations. This technique allows for improved filtering and detection of the correct position encoding code sequence.

Now referring toFIG. 16, an example system1600is shown, which may include one or more of the example devices mentioned below in accordance with present principles. The first of the example devices included in the system1610is a consumer electronics (CE) device such as an audio video device (AVD)1612such as but not limited to an Internet-enabled TV with a TV tuner (equivalently, set top box controlling a TV). However, the AVD1612alternatively may be an appliance or household item, e.g. computerized Internet enabled refrigerator, washer, or dryer. The AVD1612alternatively may also be a computerized Internet enabled (“smart”) telephone, a tablet computer, a notebook computer, a wearable computerized device such as e.g. computerized Internet-enabled watch, a computerized Internet-enabled bracelet, other computerized Internet-enabled devices, a computerized Internet-enabled music player, computerized Internet-enabled head phones, a computerized Internet-enabled implantable device such as an implantable skin device, etc. Regardless, it is to be understood that the AVD1612is configured to undertake present principles (e.g. communicate with other CE devices to undertake present principles, execute the logic described herein, and perform any other functions and/or operations described herein).

Accordingly, to undertake such principles the AVD1612can be established by some or all of the components shown inFIG. 16. For example, the AVD1612can include one or more displays1614that may be implemented by a high definition or ultra-high definition “4K” or higher flat screen and that may be touch-enabled for receiving user input signals via touches on the display. The AVD1612may include one or more speakers1616for outputting audio in accordance with present principles, and at least one additional input device1618such as e.g. an audio receiver/microphone for e.g. entering audible commands to the AVD1612to control the AVD1612. The example AVD1612may also include one or more network interfaces1620for communication over at least one network1622such as the Internet, an WAN, an LAN, etc. under control of one or more processors1624including. A graphics processor1624A may also be included. Thus, the interface1620may be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as but not limited to a mesh network transceiver. It is to be understood that the processor1624controls the AVD1612to undertake present principles, including the other elements of the AVD1612described herein such as e.g. controlling the display1614to present images thereon and receiving input therefrom. Furthermore, note the network interface1620may be, e.g., a wired or wireless modem or router, or other appropriate interface such as, e.g., a wireless telephony transceiver, or Wi-Fi transceiver as mentioned above, etc.

In addition to the foregoing, the AVD1612may also include one or more input ports1626such as, e.g., a high definition multimedia interface (HDMI) port or a USB port to physically connect (e.g. using a wired connection) to another CE device and/or a headphone port to connect headphones to the AVD1612for presentation of audio from the AVD1612to a user through the headphones. For example, the input port1626may be connected via wire or wirelessly to a cable or satellite source1626aof audio video content. Thus, the source1626amay be, e.g., a separate or integrated set top box, or a satellite receiver. Or, the source1626amay be a game console or disk player containing content that might be regarded by a user as a favorite for channel assignation purposes described further below. The source1626awhen implemented as a game console may include some or all of the components described below in relation to the CE device1644.

The AVD1612may further include one or more computer memories1628such as disk-based or solid state storage that are not transitory signals, in some cases embodied in the chassis of the AVD as standalone devices or as a personal video recording device (PVR) or video disk player either internal or external to the chassis of the AVD for playing back AV programs or as removable memory media. Also in some embodiments, the AVD1612can include a position or location receiver such as but not limited to a cellphone receiver, GPS receiver and/or altimeter1630that is configured to e.g. receive geographic position information from at least one satellite or cellphone tower and provide the information to the processor1624and/or determine an altitude at which the AVD1612is disposed in conjunction with the processor1624. However, it is to be understood that that another suitable position receiver other than a cellphone receiver, GPS receiver and/or altimeter may be used in accordance with present principles to e.g. determine the location of the AVD1612in e.g. all three dimensions.

Continuing the description of the AVD1612, in some embodiments the AVD1612may include one or more cameras2632that may be, e.g., a thermal imaging camera, a digital camera such as a webcam, and/or a camera integrated into the AVD1612and controllable by the processor1624to gather pictures/images and/or video in accordance with present principles. Also included on the AVD1612may be a Bluetooth transceiver1634and other Near Field Communication (NFC) element1636for communication with other devices using Bluetooth and/or NFC technology, respectively. An example NFC element can be a radio frequency identification (RFID) element.

Further still, the AVD1612may include one or more auxiliary sensors1637(e.g., a motion sensor such as an accelerometer, gyroscope, cyclometer, or a magnetic sensor, an infrared (IR) sensor, an optical sensor, a speed and/or cadence sensor, a gesture sensor (e.g. for sensing gesture command), etc.) providing input to the processor1624. The AVD1612may include an over-the-air TV broadcast port1638for receiving OTA TV broadcasts providing input to the processor1624. In addition to the foregoing, it is noted that the AVD1612may also include an infrared (IR) transmitter and/or IR receiver and/or IR transceiver1642such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering the AVD1612.

Still referring toFIG. 16, in addition to the AVD1612, the system1600may include one or more other CE device types. In one example, a first CE device1644may be used to send computer game audio and video to the AVD1612via commands sent directly to the AVD1612and/or through the below-described server while a second CE device1646may include similar components as the first CE device1644. In the example shown, the second CE device1646may be configured as a VR headset worn by a player1647as shown. In the example shown, only two CE devices1644,1646are shown, it being understood that fewer or greater devices may be used. For example, principles below discuss multiple players1647with respective headsets communicating with each other during play of a computer game sourced by a game console to one or more AVD1612, as an example of a multiuser voice chat system. Note that each laser/illuminator assembly and each sensor/film assembly may incorporate one or more components of the CE device1644such as appropriate processors, computer storage, and communication

In the example shown, to illustrate present principles all three devices1612,1644,1646are assumed to be members of an entertainment network in, e.g., a home, or at least to be present in proximity to each other in a location such as a house. However, present principles are not limited to a particular location, illustrated by dashed lines1648, unless explicitly claimed otherwise. Any or all of the devices inFIG. 16can implement any one or more of the lasers, films, and sensors described previously.

The example non-limiting first CE device1644may be established by any one of the above-mentioned devices, for example, a portable wireless laptop computer or notebook computer or game controller (also referred to as “console”), and accordingly may have one or more of the components described below. The first CE device1644may be a remote control (RC) for, e.g., issuing AV play and pause commands to the AVD1612, or it may be a more sophisticated device such as a tablet computer, a game controller communicating via wired or wireless link with the AVD1612, a personal computer, a wireless telephone, etc.

Accordingly, the first CE device1644may include one or more displays1650that may be touch-enabled for receiving user input signals via touches on the display. The first CE device1644may include one or more speakers1652for outputting audio in accordance with present principles, and at least one additional input device1654such as e.g. an audio receiver/microphone for e.g. entering audible commands to the first CE device1644to control the device1644. The example first CE device1644may also include one or more network interfaces1656for communication over the network1622under control of one or more CE device processors1658. A graphics processor1658A may also be included. Thus, the interface1656may be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface, including mesh network interfaces. It is to be understood that the processor1658controls the first CE device1644to undertake present principles, including the other elements of the first CE device1644described herein such as e.g. controlling the display1650to present images thereon and receiving input therefrom. Furthermore, note the network interface1656may be, e.g., a wired or wireless modem or router, or other appropriate interface such as, e.g., a wireless telephony transceiver, or Wi-Fi transceiver as mentioned above, etc.

In addition to the foregoing, the first CE device1644may also include one or more input ports1660such as, e.g., a HDMI port or a USB port to physically connect (e.g. using a wired connection) to another CE device and/or a headphone port to connect headphones to the first CE device1644for presentation of audio from the first CE device1644to a user through the headphones. The first CE device1644may further include one or more tangible computer readable storage medium1662such as disk-based or solid state storage. Also in some embodiments, the first CE device1644can include a position or location receiver such as but not limited to a cellphone and/or GPS receiver and/or altimeter1664that is configured to e.g. receive geographic position information from at least one satellite and/or cell tower, using triangulation, and provide the information to the CE device processor1658and/or determine an altitude at which the first CE device1644is disposed in conjunction with the CE device processor1658. However, it is to be understood that that another suitable position receiver other than a cellphone and/or GPS receiver and/or altimeter may be used in accordance with present principles to e.g. determine the location of the first CE device1644in e.g. all three dimensions.

Continuing the description of the first CE device1644, in some embodiments the first CE device1644may include one or more cameras1666that may be, e.g., a thermal imaging camera, a digital camera such as a webcam, and/or a camera integrated into the first CE device1644and controllable by the CE device processor1658to gather pictures/images and/or video in accordance with present principles. Also included on the first CE device1644may be a Bluetooth transceiver1668and other Near Field Communication (NFC) element1670for communication with other devices using Bluetooth and/or NFC technology, respectively. An example NFC element can be a radio frequency identification (RFID) element.

Further still, the first CE device1644may include one or more auxiliary sensors1672(e.g., a motion sensor such as an accelerometer, gyroscope, cyclometer, or a magnetic sensor, an infrared (IR) sensor, an optical sensor, a speed and/or cadence sensor, a gesture sensor (e.g. for sensing gesture command), etc.) providing input to the CE device processor1658. The first CE device1644may include still other sensors such as e.g. one or more climate sensors1674(e.g. barometers, humidity sensors, wind sensors, light sensors, temperature sensors, etc.) and/or one or more biometric sensors1676providing input to the CE device processor1658. In addition to the foregoing, it is noted that in some embodiments the first CE device1644may also include an infrared (IR) transmitter and/or IR receiver and/or IR transceiver1678such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering the first CE device1644. The CE device1644may communicate with the AVD1612through any of the above-described communication modes and related components.

The second CE device1646may include some or all of the components shown for the CE device1644. Either one or both CE devices may be powered by one or more batteries.

Now in reference to the afore-mentioned at least one server1680, it includes at least one server processor1682, at least one tangible computer readable storage medium1684such as disk-based or solid state storage, and at least one network interface1686that, under control of the server processor1682, allows for communication with the other devices ofFIG. 16over the network1622, and indeed may facilitate communication between servers and client devices in accordance with present principles. Note that the network interface1686may be, e.g., a wired or wireless modem or router, Wi-Fi transceiver, or other appropriate interface such as, e.g., a wireless telephony transceiver.

Accordingly, in some embodiments the server1680may be an Internet server or an entire server “farm”, and may include and perform “cloud” functions such that the devices of the system1600may access a “cloud” environment via the server1680in example embodiments for, e.g., network gaming applications. Or, the server1680may be implemented by one or more game consoles or other computers in the same room as the other devices shown inFIG. 16or nearby.

The methods herein may be implemented as software instructions executed by a processor, suitably configured application specific integrated circuits (ASIC) or field programmable gate array (FPGA) modules, or any other convenient manner as would be appreciated by those skilled in those art. Where employed, the software instructions may be embodied in a non-transitory device such as a CD ROM or Flash drive. The software code instructions may alternatively be embodied in a transitory arrangement such as a radio or optical signal, or via a download over the internet.

It will be appreciated that whilst present principals have been described with reference to some example embodiments, these are not intended to be limiting, and that various alternative arrangements may be used to implement the subject matter claimed herein.