Embodiments of a display device with a vision system are disclosed. In one example, a display device comprises a plurality of pixels for displaying light in a first polarization state, a plurality of photodetectors, where each photodetector in the plurality of photodetectors includes a polarization filter configured to transmit light in a second polarization state to the photodetector and reject light in other polarization states, a plurality of lenslets, where each lenslet in the plurality of lenslets focuses light in the second polarization state onto at least one photodetector in the plurality of photodetectors, a material surrounding the plurality of lenslets, where, for each lenslet in the plurality of lenslets, indices of refraction of the lenslet and the material are substantially equal for light in the first polarization state, and where indices of refraction of the lenslet and the material are different for light in the second polarization state.

BACKGROUND

Display devices, such as liquid crystal displays (LCD's) and organic light emitting diodes (OLED) displays, may be outfitted with image sensors or vision systems so that the display device may display and capture off-screen images. For example, if such a display device is coupled to a computing system, the captured images may permit a user to interact with the system, e.g., via user motions or user touch.

In one example approach, a camera may be disposed adjacent to an edge of a display device to capture off-screen images. However, in such an approach, as a user approaches the screen e.g., as a user's finger approaches the screen, the point of view of the camera may be too oblique to capture images of the finger as it moves close to the surface of the screen.

SUMMARY

Various embodiments are disclosed that relate to a display device which includes a vision system. In one example approach, a display device is disclosed comprising a plurality of pixels for displaying light in a first polarization state, a plurality of photodetectors, where each photodetector in the plurality of photodetectors includes a polarization filter configured to transmit light in a second polarization state to the photodetector and reject light in other polarization states, and a plurality of lenslets. Each lenslet focuses light in the second polarization state onto at least one of the photodetectors. Surrounding each lenslet is a material, with the indices of refraction of the lenslet and the material being substantially equal for light in the first polarization state, and different for light in the second polarization state.

DETAILED DESCRIPTION

FIG. 1shows example interaction100of a user102with a display device104. Display device may be a liquid crystal display (LCD), organic light emitting diodes (OLED) display, or any other suitable display device, for example. Display device104includes a screen108, which may include one or more image capture devices, such as photodetectors or cameras as described in more detail below. The image capture device(s) on screen108may be configured to capture off-screen images, e.g., fingers of user102as the user approaches, hovers over, or touches the screen.

In some examples, display device104may be included in a computing system106, e.g., as described in more detail below with respect toFIG. 4. In such a case, images captured by the image sensors on the screen of the display device may be processed by computing system106, e.g., to permit a user, such as user102, to interact with the system or to record and capture image and video data for processing.

As remarked above, in one example approach, a camera may be disposed adjacent to an edge, e.g., edge110, of display device to capture off-screen images. For example, such a camera may be a web camera, depth camera, or other suitable image capture device. However, in such an approach, as a user approaches the screen, e.g., as a user's finger approaches the screen, the field of view of the camera may be too oblique to capture images of the finger as it moves close to the surface of the screen.

In another example approach, photodetectors may be placed or positioned adjacent to each pixel on the screen of the display device. However, photodetectors may only detect the presence or absence of light thus may detect light emitted from the display device which may drown out incoming light. In still another example approach, lenses may be disposed in front of photodetectors on the screen of the display device. However, in this approach, the lenses may obstruct or distort light being emitted by the display device.

In view of the above, birefringent (double refracting) lenses may be employed together with photodetectors in a display device so that outbound light emitted by the display device remains substantially undistorted and inbound light is imaged by the display device even at close range.

For example,FIG. 2shows a cross-section200of an embodiment of a display device which incorporates birefringent lenses and photodetectors in a display device.

The display device shown inFIG. 2includes a plurality of display pixels for displaying or emitting light. The pixels may be of any suitable dimension, e.g., a diameter of a pixel may be approximately 1 millimeter in length. For example, pixels202,204, and206are shown inFIG. 2. In some examples, the plurality of pixels may emit substantially polarized light. For example, pixels202,204, and206may display light in a first polarization state.

The display device also includes a plurality of photodetectors for receiving incoming light. For example, photodetectors208and210are shown inFIG. 2.

The display device further includes a plurality of lenslets. For example, lenslet212and lenslet214are shown inFIG. 2. In some examples, each lenslet may be composed of a birefringent material. For example, the lenslets may be composed of a mixture of photopolymer and liquid crystal which has been aligned then cured with ultraviolet. As another example, the lenslets may be composed of reactive mesogen materials, for example. Each lenslet includes at least one photodetector in the focal plane of the lenslet.

A material216surrounds the plurality of lenslets and is adjacent to a surface218of the display device. Material216may be composed of any suitable material. In some examples, material216may be composed of a birefringent material, such as the example materials given above; whereas, in other examples material216may not be composed of a birefringent material. Further, in some examples, an index of refraction of material216may be lower than an index of refraction of the lenslets.

The lenslets surrounded by the material form a polarization splitting prism film which behaves as if transparent to outgoing light from the display and focuses incoming orthogonally polarized light onto the photodetectors. Thus, either the lenslets or the material, or both the lenslets and the material may be composed of a birefringent material.

In some examples, such as shown inFIG. 2, a single lenslet may span or cover one photodetector and one pixel. However, in other examples, such as shown inFIG. 3described below, a single lenslet may span a plurality of pixels and/or a plurality of photodetectors.

As remarked above, each lenslet is composed of a birefringent, or double refracting material. That is, each lenslet may be configured to refract light with a first polarization by a first amount and refract light with a second polarization by a second amount. Namely, a birefringent lenslet may have different indices of refraction for light of different polarizations.

Further, as remarked above, light emitted from the pixels may have a first polarization. Thus, by a suitable choice of material for material216surrounding the lenslets, it can be arranged so that the refractive indices of material216and the lenslets are substantially equal for light of the first polarization state of outgoing light. In this way, the light emitted from the pixels, e.g., as shown at220inFIG. 2, may pass through the display device substantially unchanged.

However, for light of a second polarization, different from the first polarization, the indices of refraction of the lenslet and the material may be different. In this way, the second polarization component of incoming light, e.g., as shown at222inFIG. 2, may be refracted and focused by a lenslet onto one or more photodetectors beneath the lenslet.

The first and second polarizations described above may be any suitable types of polarizations, e.g., they may be linearly polarized, circularly polarized, elliptically polarized, etc. However, the first and second polarizations may be substantially orthogonal. Further, in order to increase a focusing power of the lenslets, the difference between the indices of refraction of the lenslet and the material for the second polarization may be increased as much as possible.

Each photodetector may further include a polarization filter (or polarizer) configured to transmit light in the second polarization state to the photodetector and reject light in other polarization states. For example, as shown inFIG. 2photodetector208includes polarization filter224and photodetector210includes polarization filter226. In this way, the second polarization component of incoming light may be transmitted to the photodetector while light emitted from the pixels may be rejected.

However, depending on the size of the pixels and a thickness of the display device, e.g., a thickness of 1 centimeter may be desired, the focal length of the lenslets may be so small so as to lead to aperture diffraction which may distort light emitted from the pixels and limit resolution in the far field. Thus in some examples, larger lenslets may be employed which span a plurality of pixels and multiple photodetectors may be employed for each lenslet to increase image capture resolution, such as shown inFIG. 3described below.

Turning now toFIG. 3, a cross-section300of another embodiment of a display device in accordance with the disclosure is shown. As above, the display device includes a plurality of lenslets surrounded by a material316adjacent to a surface318of the display device.

In the example shown inFIG. 3, a single birefringent lenslet302spans or covers two pixels304and306. Further, two photodetectors308and310are used to capture the second polarization component of incoming light focused by lenslet302. As described above, each photodetector includes a polarization filter to transmit the second polarization component of incoming light, e.g. as shown at322inFIG. 3, to the photodetector while rejecting light of other polarizations. For example, photodetector308includes polarization filter330and photodetector310includes polarization filter332. Further, since the indices of refraction of lenslet302and material316are substantially equal for the first polarization state of outgoing light, shown at320inFIG. 3, the outgoing light is substantially unchanged as it leaves the display device.

FIG. 4schematically shows a nonlimiting computing system400which may include a display device, such as display device104shown inFIG. 1. Computing system400is shown in simplified form. It is to be understood that virtually any computer architecture may be used without departing from the scope of this disclosure. In different embodiments, computing system400may take the form of a mainframe computer, server computer, desktop computer, laptop computer, tablet computer, home entertainment computer, network computing device, mobile computing device, mobile communication device, gaming device, etc.

Computing system400includes a logic subsystem402and a data-holding subsystem404. Computing system400also includes a display subsystem406, and may additionally include other components not shown inFIG. 4. Computing system400may also optionally include user input devices such as keyboards, mice, game controllers, cameras, microphones, and/or touch screens, for example.

Logic subsystem402may include one or more physical devices configured to execute one or more instructions. For example, the logic subsystem may be configured to execute one or more instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more devices, or otherwise arrive at a desired result.

The logic subsystem may include one or more processors that are configured to execute software instructions. Additionally or alternatively, the logic subsystem may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic subsystem may be single core or multicore, and the programs executed thereon may be configured for parallel or distributed processing. The logic subsystem may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and/or configured for coordinated processing. One or more aspects of the logic subsystem may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

Data-holding subsystem404may include one or more physical, non-transitory, devices configured to hold data and/or instructions executable by the logic subsystem to implement the herein described methods and processes. When such methods and processes are implemented, the state of data-holding subsystem404may be transformed (e.g., to hold different data).

Data-holding subsystem404may include removable media and/or built-in devices. Data-holding subsystem404may include optical memory devices (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory devices (e.g., RAM, EPROM, EEPROM, etc.) and/or magnetic memory devices (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), among others. Data-holding subsystem404may include devices with one or more of the following characteristics: volatile, nonvolatile, dynamic, static, read/write, read-only, random access, sequential access, location addressable, file addressable, and content addressable. In some embodiments, logic subsystem402and data-holding subsystem404may be integrated into one or more common devices, such as an application specific integrated circuit or a system on a chip.

FIG. 4also shows an aspect of the data-holding subsystem in the form of removable computer-readable storage media408, which may be used to store and/or transfer data and/or instructions executable to implement various processes. Removable computer-readable storage media408may take the form of CDs, DVDs, HD-DVDs, Blu-Ray Discs, EEPROMs, and/or floppy disks, among others.

When included, display subsystem406may be used to present a visual representation of data held by data-holding subsystem404and capture image data as described herein. Display subsystem406may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic subsystem402and/or data-holding subsystem404in a shared enclosure, or such display devices may be peripheral display devices.