Target identification for sending content from a mobile device

A method for target identification for a connectable device is carried out at a mobile device. A sensor on a mobile device receives one or more infrared emissions from one or more infrared light sources, each light source being provided by a connectable device in an area. One or more processors in the mobile device analyze the one or more infrared emissions to determine a target connectable device at a target position in the area. The mobile device receives a blink pattern of infrared emissions from the target connectable device. One or more processors in the mobile device decode the blink pattern to determine a reference for the target connectable device to enable wireless connection by the mobile device to the target connectable device for data transfer.

BACKGROUND

This invention relates to the field of sending content to a target device from a mobile device. In particular, the invention relates to target identification when sending content to a target device from a mobile device.

Currently if a user of a mobile device wants to send content or messages from the mobile device to another device (for example, sharing visuals with a screen or music with a HiFi, etc.) they have to tell the device manually which consumer to target. This is usually achieved by picking an identifier (ID) of the intended target from a list of targets that the mobile device can currently connect to.

There are two problems with this. The first is that is that the user has to know the ID of their intended target and that is often not the case (for example, with conferencing facilities, etc.). The second is that the user has to pick from a list of connectable device ID's and, in some venues, this list may be large and so this search may take some time and is error prone.

It is known to display a target device's connection identifier as a Quick Response (QR) code on the device. This however has drawbacks. A user must be at a close range to the QR code and therefore the device in order to scan a QR code. Also, a QR code is an obtrusive unsightly label to have to display on a device. Finally, it is hard to change a QR code once it is printed so the device owner cannot set a target device ID of their choosing and is unable to change the code easily once it is displayed. These leads to problems of uniqueness.

Traditional high frequency (HF) infrared (IR) technology is used to send commands between devices, for example, from a television remote control to the television. It consists of an IR light-emitting diode (LED) on the remote that blinks at many thousands of times a second to communicate long bit patterns quickly (many bytes in a fraction of a second) to the television which receives them via a light dependent resistor (LDR).

This allows a high bit rate because the LDR is a simple component that changes its resistance when IR light hits the sensor. The resistance of an LDR can be sampled at many thousands of times a second (kHz) and therefore can receive signals from LEDs with high frequency modulation rates. This technique can also be used to transfer small amounts of data, for example old mobile phones used to use this technology to exchange contact details or “business cards”.

The problem with using this technology to communicate connection information for a device is that an LDR only determines whether or not light is hitting the sensor (i.e. it is either on or off). It provides no information of where the light source is in relation to the sensor or the device (for example, light can often bounce of walls and round corners and can still be received) and it is useless if there is more than one IR source in range since they will interfere with each other.

Therefore, there is a need in the art to address the aforementioned problems.

SUMMARY

In an embodiment of the present invention, a method and/or computer program product for target identification for a connectable device is carried out at a mobile device. A sensor on a mobile device receives one or more infrared emissions from one or more infrared light sources, each light source being provided by a connectable device in an area. One or more processors in the mobile device analyze the one or more infrared emissions to determine a target connectable device at a target position in the area. The mobile device receives a blink pattern of infrared emissions from the target connectable device. One or more processors in the mobile device decode the blink pattern to determine a reference for the target connectable device to enable wireless connection by the mobile device to the target connectable device for data transfer.

In an embodiment of the present invention, a system for target identification for a connectable device is provided at a mobile device. The system comprises: a sensor for receiving one or more infrared emissions from one or more infrared light sources, each light source being provided by a connectable device in an area; a light analyzer for analyzing the one or more infrared emissions to determine a target connectable device at a target position in the area; a sensor data receiver for receiving a blink pattern of infrared emissions from the target connectable device; and a pattern decoder for decoding the blink pattern to determine a reference for the target connectable device to enable wireless connection by the mobile device to the target connectable device for data transfer.

DETAILED DESCRIPTION

The described method and system enable an intended target device for wireless content sharing to be identified by a mobile device by pointing the mobile device at the intended target. The method by which this can be achieved is through using an infrared (IR) light emitter or source such as a light-emitting diode (LED) mounted on a prominent/visible location on a target device, such as a screen, which provides a blink pattern in the form of a sequence of on/off blinks of the light to indicate the target device's identity.

In a simple embodiment, the blink pattern may be an encoding of the target device's ID. In another embodiment, details of the blink pattern may be broadcast with the target device's ID. Either way the mobile device has a way of matching blink patterns to connectable target device IDs.

When an application on the mobile device wants to connect to a target device it prompts the user to point the mobile device's camera at the intended target. Since digital cameras can see IR light (provided it does not have an IR filter), it can see the blinking LED on the target device.

A simple threshold filter may be used to take the raw data feed from the camera and remove any non-IR data and then make any pixel below a certain intensity black and above it white. This allows the LED to be clearly identified.

Once an LED has been identified in the view, the application may use the blink pattern to determine which target device the user is intending to connect with and can proceed to connect to it in the usual way.

High frequency (HF) infrared (IR) transmission as known from traditional IR ports may be used to broadcast a connectable device ID. To be able to determine the location of an IR source in a scene and be able to identify and differentiate multiple sources, a sensor is required that can give a 2D image of a scene so x,y coordinates for each source can be obtained. The more complex the data that a sensor provides the longer it takes to sample and so the sample rate decreases. A charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensor found in a typical mobile device only samples at around 30 Hz, therefore a lower frequency LED source is required than traditional HF IR.

Referring toFIG. 1, a diagrammatic representation of an example embodiment of the described system100is shown. A mobile device110is shown which requires connection to a target device120(shown in this example as a television).

The target device120may include an IR light emitter121, which emits a blink pattern of light. The IR light emitter121may be an IR LED. The IR LED may be of the form used with HF IR technology with the difference that the modulation of the blink pattern is at a lower frequency in order to be able to be received by the sensor112.

The mobile device110includes a connection system111for connection to a target device and a sensor112for sensing the blink pattern of light from the IR light emitter121. The sensor112may be an existing camera sensor of a mobile device110or it may be a separate sensor for this purpose. The sensor112may receive the blink pattern and also the position in the scene.

FIG. 1illustrates an area of capture113of the sensor112of the mobile device110. The mobile device110is pointed towards the target device120so that the area of capture113includes the position of the IR light emitter121.

The connection system111may receive the blink pattern emitted by the IR light emitter121and identify the target device120from the blink pattern enabling wireless connection113,123and data sharing between the mobile device110and the target device120.

Referring toFIG. 2, a flow diagram200shows an example embodiment of the described method as carried out at a mobile device. A user of the mobile device may activate201an application or system to connect to a target device in order to share data or content wirelessly.

The application provided on the mobile device may prompt202a user to point at a target device to which the mobile device is to be connected via a wireless connection.

A sensor of the mobile device may be activated203. The sensor may obtain a 2D image of the area pointed to by the mobile device. The received light at the sensor may be analyzed204to identify an IR light emitter of a target device. The analysis204may include applying a filter to the received light to remove any non-IR data from the raw data and to enhance the IR light received.

If there are multiple possible target devices with IR light emitters, further analysis203may distinguish the actual target device's position in the area of the image. For example, if multiple targets are in view then the light source closest to the centre of the image may be selected. This provides the system with some finesse when selecting targets in a tightly packed environment. Other details of further analysis are described below.

A blink pattern may be received205from the intended target device and decoded206to reference the target device.

The target device may then be connected to207by the mobile device using the reference for the target device for subsequent wireless communication of content to the target device.

Referring toFIG. 3, a block diagram300shows an example embodiment of the described system.

A connection system111may be provided for a mobile device110. The mobile device110may have a sensor112in the form of an existing camera sensor or a separate dedicated sensor. The sensor112may be capable of sensing IR light emitted by another device and for sensing a blink pattern of the IR light. The sensor112in the form of a dedicated sensor may have a built in filter to accurately isolate an IR light emission.

The connection system111may include a user interface301for providing alerts and instructions to a user and for allowing a user to input settings. A sensor activator302may be provided in the connection system111to activate the sensor112in order to attempt to identify and connect wirelessly to a target device.

A sensor data receiver303of the connection system111may receive data from the sensor112relating to light emissions it has picked up. A light analyzer304may be provided to analyze the light emissions picked up by the sensor112. The light analyzer304may include a filter305for enhancing the IR light and a light selector306for selecting a light emitter from multiple sensed emitters.

The connection system111may also include a pattern decoder307for analyzing and decoding the blink pattern contained in the received IR light data and associating the blink pattern with a reference ID for the target device. Target reference IDs and the associated blink patterns may be stored in a storage module308.

The connection system111may also include a device connector309for connecting wirelessly with a target device using a target reference ID.

Referring toFIG. 4, an exemplary system for implementing aspects of the invention includes a data processing system400suitable for storing and/or executing program code including at least one processor401coupled directly or indirectly to memory elements through a bus system403. The memory elements may include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.

The memory elements may include system memory402in the form of read only memory (ROM)404and random access memory (RAM)405. A basic input/output system (BIOS)406may be stored in ROM404. System software407may be stored in RAM405including operating system software408. Software applications410may also be stored in RAM405.

The system400may also include a primary storage means411such as a magnetic hard disk drive and secondary storage means412such as a magnetic disc drive and an optical disc drive. The drives and their associated computer-readable media provide non-volatile storage of computer-executable instructions, data structures, program modules and other data for the system400. Software applications may be stored on the primary and secondary storage means411,412as well as the system memory402.

The computing system400may operate in a networked environment using logical connections to one or more remote computers via a network adapter416.

Input/output devices413may be coupled to the system either directly or through intervening I/O controllers. A user may enter commands and information into the system400through input devices such as a keyboard, pointing device, or other input devices (for example, microphone, joy stick, game pad, satellite dish, scanner, or the like). Output devices may include speakers, printers, etc. A display device414is also connected to system bus403via an interface, such as video adapter415.

In a confined space (for example, a living room), there may be many devices (TV, HiFi, etc.) in close proximity. A sensor is therefore required that can see IR sources with high directional finesse which is provided by a digital camera using a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensor, commonly found on most mobile devices. A digital camera's field of view is much smaller than a traditional IR port, and the x,y coordinates of an IR source can be determined in the image of a scene. This means that if multiple IR sources are visible, in it is possible to determine which one is closest to the centre of the image and therefore the intended target.

In the described method and system, the IR source is blinking, therefore a number of frames may have to be sampled and aggregated to make sure that all the IR sources in the scene have been seen. An IR source is considered to be the same as one in the previous frame(s) if it is within a given radius of its last known position. The radius may grow with the amount of time that has passed since the IR source was last seen.

This is illustrated inFIG. 5, which shows a schematic diagram500of four frames510,520,530,540captured by a mobile device's sensor.

In frame1510, a first IR source511is captured. In frame2520, a second IR source522is captured but the first IR source511is not captured. A circle523is recorded with a radius calculated from the position of the first IR source511in frame1510. In frame3530, the second IR source522is also captured but the first IR source511is still not captured. A larger circle533is recorded with a greater radius calculated from the position of the first IR source511in frame1510.

In frame4540, the second IR source522is captured as well as another IR source which is determined to be the first IR source511. The IR source is determined to be the first IR source511as it is within a yet greater circle543with an increased radius from the position of the first IR source511in frame1510.

A camera capturing video will typically sample at about 30 Hz. Therefore, if the IR source of a target device blinks faster than 30 Hz, some of its transmission may be missed. An optimum rate of transmission of 15-20 Hz may be provided to avoid any errors from being out of synchronization with the camera's sampling. There may also be some buffer bits in the message so that the start of the message can be identified. This reduces the bit rate for data transfer from tens of thousands of bits per second with traditional high frequency IR, to just tens of bits per second. This means that if a connectable device with the ID: “Tonys_TV” were to broadcast its ID in ASCII via IR at 20 Hz to a camera it would take more than 3 seconds to receive the full ID ((8 bits per char×8 chars)/20 Hz) which is longer than a user would wish to wait.

The described method enables a target device to broadcast a low frequency IR blink pattern that a mobile device can receive through a camera and use to uniquely identify the connectable target device, so that it knows precisely which, out of many connectable devices, it is pointing at. The IR blink pattern should take no more than a second for the camera to receive so that the information can be transferred in a timely fashion.

The target device may use a scheme for delivering a blink pattern with enough possible variations, such that each device can have a unique pattern with regards to the set of connectable devices in the area. The scheme should use a minimal number of bits so that to observe an entire blink pattern takes less than a second. Also it should not allow too many consecutive zero bits because this would mean that the IR source would be off for long periods and make it harder to identify/track in the camera's view.

An example scheme, is as follows:

12 Bit loop

This scheme completes in under a second at 15 Hz, has a way of identifying the start and never has more than 5 zero bits in a row. If the IR source cannot be seen for too many frames/bits, it becomes harder to track. It has 36 unique codes which should be sufficient since it is unlikely that there would be 36 connectable devices in one small area.

The blink pattern is correlated to a target device ID. In one embodiment, the protocol that the target devices use to allow things to connect to them (e.g. Bluetooth) is required to be modified. These protocols typically have a mechanism by which a connectable device can broadcast details about itself (name, ID, MAC address, etc.), which a mobile device can then use to connect to it. By modifying the protocol it would be possible to add information about what blink pattern the connectable device is currently using thus allowing the mobile device to correlate an observed blink pattern with a connectable device in the area and then connect to it.

To ensure no two target devices use the same blink pattern, a command may be added to the connection protocol to allow a mobile device to request that a connectable device change its blink pattern. This would mean that, if two target devices in a scanning range were both using the same blink pattern, as the user activates “point to connect” mode on mobile device, the device could quickly scan the connection protocol, spot they were both using the same blink pattern, and tell one of them to change to another pattern before it used its camera to scan for IR sources. It would be useful if the interface for this not only allows mobile devices to request changes blink pattern but to also provide information about blink patterns that it knows are in use.

Mobile device A is in range of connectable devices 1, 2 and 3. A uses the connection protocol's ID packet to see that the connectable devices are using the following blink patterns:

1 using pattern X

2 using pattern Y

3 using pattern X

“A” may then tell 3 to change its pattern to something other than X or Y.

3 may then change to another blink pattern, (not X or Y) e.g. Z.

The blink patterns in this system are now unique and A can tell the user to “point to connect”.

A possible variation of this technology would be for the mobile devices to tell ALL connectable devices in range, what blink pattern to use. A mobile device would only do this as it enters “point to connect” mode. Using this method, connectable devices would only have to blink their IR sources when requested by a mobile device that wishes to use it to identify them reducing the amount of “IR pollution” from the devices.

Since the LED is IR and outside the human visual range, it does not spoil the aesthetics of the target device. The LED would blink in a pattern unique to the target device.

Higher frequency sensors for digital cameras with very high frame rates may be provided in a mobile device in which case a higher frequency IR light emitter may be used to communicate larger volumes of data in the required time whilst still determining the source's position in the scene.

According to a first aspect of the present invention there is provided a method for target identification for a connectable device, wherein the method is carried out at a mobile device and the method comprising: receiving via a sensor one or more infrared emissions from one or more infrared light sources, each light source being provided by a connectable device in an area; analyzing the one or more infrared emissions to determine a target connectable device at a target position in the area; receiving a blink pattern of infrared emissions from the target connectable device; decoding the blink pattern to determine a reference for the target connectable device to enable wireless connection by the mobile device to the target connectable device for data transfer.

The step of analyzing the one or more infrared emissions may include filtering a received image to filter the received infrared emissions in the image. The step of analyzing the one or more infrared emissions may also include selecting an infrared emission closest to a centre of a received image.

The method may include activating a connection process and prompting a user to point the mobile device at the target connectable device.

The step of receiving via a sensor may receive an image captured by a camera of the mobile device and may receive a sampling of multiple frames at regular intervals.

An infrared source may be determined to be the same source if it appears in multiple frames within a circle of increasing radius in each frame.

The blink pattern may be transmitted at a frequency less than or equal to the sample rate of the sensor. The blink pattern may also include some buffer bits to identify the start of the blink pattern.

The blink pattern may be an encoding of the connectable device's identifier or a pattern referenced to the connectable device.

A group of connectable devices in a given location may have a set of unique blink patterns.

The method may include receiving a broadcast protocol of a connectable device including a blink pattern.

The method may further include instructing a connectable device to change its blink pattern.

According to a second aspect of the present invention there is provided a system for target identification for a connectable device, wherein the system is provided at a mobile device, comprising: a sensor for receiving one or more infrared emissions from one or more infrared light sources, each light source being provided by a connectable device in an area; a light analyzer for analyzing the one or more infrared emissions to determine a target connectable device at a target position in the area; a sensor data receiver for receiving a blink pattern of infrared emissions from the target connectable device; a pattern decoder for decoding the blink pattern to determine a reference for the target connectable device to enable wireless connection by the mobile device to the target connectable device for data transfer.

The system may include at least one target device having an infra-red light source for transmitting a blink pattern which references the target device for wireless connection by the mobile device.

The light analyzer for analyzing the one or more infrared emissions may include a filter for filtering a received image to filter the received infrared emissions in the image. The light analyzer for analyzing the one or more infrared emissions may also include a light selector for selecting an infrared emission closest to a centre of a received image.

The system may include a user interface for activating a connection process and prompting a user to point the mobile device at the target connectable device.

The sensor may be a camera of the mobile device. The sensor data receiver may receive a sampling of multiple frames at regular intervals. The sensor data receiver may determine an infrared source to be the same source if it appears in multiple frames within a circle of increasing radius in each frame.

According to a third aspect of the present invention there is provided a computer program product for a mobile device for target identification for a connectable device, the computer program product comprising: a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method according to the first aspect of the present invention.

According to a fourth aspect of the present invention there is provided a computer program stored on a computer readable medium and loadable into the internal memory of a digital computer, comprising software code portions, when said program is run on a computer, for performing the method of the first aspect of the present invention.

The described aspects of the invention provide the advantage of identifying the intended target of data shared from a mobile device by pointing the mobile device at the target device. The described invention also provides the advantage of enabling flexibility of the connection identifier.