Apparatus and methods of displaying messages for electronic devices

An apparatus of transmitting messages from an electronic device and display it on a mobile device, and a method of doing the same are disclosed. An exemplifying electronic device has an encoder to encode the data and a drive to modulate the signals into temporal and spatial light patterns according to the transmitted message. The light signals are then emitted from a group of LEDs on the electronic device. A video camera on a mobile device captures the light signals in the form of temporal and spatial patterns. A software application decodes the message and displays the message on the mobile device screen.

TECHNICAL FIELD

This present invention relates in general to the field of information transmission to mobile devices, and more specifically, to the technique of driving indicator lights to deliver messages from an electronic device to a mobile device.

DISCUSSION OF RELATED ART

Many devices need to indicate its status or properties to the user. The methods to present a device's static properties have been developed, such as linear barcodes and 2-dimensional barcodes (U.S. Pat. No. 4,939,354, U.S. Pat. No. 5,053,609, U.S. Pat. No. 5,124,536, U.S. Pat. No. 5,726,435). Users can use a mobile device software application to capture the barcode pattern through its video camera and decode the message.

However, besides static properties, many electronic devices need to dynamically show messages about its operating condition and alert users of error conditions. Due to limited device front panel space or restricted costs, a message is often delivered using one or more indicator lights, for example, lighting or blinking several small light bulbs or Light Emitting Diodes (LEDs). The simple indicator lights cannot provide detailed information to the user. A more sophisticated optical data communication method may be used to deliver detailed messages to the user.

Methods for visible light communication have been used for transmitting messages (US patent 2008/0131140A1, US patent 2009/0269073A1, US patent 2009/0310971A1, US patent 2010/0196018A1, US patent 2010/0284697A1, US patent 2010/0327780A1, US patent 2011/0243570A1, US patent 2012/0091896A1, US patent 2012/0269520A1). A method to transfer the visible light pattern through optical cables is also described (US patent 2011/0052195A1). However, these methods require dedicated light sensors or optical cables to receive the data.

An electronic device or equipment usually needs to present messages to the user, either in normal operation or in fault condition. For example, electrical powered equipment needs to indicate whether its power supply is in good condition or in a failure mode. An automobile needs to report its mechanical and electrical condition to the driver. Due to limited front panel space on a device or restricted costs, a message is usually delivered by a few simple indicator lights, such as lighting or blinking of small light bulbs or Light Emitting Diodes (LEDs). However, such simple mechanisms may not provide clear and detailed information for the device. For example, a simple blinking light on a network router cannot provide users with detailed information about its network address, its software version or its network conductivity, which are essential to the user in setup and configuration processes. In particular, normal means of accessing device information over a network connection becomes non-functional during troubleshooting.

Adding an alphanumerical display panel to electronic devices is costly and sometime is prohibited by the limited front panel space. Most devices use LEDs or other light indicators to alert the user of working condition or errors. Some devices use a special electronic port to connect to a dedicated monitoring device to retrieve the device's information. For example, most modem motor vehicles use an electronic processor to monitor and diagnose the mechanical and electrical problems of the vehicle. However, to retrieve the Diagnostic Trouble Code (DTC) from the vehicle, one needs a special device to connect to the vehicle's On-Board Diagnostics (OBD) port. The process of getting the vehicle DTC is cumbersome and requires a trained mechanical specialist to perform. It is highly desirable if an ordinary motor vehicle driver can simply point the video camera of a mobile device to the LED lights flashing on the dashboard and read out the fault cause and recommended repair steps. The indicator lights serve both an alerting signal for quick visual assessment of the device condition and as the light transmitters to deliver detailed data to a mobile device for message retrieval and display.

Therefore, there is a need for an apparatus and thereof method to display the dynamical information with the current compatible mobile devices.

SUMMARY

Consistent with embodiments of the present invention, an apparatus for displaying messages from a data source is provided. In some embodiments, the apparatus comprises an electronic device configured to convert signals from the data source to an array of optical signals, the electronic device comprising a data collector module collecting data from the data source, an encoder module generating encoded signals according to the received data source signals, a drive module receiving the encoded signals, and a plurality of light emitting devices being powered and modulated by the drive module according the encoded signals; and a mobile device configured to process video imaging, comprising:

a camera receiving the emitted signals from the plurality of light emitting devices, a decoder decoding the video signals, and a display drive module sending the decoded signals to a display screen.

In some embodiment, the drive module drives the plurality of light emitting devices to produce a temporal pulse sequence of signals. In another embodiment, the drive module drives the plurality of light emitting devices and produces spatial patterns of signals comprising a single signal, a linear array signal, or two dimensional array signals.

Consistent with some embodiments of the present invention, a method of displaying messages from a data source on a mobile device comprises: collecting data from the data source by a data collector in the electronic device; encoding the data to generate modulated control signals according to the collected data from the data source; driving a plurality of light emitting devices to emit optical signals with the modulated control signals, wherein the optical signals comprise a temporal sequence, a spatial array patterns, and a combination of temporal sequence and spatial patterns; providing a video imaging device on the mobile device; recording the optical signals on the video imaging device; decoding the optical signals with a decoder in the mobile device; converting the decoded data into a human readable message; and displaying the message on a display screen on the mobile device.

Another embodiment of the method comprises providing a wireless network controller configured to access internet-provided reference information and to convert the information to human readable messages on the display screen.

Another embodiment of the method comprises providing a clock generator configured to create clock reference signals, wherein the clock reference signals are transmitted in a separate stream from the encoded data from the data collector and are sent to a light emitter such as a LED from a clock light driver.

Another embodiment of the method further provides a clock generator configured to create clock reference signals, wherein the clock reference signals are intertwined with the encoded signals and displayed on the plurality of light emitting devices.

Consistent with another embodiment of the present invention, an apparatus for transmitting data from a data source to a video camera, comprises of an electronic device configured to convert data from the data source to an array of optical signals, the electronic device comprising: a data collector module collecting data from the data source, an encoder module generating encoded signals according to the received data from the data source, a drive module receiving the encoded signals, and a plurality of light emitting devices being powered and modulated by the drive module according the encoded signals, wherein the modulating rate is less than half of the video camera frame rate.

DETAILED DESCRIPTION

The present invention describes an apparatus and a method thereof to enhance the message delivery mechanism from indicator lights. The method can apply to an indicator light or a group of indicator lights. In the case of a group of indicator lights, the indicator lights form a linear array or a 2-dimensional matrix. The internal message of a device is coded in a sequence of control signals to modulate the current or voltage applied to the indicator lights. As a result, each indicator light temporally varies its light intensity or color. The group of the indicator lights also presents a spatial pattern. A video camera, which is commonly found in a mobile device or a personal computer, is used to capture the temporal and spatial light patterns emitted from the light indicators. A software application decodes the message and displays the resulting information on the mobile device's screen. The software application can also automatically query for relevant information from the external data sources such as a local database or a remote Internet-accessible database through the mobile wireless connectivity and augments the display with more comprehensive information.

One embodiment of the invention is illustrated inFIGS. 1 and 2. A group of light sources such as LEDs is organized in a 2-dimensional array11on front panel13of electronic device10. A LED drive circuit16depicted inFIG. 2modulates the current of each LED12in the LED array11, therefore changes the light intensity emitted from LED12. A mobile device20, such as a smart phone, an iPhone, an iPad, a Blueberry, or a personal computer, is used to capture the temporal and spatial light patterns through the embedded video camera21inFIG. 2. The captured video31is displayed onto the mobile device screen30for aiming and targeting purposes. A decoding process implemented in the mobile device20, described later, recovers the captured light patterns to a human readable message32and displays the message32on the mobile device screen30.

An alternative embodiment can use multicolor LEDs in the LED array11implementation. The LED color and light intensity can be combined to enhance the information transmission throughput.

FIG. 2exemplifies the process to encode messages and transmit light at the electronic device10in the example embodiment. The system data collector14monitors the device condition and records all the data required for display. The data are encoded to a binary sequence at the encoder module15. To minimize the average light density variation, DC-balanced encoding schemes, such as Manchester coding or 8b/10b coding, can be used. The encoded data are sent to the LED drive module16. The output ports of LED drive module16are connected to individual LED lights12of the LED array11at the front panel of the device10. The LED drive module16supplies electrical current to the LEDs12. The electrical current delivered to each LED is modulated at the LED drive module16by the coded data sequence from the encoder module15; therefore, light intensity emitted from LED12is modulated in time sequence, and meanwhile the entire LED array11presents a dynamic spatial pattern. A video camera21from the mobile device20is used to capture the modulated light emitted from the LED array11. The LED modulation frequency should be lower than half of the camera21capturing frame rate according to the Nyquist sampling theorem.

To ensure the reliability of capturing the encoded data from the LED lights, one or few LEDs in the LED array can be modulated by the electrical current at a fixed frequency supplied by the LED drive module as the reference clock.FIG. 3depicts an exemplary embodiment having one reference clock, displaying in a LED. The system data collector214in the electronic device210observes the device condition and records all the data required for display. The data are encoded to a binary sequence at the encoder module215. The encoded data are sent to the data LED drive module216. The output ports of data LED drive module216are connected to individual LED lights212of the LED array211except the reference clock LED219at the front panel213of the device210. A reference clock generator217creates a periodical reference clock wave. The reference clock wave modulates clock LED219light emitting through the clock LED drive218. The reference clock light wave emitted from LED219provides a timing reference to the receiving camera.

FIG. 4shows the message decoding process in a display device20(a mobile device, for example) The on-board video camera21captures the light pattern emitted from the LED array11on the electronic device10inFIG. 2. The captured video data are stored in the video frame buffer22and each video frame is forwarded to the decoder module23. The original message sent from the electronic device10is recovered at the decoder module23by decoding the temporal and spatial light patterns using the corresponding decoding protocol. The decoded message is sent to the display driver24and presented on the display screen25. Optionally, the decoded message from the decoder module23is sent in parallel to the wireless network controller26. Through the wireless network controller26relevant information is retrieved from Internet-accessible data sources. The enriched information is fed back to the display driver24and added to the display screen25to provide the user with richer reference material.

To simplify message transmission and lower building cost, a single LED112may be used to deliver data as shown inFIG. 5. In this embodiment the encoded sequence data from the electronic device110modulate the electrical current to the single LED112. The video camera of the mobile device120captures the light emitted from the LED112. The decoding process detects the temporal sequence data to recover the original message. The video image131of the LED and the recovered message132are superimposed on the mobile device screen130. Because only a single LED is used, it takes a longer time to transmit LED light sequence and video capture time is longer too.

Another exemplary application of the invention is transmitting traffic information from regular traffic lights. Using the light modulation frequency higher than the frequency range perceivable by human eyes, the driver will not notice interference from the information transmission. A video camera with the light data decoding capability can deliver traffic information to the driver.

FIG. 6shows an application example of the invention in traffic message delivery. In this example the traffic lights70are used for transmitting the traffic condition, road states and accident information to the driver in a motor vehicle60. Nowadays, most light bulbs in traffic lights are replaced by LEDs or other energy saving emitters, meanwhile light emission can be easily modulated by an electrical pulse sequence. The encoded message is converted to an electrical pulse sequence and is sent to the LED lights71to modulate the light transmission. If the electrical pulses rate exceeds the frequency range perceivable by human eyes, 30 Hz approximately, the driver of the vehicle would not feel any flickering of the modulated traffic lights. A receiving device40with a video camera (not shown) mounted above the windshield61captures the emitting light pulses and decodes the message. The video image of the traffic light51and the traffic information52are displayed on the receiving device screen50or read aloud to the user.

Video camera-equipped smart phones and mobile devices are very popular nowadays. Using a mobile device's video camera to capture and display messages from electronic devices not only lowers the manufacturing cost of electronic devices, it also provides the user a simpler and more intuitive way to recognize equipment's conditions and to catch error messages.

The foregoing description is intended to illustrate but not to limit the scope of the invention, which is defined by the scope of the appended claims. Other embodiments are within the scope of the disclosure. Numerous variations and modifications within the scope of the present disclosure and the embodiments described therein are possible.