APPARATUS AND METHOD FOR STREAMING LIVE IMAGES, AUDIO AND META-DATA

A method for streaming and viewing a user's video and audio experiences includes removably mounting an image recording device in close proximity to a user's eyes such that the image recording device is operable to record the user's visual and audio experiences. A real time signal is created by the image recording device, including at least one of video footage, still images, or audio captured by the image recording device. The real time signal is streamed from the image recording device to a server using at least one communications network. The real time signal is transmitted from the server to a remote communication device for producing an output that is perceptible to a viewer so that said viewer can experience the user's environment and the user's relationship to the environment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to the drawings, the present invention will now be described in detail with reference to the disclosed embodiment.

The present invention relates to an apparatus and method for streaming live video footage, still images, audio, date, time and sensor data (wherein data, time, and sensor data are referred to as “metadata”) of a user's visual and audio experiences, as shown inFIGS. 1-3. In order to capture the video footage, still images, audio, and metadata, the present invention provides an image recording device10. The image recording device10includes at least one image sensor12, which can be a digital image sensor that is operable record still images or videos. The image sensor12can be a portion of a digital video camera or a still digital camera that is incorporated in the image recording device10. The image sensor12is connected to a mounting structure14, such as a pair of eyeglasses16, goggles, face mask, helmet, etc., that can be releasably mounted in close proximity to the eyes of a user (not shown).

By mounting the image recording device10close to the user's eyes, the user's visual and audio experiences can be captured by the image recording device10. Once the image recording device10begins to transmit and/or record video footage, still images, audio sounds, and/or meta-data, the transmission is converted into a data stream that may be sent as a real time electronic signal, or the transmission may be recorded by the image recording device10and sent as an electronic signal at a later time. If the transmission is sent immediately, the data stream may be streamed as a live or real time electronic signal to a first communication device. The first communication device can be a local communication device (i.e., located in the proximity of the image recording device10). As one example, the first communication device can be an integral part of the image recording device10. As another example, the first communication device can be a personal communication device, such as a smart phone, tablet, computer, etc. that receives the transmission from the image recording device and relays the transmission. In one example, the first communication device may include a multi-core processor that is incorporated in the image recording device10, and allows the processor to stream the real time signal directly to a web server using, for example, a cellular data modem.

The first communication device may receive and send the data stream to at least one web server using at least one communication network, such as the internet. The webserver can include or communicate with a website that allows viewers to access and view the data stream. Once the data stream is received by the website, the images, sound, and/or metadata provided by the electronic signal may be viewed and heard on the website by viewers accessing the website through a second communication device (not shown), such as a computer and/or a smart phone thereby allowing the viewer of the website to experience the user's environment and their relationship with the environment. A content distribution network may be utilized in the web server to handle real-time transcoding of the video stream to match the viewer's viewing capabilities. The viewers may utilize the second communication device to communicate with the user while the image recording device10is transmitting a live stream of video footage, still images, audio, and meta-data. Thus, the present invention allows the viewers to experience the user's visual and audio experiences in real time while also allowing the viewers to communicate with the user in real time.

In order to secure the image recording device10to the user, the mounting structure14may take on the form of eyeglasses16, as seen inFIG. 1. The eyeglasses16include a frame18that may be molded of conventional plastic or any other light weight, high strength material. The frame18includes a frame front20that surrounds and secures a pair of lenses22and a pair of similar frame temples24that extend from and are connected to the frame front20by a pair of hinges26. The lenses22may be conventional sunglass lenses, or the lenses22may comprise LCD/LED screens to allow for the projection of images or text on the screens for the user to view. The frame temples24extend to the top and/or rear of the ears (not shown) of the user to secure the eyeglasses16to the user's head (not shown). The frame temples24are sufficiently wide that the image recording device10can be connected to an inside surface of the frame temples24between the user's head and the frame temples24. The image recording device10may be secured within a protective casing or enclosure28which is attached to or integrally formed in the frame18(near positions16,18or26), or the image recording device10may be attached directly to the frame18. One image recording device10may be mounted on either or both sides of the frame18. As an alternative, by having two image recording devices10mounted to the frame18of the eyeglasses16, a video signal having two distinct video sources may be created to provide either stereoscopic playback (3D) or panoramic viewing. If two image recording devices10are utilized, then the image recording devices10may be attached electronically, for example, through the use of electrical wires (not shown). The electrical wires may extend along or within the frame18of the eyeglasses16. A lens of the image sensor12of each of the image recording devices10extends forward toward the lenses22of the eyeglasses16so that the image recording devices10can transmit what the user is viewing and hearing. A focusing device may be added to lenses to control the focus on both lenses at the same time via a hand-held device or device mounted on the frame18of the eyeglasses. The focusing device may be controlled by the user, may be controlled remotely by a technician, may be controlled using an algorithm, or in any other manner.

As previously noted, the image recording device10may include a digital video camera and/or a digital still camera with audio recording capability. The lens of the image recording device10projects an image to a digital image sensor (not shown) of the image recording device10. The image recording device10further includes an image processor (not shown) that converts the signal from the image sensor to an electronic image signal, as will be further discussed later in the specification. The image processor is electrically connected to a controller or processor30of the image recording device10, as seen inFIG. 3. The processor30is electrically connected to the controls of the image recording device10, such as a power switch (not shown), status lights (not shown), internal hard drive (not shown), memory cards (not shown), flash memory (not shown),input/output interfaces29, playback controls (not shown), zoom lens controls (not shown), etc. The processor30may provide feedback and status information to the user via LED lights or audible notifications. The processor30may also act as a standalone computer when not being utilized to control the image recording device10. The standalone computer could utilize a shared internet connection from a smart phone or utilize a WiFi connection. The image recording device10may utilize the internal hard drive or memory cards for recording the images being transmitted, or the images recorded may be streamed directly to one or more web servers on the internet. By providing an internal hard drive or memory card, the user may record and review the images before sending the data stream to the internet, or the user may edit the images before sending the data stream to the internet. The input/output interfaces may allow for a USB port connection so that a computer can be connected directly to the image recording device10for reviewing, editing, and sending the image recording. The input/output interfaces may allow for other types of connectivity such as infrared, near field communications, acoustic data communications, as well as a type of brain control interface. In addition, the user may record an event, and then subsequently decide to send the recording to the internet. The image recording device10is powered by a conventional battery source.

The image recording device can also record and transmit metadata. As one example, the image recording device10can incorporate a geo location sensitive device such as, for example, a Global Positioning System receiver, which can output geo location metadata. Other types of metadata can be recorded and transmitted by the image recording device10.

In order to create a data stream comprising the video and audio signals from the image recording device10, the present invention provides a “Reduced Instruction Set Computer” central processing unit as well as at least one hardware based video and audio compressor to process the multiple video and audio signals into a single merged video and audio signal referred to as the data stream. The methods for displaying such video and audio signals may include Composite, S-Video, RGB, HDMI, VGA and multiple integrated connections for built-in displays, such as LCD, LED, and OLED. The data stream is then transmitted to the first communication device, such as the smart phone. The smart phone may transmit the data stream over cellular phone data connections to a web server on the internet where viewers can view the data stream in real time using a web browser or custom application. Again, the web server may act as a content distribution network which may format the data stream into a format that can be viewed by the viewer's viewing device. Any number of real time operating systems may be utilized to transmit the data stream in real time, including but not limited to, Ubuntu, Android, WindowsCE, Windows Mobile and a custom Linux derivative. A second communication device, such as a smart phone or computer, may then be utilized by the viewers to communicate with the user in real time. Thus, the viewers may communicate with the user in real time while the user is recording, playing or streaming the data stream over the internet.

The input/output interface of the image recording device10of the present invention could also operate wirelessly through the use of Bluetooth or Wi-Fi technology or other types of wireless transmission technology that may be required for industry or purpose specific reasons, such as using a more secure wireless radio system for law enforcement. The present invention may also utilize more exotic methods or communication such as gravity waves possibly in the form of surface waves or internal waves as well as gravitational radiation as detectable by devices such as the “Weber Bar” or “MiniGrail”. Bluetooth is a proprietary open wireless technology that is standard for exchanging data over short distances using short wavelength radio transmissions from fixed and mobile devices, creating personal area networks (PANs) with high levels of security. The wireless connection could allow the present invention to act as a wireless hands-free headset that could act as stereo headphones and be connected to and communicate with other devices such as computers, cell phones, MP3 players and smart phones via Bluetooth technology. The wireless connection can be used to transmit the real time signal to the first communication device. The wireless connection can be used to stream the real time signal directly to the web server via the internet. The Bluetooth versions of the present invention can communicate with and include: Bluetooth v1.0, v1.0B, v1.1, v1.2, v2.0+EDR, v2.1+EDR, and v3.0+HS which integrates the Bluetooth module with the 802.11 (Wi-Fi) module for specific tasks. The Bluetooth profiles of the present invention will communicate with SDP, HCI, HID, A2DP, and HSP. It should be noted that Bluetooth and WiFi may both be provided on the same recording device10.

Although the present invention may use a conventional power source, the present invention may also utilize any combination of alternative energy sources, such as solar cells, thermoelectric generators, magnetic based generators, etc., for primary power, as well as using such energy sources as a secondary power source for charging a conventional battery.

In use, the present invention is utilized by having the user place the mounting structure14, such as the eyeglasses16, on the user's head. As seen in the flow chart ofFIG. 2, the power switch is turned on to activate operation of the image sensor12of the image recording device10, as stated in block31. The user can then decide, as stated in decision block32, whether to stream and/or record and store the data stream live to the first communication device, such as the smart phone, as stated in block34, or whether to record and store the images on the recording media of the image recording device10, as stated in block36. If the user decides to record the images, then the data stream can be sent to the content delivery network via the smart phone, as stated in block34, at a later time. If the user decides not to stream the data and also not to record, then the device will enter an idle mode where it buffers the data stream. At some later time, the user could decide to begin streaming the buffered data stream; in essence, this allows the viewer to capture an event that has already taken place even though the user was not actively streaming or recording. However, if the user decides to immediately stream the data stream to the smart phone, as seen in block34, then the settings of the image sensor12may be adjusted on the smart phone, as stated in block38. The smart phone transmits the data stream to at least one web server on the internet. In order for the viewers to view the data stream on the internet, the viewers log onto a predetermined and configured website, as stated in block40. The data stream may be configured and displayed on the website, and various categories may be accessed by the viewers through the second communication device, such as the smart phone or computer, as stated in block42. The website may provide a function wherein the user and/or the viewers can notify or alert their friends of the live streaming of the particular data stream, as stated in decision block44. If the users decide not to notify their friends, then the data stream may be viewed by the viewers, as seen in block48. If the users decide to alert their friends, then the friends are alerted, as stated in block46, and all viewers can begin viewing the data stream, as stated in block48. The viewers may be given the ability to communicate with the user via the second communication device, such as a smart phone or a computer or any other device that is operable to output the data stream in a form that is perceptible to the viewer, so that the viewer can experience the user's environment and the user's relationship to the environment.

FIG. 4shows an example of an environment for implementation of a system for streaming and viewing a user's visual and audio experiences, which incorporates the methods and apparatuses described previously. The system can include the image recording device10and a server50that hosts a website52. The image recording device10can communicate with the server50via a network54. The network54can be or include the internet and can also be or include any other type of network operable to transmit signals and/or data. At least one remote device56is also in communication with the server50via the network. The remote device56can be used by a viewer to received and view a real time signal, such as a video stream, from the image recording device10. It is anticipated that numerous (i.e., thousands) of remote devices could be connected to the server50. As previously described, the image recording device10can create a real time signal including at least one of video footage, still images, audio, and/or metadata that is captured by the image recording device10. This real time signal is transmitted to the server50over the network54. The server50can format the real time signal for transmission via the website52. A viewer that is using the remote device56can connect to the web site52that is hosted by the server50. Using the website52, the real time signal can be transmitted to the remote device56from the server50.

In the example shown inFIG. 4, the real time signal is transmitted directly from the image recording device10to the server50via the network54. In another example, which is shown inFIG. 5, the real time signal can be transmitted to the server50from the image recording device indirectly.

As shown inFIG. 5, the image recording device10is in communication with a personal communication device58, such as a smart phone. The communication between image recording device10and the personal communication device58can be a wireless connection or can be a wired connection using any suitable protocol. As one example, the image recording device can communicate with the personal communication device58via a wireless protocol such as Bluetooth. Using such a protocol, the image recording device10transmits the real time signal to the personal communications device58. The personal communication device58relays the real time signal that is received from the image recording device10to the server50. Operation of this system is otherwise as described in connection withFIG. 4.

Although described herein as a single server, it should be understood that a web server, such as the server50, can be implemented in the form of multiple computers, processors, or other systems working in concert.

An example of a device that can be used as a basis for implementing the systems and functionality described herein, such as the server52, the remote devices56, and the personal computing device58is a conventional computing device1000, as shown inFIG. 6. In addition, a computing device such as the computing device1000can be incorporated in the image recording device10. The conventional computer1000can be any suitable conventional computer, such as in the form of a desktop computer, server computer, laptop computer, tablet computer, or smart phone. As an example, the conventional computer1000can include a processor such as a central processing unit (CPU)1010and memory such as RAM1020and ROM1030. A storage device1040can be provided in the form of any suitable computer readable medium, such as a hard disk drive. One or more input devices1050, such as a keyboard and mouse, a touch screen interface, etc., allow user input to be provided to the CPU1010. A display1060, such as a liquid crystal display (LCD) or a cathode-ray tube (CRT), allows output to be presented to the user. The input devices1050and the display1060can be incorporated in a touch sensitive display screen. A communications interface1070is any manner of wired or wireless means of communication that is operable to send and receive data or other signals using the network108. The CPU1010, the RAM1020, the ROM1030, the storage device1040, the input devices1050, the display1060and the communications interface1070are all connected to one another by a bus1080.

It is anticipated that various features of the website may allow viewers to interact with the website in various ways. For instance, a crowd source decision making feature may allow viewers to give suggestions to a user's text question, such as “Which way should I go?” The viewers can give individual answers, and the website could determine a mean result which could be published and sent to the user. Other features can provide various alerts to viewers based on certain trigger values the ambient light detects a higher level of light for a high action event, then the viewer could be notified. The alert function can also alert viewers as to certain users streaming data from the image recording device10. Location based advertising can also be utilized wherein advertisement could be displayed based on the location of the user or items the user may view.

It is also anticipated that the apparatus and method of the present invention can be utilized for numerous applications. For instance, the present invention can be utilized for social networking wherein video footage can be streamed and archived as opposed to posting a few words on a website blog. Other applications include real time shared experiences, such as sharing activities with friends and family in real time. The field of medical triage and assistance could benefit from the present invention, as doctors and nurses could actually see what an EMT or paramedic is experiencing so as to provide sound advice to the EMT or paramedic in extreme medical situations. Surgeons can stream live video from a surgery so that colleagues can watch and comment from off-site. Field reporters can use the present invention to send high quality, high definition video in real time without the need for setting up or utilizing bulky video cameras. Remote skilled workers can benefit by obtaining assistance in the field by sending live streaming video of a specific task or problem that needs to be resolved or corrected. Celebrities can stream their activities in real time for fans to view their experiences. Law enforcement officers can stream live video of dangerous situations for comment or analysis by ranking personnel. The film and movie industry can benefit from the present invention by having live video streamed off-site to a director so that the director can direct and capture video footage. Sporting events and participants can provide live streaming video so that viewers can experience the activities of the sporting event. The retail industry can also benefit by having users provide live streaming video of clothing or other items for purchase so that viewers can comment and provide advice to the user on the purchase of such products. The present invention is not limited to the above applications, but rather, the above-noted applications are but an example of the applications in which the present invention may be utilized.