Patent ID: 12256126

In the drawings, like reference numbers generally indicate identical or similar elements. Additionally, generally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.

DETAILED DESCRIPTION

FIG.1illustrates a block diagram of a multimedia environment102, according to some embodiments. In a non-limiting example, multimedia environment102is directed to streaming media such as movies, TV shows, games, audio books, and music, to name just some examples.

The multimedia environment102may include one or more media devices104. In an embodiment, media device104is a speaker. However, this disclosure is not limited to that example embodiment. In other embodiments, the media device104may be a streaming media device, DVD device, audio/video playback device, cable box, appliance, internet of things (IoT) device, and/or digital video recording device, to name just a few examples.

Media device104may include one or more speakers110, one or more audio and video decoders112, one or more transceivers114and buffer memory116. In some embodiments, the media device104responds to audio commands from users105and, accordingly, includes a microphone to receive audio data (not shown inFIG.1). In other embodiments, the media device104is not audio responsive and does not include a microphone.

In some embodiments, media device104may be coupled to a display device106. Display device106may be a monitor, television, computer, smart phone, tablet, wearable (such as a watch), and/or projector, to name just a few examples. In some embodiments, media device104can be a part of, integrated with, operatively coupled to, and/or connected to its respective display device106.

Each media device104may be configured to communicate with network134. In various embodiments, the network134can include, without limitation, wired and/or wireless intranet, extranet, Internet, cellular, Bluetooth and/or any other short range, long range, local, regional, global communications network, as well as any combination thereof.

The multimedia environment102may include one or more content servers138(also called content providers138). Each content server138may be configured to communicate with network134.

Each content server138may store content140and metadata142. Content140may include any combination of music, videos, movies, TV programs, multimedia, images, still pictures, text, graphics, gaming applications, advertisements, programming content, public service content, government content, local community content, software, and/or any other content or data objects in electronic form.

Metadata142may include data about content140. For example, metadata142may include associated or ancillary information indicating or related to writer, director, producer, composer, artist, actor, summary, chapters, production, history, year, trailers, alternate versions, related content, applications, and/or any other information pertaining or relating to the content140. Metadata142may also or alternatively include links to any such information pertaining or relating to the content140. Metadata142may also or alternatively include one or more indexes of content140, such as but not limited to a trick mode index.

Users105may interact with the media device104and/or the display device106using, for example, a remote control107. The remote control107can be any component, part, apparatus and/or method for controlling the media device104and/or display device106, such as a remote control, a tablet, laptop computer, smartphone, wearable, on-screen controls, integrated control buttons, audio controls, or any combination thereof, to name just a few examples. In some embodiments, the remote control107is battery operated. Users105may use the remote control107to interact with the media device104and/or display device106to select and playback content, such as a movie, TV show, music, book, application, game, etc.

Remote control107may be considered a smart remote control given its functionality described herein.

Remote control107may be audio responsive and respond to audio commands spoken by users105and/or audibly output by devices in the multimedia environment102, such as the display device106. Accordingly, remote control107may include a microphone126for receiving audio data.

Remote control107may also include one or more speakers118, one or more audio and video decoders120, one or more transceivers122, a talk button124, buffer memory128, a motion sensor130, and/or an audio processing module132. The remote control107may include other components, such as one or more processors or other control hardware for performing the functions described herein.

Each audio decoder112,120in the media device104and remote control107may be configured to decode audio of one or more audio formats, such as but not limited to AAC, HE-AAC, AC3 (Dolby Digital), EAC3 (Dolby Digital Plus), WMA, WAV, PCM, MP3, OGG GSM, FLAC, AU, AIFF, and/or VOX, to name just some examples.

Each video decoder112,120in the media device104and remote control107may be configured to decode video of one or more video formats, such as but not limited to MP4 (mp4, m4a, m4v, f4v, f4a, m4b, m4r, f4b, mov), 3GP (3gp, 3gp2, 3g2, 3gpp, 3gpp2), OGG (ogg, oga, ogv, ogx), WMV (wmv, wma, asf), WEBM, FLY, AVI, QuickTime, HDV, MXF (OP1a, OP-Atom), MPEG-TS, MPEG-2 PS, MPEG-2 TS, WAV, Broadcast WAV, LXF, GXF, and/or VOB, to name just some examples. Each video decoder112,120may include one or more video codecs, such as but not limited to H.263, H.264, HEV, MPEG1, MPEG2, MPEG-TS, MPEG-4, Theora, 3GP, DV, DVCPRO, DVCPRO, DVCProHD, IMX, XDCAM HD, XDCAM HD422, and/or XDCAM EX, to name just some examples.

The multimedia environment102may include one or more audio processing server(s)136. Each audio processing server136, as well as the audio processing module132in the remote control107, may be configured to receive audio data, and to recognize wake words as well as audio commands contained in the received audio data. Examples of recognizing wake words are described in U.S. patent application Ser. No. 15/895,047 titled “Trigger Word Detection With Multiple Digital Assistants,” filed Feb. 13, 2018; and U.S. patent application Ser. No. 15/341,552 titled “Improved Reception Of Audio Commands,” filed Nov. 2, 2016, which are herein incorporated by reference in their entireties.

FIG.2is a flowchart of a method202for operating a media device using a smart remote control. In some embodiments, the smart remote control is audio responsive and the media device is not audio responsive. In other words, the smart remote control is configured to respond to audio commands, and the media device is not configured to respond to audio commands. Method202can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG.2, as will be understood by a person of ordinary skill in the art. Method202shall be described with reference toFIG.1. However, method202is not limited to that example embodiment.

In204, the user105may press the talk button124on the remote control107to indicate his intent to speak an audio command to the media device104. But, according to some embodiments, the user105speaks the audio command to the remote control107rather than the media device104. Thus, the user105does not need to be physically proximate to the media device104to speak such audio command. Rather, the user105needs to only be physically proximate to the remote control107to speak audio commands. Such arrangement is advantageous because, for example, the user105can be carrying the remote control107and issuing audio commands to the media device104, while in parts of his house not containing the media device104.

It is noted that, whileFIG.2is described herein as the user105issuing audio commands to the media device104, this is done solely for illustrative purposes. In other embodiments, the user105may instead issue audio commands to any device in multimedia environment102, such as but not limited to the display device106.

In some embodiments, the pressing of the talk button124begins a listen time period205during which time the remote control107processes and analyzes audio data received via microphone126to recognize audio commands spoken by the user105or audibly output by a device in multimedia environment102(such as the display device106for example). Such processing and analysis of received audio data to recognize audio commands contained therein is sometimes referred to herein as listening for audio commands. The listen time period205begins at204(when the user105presses the talk button124) and ends X seconds later at212. X may be 5 or 10 seconds, for example, or any other predetermined time period.

Accordingly, in206, in response to the talk button124being pressed, the remote control107may begin listening for audio commands in the audio data received via microphone126. As part of206, the remote control107may turn on the microphone126.

In some embodiments, the remote control107only listens for audio commands during the listen time period205, and not outside the listen time period205. As a result, more accurate and reliable recognition of audio commands is achieved because the remote control107knows when the user105intends to speak audio commands. Also, because the remote control107only listens for audio commands during the listen time period205, rather than constantly at all times, power consumption is reduced. This is especially important since the remote control107may be a battery operated device.

In some embodiments, audio commands may include wake words, as well as commands that follow the wake words. In other embodiments, the pressing of the talk button124itself represents the wake word, in which case audio commands refer to the commands that follow the wake word. In other words, the remote control107may consider detection of the talk button124being pressed as receipt of the wake word.

In208, the user105may speak a command, or a device in the multimedia environment102(such as the display device106) may audibly output a command. This audible command represents audio data.

In210, the remote control107may receive audio data via the microphone126.

In212, at the end of the listen time period205, the remote control107may stop listening for audio commands in the received audio data. In other words, at212, the remote control107stops analyzing audio data received via microphone126to recognize audio commands contained therein. In some embodiments, at212, the remote control107turns off the microphone126.

In214, the remote control107may process and analyze the audio data received in210to recognize audio commands contained therein. The remote control107may perform step214locally by using the audio processing module132, or remotely by sending via the network134the received audio data to the audio processing server136for processing, or a combination thereof, using any speech recognition process, technique, hardware, software or solution, including but not limited to the examples described in U.S. patent application Ser. No. 15/895,047 titled “Trigger Word Detection With Multiple Digital Assistants,” filed Feb. 13, 2018; and U.S. patent application Ser. No. 15/341,552 titled “Improved Reception Of Audio Commands,” filed Nov. 2, 2016, which are herein incorporated by reference in their entireties.

In216, the remote control107may audibly output, through speakers118, the audio commands recognized in step214. The remote control107may take this action to confirm the audio commands with the user105. (In some embodiments, the remote control107may confirm audio commands using other hardware, software, solution, systems, processes and techniques, such as that described in U.S. patent application Ser. No. 15/646,379 titled “Controlling Visual Indicators In An Audio Responsive Electronic Device, and Capturing and Providing Audio Using an API, By Native and Non-Native Computing Devices and Services,” filed Jul. 11, 2017, which is herein incorporated by reference in its entirety.)

In some embodiments, as further discussed below with respect toFIG.3, when audibly outputting such confirmation messages via speakers118, the remote control107may adjust the volume level of the messages to match the volume level of the audio data received in210. For example, if the user105whispered the command in208, then the remote control107will adjust the volume level of the confirmation message to match the volume level of the whisper. Similarly, if the user105spoke the command in a loud voice in208, then the remote control107will adjust the volume level of the confirmation message to match the volume level of the loud voice. In this manner, it is more likely that the remote control107will output confirmation message at a volume level suitable for the audible conditions currently existing around the user105. It is noted that such operation may not be limited to just confirmation messages, but may instead apply to anything output by the remote control107via speakers118, including status messages, administrative messages, content such as music and the audio track of movies and TV programs, etc., as well as any combination or sub-combination thereof.

The remote control107may then process the audio commands that were recognized in step214. Such processing may involve causing the media device104, display device106and/or other devices in multimedia environment102to take action consistent with the audio commands. For example, assume that the command spoken in208is a request for content, such as music or a movie (this is indicated by218). Then in this example, the remote control107in220may request the content from content server(s)138via network134. In some embodiments, the remote control107may perform220by requesting the content directly from the content server(s)138. In other embodiments, the remote control107may perform220by sending a message to the media device104, to command the media device104to request the content from content server(s)138via network134.

In222, the content server(s)138may stream the requested content to the remote control107and the media device104via the network134. In other embodiments, the content server(s)138may stream the requested content to the remote control107, and the remote control107may then transmit the received content to the media device104(and/or other devices in the multimedia environment102). In these other embodiments, where the remote control107has at least two transceivers122, the remote control107may use a first transceiver122to receive the content from the content server(s)138via the network134, and may use a second transceiver122to transmit the received content to the media device104. Where the remote control107has a single transceiver122, the remote control107may switch the transceiver122between receiving the content from the content server(s)138via the network134, and transmitting the received content to the media device104. In any of these embodiments, buffer memories116,128may be used to store the content for either real-time or on-demand playback.

In224, the media device104may play the received content using the speakers110. Only the speakers110may be needed for playback if the received content is music, for example. If the received content is a movie or TV program, then the display device106may also be used for the playback.

In some embodiments, in226, the remote control107may play the received content using the speakers118. Similar to that discussed above with respect to step216, the remote control107may adjust the volume level of such playback to match the volume level of the audio data received in210. For example, if the user105whispered the command in208, then the remote control107will adjust the volume level of the playback to match the volume level of the whisper. Similarly, if the user105spoke the command in a loud voice in208, then the remote control107will adjust the volume level of the playback to match the volume level of the loud voice. In this manner, it is more likely that the remote control107will play the received content at a volume level suitable for the audible conditions currently existing around the user105at the time step226is performed.

Such operation shall now be described in greater detail with respect to method302inFIG.3. Method302can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG.3, as will be understood by a person of ordinary skill in the art. Method302shall be described with reference toFIGS.1and2. However, method302is not limited to those example embodiments.

In304, the audio processing module132in the remote control107may determine the volume level of audio data received via microphone126. The audio processing module132may perform this function whenever audio data is received via microphone126, such as during step210inFIG.2.

In306, whenever audibly outputting sounds over speakers118, the remote control107may adjust the volume level of such sounds to match the volume level determined in304. Step306may be performed during steps216and226ofFIG.2, for example. The processing of step306may apply to all sounds, or just some sounds. For example, the volume level may be adjusted per step306when outputting confirmation messages (as in step216), but not adjusted when playing back content (as in step226). Instead, in such embodiments, content may be played back at step226at a volume level matching the volume level at the media device104. In some embodiments, the user105may indicate the type of messages and/or content where the volume adjustment of step306is to be performed.

Some embodiments include methods for using motion of the remote control107to change the volume level at the remote control107and/or the media device104. For example, in308, a volume changing motion of the remote control107may be detected. A volume changing motion may be any predefined motion to indicate a desire to change the volume, such as moving the remote control107up or down, right or left, forward or back, etc. For example, movement to the right may indicate volume up, whereas movement to the left may indicate volume down. The magnitude of the volume change may be indicated by the extent that the remote control107is moved, and/or the speed of the detected motion. For example, the greater or faster the motion, the more the volume is changed, and vice versa. Movements of the remote control107as described herein may be detected by motion sensor130, which may include any mechanism, module, system, software, hardware or solution for detecting motion of the remote control107.

Accordingly, in308, the remote control308may detect that it has moved by using the motion sensor130. The remote control308then may compare such detected movement to a database of volume changing motions (such database may be stored in buffer128or other storage module of the remote control107), to determine whether the detected movement matches one of the volume changing motions in the database.

If there is a match, then in310, the remote control107may change the volume based on the detected volume changing motion. The volume may be changed at speakers118of the remote control107and/or the speakers110of the media device104(as specified by the user105via preference settings, for example). To change the volume level of the speakers110of the media device104, the remote control107may transmit an appropriate command/message to the media device104to take such action.

It is noted that, in large part, the functions and operations were described above as being performed by the remote control107. However, this disclosure is not limited to that example embodiment. In other embodiments, the functions and operations may be performed by other devices in multimedia environment102. For example, in some embodiments, the media device104may be audio responsive and include a talk button and a microphone, and may perform the functions and operations described above as being performed by the remote control107.

Example Computer System

Various embodiments and/or components therein can be implemented, for example, using one or more computer systems, such as computer system400shown inFIG.4. Computer system400can be any computer or computing device capable of performing the functions described herein. For example, one or more computer systems400or portions thereof can be used to implement any embodiments ofFIGS.1-3, and/or any combination or sub-combination thereof, such as the remote control107and/or the media device104.

Computer system400includes one or more processors (also called central processing units, or CPUs), such as a processor404. Processor404is connected to a communication infrastructure or bus406.

One or more processors404can each be a graphics processing unit (GPU). In some embodiments, a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications. The GPU can have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.

Computer system400also includes user input/output device(s)403, such as monitors, keyboards, pointing devices, etc., that communicate with communication infrastructure406through user input/output interface(s)402.

Computer system400also includes a main or primary memory408, such as random access memory (RAM). Main memory408can include one or more levels of cache. Main memory408has stored therein control logic (i.e., computer software) and/or data.

Computer system400can also include one or more secondary storage devices or memory410. Secondary memory410can include, for example, a hard disk drive412and/or a removable storage device or drive414. Removable storage drive414can be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.

Removable storage drive414can interact with a removable storage unit418. Removable storage unit418includes a computer usable or readable storage device having stored thereon computer software (control logic) and/or data. Removable storage unit418can be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device. Removable storage drive414reads from and/or writes to removable storage unit418in a well-known manner.

According to an exemplary embodiment, secondary memory410can include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system400. Such means, instrumentalities or other approaches can include, for example, a removable storage unit422and an interface420. Examples of the removable storage unit422and the interface420can include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.

Computer system400can further include a communication or network interface424. Communication interface424enables computer system400to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number428). For example, communication interface424can allow computer system400to communicate with remote devices428over communications path426, which can be wired and/or wireless, and which can include any combination of LANs, WANs, the Internet, etc. Control logic and/or data can be transmitted to and from computer system400via communication path426.

In some embodiments, a non-transitory, tangible apparatus or article of manufacture comprising a tangible computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system400, main memory408, secondary memory410, and removable storage units418and422, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system400), causes such data processing devices to operate as described herein.

Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art(s) how to make and use embodiments of this disclosure using data processing devices, computer systems and/or computer architectures other than that shown inFIG.4. In particular, embodiments can operate with software, hardware, and/or operating system implementations other than those described herein.

CONCLUSION

It is to be appreciated that the Detailed Description section, and not the Summary and Abstract sections, is intended to be used to interpret the claims. The Summary and Abstract sections can set forth one or more but not all exemplary embodiments as contemplated by the inventors, and thus, are not intended to limit this disclosure or the appended claims in any way.

While this disclosure describes exemplary embodiments for exemplary fields and applications, it should be understood that the disclosure is not limited thereto. Other embodiments and modifications thereto are possible, and are within the scope and spirit of this disclosure. For example, and without limiting the generality of this paragraph, embodiments are not limited to the software, hardware, firmware, and/or entities illustrated in the figures and/or described herein. Further, embodiments (whether or not explicitly described herein) have significant utility to fields and applications beyond the examples described herein.

Embodiments have been described herein with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined as long as the specified functions and relationships (or equivalents thereof) are appropriately performed. Also, alternative embodiments can perform functional blocks, steps, operations, methods, etc. using orderings different than those described herein.

References herein to “one embodiment,” “an embodiment,” “an example embodiment,” or similar phrases, indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of persons skilled in the relevant art(s) to incorporate such feature, structure, or characteristic into other embodiments whether or not explicitly mentioned or described herein. Additionally, some embodiments can be described using the expression “coupled” and “connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, some embodiments can be described using the terms “connected” and/or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

The breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.