Visible indicator for importance of audio

In one aspect, a device includes a processor, a video display accessible to the processor, and storage accessible to the processor and comprising instructions executable by the processor to present an indication on the video display of an importance of audio to video relative to effective user consumption.

FIELD

The present application relates to technically inventive, non-routine solutions that are necessarily rooted in computer technology and that produce concrete technical improvements.

BACKGROUND

As recognized herein, videos are commonly shared among users and user groups via social media. Present principles understand that for some video media, the audio component is critical for consumption, whereas in other cases the audio is not necessary, and the media can be consumed satisfactorily without sound. Two examples of such video include public settings and ephemeral content. However, as understood herein with respect to ephemeral content in particular, video once consumed is no longer available, including any accompanying audio.

There are currently no adequate solutions to the foregoing computer-related, technological problem.

SUMMARY

Accordingly, in one aspect a device includes at least one processor, a video display accessible to the at least one processor, and storage accessible to the at least one processor that includes instructions executable by the at least one processor to present a visible indication on the video display of an importance of audio to video presented on the video display.

In example embodiments, the instructions are executable to present on the video display a first visible indication for a first video indicating audio is important to the first video, and present on the video display a second visible indication for a second video indicating audio is not important to the second video. The first visible indication can include a first color and the second visible indication can include a second color.

In some examples, the visible indication is not selectable to toggle audio on and off. In other examples, the visible indication is selectable to toggle audio on and off.

In example implementations the visible indication includes a halo around a symbol. In other example implementations the visible indication includes a halo around a scrolling video pane.

Visible indication may be associated with video responsive to classification input from a user input device. Or, visible indication may be associated with video responsive to classification input from at least one neural network.

In another aspect, at least one computer readable storage medium (CRSM) that is not a transitory signal includes instructions executable by at least one processor to identify a first audio video content as being in a first classification, and based at least in part on identifying the first audio video content as being in the first classification, juxtapose a first visible indicator with the first audio video content. The instructions also are executable to identify a second audio video content as being in a second classification, and based at least in part on identifying the second audio video content as being in the second classification, juxtapose a second visible indicator with the second audio video content. The first classification includes a first audio play recommendation and the second classification includes a second audio play recommendation.

In another aspect, a method includes presenting on a video display a first indicator indicating that a respective first video is recommended for play without audio accompanying the first video, and presenting on the video display a second indicator indicating that a respective second video is recommended for play with audio accompanying the second video.

DETAILED DESCRIPTION

Note with respect to any computer systems discussed herein that a system may include server and client components, connected over a network such that data may be exchanged between the client and server components. The client components may include one or more computing devices including televisions (e.g., smart TVs, Internet-enabled TVs), computers such as desktops, laptops and tablet computers, so-called convertible devices (e.g., having a tablet configuration and laptop configuration), and other mobile devices including smart phones. These client devices may employ, as non-limiting examples, operating systems from Apple Inc. of Cupertino Calif., Google Inc. of Mountain View, Calif., or Microsoft Corp. of Redmond, Wash. A Unix® or similar such as Linux® operating system may be used. These operating systems can execute one or more browsers such as a browser made by Microsoft or Google or Mozilla or another browser program that can access web pages and applications hosted by Internet servers over a network such as the Internet, a local intranet, or a virtual private network.

As used herein, instructions refer to computer-implemented steps for processing information in the system. Instructions can be implemented in software, firmware or hardware, or combinations thereof and include any type of programmed step undertaken by components of the system; hence, illustrative components, blocks, modules, circuits, and steps are sometimes set forth in terms of their functionality.

A processor may be any general purpose single- or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, and control lines and registers and shift registers. Moreover, any logical blocks, modules, and circuits described herein can be implemented or performed with a general purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device such as an application specific integrated circuit (ASIC), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor can also be implemented by a controller or state machine or a combination of computing devices. Thus, the methods herein may be implemented as software instructions executed by a processor, suitably configured application specific integrated circuits (ASIC) or field programmable gate array (FPGA) modules, or any other convenient manner as would be appreciated by those skilled in those art. Where employed, the software instructions may also be embodied in a non-transitory device that is being vended and/or provided that is not a transitory, propagating signal and/or a signal per se (such as a hard disk drive, CD ROM or Flash drive). The software code instructions may also be downloaded over the Internet. Accordingly, it is to be understood that although a software application for undertaking present principles may be vended with a device such as the system100described below, such an application may also be downloaded from a server to a device over a network such as the Internet.

Software modules and/or applications described by way of flow charts and/or user interfaces herein can include various sub-routines, procedures, etc. Without limiting the disclosure, logic stated to be executed by a particular module can be redistributed to other software modules and/or combined together in a single module and/or made available in a shareable library.

Logic when implemented in software, can be written in an appropriate language such as but not limited to C # or C++, and can be stored on or transmitted through a computer-readable storage medium (that is not a transitory, propagating signal per se) such as a random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage such as digital versatile disc (DVD), magnetic disk storage or other magnetic storage devices including removable thumb drives, etc.

Now specifically in reference toFIG. 1, an example block diagram of an information handling system and/or computer system100is shown that is understood to have a housing for the components described below. Note that in some embodiments the system100may be a desktop computer system, such as one of the ThinkCentre® or ThinkPad® series of personal computers sold by Lenovo (US) Inc. of Morrisville, N.C., or a workstation computer, such as the ThinkStation®, which are sold by Lenovo (US) Inc. of Morrisville, N.C.; however, as apparent from the description herein, a client device, a server or other machine in accordance with present principles may include other features or only some of the features of the system100. Also, the system100may be, e.g., a game console such as XBOX®, and/or the system100may include a mobile communication device such as a mobile telephone, notebook computer, and/or other portable computerized device.

As shown inFIG. 1, the system100may include a so-called chipset110. A chipset refers to a group of integrated circuits, or chips, that are designed to work together. Chipsets are usually marketed as a single product (e.g., consider chipsets marketed under the brands INTEL®, AMD®, etc.).

In the example ofFIG. 1, the chipset110has a particular architecture, which may vary to some extent depending on brand or manufacturer. The architecture of the chipset110includes a core and memory control group120and an I/O controller hub150that exchange information (e.g., data, signals, commands, etc.) via, for example, a direct management interface or direct media interface (DMI)142or a link controller144. In the example ofFIG. 1, the DMI142is a chip-to-chip interface (sometimes referred to as being a link between a “northbridge” and a “southbridge”).

The interfaces of the I/O hub controller150may provide for communication with various devices, networks, etc. For example, where used, the SATA interface151provides for reading, writing or reading and writing information on one or more drives180such as HDDs, SDDs or a combination thereof, but in any case the drives180are understood to be, e.g., tangible computer readable storage mediums that are not transitory, propagating signals. The I/O hub controller150may also include an advanced host controller interface (AHCI) to support one or more drives180. The PCI-E interface152allows for wireless connections182to devices, networks, etc. The USB interface153provides for input devices184such as keyboards (KB), mice and various other devices (e.g., cameras, phones, storage, media players, etc.).

Additionally, though not shown for simplicity, in some embodiments the system100may include a gyroscope that senses and/or measures the orientation of the system100and provides input related thereto to the processor122, as well as an accelerometer that senses acceleration and/or movement of the system100and provides input related thereto to the processor122. Still further, the system100may include an audio receiver/microphone that provides input from the microphone to the processor122based on audio that is detected, such as via a user providing audible input to the microphone, and a camera that gathers one or more images and provides input related thereto to the processor122. The camera may be a thermal imaging camera, an infrared (IR) camera, a digital camera such as a webcam, a three-dimensional (3D) camera, and/or a camera otherwise integrated into the system100and controllable by the processor122to gather pictures/images and/or video. Also, the system100may include a GPS transceiver that is configured to communicate with at least one satellite to receive/identify geographic position information and provide the geographic position information to the processor122. However, it is to be understood that another suitable position receiver other than a GPS receiver may be used in accordance with present principles to determine the location of the system100.

It is to be understood that an example client device or other machine/computer may include fewer or more features than shown on the system100ofFIG. 1. In any case, it is to be understood at least based on the foregoing that the system100is configured to undertake present principles.

Turning now toFIG. 2, example devices are shown communicating over a network200such as the Internet in accordance with present principles. It is to be understood that each of the devices described in reference toFIG. 2may include at least some of the features, components, and/or elements of the system100described above. Indeed, any of the devices disclosed herein may include at least some of the features, components, and/or elements of the system100described above.

FIG. 2shows a notebook computer and/or convertible computer202, a desktop computer204, a wearable device206such as a smart watch, a smart television (TV)208, a smart phone210, a tablet computer212, and a server214such as an Internet server that may provide cloud storage accessible to the devices202-212. It is to be understood that the devices202-214are configured to communicate with each other over the network200to undertake present principles.

Referring toFIG. 3, overall logic is shown that may be executed by a device such as any in the system100in accordance with present principles. Beginning at block300, one or more videos and/or links to videos are received. Each video/link is identified at block302as belonging to one of at least two and in some embodiments three or more classifications pertaining to how necessary audio accompanying the video is to be able to enjoy or effectively “consume” the video. In an example, a first classification is that audio accompanying the video is recommended, i.e., should be played along with the video for effective consumption of the video, while a second classification is that audio accompanying the video need not be played for effective consumption of the video. A third classification may be that it is uncertain whether audio accompanying the video should be played for effective consumption of the video.

An example of a video in the first classification is a news feed, in which the principal import of the audio-video content is an audio report by a reporter in the video. An example of a video in the second classification is an extreme skiing video, in which the principal import of the audio-video content is the video action of the skier and not necessarily the audio, which typically is a background music track. An example of a video in the third classification may be a video that the logic cannot readily identify as being in the first or second classifications, or a video that may have been classified directly into the third classification by a user or a neural network as disclosed below.

In any case, proceeding from block302to block304, a visible indicator associated with the classification at block302is presented adjacent or on the video.

FIGS. 4 and 5illustrate two techniques for classifying videos. InFIG. 4a UI400is presented on, e.g., a touch screen display allowing a user to correlate, via the touch screen display or other user input device, types of videos and/or individual videos with types of audio importance indicators. In the simplified example ofFIG. 4, a first column402indicating that audio is always recommended for playing is shown, under which are entries404of types of videos the user wishes to be grouped in the “always audio” classification. The entries404may be manually entered by a user or may be presented automatically in a list from which the user can select videos and/or video types to associate with the “always audio” classification. These are but two example techniques for associating videos with audio recommendations. Similarly, a second column406indicating that audio is never recommended for playing is shown, under which are entries408of types of videos the user wishes to be grouped in the “never audio” classification. Additional columns for additional audio recommendation classifications may be included.

FIG. 5shows an alternate technique that uses one or more neural networks (NN) to learn what types of videos should be associated with what kind of audio play recommendation. Commencing at block500, a training set of videos is input into the NN pre-classified by a trainer into the desired audio recommendation classifications, e.g., “always audio”, “never audio”, and “unsure”. Moving to block502, an untagged video is input to the NN and at block504a prediction is received from the NN as to what audio recommendation classification the video is given. This prediction is manually tagged by a trainer at block506and fed back to the NN for further training.

In executing the logic ofFIG. 5, image recognition (including whether setting is public, indicating no audio recommended or private indicating audio recommended), activity recognition (e.g., video motion vectors above a threshold indicate no audio needed), and audio recognition may be used to make a prediction as to whether audio accompanying video is recommended (or not) to be played to effectively consume the media. The audio analysis further may be based on how “full” the audio channel is, particularly the modulation of audio needed to separate noise from important sound relative to the media.

Now referring toFIG. 6, a video display600is shown that may be implemented by any of the example displays described herein. As shown, a list of video selectors600may be presented with each selector600typically including a thumbnail still or video image of the underlying video that can be invoked by selecting, via touch or cursor input or other means, the desired selector600. Adjacent each video selector600is a respective visible indication602indicating the importance of accompanying audio to effectively consuming the video associated with the respective selector600.

Thus, a first visible indication can be presented for a first video indicating audio is important to the first video, while a second visible indication can be presented for a second video indicating audio is not important to the second video.

In some implementations the visible indication602is not selectable to toggle audio on and off. In such embodiments, a separate audio on/off toggle symbol604may be presented that can be selected to toggle audio on and off for the respective video. In other implementations the visible indication602is selectable to toggle audio on and off, in which case the audio on/off toggle symbols604can be omitted.

In the example shown, the visible indication includes a hollow “halo”606around a symbol608such as a square. In other examples the halo606may be omitted. In any case the halo606and/or symbol608have different visible appearances depending on whether audio is recommended for the associated video. For example, for a video identified as being in the “always audio” recommendation category (in the example shown, video A), the respective halo606/symbol608may be a first color such as red, whereas for a video identified as being in the “never audio” recommendation category (in the example shown, video C), the respective halo606/symbol608may be a second color such as green. For a video not identified as being in either the “always audio” or “never audio” recommendation category (in the example shown, video B), the respective halo606/symbol608may be a third color such as yellow.

As examples, assume that the video selectors600inFIG. 6are received from a social network that provides ephemeral videos. Absent present principles the user would not know which media is recommended as being acceptable to open without sound and still be able to be consumed and enjoyed. With the present visible indicators602the user is informed which media should be consumed with sound and which is OK to be consumed without sound.

In the case of video A inFIG. 6, assume the system has detected it as being a selfie video with the cameraman speaking into the camera. The system determines based on audio and visual analysis that sound is important to consume this media and consequently is associated with the “A” indicator inFIG. 6. Assume that the system detects video C as being a video of someone holding his phone out of the window of a car filming a landscape. The audio likely would be loud and indiscernible due to the wind noise, and the system therefore identifies that video C is OK to consume without audio and so the “C” visible indicator is presented adjacent video C. In contrast, assume that the system identifies video B as a video of someone doing a handstand with cheering in the background, identifying the video as one that may be enjoyed with audio or without audio and consequently presenting the “B” indicator.

FIGS. 7 and 8provide further illustration. InFIG. 7a UI700may be presented responsive to video C having been selected fromFIG. 6to present, as indicated at702, video C. A notice704may be presented indicating that video C has been selected and that the audio recommendation is “no audio”, with any audio accompanying the video being automatically muted upon selection of video C. An audio on selector706may be presented and is selectable to audibly present the audio accompanying the video C.

InFIG. 8a UI800may be presented responsive to video A having been selected fromFIG. 6to present, as indicated at802, video A. A notice804may be presented indicating that video A has been selected and that the audio recommendation is “audio on”, with any audio accompanying the video being automatically audibly played upon selection of video A. An audio off selector806may be presented and is selectable to mute the audio accompanying the video A.

FIG. 9illustrates a display presenting current video pane900playing a current video and a partial video pane902representing approaching media and potentially scrolling upward as indicated by the arrow904to replace the current video pane. The partial video pane902presents a still or video in only a top portion of a full video pane in the non-limiting example shown. An audio recommendation indicator904is presented on or near the upcoming video pane902indicating whether audio is or is not recommended for consumption with the video. In the example shown the indicator904is a halo that extends completely along all visible edges around the video pane902, color-coded consistent with principles above to indicate the audio play recommendation.

Thus for exampleFIG. 9may illustrate a newsfeed that has autoplay enabled.FIG. 9helps the user to determine if the user should continue scrolling and denote if the user is approaching media that will have sound/require sound. The system can split newsfeed into ‘sound necessary’ and ‘sound not necessary’. The user can toggle between these two modes. The system can order the newsfeed sorting the content with audio needed from lowest to highest or vice versa.

In some examples, a device operating consistent with present principles may even present a UI on its display to configure one or more settings of the device to operate consistent with present principles. For example, an option may be presented on a settings UI that may be selectable to enable or configure the device to undertake present principles, e.g., undertake the logic ofFIGS. 3 and 5and present the UIs ofFIGS. 4 and 6-9.

It may now be appreciated that present principles provide for an improved computer-based user interface that improves the functionality and ease of use of the devices disclosed herein. The disclosed concepts are rooted in computer technology for computers to carry out their functions.

It is to be understood that whilst present principals have been described with reference to some example embodiments, these are not intended to be limiting, and that various alternative arrangements may be used to implement the subject matter claimed herein. Components included in one embodiment can be used in other embodiments in any appropriate combination. For example, any of the various components described herein and/or depicted in the Figures may be combined, interchanged or excluded from other embodiments.