Non-linear navigation of videos

An embodiment of the invention may include a method, computer program product and computer system for non-linear video navigation. The method, computer program product and computer system may include a computing device that collects source content and source content metadata from a content device. The computing device may extract the metadata from the source content and store the source content and the metadata on a server. The computing device may receive user profile data from a user device. The computing device may map the semantic data of the source content, the metadata, and the user profile data to a knowledge graph. The computing device may create navigational panels for navigating the source content based on the knowledge graph, the navigational panels enabling non-linear navigation of the source content by a user. The computing device may provide the navigational panels to the user device.

BACKGROUND

The present invention relates generally to a method, system, and computer program for non-linear navigation of video content. More particularly, the present invention relates to a method, system, and computer program for navigating video sub-content based on a user's proficiency, content presentation styles and content concept categories.

Education has become increasingly digital with classes being offered via video instead of through live classes. Such educational videos may be presented live or may be recorded and stored on a remote server and accessed by a user via a network. Educational videos allow users to attend classes on their own time and thus expand a user's opportunities beyond what was available to them via traditional live classes. Educational videos may contain content which the user already knows or has already seen and a user may not be able to efficiently navigate through the video to point of interest.

For example, a user may want to go back and re-watch portions of an educational video, which may be difficult as the user would need to fast forward and rewind through the video to find the user's desired content. Such rigid linearity of educational videos restricts users' ability to efficiently navigate through the video for desired content and provides a less than desirable educational source.

BRIEF SUMMARY

Non-linear video navigation is a process that allows a user to navigate video content in a personalized order. Non-linear video navigation thus enables a user to view specific parts within video content in any order that is suitable to a user's needs.

An embodiment of the invention may include a method, computer program product and computer system for non-linear video navigation. The method, computer program product and computer system may include a computing device that collects source content and source content metadata from at least one content device by a server communicating with the content device by a communication network. The computing device may extract the metadata from the source content and store the source content and the metadata on the server. The computing device may receive user profile data from at least one user device connected the server by a communication network. The computing device may map the semantic data of the source content, the metadata of the source content, and the user profile data to a knowledge graph. The computing device may create navigational panels for navigating the source content based on the knowledge graph, the navigational panels enabling non-linear navigation of the source content by a user. The computing device may provide the navigational panels to the user device by the server for navigation by the user. The computing device may monitor the user's interaction with the navigational panels on the user device and update the knowledge graph and user profile data with the user's interaction with the navigational panels and progression through the source content.

DETAILED DESCRIPTION

FIG. 1illustrates a non-linear video navigation system100, in accordance with an embodiment of the invention. In an example embodiment, non-linear video navigation system100includes a content device110, a user device120, and a server130interconnected via a network140.

In the example embodiment, the network140is embodied as the Internet, representing a worldwide collection of networks and gateways to support communications between devices connected to the Internet. The network140may include, for example, wired, wireless or fiber optic connections. In other embodiments, network140may be implemented as an intranet, a local area network (LAN), or a wide area network (WAN). In general, network140can be any combination of connections and protocols that will support communications between content device110, user device120, and server130.

Content device110may contain source content112. Content device110may be a desktop computer, a notebook, a laptop computer, a tablet computer, a handheld device, a smart-phone, a thin client, or any other electronic device or computing system capable of storing audio, visual, or textual content and receiving and sending that content to and from other computing devices such as user device120and server130via network140. In some embodiments, content device110includes a collection of devices, or data sources, in order to collect audiovisual content. Content device110is described in more detail with reference toFIG. 4.

Source content112is a collection of audiovisual content including, but not limited to, audio, visual, and textual content. Source content112may be, for example, educational videos, training videos, movies, and presentations, etc. Source content112is located on content device110that can be accessed via network140. In accordance with an embodiment of the invention, source content112may be located on one or a plurality of content devices110.

User device120may include user interface122, for example, a graphical user interface. User device120may be a desktop computer, a notebook, a laptop computer, a tablet computer, a handheld device, a smart-phone, a cellular phone, a landline phone, a thin client, or any other electronic device, computing system, wired or wireless device capable of receiving and sending content to and from other computing devices, such as content device110, and server130, via network140. User device120is described in more detail with reference toFIG. 4.

User interface122includes components used to receive input from a user and transmit the input to non-linear video navigation program134residing on server130, or conversely to receive information from non-linear video navigation program134and display the information to the user. In an example embodiment, user interface122uses a combination of technologies and devices, such as device drivers, to provide a platform to enable users of user device120to interact with non-linear video navigation program134. In the example embodiment, user interface122receives input, such as textual input received from a physical input device, such as a keyboard. In another embodiment, user interface122may receive input via tracking ocular movement of a user.

Server130includes non-linear video navigation program134and database132. In the example embodiment, Server130may be a desktop computer, a notebook, a laptop computer, a tablet computer, a thin client, or any other electronic device or computing system capable of storing compiling and organizing audio, visual, or textual content and receiving and sending that content to and from other computing devices, such as content device110and user device120, via network140. Server130is described in more detail with reference toFIG. 4.

Database132is a collection of source content112and source content metadata114. Metadata114stored on database132may include but is not limited to metadata describing video segments concepts, styles, transcripts, objects, complexity scores, segment timestamp boundaries, and OCR text, obtained from processing source content112by non-linear video navigation program134or manual processing. Further, database132may also contain user profile data136and knowledge graphs. User profile data136may include any information about the user including, but not limited to the user's age, gender, education, learning preferences, and learning proficiency, etc. The knowledge graphs are data graphs comprised of nodes, i.e. different topics, and flows, i.e. relationships, between the nodes. The nodes and flows provide a map of the relationships between various pieces of data, such as, but not limited to, user profile data136and video content metadata. For example, an educational video may contain two topics, each having various subtopics. The knowledge graph may map each subtopic to a node and then identify the relationships, i.e. flows, between each subtopic. Further, the knowledge graph may be updated to include user profile data136and the video content metadata. For example, user profile data136may indicate a proficiency in one of the two topics presented in the video content. Thus, the knowledge graph may be updated to reflect what video content may be of interest to the user based on the user's proficiency. Knowledge graphs are described in more detail with reference toFIG. 3.

Non-linear video navigation program134is a program capable of enabling personalized, i.e. non-linear, navigation of source content112that has been collected from content device110and stored on database132on user device120. Further, the non-linear navigation of source content112may be based on user profile data136and knowledge graphs stored on database132.

Referring toFIG. 2, a method200for non-linear video navigation, in accordance with an embodiment of the invention includes collecting source content at step S210, non-linear video navigation program134collects source content112from content device110. Non-linear video navigation program134may collect source content112from one or more sources, or from one or more devices. For example, non-linear video navigation program134may collect source content112from one or more video databases including, but not limited to, Massive Open Online Course (MOOC) databases, online college course databases, educational video databases, professional training and education video databases, etc. In an alternative embodiment, source content112may be collected by server130and stored in database132and non-linear video navigation program134receives source content112from database132.

Referring to step S216, non-linear video navigation program134receives user profile data136. User profile data136may be inputted via a user interface122on a user device120by a user. User profile data136may include, but is not limited to, age, gender, education, and learning preferences, learning proficiency, etc.

Referring to step S218non-linear video navigation program134receives the user's source content selection. The user may select a specific video from source content112or the user may enter a general topic into a search function on user interface122and be presented with several videos to choose from. For example, a user may type in a search term such as “patents” into user interface122and non-linear video navigation program134may search all source content112stored in database132and present the user with a list of all source content112pertaining to “patents.”

Referring to step S220, non-linear video navigation program134maps the semantic data of source content112and user profile data136to a knowledge graph. Semantic data may include, but is not limited to, audio and textual data. Non-linear video navigation program134may use a variety of word embedding methods and algorithms including, but not limited to, methods and algorithms using recurrent neural networks (RNNs), long short term memory (LTSM), and/or gated recurrent networks (GRUs). Non-linear video navigation program134may utilize a neural network to map the semantic data to a knowledge graph including, but not limited to, word2vec and Global Vectors for Word Representation (GloVe) to embed the semantic data of source content112into vectors of real numbers. Word2vec may utilize either continuous bag-of-words (CBOW) architecture model or continuous skip-gram (C-SG) model architecture to embed the semantic data of source content112. CBOW enables the prediction of a word based its surrounding words. C-SG enables the prediction of surrounding words based on a single word. The knowledge graph may consist of nodes and flows between the nodes, where each node represents a different topic contained within source content112. For example, source content112may cover three distinct topics, non-linear video navigation program134would map the semantic data for each distinct topic to a single node. Non-linear video navigation program134may identify the relationships between the nodes based on the order or presentation within source content112. Alternatively, non-linear video navigation program134may identify the relationships between the nodes based on the semantic data mapping. Further, the knowledge graph includes the mapped semantic data of user profile data136; thus, the knowledge graph is specifically tailored to each individual user.

Referring to step S222, non-linear video navigation program134creates navigational panels on server130for source content112selected by the user based on the created knowledge graph. The navigational panels may include a table of contents of source content112, timestamps of source content112, a knowledge graph of concepts contained within source content112, concept categories of source content112, and details of the concepts contained within source content112. The navigational panels are created using the meta-data extracted from source content112selected by the user and the knowledge graph. The navigational panels may be created using, but not limited to, the user's experience, and the user's preferred presentation style. For example, a user may select an educational video from source content112covering sections 35 U.S.C. §§ 101, 102, and 103 of the United States Patent Laws. However, the user's profile may indicate proficiency in 35 U.S.C. § 101 of the United States Patent Laws. Thus, non-linear video navigation program134may customize the navigational panels to skip all content pertaining to section 101. In an alternate embodiment, non-linear video navigation program134may grey out or otherwise distinguish concepts already mastered as indicated in the user's profile data; e.g. section 35 U.S.C. § 101 of the United State Patent Laws in the example above. The navigational panels may be displayed to the user via user interface122and allow the user to interact with source content112in a customized fashion. The navigational panels of non-linear video navigation program134are described in more detail with reference toFIG. 3.

Referring to step S224, non-linear video navigation program134sends the non-linear video navigational panels to user device120via network140to be displayed to the user on user interface122.

Referring to step S226, non-linear video navigation program134monitors the user's actions on user device120and user interface122. Non-linear video navigation program134may monitor a user's actions through the user's interactions with user device120and user interface122. For example, non-linear video navigation program134may monitor the user's progression through source content112. In an alternate embodiment, non-linear video navigation program134may monitor the user's actions using intuitive interfaces towards accessing within a video using eye-tracker. For example, non-linear video navigation program134may utilize eye tracking interfaces to monitor a user's interaction with and navigation of source content112.

Referring to step S228, non-linear video navigation program134updates user profile data136and the knowledge graph based on the user's actions on user device120and user interface122. For example, a user may have watched and completed a first segment of source content112selected by the user, non-linear video navigation program134may update user profile data136indicating proficiency in the subject area of first segment, which in turn updates the knowledge graph indicating the user's proficiency in that subject.

FIG. 3illustrates an example knowledge graph138, in accordance with the invention. Knowledge graph138may be data graphs comprised of nodes312a-f, i.e. different topics, and flows, i.e. relationships, between the nodes. The nodes and flows provide a map of the relationships between various pieces of data, such as, but not limited to, user profile data136and video content metadata114.

For example, knowledge graph138may be a data graph of an educational video containing topics310a-f. Knowledge graph138may map each topic310to a node312and then identify the relationships, i.e. flows, between each subtopic. Further, knowledge graph138may include user profile data136and the metadata114relevant to each topic310.

FIG. 4illustrates an example navigational panel410of user interface122, in accordance with the invention. Navigational panel410may be divided into table of contents panel412, non-linear video display414, index panel416, knowledge graph panel418, concept categories panel430, and concept details panel434.

For example, source content112selected by the user may cover concepts420,422,424,426, and428. Non-linear video navigation program134may create navigational panel410having table of contents panel412showing the order in which concepts420-428appear in source content112selected by the user. In another embodiment, table of contents panel412may show time stamps and/or duration of each concept420-428.

Navigational panel410may have non-linear video display414, which displays to the user the selected portions of source content112selected by the user.

Navigational panel410may have index panel416. Index panel416may contain a list of terms associated with concepts420-428.

Navigational panel410may have knowledge graph panel418illustrating the mapped relationships between concepts420-428of source content112selected by the user. For example, concept420and concept422may be necessary to understand concept424. Further, concept424and concept426may be necessary to understand concept428. In an example embodiment of the invention, non-linear video navigation program134may determine that the user is proficient in, or has previously viewed, concepts420,422, and424based on user profile data136and customize navigational panel accordingly. In the example illustrated, non-linear video navigation program134may create navigational panel410so that the user would only see the portions of the source content112selected by the user associated with concept426, with all other concepts greyed out or otherwise separated. It should be appreciated that while concepts420-424, and428are greyed out, the user may still click on them via user interface122and navigate through those concepts.

Navigational panel410may have concept categories panel430illustrating how each concept420-428is structured within source content112selected by the user. Concept categories432may include, but are not limited to, an introduction, definitions, details, examples, and quizzes. For example, concept426of source content112selected by the user may contain concept categories432a-e. In an example embodiment of the invention, non-linear video navigation program134may determine that the user is proficient in, or has previously viewed, concepts categories432abased on user profile data136and customize navigational panel410accordingly. In the example illustrated, non-linear video navigation program134may create navigational panel410so that the user would only see the concept categories432of the source content112selected by the user not previously viewed or which the user is not proficient in, with all other concepts categories432are greyed out or otherwise separated. It should be appreciated that while concepts category432ais greyed out, the user may still click on concept category432avia user interface122and navigate through that concept category.

Navigational panel410may have concept details panel434illustrating the different presentation styles of source content112selected by the user. Source content112selected by the user may be presented in many ways including, but is not limited to, figures, written descriptions, examples, interactive questions, animations, etc. Concept details panel434allows a user to select their preferred presentation style436for source content112selected by the user so long as source content112selected by the user contains such presentation styling. For example, a user may indicate that their preferred presentation style436is interactive questions. Thus, non-linear video navigation program134may prioritize any interactive features of source content112selected by the user.

FIG. 5depicts a block diagram of components of content device110, user device120, and server130, in accordance with an illustrative embodiment of the present invention. It should be appreciated thatFIG. 4provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

Content device110, user device120, and server130may include communications fabric702, which provides communications between computer processor(s)704, memory706, persistent storage708, communications unit712, and input/output (I/O) interface(s)714. Communications fabric702can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, communications fabric702can be implemented with one or more buses.

The programs non-linear video navigation program134and database132in server130; source content112in content device110; and user interface122stored in user device120are stored in persistent storage708for execution by one or more of the respective computer processors704via one or more memories of memory706. In this embodiment, persistent storage708includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage708can include a solid state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage media that is capable of storing program instructions or digital information.

The media used by persistent storage708may also be removable. For example, a removable hard drive may be used for persistent storage708. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer onto another computer-readable storage medium that is also part of persistent storage708.

Communications unit712, in these examples, provides for communications with other data processing systems or devices. In these examples, communications unit712includes one or more network interface cards. Communications unit712may provide communications through the use of either or both physical and wireless communications links. The programs non-linear video navigation program134and database132in server130; source content112in content device110; and user interface122stored in user device120may be downloaded to persistent storage708through communications unit712.

I/O interface(s)714allows for input and output of data with other devices that may be connected to server130, content device110and user device120. For example, I/O interface714may provide a connection to external devices720such as a keyboard, keypad, a touch screen, and/or some other suitable input device. External devices720can also include portable computer-readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention, e.g., the programs non-linear video navigation program134and database132in server130; source content112in content device110; and user interface122stored in user device120, can be stored on such portable computer-readable storage media and can be loaded onto persistent storage708via I/O interface(s)714. I/O interface(s)714can also connect to a display722.

Display722provides a mechanism to display data to a user and may be, for example, a computer monitor.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

While steps of the disclosed method and components of the disclosed systems and environments have been sequentially or serially identified using numbers and letters, such numbering or lettering is not an indication that such steps must be performed in the order recited, and is merely provided to facilitate clear referencing of the method's steps. Furthermore, steps of the method may be performed in parallel to perform their described functionality.