Patent Publication Number: US-2020280599-A1

Title: Media platform for adding synchronized content to media with a duration

Description:
RELATED APPLICATIONS 
     This patent application is a continuation of U.S. patent application Ser. No. 16/000,546, filed on Jun. 5, 2018, entitled MEDIA PLATFORM FOR ADDING SYNCHRONIZED CONTENT TO MEDIA WITH A DURATION, which is a continuation of U.S. patent application Ser. No. 14/490,639, filed on Sep. 18, 2014, entitled MEDIA PLATFORM FOR ADDING SYNCHRONIZED CONTENT TO MEDIA WITH A DURATION, and patented as U.S. Pat. No. 9,998,518 on Jun. 12, 2018, all of which are hereby incorporated by reference in their entireties. 
    
    
     BACKGROUND 
     1. Field 
     This disclosure generally relates to the field of computing systems. More particularly, the disclosure relates to media players. 
     2. General Background 
     Current media systems are limited in their ability to allow authors to customize content for videos displayed in video players. For instance, an author may want to add content to an existing video so that a user may view a video with the additional content. That author currently has to be able to manually prepare code, e.g., html code, that would add such content with the video. 
     Such a manual coding process is cumbersome and tedious. The author has to prepare significant amounts of code to perform even simple tasks. Further, such an authoring process does not have a visual design component. As a result, the authoring process is more concentrated on coding rather than designing the visual appearance of a layout with a video and additional content. 
     The authoring process is also limited to authors that are familiar with coding. Therefore, potential authors are prevented from authoring layouts for videos and additional content. Thus, current media systems are not adequate for providing the ability to add content to media. 
     SUMMARY 
     In one embodiment, a video authoring system has an authoring computing device that generates a visual synchronization layout, via a graphical user interface without user-based inputting of computer executable code, for a video with a time duration and for content to be added to the video at one or more time-based positions within the video. The video authoring system also has a processor, in operable communication with the authoring computing device, that sends data associated with the visual synchronization layout to a remote server that automatically generates a manifest based on the data associated with the visual synchronization layout. The manifest is computer executable code that is executed by a media player at a computing device so that the media player plays the video in synchronization with the content. 
     In another embodiment, a computer program product is also provided. The computer program product includes a non-transitory computer usable device that stores a computer readable program. The computer readable program when executed on a computer causes the computer to perform the functionality of the video authoring system. In yet another embodiment, a process is provided to perform the functionality of the video authoring system. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The above-mentioned features of the present disclosure will become more apparent with reference to the following description taken in conjunction with the accompanying drawings wherein like reference numerals denote like elements and in which: 
         FIG. 1  illustrates a media configuration. 
         FIGS. 2A-2C  illustrate examples of screen displays that may be utilized during the authoring process at the author computing device illustrated in  FIG. 1 . 
         FIG. 2A  illustrates a media configuration display screen. 
         FIG. 2B  illustrates a popup configuration display screen that may be utilized by the author  102  to generate content synchronization data. 
         FIG. 2C  illustrates a popup generation display screen. 
         FIGS. 3A and 3B  illustrate screen displays displayed by the media player illustrated in  FIG. 1 . 
         FIG. 3A  illustrates a media player display screen. 
         FIG. 3B  illustrates a media player display screen that displays images of each popup. 
         FIG. 4  illustrates a content data synchronization configuration. 
         FIG. 5  illustrates a process that is utilized to provide content modification and interaction. 
     
    
    
     DETAILED DESCRIPTION 
     A media system may be utilized to add content to media with a time duration. The media system allows authors to visually generate a layout for both media with a time duration, e.g., video and/or audio, and additional content through a user interface. The authors are not required to have any understanding of coding. 
       FIG. 1  illustrates a media configuration  100 . The media configuration  100  includes an author computing device  104  for an author  102  and a user computing device  116  for a user  114 . The author  102  utilizes the author computing device  104  to author a layout for media with a time duration and additional content so that the user  114  may play media with the additional content at the user computing device  116 . For ease of illustration, the author computing device  104  and the user computing device  116  are illustrated as different devices for an author  102  that is distinct from a user  114 . If the author  102  and the user  114  are the same person, a single device may be utilized for the author computing device  104  and the user computing device  116 . For example, the author may want generate a preview of the layout or generate the layout for personal consumption. 
     The author computing device  104  may be a personal computer, laptop computer, smartphone, smartwatch, tablet device, other type of mobile computing device, etc. The author  102  accesses an authoring editor  106  to prepare a layout for media and additional content. In one embodiment, the authoring editor  106  is a set of code, e.g., a software application, that is stored on the author computing device  104 . For example, the author  102  utilizing a tablet device for the author computing device  104  may download a software application for the authoring editor  106  that is then stored on the author computing device  104 . As another example, the tablet device may have the authoring editor  106  preloaded. In another embodiment, the authoring editor  106  may be stored in a cloud computing environment and then accessed and rendered via a web browser on the author computing device  104 . 
     The media configuration  100  also has a content server  108  and a media server  122 . The content server  108  communicates with the author computing device  104  to store data for the layout generated by the author computing device  104 . The media server  122  stores the media, e.g., video and/or audio, to which content is being added. Therefore, distinct servers  108  and  122  are utilized to store each of the additional content and media. As a result, the media configuration  100  allows for authoring a layout independently of any particular media server or media player. 
     The authoring editor  104  receives media from the video server  122 . For example, the author  102  searches a website for a particular cooking video and then plays that cooking video within the authoring editor  106  to determine content synchronization data, i.e., times during the cooking video to place the additional content so that the additional content will appear in a synchronized manner at those times during playback of the media. For instance, the additional content may be a link to a website to purchase the cooking items that appear in the media at the times provided by the synchronization data. The additional content may be a variety of content, e.g., images, text, questions to the viewer, links to social media to share data, etc. The authoring editor  106  may be accessed from a cloud computing environment such as the content server  108  and rendered in a web browser on the author computing device  102 . 
     The author computing device  104  sends the content synchronization data to the content server  108 . The content server  108  has a content synchronization Application Programming Interface (“API”)  110  and a content database  112 . The content server  108  stores the content synchronization data in the content database  112 . For instance, the content server  108  may generate a new entry in the content database  112  for a new layout received from the author computing device  104 . The content server  108  may assign the new layout an identifier, e.g., a number. The content server  108  may then store content synchronization data for that layout identifier and any updates to the content synchronization data according to that layout identifier. Therefore, the author  102  may work on different layouts and store new or updated content synchronization data based upon the layout identifier in the content server database  112 . The content synchronization data may be stored until a request is received from the user computing device  116 . 
     The user computing device  106  includes a media API  118  and a media player  120 . The media player  120  may be downloaded from the content server  108  or another server, may be prestored on the computing device  106 , or may be accessed through a cloud computing environment such as the content server  108  and rendered on the user computing device  116 . The media player  120  is preprogrammed with logic or provided with logic that provides a link to the content server  108 . For instance, the media player  120  may be embedded in a website, e.g., a cooking store website. The user  114  utilizes the user computing device  116  to access the website through a web browser and render the media player  120  in the web browser. The media player has logic that is utilized to request the additional content data from the content server  108 . In one embodiment, the media player  120  stores a particular layout identifier and sends that layout identifier as part of the request. Upon receiving the request, the content server  108  retrieves the content synchronization data associated with the layout identifier from the content database  112 . The content server  108  then utilizes the content synchronization API  110  to automatically generate a manifest, e.g., code, based upon the content synchronization data. For instance, JavaScript Object Notation (“JSON”) may be utilized as the code for the manifest. The author  102  provides inputs through a user interface as to the additional content and the times for the additional content to be played in synchronization with the media. The content server  108  may provide the human readable content synchronization data to the content synchronization API  110  to generate a manifest in JSON. The content server  110  may then send the manifest to the media player  120  at the user computing device  116 . The media player  120  then automatically generates HTML code so that the additional content may be rendered by the media player  120 . 
     The user computing device  116  receives the media for playback from the media server  122 . Further, the user computing device  116  may utilize the media API  118  to obtain media time data from a media database  124  at the media server  122  that stores the media and associated time data. For instance, the media player  120  streams the media from the media server  122  for playback, obtains the additional content from the content server  108 , obtains the content synchronization data from the content server  108 , and obtains the current media time playback data from the media server  122 . Therefore, the media player  120  may render the layout determined by the author  102 , find out the current time of media playback from the media server  122 , and render the additional content based upon that current media playback time and the content synchronization data. The media player  120  may be embedded in a variety of contexts, e.g., websites, blogs, etc. 
       FIGS. 2A-2C  illustrate examples of screen displays that may be utilized during the authoring process at the author computing device  104  illustrated in  FIG. 1 .  FIG. 2A  illustrates a media configuration display screen  200 . The media configuration display screen  200  is utilized by the author  102  to determine the particular media upon which the author  102  wants to base a layout. For instance, the author  104  may input a URL link to a video at a media input field  201 . 
     The author  104  may then preview the media through a media player  202 . The media player  202  has a display screen  203 , a timeline indicium  204 , a play/pause indicium  205 , an audio volume indicium  206 , a timeline time indicium  207 , and a screen dimension adjustment indicium  208 . The timeline indicium  204  displays the time in the video at which the timeline indicium  204  is positioned. 
       FIG. 2B  illustrates a popup configuration display screen  220  that may be utilized by the author  102  to generate content synchronization data. The author  102  may preview the media in the media player  202  in the popup configuration display screen  202 . After reaching a position in the media at which the author wants to add content, the author  102  may generate a new popup by selecting a new popup indicium  221 . The author  102  may then utilize a timeline synchronization display  222  to add a popup  223  at the particular times that the author wants the popup synchronized with the media content. For instance, the author  102  may preview a cooking video and notice that a pair of cooking tongs are displayed for a duration from twenty seconds to thirty five seconds of the video. The author  102  may then utilizer a timeline slider, e.g., a cursor or other dragging feature, to drag the section of the timeline synchronization display  222  corresponding to the duration of twenty seconds to thirty five seconds. The popup  223  is then displayed in the timeline synchronization display  222  for that duration. The user  114  will then view the popup  223  during that time period in synchronization with playback of the media. For instance, the popup  223  may be a link to a webpage for product purchase of the pair of tongs. The user  114  may click on the popup  223  to obtain an expanded popup screen to get more information about the product and/or be redirected to a webpage at which the user  114  may purchase the product. 
     Although a single popup  223  has been illustrated for ease of illustration, the author  102  may insert a plurality of popups in the timeline synchronization display  222 . The plurality of popups may be at the same time, different times, and/or overlapping times. Further, the popups may be different, e.g., links to webpages, images, text, questions, etc. Therefore, the author  102  is able to visually add content to media in a synchronized manner by visually adding popups to a timeline without having to manually prepare any code. 
       FIG. 2C  illustrates a popup generation display screen  240 . The popup generation display screen  240  allows the author  102  to input information for the popup  223  illustrated in  FIG. 2B . For instance, the author  102  may input a URL link for product purchase at a product purchase URL input field  241 , a title at a title input field  242 , a popup image to be displayed for the popup  223  at a popup image field  243 , and a product description at a product description field  244 . 
     The author  102  may also provide click through requirements through various screen displays. For example, the author  102  may provide payment to an entity operating the content server  108  based upon a quantity of clicks of a particular popup. The author  102  may provide a budget input so that the content server  108  is aware of when the content synchronization data should or should not be provided depending upon a quantity of clicks and the budget input. Further, the author  102  may also provide scheduling requirements through various screen displays such that certain popups should or should not be available at certain times. The content server  108  is then aware of when to send or not send certain portions of the content synchronization data, e.g., the author  102  may only want a particular popup for merchandise to be displayed during a sale. 
       FIGS. 3A and 3B  illustrate screen displays displayed by the media player  120  illustrated in  FIG. 1 .  FIG. 3A  illustrates a media player display screen  300 . The media player display screen  300  displays the synchronized layout prepared by the author  102 . The media player display screen  300  displays the media player  120 . The media player  120  has a display screen  301 , a timeline indicium  302 , a play/pause indicium  303 , an audio volume indicium  304 , a timeline time indicium  305 , and a screen dimension adjustment indicium  306 . The timeline indicium  302  displays the time in the video at which the timeline indicium  302  is positioned. 
     A popup image display screen  307  displays a popup that is synchronized with a current time as provided for by the layout authored by the author  102  with the content synchronization data received from the manifest. After the duration for the popup image completes, a next popup may be displayed according to a subsequent time duration provided by the content synchronization data from the manifest. Although a single popup image display screen  307  is displayed for ease of illustration, multiple popup image displays screens  307  or multiple popups may be displayed in the popup image display screen  307  if the author  102  authored multiple popups for a particular time duration of the media. 
     In one embodiment, a plurality of popup timeline status displays may be utilized. As each popup is being displayed, a popup timeline indicium in a corresponding timeline status display may be displayed. The user  102  may click on the popup image display screen  307  to obtain an expanded view and obtain more information about the popup. 
       FIG. 3B  illustrates a media player display screen  350  that displays images of each popup. The user  102  may select from a plurality of popup images  352  to obtain an expanded view and obtain more information about the selected popup. 
       FIG. 4  illustrates a content data synchronization configuration  400 . In one embodiment, the content data synchronization configuration  400  is implemented utilizing a general purpose computer, e.g., a server computer, or any other hardware equivalents. Thus, the content data synchronization configuration  400  comprises a processor  402 , various input/output devices  404 , e.g., audio/video outputs and audio/video inputs, storage devices, including but not limited to, a tape drive, a floppy drive, a hard disk drive or a compact disk drive, a receiver, a transmitter, a speaker, a display, an image capturing sensor, e.g., those used in a digital still camera or digital video camera, a clock, an output port, a user input device such as a keyboard, a keypad, a mouse, and the like, or a microphone for capturing speech commands, a memory  406 , e.g., random access memory (“RAM”) and/or read only memory (ROM), a data storage device  408 , and content data synchronization code  410 . 
     The content data synchronization code  410  may be implemented as a set of computer readable instructions that may be utilized by the processor  402  in the memory  406  to perform various actions associated with content modification and interaction. The content data synchronization code  410  may be represented by one or more software applications, where the software is loaded from a storage medium, e.g., a magnetic or optical drive, diskette, or non-volatile memory, and operated by the processor  402  in the memory  406  of the computer. As such, the content data synchronization code  410  including associated data structures of the present disclosure may be stored on a computer readable medium, e.g., RAM memory, magnetic or optical drive or diskette and the like. As an example, the content data synchronization code  410  may be implemented as an application that is downloaded onto a smartphone or tablet device. 
     Alternatively, the content data synchronization code  410  may be implemented as one or more physical devices that are coupled to the processor  402 . The content data synchronization code  410  may be utilized to implement any of the configurations herein. 
     The content data synchronization configuration  400  may be implemented on a computing device. A computing device may be a server computer, personal computer, laptop, notebook, smartphone, smartwatch, tablet device, other type of mobile device, etc. 
       FIG. 5  illustrates a process  500  that is utilized to provide content data synchronization. At a process block  502 , the process  500  stores, at a content database, content synchronization data received from a graphical user interface. The content synchronization data has a location of media, a set of content, a content start time, and a content end time. The content start time is a time in the media at which play of the content is to start. The content end time is a time in the media at which the play of the content is to end. Further, at a process block  504 , the process  500  automatically generates, with a processor, a manifest for synchronizing the play of the content with play of the media according to an Application Programming Interface. 
     The processes described herein may be implemented in a general, multi-purpose or single purpose processor. Such a processor will execute instructions, either at the assembly, compiled or machine-level, to perform the processes. Those instructions can be written by one of ordinary skill in the art following the description of the figures corresponding to the processes and stored or transmitted on a computer readable medium such as a computer readable storage device. The instructions may also be created using source code or any other known computer-aided design tool. A computer readable medium may be any medium capable of carrying those instructions and include a CD-ROM, DVD, magnetic or other optical disc, tape, silicon memory, e.g., removable, non-removable, volatile or non-volatile, packetized or non-packetized data through wireline or wireless transmissions locally or remotely through a network. A computer is herein intended to include any device that has a general, multi-purpose or single purpose processor as described above. 
     It is understood that the processes, systems, apparatuses, and compute program products described herein may also be applied in other types of processes, systems, apparatuses, and computer program products. Those skilled in the art will appreciate that the various adaptations and modifications of the embodiments of the processes, systems, apparatuses, and compute program products described herein may be configured without departing from the scope and spirit of the present processes and systems. Therefore, it is to be understood that, within the scope of the appended claims, the present processes, systems, apparatuses, and compute program products may be practiced other than as specifically described herein.