Elastic cloud video editing and multimedia search

Technologies for cloud-based media search and editing include a video editor configured to build a media query and associate the media query with a dynamic content slot of a media program. When generating video output based on the media program, the video editor transmits the media query to a cloud analytics server and receives 5 search results identifying one or more media clips produced by a number of mobile computing devices. The video editor may display a list of clips for selection by the user or may automatically include one of the clips in the output. The cloud analytics server transmits an acceptance policy defining criteria for acceptable media, based on the media query, to the mobile computing devices. The mobile computing devices configure 10 capture settings according to the acceptance policy and may display a user interface to assist the user in capturing acceptable media. Other embodiments are described and claimed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a national stage entry under 35 USC §371(b) of International Application No. PCT/US2013/041865, which was filed May 20, 2013.

BACKGROUND

Video production is typically laborious and time-consuming. When assembling a video program from a collection of video clips, a video producer must organize a large number of video clips, evaluate each video clip for content, and arrange the selected video clips as desired. Particularly for live video production, typical systems require large numbers of human operators to incorporate large numbers of cameras or video sources into a single video production.

Cloud computing services allow for data processing and storage distributed over a ubiquitous network such as the Internet. Cloud computing may be scalable to grow or shrink available computing and storage resources according to current demand. Cloud computing services may collect and distribute content produced by numerous mobile devices with persistent network connections. Such “crowd-sourced” content may provide numerous unique, individualized views of the same event.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring now toFIG. 1, in one embodiment, a system100for cloud-enabled multimedia search and video editing includes a video editor102, a cloud analytics server104, and a number of mobile computing devices106, all in communication with each other over a network108. In use, as discussed in more detail below, the mobile computing devices106capture media data such as video clips, audio clips, or still images, and upload the media data to the cloud analytics server104. The cloud analytics server104processes the uploaded media data, producing a searchable database of media data and related metadata. The video editor102edits a media program including one or more dynamic content slots. The video editor102builds a media query associated with each dynamic content slot and submits the media queries to the cloud analytics server104. The media queries may include search terms based on media content and metadata, or may be based on similarity to a given sample media clip as discussed in more detail below. The video editor102receives search results from the cloud analytics server104and inserts the search results into the associated dynamic content slot when generating video output. Thus, the video editor102may allow for near-real-time search and incorporation of cloud-based media into a video production. Although illustrated as including one video editor102, in some embodiments the system100may include any number of video editors102that may produce independent video output (i.e., a many-to-many-to-many relationship).

The disclosed technologies may allow a video producer to incorporate crowd-sourced content into a video production without requiring cost-prohibitive or overwhelming manual review of submitted content. Incorporation of near-real-time crowd-sourced content into a video production may allow for inclusion of unique and diverse views of notable events that could otherwise go unnoticed. Those diverse views may increase the artistic, entertainment, and informative value of video productions.

The video editor102may be embodied as any type of computer device for performing the functions described herein. For example, the video editor102may be embodied as, without limitation, a computer, a desktop computer, a workstation, a laptop computer, a notebook computer, a tablet computer, a smartphone, a distributed computing system, a multiprocessor system, a consumer electronic device, and/or any other computing device configured to perform cloud-enabled video editing. As shown inFIG. 1, the video editor102includes a processor120, an input/output subsystem122, a memory124, a data storage device126, a communication circuit128, and a display130. Of course, the video editor102may include other or additional components, such as those commonly found in a computer workstation (e.g., various input/output devices), in other embodiments. Additionally, in some embodiments, one or more of the illustrative components may be incorporated in, or otherwise form a portion of, another component. For example, the memory124, or portions thereof, may be incorporated in the processor120in some embodiments.

The processor120may be embodied as any type of processor capable of performing the functions described herein. For example, the processor120may be embodied as a single or multi-core processor(s), digital signal processor, microcontroller, or other processor or processing/controlling circuit. Similarly, the memory124may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, the memory124may store various data and software used during operation of the video editor102such as operating systems, applications, programs, libraries, and drivers. The memory124is communicatively coupled to the processor120via the I/O subsystem122, which may be embodied as circuitry and/or components to facilitate input/output operations with the processor120, the memory124, and other components of the video editor102. For example, the I/O subsystem122may be embodied as, or otherwise include, memory controller hubs, input/output control hubs, firmware devices, communication links (i.e., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.) and/or other components and subsystems to facilitate the input/output operations. In some embodiments, the I/O subsystem122may form a portion of a system-on-a-chip (SoC) and be incorporated, along with the processor120, the memory124, and other components of the video editor102, on a single integrated circuit chip.

The data storage device126may be embodied as any type of device or devices configured for short-term or long-term storage of data such as, for example, memory devices and circuits, memory cards, hard disk drives, solid-state drives, or other data storage devices. The data storage device126may store, permanently or temporarily, media data received from the cloud analytics server104in response to media queries.

The communication circuit128of the video editor102may be embodied as any communication circuit, device, or collection thereof, capable of enabling communications between the video editor102, the cloud analytics server104, and/or other remote devices. The communication circuit128may be configured to use any one or more communication technology (e.g., wireless or wired communications) and associated protocols (e.g., Ethernet, Bluetooth®, Wi-Fi®, WiMAX, etc.) to effect such communication.

The display130of the video editor102may be embodied as any type of display capable of displaying digital information such as a liquid crystal display (LCD), a light emitting diode (LED), a plasma display, a cathode ray tube (CRT), or other type of display device. In some embodiments, the display130may be coupled with a touch screen to facilitate user interaction.

The cloud analytics server104is configured to provide services including sorting and processing media data uploaded by the mobile computing devices106. The cloud analytics server104may be embodied as any type of server computing device, or collection of devices, capable of performing the functions described herein. As such, the cloud analytics server104may be embodied as a single server computing device or a collection of servers and associated devices. For example, in some embodiments, the cloud analytics server104may be embodied as a “virtual server” formed from multiple computing devices distributed across the network108and operating in a public or private cloud. Accordingly, although the cloud analytics server104is illustrated inFIG. 1and described below as embodied as a single server computing device, it should be appreciated that the video editor102may be embodied as multiple devices cooperating together to facilitate the functionality described below.

The cloud analytics server104includes compute resources140and storage resources142that may be used to process and search media data. The compute resources140may be embodied as physical processors, servers, virtual machines, or any other physical or virtual computing resource capable of performing the functions described herein. The cloud analytics server104may allocate or de-allocate compute resources140based on workload or other factors. The storage resources142may be embodied as physical storage devices, virtual disks, network servers, or any other form of data storage that may be used by the compute resources140. In some embodiments, the storage resources142may be distributed among several geographical locations. Because the compute resources140and the storage resources142are scalable on demand, the cloud analytics server104may be known as an “elastic” cloud service provider.

The cloud analytics server104further includes media data144and metadata146. The media data144is a repository for media data uploaded by the mobile computing devices106. The metadata146is, a repository for data related to the media data144, for example, semantic data describing the content of the media data144, location data associated with the media data144, or technical information relating to the media data144such as the type of camera used to record the media data144, quality information, file format information, or the like. Metadata146may be embedded in the media data144, stored separately from the media data144, or some combination of the two. The media data144and/or the metadata146may be stored using the storage resources142of the cloud analytics server104, or may be stored in one or more data servers accessible by the cloud analytics server104over the network108. As described above, in some embodiments, the media data144and/or the metadata146may be distributed across data servers located in multiple geographic locations.

Each of the mobile computing devices106is configured to capture media data and transmit the captured media data to the cloud analytics server104. Each mobile computing device106may process captured media data prior to transmission to ensure the captured media data satisfies requirements of the video editor102. Additionally, each mobile computing device106may include a user interface to assist a user in capturing media data desired by the video editor102. Each mobile computing device106may be embodied as any type of mobile computing device capable of performing the functions described herein, including, without limitation, a computer, a smartphone, a tablet computer, a laptop computer, a notebook computer, a cellular telephone, a handset, a messaging device, a vehicle telematics device, a network appliance, a web appliance, a distributed computing system, a multiprocessor system, a processor-based system, and/or a consumer electronic device. Illustratively, a mobile computing device106includes a processor160, an I/O subsystem162, a memory164, a data storage166, a communication circuit168, a display170, and/or other components and devices commonly found in a smartphone or similar computing device. Those individual components of the mobile computing device106may be similar to the corresponding components of the video editor102, the description of which is applicable to the corresponding components of the mobile computing device106and is not repeated herein so as not to obscure the present disclosure.

The mobile computing device106further includes a camera172, a microphone174, and location circuitry176. The camera172may be embodied as a digital camera or other digital imaging device integrated with the mobile computing device106. The camera172includes an electronic image sensor, such as an active-pixel sensor (APS), e.g., a complementary metal-oxide-semiconductor (CMOS) sensor, or a charge-coupled device (CCD). The camera172may be capable of capturing still images and/or video. The microphone174includes a transducer to capture audible sound waves as sound information on the mobile computing device106. Instead of a microphone174, in other embodiments, the mobile computing device106may include any sensor capable of capturing audio signals, such as a line input jack, an analog-to-digital converter (ADC), or other type of audio sensor. The microphone174may be capable of capturing standalone audio or, in conjunction with the camera172, audio information for a video recording.

In the illustrative embodiment, the mobile computing device106includes location circuitry176. The location circuitry176may be embodied as any type of sensor capable of determining the precise or approximate position of the mobile computing device106. For example, the location circuitry176may be embodied as a global positioning system (“GPS”) receiver, capable of determining the precise coordinates of the mobile computing device106. In other embodiments, the location circuitry176may triangulate the position of the mobile computing device106using distances or angles to cellular network towers with known positions, provided by the communication circuit168. In other embodiments, the location circuitry176may determine the approximate position of the mobile computing device106based on association to wireless networks with known positions, using the communication circuit168.

As discussed in more detail below, the video editor102, the cloud analytics server104, and the mobile computing devices106are configured to transmit and receive data with the other devices of the system100over the network108. The network108may be embodied as any number of various wired and/or wireless networks. For example, the network108may be embodied as, or otherwise include, a wired or wireless local area network (LAN), a wired or wireless wide area network (WAN), and/or a publicly-accessible, global network such as the Internet. As such, the network108may include any number of additional devices, such as additional computers, routers, and switches, to facilitate communications among the devices of the system100.

Referring now toFIG. 2, in one embodiment, the video editor102establishes an environment200during operation. The illustrative environment200includes a query builder module202, a video editing module206, and a video output module208. The various modules of the environment200may be embodied as hardware, firmware, software, or a combination thereof.

The query builder module202is configured to build media queries based on selections received from a user of the video editor102. Each media query may be used to dynamically search for media information stored and/or indexed by the cloud analytics server104. Each media query may be embodied as a set of query parameters selected by the user, including metadata query parameters, media query parameters, and sample media query parameters, as described below. The media query may be submitted to the cloud analytics server104to be run immediately, or scheduled in advance to run at a later date and time. In some embodiments, those functions may be performed by sub-modules, for example, by a query scheduling module204.

The video editing module206is configured to manage and edit a media program including dynamic content slots. The video editing module206may provide a timeline interface to the media program, allowing the user to lay out pre-programmed media and dynamic content slots. The video editing module206associates media queries built by the query builder module202with dynamic content slots of the media program.

The video output module208is configured to render the media program produced by the video editing module206into a video output in a suitable format. The video output may include a video data file, a video data stream, or a video stream suitable for live broadcast. The video output module208incorporates media data received from the cloud analytics server104in response to the media query into the video output at the associated dynamic content slot.

Referring now toFIG. 3, in one embodiment, the cloud analytics server104establishes an environment300during operation. The illustrative environment300includes an analysis module302, a search module304, and a resource scheduling module306. The various modules of the environment300may be embodied as hardware, firmware, software, or a combination thereof.

The analysis module302is configured to receive media data144uploaded from the mobile computing devices106and analyze the media data144to allow for searching. The analysis module302may perform any analysis on the media data that may facilitate searching; for example, the analysis module302may perform facial detection, object detection, text recognition, color balance and saturation analysis, and audio tune analysis. The analysis module302may also evaluate the focus, stability, exposure, or lighting of the media data144, determine whether the media data144includes audio, and evaluate image data for motion. The analysis module302may store the results of analysis in the metadata146.

The search module304is configured to search the media data144based on a media query received from the video editor102and return the search results. As described above, the media query may be based on metadata query parameters, media query parameters, or sample media query parameters. The search module304may search against the media data144, the metadata146, or any combination of the two.

The resource scheduling module306is configured to schedule compute resources140and/or storage resources142based on requests received from the video editor102. For example, the resource scheduling module306may pre-allocate compute resources140and/or storage resources142in anticipation of receiving a large amount of uploaded media data near the scheduled date of a media query. Similarly, the resource scheduling module306may pre-allocate compute resources140and/or storage resources142that are geographically close to the scheduled location of the media query. In some embodiments, the resource scheduling module306may schedule resources automatically based on the media query.

Referring now toFIG. 4, in one embodiment, each mobile computing device106establishes an environment400during operation. The illustrative environment400includes a media acquisition module402and in some embodiments may include a metadata module410. The various modules of the environment400may be embodied as hardware, firmware, software, or a combination thereof.

The media acquisition module402is configured to capture media data144using sensors of the mobile computing device106such as the camera172and microphone174. The media acquisition module402is further configured to upload captured media data144to the cloud analytics server104. In some embodiments, the media acquisition module402may process the captured media data144prior to uploading, to ensure that an acceptance policy created by the video editor102has been satisfied. The media acquisition module402is further configured to present a user interface to facilitate capturing media data144. The user interface may provide instructions or other assistance to ensure captured media data144satisfies the acceptance policy of the video editor102. In some embodiments, those functions may be performed by sub-modules, for example, by a media capture module404, a media upload module406, or a user interface module408.

In some embodiments, the metadata module410is configured to embed metadata146into the captured media data144prior to upload. The metadata146may include technical attributes of the mobile computing device106or its components such as the camera172or the microphone174. The metadata146may include technical attributes of the captured media data144, such as exposure, focus, aperture, or shutter settings. The metadata146may include location information, for example GPS coordinates provided by the location circuitry176. In some embodiments, the metadata146may include information provided by other sensors of the mobile computing device106, for example, information on altitude, motion, temperature, compass direction, or spatial orientation. Additionally, in some embodiments, the metadata146may include metadata entered by the user, for example tags, labels, or other annotations.

Referring now toFIG. 5, in use, the video editor102may execute a method500for cloud-enabled video editing. The method500begins with block502, in which the video editor102builds a media query based on query parameters selected by the user of the video editor102. The video editor102may present a user interface to allow the user to configure the media query. For example, referring toFIG. 6, diagram600illustrates one embodiment of a media query builder602. The control groups604,606,608of the media query builder602illustrate potential query parameters that may be selected by the user.

Referring back toFIG. 5, in block504, in some embodiments, the video editor102receives metadata-based query parameters from the user. Metadata-based query parameters may be used to search indexed description data of the metadata146that is related to the media data144, but not necessarily embodied in the content of the media data144. For example, as illustrated in control group604ofFIG. 6, metadata-based queries may be based on search keywords, plaintext description, technical metadata attributes, a sentiment factor, location, or date and time. Search keywords and plaintext description may be associated manually with the media data144, or may be associated through automated analysis by the cloud analytics server104. Technical metadata attributes, for example white balance settings, frame rate, shutter settings, aperture settings, and the like, may be supplied by the mobile computing device106at the time of capture. The sentiment factor may be assigned algorithmically by the cloud analytics server104based on content analysis of the media data144, or may be derived from public input. Location and date/time information may also be supplied by the mobile computing device106at the time of capture.

Referring back toFIG. 5, in block506, in some embodiments, the video editor102receives content-based media query parameters from the user. Media query parameters may be used to search information derived through analysis of the media data144itself. Such analysis may be performed by the cloud analytics server104ahead of time, to improve search performance. For example, as illustrated in control group606ofFIG. 6, media queries may be based on focus, stability, lighting or exposure, audio, or motion. Focus and stability may be used to restrict the search to media clips that are in-focus and stable, that is, media clips captured without excessive camera shaking. Lighting and exposure may restrict the search to media clips captured under appropriate lighting conditions. As an example, audio may be used to return only media clips including an audio track. As another example, motion may be used to return only media clips with motion detected, for example media including physical gestures, media panning across a scene, or media including moving objects within a scene. As another example, media queries may be based on gestures detected within the media clips. Such media query parameters may be used by the video editor102to restrict the search to media clips having desirable quantities and to exclude media clips unsuitable for use in the output video.

In block508of the method500ofFIG. 5, in some embodiments; the video editor102receives a sample media query parameter and associated matching options from the user. The sample media may be used to search for similar media clips given a stock media clip as a reference. The cloud analytics server104may process the sample media similarly to the media data144, and use the processing results as a basis for comparison. For example, as illustrated in control group608ofFIG. 6, sample media queries may include options to match media clips based on facial recognition, object recognition, text recognition, color balance, color saturation, or audio tune. The video editor102may supply a sample media clip including an image of a person's face, e.g., a celebrity or newsworthy individual. Facial recognition may search for media clips also including that person's face. Similarly, the sample media clip may include images of a particular object or written text that may be the basis for a search. Color balance and color saturation matching may be used to search for media clips containing similar color balance and/or color saturation. Matching colors may allow the resulting media clips to fit in well with the rest of the media program.

Referring back toFIG. 5, in block510, in some embodiments, the video editor102may receive a scheduled date and/or location for the media query. As illustrated by control group616ofFIG. 6, in some embodiments the user of the video editor102may schedule the search for a future time, or may submit the search immediately. The scheduled date and/or location may correspond to a future broadcast or live performance of the media program. As described below, the scheduled date may be used to coordinate the compute resources140and/or the storage resources142of the cloud analytics server104. Additionally, in some embodiments, the location for the media query may be fine grained and may include orientation information, allowing selection of media from particular viewpoints of a large event. For example, the location may specify locations and viewing angles in a stadium used for a live sporting event. Thus, the media query may build a panoramic view of an event from multiple view points.

In block512of the method500ofFIG. 5, after building the media query, the video editor102associates the media query with a dynamic content slot in the media program. The dynamic content slot represents a portion of the media program that will be replaced with dynamic content as the video output is generated. For example, referring toFIG. 7, diagram700illustrates a media program702. The media program702includes three pre-programmed content segments704,708,712. Dynamic content slots706,710are positioned between the pre-programmed content segments704,708,712. Each dynamic content slot706,710has a duration and an associated media query. In a user interface of the video editor102, each dynamic content slot706,710may be represented with an icon derived from the media query, such as a thumbnail representation of a sample media clip. For example, when producing a news program on a hurricane, the content segment704may include a news anchor introduction. The dynamic content slot706may be associated with a media query based on the time and location of the hurricane, searching for severe weather and evacuation. Stock footage of a hurricane may be provided as a sample media clip. Content segment708may be a pre-recorded correspondent segment. Dynamic content slot710may include a media query based on the location of the hurricane, searching for themes of recovery and hope. Content segment712may include a news anchor conclusion.

Referring back toFIG. 5, in block514, the video editor102transmits the media query to the cloud analytics server104. As described above, the media query may be submitted for immediate evaluation and search, or may be submitted for evaluation and search at a scheduled time. In some embodiments, in block516, the video editor102may request scheduling of cloud resources of the cloud analytics server104based on the scheduled query. For example, the user of the video editor102may anticipate a large number of media submissions associated with the date and location of a live event. Accordingly, the video editor102may request that the cloud analytics server104pre-allocate compute resources140to be available at the date or location of the scheduled query. In such example, the cloud analytics server104may prepare additional virtual machines on the date of the scheduled query, or move virtual machines to a cloud datacenter near the location of the scheduled query. Additionally, or alternatively, the video editor102may request the cloud analytics server104to pre-allocate storage resources142based on the date and location of the scheduled query. Scheduling of cloud resources may be requested based on a user selection, for example, the checkbox614ofFIG. 6.

In some embodiments, in block518of the method500ofFIG. 5, the video editor102may request that the query parameters of the media query be pushed by the cloud analytics server104to the mobile computing devices106. Some or all of the query parameters may represent an acceptance policy of the video editor102; that is, the video editor102may only accept media clips satisfying those query parameters. In such embodiments, the mobile computing devices106may apply the query parameters of the acceptance policy to captured media clips prior to uploading to the cloud analytics server104. By applying query parameters before uploading media clips, the quality of uploaded media clips may be improved, and the processing required by the cloud analytics server104may be reduced. As illustrated by checkbox612ofFIG. 6, the video editor102may select to push all query parameters to the mobile computing devices106. In other embodiments, the video editor102may select a subset of the query parameters to be pushed to the mobile computing devices106; for example, media clip format and resolution requirements.

Some time after transmitting the media query, in block520the video editor102receives query results from the cloud analytics server104. The query results identify media clips generated by mobile computing devices106that satisfy the query parameters of the media query. The query results may be sorted in order of relevance, date created, or otherwise. In some embodiments, the query results may include a reference to matching media clips. In other embodiments, the query results may include the media data144for the matching media clips. In some embodiments, in block522, the video editor102may present the search results to the user through a user interface. The user may provide a selection indicating the desired media clip. For example, in some embodiments the video editor102may present a list of the top three to five search results, including thumbnails or other representations of the matching media clips, on the display130of the video editor102. The user may be able to quickly select a desired clip from among the search results. In other embodiments, rather than presenting the search results to the user, the video editor102may automatically select a search result without further direction from the user, for example, the best-matching search result. The video editor102may determine whether or not to display the search results based on a selection from the user, for example, as illustrated by radio button group610ofFIG. 6.

In block524, the video editor102outputs the selected media clip into the associated dynamic content slot of the media program. As described above, the selected media clip may be selected by the user through a user interface, or may be selected automatically. In block526, in some embodiments, the video editor102inserts the selected media clip into a video output generated based on the media program. The video editor102may render the video output to an output media file suitable for later playback. In some embodiments, the video editor102may render video output that is immediately or nearly immediately transmitted to a video device of a video consumer. For example, the video editor102may produce a live video stream that is broadcast over television, or streamed to video consumers over the network108. After generating the video output, the method500loops back to block502to continue building media queries.

Referring now toFIG. 8, in use, the cloud analytics server104may execute a method800for receiving and processing media queries from the video editor102. The method800begins with block802, in which the cloud analytics server104determines whether a media query has been received from the video editor102. If not, the method800loops back to continue waiting for media queries in block802. If so, the method800advances to block804.

In block804, the cloud analytics server104parses the media query received from the video editor102. The cloud analytics server104may interpret and evaluate each query parameter of the media query. In some embodiments, in block806, the cloud analytics server104may analyze a sample media clip supplied with the media query. The cloud analytics server104may perform the same analysis on the sample media clip as is performed on media clips submitted by the mobile computing devices106, as described below in connection withFIG. 9.

In some embodiments, in block808, the cloud analytics server104may transmit query parameters of the media query to the mobile computing devices106. For example, in some embodiments, the cloud analytics server104may maintain a list of mobile computing devices106and may push the query parameters to each. In some embodiments, the cloud analytics server104may store the query parameters and supply the query parameters to the mobile computing devices106when requested. Additionally, although illustrated as occurring in block808, in other embodiments the cloud analytics server104may provide the query parameters to the mobile computing devices106at other times.

After some time, in block810the cloud analytics server104searches the media data144based on the media query. The cloud analytics server104may perform such search using the metadata146, or by directly analyzing the media data144. In some embodiments, in block812, the cloud analytics server104searches the media data144based on metadata query parameters of the media query. For example, as described above in connection withFIG. 5, metadata query parameters may include search keywords, plaintext description, technical metadata attributes, a sentiment factor, location, or date and time. In some embodiments, in block814, the cloud analytics server104searches the media data144based on media query parameters. For example, as described above in connection withFIG. 5, the media query parameters may be based on focus, stability, lighting or exposure, audio, or motion of the media data144itself. In some embodiments, in block816, the cloud analytics server104searches the media data144based on sample media query parameters. For example, as described above in connection withFIG. 5, the sample media query parameters may include a sample media clip and matching options based on facial recognition, object recognition, text recognition, color balance, color saturation, or audio tune.

After searching the media data144, in block818, the cloud analytics server104ranks and sorts the search results. The cloud analytics server104may rank the search results based on the best match to the media query, the date of the media clip, or any other ranking criteria. In block820, the cloud analytics server104transmits the search results to the video editor102. In some embodiments, the cloud analytics server104may send a list of top-ranked search results. In other embodiments, the cloud analytics server104may send a single search result. In some embodiments, in block822the cloud analytics server104transmits the media data144of a selected media clip. As described above in connection withFIG. 5, the selected media clip may be the single top-ranked media clip, or may be a media clip selected by the user of the video editor102. After transmitting the search results, the method800loops back to block802to continue waiting for additional media queries.

Referring now toFIG. 9, in use, the cloud analytics server104may execute a method900for receiving and processing media clips from the mobile computing devices106. The method900begins with block902, in which the cloud analytics server104determines whether a media clip upload has been received from a mobile computing device106. If not, the method900loops back to continue waiting for media clip uploads in block902. If so, the method900advances to block904.

In some embodiments, in block904, the cloud analytics server104pre-processes media data144received from a mobile computing device106. In some embodiments, such pre-processing may be embodied as any analysis or evaluation of the media data that may be performed quickly and without using excessive compute resources140. For example, pre-processing may be embodied as evaluating metadata146or format data associated with the media data144. In such example, the cloud analytics server104may evaluate a source location associated with the media data144, a media format of the media data144, or a resolution of the media data144. In other embodiments, pre-processing may include evaluating capabilities of the mobile computing device106. For example, the cloud analytics server104may evaluate the processing capabilities or available network bandwidth of the mobile computing device106. Information on capabilities of the mobile computing device106may be received with the media data144, as described below in connection withFIG. 10.

In block906, the cloud analytics server104determines whether to reject the uploaded media data144. The cloud analytics server104may reject the media data144based on the results of the pre-processing of block904. Criteria for rejection may be based on one or more media queries that have been received from the video editor102. For example, the cloud analytics server104may reject media data144that is not from a particular source location, that is supplied in an incorrect format, or that does not have a required resolution. In some embodiments, the cloud analytics server104may reject media data144from mobile computing devices106that do not have desired capabilities; for example, mobile computing devices106with insufficient processing power or network bandwidth. Rejecting media data144prior to extensive processing may conserve compute resources140and/or reduce the amount of media data144and metadata146that must be stored in the storage resources142. If the cloud analytics server104determines to reject the media data144, the method900loops back to block902to continue waiting for media uploads. If the cloud analytics server104determines not to reject the media data144, the method900advances to block908.

In block908, the cloud analytics server104post-processes the uploaded media data144. Post-processing may include computing-resource-intensive calculations performed on the media data144. Such post-processing may be scheduled or distributed among the compute resources140of the cloud analytics server104. Results of post-processing may be indexed or stored in the metadata146, to improve search performance. In some embodiments, in block910the cloud analytics server104processes video image content of the uploaded media data144. That image analysis may be used to evaluate query parameters received from the video editor102. For example, the cloud analytics server104may analyze the media clip to determine whether it is in focus, determine whether the camera is stable, determine lighting or exposure information, or determine whether the media clip includes motion. The cloud analytics server104may further perform facial detection analysis, object detection analysis, optical character recognition and other text recognition analysis, and analysis of image color and saturation. In some embodiments, in block912, the cloud analytics server104analyzes the audio content of the uploaded media data144. Again, that audio analysis may be used to evaluate query parameters received from the video editor102. For example, the cloud analytics server104may determine whether the media data144includes audio information, and may perform audio tune analysis of the media data144. Audio tune analysis may include identifying musical tones in the audio data as well as determining how closely the audio data matches a musical tone. In some embodiments, in block914, the cloud analytics server104may process metadata146embedded in the uploaded media data144. That metadata analysis may be used to evaluate query parameters. For example, the cloud analytics server104may evaluate location coordinates embedded in the media data144or may evaluate technical attributes of the recording device that are embedded in the media data144.

In block916, the cloud analytics server104determines whether to reject the uploaded media data144. The cloud analytics server104may reject the media data144based on the results of the post-processing of block916, as well as any result from pre-processing of block904. As described above in connection with block906, criteria for rejection may be based on one or more media queries that have been received from the video editor102. The uploaded media data144may be rejected after post-processing to store only media data144that satisfies a media query, or to prevent storage of cumulative media data144. If the cloud analytics server104determines to reject the media data144, the method900loops back to block902to continue waiting for additional uploaded media data144. If the cloud analytics server104determines not to reject the media data144, the method900advances to block918.

In block918, the cloud analytics server104stores the uploaded media data144. As described above in connection withFIG. 5, storage resources142for the media data144may be pre-allocated based on a scheduled date and location of a media query. For example, storage resources142may be located in a datacenter geographically close to a scheduled live event. In some embodiments, in block920the cloud analytics server104stores a media retention policy associated with the media data144. The media retention policy may dictate how long the media data144is stored and when the media data144is deleted. For example, the media retention policy may dictate that the media data144be deleted some time after the scheduled date of a media query. Thus, the media retention policy may conserve storage resources142of the cloud analytics server104. In block922, the cloud analytics server104stores metadata146associated with the uploaded media data144. As described above, the metadata146may be searched to return associated media data144as a search result to the video editor102. After storing the media data144and the metadata146, the method900loops back to902to continue waiting for uploaded media data144.

Referring now toFIG. 10, in use, a mobile computing device106may execute a method1000for capturing media clips and uploading the media clips to the cloud analytics server104. The method1000begins with block1002, in which the mobile computing device106requests query parameters from the cloud analytics server104. The query parameters derive from the media query produced by the video editor102. The received query parameters thus represent an acceptance policy describing media data144that could be provided to the video editor102. Although illustrated as requesting the query parameters from the cloud analytics server104, in some embodiments, the cloud analytics server104may push query parameters to the mobile computing device106without a request. In some embodiments, in block1004, the mobile computing device106may transmit capabilities information to the cloud analytics server104. As described above, the capabilities information may include information describing the processing capability of the mobile computing device106or concerning available network bandwidth.

In block1006, the mobile computing device106determines whether query parameters have been received from the cloud analytics server104. If not, the method1000advances to block1012, as described below. If so, the method1000branches to block1008.

In block1008, the mobile computing device106configures media capture settings based on the query parameters. By configuring capture settings, the mobile computing device106may ensure that captured media data144is usable by the video editor102prior to capturing. For example, the mobile computing device106may configure media format, resolution, lighting, or exposure settings according to query parameters. Those query parameters are derived from the media query produced by the video editor102.

In some embodiments, in block1010the mobile computing device106displays a media capture assistance user interface based on the query parameters. The media capture assistance user interface may be embodied as words or symbols displayed on the display170of the mobile computing device106. For example, the mobile computing device106may display textual instructions based on the query parameters. Continuing that example, the query parameters may include a search for a person smiling and waving at the camera. In that example, the mobile computing device106may prompt the user to direct the subject to smile and wave at the camera. In another example, the query parameters may indicate that only in-focus, stable images will be accepted. In that example, the mobile computing device106may display text or symbols directing the user to correct focus or stability problems. After block1010, the method1000advances to block1012.

In block1012, the mobile computing device106captures media data144. The mobile computing device106may capture video clips, audio clips, still images, or any other media data144. Media capture may be initiated by a user interface interaction or automatically without further direction from the user.

In block1014, the mobile computing device106processes the captured media data144. Processing may be minimal, such as converting the captured media data144to an appropriate file format, or involved, such as performing content analysis such as facial recognition, object detection, audio tune analysis, or the like. The extent of processing performed by the mobile computing device106may depend on the capabilities of the mobile computing device106. In some embodiments, in block1016the mobile computing device106tags the captured media data144with metadata146. The metadata146may include technical information identifying the mobile computing device106or providing capture information such as white balance, exposure, aperture, image quality attributes, or similar. In some embodiments, the metadata146may include location information. For example, the metadata146may include GPS coordinates provided by the location circuitry176.

In block1018, the mobile computing device106determines whether to reject the captured media data144. The mobile computing device106applies the received query parameters to the captured media data144to determine whether the acceptance policy of the video editor102has been satisfied. In many embodiments, no captured media data144will be rejected, because the media capture settings were correctly set in block1008based on the query parameters. In some embodiments, the media data144may be rejected based on processing performed in block1016. For example, media data144may be rejected if facial recognition technology has not identified any faces in the captured media data144. Rejection of media data144by the mobile computing device106may reduce demand on the compute resources140of the cloud analytics server104. Of course, if no query parameters have been received by the mobile computing device106, no captured media data144may be rejected. If the mobile computing device106determines to reject the captured media data144, the method1000loops back to block1002to continue capturing media data144. If the mobile computing device106determines not to reject the captured media data144, the method1000advances to block1020. In block1020, the mobile computing device106uploads the captured media data144to the cloud analytics server104. After upload, the method1000loops back to block1002to continue capturing media data144.

EXAMPLES

Example 1 includes a video editor comprising a query builder module to build a media query including a query parameter selected by a user of the video editor; a video editing module to associate the media query with a dynamic content slot of a media program; transmit the media query to a cloud analytics service; and receive query results from the cloud analytics service in response to the media query, the query results to identify a media clip produced by a mobile computing device; and a video output module to generate a video output based on the media program, the video output to include the media clip identified in the query results inserted in the dynamic content slot.

Example 2 includes the subject matter of Example 1, and wherein the media clip comprises at least one of: a video clip, an audio clip, or a still image produced by the mobile computing device.

Example 3 includes the subject matter of any of Examples 1 and 2, and wherein to generate the video output comprises to automatically insert the media clip identified in the query results into the dynamic content slot without direction from the user.

Example 4 includes the subject matter of any of Examples 1-3, and wherein the video editing module is further to present a user interface including the query results; and receive a user selection of a selected media clip identified in the query results; wherein to generate the video output comprises to generate the video output including the selected media clip inserted in the dynamic content slot.

Example 5 includes the subject matter of any of Examples 1-4, and wherein the video editing module and the video output module are to receive the query results and to generate the video output contemporaneously with each other.

Example 6 includes the subject matter of any of Examples 1-5, and wherein to generate the video output comprises to transmit the video output to a video device of a video consumer.

Example 7 includes the subject matter of any of Examples 1-6, and wherein the query parameter comprises at least one of: (i) a metadata query parameter to search description data associated with the media data, (ii) a media query parameter to search content data derived from the media data, or (iii) a sample media query parameter based on a sample media clip selected by the user.

Example 8 includes the subject matter of any of Examples 1-7, and wherein the media query parameter comprises a media query parameter selected from: a keyword, embedded metadata produced by the mobile computing device, a sentiment factor, a location, or a date.

Example 9 includes the subject matter of any of Examples 1-8, and wherein the media query parameter comprises a media query parameter selected from: an in-focus requirement, a stability requirement, a lighting requirement, an audio information requirement, or a motion requirement.

Example 10 includes the subject matter of any of Examples 1-9, and wherein the sample media query parameter comprises a sample media query parameter selected from: a face detection matching option, an object detection matching option, a text detection matching option, a color matching option, or an audio tune matching option.

Example 11 includes the subject matter of any of Examples 1-10, and wherein to build the media query comprises to receive a scheduled output date for the media query from the user, and generation of the video output occurs at the scheduled output date.

Example 12 includes the subject matter of any of Examples 1-11, and wherein to transmit the media query to the cloud analytics service comprises to request scheduling of cloud resources based on the scheduled output date of the media query.

Example 13 includes the subject matter of any of Examples 1-12, and wherein to request scheduling of cloud resources comprises at least one of to request pre-allocation of cloud compute resources based on the scheduled output date of the media query; request pre-allocation of cloud storage resources based on the scheduled output date of the media query; request pre-allocation of cloud compute resources based on a location of the media query; or request pre-allocation of cloud storage resources based on a location of the media query.

Example 14 includes a cloud analytics server comprising a search module to receive a media query from a video editor, the media to include an acceptance policy that defines criteria of acceptable media data; and an analysis module to transmit the acceptance policy to a plurality of mobile computing devices; and receive media data transmitted from a mobile computing device in response to receiving the acceptance policy.

Example 15 includes the subject matter of Example 14, and wherein the media data comprises at least one of: video data, audio data, or still image data.

Example 16 includes the subject matter of any of Examples 14 and 15, and wherein to transmit the acceptance policy comprises to cause the mobile computing device to adjust media capture settings based on the acceptance policy.

Example 17 includes the subject matter of any of Examples 14-16, and wherein to transmit the acceptance policy further comprises to cause the mobile computing device to set a format of the media data based on a format parameter of the acceptance policy.

Example 18 includes the subject matter of any of Examples 14-17, and wherein to transmit the acceptance policy further comprises to cause the mobile computing device to set a resolution of the media data based on a resolution parameter of the acceptance policy.

Example 19 includes the subject matter of any of Examples 14-18, and wherein to transmit the acceptance policy comprises to cause the mobile computing device to display a capture assistance user interface based on the acceptance policy.

Example 20 includes the subject matter of any of Examples 14-19, and wherein to transmit the acceptance policy further comprises to cause the mobile computing device to display instructions that define the content of the media data based on the acceptance policy.

Example 21 includes the subject matter of any of Examples 14-20, and wherein to transmit the acceptance policy comprises to cause the mobile computing device to transmit the media data in response to the mobile computing device determining that the acceptance policy is satisfied.

Example 22 includes the subject matter of any of Examples 14-21, and wherein to transmit the acceptance policy further comprises to cause the mobile computing device to determine whether a capability of the mobile computing device satisfies a capability parameter of the acceptance policy.

Example 23 includes the subject matter of any of Examples 14-22, and wherein to determine whether a capability satisfies the capability parameter of the acceptance policy comprises at least one of to: (i) determine whether a processing capability of the mobile computing device satisfies the capability parameter, or (ii) determine whether a network connection quality of the mobile computing device satisfies the capability parameter.

Example 24 includes the subject matter of any of Examples 14-23, and wherein to determine that the acceptance policy is satisfied comprises, to determine, on the mobile computing device, whether the acceptance policy is satisfied based on at least one of: detection of faces represented in the media data, detection of objects represented in the media data, detection of text represented in the media data, or detection of an audio tune represented in the media data.

Example 25 includes the subject matter of any of Examples 14-24, and wherein the analysis module is further to process the media data; determine whether the media data satisfies the acceptance policy based on processing of the media data; and reject a portion of the media data in response to a determination that the portion of the media data does not satisfy the acceptance policy.

Example 26 includes the subject matter of any of Examples 14-25, and wherein to process the media data comprises at least one of to: detect faces represented in the media data, detect objects represented in the media data, detect text represented in the media data, or detect an audio tune represented in the media data.

Example 27 includes the subject matter of any of Examples 14-26, and, wherein the analysis module is further to determine whether the media data satisfies the acceptance policy prior to processing of the media data.

Example 28 includes the subject matter of any of Examples 14-27, and wherein to determine whether the media data satisfies the acceptance policy prior to processing of the media data comprises at least one of to determine whether metadata associated with the media data includes location data satisfying a location parameter of the acceptance policy; determine whether a format of the media data satisfies a format parameter of the acceptance policy, or determine whether capability information received from the mobile computing device satisfies a capability parameter of the acceptance policy, the capability information to define a media capture capability of the mobile computing device.

Example 29 includes the subject matter of any of Examples 14-28, and wherein to determine whether the format matches the format parameter comprises to determine whether a resolution of the media data satisfies a resolution parameter of the acceptance policy.

Example 30 includes the subject matter of any of Examples 14-29, and wherein the capability information comprises at least one of: information on a processing capability of the mobile computing device, or information on a network connection quality of the mobile computing device.

Example 31 includes a method for media editing, the method comprising building, on a video editor, a media query including a query parameter selected by a user of the video editor; associating, on the video editor, the media query with a dynamic content slot of a media program; transmitting the media query from the video editor to a cloud analytics service; receiving, on the video editor, query results from the cloud analytics service in response to the media query, the query results identifying a media clip produced by a mobile computing device; and generating, on the video editor, a video output based on the media program, the video output including the media clip identified in the query results inserted in the dynamic content slot.

Example 32 includes the subject matter of Example 31, and wherein generating the video output comprises automatically inserting the media clip identified in the query results into the dynamic content slot without direction from the user.

Example 33 includes the subject matter of any of Examples 31 and 32, and wherein receiving the query results and generating the video output occur contemporaneously with each other.

Example 34 includes the subject matter of any of Examples 31-33, and wherein generating the video output comprises transmitting the video output to a video device of a video consumer.

Example 35 includes the subject matter of any of Examples 31-34, and further including presenting, on the video editor, a user interface including the query results; and receiving, on the video editor, a user selection of a selected media clip identified in the query results; wherein generating the video output comprises generating the video output including the selected media clip inserted in the dynamic content slot.

Example 36 includes the subject matter of any of Examples 31-35, and wherein presenting the user interface including the query results and generating the video output occur contemporaneously with each other.

Example 37 includes the subject matter of any of Examples 31-36, and wherein generating the video output comprises transmitting the video output to a video device of a video consumer.

Example 38 includes the subject matter of any of Examples 31-37, and wherein building the media query including the query parameter comprises at least one of building a media query including a metadata query parameter to search description data associated with the media data; building a media query including a media query parameter to search content data derived from the media data; or building a sample media query parameter based on a sample media clip selected by the user.

Example 39 includes the subject matter of any of Examples 31-38, and wherein building the media query including the metadata query parameter comprises including a metadata query parameter selected from: a keyword, embedded metadata produced by the mobile computing device, a sentiment factor, a location, or a date.

Example 40 includes the subject matter of any of Examples 31-39, and wherein building the media query including the media query parameter comprises including a media query parameter selected from: an in-focus requirement, a stability requirement, a lighting requirement, an audio information requirement, or a motion requirement.

Example 41 includes the subject matter of any of Examples 31-40, and wherein building the media query including the query parameter further comprises including a sample media query parameter selected from: a face detection matching option, an object detection matching option, a text detection matching option, a color matching option, or an audio tune matching option.

Example 42 includes the subject matter of any of Examples 31-41, and wherein building the media query comprises receiving a scheduled output date for the media query from the user; and generating the video output occurs at the scheduled output date.

Example 43 includes the subject matter of any of Examples 31-42, and wherein transmitting the media query to the cloud analytics service comprises requesting scheduling of cloud resources based on the scheduled output date of the media query.

Example 44 includes the subject matter of any of Examples 31-43, and wherein requesting scheduling of cloud resources comprises at least one of requesting pre-allocation of cloud compute resources based on the scheduled output date of the media query; requesting pre-allocation of cloud storage resources based on the scheduled output date of the media query; requesting pre-allocation of cloud compute resources based on a location of the media query; or requesting pre-allocation of cloud storage resources based on a location of the media query.

Example 45 includes the subject matter of any of Examples 31-44, and wherein receiving the query results comprises receiving query results identifying at least one of: a video clip, an audio clip, or a still image produced by the mobile computing device.

Example 46 includes a method for collecting media captured by a plurality of mobile computing devices, the method comprising receiving, with a cloud analytics server from a video editor, a media query including an acceptance policy that defines criteria of acceptable media data; transmitting the acceptance policy from the cloud analytics server to the plurality of mobile computing devices; and receiving, with the cloud analytics server, media data transmitted from a mobile computing device in response to receiving the acceptance policy.

Example 47 includes the subject matter of Example 46, and wherein transmitting the acceptance policy comprises causing the mobile computing device to adjust media capture settings based on the acceptance policy.

Example 48 includes the subject matter of any of Examples 46 and 47, and wherein transmitting the acceptance policy further comprises causing the mobile computing device to set a format of the media data based on a format parameter of the acceptance policy.

Example 49 includes the subject matter of any of Examples 46-48, and wherein transmitting the acceptance policy further comprises causing the mobile computing device to set resolution of the media data based on a resolution parameter of the acceptance policy.

Example 50 includes the subject matter of any of Examples 46-49, and wherein transmitting the acceptance policy comprises causing the mobile computing device to display a capture assistance user interface based on the acceptance policy.

Example 51 includes the subject matter of any of Examples 46-50, and wherein transmitting the acceptance policy further comprises causing the mobile computing device to display instructions that define the content of the media data based on the acceptance policy.

Example 52 includes the subject matter of any of Examples 46-51, and wherein transmitting the acceptance policy comprises causing the mobile computing device to transmit the media data in response to the mobile computing device determining that the acceptance policy is satisfied.

Example 53 includes the subject matter of any of Examples 46-52, and wherein transmitting the acceptance policy further comprises causing the mobile computing device to determine whether a capability of the mobile computing device satisfies a capability parameter of the acceptance policy.

Example 54 includes the subject matter of any of Examples 46-53, and wherein to determine whether a capability satisfies the capability parameter of the acceptance policy comprises at least one of: (i) to determine whether a processing capability of the mobile computing device satisfies the capability parameter, or (ii) to determine whether a network connection quality of the mobile computing device satisfies the capability parameter.

Example 55 includes the subject matter of any of Examples 46-54, and wherein determining that the acceptance policy is satisfied comprises, determining, on the mobile computing device, whether the acceptance policy is satisfied based on at least one of: detecting faces represented in the media data, detecting objects represented in the media data, detecting text represented in the media data, or detecting an audio tune represented in the media data.

Example 56 includes the subject matter of any of Examples 46-55, and further including processing, with the cloud analytics server, the media data; determining, with the cloud analytics server, whether the media data satisfies the acceptance policy based on processing the media data; and rejecting, with the cloud analytics server, a portion of the media data in response to determining that the portion of the media data does not satisfy the acceptance policy.

Example 57 includes the subject matter of any of Examples 46-56, and wherein processing the media data comprises at least one of: detecting faces represented in the media data, detecting objects represented in the media data, detecting text represented in the media data, or detecting an audio tune represented in the media data.

Example 58 includes the subject matter of any of Examples 46-57, and further including determining, with the cloud analytics server, whether the media data satisfies the acceptance policy prior to processing the media data.

Example 59 includes the subject matter of any of Examples 46-58, and wherein determining whether the media data satisfies the acceptance policy prior to processing the media data comprises at least one of (i) determining whether metadata associated with the media data includes location data satisfying a location parameter of the acceptance policy; (ii) determining whether a format of the media data satisfies a format parameter of the acceptance policy; or (iii) receiving, on the cloud analytics server, capability information from the mobile computing device, the capability information defining a media capture capability of the mobile computing device, and determining whether the capability information satisfies a capability parameter of the acceptance policy.

Example 60 includes the subject matter of any of Examples 46-59, and wherein determining whether the format matches the format parameter comprises determining whether a resolution of the media data satisfies a resolution parameter of the acceptance policy.

Example 61 includes the subject matter of any of Examples 46-60, and wherein receiving the capability information comprises at least one of: (i) receiving information on a processing capability of the mobile computing device or (ii) receiving information on a network connection quality of the mobile computing device.

Example 62 includes the subject matter of any of Examples 46-61, and wherein receiving the media data comprises receiving at least one of: video data, audio data, or still image data.

Example 64 includes one or more machine readable storage media comprising a plurality of instructions stored thereon that in response to being executed result in a computing device performing the method of any of Examples 31-62.

Example 65 includes a computing device comprising means for performing the method of any of Examples 31-62.