Abstract:
A computer-implemented method involving an object-based editing environment for defining a digital video as a linear arrangement of discrete media objects, each digital media object electronically stored as a digital file. The method includes displaying a graphical representation of a first digital media object within the editing environment; displaying one or more user interface icons associated with the digital media object; displaying a first menu in response to receiving a first user input; receiving second user input associated with the first menu and identifying a second digital media object; displaying a graphical representation of the second digital media object within the editing environment; and generating a digital video comprising the first digital media object and the second digital media object.

Description:
CLAIM OF PRIORITY 
     This application is the U.S. national stage under 35 USC §371 of International Application Number PCT/US2012/050155, filed on Aug. 9, 2012 which claims the benefit under 35 USC §119(e) of U.S. Provisional Patent Application Ser. No. 61/521,624, filed on Aug. 9, 2011. The entire contents of each of the foregoing applications are incorporated by reference. 
    
    
     TECHNICAL FIELD 
     This specification generally relates to digital media editing particularly for mobile processing devices. 
     SUMMARY 
     In general, innovative aspects of the subject matter described in this specification can be embodied in methods that include the actions of providing an object-based editing environment for defining a digital video as a linear arrangement of discrete media objects, each digital media object electronically stored as a digital file, displaying a graphical representation of a first digital media object within the editing environment, displaying one or more user interface icons associated with the digital media object, receiving first user input, the first user input corresponding to a first interface icon of the one or more interface icons, in response to receiving the first user input, displaying a first menu, receiving second user input associated with the first menu, the second user input identifying a second digital media object, displaying a graphical representation of the second digital media object within the editing environment, and generating a digital video comprising the first digital media object and the second digital media object. Other embodiments of these aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices. 
     These and other embodiments may each optionally include one or more of the following features. For instance, the first digital media object comprises one of a first digital video, a first digital image, a first title and a first transition; the second digital media object includes one of a second digital video, a second digital image, a second title and a second transition, the first interface icon includes an insert icon indicating insertion of the second media object before or after the first digital media object; generating the digital video includes merging the first and second digital objects; the first menu comprises a plurality of option buttons, each option button corresponding to a different type of media object; the second user input includes a selection of an option button, the second media object being of a type of media object corresponding to the option button; the first menu is a contextual menu, the content of which is based on a type of the first interface icon; the first interface icon is an insertion icon; actions further include: receiving third user input, the third user input corresponding to a second interface icon of the one or more interface icons, and in response to receiving the third user input, displaying a second menu, the second menu is a contextual menu, the content of which is based on a type of the first media object; the first menu is displayed within the graphical representation of the first media object; and actions further include displaying an overview timeline, the overview timeline including thumbnail representations of each of the first and second media objects. 
     The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIGS. 1A and 1B  illustrate an example mobile processing device. 
         FIG. 2  depicts an exemplary process for editing digital media that can be performed by a mobile device application running on the mobile processing device of  FIGS. 1A and 1B . 
         FIGS. 3A-3F  depict the mobile processing device of  FIGS. 1A and 1B  running an example mobile device application. 
       Like reference numbers represent corresponding parts throughout. 
     
    
    
     DETAILED DESCRIPTION 
     Implementations of the present disclosure are generally directed to mobile device applications for enabling video editing to be performed on suitable mobile processing devices. Such mobile device applications can provide an object based environment where distinct digital media units (e.g., still images, video clips, audio clips, etc.) can be imported and edited. These mobile device applications can also be operable to merge the imported digital media units based on modifications achieved in the editing environment. 
       FIGS. 1A and 1B  illustrate an example mobile processing device  100 . In the depicted example, the mobile processing device  100  is provided as a smartphone. It is appreciated, however, implementations of the present disclosure can be realized using tablet computing devices, laptop computers, personal digital assistants (PDAs), mobile phones, or any one of many other known types of suitable mobile processing devices. As shown, the mobile processing device  100  incudes one or more processors  102 , memory  104  (e.g., computer readable media, read-only memory (ROM), random access memory (RAM), flash memory, a removable memory card (e.g., SD card), a hard disk, or any combination thereof), a transmitter  106 , a receiver  108 , a keypad  110  (e.g., a soft key), a display  112 , a user-interface (UI) module  114  (e.g., a graphical user interface module for touchscreen devices), speakers  116 , a microphone  118 , sensors  120 , and a battery  122 . In some examples, the processor  102  can control each component of the mobile processing device  100  and can access memory  104  to run applications  124 , and/or retrieve user data  126 , which can be stored in the memory. The transmitter  106  and the receiver  108  transmit and receive signals (e.g., via one or more antennas) to other communication devices, base stations, computers, satellites (e.g., GPS satellites), and/or wireless access points. The keypad  110 , the display  112 , the speaker  116 , and the microphone  118  are input/output devices for communicating with a user. The battery  120  provides power to appropriate components of the mobile processing device. 
     In some examples, the processor  102  can include a display manager  128  and an input manager  130 . The display manager  128  can be operable to manage displayed on the display  112 . In some implementations, an operating system running on the mobile device  102  can use the display manager  128  to arbitrate access to the display  112  for a number of applications  124  running on the mobile processing device  100 . In some examples, a number of applications  124  can run concurrently on the mobile processing device  100 , with each application being shown to the user in respective windows on the display  112 . In such examples, the display manager  128  can control which portions of each application  124  are shown on the display  112 . In some implementations, the display manager  128  can control the orientation (i.e., portrait orientation or landscape orientation) of the display  112 . In some examples, the input manager  130  can be operable control the handling of data received from a user via one or more input devices of the mobile processing device  100 . For example, the input manager  130  can determine which application  124  should be invoked as a result of user received input. In some implementations, the UI module  114  can coordinate with the input devices to a interpret user input. The interpreted input can be evaluated by the input manager  130  to determine which application  124  should be invoked. 
       FIG. 2  depicts an example process  200  for editing digital media that can be performed by one or more mobile device applications executed on the mobile processing device  100 . As shown, the process  200  includes obtaining a plurality of digital media units, at operation  202 . Digital media units can be obtained by any one of several suitable procedures. In some examples, a digital media unit can include a digital video file, a digital image file and/or a digital audio file. In some implementations, digital media units can be obtained by accessing the user data  126  stored in memory  104 . That is, digital video files, digital image files and/or a digital audio files can be stored in memory  104 . In some examples, the mobile device application can enable a user to utilize the input components to select one or more digital media units stored in memory  104 . In some examples, the mobile device application can copy the selected digital media units and store them in a suitable directory, or simply reference their location in memory  104 . In some implementations, digital media units can also be obtained through a communications link (e.g., a wired or wireless communications link). For example, the mobile device application can enable a user to utilize the input components of mobile processing device  100  to select specific media units for downloading from a remote server or a computing system (e.g., a personal computer communicatively coupled to mobile processing device  100 ). The downloaded digital media units can be stored in memory  104 . In some examples, digital media units can be created using and stored to the mobile processing device  100 . For example, a digital camera can be used to capture digital images and/or digital video and store them as digital files on the mobile processing device  100 . As another example, a microphone of the mobile processing device  100  can capture audio and store it in a digital audio file. 
     As shown, the process  200  also includes providing an editing environment, at operation  204 . In some examples, the editing environment is an object based editing environment. In such examples, obtained digital media units can be represented as discrete objects arranged in the editing environment. In some implementations, a number of the discrete objects are arranged linearly by the mobile device application in a prescribed or random order. In some implementations, the mobile device application can enable users to rearrange the objects within the editing environment by utilizing the input components of the mobile processing device  100 . In some examples, the digital media units can be arranged linearly, end-to-end, or in an overlapping manner by users. The mobile device application can also enable users to introduce additional objects to the editing environment. For example, objects representing additional digital media units can be introduced. In some implementations, objects in the editing environment can represent elements other than digital media units. For example, objects can represent effects or actions (such as transitions) to be implemented. Such objects can be imported manually by users, or automatically by the mobile device application. 
     As shown, the process  200  further includes merging the obtained plurality of digital media units to form a substantially continuous digital media unit, at operation  206 . For example individual video clips, audio clips, and/or or still images can be merged (merged still images may be provided in the form of a slide show where each image is displayed for a predetermined duration). In some implementations, the plurality of digital media units are stitched together piece-by-piece according to the arranged configuration of representative objects in the editing environment. Further, in some examples, where an object in the editing environment represents a transition effect, the resulting “merge” of the pre-input media and the post-input media for that transition can be created based on the type of transition selected. The amount of overlap can be defined by the duration property for that transition object. 
       FIGS. 3A-3F  depict the mobile processing device  100  running an example mobile device application (e.g., an application  124  stored in memory  104 ). As shown (for example in  FIGS. 3A-3C ), the mobile device application presents an editing environment  302  to users via the display  112  of the mobile processing device  100 . A number of discrete virtual objects  304  (e.g.,  304   a  and  304   b ) are disposed in a linear configuration within the editing environment  302 . In this example, the objects  304  represent distinct digital media units available from memory  104  of the mobile processing device  100 . In some implementations, the objects  304  are movable within editing environment  302 . Therefore, as described above, the objects  304  can be rearranged in any prescribed order by a user. For example, a “drag-and-drop” technique can be executed by a user to rearrange the objects  304 . 
     As shown, the mobile device application can provide a menu bar  306  including a number of selectable menu items. For example, selecting menu item  308  (“Projects”) enables a user to access previously saved projects (i.e., instances of editing environment  302 ), or to create new projects. Various projects can include representations of the same, or different, digital media units. Selecting menu item  310  enables the user to adjust any one of several default settings for the project (e.g., screen orientation, video or sound quality, etc.). Selecting menu item  312  enables the user to view a help file. The help file can include various resources such as troubleshooting techniques, software patch or version notes, etc. Selecting menu item  314  (“Preview”) enables the user to observe a preliminary version, or preview of a merged digital media unit (as described in greater detail below). In this example, the menu bar  306  also includes selectable toggle items  316   a  (“Video”) and  316   b  (“Audio”) for defining the type of digital media units that are displayed for editing. 
     In some implementations, the mobile device application can also provide an overview bar  318 . As shown, the overview bar  318  includes a number of respective selectable tiles  320  corresponding to each object  304  in the editing environment  302 . In some examples, selecting any one of tiles  320  simultaneously selects the corresponding object  304  for detailed display in the editing environment  302 . 
     In some examples, users can add representations of additional digital media units (or other content) to the editing environment  302 . Users can also modify digital media units that are already represented in the editing environment  302 . For example, and with reference to  FIG. 3B , when an existing object (e.g., object  304   a ) is selected and displayed, various selectable editing icons are displayed within the editing environment  302 . In the depicted embodiment, the editing icons include an insert icon  322   a , an insert icon  322   b  and an editing icon  324 . Selecting either of insertion icons  322   a ,  322   b  enables the user to insert a digital media unit into editing environment  302  before or after the displayed object, respectively. In some examples, a menu is displayed in response to user selection of an insertion icon  322   a ,  322   b , which menu includes object, transition and title options. The user can select object from the menu to insert select a digital media object for insertion. The user can select transition from the menu to select a transition for insertion. The user can select title from the menu to input a title for insertion. 
     Selecting editing item  324  enables the user to modify the digital media unit represented by the selected object. In some implementations, selecting editing item  324  allows the user to access one or more contextual editing options. For example, when a selected object is representative of a video clip, the mobile device application enables users to: add and/or remove frames, adjust a default playback speed, pan and/or zoom (e.g., via the Ken Burns effect or traditional cropping), etc. Similarly, when the selected object is representative of a still image, the mobile device application enables the user to: pan and/or zoom (e.g., via the Ken Burns effect or traditional cropping), add filters, etc. 
       FIG. 3C  shows a place holder for an additional object  326  to be imported into editing environment  302  and inserted between objects  304   a  and  304   b . For example, the depiction of  FIG. 3C  is provided in response to user selection of the insert icon  322   b  of  FIG. 3B , and making a selection from the displayed menu. In the depicted example, the object  326  can be representative of a transition effect to be inserted between the digital media units represented by the objects  304   a  and  304   b . As noted above, the object  326  can be inserted manually by a user utilizing editing items  322   a ,  322   b . In some examples, insertion of an object can be achieved automatically by the mobile device application. As shown, when the object  326  is selected, various selectable editing icons  322   a ,  322   b ,  324  are displayed within editing environment  302 . Selecting either of editing items  322   a ,  322   b  enables the user to import additional content (e.g., additional representative objects) into editing environment  302 . 
     Selecting editing icon  324  enables the user to modify a default transition effect represented by object  326 .  FIG. 3D  shows a menu of selectable items  332  for indicating a desired type of transition effect (e.g., “Default”, “Crossfade”, “Slide Left”, “Fall”, “Grow”, “Shrink”, etc.). The transition effect provides a visual transition between objects when the video is played. In some implementations, the transitions represented by selectable items  332  are organized in a library stored in memory  104  of mobile processing device  100 . In some examples, additional transitions can be added to the library (for example, additional transitions can be downloaded from a remote server or a computing system) and represented in the menu by additional selectable items. 
     As described above, selectable toggle items  316   a  (“Video”) and  316   b  (“Audio”) can define the type of digital media units that are displayed for editing.  FIG. 3E  shows editing environment  302  when item  316   b  is selected. In the depicted example, audio slots  328   a ,  328   b  are provided and recording slot  330  is provided. The audio slots  328   a ,  328   b  can each be populated with an audio object. In the depicted example, the audio slots  328   a ,  328   b  are populated by audio objects represented by audio representations  338   a ,  338   b , respectively, and the recording slot  330  is empty. In this example, audio representations  338   a ,  338   b  are provided in the form of linear segments aligned with a video timeline  340  (e.g., a frame-by-frame timeline). In some implementations, however, audio representations can be provided in the form of discrete objects that are movable within editing environment  302 . In some example, when an audio slot is empty, an insert icon is displayed. In response to user selection of the insert icon, an audio object selection menu can be displayed, which the user can navigate to select an audio object for populating the audio slot. 
     Audio editing icons  334   a ,  334   b ,  336  are displayed. The audio editing icons  334   a ,  334   b ,  336  and audio representations  338   a ,  338   b  overlay frames of the objects that make up the video. Selecting either of editing icons  334   a ,  334   b  enables the user to edit an audio track represented by the audio representation  338   a ,  338   b , respectively. In some examples, audio editing options include accelerating or decelerating the audio playback speed and/or setting start and end time points for the respective audio object relative to a timeline of the underlying video. Selecting icon  336  enables the user to record an audio file and import a representation of the recorded file into editing environment  302 . 
     As described above, selecting the appropriate menu item enables the user to observe a preliminary version of the merged digital media unit. The merged digital media unit can include a combination of the digital media units and effects embodied by the representations imported into editing environment  302 . In some implantations, a number of the digital media units and effects are combined in a chronological order prescribed by the arrangement of discrete objects in editing environment  302 .  FIG. 3F  shows a preview screen presented to users via display  112  of mobile processing device  100 . As shown, the preview screen includes a selectable media item  342 . Selecting the media item can allow users to observe the merged digital media unit. Additionally, in this example, menu bar  306  includes selectable menu item  344  (“Export”), in addition to the various other menu items described above. Selecting menu item  344  can allow users to export the merged digital media unit to a specified location in memory  104  of mobile processing device  100 . 
     Although described in the context of a mobile device application operating on a mobile processing device, it is appreciated that implementations of the present disclosure, and all of the functional operations provided herein, can be realized in any suitable type of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the present disclosure can also be realized as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. 
     A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network. 
     The processes and logic flows described in this disclosure can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). 
     Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. Elements of a computer can include a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio player, a Global Positioning System (GPS) receiver, to name just a few. Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. 
     To provide for interaction with a user, implementations of the present disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. 
     While this disclosure includes some specifics, these should not be construed as limitations on the scope of the disclosure or of what can be claimed, but rather as descriptions of features of example implementations of the disclosure. Certain features that are described in this disclosure in the context of separate implementations can also be provided in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be provided in multiple implementations separately or in any suitable sub-combination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination. 
     Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. 
     Thus, particular implementations of the present disclosure have been described. Other implementations are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. A number of implementations have been described. Nevertheless, it will be understood that various modifications can be made without departing from the spirit and scope of the disclosure. For example, various forms of the flows shown above can be used, with steps re-ordered, added, or removed. Accordingly, other implementations are within the scope of the following claims.