Abstract:
The present invention provides an easy method for creating animations from drawings. A user utilizes a user interface of an electronic media to draw a first line, then to go back in the recording&#39;s timeline, and then to draw a second line, such that a playback of the recording shows at least some portion of the first and second lines being drawn simultaneously. This allows a user to easily create animations from drawings for the purpose of visualization, art, entertainment, or encoding of synchronized motion. The invention allows for various ways in which the computer can receive a user&#39;s drawing events, in which drawing events are associated with timelines to create a recording, in which drawing events are displayed, and in which the playback of drawing events is saved.

Description:
REFERENCE DATA 
       [0001]    The present application claims priority from Swiss patent application CH00599/10 filed Apr. 23, 2010, the contents whereof are hereby incorporated by reference in their entirety. 
       PREAMBLE 
       [0002]    The headings are for reference purposes only and do not affect in any way the meaning or interpretation. Whenever the words “include”, “including”, “includes”, “e.g.”, “for instance”, “for example”, “among others”, “such as” are used, they are deemed to be followed by the words “without limitation”. The use of “or” is not intended to be exclusive, unless expressively indicated otherwise. 
       TECHNICAL DOMAIN 
       [0003]    The invention generally relates to the field of electronic arts. More specifically, the present invention relates to a computer or a server with a function for inputting and displaying drawn graphics or images. 
       BRIEF SUMMARY OF THE INVENTION 
       [0004]    The present invention provides an easy method for creating animations from drawings by allowing to replay and edit consecutively drawn lines in a non-consecutive way for the purpose of art, or visualization or encoding of synchronized motion. This is achieved by providing a user with a method and means to directly manipulate a drawings timelines. 
       BACKGROUND OF THE INVENTION  
     Related Art 
       [0005]    The present invention relates to two fields: Digital drawing and digital animation. 
       Digital Drawing 
       [0006]    Digital drawing has become increasingly popular due to advances in hardware (e.g., touch-screen devices, drawing pads, high-resolution screens) and in software (e.g., browser-based applications, online portals). This has resulted in the formation of large online communities (e.g., deviantart.com, sketchfu.com, odopod.com) dedicated to the creation and sharing of digital drawings, with thousands of drawings per day. Current state-of-the art drawing tools such as Adobe Photoshop or those offered by the aforementioned websites feature many advanced tools (e.g., brushes, layers, filters) for the creation and manipulation of graphics. However, their focus is the creation of static images. The possibilities for creating moving graphics are restricted to applying functions to existing graphics (e.g., equation-based movement, morphing, interpolation), combining multiple individual graphics into an animation (e.g., animated GIFs), or playback of a recording of the graphic&#39;s creation. 
         [0007]    WO2008112847 discloses a creative tool and system that may be utilized for entertainment purposes by a user. The creative tool utilizes an electronic media to allow a user to draw one or more lines with a user interface on which a moving object such as a sled may ride after the completion of the drawing by the user. The creative tool and system employs a computer program that may allow the user to input a set of parameters using a free hand drawing tool that is then graphically displayed to the user after they have input the parameters. Additionally, the input parameters are subject to natural velocity and trajectory forces from the moving object along the graphical representation displayed to the user. Additionally, the moving object is central on the graphical display and will continue until the parameters are interrupted or velocity and trajectory provide deviation from the user&#39;s parameters. 
         [0008]    U.S. Pat. No. 6,373,490 discloses a graphical rendering system and method which stores generated and calculated path parameters along the entire user drawn path. These stored parameters are then editable or otherwise filtered at any point so as to create for the user different effects without the necessity of redrawing any portion of the path. Color changes or blends can be created along the length of the path or at any specific segment such as overdraws where the color is darker due to the overdraw. The system and method allow for changing of pen type and for giving different weights to certain of the stored parameters depending upon the currently selected pen type, color choice, light source shading, or desired rendered media. 
         [0009]    U.S. Pat. No. 7,116,844 discloses an invention relating to obtaining vector data for a hand-drawn curve having coordinate values input from a pointing device and a thickness and/or density in accordance with pen pressure values. In order to reduce the amount of data input from the pointing device and obtain vector data therefor in real time, the coordinate data and pen pressure data are extracted independently of each other so as to reduce the amount of data stored in a memory and used for plotting or display. 
       Digital Animation 
       [0010]    Digital animation, as described in “Encyclopedia of Animation Techniques” by Richard Taylor (2004) and “Computer animation: algorithms and techniques” by Rick Parent (2001), is historically based on frames, i.e. images with identical size displayed in a specific order and timing. Current state-of-the-art animation uses two main methods: Keyframe animation and motion capture. Keyframe animation relies on the definition of specific frames of special importance, with intermediate frames created in a separate step called “inbetweening” or “tweening” to give the appearance that a first keyframe evolves smoothly into a following keyframe. Traditionally inbetweening was performed using light tables, modern methods use computers to allow for easier drawing (using e.g., onion skinning) or automatic creation (using e.g., morphing software) of tweens. Motion capture uses a camera to record frames of the motion of people (e.g., a dancer) or objects (e.g., a paper airplane). The resulting frames are then post-processed using specialized software. Alternative techniques are based on applying functions to graphics as mentioned above. While current state-of-the-art tools such as Autodesk Max3D Studio, Canvas X, Adobe After Effects feature many ways of moving and animating graphics, they focus on the use of keyframes, which requires a complex, technical process not well suited for many users and not well suited for many applications such as those described below. 
         [0011]    US20070109304 discloses a method and system producing animations based on drawings. It proposes services to casual users, having little or no experience in creating animations or using animation software packages. One or more individuals create one or more drawings that will serve as the basis for the animation and a storyline for the characters and other elements in the drawings. The drawings and storyline are sent to an animation services system, along with optional audio selections. At the animation services system, the drawings are opened using image processing software, parsed into a number of logical subunits, and animated using image animation software. Sound effects, music and other audible components are then added to the animation. The completed animation is then provided to the individuals and optionally posted on a website accessible through the Internet. In another aspect, the invention includes a software system for receiving requests for animation services from clients, assigning those requests to animators, and providing the completed animations to the clients. 
         [0012]    US20040174365 discloses methods and systems that in accordance with the present invention provide computer animation in which drawing lines or marks are erased after a predetermined time interval. When the lines or marks are automatically erased after a set interval, animations may be created with relatively little effort. Methods and systems in accordance with the present invention provide an easy, intuitive way to create animations by recording drawing events in sequence and playing back these events at a defined rate while erasing old events after a predetermined interval. The persistence interval, which can be defined, for example, in time or by a function of the relative order the line was drawn (“line drawn index”), and the playback rate may be defined by the creator of the animation for each drawing event. The result of this process is a dynamically changing drawing which can be replayed as an animation. 
         [0013]    US20030016787 discloses a method and apparatus for producing an overall image from a number of partial images which are produced successively for common playback, a first partial image is produced and stored, and at least a portion of the first partial image, that adjoins a second partial image to be produced, is played back. A survey image is produced for the second partial image and this survey image is simultaneously played back with the first partial image, with overlapping regions adjoining each other. From this simultaneous playback, the correct orientation of the exposure region for the second partial image is selected, and the image acquisition device is correspondingly positioned and the second partial image is produced. The first and second partial images are then joined to form the overall image. 
         [0014]    A common problem with many related art solutions is the complexity of their usage and implementation, which makes them unsuitable for non-expert users, hinders their implementation on many hardware platforms (e.g., mobile devices) and hinders their implementation on many software platforms (e.g., browsers). 
         [0015]    Another common problem with many related art solutions is that they are based on frames, or require the definition of keyframes, or require the definition of animation variables (“Avars”). All of these require an additional, intermediate, ad-hoc step of separating the timeline into intervals, resulting in additional workload, increased complexity of usage and implementation, and reduced applicability. 
         [0016]    Another common problem with many related art solutions is that they require user input to help define a function (e.g., morphing, scanimate, interpolation, transformation, orientation, or kinematic modeling) to alter or animate a drawing, requiring an additional step and increasing complexity. 
         [0017]    Another common problem with many related art solutions is that they require input to be presented in a serial format. 
         [0018]    Another common problem with many related art solutions is that they are limited to the animation of objects. This hinders or prevents their applicability to drawing, writing or painting. 
         [0019]    One aim of the present invention is thus to provide a user-friendly and easy-to-implement way of producing animations based on drawing simple shapes such as lines in combination with a way to access the drawing&#39;s timelines. 
         [0020]    Another aim of the present invention is to allow the production of animations without the use of frames in general, and in particular without the constraints and effort imposed by the use of keyframes. 
         [0021]    Another aim of the present invention is to allow animation without user definition of a function. 
         [0022]    Another aim of the present invention is to provide an easy method for input to be presented in parallel. 
         [0023]    Another aim for the present invention is to provide an easy method applicable to objects, shapes, drawings, writing and paintings. 
         [0024]    Another aim of the present invention is to allow users to produce animations based on drawing simple shapes such as lines in combination with a way to access another operation, linked to or separate from, the drawing&#39;s timeline, such as a zoom, rotation or shift operation. 
       BRIEF DESCRIPTION 
       [0025]    The present invention utilizes a user interface of an electronic media to allow a user to input and record information by drawing a first line, then to go back in the timeline of the recorded input, and then to input additional information by drawing a second line, such that a playback of the two timelines shows at least some portion of both lines being drawn simultaneously. This provides an easy method for a user to create animations from drawings for the purpose of visualization, art, entertainment, or encoding of synchronized motion. 
         [0026]    It is another object of the present invention to provide means to link timelines to processing rules for inputting, recording, altering and playing back of an animation. 
         [0027]    It is another object of the present invention to provide means for control of the timelines. In particular it allows to use the starting or end points in the timelines of drawing events to switch between modes of advancing through the timelines, or to jump to defined positions in the timelines. 
         [0028]    In order to achieve the above, the present invention allows for various ways of inputting, recording, altering and playing back the relation of drawing events with their associated timelines. In particular, the present invention allows for recording, alteration and playback while advancing through the timelines at different speeds, forwards and backwards, as well as pausing the progression in the timelines. 
         [0029]    In order to achieve the above, the present invention also allows for various types of drawing events including objects, letters, symbols and lines of unitary length (points). 
         [0030]    The present invention may be used as an enhancement and in combination with tools known from the prior art for the inputting, recording, altering and playing back of drawings and animations. In particular, it allows for various ways of controlling the timelines and of easy input, recording, altering and playback of sub-sections of drawing events in space or time. In particular, the present invention allows to use the timelines of drawing events to link drawing events created at different times, by different users in different locations in both time (e.g., one after the other, partially overlapping, etc.) and space (e.g., one next to the other, partially overlapping, etc.). 
     
    
     
       DESCRIPTION OF THE FIGURES 
         [0031]      FIG. 1  shows a screenshot of a sample embodiment of the application with important elements for user interaction. 
           [0032]      FIG. 2  shows a screenshot of a sample embodiment of an element for user interaction that allows to change the drawing&#39;s playback speed. 
           [0033]      FIG. 3  shows a screenshot of a sample embodiment of an element that allows a user to share their animation with other users via popular online networks. 
           [0034]      FIG. 4  shows a screenshot of another, different sample embodiment of the drawing application with important elements for user interaction. 
           [0035]      FIGS. 5 and 6  show two screenshots of another, different sample embodiment of the drawing application, in which the scrollbar is used to control a zoom function instead of the drawings timeline. 
       
    
    
     DETAILED DESCRIPTION 
       [0036]    In a first, preferred embodiment intended for an application and described with reference to  FIG. 1 , the present invention allows a sports team coach to quickly and easily sketch the movement of players  101  and a ball  102  on her mobile touch screen device. The mobile touch screen device includes a programmable processor and has access to a memory area containing a software program to execute the inventive method when they are executed by the processor. Furthermore, the mobile touch screen device includes a touch screen, or other input means, arranged to receive indications of drawing events or other commands, via a suitable user interface. The invention is suitable for mobile or portable devices, like tablets, smartphones, videogame consoles and so on, and is described herewith with reference to portable devices for simplicity, but it not limited to those devices and applies as well to personal computers or any suitable programmable device. It must be understood also that the inventive steps need not be executed on a single device, but that the invention can be implemented in a distributed environment, for example as a web service, that is executed in one or several servers, receiving indications of drawing events or other instructions from remote users via the internet. 
         [0037]    The inventive method proceeds by the coach drawing a first line to illustrate the movement of player  1  on a canvas representing the playing field  103 ; the mobile touch screen device receives indications of this first event, and this event is associated with a first timeline. The coach then uses a scrollbar  104  to return to the beginning of the animation, and draws a second line to illustrate the movement of player  2 ; the mobile touch screen devices receives indications of this second drawing event, which is then associated with a second timeline. Then again using the scrollbar  104  to return to an earlier moment in the animation, the coach draws a third line illustrating the movement of the ball  102  between players  101 , which is associated with a further timeline. The three timelines are associated with one another in a relation of synchronicity, according to the movements of the scrollbar  104 . The coach can then play back  105  the complete animation to show the three timelines of the players  101  and the ball  102  simultaneously according to the association of the timelines. In this first, preferred embodiment recording of subsequent timelines happens during replay of previous timelines. In addition to lines, the coach can also easily add and remove other moving objects (e.g., players of the opposing team)  106  or static objects (e.g., the goal)  107  or static drawings  108  to the animation. 
         [0038]    This preferred embodiment also has provisions to easily alter an animation using simple actions such as drag-and-drop, copy-paste, scaling, or distortion operations. For example, the coach can edit lines by using the scrollbar  104  to move to a specific point in time, then clicking on the end point of a line to select the current position of a player  101 , and then redraw the portion of the timeline she wants to edit. As another example, the coach can edit the same line by clicking the line to select it and then use a slider  403 , visible on  FIG. 4 , to increase or decrease its speed. As other examples, the coach can alter starting points, end points, speeds, and accelerations of lines or objects or of their sub-elements, resulting in gradual or instantaneous changes in the final animation. In all cases, the user&#39;s operations will result in a re-association of lines and objects with the timeline. 
         [0039]    This preferred embodiment also has provisions to easily alter the playback speed of the animation  201 . For example, the coach can use the interface element represented in  FIG. 2  to select a desired total duration for her animation, which will result in a re-association of all graphic elements with the timeline so that playback of the entire animation occurs in the desired time interval. This preferred embodiment also has provisions to share the animation via a network  301 . For example, the coach can use the interface element illustrated by  FIG. 3  to upload her animation to “YouTube” or other video sharing services. 
         [0040]    In an exemplary embodiment, the association of the timelines of the drawing events is such that recording, alteration and playback can be made while advancing at different speeds, forwards and backwards, as well as paused and stopped, through the animation&#39;s timeline or the timelines of its individual elements. For example, drawing events input while a “Pause” key  105  is pressed may be set to appear instantly in the animation. Drawing events input while a slow movement speed is selected may appear rapidly during the animation. Drawing events input while a fast movement speed  105  is selected may appear slowly during the animation. Drawing events input while a reverse movement speed  105  is selected will appear in reverse during the animation. Using methods as known from the prior art from selecting sub-portions of drawings in space (e.g. layers, selections, crop areas, groups, etc.), specific sub-portions of the animation may be edited independently in both space and time. 
         [0041]    In an exemplary embodiment, the present invention has provisions for altering the speed at which time passes while drawing (e.g., as a function of the pressure of the pen, as a function of drawing speed, as a function of user input for example through a second scrollbar  403 , ( FIG. 4 ), etc.). 
         [0042]    As a simple example, animation time may only advance when a pen is touching the drawing area and stand still otherwise. As a slightly more complex example, animation time may advance as a logarithmic function of the users drawing speed. As yet another example, animation time may advance as a spatial function of the user input, such that time advances slowly when drawing close to the center but more quickly when drawing at the borders of the image. 
         [0043]    In an exemplary embodiment, illustrated by  FIGS. 5 and 6 , movement in time is linked to a predefined processing rule for information display. For example, each point in time may be linked with a predefined zoom level, such that playback of a drawn circle will result in an animation of flying into a spiral and playback of a line drawn radially from the zoom center at the right speed will appear as a point in the animation. Other examples of predefined processing rules include rotation, panning/shifting/drift operations, mirroring, or other more complex operations and transformations of the canvas described by a function or algorithm. 
         [0044]    For example, one such processing rule may make portions of lines disappear 5 seconds after their starting point in the animation, 5 seconds before their end point, or after 5 seconds in the animation&#39;s timeline. 
         [0045]    In another exemplary embodiment intended for entertainment, the present invention is used in the fashion of computer-implemented online or offline games. In this embodiment the timeline of one animation may be linked to that of another such that both animations are shown in parallel or in sequence both, in time or in space. For example, to create a challenging and entertaining way of drawing an animation, a user may be given the end point of another user&#39;s animation as a starting point for her animation. The user would then create her own animation and mark it as completed, upon which another, different user would be given the new end point as a starting point. This allows multiple users to collaboratively create a single animated video sequence with smooth transitions between their individual contributions. Other additional constraints, such as using a color palette with all except one of the previous&#39; animation&#39;s colors, may be imposed. 
         [0046]    As another, related example, a user may be given another user&#39;s animation as a part of their animation (e.g., as the left half of their animation or as a square at the center). The user would then create her own animation in such a way that it connects to the previous animation in space and in time (e.g., such that lines drawn in the previous animation continue in the user&#39;s animation). 
         [0047]    As another example of creating a challenging and entertaining way of drawing an animation, the timeline may be linked to an operation in space. For example, the timeline may be linked to a zoom operation, with each point in time linked to a specific zoom level. In this example, a pen placed at the exact center of the drawing area will start the timeline and result in an animation of an expanding filled circle, and a pen placed next to the center of the drawing area will result in a radial line. 
         [0048]    As another example, the timeline may be linked to an operation in time. For example, the timeline may be entirely controlled by a predefined processing rule. In this example, a user may be presented with a continuously running 10 second long animation that will automatically loop back to its beginning. In this example, a pen placed at the exact center of the drawing area for 5 seconds will result in a dot that appears and disappears in a 5 second interval. 
         [0049]    Adaptation and combination of these examples gives rise to a large number of possibilities, where drawing events are associated with timelines by various functions. 
         [0050]    With reference to  FIG. 4 , in another exemplary embodiment intended for the creation of art, the present invention allows an artist to create an animated drawing by enhancing the typical features of modern drawing applications (color selector, color picker, undo, redo, layers, brushes, erasers  109  (visible on  FIG. 1 ), and the like  401 ) with a way to manipulate the animation&#39;s timeline  402 . 
         [0051]    In this embodiment, the enhancement is introduced as a slider  402 ,  102 , with buttons  105  (visible in  FIG. 1 ). This embodiment also has an additional option to quickly returning to a set point in time by either setting and activating a set-point using a double-click on the slider  402 ,  102 , or by use of a special button at the end point of the slider  402 , or by use of a shortcut key or any suitable interface element. Optionally, this set point in time may be defined as the starting point of the last line drawn and, optionally, programmed to return to this point upon reaching the end point of the last line drawn. This last option would, for example, allow a user to easily create an animation of growing grass by drawing many individual lines representing individual blades of grass, which would then automatically appear to grow in parallel in the animation. 
         [0052]    In an exemplary embodiment, the invention includes a method to project the entire animation onto one image (i.e., collapsing the entire timeline to a single point in time), such that the drawings created at predefined zoom levels will result in a single drawing with multiple levels of detail. 
         [0053]    In an exemplary embodiment, the user interface is a touch screen, drawing tablet, keyboard, mouse, joystick, or similar device. 
         [0054]    In an exemplary embodiment, the input information is provided by the user via the movement of a curser, an object such as a ball, the motion of a screen character, or the definition of a rule or function. For example, in a first step a user could be asked to draw a landscape. In a second step, a ball rolling down a predefined landscape governed by the functions describing the laws of Physics could provide additional drawing events for the animation. In another, related example, input information is a function of the timing of another animation. 
         [0055]    In an exemplary embodiment, the input information is provided by a combination of user input and a predetermined rule. For example, a user may input a drawing event which will be associated to a timeline. A predetermined rule will move segments towards the bottom of the image depending on their age, such that lines continuously drift towards the bottom throughout the animation, such that a series of dots drawn at the top of the canvas throughout the animation will result in an animation of falling dots similar to rain. 
         [0056]    In an exemplary embodiment, the present invention allows the input information to be a letter of the alphabet or other symbol. 
         [0057]    In an exemplary embodiment, the user uses a slider, scrollbar, mouse wheel, button or similar method to go back and forward in time. 
         [0058]    In an exemplary embodiment, the time recording can be coupled to another timeline, such as that from another animation for an even more advanced control of the timeline. 
         [0059]    In an exemplary embodiment, the method includes provisions to use methods for the manipulation of drawings and animations as known from the prior art (e.g., path smoothing (matching spline angles) and trajectory smoothing (matching drawing speed and acceleration) of drawn, edited and joined lines, tweening, adding and removing items from the animation, editing items from the animation individually or simultaneously, copy-paste, distortion, etc.). 
         [0060]    In an exemplary embodiment, the present invention has provisions for drawing in more than two dimensions. For example, using features as explained above, a user may draw a cube in three dimensions by first pausing the animation and inputting a square, then setting a time marker on the animation slider, then unpausing the animation, then placing the pen on the first corner for 1 second, then using a shortcut key to returning to the time defined by the time marker, then placing the pen on the second corner for 1 second, then again using the shortcut key to return to the time defined by the time marker, then repeating the process for corners  3  and  4 , then pausing the animation and then drawing a final square.