PATENT DOCUMENT

Publication Number: US-8717305-B2
Application Number: US-4206708-A
Country: US
Kind Code: B2

Title: Touch event model for web pages

Abstract:
One or more touch input signals can be obtained from a touch sensitive device. A touch event model can be used to determine touch and/or gesture events based on the touch input signals. The touch and gesture events can be associated with touch input signals generated from different regions of a web page displayed on the touch sensitive device. Access can be provided to at least one touch or gesture event through a programming interface.

Claims:
What is claimed is: 
     
       1. A method performed by a device that includes one or more processors, memory, and a touch-sensitive display, the method comprising:
 receiving simultaneous touch input signals associated with two or more regions of a web page displayed on the touch-sensitive display, wherein each region is associated with a respective node in a document object model and the web page includes instructions for processing one or more gesture events; 
 generating two or more separate touch events associated with the two or more regions of the web page based on the touch input signals; 
 generating a gesture event by combining the two or more separate touch events associated with the two or more regions of the web page; and 
 sending the gesture event to the web page for processing the gesture event, generated by combining the two or more separate touch events associated with the two or more regions of the web page, by executing at least some of the instructions included in the web page. 
 
     
     
       2. The method of  claim 1 , where the touch events are from a group of touch events consisting of a touch start event, a touch move event, a touch end event and a touch cancel event. 
     
     
       3. The method of  claim 1 , further comprising generating a sequence of touch events, where the sequence of touch events includes a touch start event, one or more touch move events, and a touch end or touch cancel event. 
     
     
       4. The method of  claim 1 , where the touch events occur at substantially the same time. 
     
     
       5. The method of  claim 1 , wherein the web page regions are associated with nodes in the document object model and the nodes are at the same or different hierarchical levels of the document object model. 
     
     
       6. The method of  claim 1 , wherein the web page corresponds to a markup language document with attributes that define event actions and that are associated with the gesture event. 
     
     
       7. The method of  claim 6 , where the web page is displayed in a browser of the device. 
     
     
       8. The method of  claim 6 , where the event actions include running JavaScript. 
     
     
       9. The method of  claim 6 , wherein the web page corresponds to an HTML document with attributes in one or more HTML tags that define event actions and that are associated with the gesture event. 
     
     
       10. The method of  claim 1 , where each touch event is associated with a single touch input signal. 
     
     
       11. The method of  claim 10 , where each touch input signal is generated in response to touch input generated by a single finger touching the touch sensitive device. 
     
     
       12. The method of  claim 1 , wherein the two or more regions of the web page do not include an event handler to process the gesture event. 
     
     
       13. A system comprising:
 at least one processor; and 
 a computer-readable medium coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, cause the system to perform operations comprising:
 receiving simultaneous touch input signals associated with two or more regions of a web page displayed on a touch-sensitive display, wherein each region is associated with a respective node in a document object model and the web page includes instructions for processing one or more gesture events; 
 generating two or more separate touch events associated with the two or more regions of the web page based on the touch input signals; 
 generating a gesture event by combining the two or more separate touch events associated with the two or more regions of the web page; and 
 sending the gesture event to the web page for processing the gesture event, generated by combining the two or more separate touch events associated with the two or more regions of the web page, by executing at least some of the instructions included in the web page. 
 
 
     
     
       14. The system of  claim 13 , wherein the touch events are from a group of touch events consisting of a touch start event, a touch move event, a touch end event and a touch cancel event. 
     
     
       15. The system of  claim 14 , wherein the operations include generating a sequence of touch events, and the sequence of touch events include a touch start event, one or more touch move events, and a touch end or touch cancel event. 
     
     
       16. The system of  claim 14 , where the touch events occur at substantially the same time. 
     
     
       17. The system of  claim 13 , wherein the two or more regions of the web page do not include an event handler to process the gesture event. 
     
     
       18. A system, comprising:
 one or more processors; and 
 memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:
 receiving simultaneous touch input signals associated with two or more regions of a web page displayed on a touch-sensitive display, wherein each region is associated with a respective node in a document object model and the web page includes instructions for processing one or more gesture events; 
 generating two or more separate touch events associated with the two or more regions of the web page based on the touch input signals; 
 generating a gesture event by combining the two or more separate touch events associated with the two or more regions of the web page; and 
 sending the gesture event to the web page for processing the gesture event, generated by combining the two or more separate touch events associated with the two or more regions of the web page, by executing at least some of the instructions in the web page. 
 
 
     
     
       19. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by at least one processor of a device, cause the device to perform operations comprising:
 receiving simultaneous touch input signals associated with two or more regions of a web page displayed on a touch-sensitive display, wherein each region is associated with a respective node in a document object model and the web page includes instructions for processing one or more gesture events; 
 generating two or more separate touch events associated with the two or more regions of the web page based on the touch input signals; 
 generating a gesture event by combining the two or more separate touch events associated with the two or more regions of the web page; and 
 sending the gesture event to the web page for processing the gesture event, generated by combining the two or more separate touch events associated with the two or more regions of the web page, by executing at least some of the instructions in the web page. 
 
     
     
       20. The computer-readable storage medium of  claim 19 , wherein the touch events are from a group of touch events consisting of a touch start event, a touch move event, a touch end event and a touch cancel event. 
     
     
       21. The computer-readable storage medium of  claim 19 , wherein the operations include generating a sequence of touch events, and the sequence of touch events include a touch start event, one or more touch move events, and a touch end or touch cancel event. 
     
     
       22. The computer-readable storage medium of  claim 19 , wherein the touch events occur at substantially the same time. 
     
     
       23. The computer-readable medium of  claim 19 , wherein the two or more regions of the web page do not include an event handler to process the gesture event.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     The subject matter of this application is related to application Ser. No. 12/042,237, for “Touch Event Processing for Web Pages” filed Mar. 4, 2008, and application Ser. No. 12/042,299, for “Touch Event Model Programming Interface” filed Mar. 4, 2008, the contents of which are herein incorporated by reference. 
     TECHNICAL FIELD 
     This subject matter is generally related to web browsing services. 
     BACKGROUND 
     Web pages are created using a markup language which provides a means to describe a structure of text-based information in a document and to supplement that text with interactive forms, embedded images, and other objects. One popular markup language is HyperText Markup Language (HTML) which is written in the form of tags surrounded by angle brackets. HTML can describe the appearance and semantics of a web page, and can include embedded scripting language code (e.g., JavaScript®) which can affect the behavior of web browsers and other HTML processors. JavaScript® provides developers with the ability to add mouse event handlers or event listeners in a web page. These mouse event handlers can be assigned to specific regions of the web page and configured to receive mouse events in those regions, such as mouse up or mouse down events. 
     By contrast, web pages that are navigated with a touch sensitive device often need to respond to touch events generated by a user touching a web page with one or more fingers and making gestures. Conventional mouse event handlers cannot correctly interpret these touch events. Thus touch events require a different touch event model to correctly interpret touch events and to allow developers to fully utilize the capabilities of a touch sensitive display or device. 
     SUMMARY 
     One or more touch input signals can be obtained from a touch sensitive device. A touch event model can be used to determine touch and/or gesture events based on the touch input signals. The touch and/or gesture events can be associated with touch input signals generated from different regions of a web page displayed on the touch sensitive device. Access can be provided to at least one touch or gesture event through a programming interface. 
     In some implementations, a method includes: obtaining one or more touch input signals from a touch sensitive device, determining two or more touch events based on the touch input signals, where the touch events are associated with touch input signals associated with two or more regions of a web page displayed on the touch sensitive device, and providing access to at least one touch event through a programming interface. 
     In other implementations, a method includes: obtaining one or more touch input signals from a touch sensitive device, determining two or more touch events based on the touch input signals, where the touch events are associated with touch input signals associated with two or more regions of a web page displayed on the touch sensitive device, and sending the two or more touch events to the web page for processing. 
     Other implementations are disclosed which are directed to systems, methods and computer-readable mediums. 
    
    
     
       DESCRIPTION OF DRAWINGS 
         FIG. 1  illustrates an example web page document. 
         FIG. 2  illustrates a processing stack of an example multi-touch capable device. 
         FIG. 3  is a flow diagram of an example process for processing touch events. 
         FIG. 4  illustrates an example multi-touch capable device. 
         FIG. 5  is a block diagram of an example network operating environment for the multi-touch capable device of  FIG. 4 . 
         FIG. 6  is a block diagram of an example implementation of the multi-touch capable device of  FIG. 4 . 
     
    
    
     DETAILED DESCRIPTION 
     Example Web Page Structure and DOM 
       FIG. 1A  shows an example web page  100  that can be displayed on a browser. The browser may be hosted on a portable device, such as the multi-touch capable device  400  of  FIG. 4 . One or more elements, element  102  (“element 1”), element  104  (“element 2”), and element  106  (“element 3”) can be displayed on the web page  100 . The elements  102 ,  104 ,  106  can correspond to regions of the web page  100  that a user can select and additional functionality can be provided as a result of the selection. The elements can, for example, correspond to buttons on the web page  100 . The elements can also be nested so that one element contains another element. For example, element  104  contains element  108 . In the example shown, the element  108  is a scrubber control nested within element  104  which can be a media player user interface, for example. 
     In some implementations, a user may perform various functions with the elements on the web page  100  using a finger instead of a mouse. The user can, for example, touch the elements of the web page  100  using the touch sensitive display  402  shown in  FIG. 4 . In one example, the user can select an element by touching the element with one or more fingers and/or making a gesture, such as a swiping, pinching or rotating motion. To recognize touch input signals, certain areas of the web page  100  may be associated with touch event handlers. This can be accomplished with a DOM and embedded scripting language, as will be described in reference to  FIG. 1B . 
       FIG. 1B  is an exemplary DOM  150  associated with the web page  100 . The DOM  150  provides a structural representation of the web page  100  and describes the web page content as a set of objects that a scripting language (e.g., JavaScript®) can interpret. In some implementations, the DOM  150  provides access to the structure of a web page by mapping the elements  102 ,  104 ,  106 ,  108  in the web page  100  to individual nodes of a tree. For example, element  102  corresponds to node  154 . Element  104  corresponds to node  156 . Element  106  corresponds to node  158 . Element  108  corresponds to node  160 . The root node  152  corresponds to the overall web page  100 . 
     In some implementations, one or more elements  102 ,  104 ,  106 ,  108  in web page  100  can be associated with one or more corresponding touch event handler by associating corresponding nodes in the DOM  150  with the touch event handlers. Touch event handlers can be inserted into HTML tags of the web page  100  and can execute a scripting language to perform an action when, for example, a user touches or gestures within one of the elements on the web page  100 . For example, JavaScript® can work with the DOM  150  to attach actions to different touch events. 
     In some implementations, one or more elements  102 ,  104 ,  106 ,  108  can receive touch input which is detected by an event handler or listener. The touch input can be detected and processed into touch events by a touch event model which can be implemented in one or more layers of a software stack, as described in reference to  FIG. 2 . The touch events can be further processed by the web page  100 . The touch events can be in a format (e.g., attributes) that are easier to use in an application than raw touch input signals generated by the touch sensitive device. For example, each touch event can include a set of coordinates at which a touch is currently occurring. 
     Each element in the web page  100 , and its associated event handler, can receive, process and handle touch events. For example, if driver  202  ( FIG. 2 ) senses a touch point  110  associated with element  102 , or a touch point  112  associated with element  104 , the event handlers associated with elements  102  or  104  can each receive a separate touch event indicating that the element has been touched and can optionally send the touch event to the web page  100  for further processing. In some implementations, if a region of the web page  100  is touched that does not correspond to an event handler, a browser in the applications layer  214  can process the input, instead of the web page  100 . 
     In some implementations, a touch event can be detected per finger per node in the DOM  150 . For example, a user can touch the touch sensitive display  402  at touch point  110  and touch point  112  at substantially the same time, and two separate touch events can be detected by the touch event model. A separate touch event can be detected for touch point  110  and touch point  112  because each node  102  and node  104  in the DOM  150  are associated with a separate touch event handler. 
     In some implementations, touch events can be delivered to the web page  100  as EventTargets. Some examples of touch events can include touchstart, touchmove, touchend, and touchcancel. Other touch events are possible. Touchstart is a touch event that is detected when a user first places a finger on the touch sensitive display  402  in a region on the web page  100  that is associated with an event handler. When the user moves his finger around on the web page  100 , one or more touchmove events can be detected. When the user raises his finger off of the web page  100 , a touchend event is detected. The touchcancel can be detected when the system interrupts regular event handling. For example, a touchcancel event can occur when the touch sensitive display  402  is locked out to prevent inadvertent touches. 
     In some implementations, gesture events can also be detected by combining two or more touch events. Like touch events, the gesture events (GestureEvents) can also be delivered to the web page  100  as EventTargets. Some examples of gesture events can be gesturestart, gesturechange, and gestureend. The gesture events can contain scale and/or rotation information. The rotation information can include a rotation value that is a relative delta in degrees. An element on the webpage  100  can be dynamically rotated base on the rotation value. The scale information can include a scaling value that is a relative delta in document pixels. An element on the webpage  100  associated with the gesture event can be dynamically resized based on the scaling value. Other gesture events are possible. 
     In some implementations, a touch list can be received that includes touch event data to identify one or more touches on the web page  100 . The touch event data can include a touch identifier and at least one set of touch location coordinates. The touch list can also include touch event data to a touch event target associated with each touch. In some implementations, the one set of touch location coordinates can include client coordinates, page coordinates, and screen coordinates. In some implementations, the touch event data can identify one or more changed touches. 
     In some implementations, GestureEvents can be sent to the web page  100  before TouchEvents. For example, if a user places fingers on touch point  110  and touch point  112 , then makes a rotating gesture with those fingers clockwise or counterclockwise on the touch sensitive display, the touch event model detects these multiple touch events and combines the touch events into a gesture event. The gesture event can then be sent to the web page  100 , followed by the touch events that were combined to form the gesture event. This way, a developer has access to a gesture event and the individual touch events of the gesture event which provides the developer with more flexibility when developing a web application. 
     In some implementations, the touch events are received in the following order: a touchstart event, one or more touchmove events and a touchend or touchcancel event. Using the example of  FIG. 1A , when the user touches touch point  110 , a first touchstart event is detected by a first touch event handler associated with element  102 . When the user touches touch point  112 , a second touchstart event is detected by a second touch event handler associated with element  104 . As the user rotates her fingers without lifting her fingers, the first and second touch event handlers detect touchmove events which can be interpreted by the touch event model as a rotation gesture event. When the user finishes the rotation and lifts her fingers from the web page  100 , the first and second touch event handlers detect touchend events. All or some of these touch events can be made available to developers through a touch event Application Programming Interface (API). The touch API can be made available to developers as a Software Development Kit (SDK) or as part of an application (e.g., as part of a browser tool kit). The touch event API can rely on other services, frameworks and an operating system to perform its various functions. These services, frameworks and operating system can be part of a software or processing stack, as described in reference to  FIG. 2 . where the touch events are associated with attributes that can be inserted in documents to define event actions in an application. 
     Example IDL 
     An example touch event model will now be described in Interface Description Language (IDL). The functionality and data structures of the IDL can be accessed through an API by a web designer or application developer. Access to touch events and/or gesture events can be associated with attributes that can be inserted in an markup language document (e.g., HTML, XML) to define event actions in an application. For example, the attributes can be inserted in one or more HTML tags in an HTML document for generating the web page displayed on the touch sensitive display  402 . The event actions can include running an embedded script (e.g., JavaScript®). 
     
       
         
           
               
             
               
                   
               
             
            
               
                 interface [ 
               
               
                  Conditional=TOUCH_EVENTS, 
               
               
                  GenerateConstructor 
               
               
                 ] TouchEvent : UIEvent { 
               
               
                  void initTouchEvent(in AtomicString type, 
               
            
           
           
               
               
            
               
                   
                 in boolean canBubble, 
               
               
                   
                 in boolean cancelable, 
               
               
                   
                 in DOMWindow view, 
               
               
                   
                 in long detail, 
               
               
                   
                 in long screenX, 
               
               
                   
                 in long screenY, 
               
               
                   
                 in long clientX, 
               
               
                   
                 in long clientY, 
               
               
                   
                 in boolean ctrlKey, 
               
               
                   
                 in boolean altKey, 
               
               
                   
                 in boolean shiftKey, 
               
               
                   
                 in boolean metaKey, 
               
               
                   
                 in TouchList touches, 
               
               
                   
                 in TouchList targetTouches, 
               
               
                   
                 in TouchList changedTouches, 
               
               
                   
                 in long scale, 
               
               
                   
                 in long rotation); 
               
            
           
           
               
            
               
                  readonly attribute TouchList touches;  // all touches 
               
               
                  readonly attribute TouchList targetTouches; // all touches in this 
               
               
                  TouchEvent Target 
               
               
                  readonly attribute TouchList changedTouches; // all touches changed 
               
               
                  in the current event 
               
               
                  readonly attribute long scale; 
               
               
                  readonly attribute long rotation; 
               
               
                  readonly attribute boolean ctrlKey; 
               
               
                  readonly attribute boolean shiftKey; 
               
               
                  readonly attribute boolean altKey; 
               
               
                  readonly attribute boolean metaKey; 
               
               
                 }; 
               
               
                 interface [ 
               
               
                  Conditional=TOUCH_EVENTS, 
               
               
                 ] Touch { 
               
               
                  readonly attribute EventTarget target; 
               
               
                  readonly attribute long identifier; 
               
               
                  readonly attribute long clientX; 
               
               
                  readonly attribute long clientY; 
               
               
                  readonly attribute long pageX; 
               
               
                  readonly attribute long pageY; 
               
               
                  readonly attribute long screenX; 
               
               
                  readonly attribute long screenY; 
               
               
                 }; 
               
               
                 interface [ 
               
               
                  Conditional=TOUCH_EVENTS, 
               
               
                  HasIndexGetter, 
               
               
                 ] TouchList { 
               
               
                  readonly attribute unsigned long length; 
               
               
                  Touch  item(in unsigned long index); 
               
               
                 }; 
               
               
                 interface [ 
               
               
                  Conditional=TOUCH_EVENTS, 
               
               
                  GenerateConstructor 
               
               
                 ] GestureEvent : UIEvent { 
               
               
                  void initGestureEvent( in AtomicString type, 
               
            
           
           
               
               
            
               
                   
                 in boolean canBubble, 
               
               
                   
                 in boolean cancelable, 
               
               
                   
                 in DOMWindow view, 
               
               
                   
                 in long detail, 
               
               
                   
                 in long screenX, 
               
               
                   
                 in long screenY, 
               
               
                   
                 in long clientX, 
               
               
                   
                 in long clientY, 
               
               
                   
                 in boolean ctrlKey, 
               
               
                   
                 in boolean altKey, 
               
               
                   
                 in boolean shiftKey, 
               
               
                   
                 in boolean metaKey, 
               
               
                   
                 in EventTarget target, 
               
               
                   
                 in long scale, 
               
               
                   
                 in long rotation); 
               
            
           
           
               
            
               
                  readonly attribute EventTarget target; 
               
               
                  readonly attribute long scale; 
               
               
                  readonly attribute long rotation; 
               
               
                  readonly attribute boolean ctrlKey; 
               
               
                  readonly attribute boolean shiftKey; 
               
               
                  readonly attribute boolean altKey; 
               
               
                  readonly attribute boolean metaKey; 
               
               
                 }; 
               
               
                 In Document.idl: 
               
            
           
           
               
               
            
               
                 Touch 
                 createTouch(in EventTarget target, 
               
            
           
           
               
               
            
               
                   
                 in long identifier, 
               
               
                   
                 in long clientX, 
               
               
                   
                 in long clientY, 
               
               
                   
                 in long pageX, 
               
               
                   
                 in long pageY, 
               
               
                   
                 in long screenX, 
               
               
                   
                 in long screenY) 
               
            
           
           
               
            
               
                  raises (DOMException); 
               
               
                 [Custom] TouchList  createTouchList( ) 
               
               
                  raises (DOMException); 
               
               
                   
               
            
           
         
       
     
     The following is example of HTML code snippet for processing touch events using the example IDL above. The HTML below shows, for example, that the touch event listener TouchStart and GestureStart were added to an element with the HTML code:
     this.element.addEventListener(‘touchstart’, function(e) {return self.onTouchStart(e)}, false);   this.element.addEventListener(‘gesturestart’, function(e) {return self.onGestureStart(e)}, false);   

     The HTML code corresponding to the IDL above may be as follows: 
     
       
         
           
               
             
               
                   
               
             
            
               
                 &lt;!DOCTYPE html PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN” 
               
               
                 “http://www.w3.org/TR/html4/loose.dtd”&gt; 
               
               
                 &lt;html lang=“en”&gt; 
               
               
                 &lt;head&gt; 
               
               
                  &lt;meta http-equiv=“Content-Type” content=“text/html; charset=utf-8”&gt; 
               
               
                   &lt;meta name=“viewport” content=“initial-scale=1.0” /&gt; 
               
               
                   &lt;title&gt;Transform Gestures&lt;/title&gt; 
               
               
                   &lt;style type=“text/css” media=“screen”&gt; 
               
               
                    .box { 
               
               
                     position: absolute; 
               
               
                     height: 150px; 
               
               
                     width: 150px; 
               
               
                     background-color: blue; 
               
               
                    } 
               
               
                    .box:active { 
               
               
                     background-color: red; 
               
               
                    } 
               
               
                    body { 
               
               
                     margin: 0px; 
               
               
                    } 
               
               
                    #container { 
               
               
                     position: absolute; 
               
               
                     width: 100%; 
               
               
                     height: 100%; 
               
               
                    } 
               
               
                    #main-box2 { 
               
               
                     top: 10px; 
               
               
                     left: 155px; 
               
               
                     background: red; 
               
               
                     z-index: 1; 
               
               
                    } 
               
               
                   &lt;/style&gt; 
               
               
                   &lt;script type=“text/javascript” charset=“utf-8”&gt; 
               
               
                    var trackedObjectCount = 0; 
               
               
                    function Box(inElement) 
               
               
                    { 
               
               
                     var self = this; 
               
               
                     this.element = inElement; 
               
               
                     this.scale = 1.0; 
               
               
                     this.rotation = 0; 
               
               
                     this.position = ‘0,0’; 
               
               
                     this.element.addEventListener(‘touchstart’, function(e) { return self.onTouchStart(e) }, false); 
               
               
                     this.element.addEventListener(‘gesturestart’, function(e) { return self.onGestureStart(e) }, 
               
               
                 false); 
               
               
                    } 
               
               
                    Box.prototype = { 
               
               
                     // position strings are “x,y” with no units 
               
               
                     get position( ) 
               
               
                     { 
               
               
                      return this._position; 
               
               
                     }, 
               
               
                     set position(pos) 
               
               
                     { 
               
               
                      this._position = pos; 
               
               
                      var components = pos.split(‘,’) 
               
               
                      var x = components[0]; 
               
               
                      var y = components[1]; 
               
               
                      const kUseTransform = true; 
               
               
                      if (kUseTransform) { 
               
               
                       this.element.style.webkitTransform = ‘rotate(‘ + this.rotation + ’deg) scale(‘ + this.scale + ’) 
               
               
                 translate(‘ + x + ’px, ‘ + y + ’px)’; 
               
               
                      } 
               
               
                      else { 
               
               
                       this.element.style.left = x + ‘px’; 
               
               
                       this.element.style.top = y + ‘px’; 
               
               
                      } 
               
               
                     }, 
               
               
                     get x( ) 
               
               
                     { 
               
               
                      return parseInt(this._position.split(‘,’)[0]); 
               
               
                     }, 
               
               
                     set x(inX) 
               
               
                     { 
               
               
                      var comps = this._position.split(‘,’); 
               
               
                      comps[0] = inX; 
               
               
                      this.position = comps.join(‘,’); 
               
               
                     }, 
               
               
                     get y( ) 
               
               
                     { 
               
               
                      return parseInt(this._position.split(‘,’)[1]); 
               
               
                     }, 
               
               
                     set y(inY) 
               
               
                     { 
               
               
                      var comps = this._position.split(‘,’); 
               
               
                      comps[1] = inY; 
               
               
                      this.position = comps.join(‘,’); 
               
               
                     }, 
               
               
                     filterEvent: function(e) 
               
               
                     { 
               
               
                      // Prevent the browser from doing its default thing (scroll, zoom) 
               
               
                      e.preventDefault( ); 
               
               
                      // Event listeners are added at the document level, so we receive gesturechange events for 
               
               
                 other elements. 
               
               
                      return (e.target == this.element); 
               
               
                     }, 
               
               
                     onTouchStart: function(e) 
               
               
                     { 
               
               
                      if (!this.filterEvent(e)) 
               
               
                       return false; 
               
               
                      // Start tracking when the first finger comes down in this element 
               
               
                      if (e.targetTouches.length != 1) 
               
               
                       return false; 
               
               
                      this.startX = e.targetTouches[0].clientX; 
               
               
                      this.startY = e.targetTouches[0].clientY; 
               
               
                      var self = this; 
               
               
                      if (!(“touchMoveHandler” in this)) { 
               
               
                       this.touchMoveHandler = function(e) { return self.onTouchMove(e) } 
               
               
                       this.touchEndHandler = function(e) { return self.onTouchEnd(e) } 
               
               
                      } 
               
               
                      document.addEventListener(‘touchmove’, this.touchMoveHandler, false); 
               
               
                      document.addEventListener(‘touchend’, this.touchEndHandler, false); 
               
               
                      trackedObjectCount++; 
               
               
                      return false; 
               
               
                     }, 
               
               
                     onTouchMove: function(e) 
               
               
                     { 
               
               
                      if (!this.filterEvent(e)) 
               
               
                       return false; 
               
               
                      // Don&#39;t track motion when multiple touches are down in this element (that&#39;s a gesture) 
               
               
                      if (e.targetTouches.length != 1) 
               
               
                       return false; 
               
               
                      var leftDelta = e.targetTouches[0].clientX − this.startX; 
               
               
                      var topDelta = e.targetTouches[0].clientY − this.startY; 
               
               
                      var newLeft = (this.x) + leftDelta; 
               
               
                      var newTop = (this.y) + topDelta; 
               
               
                      this.position = newLeft + ‘,’ + newTop; 
               
               
                      this.startX = e.targetTouches[0].clientX; 
               
               
                      this.startY = e.targetTouches[0].clientY; 
               
               
                      return false; 
               
               
                     }, 
               
               
                     onTouchEnd: function(e) 
               
               
                     { 
               
               
                      if (!this.filterEvent(e)) 
               
               
                       return false; 
               
               
                      // Stop tracking when the last finger is removed from this element 
               
               
                      if (e.targetTouches.length &gt; 0) 
               
               
                       return false; 
               
               
                      document.removeEventListener(‘touchmove’, this.touchMoveHandler, false); 
               
               
                      document.removeEventListener(‘touchend’, this.touchEndHandler, false); 
               
               
                      trackedObjectCount−−; 
               
               
                      return false; 
               
               
                     }, 
               
               
                     onGestureStart: function(e) 
               
               
                     { 
               
               
                      if (!this.filterEvent(e)) 
               
               
                       return false; 
               
               
                      var self = this; 
               
               
                      if (!(“gestureChangeHandler” in this)) { 
               
               
                       this.gestureChangeHandler = function(e) { return self.onGestureChange(e) } 
               
               
                       this.gestureEndHandler = function(e) { return self.onGestureEnd(e) } 
               
               
                      } 
               
               
                      document.addEventListener(‘gesturechange’, this.gestureChangeHandler, true); 
               
               
                      document.addEventListener(‘gestureend’, this.gestureEndHandler, true); 
               
               
                      return false; 
               
               
                     }, 
               
               
                     onGestureChange: function(e) 
               
               
                     { 
               
               
                      if (!this.filterEvent(e)) 
               
               
                       return false; 
               
               
                      // Only interpret gestures when tracking one object. Otherwise, interpret raw touch events 
               
               
                      // to move the tracked objects. 
               
               
                      if (trackedObjectCount == 1) { 
               
               
                       this.scale += e.scaling * 0.01; 
               
               
                       this.rotation += e.rotation / 2; 
               
               
                       this.position = this.position; 
               
               
                      } 
               
               
                       return false; 
               
               
                      }, 
               
               
                      onGestureEnd: function(e) 
               
               
                      { 
               
               
                       if (!this.filterEvent(e)) 
               
               
                        return false; 
               
               
                       document.removeEventListener(‘gesturechange’, this.gestureChangeHandler, true); 
               
               
                       document.removeEventListener(‘gestureend’, this.gestureEndHandler, true); 
               
               
                       return false; 
               
               
                      }, 
               
               
                     } 
               
               
                     function loaded( ) 
               
               
                     { 
               
               
                      new Box(document.getElementById(‘main-box’)); 
               
               
                      new Box(document.getElementById(‘main-box2’)); 
               
               
                     } 
               
               
                     window.addEventListener(‘load’, loaded, true); 
               
               
                    &lt;/script&gt; 
               
               
                   &lt;/head&gt; 
               
               
                  &lt;body&gt; 
               
               
                   &lt;div id=“container”&gt; 
               
               
                    &lt;div id=“main-box” class=“box”&gt;&lt;/div&gt; 
               
               
                    &lt;div id=“main-box2” class=“box”&gt;&lt;/div&gt; 
               
               
                   &lt;/div&gt; 
               
               
                  &lt;/body&gt; 
               
               
                 &lt;/html&gt; 
               
               
                   
               
            
           
         
       
     
     Example Processing Stack for Multi-Touch Device 
       FIG. 2  is a diagram of a processing stack of an example multi-touch capable device. The touch event model described above can be implemented in one or more regions of the processing stack and user various resources in the stack. The hardware  200  layer can include various hardware interface components, such as a touch sensitive or enabled device or touch sensitive display. The touch sensitive device can include a display and a panel that senses multiple touches simultaneously. The hardware layer  200  can also include an accelerometer for detecting an orientation of the touch sensitive display or device (e.g., portrait, landscape). Thus signals indicative of orientation can be used by the touch event model to scale web pages for optimum display. 
     One or more drivers in a driver layer  202  can communicate with the hardware  200 . For example, the drivers can receive and process touch input signals generated by the touch sensitive display or device in the hardware layer  200 . A core Operating System (OS)  204  can communicate with the driver(s). The core OS  204  can process raw input data received from the driver(s). In some embodiments, the drivers can be considered to be a part of the core OS  204 . 
     A set of OS application programming interfaces (APIs)  206  can communicate with the core OS  204 . These APIs can be a set of APIs that are usually included with operating systems (such as, for example, Linux or UNIX APIs). A set of core foundation APIs  208  can utilize the OS APIs  206 , and a set of foundation APIs  210  can utilize the core foundation APIs  208 . 
     Web page software development kit (SDK)  210  can include a set of APIs designed for use by applications running on the device. The touch event APIs can, for example, be included in the Web page SDK  210 . The APIs of the Web page SDK  210  can utilize the foundation APIs  208 . The Web page SDK  210  can, for example, include Web KIT provided by Apple Inc. The Web page SDK  210  can be offered as an API or can be accessible through an application, for example, a browser such as SAFARI®, provided by Apple Inc. 
     Applications  214  running on the device can utilize the APIs of the Web page SDK  210  to create web pages. The APIs of the Web page SDK  210  can, in turn, communicate with lower level elements, ultimately communicating with the touch sensitive display or device and various other user interface hardware. While each layer can utilize the layer underneath it, that is not always required. For example, in some embodiments, applications  214  can occasionally communicate with OS APIs  206 . 
     Example Touch Event Process 
       FIG. 3  is a flow diagram of a process  300  for providing access to touch and/or gesture events though an API. The process  300  can begin by obtaining one or more touch input signals ( 302 ). The touch input signals can be obtained from a touch sensitive display or device. A determination of touch events and/or gestures can be made based on the touch input signals using a touch event model ( 304 ). The touch events can be associated with regions of a web page displayed on a touch sensitive display or device. For example, the touch sensitive display can be a display on a mobile phone, and a touch sensitive device can be a touch sensitive pad on a notebook computer. Access to touch events and/or gesture events can be provided through a programming interface ( 306 ). For example, with respect to the HTML snippet described above with reference to  FIG. 2 , the snippet can be inserted into an HTML document by the web developer to provides the developer with access to touch and/or gesture events. The touch events and/or gesture events can be further processed by code in the HTML document to initiate event actions ( 306 ). 
     Mobile Device Overview 
       FIG. 4  is a block diagram of an example multi-touch capable device  400 . In some implementations, the multi-touch capable device  400  includes a touch sensitive display  402 . The touch sensitive display  402  can implement liquid crystal display (LCD) technology, light emitting polymer display (LPD) technology, or some other display technology. The touch sensitive display  402  can be sensitive to haptic and/or tactile contact with a user. 
     In some implementations, the touch sensitive display  402  can comprise a multi-touch sensitive display  402 . A touch sensitive display  402  can, for example, process multiple simultaneous touch points, including processing data related to the pressure, degree and/or position of each touch point. Such processing facilitates gestures and interactions with multiple fingers, chording, and other interactions. Other touch sensitive display technologies can also be used, e.g., a display in which contact is made using a stylus or other pointing device. Some examples of multi-touch sensitive display technology are described in U.S. Pat. Nos. 6,323,846, 6,570,557, 6,677,932, and U.S. Patent Publication 2002/0015024A1, each of which is incorporated by reference herein in its entirety. In some implementations, the multi-touch capable device  400  can display one or more graphical user interfaces on the touch sensitive display  402  for providing the user access to various system objects and for conveying information to the user. 
     Example Multi-Touch Capable Device Functionality 
     In some implementations, the multi-touch capable device  400  can implement multiple device functionalities, such as a telephony device, an e-mail device, a network data communication device, a Wi-Fi base station device, and a media processing device. In some implementations, the multi-touch capable device  400  can include a web browser  404  for displaying web pages (e.g., web page  100 ). The touch sensitive display  402  can receive touch input signals made on the web page  100  and the touch model described above can be used to determine touch and/or gesture events based on the touch input signals. In some implementations, the multi-touch capable device  400  can implement network distribution functionality. In some implementations, the touch sensitive display  402  can be locked down when the multi-touch capable device  400  is proximate to the user&#39;s ear. This lockdown would cause a touchcancel event as described in reference to  FIG. 1B . 
     In some implementations, an accelerometer  472  can be utilized to detect movement of the multi-touch capable device  400 , as indicated by the directional arrow  474 . Accordingly, display objects and/or media can be presented according to a detected orientation, e.g., portrait or landscape. In some implementations, the multi-touch capable device  400  may include circuitry and sensors for supporting a location determining capability, such as that provided by the global positioning system (GPS) or other positioning systems (e.g., systems using Wi-Fi access points, television signals, cellular grids, Uniform Resource Locators (URLs)). In some implementations, a positioning system (e.g., a GPS receiver) can be integrated into the multi-touch capable device  400  or provided as a separate device that can be coupled to the multi-touch capable device  400  through an interface to provide access to location-based services. The multi-touch capable device  400  can also include one or more wireless communication subsystems. 
     In some implementations, a port device, e.g., a Universal Serial Bus (USB) port, or a docking port, or some other wired port connection, can be included. The port device can, for example, be utilized to establish a wired connection to other computing devices, such as other multi-touch capable devices  400 , network access devices, a personal computer, a printer, or other processing devices capable of receiving and/or transmitting data. In some implementations, the port device allows the multi-touch capable device  400  to synchronize with a host device using one or more protocols, such as, for example, the TCP/IP, HTTP, UDP and any other known protocol. 
     Network Operating Environment 
       FIG. 5  is a block diagram of an example network operating environment  600  for the multi-touch capable device  400  of  FIG. 4 . The multi-touch capable device  400  of  FIG. 4  can, for example, communicate over one or more wired and/or wireless networks  510  in data communication. For example, a wireless network  512 , e.g., a cellular network, can communicate with a wide area network (WAN)  514 , such as the Internet, by use of a gateway  516 . Likewise, an access point  518 , such as an 802.11g wireless access point, can provide communication access to the wide area network  514 . In some implementations, both voice and data communications can be established over the wireless network  512  and the access point  518 . For example, the multi-touch capable device  400   a  can place and receive phone calls (e.g., using VoIP protocols), send and receive e-mail messages (e.g., using POP3 protocol), and retrieve electronic documents and/or streams, such as web pages, photographs, and videos, over the wireless network  512 , gateway  516 , and wide area network  514  (e.g., using TCP/IP or UDP protocols). Likewise, the multi-touch capable device  400   b  can place and receive phone calls, send and receive e-mail messages, and retrieve electronic documents over the access point  518  and the wide area network  514 . In some implementations, the multi-touch capable device  400  can be physically connected to the access point  518  using one or more cables and the access point  518  can be a personal computer. In this configuration, the multi-touch capable device  400  can be referred to as a “tethered” device. 
     The multi-touch capable devices  400   a  and  400   b  can also establish communications by other means. For example, the multi-touch capable device  400   a  can communicate with other wireless devices, e.g., other multi-touch capable devices  400 , cell phones, etc., over the wireless network  512 . Likewise, the multi-touch capable device  400   a  and  400   b  can establish peer-to-peer communications  520 , e.g., a personal area network, by use of one or more communication subsystems, such as the Bluetooth™ communication device  488  shown in  FIG. 4 . Other communication protocols and topologies can also be implemented. 
     The multi-touch capable device  400  can, for example, communicate with a network resource  530  over the one or more wired and/or wireless networks  510 . For example, the network resource can be a web server for delivering web pages which can be touched via the touch model, as described in reference to  FIGS. 1-2 . 
     Other services can also be provided, including a software update service that automatically determines whether software updates exist for software on the multi-touch capable device  400 , then downloads the software updates to the multi-touch capable device  400  where it can be manually or automatically unpacked and/or installed. 
     Example Mobile Device Architecture 
       FIG. 6  is a block diagram  600  of an example implementation of the multi-touch capable device  400  of  FIG. 4 . The multi-touch capable device  400  can include a memory interface  602 , one or more data processors, image processors and/or central processing units  604 , and a peripherals interface  606 . The memory interface  602 , the one or more processors  604  and/or the peripherals interface  606  can be separate components or can be integrated in one or more integrated circuits. The various components in the multi-touch capable device  400  can be coupled by one or more communication buses or signal lines. 
     Sensors, devices and subsystems can be coupled to the peripherals interface  606  to facilitate multiple functionalities. For example, a motion sensor  610 , a light sensor  612 , and a proximity sensor  614  can be coupled to the peripherals interface  606  to facilitate the orientation, lighting and proximity functions described with respect to  FIG. 4 . Other sensors  616  can also be connected to the peripherals interface  606 , such as a positioning system (e.g., GPS receiver), a temperature sensor, a biometric sensor, or other sensing device, to facilitate related functionalities. 
     A camera subsystem  620  and an optical sensor  622 , e.g., a charged coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) optical sensor, can be utilized to facilitate camera functions, such as recording photographs and video clips. 
     Communication functions can be facilitated through one or more wireless communication subsystems  624 , which can include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem  624  can depend on the communication network(s) over which the multi-touch capable device  400  is intended to operate. For example, a multi-touch capable device  400  may include communication subsystems  624  designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth™ network. In particular, the wireless communication subsystems  624  may include hosting protocols such that the device  500  may be configured as a base station for other wireless devices. 
     An audio subsystem  626  can be coupled to a speaker  628  and a microphone  630  to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions. 
     The I/O subsystem  640  can include a touch screen controller  642  and/or other input controller(s)  644 . The touch-screen controller  642  can be coupled to a touch screen  646 . The touch screen  646  and touch screen controller  642  can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen  646 . 
     The other input controller(s)  644  can be coupled to other input/control devices  648 , such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) can include an up/down button for volume control of the speaker  628  and/or the microphone  630 . 
     In one implementation, a pressing of the button for a first duration may disengage a lock of the touch screen  646 ; and a pressing of the button for a second duration that is longer than the first duration may turn power to the multi-touch capable device  400  on or off. The user may be able to customize a functionality of one or more of the buttons. The touch screen  646  can, for example, also be used to implement virtual or soft buttons and/or a keypad or keyboard. 
     In some implementations, the multi-touch capable device  400  can present recorded audio and/or video files, such as MP3, AAC, and MPEG files. In some implementations, the multi-touch capable device  400  can include the functionality of an MP3 player, such as an iPod™. The multi-touch capable device  400  may, therefore, include a 32-pin connector that is compatible with the iPod. Other input/output and control devices can also be used. 
     The memory interface  602  can be coupled to memory  650 . The memory  650  can include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and/or flash memory (e.g., NAND, NOR). The memory  650  can store an operating system  652 , such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks. The operating system  652  may include instructions for handling basic system services and for performing hardware dependent tasks. 
     The memory  650  may also store communication instructions  654  to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers. The memory  650  may include graphical user interface instructions  656  to facilitate graphic user interface processing; sensor processing instructions  658  to facilitate sensor-related processing and functions; phone instructions  660  to facilitate phone-related processes and functions; electronic messaging instructions  662  to facilitate electronic-messaging related processes and functions; web browsing instructions  664  to facilitate web browsing-related processes and functions; media processing instructions  666  to facilitate media processing-related processes and functions; GPS/Navigation instructions  668  to facilitate GPS and navigation-related processes and instructions; camera instructions  670  to facilitate camera-related processes and functions; and/or other messaging instructions  672  to facilitate processes and functions, as described in reference to  FIGS. 1-5 . 
     Each of the above identified instructions and applications can correspond to a set of instructions for performing one or more functions described above. These instructions need not be implemented as separate software programs, procedures or modules. The memory  650  can include additional instructions or fewer instructions. Furthermore, various functions of the multi-touch capable device  400  may be implemented in hardware and/or in software, including in one or more signal processing and/or application specific integrated circuits. 
     The features described can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The features can be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device or in a propagated signal, for execution by a programmable processor; and method steps can be performed by a programmable processor executing a program of instructions to perform functions of the described implementations by operating on input data and generating output. 
     The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. 
     Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors or cores, of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits). 
     To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer. 
     The features can be implemented in a computer system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include, e.g., a LAN, a WAN, and the computers and networks forming the Internet. 
     The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. 
     A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of one or more implementations may be combined, deleted, modified, or supplemented to form further implementations. As yet another example, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.

Metadata:
Filing Date: 20080304
Publication Date: 20140506
Grant Date: 20140506
Priority Date: 20080304
Inventors: WILLIAMSON RICHARD
BOLSINGA GREGORY DENNIS
OMERNICK TIM
Assignee: APPLE INC
CPC Classifications: [{"code": "G06F3/04883", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04886", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F16/9577", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F16/9577", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04886", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04883", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F40/143", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F3/0488", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F40/143", "inventive": true, "first": true, "tree": "[]"}]
Family ID: 40743219