Source: https://patents.justia.com/patent/10558321
Timestamp: 2020-02-28 06:55:55
Document Index: 706379293

Matched Legal Cases: ['§ 119', 'Application No. 11829890', 'Application No. 2013', 'Application No. 201180058029', 'Application No. 201180058029', 'Application No. 201180058029', 'Application No. 201180056896', 'Application No. 201180056896', 'Application No. 201180056896', 'Application No. 2013', 'Application No. 201180057378', 'Application No. 201180057378', 'Application No. 11829757', 'Application No. 11829844', 'Application No. 11829763', 'Application No. 2011800515271', 'Application No. 11829890', 'Application No. 2013', 'Application No. 201180058029', 'Application No. 11829839', 'Application No. 2013', 'Application No. 201180056896', 'Application No. 2013', 'Application No. 201180057652', 'Application No. 2013', 'Application No. 201180057378', 'Application No. 11829757', 'Application No. 2013', 'Application No. 201180056896', 'Application No. 201180058029', 'Application No. 201180056896', 'Application No. 201180056896', 'Application No. 11829844']

US Patent for Drag move gesture in user interface Patent (Patent # 10,558,321 issued February 11, 2020) - Justia Patents Search
Justia Patents Tiling Or Modular Adjacent DisplaysUS Patent for Drag move gesture in user interface Patent (Patent # 10,558,321)
Aug 28, 2015 - Z124
Unified desktop big brother application pools
Mobile device off-screen gesture area
The present application is a continuation of and claims priority to U.S. patent application Ser. No. 13/223,697, filed on Sep. 1, 2011, entitled “DRAG MOVE GESTURE IN USER INTERFACE,” which claims the benefits of and priority, under 35 U.S.C. § 119(e), to U.S. Provisional Application Ser. No. 61/389,000, filed Oct. 1, 2010, entitled “DUAL DISPLAY WINDOWING SYSTEM;” 61/389,117, filed Oct. 1, 2010, entitled “MULTI-OPERATING SYSTEM PORTABLE DOCKETING DEVICE;” 61/389,087, filed Oct. 1, 2010, entitled “TABLET COMPUTING USER INTERFACE.” Each of the aforementioned documents is incorporated herein by this reference in their entirety for all that they teach and for all purposes.
In one embodiment, a method provides the steps of:
(a) receiving, by one or both of a gesture capture region and a touch sensitive display, a gesture (which is commonly a drag gesture); and
(b) in response, ceasing to display a first displayed image on a first touch sensitive display of a communication device and commencing to display the first displayed image on a second touch sensitive display of the communication device, the first displayed image being one or both of a desktop and window of an application.
(a) receiving, by one or more of a gesture capture region and a touch sensitive display, a gesture (which is commonly a drag gesture);
(b) determining that a dual display communication device is in a closed state, wherein, in the closed state, a first touch sensitive display is enabled but a second touch sensitive display is disabled; and
(c) in response, ceasing to display a first displayed image on the first touch sensitive display and displaying a different second displayed image on the first touch sensitive display, wherein the first displayed image is one or more of a desktop and window of an application.
In one embodiment, a dual display communication device includes:
(i) a gesture capture region to receive a gesture; and
(ii) a first touch sensitive display to receive a gesture and display displayed images, wherein the displayed images are one or more of a desktop and window of an application;
(iii) a second touch sensitive display to receive a gesture and display displayed images;
(iv) middleware operable to perform one or more of the following operations:
(A) in response to a received gesture (which is commonly a drag gesture), commencing to display the first displayed image on the second touch sensitive display; and
(B) determine that the dual display communication device is in a closed state, wherein, in the closed state, a first touch sensitive display is enabled but a second touch sensitive display is disabled and in response to the received gesture, displaying a different second displayed image on the first touch sensitive display.
In one configuration, the first displayed image is a desktop, the gesture is received by the touch sensitive display, and the desktop is one of a plurality of an ordered set of desktop images.
In one configuration, the displayed image is a window, the gesture is received by the gesture capture region, the gesture capture region is unable to display any displayed image, and the window is one of a plurality of an ordered set of windows.
In one configuration, a gesture received by the gesture capture region enables navigation through an ordered set of desktops or windows and a gesture received by the touch sensitive display enables navigation through an ordered set of the other of desktops or windows.
In one configuration, the communication device has opened and closed states, in the opened state, both the first and second touch sensitive displays are enabled, in the closed state, the first touch sensitive display is enabled but the second touch sensitive display is disabled, and wherein the communication device is currently in the opened state.
In one configuration, the first and second touch sensitive displays are each in the portrait display orientation, the first touch sensitive display thereafter displays a second displayed image different from the first displayed image, a third displayed image different from the first displayed image was previously displayed by the second touch sensitive display, and the first and second displayed images, in response, are in active positions and the third displayed image is in an inactive position.
In one configuration, the first touch sensitive display is in the portrait display orientation, the communication device has opened and closed states, in the opened state, both the first and second touch sensitive displays are enabled, the first displayed image was previously in an active state and the second displayed image in an inactive state, and in response, the first displayed image is now in the inactive state and the second displayed image is now in the active state. The present disclosure can provide a number of advantages depending on the particular aspect, embodiment, and/or configuration. The communication display can enable an effective, convenient, and intuitive methodology for a user to navigate through or reorder display objects, such as stacks or ordered sets of windows and/or desktops, using a dual display communication device, particularly a cellular or wireless phone.
The term “desktop” refers to a metaphor used to portray systems. A desktop is generally considered a “surface” that typically includes pictures, called icons, widgets, folders, etc. that can activate show applications, windows, cabinets, files, folders, documents, and other graphical items. The icons are generally selectable to initiate an task through user interface interaction to allow a user to execute applications or conduct other operations.
FIGS. 7A-B are a series of portrait display orientation screen shots according to an embodiment;
FIGS. 8A-D are a series of portrait display orientation screen shots according to an embodiment;
FIGS. 9A-D are a series of portrait display orientation screen shots according to an embodiment;
FIGS. 10A-C are a series of portrait display orientation screen shots according to an embodiment;
FIGS. 11A-C are a series of portrait display orientation screen shots according to an embodiment;
FIGS. 12A-B are a series of landscape display orientation screen shots according to an embodiment;
FIG. 13 is a flow chart representing an embodiment;
FIG. 14 is a flow chart representing an embodiment;
FIG. 15A-C are a series of landscape display orientation screen shots according to an embodiment;
FIG. 16A-C are a series of landscape display orientation screen shots according to an embodiment; and
FIG. 17A is representation of a logical window stack;
FIG. 17B is another representation of an embodiment of a logical window stack;
FIG. 17C is another representation of an embodiment of a logical window stack;
FIG. 17D is another representation of an embodiment of a logical window stack;
FIG. 17E is another representation of an embodiment of a logical window stack;
FIG. 18 is block diagram of an embodiment of a logical data structure for a window stack;
FIG. 19 is a flow chart of an embodiment of a method for creating a window stack; and
FIG. 20 depicts a window stacking configuration according to an embodiment.
One or more display controllers 216a, 216b may be provided for controlling the operation of the touch sensitive screens 104 and 108, including input (touch sensing) and output (display) functions. In the exemplary embodiment illustrated in FIG. 2, a separate touch screen controller 216a or 216b is provided for each touch screen 104 and 108. In accordance with alternate embodiments, a common or shared touch screen controller may be used to control each of the included touch sensitive screens 104 and 108. In accordance with still other embodiments, the functions of a touch screen controller may be incorporated into other components, such as a processor 204.
The device 100 manages desktops and/or windows with at least one window stack 1700, 1728, as shown in FIGS. 17A and 17B. A window stack 1700, 1728 is a logical arrangement of active and/or inactive windows for a multi-screen device. For example, the window stack 1700, 1728 may be logically similar to a deck of cards, where one or more windows or desktops are arranged in order, as shown in FIGS. 17A and 17B. An active window is a window that is currently being displayed on at least one of the touch sensitive displays 110, 114. For example, windows 104 and 108 are active windows and are displayed on touch sensitive displays 110 and 114. An inactive window is a window that was opened and displayed but is now “behind” an active window and not being displayed. In embodiments, an inactive window may be for an application that is suspended, and thus, the window is not displaying active content. For example, windows 1712, 1716, 1720, and 1724 are inactive windows.
A window stack 1700, 1728 may have various arrangements or organizational structures. In the embodiment shown in FIG. 17A, the device 100 includes a first stack 1760 associated with a first touch sensitive display 110 and a second stack associated with a second touch sensitive display 114. Thus, each touch sensitive display 110, 114 can have an associated window stack 1760, 1764. These two window stacks 1760, 1764 may have different numbers of windows arranged in the respective stacks 1760, 1764. Further, the two window stacks 1760, 1764 can also be identified differently and managed separately. Thus, the first window stack 1760 can be arranged in order from a first window 1704 to a next window 1720 to a last window 1724 and finally to a desktop 1722, which, in embodiments, is at the “bottom” of the window stack 1760. In embodiments, the desktop 1722 is not always at the “bottom” as application windows can be arranged in the window stack below the desktop 1722, and the desktop 1722 can be brought to the “top” of a stack over other windows during a desktop reveal. Likewise, the second stack 1764 can be arranged from a first window 1708 to a next window 1712 to a last window 1716, and finally to a desktop 1718, which, in embodiments, is a single desktop area, with desktop 1722, under all the windows in both window stack 1760 and window stack 1764. A logical data structure for managing the two window stacks 1760, 1764 may be as described in conjunction with FIG. 18.
Another arrangement for a window stack 1728 is shown in FIG. 17B. In this embodiment, there is a single window stack 1728 for both touch sensitive displays 110, 114. Thus, the window stack 1728 is arranged from a desktop 1758 to a first window 1744 to a last window 1756. A window can be arranged in a position among all windows without an association to a specific touch sensitive display 110, 114. In this embodiment, a window is in the order of windows. Further, at least one window is identified as being active. For example, a single window may be rendered in two portions 1732 and 1736 that are displayed on the first touch sensitive screen 110 and the second touch sensitive screen 114. The single window may only occupy a single position in the window stack 1728 although it is displayed on both displays 110, 114.
Yet another arrangement of a window stack 1760 is shown in FIGS. 17C through 17E. The window stack 1760 is shown in three “elevation” views. In FIG. 17C, the top of the window stack 1760 is shown. Two sides of the window stack 1760 are shown in FIGS. 17D and 17E. In this embodiment, the window stack 1760 resembles a stack of bricks. The windows are stacked on each other. Looking from the top of the window stack 1760 in FIG. 17C, only the top most windows in the window stack 1760 are seen in different portions of the composite display 1764. The composite display 1764 represents a logical model for the entire display area of the device 100, which can include touch sensitive display 110 and touch sensitive display 114. A desktop 1786 or a window can occupy part or all of the composite display 1764.
In the embodiment shown, the desktop 1786 is the lowest display or “brick” in the window stack 1760. Thereupon, window 1 1782, window 2 1782, window 3 1768, and window 4 1770 are layered. Window 1 1782, window 3 1768, window 2 1782, and window 4 1770 only occupy a portion of the composite display 1764. Thus, another part of the stack 1760 includes window 8 1774 and windows 5 through 7 shown in section 1790. Only the top window in any portion of the composite display 1764 is actually rendered and displayed. Thus, as shown in the top view in FIG. 17C, window 4 1770, window 8 1774, and window 3 1768 are displayed as being at the top of the display in different portions of the window stack 1760. A window can be dimensioned to occupy only a portion of the composite display 1760 to “reveal” windows lower in the window stack 1760. For example, window 3 1768 is lower in the stack than both window 4 1770 and window 8 1774 but is still displayed. A logical data structure to manage the window stack can be as described in conjunction with FIG. 18.
A logical data structure 1800 for managing the arrangement of windows or desktops in a window stack is shown in FIG. 18. The logical data structure 1800 can be any data structure used to store data whether an object, record, file, etc. The logical data structure 1800 can be stored in any type of database or data storage system, regardless of protocol or standard. In embodiments, the logical data structure 1800 includes one or more portions, fields, attributes, etc. that store data in a logical arrangement that allows for easy storage and retrieval of the information. Hereinafter, these one or more portions, fields, attributes, etc. shall be described simply as fields. The fields can store data for a window identifier 1804, dimensions 1808, a stack position identifier 1812, a display identifier 1816, and/or an active indicator 1820. Each window in a window stack can have an associated logical data structure 1800. While only a single logical data structure 1800 is shown in FIG. 18, there may be more or fewer logical data structures 1800 used with a window stack (based on the number of windows or desktops in the stack), as represented by ellipses 1824. Further, there may be more or fewer fields than those shown in FIG. 18, as represented by ellipses 1828.
Dimensions 1808 can include dimensions for a window in the composite display 1760. For example, the dimensions 1808 can include coordinates for two or more corners of the window or may include one coordinate and dimensions for the width and height of the window. These dimensions 1808 can delineate what portion of the composite display 1760 the window may occupy, which may the entire composite display 1760 or only part of composite display 1760. For example, window 4 1770 may have dimensions 1880 that indicate that the window 1770 will occupy only part of the display area for composite display 1760, as shown in FIGS. 17C through 17E. As windows are moved or inserted in the window stack, the dimensions 1808 may change.
A stack position identifier 1812 can be any identifier that can identify the position in the stack for the window or may be inferred from the window's control record within a data structure, such as a list or a stack. The stack position identifier 1812 can be a GUID, a numeric ID, an alphanumeric ID, or other type of identifier. Each window or desktop can include a stack position identifier 1812. For example, as shown in FIG. 17A, window 1 1704 in stack 1 1760 can have a stack position identifier 1812 of 1 identifying that window 1704 is the first window in the stack 1760 and the active window. Similarly, window 6 1724 can have a stack position identifier 1812 of 3 representing that window 1724 is the third window in the stack 1760. Window 2 1708 can also have a stack position identifier 1812 of 1 representing that window 1708 is the first window in the second stack 1764. As shown in FIG. 17B, window 1 1744 can have a stack position identifier 1812 of 1, window 3, rendered in portions 1732 and 1736, can have a stack position identifier 1812 of 3, and window 6 1756 can have a stack position identifier 1812 of 6. Thus, depending on the type of stack, the stack position identifier 1812 can represent a window's location in the stack.
A display identifier 1816 can identify that the window or desktop is associated with a particular display, such as the first display 110 or the second display 114, or the composite display 1760 composed of both displays. While this display identifier 1816 may not be needed for a multi-stack system, as shown in FIG. 17A, the display identifier 1816 can indicate whether a window in the serial stack of FIG. 17B is displayed on a particular display. Thus, window 3 may have two portions 1732 and 1736 in FIG. 17B. The first portion 1732 may have a display identifier 1816 for the first display while the second portion 1736 may have a display identifier 1816 for the second display 114. However, in alternative embodiments, the window may have two display identifier 1816 that represent that the window is displayed on both of the displays 110, 114, or a display identifier 1816 identifying the composite display. In another alternate embodiment, the window may have a single display identifier 1816 to represent that the window is displayed on both of the displays 110, 114.
Similar to the display identifier 1816, an active indicator 1820 may not be needed with the dual stack system of FIG. 17A, as the window in stack position 1 is active and displayed. In the system of FIG. 17B, the active indicator 1820 can indicate which window(s) in the stack is being displayed. Thus, window 3 may have two portions 1732 and 1736 in FIG. 17. The first portion 1732 may have an active indicator 1820 while the second portion 1736 may also have an active indicator 1820. However, in alternative embodiments, window 3 may have a single active indicator 1820. The active indicator 1820 can be a simple flag or bit that represents that the window is active or displayed.
An embodiment of a method 1900 for creating a window stack is shown in FIG. 19. While a general order for the steps of the method 1900 is shown in FIG. 19. Generally, the method 1900 starts with a start operation 1904 and ends with an end operation 1928. The method 1900 can include more or fewer steps or can arrange the order of the steps differently than those shown in FIG. 19. The method 1900 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer readable medium. Hereinafter, the method 1900 shall be explained with reference to the systems, components, modules, software, data structures, user interfaces, etc. described in conjunction with FIGS. 1-18.
The Display Configuration Module 568 may use the input from these modules and evaluate the current window stack 1760 to determine the best place and the best dimensions, based on a visibility algorithm, to open the window. Thus, the Display Configuration Module 568 determines the best place to put the window at the top of the window stack 1760, in step 1916. The visibility algorithm, in embodiments, determines for all portions of the composite display, which windows are at the top of the stack. For example, the visibility algorithm determines that window 3 1768, window 4 1770, and window 8 1774 are at the top of the stack 1760 as viewed in FIGS. 17C through 17E. Upon determining where to open the window, the Display Configuration Module 568 can assign a display identifier 816 and possibly dimensions 808 to the window. The display identifier 816 and dimensions 808 can then be sent back to the Task Management Module 540. The Task Management Module 540 may then assign the window a stack position identifier 812 indicating the windows position at the top of the window stack.
In embodiments, the Task Management Module 540 sends the window stack information and instructions to render the window to the Window Management Module 532. The Window Management Module 532 and the Task Management Module 540 can create the logical data structure 800, in step 1924. Both the Task Management Module 540 and the Window Management Module 532 may create and manage copies of the window stack. These copies of the window stack can be synchronized or kept similar through communications between the Window Management Module 532 and the Task Management Module 540. Thus, the Window Management Module 532 and the Task Management Module 540, based on the information determined by the Multi-Display Management Module 524, can assign dimensions 808, a stack position identifier 812 (e.g., window 1 1782, window 4 1770, etc.), a display identifier 816 (e.g., touch sensitive display 1 110, touch sensitive display 2 114, composite display identifier, etc,), and an active indicator 820, which is generally always set when the window is at the “top” of the stack. The logical data structure 800 may then be stored by both the Window Management Module 532 and the Task Management Module 540. Further, the Window Management Module 532 and the Task Management Module 540 may thereinafter manage the window stack and the logical data structure(s) 800.
FIG. 20 depicts a further window stacking configuration. A plurality of windows 1, 2, 3, 4, 5, 6, 7, and 8, whether from the same or different multi-screen or single-screen applications, are depicted. The touch sensitive display 110 currently has window 4 in the active display position while touch sensitive display 114 currently has window 5 in the active display position. The stack for touch sensitive display 110, from top to bottom, has window 4 in the active display position and windows 3, 2, and 1 positioned behind it. The stack for touch sensitive display 114, from top to bottom, has window 5 in the active display position and windows 6, 7, and 8 positioned behind it.
Various window navigation configurations will now be discussed with reference to FIGS. 7-12 and 15-16. The various configurations depict examples relating to navigating among independent or hierarchically related displayed images. In the various examples, middleware 520, particularly one or more of the Multi-Display Management (MDM) class 524, a Surface Cache class 528, a Window Management class 532, an Activity Management class 536, and an Application Management class 540, independently or collectively, detect a stimulus, such as receipt of user input, particularly user gestures (step 1400 of FIG. 14), determine a current state of the device 100, display 110, 114 or screen 104, 108 (such as a state of frame buffers 548, activity stack 552, and/or event buffer 556), a context associated with the stimulus, and/or a sensed location of a gesture (step 1404), map one or more rule sets to the user device state, display state, screen state, stimulus context, and/or sensed gesture location to determine an appropriate functional result to be performed, (step 1408), and perform or cause to be performed the functional result (step 1412).
Referring to FIGS. 8A-D, the primary and secondary touch sensitive displays 110 and 114 are in portrait display orientation and provide first and second desktops D1 and D2, respectively (FIG. 8A). The first and second desktops form an ordered set of desktops. The secondary screen 108 detects a user gesture on the touch sensitive display 114. In one example, the user gesture 700 is a drag and, in another example, the user gesture is a flick. As shown in FIG. 8B, the second desktop D2 has moved in the direction indicated to an active display position (such as in a respective frame buffer or stack) or state for the primary screen and is displayed by the touch sensitive display 110, and the first desktop D1 has moved from an active to an inactive display position for the primary screen. A third desktop D3 has moved from an inactive to an active display position for the secondary screen and is now displayed by the touch sensitive display 114. Commonly, only a drag beyond approximately one-half of a display's or screen's width is considered to be a successful desktop navigation operation.
As further shown in FIG. 8C, the secondary screen 108 detects a further user gesture on the touch sensitive display 114. As shown in FIG. 8D, the third desktop D3 has moved in the direction indicated to an active display position for the primary screen and is displayed by the touch sensitive display 110, and the second desktop D2 has moved from an active display position to an inactive position for the primary screen. A fourth desktop D4 has moved from an inactive to an active display position for the secondary screen and is now displayed by the touch sensitive display 114.
Although not shown, the user could use an opposing gesture sensed in the touch sensitive display 110 to move a desktop from left-to-right. For example, in FIG. 8B the user could use a drag or flick gesture 700, 704 in the touch sensitive display 110 to cause the first desktop D1 to move from an inactive display position to an active display position for the touch sensitive display 110, thereby causing the second desktop D2 to move to an active position for the touch sensitive display 114 and the third desktop D3 to move from an active to an inactive display position for the secondary screen.
In one configuration shown in FIGS. 7A-B, gestures received by a gesture capture region 120 or 124 do not allow navigation between desktops in a desktop stack. Stated another way, gestures received by a touch sensitive display enable navigation through an ordered set or stack of desktops but not through an ordered set or stack of windows. As discussed below, gestures received by a gesture capture region enable navigation through the ordered set or stack of windows but not through the ordered set or stack of desktops.
Although drag and flick gestures are depicted in FIG. 7-8, it is to be understood that only one gesture is required and that other gestures may be employed.
Unlike desktop navigation, gestures received by a gesture capture region 120 or 124 are used to move windows from one screen to the other (e.g., from one active or inactive position to another). As discussed below, this rule applies when the device 100 is in either the closed or opened state.
Referring now to FIGS. 9A-D, a (minimized) first window 1 is displayed by the touch sensitive display 110, and a second desktop D2 by the touch sensitive display 114. The first window 1 is minimized in that it is displayed by only one of the primary and secondary screens 104 and 108. In the examples discussed herein, the windows and desktops can be maintained in separate display stacks or ordered sets. In FIG. 9A, a gesture, such as a drag or flick gesture 700, 704 is received by a gesture capture region 120. In response and as shown in FIGS. 9B and 9C, the first window 1 moves in the direction indicated (as shown by the cross-thatched displayed image) from an active display position of the touch sensitive display 110 (and primary screen 104) to an active display position of the touch sensitive display 114 (and secondary screen 108). As shown by FIG. 9D, the first desktop D1 is uncovered by the first window 1 and moves from an inactive to an active display position of the touch sensitive display 110 (and primary screen 104), and the second desktop D2 is now covered by the first window 1 and moves from an active to an inactive display position for the secondary screen. The second desktop D2 could be returned to an active display position for the secondary screen by providing an opposing user gesture to the gesture capture region 124. This would return the first window 1 to the active display position of the primary screen, as shown in FIG. 9A.
In one configuration, a well is displayed ahead of the window as the window moves from the touch sensitive display 110 to touch sensitive display 114. The well is a displayed image that is unable to receive user input or provide output to the user. It typically, though not necessarily, is a substantially solid colored displayed image. The well expands or slides from behind the window and moves towards the target touch sensitive display. When and as the well expands or moves, a seam 800 between the first and second touch sensitive displays and their respective display images is dimmed out completely to reveal the well background. When the well has moved and occupies fully the target touch sensitive display (in this example touch sensitive display 114) so that the prior display image is completely dimmed out or covered by the well (the prior display image is, in the example, the second desktop 114), the first window 1 moves over to occupy the target touch sensitive display. Until that time, the first window 1 remains as the displayed image for the source touch sensitive display (in this example touch sensitive display 110). The well is an affordance generally used for multi-screen applications and not for single-screen applications.
Referring to FIGS. 10A-C, window navigation when the device 100 is in the closed state is depicted. In the following examples, an object stack includes first, second, and third windows 1, 2, 3. In FIG. 10A, the touch sensitive display 110 for the primary screen displays the first window 1. A gesture, shown as a drag 700 (though a flick 704 could be used), is received by the gesture capture region 120 for the primary screen 104. The gesture causes the window 1 to move to an inactive display position and the window 2 to move to an active display position on touch sensitive display 110.
With reference to FIG. 10B, the touch sensitive display 110 for the primary screen displays the second window 2. A gesture, shown as a drag 700 (though a flick 704 could be used), is received by the gesture capture region 120 for the primary screen 104. In response, the second window 2 moves in the direction indicated, causing the third window 3 to move into the active display position in the window stack, whereby the touch sensitive display 110 now displays the third window 3.
In FIG. 10C, the touch sensitive display 110 for the primary screen displays the third window 3. A gesture, shown as a drag 700 (though a flick 704 could be used), is received by the gesture capture region 120 for the primary screen 104 (not shown). The user, by the gesture, attempts to move “beyond” the object stack for the primary screen 104. The gesture causes the third window 3 (or a desktop at an end of the stack) to move partially off screen (to the left) followed by a rapid return, such as by a snap or rubber-band effect, to the primary display 104. This result communicates to the user that the window (or desktop) cannot be moved in the indicated direction as the end of the object stack has been reached. A similar result would have been realized in FIG. 10A had the drag gesture 700 been in the opposite direction.
Referring to FIGS. 11A-C, window navigation when the device 100 is in the open state is depicted. In the following examples, an object stack shown for the sake of simplicity as being common to both touch sensitive displays 110, 114 includes first desktop D1, (minimized) first and second windows 1, 2, and second desktop D2. In FIG. 11A, the touch sensitive display 110 for the primary screen displays the first desktop D1, and the touch sensitive display 114 for the primary screen displays the first window 1. A gesture, shown as a drag 700 (though a flick 704 could be used), is received by the gesture capture region 124 for the secondary screen 108 (or by the gesture capture region 120 for the primary screen 104). Referring to FIG. 11B, the gesture causes the first window 1 to move in the direction indicated, causing the first window 1 to move from the active display position for the secondary screen 108 to the active display position for the primary screen 104 and the first desktop D1 to an inactive display position for the primary screen 104. The second window 2 has moved from an inactive display position into the active display position for the secondary screen 108, whereby the touch sensitive display 114 now displays the second window 2.
With further reference to FIG. 11B, the gesture capture region 124 (or the gesture capture region 120 for the primary screen 104) receives a further gesture from the user, depicted as a drag gesture 700 (though a flick gesture 704 could be employed). Referring to FIG. 11C, the gesture causes the second window 2 to move in the direction indicated, causing the second window 2 to move into the active display position for the primary screen 104 and the first window 1 to an inactive display position for the primary screen 104. The second desktop D2 has moved from an inactive display position into the active display position for the secondary screen 108, whereby the touch sensitive display 114 now displays the second desktop D2.
In FIG. 11C, a gesture, shown as a drag 700 (though a flick 704 could be used), is received by the gesture capture region 124 for the primary screen 104 (or by the gesture capture region 120 for the primary screen 104). The user, by the gesture, attempts to move “beyond” the stack for the secondary screen 108. The gesture causes the second desktop 2 to move partially off screen (to the left) followed by a rapid return, such as by a snap or rubber-band effect, to the secondary screen 108. This result communicates to the user that the windows cannot be moved in the indicated direction as the end of the stack has been reached.
In the examples above, a gesture in the opposing direction would cause an opposite result. For example, in FIG. 11B, a drag in the opposite direction would cause the touch sensitive display 110 of the primary screen to display the second window 2, and the touch sensitive display 114 of the primary screen to display the second desktop D2.
In the examples above, it is to be understood that one or more other gesture(s) may accomplish the same functional result. The mapping of specific gestures to produce a specified result may be varied to produce a virtually endless array of possible relationships.
Navigation in landscape display orientation is, in one configuration, similar to navigating in portrait display orientation. One difference is the orientation of the displayed image stack, which may be oriented clockwise or counter-clockwise, depending on the direction of rotation of the device 100. Referring to FIGS. 16A-B, the device 100 is open and the primary and secondary touch sensitive displays 110 and 114 are in landscape display orientation and provide first and second desktops D1 and D2, respectively (FIG. 16A). The first and second desktops form an ordered set of desktops. The secondary screen 108 (or primary screen 104) detects a user gesture on the touch sensitive display 114. In one example, the user gesture 700 is a drag and, in another example, the user gesture is a flick. As shown in FIG. 16B, the second desktop D2 has moved in the direction indicated to an active display position (such as in a respective frame buffer or stack) or state for the primary screen and is displayed by the touch sensitive display 110, and the first desktop D1 has moved from an active to an inactive display position for the primary screen. A third desktop D3 has moved from an inactive to an active display position for the secondary screen and is now displayed by the touch sensitive display 114. Commonly, only a drag beyond approximately one-half of a display's or screen's height is considered to be a successful desktop navigation operation.
As further shown in FIG. 16B, the secondary screen 108 (or primary screen 104) detects a further user gesture on the touch sensitive display 114. As shown in FIG. 16C, the third desktop D3 has moved in the direction indicated to an active display position for the primary screen and is displayed by the touch sensitive display 110, and the second desktop D2 has moved from an active display position to an inactive position in a for the primary screen. A fourth desktop D4 has moved from an inactive to an active display position for the secondary screen and is now displayed by the touch sensitive display 114.
Although not shown, the user could use an opposing gesture sensed in the touch sensitive display 110 to move a desktop from top-to-bottom. For example, in FIG. 16B the user could use a downward drag or flick gesture 700, 704 in the touch sensitive display 110 to cause the first desktop D1 to move from an inactive display position to an active display position for the touch sensitive display 110, thereby causing the second desktop D2 to move to an active position for the touch sensitive display 114 and the third desktop D3 to move from an active to an inactive display position for the secondary screen.
A further example will be discussed with reference to FIGS. 12A-B. The object stack includes first and second windows 1, 2. First and second windows 1, 2 are controlled by different single-screen applications. In other configurations, the first and second windows are controlled by a common multi-screen application or by different multi-screen and single-screen applications. In FIG. 12A, the primary screen 104 displays the first desktop D1, and the secondary screen 108 displays first window 1. A drag (or flick) gesture is received by the gesture capture region 120 or 124. In response and as shown in FIG. 12B, the primary screen 104 now displays the first window 1, and the secondary screen 108 now displays the second window 2. Stated another way, the first and second desktops D2 and D3 are in inactive display positions while the first and second windows are in active display positions.
Referring to FIGS. 15A-C, window navigation when the device 100 is in the closed state and landscape display orientation is depicted. In the following examples, the object stack includes first, second, and third windows 1, 2, 3. First, second, and third windows 1, 2, 3 are controlled by different single-screen applications. In other configurations, the first, second, and third windows are controlled by a common multi-screen application or by different multi-screen and single-screen applications. In FIG. 15A, the touch sensitive display 110 for the primary screen displays the first window 1. A gesture, shown as an upward drag 700 (though an upward flick 704 could be used), is received by the gesture capture region 120 for the primary screen 104. The gesture causes the window 1 to move to an inactive display position and window 2, the next window in the stack, to move to an active display position.
With reference to FIG. 15B, the touch sensitive display 110 for the primary screen now displays the second window 2. A gesture, shown as an upward drag 700 (though an upward flick 704 could be used), is received by the gesture capture region 120 for the primary screen 104. In response, the second window 2 moves in the direction indicated to an inactive display position, causing the third window 3 to move into the active display position in the window stack, whereby the touch sensitive display 110 now displays the third window 3 (FIG. 15C).
In FIG. 15C, a further gesture, particularly an upward drag 700 or flick 704, received by the gesture capture region 120 for the primary screen 104 would attempt to move “beyond” the object stack for the primary screen 104. The gesture would cause the third window 3 to move partially off screen (upwards) followed by a rapid return, such as by a snap or rubber-band effect, to the primary display 104. This result communicates to the user that the windows cannot be moved into the indicated direction as the end of the object stack has been reached.
An example will now be discussed with reference to FIG. 13. A drag gesture 700 is received in step 1300 by one or more of a gesture capture region 120, 124 and a touch sensitive display 110, 114. In response, middleware 520, for each active touch sensitive display, moves, in step 1304, each currently displayed image from an active to an inactive display position or to an active display position of the other touch sensitive display. The logic then terminates in step 1308.
For example in one alternative embodiment, the desktops are moved by gestures received in a gesture capture region 120, 124, and windows are moved by gestures received in a touch sensitive display 110, 114.
receiving a first gesture through one of a first touch sensitive screen of a communications device or a second touch sensitive screen of the communications device, wherein the first touch sensitive screen comprises a first display area and a first gesture capture region within the first touch sensitive screen but which is outside of the first display area and not capable of rendering a displayed image, wherein the second touch sensitive screen comprises a second display area and a second gesture capture region within the second touch sensitive screen but which is outside of the second display area and not capable of rendering a displayed image, and wherein the first gesture is received though one of the first or second display areas or one of the first or second gesture capture areas;
in response to receiving the first gesture, ceasing to display a first displayed desktop on the first display area of the first touch sensitive screen and commencing to display the first displayed desktop on the second display area of the second touch sensitive screen;
receiving a second gesture though the other of the first or second display areas or the first or second gesture capture areas; and
in response to receiving the second gesture, ceasing to display a first displayed window of an application on the first display area of the first touch sensitive screen and commencing to display the first displayed window of the application on the second display area of the second touch sensitive screen,
wherein the first displayed window on the second display area of the second touch sensitive screen at least partially obscures the first displayed desktop on the second display area of the second touch sensitive screen,
wherein the gesture received through the first or second gesture capture region enables navigation through an ordered set of desktops, and
wherein the gesture received through the first or second display areas enables navigation through an ordered set of windows.
2. The method of claim 1, wherein the first displayed desktop is one of the ordered set of desktops.
3. The method of claim 1, wherein the application window is one of the ordered set of windows.
4. The method of claim 3, wherein the gesture received by the first or second gesture capture regions enables navigation through only the ordered set of desktops and the first or second gestures received by the touch sensitive display enables navigation through only the ordered set of windows.
5. The method of claim 1, wherein the communication device is in an opened state.
6. The method of claim 1, wherein the received first gesture is a drag gesture, wherein the first and second touch sensitive screens are each in the portrait display orientation, wherein the first display area of the first touch sensitive screen thereafter displays a second displayed desktop different from the first displayed desktop, wherein a third displayed desktop different from the first displayed desktop was previously displayed by the second display area of the second touch sensitive screen, wherein the first and second displayed desktops, in response, are in active positions and the third displayed desktop is in an inactive position.
determining, by a sensor, that a dual display communication device is in an open state, wherein, in the open state, a first touch sensitive screen of the communication device and a second touch sensitive screen of the communication device are enabled, wherein the first touch sensitive screen comprises a first display area and a first gesture capture region within the first touch sensitive screen but which is outside of the first display area and not capable of rendering a displayed image and wherein the second touch sensitive screen comprises a second display area and a second gesture capture region within the second touch sensitive screen but which is outside of the second display area and not capable of rendering a displayed image;
receiving a first gesture though one of the first or second display areas or one of the first or second gesture capture areas;
in response to receiving the first gesture, ceasing to display a first displayed desktop of a desktop stack on the first display area and displaying a different second displayed desktop of the desktop stack on the first display area;
in response to receiving the first gesture, ceasing to display the different second displayed desktop of the desktop stack on the second display area and displaying a different third desktop of the desktop stack on the second display area;
in response to receiving the second gesture, ceasing to display a first displayed window of an application on the first display area and commencing to display the first displayed window of the application on the second display area,
wherein the first displayed window on the second touch sensitive display at least partially obscures the first displayed desktop on the second display area,
wherein the gesture received through the first or second display area enables navigation through an ordered set of windows.
9. The method of claim 8, wherein the first gesture is received through the first of second gesture capture region, and wherein the first and second desktops are adjacent members of an ordered set of desktops in a desktop stack.
10. The method of claim 8, wherein the first gesture received enables navigation through the ordered set of desktops in the desktop stack.
11. The method of claim 8, wherein the received first gesture is a drag gesture, wherein the first touch sensitive screen is in the portrait display orientation, wherein the first displayed desktop was previously in an active state and the second displayed desktop in an inactive state, and wherein, in response, the first displayed desktop is now in the inactive state and the second displayed desktop is now in the active state.
12. A non-transient computer readable medium comprising processor executable instructions operable to perform the steps of claim 8.
13. A dual display communication device, comprising:
a first touch sensitive screen comprising a first display area and a first gesture capture region within the first touch sensitive screen but which is outside of the first display area and not capable of rendering a displayed image;
a second touch sensitive screen touch sensitive screen but which is outside of the second display area and not capable of rendering a displayed image; and
middleware operable to perform the following operations: determine, based on a sensor, that the dual display communication device is in an open state, wherein, in the open state, the first display area is enabled and the second display area is enabled; receive a first gesture though one of the first or second display areas or one of the first or second gesture capture areas; in response to receiving the first gesture, ceasing to display a first displayed desktop on the first display area and commencing to display the first displayed desktop on the second display area; receive a second gesture though the other of the first or second display areas or the first or second gesture capture areas; and in response to receiving the second gesture, ceasing to display a first displayed window of an application on the first display area and commencing to display the first displayed window of the application on the second display area,
wherein the first displayed window on the second display area at least partially obscures the first displayed desktop on the second display area,
wherein the gesture received through the first or second gesture capture region enables navigation through an ordered set of windows, and
wherein the gesture received through the first or second display area enables navigation through an ordered set of desktops.
14. The device of claim 13, wherein the first gesture is received through the first or second display areas, and wherein the first displayed desktop is one of the ordered set of desktops.
15. The device of claim 13, wherein the second gesture is received through the first or second gesture capture regions, and wherein the application window of the application is one of the ordered set of windows.
16. The device of claim 13, wherein the second gesture is received through the first or second gesture capture regions, and wherein the first and second windows are adjacent members of the ordered set of windows.
17. The device of claim 13, wherein the received first gesture is a drag gesture, wherein the first touch sensitive display is in the portrait display orientation, wherein the communication device has opened and closed states, wherein the first displayed desktop was previously in an active state, and wherein, in response, the first displayed desktop is now in the inactive state.
18. The device of claim 13, wherein the received first gesture is a drag gesture, wherein a gesture received by the first and second gesture capture regions enables navigation through only the ordered set of windows and a gesture received by the first and second display areas enables navigation through only the ordered set of desktops.
a second touch sensitive screen comprising a second display area and a second gesture capture region within the second touch sensitive screen but which is outside of the second display area and not capable of rendering a displayed image;
a processor coupled with the first touch sensitive screen and the second touch sensitive screen; and
a memory coupled with and readable by the processor and storing therein a set of instructions which, when executed by the processor, causes the processor to: receive a first gesture through one of the first touch sensitive screen or the second touch sensitive screen of the communications device, wherein the first gesture is received though one of the first or second display areas or one of the first or second gesture capture areas; in response to receiving the first gesture, cease to display a first displayed desktop on the first display area of the first touch sensitive screen and commence to display the first displayed desktop on the second display area of the second touch sensitive screen; receive a second gesture though the other of the first or second display areas or the first or second gesture capture areas; and in response to receiving the second gesture, cease to display a first displayed window of an application on the first display area of the first touch sensitive screen and commence to display the first displayed window of the application on the second display area of the second touch sensitive screen, wherein the first displayed window on the second display area of the second touch sensitive screen at least partially obscures the first displayed desktop on the second display area of the second touch sensitive screen, wherein the gesture received through the first or second gesture capture regions enables navigation through an ordered set of desktops, and wherein the gesture received through the first or second display areas enables navigation through an ordered set of windows.
6243645 June 5, 2001 Moteki et al.
6385331 May 7, 2002 Harakawa et al.
6861946 March 1, 2005 Verplaetse et al.
6977643 December 20, 2005 Wilbrink et al.
7917584 March 29, 2011 Arthursson
7936341 May 3, 2011 Weiss
7938721 May 10, 2011 Miyamoto et al.
8171427 May 1, 2012 Chung et al.
8194043 June 5, 2012 Cheon et al.
8291344 October 16, 2012 Chaudhri
8504936 August 6, 2013 Gimpl et al.
8527892 September 3, 2013 Sirpal et al.
8599106 December 3, 2013 Gimpl et al.
8599151 December 3, 2013 Narita
8648825 February 11, 2014 Sirpal et al.
8650508 February 11, 2014 Lim
8704781 April 22, 2014 Kii
8739053 May 27, 2014 Chen et al.
8762896 June 24, 2014 Lee et al.
8786559 July 22, 2014 Hogan
8810533 August 19, 2014 Chen
8856688 October 7, 2014 Tseng
9019214 April 28, 2015 Sirpal
9026923 May 5, 2015 Sirpal et al.
9046992 June 2, 2015 Sirpal
9052801 June 9, 2015 Sirpal et al.
9075558 July 7, 2015 Reeves et al.
9182884 November 10, 2015 Westerman et al.
20020075289 June 20, 2002 Hatori et al.
20030179243 September 25, 2003 Numano
20030182597 September 25, 2003 Coha et al.
20040172279 September 2, 2004 Carolan et al.
20050030255 February 10, 2005 Chiu
20060161868 July 20, 2006 Van Dok et al.
20060206825 September 14, 2006 Karlheinz et al.
20060227106 October 12, 2006 Hashimoto et al.
20070192749 August 16, 2007 Baudisch
20080242359 October 2, 2008 Seol et al.
20090006292 January 1, 2009 Block
20090027354 January 29, 2009 Perski et al.
20090204925 August 13, 2009 Bhat
20090209350 August 20, 2009 Kelly et al.
20090259967 October 15, 2009 Davidson et al.
20090289874 November 26, 2009 Ha
20090298537 December 3, 2009 Choi
20090303187 December 10, 2009 Paliakoff
20090303231 December 10, 2009 Robinet et al.
20100083154 April 1, 2010 Takeshita
20100115473 May 6, 2010 Reeves et al.
20100162128 June 24, 2010 Richardson et al.
20100175018 July 8, 2010 Petschnigg et al.
20100180297 July 15, 2010 Levine et al.
20100227650 September 9, 2010 Kim et al.
20100231536 September 16, 2010 Chaudhri et al.
20100248788 September 30, 2010 Yook et al.
20100298032 November 25, 2010 Lee
20100321319 December 23, 2010 Hefti
20110006971 January 13, 2011 Ebey et al.
20110025601 February 3, 2011 Wilson et al.
20110039603 February 17, 2011 Kim et al.
20110047459 February 24, 2011 Van Der Westhuizen
20110069021 March 24, 2011 Hill
20110074696 March 31, 2011 Rapp et al.
20110074710 March 31, 2011 Weeldreyer et al.
20110078624 March 31, 2011 Missig
20110081889 April 7, 2011 Gao et al.
20110090255 April 21, 2011 Wilson et al.
20110107272 May 5, 2011 Aguilar et al.
20110125398 May 26, 2011 Bos
20110209058 August 25, 2011 Hinckley et al.
20110209099 August 25, 2011 Hinckley et al.
20110209100 August 25, 2011 Hinckley et al.
20110209101 August 25, 2011 Hinckley et al.
20110209104 August 25, 2011 Hinckley et al.
20110228463 September 22, 2011 Matagne
20110260997 October 27, 2011 Ozaki
20110265021 October 27, 2011 Chien et al.
20110285631 November 24, 2011 Imamura et al.
20120050779 March 1, 2012 Tani et al.
20120056817 March 8, 2012 Griffin et al.
20120081293 April 5, 2012 Sirpal et al.
20120081306 April 5, 2012 Sirpal et al.
20120081310 April 5, 2012 Schrock
20120084678 April 5, 2012 Sirpal et al.
20120084679 April 5, 2012 Sirpal et al.
20120084680 April 5, 2012 Gimpl et al.
20120084690 April 5, 2012 Sirpal et al.
20120084700 April 5, 2012 Sirpal et al.
20120084725 April 5, 2012 Sirpal et al.
20120084736 April 5, 2012 Sirpal
20120242599 September 27, 2012 Seo et al.
20150020021 January 15, 2015 Marr et al.
20160034176 February 4, 2016 Sirpal
20160062593 March 3, 2016 Schrock
2662553 December 2004 CN
1573648 February 2005 CN
1933883 March 2007 CN
1941974 April 2007 CN
1949905 April 2007 CN
101038737 September 2007 CN
101661383 March 2010 CN
101727268 June 2010 CN
102084325 June 2011 CN
102150096 August 2011 CN
19739284 March 1999 DE
1729204 December 2006 EP
2166437 March 2010 EP
2169949 March 2010 EP
2214088 August 2010 EP
2226713 September 2010 EP
2309368 April 2011 EP
2309369 April 2011 EP
H09-305262 November 1997 JP
H11-242539 September 1999 JP
2002-259001 September 2002 JP
2003-280622 October 2003 JP
2005-284886 October 2005 JP
2007-109240 April 2007 JP
2009-211547 September 2009 JP
2010-015239 January 2010 JP
2010-039636 February 2010 JP
2010-92340 April 2010 JP
2010-134039 June 2010 JP
2010-160581 July 2010 JP
2010-211400 September 2010 JP
WO 2005/119404 December 2005 WO
WO 2009/067224 May 2009 WO
WO 2009/137419 November 2009 WO
WO 2010/001699 January 2010 WO
WO 2010/010835 January 2010 WO
WO 2010/028405 March 2010 WO
WO 2010/028406 March 2010 WO
WO 2010/111391 September 2010 WO
Official Action for European Patent Application No. 11829890, dated Dec. 5, 2017, 8 pages.
Official Action (with English translation) for Japanese Patent Application No. 2013-531849, dated May 10, 2016, 5 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180058029, dated May 10, 2016, 5 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180058029, dated Jan. 20, 2017, 12 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180058029, dated Oct. 25, 2017, 11 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180056896, dated Mar. 18, 2016, 24 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180056896, dated Oct. 19, 2016, 24 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180056896, dated Apr. 21, 2017, 28 pages.
Official Action (with English translation) for Japanese Patent Application No. 2013-531788, dated Jul. 26, 2016, 8 pages.
Offical Action (with English translation) for Chinese Patent Application No. 201180057378, dated Oct. 13, 2016, 15 pages.
Decision to Grant (with English translation) for Chinese Patent Application No. 201180057378, dated Mar. 28, 2017, 5 pages.
Official Action for European Patent Application No. 11829757.1, dated Dec. 11, 2017 4 pages.
U.S. Appl. No. 14/945,844, filed Nov. 19, 2015, Gimpl et al.
“Lapdock™ for Motorola ARIX,” at http://www.motorola.com/Consumers/US-EN/Consumer-Product-and-Services/Mobile . . . , accessed Apr. 18, 2011, 1 page.
“Motorola ATRIX 4G Laptop Dock Review,” at http://www.phonearena.com/reviews/Motorola-Atrix-4G-Laptop-Dock-Review_id2667, Mar. 2, 2011, 6 pages.
Website entitled, “Sony Tablet,” at store.sony.com/webapp/wcs/stores/servlet/CategoryDisplay?catalogId=10551&storeId=10151&Iangld=-1&categoryId=8198552921644795521, 2011, 3 pages.
Bretzner et al., “Hand Gesture Recognition using Multi-Scale Colour Features, Hierarchical Models and Particle Filtering,” Proceedings Fifth IEEE Conference on Automatic Face and Gesture Recognition, 2002, pp. 1-6.
Burns, C., “Motorola ATRIX 4G Laptop Dock Review,” at http://androidcommunity.com/motorola-atrix-4g-laptop-dock-review-20110220/, Feb. 20, 2011, 5 pages.
Catacchio, Chad, “This smartphone has two huge screens . . . that rotate,” The Next Web at http://thenextweb.com/asia/2010/10/07/this-smartphone-has-two-huge-screens-that-rotate/, Jul. 21, 2011, 2 pages.
Stein, S., “How does the Motorola Atrix 4G Lapdock compare with a laptop?” Crave—CNET, at http://news.cnet.com/8301-17938_105-20031251-1.html, Feb. 9, 2011, 7 pages.
International Search Report and Written Opinion for International (PCT) Patent Application No. PCT/US11/53937, dated Feb. 27, 2013 11 pages.
International Preliminary Report on Patentability for International (PCT) Patent Application No. PCT/US11/53937, dated Apr. 11, 2013 10 pages.
International Search Report and Written Opinion for International (PCT) Patent Application No. PCT/US2011/053771, dated Feb. 24, 2012 9 pages.
International Preliminary Report on Patentability and Written Opinion for International (PCT) Patent Application No. PCT/US2011/053771, dated Apr. 11, 2013 8 pages.
International Search Report and Written Opinion for International (PCT) Patent Application No. PCT/US2011/053773, dated Feb. 14, 2012 11 pages.
International Preliminary Report on Patentability for International (PCT) Patent Application No. PCT/US2011/053773, dated Apr. 11, 2013 12 pages.
International Search Report and Written Opinion for International (PCT) Patent Application No. PCT/US11/53776, dated Feb. 13, 2012 8 pages.
International Preliminary Report on Patentability for International (PCT) Patent Application No. PCT/US11/53776, dated Apr. 11, 2013 7 pages.
International Search Report and Written Opinion for International (PCT) Patent Application No. PCT/US11/53779, dated Feb. 13, 2012 7 pages.
International Preliminary Report on Patentability for International (PCT) Patent Application No. PCT/US11/53779, dated Apr. 11, 2013 6 pages.
International Search Report and Written Opinion for International (PCT) Patent Application No. PCT/US11/53781, dated Feb. 13, 2012 8 pages.
International Preliminary Report on Patentability for International (PCT) Patent Application No. PCT/US11/53781, dated Apr. 11, 2013 7 pages.
European Search Report for European Patent Application No. 11829844.7, dated Jul. 16, 2014, 9 pages.
International Search Report for International (PCT) Patent Application No. PCT/US2011/052932, dated Apr. 27, 2012 3 pages.
International Preliminary Report on Patentability and Written Opinion for International (PCT) Patent Application No. PCT/US2011/052932, dated Apr. 11, 2013 7 pages.
Extended European Search Report and Search Opinion for European Patent Application No. 11829763.9, dated Jun. 26, 2014 7 pages.
International Search Report for International (PCT) Patent Application No. PCT/US2011/053032, dated Apr. 27, 2012 3 pages.
International Preliminary Report on Patentability and Written Opinion for International (PCT) Patent Application No. PCT/US2011/053032, dated Apr. 11, 2013 5 pages.
International Search Report for International (PCT) Patent Application No. PCT/US2011/052815, dated Apr. 27, 2012 3 pages.
International Preliminary Report on Patentability and Written Opinion for International (PCT) Patent Application No. PCT/US2011/052815, dated Apr. 11, 2013 6 pages.
International Search Report for International (PCT) Patent Application No. PCT/US2011/052802, dated Apr. 27, 2012 4 pages.
International Preliminary Report on Patentability and Written Opinion for International (PCT) Patent Application No. PCT/US2011/052802, dated Apr. 11, 2013 7 pages.
Official Action for U.S. Appl. No. 13/247,977, dated Apr. 2, 2013 17 pages.
Final Action for U.S. Appl. No. 13/247,977, dated Aug. 1, 2013 22 pages.
Official Action for U.S. Appl. No. 13/247,977, dated Mar. 26, 2014 17 pages.
Final Action for U.S. Appl. No. 13/247,977, dated Oct. 23, 2014 20 pages.
Official Action for U.S. Appl. No. 13/223,747, dated May 31, 2013 22 pages.
Final Action for U.S. Appl. No. 13/223,747, dated Dec. 18, 2013 22 pages.
Official Action for U.S. Appl. No. 13/223,747, dated Jun. 10, 2014 25 pages.
Notice of Allowance for U.S. Appl. No. 13/223,747, dated Dec. 3, 2014 8 pages.
Official Action for U.S. Appl. No. 13/223,674, dated Nov. 20, 2012 19 pages.
Final Action for U.S. Appl. No. 13/223,674, dated Jun. 7, 2013 21 pages.
Official Action for U.S. Appl. No. 13/223,674, dated Aug. 1, 2014 24 pages.
Notice of Allowance for U.S. Appl. No. 13/223,674, dated Nov. 26, 2014 13 pages.
Notice of Allowance for U.S. Appl. No. 13/247,822, dated Oct. 2, 2013, 15 pages.
Official Action for U.S. Appl. No. 13/223,809, dated May 1, 2013, 17 pages.
Final Action for U.S. Appl. No. 13/223,809, dated Nov. 20, 2013, 20 pages.
Office Action for U.S. Appl. No. 13/223,809, dated Jun. 16, 2014, 18 pages.
Office Action for U.S. Appl. No. 13/223,727, dated May 14, 2013, 11 pages.
Final Action for U.S. Appl. No. 13/223,727, dated Nov. 20, 2013, 13 pages.
Official Action for U.S. Appl. No. 13/223,727, dated Jun. 16, 2014, 13 pages.
Notice of Allowance for U.S. Appl. No. 13/223,727, dated Nov. 20, 2014, 8 pages.
Official Action for U.S. Appl. No. 13/223,697, dated May 21, 2013, 23 pages.
Final Action for U.S. Appl. No. 13/223,697, dated Dec. 2, 2013, 23 pages.
Official Action for U.S. Appl. No. 13/223,697, dated Apr. 1, 2014, 23 pages.
Final Action for U.S. Appl. No. 13/223,697, dated Oct. 14, 2014 16 pages.
Official Action for U.S. Appl. No. 13/629,085, dated Sep. 12, 2014 10 pages.
Official Action for U.S. Appl. No. 13/629,173, dated May 22, 2014, 14 pages.
Final Action for U.S. Appl. No. 13/629,173, dated Sep. 12, 2014 17 pages.
Official Action for U.S. Appl. No. 12/948,667, dated Sep. 26, 2012 15 pages.
Final Action for U.S. Appl. No. 12/948,667, dated May 24, 2013 17 pages.
Official Action for U.S. Appl. No. 12/948,667, dated Jul. 24, 2014 17 pages.
Official Action for U.S. Appl. No. 12/948,675, dated Oct. 31, 2014 15 pages.
Official Action for U.S. Appl. No. 12/948,676, dated Oct. 11, 2012 8 pages.
Final Action for U.S. Appl. No. 12/948,676, dated May 9, 2013 12 pages.
Office Action for U.S. Appl. No. 12/948,676, dated Sep. 26, 2013 11 pages.
Final Action for Appl. No. 12/948,676, dated Apr. 24, 2013 15 pages.
Office Action for U.S. Appl. No. 12/948,684, dated Dec. 20, 2013 12 pages.
Final Action for Appl. No. 12/948,684, dated Mar. 29, 2013 13 pages.
Official Action for U.S. Appl. No. 12/948,684, dated Sep. 26, 2013 12 pages.
Official Action for U.S. Appl. No. 12/948,684, dated May 9, 2014 8 pages.
Official Action for U.S. Appl. No. 12/948,684, dated Oct. 3, 2014 18 pages.
Official Action for U.S. Appl. No. 13/187,026, dated Jun. 18, 2013 5 pages.
Official Action for U.S. Appl. No. 13/187,026, dated Nov. 20, 2013 10 pages.
Notice of Allowance for U.S. Appl. No. 13/187,026, dated Mar. 10, 2014 8 pages.
“Connect the Brix Phones together for a Larger Media Screen,” 2014, retrieved Jul. 13, 2014 from http://walyou.com/connect-the-brix-phones-together-for-a-larger-media-screen/, 5 pages.
“Microsoft Patents Dual-Screen Phone From the Future (Video & Pics),” retrieved Jul. 13, 2014 from www.wp7connect.com/2011/07/09/microsoft-patents-dual-screen-phone-from-the-future-video-pics/, 2 pages.
“Palm Pre employing HTML5 equips multi-task operation,” (English translated title) Nikkei Business Publications, Inc., Aug. 10, 2009, vol. 1010, pp. 38-43.
Hinckley et al., “Stitching: Pen Gestures that Span Multiple Displays,” Proc. of the Working Conference on Advanced Visual Interfaces, Jan. 1, 2004, pp. 23-31.
Webb, “Samsung files patent for a phone with Two Screens to make reading, gaming and conference calls easier,” MAILOnline, Dec. 18, 2013, retrieved Jul. 13, 2014 from http://www.dailymail.co.uk/sciencetech/article-2525812/Double-vision-Samsung-unveils-phone-two-screens-easier-reading-gaming-video-conferencing.html, 19 pages.
Official Action (with English translation) for Chinese Patent Application No. 2011800515271, dated Dec. 31, 2015, 22 pages.
Extended European Search Report for European Patent Application No. 11829890, dated Jun. 8, 2015, 8 pages.
Official Action (with English translation) for Japanese Patent Application No. 2013-531849, dated Oct. 27, 2015, 8 pages.
Official Action for Mexican Patent Application No. MX/a/2013/003249, dated Jan. 14, 2016, 3 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180058029, dated Sep. 6, 2015, 15 pages.
Extended European Search Report for European Patent Application No. 11829839.7, dated May 11, 2015, 6 pages.
Official Action (with English translation) for Japanese Patent Application No. 2013-531785, dated Dec. 1, 2015, 3 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180056896, dated Jul. 21, 2015, 26 pages.
Official Action (with English translation) for Japanese Patent Application No. 2013-531788, dated Nov. 4, 2015, 10 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180057652.3, dated Nov. 20, 2015, 24 pages.
Official Action (with English translation) for Japanese Patent Application No. 2013-531674, dated Nov. 10, 2015, 7 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180057378, dated Oct. 27, 2015, 16 pages.
Extended European Search Report and Search Opinion for European Patent Application No. 11829757.1, dated May 7, 2015 8 pages.
Notice of Allowance (with English translation) for Japanese Patent Application No. 2013-531667, dated Nov. 10, 2015, 2 pages.
Official Action (with English summary) for Mexican Patent Application No. MX/a/2013/003424, dated Jan. 13, 2016, 5 pages.
Official Action for U.S. Appl. No. 13/247,977, dated Jul. 13, 2015 21 pages.
Final Action for U.S. Appl. No. 13/247,977, dated Dec. 10, 2015 20 pages.
Final Action for U.S. Appl. No. 13/223,809, dated Dec. 3, 2014, 20 pages.
Official Action for U.S. Appl. No. 13/223,809, dated Jun. 4, 2015, 22 pages.
Notice of Allowance for U.S. Appl. No. 13/629,085, dated Feb. 6, 2015 24 pages.
Official Action for U.S. Appl. No. 13/629,173, dated Jan. 30, 2015 20 pages.
Final Action for U.S. Appl. No. 13/629,173, dated Jun. 5, 2015 18 pages.
Notice of Allowance for U.S. Appl. No. 12/948,667, dated Jan. 7, 2015 10 pages.
Final Action for U.S. Appl. No. 12/948,675, dated May 6, 2015 18 pages.
Official Action for U.S. Appl. No. 12/948,676, dated Jan. 6, 2015 16 pages.
Final Action for U.S. Appl. No. 12/948,676, dated Aug. 20, 2015 29 pages.
Final Action for U.S. Appl. No. 12/948,684, dated Feb. 27, 2015 23 pages.
Notice of Allowance for U.S. Appl. No. 12/948,684, dated Feb. 16, 2016 5 pages.
Official Action for U.S. Appl. No. 14/844,970, dated Mar. 8, 2018, 22 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180056896, dated May 24, 2018, 20 pages.
Final Action for U.S. Appl. No. 14/844,970, dated Aug. 16, 2018, 25 pages.
Notice of Allowance (with English translation) for Chinese Patent Application No. 201180058029, dated Mar. 9, 2018, 5 pages.
Official Action (with English translation) for Chinese Patent Application No. 201180056896.X, dated Nov. 29, 2018 7 pages.
Official Action for U.S. Appl. No. 14/819,712, dated Dec. 21, 2018 17 pages.
Final Action for U.S. Appl. No. 14/844,970, dated May 23, 2019, 25 pages.
Official Action for U.S. Appl. No. 12/948,676, dated Apr. 5, 2019 24 pages.
Notice of Allowance (with English translation) for Chinese Patent Application No. 201180056896.X, dated Mar. 5, 2019 5 pages.
Official Action for European Patent Application No. 11829844.7, dated Apr. 5, 2019, 8 pages.
Patent Publication Number: 20160062554
Assignee: Z124 (Georgetown)
Inventors: Sanjiv Sirpal (Oakville), Alexander de Paz (Burlington), Paul E. Reeves (Oakville), Maxim Marintchenko (Vaughan)
Application Number: 14/839,727
International Classification: G06F 3/0481 (20130101); G06F 3/0482 (20130101); G06F 3/14 (20060101); G06F 3/0484 (20130101); G06F 3/0488 (20130101); G06F 3/041 (20060101); G06F 3/01 (20060101); G06F 3/0486 (20130101); G06F 3/16 (20060101); G06F 3/0483 (20130101); G06F 1/16 (20060101);