Patent Description:
Furthermore, <CIT> discloses a computer that includes a processor and first and second screen panels. The processor is provided in the first screen panel. The computer also includes a connector hinge by which the first and second screen panels are reversibly connected with each other. The screen panels each are individually capable of performing wireless communication. In one example implementation, when a user desires to operate the computer as a laptop computer, the two screen panels may be inserted and locked into the two screen panel support structures via connector switches. The user may position a first of the screen panels, e.g., onto a work surface, e.g., the user's lap, and may position a keyboard onto the first screen panel to obtain a laptop configuration, wherein the keyboard may lie over one of the screens and can be removed from it at will whenever needed.

Additionally, <CIT> discloses a system and method that utilize a model of user input from a touch sensor capable of sensing location of a finger or object above a touch surface. In the electronic device, data representative of current user input to the electronic device is created. The model of user input is applied to the data representative of current user input to create data reflecting a prediction of a future user input event. That data is used to identify at least one particular response associated with the predicted future user input event. Data useful to implement graphical and application state changes is cached in a memory of the electronic device, the data including data reflecting a particular response associated with the predicted future user input. The cached data is retrieved from the memory of the electronic device and is used the data to implement the state changes.

The accompanying drawings illustrate implementations of the concepts conveyed in the present document. Features of the illustrated implementations can be more readily understood by reference to the following description taken in conjunction with the accompanying drawings. Like reference numbers in the various drawings are used wherever feasible to indicate like elements. Further, the left-most numeral of each reference number conveys the FIG. and associated discussion where the reference number is first introduced. Where space permits, elements and their associated reference numbers are both shown on the drawing page for the reader's convenience. Otherwise, only the reference numbers are shown.

The present concepts relate to devices, such as computing devices and associated input devices. The computing device can include one or more displays. The displays can be managed as one or more units. The units of the displays can be managed for a specific functionality. For instance, one unit can be used to present content as a graphical user interfaces (GUIs) while another unit is used to present other content as another GUI and still another unit can be used as an input device (e.g., the unit may function as a trackpad). The overall portion of the display dedicated to each unit and/or the location of the unit can be varied depending upon various conditions. For instance, in some cases, some units may be blocked from view. For example, a user may place the input device on the display. Units affected by the blockage can be reconfigured (e.g., their size and/or location on the display can be changed) to accommodate the blockage. The unit underlying the input device may be managed in a manner that reflects that it is not visible to the user.

Sometimes users can be disoriented or confused by sudden reconfiguration of the units. As such, the present concepts can provide visual indications to the user that indicate how the units are going to be reconfigured (e.g., before the input device is physically positioned on the display). The visual indications may be accomplished with one or more transition GUIs that reflect unit size and/or location on the display so that the user can understand what is happening. Further, the presence of the input device may make some units unsuitable for their present use. In such cases, the present concepts can assign an alternative functionality to those units. These aspects and others are described in more detail below.

Introductory <FIG> shows a system <NUM> that can include an example computing device <NUM> and an associated input device <NUM> that can communicate with computing device <NUM>. In this example, the associated input device <NUM> is a wireless detachable input device that can communicate with computing device <NUM> via one or more wireless technologies, such as Bluetooth, and/or near field communication (NFC), among others. In this case, the computing device <NUM> has first and second portions <NUM> and <NUM> that are rotatably secured together by a hinge assembly <NUM>. Other example computing devices could be manifest as a single portion (e.g., a tablet) or more than two portions. Example input devices <NUM> can include a keyboard, a touch pad, and/or a game controller, among others.

Displays <NUM> can be positioned on the first and/or second portions <NUM> and <NUM>. Displays often are touch sensitive displays that can detect pressure and/or proximity of nearby objects. Display <NUM>(<NUM>) can extend from a hinge end <NUM> to a distal end <NUM> to define a length L<NUM> and can also define a width W<NUM> that lies orthogonal to the length L<NUM>. Display <NUM>(<NUM>) can extend from a hinge end <NUM> to a distal end <NUM> to define a length L<NUM> and can also define a width W<NUM> that lies orthogonal to the length L<NUM>. The length and the width can define the display area (e.g., rectangle) of the display. Input device <NUM> can have a width W<NUM> that lies orthogonal to the length L<NUM>. In the illustrated case, the width of the input device is approximately the same as the width of the displays. In other cases, the input device could be narrower or wider than the displays.

System <NUM> can also include sensors <NUM> that can convey information about the position and/or movement of input device <NUM> relative to computing device <NUM>. In this example, sensors <NUM>(<NUM>) and <NUM>(<NUM>) are positioned on computing device <NUM> and sensors <NUM>(<NUM>) and <NUM>(<NUM>) are positioned on input device <NUM>. In one implementation, the sensors can be manifest as near field communication (NFC) sensors. NFC sensors can utilize electromagnetic induction between two loop antennas to communicate data between two devices. However, the present concepts can leverage changes to the electromagnetic fields to convey information about the position and/or movement of the input device <NUM> relative to the computing device <NUM>. This technique can be effective out to around <NUM>-<NUM> centimeters, for example. Alternatively, Bluetooth sensors (e.g., Bluetooth Low Energy (BLE)) beacons can be employed to track the location and/or movement of the input device relative to the computing device. Further, as mentioned above, the displays can have pressure and/or proximity sensors that can contribute position and/or movement information.

In this configuration, units <NUM> of displays <NUM> (e.g., of the display area) can be managed for specific functions. In <FIG>, unit <NUM>(<NUM>) includes all of display <NUM>(<NUM>) and unit <NUM>(<NUM>) includes all of display <NUM>(<NUM>). This relationship between the units and the displays can be fluid as will become apparent through the description below (e.g., the size, location, and/or number of units can change).

For purposes of explanation, assume that unit <NUM>(<NUM>) is being utilized to present a first graphical user interface (GUI) <NUM>(<NUM>) and unit <NUM>(<NUM>) is being utilized to present a second GUI <NUM>(<NUM>). For instance, the first GUI might be associated with a first application and the second GUI might be associated with a second application.

<FIG> capture specific instances of a process that may extend for a time duration. From one perspective, the instances captured in the series of <FIG> may be viewed as snapshots from a video progression from a time starting at <FIG> (e.g., original configuration and culminating at <FIG> (e.g., updated configuration)). Starting with <FIG>, the input device <NUM> is physically separated from the computing device <NUM>. The user can utilize the input device to control the computing device, such as the GUIs <NUM>. For instance, GUI 126A(<NUM>) could be associated with a first application and GUI 126A(<NUM>) could be associated with a second application. Now assume that the user decides to move the input device <NUM> and place it on the computing device's second portion <NUM>.

<FIG> shows user <NUM> moving the input device <NUM> toward the computing device <NUM> (in this case toward the second portion <NUM>). This movement, such as location and direction of movement, has been detected by the sensors <NUM>. Computing device <NUM> can predict that the movement will result in the input device <NUM> positioned on and occluding a portion of the second display <NUM>(<NUM>). As such, units on this display can be moved, such as to first display <NUM>(<NUM>). However, instantaneous reconfiguration of the units may disorient the user. Thus, GUIs 126B(<NUM>) and 126B(<NUM>) can be termed transition GUIs, which show an intervening configuration of the units 124B from the initial configuration (<FIG>) to the updated configuration of reconfigured units 124E of <FIG>.

<FIG> shows the initial transition GUIs 126B(<NUM>) and 126B(<NUM>). Transition GUI 126B(<NUM>) is moving to the left and GUI 126B(<NUM>) is moving to the right. At this point, each transition GUI 126B occupies a portion of the first display <NUM>(<NUM>) and the second display <NUM>(<NUM>). Here, transition GUI 126B(<NUM>) occupies a majority of first display <NUM>(<NUM>) but has moved to the left to create a space for transition GUI 126B(<NUM>). Similarly, transition GUI 126B(<NUM>) has moved to the right on second display <NUM>(<NUM>) where it still occupies a majority of the space and it also occupies the rightmost portion of first display <NUM>(<NUM>). In this case, subsequent transition GUIs are shown in <FIG>. The transition GUIs can provide a visual indication to the user that explains how the units are being reconfigured due to the predicted changing conditions (e.g., changing position of the input device).

To summarize in this example scenario starting at <FIG>, unit <NUM>(<NUM>) occupies all of the first display <NUM>(<NUM>) and unit <NUM>(<NUM>) occupies all of second display <NUM>(<NUM>) (e.g., the units are vertically arranged one atop the other). Eventually, in <FIG> the first unit <NUM>(<NUM>) and the second unit <NUM>(<NUM>) are both on first display <NUM>(<NUM>) in a side-by-side (e.g., horizontal relationship). <FIG> show the visual indictors that are presented to help the reader understand what is happening.

<FIG> shows a subsequent view where the user is continuing to move the input device <NUM> toward the computing device <NUM>. At this point transition GUIs 126C are positioned side-by-side on both displays <NUM>. The transition GUIs 126C are positioned on the first display <NUM>(<NUM>) and extend down onto second display <NUM>(<NUM>).

<FIG> shows a subsequent point where the input device <NUM> is partly over the second display <NUM>(<NUM>). The presence of the input device (e.g., sensed position) can be sensed by sensors <NUM> and/or by sensors of display <NUM>(<NUM>) (e.g., in the case where the display is a touch sensitive display). Transition GUIs 126D(<NUM>) and 126D(<NUM>) have moved away from the input device <NUM> and an additional transition GUI 126D(<NUM>) indicates a third unit 124D(<NUM>) and additional transition GUI indicates a fourth unit 124D(<NUM>) on the second display <NUM>(<NUM>). The third unit 124D(<NUM>) can be used for various functions, such as a touch sensitive tool bar, or a touch pad, among others. The fourth unit 124D(<NUM>) can be an area blocked by the input device and thus not used to present content or for receiving touch gestures (e.g., no corresponding GUI). In some implementations, the blocked fourth unit 124D(<NUM>) can be powered down (e.g., operated in a lower power state) to reduce resource usage (e.g., battery usage). For instance, the occluded fourth unit 124D(<NUM>) of the display could be powered down by turning off pixels in the case where the display is an OLED display, or turning off certain LEDs for an LCD display, among others.

<FIG> shows a subsequent point where the input device <NUM> is positioned on the second display <NUM>(<NUM>). Thus, fourth unit 124E(<NUM>) is blocked by the input device. Transition GUIs 126E(<NUM>) and 126E(<NUM>) are now side-by-side on the first display <NUM>(<NUM>). Transition GUI 126E(<NUM>) shows that unit 124E(<NUM>) of second display <NUM>(<NUM>) will be managed for a new functionality. The new functionality can be determined based upon the dimensions of the unit and/or be defined by the user. For instance, the dimensions of unit 124E(<NUM>) may not lend themselves to presenting content and may be better suited to other functionalities, such as a touch sensitive tool bar. The touch sensitive tool bar can be generic (e.g., tied to an operating system of the computing device). Alternatively, the touch sensitive tool bar could relate to a specific application. For instance, GUI 126E(<NUM>) could be dedicated to presenting content of an application, such as a word processing application, for example. GUI 126E(<NUM>) could present the controls for the application on the touch sensitive tool bar. Thus, more display area of first unit 124E(<NUM>) could be dedicated to presenting content at an aspect ratio that lends itself to content presentation and display area of the third unit 124E(<NUM>) can be used for the control aspects of the application in an aspect ratio that lends itself to tool bars.

<FIG> shows a subsequent point where the user slid the input device <NUM> up the second display <NUM>(<NUM>) against the first display <NUM>(<NUM>). Unit 124F(<NUM>) is now blocked by the input device and unit 124F(<NUM>) of the second display is now available to perform a functionality. Thus, unit 124F(<NUM>) can now include a GUI 126F(<NUM>) while unit 124F(<NUM>) does not. In this case, the functionality may be touch pad functionality to complement the input device, for instance. <FIG> and <FIG> show that any positioning of the input device <NUM> can be accommodated utilizing multiple units. For instance, if the input device was positioned at an intervening position, additional units could be managed above and below the input device.

<FIG> shows another system <NUM> that can include an example computing device <NUM> and an associated input device <NUM> that can communicate with computing device <NUM>. In this case, the input device <NUM> can be rotatably secured to the computing device by an input hinge <NUM>. One or more sensors <NUM> can provide sensor data relating to the orientation of the input device <NUM> to the computing device. In one implementation, the sensors can be positioned in the hinge itself. In another implementation, the sensors <NUM> can be positioned in the input device <NUM> and the computing device <NUM>, such as proximate to the input hinge <NUM>.

Starting at <FIG>, the input device <NUM> is positioned against the second portion <NUM> on the opposite side from the second display <NUM>(<NUM>) (e.g., input device is in a storage position). The first display <NUM>(<NUM>) is being managed as a single unit 224A(<NUM>). First GUI 226A(<NUM>) can be presented on first unit 224A(<NUM>). The second display <NUM>(<NUM>) is being managed as a single unit 224A(<NUM>). Second GUI 226A(<NUM>) can be presented on second unit 224A(<NUM>).

<FIG> shows the input device <NUM> rotated <NUM> degrees from the orientation of <FIG>. At this point, the first and second units 224B(<NUM>) and 224B(<NUM>) remain unchanged from <FIG>.

<FIG> shows an additional <NUM> degrees of rotation of the input device <NUM> from the orientation of <FIG>. This may be a working orientation, where the user positions the second portion <NUM> and the input device <NUM> on a work surface and uses the input device to control the computing device <NUM>. The first and second units 224C(<NUM>) and 224C(<NUM>) remain unchanged from <FIG>.

<FIG> shows an additional <NUM> degrees of rotation of the input device <NUM> from the orientation of <FIG> (<NUM> degrees of rotation relative to <FIG>). At this point, computing device <NUM> can predict that the user will continue to rotate the input device and that the input device will visually and/or physically block some of second display <NUM>(<NUM>). As such, units 224D(<NUM>) and 224D(<NUM>) can be changed as shown by GUIs 226D(<NUM>) and 226D(<NUM>). In this example, unit 224D(<NUM>) is narrowed from right to left and unit 224D(<NUM>) is extended up onto the vacated area of first display <NUM>(<NUM>). This is visually represented by transition GUIs 226D(<NUM>) and 226D(<NUM>).

<FIG> shows the input device <NUM> rotated to the <NUM>-degree orientation. The first unit 224E(<NUM>) as visualized by transition GUI 226E(<NUM>) is further narrowed and second unit 224E(<NUM>) continues to fill the occupied area as represented by transition GUI 226E(<NUM>). Further, a third unit 224E(<NUM>) is introduced on the upper left portion of second display <NUM>(<NUM>) as evidenced by transition GUI 226E(<NUM>). Thus, <FIG> and <FIG> show the user the changes to the displays to compensate for the presence of the input device <NUM> on the second portion <NUM> before such occurrence actually happens. Thus, the user can see what is changing and how it is changing.

<FIG> shows the input device <NUM> rotated <NUM> degrees from the orientation of <FIG> and now contacting the second display <NUM>(<NUM>). Units 224F(<NUM>) and 224F(<NUM>) as visualized by GUIs 226F(<NUM>) and 226F(<NUM>) are arranged side-by-side on the first display <NUM>(<NUM>). A third unit 224F(<NUM>) visualized by GUI 226F(<NUM>) covers areas of the second display that are not blocked by the input device. The shape of the third unit may lend itself to specific functions. For instance, the long narrow shape may facilitate a toolbar functionality for either or both of GUIs 226F(<NUM>) and 226F(<NUM>) better than associating the third unit with another application. The transition GUIs of <FIG> and <FIG> allow the user to be immediately ready to use the computing device as configured in <FIG> rather than the user having to pause to figure out how the computing device was reconfigured. Thus, the user can immediately start reading the content of GUIs 226F(<NUM>) and/or 226F(<NUM>) and/or using the input device <NUM>.

<FIG> collectively show another scenario where system <NUM> can create visual indications for the user as the configuration of the computing device changes.

<FIG> shows computing device <NUM> in a 'notebook' orientation with the second portion <NUM> positioned on a horizontal surface (HS) and the first portion <NUM> oriented about <NUM>-<NUM> degrees from the second portion. In this case, input device <NUM> is positioned in front of, but separated from, the second portion on the horizontal surface. The computing device can employ sensors <NUM>. In this case, the sensors can detect the orientation of the first and second portions in absolute terms and/or relative to one another. For instance, the sensors could be manifest as microelectromechanical systems (MEMS) sensors, such as inertial measurement units and/or inertial navigation systems, among others.

In this case, the first display <NUM>(<NUM>) is being managed as a single unit 324A(<NUM>) with a single GUI 326A(<NUM>). Similarly, second display <NUM>(<NUM>) is being managed as a single unit 324A(<NUM>) with a single GUI 326A(<NUM>). Note that in this case, text of the GUIs 326A is arranged horizontally. In this configuration, the horizontal orientation of the text runs parallel to a length of the displays <NUM> (e.g., parallel to the hinge axis (HA)).

<FIG> shows user <NUM> grasping the first portion <NUM> to change the configuration of the computing device <NUM>.

<FIG> shows the user turning the computing device <NUM> clockwise. The sensors <NUM> can detect this rotation. The computing device can make a prediction from the rotation and control the units <NUM> accordingly. Further, the computing device can provide indications to the user how the units will be controlled based upon the prediction. For instance, the prediction may be that the user will continue to rotate the computing device into an 'open book' configuration where the hinge axis is vertical (e.g., analogous to an open book where the spine is vertical and the text is horizontal on pages that are side by side). In this case, transition GUIs 326C(<NUM>) and 326C(<NUM>) show the text remaining horizontal despite the rotation of the computing device as an indication of how the units 324C are being managed.

<FIG> shows the user's rotation of the computing device <NUM> completed with the hinge axis (HA) now vertical and the first and second portions <NUM> and <NUM> supporting the computing device on the horizontal surface. In this case, the user does not see any sudden flipping of the text on GUIs 326D, instead the transition GUIs provided indications to the user how the units 324D and the GUIs where being managed during the configuration change. Further, in this case, the prediction allowed the GUI orientation to be maintained throughout the rotation despite the rotation of the device, thereby further maintaining continuity and predictability for the user.

In some implementations, there may be a range of relative orientations between the first and second portions <NUM> and <NUM> that can stably support the computing device <NUM>. For instance, the range might include angles between <NUM> degrees and <NUM> degrees, for instance. At other angles, the computing device may be prone to tipping over. Recall that in some implementations, the sensors <NUM> can detect the orientation of the first and second portions and the relative angle between the first and second portions and the orientations. For instance, the sensor data may indicate that the computing device is in a vertical open book orientation (such as in <FIG>) and the first and second portions are at an angle of <NUM> degrees relative to one another. When the computing device is in the vertical open book orientation, a sensory warning may be emitted if the orientation is outside of the stable range. For instance, if the user opens the angle to <NUM> degrees an audible warning may be emitted that the user should reduce the angle so that the computing device does not fall over.

<FIG> show how the computing device can be seamlessly transitioned from the displays positioned over and under (e.g., <FIG>) to side-by-side (e.g., <FIG>) as desired by the user. In either case, the input device can augment the functionality of the computing device so that all of the display area can be managed for content presentation as desired by the user.

<FIG> shows a system <NUM> that can accomplish display management concepts. For purposes of explanation, system <NUM> can include various computing device <NUM>, such as computing device <NUM>(<NUM>)-<NUM>(<NUM>). Computing device <NUM>(<NUM>) is similar to computing device <NUM>, <NUM>, and/or <NUM> introduced above that has two displays. Computing device <NUM>(<NUM>) is laptop computing device with a single display. Computing device <NUM>(<NUM>) is a smart phone type device. Computing device <NUM>(<NUM>) is a server type computer. Computing devices <NUM> can communicate via one or more networks (represented by lightning bolts <NUM>). In some cases, parentheticals are utilized after a reference number to distinguish like elements. Use of the reference number without the associated parenthetical is generic to the element.

<FIG> shows two device configurations <NUM> that can be employed by devices <NUM>. Individual devices <NUM> can employ either of configurations <NUM>(<NUM>) or <NUM>(<NUM>), or an alternate configuration. (Due to space constraints on the drawing page, one instance of each configuration is illustrated rather than illustrating the device configurations relative to each device <NUM>). Briefly, device configuration <NUM>(<NUM>) represents an operating system (OS) centric configuration. Configuration <NUM>(<NUM>) represents a system on a chip (SOC) configuration. Configuration <NUM>(<NUM>) is organized into one or more applications <NUM>, operating system <NUM>, and hardware <NUM>. Configuration <NUM>(<NUM>) is organized into shared resources <NUM>, dedicated resources <NUM>, and an interface <NUM> there between.

In either configuration <NUM>, the device <NUM> can include storage/memory <NUM>, a processor <NUM>, and/or a unit manager <NUM>. In some cases, the unit manager can include a configuration prediction module <NUM>.

The unit manager <NUM> can generate and manage units on one or more displays of the computing device <NUM>. The unit manager can decide how many units to employ on the displays, the dimensions of the units, and/or the function of the units. The unit manager can utilize various parameters as input to make these decisions. For instance, the unit manager can utilize various parameters about the configuration of the computing device. In some cases, the configuration prediction module <NUM> can analyze the parameters to predict a future configuration of the computing device. The unit manager <NUM> can compute how to manage the displays for the future configuration. The unit manager can generate transition GUIs to indicate to the user how the displays will be managed so the user is not surprised and/or confused by the changes.

In some configurations, each of computing devices <NUM> can have an instance of the unit manager <NUM>. In other cases, a remote unit manager may determine how to manage the computing device's displays. For instance, computing device <NUM>(<NUM>) could run a unit manager that receives sensed parameters from computing device <NUM>(<NUM>), determines how to manage the display of computing device <NUM>(<NUM>), and sends the display management information to computing device <NUM>(<NUM>).

The term "device," "computer," or "computing device" as used herein can mean any type of device that has some amount of processing capability and/or storage capability. Processing capability can be provided by one or more processors that can execute data in the form of computer-readable instructions to provide a functionality. Data, such as computer-readable instructions and/or user-related data, can be stored on storage, such as storage that can be internal or external to the device. The storage can include any one or more of volatile or non-volatile memory, hard drives, flash storage devices, and/or optical storage devices (e.g., CDs, DVDs, etc.), remote storage (e.g., cloud-based storage), among others. As used herein, the term "computer-readable media" can include signals. In contrast, the term "computer-readable storage media" excludes signals. Computer-readable storage media includes "computer-readable storage devices. " Examples of computer-readable storage devices include volatile storage media, such as RAM, and non-volatile storage media, such as hard drives, optical discs, and/or flash memory, among others.

Examples of computing devices <NUM> can include traditional computing devices, such as personal computers, desktop computers, servers, notebook computers, cell phones, smart phones, personal digital assistants, pad type computers, mobile computers, smart devices etc. and/or any of a myriad of ever-evolving or yet to be developed types of computing devices.

As mentioned above, configuration <NUM>(<NUM>) can be thought of as a system on a chip (SOC) type design. In such a case, functionality provided by the computing device can be integrated on a single SOC or multiple coupled SOCs. One or more processors <NUM> can be configured to coordinate with shared resources <NUM>, such as memory/storage <NUM>, etc., and/or one or more dedicated resources <NUM>, such as hardware blocks configured to perform certain specific functionality. Thus, the term "processor" as used herein can also refer to central processing units (CPUs), graphical processing units (GPUs), field programmable gate arrays (FPGAs), controllers, microcontrollers, processor cores, and/or other types of processing devices.

Generally, any of the functions described herein can be implemented using software, firmware, hardware (e.g., fixed-logic circuitry), or a combination of these implementations. The term "component" as used herein generally represents software, firmware, hardware, whole devices or networks, or a combination thereof. In the case of a software implementation, for instance, these may represent program code that performs specified tasks when executed on a processor (e.g., CPU or CPUs). The program code can be stored in one or more computer-readable memory devices, such as computer-readable storage media. The features and techniques of the component are platform-independent, meaning that they may be implemented on a variety of commercial computing platforms having a variety of processing configurations.

<FIG> is a block diagram of an example implementation of unit manager <NUM>. The unit manager <NUM> can include a GUI host visual tree (also referred to as a GUI host) <NUM> that can include or reference other GUI host visual trees <NUM> for any number of GUI controls for different types of computing devices, such as <NUM>, <NUM>, <NUM> of <FIG>. In some implementations, the GUI host visual tree <NUM> can include a GUI control root visual <NUM> that can include Windows. Composition and/or root Extensible Application Markup Language (XAML) code, for example. The GUI control root visual <NUM> can declaratively or imperatively generate various elements of a unit (e.g., units <NUM>, <NUM>, and/or <NUM> introduced above starting with <FIG>). For instance, when considering a unit, the GUI control root visual <NUM> can distinguish unit content <NUM> from unit setting <NUM>. The unit setting, sometimes referred to as chrome, tends to be positioned around the content. The unit setting can be specific to an application (app centric <NUM>), such as an application tool bar and/or generic <NUM>, such as desktop background, and/or tool bars, taskbars, icons, etc. The unit manager <NUM> can also incorporate XAMI, code for generating visuals that are running outside of a process such as component applications and top-level applications.

The generic setting <NUM> can relate to the desktop background and can include any suitable image, any number of links or shortcuts to locally stored files, links to directories, and the like. Taskbars can include a link to a digital assistant, a task view illustrating open applications, a set of icons corresponding to applications being executed, and various icons corresponding to applications and hardware features that are enabled each time a device receives power.

In one example, the unit manager <NUM> can configure unit sets <NUM> based at least in part on input from the prediction module <NUM> and/or on a list of applications (<NUM>, <FIG>) being executed on the computing device (e.g., running appslist <NUM>). The unit sets <NUM> can include one or more units (<NUM>, <FIG>, <NUM>, <FIG>, <NUM>, <FIG>) for the computing device at a specific instance in time. The list of applications <NUM> can indicate a number of applications for which units <NUM> may be managed with associated GUIs (<FIG>, <NUM>, <NUM>, <FIG>, <NUM>, <FIG>).

The prediction module <NUM> can analyze various parameters <NUM> to determine the configuration of the computing device at that instant and to predict a future configuration. The parameters <NUM> can be obtained from and/or relate to input device <NUM>, display <NUM>, sensor <NUM>, and/or other sources <NUM>. For instance, the prediction module <NUM> can utilize sensor information to determine that the input device is moving toward the computing device and is likely to be positioned upon the device in a manner that blocks an area of one of the displays <NUM>.

The unit manager <NUM> can utilize this prediction to define that area as a unit and to manage that unit consistent with the predicted blockage (e.g., don't plan to present an application on the blocked unit because the user won't be able to see it). The unit manager can utilize parameters, the number and/or type of applications that are running, the size, shape, and/or locations of the units, and/or the location and/or relative movement of the input device, among others, to compute how to manage the units.

For instance, the unit manager <NUM> may determine that the long and narrow shape of third unit 124E(<NUM>) in <FIG> may not lend itself to content presentation. However, such a long and narrow shape may lend itself for presentation of a tool bar. Thus, unit manager <NUM> may manage such a space as a generic tool bar (e.g., allows the user to interact with the operating system). Alternatively, the unit manager could manage the space as a toolbar associated with another unit, such as unit 124E(<NUM>). For example, all of unit 124E(<NUM>) could be managed for presenting content associated with an application and unit 124E(<NUM>) could be used to present the toolbar for the application. As such, the display area is better used than is possible in traditional scenarios. For example, unit 124E(<NUM>) may have an aspect ratio that is preferable for content presentation and more display area in unit 124E(<NUM>) can be dedicated to presenting content, because the toolbar is moved to unit 124E(<NUM>), which is less desirable for content presentation and would otherwise be `wasted space' or 'under utilized space.

The unit manager <NUM> can compute the unit sets <NUM> based at least in part on the parameters <NUM> described above. As mentioned, there may be a one-to-one relationship between units and applications, such as unit <NUM>(<NUM>) and application <NUM>(<NUM>). Alternatively, multiple units can be associated with an application, such as units <NUM>(<NUM>) and <NUM>(<NUM>) associated with application <NUM>(<NUM>). In some examples one unit is on one display and the other unit is on another display, but they could be on the same display. Further, units can be associated with the operating system <NUM>, such as unit <NUM>(N) (where 'N' indicates that any number of units can be employed) associated with operating system <NUM>, such as to display a tool bar, function as a touch pad, and/or display operating system content.

As mentioned above, the unit manager <NUM> can generate transition GUIs (see for example GUIs 126B and 126C of <FIG> and <FIG>) that can help the user understand changes that are being made to the units. The transition GUIs may be full featured GUIs (e.g., include all of the details and/or features of the original GUI (see GUIs 126A of <FIG>) and the updated GUI (see GUIs 126F of <FIG>). Alternatively, the transition GUIs may be simplified GUIs. For instance, the transition GUIs may maintain the basic appearance of the original and updated GUIs without the details and/or functionality. For example, the transition GUIs may have a lower resolution than the original and/or updated GUIs. In another example, the user may not be able to interact with the transition GUI, such as typing content during the transition.

In some implementations, the unit manager <NUM> can manage the displays as a set of dynamically variable units. The unit manager can vary the size, aspect ratio, location, and/or function of individual units based upon present and/or predicted future conditions of the displays. The unit manager can utilize a set of private application programming interfaces (APIs) to manage the units. These private APIs can allow the unit manager to facilitate interactions between applications, the operating system and the units.

<FIG> illustrates a flowchart of a display control technique or method <NUM>. At block <NUM>, the method can manage a display as a unit. The method can manage a single display or multiple displays. Each display can be managed as one or more units.

At block <NUM>, the method can identify a change in a condition associated with the display. In one case, the change in condition could be that an input device or other object is going to block an area of the display. In other cases, the change in condition can be a change in orientation of the device. For instance, the user may be rotating the device from a landscape orientation to a portrait orientation or vice versa, among others.

At block <NUM>, the method can alter at least one of a location, aspect ratio, and/or size of the unit based on the condition. Some implementations can enhance the user experience by providing transitions that are intended to help the user understand the alterations that are being performed. For instance, transition GUIs can be presented on the units that visualize the changes in a manner that the user can understand rather than the display suddenly switching from one visual configuration to a completely different visual configuration.

The described methods can be performed by the systems and/or elements described above and/or below, and/or by other display management devices and/or systems.

Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof, such that a device can implement the method. In one case, the method is stored on one or more computer-readable storage medium/media as a set of instructions (e.g., computer-readable instructions or computer-executable instructions) such that execution by a processor of a computing device causes the computing device to perform the method.

The present computing device display management concepts can be utilized with any type of computing devices and/or associated input devices, such as but not limited to notebook computers, smart phones, wearable smart devices, tablets, and/or other types of existing, developing, and/or yet to be developed devices.

Various methods of manufacture, assembly, and/or use for these devices and associated input devices are contemplated beyond those shown above relative to <FIG>.

Claim 1:
A computing device (<NUM>), comprising:
a display (<NUM>);
storage (<NUM>); and,
a processor (<NUM>) configured to execute computer readable instructions stored on the storage (<NUM>) to perform operations comprising:
managing (<NUM>) the display (<NUM>) as an original unit (124A) presenting at least one original graphical user interface ,GUI, (126A), and
identifying (<NUM>) a change in a current occlusion condition associated with the display (<NUM>),
characterized by
predicting a future occlusion condition associated with the display (<NUM>) based at least on the change, the future occlusion condition being different from the current occlusion condition,
computing (<NUM>) a future unit (124F) presenting at least one updated GUI (126F) based at least on the future occlusion condition, the future unit (124F) being different from the original unit (124A) in at least one of a location, aspect ratio, or size, and
presenting a transition unit (124D) with at least one transition GUI (126D) on the display (<NUM>), the transition unit (124D) having a location, aspect ratio and size in between those of the original unit (124A) and the future unit (124F).