Display environment for a plurality of display devices

A display environment for a plurality of display devices is described. In one or more implementations, a display environment of an operating system of a computing device is configured to display a plurality of shells that support user interaction with the operating system by launching a first shell for display on a first display device of the computing device and launching a second shell for display on a second display device of the computing device such that the first and second shells are displayable simultaneously by the computing device.

BACKGROUND

Users have access to an ever increasing functionality that may be provided by computing devices. From browsers to word processors, spreadsheets, calendaring and email applications, and so on, a user may execute a wide number of applications at any one time to view information from these applications.

In order to increase the amount of information that is viewable at a particular time, a user may employ a plurality of display devices (e.g., monitors) that are communicatively coupled to a single computing device. In this way, a user increases an amount of area that is available to display data from the applications that are executed on the computing device. However, this increase in display area could also negatively affect traditional techniques for user interaction that were generally based on a single display device.

SUMMARY

A display environment for a plurality of display devices is described. In one or more implementations, a display environment of an operating system of a computing device is configured to display a plurality of shells that support user interaction with the operating system by launching a first shell for display on a first display device of the computing device and launching a second shell for display on a second display device of the computing device such that the first and second shells are displayable simultaneously by the computing device.

In one or more implementations, an input is received by a computing device to initiate execution of an application by the computing device. A determination is made as to which of a plurality of display devices of the computing device with which a user interacted to provide the input. Execution of the application is launched to display a user interface associated with the application on the determined display device.

In one or more implementations, one or more computer readable storage media comprise instructions stored thereon that, in response to execution by a computing device, cause the computing device to configure a display environment of an operating system executed by the computing device for display of launch functionality configured to initiate execution of an application, the launch functionality displayable on each of a plurality of display devices that output the display environment of the computing device without reconfiguring settings of the launch functionality to specify a particular one of the plurality of display devices that is to display the launch functionality.

DETAILED DESCRIPTION

Overview

Users often employ a plurality of display devices when interacting with a single computing device in order to increase an amount of display area that is available to display data to the user. However, traditional display environments were often designed with a single display device in mind. Therefore, even though the display environment could be expanded across a plurality of display devices, this expansion could hinder interaction supported by the traditional display environment.

A display environment for a plurality of display devices is described. In one or more implementations, an operating system is configured to support a plurality of shells via which a user may interact with the operating system. Accordingly, the display environment output by the operating system may be configured to support these shells. For example, the display environment may be configured to display each of the shells on a different display device such that a user may view data from the shells simultaneously.

Further, the display environment may be configured to make a switch between shells responsive to a user action. For instance, a user may drag a window from a display device that outputs a productive shell (e.g., a desktop shell) to a display device that outputs a consumptive shell, e.g., an immersive shell configured primarily to support touch interaction. This may cause the display device that displayed the consumptive shell to switch to a productive shell. In another instance, an application for output in a consumptive shell may be launched for display on a display device that currently outputs a productive shell. In this instance, the display device may also make a switch, but in this case from a productive shell to a consumptive shell. Further discussion of display of different shells, consumptive and productive shells, and so on may be found in relation to the following discussion.

In one or more implementations, the display environment may be configured to display applications on display devices from which the application was launched. For example, a computing device may be communicatively coupled to a plurality of display devices as described above. An input may be received from a particular one of the display devices to launch an application, such as a selection of a representation of the application in a launch menu displayed on the particular display device. In this example, the launch menu (e.g., a system UI) may be invoked on any of the display devices. Accordingly, data from execution of the application may be displayed on the particular display device. Thus, a user may readily choose which display device is to display data from an application by initiating execution of the application on that display device. Further discussion of this implementation may also be found in relation to the following discussion.

Example Environment

FIG. 1is an illustration of an environment100in an example implementation that is operable to employ the display environment techniques described herein. The illustrated environment100includes a computing device102having a processing system104and a computer-readable storage medium that is illustrated as a memory106although other confirmations are also contemplated as further described below.

The computing device102may be configured in a variety of ways. For example, a computing device may be configured as a computer that is capable of communicating over a network, such as a desktop computer, a mobile station, an entertainment appliance, a set-top box communicatively coupled to a display device, a wireless phone, a game console, and so forth. Thus, the computing device102may range from full resource devices with substantial memory and processor resources (e.g., personal computers, game consoles) to a low-resource device with limited memory and/or processing resources (e.g., traditional set-top boxes, hand-held game consoles). Additionally, although a single computing device102is shown, the computing device102may be representative of a plurality of different devices, such as multiple servers utilized by a business to perform operations such as by a web service, a remote control and set-top box combination, an image capture device and a game console configured to capture gestures, and so on.

The computing device102is further illustrated as including an operating system108. The operating system108is configured to abstract underlying functionality of the computing device102to applications110that are executable on the computing device102. For example, the operating system108may abstract the processing system104, memory106, display devices112,114, network, and other functionality of the computing device102such that the applications110may be written without knowing “how” this underlying functionality is implemented. The application110, for instance, may provide data to the operating system108to be rendered and displayed by the display devices112,114without understanding how this rendering will be performed. The operating system108may also represent a variety of other functionality, such as to manage a file system and user interface that is navigable by a user of the computing device102.

The operating system108is also illustrated as including a user experience module116which is representative of functionality of the computing device102to provide a plurality of shells via which a user may interact with the operating system108and the applications110and other data of the computing device102, both local to the device as well as remotely via a network. The shells are configured to accept inputs to interact with the operating system108and applications110of the computing device102to access functionality of the computing device102, such as the abstracted functionality described above. Examples of such shells include a consumptive shell118and a productive shell120.

The consumptive shell118may be configured to support interaction with the applications110with little to no window frame. Additionally, the consumptive shell118may support interaction with the applications110without requiring the user to manage a corresponding window frame's layout, primacy of the window with respect to other windows (e.g., whether a window is active, in front of behind other windows, an order of the windows, and so on). Although illustrated as part of the operating system108, the user experience module116may be implemented in a variety of other ways, such as a stand-alone module, remotely via a network, and so forth.

In one or more implementations, the consumptive shell118of the operating system108is configured such that it is not closeable or capable of being uninstalled apart from the operating system108. Additionally, the consumptive shell118may be configured to consume a significant portion of an available display area of a display device, as shown through display of the consumptive shell118on the first display device112. A user may interact with the consumptive shell118in a variety of ways, such as via a cursor control device, using one or more gestures, using speech recognition, capture using one or more depth-sensing cameras, and so on.

Thus, the user experience module116may manage the consumptive shell118in which content of applications110may be presented and the presentation may be performed without requiring a user to manage size, location, primacy, and so on of windows used to display the content of the applications110.

For example, as shown on a user interface displayed by the first display device112, a user interface is shown that is configured to display data from two applications in a “snapped” configuration. In this example, both of the applications that correspond to the data are enable to actively execute by the computing device102while execution is suspended for other of the applications110that do not currently display data. A gutter displayed as a line between the two applications (e.g., “weather” and “social network”) is disposed between the displays of the data that may be moveable to change an amount of display area consumed by applications on the first display device112, respectively, in the consumptive shell118.

The consumptive shell118may support a wide variety of functionality to provide an immersive experience for a user to access the applications110. In the following discussion, this functionality is discussed in relation to leveraging these techniques to enable interaction with a productive shell120. It should be readily apparent, however, that these techniques may be employed for managing interaction with the applications110themselves without departing from the spirit and scope thereof.

As illustrated, the user experience module116also supports a productive shell120, an example of which is shown as a desktop displayed on the second display device114. The productive shell120is representative of another configuration of a user interface output by the operating system108when in this example is to interact with the applications110and other data. For example, the productive shell120may be configured to present applications and corresponding data through windows having frames. These frames may provide controls through which a user may interact with an application as well as controls enabling a user to move and size the window. The productive shell120may also support techniques to navigate through a hierarchical file structure through the use of folders and represent the data and applications through use of icons. In one or more implementations, the productive shell120may also be utilized to access applications110that are configured specifically for interaction via the productive shell120and not configured for access via the consumptive shell118, although other implementation are also contemplated.

In one or more of the techniques described herein, the user experience module116is configures a display environment that that may employ a plurality of display devices, such as the first and second display devices112,114as illustrated although greater numbers of display devices are also contemplated. Further discussion of the display environment may be found beginning in relation toFIG. 2below.

For example, the computing device102may also include an entity (e.g., software) that causes hardware of the computing device102to perform operations, e.g., processors, functional blocks, and so on. For example, the computing device102may include a computer-readable medium that may be configured to maintain instructions that cause the computing device, and more particularly hardware of the computing device102to perform operations. Thus, the instructions function to configure the hardware to perform the operations and in this way result in transformation of the hardware to perform functions. The instructions may be provided by the computer-readable medium to the computing device102through a variety of different configurations.

One such configuration of a computer-readable medium is signal bearing medium and thus is configured to transmit the instructions (e.g., as a carrier wave) to the hardware of the computing device, such as via a network. The computer-readable medium may also be configured as a computer-readable storage medium and thus is not a signal bearing medium. Examples of a computer-readable storage medium include a random-access memory (RAM), read-only memory (ROM), an optical disc, flash memory, hard disk memory, and other memory devices that may use magnetic, optical, and other techniques to store instructions and other data.

FIG. 2illustrates a system200in an example implementation of the computing device102ofFIG. 1as employed a plurality of display devices to display a plurality of shells of an operating system simultaneously. The system200includes the first and second display devices112,114of the computing device102ofFIG. 1. The first display device112is illustrated as displaying a start screen of the consumptive shell118while the second display device114is illustrated as displaying a desktop of the productive shell120.

The start screen includes a plurality of tiles that represent applications that are selectable for execution by the computing device102. A variety of other representations are also contemplated without departing from the spirit and scope thereof. Selection of the tiles may be accomplished in a variety of ways. For example, a user may utilize a cursor control device, such as a mouse or track pad, to navigate a cursor to be displayed “over” a tile to be selected and “click” the tile to indicate the selection. In another example, gestures may be supported by the computing device102, such as by tapping a desired representation to launch a corresponding application.

Responsive to the selection, the corresponding application110may be launched for execution by the computing device102. An example of this is illustrated in the example system300ofFIG. 3in which the social network application is launched into the consumptive shell118on the first display device112. The social network application is illustrated as providing a user interface that consumes approximately an entirety of a display area of the first display device112. Additionally, the user interface of the social network application as illustrated does not include a window or “chrome” that is displayed along with the data of the application.

This technique may be repeated to launch a plurality of applications110for execution by the computing device102. For example, a user may return to the start screen illustrated inFIG. 2and select additional applications for execution, such as through selecting a “start” selection in a launch menu302of commands (e.g., system UI or “charms”) as illustrated. Responsive to each selection, additional user interfaces may be output that substantially consume the available display area of the first display device112.

Thus, the launch menu302may be used to initiate execution of an application through interaction using a particular display device and have the application launched on that display device. In this example, for instance, the start screen was displayed on the first display device112and a selection was made of the social network application on the first display device112. In response, the user experience module116causes data from the selected application (e.g., the user interface application) to be displayed on the corresponding display device, e.g., the first display device in this example. Continuing with the example above successive launch of the applications in the consumptive shell118on the first display device112may also be performed using similar techniques.

A user may then navigate through the applications that have been selected for execution in the consumptive shell118in a variety of ways. For example, a user may interact with a left edge of the first display device112to cause the display of the social network application to be replaced with a display that corresponds to another application that was launched by the user as described above.

This interaction may be performed in a variety of ways, such as through maneuvering a cursor proximal to the left edge, a “swipe” gesture involving the left edge, and so on. This replacement may be performed non-modally by automatically cycling through the applications, modally through output of a menu having representations of the selected applications110, and so on. Although interaction with the left edge is described for navigation through the applications110, it should be readily apparent that different edges of the first display device112may be utilized to navigate through applications in the consumptive shell118as well as different navigation techniques without departing from the spirit and scope thereof.

As described above, launch functionality may be supported by the user experience module116for each of the plurality of display devices. As further illustrated inFIG. 3, for instance, a user may also cause output of a launch menu304on the second display device114. The launch menu304may also include an option to cause the start screen ofFIG. 2to be output on the second display device114, as further shown inFIG. 4.

FIG. 4depicts a system400in an example implementation in which a consumptive shell118is depicted as being output on the first display device112and a start screen is illustrated as being output on the second display device114. As previously described the start screen may include a variety of different representations (e.g., tiles) that may be selected to initiate launch of a represented application110. These applications may be configured for output in different shells. For example, the social network application as previously described may be configured for output in the consumptive shell118. Accordingly, when the application is launched on a particular display device, that display device may configured for interaction using the corresponding shell, e.g., the consumptive shell118in this instance. Similar techniques may also be used to launch an application in a productive shell.

For example, a user may select a tile (e.g., a spreadsheet application) as before to cause data from the execution of the application to be output on the corresponding display device, which in this instance is the second display device114as illustrated inFIG. 4. This selection may be accomplished in a variety of ways, such as through a cursor control device, a gesture, and so on. In response to the selection of the representation, the user experience module116may determine which display device is being used to launch the application, which application is being launched, as well as which shell is to be utilized to interact with the application. Continuing with the illustrated example, the user experience module116may determine that the second display device114is being used to launch a spreadsheet application that is to be executed in a productive shell120, a result of which is shown inFIG. 5.

FIG. 5depicts a system500in an example implementation in which an application configured for interaction via a consumptive shell118is displayed on a first display device112of the computing device102and another application that is configured for interaction via a productive shell120is displayed on a second display device120. As shown in this figure, applications from different shells of an operating system108may be accessed using respective display devices.

Further, as described in the preceding examples the display devices112,114may be “set” to output a particular one of a plurality of shells based on a user action, which was through interaction with a launch menu. A variety of other user interactions may also be leveraged by the user experience module116to determine which shell is to be used by a respective one of a plurality of display devices, another example of which may be found beginning in relation to the following figure.

FIG. 6depicts a system600in an example implementation in which a user interaction is detected by a user experience module116as indicating an intent to cause a switch between shells of an operating system108. In this example, a first display device112is used to display a user interface for interaction with a productive shell120and a second display device114is used to display a user interface for interaction with a consumptive shell118. The first display device112, for instance, is illustrated as displaying data from a spreadsheet application, which includes a plurality of windows and “chrome” within a primary window602that corresponds to the application.

A user may select the window602(e.g., the window containing the spreadsheet application) in a variety of ways to move the window602for display on a different location of the display environment. For example, a user may utilize a “click-and-drag” technique using a cursor control device, a “tap-and-slide” gesture, and so on. In response, the window602may then be moved across the first display device112by the user experience module116as further shown in the following figure.

FIG. 7depicts a system700in an example implementation showing movement of a window from a first display device112that displays data from a productive shell120to a second display device114that displays data from a consumptive shell118. In this example, movement of the window602of the spreadsheet application is shown over a window702that corresponds to a photo application in the productive shell120displayed on the first display device112.

In this way, the windows602,702provide support of a display of depth such that data from the one application appears to be displayed over at least a portion of data from another application. Similar techniques may be utilized to display data from the same application, such as the sub-windows within the window602of the spreadsheet application as illustrated. As stated previously, the consumptive shell, outside of the productive shell, is not so configured but rather is configured primarily for touch (although a cursor control device may also be utilized) in comparison with the productive shell.

The user experience module116may then detect movement of the window602past a threshold, e.g., a cursor and/or touch point is within a predefined range of an edge of the first display device602, a cursor “crosses over” into the second display device114while selecting the window602, and so on. In response, the user experience module116may cause a switch from the consumptive shell118in the second display device114to the productive shell120, a result of which is shown in the following figure.

FIG. 8depicts a system800in an example implementation showing first and second display devices112,114as displaying data from a productive shell120. The second display device114in this example is shown as display the window602of the spreadsheet application over a desktop in the productive shell120. The first display device112is also still illustrated as displaying the window702of the photo application in the productive shell120. Thus, in the example described in relation toFIGS. 6-8a user interaction that involves movement of the window from one display device to another is used to indicate that the display device is to be placed in a corresponding shell of the operating system108if not already so configured. Applications that were already executed within the previous shell may be addressed in a variety of ways, such as minimized, formed for execution within a corresponding window if such execution is supported, cause execution of the application to be ceased, and so on. Thus, a variety of different techniques may be leveraged by the user experience module to support a display environment for a plurality of display devices, further discussion of which may be found in relation to the following procedures.

Example Procedures

FIG. 9depicts a procedure900in an example implementation in which a display environment is configured to support different shells of an operating system on different display devices. A display environment of an operating system of a computing device is configured to display a plurality of shells that support user interaction with the operating system (block902). For example, a first shell may be launched for display on a first display device of the computing device (block904), e.g., by a user experience module116. Likewise, a second shell may be launched for display on a second display device of the computing device such that the first and second shells are displayable simultaneously by the computing device (block904). The shells may be configured in a variety of ways by the user experience module116.

For example, the shells may include consumptive (e.g., immersive) and productive (e.g., desktop) shells118,120. As described above, the consumptive shell may be configured to support user interaction that is based primarily on touch in comparison with user interaction supported by the productive shell that is based primarily on use of a cursor control device. Additionally, the productive shell may be configured to support display of a plurality of windows that correspond to a plurality of applications that are executable within the productive shell and the consumptive shell, outside of the productive shell, is not so configured. Rather, the consumptive shell may be configured to provide a substantially “full screen” experience with limited chrome, may support “snapping” behavior, and so on. Further, the productive shell may be configured to employ chrome and a task bar and the consumptive shell, outside of the productive shell, is not so configured. Yet further, the productive shell may be configured to support a display of depth such that data from one application appears to be displayed over at least a portion of data from another application and the consumptive shell, outside of the productive shell, is not so configured. A variety of other examples are also contemplated of differences between the consumptive118and productive120shells of the operating system108.

Additionally, an input may be detected that describes a user action to initiate a change from one to another of the plurality of shells (block908). The change may then be made from the one to the other of the plurality of shells for at least one of the first or second display devices (block910). A variety of different user actions may be detected. For example, a user may interact with launch functionality such as a launch menu as described in relation toFIG. 3to select particular application to be launched in a respective display device. A shell that corresponds to the application (e.g., via which execution of the application is accessible) may then be launched by the user experience module116. In one or more implementations, applications110are configured to execute within a particular shell and therefore and not executable “outside” that shell, i.e., in a different shell. Other implementations are also contemplated.

In another example as described in relation toFIGS. 6-8, movement of an item that is executable within one shell to another display device that is currently associated with another shell may cause a change to the “correct” shell. For instance, movement of a window from a display device that displayed a productive shell120to a display device that displays a consumptive shell118may cause a change to the productive shell on that device. Other examples are also contemplated. Naturally, although first and second display devices are described in the previous example, a greater number of display devices and shells are also contemplated which may also leverage the techniques described above.

FIG. 10depicts a procedure1000in an example implementation in which a display environment is configured to support display of data on a particular one of a plurality of display devices based on which display device support the interaction to provide the input to launch the application. An input is received by a computing device to initiate execution of an application by a computing device (block1002). A user, for instance, may select a representation of an application in a start screen, using launch functionality (e.g., a launch menu, system UI), and so on. The user experience module116, for example, may configure a display environment such that the launch menu is accessible via a plurality (e.g., each) of display devices of the computing device102.

Further, in one or more implementations this functionality may be supported without manually setting configuration settings to make the change. Previously, a user could select which of a plurality of display devices was to display a launch menu, e.g., as a pop-up menu, task bar, and so on. However, this selection prevented output on other display devices currently being used. Thus, although the launch menu could be output on different display devices a user was first forced to configure which device was to be allowed to output the launch menu. However, in this implementation each display device may display the launch menu, such as through detection of interaction of a user with an edge (e.g., bottom edge) of the display device using a cursor control device, gesture, and so on.

Accordingly, a determination may be made as to which of a plurality of display devices of the computing device with which a user interacted to provide the input (block1004). Continuing with the previous example, the user experience module116may determine which display device outputted the start menu, launch menu, and so on and also via which device an input was received selecting a particular application, e.g., a representation of the application in the task bar, launch menu, start menu, and so on.

Execution of the application may then be launched to display a user interface associated with the application on the determined display device (block1006). The user experience module116, for instance, may display a window associated with the application in a productive shell120, employ an immersive experience using a consumptive shell118, and so on. A variety of other examples are also contemplated without departing from the spirit and scope thereof.

Example Device

FIG. 11illustrates various components of an example device1100that can be implemented as any type of computing device as described with reference toFIGS. 1-8to implement embodiments of the display environment techniques described herein. Device1100includes communication devices1102that enable wired and/or wireless communication of device data1104(e.g., received data, data that is being received, data scheduled for broadcast, data packets of the data, etc.). The device data1104or other device content can include configuration settings of the device, media content stored on the device, and/or information associated with a user of the device. Media content stored on device1100can include any type of audio, video, and/or image data. Device1100includes one or more data inputs1106via which any type of data, media content, and/or inputs can be received, such as user-selectable inputs, messages, music, television media content, recorded video content, and any other type of audio, video, and/or image data received from any content and/or data source.

Device1100also includes communication interfaces1108that can be implemented as any one or more of a serial and/or parallel interface, a wireless interface, any type of network interface, a modem, and as any other type of communication interface. The communication interfaces1108provide a connection and/or communication links between device1100and a communication network by which other electronic, computing, and communication devices communicate data with device1100.

Device1100includes one or more processors1110(e.g., any of microprocessors, controllers, and the like) which process various computer-executable instructions to control the operation of device1100and to implement embodiments of the techniques described herein. Alternatively or in addition, device1100can be implemented with any one or combination of hardware, firmware, or fixed logic circuitry that is implemented in connection with processing and control circuits which are generally identified at1112. Although not shown, device1100can include a system bus or data transfer system that couples the various components within the device. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures.

Device1100also includes computer-readable media1114, such as one or more memory components, examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of a read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and a disk storage device. A disk storage device may be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a recordable and/or rewriteable compact disc (CD), any type of a digital versatile disc (DVD), and the like. Device1100can also include a mass storage media device1116.

Computer-readable media1114provides data storage mechanisms to store the device data1104, as well as various device applications1118and any other types of information and/or data related to operational aspects of device1100. For example, an operating system1120can be maintained as a computer application with the computer-readable media1114and executed on processors1110. The device applications1118can include a device manager (e.g., a control application, software application, signal processing and control module, code that is native to a particular device, a hardware abstraction layer for a particular device, etc.). The device applications1118also include any system components or modules to implement embodiments of the techniques described herein. In this example, the device applications1118include an interface application1122and an input/output module1124that are shown as software modules and/or computer applications. The input/output module1124is representative of software that is used to provide an interface with a device configured to capture inputs, such as a touchscreen, track pad, camera, microphone, and so on. Alternatively or in addition, the interface application1122and the input/output module1124can be implemented as hardware, software, firmware, or any combination thereof. Additionally, the input/output module1124may be configured to support multiple input devices, such as separate devices to capture visual and audio inputs, respectively.

Device1100also includes an audio and/or video input-output system1126that provides audio data to an audio system1128and/or provides video data to a display system1130, which may include a plurality of display device1132,1134. The audio system1128and/or the display system1130can include any devices that process, display, and/or otherwise render audio, video, and image data. Video signals and audio signals can be communicated from device1100to an audio device and/or to a display device via an RF (radio frequency) link, S-video link, composite video link, component video link, DVI (digital video interface), analog audio connection, or other similar communication link. In an embodiment, the audio system1128and/or the display system1130are implemented as external components to device1100. Alternatively, the audio system1128and/or the display system1130are implemented as integrated components of example device1100.

CONCLUSION