MULTIPLE-DISPLAY UNIFICATION SYSTEM AND METHOD

A method for providing a cohesive user experience when using multiple displays includes displaying a first graphical element on a first display screen, and a second graphical element on a second display screen. The method enables a user to move at least one of the first and second graphical elements until the first and second graphical elements are substantially aligned with one another across the first and second display screens. The method further enables the user to resize at least one of the first and second graphical elements until the first and second graphical elements are scaled similarly on the first and second display screens. The method automatically adjusts a relative understood position of the first and second display screens and scales graphical objects displayed across the first and second display screens. A corresponding system and computer program product are also disclosed.

BACKGROUND

Field of the Invention

This invention relates to systems and methods for providing a more cohesive experience when using multiple display screens.

Background of the Invention

As high quality display screens become more affordable and mobile devices increasingly more common, multiple display screens are frequently used together to achieve a larger display area or to extend the functionality of mobile devices such as laptops, tablets, and cell phones. Companies often join together a number of smaller displays to create large digital billboards, banners, and signage. In such applications, it is important that each display screen be precisely aligned to avoid distorted images and jarring movement when visual objects move between the displays.

Companies and individuals may in certain cases decide to forego traditional desktop personal computers and workstations and instead utilize laptops with additional monitors to increase mobility without sacrificing productivity. In such situations, it is important to have correctly aligned display screens to avoid frustrating inconsistencies when visual objects are moved between the display screens. This becomes particularly important with laptops since they are frequently moved and are rarely put back in the same location relative to an external monitor. Any iterative solution to this problem is infeasible as it would regularly waste a significant amount of time.

It may also be important to have consistent sizing of visual elements across monitors, especially in design and content creation applications where additional display screens are used to increase the size of the digital canvas that is available. In such cases, a misalignment of even a few pixels can lead to additional work and high screen resolution becomes essential to provide a correct level of detail.

In view of the foregoing, what are needed are systems and methods to provide a more cohesive experience when using multiple displays. Ideally, such systems and methods will enable a user to quickly align display screens and correctly size visual elements across multiple display screens.

SUMMARY

The invention has been developed in response to the present state of the art and, in particular, in response to the problems and needs in the art that have not yet been fully solved by currently available systems and methods. Accordingly, systems and methods have been developed to provide a more cohesive user experience when using multiple displays. The features and advantages of the invention will become more fully apparent from the following description and appended claims, or may be learned by practice of the invention as set forth hereinafter.

Consistent with the foregoing, a method for providing a cohesive user experience when using multiple displays is disclosed. In one embodiment, such a method includes displaying a first graphical element on a first display screen, and a second graphical element on a second display screen. The method enables a user to move at least one of the first and second graphical elements until the first and second graphical elements are substantially aligned with one another across the first and second display screens. The method further enables the user to resize at least one of the first and second graphical elements until the first and second graphical elements are scaled similarly on the first and second display screens. In accordance with the movement and resize of the graphical elements, the method automatically adjusts a relative understood position of the first and second display screens and scales graphical objects displayed across the first and second display screens.

A corresponding system and computer program product are also disclosed and claimed herein.

DETAILED DESCRIPTION

The present invention may be embodied as a system, method, and/or computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

The computer readable program instructions may execute entirely on a user's computer, partly on a user's computer, as a stand-alone software package, partly on a user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter scenario, a remote computer may be connected to a user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

Referring toFIG. 1, one example of a computing system100is illustrated. The computing system100is presented to show one example of an environment where a system and method in accordance with the invention may be implemented. The computing system100may be embodied as a mobile device100such as a smart phone or tablet, a desktop computer, a workstation, a server, or the like. The computing system100is presented by way of example and is not intended to be limiting. Indeed, the systems and methods disclosed herein may be applicable to a wide variety of different computing systems in addition to the computing system100shown. The systems and methods disclosed herein may also potentially be distributed across multiple computing systems100.

As shown, the computing system100includes at least one processor102and may include more than one processor102. The processor102may be operably connected to a memory104. The memory104may include one or more non-volatile storage devices such as hard drives104a, solid state drives104a, CD-ROM drives104a, DVD-ROM drives104a, tape drives104a, or the like. The memory104may also include non-volatile memory such as a read-only memory104b(e.g., ROM, EPROM, EEPROM, and/or Flash ROM) or volatile memory such as a random access memory104c(RAM or operational memory). A bus106, or plurality of buses106, may interconnect the processor102, memory devices104, and other devices to enable data and/or instructions to pass therebetween.

To enable communication with external systems or devices, the computing system100may include one or more ports108. Such ports108may be embodied as wired ports108(e.g., USB ports, serial ports, Firewire ports, SCSI ports, parallel ports, etc.) or wireless ports108(e.g., Bluetooth, IrDA, etc.). The ports108may enable communication with one or more input devices110(e.g., keyboards, mice, touchscreens, cameras, microphones, scanners, storage devices, etc.) and output devices112(e.g., displays, monitors, speakers, printers, storage devices, etc.). The ports108may also enable communication with other computing systems100.

In certain embodiments, the computing system100includes a wired or wireless network adapter114to connect the computing system100to a network116, such as a LAN, WAN, or the Internet. Such a network116may enable the computing system100to connect to or communicate with one or more servers118, workstations120, personal computers120, mobile computing devices, or other devices. The network116may also enable the computing system100to connect to or communicate with another network by way of a router122or other device122. Such a router122may allow the computing system100to communicate with servers, workstations, personal computers, or other devices located on different networks.

Referring toFIG. 2, as previously mentioned, multiple display screens are frequently used together to achieve a larger display area or to extend the functionality of mobile devices such as laptops, tablets, or cell phones. In some cases, multiple displays may be joined together to create large digital billboards, banners, and signage. Companies and individuals may in certain cases forego traditional desktop personal computers and workstations and instead utilize laptops with additional monitors to increase mobility. In other cases, multiple monitors may be used with desktop personal computers or workstations to increase an amount of screen area to display documents and applications. In such situations, it is important to have correctly aligned display screens to avoid frustrating inconsistencies when visual objects are displayed or moved across the display screens. This becomes particularly important with laptops since they are frequently moved and are rarely put back in the exact same location relative to an external monitor.

FIG. 2shows a typical scenario of a laptop computer200coupled to an external monitor202. As shown, a display screen204aof the laptop computer200is substantially smaller than a display screen204bof the external monitor202. Furthermore, the display screen204aof the laptop computer200sits substantially below the display screen204bof the external monitor202. In addition to the size difference between the display screens204a,204b, the display screens204a,204bmay have differing resolutions and pixel densities (e.g. pixels per inch). The result is that the same graphical objects (windows, cursors, widgets, etc.) may appear to be different sizes when displayed on the display screens204a,204b. The graphical objects may also appear disjointed when spanning the display screens204a,204bor when moved across the display screens204a,204b. In many cases, a graphical object may not follow a straight line as it moves across the display screens204a,204b, instead following a disjointed path. A graphical object may also change size as it moves across the display screens204a,204b, potentially displaying different portions of the graphical object at a different scale. This, in turn, may decrease efficiency, cause annoyances to a user, and possibly cause mistakes in a work product.

In order to address the problems described above and provide a more cohesive user experiences when using multiple display screens204a,204b, systems and methods in accordance with the invention may compensate for physical misalignments and/or differing resolutions or pixel densities of multiple display screens204a,204b. As shown inFIG. 2, systems and methods in accordance with the invention may display a graphical element206, in this example a rectangle206, on each display screen204. These graphical elements206a,206bmay be manipulated (e.g., moved, resized, etc.) by a user with a mouse, cursor, stylus, or the like, to compensate for the physical misalignments and/or differing resolutions or pixel densities of the display screens204a,204b.

For example, as shown inFIG. 3, a user may initially move the graphical element206bon the display screen204bto visually align it with the graphical element206aon the other display screen204a. In the illustrated example, the graphical element206bis moved down with a cursor, finger, or the like, so that its lower edge is substantially aligned with the lower edge of the graphical element206a. The graphical element206amay also be moved if needed to visually align its lower edge with the lower edge of the graphical element206b. Thus, alignment of the graphical elements206a,206bmay require movement of one or more of the graphical elements206a,206bwith a cursor, finger, stylus, or the like. Alignment is not limited to the lower edge of the graphical elements206. In other embodiments, the upper edges of the graphical elements206a,206bmay be initially aligned or the graphical elements206a,206bmay be brought into approximate alignment and the edges may be more exactly aligned during the resize step illustrated inFIG. 4.

Referring toFIG. 4, once some portion of the graphical elements206a,206bare substantially aligned, one or more of the graphical elements206a,206bmay be resized so that they appear to the user to be the same size or substantially the same size on each of the display screens204a,204b. In certain embodiments, the graphical elements206a,206bmay be resized by dragging a corner of the graphical elements206a,206b, similar to resizing windows in many operating systems. The graphical elements206a,206bmay, in certain embodiments, maintain their same shape or proportions when they are resized although this is not necessary in all embodiments. In other embodiments, an upper or lower edge of the graphical elements206a,206bmay be dragged up or down to resize the graphical elements206a,206buntil they have the same vertical dimensions or substantially the same vertical dimensions.

Once the graphical elements206a,206bare aligned and resized so that their top and bottom edges are substantially visually aligned with one another, a monitor unification module1100in accordance with the invention, as will be explained in more detail in association withFIG. 11, may adjust a relative understood position of the display screens204a,204band scale objects on the display screens204a,204bso that they appear to be substantially the same size when they move across the display screens204a,204b. Specifically, the monitor unification module1100may compensate for the different physical alignments and differing resolutions and pixel densities of the display screens204a,204b. This will provide a much more cohesive user experience and cause the separate display screens204a,204bto act more like one large display screen204.

For example, as shown inFIG. 5, after the monitor unification module1100performs the above-described compensation, a cursor500or other graphical object500that moves across the display screens204a,204bmay follow a straight line, regardless of the vertical position of the cursor500or graphical object500as it travels across the display screens204a,204b. Furthermore, graphical objects600such as windows600may be scaled similarly on both display screens204a,204bwhen they travel across or span the display screens204a,204b, as shown inFIG. 6.

In certain embodiments, the monitor unification module1100may take into account bezels around the display screens204a,204bas well as any distance between the bezels of the two display screen204a,204b. The thickness of the bezels, as well as the distance between the bezels of the monitors200,202, may affect the appearance of graphical objects as they move across the display screens204a,204b. For example, a cursor moving at an angle from one display screen204ato the other204bmay not appear to travel along a straight line unless the bezels and any distance between the bezels is taken into account.

Bezels and any distance between the bezels may, in certain embodiments, be ignored (treated like they don't exist) for graphical objects that span the display screens204a,204b. This would enable all of a graphical object to be seen when spanning the display screens204a,204b. In other embodiments, the bezels and distance between the bezels may obscure part of a graphical object as it spans or passes across the display screens204a,204b. This may provide a more realistic appearance and allow the graphical objects to maintain their geometric proportions, while hiding from view part of the graphical objects. The amount of the graphical objects hidden from view may be reduced by using monitors200,202with thinner bezels and/or less distance the bezels (such as by having the bezels abut another). In certain embodiments, the monitor unification module1100may gather information about bezel width and any distance between the bezels from a user, from the monitor itself (such as from Extended Display Identification Data (EDID) data), or other sources so that these factors may be taken into account when displaying or translating graphical objects across the display screens204a,204b.

Referring toFIGS. 7 through 9, the systems and methods described herein may in certain embodiments be adapted to work with more than two display screens204. For example,FIG. 7shows a scenario using three display screens204a-c. Graphical elements204a-dmay be displayed on the display screens204a-cfor the purposes of “unifying” the display screens. As shown, the display screen204bmay include two graphical elements206b,206cto enable it to be unified with the display screen204ato the left and the display screen204cabove.

To unify the display screens204a,204bto compensate for differences in physical alignment, resolution, and/or pixel density, the graphical elements206a,206bmay be aligned and resized in the manner previously described, as shown inFIG. 8. As can be observed inFIG. 8, when the graphical element206bis resized to match the size of the graphical element206a, the other graphical element206con the display screen204bmay automatically change in size to reflect the resize. That is, the graphical elements206b,206cmay maintain the same proportions relative to one another.

After the graphical elements206a,206bhave been aligned and resized to match one another, thereby causing the display screens204a,204bto be “unified” with one another, the display screens204b,204cmay then be unified. This may be accomplished by aligning and resizing the graphical elements206c,206d. Because the display screens204b,204care in a stacked configuration, the graphical elements206c,206dmay be moved horizontally as opposed to vertically. Because resizing the graphical element206cwill cause a corresponding resize of the graphical element206b(and thereby possibly cause a mismatch with graphical element206a), only the graphical element206dmay be resized to match the graphical element206c. Thus, the graphical element206dmay be aligned with the graphical element206cand resized to substantially match the size of the graphical element206c, as shown inFIG. 9. This will unify the display screens204b,204cwithout changing the relationship between the display screens204a,204b. This final step will result in all of the display screens204a-cacting as a single large display.

Referring toFIG. 10, in certain embodiments, a camera1000(shown in this embodiments as a camera1000on a mobile computing device1002) may be used to assist in unifying multiple display screens204, possibly without requiring a user to move around and/or resize graphical elements206on the display screens204. The monitor unification module1100previously discussed may cause one or more graphical elements206to be displayed on each display screen204in the group. A user may then take a photograph of the display screens204and associated graphical elements206. The photograph may be communicated to the monitor unification module1100which may analyze the photograph, including the sizes of the display screens204and graphical elements206thereon. The monitor unification module1100may also analyze the relative positions and alignments of the display screens204. Based on this information as well as information such as screen resolutions and pixel densities (which may be acquired, for example, from a user or EDID data), the monitor unification module1100may adjust a relative understood position of the display screens204and scale graphical objects on the display screens204so that they appear to be same size when travelling across or spanning the display screens204. Stated otherwise, the monitor unification module1100may “unify” the display screens204such that they perform like one large display screen204.

Referring toFIG. 11, the monitor unification module1100previously discussed may include one or more sub-modules to provide various features and functions. These sub-modules may be implemented in hardware, software, firmware, or combinations thereof. As shown, these sub-modules may include one or more of a monitor detection module1102, characteristic determination module1104, element presentation module1106, movement module1108, resize module1110, alignment calculation module1112, scaling calculation module1114, and snapshot module1116. The sub-modules are presented by way of example and not limitation. The monitor unification module1100may include more or fewer sub-modules than those illustrated, or the functionality of the sub-modules may be combined or split into additional sub-modules.

The monitor detection module1102may be configured to detect monitors that are connected to a computing system100. This detection may occur upon request or automatically when a monitor is connected to the computing system100. The characteristic determination module1104may determine characteristics, such as resolution, pixel density, screen size, bezel width, and the like, associated each of the monitors that are connected to the computing system100. In certain embodiments, these characteristics are pulled from the monitor itself, such as from a memory within the monitor. For example, EDID data which may indicate vertical and horizontal lengths of a monitor as well as its resolution, may be pulled from the monitor. In other embodiments, the characteristic determination module1104determines models of the monitors and looks up information about the particular models in a database stored on the computing system100or pulled from an external source such as the Internet. In yet other embodiments, the characteristic determination module1104enables a user to manually input characteristics about the monitors. These characteristics may be helpful to “unify” the display screens204and ensure that graphical objects appear correctly when translated or displayed across the display screens204.

The element presentation module1106may present graphical elements206on the display screens204for the purpose of unifying the display screens204. These graphical elements206may take on various forms and not limited to any particular shape or orientation. In certain embodiments, the graphical elements206are rectangles as illustrated inFIGS. 2 through 9. These graphical elements206may be manipulated (moved, resized, etc.) in order to unify the display screens204. The movement module1108may enable a user to move the graphical elements206on the display screens204. For side-by-side monitors, the movement module1108may enable vertical translation of the graphical elements206. For stacked monitors, the movement module1108may enable horizontal translation of the graphical elements206. The resize module1110, by contrast, may enable resizing of the graphical elements206. In certain embodiments, this may be accomplished by selecting and dragging a corner or edge of the graphical elements206until they achieve a desired size.

Based on the way the graphical elements206are moved and resized, the alignment calculation module1112may adjust a relative understood position of the display screens204(i.e., adjust an understanding of the computing system100as to where the display screens204are located relative to one another). This will enable compensation for any misalignment. This, in turn, will enable a graphical object (e.g., cursor, window, image, etc.) to transition from one display screen204to another along a straight line without jumping or lurching during the transition. The scaling calculation module1114, by contrast, may scale graphical objects on the display screens204so that they appear to be the same size or substantially the same size when spanning or transitioning across the display screens204. The scaling calculation module1114may take into account the dimensions, resolutions, and/or pixel densities of the display screens204when making this calculation.

The snapshot module1116may enable a snapshot to be taken of multiple monitors and associated graphical elements206as was discussed in association withFIG. 10. This snapshot may be analyzed by the alignment calculation module1112and scaling calculation module1114to determine the physical alignment between the monitors and the scaling that needs to occur to make graphical objects appear to be the same size or substantially the same size as they pass therebetween. In certain embodiments, this may be accomplished without having to move and/or resize the graphical elements206on the display screens204. That is, the alignment calculation module1112and scaling calculation module1114may analyze the snapshot and determine, from the size and relative position of the monitors, the size and position of the graphical elements206on the display screens204, and the resolutions and/or pixel densities of the monitors, the adjustments that need to be made to make the monitors correctly function together.

Different variations of the disclosed systems and methods are possible. For example, it is contemplated that the monitor unification module1100could enable different monitors in a group to maintain their current resolutions and size graphical objects contained entirely on their display screens204in accordance with their native resolutions. Thus, a graphical object on a first monitor may appear to be a first size while the same graphical object on a second monitor may appear to be a second size different from the first size. By contrast, when the graphical object is translated across the first monitor and second monitor, the monitor unification module1100may gradually shrink or enlarge the graphical object from the first size to the second size. Any graphical object that spans the first and second monitor may have a size between the first and second size, depending on much of the graphical object is displayed on the first monitor and second monitor. While spanning the first and second monitors, the graphical object may be scaled similarly on each of the first and second monitors so that it appears in a normal non-disjoint manner. Alternatively, a graphical object that is moved from a first monitor to a second monitor may snap down or up in size to the resolution of the monitor to which it is being moved once the graphical object is entirely contained within the destination display screen204. Before it is entirely contained within the destination display screen, the graphical object may be scaled in accordance with the source or originating display screen204.

The flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. Other implementations may not require all of the disclosed steps to achieve the desired functionality. It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.