Patent Publication Number: US-8990363-B1

Title: Decomposition and recomposition for cross-platform display

Description:
BACKGROUND 
     1. Field of the Invention 
     The present invention generally relates to cross-platform display. More specifically, the present invention relates to decomposition and recomposition for cross-platform display. 
     2. Description of the Related Art 
     Individuals currently have a variety of options for communicating and carrying out transactions. Such options may include traditional desktop computing devices, as well as various mobile devices (e.g., mobile phones, smartphones, tablets). In fact, many individuals may use multiple computing and mobile devices at home, work, and on the move. For example, an individual may use a desktop computer at work, a laptop computer at home, and one or more mobile devices (e.g., smartphone, tablet) elsewhere. As such, people have come to expect to be able to have access to data and computing resources so to perform most computing tasks anywhere. 
     One difficulty in meeting such an expectation is that the various computing devices may not all have the same capabilities. For example, such devices may run different operating systems/platforms and applications. Such differences may make it difficult to support the same tasks across such devices. One solution has been to provide remote desktops where a first device runs the applications and a second device receives the visual display that appears on the first device over a communication network (e.g., Internet). Such remote desktops can allow users to access and control resources and data on the first device at a remote location using a second (e.g., portable) device. 
     One drawback to such an approach arises from the fact that such devices are generally used differently, so applications may be optimized for one type of device, but not another. For example, the different devices may have different sizes and input options (e.g., keyboard, keypad, touchscreen). The display of one device may not be optimized for a second device. For example, if a desktop computer display is shrunk to fit on a smartphone screen, the shrunken size may be difficult for the user to read or discern what is being displayed. Alternatively, if the display is not shrunken, the smartphone may only be able to display a portion of the original display at a time, which also adds to the difficulty in reading and discerning what is being displayed. While some devices allow for manual adjustment of the display by the user, changing displays and images may require the user to continually re-adjust the display, which may be unwieldy and inconvenient. Such is the case in many applications where content only uses a portion of the screen and user interface portions (e.g., toolbars, status bars, scroll bars, rulers). 
     There is, therefore, a need in the art for improved systems and methods for decomposition and recomposition for cross-platform display. 
     SUMMARY OF THE CLAIMED INVENTION 
     Embodiments of the present invention include systems and methods of decomposition and recomposition for cross-platform display. Data including information regarding a display of the host device may be received. Such information may be used to identify multiple portions of the host device display. Each portion is determined to be associated with a description. Each description is mapped to a component that may be used by the client device. Instructions may be generated for the client device, such that the client device display may include a corresponding display that is reconfigured in an optimal way for the client device based on the mapped descriptions. 
     Various embodiments of the present invention include methods of decomposition and recomposition for cross-platform display. Such methods may include receiving data including information regarding a display of a host device, executing instructions to identify multiple portions of the display of the host device, determine an associated description for each portion, map the description to a corresponding component for the client device, and generate instructions for a display of a client device based on the mapped descriptions. As a result, the client device display may correspond to the host device display, though reconfigured based on the mapped descriptions. 
     Embodiments of the present invention may further include systems of decomposition and recomposition for cross-platform display. Such systems may include a host device and a client device requesting information from a host device. The host device may provide data including information regarding a display of the host device. Such information may be used to identify multiple portions of the display of the host device, determine an associated description for each portion, map the description to a corresponding component for the client device, and generate instructions for a display of a client device based on the mapped descriptions. Execution of such instructions by the client device may result in a display corresponding to the host device display, though reconfigured based on the mapped descriptions. In some embodiments, an intermediate device (e.g., server) may receive the request from the client device, retrieve information from the host device, identify multiple portions of the display of the host device, determine an associated description for each portion, map the description to a corresponding component for the client device, generate instructions for a display of a client device based on the mapped descriptions, and send the instructions to the client device. 
     Other embodiments of the present invention include non-transitory computer-readable storage media on which is embodied instructions executable to perform a method of decomposition and recomposition for cross-platform display as previously set forth above. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates a network environment in which an exemplary system of decomposition and recomposition for cross-platform display may be implemented. 
         FIG. 2  is a flowchart illustrating an exemplary method of decomposition and recomposition for cross-platform display. 
         FIG. 3A  is a screenshot of an exemplary display of a host device. 
         FIG. 3B  is a screenshot of a display of a client device that is based on instructions generated in accordance with an exemplary method of decomposition and recomposition for cross-platform display. 
         FIG. 3C  is another screenshot of a display of a client device that is based on instructions generated in accordance with an exemplary method of decomposition and recomposition for cross-platform display. 
     
    
    
     DETAILED DESCRIPTION 
     Decomposition and recomposition for cross-platform display are provided. Data including information regarding a display of the host device may be received. Such information may be used to identify multiple portions of the host device display. Each portion is determined to be associated with a description. Each description is mapped to a component on the client device. Instructions may be generated for the client device, such that the client device display may include a corresponding display that is reconfigured in an optimal way for the client device based on the mapped descriptions. 
       FIG. 1  illustrates a network environment  100  in which a system for decomposition and recomposition for cross-platform display may be implemented. Network environment  100  may include a communication network  110 , one or more user devices  120 A-C, and a server  130 . Devices in network environment  100  may communicate with each other via communications network  110 . 
     Communication network  110  may be a local, proprietary network (e.g., an intranet) and/or may be a part of a larger wide-area network (e.g., the cloud). The communications network  110  may be a local area network (LAN), which may be communicatively coupled to a wide area network (WAN) such as the Internet. The Internet is a broad network of interconnected computers and servers allowing for the transmission and exchange of Internet Protocol (IP) data between users connected through a network service provider. Examples of network service providers are the public switched telephone network, a cable service provider, a provider of digital subscriber line (DSL) services, or a satellite service provider. Communications network  110  allows for communication between the various components of network environment  100 . 
     Users may use any number of different electronic user devices  120 A-C, such as general purpose computers, mobile phones, smartphones, personal digital assistants (PDAs), portable computing devices (e.g., laptop, netbook, tablets), desktop computing devices, handheld computing devices, or any other type of computing device capable of communicating over communication network  110 . User devices  120  may also be configured to access data from other storage media, such as memory cards or disk drives as may be appropriate in the case of downloaded services. User device  120  may include standard hardware computing components such as network and media interfaces, non-transitory computer-readable storage (memory), and processors for executing instructions that may be stored in memory. 
     User device  120 A is illustrated as a mobile phone or smartphone, while user device  120 B is illustrated as a tablet computing device and client device  120 C is illustrated as a desktop device. As can be seen, each user device  120  is sized differently and/or has different input options. Exemplary embodiments of the present invention allow for tasks and applications that are specific to one user device  120  (e.g., desktop or laptop operating in a Microsoft Windows® environment) to be used and optimized for another user device  120  (e.g., touchscreen tablet operating in an Apple iOS® environment). 
     Each user device  120  may act as a host device interacting with a client device;
         likewise, each user device  120  may act as the client device in communication with a host. A user device  120  may further include a client application, a client 3D library, and a client display driver. Collectively, these elements may enable the user device and the user to consume computer graphics resources or services provided by server  130 .       

     Server  130  may include any type of server or other computing device as is known in the art, including standard hardware computing components such as network and media interfaces, non-transitory computer-readable storage (memory), and processors for executing instructions or accessing information that may be stored in memory. The functionalities of multiple servers may be integrated into a single server. Any of the aforementioned servers (or an integrated server) may take on certain client-side, cache, or proxy server characteristics. These characteristics may depend on the particular network placement of the server or certain configurations of the server. 
     Server  130  may associated with the same user and located in the same local network as user device  120 C. Alternatively, server  130  may be located remotely (e.g., in the cloud) and may be associated with a third party that provides services in accordance with embodiments of the present invention. In some instances, the services may be provided via software (e.g., software as a service) downloaded from server  130  to one or more user devices  120 . Updated software may similarly be downloaded as the updates become available or as needed. 
     Server application may represent an application executing (“running”) on server  130 . The functionality of server application may be visible to and accessible by client  120  via application publishing over the cloud (e.g., communication network  110 ), such as that supported by GraphOn GO-Global, Microsoft Remote Desktop Services, and Citrix XenApp. Examples of server application may include a computer-aided design (CAD) application, such as AutoCAD® (by Autodesk, Inc. of San Rafael, Calif.) or Cadence Virtuoso (by Cadence Design Systems of San Jose, Calif.), a medical clinical workflow application such as Symbia.net (by Siemens AG of Munich, Germany), an interactive mapping application such as Google Earth (by Google, Inc of Mountain View, Calif.), or a 3D game. 
       FIG. 2  illustrates a method  200  for decomposition and recomposition for cross-platform display. The method  200  of  FIG. 2  may be embodied as executable instructions in a non-transitory computer readable storage medium including but not limited to a CD, DVD, or non-volatile memory such as a hard drive. The instructions of the storage medium may be executed by a processor (or processors) to cause various hardware components of a computing device hosting or otherwise accessing the storage medium to effectuate the method. The steps identified in  FIG. 2  (and the order thereof) are exemplary and may include various alternatives, equivalents, or derivations thereof including but not limited to the order of execution of the same. 
     In method  200  of  FIG. 2 , a request is sent to host device, responsive data is received, the responsive data is evaluated to identify multiple portions of the display of the host device, determine an associated description for each portion, map the description to a corresponding component for the client device, and generate instructions for a display of a client device based on the mapped descriptions. The display instructions are executable by a client device to generate a display corresponding to the host device display, though reconfigured for the client device. In some embodiments, reconfiguration may include hiding, moving, or displaying various portions in a manner that is specific to the client device and that therefore appears different from the corresponding portion(s) of the host device. 
     In step  210 , a request initiated at a requesting client device (e.g., user device  120 A) is sent to a host device. The host device may be another user device  120  (e.g., user device  120 C) and may host various types of data and resources, any of which may be subject of the request sent by the requesting client device  120 A. For example, a mobile device  120 A may request access to certain host data or may request that a task be performed by host device  120 C (and/or host device applications). 
     In step  220 , data responsive to the request is received. The responsive data includes information regarding a display of the host device  120 C, which may have changed in response to the request sent in step  210 . For example, where the mobile device  120 A requests access to a document hosted on host device  120 C, the display may include a window presenting at least a portion of the requested document. Similarly, where the mobile device  120 A requests that host device  120 C perform a task (e.g., launch an application), the display may include a window presenting a user interface associated with the application. In some embodiments, the data may include descriptions of the host device display, which may include XML-based (extensible markup language) descriptions. 
     In step  230 , portions of the display of the host device  120 C are identified. The display of the host device  120 C may additionally include one or more interface portions. For example, a display of a Microsoft® Word document may include a portion displaying the actual content (e.g., the Word document), in addition to various toolbars, status bars, scroll bars, rulers, etc. that may be present. 
       FIG. 3A  illustrates a screenshot of an exemplary display of a host device. Depending on the application(s) currently running on host device  120 C, there may be any number of content or interface portions that may be present in the display. The responsive data received in step  220  may include information indicating how many portions and what kind are currently associated with the display of host device  120 C. As illustrated,  FIG. 3A  illustrates a content portion, as well as a menu bar, a tool bar, a status bar, and a scroll bar. 
     In step  240 , a description is determined to be associated with each of the identified portions of the host device display. For example, the tool bar in  FIG. 3A , as well as its associated tools may be associated with particular commands or instructions. The data received in step  220  (e.g., XML-based descriptions) may indicate or otherwise be used to determine which commands or instructions are associated with each portion (and/or sub-portion). This data may come primarily from a tool that can be used prior to publishing an application for remote access. This tool may utilize a combination of data sources including but not limited to application resources, window class, and tree view tagging. Application resources may include menus and dialogs within the application resources that would be used to identify relevant information. With respect to window class a running application may have certain window types that can be identified via the “Window Class”. Tree view tagging tool may provide a tree view of the window hierarchy of an application in which certain portions of the application can be tagged as to how they should be presented on the various devices such as the main content area, toolbar area(s), etc. 
     In step  250 , the descriptions are remapped to an associated component for use by client device  120 A. The component may or may not be immediately displayed on the client device  120 A. For example, a menu tab (e.g., Menu 1) displayed on a host device display may be hidden on the client device display until requested by the user.  FIGS. 3B and 3C  are client device displays generally corresponding to the host device display of  FIG. 3A . As can be seen in  FIG. 3B , the scroll bar of the host device display has been omitted. In addition, the menu bar, tool bar, and status bar of the host device display have been reconfigured into a single bar. The particular menus (e.g., Menu 1, Menu 2, and Menu 3) are not immediately visible in  FIG. 3B . As can be seen in  FIG. 3C , the particular menus (and sub-menus/menu options) may be reconfigured into pop-up windows that may appear, for example, when requested by a user. Such mapping for use by client device  120 A may be based on automatic logic and/or developer-drive logic. Such logic may further be specific to the particular applications, hosts, clients, or host-client combinations. 
     In step  260 , instructions regarding the display of the client device  120 A are generated and provided to client device  120 A for execution. The instructions provide for a corresponding display (e.g., of a content portion) of the host device  120 C display with omitted or reconfigured portions (e.g., toolbars, status bars, scroll bars, rulers). Other adjustments of the client device  120 A display are also possible, including re-centering, adjusting text size, and other display adjustments known in the art. 
     Various embodiments of the present invention allow for the method  200  to be performed by an intermediary device (e.g., server  130 ) which may be associated with the host device or reside elsewhere in the network (e.g., in the cloud). For example, server  130  may receive a request from a requesting client device  120 A, forward to a host client device  120 C, receive responsive data from host client device  120 C, use the responsive data to identify the portions of the host device display, determine an associated description for each portion, map to a component for use by client device  120 A, and generate the instructions for the client device  120 A display based on the map, which may then be sent to client device  120 A. Upon execution, the client device display may correspond to a display (e.g., a content portion) on the host device  120 C, but the client device  120 A may omit or reconfigure one or more portions (e.g., such that the portion is hidden until requested). 
     Alternatively, the method  200  may be performed by software downloaded to a user device  120 . For example, software located at either requesting client device  120 A or host device  120 C may provide a request initiated at the requesting client device  120 A to host client device  120 C, receive data responsive to the request as provided by host client device  120 C, use the responsive data to identify the portions of the host device display, determine an associated description for each portion, map to a component for use by client device  120 A, and generate the instructions for client device  120 A display based on the map, which may then be sent or otherwise provided to client device  120 A for execution. The result may include a content portion corresponding to the host device  120 C display, but other portion(s) may be omitted or reconfigured as compared to the host device  120 C display. 
     The present invention may be implemented in an application that may be operable using a variety of devices. Non-transitory computer-readable storage media refer to any medium or media that participate in providing instructions to a central processing unit (CPU) for execution. Such media can take many forms, including, but not limited to, non-volatile and volatile media such as optical or magnetic disks and dynamic memory, respectively. Common forms of non-transitory computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM disk, digital video disk (DVD), any other optical medium, RAM, PROM, EPROM, a FLASHEPROM, and any other memory chip or cartridge. 
     Various forms of transmission media may be involved in carrying one or more sequences of one or more instructions to a CPU for execution. A bus carries the data to system RAM, from which a CPU retrieves and executes the instructions. The instructions received by system RAM can optionally be stored on a fixed disk either before or after execution by a CPU. Various forms of storage may likewise be implemented as well as the necessary network interfaces and network topologies to implement the same. 
     While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. The descriptions are not intended to limit the scope of the invention to the particular forms set forth herein. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments. It should be understood that the above description is illustrative and not restrictive. To the contrary, the present descriptions are intended to cover such alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims and otherwise appreciated by one of ordinary skill in the art. The scope of the invention should, therefore, be determined not with reference to the above description, but instead should be determined with reference to the appended claims along with their full scope of equivalents.