Abstract:
Method and systems of providing remote access to an application in a tiered remote access framework that includes a server tier and a client tier. In the server tier, an application that is remotely accessed and a server remote access application execute on a server. In the client tier, a client remote access application executes on a client device. A server SDK may be associated with the application in the server tier, where the server SDK being adapted to communicate display information to the client tier. A client SDK may be associated with a client application executing on the client device, where the client SDK receives the display information from the server tier. The client device connects to the server at an enumerated Uniform Resource Locator (URL) to initiate the reception of the display information.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of U.S. patent application Ser. No. 13/632,245, filed Oct. 1, 2012, and entitled “Uncoupled Application Extensions Including Interactive Digital Surface Layer for Collaborative Remote Application Sharing and Annotating.” The present application also claims priority to U.S. Provisional Patent Application No. 61/541,540, filed Sep. 30, 2011 and U.S. Provisional Patent Application No. 61/623,108, filed Apr. 12, 2012, each entitled “Uncoupled Application Extensions Including Acetate Layer for Collaborative Remote Application Sharing and Annotating.” The above applications are incorporated herein by reference in their entireties. 
    
    
     BACKGROUND 
     Ubiquitous remote access to services, application programs and data has become commonplace as a result of the growth and availability of broadband and wireless network access. As such, users are accessing application programs and data using an ever-growing variety of client devices (e.g., mobile devices, tablet computing devices, laptop/notebook/desktop computers, etc.). Data may be communicated to the devices from a remote server over a variety of networks including, 3G and 4G mobile data networks, wireless networks such as WiFi and WiMax, wired networks, etc. Clients may connect to a server offering the services, applications programs and data across many disparate network bandwidths and latencies. 
     In such an environment, applications may also be shared among remote users in a collaborative session. However, when collaborating, users may be limited solely to the functionalities provided by the shared application, thus limiting the collaborative session. 
     SUMMARY 
     Disclosed herein are systems and methods for providing remote access to an application in a tiered remote access framework. In accordance with the present disclosure, a method of providing remote access to an application is disclosed. The method may include providing a tiered remote access framework comprising a server tier in which an application that is remotely accessed and a server remote access application execute on a server, and a client tier in which a client remote access application executes on a client device; providing a server SDK that is associated with the application in the server tier, the server SDK being adapted to communicate display information to the client tier; and providing a client SDK that is associated with a client application executing on the client device, the client SDK being adapted to receive the display information from the server tier. The client device connects to the server at an enumerated Uniform Resource Locator (URL) associated with the application program to initiate the reception of the display information. 
     In accordance with other aspects, there is disclosed a method of providing remote access to an application. The method may include providing a tiered remote access framework that includes a server tier in which an application that is remotely accessed and a server remote access application execute on a server; and a client tier in which a client remote access application executes on a client device. The method also includes enumerating a Uniform Resource Locator (URL) that is associated with the application in the server tier; receiving a connection from the client device at the enumerated URL; and communicating display information associated with the application to the client remote access application. 
     Other systems, methods, features and/or advantages will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features and/or advantages be included within this description and be protected by the accompanying claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views. 
         FIG. 1  is a simplified block diagram illustrating a system for providing remote access to an application at a remote device via a computer network; 
         FIG. 2  is a state model in accordance with the present disclosure; 
         FIG. 3A  illustrates additional aspects of the system of  FIG. 1 ; 
         FIG. 3B  illustrates the data flow of the extension state model and the application state model within the environment of  FIGS. 1 and 2 ; 
         FIG. 4A  illustrates a tree within an XML state model document; 
         FIGS. 4B and 4C  illustrate example axial and coronal medical image views associated with the XML state model of  FIG. 4A ; 
         FIGS. 5A and 5B  illustrate an example user interface containing an application that may be shared in a collaborative session between users of client computing devices; 
         FIG. 6A  illustrates a flow diagram of example operations performed within the system of  FIGS. 1-5 ; 
         FIG. 6B  illustrates a flow diagram of other example operations performed within the system of  FIGS. 1-5 ; and 
         FIG. 7  illustrates an exemplary computing device. 
     
    
    
     DETAILED DESCRIPTION 
     Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present disclosure. While implementations will be described for remotely accessing applications, it will become evident to those skilled in the art that the implementations are not limited thereto, but are applicable for remotely accessing any type of data or service via a remote device. 
     Referring to  FIG. 1 , a system  100  for providing remote access to an application, data or other service via a computer network. The system comprises a client computing device  112 A or  112 B, such as a wireless handheld device such as, for example, an IPHONE  112 A or a BLACKBERRY  112 B connected via a computer network  110  such as, for example, the Internet, to a server  102 B. Similarly, the client computing devices may also include a desktop/notebook personal computer  112 C or a tablet device  112 N that are connected by the communication network  110  to the server  102 B. It is noted that the connections to the communication network  110  may be any type of connection, for example, Wi-Fi (IEEE 802.11x), WiMax (IEEE 802.16), Ethernet, 3G, 4G, etc. 
     The server  102 B is connected, for example, via the computer network  110  to a Local Area Network (LAN)  109  or may be directly connected to the computer network  110 . For example, the LAN  109  is an internal computer network of an institution such as a hospital, a bank, a large business, or a government department. Typically, such institutions still use a mainframe computer  102 A and a database  108  connected to the LAN  109 . Numerous application programs  107 A may be stored in memory  106 A of the mainframe computer  102 A and executed on a processor  104 A. Similarly, numerous application programs  107 B may be stored in memory  106 B of the server  102 B and executed on a processor  104 B. The application programs  107 A and  107 B may be “services” offered for remote access. The mainframe computer  102 A, the server  102 B and the client computing devices  112 A,  112 B,  112 C or  112 N may be implemented using hardware such as that shown in the general purpose computing device of  FIG. 7 . 
     A client remote access application  121 A,  121 B,  121 C,  121 N may be designed for providing user interaction for displaying data and/or imagery in a human comprehensible fashion and for determining user input data in dependence upon received user instructions for interacting with the application program using, for example, a graphical display with touch-screen  114 A or a graphical display  114 B/ 114 N and a keyboard  116 B/ 116 C of the client computing devices  112 A,  112 B,  112 C,  112 N, respectively. For example, the client remote access application is performed by executing executable commands on processor  118 A,  118 B,  118 C,  118 N with the commands being stored in memory  120 A,  120 B,  120 C,  120 N of the client computing devices  112 A,  112 B,  112 C,  112 N, respectively. 
     Alternatively or additionally, a user interface program is executed on the server  102 B (as one of application programs  107 B) which is then accessed via an URL by a generic client application such as, for example, a web browser executed on the client computing device  112 A,  112 B. The user interface is implemented using, for example, Hyper Text Markup Language HTML 5. In some implementations, the server  102 B may participate in a collaborative session with the client computing devices  112 A,  112 B,  112 C . . .  112 N. For example, the aforementioned one of the application programs  107 B may enable the server  102 B to collaboratively interact with the application program  107 A or another application program  107 B and the client remote access applications  121 A,  121 B,  121 C,  121 N. As such, the server  102 B and each of the participating client computing devices  112 A,  112 B,  112 C . . .  112 N may present a synchronized view of the display of the application program. 
     The operation of a server remote access application  111 B with the client remote access application (any of  121 A,  121 B,  121 C,  121 N, or one of application programs  107 B) is performed in cooperation with a state model  200 , as illustrated in  FIG. 2 . An example of the server remote access program is provided in PUREWEB, available from Calgary Scientific, Alberta, Canada. When executed, the client remote access application updates the state model  200  in accordance with user input data received from a user interface program. The remote access application may generate control data in accordance with the updated state model  200 , and provide the same to the server remote access application  111 B running on the server  102 B. 
     Upon receipt of application data from an application program  107 A or  1097 B, the server remote access application  111 B updates the state model  200  in accordance with the screen or application data, generates presentation data in accordance with the updated state model  200 , and provides the same to the client remote access application  121 A,  121 B,  121 C,  121 N on the client computing device. The state model  200  comprises an association of logical elements of the application program with corresponding states of the application program, with the logical elements being in a hierarchical order. For example, the logical elements may be a screen, a menu, a submenu, a button, etc. that make up the application program user interface. This enables the client device, for example, to natively display the logical elements. As such, a menu of the application program that is presented on a mobile phone will look like a native menu of the mobile phone. Similarly, the menu of the application program that is presented on desktop computer will look like a native menu of the desktop computer operating system. 
     The state model  200  is determined such that each of the logical elements is associated with a corresponding state of the application program  107 A or  107 B. The state model  200  may be determined such that the logical elements are associated with user interactions. For example, the logical elements of the application program are determined such that the logical elements comprise transition elements with each transition element relating a change of the state model  200  to one of control data and application representation data associated therewith. 
     In some implementations, two or more of the client computing devices  112 A,  112 B,  112 C . . .  112 N and/or the server  102 B may collaboratively interact with the application program  107 A or  107 B. As such, by communicating state information between each of the client computing devices  112 A,  112 B,  112 C . . .  112 N and/or the server  102 B and/or the mainframe computer  102 A participating in a collaborative session, each of the participating client computing devices  112 A,  112 B,  112 C . . .  112 N may present a synchronized view of the display of the application program  107 A or  107 B. 
     In accordance with some implementations, the system  100  may provide for uncoupled or decoupled application extensions. Such extensions are provided as part of the server remote access application  111 B (e.g., as a plug-in), the client remote access applications  121 A,  121 B,  121 C,  121 N (e.g., as part of a client software development kit (SDK)), one of the applications  107 B (e.g., as part of a server SDK), or combinations thereof to provide features and functionalities that are otherwise are not provided by the application programs  107 A or  107 B. These are described more fully with regard to  FIGS. 3A-3B , below. These features and functionalities may be provided without a need to modify the application programs  107 A or  107 B, as they are integral with the remote access applications. As such, the uncoupled application extensions are agnostic to the application itself, i.e., the application extensions do not depend on the application being displayed within the server remote access application  111 B and client remote access application  121 A,  121 B,  121 C,  121 N. Further, the application extensions may be made available within controls presented by the server remote access application  111 B or client remote access application  121 A,  121 B,  121 C,  121 N. 
     For example, an “interactive digital surface layer” may be provided as an application extension to enable participants to a collaborative session to make annotations on top of the application running in the session. Such a layer may function as an “acetate layer” in that it is transparently overlaid on top of the application running in the session. The interactive digital surface layer functions like a scribble tool to enable a user to draw lines, arrows, symbols, scribbles, etc. on top of an application to provide collaboration of both the application and the interactive digital surface layer. As will be described below with reference to  FIGS. 4A and 4B , the interactive digital surface layer is available as a control within the environment of  FIG. 1 . 
       FIGS. 3A-3B  illustrate aspects of the system  100  of  FIGS. 1-2  in greater detail.  FIG. 3A  illustrates the system  100  as having a tiered software stack. The client remote application  121 A,  121 B,  121 C,  121 N may sit on top of a client software development kit (SDK)  304  in a client tier. The client tier communicates to the server remote access application  111 B in a server tier. The server tier communicates to a state manager  308  sitting on top of the applications  107 A/ 107 B and a server SDK  312  in an application tier. As noted above, the application extensions may be implemented in any of the tiers, i.e., within the server tier as a plug-in  306 , the client tier as client application extension  302 , the application tier as application extension  310 , or combinations thereof. The state model  200  is communicated among the tiers and may be modified in any of the tiers by the application extensions  302  and  310 , and the plug-in  306 . 
     In yet another example, in the application tier, the application extension  310  may be a separate executable program that includes new features to enhance the applications  107 A/ 107 B. The application extension  310  may consume the state model  200  and produce its own document  314  (i.e., a state model of the application extension  310 ) that may include: (1) information from the state model  200  and information associated with the application extension  310 , (2) only information associated with the application extension  310 , or (3) a combination of some of the state model information and information associated with the extension state model  314 . The extension state model  314  may be communicated to the server remote access application  111 B, where the server remote access application  111 B may compose an updated state model  200  to include the information in the extension state model  314 . Alternatively or additionally, the client remote access application  121 A,  121 B,  121 C,  121 N may receive both the state model  200  and the extension state model  314 , and the client remote access application may compose an updated state model  200  to include the information in the extension state model  314 . 
       FIG. 3B  illustrates the data flow of the extension state model  314  and the application state model  200  within the environment of  FIGS. 1 and 2 . As illustrated, the application extensions  302  communicate the extension state model  314  between the client computing devices, e.g., client computing device  121 A and client computing device  121 B. For example, client remote access applications  121 A and  121 B may be the communication mechanism used by the application extensions  310  to enable synchronization of the extension state model  314 . Concurrently, the application state model  200  is communicated between the client computing device  121 A, the client computing device  121 B and, e.g., the server  102 . As such, in accordance with the data flow of  FIG. 3B , the states of the extension  310  and the application  107 B may synchronized between the various participants in a collaborative session. Additionally or alternatively, the states are separately maintained to uncouple the extension  310  from the application  107 B. 
     In general, the interactive digital surface layer may be used for any purpose to provide information to a user and/or provide features and functionalities that are independent of the application being shared by users in a collaborative session. The interactive digital surface layer may provide such features concurrently with the execution of the application or as a stand-alone application. For example, the interactive digital surface layer may be used to enable users to annotate a display as the application is executing, to enable pointing operations by the user, and to provide notifications about status or user availability. The interactive digital surface layer may be enabled and disabled by a user control and may be controlled by a session leader or each participant within the collaborative session. Additional features may be provided within the framework discussed above. Details of the above-noted features will now be provided. 
       FIG. 4A  illustrates a tree within an XML state model document that describes an uncoupled application extension, such as the interactive digital surface layer.  FIGS. 4B and 4C  illustrate example axial and coronal medical image views associated with the XML state model of  FIG. 4A  to illustrate the views presented to users within a collaborative session. Within the XML tree, there is a collaboration node defied that includes one or more sessions. The sessions are associated with the application extensions, such as the interactive digital surface layer. The participants in the sessions are identified by a UserInfo tag, and may be, for example Glen and Jacquie. Each participant is assigned a default color (DefaultColor) to represent the user&#39;s annotations within the interactive digital surface layer (e.g., blue for Glen and green for Jacquie). Any displayable color may be selected as a default color for participants to the collaborative session. A prioritization of colors may be defined, such that a first user is assigned blue, a second user is assigned green, a third user is assigned orange, etc. 
     Under the collaboration node there are also one or more views defined. In the example of  FIG. 4A , Glen and Jacquie may be collaborating within a medical imaging application. As such, there may be two views defined—an axial view and a coronal view (see,  FIGS. 4B and 4C ). Sessions are associated with each of the views, where the sessions include the users to the collaboration. For the axial view, Glen&#39;s session has associated therewith a cursor position (CP) and certain markups, e.g., a scribble, arrow and circle. In the axial view, Jacquie has an associated cursor position, but since she has not made any markups to the interactive digital surface layer, there is no additional information associated with Jacquie&#39;s axial session view. Under the coronal session, each user only has a cursor position associated therewith. 
     The above information is displayed by the client remote access application as illustrated in axial views of  FIGS. 4B and 4C , which may be displayed on a client computing device associated with Glen and Jacquie, respectively. In  FIG. 4B , Glen may be viewing the application on a client computing device such as a laptop, which has a mid-sized display. As such, Glen is able to view both the axial view and the coronal view at the same time. With reference to  FIG. 4C , Jacquie may be viewing the application on a smaller computing device, such as a handheld wireless device. As such, only the axial view may be presented due to the more limited display area of such a device. 
     Below is an example section of a state model  200  in accordance with the tree of  FIG. 4A . The state model  200  may be represented by, e.g., an Extensible Markup Language (XML) document. Other representations of the state model  200  may be used. Information regarding the application program and interactive digital surface layer is communicated in the state model  200 . Because the interactive digital surface layer is uncoupled from the application, the information regarding the interactive digital surface layer is not part of the application state (i.e., it is abstracted from the application). Rather, the interactive digital surface layer information, i.e., the extension state information, is separately maintained in the state model  200 , as shown between the Collaboration tags. Other uncoupled application extension may be similarly defined in the state model  200 . 
     
       
         
               
             
           
               
                   
               
             
             
               
                 &lt;ApplicationState &gt; 
               
               
                   &lt;Screens&gt; 
               
               
                     &lt;Screen id=“0” name=“Axial”&gt; 
               
               
                       &lt;Fields&gt; 
               
               
                         &lt;Field name=“name” description=“Name” default=“”&gt; 
               
               
                         &lt;Type fieldType=“Text” maxChars=“128” /&gt; 
               
               
                         &lt;Validation /&gt; 
               
               
                         &lt;/Field&gt; 
               
               
                       &lt;/Fields&gt; 
               
               
                     &lt;/Screen&gt; 
               
               
                     &lt;Screen id=“1” name=“ Coronal” /&gt; 
               
               
                   &lt;/Screens&gt; 
               
               
                   &lt;Screen Data&gt; 
               
               
                     &lt;CurrentScreen id=“0” /&gt; 
               
               
                     &lt;Screen id=“0”&gt; 
               
               
                     &lt;/Screen&gt; 
               
               
                   &lt;/ScreenData&gt; 
               
               
                 &lt;/ApplicationState&gt; 
               
               
                 &lt;Collaboration&gt; 
               
               
                   &lt;Sessions&gt; 
               
               
                     &lt;UserInfo=“Glen” DefaultColor=“Blue” /&gt; 
               
               
                     &lt;UserInfo=“Jacquie” DefaultColor=“Green” /&gt; 
               
               
                   &lt;/Sessions&gt; 
               
               
                   &lt;Views&gt; 
               
               
                     &lt;Axial&gt; 
               
               
                       &lt;Sessions&gt; 
               
               
                         &lt;UserName=“Glen” CP=“XY” Markups= 
               
               
                         “Scribble Arrow Circle” /&gt; 
               
               
                         &lt;UserName=“Jacquie” CP=“XY”/&gt; 
               
               
                       &lt;/Sessions&gt; 
               
               
                     &lt;/Axial&gt; 
               
               
                     &lt;Coronal&gt; 
               
               
                       &lt;Sessions&gt; 
               
               
                         &lt;UserName=“Glen” CP=“XY” /&gt; 
               
               
                         &lt;UserName=“Jacquie” CP=“XY”/&gt; 
               
               
                       &lt;/Sessions&gt; 
               
               
                     &lt;/Coronal&gt; 
               
               
                   &lt;/Views&gt; 
               
               
                 &lt;/Collaboration&gt; 
               
               
                   
               
             
          
         
       
     
     Information regarding the application ( 107 A or  107 B) is maintained in the ApplicationState node in a first portion of the XML state model. Different states of the application program associated with the axial view and the coronal view are defined, as well as related triggers. For example, in the axial view a “field” is defined for receiving a name as user input data and displaying the same. The uncoupled collaboration states and application extension states (e.g., interactive digital surface layer) are maintained in a second portion of the XML document. 
     The state model  200  may thus contain session information about the application itself, the application extension information (e.g., interactive digital surface layer), information about views, and how to tie the annotations to specific views (e.g., scribble, arrow, circle tied to axial view). 
       FIGS. 5A and 5B  illustrate an example user interface  500  containing an application  502  that may be shared in a collaborative session. In example of  FIGS. 5A and 5B , users may be collaborating with a presentation application, such as MICROSOFT POWERPOINT in the user interface  502 . In the example, the users are creating a schedule of meals, e.g., for a conference. As shown in  FIG. 5A , a user at a first client computing device may activate a sharing control  504  using the client remote access application. The sharing control  504  may provide a list of all available applications which may be shared between users. In the example, five applications may be shared (e.g., an event viewer, the presentation application, a spreadsheet, a movie player, and a word processor). In accordance with an aspect of the present disclosure, the sharing control  504  includes a draw function  508 , which activates the interactive digital surface layer described above. The sharing control  504  may optionally include a remote view mirror  510 , which displays the user interface as shown on a remote client computing device of another user in the collaborative session. 
     If the user activates the draw function  508 , then the interactive digital surface layer is operable to receive user input to collaboratively display annotations input by the users in the session. As noted above, the annotations may be color-coded to each of the users in the collaborative session. Referring to  FIG. 5B , the user interface  500  is shown containing annotations to the application  502  in the interactive digital surface layer. For example, Glen has crossed out the word “Great” has penciled in the word “Small.” As another example, Jacquie has crossed out the location of the lunch and replace it with“?” as the location is uncertain. Thus, in view of the above, one of ordinary skill in the art would understand that any type of annotation may be made using the interactive digital surface layer. 
       FIG. 5B  further illustrates an indicator  520 , for example, an indicator to show there is network latency associated with at least one of the participants in the collaborative session. For example, a responsiveness indicator such as a spinning wheel, a greyed-out area, etc. may be presented when latency is below a predetermined threshold. Such an indicator may be useful during collaboration where individual users have different network response characteristics. For example, the indicator may be presented to all users so they know someone in a collaborative session is lagging. The indicator may be spawned by a process that examines the extension state model  314  for, e.g., timing information that may be used to calculate that a particular link has a data rate below a threshold. 
     In accordance with features of the interactive digital surface layer, users may see each other&#39;s mouse cursor. Each client may erase annotations made within the interactive digital surface layer. The interactive digital surface layer may be saved either separately from the application, or may be consumed by original application and saved in a meaningful way. The interactive digital surface layer may also be bookmarked in case a collaboration session goes down, such that it can be recovered. 
     During a collaborative session, a user may wish to point to an area of the user interface  500  without interacting with the underlying application program  107 A/ 107 B. For example, a user may be making a presentation of a slide deck and may wish to “point” to an item on the slide being displayed in the user interface  200 . The interactive digital surface layer may be used to provide such an indication to other users in the collaborative session. 
     To accommodate the above, the sending of mouse cursor position data may be separated from the sending of mouse input events to the application  107 A/ 107 B so that the position and event data can be triggered independently of one another. As such, a cursor position tool may be directed to send cursor information without input events that would otherwise cause an interaction when the user of the tablet device  112 N does not desire such interaction with the application program  107 A/ 107 B. The above may be achieved by separating a single method that updates the interactive digital surface layer for cursor position into two methods, one of which performs cursor position updates, and one of which queues the input events. Optionally or additionally, the mouse cursor may change characteristics when operating in such a mode. For example, where the mouse cursor is being used for indication purposes, the cursor may thicken, change color, change shape, blink, etc. to indicate to other users that the cursor is being used as an indicator. 
     While the above may be implemented for all types of client computers, a particular use case is where users of mobile devices having a touch-sensitive interface (e.g., tablet device  112 N) wish to indicate to other users what he or she is currently viewing on the display. Typically, a touch of a tablet device represents an interaction with the application program  107 A/ 107 B. In accordance with the above, separating the mouse cursor position data (i.e., the touch location) from the sending of mouse input events (i.e., the actual touch) enables users of tablet devices  112 N to make such an indication similar to client computers having a pointing device. 
     In another aspect that may be combined with the above or separately implemented, annotations can be created in the interactive digital surface layer without interacting with the underlying application program  107 A/ 107 B, and interactions with the underlying application program  107 A/ 107 B do not necessarily create annotations within the interactive digital surface layer. For example, with reference to  FIG. 5B , when Glen crosses the word “Great” and pencils in the word “Small” may he may not want to highlight or change the underlying spreadsheet (i.e., application program  107 A/ 107 B). Therefore, sharing control  504  may be provided with an option to disable interaction with the underlying application  107 A/ 107 B. On the other hand, Glen may want to guide Jacquie in a tutorial of how to edit the same spreadsheet. In this situation, by not disabling interaction with the application program using the sharing control  504 , the interactive digital surface layer may show a cursor to indicate where Glen is currently pointing his mouse and continue to interact with the underlying application program  107 A/ 107 B. 
     Additionally or alternatively to the drawing function above, in some implementations, the interactive digital surface layer may be used to provide other indicators, such as indications of the users within the session, the user who is controlling of the interactive digital surface layer, an indicator that a user has joined or exited a collaboration session, whether changes in image quality (JPEG downsizing) have been made, etc. In general, the indicator may be enabled by providing information in the extension state model  314  that is used to trigger the appearance and/or disappearance of the indicator. In some implementations, the indicator may be temporarily displayed to a user. In other implementations, the indicator may remain until a user action is taken, such as a gesture on the tablet device  112 N, a click of a mouse, etc. 
     In accordance with other implementations, the interactive digital surface layer may be provided access to names of users participating in a collaborative session. The name information may be contained in the extension state model  314  and displayed to other users in the collaborative session using the interactive digital surface layer extension  310 . As noted above, each user may be associated with a unique color that may be used for pointer location and mark-up within the interactive digital surface layer. Each user may toggle the visibility of other user&#39;s cursor positions. Users may clear their own mark-up, and toggle visibility of other users&#39; mark-up. Among the users, a leader may be chosen or default to an initial creator. The leader may clear all mark-up and transfer leadership. If a leader drops from the collaborative session, the leadership may transfer to a next participant in the session. 
     To enable the above, client side APIs may be provided the respond to changes in the extension state model  314  to provide the appropriate displays and controls. For example, APIs may be provided to build user interface components, render images in the interactive digital surface layer, provide a capture/clipboard feature, provide an undo/redo feature, provide for changes to image/line thickness, provide for font selection/size/attribute, select a color, provide for text boxes, provide for emoticon/icons, and provide for watermarks (e.g., confidential, draft, working copy). 
     Based on configuration settings, an API may determine whether the interactive digital surface layer will be “on” for everyone when one user turns the interactive digital surface layer on, if individual users may toggle the visibility of the interactive digital surface layer independent of other users, or if the interactive digital surface layer will be on only if the leader toggles it on. Certain configuration may enable an API to determine whether individual clients may toggle visibility of other users&#39; markup, or if only the leader may control a client&#39;s visibility on behalf of everyone. Yet other configuration may enable an API to determine whether individual clients may toggle visibility of other users&#39; cursor position, or if only the leader may control a client&#39;s cursor visibility on behalf of everyone. It may be specified that only the leader may clear all other users&#39; markup; however, anyone can clear their own markup at any time. 
     In some implementations, the state of the interactive digital surface layer may be saved. For example, the extension state model  314  may be saved at the client computing devices  112 A,  112 B,  112 C or  112 N for later retrieval. A replay option may be provided to “replay” the states within the extension state model  314  that were received over a period of time during an interactive session. Users may be enabled to save a drawing in the interactive digital surface layer and select from a list of saved images. In some implementations, the images may be saved as a set of vectors in an XML file that is loaded so a user may continue editing the drawing. 
     Thus, as described above, the present disclosure provides for an interactive digital surface layer that may be independently provided and used collaboratively by users to provide annotations on top of an application. Also, more generally, the present disclosure provides for application extensions that may be made available through a remote access mechanism. The application extensions provide enhancements to applications through the remote access mechanism without the need to modify the applications themselves. 
       FIG. 6A  illustrates a flow diagram  600  of example operations performed within the system of  FIGS. 1-5 . As described above, the server remote access application  111 B of the server  102 B provides access to one or more applications  107 A or  107 B, which is displayed by the client remote access application  121 A,  121 B,  121 C,  121 N. At  602 , a user to a collaborative session activates a control provided by the client remote access application. The control may be, e.g., the sharing control  504 . 
     At  604 , a selection is received to activate the interactive digital surface layer. For example, a user may select the draw function  508  in the sharing control  604 . At  606 , user inputs are received in the interactive digital surface layer. The user inputs may be received as lines, arrows, squiggles, circles, etc., drawn on the interactive digital surface layer by one or more of the users in the collaborative session. The interactive digital surface layer may function to collaboratively receive the inputs from the one or more users in the session. 
     At  608 , the extension state model is updated. For example, the extension state model  314  may be updated to represent the annotations made by each of the users in the collaborative session. Where the user input is a pre-defined shape, such as a circle, line, square or other geometric shape, the extension state model  314  may represent such shapes using XY coordinates. For a line, the XY coordinates may define an origin and together with a length the line may be defined. For a circle, the coordinate point may be a center, and a radius may be defined from the center to represent the circle. Squiggles may be represented by a series of small vectors, that when combined define the shape of the squiggle. The use of coordinates, rather than pixels, provides for scaling of the shapes in the various displays of the client computing devices participating in a collaborative session. 
     At  610 , the extension state model is communicated to the participants in the collaborative session. As described in  FIGS. 3A and 3B , the extension state model  314  may be passed among the client computing devices, such that each client computing device is synchronized. The state model  200  also may be passed among the client computing devices and the server, such that with respect to the application, each client computing device and the server are synchronized. At  612 , the participant displays are updated to reflect contents of the state model(s). The display of each client computing device and the server may be updated to reflect current state of the extension state model  314  and/or state model  200 . 
       FIG. 6B  illustrates a flow diagram  650  of example operations performed within the system of  FIGS. 1-5 . As described above, the server remote access application  111 B of the server  102 B provides access to one or more applications  107 A or  107 B, which is displayed by the client remote access application  121 A,  121 B,  121 C,  121 N. At  652 , a user to a collaborative session activates a control provided by the client remote access application. The control may be, e.g., the sharing control  504 . 
     At  654 , the interactive digital surface layer is activated. For example, a user may select a function in the sharing control  604  or the interactive digital surface layer may be automatically activated at step  652 . 
     At  656 , notification-related information is processed. For example, the extension state model  314  is received at the client computing device and an application may process information contained in the extension state model  314  to determine if a notification should be presented. For example, timing information in the extension state model  314  may be examined to determine if a participant in the collaborative session is experiencing network latency. As another example, an identity of a new user to a collaborative session may be indicated. 
     At  658 , a notification displayed, if necessary. If the processing at  656  determines that a notification should be presented, then the notification is presented to the user in the interactive digital surface layer. For example, network latency may be greater than a predetermined threshold; as such that indicator  520  may be presented to the user. 
     The processes at  656  and  658  may be repeated to determine if the notification should be removed or the notification may be removed after a predetermined time-out period has expired. 
       FIG. 7  shows an exemplary computing environment in which example embodiments and aspects may be implemented. The computing system environment is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality. 
     Numerous other general purpose or special purpose computing system environments or configurations may be used. Examples of well known computing systems, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network personal computers (PCs), minicomputers, mainframe computers, embedded systems, distributed computing environments that include any of the above systems or devices, and the like. 
     Computer-executable instructions, such as program modules, being executed by a computer may be used. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Distributed computing environments may be used where tasks are performed by remote processing devices that are linked through a communications network or other data transmission medium. In a distributed computing environment, program modules and other data may be located in both local and remote computer storage media including memory storage devices. 
     With reference to  FIG. 7 , an exemplary system for implementing aspects described herein includes a computing device, such as computing device  700 . In its most basic configuration, computing device  700  typically includes at least one processing unit  702  and memory  704 . Depending on the exact configuration and type of computing device, memory  704  may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in  FIG. 7  by dashed line  706 . 
     Computing device  700  may have additional features/functionality. For example, computing device  700  may include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in  FIG. 7  by removable storage  708  and non-removable storage  710 . 
     Computing device  700  typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by device  700  and includes both volatile and non-volatile media, removable and non-removable media. 
     Computer storage media include volatile and non-volatile, and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Memory  704 , removable storage  708 , and non-removable storage  710  are all examples of computer storage media. Computer storage media include, but are not limited to, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device  700 . Any such computer storage media may be part of computing device  700 . 
     Computing device  700  may contain communications connection(s)  712  that allow the device to communicate with other devices. Computing device  700  may also have input device(s)  714  such as a keyboard, mouse, pen, voice input device, touch input device, etc. Output device(s)  716  such as a display, speakers, printer, etc. may also be included. All these devices are well known in the art and need not be discussed at length here. 
     It should be understood that the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination of both. Thus, the methods and apparatus of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the presently disclosed subject matter. In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. One or more programs may implement or utilize the processes described in connection with the presently disclosed subject matter, e.g., through the use of an application programming interface (API), reusable controls, or the like. Such programs may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language and it may be combined with hardware implementations. 
     Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.