Abstract:
In the new architecture, a client that desires access to a piece of hardware sends HTTP requests to a device server. The server receives a HTTP requests, accesses a device on behalf of the client, and send the device&#39;s response back to the client in the form of an HTTP response. This architecture has three primary advantages. First, it offers a clean interface for clients where all devices are accessed as if they are web servers. Second, it helps make devices more secure whereby clients can be prevented from accessing hardware directly, and all device access is forced through the narrow HTTP access protocol and checked for compliance with a desired security policy. Third, since HTTP allows local and remote servers to be contacted, the proposed architecture makes it easy for clients to communicate with devices that are not physically co-resident with the client but which are accessible via a network connection.

Description:
BACKGROUND 
       [0001]    Since its origins in the 1960s and commercialization in the 1990s, the Internet has expanded worldwide to become ubiquitous in social and business activities. When combined with wireless and high speed data links, many new applications have emerged. While the Internet was initially conceived as a communication medium for collaboration between users and applications, it relied on early web browsers that were limited to displaying static content to users. Web browsers now provide an increasingly rich execution platform that supports a variety of programming languages which allow web pages to respond to user input, dynamically update display screens, and issue asynchronous requests for new information. Web browsers can receive and transmit information between applications and the web browsers over the Internet in near real-time. Examples of these applications include handheld game controllers, meteorological data collection sites, webcam devices, and mobile phones and tablet computers that include cameras, Global Positioning System (GPS) receivers and removable data storage. 
         [0002]    Despite increased sophistication and complexity of these applications, web browsers have been slow to allow low-level hardware devices to be accessed using JavaScript, the most popular client-side scripting language. This limitation has become particularly problematic as mobile processing units such as cellular phones and tablet computers include applications that leverage data from hardware devices including accelerometers, GPS receivers and removable storage. Concurrently, mobile processing units include increasingly powerful computational and storage capabilities. As the use of such mobile processing units increase, the requirement to access data from hardware devices will also increase. 
         [0003]    In General, software programs must access hardware devices to interact with the outside world. Examples of hardware devices include hard disks, computer monitors, accelerometers, cameras, and microphones. Programs typically access hardware directly through the use of low-level assembly instructions, or indirectly through services provided by the underlying operating system. These traditional ways of device access arc problematic for three reasons. First, these low-level programming interfaces are often inconsistent and difficult to use, making it hard for developers to write programs that access hardware in sophisticated ways. Second, these programming interfaces often grant the software program wide-ranging capabilities to read and write to hardware devices. If the software program is buggy or malicious, it can wreak havoc with other programs and the user&#39;s data through the improper use of hardware devices. Third, low-level interfaces make it difficult for programs to interact with devices that are not directly co-located with the machine that is running a software program. For example, accessing a locally resident microphone is easily expressed, but accessing a microphone on a remote machine is difficult to express or impossible. 
         [0004]    Ideally, applications could access hardware devices in a location-agnostic way using network protocols. These protocols would provide a narrow, simple interface, making them easy to secure and obviating the need for a large trusted computing base such as a browser. 
       SUMMARY 
       [0005]    The invention disclosed herein solves the problems discussed above by introducing a new programming model. Within this model, hardware devices are treated like a web server where client programs access the devices by sending Hypertext Transfer Protocol (HTTP) requests to the devices. More specifically, a program known as a hardware device server acts as an intermediary between a client program and each hardware device. To access a particular device, a client program sends an HTTP request to an appropriate hardware device server. The hardware device server examines the request and authenticates the client in some way. If the authentication is successful, the hardware device server accesses the particular hardware device as specified by the client program, and returns a result in a HTTP response sent to the requesting client program. 
         [0006]    This architecture solves the three problems identified earlier. First, the programming interface is much cleaner, since all hardware devices arc uniformly presented to client applications as web servers which use HTTP as a communication protocol. Second, the new architecture improves security, since clients can be sandboxed (i.e., prevented from directly accessing hardware). The hardware device server can prevent unauthorized access by safeguarding the narrow HTTP protocol. Third, since HTTP can trivially be used to access both local and remote web servers, the new architecture makes it trivial for clients to access local and remote devices in the same way. 
         [0007]    One embodiment of the disclosed invention allows a web page to securely access hardware devices. The web page can include a device protocol translator which engages an HTTP device protocol with hardware device servers running locally or on remote machines. To access hardware, the page passes a request to the device protocol translator. The device protocol translator crafts the appropriate HTTP request and sends it to the hardware device server. If the requests are authorized, the hardware device server performs the requested actions and returns any data and status information to the requesting web page using a standard HTTP response. The device commands and responses may or may not include data. Users may authorize individual web domains to access each hardware device, and the hardware device server may authenticate each hardware request by ensuring that an HTTP referrer field represents an authorized domain. The device server may also inform the user about which domains are currently authorized and actively using the system. For example, the device server may use sensor widgets, which are small graphical icons that change their visual appearance and make a sound when hardware devices are accessed. 
         [0008]    A hardware device server is a computing device implemented in either hardware or software that enables applications to access hardware devices by issuing HTTP requests to the hardware device server. A device protocol translator may provide an API to a web application which allows the web application to issue high-level device requests without having to know the low-level details of the HTTP device access protocol. The device protocol translator may be implemented by a JavaScript library. The hardware device server is an application which runs in a separate process on a local or remote machine. The hardware device server may receive commands from the device protocol translator, process the commands and cause designated hardware devices to respond according to the commands. To access hardware devices, applications or device protocol translators send hardware device requests to the hardware device server in the form of HTTP requests and, if the requests are authorized, the hardware device server performs the requested actions and returns any data and status information to the requesting web page with a standard HTTP response. The device commands and responses may or may not include data. Users may authorize individual web domains to access each hardware device, and the hardware device server authenticates each hardware request by ensuring that a requester represents an authorized domain. 
         [0009]    The hardware device server does not require an associated browser to be trustworthy, and indeed, the user or the device server itself can sandbox the browser such that the browser cannot access hardware devices directly, and instead must access all such devices using the HTTP access protocol; in this fashion, the device server tightly restricts what the browser can and cannot do. However, a corrupted or malicious browser which contains snooped referrer fields from authorized user requests can send requests to the hardware device server. The hardware device server may use authentication processes to limit these attacks. Methods for authentication and authorization may include manifest authorization, security token exchanges, sensor widget notifications and iframes for device handling encapsulation. Before a web page can access hardware devices, it is desirable that any request be authenticated by the hardware device server. 
         [0010]    This Summary is provided to introduce a selection of concepts in a simplified form that arc further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed matter, nor is it intended to be used to limit the scope of the claimed subject matter. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0011]    These and other features, aspects and advantages of the present invention will become better understood with regard to the following description and accompanying drawings wherein: 
           [0012]      FIG. 1A  is a functional diagram of a software architecture with a web server computer connected to a processing unit and a remotely connected processing unit attached to the processing unit for controlling sensors, processors and storage devices over the Internet; 
           [0013]      FIG. 1B  is a functional diagram of a software architecture with a web server computer connected to a remotely connected processing unit for controlling sensors, processors and storage devices; 
           [0014]      FIG. 1C  is a functional diagram of a software architecture with a web server computer having an embedded hardware device server and web application for controlling sensors, processors and storage devices local to the web server; 
           [0015]      FIG. 2A  represents an exemplary hardware block diagram of a processing unit and  FIG. 2B  represents an exemplary hardware block diagram of a remotely connected processing unit that may communicate with the processing unit for controlling sensors, processors and storage devices; 
           [0016]      FIG. 3  is a flow chart illustrating an exemplary implementation of the operation of API calls to a device protocol translator for controlling sensors, processors and storage devices by a hardware device server; and 
           [0017]      FIG. 4  is a flow chart illustrating an exemplary implementation of the operation of the hardware device server. 
       
    
    
     DETAILED DESCRIPTION 
       [0018]    The disclosed hardware device server may be a software program or equivalent hardware implementation that enables applications to access hardware devices in a manner similar to that used for accessing a web server. The hardware device server acts as a mediator between software applications and hardware devices enabling the hardware devices to be accessed by HTTP network protocols. The platform containing a hardware device of interest runs the hardware device server, which is a small program that exposes an externally visible web server interface to software applications. Other software applications may access a hardware device by sending HTTP requests to a hardware device server associated with the targeted hardware device. The hardware device server accesses the hardware on behalf of the requesting software application and sends a HTTP response back to the requesting software application. The hardware device server may or may not reside on the same platform as the requesting software application. Thus, a requesting software application may use the same HTTP protocol to access local devices and remote devices. 
         [0019]    The following detailed description illustrates a very specific implementation of the hardware and software environment surrounding the disclosed hardware device server that is illustrated in the context of an Internet application accessing a web page running within a browser, and communicating with hardware devices using a HTTP protocol. Note, however, that the core inventive entity is not the specific implementation including the Internet and a web server. The disclosed invention describes the notion of an abstract client application, such as a web page, a C program, or a Python program, accessing hardware via HTTP messages exchanged with a hardware device server in order to access hardware devices associated with the hardware device server. It should be noted that the disclosed device server may be a standalone software program or an equivalent hardware implementation. 
       Software Architecture 
       [0020]    This detailed description below is directed to methods, systems and devices that are particularly useful for accessing local sensors, processors and storage devices by processing units that may be accessed via a HTTP protocol interface. It includes descriptions of hardware and software architectures, application programming interface (API) calls for accessing sensors, processors and storage devices, authentication of hardware requests, and hardware device server implementation. These methods, systems and devices rely on a hardware device server that includes an interface to the localhost domain which uses HTTP Internet protocol as a hardware access protocol for accessing hardware devices, such as sensors, processors and storage devices. The hardware device server is a relatively simple native code application that runs in a separate process on the local machine and exports a web server interface on the localhost domain. This configuration provides device-aware, cross-platform web pages that require no installation and whose security does not depend on a huge trusted computing base such as a web browser. 
         [0021]      FIG. 1A  is a functional diagram of a software architecture  100 , where a web server computer  110  connected to a user&#39;s machine  140  via a network  118  using HTTP Internet protocol for communication  158 . The user&#39;s machine  140  includes an operating system  142  that provides services to a web browser  144 , and the web browser enables operation of the web application  146 . The web application  146  accesses a local hardware device server  156  and a remote hardware device server  124  via a device protocol translator  148 . After the web server computer  110  delivers a web application  146  to the user&#39;s machine  140 , the user  105  interacts with the web application  146 . The device protocol translator  148  is a particular client side implementation of a HTTP device protocol. The web application  146  includes the device protocol translator  148  that accepts API calls  147  from the web application  146 . The API calls  147  are described below in relation to Table 1 and  FIG. 3 . The device protocol translator  148  transmits commands using HTTP protocol  154  to a local hardware device server  156  for controlling various hardware devices such as sensors  130 , processors  132  and storage devices  134  attached to the mobile hardware device  140 . The various hardware devices are hardwired to interface with the hardware device servers  124 ,  156 . The device protocol translator  148  may also transmits commands using HTTP protocol  136  to the remote hardware device server  124  for controlling various hardware devices such as sensors  130 , processors  132  and storage devices  134  attached to remotely connected processing unit  120 . The transmitted commands to the hardware device servers using HTTP protocol are described below in relation to Table 2. The sensor devices  130  may include cameras, microphones, UPS devices, accelerometers, and the like. Processor devices  132  may include other processor cores, graphical processor units, floating point processors, and the like. Storage devices  134  may include internal storage or various removable storage media. The remotely connected hardware device  120  also includes an operating system  122  for controlling operation of the remote hardware device server  124 . The device protocol translator  148  may be implemented with a JavaScript library. 
         [0022]    When a user  105  wants to access data from sensors  130 , processors  132  and storage devices  134  attached to the user&#39;s machine  140  or the remotely connected processing unit  120 , a web application  146  is sent from the web server computer  110  via the network  118  to the web browser  144  in the user&#39;s machine  140 . The web browser  144  commands the web application  146  via the device protocol translator  148  to access the sensors  130 , processors  132  and storage devices  134  connected to either the processing unit  140  or the remotely connected processing unit  120 . The device protocol translator  148  then accesses a selected hardware device  130 ,  132 ,  134  in either the processing unit  140  or the remotely connected processing unit  120  via the hardware device servers  124 ,  156 . 
         [0023]    The hardware device servers  124 ,  156  are small native code applications that reside in separate processes from the web browser  144 . They execute hardware requests on behalf of the web application  146  via the device protocol translator  148 . The device protocol translator  148  authenticates requests from the authentication engines  128  of the hardware device servers  124 ,  156  for allowing authorized web domains access the hardware devices  130 ,  132 ,  134  attached to the user&#39;s machine  140  and the remotely connected processing unit  120 . Device manifests, sensor widgets and authentication tokens may be used for authenticating requests. The hardware device servers  156 ,  124  have to be local to the hardware devices, but not local to client host  146 . 
         [0024]      FIG. 1B  is a functional diagram of a software architecture  102  where a web server computer  112  includes a web application  160  and device protocol translator  160  connected to a remote processing device  120 . The web server computer  112  includes a web application  160  for accessing hardware devices and a device protocol translator  162  that presents an API to the web application  160 . After the web server computer  112  instantiates a web application  160 , the user  105  interacts with the web application  160 . The device protocol translator  148  is a particular client side implementation of a HTTP device protocol. The device protocol translator  162  transmits commands using HTTP protocol  136  to a hardware device server  124  within the remotely connected processing unit  120 . The hardware device server  124  is controlled by an operating system  122  and also includes an authentication engine  128 . Before a web page  160  can access hardware devices, requests must be authenticated by the authentication engine  128  by identifying a requestor as being legitimate. The web server computer  112  includes an operating system that provides services to applications running on the web server computer  112 . 
         [0025]      FIG. 1C  is a functional diagram of a software architecture  103  where a web server computer  114  includes an embedded hardware device server  188  and web application  180  for controlling sensors  130 , processors  132  and storage devices  134  local to the web server computer  114 . After the web server computer  114  instantiates a web application  180 , the user  105  interacts with the web application  180 . The web server computer  114  includes the web application  180  for accessing hardware devices and a device protocol translator  182  that presents an API to the web application  180 . The device protocol translator  148  is a particular client side implementation of a HTTP device protocol. The device protocol translator  182  transmits commands using HTTP protocol  186  to a hardware device server  188  within the web server computer  114  for controlling various hardware devices such as the sensors  130 , processors  132  and storage devices  134  attached to web server computer  114 . The device server  188  also includes an authentication engine  128  for authenticating the requestor of hardware device server requests. The web server computer  114  includes an operating system that provides services to applications running on the web server computer  114 . 
       Hardware Architecture 
       [0026]      FIG. 2A  and  FIG. 2B  are block diagrams  200 ,  202  of exemplary processing units  210 ,  212  for controlling sensors, processors and storage devices over the Internet and local to web server computers.  FIG. 2A  represents a configuration  210  with a capability of implementing the operations of a user&#39;s machine ( 140  in  FIG. 1A ) and  FIG. 2B  represents a configuration  212  with a capability of implementing the operations of the remotely connected processing unit ( 120  in  FIG. 1A  and  FIG. 1B ).  FIG. 2B  is a subset of the configuration shown in  FIG. 2A , and operates in accordance with the description of  FIG. 2A . The memory  250  shown in  FIG. 2A  may be a combination of volatile or non-permanent memory and non-volatile or permanent memory. Generally, the operating system  252 , web browser  254  and hardware device server  260  arc installed in non-volatile memory. The web application  256  and device protocol translator  258  located within the web application  256  are generally downloaded and installed in non-volatile memory. The processor  220  is controlled by the operating system  252 , and controls the other functions shown in  FIG. 2A . The web application interface  232  connects to a web server computer via the Internet, the remote interface  234  connects between a remotely connected processing unit and a local processing unit such as shown in  FIG. 2B  ( 120  in  FIG. 1A  and  FIG. 1B ), the sensor interface  236  connects to hardware sensor devices, the external processor interface  238  connects to processor hardware devices external to the user&#39;  210  (or  212  in  FIG. 2B ), and the storage device interface  240  connects to hardware storage devices. 
       API Calls 
       [0027]    The API calls merely represent examples of interfaces that a client application could invoke in order to access a hardware device server. Different applications may require definition of a different set of APIs. The API call definition is determined by the device protocol translator that connects to the hardware device server. Implementation of the device protocol translator for determining the selected API call may be by a JavaScript library. 
         [0028]    Table 1 is a summary of exemplary API calls to a client side implementation of HTTP device protocol for accessing hardware devices via the hardware device server by a web application.  FIG. 3  is a flow chart illustrating an exemplary implementation of the operation of API calls to the device protocol translator for implementation of HTTP device protocols for controlling sensors, processors and storage devices by a hardware device server. The API calls shown in Table 1 and illustrated in  FIG. 3  define the interface (see  147  in  FIG. 1 ) between a web application and the a device protocol translator (see  146  and  148 , respectively, in  FIG. 1 ). The process is started  310  when a requester application is authenticated and a session is created  312 . All API calls require authentication and manifest validation  314  from a user to receive permission to access certain devices, as shown in the first and second set of calls in Table 1 and  FIG. 3 . Authentication is used for identifying a requestor as being legitimate while the session is open. A manifest is a list of devices that a page wishes to access. A manifest validation is an authorization by a user for a requesting domain to access specific hardware devices. Before a web page can initiate hardware requests to the device protocol translator  310 , it must be authenticated via createSession ( )  312 , which authenticated the legitimacy of the web page. Then it must send its device manifest to the hardware device server via requestAccess ( )  314 . The hardware device server presents the manifest to the user and asks for validation of the requested hardware permissions if the user has not previously validated the hardware permissions to the hardware device server. Prior to granting each device request, each request is requires authentication and validation  312 ,  314 ,  340 . Upon completion of a session and there are no more requests  340 , the API call destroySession ( ) cancels authentication  348  and ends the session  350 . If the hardware device server does not receive a valid authentication or manifest validation, it ignores the createSession ( ) call. 
         [0000]    
       
         
               
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                   
                 CALL 
                 DESCRIPTION 
               
               
                   
               
             
             
               
                 1 
                 createSession ( ) 
                 Authenticate requester. 
               
               
                   
                 destroyS&#39;ession ( ) 
                 Cancel authentication. 
               
               
                 2 
                 requestAccess (manifest) 
                 Ask for permission to access certain  
               
               
                   
                   
                 devices. 
               
               
                 3 
                 singleQuery (name, params) 
                 Get a single sensor sample. 
               
               
                   
                 continuous Query (name,  
                 Start periodic fetch of sensor samples. 
               
               
                   
                 params, period) 
                   
               
               
                 4 
                 startSensor (name) 
                 Turn on sensor. 
               
               
                   
                 stopSensor (name) 
                 Turn off sensor. 
               
               
                   
                 sensorAdded (name) 
                 Upcall fired when a sensor is added. 
               
               
                   
                 sensorRemoved (name) 
                 Upcall fired when a sensor is removed. 
               
               
                   
                 getSensorList ( ) 
                 Get available sensors. 
               
               
                 5 
                 enqueueKernel (kernel) 
                 Queue a computation kernel for execution. 
               
               
                   
                 setKernelData (parameters) 
                 Set the input data for the computation  
               
               
                   
                   
                 pipeline. 
               
               
                   
                 executeKernels ( ) 
                 Run the queued kernels on the input data. 
               
               
                 6 
                 put (storename, key, value) 
                 Put value by key. 
               
               
                   
                 get (storename, key) 
                 Get value by key. 
               
               
                   
               
             
          
         
       
     
       Sensor API 
       [0029]    The API may control several device types  316 . To provide access to sensors  318  like cameras, accelerometers, and GPS units and the like, the API provides a single-sample query interface  324 ,  344  and a continuous query interface  324 ,  330  as shown in the third set of calls in Table 1 and in  FIG. 3 . The API calls singleQuery (name, params) and continuous Query (name, params, period) accept an application-defined callback which is invoked when the data has arrived from the sensor. The data, for example, may be picture information if the sensor is a camera or position information if the sensor is a GPS receiver. The singleQuery (name, params) call accepts the name of the sensor device to query and a device-specific params value which controls sensor-specific properties such as audio sampling rate of data from a microphone. The continuous Query (name, params, period) call also accepts the name of the sensor device to query and a device-specific params value, and also includes an additional parameter representing the period of the sampling rate frequency for continuous query calls when the continuous query is started  330 . 
         [0030]    The fourth set of calls in Table 1 and shown in  FIG. 3  provide a sensor management interface. The API calls startSensor (name) and stopSensor (name) provide power controls that allow a web page to turn on or shut off devices that is not needed. The hardware device server ensures that a device is left on if at least one application still needs it. The API calls sensorAdded (name) and sensorRemoved (name) let applications register callbacks which provide notification to the application when devices arrive or leave. These events are common for off-platform devices such as Bluetooth headsets and some shoe sensors that track motion and distance traveled of the individual wearer. Inserting and removing USB port devices may also cause callbacks to provide notification of this status in some applications. The API call getSensorList ( ) produces a list of all available sensors available to a hardware device server. 
       Processor API 
       [0031]    Multi-core processors and programmable GPUs  320  are available on desktops and transitioning to mobile devices. Simple multi-processor computing models arc available for exporting to mobile devices. A kernel represents a computational task to run on a core. Kernels are restricted to executing two types of predefined functions, primitive functions and built-in functions. Primitive functions are geometric, trigonometric, and comparator operations. Built-in functions are higher-level functions that have been identified as being particularly useful for processing hardware data, and may include the use of primitive functions. For example, built-in functions may include Fast Fourier Transform signal transforms and matrix operations. 
         [0032]    Considering the fifth set of calls in Table 1, a web page passes a kernel to the API by calling enqueueKernel (kernel),  326  as shown in  FIG. 3 . To execute a parallel vector computation with that kernel, the web page calls setKernelData (parameters) with vector arguments  332 . A new copy of the kernel will be instantiated for each argument, the kernels being run in parallel. A web page can also create a computational pipeline by calling enqueueKernel (kernel) multiple times with the same or different kernel. The kernels inputs and outputs will be chained in the order that the kernels were passed to enqueueKernel (kernel). The web page sets the input data for the pipeline by passing a scalar value to setKernelData (parameters). Once an application has created its kernels and set their input data, it calls executeKernels ( ) to start the computation to run the queued kernels on the input data  336 . The kernels are distributed to various cores in the system, cross-kernel communication is coordinated, and an application-provided callback is executed when the computation is complete  342 . 
       Storage API 
       [0033]    The sixth and final set of API calls in Table 1 and  FIG. 3  provide a key/value storage interface  322 . The namespace is partitioned by web domain and by storage device. A web domain can only access data that resides in its partitions. To support removable devices, connection and disconnection notification events arc provided similar to off platform sensors. The API call put (storename, key, value) stores a value by key in storename  328 ,  346 , and get (storename, key) accesses a value by key in storename  338 . 
       Authenticating Hardware Requests 
       [0034]    Privilege separation is used to provide a web page with hardware access while reducing security vulnerabilities. It lets parts of an application run with different levels of privilege, which may be used to prevent non-privileged or low-privileged parts of an application from gaining unauthorized access to parts with higher privileges. The web page and an enclosing web browser that executes the page&#39;s code are both considered to be unfrosted. The web browser is placed in a sandbox which limits its access to certain services, including preventing direct access to hardware devices controlled by the hardware device server. The small, native code hardware device server resides in a separate process from the web browser, and executes hardware requests on behalf of the device protocol translator embedded within the web page application and browser, exchanging data with the device protocol translator via HTTP protocol and HTTP commands. The device protocol translator defines and implements the API described above. It provides for authentication purposes and translates page-initiated hardware requests into HTTP fetches. The hardware device server requires three housekeeping tasks to occur prior to enabling access to the hardware devices. These tasks include manifest authorization, session establishment and session teardown. It should be noted that remote or off-platform devices (see  120  in  FIG. 1 ) are seamlessly accessed in the same manner as the local devices. 
         [0035]      FIG. 4  is a flow chart illustrating an exemplary implementation of the operation of a hardware device server. It represents an exemplary life cycle of a session of the hardware device server. At the start of a session  410 , the hardware device server receives request to create a session  415 . To create a session, a manifest validation  425  and requester authentication  435  arc required to allow access to certain devices. A manifest validation  425  is an authorization by a user for allowing a requesting domain to access specific hardware devices. Upon receipt of a request for a session  415  by the hardware device server from a web page, the web page includes a device manifest in its HTTP request. The manifest is merely a list of devices that the page wishes to access. The hardware device server compares the received manifest with previously stored authenticated manifests  420 . If a match is found between the received manifest and a stored manifest  430 , the next step of creating a new capability token and sending it to the requesting domain  435  is conducted. If a match is not found  430  (normally because this is the first time the web page is requesting a session), the hardware device server presents this new manifest to the user ( 105  in  FIG. 1 ) and requests a grant of the specified access permissions to the web page&#39;s domain  440 . If the requested manifest permission is authenticated by a grant from the user  450 , the hardware device server stores the authenticated manifest  460  in a database and continues to the next step of requesting authentication from the user  435 . If the user does not authenticate the request, the session is ended  490 . If a web page requests access to a new device, the manifest will be authenticated as previously described. Subsequent web page requests  480  for devices in the manifest will not require explicit user action. The manifest validation and request authentication  465  are implemented by the authentication engine ( 128  in  FIG. 1 ) within the hardware device server. 
         [0036]    If the manifest is authorized  450 , the hardware device server sends an authentication request to the requesting domain  435 . Any subsequent request by the requesting web page must be authenticated by comparing the received authentication with a previously stored corresponding authentication  470 . Authentication must be received  445  before a session may be established  455 . 
         [0037]    Before a web page can open a session and initiate hardware requests to the device protocol translator, it must get a new authentication from the hardware device server  435 . Once a session is established  455 , each request must be authenticated  465  by comparing the received authentication with a corresponding authentication stored in memory  470 . If a mapping does not exist between the received authentication and that stored in memory, the request is ignored by the hardware device server  470 ,  485 . Authentication identifies a requestor as being legitimate for every request made while the session is open. If a mapping does exist between the received authentication and that stored in memory  470 , the request is validated and the appropriate hardware device is accessed  475 . If additional requests are made  480 , another authentication  465  is required. Upon receipt of a request to terminate the session  480 , access to the selected hardware device is terminated and the session is ended  485 . 
         [0038]    Since the hardware device server receives requests expressed as HTTP fetches, a natural way for the hardware device server to authenticate requester identification is to inspect the referrer field in the HTTP request. This is a standard HTTP field which indicates the URL, and thus the domain, of the page that generated the request. Unfortunately, a misbehaving browser can subvert this authentication scheme by examining which domains successfully receive hardware data, and then generating fake requests containing these snooped referrer fields. This is essentially a replay attack on a weak authenticator. 
         [0039]    To prevent these replay attacks, the hardware device server grants authentication to each authorized domain to prevent access by unauthorized domains. Before a page in a trusted domain can access the hardware device server, it must send a session establishment message  415  to the hardware device server. The hardware device server examines the referrer of the HTTP message and checks whether the domain has already been granted authentication. If not, the server generates an authentication, stores the mapping between the domain and that authentication, and sends the authentication to the page. Later, when the page sends an actual hardware request to establish a session  455 , it includes the authentication in its HTTP request. If the authentication does not match the mapping found in the hardware device server&#39;s table  470 , the hardware device server ignores the hardware request  485 . If the authentication does match  470 , a session may be established  455  and access to sensors, processors and storage devices is enabled  475 . 
         [0040]    A page sends a session tear down message to the hardware device server  480  when it no longer needs to access hardware attached to the hardware device server. Upon receipt of the tear down message  480 , the server deletes the relevant domain/token mapping and the session ends  485 . The device protocol translator detects when a page is about to unload by registering a handler for the device protocol translator unload event. 
         [0041]    Given the capability scheme, a misbehaving browser that can only spoof referrers cannot fraudulently access hardware, since the browser must also steal another domain&#39;s token or retrieve a new one from the hardware device server. However, nothing prevents a browser from autonomously downloading a new authentication in the background under the guise of an authorized domain, and then using this authentication in its HTTP request to the hardware device server. 
         [0042]    Although the device protocol translator hides the details of the HTTP protocol used for accessing the hardware device server from the web application, the web application can still issue HTTP requests directly to the hardware device server without going through the device protocol translator. This may disrupt logic in the device protocol translator that buffers device output or otherwise attempts to optimize device accesses. If the browser is not compromised, access attempts can be forced to go through the device protocol translator using techniques by splitting the device protocol translator into two pieces, the public component that exports a standardized interface and a private component that is generated by the hardware device server and handles platform-specific optimizations and quirks. For example, if the device protocol translator is implemented as a JavaScript library running inside a web browser, the library can create an HTML frame in the localhost domain that cannot be accessed by the outer application frame that runs in the domain of the application&#39;s originating web server. The private component can be placed in the localhost frame, and the public component can be placed in the enclosing application frame, allowing the two frames to pass messages to each other, but keeping secrets in the localhost frame safe from the untrusted outer frame. The application passes device access requests to the public component; in turn, the public component passes the request to the private component. The private component, which has established a secret with the device server, includes this secret in the forwarded application-generated device request. Thus, the device server can detect that the request originated from the private component, and not from a web application that is trying to directly issue requests without going through the device protocol translator. 
       Hardware Device Server Implementation 
       [0043]    The device protocol translator encodes device requests to the hardware device server as HTTP commands using simple XML strings embedded in HTTP requests. Whereas API calls ( 147  in  FIG. 1 ) constitute the interface between a web application ( 146  in  FIG. 1 ) and the device protocol translator ( 148  in  FIG. 1 ), the HTTP commands ( 154  and  136  in  FIG. 1 ) described herein constitute the interface between the device protocol translator ( 148  in  FIG. 1 ) and the hardware device server ( 156  and  124  in  FIG. 1 ). The API calls shown in Table 1 and illustrated in  FIG. 3  define the interface (see  147  in  FIG. 1 ) between a web application and the device protocol translator (see  146  and  148 , respectively, in  FIG. 1 ). The device protocol translator ( 148  in  FIG. 1 ) transmits commands using HTTP protocol ( 154  and  136  in  FIG. 1 ) to a local hardware device server and remotely connected hardware device server ( 124  and  156  in  FIG. 1 ) for controlling various hardware devices. The commands using HTTP protocol are described below in relation to Table 2. Table 2 is an XML template for accessing the hardware device server. 
         [0044]    An exemplary XML string for a command sent to a hardware device server from a device protocol translator requesting an action and a corresponding result sent from the hardware device server to the device protocol translator that includes audio data generated by a microphone are shown in Table 2. Each request includes a command and a corresponding result. A command may include a security authenticate as described above, an action to be performed such as “record”, a target device such as a “microphone”, and optional device-specific params field such as recording “durationLength” and optional data field to be sent to recipient devices such as memory devices. The optional data field is not used in this example. When a hardware device server responds to a command, the responses arc also encoded as a simple XML string. In response to the exemplary command shown in Table 2, a result may be returned to the device protocol translator from the hardware device server that may include a value field indicating whether the associated command succeeded or failed. The result may also include a data field such as the requested audio data from a microphone. 
         [0000]                                TABLE 2                               &lt;command&gt;                &lt;authenticate&gt;0fdb53ac3d2...&lt;/authenticate&gt;                &lt;action&gt;record&lt;/action&gt;                &lt;device&gt;microphone&lt;/device&gt;                &lt;params&gt;durationLength=10&lt;/params&gt;                &lt;data&gt;&lt;/data&gt;               &lt;/command&gt;               &lt;result&gt;                &lt;value&gt;OK&lt;/value&gt;                &lt;data&gt;bcd32abfe029e...&lt;/data&gt;               &lt;/result&gt;                        
On the client side, the device protocol translator encapsulates the low level decoding routines and other protocol functionality and responds to API calls from a web page with any requested data or status information.
 
         [0045]    Although the subject matter has been described in language specific to structural features and methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.