Multimodal or multi-device configuration

Embodiments of the present invention provide methods, system and machine-readable media for dynamic configuration and/or support for application using multiple modes and/or devices. Generally speaking, dynamic configuration and/or support for an application using multiple modes and/or devices can comprise accessing a multimodal and/or multi-device application via a user agent of a user device. The user agent can determine a plurality of modes or devices supported by the application and request multimodal services based on the modes or devices supported by the application. A multimodal/multi-device infrastructure communicatively coupled with the user agent can be adapted to register components of a multimodal service in a central repository. In response to the request for multimodal services from the user agent of the user device, the multimodal/multi-device infrastructure can locate services supporting the one or more modes or devices based on the registered components of the multimodal service in the central repository.

BACKGROUND OF THE INVENTION

Embodiments of the present invention relate generally to multimodal and/or multiple-device applications and more particularly to providing dynamic configuration and/or support for applications using multiple modes and/or devices.

Multimodal applications are applications that utilize multiple modes of operation or input/output such as presenting a Graphical User Interface (GUI), a voice interface, text, handwriting, and/or other modalities. Multi-device applications are applications that can be executed, simultaneously or not, on multiple devices of different types such as a cell phone, Personal Digital Assistant (PDA), personal computer, etc. Furthermore, an application can combine multiple modes and multiple devices to, for example, present different interfaces to different types of devices, present multiple interfaces, present multiple interfaces of certain types to only particular types of devices, etc.

Generally speaking, the execution model of multimodal and multi-device applications can be described as follows. First, a user interacts with one of the available modalities. For example, the user accesses a web page via his personal computer. Interaction events resulting from the user's interactions with this modality or device are generated by an agent for that mode or device. The interaction events are passed to a synchronization manager that handles a representation of the interaction event and determines the impact of the interaction based on the state of the application and synchronization scheme. The synchronization manager updates its internal representation of the state of the application and sends update events to registered agents or applications for each of the other modes or devices. Any interaction or change resulting from the user's interaction with the first mode or device is then performed by the agent(s) for the other modes or devices based on the update events. Therefore, changes or interactions with the application made in one mode or via one device can be propagated through the synchronization manager to the agents for the other modes or devices to update those modes or devices.

However, such a model assumes that the configuration, i.e., the devices and modalities and the agents for each, have been established and remain static. That is, the synchronization manager is hard-coded to interact with the application and the modes and devices it is able to use as well as the individual agents for each modality or device. There is no way for the manager to dynamically discover and utilize an agent for a particular modality or device based on the application and the modes or devices it may support or utilize. Rather, to add or change modalities, the application, synchronization manager, and agents must be coded or re-coded. Hence, there is a need for methods and systems that allow dynamic configuration and/or support for application using multiple modes and/or devices.

BRIEF SUMMARY OF THE INVENTION

Systems, methods, and machine-readable media are disclosed for dynamic configuration and/or support for applications using multiple modes and/or devices. In one embodiment, a system for providing multimodal configuration and support can comprise a system having stored therein a multimodal application. A user agent can be communicatively coupled with the system. The user agent can be adapted to access the multimodal application, determine a plurality of modes or devices supported by the application, and request multimodal services based on the modes or devices supported by the application. The system can also include a multimodal infrastructure communicatively coupled with the user agent. The multimodal infrastructure can be adapted to register components of a multimodal service in a central repository and, in response to the request for multimodal services from the user agent, locate services supporting the one or more modes or devices based on the registered components of the multimodal service in the central repository.

According to one embodiment, the multimodal infrastructure can be further adapted to launch a multimodal manager in response to the request for multimodal services from the user agent. The user agent can be adapted to initiate a session with the multimodal manager of the multimodal infrastructure and provide application data from the multimodal application to the multimodal manager. The multimodal manager of the multimodal infrastructure, in response to receiving the application data from the user agent, can determine one or more components for supporting execution of the application, initiate the one or more components, and synchronize the one or more components with the application data from the user agent.

According to another embodiment, a method of providing multimodal configuration and support can comprise accessing a multimodal application. A plurality of modes or devices supported by the application can be determined. Determining the plurality of modes or devices supported by the application can be based, for example, on information explicitly provided by the application, by parsing application data and inferring the plurality of modes or devices, and/or based on information obtained from an external source. Multimodal services can be requested based on the modes or devices supported by the application. Information identifying one or more multimodal services related to the modes or devices supported by the application can be read from a central repository. The method can further include initiating a session with a multimodal manager of a multimodal infrastructure and providing application data from the multimodal application to the multimodal manager.

According to yet another embodiment, a method of providing multimodal configuration and support can comprise registering components of a multimodal service in a central repository. A request for multimodal services can be received from a user agent. The request can indicate one of more modes or devices. In response to the request for multimodal services from the user agent, services supporting the one or more modes or devices can be located based on the registered components of the multimodal service in the central repository.

The method can further include launching a multimodal manager in response to the request for multimodal services from the user agent and initiating a session between the multimodal manager and the user agent. According to one embodiment, in response to receiving application data related to a multimodal application from the user agent, one or more components for supporting execution of the application can be determined. The one or more components can be initiated and synchronized with the application data from the user agent.

According to one embodiment, locating services supporting the one or more modes or devices based on the registered components of the multimodal service in the central repository can be based on load balancing between the services. In such a case, the method can further comprise dynamically balancing the load between services. In other cases, locating services supporting the one or more modes or devices based on the registered components of the multimodal service in the central repository can be based on availability of the services.

The method can further comprise activating or deactivating the multimodal manager based on preferences of a user of the user device, application data from the user agent of the user device, a request from the user agent of the user device, etc. The preferences of the user of the user device can comprise server-side preferences, user agent preferences, multimodal manager preferences, etc. According to one embodiment, the method can further comprise ending the session between the multimodal manager and the user agent based on the user agent deactivating itself or deactivating another user agent.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention provide methods, system and machine-readable media for dynamic configuration and/or support for applications using multiple modes and/or devices. Generally speaking, dynamic configuration and/or support for applications using multiple modes and/or devices can comprise accessing a multimodal and/or multi-device application via a user agent. In some cases, the user agent may be located and/or operate on a user device. In other cases, the user agent may be on a server such as a server providing the multimodal and/or multi-device application or another server. Regardless of its location, the user agent can determine a plurality of modes or devices supported by the application and request multimodal services based on the modes or devices supported by the application. A multimodal/multi-device infrastructure can communicatively coupled with the user agent and can be adapted to register components of a multimodal service in a central repository. In response to the request for multimodal services from the user agent, the multimodal/multi-device infrastructure can locate services supporting the one or more modes or devices based on the registered components of the multimodal service in the central repository.

According to one embodiment, the multimodal/multi-device infrastructure can be further adapted to launch a multimodal manager in response to the request for multimodal services from the user agent. The user agent can be adapted to initiate a session with the multimodal manager of the multimodal/multi-device infrastructure and provide application data from the multimodal application to the multimodal manager. The multimodal manager of the multimodal/multi-device infrastructure, in response to receiving the application data from the user agent, can determine one or more components for supporting execution of the application, initiate the one or more components, and synchronize the one or more components with the application data from the user agent.

As used herein the term user agent refers to a logical component that can render presentation data into physical effects that can be perceived and interacted with by the user; and/or that can capture physical actions from the user and interpret them as interactions that can be reflected into presentation updates and/or update a data model. It may provide one or multiple modality and support input only, output only or combined input and output. Multimodal/multi-device configuration refers to a device, system, component, etc. that performs configuration of resources (multimodal/multi-device infrastructure) that supports multimodal and/or multi-device user interaction.

Embodiments of the present invention may be implemented in a wide variety of environments and systems. For example, various embodiments may be presented through an Internet web browser and/or a web service. Other types of client-server, peer-to-peer, or other environments are considered to be equally suitable for implementing embodiments of the present invention. Furthermore, components of the system described herein may be implemented in various manners. For example, while illustrated and discussed herein as being located and/or executed on a user device, it should be understood that a user agent for accessing a multimodal application need not be located on and/or executed by a user device. Rather, this implementation is offered by way of example and not limitation. In other implementations, a user agent for accessing a multimodal and/or multi-device application can reside and/or operate on other components such as the server or other system providing the application. Other variations are also contemplated and considered to be within the scope of the present invention.

FIG. 1is a block diagram illustrating components of an exemplary operating environment in which various embodiments of the present invention may be implemented. The system100can include one or more user computers105,110, which may be used to operate a client, whether a dedicate application, web browser, etc. The user computers105,110can be general purpose personal computers (including, merely by way of example, personal computers and/or laptop computers running various versions of MICROSOFT WINDOWS™ and/or MACINTOSH™ operating systems) and/or workstation computers running any of a variety of commercially-available UNIX™ or UNIX™-like operating systems (including without limitation, the variety of LINUX™ operating systems). These user computers105,110may also have any of a variety of applications, including one or more development systems, database client and/or server applications, and web browser applications. Alternatively, the user computers105,110may be any other electronic device, such as a thin-client computer, Internet-enabled mobile telephone, and/or personal digital assistant, capable of communicating via a network (e.g., the network115described below) and/or displaying and navigating web pages or other types of electronic documents. Although the exemplary system100is shown with two user computers, any number of user computers may be supported.

In some embodiments, the system100may also include a network115. The network may can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially-available protocols, including without limitation TCP/IP, SNA, IPX™, APPLETALK™, and the like. Merely by way of example, the network115maybe a local area network (“LAN”), such as an Ethernet network, a Token-Ring network and/or the like; a wide-area network; a virtual network, including without limitation a virtual private network (“VPN”); the Internet; an intranet; an extranet; a public switched telephone network (“PSTN”); an infra-red network; a wireless network (e.g., a network operating under any of the IEEE 802.11 suite of protocols, the BLUETOOTH™ protocol known in the art, and/or any other wireless protocol); and/or any combination of these and/or other networks.

The system may also include one or more server computers120,125,130which can be general purpose computers and/or specialized server computers (including, merely by way of example, PC servers, UNIX™ servers, mid-range servers, mainframe computers rack-mounted servers, etc.). One or more of the servers (e.g.,130) may be dedicated to running applications, such as a business application, a web server, application server, etc. Such servers may be used to process requests from user computers105,110. The applications can also include any number of applications for controlling access to resources of the servers120,125,130.

In some embodiments, an application server may create web pages dynamically for displaying on an end-user (client) system. The web pages created by the web application server may be forwarded to a user computer105via a web server. Similarly, the web server can receive web page requests and/or input data from a user computer and can forward the web page requests and/or input data to an application and/or a database server. Those skilled in the art will recognize that the functions described with respect to various types of servers may be performed by a single server and/or a plurality of specialized servers, depending on implementation-specific needs and parameters.

The system100may also include one or more databases135. The database(s)135may reside in a variety of locations. By way of example, a database135may reside on a storage medium local to (and/or resident in) one or more of the computers105,110,115,125,130. Alternatively, it may be remote from any or all of the computers105,110,115,125,130, and/or in communication (e.g., via the network120) with one or more of these. In a particular set of embodiments, the database135may reside in a storage-area network (“SAN”) familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to the computers105,110,115,125,130may be stored locally on the respective computer and/or remotely, as appropriate. In one set of embodiments, the database135may be a relational database, such as ORACLE™ 10g, that is adapted to store, update, and retrieve data in response to SQL-formatted commands.

FIG. 2is a block diagram illustrating functional components of a system for providing multimodal and/or multi-device configuration and support according to one embodiment of the present invention. This example represents, conceptually, components that may be implemented in an environment such as described above with reference toFIG. 1or any other suitable environment for providing multimodal and/or multi-device services and/or applications.

In this example, the system200for providing multimodal configuration and support comprises a backend system210, a user device215, and components of a multimodal/multi-device infrastructure205. The user device215can be communicatively coupled with the backend system210and the components of the multimodal/multi-device infrastructure205via a communication network such as described above or any other suitable communications media. While shown here as located on user device215, it should be noted that this first user agent240can be located and/or operated by other components of the system200. For example the user agent240can be located and/or operated on the backend system210or another server (not shown here). Other variations are also contemplated and considered to be within the scope of the present invention.

The backend system210, such a web server or other system, can have stored therein a multimodal and/or multi-device application245providing support for multiple modes of operation or input/output. For example, the application245may support multiple interface modes such as a Graphical User Interface (GUI), a voice interface, text, handwriting, and/or other modalities. Additionally, the application245may provide support for execution on multiple devices of different types such as a cell phone, Personal Digital Assistant (PDA), personal computer, etc.

The user agent240may send a request Universal Resource Identifier (URI)250to the backend system210which in turn can provide255the application245or access to the application245for execution on the backend system210or user device215. Once the application245or access to the application245has been received, the user agent240can determine that a plurality of modes or devices is needed to support the application245. The user agent240can determine the plurality of modes or devices to support the application245based on information explicitly provided by the application245, by parsing application data and inferring the plurality of modes or devices, by reading information from a source separate from the user agent240and the multimodal/multi-device infrastructure205such as a Universal Description, Discovery, and Integration (UDDI) repository associated with a multimodal service or other catalog of modes or devices, or by other methods. For example, the user agent can determine the plurality of modes or devices supported by the application based on information obtained from a Uniform Resource Locator (URL) for the application, information obtained from an extension of a file returned by the application, information obtained from a content of a file, etc. The user agent240can then request260multimodal and/or multi-device capabilities from a set of multimodal/multi-device infrastructure components205based on the modes or devices supported by the application245to have the multimodal / multi-device infrastructure configured with the right features based on the needs of the application.

The multimodal/multi-device infrastructure components205can be a registered in a central repository or registrar220. For example, the registrar220can be an addressable component where the multimodal and multi-device components of205, such as agents235and engines230, that can support setup of a multimodal/multi-device infrastructure can register their address or other identifying information through which they can be located and/or accessed. In response to the request260for multimodal services from the user agent240, the components of the multimodal/multi-device infrastructure205supporting the one or more modes or devices supported by the application245can be located and configured. Locating these components can be based on the registered components of the multimodal service in the registrar220. Alternatively, the registrar220may be made available to or accessible by the user agent240which can look-up or otherwise discover the registered components that are already configured. In such a case, the user agent only establishes a session with a block of components already correctly configured.

Once the registered components have been identified, a multimodal manager225can be launched. Again, there are multiple options or variations. For example, the components may be launched already, i.e., already executing, by the multimodal and/or multi-device infrastructure and registered as being available. Alternatively, the capabilities and/or locations of the components may be registered but not yet launched. In such a case, when the request is received, the corresponding component can be identified and launched. In both cases, registration or invocation may be routed to an available component using load balancing techniques.

The user agent240of the user device215can initiate265a session with the multimodal manager225and provide application data from the application245to the multimodal manager225. Additionally, the multimodal manager225, in response to receiving the application data from the user agent240of the user device215, can determine one or more components for supporting execution of the application such as user agent235and processing engine230. These components can be setup following the same flow and approach as used to setup the multimodal manager225but now the multimodal manager225can be used to setup these other components. The user agent235of the multimodal/multi-device infrastructure205provides a copy or other rendition of the application245or application data that is available to the multimodal/multi-device infrastructure components205. Processing engine230is the logical component that transforms232user physical I/O to a format useful for the user agent235, e.g. voice to text, ink to text, display, keyboard, T9, TTS, etc. Importantly, while only one agent235and processing engine230is shown here, any number of agents and processing engines may be used depending upon the implementation and the modes and/or devices being supported.

Once the multimodal/multi-device manager225determines which agent(s)235and processing engine(s)230are to be used to support execution of the application245, the multimodal/multi-device manager225can initiate275the one or more components235and230, and synchronize270them with the application data from the user agent240of the user device215. The multimodal manager225then coordinates data and synchronizes execution flow to and from the user agent240and other components of the system200during execution of the application245.

According to one embodiment, the multimodal/multi-device infrastructure205can be further adapted to activate or deactivate the multimodal manager225. Multimodal activation refers to the act of making a user agent235available to the multimodal manager225. Multimodal de-activation refers to the act of making a user agent235unavailable to the multimodal manager225. Activation and/or deactivation can be based on preferences of a user of the user device215. The preferences of the user of the user device215can comprise server-side preferences, user agent preferences, multimodal manager preferences, etc. Alternatively or additionally, activation and/or deactivation can be based on application data from the user agent240of the user device215or based on a request from the user agent240of the user device215. Furthermore, the multimodal/multi-device infrastructure205can be further adapted to end the session between the multimodal manager225and the user agent240of the user device215based on the user agent240of the user device215deactivating itself or based on the user agent240of the user device215deactivating another user agent.

FIG. 3illustrates an exemplary computer system300, in which various embodiments of the present invention may be implemented. The system300may be used to implement any of the computer systems described above such as the backend system210, user device215, multimodal/multi-device infrastructure205, etc.. The computer system300is shown comprising hardware elements that may be electrically coupled via a bus355. The hardware elements may include one or more central processing units (CPUs)305, one or more input devices310(e.g., a mouse, a keyboard, etc.), and one or more output devices315(e.g., a display device, a printer, etc.). The computer system300may also include one or more storage device320. By way of example, storage device(s)320may be disk drives, optical storage devices, solid-state storage device such as a random access memory (“RAM”) and/or a read-only memory (“ROM”), which can be programmable, flash-updateable and/or the like.

The computer system300may additionally include a computer-readable storage media reader325a, a communications system330(e.g., a modem, a network card (wireless or wired), an infra-red communication device, etc.), and working memory340, which may include RAM and ROM devices as described above. In some embodiments, the computer system300may also include a processing acceleration unit335, which can include a DSP, a special-purpose processor and/or the like.

The computer-readable storage media reader325acan further be connected to a computer-readable storage medium325b, together (and, optionally, in combination with storage device(s)320) comprehensively representing remote, local, fixed, and/or removable storage devices plus storage media for temporarily and/or more permanently containing computer-readable information. The communications system330may permit data to be exchanged with the network320and/or any other computer described above with respect to the system300.

The computer system300may also comprise software elements, shown as being currently located within a working memory340, including an operating system345and/or other code350, such as an application program (which may be a client application, web browser, mid-tier application, RDBMS, etc.). It should be appreciated that alternate embodiments of a computer system300may have numerous variations from that described above. For example, customized hardware might also be used and/or particular elements might be implemented in hardware, software (including portable software, such as applets), or both. Further, connection to other computing devices such as network input/output devices may be employed. Software of computer system300may include code350for implementing processes for dynamically providing multimodal and/or multi-device configuration and support.

Generally speaking, dynamic configuration and/or support for an application using multiple modes and/or devices can comprise accessing a multimodal and/or multi-device application via a user agent of a user device. The user agent can determine a plurality of modes or devices supported by the application and request multimodal services based on the modes or devices supported by the application. A multimodal/multi-device infrastructure communicatively coupled with the user agent can be adapted to register components of a multimodal service in a central repository. In response to the request for multimodal services from the user agent of the user device, the multimodal/multi-device infrastructure can locate services supporting the one or more modes or devices based on the registered components of the multimodal service in the central repository.

According to one embodiment, the multimodal/multi-device infrastructure can be further adapted to launch a multimodal manager in response to the request for multimodal services from the user agent of the user device. The user agent of the user device can be adapted to initiate a session with the multimodal manager of the multimodal/multi-device infrastructure and provide application data from the multimodal application to the multimodal manager. The multimodal manager of the multimodal/multi-device infrastructure, in response to receiving the application data from the user agent of the user device, can determine one or more components for supporting execution of the application, initiate the one or more components, and synchronize the one or more components with the application data from the user agent of the user device.

FIG. 4is a flowchart illustrating, at a high level, a process for providing multimodal and/or multi-device configuration and support according to one embodiment of the present invention. In this example, a multimodal and/or multi-device service is first deployed405by a service provider deployment team. That is, the service provider makes available for use a multimodal/multi-devices infrastructure components205. Alternatively, the application and the multimodal/multi-device infrastructure components may be implemented on the same device. Also, the addresses and capabilities of the different components that the service provider make available to support its multimodal and multi-device services are registered410such as in registrar220or a central repository within or separate from the multimodal/multi-devices infrastructure205.

An application developer writes and deploys415a multimodal and/or multi-device application245. This application is made available on a backend server210such as a web application server, through, for example, a URI. A user accesses420the multimodal and/or multi-device application245by entering or clicking/selecting a URI on one of its user agents240. The web application server serves the multimodal and/or multi-device application245to the user's device215.

The user agent240that initiated the request determines425the other components and configurations supported by the application245. This can be achieved in multiple ways. For example, the application data could contain information explicitly provided by the application developer including, possibly, details on the capabilities and/or configurations of the other components. Alternatively or additionally, the user agent240may be able to extract details on what other modalities, components and their capabilities to involve by parsing the received application data. Alternatively or additionally, the user agent240may be able to extract that information from another source such as, for example, a UDDI repository associated to the service.

The user agent240then discovers430from the multimodal registrar220the available components. This can be accomplished by the user agent240querying or otherwise accessing the information in the registrar220. Alternatively, it may involve the user agent240requesting the multimodal/multi-device infrastructure to look up or otherwise access the information in the registrar220and provide it to the user agent240. In some implementations, discovery430may involve a negotiation based on possible configurations allowed by the application, the user, or the service provider. In such a case, discover can involve exchanges between the party of possible and desired configurations until a selected one is accepted by all parties. Additionally, discovery430may involve, for example, load balancing of the different components. That is, the discovered resource or service can be based on load balanced, availability, etc.

Upon discovering430a candidate multimodal manager225, the user agent240initiates435a multimodal session with the assigned multimodal manager225. After the multimodal session is initiated435, the user agent240initiates the multimodal service by providing440the application data and its current state to the multimodal manager225. The multimodal manager225uses this data to initiate and synchronize445other components of the system such as user agent(s)235and processing engine(s)230used to support the other modes and/or devices. Additionally or alternatively, the multimodal manager can determine from the application data that more capabilities are supported by the application. In such a case, the multimodal manager can discover components to support these capabilities, setup a session with these components, and then pass data to these components, i.e., repeating the steps above to link the other user agents to the multimodal manager.

At this point, the user agent240is part of a multimodal or multi-device infrastructure configuration, provisioned with the multimodal or multi-device application. It is ready for presentation and interaction with the user. During the multimodal session, according to one embodiment, the different modalities may be activated or de-activated by the multimodal manager. This may be based on preferences of the user made available to the multimodal manager, the multimodal or multi-device application data and state, requests to activate or deactivate sent from a user agent about itself or about another user agent or modality.

Preferences of the users, in terms of what modality to use etc, can be server-side preferences that determine, for the page served to the user, what modalities/user agents should be used and for what purposes. The preferences can also be user agent preferences that may result in the user agent of the user device requesting particular modalities or user agents to be activated by the multimodal manager at some moment or for some purpose. This is can be done by a special user interface that lets the user select settings that are then passed to the multimodal manager and applied as needed. Furthermore, the preferences can be multimodal manager preferences that allow the multimodal manager to activate or deactivate user agents or modalities as needed based on the moment or application/interaction state.

FIG. 5is a flowchart illustrating a process for a user agent to obtain multimodal and/or multi-device configuration and support according to one embodiment of the present invention. In this example, the process begins with accessing505a multimodal application. As noted above, a user can accesses a multimodal and/or multi-device application by entering or clicking/selecting a URI on one of its user agents. In turn, the server provides the application or access to the application to the user agent of the user's device.

A plurality of modes or devices supported by the application can be determined510. Determining510the plurality of modes or devices supported by the application can be based on information explicitly provided by the application, information obtained by parsing application data and inferring the plurality of modes or devices, or information obtained from a separate repository of information related to a multimodal and/or multi-device service. For example, the application data could contain information explicitly provided by the application developer including, possibly, details on the capabilities and/or configurations of the other components. Alternatively or additionally, the user agent may be able to extract details on what other modalities, components and their capabilities to involve by parsing the received application data. Alternatively or additionally, the user agent may be able to extract that information from another source such as, for example, a UDDI repository associated to the service.

Information identifying one or more multimodal services related to the modes or devices used by the application can be read520from a central repository. This can be accomplished by querying or otherwise accessing the information in the registrar. Alternatively, it may involve requesting the multimodal/multi-device infrastructure to look up or otherwise access the information in the registrar220.

Once the multimodal services related to the modes or devices supported by the application are found, a session can be initiated525with a multimodal manager of the multimodal infrastructure and application data from the multimodal application can be provided530to the multimodal manager.

FIG. 6is a flowchart illustrating a process for a set of multimodal and/or multi-device infrastructure components to provide multimodal and/or multi-device configuration and support according to one embodiment of the present invention. In this example, the process begins with registering600components of a multimodal service in a central repository. That is, the addresses and capabilities of the different components that the service provider make available to support its multimodal and multi-device services can be registered600such as in a registrar or a central repository.

A request for multimodal services can be received605from a user agent indicating one of more modes or devices supported by a multimodal application. In response to a request for multimodal services from a user agent of a user device, components supporting the one or more modes or devices can be located610based on the registered components of the multimodal service in the central repository. Locating these components can be based on the registered components of the multimodal service in the registrar220. Alternatively, the registrar220may be made available to or accessible by the user agent240which can look-up or otherwise discover the registered components that are already configured. In such a case, the user agent only establishes a session with a block of components already correctly configured.

A multimodal manager can be launched615in response to the request for multimodal services from the user agent of the user device and a session can be initiated620between the multimodal manager and the user agent of the user device. In response to receiving625application data related to a multimodal application from the user agent, one or more components for supporting execution of the application can be determined630by the multimodal manager. The one or more components can be initiated635and synchronized640with the application data from the user agent of the user device by the multimodal manager. That is, once the multimodal manager determines630which components are to be used to support execution of the application, the multimodal manager can initiate the components and synchronize them with the application data from the user agent. The multimodal manager then coordinates data and synchronizes execution flow to and from the user agent and other components during execution of the application.