Patent Description:
The market for user terminals is driven by the desire to provide users with increasingly advanced communication capabilities and other operational features within the constraints of a portable handheld form factor. The development requirements for user terminal are increasingly complex as designers seek to integrate a greater variety of user interfaces and advanced operational features within the portable handheld form factor. Advancements in operational features have required more highly integrated and faster processing circuits with greater circuit densities, which becomes more difficult under constraints on costs and power consumption.

This all-inclusive feature-rich approach for user terminal development does not satisfy all of the myriad of differing desires held by consumers. Moreover, the always-connected expectations of today's society obligates users to vigilantly keep their user terminals within reach or risk being unable to timely receive or initiate communication services.

Some approaches have been proposed which seek to replace the conventional user terminal with other types of user interfaces which can interface with server based services. For example, Sun Microsystems has proposed thin client stateless terminals through any one of which a user can access a user application executed by a server. The terminals include an integrated smart card reader which operates to authenticate users and enable a user to remove a smart card (logging the user off one terminal) to pause a session and subsequently insert the smart card into any other terminal (logging the user on the other terminal) to promptly resume the session from where the user left off.

<CIT> discloses a user terminal emulation server maintaining a database identifying network addresses, UI capabilities, and communication protocols, of I/<NUM> user devices. Communication sessions are established between a user terminal emulation application and a network entity and I/<NUM> user devices proximately located to a user and provide a combined I/<NUM> user interface. Delay profiles are determined between the application and the I/<NUM> user devices. A downlink flow from the network entity is split into a plurality of downlink flow components assigned to the I/<NUM> user devices. For each of the downlink flow components, the server formats the component for transmission to the assigned I/<NUM> user device, initiates transmission of the formatted downlink flow component to the assigned I/<NUM> user device, and controls timing for when the formatted downlink flow component is transmitted to the assigned I/<NUM> user device based on the delay profile associated with the assigned I/<NUM> user device.

Objects of the invention are a user terminal emulation server, a corresponding method and a corresponding computer program product as claimed in the appended independent claims. Preferred embodiments are covered by the appended dependent claims.

Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying drawings. In the drawings:.

Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of various present inventive concepts to those skilled in the art.

Known prior-existing solutions do not acceptably support cloud-based software-defined smartphones or similar, in a way that can provide sufficiently seamless handoff of communication services between individual input and/or output ("I/O") devices ("IODs") or a set of I/O user devices which are proximately located to a user. When I/O user devices can be made available to a user, an unacceptable delay can occur between detecting presence of the user and establishing a communication service through the I/O user devices. Moreover, handoff between I/O user devices may not be available or may be supported in a discontinuous manner that results in dropped communication service or interruption of the communication service as a user moves.

Various embodiments disclosed herein are directed to providing a user terminal emulation server, e.g., as a cloud server resource, which emulates a user terminal using individual I/O user devices or a set of I/O user devices that are proximately located to a user and have requisite user interface (UI) capabilities to provide a user with the ability to obtain a communication service provided by a network entity. These embodiments can predict that an I/O user device or a set of I/O user devices will become proximately located to the user, and send a signal to prepare the I/O user device or the group of I/O user devices for use in a communication service in advance of the user becoming proximately located to the I/O user device. Upon arrival of the user, the I/O user device or the group of I/O user devices can be configured to already be used to provide the communication service of the network entity for the user or can more instantaneously be made available for providing the communication service.

A user can receive and initiate communication services without the necessity of a traditional all-inclusive feature-rich user terminal, i.e., a conventional smartphone, mobile phone, tablet computer, etc. The user terminal emulation server can adaptively combine the available UI capabilities of I/O user devices that are predicted to become proximate to a user to support provision of a communication service to the user before the user becomes proximately located to the I/O user devices. Dynamic allocation of I/O user interface capabilities of I/O user devices whenever and wherever they are predicted to become proximately located to a user enables efficient and flexible use of existing hardware, such as televisions, conference phones, laptops, surveillance cameras, connected household appliances, connected cars, etc., that is capable of providing some UI functionality to user during a communication service. The user thereby has reduced need or no need to carry an expensive and all-inclusive user terminal, e.g. smart phone, that includes all necessary UI capabilities, display device, keyboard, speakers, etc. The user may instead carry a hardware device which operates to identify the user, referred to as a "UserTag", over a wireless communication interface, such as a near field communication (NFC) interface, to one or more of the I/O user devices.

Various embodiments disclosed herein may disrupt the traditional handset-centric mobile communication industry as the features and capabilities of what forms a user terminal are not constrained to the domain of mobile phone manufacturers. A user terminal emulation server can operate to provide a user terminal, which can also be referred to as a "SoftUE" or "soft phone", by means of a user terminal emulation application that is run by the user terminal emulation server.

<FIG> illustrates a system with a user terminal emulation server <NUM> that operationally uses individual I/O user devices <NUM> or integrates sets of I/O user devices <NUM> which are proximately located to users to logically emulate user terminals providing communication services through network entities <NUM> in accordance with some embodiments of the present disclosure. <FIG> is a flowchart of operations that may be performed by the user terminal emulation server <NUM> to signal an I/O user device <NUM> to prepare for using an I/O user interface of the I/O user device <NUM> to provide a first communication service for a user through the network entity <NUM>, in accordance with some embodiments of the present disclosure.

Referring to <FIG> and <FIG>, the user terminal emulation server <NUM> may be a cloud resource that is remotely networked to the I/O user devices <NUM>, or may be more proximately located on a shared network with the I/O user devices <NUM>, e.g., in an edge cloud. The user terminal emulation server <NUM> is configured to register <NUM> a user with a network entity <NUM> that provides communication services for the user, such as by registering the user with a Voice Over Internet Protocol (VoIP) service, videoconference service (e.g., Teams, Skype, etc.), media content streaming service (e.g., Netflix, HBO, Hulu, etc.), gaming service (e.g., online game application), etc. The communication services can include, without limitation, Voice over IP and/or Video Over IP calling (e.g., Skype, Microsoft Teams, etc.), streaming media (e.g., Netflix, HBO, Hulu, etc.), online gaming, etc. The user terminal emulation server <NUM> is further configured to predict <NUM> that one or more of the I/O user devices <NUM> will become proximately located to the user and, based on the prediction that the I/O user device <NUM> will become proximately located to the user, determine <NUM> that the I/O user device <NUM> has an I/O user interface that satisfies a capability criterion to enable the user to use a first one of the communication services provided by the network entity <NUM>. Moreover, based on the satisfaction of the capability criterion and before the user is predicted to become proximately located to the I/O user device <NUM>, the user terminal emulation server <NUM> is further configured to signal <NUM> the I/O user device <NUM> to prepare for using the I/O user interface to provide the first communication service for the user through the network entity <NUM>.

Users may carry a hardware tag, a. "UserTag" in <FIG>, which is capable of transmitting a user identifier through a communications interface, such as a near-field communications interface (e.g., Bluetooth, Bluetooth low energy (BLE), near field communication (NFC), RFID, etc., or combinations thereof), for receipt by one or more of the I/O user devices <NUM> which are proximately located to the user. One type of UserTag can be a simple stand-alone electronic device having limited capability for transmitting an identifier through a near-field communications interface. Another type of UserTag can be a smartphone or smartwatch having cellular connectivity that transmits a cellular identity (e.g., from a SIM card) or an application identity through a cellular interface or a near-field communications interface.

In one example embodiment which will be explained in further detail below, the user terminal emulation server <NUM> can operate to provide seamless handoff of a videoconference service is presently being provided to a first user through the network entity <NUM> from a first I/O user device <NUM> to a second I/O user device <NUM> as the first user walks down a building corridor. For example, the first I/O user device <NUM> may be located at an entrance to the building corridor and the second I/O user device <NUM> may be located further down the building corridor. The user terminal emulation server <NUM> determines that the first user is presently proximately located to the first I/O user device <NUM> based on receiving a report from the first I/O user device <NUM> which provides the user identifier of the sensed UserTag#<NUM> associated with the first user. Responsive to an incoming request from the network entity <NUM> or from the first user to establish a videoconference service, the user terminal emulation server <NUM> uses the I/O user interface of the first I/O user device <NUM> to provide the videoconference service.

The user terminal emulation server <NUM> predicts the second I/O user device <NUM> will become proximately located to the user, such as based on being configured to know relative locations of the first and second I/O user devices <NUM> and/or from learning that users become proximately located to the second I/O user device <NUM> within a threshold amount of time after being proximately located to the first I/O user device <NUM>. Alternatively or additionally, the user terminal emulation server <NUM> may predict the second I/O user device <NUM> will become proximately located to the user based on being informed of a location of the user by a satellite-based positioning system (e.g., GNSS, GPS, etc.), cellular-based positioning system, etc..

The user terminal emulation server <NUM> can determine that the second I/O user device <NUM> has an I/O user interface that satisfies a capability criterion to enable the first user to use the videoconference service provided by the network entity <NUM>, and can signal the second I/O user device <NUM> to prepared for using the I/O user interface to provide the videoconference service for the first user. For example, as will be explained further detail below, the user terminal emulation server <NUM> may initiate routing of videoconference service traffic to the second I/O user device <NUM> before the first user is predicted to become proximately located to the second I/O user device <NUM>. The second I/O user device <NUM> may perform one of the following operations with the videoconference service traffic: initiate playout of the videoconference service traffic through the I/O user interface; buffer the videoconference service traffic for subsequent playout through the I/O user interface responsive to the user becoming proximately located to the second I/O user device <NUM>; and discard the videoconference service traffic until the user becomes proximately located to the second I/O user device <NUM>.

The identity of a user, e.g., the first user, may alternatively or additionally be operationally determined by biometrics operations performed by, e.g., one or more of the I/O user devices <NUM>. The biometrics operations may include, without limitation, one or more of voice recognition, image/face recognition, eye recognition, fingerprint recognition, or a combination thereof. The user identity may be determined based on credential provided by the user when, e.g., logging into an application or account. The user identity may be determined from information provided by a cell phone, such as from the subscription SIM, and the user's proximity of the cell phone to one or more of the I/O user devices <NUM> can be determined using the phone's near-field communications (NFC) capability.

A user identifier, a UserTag identifier, and a user terminal application can be logically associated with each other in a database repository <NUM> during a user registration process or as part of another setup process. For example, during a user registration process a user may obtain an account login identifier (serving as the user identifier) that is registered in the database repository <NUM> as being associated with a UserTag identifier for a physical UserTag that has been provided to (e.g., purchased by) the user and being associated with a user terminal application that emulates a user terminal having defined capabilities (e.g., a cell phone providing cellular and over-the-top voice-over-IP communication services).

Operations that can be performed by the user terminal emulation server <NUM> to provide communication services to users are described below with further reference to <FIG>. Referring to <FIG> and <FIG>, the user terminal emulation server <NUM> maintains <NUM> a database repository <NUM> that identifies network addresses of I/O user devices <NUM> and further identifies UI capabilities of the I/O user devices <NUM>, based on content of received registration messages. The capabilities of the I/O user devices <NUM> may be logically arranged in the repository <NUM> based on the type of UI capability provided, e.g., display device, microphone, headphone or ear speaker (e.g., Bluetooth ear speaker), loudspeaker, keyboard, and may be further arranged based on quality of service characteristics provided by the UI capability (e.g., mono or stereo speaker, speaker maximum volume, display screen size, display maximum resolution, etc.).

An I/O user device <NUM> may communicate a registration message, containing its network address and UI capabilities, to the user terminal emulation server <NUM> responsive to an initial set up operation, to being connected to a new communication network, and/or responsive to another defined event for triggering generation of a registration message. The registration messages may include a geographic location of the I/O user device <NUM>, which can be stored in the database repository <NUM>. The I/O user devices <NUM> may communicate with the server <NUM> via a data network (e.g., Internet and/or private network) using a WiFi transceiver, Bluetooth transceiver, cellular transceiver, light communication transceiver (LiFi), and/or another RF or light communications transceiver.

The user terminal emulation server <NUM> registers (e.g., <NUM> in <FIG>) a user with the network entity <NUM> providing communication services, such as by registering <NUM> an identity of a user with the network entity <NUM> and may further register a network address of a user terminal emulation application <NUM> with the network entity <NUM>. The network entity <NUM> provides a communication service function <NUM> which may, for example, correspond to an over-the-top Voice Over Internet Protocol (VoIP) service, videoconference service (e.g., Teams, Skype, etc.), media content streaming service (e.g., Netflix, HBO, Hulu, etc.), gaming service (e.g., online game application), Internet browser service, a cellular communication service, etc. The user terminal emulation application <NUM> is executed by the user terminal emulation server <NUM>. A user terminal emulation application <NUM> may run one or more applications that are normally run by a smart phone, such as a VoIP service application, a video conference service application, media content streaming application, online game application, Internet browser application, etc..

As illustrated in <FIG>, in some embodiments a different instantiation of the user terminal emulation application <NUM> is hosted by the server <NUM> for each user who is to be provided communication services (i.e., illustrated user terminal emulation applications #<NUM>-#N corresponding to users <NUM>-N). The user terminal emulation application <NUM> may perform registration of the user with the network entity <NUM> and setup of a communication service with a user responsive to communication requests in accordance with the operations of <FIG>.

When the communication service function <NUM> of the network entity <NUM> is a VoIP service, the operation to register <NUM> the network address of the user terminal emulation application and the identity of the user with the network entity <NUM> can include registering the network address of the user terminal emulation application <NUM> and the identity of the user with a network server of a VoIP communication service provider.

When the communication service function <NUM> of the network entity <NUM> is a cellular communication service, the operation to register <NUM> the network address of the user terminal emulation application and the identity of the user with the network entity <NUM> can include registering the network address of the user terminal emulation application <NUM> and the identity of the user with a Home Subscriber Server (HSS) or other network node of a core network operated by a cellular communication service provider.

The user terminal emulation server <NUM> may receive the registration messages from the I/O user devices <NUM> using the Session Initiation Protocol (SIP)/Session Description Protocol (SDP), where each of the registration messages identifies the network address and the UI capability of one of the I/O user devices <NUM>. The communication request may be received from the network entity <NUM> using the SIP/SDP, and the operation to provide communication sessions between the user terminal emulation application <NUM> and each of the I/O user devices <NUM> in the set, and between the user terminal emulation application <NUM> and the requesting user terminal may be performing using the SIP/SDP.

A registration message from an I/O user device <NUM> can include, for example, an IP address and port number, MAC address, fully qualified domain name (FQDN), and/or another network address, and further include information identifying the UI capability of the I/O user device <NUM>. The I/O user device <NUM> may respond to becoming powered-on by communicating the registration message to the user terminal emulation server <NUM>.

The user terminal emulation server <NUM> receives <NUM> a communication request from the network entity <NUM> for establishing a communication service between the user and a requesting user terminal, e.g., a cellular phone, computer with Skype application, etc. Responsive to the communication request, the user terminal emulation server <NUM> identifies <NUM> a set of I/O user devices among the I/O user devices <NUM> identified by the repository <NUM> that are determined to be proximately located to a location of the user or are predicted to become proximately located to the user. The user terminal emulation server <NUM> further determines, based on the UI capabilities identified by the repository <NUM> for the set of I/O user devices and based on content of the communication request, that the set of I/O user devices satisfies a combined capability criterion for being combinable to provide a combined I/O user interface for the user to interface with the user terminal emulation application <NUM> to provide the communication service.

Based on determining that the set of I/O user devices satisfies the combined capability criterion, the user terminal emulation server <NUM> provides <NUM> communication sessions between the user terminal emulation application <NUM> and the I/O user devices in the set and between the user terminal emulation application <NUM> and the requesting user terminal via the network entity <NUM>. The communication request that is received <NUM> by the user terminal emulation application <NUM> may contain an indication of a minimum UI capability that must be provided to the user during the communication service, such as: speaker only; combination of speaker and microphone; display only; combination of display device, speaker, and microphone; etc. The combined capability criterion which is used by the server <NUM> to determine whether a communication service can be provided and by which set of I/O user devices, may thereby be defined based on the minimum UI capability that is indicated by the communication request.

The user terminal emulation server <NUM> then routes <NUM> communication traffic that is received from at least one of the I/O user devices in the set toward the requesting user terminal via the network entity <NUM>. For each data type that is received as communication traffic from the requesting user terminal, the user terminal emulation server <NUM> selects one of the I/O user devices from among the set of I/O user devices based on matching characteristics of the data type to the UI capabilities identified by the repository <NUM> for the one of the I/O user devices, and then routes the data of the data type toward the network address of the selected one of the I/O user devices.

As will be explained in further detail below, the server <NUM> may also combine <NUM> data streams are received from the I/O user devices in the set, and route the combined data streams towards the requesting user terminal, e.g., via the network entity <NUM>.

The user terminal emulation server <NUM> (e.g., the application <NUM> or an I/O user device handler described below) may be responsible for tracking which I/O user devices are presently proximately located to a present location of the user and predicting which other I/O user devices will become proximately located to the user. <FIG> is a flowchart of corresponding operations. The server <NUM> can receive <NUM> presence reports from individual ones of the I/O user devices containing their network address and an identifier of a user who is determined by the I/O user device to be proximately located. For example, an I/O user device may read a hardware tag, also referred to herein as a "UserTag", through a NFC communication interface, may sense biometric information from the user, identify the user through facial recognition performed on a video stream from a camera, and/or may perform other operations to detect presence of a user and to identify the user. Responsive to the presence reports, the server <NUM> updates <NUM> the repository <NUM> to indicate which user identifiers are proximately located to which of the I/O user devices.

With further reference to the example system of <FIG>, a set of I/O user devices <NUM> has been determined by the instantiated user terminal emulation application #<NUM> to be proximately located to a location of a first user carrying UserTag#<NUM>, and to further have UI capabilities that are combinable to satisfy the combined capability criterion for providing a combined I/O user interface for the first user to use during a requested communication service. Application #<NUM> responsively uses that set of I/O user devices <NUM> to provide a combined I/O user interface for use by the first user during a communication service via network entity <NUM> between the first user and another user terminal.

Similarly, another set of I/O user devices <NUM> has been determined by the instantiated user terminal emulation application #<NUM> to be proximately located to a location of a second user carrying UserTag#<NUM>, and to further have UI capabilities that are combinable to satisfy the combined capability criterion for providing a combined I/O user interface for the second user to use during a requested communication service. Application #<NUM> responsively uses that set of I/O user devices <NUM> to provide a combined I/O user interface for use by the second user during a communication service via network entity <NUM> between the second user and yet another user terminal.

<FIG> also illustrates that another set of I/O user devices <NUM> is not proximately located to either UserTag#<NUM> or UserTag#<NUM>. This other set of I/O user devices <NUM> may be predicted to become proximately located to the first or second user, e.g., within a threshold time, and therefore available for use by the user terminal emulation server <NUM> for providing the communication service to that user in the future. In accordance with some embodiments, the user terminal emulation application <NUM> (e.g., user terminal emulation application #<NUM> or #<NUM>) and signaled the other set of I/O user devices <NUM> prepare them to use their respective I/O user interfaces provide the communication service for that user, such as will be described further below.

As explained above, the communication request which is requesting the establishment of a communication service with an identified user may be initiated by the network entity <NUM> using the identity of the user, and maybe also using the network address of the user terminal emulation application, which was earlier registered with the network entity <NUM>. However, the communication request may additionally or alternatively be generated by one of the I/O user devices <NUM> responsive to a command received from a proximately located user. For example, a user may operate a user interface provided by one of the I/O user devices <NUM> to initiate a combined audio and video call with another user. The user terminal emulation server <NUM> receives the communication request along with the identity of the user, which may be sensed via the UserTag. The application <NUM> performs the identifying <NUM>, providing <NUM>, routing <NUM>, selecting <NUM>, and combining <NUM> operations described above for <FIG> to set up and operate a communication service between the user and the other user via the network entity <NUM>.

Further example systems and related operations will now be described to further illustrate how I/O user devices having different UI capabilities can be operationally combined to provide a combined UI that can be used by a user to satisfy the communication requirements of a communication service.

Further illustrative operations are described regarding an example embodiment in which a speaker device is one of the I/O user devices <NUM> in the set capable of playing a received audio stream and a microphone device is another one of the I/O user devices <NUM> in the set capable of sensing audio to output a microphone stream. Operations by the user terminal emulation server <NUM> (e.g., by one of the user terminal emulation applications) include updating the repository <NUM> based on content of registration messages from the speaker device and the microphone device to identify network addresses of the speaker device and the microphone device, and to identify UI capabilities of the speaker device as having a speaker capability and the microphone device as having a microphone capability. The speaker UI capabilities may identify a number of speakers provided, sound loudness capability, and/or other operational characteristics. The microphone UI capabilities may identify a number of microphones provided, sensitivity the microphones, and/or other operational characteristics. The speaker device and the microphone device are each identified as belonging to the set of I/O user devices that are determined to be proximately located to or are predicted to become proximately located to the location of the user (e.g., UserTag#<NUM>) and are further determined, based on the UI capabilities identified by the repository <NUM>, to satisfy the combined capability criterion for being combinable to provide a combined I/O UI for the user to interface with the user terminal emulation application <NUM> to provide the communication service. Based on determining that the speaker device and the microphone device satisfy the combined capability criterion, further operations are performed to route a microphone stream received from the microphone device toward the requesting user terminal (e.g., via network entity <NUM>). When an audio stream is received as communication traffic from the requesting user terminal the operations select the speaker device based on matching an audio characteristic of the audio stream to the speaker capability identified by the repository for the speaker device, and then route the audio stream toward the network address of the speaker device.

The example embodiment may include, when a display device is one of the I/O user devices in the set capable of displaying a received video stream, the operations to update the repository <NUM> based on content of registration messages to identify network addresses of the display device, and to identify UI capabilities of the display device as having a display capability. The display UI capabilities may identify a screen display size, aspect ratio, pixel resolution, video frame rates supported, whether display device supports shared user support via split screen configuration, and/or other operational characteristics. The display device is also identified as among the set of I/O user devices that is determined to be proximately located to or predicted to become proximately located to the location of the user and are further determined, based on the UI capabilities identified by the repository <NUM>, to satisfy the combined capability criterion for being combinable to provide the combined I/O UI for the user to interface with the user terminal emulation application <NUM> to provide the communication service. Based on determining that the speaker device, the display device, and the microphone device satisfy the combined capability criterion, further operations respond to receipt of video stream as communication traffic from the requesting user terminal by selecting the display device based on matching a video characteristic of the video stream to the display capability identified by the repository <NUM> for the display device, and then routing the video stream toward the network address of the display device.

In the example embodiment the operations for routing the audio stream and the video stream toward the network addresses of the speaker device and the display device, respectively, may include when audio data and video data are received within a same stream from the requesting user terminal through a first communication session: separating the audio data from the video data; routing the audio data toward the network address of the speaker device through a second communication session; and routing the video data toward the network address of the display device through the second communication session or a third communication session.

The example embodiment may include, when a camera device is one of the I/O user devices in the set capable of outputting a camera stream, the operations update the repository <NUM> based on content of a registration message to identify a network address of the camera device and to identify a UI capability of the camera device as having a camera capability. The camera UI capabilities may identify a camera pixel count, resolution, image quality, light sensitivity, and/or other operational characteristics. The camera device is further identified as a member of the set of I/O user devices that are determined to be proximately located to the location of the user and is further determined, based on the UI capability identified by the repository <NUM>, to satisfy the combined capability criterion for being combinable with the other I/O user devices in the set to provide the combined I/O UI for the user to interface with the user terminal emulation application <NUM> to provide the communication service. Based on determining that the camera device satisfies the combined capability criterion, further operations are performed to route the camera stream received from the camera device toward the requesting user terminal, e.g., via the network entity <NUM>.

The operations for routing the microphone stream received from the microphone device and the camera stream received from the camera device toward the requesting user terminal, can include: receiving the microphone stream from the microphone device through a first communication session; receiving the camera stream from the camera device through the first communication session or a second communication session; combining the microphone stream and camera stream in a combined stream; and routing the combined stream toward the requesting user terminal through a third communication session, e.g., via the network entity <NUM>.

The example embodiment may include, when a keyboard device is one of the I/O user devices in the set capable of outputting key selection data responsive to key selections by a user among keys of the keyboard device, the operations can update the repository <NUM> based on content of a registration message to identify a network address of the keyboard device and to identify a UI capability of the keyboard device as having a keyboard capability. The keyboard device capabilities may identify a key count, indication of whether the keyboard is a physical keyboard or a touch sensitive input device, and/or other keyboard capabilities. The keyboard device is further identified as a member of the set of I/O user devices that are determined to be proximately located to the location of the user or are predicted to become proximately located to the user, and is further determined, based on the UI capability identified by the repository <NUM>, to satisfy the combined capability criterion for being combinable with the other I/O user devices in the set to provide the combined I/O UI for the user to interface with the user terminal emulation application <NUM> to provide the communication service. Based on determining that the keyboard device satisfies the combined capability criterion, further operations are performed to identify commands formed by the key selection data received from the keyboard and to perform operations that have been predefined as being triggered based on receipt of the identified commands.

The operations for routing the key selection data received from the keyboard device and microphone stream received from the microphone device, may include: receiving the key selection data from the keyboard device through a first communication session receiving the microphone stream from the microphone device through the first communication session or a second communication session; combining the key selection data and the microphone stream in a combined stream; and routing the combined stream toward the requesting user terminal through a third communication session, e.g., via the network entity <NUM>.

<FIG> is a block diagram illustrating the user terminal emulation server <NUM> as an element of an operator service node <NUM> within a cellular system <NUM>. Referring to <FIG>, the communication service function of the network entity <NUM> (<FIG>) may be provided by the operator service node <NUM> or may be reached through external infrastructure <NUM>, e.g., the Internet and/or private network. The server <NUM> may, for example, be implemented in the radio access network <NUM> to provide edge computing with faster responsiveness or may be implemented within another node of the cellular system <NUM>. The user terminal emulation server <NUM> can include an I/O user device handler (IODH) <NUM>, a control function (CF) <NUM>, the instantiated user terminal emulation applications <NUM>, and a service gateway (GW) <NUM>. A user terminal emulation application <NUM> may run one or more applications that are normally run by a smart phone, such as a Netflix application, Facebook application, Skype application, Internet browser application, etc..

The IODH <NUM> may perform operations to manage the I/O user devices, such as to handle maintenance (<NUM> in <FIG>) of the repository <NUM> and/or perform registration (<NUM> in <FIG>) of the users and possibly also user terminal emulation applications <NUM>. For example, the IODH <NUM> may operate to register with a Skype service server the IP address of a Skype application, which is run by or interfaced to the user terminal emulation application <NUM>, and the user's Skype name. The CF <NUM> may be responsible for assigning an IP address to each user terminal emulation application <NUM>. The IP address to be assigned by the CF <NUM> may be received from the core network <NUM> functionality such as a PDN-GW. The service GW <NUM> may interconnect the user terminal emulation server <NUM> to a PSTN network, packet data network gateway of a 3GPP (3rd Generation Partnership Project) system, etc. The cellular system <NUM> can include a Core Network <NUM> having a Home Subscriber Server (HSS), a Policy and Charging Roles Function (PCRF), gateway (GW) and Mobility Management Entity (MME) providing control signaling related to mobile terminal mobility and security for the radio access. The HSS contains subscriber-related information and provides support functionality for user authentication and user access to the system. The PCRF enables QoS control per data flow and radio bearer, by setting QoS criteria for each data flow, based on operator set policies and subscriber information. The GW can include a Serving GW (S-GW) and a Packet Data Network GW (PDN-GW), where the S-GW interconnects the core network <NUM> with the radio access network <NUM> and routes incoming and outgoing packets for the I/O user devices <NUM> and/or <NUM> and the user terminals <NUM>. The PDN-GW interconnects the core network <NUM> with external infrastructure <NUM>, such as the Internet, and allocates IP-addresses and performs policy control and charging.

Some I/O user devices <NUM> having cellular communication capability can communicate via, e.g., eNBs or other radio access nodes of a Radio Access Network <NUM> with the operator service node <NUM> via the core network <NUM>. In the system of <FIG>, the user terminal emulation server <NUM> may handle set up of a communication service between a selected set of the I/O user devices, which are determined to be proximate to a user or predicted to become proximately located to the user, and a remote user terminal <NUM> (e.g., smart phone) via the cellular system <NUM>.

<FIG> is a block diagram illustrating the user terminal emulation server <NUM> communicating in a different manner with various elements of a cellular system <NUM>, which may operate as the network entity <NUM> (<FIG>), to provide communication services in accordance with some embodiments of the present disclosure. The system of <FIG> differs from the system of <FIG> by the user terminal emulation server <NUM> being an Internet service within external infrastructure <NUM> outside of the cellular system <NUM>. In the system of <FIG>, the CF <NUM> may determine the IP address to be assigned to different ones of the user terminal emulation applications <NUM> based on signaling from the Internet service within the external infrastructure <NUM>.

The above and other operations will now be described in further detail in the context of three different example "use cases": <NUM>) incoming call scenario; <NUM>) outgoing call scenario; and <NUM>) share of I/O user devices scenario (sharing of physical resources and/or capabilities).

This use case involves a user, with a UserTag or other way of being identified, being proximately located to or predicted to become proximately located to I/O user devices <NUM> having different UI capabilities when an incoming call is received by the user terminal emulation server. Although operations are explained below in the context of identifying a user through a physical UserTag carried by the user, these operations are not limited thereto and may be used with any other way of identifying a user, such as by sensing biometric information that identifies the user.

A user terminal emulation application <NUM> may be instantiated or otherwise activated responsive by an incoming call (service, session) targeting the UserTag. The user terminal emulation application <NUM> can identify subscriptions associated with the UserTag (i.e. the physical user) and preferred methods of communication (e.g., audio not video, audio and video, etc.) that have been specified by the user, and determines the UI capabilities of the I/O user devices that will be needed to satisfy the UI capabilities which may be specified for the incoming communication session. The user terminal emulation application <NUM> may ask the IODH to identify which I/O user devices <NUM> are proximately located to the UserTag or are predicted to become proximately located to the UserTag, and may further ask the IODH to determine or may determine itself whether the identified I/O user devices <NUM> are combinable to satisfy the UI capabilities specified by the incoming communication session. The user terminal emulation application <NUM> and/or the IODH may receive an ACK or NACK back on whether a sufficient set of I/O user devices <NUM> can be used to provide the communication service. If ACK, then the IODH may also sets the state of the I/O user devices <NUM> in the set to in-use to avoid another user terminal emulation application <NUM> attempting to utilize the same I/O user devices <NUM> as which are presently in use or about to become in use. In case of NACK, the user terminal emulation application <NUM> and/or the IODH can take different actions to setup a reduced UI capability communication service with the user depending on user settings, e.g. only allow sound-based communications instead of a combination of sound and video responsive to when no display device is presently available for use. An example of no display device being available may occur when the only display device that is proximately located to the user is presently being used by another user to receive information from another user terminal emulation application during an ongoing communication service or when no display device is proximately located to the user.

<FIG> is a combined flowchart of operations and related data flows between UserTags, I/O user devices, and the user terminal emulation server in accordance with some embodiments of the present disclosure. Referring to <FIG>, a UserTag enters a room and signals <NUM> its presence to any proximately located and capable I/O user device in the room using a discovery beacon signal. Alternatively, one or more of the I/O user devices determines presence of the UserTag by polling <NUM>, such as by periodically transmitting discover beacon signals that trigger responsive signaling by the UserTag. The I/O user devices that receive signaling indicated presence of the UserTag report <NUM> to the IODH in the user terminal emulation server along with a network address of the I/O user device (e.g., IP address, port number, MAC address, FQDN, etc.). The user terminal emulation application corresponding to the specific user (i.e., the UserTag) is updated <NUM> with respect to the detected user's presence. The IODH may operate to receive the notifications from the I/O user devices proximately located to the UserTag. Further UI capability discovery (synchronization) communications <NUM> are performed between the user terminal emulation server and the I/O user devices which reported the user's presence and/or with I/O user devices that predicted to become proximately located to the user such as based on the user presently being proximately located to another one of the I/O user devices. The I/O user devices are associated to the user in the repository, along with associated indications subscriptions, combinable UI capabilities provided by the set of I/O user devices which are proximately located to or predicted to become proximately located to the UserTag. By operation <NUM> the user via the UserTag is now known to be reachable within the system through an identified set of I/O user devices with identified UI capabilities (e.g., speakers yes/no, display yes/no, microphone yes/no, keyboard yes/no, etc.), thereby creating a logical virtualized user terminal through which the user may be provided in a communication service. The user may initiate a communication service through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.

In operation <NUM>, an incoming session (e.g., video call) from a requesting user terminal which is directed to the user (UserTag) arrives at the user terminal emulation server for the user carrying the UserTag. In operation <NUM> the combinable UI capabilities of the available I/O user devices is compared to the UI requirements of the incoming session. When the UI requirements of the incoming session are not satisfied by the combinable UI capabilities of the I/O user devices, the user terminal emulation server may renegotiate <NUM> the required UI capabilities (e.g., QoS) of the incoming session. In contrast, when the UI requirements of the incoming session are satisfied by the combinable UI capabilities of the I/O user devices the user terminal emulation server prompts, via one or more of the available I/O user devices (e.g., a pre-selected answer device), the user carrying the UserTag to provide a session request answer (ACK/NACK). The user responds through the pre-selected answer device <NUM> to accept (ACK) or reject (NACK) the incoming session, to provide signaling <NUM> to the user terminal emulation server. When an ACK is received, operations <NUM> route an audio stream from the requesting user terminal to one of the I/O user devices in the set that has a speaker capability via one or more sessions <NUM>, and routes a video stream from the requesting user terminal to another one of the I/O user devices in the set that has a display capability via one or more sessions <NUM>. A data stream that is received from one of I/O user devices in the set through a one or more sessions <NUM> is routed <NUM> toward the requesting user terminal. When two or more data streams are received through one or more sessions <NUM> from the I/O user devices, they can be combined into a combined data stream that is routed <NUM> toward the requesting user terminal.

The user terminal emulation server may perform operations <NUM> to continuously monitor presence of the I/O user devices to determine when one or more of I/O user devices are no longer proximately located to the user such that they can no longer be included as part of the combined UI be provided during the ongoing communication session. The user terminal emulation server may substitute the UI capability of another I/O user device to the set being used by the user for the ongoing communication session responsive to a previous member of the set no longer having required presence. The user terminal emulation server may substitute the UI capability of another I/O user device that is predicted to become proximately located to the user as the user moves, and signal the other I/O user device to prepare for using its I/O user interface to support the ongoing communication session. As will be explained in further detail below, the user terminal emulation server can signal the I/O user device that is predicted to become proximately located to the user, to prepare the I/O user device for using the I/O user interface to provide the ongoing communication session for the user.

This use case involves a user, with a UserTag, being proximately located to or predicted to become proximately located to I/O user devices <NUM> having different UI capabilities when an outgoing call (communication session) is received by the user terminal emulation server. The I/O user devices <NUM> are associated to the identified user via the user terminal emulation server <NUM> which handles all communications sessions for the user while the associated I/O user devices <NUM> are managed by an IODH.

A user terminal emulation application <NUM> may be instantiated or otherwise activated responsive by an outgoing call being requested by a user carrying the UserTag. The user may initiate an outgoing call through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to a proximately located I/O user device.

The user terminal emulation application <NUM> can identify subscriptions associated with the UserTag (i.e. the physical user) and preferred methods of communication (e.g., audio not video, audio and video, etc.) that have been specified by the user, and determines the UI capabilities of the I/O user devices that will be needed to satisfy the UI capabilities which may be specified for the outgoing call. The user terminal emulation application <NUM> may ask the IODH to identify which I/O user devices <NUM> are proximately located to the UserTag and further ask, e.g., depending upon the answer, which other I/O user devices <NUM> are predicted to become proximately located to the user. The user terminal emulation application <NUM> may further ask the IODH to determine or may determine itself whether the identified I/O user devices <NUM> and/or other I/O user devices <NUM> are combinable to satisfy the UI capabilities specified by the outgoing call. The user terminal emulation application <NUM> and/or the IODH may receive an ACK or NACK based on whether a sufficient set of I/O user devices <NUM> can be used to provide the communication service. If ACK, then the IODH also sets the state of the I/O user devices <NUM> in the set to in-use to avoid another user terminal emulation application <NUM> attempting to utilize the same I/O user devices <NUM> as which are presently in use or about to be in use. In case of NACK, the user terminal emulation application <NUM> and/or the IODH can take different actions to setup a reduced UI capability communication service with the user depending on user settings, e.g. only allow sound instead of the preferred sound and video responsive to when no display device is presently available for use (e.g., when presently used by another user terminal emulation application <NUM> or when none is proximately located to the UserTag or predicted to become proximately located to the UserTag with a threshold time).

<FIG> is a combined flowchart of operations for an outgoing call and related data flows between UserTags, I/O user devices, and the user terminal emulation server in accordance with some embodiments of the present disclosure. Referring to <FIG>, a UserTag enters a room and signals <NUM> its presence to any proximately located and capable I/O user device in the room using a discovery beacon signal. Alternatively, one or more of the I/O user devices determines presence of the UserTag by polling <NUM>, such as by periodically transmitting discover beacon signals that trigger responsive signaling by the UserTag. The I/O user devices that receive signaling indicated presence of the UserTag report <NUM> to the IODH in the user terminal emulation server along with a network address of the I/O user device (e.g., IP address, port number, MAC address, FQDN, etc.). The user terminal emulation application corresponding to the specific user (i.e., the UserTag) is updated <NUM> with respect to the detected user's presence.

The IODH may operate to receive the notifications from the I/O user devices proximately located to the UserTag. Further UI capability discovery (synchronization) communications <NUM> are performed between the user terminal emulation server and the I/O user devices. The I/O user devices are associated to the user in the repository, along with associated indications subscriptions, combinable UI capabilities provided by the set of I/O user devices which are proximately located to the UserTag. By operation <NUM> the user via the UserTag is now known to be reachable within the system through an identified set of I/O user devices with identified UI capabilities (e.g., speakers yes/no, display yes/no, microphone yes/no, keyboard yes/no, etc.), thereby creating a logical virtualized user terminal through which the user may be provided in a communication service. The IODH may further predict what other I/O user devices will become proximately located to the user, such as within a threshold time and have combinable UI capabilities that satisfy the capability requirements. The user may initiate a communication service through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.

In operation <NUM>, a user carrying the UserTag uses the UI of one of the I/O user devices to trigger <NUM> an outgoing call (e.g., video call), which triggers signaling <NUM> of the outgoing call to the user terminal emulation server. In operation <NUM> the IODH queries the user (e.g., displays a message, generates a sound, etc.) through one of the I/O user devices proximately located to the user to request the user to select among available types of communication methods that can be presently used for the outgoing call. One of the I/O user devices provides responsive signaling <NUM> to the IODH indicating the user's selected type of communication method for the outgoing call. In operation <NUM> the user terminal emulation server communicates an outgoing session stream request to the network entity <NUM>, where the request may include an identifier of the calling user, identifier of the user terminal of the called user, and a quality of service for the communication session. In operation <NUM>, the user terminal emulation server receives a communication session acceptance (ACK) or denial (NACK) from the network entity <NUM>. When the communication session is denied, the user terminal emulation server may attempt to renegotiate <NUM> the requested communication session such as at a lower quality of service.

When the communication session is accepted (ACK), for each data type that is received <NUM> as communication traffic from the requesting user terminal, the user terminal emulation server selects one of the I/O user devices from among the set of I/O user devices based on matching characteristics of the data type to the UI capabilities identified by the repository for the one of the I/O user devices, and then routes <NUM> the data of the data type toward the network address of the selected one of the I/O user devices. The data originating ones of the I/O user devices transmit <NUM> data stream through one or more sessions <NUM> to the user terminal emulation server <NUM>, which may combine <NUM> the data streams into a combined data stream that is routed <NUM> toward the called user terminals via the network entity <NUM>.

The user terminal emulation server may continuously monitor <NUM> presence of the I/O user devices to determine when one or more of I/O user devices is no longer proximately located to the user such that it can no longer be included as part of the combined UI to be provided during the ongoing communication session and/or to determine when one or more of predicted soon-to-be proximately located I/O user devices becomes proximately located to the user such that it can be included as part of the combined UI to be provided during the ongoing communication session. The user terminal emulation server may substitute the UI capability of another I/O user device, e.g., an I/O user device that is predicted to become proximately located to the user, to the set being used by the user for the ongoing communication session responsive to a previous member of the set no longer having required presence. As will be explained in further detail below, the user terminal emulation server can signal the I/O user device that is predicted to become proximately located to the user, to prepare the I/O user device for using the I/O user interface to provide the ongoing communication session for the user.

The third use case is directed to a scenario where two or more users are located in a physical area having a number of I/O user devices with combined UI capabilities that are insufficient to satisfy the UI requirements needed to support time overlapping communication sessions without sharing of some of the I/O user devices by the two users. In this case, both users' ID entities, i.e. UserTags will be detected in the proximity of some I/O user devices which the IODH then associates to the users' respective user terminal emulation applications <NUM>.

In the illustration of <FIG>, user terminal emulation application #<NUM> is already handling an ongoing communication session in which some of the associated I/O user devices are allocated and in use by a first user corresponding to UserTag #<NUM>; this means the example starts after block <NUM> in <FIG>. At that point in time, another second user carrying UserTag#<NUM> enters the physical area.

In block <NUM> the UserTag#<NUM> becomes proximately located to or is predicted to become proximately located to at least one of the I/O user devices being presently utilized by user terminal emulation application #<NUM>. For example, the at least one of the I/O user devices detects UserTag#<NUM> or another I/O user device detects UserTag#<NUM> and the UserTag#<NUM> is predicted to become proximately located to the at least one of the I/O user devices, e.g., within a threshold time. The user terminal emulation application #<NUM> is responsively instantiated on the user terminal emulation server <NUM> by the IODH.

In block <NUM>, a request for a new communication session is received which is intended for the user terminal emulation application #<NUM>. In block <NUM> the IODH considers priorities among the user terminal emulation applications have been instantiated. The IODH compares subscriptions of currently hosted (ongoing) user terminal emulation application #<NUM> sessions to those of the new incoming request for a communication session towards the user terminal emulation application #<NUM>.

If a prioritization is not identified, the user terminal emulation applications are treated equally so that available I/O user devices with their UI capabilities are allocated on a first-come-first-served manner. In contrast, if a prioritization is identified, the IODH operates to: assign a priority to certain I/O user devices for the user terminal emulation application having the highest priority according to defined QoS/capability prioritization criteria; with certain UI capabilities present at certain types of I/O user devices, some UI capabilities are operationally shared between the first and second users (e.g., a sufficiently large display screen can be split in half with the halves respectively assigned to the first and second users).

In the present scenario, the user terminal emulation applications #<NUM> and #<NUM> are treated equally without prioritization, which applies to all operations within reference block <NUM>.

In block <NUM> the IODH assesses the request to capabilities of the second user and user terminal emulation application #<NUM> (considering prioritization's established in the previous step, if any). If the incoming request is not supported (e.g., there are no available or insufficient available I/O user devices determined (via query <NUM> to I/O user devices) to satisfy the required UI capabilities) by the user terminal emulation application #<NUM> irrespective of any priorities, the IODH may operate to negotiate UI capabilities/QoS with the network entity which sent the session request for the second user. In contrast, if the incoming request is supported by the UI capabilities provided by the user device emulation application #<NUM> for the requested communication session, as determined by the user terminal emulation application #<NUM> querying <NUM> the available I/O user devices, then an ACK is communicated to the network entity <NUM>.

If the incoming communication request can be supported by the user terminal emulation application #<NUM> when granted some priority for use of one or more of the I/O user devices being presently used by the user terminal emulation application #<NUM>, the operations of block <NUM> can be performed to preempt or cause sharing of the one or more of the I/O user devices that are presently being used by the user terminal emulation application #<NUM>. The IODH and/or the user terminal emulation application #<NUM> may then renegotiate <NUM> with the network entity <NUM> the required UI capabilities of the communication session of the user terminal emulation application #<NUM> for the first user. The user terminal emulation application #<NUM> may receive a session ACK/NACK generated <NUM> by the one or more of the preempted or shared I/O user devices with the user terminal emulation application #<NUM> indicating what UI capabilities are available for use by the user terminal emulation application #<NUM>, and may responsively terminate the existing communication session if the required UI capability provided by the I/O user devices is no longer sufficient for use by the user terminal emulation application #<NUM> for the communication session.

In block <NUM>, the user terminal emulation application #<NUM> queries the second user through a preselected one of the I/O user devices for whether the requested communication session is accepted. The preselected one of I/O user devices may display <NUM> a prompt querying the second user for an answer, which is provided <NUM> as an ACK/NACK to the user terminal emulation application #<NUM>. When the second user accepts the communication session (ACK), a communication service is established between the second user and a remote user terminal through the user terminal emulation application #<NUM> and the network entity <NUM>.

An I/O user device presence monitor may operate as a function of the IODH to monitor (continuously, periodically, or responsive to occurrence of a defined event) an ongoing communication session to ensure that all UI capabilities provided by the set of I/O user devices remains proximately located and operationally available to the user.

Exchange of messages between the different system entities can be carried out using the Session Initiation Protocol (SIP) with the Session Description Protocol (SDP) with possible some minor changes regarding the current supported methods in the protocols and media formats. Using SIP/SDP may be advantageous as the connections between the I/O user devices and user terminal emulation applications can be a SIP session which may set up in a manner similar to that between two VoIP clients.

More generally, the operations performed by the user terminal emulation server can include receiving another communication request from the network entity for establishing another communication service between another user and another requesting user terminal. Responsive to the another communication request, the operations determine whether another set of I/O user devices among the I/O user devices identified by the repository are determined to be proximately located to a location of the another user or are predicted to become proximately located to the another user and available for use for the another communication service. A further determination is made, based on the UI capabilities identified by the repository for the another set of I/O user devices, that the set of I/O user devices satisfy the combined capability criterion for being combinable to provide a combined I/O UI for the another user to interface with another user terminal emulation application to provide the another communication service. Based on determining that no other set of I/O user devices is determined to satisfy the combined capability criterion, available for use by the another communication service, and proximately located to or predicted to become proximately located to the location of the another user, the operations responsively configure one of the I/O user devices in the set of I/O user devices that is proximately located to or predicted to become proximately located to the another user but is currently being used by the user, to operate to provide a shared UI that is used by the user while the communication service is continuing to be provided to the user and that is further used by the another user while the another communication service is provided to the another user.

In one example, information that is directed to the user can be routed for display in one half of a screen of the display device, and information that is directed to the other user can be routed for display in the other half of the screen of the display device. In another example, a keyboard can be shared by two users who identify themselves via the keyboard (e.g., by typing a user ID, scanning the UserTag, biometric scan, etc.) at the time they are entering information, so that the server can selectively route the keyboard entries to the correct one of the two user terminal emulation applications.

The above description has explained various operations for determining that an I/O user device has an I/O user interface that satisfies a capability criterion to enable a user to use a communication service through a network entity, and to use the I/O user device for the communication service. Further operations are now described for predicting that an I/O user device will become proximately located to a user and responsively signaling the I/O user device to prepare for using the I/O user interface to provide the communication service for the user.

<FIG> illustrates a combined flowchart of operations and related data flows between UserTags, I/O user devices, and the user terminal emulation server <NUM> to predict that a user will become proximately located to an I/O user device and to responsively signal the I/O user device to prepare for using the I/O user interface to provide the communication service for the user, in accordance with some embodiments of the present disclosure.

Referring to <FIG>, a first I/O user device (IOD#<NUM>) detects <NUM> a UserTag transported by the user, and sends <NUM> to the user terminal emulation server <NUM> IOD data that is obtained from the UserTag or determined based on sensing characteristics of the UserTag. The IOD data may include a user identifier, user preferences relating to one or more types of communication services, and/or user mobility information (e.g., location, speed of movement, direction of movement, etc.). The user preferences indicated by the IOD data may include one or more of: whether the user prefers to use headset audio or loudspeaker audio, the user's speaker loudness preference, whether the user prefers to participate in online meetings using or without using camera video, the user's written language preference, the user's keyboard layout configuration preference, the user's video characteristic preference (e.g., minimum text size preference), etc..

In the example of <FIG>, the IOD data is received by an I/O user device handler (IODH) <NUM> which associates <NUM> the first I/O user device (IOD#<NUM>) with a user terminal emulator <NUM>, and sends <NUM> a notification to the user terminal emulator <NUM> indicating that the first I/O user device (IOD#<NUM>) is presently proximately located to the user and available to provide a communication service for the user. The IODH <NUM> sends <NUM> a prediction request containing at least part of the IOD data, such as the user identifier and an indication that the first I/O user device (IOD#<NUM>) is presently proximately located to the user.

In some embodiments, the prediction request is provided to a machine learning model <NUM>, which predicts <NUM> (also <NUM> in <FIG>) which, if any, other I/O user device(s) will become proximately located to the user based on content of the received prediction request. In one embodiment, the machine learning model <NUM> determines one or more I/O user devices that satisfy a criterion for having a probability which satisfies a probability threshold for becoming within a threshold proximity range of the user within a threshold time. The machine learning model <NUM> provides <NUM> a prediction response to the IODH <NUM>, which in the example of <FIG> indicates that a second I/O user device (IOD#<NUM>) satisfies the criterion. The prediction response can identify a list of any I/O user devices which satisfy the determinations by the machine learning model <NUM>.

The IODH <NUM> decides <NUM> whether to associate the second I/O user device (IOD#<NUM>) with the user terminal emulator <NUM> based on the prediction response and a determination (e.g., <NUM> in <FIG>) whether the second I/O user device (IOD#<NUM>) has an I/O user interface that satisfies a capability criterion to enable the user to use a communication service through the network entity <NUM>. When the IODH <NUM> decides <NUM> to associate the second I/O user device (IOD#<NUM>) with the user terminal emulator <NUM>, the IODH <NUM> signals <NUM> (e.g., <NUM> in <FIG>) the second I/O user device (IOD#<NUM>) to prepare for using the I/O user interface to provide the communication service for the user through the network entity <NUM>, and can further send <NUM> a notification to the user terminal emulator <NUM> indicating that the second I/O user device (IOD#<NUM>) is available or may soon become available to provide a communication service for the user. In this manner, the user terminal emulator <NUM> is made aware that the second I/O user device (IOD#<NUM>) can be used to set up a communication service with the user or to hand off an ongoing communication service involving the user, such as from using the first I/O user device (IOD#<NUM>) to instead using the second I/O user device (IOD#<NUM>). For example, an audio and/or video stream that was being routed to the first I/O user device (IOD#<NUM>) may then be additionally or alternatively routed to the second I/O user device (IOD#<NUM>), such as responsive to the first I/O user device (IOD#<NUM>) ceasing to detect the presence of the UserTag, e.g., for at least a threshold time and/or responsive to the second I/O user device (IOD#<NUM>) detecting new presence of the UserTag.

In some embodiments, various I/O user devices may be configured to communicate directly with one another to perform for the user terminal emulation server <NUM>. For example, the IODH <NUM> may signal <NUM> (e.g., <NUM> in <FIG>) the first I/O user device (IOD#<NUM>) to trigger the first I/O user device (IOD#<NUM>) to, in turn, signal the second I/O user device (IOD#<NUM>) to prepare for using the I/O user interface of the second I/O user device (IOD#<NUM>) to provide the communication service for the user. Alternatively or additionally, the first and second I/O user devices may communicate between them to coordinate hand off between then for providing an I/O user interface for the user for an ongoing communication service.

Further operations are now described for predicting that an I/O user device will become proximately located to a user and responsively controlling routing of a traffic for an ongoing communication service to the I/O user interface, in accordance with some embodiments.

<FIG> illustrates a combined flowchart of operations and related data flows between a UserTag, I/O user devices, and the user terminal emulation server <NUM> in accordance with some embodiments of the present disclosure.

Referring to <FIG>, a user associated with the UserTag is using a first I/O user device (IOD#<NUM>) or group of IODs (collectively referred to as IOD#<NUM>) for a communication session <NUM> established to transport traffic of an ongoing communication service provided through the user terminal emulation server <NUM> and the network entity <NUM> of, e.g., <FIG>. The user terminal emulation server <NUM> and, more particularly, the IODH <NUM> sends <NUM> a prediction request containing at least part of IOD data, such as the user identifier and an indication that the first I/O user device (IOD#<NUM>) is presently proximately located to the user.

In some embodiments, the prediction request is provided to the machine learning model <NUM>, which predicts <NUM> (also <NUM> in <FIG>) which, if any, other I/O user device(s) will become proximately located to the user based on content of the received prediction request. In one embodiment, the machine learning model <NUM> determines one or more I/O user devices that satisfy a criterion for having a probability which satisfies a probability threshold for becoming within a threshold proximity range of the user within a threshold time. The machine learning model <NUM> provides <NUM> a prediction response to the IODH <NUM>, which in the example of <FIG> indicates that a second I/O user device (IOD#<NUM>) satisfies the criteria. The prediction response can identify a list of any I/O user devices which satisfy the determinations by the machine learning model <NUM>.

The IODH <NUM> decides <NUM> whether to associate the second I/O user device (IOD#<NUM>) with the user terminal emulator <NUM> based on the prediction response and a determination (e.g., <NUM> in <FIG>) whether the second I/O user device (IOD#<NUM>) has an I/O user interface that satisfies a capability criterion to enable the user to use the ongoing communication service through the network entity <NUM>. When the IODH <NUM> decides <NUM> to associate the second I/O user device (IOD#<NUM>) with the user terminal emulator <NUM>, the IODH <NUM> may repetitively signal 1110a-1110n (e.g., <NUM> in <FIG>) the second I/O user device (IOD#<NUM>) to prepare for using the I/O user interface to serve as a user interface for at least some of the traffic of the ongoing communication session <NUM>. The repetitive signaling 1110a-1110n may serve to query the second I/O user device (IOD#<NUM>) whether the user has become proximately located, to wake up the second I/O user device (IOD#<NUM>) for identifying when the user has become proximately located, for paging the second I/O user device (IOD#<NUM>), and/or to enable faster establishment of a communication session with the IODH <NUM> when handoff of the communication services is to be performed to the second I/O user device (IOD#<NUM>).

The IODH <NUM> may further send <NUM> a notification to the user terminal emulator <NUM> indicating that the second I/O user device (IOD#<NUM>) is available or may soon become available to provide the communication service of the communication session <NUM> for the user. In this manner, the user terminal emulator <NUM> is made aware that the second I/O user device (IOD#<NUM>) can be used for hand off of the ongoing communication service involving the user, such as by routing traffic for the communication service to the second I/O user device (IOD#<NUM>) and ceasing routing of the traffic the first I/O user device (IOD#<NUM>). For example, an audio and/or video stream that was being routed to the first I/O user device (IOD#<NUM>) may then be additionally or alternatively routed to the second I/O user device (IOD#<NUM>), such as responsive to the first I/O user device (IOD#<NUM>) ceasing to detect the presence of the UserTag, e.g., for at least a threshold time and/or responsive to the second I/O user device (IOD#<NUM>) detecting new presence of the UserTag.

As further illustrated in the example scenario of <FIG>, the second I/O user device (IOD#<NUM>) detects <NUM> presence of the UserTag and receives the IOD data which can include the user ID and preferences as described above. The second I/O user device (IOD#<NUM>) sends <NUM> the IOD data (e.g., user ID) to the IODH <NUM> which can provide <NUM> a notification to the user terminal emulator <NUM> that the second I/O user device (IOD#<NUM>) is now proximately located to the user (e.g., user ID). The user terminal emulator <NUM> responsively decides <NUM> to route traffic (user data) for the ongoing communication service to the second I/O user device (IOD#<NUM>) by, for example, establishing a communication session with the second I/O user device (IOD#<NUM>).

In some embodiments, the operation for signaling (e.g., <NUM> in <FIG>, <NUM> in <FIG>, 1110a-n in <FIG>) an I/O user device (e.g., second I/O user device (IOD#<NUM>)) to prepare for using its I/O user interface to provide a communication service for the user, includes sending (e.g., <NUM> in <FIG>, 1110a-n in <FIG>) a message configured to trigger the I/O user device to report to the user terminal emulation server <NUM> when the user has become proximately located.

As illustrated in the operational scenario of <FIG>, the signaling (e.g., <NUM> in <FIG>, <NUM> in <FIG>, 1110a-n in <FIG>) can include repetitively sending (1110a-n) the message, and adjusting a rate at which the message is repetitively sent based on at least one of: a determined probability that the I/O user device will become proximately located to the user; and a prediction of when the I/O user device will become proximately located to the user.

In some other embodiments, the user terminal emulation server <NUM> starts routing traffic for communication service to an I/O user device before the user is predicted to become proximately located to the I/O user device. In one example embodiment, operations for signaling (<NUM> in <FIG>, <NUM> in <FIG>, 1110a-n in <FIG>) the I/O user device to prepare for using the I/O user interface to provide a communication service for the user through the network entity <NUM>, includes initiating routing of communication traffic of the communication service to the I/O user device before the user is predicted to become proximately located to the I/O user device. The I/O user device is configured to perform one of the following with the communication traffic: initiate playout (e.g., audio output through speaker, video output through video display, etc.) of the communication traffic through the I/O user interface; buffer the communication traffic for subsequent playout through the I/O user interface responsive to the user becoming proximately located to the I/O user device; and discard the communication traffic until the user becomes proximately located to the I/O user device.

Preparation of the I/O user device responsive to the signaling can include establishing a communication session with I/O user device which connects the I/O user device to the user terminal emulation server <NUM> before the user is predicted to become proximately located to the I/O user device. In some embodiments, traffic for the communication session may not start to be routed to the I/O user device until the user's proximity, e.g., userTag, is reported. In one example embodiment, the user terminal emulation server <NUM>, e.g., user terminal emulator <NUM> and/or the IODH <NUM>, establishes a communication session with the I/O user device (e.g., second I/O user device (IOD#<NUM>)) which connects the I/O user device to the user terminal emulation server <NUM> before the user is predicted to become proximately located to the I/O user device. Responsive to determining that the user has become proximately located to the I/O user device, the user terminal emulation server <NUM> initiates routing of communication traffic of the communication service through the communication session established with the I/O user device (e.g., second I/O user device (IOD#<NUM>)) and ceases routing of the communication traffic of the communication service through another communication session established with a previously proximate I/O user device (e.g., first I/O user device (IOD#<NUM>)) that is no longer proximately located to the user.

The signaling to prepare the I/O user device may contain configuration and/or setup information that is used by the I/O user device to become operationally ready for using its I/O user interface to provide the communication service for the user. For example, in one embodiment the signaling (<NUM> in <FIG>, <NUM> in <FIG>, 1110a-n in <FIG>) includes communicating <NUM> device configuration data to the I/O user device which configures the I/O user device for using the I/O user interface to provide the first communication service. In a further embodiment, the device configuration data communicated to the I/O user device is adapted to configure at least one of the following operations of the I/O user device:.

Various ways that the machine learning model <NUM> can be trained and used to predict proximity of a user to one or more I/O user devices are now explained.

In some embodiments, the machine learning model <NUM> includes a decentralized federated learning architecture with hierarchical associations between layers of machine learning models. A layer of local machine learning models process and are trained to track information indicating users' habits for using communication services (e.g., which I/O user devices the users usually use as a function of various types of communication services, as a function of time of day and/or day of week, as a function of how many people and/or which particular people are participating in the communication service). As the local machine learning models are updated based on observed user activity, model parameters are also transmitted to a higher level machine learning model which may reside in the IODH <NUM> which has been trained with knowledge about the relative locations of the I/O user devices and the associated local machine learning models that more locally monitor I/O user device usage. Each of the models may be configured as a Deep Neural Network (DNN) or Convolutional Neural Network (CNN).

The division of responsibility between the hierarchical layers of models can be configured according to geographic constraints, e.g., so that "sub IODHs" have access to process information relating to only a city constrained list of locations of I/O user devices, and are trained based on user usage of the I/O user devices within that city. That city-based model can be then in-turn be connected to a higher level "municipality model" which encompasses the plurality of cities or directly to a "country model" which encompasses a plurality of municipalities, and so on. The architectural layout of the hierarchical machine learning models may be manually setup by one or more model owners and/or can be dynamically self-configured depending on the number of users, number of I/O user devices, observed patterns of mobility between the I/O user devices, etc. City, municipality, etc. are non-limiting examples and any grouping of I/O user devices to machine learning models can be used. The grouping may be based on data rates (number of I/O user devices communicating with the IODH <NUM>, etc.), number of users, and/or local/regional regulatory borders.

Associations between I/O user devices and individual ones of the machine learning models may be remapped over time to reflect changes in location of the I/O user devices, such as when some of I/O user devices are not geographically fixed but instead mobile.

A repository database <NUM> (e.g., <FIG>) containing locations and UI capabilities of I/O user devices may be organized in a data structure similar to a look-up-table that is continuously updated with new locations of I/O user devices. Alternatively, an improved approach may use a network model database or hierarchical model that is built up by the IODH <NUM> when new I/O user devices are being added. This makes the repository database <NUM> better at handling multiple access at the same time, since the lookup will start the traverse from the datapoints around it (geography-wise) and then search from there, until the search criteria has been met.

The machine learning model <NUM> can be trained based on time ordered sequences of I/O user devices which have been historically observed to sequentially become proximate to users, e.g., first I/O user device near doorway detects proximity with users, then second I/O user device further down a hallway detects proximity with some or all of those users, then a third I/O user device further down the hallway detects proximity with some or all of those users, then a fourth I/O user device in a main conference room further down the hallway detects proximity with some or all of those users, etc. In one embodiment, the operation predicting (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) that the I/O user device will become proximately located to the user, includes processing (<NUM>, <NUM>) information indicating present proximity of the user to another I/O user device through the machine learning model <NUM> that has been trained based on time ordered sequences of I/O user devices that have been historically observed to become proximately located to users, and predicting (<NUM>, <NUM>, <NUM>, <NUM>) that the I/O user device will become proximately located to the user based on output of the machine learning model <NUM> from processing the information.

The rate at which a message is repetitively sent for the signaling (<NUM>, <NUM>, 1110a-n) to prepare an I/O user device, can be adjusted based on output of the machine learning model <NUM> indicating probability the user will be proximately located to the I/O user device. In one embodiment, the operations repetitively send (1110a-1110n) a message configured to trigger the I/O user device to prepare the I/O user device for using the I/O user interface to provide the first communication service for the user. The operations process (<NUM>, <NUM>) information indicating present proximity of the user to another I/O user device through the machine learning model <NUM> that has been trained based on time ordered sequences of I/O user devices identified in the repository that have been historically observed to become proximately located to users, to output an indicated probability that the user will be proximately located to the I/O user device. The operations then adjust a rate at which the message is repetitively sent based on the indicated probability.

The machine learning model <NUM> may be trained based on historical observations of which users have been proximately located to which I/O user devices as a function of time and date. In one embodiment, the operations to predict that the I/O user device will become proximately located to the user, include processing (<NUM>, <NUM>) a time of day and date (where "date" can be the day of week, day of month, or day of year) through the machine learning model <NUM> that has been trained based on historically observed proximity between identified users and identified I/O user devices at identified times and dates, to output probability predictions of which of the identified users are predicted to become proximately located to identified I/O user devices. The operations then predict (<NUM>, <NUM>, <NUM>, <NUM>) whether the I/O user device will become proximately located to the user based on an identity of the user and the probability predictions output by the machine learning model <NUM>.

The machine learning model <NUM> may be trained based on geographic locations of the I/O user devices, so a user's present geographic location can generate a prediction of which, if any, I/O user devices are likely to become proximately located to the user. In one embodiment, the operations to predict that the I/O user device will become proximately located to the user, include processing (<NUM>, <NUM>) a present geographic location of the user through the machine learning model <NUM> that has been trained based on geographic locations of the I/O user devices. The operations then predict (<NUM>, <NUM>, <NUM>, <NUM>) whether the I/O user device will become proximately located to the user based on an output of the machine learning model <NUM> from processing the present geographic location of the user.

The machine learning model <NUM> may be trained based on capabilities of I/O user devices which are required to satisfy defined types of communication services. One embodiment, the operations to determine that the I/O user interface of the I/O user device satisfies the capability criterion, include processing (<NUM>, <NUM>) a user interface capability of the I/O user interface of the I/O user device and an indication of the first one of the communication services through the machine learning model <NUM> that has been trained based on user interface capabilities that are required to satisfy different types of communication services. The operations then determine (<NUM>, <NUM>, <NUM>, <NUM>) whether the capability criterion is satisfied based on an output of the machine learning model <NUM> from processing the user interface capability of the I/O user interface and the indication of the first one of the communication services.

The machine learning model <NUM> may be trained based on sets of I/O user devices which have been historically selected by users when using a communication service. In one embodiment, the operations process (<NUM>, <NUM>, <NUM>) a predicted next location of the user through the machine learning model <NUM> that has been trained based on historically observed sets of the I/O user devices that have been selected for use by users who were proximately located to the next location when using a communication service, to output candidate sets of the I/O user devices. The operations then select (<NUM>, <NUM>) a first set of the I/O user devices, from among the candidate sets of the I/O user devices, which have a set of I/O user interfaces that satisfy the capability criterion to enable the user to use the first one of the communication services. Before the user is predicted to become proximately located to the first set of the I/O user devices, the operations signal (<NUM>, <NUM>, 1110a-n) the first set of the I/O user devices to prepare for using the set of I/O user interfaces to provide the first communication service for the user through the network entity <NUM>.

In some embodiments, input data is processed through machine learning model <NUM> at a defined frequency returning a set of I/O user devices that is predicted to become proximately located to one or more users. Reinforced training of the machine learning model <NUM> over time can include using feedback indicating which of the I/O user devices listed in the set were chosen and/or not chosen by the IODH <NUM> and/or by a user(s) as an interface for one or more communication services. Machine learning model parameters which have been updated through the training can be provided to one or more higher level machine learning models in a hierarchical set of machine learning models, such as described above (e.g., small geographical area layer of machine learning models, connected higher layer of machine learning models for larger geographic regions, etc.).

As was described above, the user terminal emulation server <NUM> can select a set of the I/O user devices, from among candidate sets of the I/O user devices, which have a set of I/O user interfaces that satisfy a capability criterion to enable the user to use one of the communication services. Before the user is predicted to become proximately located to the set of the I/O user devices, the user terminal emulation server <NUM> can signal the set of the I/O user devices to prepare for using the set of I/O user interfaces to provide the communication service for the user through the network entity <NUM>.

An example of a set of the I/O user devices includes operationally coupling to the user terminal emulator <NUM> a big screen TV, which lacks a microphone, with a nearby smart speaker that has a microphone operable to record user commands. Together, the set of the I/O user devices satisfies the requirements of a single video conference communication device. Another example includes a set of TV screens spaced apart along a building corridor, which are sequentially prepared for use and then used to provide a communication service to a user as the user walks along the building corridor becoming proximately located to individual ones of the TV screens one-at-a-time.

Which I/O user devices are selected for inclusion in a set can also be based on how many people and/or which particular people are participating in a communication service, such as to prepare a headset speaker for use by a single local user participant or a loudspeaker for use by a plurality of local user participants. The selection may also be influenced by time of day and/or date which affects known availability of particular ones of the I/O user devices for use in providing a communication service.

A wide range of sensors may be used selectively included as part of a set of I/O user devices to be prepared for a communication service. For example, led lights and environmental sensors (temp or ventilation, etc.) may also be selected among for inclusion in the set to, for example, enable adjustment of room lighting and/or temperature in preparation for a user participating in a communication service from within the room.

Some or all operations described above as being performed by the user terminal emulation server <NUM> or the I/O user devices <NUM> may alternatively be performed by the other one, and/or by another node that is part of a cloud computing resource. For example, those operations can be performed as a network function that is close to the edge, such as in a cloud server or a cloud resource of a telecommunications network operator, e.g., in a CloudRAN or a core network, and/or may be performed by a cloud server or a cloud resource of a media provider, e.g., iTunes service provider or Spotify service provider.

<FIG> is a block diagram of components of an I/O user device <NUM> which are configured to operate in accordance with some embodiments. The I/O user device <NUM> can include a wired/wireless network interface circuit <NUM>, a near field communication circuit <NUM>, at least one processor circuit <NUM> (processor), and at least one memory circuit <NUM> (memory). The processor <NUM> is connected to communicate with the other components. The memory <NUM> stores program code <NUM> that is executed by the processor <NUM> to perform operations disclosed herein. The processor <NUM> may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor <NUM> is configured to execute the program code <NUM> in the memory <NUM>, described below as a computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a mobile electronic device. The I/O user device <NUM> can include one or more UI devices, including without limitation, a microphone <NUM>, a speaker <NUM>, a camera <NUM>, and display device <NUM>, and a user input interface <NUM>.

<FIG> is a block diagram of components of a user terminal emulation server <NUM> which are configured to operate in accordance with some embodiments. The user terminal emulation server <NUM> can include a wired/wireless network interface circuit <NUM>, a repository <NUM> (e.g., listing I/O user devices, UI capabilities of the I/O user devices, known proximities to user identifiers, etc.), a display device <NUM>, a user input interface <NUM> (e.g., keyboard or touch sensitive display), at least one processor circuit <NUM> (processor), and at least one memory circuit <NUM> (memory). The processor <NUM> is connected to communicate with the other components. The memory <NUM> stores user terminal emulation application <NUM> that is executed by the processor <NUM> to perform operations disclosed herein. The memory <NUM> also stores a proximity prediction and I/O user device (IOD) preparation module <NUM>. In some embodiments, the module <NUM> includes a machine learning model <NUM>. The processor <NUM> may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor <NUM> is configured to execute computer program instructions in the memory <NUM>, described below as a computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a mobile electronic device.

In the above description of various embodiments of present inventive concepts, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of present inventive concepts. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which present inventive concepts belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense expressly so defined herein.

Claim 1:
A user terminal emulation server (<NUM>) for providing communication services through one or more input and/or output, I/O, user devices to a user, wherein the user terminal emulation server (<NUM>) is configured to:
register the user with a network entity (<NUM>) providing the communication services;
predict that an I/O user device will become proximately located to the user; and
based on the prediction that the I/O user device will become proximately located to the user,
determine that the I/O user device has an I/O user interface that satisfies a capability criterion to enable the user to use a first one of the communication services, the user terminal emulation server being characterized in that it is further configured,
based on the satisfaction of the capability criterion and before the user is predicted to become proximately located to the I/O user device, to signal the I/O user device to prepare for using the I/O user interface to provide the first communication service for the user through the network entity (<NUM>)
wherein to signal the I/O user device to prepare for using the I/O user interface to provide the first communication service for the user through the network entity (<NUM>), the user terminal emulation server (<NUM>) is configured to:
send a message configured to trigger the I/O user device to report to the user terminal emulation server (<NUM>) when the user has become proximately located.