Patent Description:
Virtualization technology abstracts a desktop operating system (OS) and associated applications from a client computing device used to access them. For example, a virtual desktop infrastructure solution (VDI) may involve hosting a desktop on servers in a data center and delivering an image of the desktop over a network to the client computing device. The desktop image can then be rendered on the client computing device and a user of the client computing device may interact directly with the image as though the desktop and its applications are running locally on the client computing device.

This approach enables customers to streamline management and costs by consolidating and centralizing user desktops. The benefits of centralization include hardware resource optimization, reduced software maintenance, and improved security. For example, software patching and OS migrations can be applied and tested for all users in one instance. Moreover, software assets are centralized and thereby easily monitored and protected, and sensitive data is uncompromised in cases of desktop loss or theft. In addition, desktop virtualization increases users' mobility and the freedom to access virtual desktops from anywhere and any device.

Despite the many benefits afforded by virtualization technologies, some applications, however, may not be optimized for or well-supported in VDI environments. One such example is communications applications running in a VDI environment that enable real-time communications (RTC) between users (e.g., Microsoft® Teams, Slack®). RTC refers to near simultaneous exchange of information (e.g., voice, instant messaging, video, etc.,) over a network from the sender to the receiver with negligible latency.

<CIT> describes techniques for sharing audio and video device on a client endpoint device between local use and hosted virtual desktop use.

Methods, systems, and computer program products are described herein for improving the performance of a remote desktop client that is executing on a client computing device to present a user interface (UI) of a communications application that is remotely executing in a cloud computing environment. In aspects, the methods, systems, and computer program products enable the remote desktop client to conduct audio and/or video communication with a remote computing device in a peer-to-peer manner rather than via the remotely-executing communications application. In further accordance with such aspects, the methods, systems, and computer program products enable the remote desktop client to determine one or more hardware-based media processing capabilities of the client computing device and leverage such one or more hardware-based media processing capabilities in conducting the peer-to-peer audio and/or video communication with the remote computing device, which can improve the quality of such audio and/or video communication and reduce a processing burden on the client computing device. The one or more hardware-based media processing capabilities may be used, for example, to process media received from the remote computing device for rendering by the client computing device, to process media captured from a media source of the client computing device for transmission to the remote computing device, or as a basis for negotiating a media communication parameter with the remote computing device.

Further features and advantages of the invention, as well as the structure and operation of various aspects, are described in detail below with reference to the accompanying drawings. It is noted that the aspects are not limited to the specific aspects described herein. Such aspects are presented herein for illustrative purposes only. Additional aspects will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.

The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate aspects of the present application and, together with the description, further serve to explain the principles of the aspects and to enable a person skilled in the pertinent art to make and use the aspects.

The invention relates to a system and a method as set forth in the claims.

The present specification and accompanying drawings disclose one or more aspects that incorporate the features of the present invention. The scope of the present invention is not limited to the disclosed aspects. The disclosed aspects merely exemplify the present invention, and modified versions of the disclosed aspects are also encompassed by the present invention. Aspects of the present invention are defined by the claims appended hereto.

References in the specification to "one aspect," "an aspect," "an example aspect," etc., indicate that the aspect described may include a particular feature, structure, or characteristic, but every aspect may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same aspect. Further, when a particular feature, structure, or characteristic is described in connection with an aspect, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other aspects whether or not explicitly described.

In the discussion, unless otherwise stated, adjectives such as "substantially" and "about" modifying a condition or relationship characteristic of a feature or features of an aspect of the disclosure, are understood to mean that the condition or characteristic is defined to within tolerances that are acceptable for operation of the aspect for an application for which it is intended.

Numerous exemplary aspects are described as follows. Aspects are described throughout this document, and any type of aspect may be included under any section/subsection. Furthermore, aspects disclosed in any section/subsection may be combined with any other aspects described in the same section/subsection and/or a different section/subsection in any manner.

Despite the many benefits afforded by virtualization technologies, some applications, however, may not be optimized for or well-supported in VDI environments. One such example is communications applications running in a VDI environment that enable real-time communications (RTC) between users (e.g., Microsoft® Teams, Slack®). RTC as described herein refers to near simultaneous exchange of information (e.g., voice, instant messaging, video, etc.,) over a network from the sender to the receiver with negligible latency.

For example, <FIG> depicts a block diagram of an example system <NUM> for enabling video and/or audio communication via a communications application executing in a cloud computing environment. As shown in <FIG>, system <NUM> includes a cloud computing environment <NUM> that includes a virtual desktop host <NUM>, a client device <NUM>, and a remote computing device <NUM>.

For illustration purposes, cloud computing environment <NUM> is shown to include only a single virtual desktop host <NUM> but may include any number of resources. For example, cloud computing environment <NUM> may be comprised of resources (e.g., servers) running within one or more cloud data centers and/or on-premises with respect to an enterprise or organization. Additionally, in aspects, cloud computing environment <NUM> may include any type and number of other resources including resources that facilitate communications with and between servers (e.g., network switches, networks, etc.), storage by servers (e.g., storage devices, etc.), resources that manage other resources (e.g., hypervisors that manage virtual machines to present a virtual operating platform for tenants of a multi-tenant cloud, etc.), and/or further types of resources. In some aspects, virtual desktop host <NUM> may comprise a remote server or virtual machine accessible in a data center or an on-premises server.

In addition, client computing device <NUM> and remote computing device <NUM>, although respectively pictured as a laptop and a smart phone, may be any type of a mobile or stationary computing device. Example mobile computing devices include but are not limited to a Microsoft ® Surface® device, a personal digital assistant (PDA), a thin client, a laptop computer, a notebook computer, a tablet computer such as an Apple iPad™, a netbook, a mobile phone, a portable gaming device, or a wearable computing device. Example stationary computing devices include but are not limited to a desktop computer or PC (personal computer), a gaming console, or a server.

As further shown in <FIG>, client computing device <NUM> is communicatively coupled to cloud computing environment <NUM> via a network <NUM>, and remote computing device <NUM> is communicatively coupled to cloud computing environment <NUM> via a network <NUM>. Networks <NUM> and <NUM> may comprise one or more networks such as local area networks (LANs), wide area networks (WANs), enterprise networks, the Internet, etc., and may include one or more of wired and/or wireless portions. In certain scenarios, network <NUM> and network <NUM> may comprise the same network.

Client computing device <NUM>, remote computing device <NUM>, and virtual desktop host <NUM> may include at least one network interface that enables communications over networks <NUM> and <NUM>. Examples of such a network interface, wired or wireless, include an IEEE <NUM> wireless LAN (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth™ interface, a near field communication (NFC) interface, etc. Further examples of network interfaces are described elsewhere herein.

Client computing device <NUM> also includes a remote desktop client that is configured to present within a user interface of client computing device <NUM> a user interface <NUM> of a communications application that provides for video and/or audio communication between users thereof and runs on virtual desktop host <NUM>. For instance, data associated with user interface <NUM> of the communications application may be transferred to client computing device <NUM> and rendered as a user interface in a screen display of client computing device <NUM>. Any interactions (e.g., mouse and/or keyboard inputs) that a user <NUM> may have with the user interface on client computing device <NUM> that is presenting user interface <NUM> of the communications application may be tracked and transmitted to virtual desktop host <NUM>.

More specifically, user <NUM> may initiate a video and/or audio communication with a user <NUM> of remote computing device <NUM> by interacting with the user interface of client computing device <NUM> (e.g., by clicking a "phone icon" next to a contact displayed in a user interface). This interaction may be provided to virtual desktop host <NUM>, and the communications application running on virtual desktop host <NUM> may initiate and perform the video and/or audio communication with user <NUM>.

In aspects, during the video and/or audio communication between user <NUM> and user <NUM>, device sources (such as a video camera) on client computing device <NUM> and remote computing device <NUM> may capture images appearing in front of the device sources. As depicted in <FIG>, video captured of users <NUM> and <NUM> may be transferred through virtual desktop host <NUM> to the computing device of the other user for presenting on a display screen of the other user's computing device. For instance, video of user <NUM> captured by the device source on remote computing device <NUM> may be transferred to virtual desktop host <NUM> and processed (e.g., decoded) on virtual desktop host <NUM> and rendered locally as part of user interface <NUM> of the communications application. In some aspects, screen scraping may be used to collect screen display data associated with the video of user <NUM> rendered in user interface <NUM> for rendering the video of user <NUM> in the user interface of client computing device <NUM> so that user <NUM> may contemporaneously view the video of user <NUM> as it is captured by a device source.

This video and/or audio communication approach described above with reference to <FIG> may create latency issues (especially for high resolution video and audio experiences) as video and/or audio communication between client computing device <NUM> and remote computing device <NUM> is transferred through network <NUM>, virtual desktop host <NUM>, and network <NUM>. Moreover, this approach may increase the amount of time and resources spent on processing the media. For example, when virtual desktop host <NUM> receives media, it may need to be decoded and when virtual desktop host <NUM> transmits the media, it may need to be encoded.

Implementing a peer-to-peer video and/or audio communication approach between client computing device <NUM> and remote computing device <NUM> may alleviate the latency issues discussed above as well as reduce the processing workload on virtual desktop host <NUM>. For example, <FIG> depicts a block diagram of an example system <NUM> for enabling peer-to-peer audio and/or video communication between a client computing device interacting with a communications application in the cloud and a remote computing device.

As described above with reference to <FIG>, user <NUM> may initiate a video and/or audio communication with user <NUM> of remote computing device <NUM> by interacting with the user interface of client computing device <NUM>. This interaction is provided to virtual desktop host <NUM> and prompts the communications application running on virtual desktop host <NUM> to initiate the media communication with remote computing device <NUM>. However, in <FIG>, the media communication call is intercepted and redirected to client computing device <NUM> to enable peer-to-peer media communication between client computing device <NUM> and remote computing device <NUM>. The remote desktop client of client computing device <NUM> is configured to receive the redirected communication from the communications application for the purposes of enabling peer-to-peer audio and/or video communication between client computing device <NUM> and remote computing device <NUM>.

Once client computing device <NUM> and remote computing device <NUM> are connected for the purposes of audio and/or video communication (a process that will be described in more detail below with reference to <FIG> and <FIG>), client computing device <NUM> and remote computing device <NUM> may communicate video and/or audio with each other via a network <NUM> (which may be the same as, similar to, or entirely different from networks <NUM> and <NUM> of <FIG>).

By enabling peer-to-peer audio and/or video communication between client computing device <NUM> and remote computing device <NUM>, media communication through virtual desktop host <NUM> is avoided, thereby reducing latency associated with such audio and/or video communication and also reducing a processing burden on virtual desktop host <NUM>. Furthermore, as part of shifting the media communication connection between client computing device <NUM> and remote computing device <NUM> to a peer-to-peer channel, aspects described herein also enable local hardware-based media processing capabilities (e.g., a hardware-based video or audio codec) of client computing device <NUM> to be discovered and leveraged by the remote desktop client to better carry out such communication. As used herein, the term media is to be understood to encompass at least video, audio or a combination thereof. Because such aspects can leverage the hardware-based media processing capabilities of client computing device <NUM>, the quality and efficiency of media processing by client computing device <NUM> can be improved as compared to a software-based media processing approach. Moreover, utilizing such hardware-based media processing capabilities of client computing device <NUM> can reduce a burden on a processor of client computing device <NUM> which will not need to run a corresponding software-based media processing capability instead.

<FIG> will now be described, with continued reference to <FIG>, to help illustrate the process of redirecting communication from the communications application running on virtual desktop host <NUM> to client computing device <NUM>. <FIG> depicts a block diagram of an example system <NUM> for enabling redirection of communication from a communications application for the purposes of enabling peer-to-peer audio and/or video communication between a client computing device and a remote computing device.

As shown in <FIG>, cloud computing environment <NUM> includes virtual desktop host <NUM> and a plugin <NUM>. As further shown in <FIG>, plugin <NUM> includes redirection shim(s) <NUM>. Redirection shim(s) <NUM> are configured to intercept communication (e.g., API calls) initiated by the communications application running on virtual desktop host <NUM> and redirect such calls to client computing device <NUM> via network <NUM>. For example, if the communications application initiates video and/or audio communication with remote computing device <NUM>, then redirection shim(s) <NUM> will intercept the video and/or audio communication request, modify the request if necessary (e.g., replacing certain script objects with different implementations), and redirect it to client computing device <NUM> over a communication channel <NUM> (via network <NUM>).

In an aspect, the communications application may load plugin <NUM> at runtime if particular conditions are met (e.g., if the communications application is running on a particular virtual desktop environment and if multi-session users is enabled). Once loaded, plugin <NUM> may attempt to establish communication channel <NUM> (e.g., establish a dynamic virtual channel (DVC)) with client computing device <NUM> to enable communication between virtual desktop host <NUM> and client computing device <NUM>. In aspects, plugin <NUM> may make Remote Procedure calls (RPCs) through a WebSockets server process to client computing device <NUM>.

To provide a more detailed perspective of client computing device <NUM>, <FIG> will now be described. <FIG> depicts client computing device <NUM> including a plugin <NUM> to a remote desktop client <NUM> that enables peer-to-peer communication between client computing device <NUM> and a remote computing device, determines a hardware-based media processing capability of client computing device <NUM>, and uses the detected hardware-based media processing capability to process media. Plugin <NUM> comprises a set of components <NUM> that are non-OS specific and compatible with any platform and a set of components <NUM> that are OS specific and would need to be reimplemented for each platform. Set of components <NUM> that are non-OS specific comprises: an RTC listener <NUM>, an RTC manager <NUM>, a window manager <NUM>, and an RTC component <NUM>.

As further shown in <FIG>, client computing device <NUM> also includes a remote desktop client <NUM>. As previously described, remote desktop client <NUM> of client computing device <NUM> is configured to receive redirected communication on communication channel <NUM> from the communications application (i.e., through plugin <NUM> of cloud computing environment <NUM> in <FIG>) for the purposes of enabling peer-to-peer audio and/or video communication between client computing device <NUM> and a remote computing device (e.g., remote computing device <NUM> in <FIG>).

In the process of enabling peer-to-peer communication between client computing device <NUM> and remote computing device <NUM>, RTC listener <NUM> is configured to listen on communication channel <NUM> for the redirected communication and provide a stream of the redirected communication to RTC manager <NUM> once received. RTC manager <NUM> is configured to receive the stream of redirected communication (e.g., including serialized RPC call information) and translate it into a format compatible with a framework for enabling real-time communication (e.g., WebRTC). RTC manager <NUM> then provides the translated communication (e.g., WebRTC API calls) to RTC component <NUM>. RTC component <NUM> is configured to connect client computing device <NUM> and remote computing device <NUM> for the purposes of audio and/or video communication based on the translated communication (e.g., WebRTC API calls) provided from RTC manager <NUM>. RTC component <NUM> may establish a communication channel <NUM> between client computing device <NUM> and remote computing device <NUM> for the purposes of audio and/or video communication.

Set of components <NUM> of plugin <NUM> that are OS specific comprise: a media capture component <NUM>, a media renderer component <NUM>, and a window manager component <NUM>. After client computing device <NUM> and remote computing device <NUM> are connected for audio and/or video communication, media capture component <NUM> and media renderer component <NUM> are used to detect a hardware-based media processing capability of client computing device <NUM> and use the hardware-based media processing capability to optimize audio and/or video communication between client computing device <NUM> and remote computing device <NUM>.

Media capture component <NUM> determines a hardware-based media processing capability of client computing device <NUM> by using OS-specific APIs to determine what is available on client computing device <NUM>. After determining what is available on client computing device <NUM>, media capture component <NUM> is configured to inform a source reader <NUM> of any information (such as formatting) related to processing of captured media and any configuring of hardware needed to leverage the hardware-based media processing capability to process captured media. Source reader <NUM> is configured to capture media from a video camera <NUM> and other device sources (e.g., a microphone) of client computing device <NUM>.

To help illustrate, say media capture component <NUM> determines that client computing device <NUM> has a hardware-based media processing capability to transcode captured video to a lower resolution/quality. Media capture component <NUM> may inform source reader <NUM> that the original captured media needs to be decoded to an intermediate uncompressed format (e.g., PCM for audio or YUV for video) and then encoded into a target format. Source reader <NUM> may then perform the transcoding and provide the processed video to media capture component <NUM>. In capturing video, source reader <NUM> may act like a frame server and stream captured video to media capture component <NUM>.

Media capture component <NUM> is further configured to receive from source reader <NUM> processed and/or unprocessed media captured from video camera <NUM> and provide the captured media to RTC component <NUM>. RTC component <NUM> is configured to prepare the captured media for transmission to remote computing device <NUM>. As shown in <FIG>, RTC component <NUM> may send the captured media through a socket API <NUM> over communication channel <NUM> to remote computing device <NUM>.

Media renderer component <NUM> may determine a hardware-based media processing capability of client computing device <NUM> by using OS-specific APIs to determine what is available on client computing device <NUM>. After determining what is available on client computing device <NUM>, media renderer component <NUM> is configured to inform a media engine <NUM> of any information (such as formatting) related to processing of received media and any configuring of hardware needed to leverage the hardware-based media processing capability to process received media.

Media renderer component <NUM> is further configured to provide a specific source of media received from RTC component <NUM> to media engine <NUM>. Media engine <NUM> will then build up a media pipeline and set it up for rendering. Media engine <NUM> is configured to provide the specific source (compressed or decompressed) to a media stream source <NUM>. Media stream source <NUM> is configured to provide the media source to hardware needed to leverage the hardware-based media processing capability to process the received media. In an aspect, media stream source <NUM> may use a frame server approach for producing video media.

As shown in <FIG>, media stream source <NUM> is connected to a hardware decoder <NUM>. In some aspects, if the received media is compressed, then hardware decoder <NUM> may decode the received media. Additionally, media stream source <NUM> may provide the received media to other hardware-based media processing capabilities, such as hardware-accelerated video processing (e.g., resizing the frames, color conversions, etc.,) or hardware-accelerated audio processing. As depicted in <FIG>, the processed media may be provided to remote desktop client <NUM> for rendering in its user interface <NUM>. Alternatively or additionally, the processed media can be rendered at a display <NUM>. Window manager <NUM> and window manager component <NUM> are configured to monitor and track any changes (e.g., window movements, window resizing, pop-up menus, etc.,) needed to be performed on video rendered in user interface <NUM>.

Client computing device <NUM> is described in further detail as follows with respect to <FIG>. <FIG> shows a flowchart <NUM> of a method for enabling peer-to-peer communication between a client computing device and a remote computing device, determining a hardware-based media processing capability of a client computing device, and using the detected hardware-based media processing capability to process media.

Flowchart <NUM> begins with step <NUM>. In step <NUM>, a redirected communication is received from the communications application for purposes of enabling peer-to-peer audio and/or video communication between the client computing device and a remote computing device as opposed to audio and/or video communication via the communications application. For example, and with continued reference to <FIG>, remote desktop client <NUM> receives a redirected communication on communication channel <NUM> from the communications application for purposes of enabling peer-to-peer audio and/or video communication between client computing device <NUM> and remote computing device <NUM> as opposed to audio and/or video communication via the communications application.

In step <NUM>, the client computing device and the remote computing device are connected for the purposes of audio and/or video communication based on the redirected communication using a framework for enabling real-time communication. For example, and with continued to <FIG>, RTC component <NUM> connects client computing device <NUM> and remote computing device <NUM> for the purposes of audio and/or video communication based on the redirected communication using a framework for enabling real-time communication. In one aspect, the redirected communication includes WebRTC API calls intercepted from the communications application and APIs and protocols of WebRTC is the framework used for connecting client computing device <NUM> and the remote computing device <NUM>.

In step <NUM>, a hardware-based media processing capability of the client computing device is determined. For example, and with continued to <FIG>, media renderer component <NUM> determines a hardware-based media processing capability of client computing device <NUM> by using OS-specific APIs to determine what is available on client computing device <NUM>. Such hardware-based media processing capability of client computing device <NUM> may comprise, for example and without limitation, a video codec (comprising a video decoder), an audio codec (comprising an audio decoder), hardware-accelerated video processing, or hardware-accelerated audio processing.

In step <NUM>, media transmitted from the remote computing device is received. For example, and with continued to <FIG>, media renderer component <NUM> receives media transmitted from remote computing device <NUM> from RTC component <NUM>.

In step <NUM>, the hardware-based media processing capability is used to process the received media from the remote computing device for rendering by the client computing device. For example, and with continued to <FIG>, media stream source <NUM> uses the hardware-based media processing capability to process the received media from remote computing device <NUM> for rendering by client computing device <NUM>. To help illustrate, media stream source <NUM> is connected to a hardware decoder <NUM>. In some aspects, if the received media is compressed, then hardware decoder <NUM> may decode the received media. Additionally, media stream source <NUM> may provide the received media to other hardware-based media processing capabilities, such as hardware-accelerated video processing (e.g., resizing the frames, color conversions, etc.,) or hardware-accelerated audio processing.

<FIG> shows a flowchart <NUM> of a method for negotiating a media communication parameter with a remote computing device based on a determined hardware-based media processing capability of a client computing device. Flowchart <NUM> begins with step <NUM>. In step <NUM>, another hardware-based video processing capability of the client computing device is determined. For example, and with continued to <FIG>, RTC manager <NUM> may determine another hardware-based video processing capability of client computing device <NUM>. RTC manager <NUM> may determine the other hardware-based media processing capability through an API of the framework for enabling real-time communication.

In step <NUM>, a media communication parameter is negotiated with the remote computing device based on the other hardware-based media processing capability. For example, and with continued to <FIG>, RTC manager <NUM> negotiates a media communication parameter with remote computing device <NUM> based on the other hardware-based media processing capability. For example, RTC manager <NUM> may determine that client computing device <NUM> has a particular decoder (e.g., VP9 decoder) that remote computing device <NUM> does not have. RTC manager <NUM> may negotiate with remote computing device <NUM> to provide media in a format compatible with its particular decoder.

<FIG> shows a flowchart <NUM> of a method for determining a hardware-based media processing capability of a client computing device and using the hardware-based media processing capability to process media captured from a media source of the client computing device. Flowchart <NUM> begins with step <NUM>. In step <NUM>, another hardware-based media processing capability of the client computing device is determined. For example, and with continued to <FIG>, media capture component <NUM> determines another hardware-based media processing capability of client computing device <NUM>. Media capture component <NUM> may determine available hardware-based media processing capability of client computing device <NUM> by using APIs specific to the OS of client computing device <NUM>. Such hardware-based media processing capability of client computing device <NUM> may comprise, for example and without limitation, a video codec (comprising a video encoder), an audio codec (comprising an audio encoder), hardware-accelerated video processing, or hardware-accelerated audio processing.

In step <NUM>, the other hardware-based media processing capability is used to process media captured from a media source of the client computing device. For example, a hardware-based video or audio encoder may be used to encode video or audio, respectively, captured from a media source of client computing device <NUM>. As another example, hardware-accelerated video or audio processing may be used to accelerate the processing of video or audio, respectively, captured from a media source of client computing device <NUM>. In some aspects, and with continued to <FIG>, media capture component <NUM> is configured to inform source reader <NUM> of any information (such as formatting) related to processing of captured media and any configuring of hardware needed to leverage the hardware-based media processing capability to process captured media.

In step <NUM>, the processed media is transmitted to the remote computing device. For example, and with continued to <FIG>, RTC Component <NUM> transmits the processed media to remote computing device <NUM> over communication channel <NUM>.

Although the foregoing aspects refer to a communications application executing on a virtual desktop host, it is to be understood that the techniques described herein could be utilized in conjunction with any communications application that is executing remotely from client computing device <NUM>, regardless of whether such communications application is running on a virtual desktop or some other type of platform. That is to say, the aspects described herein are not limited to communications applications being executed on a virtual desktop but instead broadly encompass any type of communications application that may execute on a server and that enable audio and/or video communication between remotely-located users and user devices.

Furthermore, although the foregoing aspects refer to a remote desktop and associated plugin running on client computing device <NUM>, it is to be understood that the functionality provided by those components could be encapsulated within any type of software components having any type of structure or architecture. For example, the functions and features of plugin <NUM> may be included in a software component that isn't necessarily a plugin but is instead some other type of software component that is operable to work with remote desktop client <NUM>. Furthermore, the functions and features of plugin <NUM> may also be integrated directly into remote desktop client <NUM> such that an additional software component is not required beyond remote desktop client <NUM>. Still further, the function and features of remote desktop client <NUM> and plugin <NUM> may be included in an application or operating system component of client computing device <NUM> that isn't referred to as a remote desktop client or plugin but nevertheless provides similar capabilities to those components as were described herein.

Cloud computing environment <NUM>, virtual desktop host <NUM>, plugin <NUM>, client computing device <NUM>, remote desktop client <NUM>, plugin <NUM>, flowchart <NUM>, flowchart <NUM>, and/or flowchart <NUM> may be implemented in hardware, or hardware combined with one or both of software and/or firmware. For example, cloud computing environment <NUM>, virtual desktop host <NUM>, plugin <NUM>, client computing device <NUM>, remote desktop client <NUM>, plugin <NUM>, flowchart <NUM>, flowchart <NUM>, and/or flowchart <NUM> may be implemented as computer program code/instructions configured to be executed in one or more processors and stored in a computer readable storage medium. In another aspect, cloud computing environment <NUM>, virtual desktop host <NUM>, plugin <NUM>, client computing device <NUM>, remote desktop client <NUM>, plugin <NUM>, flowchart <NUM>, flowchart <NUM>, and/or flowchart <NUM> may also be implemented in hardware that operates software as a service (SaaS) or platform as a service (PaaS). Alternatively, cloud computing environment <NUM>, virtual desktop host <NUM>, plugin <NUM>, client computing device <NUM>, remote desktop client <NUM>, plugin <NUM>, flowchart <NUM>, flowchart <NUM>, and/or flowchart <NUM> may be implemented as hardware logic/electrical circuitry.

For instance, in an aspect, one or more, in any combination, of cloud computing environment <NUM>, virtual desktop host <NUM>, plugin <NUM>, client computing device <NUM>, remote desktop client <NUM>, plugin <NUM>, flowchart <NUM>, flowchart <NUM>, and/or flowchart <NUM> may be implemented together in a system on a chip (SoC). The SoC may include an integrated circuit chip that includes one or more of a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and/or further circuits, and may optionally execute received program code and/or include embedded firmware to perform functions.

<FIG> depicts an exemplary implementation of a computing device <NUM> in which aspects may be implemented. For example, components of cloud computing environment <NUM>, virtual desktop host <NUM>, plugin <NUM>, client computing device <NUM>, remote desktop client <NUM>, plugin <NUM> may each be implemented in one or more computing devices similar to computing device <NUM> in stationary or mobile computer aspects, including one or more features of computing device <NUM> and/or alternative features. The description of computing device <NUM> provided herein is provided for purposes of illustration, and is not intended to be limiting. Aspects may be implemented in further types of computer systems, as would be known to persons skilled in the relevant art(s).

A number of program modules may be stored on the hard disk, magnetic disk, optical disk, ROM, or RAM. These programs include operating system <NUM>, one or more application programs <NUM>, other programs <NUM>, and program data <NUM>. Application programs <NUM> or other programs <NUM> may include, for example, computer program logic (e.g., computer program code or instructions) for implementing virtual desktop host <NUM>, plugin <NUM>, remote desktop client <NUM>, plugin <NUM>, flowchart <NUM>, flowchart <NUM>, and/or flowchart <NUM> (including any suitable step of flowcharts <NUM>, <NUM>, and <NUM>), and/or further aspects described herein.

Display screen <NUM>, and/or any other peripheral output devices (not shown) may be used for implementing user interfaces <NUM> and <NUM>, and/or any further aspects described herein.

Aspects are also directed to such communication media that are separate and non-overlapping with aspects directed to computer-readable storage media.

Such computer programs, when executed or loaded by an application, enable computing device <NUM> to implement features of aspects discussed herein.

Aspects are also directed to computer program products comprising computer code or instructions stored on any computer-readable medium.

Claim 1:
A client computing device (<NUM>), comprising:
one or more processors;
one or more memory devices that store computer program logic for execution by the one or more processors, the computer program logic comprising:
a remote desktop client (<NUM>) that is configured to present a user interface of a communications application executing in a cloud computing environment within a user interface of the client computing device (<NUM>), wherein a user interaction with the user interface prompts the communications application to initiate communication to a remote computing device (<NUM>), the remote desktop client (<NUM>) being further configured to receive redirected communication to the remote computing device (<NUM>) from the communications application for the purposes of enabling peer-to-peer audio and/or video communication between the client computing device (<NUM>) and the remote computing device (<NUM>) as opposed to audio and/or video communication via the communications application; and
a plugin (<NUM>), comprising:
a real-time communication manager (<NUM>) configured to receive the redirected communication to the remote computing device (<NUM>) from the remote desktop client (<NUM>) and translate the redirected communication to the remote computing device (<NUM>) into a format compatible with a framework for enabling real-time communication;
a real-time communication component (<NUM>) configured to connect the client computing device (<NUM>) and the remote computing device (<NUM>) for the purposes of audio and/or video communication based on the translated communication; and
a media capture component (<NUM>) configured to determine a hardware-based media processing capability of the client computing device (<NUM>) by using an application programming interface, API, specific to the operating system, OS, of the client computing device, wherein the hardware-based media processing capability comprising one of a video codec, an audio codec, hardware-accelerated video processing or hardware-accelerated audio processing, use the hardware-based media processing capability to process media captured from a media source of the client computing device (<NUM>), and provide the processed media to the real-time communication component (<NUM>) to be transmitted to the remote computing device (<NUM>).