Patent Description:
It is with respect to this general technical environment that aspects of the present technology disclosed herein have been contemplated. Furthermore, although a general environment has been discussed, it should be understood that the examples described herein should not be limited to the general environment identified in the background.

<CIT> describes techniques for remote operation of applications using received data.

<CIT> describes techniques for displaying images from a plurality of information processing apparatus concurrently.

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description section. Additional aspects, features, and/or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description or may be learned by practice of the disclosure.

Non-limiting examples of the present disclosure describe systems, methods and devices for assisting with cross-platform drag and drop functionality. A common scenario related to multi-device use cases is the ability to "remote access" one device from another device and interact with the native experience (e.g., remote desktop or visual network computing). In examples, a user may link a host device with a remote device via a device mirroring application and a network (e.g., Bluetooth network, local area network, WiFi network, etc.). The remote device user interface may thus be mirrored on the host device, while allowing the user to interact with the remote device via the host device's native controls. According to examples described herein a user may perform a drag and drop operation that drags an object mirrored on the device mirroring application of the host device to a construct of the host device (e.g., an application, a desktop) where the object is dropped. The object may be copied from the remote device and transferred to the host device. In some examples, the object may be translated from its native format to one or more formats that are compatible with the host device and/or a construct of the host device where the object is dropped.

The invention relates to a computer-implemented method and a system as set forth in the claims. It will be understood that aspects of the disclosure falling outside of the scope of the claims may not be part of the invention but may be useful to understand the invention.

Non-limiting and non-exhaustive examples are described with reference to the following figures:.

Various aspects will be described in detail with reference to the drawings, wherein like reference numerals represent like parts and assemblies throughout the several views. Reference to various aspects does not limit the scope of the claims attached hereto. Additionally, any examples set forth in this specification are not intended to be limiting and merely set forth some of the many possible aspects for the appended claims.

Examples of the disclosure provide systems, methods, and devices for assisting with cross-platform drag and drop functionality utilizing a platform mirroring application. As used herein, "platform" describes a specific environment comprised of a set of computer hardware and an operating system on which software is executed. For example, a first computing device may be said to run on a first platform that is different than a second platform if it has a unique set of one or both of: hardware components and/or an operating system (including an operating system version), and a second computing device may be said to run on a second platform that is different from the first platform if it has a set of one or both of: hardware components and/or an operating system (including an operating system version), that is different than the first computing device.

A "host device" as described herein is a device that a user may control and/or view a copy, in real or near real-time, of a user interface of a "remote device". A "remote device" as described herein is a device that may have its user interface copied, in real or near real-time, on a device mirroring application of a host device. The remote device may be controlled via the native controls of the host device via the device mirroring application.

According to examples, a host device and a remote device may each run a device mirroring application. The mirroring application may run in the background of the remote device, while a window mirroring the user interface of the remote device may be displayed on a user interface of the host device. The remote device and the host device may connect with one another (e.g., via Bluetooth, via network connection, via Wi-Fi, etc.). A user may initiate a drag and drop operation via the host device and the mirroring application. For example, a user may left click, touch and hold, etc. on an object in the device mirroring application. In some examples, the object may be currently displayed on the user interface of the remote device. In other examples, the display of the remote device may be non-active or in an off state, but the object would be displayed if the display were transitioned to an active, or on, state. This "start" of a drag and drop operation is described herein as a "drag-initiating input". When the drag initiating input is received at the mirroring application on the host device, an indication of that input may be sent by the host device to the remote device. The remote device may then copy the object corresponding to the object element that the drag input was received in relation to on the host device and save that object to a temporary storage location on the remote device. In some examples, the temporary storage location may comprise an electronic clipboard. In other examples, the object may be made available for transfer to the host device and/or saved to other storage locations (e.g., via a user interface accessibility tree, via an application-specific API, via a clipboard manager application, etc.).

As the object element is being dragged/moved across the device mirroring application on the host device, the remote device may receive an indication of that movement from the host device and move the display of its corresponding object in relation to that movement in real time or near real time. When the object element in the device mirroring application reaches the border of the device mirroring application and the native constructs of the host device, an indication of that border being reached may be sent from the host device to the remote device. The remote device may transfer the copied object in temporary storage to the host device. The host device may store the copied object in its own temporary storage location. In examples, the temporary storage location on the host device may comprise an electronic clipboard. In other examples, the object may be stored in a different temporary storage location (e.g., a temporary storage location associated with a clipboard manger application, a temporary storage location associated with a different application, etc.). When the drag operation being performed on the host device causes the object element to cross the border between the device mirroring application and the native constructs of the host device, the object may be caused to be surfaced on the native constructs of the host device, while disappearing from the device mirroring application. An indication that the object element has moved over into the host device's native constructs may be sent to the remote device, and the remote device may then no longer display all of the corresponding object and/or the portion of the corresponding object that has crossed the border between the device mirroring application and the native constructs of the host device.

A translation engine associated with the host device may determine whether the object copied to temporary storage on the host device is in a format that is compatible with the host device and/or a construct that the object is being dropped into. If the object is in a format that is compatible with the host device, no translation may be necessary, and the object may be dropped into the construct of the host device in the format it was received by the host device in. If the object is not in a format that is compatible with the host device, the translation engine may translate the object into one or more formats that are compatible with the host device. In additional examples, the translation engine may identify a secondary source location of an object corresponding to the object element that is being dropped (e.g., if the translation engine is incapable of translating the object into a compatible format; if network conditions are poor; depending on a size of the object). When the object is dropped in the host device construct, the translation engine and/or the device mirroring application may convert the user interface element corresponding to the object into a link or shortcut where the object can be located at the secondary source. When a drop command (e.g., release of left click, removal of touch input in relation to object element) is received at the host device over the host device's native construct (e.g., over an application or shell construct), the object may be inserted, embedded and/or saved to the location where the drop command was received. In some examples, the object may be deleted from the remote device. In other examples, the object may be retained at the remote device.

The systems, methods, and devices described herein provide technical advantages for transferring objects across computing devices operating on different platforms. Processing costs (i.e., CPU cycles) associated with transferring objects from a remote device operating on a first platform to a host device operating on a second platform are reduced in that an object does not need to be manually converted to a compatible file type for reading by the host device. Rather, a translation engine may identify compatible format types across devices and applications and automatically translate an object from a first, non-compatible format, to one or more compatible formats. Additionally, rather than requiring that an object from a remote device be copied to a removable storage device (e.g., a USB drive) and physically transferred via that removable storage device, the current mechanisms allow for object transfer interactions across different platforms via wireless communication and a drag and drop mechanism that is familiar to users. Further, whereas most solutions related to cross platform object transfers only provide for limited data payload types to be transferred (e.g. limited image types, plain text), the mechanisms described herein may be applied to a vast number of payload types due to the utilization of a translation engine. Additionally, the mechanisms described herein provide for a more secure transfer of objects across different platforms, as objects may be transferred directly from a first platform device to a second platform device, whereas even limited payload solutions in the field had traditionally required that an object be saved to the cloud.

<FIG> is a schematic diagram illustrating an example distributed computing environment <NUM> for assisting with cross-platform drag and drop functionality utilizing a platform mirroring application. Computing device <NUM> includes first drag and drop sub-environment <NUM> and second drag and drop sub-environment <NUM>. First drag and drop sub-environment <NUM> includes remote computing device <NUM> and host computing device 112A. Remote computing device <NUM> runs on a first platform and host computing device 112A runs on a second platform that is different from the first platform. Second drag and drop sub environment <NUM> includes host computing device 112B, which is the same device as host computing device 112B after a drag and drop operation has been completed.

Remote computing device <NUM> and host computing device 112A may establish a connection with one another. The connection may be a network connection, a Bluetooth connection, a Wi-Fi connection, a local area network connection, etc. In some examples, the connection may be encrypted. Remote computing device <NUM> and host computing device 112A are each executing a device mirroring application. The device mirroring application provides mechanisms for mirroring, in real time or near real time, the user interface of remote computing device <NUM> on the user interface of host computing device 112A. Various operations may be performed in the constructs of the device mirroring application on host computing device <NUM> to control and/or interact with operations and content on remote computing device 112A. The device mirroring application is illustrated on computing device 112A at device mirroring application window <NUM>. The device mirroring application may run in the "background" of remote computing device <NUM>. That is, there need not be an application window open on remote computing device <NUM> for it to perform one or more of the operations described herein.

Remote computing device <NUM> displays a plurality of objects on its user interface. Those objects are object <NUM>104A object <NUM><NUM>, object <NUM><NUM> and object <NUM><NUM>. That user interface is mirrored on device mirroring application window <NUM> on the user interface of host computing device 112A via the connection between the two devices. Mirroring application window <NUM> is positioned adjacent to and/or partially overlapping native application window <NUM> on host computing device 112A. Native application window <NUM> displays an open application (Application A) that is native to host computing device 112A. There are two objects displayed by Application A - object <NUM><NUM> and object <NUM><NUM>.

In some examples, a user may want to move one or more objects (e.g., file objects, folder objects, application objects, document objects, image objects, video objects) from remote computing device <NUM> to host computing device 112A, or from host computing device 112A to remote computing device <NUM>. One of the most user-friendly means for accomplishing the movement and/or transition of objects from a first location to a second location on a computing device is the drag and drop interaction whereby an object is selected, a user interface element corresponding to the object is dragged to a location where the user would like to save and/or embed the object (e.g., in a desktop shell construct, in a file browsing shell construct, in an application construct), the user interface element corresponding to the object is unselected (i.e., "dropped"), that user interface element is converted to a shortcut to the object in that new location (or the object is directly embedded in an application), and the object file is saved to a location corresponding to the drop. However, remote computing device <NUM> and host computing device 112A may run different operating systems, which may not have one-to-one file navigation and/or execution compatibility. Additionally, one or more constructs that an object may be dragged to on host computing device 112A may not support the object type of an object from remote computing device <NUM>.

The systems, methods and devices described herein provide mechanisms for overcoming the hurdles described above with regard to cross-platform drag and drop experiences. In the illustrated example, a user initiates, via a drag-initiating input, a drag and drop action of object <NUM>104A from device mirroring application window <NUM>. As described herein, a drag-initiating input may comprise one or more of: a left click and hold on an object, a selection of an object and subsequent movement of the object, a touch and hold on an object, and/or a verbal command to initiate a drag of an object, for example. A user interface element corresponding to object <NUM>104A is then moved/dragged from device mirroring application window <NUM> to Application A as illustrated by object <NUM>104B in second drag and drop sub-environment <NUM>. When the user interface element corresponding to object <NUM>104B is dropped (e.g., the left click is released, the touch and hold is removed from the display of computing device 112B), object <NUM>104B is embedded directly into Application A along with object <NUM> and object <NUM>, object <NUM>104B is saved to a location corresponding to the file open in Application A where object <NUM>104B was dropped, and/or a user interface element corresponding to object <NUM>104B is embedded into Application A which is converted to a shortcut to a file corresponding to object <NUM>104B. Specific mechanisms by which these operations are accomplished are more fully discussed in relation to <FIG> below.

<FIG> is a schematic diagram illustrating an example distributed computing environment <NUM> for assisting with cross-platform drag and drop functionality, starting with a drag-initiating input and continuing through a drag in a platform mirroring application. <FIG>, <FIG> and <FIG> include drag and drop element numbers (e.g., <NUM>, <NUM>, <NUM>(a), <NUM>(b)). The drag and drop element numbers are provided for ease of illustration, but it should be understood that the operations associated with those element numbers may be, but are not necessarily, performed in the numbered and/or lettered order.

Distributed computing environment <NUM> includes platform A <NUM>, which is representative of a first computing device that operates on a first computing platform, and platform B, which operates on a second computing platform that is different than the first computing platform. Platform A includes an object (Object <NUM>206A) on its user interface. Platform A <NUM> also includes temporary storage <NUM>, which may be utilized to temporarily store data associated with (e.g., generated by, downloaded by, received by, sent by, copied by) platform A <NUM>. Platform A also includes (although not shown) a platform mirroring application that may be running in the background and performing one or more of the operations described herein. Platform B includes platform A mirroring application <NUM>. Platform A and platform B are communicatively connected (e.g., via Bluetooth, via network connection, via Wi-Fi). In some examples, platform A <NUM> may transfer, via its platform mirroring application, an indication of content that is being currently displayed on the user interface of platform A <NUM>. In other examples, if platform A <NUM> is in a non-active, or off, state, platform A <NUM> may transfer, via its platform mirroring application, an indication of content that would be currently displayed on the user interface of platform A <NUM> if it was in an active, or on, state. That information may be received by platform A mirroring application <NUM>, which may then cause platform B <NUM> to display a mirrored copy of the user interface (or if in a non-active state, what the user interface would appear like if in an active state) of platform A in a window corresponding to platform A mirroring application <NUM>. Thus, platform B <NUM> displays object <NUM>206C in platform A mirroring application <NUM>.

In this example, a user initiates a drag and drop of object <NUM>206C from platform B <NUM>, and specifically, from platform A mirroring application <NUM>. The user may initiate the drag of object <NUM>206C via various mechanisms (e.g., a left mouse click and hold on the object, a touch and hold on the object, a touch hold and move on the object). An indication that this "triggering event", herein referred to as a drag-initiating input, is sent from platform B to Platform A as indicated by drag and drop element <NUM>. Platform A receives that indication, and copies object <NUM>206A to temporary storage <NUM>, as illustrated by object <NUM>206B and by drag and drop element <NUM>(a). When object <NUM>206C is moved/dragged in association with the drag-initiating input (as illustrated by drag and drop element <NUM>(b)), platform A receives that indication from platform B, and the user interface element corresponding to object <NUM>206A is moved on the user interface of platform A in accordance with that movement/dragging, as illustrated by drag and drop element <NUM>(b*). In some examples, the user interface element corresponding to an object that is being dragged may be highlighted, shadowed, and/or otherwise indicated as being in a drag state while the drag event is occurring. Additional details regarding a drag and drop flow according to the aspects described herein are provided in relation to <FIG>.

<FIG> is a schematic diagram illustrating an example distributed computing environment <NUM> for assisting with cross-platform drag and drop functionality, starting from an object UI element being dragged through a platform mirroring application boundary between the mirroring application and a native application and/or shell construct. In this example, the drag of object <NUM>306C has begun and object <NUM>306C has met a boundary of the platform A mirroring application <NUM>, as illustrated by drag and drop element <NUM>(a). Platform B <NUM> then sends an indication to platform A <NUM> that object <NUM>306C has reached the boundary of platform A mirroring application <NUM>, as illustrated by drag and drop element <NUM>(b). Any portion of object <NUM>306C that crosses that boundary may be caused to be hidden on the user interface of platform A, as indicated by drag and drop element <NUM>(c). Additionally, any portion of object <NUM>306C that crosses that boundary over into the native constructs of platform B <NUM> (e.g., a native application, a desktop, a file browsing construct) may be caused to be displayed as being overlaid those constructs, as illustrated by drag and drop element <NUM>(d). Thus, in some examples, a portion of object <NUM>306C may be displayed in platform A mirroring application <NUM> while a portion over the boundary may be displayed in the native constructs of platform B <NUM>. Alternatively, in some examples, once object <NUM>306C crosses the boundary, the entirety of object <NUM>306C may be caused to be displayed in the native constructs of platform B <NUM> and removed from the display on platform A mirroring application <NUM>. When platform A receives an indication that object <NUM>306C has met and/or gone over the boundary between platform A mirroring application <NUM> and the native constructs of platform B <NUM>, object <NUM>306B in temporary storage <NUM> may be sent to temporary storage <NUM> of platform B, as indicated by object <NUM>306D in temporary storage <NUM>, and illustrated by drag and drop element <NUM>(e).

In some examples, object <NUM>306D may be stored in a format that is not compatible with the construct it is being dragged to in platform B <NUM> (e.g., an application construct, a shell construct). For example, if object <NUM>306C is being dragged over a word processing application that is native to platform B, and object <NUM>306D is a word processing document that is in a format that is native to platform A, platform B may not be capable of reading/editing object <NUM>306D in its current format. One or more operations associated with translation engine <NUM> executed by platform B may be applied to object <NUM>306D. Translation engine <NUM> may perform operations associated with identifying one or more formats of an object, determining one or more formats that are readable/executable by a target application (i.e., an application where an object is being dropped), and translating an object into one or more formats that are compatible with a target application. In additional examples, if translation engine <NUM> cannot identify a compatible format that it is capable of translating an object into, translation engine <NUM> may identify a corresponding object from a secondary source (e.g., an app store, an cloud storage platform, a web application, etc.) and provide a shortcut and/or link to that source. In this example, translation engine <NUM> is capable of translating object <NUM>306D to a format that is compatible with the construct that object <NUM>306C is being dragged over/into in platform B. As such, translation engine <NUM> translates object <NUM>306D into that format, as illustrated by drag and drop element <NUM>(f). In some examples, translation engine <NUM> may translate the object into one or more compatible formats. Additional details regarding a drag and drop flow according to the aspects described herein are provided in relation to <FIG>.

<FIG> is a schematic diagram illustrating an example distributed computing environment <NUM> for assisting with cross-platform drag and drop functionality, starting from a drop object command in the native constructs of a computing device and continuing through the saving of that object to the computing device. In this example, object <NUM>406D has been translated by translation engine <NUM> into a format that is compatible with the construct it has been dragged into in platform B <NUM>. Object <NUM>406C is dropped in that native construct of platform B (e.g., it is dropped in an application or shell construct of platform B), as illustrated by drag and drop elements <NUM> and <NUM>. In some examples, the object may not be translated until the drop command has been received. In other examples, the object may be translated prior to the drop command being received. The drop command may be a release of a mouse click, a removal of a touch on the display of the device of platform B, etc. Object <NUM>406D may then be embedded into the construct of platform B <NUM> where it was dropped. In some examples, a user interface element corresponding to the object may be converted to a shortcut to the object file and inserted at the location where the drop command was received. In additional examples, the object may be saved to a location corresponding to the location where the drop command was received, as indicated by object <NUM>406E in platform B application/shell construct <NUM>, and as illustrated by drag and drop element <NUM>. In some examples, the object that was dropped in platform B may be deleted from platform A. In other examples, the object that was dropped in platform B may be retained by platform A.

<FIG> is an exemplary method 500A for assisting with cross-platform drag and drop functionality from the host computing device. The method 500A begins at a start operation and flow continues to operation 502A where a connection with a second device is established by a first device. In this example, the first device is the host device and the second device is the remote device. In some examples, the connection may be automatically established based on a same user account be signed into a device mirroring application that is running on the first device and a device mirroring application that is running on the second device. In additional examples, the connection may be established via a network setting associated with the user account. The connection may be one of: a WiFi connection, a Bluetooth connection, a LAN connection, etc..

From operation 502A flow continues to operation 504A where a mirroring application that mirrors the user interface of the second device and partially overlaps the user interface of the first device is surfaced on a user interface of the first device. In examples where the first device is connected to one or more external and/or secondary display devices the mirroring application may be surfaced on one of the external and/or secondary display devices. In some examples, the user interface of the second device may be in an active, or on, display state when it is mirrored by the first device. In other examples, the user interface of the second device may be in an inactive, or off, display state when it is mirrored by the first device.

From operation 504A flow continues to operation 506A where a drag-initiating input of a user interface element corresponding to an object of the second device is received. The drag-initiating input may comprise one or more of: a left click and hold on an object, a selection of an object and subsequent movement of the object, a touch and hold on an object, and/or a verbal command to initiate a drag of an object, for example. In some examples, the drag-initiating input may initiate the copying of the object to temporary storage by the remote device. A temporary storage location where the object is copied to may be a clipboard, an application-specific storage location, a cloud location, or another temporary storage location, for example.

From operation 506A flow continues to operation 508A where a drag input dragging the user interface element corresponding to the object over a boundary of the mirroring application is received. The drag input may comprise one or more of: a mouse movement in association with an interacted-with object, and a non-interrupted touch input and movement on a touch-enabled display in association with an interacted-object, for example.

From operation 508A flow continues to operation 510A where a second user interface element corresponding to the object is surfaced at a location of the user interface of the first device (or display device connected to the first device) corresponding to a location where the drag input crossed the boundary of the mirroring application.

From operation 510A flow continues to operation 512A where the object is received by the host device from the remote device.

From operation 512A flow continues to operation 514A where a drop input of the user interface element is received on a construct displayed on the user interface of the first device. The object may be dropped into an application construct or an operating shell construct of the host device. The drop input may comprise one of: a release of a mouse click, a release of a touch input from a display device, a voice instruction, etc..

From operation 514A flow continues to operation 516A where the object is translated to a format that is compatible with the construct. That is, in some examples, the object may be stored on the second device in a format that is not compatible with a construct where the object is being dropped on the first device. A translation engine that resides on one or both of the remote and/or host devices, and/or a device in the cloud (e.g., a server device), may translate the object to a compatible format. The translation may comprise converting the object to a different file type, adding file features to the object that are readable by both devices and/or different platforms, etc..

From operation 516A flow continues to operation 518A where the object is saved to a location of the first device corresponding to the construct. The object may be saved in a translated format.

From operation <NUM> flow moves to an end operation and the method 500A ends.

<FIG> is an exemplary method 500B for assisting with cross-platform drag and drop functionality from the remote computing device. The method 500B begins at a start operation and flow continues to operation 502B where a connection between a first device and a second device is established. In this example, the first device is the host device and the second device is the remote device.

From operation 502B flow continues to operation 504B where an indication that a drag-initiating event of a user interface element corresponding to an object on the second device has been received by a mirroring application of the first device is received by the second device.

From operation 504B flow continues to operation 506B where an indication that a drag input of the user interface element corresponding to the object has been received by the mirroring application of the first device is received by the second device.

From operation 506B flow continues to operation 508B where the object is copied by the second device.

From operation 508B flow continues to operation 510B where an indication that the user interface element corresponding to the object has crossed a boundary separating the mirroring application on the first device and a native construct on the user interface of the first device during the drag input is received.

From operation 510B flow continues to operation 512B where the object is sent from the second device to the first device.

From operation 512B flow moves to an end operation and the method 500B ends.

<FIG> and <FIG> illustrate a mobile computing device <NUM>, for example, a mobile telephone, a smart phone, wearable computer (such as smart eyeglasses), a tablet computer, an e-reader, a laptop computer, or other AR compatible computing device, with which aspects of the disclosure may be practiced. With reference to <FIG>, one aspect of a mobile computing device <NUM> for implementing the aspects is illustrated. In a basic configuration, the mobile computing device <NUM> is a handheld computer having both input elements and output elements. The mobile computing device <NUM> typically includes a display <NUM> and one or more input buttons <NUM> that allow the user to enter information into the mobile computing device <NUM>. The display <NUM> of the mobile computing device <NUM> may also function as an input device (e.g., a touch screen display). If included, an optional side input element <NUM> allows further user input. The side input element <NUM> may be a rotary switch, a button, or any other type of manual input element. In alternative aspects, mobile computing device <NUM> may incorporate more or fewer input elements. For example, the display <NUM> may not be a touch screen in some aspects. In yet another alternative aspect, the mobile computing device <NUM> is a portable phone system, such as a cellular phone. The mobile computing device <NUM> may also include an optional keypad <NUM>. Optional keypad <NUM> may be a physical keypad or a "soft" keypad generated on the touch screen display. In various aspects, the output elements include the display <NUM> for showing a graphical user interface (GUI), a visual indicator <NUM> (e.g., a light emitting diode), and/or an audio transducer <NUM> (e.g., a speaker). In some aspects, the mobile computing device <NUM> incorporates a vibration transducer for providing the user with tactile feedback. In yet another aspect, the mobile computing device <NUM> incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device.

<FIG> is a block diagram illustrating the architecture of one aspect of a mobile computing device. That is, the mobile computing device <NUM> can incorporate a system (e.g., an architecture) <NUM> to implement some aspects. In one aspect, the system <NUM> is implemented as a "smart phone" capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players). In some aspects, the system <NUM> is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.

One or more application programs <NUM> may be loaded into the memory <NUM> and run on or in association with the operating system <NUM>. Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. The system <NUM> also includes a non-volatile storage area <NUM> within the memory <NUM>. The non-volatile storage area <NUM> may be used to store persistent information that should not be lost if the system <NUM> is powered down. The application programs <NUM> may use and store information in the non-volatile storage area <NUM>, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on the system <NUM> and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area <NUM> synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory <NUM> and run on the mobile computing device <NUM>, including instructions for providing and operating a user experience transformation computing platform.

The visual indicator <NUM> may be used to provide visual notifications, and/or an audio interface <NUM> may be used for producing audible notifications via the audio transducer <NUM>. In the illustrated aspect, the visual indicator <NUM> is a light emitting diode (LED) and the audio transducer <NUM> is a speaker. These devices may be directly coupled to the power supply <NUM> so that when activated, they remain on for a duration dictated by the notification mechanism even though the processor <NUM> and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. The audio interface <NUM> is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to the audio transducer <NUM>, the audio interface <NUM> may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with aspects of the present disclosure, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below. The system <NUM> may further include a video interface <NUM> that enables an operation of an on-board camera <NUM> to record still images, video stream, and the like.

<FIG> is a block diagram illustrating physical components (e.g., hardware) of a computing device <NUM> with which aspects of the disclosure may be practiced. The computing device components described below may have computer executable instructions for assisting with the dragging and dropping of objects across the constructs of devices operating on different platforms via a device mirroring application. In a basic configuration, the computing device <NUM> may include at least one processing unit <NUM> and a system memory <NUM>. Depending on the configuration and type of computing device, the system memory <NUM> may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. The system memory <NUM> may include an operating system <NUM> suitable for running one or more cross-platform mirroring applications. The operating system <NUM>, for example, may be suitable for controlling the operation of the computing device <NUM>. Furthermore, aspects of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated in <FIG> by those components within a dashed line <NUM>. The computing device <NUM> may have additional features or functionality. For example, the computing device <NUM> may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in <FIG> by a removable storage device <NUM> and a non-removable storage device <NUM>.

As stated above, a number of program modules and data files may be stored in the system memory <NUM>. While executing on the processing unit <NUM>, the program modules <NUM> (e.g., cross-platform mirroring application <NUM>) may perform processes including, but not limited to, the aspects, as described herein. According to examples, drag indication monitor <NUM> may perform one or more operations associated with identifying when a drag-initiating command has been received in relation to an object element displayed in a device mirroring application of a host device. Boundary engine <NUM> may perform one or more operations associated with identifying when an object element has reached a boundary of a device mirroring application and the native constructs of a host device. Translation engine <NUM> may perform one or more operations associated with identifying whether an object received from a remote device is compatible with a host device, translating the object into one or more compatible formats with the host device if the format the object was received in is not compatible with the host device. UI modification engine <NUM> may perform one or more operations associated with modifying an object element displayed in a device mirroring application to a format that is native across the boundary of a host device.

Furthermore, aspects of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, aspects of the disclosure may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in <FIG> may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or "burned") onto the chip substrate as a single integrated circuit. When operating via an SOC, the functionality, described herein, with respect to the capability of client to switch protocols may be operated via application-specific logic integrated with other components of the computing device <NUM> on the single integrated circuit (chip). Aspects of the disclosure may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. In addition, aspects of the disclosure may be practiced within a general purpose computer or in any other circuits or systems.

The computing device <NUM> may also have one or more input device(s) <NUM> such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc. The output device(s) <NUM> such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. The computing device <NUM> may include one or more communication connections <NUM> allowing communications with other computing devices <NUM>. Examples of suitable communication connections <NUM> include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.

<FIG> illustrates one aspect of the architecture of a system for processing data received at a computing system from a remote source, such as a personal/general computer <NUM>, tablet computing device <NUM>, or mobile computing device <NUM>, as described above. Content displayed at server device <NUM> may be stored in different communication channels or other storage types. For example, various documents may be stored using a directory service <NUM>, a web portal <NUM>, a mailbox service <NUM>, an instant messaging store <NUM>, or a social networking site <NUM>. The program modules <NUM> may be employed by a client that communicates with server device <NUM>, and/or the program modules <NUM> may be employed by server device <NUM>. The server device <NUM> may provide data to and from a client computing device such as a personal/general computer <NUM>, a tablet computing device <NUM> and/or a mobile computing device <NUM> (e.g., a smart phone) through a network <NUM>. By way of example, the computer system described above with respect to <FIG> may be embodied in a personal/general computer <NUM>, a tablet computing device <NUM> and/or a mobile computing device <NUM> (e.g., a smart phone). Any of these aspects of the computing devices may obtain content from the store <NUM>, in addition to receiving graphical data useable to be either pre-processed at a graphic-originating system, or post-processed at a receiving computing system.

The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The aspects, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the best mode of claimed disclosure. The claimed disclosure should not be construed as being limited to any aspect, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an aspect with a particular set of features. Having been provided with the description and illustration of the present disclosure, one skilled in the art may envision variations, modifications, and alternate aspects falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed disclosure.

Claim 1:
A computer-implemented method for assisting with cross-platform drag and drop functionality, the computer-implemented method comprising:
displaying (504A), on a user interface of a first device, a mirroring application that mirrors a user interface of a second device and partially overlaps the user interface of the first device;
receiving (506A) a drag-initiating input of a user interface element corresponding to an object of the second device;
copying the object to a temporary storage location of the second device upon receiving the drag-initiating input;
receiving (508A) a drag input dragging the user interface element corresponding to the object over a boundary of the mirroring application;
displaying (510A) a second user interface element corresponding to the object at a location of the user interface of the first device corresponding to a location where the drag input crossed the boundary of the mirroring application;
receiving, by the second device, an indication that the object has crossed the boundary of the mirroring application on the first device;
transferring the object copied in the temporary storage location of the second device from the second device to the first device upon receiving the indication;
saving the object to a temporary storage location of the first device;
receiving (514A) a drop input of the user interface element on a construct displayed on the user interface of the first device;
translating (516A) the object to a format compatible with the construct; and
saving (518A) the object to a location of the first device corresponding to the construct.