Continuous and concurrent device experience in a multi-device ecosystem

Various embodiments generally relate to decentralized operating systems spanning multiple devices. More specifically, some embodiments relate to continuous and concurrent device experiences across the multiple devices. For example, a user may be streaming a movie through a service provided via a smart television. When the user's phone detects an incoming video call, the television can recognize that a video call is coming in and ask the user if the user wants to answer the call. If the user responds in the affirmative, the television can pause the streaming video from the service provide allowing the user to answer their phone. In some cases, the television can be used as the display screen to display the video call while the local phone resources (e.g., video camera) can be used as the recording device.

TECHNICAL FIELD

Various embodiments generally relate to decentralized operating systems spanning multiple devices. More specifically, some embodiments relate to continuous and concurrent device experiences across the multiple devices.

BACKGROUND

Computing devices such as mobile phones, smartphones, and tablet computers have become ubiquitous in today's society. In fact, many individuals use a number of computing devices to perform various tasks during the course of their day. For instance, a person/user may use a laptop for work, a tablet for entertainment, a smartphone for phone calls, etc. Further, given the convergence of capabilities amongst the various computing devices, the user may use any of the computing devices to perform many of their tasks. For example, devices such as smart televisions, mobile phones, tablets, laptops, and others may be capable of streaming video from the internet.

While computing devices have become able to communicate through various communication protocols, there has been no efficient way for these devices to interoperate. Instead, each computing device has traditionally been completely independent. For example, each computing device typically has had its own operating system to manage the unique hardware resources available only to that computing device.

SUMMARY

Various embodiments generally relate to decentralized operating systems spanning multiple devices. More specifically, some embodiments relate to continuous and concurrent device experiences across the multiple devices. Some embodiments provide for a method that includes executing a first activity session on a first computing device. The first computing device may be part of a group of multiple computing devices having a decentralized operating system that spans the multiple computing devices. The decentralized operating system is responsible for managing the hardware resources of the multiple devices. As such, when a second activity session is being initiated at one or more of the computing devices in the group of multiple computing devices, the decentralized operating system can manage how the second activity session is processed (e.g., which resources are utilized). In some embodiments, a determination can be made as to whether to switch from the first activity session at the first computing device to the second activity session. The first activity session on the first computing device may be paused, for example, while the second activity session is allowed to begin.

For example, a user may be streaming a movie through a service provided via a smart television. When the user's phone detects an incoming call, the television can recognize that a call is coming in and ask the user if the user wants to answer the call. If the user responds in the affirmative, the television can pause the streaming video allowing the user to answer their phone.

In other embodiments, an activity session on a first computing device may be transferred to a second computing device within a multi-device ecosystem. In some embodiments, the multi-device ecosystem may have a decentralized operating system that spans the multiple computing devices capable of facilitating the session transfer. For example, a user can be using a phone to stream music. When the user walks in the front door of their house, the user may want to transfer the streaming music session to a home sound system. Various embodiments allow a user to set customized rules indicating when a transfer should happen automatically or when the user should be prompted for confirmation before initiating the transfer. The transferred session can continue at current state. The decentralized operating system facilitates the device discovery making the transfer easier. For example, in some embodiments, the decentralized operating system can determine the capabilities of each device and determine if a transfer would be successful.

Embodiments of the present technology also include computer-readable storage media containing sets of instructions to cause one or more processors to perform the methods, variations of the methods, and other operations described herein.

While multiple embodiments are disclosed, still other embodiments of the present technology will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative embodiments of the technology. As will be realized, the technology is capable of modifications in various aspects, all without departing from the scope of the present technology. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not restrictive.

The drawings have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be expanded or reduced to help improve the understanding of the embodiments of the present invention. Similarly, some components and/or operations may be separated into different blocks or combined into a single block for the purposes of discussion of some of the embodiments of the present invention. Moreover, while the invention is amenable to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are described in detail below. The intention, however, is not to limit the invention to the particular embodiments described. On the contrary, the invention is intended to cover all modifications, equivalents, and alternatives falling within the scope of the invention as defined by the appended claims.

DETAILED DESCRIPTION

Various embodiments generally relate to decentralized operating systems spanning multiple devices. More specifically, some embodiments relate to continuous and concurrent device experiences across the multiple devices. Some embodiments provide for a method that includes executing a first activity session on a first computing device. The first computing device may be part of a group of multiple computing devices having a decentralized operating system that spans the multiple computing devices. The decentralized operating system is responsible for managing the hardware resources of the multiple devices. As such, when a second activity session is being initiated at one or more of the computing devices in the group of multiple computing devices, the decentralized operating system can manage how the second activity session utilizes available resources, if the second activity session should be terminated, if the second activity session should be allowed to begin, etc. In some embodiments, a determination can be made as to whether to switch from the first activity session at the first computing device to the second activity session. The first activity session on the first computing device may be paused, for example, while the second activity session is allowed to begin.

For example, a user may be streaming a movie through a service provided via a smart television. When the user's phone detects an incoming call, the television can recognize that a call is coming in and ask the user if the user wants to answer the call. If the user responds in the affirmative, the television can pause the streaming video allowing the user to answer their phone.

In other embodiments, an activity session on a first computing device may be transferred to a second computing device within a multi-device ecosystem. In some embodiments, the multi-device ecosystem may have a decentralized operating system that spans the multiple computing devices capable of facilitating the session transfer. For example, a user can be using a phone to stream music. When the user walks in the front door of their house, the user may want to transfer the streaming music session to a home sound system. Various embodiments allow a user to set customized rules indicating when a transfer should happen automatically or when the user should be prompted for confirmation before initiating the transfer. The transferred session can continue at the current state of the activity session. The decentralized operating system facilitates the device discovery making the transfer easier. For example, in some embodiments, the decentralized operating system can determine the capabilities of each device and determine if a transfer would be successful.

For example, some embodiments allow a user to interact with a browser page, a file, etc. on one device and then pick up the session at a second device. In some embodiments for example, a transfer icon may be displayed that allows the user to pull the experience off the previous device and continue to engage and interact with it as if the entire experience was on the second device.

In some embodiments, computing devices can be transformed into peripherals devices for receiving inputs and generating outputs. For example, a CPU/GPU may be used as a peripheral. As such, when one device is plugged in and the other one is not, the CPU/GPU of the device that is plugged in can be used for processing. As another example, a user can use two or more devices at the same time to do the same/similar thing. These techniques can allow applications and developers to see that there are multiple devices running the application at the same time. Still yet, these techniques can also allow a first device to take a screenshot of another device.

Peripherals can also recognize whether they are near each other (NFC) and communicate with each other and can optimize performance based on peripheral location. For example, nearby peripherals can utilize the more charged peripheral for chatting/mapping/etc. Suppose you have a tablet and N devices, one per player of a game. The tablet can show a shared game state (e.g., a tile board for a board game, or the cards on the table for a card game), and each device can have the secret state (e.g. cards in hand) of each player. In some embodiments, the computing devices can be placed into a hierarchy along with specified directional communication flows in order to facilitate an activity.

As another example, these techniques allow a user's device with a camera to be set up somewhere. Then, the user can control that camera from a second device in the user's possession, including being able to look remotely through the camera's viewfinder, setting various camera parameters, and actually taking photos or shooting video.

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent, however, to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details. While, for convenience, embodiments of the present invention are described with reference to a multi-device ecosystem having a decentralized operating system, embodiments of the present invention are equally applicable to various other configurations.

Terminology

General Description

FIG. 1illustrates an example of a network-based environment100of multiple computing devices having a decentralized operating system in accordance with various embodiments of the technology. As illustrated inFIG. 1, network-based environment100may include one or more computing devices110A-110N (such as a mobile device, a mobile phone, a tablet computer, a laptop, a mobile media device, a mobile gaming device, a vehicle-based computer, a smart television, a wearable computing technology, etc.). In accordance with various embodiments, computing devices110A-110N may have a decentralized operating system (“DCOS”) that manages multiple computing devices110A-110N as a single entity115. As a result, each of the computing devices110A-110N can use resources from any of the other computing devices.

An application can be installed on an application layer of the DCOS. As a result, any of the computing devices being managed by the DCOS can invoke the application installed on the application layer of the DCOS. Thus, a separate installation of the application on each of the computing devices is not required to execute the application through any of the computing devices if the application is installed through the DCOS managing the computing devices.

In some embodiments, an instance of the DCOS executes using the hardware resources of the multiple computing devices and manages the hardware resources of all the computing devices as a single entity115. The various components of the DCOS (e.g., file system of the DCOS) can be stored across the hardware resources of the multiple computing devices. For instance, the data associated with a file system of the DCOS can be stored across the memory resources of the various computing devices. The DCOS gathers any required data associated with the file system from across the computing devices when needing to provide access the file system.

Similarly, the execution of an instance of the DCOS can be performed using the hardware resources that are available across the computing devices. For instance, when a thread associated with the executing instance of this DCOS wants to execute a function, a kernel of the DCOS translates the function into data processing instructions and coordinates the various local OS kernels managing the hardware resources (e.g., CPU, memory, etc.) of each of the computing devices to execute the instructions.

In one or more embodiments, the DCOS can manage the multiple computing devices110A-110N as a single entity115by networking all of the multiple computing devices110A-110N. The multiple computing devices110A-110N may communicate with each other through a network120. The computing devices110A-110N can include network communication components that enable the computing devices110A-110N to communicate with network120or other electronic devices by transmitting and receiving wireless signals using licensed, semi-licensed or unlicensed spectrum over network120. In some cases, network120may be comprised of multiple networks, even multiple heterogeneous networks, such as one or more border networks, voice networks, broadband networks, service provider networks, Internet Service Provider (ISP) networks, and/or Public Switched Telephone Networks (PSTNs), interconnected via gateways operable to facilitate communications between and among the various networks. Network120can also include third-party communications networks such as a Global System for Mobile (GSM) mobile communications network, a code/time division multiple access (CDMA/TDMA) mobile communications network, a 3rd or 4th generation (3G/4G) mobile communications network (e.g., General Packet Radio Service (GPRS/EGPRS), Enhanced Data rates for GSM Evolution (EDGE), Universal Mobile Telecommunications System (UMTS), or Long Term Evolution (LTE) network), or other communications network.

FIG. 2is a block diagram illustrating an example of components in a network-based environment of multiple computing devices in accordance with one or more embodiments of the technology. The environment200can include multiple computing devices110A-110N that are managed as a single entity by an instance of a DCOS210that executes using the hardware resources of the multiple computing devices110A-110N. As illustrated inFIG. 2, the DCOS210may be interposed between the executing applications and the physical hardware resources of the various computing devices110A-110N to manage/execute the physical hardware resources on behalf of the executing applications. The DCOS210can include a DCOS kernel220that allows the DCOS210to manage the hardware resources of the various computing devices110A-110N as a single entity.

The computing devices110A-110N can be a desktop computer, a laptop computer, a tablet computer, a game console, a smart phone, a personal digital assistant, wearable computer, or other electronic devices capable of running computer applications, as contemplated by a person having ordinary skill in the art. The hardware resources of the various computing devices110A-110N can include central processing units (“CPU”), storage devices, graphics processing units (“GPU”), network communication hardware, a display screen, a video camera, a microphone, accelerometers, gyroscopes, global positioning receivers, environmental sensors, actuators, etc. In some embodiments, the DCOS kernel220manages memory access for the storage modules in the various computing devices110A-110N, determines which applications get access to which hardware resources, maintains a file system to manage the user/application data, and sets up or resets the operating states of CPUs (of computing devices110A-110N) for optimal operation at all times. In various embodiments, DCOS kernel220includes firmware and device drivers that control hardware resources of computing devices110A-110N at a basic level.

In one or more embodiments, DCOS210includes resource discovery and management application programming interfaces (“API”)230that can interact with local OS kernels (e.g., a Linux kernel) that each directly manage the hardware resources of a respective computing device110A-110N. Resource discovery and management API230can provide the interface for generating resource queries and sending instructions for managing the local resources. In some embodiments, DCOS kernel220can use the local OS kernels to control the various hardware resources to perform the tasks/services provided by the DCOS kernel220.

In various embodiments, DCOS kernel220maintains a mapping between the various local OS kernels and the hardware resources that are managed by the local OS kernels. DCOS kernel220may use the mapping and the resource discovery and management API230to send a request to the local OS kernel that manages the local resources. The local OS kernel then uses firmware to control the resource (e.g., a hard drive, sensor, actuator, etc.).

In embodiments, DCOS210can include libraries240that enable various operating system related tasks, e.g., file manipulation, to be performed. In some embodiments, the libraries240can be written in C, C++, or another programming language. The libraries240include, for example, a library to manage the display and windowing manager, a library to support various audio and video formats and codecs including their playback and recording, a library for rendering 2D and 3D graphics, a database library to support a database engine, etc. In embodiments, the libraries240can be stored by DCOS210across the storage devices of the computing devices110A-110N.

In some embodiments, the DCOS210can include an application framework250that offers an array of APIs that can be re-used by developers to perform various standard functions. The applications executing on DCOS210can interact directly with the application framework250when a standard function included within the application framework250is to be executed. In various embodiments, the application framework250may include a manager that manages the activities that govern the application life cycle, a manager that enables the applications to create customized alerts, views that are used to create layouts, including components such as grids, lists, buttons, etc., a manager to manage external resources used by the applications, a content provider to enable sharing of data between applications, etc. Resource discovery and management API may be a part of application framework250in some embodiments.

In accordance with various embodiments, the application framework250can be stored by the DCOS210across the storage devices of the computing devices110A-110N. It should be noted that the above description of the various components of the DCOS210are not intended to be exhaustive. The DCOS210can include other components that are required for the DCOS210to function or can improve the performance of the DCOS210as contemplated by a person having ordinary skill in the art. Further, the architecture of the DCOS210provided inFIG. 2is intended as an illustrative example. Therefore, the DCOS210can include other architectures as contemplated by a person having ordinary skill in the art.

As illustrated inFIG. 2, the environment200can include an application layer260consisting of the various computer applications installed on the DCOS210and stored using the storage devices of the computing devices110A-110N. The computer applications installed on DCOS210can include applications for general productivity and information retrieval, including email, calendar, contacts, and stock market and weather information. The computer applications can also include applications in other categories, such as mobile games, factory automation, GPS and location-based services, banking, order-tracking, ticket purchases or any other categories as contemplated by a person having ordinary skill in the art.

In some embodiments, an instance of DCOS210executes using the hardware resources of the multiple computing devices110A-110N and manages the hardware resources of all the computing devices110A-110N as a single entity. For instance, a thread associated with DCOS210can be executed by the DCOS210with the resources associated with one of the local OS kernels, where the thread-related functions are executed by a CPU associated with the local OS kernel. When the thread wants to execute a function, the DCOS kernel210translates the function into data processing instructions and coordinates with the local OS kernel managing the hardware resources (e.g., CPU, memory, etc.) to execute the instructions.

FIG. 3is a flow diagram illustrating various operations300for a continuous device experience in accordance with some embodiments of the technology. As illustrated inFIG. 3, a session activity is occurring on a first computing device during activity operation310. The session can include any type of activity that the first computing device is capable of performing. For example, the session may include a user playing a game, streaming of music or video, preparing a document, a phone call, web browsing, recording a video, recording audio, or other activity.

When detection operation320detects a triggering event, a second computing device can be selecting during selection operation330. In some embodiments, the second computing device picked during selection operation330may be chosen automatically or with user input (e.g., through a graphical user interface screen). The triggering events can include locational events, time-based events, requests to initiate other sessions, environmental triggers, and others. In some embodiments, the triggering events may be combined into more complicated triggering rules which need to be satisfied to generate the triggering event. The triggering events may be set by default or customized by the user. For example, a user may specify that when they are streaming music on their phone and arrive at home (i.e., a location based trigger) that the streaming music session is transferred to a home theater system. As another example, a triggering rule may be customized by the user to let the user know when there are alternative devices within a certain distance that have better speakers when the user is participating in a streaming music session.

Once the second device has been selected, either automatically or through a user selection, the session can be transferred from the first computing device to the second computing device during transfer operation340. In some embodiments, the session transfer may be managed by the DCOS. The DCOS can determine the capabilities of each computing device and alter the session as appropriate. The DCOS can create and manage the transfer requests which can include fields such as, but not limited to, session ID, originating device, destination device, transfer initiation time, transfer complete time, session parameters, and other fields.

FIG. 4illustrates an example of a continuous device experience in accordance with various embodiments of the technology. As illustrated inFIG. 4, first device400is performing a session410. Upon processing the transfer request, session410can be seamlessly transferred to second device420.

FIG. 5is a flow diagram illustrating various operations500for processing a session transfer request in accordance with one or more embodiments of the technology. As illustrated inFIG. 5, when detection operation510detects a triggering event, this causes identification operation520to determine devices that are capable of supporting a session transfer from a first device. In order to determine device capabilities, for example, one or more embodiments can send resource queries to the multiple computing devices. In some cases, the resource queries may only go to a subset of the multiple computing devices (e.g., based on geographical location, communication bandwidth, or other factors). Then, generation operation530generates a graphical user interface that allows a user to select to transfer a session from the first computing device to an alternative device identified by identification operation520.

Determination operation540determines if the user has requested a transfer of the session in response to the presentation of the graphical user interface. If determination operation540determines that no transfer request has been made, then determination operation540branches to maintain operation550where the session is maintained at the current device. If determination operation540determines that a transfer request has been made, then determination operation540branches to transfer operation560where the session is transferred to the selected computing device.

Success operation570determines if the session has been successfully transferred to the selected computing device. If success operation570determines that the transfer was unsuccessful, then success operation570branches to maintain operation550where the session is maintained at the first computing device. If success operation570determines that the transfer was successful, then success operation570branches to resume operation580where the session is resumed at the selected computing device.

FIG. 6is a flow diagram illustrating various operations600for a concurrent device experience in accordance with some embodiments of the technology. During performance operation610, an activity is being performed on a first computing device. During receiving operation620, the DCOS may receive an indication of a second activity being initiated or performed on a second computing device. Determination operation630determines if activities should be switched. If determination operation630determines that the activity on the first device should not be switched for the second activity, determination operation630branches to continuation operation640where the activity on the first computing device is continued. If determination operation630determines that the activity on the first device should be switched to the second activity, determination operation630branches to pause operation650where the activity on the first computing device can be paused. The second activity can be initiated (maybe even on the first computing device if supported) during initiation operation660.

For example, while a user is watching television, a call might arrive on their phone. The DCOS can recognize that a call is coming in on the phone and alert the user via the television. If the television is capable of supporting a phone call, then the user may also be prompted to have the phone call answered through the television. As another example, if the phone session is a video chat, the television may be used as the display screen and for presenting the audio from the other person while a microphone, camera, or other recording device in the phone is used to record audio/video of the user.

Exemplary Computer System Overview

FIG. 7is a block diagram of a computer system as may be used to implement features of some embodiments of the disclosed technology. The computing system700may include one or more central processing units (“processors”)710, memory720, input/output devices750(e.g., keyboard and pointing devices, display devices), storage devices740(e.g., disk drives, flash memory, etc.), and network adapters760(e.g., network interfaces) that are connected to an interconnect730. The interconnect730is illustrated as an abstraction that represents any one or more separate physical buses, point to point connections, or both connected by appropriate bridges, adapters, or controllers. The interconnect730, therefore, may include, for example, a system bus, a Peripheral Component Interconnect (PCI) bus or PCI-Express bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), IIC (I2C) bus, an Institute of Electrical and Electronics Engineers (IEEE) standard1394bus, also called “Firewire”, or any type of cellular, IP-based or converged telecommunications network, including but not limited to Global System for Mobile Communications (GSM), Worldwide Interoperability for Microwave Access (WiMAX), etc.

The instructions stored in memory720can be implemented as software and/or firmware to program the processor(s)710to carry out actions described above. In some embodiments, such software or firmware may be initially provided to the processing system700by downloading it from a remote system through the computing system700(e.g., via network adapter760).

The terms used in this specification generally have their ordinary meanings in the art, within the context of the disclosure, and in the specific context where each term is used. Certain terms that are used to describe the disclosure are discussed below, or elsewhere in the specification, to provide additional guidance to the practitioner regarding the description of the disclosure. It will be appreciated that the same thing can be said in more than one way. One will recognize that “memory” is one form of a “storage” and that the terms may on occasion be used interchangeably.

The above description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of the disclosure. However, in certain instances, well-known details are not described in order to avoid obscuring the description. Further, various modifications may be made without deviating from the scope of the technology. Accordingly, the technology is not limited except as defined by the appended claims.