Patent Description:
Computing systems made up of one or more computing devices such as head-mountable devices typically have multiple processes being managed by multiple processors. The multiple processes function together to create a user experience. Document <CIT> relates to system and method for managing a plurality of clients, wherein a request to implement a change in configuration data is received from a user, the configuration data relates to an operation of a client, and wherein a notification service associated with the client is identified and notified of the change in the configuration data.

Particular embodiments are defined by the subject-matter of the dependent claims. Embodiments which do not fall within the scope of the claims are to be interpreted merely as examples useful for understanding the invention.

A physical environment refers to a physical world that people can sense and/or interact with without aid of electronic devices. The physical environment may include physical features such as a physical surface or a physical object. For example, the physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and/or interact with the physical environment such as through sight, touch, hearing, taste, and smell. In contrast, an extended reality (XR) environment refers to a wholly or partially simulated environment that people sense and/or interact with via an electronic device. For example, the XR environment may include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, and/or the like. With an XR system, a subset of a person's physical motions, or representations thereof, are tracked, and, in response, one or more characteristics of one or more virtual objects simulated in the XR environment are adjusted in a manner that comports with at least one law of physics. As one example, the XR system may detect head movement and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment. As another example, the XR system may detect movement of the electronic device presenting the XR environment (e.g., a mobile phone, a tablet, a laptop, or the like) and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment. In some situations (e.g., for accessibility reasons), the XR system may adjust characteristic(s) of graphical content in the XR environment in response to representations of physical motions (e.g., vocal commands).

There are many different types of electronic systems that enable a person to sense and/or interact with various XR environments. Examples include head mountable systems, projection-based systems, heads-up displays (HUDs), vehicle windshields having integrated display capability, windows having integrated display capability, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones/earphones, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop/laptop computers. A head mountable system may have one or more speaker(s) and an integrated opaque display. Alternatively, a head mountable system may be configured to accept an external opaque display (e.g., a smartphone). The head mountable system may incorporate one or more imaging sensors to capture images or video of the physical environment, and/or one or more microphones to capture audio of the physical environment. Rather than an opaque display, a head mountable system may have a transparent or translucent display. The transparent or translucent display may have a medium through which light representative of images is directed to a person's eyes. The display may utilize digital light projection, OLEDs, LEDs, uLEDs, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium may be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In some implementations, the transparent or translucent display may be configured to become opaque selectively. Projection-based systems may employ retinal projection technology that projects graphical images onto a person's retina. Projection systems also may be configured to project virtual objects into the physical environment, for example, as a hologram or on a physical surface. <FIG> is a block diagram depicting components of a computing device according to aspects of the subject technology. Not all of the depicted components may be used in all implementations, however, and one or more implementations may include additional or different components than those shown in the figure. Variations in the arrangement and type of the components may be made without departing from the scope of the claims as set forth herein. Additional components, different components, or fewer components may be provided.

As depicted in <FIG>, computing device <NUM> includes system on a chip (SOC) <NUM>, SOC <NUM>, manifest server <NUM>, and input/output (IO) processors <NUM>, <NUM>, and <NUM>. IO processors <NUM>, <NUM>, and <NUM> are configured to control input and/or output operations of computing device <NUM>. For example, computing device <NUM> may be a smartphone, a tablet device, or a wearable device such as a head-mountable device (as examples) with the IO processors controlling cameras, displays, content rendering, etc. The subject technology is not limited to this number of IO processors and may be implemented with more or fewer IO processors than those depicted in <FIG>.

In the example of <FIG>, manifest server <NUM> is configured to control the timing of IO processors <NUM>, <NUM>, and <NUM> performing operations to change an operating state of computing device <NUM> from a first state to a second state. In one or more implementations, the transitions may be between modes such as a virtual reality mode and a mixed reality mode, between different frame rates being displayed, enabling/disabling the display(s), etc. Transitions may also, or alternatively, include transitions to or from a bounded two-dimensional display mode from or to a bounded partial three-dimensional display mode, to or from a bounded two-dimensional display mode from or to a bounded three-dimensional display mode, to or from a bounded partial three-dimensional display mode from or to a bounded three-dimensional display mode, to or from a bounded display mode from or to a full-screen mixed reality display mode, to or from bounded display mode from or to a full-screen virtual reality display mode, to or from a full-screen mixed reality display mode from or to a full-screen virtual reality display mode, and/or to a pass-through video only mode from any of the above modes.

Any or all of these transitions may involve activating or deactivating one or more cameras and/or one or more sensors (e.g., depth sensors, light sensors, microphones, or the like), setting up buffers for upcoming data from the cameras and/or sensors, activating or deactivating processing operations for images, audio, display data, etc., and/or activating or deactivating other hardware and/or software elements of the computing device <NUM>. The operation of manifest server <NUM> is described in further detail below. The operations described herein in which the manifest server <NUM> and/or another manifest server compile and distribute manifests to various processors for a transition can allow multiple separate processors and/or processes to concurrently execute their respective operations for the transitions in a time-coordinated matter that prevents artifacts and/or other timing errors from affecting the user's experience of using the device. For example, the subject technology provides an effective system for coordinating the operations of multiple IO processors and/or other processors to be performed to effect a state transition of a computing device or computing system. The use of manifests to control the timing of the IO processors and/or other processors executing their respective operations makes coordination of the IO processors and/or other processors possible without requiring the IO processors and/or other processors to communicate with each other regarding their respective operations. This reduction in communications saves power in the computing system or computing device and frees up resources for other processes in the computing system or computing device.

In the example of <FIG>, manifest server <NUM> and IO processors <NUM> and <NUM> are part of SOC <NUM> and IO processor <NUM> is part of SOC <NUM>. Manifest server <NUM> and IO processors <NUM>, <NUM>, and <NUM> may include suitable logic, circuitry, and/or code that enable processing data and/or controlling operations of computing device <NUM>. These components, or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), may be implemented in hardware (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other suitable devices) and/or a combination of both hardware and software.

The example of <FIG> in which the computing device <NUM> includes the SOC <NUM>, the SOC <NUM>, the manifest server <NUM>, and the IO processors <NUM>, <NUM>, and <NUM> is merely illustrative, and, in other implementations, the computing device <NUM> may include one or more other processors, one or more other manifest servers, and/or may execute one or more software processes associated with a transition. For example, <FIG> illustrates an implementation in which the computing device <NUM> includes the SOC <NUM>, the SOC <NUM>, the manifest server <NUM>, and the IO processors <NUM>, <NUM>, and <NUM>, and additional includes one or more additional processors <NUM> on the SOC <NUM>, one or more software processes <NUM> that may be executed by the processor(s) <NUM> and/or the IO processors <NUM>, <NUM>, and <NUM>. In the example of <FIG>, the SOC <NUM> also includes one or more additional IO processors <NUM> and may include an additional manifest server, such as manifest server <NUM>.

Manifest server <NUM>, processor(s) <NUM>, software processes <NUM>, and/or IO processor <NUM> may include suitable logic, circuitry, and/or code that enable processing data and/or controlling operations of computing device <NUM>. These components, or one or more portions thereof, may be implemented in software (e.g., instructions, subroutines, code), may be implemented in hardware (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other suitable devices) and/or a combination of both hardware and software.

The subject technology may be implemented with a single SOC or with more than two SOCs, with differing or the same numbers of IO processors and/or other processors arranged in the respective SOCs. The subject technology also may be implemented with SOCs and their respective IO processors and/or other processors arranged in a computing system that includes one or more computing devices. For example, in a computing system with more than one computing device each computing device may include one or more SOCs, with each SOC including one or more IO processors. For example, although the SOC <NUM> and the SOC <NUM> are depicted as being included in the same computing device <NUM>, in other implementations, the SOC <NUM> and the SOC <NUM> may be implemented in separate computing devices.

As indicated above, manifest server <NUM> is configured to control the timing of the IO processors, other processors, and/or software processes performing operations to change an operating state of the computing device from a first state to a second state.

Briefly, in response to receiving a request to transition the computing device from a first state to a second state, the manifest server <NUM> compiles and dispatches a manifest to the IO processors (and/or other processors). In one or more implementations, the request may include a transition identifier of the requested transition. For example, <FIG> illustrates a use case in which the manifest server <NUM> provides the manifest to the IO processor <NUM>, the IO processor <NUM>, and the processor(s) <NUM> (e.g., and the software processes <NUM>) on the SOC <NUM>, and to the IO processor <NUM>, the IO processor <NUM>, and/or other IO processors on the SoC <NUM> (e.g., without involvement of the manifest server <NUM>).

The manifest includes a transition identifier, that identifies a particular transition from a first state to a second state and represents a command to make that transition, a list of operations to be performed by each of the IO processors (and/or other processors), and a time at which the IO processors (and/or other processors) should execute operations associated with the transition identifier. The manifest may be provided in a message or other communication from the manifest server(s) to one or more processors. For example, a transition identifier might identify the transition from a virtual reality mode to a mixed reality mode, a transition from a bounded mode to an unbounded or full-screen mode, a transition from one frame rate to another frame rate, and/or other transitions as described herein. System clocks are synchronized across the SOCs so that all of the IO processors and/or other processors see the same absolute time to coordinate the execution of the operations based on the times included in the manifest(s) as described herein. This process will now be described in more detail with reference <FIG>.

<FIG> is a flowchart illustrating an example process <NUM> for controlling the timing of IO processors performing operations to change an operating state of a computing device according to aspects of the subject technology. For explanatory purposes, the blocks of process <NUM> are described herein as occurring in serial, or linearly. However, multiple blocks of process <NUM> may occur in parallel. In addition, the blocks of process <NUM> need not be performed in the order shown and/or one or more blocks of process <NUM> need not be performed and/or can be replaced or supplemented by other operations.

Process <NUM> begins with initializing the manifest server (block <NUM>). In particular, the manifest server may be instantiated in a host process. Once instantiated, the manifest server queries the IO processors and/or other processors for a manifest time and an execution time associated with each transition identifier for transitions available in the system. If an IO processor or other processor has one or more operations to execute to effect a transition of the computing device corresponding to a particular transition identifier, the IO processor or other processor returns the manifest time, which represents a period of time needed by the IO processor or other processor to prepare in advance of executing the one or more operations, and the execution time, which represents the period of time needed by the IO processor or other processor to execute the one or more operations associated with the transition identifier. The one or more operations to effect a transition in a head-mountable device may include operations for frame-level synchronization across IO processors or other processors. If an IO processor or other processor is not involved in a particular transition of the computing device, the IO processor or other processor returns a failure in response to a query for the transition identifier associated with that transition, and/or returns an indication that no operations need to be performed by the IO processor or other processor for the particular transition.

Alternatively, the IO processors and/or other processors may register with the manifest server once the manifest server has been instantiated. When an IO processor or other processor is initialized, the IO processor or other processor may query an operating system of the computing device to determine if the manifest server has been instantiated. If the manifest server has not been instantiated, the IO processor or other processor may request the operating system to notify the IO processor or other processor when the manifest server is instantiated. Once the IO processor or other processor becomes aware of the manifest server, the IO processor or other processor registers with the manifest server by indicating the different transitions, identified by a transition identifier, that the IO processor or other processor executes one or more operations to effect the transitions. For each of the indicated transitions the IO processor or other processor also provides the manifest time and the execution time associated with the transition.

Once initialized, the manifest server waits to receive a request to transition the computing device from a first state to a second state (block <NUM>). The request may be received from an application or from the operating system of the computing device. Upon receiving the request, the manifest server may determine if there are any pending transitions that are still being processed by the IO processors and/or other processors. If there is a pending transition, the manifest server may wait until the IO processors and/or other processors have completed execution of the operations associated with the pending transition before proceeding in response to the received request. If the pending transition and the transition identified in the received request involve different subsets of IO processors and/or other processors executing operations to effect the respective transitions, the manifest server may proceed and have the two transitions occur in parallel. Similarly, if the operations executed by the IO processors and/or other processors are distinct for the two transitions, the manifest server may proceed and have the two transitions occur in parallel.

As part of a prepare phase, the manifest server may notify the IO processors and/or other processors of the received transition request (block <NUM>). This notification allows the IO processors and/or other processors to prepare for the upcoming manifest by performing, for example, any cleanup or state management needed before the IO processor or other processor would be ready to execute the operations corresponding to the manifest. In response to the notification, the IO processors and/or other processors return a success indicator indicating either success or failure in preparing for the manifest. If the manifest server receives a failure indication, the manifest server may retry sending the notification to the IO processor or other processor that returned the failure indication until a success indication is received or other mitigating actions are taken.

The manifest server compiles a manifest for each IO processor or other processor participating in the transition of the computing device from the first state to the second state once the prepare phase discussed above has successfully completed (block <NUM>). The manifest for a particular IO processor or other processor includes a transition identifier representing a command to transition from the first state to the second state. The manifest for a particular IO processor or other processor further includes an action time for that IO processor or other processor to execute one or more operations associated with the transition identifier. The action time may be determined by the manifest server using the manifest times provided by the IO processors and/or other processors when the manifest server was initialized. For example, the action time may be based on the maximum or longest manifest time returned by the IO processors and/or other processors. The action time may further be based on a transport time representing the period of time needed to transmit the manifests to the IO processors and/or other processors. According to aspects of the subject technology, the action time may be the maximum manifest time plus the transport time.

The manifest for an IO processor or other processor may be in the form of a table containing one or more operations to be performed by the receiving IO processor or other processor to effect the transition identified by a transition identifier. For tables containing more than one operation, the table may include a single action time to start execution of the sequence of operations, or may include a separate action time for each operation indicating when execution of the respective operation should be started. The table also may include a deadline indicating a time period after which the one or more operations are expected to be completed to effect the transition. As with the action time, the table may include a single deadline for completing all of the operations included in the table, or may include a separate deadline for each operation to complete. Using individual action times and deadlines for each of a sequence of operations indicated in the table facilitates better coordination between multiple IO processors and/or other processors that each must execute multiple operations to effect the transition.

Once the manifest server has compiled the manifests, the manifests are dispatched to the IO processors and/or other processors (block <NUM>). At the action time(s) indicated in the manifests, each of the IO processors and/or other processors executes the one or more operations associated with the transition identifier in the manifests. Upon completion of the one or more operations, the IO processors and/or other processors may return a status report to the manifest server to inform the manifest server that the IO processor or other processor has completed the operations associated with the transition identifier (block <NUM>). Alternatively, at the time of dispatching the manifests to the IO processors and/or other processors, the manifest server may set a timer to expire at the action time plus the execution time returned by the IO processors and/or other processors at initialization. Upon expiration of the timer, the manifest server may poll the IO processors and/or other processors to determine if the IO processors have completed execution of the one or more operations associated with the transition identifier. In addition to returning a successful status report to the manifest server, the IO processors and/or other processors may return an actual amount of time needed to complete the one or more operations to update the execution time obtained by the manifest server at the time of initialization.

<FIG> is a timing chart illustrating the relative timing of process <NUM> described with respect to <FIG> according to aspects of the subject technology. As discussed above, the action time (t_action, indicated between time A and time B in <FIG>) may be the maximum manifest time plus the transport time (t_transport, indicated between time C and time D in <FIG>). As indicated in <FIG>, the transport time may vary for different IO processors and/or other processors, but the action time is selected so that all of the IO processors and/or other processors begin execution of the operations to effect the transition at the same time. While the execution time (t_execute, indicated between time E and time F in <FIG>) is depicted as being the same amount of time in <FIG>, the execution time may vary between different IO processors and/or other processors.

In the example of <FIG> and <FIG>, the manifest server <NUM> of the SOC <NUM> provides the manifests to the IO processors and/or other processors of both the SOC <NUM> and the SOC <NUM>. However, in some use cases, the manifest server <NUM> and/or the SOC <NUM> may become unavailable (e.g., in a failure state and/or during a restart process of the SOC <NUM>). In one or more implementations, the manifest server <NUM> may act as a backup manifest server (e.g., responsive to a determination that the manifest server <NUM> and/or the SOC <NUM> are unavailable). For example, <FIG> illustrates an example use case in which the SOC <NUM> is in a failure state or in a restart process, and manifest server <NUM> acts as a backup manifest server for the computing device <NUM>.

As shown in <FIG>, the manifest server compiles a manifest for each of the IO processors <NUM> and <NUM> on the SOC <NUM>, and distribute the respective manifests to the IO processors <NUM> and <NUM> on the SOC <NUM>. In this way, even when the SOC <NUM> is unavailable, the IO processor <NUM>, the IO processor <NUM>, and/or other IO processors (and/or other processors) at the SOC <NUM> can continue to operate. For example, the IO processor <NUM>, the IO processor <NUM>, and/or other IO processors (and/or other processors) at the SOC <NUM> process signals from components such as the display, the camera(s), and/or other sensor(s) of the computing device that are used to provide a pass-through view of the physical environment on the display of the computing device even when the SOC <NUM> is unavailable. For example, the IO processor <NUM>, the IO processor <NUM>, and/or other IO processors (and/or other processors) at the SOC <NUM> can execute a coordinated transition from a current operating state (e.g., a VR or MR operating state) of the computing device (when the SOC <NUM> is functioning) to a pass-through operating state in which only the view of the physical environment (e.g., and an overlaid notification such as restart notification or failure mode notification) is displayed (while the SOC <NUM> is unavailable). In this way, a user of the computing device <NUM> may continue to see their physical environment even in the failure mode of the SOC <NUM>. In this example, the manifest server <NUM> may compile and distribute a manifest for a subset of the IO processors and/or other processors of the computing device <NUM> (e.g., using the manifest compilation and distribution operations of blocks <NUM>, <NUM>, and <NUM> of <FIG>), such as the IO processors and/or other processors of the SOC <NUM>.

<FIG> is a flowchart illustrating an example process <NUM> for controlling, by a backup manifest server, the timing of processors performing operations to change an operating state of a computing device according to aspects of the subject technology. For explanatory purposes, the blocks of process <NUM> are described herein as occurring in serial, or linearly. However, multiple blocks of process <NUM> may occur in parallel. In addition, the blocks of process <NUM> need not be performed in the order shown and/or one or more blocks of process <NUM> need not be performed and/or can be replaced or supplemented by other operations.

In the example of <FIG>, at block <NUM>, a backup manifest server (e.g., a second manifest server, such as manifest server <NUM> on a second SOC, such as SOC <NUM>) of a device (e.g., computing device <NUM>) may, responsive to a determination that a first manifest server (e.g., manifest server <NUM>) or a first SOC (e.g., SOC <NUM> on which the first manifest server is implemented) is unavailable, compile, for a transition (e.g., a second transition after the first transition from the first state to the second state as described in <FIG>) from a third state to a fourth state, a respective manifest for each of a subset of processors of the device. For example, the subset of the processors may be a second set of processors, other than a first set of processors implemented on a first SOC (e.g., the SOC <NUM>) of the device, that are implemented on a second SOC (e.g., SOC <NUM>) with the backup manifest server. In one or more implementations, determining that the first manifest server or the first SOC is unavailable may include receiving a fault signal (e.g., via general purpose I/O) from the first SOC at the second SOC, or detecting that first manifest server or the first SOC is unavailable at the second SOC (e.g., by identifying an absence of display updates for a predetermined period of time).

At block <NUM>, the backup manifest server may dispatch the respective manifests compiled by the second manifest server, for the second transition, to the subset of the processors.

At block <NUM>, the backup manifest server may receive status reports from the subset of the processors regarding the second transition from the third state to the fourth state. In one or more implementations, the fourth state may be a pass-through video state in which only pass-through video and a notification (e.g., a failure state notification or a restart notification) are displayed by a display of the device. In one or more implementations, the third state may be either of the first state or the second state described herein in connection with <FIG>.

In the examples of <FIG> and <FIG>, the manifest server <NUM> distributes the manifest to the IO processors of the SOC <NUM> when the SOC <NUM> and/or the manifest server <NUM> is in a failure state. In one or more other examples, the manifest server <NUM> may be operable to distribute manifests, received from the manifest server <NUM>, to the IO processors of the SOC <NUM>. For example, <FIG> illustrates an example in which the manifest server <NUM> provides the manifest(s) to the IO processor <NUM>, the IO processor <NUM>, and the processor(s) <NUM> (e.g., and the software processes <NUM>) on the SOC <NUM>, and to the manifest server <NUM> of the SOC <NUM>. In this example, the manifest server <NUM> then distributes the manifest(s), received from the manifest server <NUM>, to the IO processor <NUM>, the IO processor <NUM>, and/or other IO processors on the SoC <NUM>. In this way, the manifest server <NUM> can dispatch respective manifests to a plurality of processors, in part, by dispatching respective manifests corresponding to a second set of processors on the SOC <NUM> to the second set of processors on the SOC <NUM> via the second manifest server <NUM>.

<FIG> illustrates a block diagram showing how the manifest server <NUM> may receive a transition request from any of various user processes <NUM> (e.g., application processes) and/or system processes <NUM>. In one or more implementations, the manifest server <NUM> may compile and distribute manifests to the IO processor <NUM>, the IO processor <NUM>, and/or one or more processor(s) <NUM> on the SOC <NUM>, and may distribute manifests for the IO processor <NUM>, the IO processor <NUM>, and/or one or more other IO processors <NUM> to the manifest server <NUM>. The manifest server <NUM> may then distribute the manifests, received from the manifest server <NUM> for the IO processor <NUM>, the IO processor <NUM>, and/or one or more other IO processors <NUM> of the SOC <NUM>, to the IO processor <NUM>, the IO processor <NUM>, and/or one or more other IO processors <NUM> of the SOC <NUM>. As illustrated in <FIG>, the manifest server <NUM> may also, in some use cases, receiving a backup trigger, and begin compiling and distributing manifests for the IO processor <NUM>, the IO processor <NUM>, and/or one or more other IO processors <NUM> of the SOC <NUM> (e.g., as described above in connection with <FIG> and <FIG>).

<FIG> and <FIG> form a sequence diagram illustrating communications that may be performed for distributing/dispatching manifests as in the example of <FIG>. As shown in <FIG>, the manifest server (e.g., manifest server <NUM>) of the SOC <NUM> may request properties, for a transition having a transition identifier (transitionID), from a first SOC processor (e.g., IO processor <NUM>, IO processor <NUM>, and/or one or more processor(s) <NUM>)) of the SOC <NUM>. The first SOC processor of the SOC <NUM> may then return properties (e.g., t_manifest and t_execute) for that transition. As shown, an IO processor (e.g., IO processor <NUM> or IO processor <NUM>) may provide a request to register to a manifest server (e.g., manifest server <NUM>) of the SOC <NUM>. Responsively, the manifest server of the SOC <NUM> may request properties, for a transition having a transition identifier, from the IO processor of the SOC <NUM>. The IO processor of the SOC <NUM> may then return properties (e.g., t_manifest and t_execute) for that transition. As shown, the manifest server of the SOC <NUM> may provide the properties from the IO processor of the SOC <NUM> to the manifest server of the SOC <NUM> via a transfer process of the SOC <NUM>.

As shown in <FIG>, a requesting process (e.g., a user process <NUM> or a system process <NUM>) may initiate a system transition with a particular transition identifier, by providing a transition request to an interface for the manifest server <NUM> of the SOC <NUM>, and wait for completion of the transition. As shown, the interface for the manifest server of the SOC <NUM> may check with the manifest server of the SOC <NUM> whether any processors are registered. If no processors are registered for that transition identifier, the interface for the manifest server of the SOC <NUM> may wait and retry until a timeout occurs. If any processors are registered, the interface for the manifest server of the SOC <NUM> may schedule the transition for the particular transition identifier. In the example of <FIG>, the request is for the transition for which the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM> previously provided properties (e.g., registered).

As shown in <FIG>, the manifest server of the SOC <NUM> may then check for a pending manifest and instruct the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM> to prepare to execute a manifest. As shown, the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM> may return a success message (if operations are to be performed by that processor for the requested transition) or a failure message (if no operations are to be performed by that processor for the requested transition) to the manifest server of the SOC <NUM>. In the example of <FIG> and <FIG>, the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM> return a success message.

As shown in <FIG>, responsive to receiving the success messages, the manifest server of the SOC <NUM> compiles manifests, including a time t_action as described herein, for each of the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM>, and dispatches the manifests to the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM>. In the example <FIG>, the manifest for the IO processor of the SOC <NUM> is dispatched to the IO processor of the SOC <NUM> via the manifest server of the SOC <NUM>. As shown in <FIG>, the first SOC processor of the SOC <NUM> and the IO processor of the SOC <NUM> may each execute the requested transition at the time t_action specified in the manifests.

As shown in <FIG>, the manifest server of the SOC <NUM> may check for a deadline for completion of the transition (e.g., by comparing the difference between t_action and the current time to a margin threshold), and may start a status timer to expire at a time t_action + t_execute. As shown, the manifest server of the SOC <NUM> may return t_action to the interface, and the interface may wait until t_action expires to report to the requesting process that the requested transition is completed. As shown in <FIG>, the manifest server of the SOC <NUM> may also perform a status polling operation after expiration of the timer. In the polling operation, the manifest server of the SOC <NUM> may request a manifest status from the first SOC processor of the SOC <NUM> and from the manifest server of the SOC <NUM>. As shown, the first SOC processor of the SOC <NUM> and the manifest server of the SOC <NUM> may return statuses including a success or failure status for the manifest and an actual time t_action, and the manifest server of the SOC <NUM> may then log the received statuses.

As discussed above, the subject technology provides an effective system for coordinating the operations of multiple IO processors and/or other processors to be performed to effect a state transition of a computing device or computing system. The use of manifests to control the timing of the IO processors and/or other processors executing their respective operations makes coordination of the IO processors and/or other processors possible without requiring the IO processors and/or other processors to communicate with each other regarding their respective operations. This reduction in communications saves power in the computing system or computing device and frees up resources for other processes in the computing system or computing device.

As noted above, computing device <NUM> may be implemented as a smartphone, a tablet device, a laptop device, or a wearable device such as a head-mountable device. <FIG> is a block diagram of computing system <NUM>, that may be implemented as a smartphone, a tablet device, a laptop device, or a wearable device such as a head-mountable device, according to aspects of the subject technology. It will be appreciated that components described herein can be provided on either or both of a frame and/or a securement element of the computing system <NUM>. It will be understood that additional components, different components, or fewer components than those illustrated may be utilized within the scope of the subject disclosure.

As shown in <FIG>, the computing system <NUM> can include a processor <NUM> (e.g., control circuity) with one or more processing units that include or are configured to access a memory <NUM> having instructions stored thereon. The instructions or computer programs may be configured to perform one or more of the operations or functions described with respect to the computing system <NUM>. The processor <NUM> can be implemented as any electronic device capable of processing, receiving, or transmitting data or instructions. For example, the processor <NUM> may include one or more of: a microprocessor, a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), or combinations of such devices. As described herein, the term "processor" is meant to encompass a single processor or processing unit, multiple processors, multiple processing units, or other suitably configured computing element or elements.

The memory <NUM> can store electronic data that can be used by the computing system <NUM>. For example, the memory <NUM> can store electrical data or content such as, for example, audio and video files, documents and applications, device settings and user preferences, timing and control signals or data for the various modules, data structures or databases, and so on. The memory <NUM> can be configured as any type of memory. By way of example only, the memory <NUM> can be implemented as random access memory, read-only memory, Flash memory, removable memory, or other types of storage elements, or combinations of such devices.

The computing system <NUM> can further include a display unit <NUM> for displaying visual information for a user. The display unit <NUM> can provide visual (e.g., image or video) output. The display unit <NUM> can be or include an opaque, transparent, and/or translucent display. The display unit <NUM> may have a transparent or translucent medium through which light representative of images is directed to a user's eyes. The display unit <NUM> may utilize digital light projection, OLEDs, LEDs, uLEDs, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium may be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, the transparent or translucent display may be configured to become opaque selectively. Projection-based systems may employ retinal projection technology that projects graphical images onto a person's retina. Projection systems also may be configured to project virtual objects into the physical environment, for example, as a hologram or on a physical surface. The computing system <NUM> can include an optical subassembly configured to help optically adjust and correctly project the image based content being displayed by the display unit <NUM> for close up viewing. The optical subassembly can include one or more lenses, mirrors, or other optical devices.

The computing system <NUM> can include an input/output component <NUM>, which can include any suitable component for connecting computing system <NUM> to other devices. Suitable components can include, for example, audio/video jacks, data connectors, or any additional or alternative input/output components. The input/output component <NUM> can include buttons, keys, or another feature that can act as a keyboard for operation by the user. Input/output component <NUM> may include a microphone. The microphone may be operably connected to the processor <NUM> for detection of sound levels and communication of detections for further processing, as described further herein. Input/output component <NUM> also may include speakers. The speakers can be operably connected to the processor <NUM> for control of speaker output, including sound levels, as described further herein.

The computing system <NUM> can include one or more other sensors <NUM>. Such sensors can be configured to sense substantially any type of characteristic such as, but not limited to, images, pressure, light, touch, force, temperature, position, motion, and so on. For example, the sensor can be a photodetector, a temperature sensor, a light or optical sensor, an atmospheric pressure sensor, a humidity sensor, a magnet, a gyroscope, an accelerometer, a chemical sensor, an ozone sensor, a particulate count sensor, and so on. By further example, the sensor can be a bio-sensor for tracking biometric characteristics, such as health and activity metrics. Other user sensors can perform facial feature detection, facial movement detection, facial recognition, eye tracking, user mood detection, user emotion detection, voice detection, etc..

The computing system <NUM> can include communications circuitry <NUM> for communicating with one or more servers or other devices using any suitable communications protocol. For example, communications circuitry <NUM> can support Wi-Fi (e.g., a <NUM> protocol), Ethernet, Bluetooth, high frequency systems (e.g., <NUM>, <NUM>, and <NUM> communication systems), infrared, TCP/IP (e.g., any of the protocols used in each of the TCP/IP layers), HTTP, BitTorrent, FTP, RTP, RTSP, SSH, any other communications protocol, or any combination thereof. Communications circuitry <NUM> can also include an antenna for transmitting and receiving electromagnetic signals.

The computing system <NUM> can include a battery <NUM>, which can charge and/or power components of the computing system <NUM>. The battery can also charge and/or power components connected to the computing system <NUM>.

While various embodiments and aspects of the present disclosure are illustrated with respect to a head-mountable device, it will be appreciated that the subject technology can encompass and be applied to other electronic devices. Such an electronic device can be or include a desktop computing device, a laptop-computing device, a display, a television, a portable device, a phone, a tablet computing device, a mobile computing device, a wearable device, a watch, and/or a digital media player.

According to aspects of the subject technology, a method is provided that includes receiving a request to transition a computing system from a first state to a second state, and compiling a respective manifest for each of a plurality of processors of the computing system. Each manifest comprises a transition identifier representing a command to transition from the first state to the second state and an action time for executing one or more operations associated with the transition identifier. The respective manifests are dispatched to the plurality of processors, and status reports are received from the plurality of processors regarding the transition from the first state to the second state.

The method further includes receiving a manifest time associated with the transition identifier from each of the plurality of processors, wherein the manifest time corresponds to an advance time required by the plurality of processors before executing the one or more operations, and wherein the action time is based on the manifest time. The action time may be based on a maximum manifest time received from the plurality of processors. The action time may be further based on a transport time, wherein the transport time corresponds to a period of time between dispatching the respective manifests to the plurality of processors and receiving the respective manifests by the plurality of processors.

The method may further include receiving an execution time associated with the transition identifier from each of the plurality of processors, wherein the execution time corresponds to a period of time approximated for executing the one or more operations associated with the transition identifier. The action time may be based on the execution time associated with a previous manifest dispatched to the plurality of processors. The method may further include setting a timer based on the action time and the execution time, and polling the plurality of processors upon expiration of the timer, wherein the status reports are received from the plurality of processors in response to the polling.

The method may further include notifying the plurality of processors about the manifests before dispatching the plurality of manifests to the plurality of processors, and receiving a success indicator from the plurality of processors in response to notifying the plurality of processors about the manifests.

According to aspects of the subject technology, a non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations is provided. The operations include receiving a request to transition a computing device from a first state to a second state and compiling a respective manifest for each of a plurality of processors of the computing device. Each manifest comprises a transition identifier representing a command to transition from the first state to the second state and an action time for executing one or more operations associated with the transition identifier. The operations further include dispatching the respective manifests to the plurality of processors, polling the plurality of processors upon expiration of a timer, and receiving status reports from the plurality of processors regarding the transition from the first state to the second state in response to the polling.

The operations further include receiving a manifest time associated with the transition identifier from each of the plurality of processors, wherein the manifest time corresponds to an advance time required by the plurality of processors before executing the one or more operations, and wherein the action time is based on the manifest time. The action time may be based on a maximum manifest time received from the plurality of processors. The action time may be further based on a transport time, wherein the transport time corresponds to a period of time between dispatching the respective manifests to the plurality of processors and receiving the respective manifests by the plurality of processors.

The operations may further include receiving an execution time associated with the transition identifier from each of the plurality of processors, wherein the execution time corresponds to a period of time approximated for executing the one or more operations associated with the transition identifier. The action time may be based on the execution time associated with a previous manifest dispatched to the plurality of processors.

The operations may further include notifying the plurality of processors about the manifests before dispatching the plurality of manifests to the plurality of processors, and receiving a success indicator from the plurality of processors in response to notifying the plurality of processors about the manifests.

According to aspects of the subject technology, a device is provided that includes a memory storing a plurality of computer programs, and one or more processors configured to execute instructions of the plurality of computer programs to receive a request to transition the device from a first state to a second state, and receive, from each of a plurality of processors a manifest time associated with a transition identifier representing a command to transition from the first state to a second state. The manifest time corresponds to an advance time required by the plurality of processors before executing one or more operations associated with the transition identifier. A respective manifest is compiled for each of the plurality of processors of the device, wherein each manifest comprises the transition identifier and an action time for executing the one or more operations associated with the transition identifier, wherein the action time is based on the manifest time, the respective manifests are dispatched to the plurality of processors, and status reports are received from the plurality of processors regarding the transition from the first state to the second state.

The action time may be further based on a transport time, and wherein the transport time corresponds to a period of time between dispatching the respective manifests to the plurality of processors and receiving the respective manifests by the plurality of processors. The one or more processors may be further configured to execute instructions to receive an execution time associated with the transition identifier from each of the plurality of processors, wherein the execution time corresponds to a period of time approximated for executing the one or more operations associated with the transition identifier.

The one or more processors may be further configured to execute instructions to set a timer based on the action time and the execution time, and poll the plurality of processors upon expiration of the timer, wherein the status reports are received from the plurality of processors in response to the polling. The one or more processors may be further configured to execute instructions to notify the plurality of processors about the manifests before dispatching the plurality of manifests to the plurality of processors, and receive a success indicator from the plurality of processors in response to notifying the plurality of processors about the manifests.

The collection and transfer of data from an application to other computing devices may occur. The present disclosure contemplates that in some instances, this collected data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data can include demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, images, data or records relating to a user's health or level of fitness (e.g., vital signs measurements, medication information, exercise information), date of birth, or any other personal information.

The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users. Uses for personal information data that benefit the user are also contemplated by the present disclosure. For instance, health and fitness data may be used, in accordance with the user's preferences to provide insights into their general wellness, or may be used as positive feedback to individuals using technology to pursue wellness goals.

Despite the foregoing, the present disclosure also contemplates implementations in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, in the case of video conferencing, the present technology can be configured to allow users to select to "opt in" or "opt out" of participation in the collection of personal information data during registration for services or anytime thereafter. In addition to providing "opt in" and "opt out" options, the present disclosure contemplates providing notifications relating to the access or use of personal information. For instance, a user may be notified upon downloading an app that their personal information data will be accessed and then reminded again just before personal information data is accessed by the app.

A phrase "at least one of" preceding a series of items, with the terms "and" or "or" to separate any of the items, modifies the list as a whole, rather than each member of the list. By way of example, each of the phrases "at least one of A, B, and C" or "at least one of A, B, or C" refers to only A, only B, or only C; any combination of A, B, and C; and/or at least one of each of A, B, and C.

It is understood that the specific order or hierarchy of steps, operations, or processes disclosed is an illustration of exemplary approaches. Unless explicitly stated otherwise, it is understood that the specific order or hierarchy of steps, operations, or processes may be performed in different order. Some of the steps, operations, or processes may be performed simultaneously. The accompanying method claims, if any, present elements of the various steps, operations or processes in a sample order, and are not meant to be limited to the specific order or hierarchy presented. These may be performed in serial, linearly, in parallel or in different order. It should be understood that the described instructions, operations, and systems can generally be integrated together in a single software/hardware product or packaged into multiple software/hardware products.

In one aspect, a term coupled or the like may refer to being directly coupled. In another aspect, a term coupled or the like may refer to being indirectly coupled.

Terms such as top, bottom, front, rear, side, horizontal, vertical, and the like refer to an arbitrary frame of reference, rather than to the ordinary gravitational frame of reference. Thus, such a term may extend upwardly, downwardly, diagonally, or horizontally in a gravitational frame of reference.

Claim 1:
A method, comprising:
receiving a request to transition a computing system from a first state to a second state;
compiling a respective manifest for each of a plurality of processors of the computing system, wherein each manifest comprises a transition identifier representing a command to transition from the first state to the second state and an action time for executing, by each of the plurality of processors, one or more operations associated with the transition identifier, wherein the action time is based on a respective manifest time received from each respective processor of the plurality of processors, wherein the manifest time corresponds to an advance time required by the plurality of processors before executing the one or more operations;
dispatching the respective manifests to the plurality of processors; and
receiving status reports from the plurality of processors regarding the transition from the first state to the second state.