USER INTERFACES FOR DEVICE CONTROLS

The present disclosure generally relates to displaying user interfaces with device controls.

FIELD

The present disclosure relates generally to computer user interfaces, and more specifically to techniques for controlling external devices.

BACKGROUND

Computer systems can include hardware and/or software for providing improved techniques for a user preferring to use different input mechanisms and/or for a user having one or more impairments (e.g., motor impairment and/or visual impairment) to interact with the computer.

BRIEF SUMMARY

Users may prefer to use alternative input techniques and/or may have a limited ability to provide certain inputs to control a computer (e.g., using a mouse and/or a touchscreen). A user may configure particular accessibility settings of a computer system to allow the user to more easily control a device. A user may request to control one computer system using a different computer system (e.g., an external device).

Some techniques for controlling external devices using electronic devices, however, are generally cumbersome and inefficient. For example, some existing techniques use a complex and time-consuming user interface, which may include multiple key presses or keystrokes. Existing techniques require more time than necessary, wasting user time and device energy. This latter consideration is particularly important in battery-operated devices.

Accordingly, the present technique provides electronic devices with faster, more efficient methods and interfaces for controlling external devices. Such methods and interfaces optionally complement or replace other methods for controlling external devices. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated computing devices, such methods and interfaces conserve power and increase the time between battery charges. When a user desires to control an external (e.g., remote) device, automatically displaying a particular user interface based on the accessibility settings of the one or more devices (e.g., a local device and/or an external device) reduces the number of inputs required to control the one or more devices. Additionally or alternatively, automatically displaying a particular user interface to control an external device based on an accessibility setting improves a human-machine interface for users.

In accordance with some embodiments, a method performed at a first computer system that is in communication with a display generation component and one or more input devices is described. The method comprises: detecting, via the one or more input devices, an input corresponding to a request to control a second computer system; and in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with a determination that a first set of control criteria is met, displaying one or more first user interface objects that controls a graphical element displayed by the second computer system to navigate one or more user interfaces of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system is enabled on the second computer system; and in accordance with a determination that the first set of control criteria is not met, forgoing display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system.

In accordance with some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a first computer system that is in communication with a display generation component and one or more input devices, the one or more programs including instructions for: detecting, via the one or more input devices, an input corresponding to a request to control a second computer system; and in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with a determination that a first set of control criteria is met, displaying one or more first user interface objects that controls a graphical element displayed by the second computer system to navigate one or more user interfaces of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system is enabled on the second computer system; and in accordance with a determination that the first set of control criteria is not met, forgoing display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system.

In accordance with some embodiments, a transitory computer-readable storage medium is described. The transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a first computer system that is configured to communicate with a display generation component and one or more input devices, the one or more programs including instructions for: detecting, via the one or more input devices, an input corresponding to a request to control a second computer system; and in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with a determination that a first set of control criteria is met, displaying one or more first user interface objects that controls a graphical element displayed by the second computer system to navigate one or more user interfaces of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system is enabled on the second computer system; and in accordance with a determination that the first set of control criteria is not met, forgoing display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system.

In accordance with some embodiments, a first computer system that is configured to communicate with a display generation component and one or more input devices is described. The computer system comprises: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: detecting, via the one or more input devices, an input corresponding to a request to control a second computer system; and in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with a determination that a first set of control criteria is met, displaying one or more first user interface objects that controls a graphical element displayed by the second computer system to navigate one or more user interfaces of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system is enabled on the second computer system; and in accordance with a determination that the first set of control criteria is not met, forgoing display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system.

In accordance with some embodiments, a first computer system that is configured to communicate with a display generation component and one or more input devices is described. The computer system comprises: means for detecting, via the one or more input devices, an input corresponding to a request to control a second computer system; and means, responsive to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system, for: in accordance with a determination that a first set of control criteria is met, displaying one or more first user interface objects that controls a graphical element displayed by the second computer system to navigate one or more user interfaces of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system is enabled on the second computer system; and in accordance with a determination that the first set of control criteria is not met, forgoing display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system.

In accordance with some embodiments, a computer program product is described. The computer program product comprises one or more programs configured to be executed by one or more processors of a computer system that is that is in communication with a display generation component and one or more input devices. The one or more programs include instructions for: detecting, via the one or more input devices, an input corresponding to a request to control a second computer system; and in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with a determination that a first set of control criteria is met, displaying one or more first user interface objects that controls a graphical element displayed by the second computer system to navigate one or more user interfaces of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system is enabled on the second computer system; and in accordance with a determination that the first set of control criteria is not met, forgoing display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system.

Thus, devices are provided with faster, more efficient methods and interfaces for controlling external devices, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace other methods for controlling external devices.

DESCRIPTION OF EMBODIMENTS

There is a need for electronic devices that provide efficient methods and interfaces for controlling external devices. For example, a technique enables the display of a particular user interface based on what accessibility settings are enabled at one or more devices. Such techniques can reduce the cognitive burden on a user who seeks to control the external device, thereby enhancing productivity. Further, such techniques can reduce processor and battery power otherwise wasted on redundant user inputs.

Below,FIGS.1A-1B,2,3, and4A-4B, and5A-5Bprovide a description of exemplary devices for performing the techniques for managing event notifications.FIGS.6A-6Rillustrate exemplary user interfaces for controlling external devices.FIG.7is a flow diagram illustrating methods of displaying a user interface to control a device in accordance with some embodiments. The user interfaces inFIGS.6A-6Rare used to illustrate the processes described below, including the processes inFIG.7.

The processes described below enhance the operability of the devices and make the user-device interfaces more efficient (e.g., by helping the user to provide proper inputs and reducing user mistakes when operating/interacting with the device) through various techniques, including by providing improved visual feedback to the user, reducing the number of inputs needed to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, performing an operation when a set of conditions has been met without requiring further user input, controlling devices having one or more accessibility settings, improving the human-machine interface for a user, and/or additional techniques. These techniques also reduce power usage and improve battery life of the device by enabling the user to use the device more quickly and efficiently.

Although the following description uses terms “first,” “second,” etc. to describe various elements, these elements should not be limited by the terms. In some embodiments, these terms are used to distinguish one element from another. For example, a first touch could be termed a second touch, and, similarly, a second touch could be termed a first touch, without departing from the scope of the various described embodiments. In some embodiments, the first touch and the second touch are two separate references to the same touch. In some embodiments, the first touch and the second touch are both touches, but they are not the same touch.

I/O subsystem106couples input/output peripherals on device100, such as touch screen112and other input control devices116, to peripherals interface118. I/O subsystem106optionally includes display controller156, optical sensor controller158, depth camera controller169, intensity sensor controller159, haptic feedback controller161, and one or more input controllers160for other input or control devices. The one or more input controllers160receive/send electrical signals from/to other input control devices116. The other input control devices116optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and so forth. In some embodiments, input controller(s)160are, optionally, coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. The one or more buttons (e.g.,208,FIG.2) optionally include an up/down button for volume control of speaker111and/or microphone113. The one or more buttons optionally include a push button (e.g.,206,FIG.2). In some embodiments, the electronic device is a computer system that is in communication (e.g., via wireless communication, via wired communication) with one or more input devices. In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a trackpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors164and/or one or more depth camera sensors175), such as for tracking a user's gestures (e.g., hand gestures and/or air gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system. In some embodiments, an air gesture is a gesture that is detected without the user touching an input element that is part of the device (or independently of an input element that is a part of the device) and is based on detected motion of a portion of the user's body through the air including motion of the user's body relative to an absolute reference (e.g., an angle of the user's arm relative to the ground or a distance of the user's hand relative to the ground), relative to another portion of the user's body (e.g., movement of a hand of the user relative to a shoulder of the user, movement of one hand of the user relative to another hand of the user, and/or movement of a finger of the user relative to another finger or portion of a hand of the user), and/or absolute motion of a portion of the user's body (e.g., a tap gesture that includes movement of a hand in a predetermined pose by a predetermined amount and/or speed, or a shake gesture that includes a predetermined speed or amount of rotation of a portion of the user's body).

Device100optionally also includes one or more depth camera sensors175.FIG.1Ashows a depth camera sensor coupled to depth camera controller169in I/O subsystem106. Depth camera sensor175receives data from the environment to create a three dimensional model of an object (e.g., a face) within a scene from a viewpoint (e.g., a depth camera sensor). In some embodiments, in conjunction with imaging module143(also called a camera module), depth camera sensor175is optionally used to determine a depth map of different portions of an image captured by the imaging module143. In some embodiments, a depth camera sensor is located on the front of device100so that the user's image with depth information is, optionally, obtained for video conferencing while the user views the other video conference participants on the touch screen display and to capture selfies with depth map data. In some embodiments, the depth camera sensor175is located on the back of device, or on the back and the front of the device100. In some embodiments, the position of depth camera sensor175can be changed by the user (e.g., by rotating the lens and the sensor in the device housing) so that a depth camera sensor175is used along with the touch screen display for both video conferencing and still and/or video image acquisition.

Each of the above-identified elements inFIG.3is, optionally, stored in one or more of the previously mentioned memory devices. Each of the above-identified modules corresponds to a set of instructions for performing a function described above. The above-identified modules or computer programs (e.g., sets of instructions or including instructions) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), procedures, or modules, and thus various subsets of these modules are, optionally, combined or otherwise rearranged in various embodiments. In some embodiments, memory370optionally stores a subset of the modules and data structures identified above. Furthermore, memory370optionally stores additional modules and data structures not described above.

As used here, the term “affordance” refers to a user-interactive graphical user interface object that is, optionally, displayed on the display screen of devices100,300, and/or500(FIGS.1A,3, and5A-5B). For example, an image (e.g., icon), a button, and text (e.g., hyperlink) each optionally constitute an affordance.

As used herein, the terms “open application” or “executing application” refer to a software application with retained state information (e.g., as part of device/global internal state157and/or application internal state192). An open or executing application is, optionally, any one of the following types of applications:an active application, which is currently displayed on a display screen of the device that the application is being used on;a background application (or background processes), which is not currently displayed, but one or more processes for the application are being processed by one or more processors; anda suspended or hibernated application, which is not running, but has state information that is stored in memory (volatile and non-volatile, respectively) and that can be used to resume execution of the application.

FIGS.6A-6Rillustrate exemplary user interfaces for controlling a device, in accordance with some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes inFIG.7.

FIGS.6A-6Dillustrates exemplary graphical user interfaces for accessibility settings. The accessibility settings provide different options that can enable different modes of a device so that a user can better interact with and use devices600and614. These different options in the accessibility settings allow a user to control a local device (e.g., device600) and/or remote device (e.g., device614) in different manners.

AtFIG.6A, device600displays interface602aof an accessibility settings menu. In some embodiments, device600includes one or more features of devices100,300, and/or500. Interface602aincludes menu option604to access a setting for screen reader and menu option606to access a setting for hand gestures. As explained in greater detail herein, when screen reader is enabled, device600will output a simulated voice of the contents of a user interface being displayed (e.g., sequentially) so that a user can hear different options provided by the user interface. When hand gestures are enabled, device600will detect air-based movements or actions when the device is worn by a user (e.g., on a user's wrist and/or on a user's head). Using one or more sensors (e.g., an optical sensor (e.g., a heart rate sensor), a camera), a gyroscope, an accelerometer)), device600can distinguish between different hand gestures and execute a corresponding command. In some embodiments, device600detects the hand gestures by detecting changes in the user's wrist, blood flow, and/or heartrate, rather than detecting the gestures using a camera (e.g., detecting hand movement or finger placement using the camera). In some embodiments, device600does not use a camera that has the user's hand in the field of view of the camera to detect hand gestures.

AtFIG.6A, while displaying interface602a, device600detects input650a(e.g., a tap gesture, and/or a mouse click). Interface602ais a settings user interface that enables the user to modify how the user of device600interacts/interfaces with device600. In response to detecting input650a, device600displays interface602bof a settings menu, as depicted inFIG.6B.

AtFIG.6B, interface602bincludes menu option608to display hand gestures that are available. While displaying interface602b, device600detects input650b(e.g., a tap gesture and/or a mouse click) directed to menu option608. In response to detecting input650b, device600displays interface602cof the settings menu, as depicted inFIG.6C.

AtFIG.6C, interface602cincludes menu options for customizing a particular gesture to execute a particular command (e.g., function). Interface602calso includes a toggle610to enable (and/or disable) gestures. As illustrated inFIG.6C, toggle610is enabled. As such, device600will detect and act on hand gestures that it would otherwise not detect and/or act on had the toggle been disabled. In some embodiments, device600can distinguish between different hand gestures using one or more sensors (e.g., an accelerometer, a gyroscope, and/or an optical sensor (e.g., one or more cameras and/or a heart rate sensor)). In some embodiments, the device600detects gestures based how device600is worn. As illustrated, device600is a watch that is worn on a user's wrist. As such, in some embodiments, device600detects different hand gestures as described in and referred to as hand gestures in U.S. Provisional Application 63/221,331, which is hereby incorporated by reference in its entirety. In some implementations, hand gestures (e.g., a double-pinch gesture) can initiate commands such as: answer or end a phone call, dismiss a notification, take a photo, play or pause media, and start, pause, or resume a workout on device600(and/or device614). In some embodiments, device600(and/or device614) displays instructions that indicate the hand gesture for executing a particular command using assistive touch (e.g., double pinch to play music). For example, in response to receiving a call a device600, device600can display an instruction to answer the call by using clench gesture609c.

AtFIG.6C, functions can be customizable for multiple gestures. In some embodiments, to customize a particular hand gesture, device600provides an affordance for specific hand gestures, including a pinch affordance608afor pinch gesture609a, double-pinch affordance608bfor double-pinch gesture609b(e.g., two pinches occurring within a period of time and/or in successive order), clench affordance608cfor clench gesture609c, and/or double-clench affordance608dfor double-clench gesture609d(e.g., two clenches occurring within a period of time and/or in successive order), in some embodiments. In some embodiments, if device600were a device worn on a different part of the body, device600provides affordances for different gestures. In embodiments, where device600is a head-worn device, gestures associated with affordances608a-doptionally include a hand gestures performed in such a manner that the gestures are detected in a field of view of one or more cameras of the head-worn device. In some embodiments, for a head-word device, a single gesture includes detected movement of two hands of the user.

AtFIG.6C, while displaying interface602c, device600detects input650c(e.g., a tap gesture and/or a mouse click) on double pinch affordance608b. In response to detecting input650c, device600displays interface602dof a settings menu, as depicted inFIG.6D.

AtFIG.6D, interface602dincludes affordances612a-cfor configuring (e.g., designating or setting) a double-pinch gesture to execute a command (e.g., function and/or operation) of a visual indicator (e.g., visual indicator622) (e.g., a highlighting and/or a selector). As illustrated, affordances612a-cinclude affordance612ato customize a command for advancing the visual indicator forwards (e.g., onto a subsequent item that is in a series of items) in response to detecting the respective gesture, affordance612bto customize a command for moving the visual indicator backwards (e.g., onto a preceding item that is in the series of items) in response to detecting the respective gesture, and/or affordance612cto customize a command that initiates a selection (e.g., of an item identified by the visual indicator and/or of a highlighted item) in response to detecting the respective gesture.

AtFIG.6D, options612includes options to execute a system action (e.g., displaying a user interface for notification and/or displaying a control user interface). As illustrated, options612provides an option for configuring a gesture to display action menu612d, as described in greater detail below. Referring briefly toFIG.6C, device600is configured so a double-clench executes a command to display action menu612d.

AtFIG.6D, affordances612e-612ginclude affordances to execute an input provided by rotatable input device601in response to detecting a hand gesture. As illustrated, affordances612e-612ginclude affordance612eto configure simulating a press (e.g., depress) of rotatable input device601to a hand gesture, affordance612fto configure simulating rotation of rotatable input device601in a first direction (e.g., up and/or clockwise) to a hand gesture, and/or affordance612gto configure simulating rotation of rotatable input device601in a second direction (e.g., down and/or counter-clockwise) to a hand gesture.

AtFIG.6D, options612include options to execute a shortcut. Shortcuts can include one or more (e.g., two or three successive commands) commands that are configured by the user, such as a shortcut for opening specific applications, automatically taking an action within the application, and/or modifying system settings (e.g., silence notifications and/or turn on airplane mode). As illustrated, interface602dincludes an option to execute workout shortcut612h, as described in greater detail with respect toFIG.6F. In some embodiments, interface602includes other shortcuts, such as a commute shortcut612i.

AtFIG.6D, in some embodiments, shortcuts are configurable by device600and/or device614. In some embodiments, device600and device614are in wireless communication, where system settings, modes, locations, and/or commands (e.g., inputs and/or system actions) are communicated between the devices. In some embodiments, device600and device614are logged into the same user account and communicate over a network. In some embodiments, device600and device614communicate directly, for example, using short-range wireless communications.

AtFIG.6D, options612are displayed with respect to a double-pinch gesture. However, options612can be configured for other hand gestures, such as for a pinch gesture, a clench gesture, and a double clench gesture.

AtFIG.6D, while displaying the option to execute workout shortcut612h, device600detects an input650d(e.g., a tap and/or mouse click) on the option to execute workout shortcut612h. In response, device600configures a double pinch gesture to correspond to a command that executes workout shortcut612h. Once the double pinch gesture is corresponded to (assigned to) the command that executes workout shortcut612h, device600will execute workout shortcut612hin response to detecting that the user of device600has performed the double pinch gesture.

AtFIG.6E, a user has navigated device600to user interface616a. While displaying interface616a, device600detects double pinch gesture609b(e.g., in the air, without touching a touch-sensitive display, and/or without touching a button of device600). In response to detecting double pinch gesture609b, device600executes workout shortcut612k, which includes causing device600and/or device614to display respective workout interface616b, as depicted inFIG.6F.

AtFIG.6F, device600has executed workout shortcut612h. Workout shortcut612hoptionally includes a command to open a media application and a command play (e.g., automatically play) a song using the media application. As such, multiple commands can be executed in response to detecting a gesture (e.g., a single hand gesture). As illustrated, “workout song 1” is playing in user interface616bof the media application.

AtFIG.6F, in some embodiments, hand gestures detected by device600cause a command to be executed at device614(and/or device600). In some embodiments, device600(and/or device614) displays instructions that indicate the hand gesture for executing a particular command using assistive touch. For example, while displaying interface618b, device600(and/or device614) can display an indication that a specific gesture causes device600(and/or device614) to play and/or pause the music (e.g., double pinch to pause music) (e.g., as opposed to launch a shortcut). In some embodiments, the indication indicates that double pinch gesture609bpauses music. In some embodiments, the indication indicates that a triple pinch gesture pauses music. In some embodiments, the indication indicates that a triple pinch gesture plays the music. In some embodiments, hand gestures detected by device600cause device600to launch a shortcut. For example, in response to detecting double pinch gesture609batFIG.6E, device614executes workout shortcut612h, which causes display of the media application at device614, as depicted inFIG.6F. In some embodiments, based on a set of criteria (e.g., proximity of device600and device614, device600and device614are signed in the same user account, device600and device614are both unlocked, and/or device600and device614are paired), device600causes device614to display the media application when workout shortcut612his executed. In some embodiments, the shortcut causes display of respective media applications at both devices600and614. In some embodiments, workout shortcut causes display of the media application at only one of devices600and614(and not at the other).

AtFIG.6G, a user has navigated back to interface616aand device614is displaying interface618a. In some embodiments, device614displays a lock screen instead of interface618a. In some embodiments, device614is in a low power state and/or has a dim display (e.g., in response to timing out and/or not receiving user input for a threshold period of time) while displaying interface618aor the lockscreen.

AtFIG.6G, while displaying interface616a, device600detects an input (e.g., a series of touch inputs to display action menu612dand/or double clench gesture609d). In response to detecting the input atFIG.6G, device600displays action menu612d, as depicted inFIG.6H.

AtFIG.6H, action menu612dincludes affordances to perform specific actions. As illustrated, action menu612dincludes affordance620which, when selected, causes display of one or more shortcuts (e.g., commute shortcut612iand/or workout shortcut612h). Action menu612dalso includes affordance623which, when selected, causes display of a list of external (e.g., remote) devices that are controllable by device600, as described in greater detail below.

AtFIG.6H, device600displays visual indicator622(e.g., highlighting, shading, symbol, cross-hairs, and/or shape) to distinguish affordance624from affordance623. Visual indicator622provides an indication that device600will execute a command (e.g., selection and/or touch and drag) with respect to the user interface object (e.g., application icon, menu option, and/or affordance) that visual indicator622is associated with. In some embodiments, device600will move (e.g. translate and/or shift) visual indicator622from one user interface object to another in response to detecting user input (e.g., hand gestures). In some embodiments, device600displays an animation of visual indicator622moving. In some embodiments, device600does not display an animation of visual indicator622moving.

AtFIG.6H, prior to displaying visual indicator622on affordance623, device600displays visual indicator622on affordance624(e.g., upon initial display of action menu612d). Device600moves visual indicator622sequentially through action menu612din response to detecting an input corresponding to a request to move visual indicator622. For example, in response to detecting an input (e.g., pinch gesture609aand/or touch input650a), device600ceases to display visual indicator622at affordance624and displays visual indicator622on affordance620. In response to detecting another input (e.g., pinch gesture609aand/or touch input), device600ceases to display visual indicator622on affordance620and displays visual indicator622on affordance623.

AtFIG.6H, while displaying visual indicator622on affordance623, device600detects an input (e.g., clench gesture609c, mouse click, and/or tap650h) to select affordance623. In response to detecting the input to select affordance623, device600displays device selector menu626, as depicted inFIG.6I.

AtFIG.6I, device600displays device selector menu626. Device selector menu626provides options to select an external device, such as device614or a different external device, to control. In some embodiments, device selector menu626includes options to control a tablet computer, a phone (e.g., device614), and/or a computer (e.g., laptop and/or desktop). In some embodiments, other computer systems (e.g., smart TVs and/or a head mounted device) can be controlled by device626.

AtFIG.6I, in some embodiments, device600and device614(and/or other devices in device selector menu626) are logged into the same user account. In some embodiments, device selector menu626includes (e.g., only) options to control devices that are logged into the same account as device600. In some embodiments, device selector menu626includes options to control devices that are within a threshold distance of each other (e.g., both devices are located in the same room, home, and/or city). For instance, device600and/or device614can communicate distance based on GPS data, based on being within short-range communication range, and/or based on being connected to the same LAN. In some embodiments, device selector menu626includes a first set of device types and excludes a second set of device types different than the first set of device types. For instance, larger devices (e.g., larger display area and/or physical dimensions) than device600are included while devices with displays that are smaller than device600are excluded (even if the devices with smaller displays are within the threshold distance and/or logged into the same user account as device600). As a further example, phones, tablets, laptops, and/or desktops are included in device selector menu626while smart televisions and/or streaming computing devices for a television are not (even if the devices are within the threshold distance and/or logged into the same user account as device600).

AtFIG.6I, in some embodiments, device600and device614are in communication (e.g., wireless communication, such as LAN, Bluetooth, and/or via a network) prior to receiving the input selecting the option to control device614. In some embodiments, device600and device614are/stay in communication after receiving input ofFIG.6I(e.g., while device600controls device614atFIGS.6J-6P). In some embodiments, device600and device614stay in communication when device600is no longer controlling device614.

AtFIG.6I, in some embodiments, device600and device614are in wireless communication, where system settings (e.g., accessibility settings), modes, and/or commands (e.g., inputs, system actions, and/or shortcuts) are communicated between the devices prior to receiving an input selecting an option to control device614. In some embodiments, system settings (e.g., accessibility settings), modes, and/or commands (e.g., inputs and/or system actions) are communicated between devices600and614in response to receiving an input at device600selecting an option to control device614.

AtFIG.6I, while displaying device selector menu626, device600detects an input (e.g., clench gesture609c, mouse click, and/or tap650i) selecting an option to control device614. In response to detecting the input selecting an option to control device614, device600displays a specific user interface depending on how accessibility settings are configured at device614and/or device600. As described herein, atFIGS.6J-6L, device600displays user interface630in response to the input selecting the option to control device614and when screen reader is disabled on device600and adaptive navigation is enabled on device614. AtFIGS.6M-6P, device600displays user interface632in response to detecting the input selecting the option to control device614and when screen reader is enabled on device600and adaptive navigation is disabled on device614. Automatically displaying user interface630or user interface632based on how devices600and device614are configured causes device600to display the appropriate controls based on how devices600and device614are configured, which reduces the number of inputs at device600(and/or device614) and provides visual feedback as to how each device is configured.

AtFIGS.6J-6L, screen reader is disabled on device600and adaptive navigation is enabled on device614. Adaptive navigation is an additional (or, optionally, alternative) input setting that allows a user (e.g., a user with a motor impairment) to navigate a user interface using user-configurable controls and/or devices. Turning briefly toFIG.6R, adaptive navigation is enabled (e.g., using a toggle and/or button) under a system setting (e.g., accessibility setting) of device614. Screen reader is enabled at device600by navigating into menu option605for accessibility settings, described inFIG.6A.

AtFIG.6J, device614displays visual indicator622based on adaptive navigation being enabled at device614. In some embodiments, adaptive navigation is enabled at device614(e.g., by user input detected at device614) prior to device600controlling device614. In some embodiments, device614displays visual indicator622prior to device600controlling device614if adaptive navigation is enabled prior to device600controlling device614. In some embodiments, prior to (or after) using device600to control device614, a user controls device614using visual indicator622and inputs at device614. In some embodiments, visual indicator622displayed on device614has a similar or different appearance as visual indicator622that is displayed on device600. In some embodiments, visual indicator622that is displayed on device600ceases to be displayed in response to detecting that device600is controlling device614. In some embodiments, visual indicator622continues to be displayed on device600while device600controls device614. In such embodiments, visual indicator622at device600is controlled using hand gestures to navigate and select affordances (e.g., affordances638and/or system affordances640) which, in turn, initiates a command to move visual indicator622at device614.

AtFIG.6J, in some embodiments, a device displays notification636(e.g., a notification overlaid on a user interface) that a device is being controlled (or is controlling) another device. As illustrated, device600displays notification636indicating that device600is ready to control device614. Device614displays notification636indicating that device614is controlled by device600. In some embodiments, notification636is persistent (e.g., always displayed while device600controls device614). In some embodiments, notification636is temporary (e.g., device614displays notification636for a threshold amount of time and then ceases to display notification636once the threshold amount of time is reached).

AtFIG.6J, in response to the input received atFIG.6I, and based on screen reader being disabled at device600and adaptive navigation being enabled at device614, device600displays interface630. Interface630is an interface that includes one or more user interface objects for controlling device614. As illustrated, interface630includes one or more affordances638(e.g., next affordance638ato advance forwards, back affordance368bto advance backwards, and select affordance638c) (e.g., similar to input controls612a-612cofFIG.6D) for controlling visual indicator622displayed on device614. Interface630includes one or more system affordances640for controlling system actions (similar to system actions611ofFIG.6D) of device614. In some embodiments, one or more system affordances640include an affordance for displaying open applications (e.g., an application switcher) of device614and/or an affordance for displaying a home screen of device614.

AtFIG.6J, in some embodiments, device614responds to inputs received at device614(e.g., a user can still control device614as the user typically would despite device614concurrently being controlled by device600). For instance, controlling device614by input at600does not disable device614from detecting input received at device614. In some embodiments, device614does not respond to input at device614while controlled by device600(e.g., a user cannot control device614using input at device614based on being controlled by device600). In some embodiments, device614does not display interface630(e.g., and/or one or more of its user interface objects) while device600displays interface630.

AtFIG.6J, while displaying interface630, device600detects an input (e.g., pinch gesture609aand/or tap650jon next affordance638) to move visual indicator622. In response to detecting the input to move visual indicator622, device600initiates a command for device614to move (e.g., advance) visual indicator622in user interface618a(e.g., from mail application642ato calendar application642b), as depicted inFIG.6K.

AtFIG.6K, while displaying interface630, device600detects an input (e.g., clench gesture609cand/or tap650k) corresponding to a selection. In response to device600detecting the input corresponding to the selection, device600initiates a command for device614to perform a selection. Because visual indicator622is on calendar application642bwhen the input (e.g., clench gesture609cand/or tap650k) is received, device614selects calendar application642bwhich causes display of user interface618a, as depicted inFIG.6L.

AtFIG.6L, device614displays user interface618cof calendar application642b. In some embodiments, in response to detecting further inputs (e.g., hand gestures and/or touch inputs) at device600, device600optionally moves visual indicator622displayed on device614. As described in further detail with reference toFIG.6N, in some embodiments, visual indicator622is moved through a user interface (e.g., interface618aand/or user interface618c) more rapidly in response to device600detecting a rotational input of rotatable input device601and/or a hand gesture corresponding to (e.g., simulating) crown rotation (e.g., hand gesture corresponding to (e.g., simulating) a rotation of rotatable input device601in the first direction and/or a rotation of rotatable input device601in the second direction).

In some embodiments, interface630includes done affordance644that terminates the session of device600controlling device614using interface630. In some embodiments, device600is in communication with device614after receiving an input directed at affordance644and/or after device600is no longer controlling visual indicator622of device614. In some embodiments device600is paired with device614.

AtFIGS.6M-6P, device600displays user interface632in response to the input selecting the option to control device614. As illustrated, screen reader is enabled on device600and adaptive navigation is disabled on device614. Screen reader is an additional (or, optionally, alternative) input setting that allows a user (e.g., a user with a visual impairment) to navigate a user interface on a device. In some embodiments, screen reader includes display of visual indicator622. In some embodiments, visual indicator622for screen reader has a similar or different appearance as visual indicator622for adaptive navigation. When operating with screen reader enabled, information about a graphical user interface object (e.g., the name of the object and/or the function the object will perform) in which the visual indicator622is on will be output via a simulated voice by a device (e.g., device600, device614, a wirelessly connected headphone, and/or another device will provide audio output).

Turning briefly toFIG.6R, in some embodiments, screen reader is enabled or disabled via user input at device614(e.g., using a toggle and/or button) under a system setting (e.g., an accessibility setting) of device614.

AtFIG.6M, in some embodiments, screen reader is enabled (e.g., automatically) on device614when screen reader at device600is enabled and device600begins controlling device614. For instance, in some embodiments, when screen reader is enabled on device600and in response to device600detecting the input to select affordance623ofFIG.6I, device600initiates a command to enable screen reader on device614. In response to receiving the command to enable screen reader on device614, device614enables screen reader.

AtFIG.6M, device614provides audio output646aof mail application642a(“e-mail application”). In some embodiments, the audio output includes audio output corresponding to the function being performed (e.g., “open e-mail application”). In some embodiments, device600provides audio output646aof mail application642a(“e-mail application”) (e.g., in addition to device614providing the audio output and/or instead of device614providing the audio output).

AtFIG.6M, in some embodiments, device600and device614display notification636. As described above, in some embodiments, device614responds to input at device614(e.g., a user can still control device614despite being controlled by device600). For instance, controlling device614by input detected at device600does not disable device614from detecting input received at device614. In some embodiments, device614does not respond to input at device614while being controlled by device600(e.g., a user cannot control device614using input at device614based on being controlled by device600). In some embodiments, device614does not display interface632(e.g., and/or one or more of its user interface objects) while device600displays interface632.

AtFIG.6M, in response to screen reader being enabled at device600and adaptive navigation being disabled at device614, device600displays interface632. As illustrated, interface632includes touchpad area633for controlling device614. In some embodiments, touchpad area633is occupies an area of the user interface and corresponds to a region where the device600detects one or more touch inputs (e.g., a tap, a swipe, a two-finger swipe, and/or a two-finger rotation). The one or more touch inputs allow a user of device600to navigate user interfaces of device614. In some embodiments, the one or more touch inputs control visual indicator622(e.g., move forward, move backward, and/or select) (e.g., similar to commands associated with affordances612a-cofFIG.6D). In some embodiments, one or more touch inputs (e.g., a set of inputs and/or all of the inputs) that are detectable at device614when screen reader is enable at device614are detectable via touchpad area633(and performs the same operation had the input been detected by device614). In some embodiments, touchpad area633controls device614beyond (e.g., additionally and/or alternatively) controlling visual indicator622(e.g., execute a zoom command using a pinch and/or execute a scroll based on two finger swipe). In some embodiments, interface632includes one or more user interface objects for controlling device614. As illustrated, interface632includes one or more system affordances640for controlling system actions (similar to system actions611ofFIG.6D) of device614.

AtFIG.6M, while displaying interface632, device600detects an input (e.g., pinch609aand/or swipe650m) to move visual indicator622. In response to detecting the input to move visual indicator622, device600instructs device614to move visual indicator622from mail application642ato calendar application642b, as depicted inFIG.6N.

AtFIG.6N, device614displays visual indicator622on calendar application642b. In some embodiments, device614(and/or device600) provides audio output646bcorresponding to calendar application642b(e.g., a simulated voice used to output audio corresponding to “calendar application” and/or “open calendar application”).

AtFIG.6N, in some embodiments, visual indicator622moves through the user interface more rapidly in response to a rotational input of rotatable input device601and/or a gesture corresponding to (e.g., simulating) rotational input control (e.g., hand gesture corresponding to (simulating) rotation of rotatable input device601in the first direction and/or rotation of rotatable input device601in the second direction)). In some embodiments, while displaying interface630, device600detects an input (e.g., a hand gesture configured to execute (simulating) a rotational input and/or rotational input650nof rotatable input device601). In response to detecting the input, device600instructs device614to move visual indicator622from mail application642ato movie application642c, as depicted inFIG.6O.

AtFIG.6O, visual indicator622is displayed on movie application642c. As illustrated, device614provides audio output646corresponding to movie application642c(e.g., “movie application” and/or “open movie application”). while displaying interface632, device600detects an input (e.g., clench609cand/or tap650o) corresponding to a selection. In response to device600detecting the input corresponding to the selection, device600instructs device614to perform a selection. Because visual indicator622is on movie application642c, device614displays user interface618dof movie application642c, as depicted inFIG.6P.

AtFIG.6P, device614displays user interface618dof movie application642c. Device614provides audio output646dcorresponding to movies category648(e.g., “movies” and/or “open movies category”). In some embodiments, in response to detecting further inputs (e.g., hand gestures and/or touch inputs) at device600, device600can control visual indicator622on device614. In some embodiments, interface630includes done affordance644that terminates the session of device600controlling device614using interface632. In some embodiments, device600is in communication with device614after receiving an input directed at done affordance644and/or after device600is no longer controlling device614.

AtFIG.6Q, embodiments described with respect to device600and device614apply to device652(e.g., a smartphone) and device654(e.g., a tablet computer). In some embodiments, device652can be utilized to control device654using the similar techniques (e.g., user interfaces, inputs, and/or gestures (e.g., detected using one or more cameras656, such as a camera that has a user in a field-of-view) described with respect to device600and device614. In some embodiments, device652displays interface630or interface632based on the accessibility settings that are enabled at one or more devices. As illustrated, screen reader is enabled at device652while adaptive navigation is disabled at device654. As such, device652displays interface632to control device654and provides audio output646a(e.g., “e-mail application”). Additionally or alternatively, if screen reader is disable at device652and adaptive navigation is enabled at device654, then device652displays interface630to control device654.

AtFIG.6Q, in some embodiments, if adaptive navigation is enabled at device652(and/or device600) while screen reader is enabled at device654(and/or device614), device652(and/or device600) displays interface630(and forgoes displaying interface632) (e.g., interface630will be prioritized over interface632). In some embodiments, if adaptive navigation is enabled at device652(and/or device600) while screen reader is enabled at device654(and/or device614), device652(and/or device600) displays interface632(and forgoes displaying interface630) (e.g., interface632will be prioritized over interface630). In some embodiments, if adaptive navigation is enabled at device652while screen reader is enabled at device652, device652(and/or device600) displays interface630(and forgoes displaying interface632) (e.g., interface630will be prioritized over interface632).

AtFIG.6R, device614displays screen reader and adaptive navigation as being disabled. Screen reader or adaptive navigation can be enabled by user input at device614(e.g., using a toggle and/or button) under a system setting (e.g., an accessibility setting). AtFIG.6R, in some embodiments, when screen reader is disabled at device600and adaptive navigation is disabled at device614, device600can still control device614. In such embodiments, device600displays a user interface including one or more affordances for controlling system actions (one or more system affordances640). Additionally or alternatively, hand gestures detected by device can initiate commands to launch a shortcut at device614, such as shortcut612iand/or shortcut614h. In some implementations, hand gestures (e.g., a double-pinch gesture) can initiate commands such as: answer or end a phone call, dismiss a notification, take a photo, play or pause media, and start, pause, or resume a workout.

FIG.7is a flow diagram illustrating a method for controlling a second computer using a first computer system in accordance with some embodiments. Method700is performed at a first computer system (e.g.,100,300,500,600, and/or652) (e.g., a smartwatch, a smartphone, a tablet, a laptop computer, and/or a head mounted device (e.g., a head mounted augmented reality and/or extended reality device)) that is in communication with a display generation component (e.g., a display of device600) (e.g., a display controller, a touch-sensitive display system, a monitor, and/or a head mounted display system) and one or more input devices (e.g., a touch-sensitive surface, a keyboard, a controller, a rotatable input device, microphone, and/or a mouse). In some embodiments, the computer system is in communication with one or more sensors (e.g., one or more cameras, one or more biometric sensors (e.g., a heart rate sensor), a gyroscope, an accelerometer)). Some operations in method700are, optionally, combined, the orders of some operations are, optionally, changed, and some operations are, optionally, omitted.

As described below, method700provides an intuitive way for displaying a user interface with device controls. The method reduces the cognitive burden on a user for controlling devices, thereby creating a more efficient human-machine interface. For battery-operated computing devices, enabling a user to control devices faster and more efficiently conserves power and increases the time between battery charges.

The first computer system (e.g.,600) detects (702), via the one or more input devices, an input (e.g.,650iand/or609c) corresponding to a request to control a second computer system (e.g.,614,654) (e.g., a smartwatch, a smartphone, a tablet, a laptop computer, and/or a head mounted device (e.g., a head mounted augmented reality and/or extended reality device)). In response (704) to detecting the input corresponding to the request to control (e.g., execute commands at and/or navigate user interfaces of) the second computer system and while in communication with the second computer system (e.g., using a wired or wireless connection with the second computer system), and in accordance with a determination that a first set of control criteria is met, the first computer system displays (706) (e.g., via the display generation component) one or more first user interface objects (e.g., one or more of the user interface objects of interface630, including638a-c, and/or640) (e.g., a user interface object that is activated a first type of input and/or a user interface object that has a first size) that controls a graphical element (e.g.,622) displayed by the second computer system (e.g., cursor, pointer, and/or highlighter) (e.g., that indicates a respective function (e.g., opening an application, executing a function within an application, and/or closing the application) will be executed on the second computer system, such as when a confirmation input is received) to navigate one or more user interfaces (e.g.,618aand/or618b) of the second computer system, wherein a first control criteria of the first set of control criteria is met when a first mode of controlling a computer system (e.g., by navigating a graphical user interface and/or executing a function on the computer system) is enabled (e.g., an input setting is active, a setting that provides one or more additional (and/or alternative) options to control how input is detected, and/or a mode in which a particular input would not cause a function to be performed unless a particular input setting is active) on the second computer system (e.g., adaptive navigation is enabled on device614as described inFIGS.6J-6L).

In response (704) to detecting the input corresponding to the request to control (e.g., execute commands at and/or navigate user interfaces of) the second computer system and while in communication with the second computer system (e.g., using a wired or wireless connection with the second computer system), and in accordance with a determination that the first set of control criteria is not met, the first computer system forgoes (708) display of the one or more user interface objects that controls a graphical element to navigate the one or more user interfaces of the second computer system (e.g.,638a-cand/or640). In some embodiments, the first computer system wirelessly connects to the second computer system in response to the input corresponding to the request to control the second computer system. In some embodiments, the first mode of controlling a computer system is enabled via a system setting (e.g., the first mode is enabled or disabled through a system settings menu). In some embodiments, the first computer system uses one or more sensors (e.g., a biometric sensor (e.g., a heartrate sensor and/or an optical heartrate sensor), an accelerometer, and/or gyroscope) to detect and differentiate among various gestures performed by a hand (and/or other body part) of the user while the first computer system is worn (e.g., on the user's wrist and/or other body part) by the user. In some embodiments, the various gestures can be used to initiate (e.g., transmit and/or send) one or more commands to be executed by the second computer system. In some embodiments, the various gestures do not include input at a button or touch-sensitive surface of the computer system. In some embodiments, the one or more user interface objects that controls the graphical element displayed at the second computer system are not displayed at the second computer system (and/or are displayed only at the first computer system). In some embodiments, the first computer system (and/or the second computer system) displays an indication (e.g., a notification and/or a banner) (e.g., a non-persistent indication and/or a persistent indication) that the first computer system (and/or the second computer system) is controlling the second computer system (and/or the first computer system). In some embodiments, the first mode of the second computer system is enabled independently of communication with the first computer system. In some embodiments, the first mode of the second computer system is not enabled in response to the second computer system communicating with the first computer system. In some embodiments, the first mode of the second computer system is enabled prior the second computer system communicating to communicating with the first computer system. In some embodiments, the first mode of the second computer system is enabled after the second computer system communicates with the first computer system. In some embodiments, the first mode of the second computer system is manually enabled (e.g., by the user). In some embodiments, the first mode of the second computer system is enabled locally at the second computer system (and/or cannot be enabled via the first computer system). Automatically displaying particular controls when an accessibility setting is enabled on the second computer system allows a user to better control the second device when the user has a physical impairment and provides a user visual feedback as to the state of the second computer system, which enhances the user-machine interface for users having physical impairments and provides improved visual feedback as to the state of the second computer system.

In some embodiments, in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with the determination that the first set of control criteria is not met, the first computer system displays, via the display generation component, a control user interface object (e.g.,640) (e.g., a selectable user interface object and/or toggle) (e.g., one or more) that controls a function (e.g., navigate to a home screen, view one or more open applications (e.g., application switcher and/or an application navigation view), and/or open a notification center) of the second computer system (e.g., a system action as depicted inFIG.6Dand as described in reference toFIG.6J). In some embodiments, the user interface object that controls the function of the second computer system is displayed when the first set of control criteria is met. Automatically displaying controls when an accessibility setting is not enabled on the second computer system allows a user to better control the second computer system so as to efficiently perform different operations at the second computer system, which enhances the user-machine interface and provides improved visual feedback as to the state of the second computer system.

In some embodiments, in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with a determination that a second set of control criteria is met, the first computer system displays, via the display generation component, one or more second user interface objects (e.g., one or more of the interface objects of interface632, including633and/or640) (e.g., a user interface object that is activated using a second type of input and/or a user interface objects that has a second size) that controls a graphical element (e.g.,622) displayed by the second computer system, wherein the one or more second user interface objects is different from the one or more first user interface objects (e.g., the second user interface object is activated using at least one type of input that does not activate the first user interface object (or vice versa) and/or the second user interface object has a different (greater or smaller) size than the first user interface object), and wherein the second set of control criteria includes a first control criterion that is met when a second mode, different from the first mode, of controlling a computer system (e.g., by navigating a graphical user interface and/or executing a function on the computer system) is enabled (e.g., an input setting is active, a setting that provides one or more additional (and/or alternative) options to control how input is detected, and/or a mode in which a particular input would not cause a function to be performed unless a particular input setting is active) on the first computer system (e.g., screen reader on device600or device652is enabled as described inFIGS.6M-6Q). In some embodiments, a screen reader is enabled when a computer system is in the second mode. In some embodiments, the screen reader is not enabled when a computer system is in the first mode. In some embodiments, the computer system displays a second option (e.g., an affordance and/or toggle) (e.g., the same as the first option and/or different from the first option) that controls a function (e.g., navigate to a home screen, view one or more open applications (e.g., “application switcher” and/or an application navigation view), and/or open a notification center) of the second computer system. In some embodiments, the second option is displayed when the second set of control criteria is met. Automatically displaying particular controls when a particular accessibility setting (e.g., a screen reader or text reader) is enabled on the first computer system allows a user to better control the second computer system when the user has a physical impairment and provides a user visual feedback as to the state of the second computer system, which enhances the user-machine interface for users having physical impairments and provides improved visual feedback as to the state of the first computer system.

In some embodiments, the first set of control criteria includes a second control criterion that is met when the second set of control criteria is not met. In other words, first set of control criteria is not met and, therefore, the one or more first user interface objects are not displayed when the second set of control criteria are met. Not displaying particular controls (e.g., non-screen reader and/or non-text reader controls) at the first computer system when a particular setting (e.g., a screen reader or text reader) is enabled on the first computer system decreases the number of controls displayed at the first computer system, which enhances the user-machine interface for users having physical impairments and declutters the user interface at the first computer system.

In some embodiments, the second set of control criteria does not include a control criterion that is based on whether the second mode of controlling a computer system is enabled (or, optionally, disabled) on the second computer system. In some embodiments, the second set of control criteria is met or not met independent of whether the second mode of controlling a computer system is enabled on the second computer system. Displaying particular controls when an accessibility setting (e.g., a screen reader or text reader) is enabled on the first computer system allows a user to better control the second computer system as it prioritizes a local accessibility setting and how the user is using the first computer system based on his or her a physical impairment, which enhances the user-machine interface for users having physical impairments and provides improved visual feedback as to the state of the first computer system.

In some embodiments, in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system: in accordance with the determination that the second set of control criteria is met (and/or, optionally, in accordance with a determination that the second mode is disabled on the second computer system), the first computer system initiates a command (e.g., initiating a transmission and/or instructing) to enable the second mode on the second computer system (e.g., device600initiates a command to enable screen reader on device614inFIG.6M). Enabling a particular accessibility setting (e.g., a screen reader or text reader) at a remote device when the particular accessibility setting is enabled on a first computer system allows a user to control the second computer system according to how the user is controlling the first computer system, which enhances the user-machine interface for users having physical impairments.

In some embodiments, while the second mode is enabled on the second computer system, the one or more second user interface objects are not displayed at the second computer system (e.g., the user interface objects of interface632are not displayed on device614and/or device654). Not displaying, at a remote device, control options that are displayed at a first computer system allows a user to control the second computer system and limits the amount of control options displayed at the second computer system, which provides additional control options without cluttering the user interface.

In some embodiments, the second set of control criteria does not include a criterion that is based on whether the first mode is enabled on the second computer system (e.g., interface632is displayed at device600and/or device652regardless of whether adaptive navigation is enabled at device614and/or device654) (e.g., the one or more second user interface objects are displayed independent of the first set of criteria being met). Displaying controls independent of a particular accessibility setting that is enabled on the second computer system allows a user to better control the second computer system as the first computer system displays the one or more second user interface objects regardless of whether the first mode is enabled on the second computer system which enhances the user-machine interface for users having physical impairments and provides improved visual feedback as to the state of the first computer system.

In some embodiments, wherein the first set of control criteria includes a third criterion that the second mode is enabled on the first computer system and the first mode is enabled on the second computer system (e.g.,638a-cand/or640is displayed when screen reader is enabled on device600and adaptive navigation is enabled on device614). In some embodiments, the second set of control criteria includes a criterion that the second mode is enable at first computer system and the first mode is enabled on the second computer system. Displaying controls for a particular accessibility setting when two different modes are enabled on the respective devise allows a user to better control the second computer system as it prioritizes the display of particular interface objects at the first computer system based on how the user is using the second computer system, which enhances the user-machine interface for users having physical impairments and provides improved visual feedback as to the state of the second computer system.

In some embodiments, displaying the one or more second user interface objects includes displaying a touch input area (e.g.,633) (e.g., a touchpad area and/or trackpad area). In some embodiments, while displaying the touch input area, the first computer system detects, via the one or more input devices, a gesture (e.g.,650m) at the touch input area. In response to detecting the gesture at the touch input area, the first computer system transmits, to the second computer system, an indication of the gesture (e.g., swipe650causes device614to move visual indicator622as depicted inFIGS.6M-6N) (e.g., transmitting a location, direction, duration, and/or path of the gesture to the second computer system). In some embodiments, the touch input area is used for controlling the second computer system using motion-based gestures (e.g., a swipe and/or a rotation). In some embodiments the touch input area (e.g., for detecting motion-based gestures) allows for detecting multi-finger inputs (e.g., two finger tap and/or three finger swipe). Providing a touch input area at a first computer system, which receives input that is typically received at the remote device, allows a user to better control the second computer system through motion-based gestures received at the first computer system, which enhances the user-machine interface for users having physical impairments.

In some embodiments, in response to detecting the input corresponding to the request to control the second computer system and while in communication with the second computer system, in accordance with a determination that the first set of control criteria is met, the first computer system forgoes display of the touch input area (e.g., interface630does not include touchpad area633). In some embodiments, the one or more first user interface objects includes an affordance for controlling the graphical element displayed by the second computer system that is not included in the one or more second user interface objects. Not providing the touchpad area at the first computer system when a particular accessibility setting (e.g., an accessibility setting other than a screen reader setting) is enabled on the second computer system limits the number of unnecessary graphical objects based on the user's physical impairments, which enhances the user-machine interface for users having physical impairments by including relevant controls.

In some embodiments, the first computer system includes a rotatable input device (e.g.,601) (e.g., a rotatable input mechanism and/or crown). In some embodiments, the first computer system detects, via the rotatable input device, a rotational input (e.g.,650n). In response to detecting the rotational input, the first computer system transmits, to the second computer system, a scroll instruction to scroll content displayed at the second computer system (e.g., content displayed on device614is scrolled and/or as indicator622moves through content of a user interface displayed on device614, device614scrolls the content displayed). In some embodiments, the scroll instructions include a direction of scroll (first direction vs second direction), a rate of scroll (a first rate vs. a second rate), and/or a duration of scroll (first duration vs. second duration). In some embodiments, in response to detecting the rotational input, the computer system transmits instructions to the second computer system to move the graphical element displayed by the second computer system. Scrolling through items on a second computer system in response to a crown or wheel rotation at a first computer system allows a user to better control the second computer system as it provides an efficient way to quickly move the graphical indicator through a remote user interface, which enhances the user-machine interface for users having physical impairments and provides additional control options without cluttering the user interface.

In some embodiments, while displaying the one or more first user interface objects, the first computer system detects, via one or more sensors (e.g., a biometric sensor (e.g., a heartrate sensor and/or an optical heartrate sensor), an accelerometer, and/or gyroscope), a non-touch hand gesture (e.g.,609aand/or609cas described inFIGS.6J-6L) (e.g., a gesture made in the air, such as an air-pinch or hand clench). In response to detecting the non-touch hand gesture, the first computer system initiates a command (e.g., initiating a transmission and/or instructing the second computer system) to perform an operation (e.g., move visual indicator622as described inFIGS.6J-6Kand/or select calendar application as described inFIGS.6K-6L) (move the graphical indicator, make a selection, and/or open a notification center) at the second computer system. In some embodiments, the hand gesture does not include input at a button or touch-sensitive surface of the computer system. Controlling a second computer system in response to detecting a gesture (e.g., an air-pinch or clench) made by the user at a first computer system provides the user with additional control options without requiring the user to press a button and/or touch-sensitive surface, which enhances the user-machine interface for users having physical impairments and provides additional control options without cluttering the user interface.

In some embodiments, initiating the command to perform the operation includes: in accordance with a determination that that the non-touch hand gesture is a first type of non-touch hand gesture (e.g.,609a), the first computer system transmits, to the second computer system, a first command (e.g., move visual indicator622as described inFIGS.6J-6K); and in accordance with a determination that that the non-touch hand gesture is a second type of non-touch hand gesture (e.g.,609c) different from the first type of non-touch hand gesture, the first computer system transmits, to the second computer system, a second command (e.g., select calendar application as described inFIGS.6K-6L). Performing different operations at a second computer system in response to detecting, at a first computer system, a gesture (e.g., an air-pinch or clench) made by the user provides the user with additional control options without requiring the user to press a button and/or touch-sensitive surface to perform the same operation at the second computer system, which enhances the user-machine interface for users having physical impairments and provides additional control options without cluttering the user interface.

In some embodiments, the first computer system displays, via the display generation component, a plurality of options to control a plurality of computer systems (e.g.,626), including a first option to control the second computer system (e.g., the option to phone inFIG.6I) and a second option to control a third computer system (e.g., the option to control tablet and/or the option to control computer inFIG.6I) that is different from the first computer system and the second computer system. In some embodiments, detecting the input corresponding to a request to control the second computer system includes detecting, via the one or more input devices, selection of the first option to control the second computer system (e.g.,650iand/or609c). Displaying multiple computer system to control provides the user an ability to select which computer system the user would like to control using a respective accessibility setting, which enhances the user-machine interface for users having physical impairments and provides additional control options.

In some embodiments, displaying the plurality of options to control the plurality of computer systems includes: in accordance with a determination that a respective computer system satisfies a set of one or more display conditions, the first computer system displays, via the display generation component, an option to control the respective computer system. In some embodiments, the set of one or more display conditions includes a first display condition that is satisfied when the respective computer system is logged into the same user account as the first computer system (e.g., device600is logged into the same user account as device614, device600is logged into the same user account as a tablet computer associated with the tablet option displayed in device selector menu626, and/or device600is logged into the same user account as a computer associated with the computer option displayed in device selector menu626). In some embodiments, displaying the plurality of options to control the plurality of computer systems includes: in accordance with a determination that the respective computer system does not satisfy the set of one or more display conditions, the first computer system forgoes display of the option to control the respective computer system (e.g., devices that are not signed into the same user account are not displayed in device selector menu626). Displaying multiple computer systems based on whether the device is logged into the same user account provides the user an ability to select which computer system the user would like to control, which enhances the user-machine interface for users having physical impairments and provides additional control options.

In some embodiments, the set of one or more display conditions includes a second display condition that is satisfied when the respective computer system is within a threshold distance of the first computer system (e.g., device600is located within 200 feet of device614). In some embodiments, the first computer system receives an indication (e.g., from a server, from a global positioning satellite, and/or from the respective computer system (e.g., using a local area network and/or Bluetooth) that the respective computer system is within the threshold distance of the first computer system. Displaying multiple devices based on whether the device is within a threshold distance of the computer system provides the user an ability to select nearby devices that the user would like to control and limiting the number of inputs to identify nearby devices, which enhances the user-machine interface for users having physical impairments and reduces the number of inputs needed to perform an operation.

In some embodiments, the first computer system receives, via the one or more input devices, user input to correspond a user-specified non-touch hand gesture (e.g.,609b,609d, and/or609c) (e.g., an air gesture and/or gesture detected independent of a touch-sensitive surface of the first computer system) to a first function (e.g., a rotate) of the rotatable input device of the first computer system (e.g., device600receives an input to correspond to one of the gestures associated with affordances608a-dwith a rotation of rotatable input device601that is associated with affordance612fand/or affordance612g), where the user-specified non-touch hand gesture does not use the rotatable input device of the first computer system. In some embodiments, while the user-specified non-touch hand gesture corresponds to the first function of the rotatable input device of the first computer system: the first computer system receives, via the one or more input devices, the user-specified non-touch hand gesture. In response to receiving the user-specified non-touch hand gesture, the first computer performs a first operation (e.g., a scroll operation) that corresponds to the first function of the rotatable input device of the first computer system (e.g., the first computer system would perform the same operation had the computer system detected input using the first function (e.g., rotation) of the rotatable input device). In some embodiments, subsequent to performing the first operation that corresponds to the first function of the rotatable input device, the first computer system receives, via the one or more input devices, user input to correspond the user-specified non-touch hand gesture (e.g., an air gesture and/or gesture detected independent of a touch-sensitive surface of the first computer system) to a second function (e.g., a rotate or press input) (e.g., device600receives an input to correspond one of the gestures associated with affordances608a-dwith a press (e.g., depress) of rotatable input device601that is associated with affordance612g), different from the first function, of the rotatable input device of the first computer system, where the user-specified non-touch hand gesture does not use the rotatable input device of the first computer system. In some embodiments, while the user-specified non-touch hand gesture corresponds to the second function of the rotatable input device of the first computer system: the first computer system receives, via the one or more input devices, the user-specified non-touch hand gesture. In response to receiving the user-specified non-touch hand gesture, the computer system performs a second operation that corresponds to the second function (e.g., press input of) of the rotatable input device of the first computer system (e.g., the computer system would perform the same operation had the computer system detected input using the second function (e.g., press input) of the rotatable input device). In some embodiments, the computer system receives user input to confirm non-touch gestures that don't use the rotatable input device to mimic (or substitute for) functions of the rotatable input device. Customizing a hand gesture (e.g., an air-pinch or clench) made by the user to perform an operation typically detected by a crown or wheel provides a user with alternative ways to provide an input so as to control a device (local or remote), which enhances the user-machine interface for users having physical impairments and provides additional control options.