Patent Description:
Conventional screen content capture systems and methods may provide for the capture of image content displayed on a screen. For example, a user of a computing device may launch screen content capture functionality, or screenshot functionality, of the computing device through one or more shortcuts, series of inputs and the like. The user may then access the captured screenshot in a storage location within a file management system of the computing device. The user may access these types of conventional screen content capture and management systems without the benefit of a user interface (UI) that facilitates the capture of screen content, and the access to and management of the screen content after capture.

TechSmith "Snagit Help" (XP055877169, <NUM> June <NUM>, retrieved from URL:https://www. com/blog/wp-content/uploads/<NUM>/<NUM>/Snagit2020. <NUM>-Help. pdf) discloses to capture a screenshot or video of anything on your screen with Snagit.

<CIT> discloses a method for mediation of a screenshot capture by a client application based on policy. The method includes identifying, by a client application on a client device, a policy for mediating one or more screenshots of content displayed via the client application. An embedded browser within the client application accesses a network application of one or more servers. The method further includes intercepting, by the client application, a request to capture a screenshot of at least a portion of the network application being displayed, determining, by the client application, one or more mediation actions to perform on the screenshot responsive to the policy, performing, by the client application, the one or more mediation actions on the screenshot, and providing, by the client responsive to the request, the screenshot resulting from the one or more mediation actions.

<CIT> discloses a video screen capturing method, a video screen capturing device and a terminal. The method comprises: acquiring a screen capturing request to a video, generating a screen capturing instruction according to the screen capturing request, storing the screen capturing instruction, and thereby executing the screen capturing instruction to generate corresponding screenshots.

The matter for protection is set out in appended method claim <NUM> and appended system claim <NUM>, with optional features set out in the dependent claims appended thereto. Systems and methods, in accordance with implementations described herein, may function to facilitate access to screen capture functionality of a computing device. For example, systems and methods described herein may function to provide for more easily accessible screen capture functionality than conventional systems, making screen capture functionality accessible through a system menus of a computing device, a dedicated physical key of the computing device, through a launcher search of the computing device, from a shelf pod or tray, and the like. Systems and methods, in accordance with implementations described herein, cause a screen capture user interface (UI) to be generated for display to the user of the computing device when the screen capture mode is invoked. The screen capture UI provides for user selection of a screen capture mode including selection of a screenshot mode in which still image content is captured, or a screen recording mode in which moving image content, or video content, is recorded. The screen capture UI may provide for user selection of a user specified (or custom) screen capture area, a window or tab for capture, a page/full page for capture, or a full screen capture, with either the screenshot mode or the screen recording mode. In some examples, the screen capture UI may also provide for adjustment of the user specified screen capture area, adjustment of content captured within the user specified screen capture area, with either the screenshot mode or the screen recording mode. In some examples, the screen capture UI may also provide for selection of a scrolling screenshot mode, allowing for capture of long-form content by scrolling, for example manually scrolling or automatically scrolling, while in the screenshot mode. The screen capture UI as described herein may provide for simplified user access to expanded functionality within the screen capture functionality compared to conventional systems.

A system and method, in accordance with implementations described herein, may function to facilitate the use and management of captured screen content, post capture. For example, systems and methods as described herein may cause a quick action UI to be generated at the completion of screen content capture. The quick action UI may provide for the copying of captured content (a screenshot or screen recording) to a designated location, previewing of the captured content, editing of the captured content, sharing of the captured content, deleting of the captured content, and other such functionality, at the completion of screen capture. The quick action UI may expand on the functionality of conventional systems, and may improve user interaction with and management of screenshots and/or screen recordings, which are often captured by the user for immediate use and/or single use. The time and number of user inputs needed to manage or use the screenshots and/or screen recordings may therefore be reduced, which can facilitate a reduction in computational resources.

A system and method, in accordance with implementations described herein, may function to group multiple instances of screen content captured in the manner described above. The grouping of stacking of captured screen content may provide for batch sharing, editing, deleting and the like.

A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination thereof installed on the system that in operation causes the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

In a first general aspect, a computer-implemented method includes generating a screen capture user interface; detecting selection of a screen capture mode; and operating in the selected capture mode in response to detecting the selection of the screen capture mode. Operating in the selecting capture mode includes triggering operation in a first capture mode in response to detection of selection of the first capture mode, the first capture mode being a screenshot mode for capture of still image content output by a display of a computing device; and triggering operation in a second capture mode in response to detection of selection of the second capture mode, the second capture mode being a screen recording mode for capture of moving image content output by the display. In some examples, operating in the selected capture mode in response to detecting the selection of the screen capture mode comprises triggering operation of a content manager of the operating system in the selected screen capture mode and initiating a screen capture operation in response to detecting the selection of the screen capture mode. Optionally, triggering operation in a first capture mode comprises triggering operation of the content manager in a first capture mode. Optionally, triggering operation in a second capture mode comprises triggering operation of the content manager in a second capture mode.

In some implementations, triggering operation in the second capture mode includes triggering screen recording of video content including moving image content. In some implementations, triggering operation in the second capture mode includes triggering of screen recording of video content including only moving image content. In some implementations, triggering operation in the second capture mode includes triggering of screen recording of video content including moving image content and audio content associated with the video content. In some implementations, triggering screen recording of video content may include recording at least a portion of the moving image content output by the display and audio content associated with the moving image content output by the display. in some implementations, triggering screen recording of video content may include triggering operation of a microphone of the computing device and recording at least a portion of the moving image content output by the display and audio content detected by the microphone. In some implementations, triggering screen recording of video content can include the capture of audio content embedded in or inherent in the video content.

Generating the screen capture user interface includes generating a first section of the screen capture user interface providing for selection of the first capture mode or the second capture mode; and generating a second section of the screen capture user interface providing for selection of a screen capture area. Generating the second section of the screen capture user interface may include providing for selection of a full screen capture function, a custom capture function, or a window capture function. Detecting selection of the screen capture mode may include detecting selection of the custom capture function, the method further comprising defining a capture area associated with the display of content, including detecting a point input identifying an initial capture point of the capture area; detecting a drag input from the point input; detecting a release of the point input identifying a final capture point of the capture area; and defining the capture area based on a trajectory and a distance between the initial capture point and the final capture point. In some implementations, defining the capture area also includes detecting a first adjustment input at at least one of a corner of a boundary defining the capture area or a side of the boundary of the capture area; and adjusting at least one of a size or a shape of the capture area in response to the detected first adjustment input. Defining the capture area may also include detecting a second adjustment input within the boundary of the capture area; and adjusting content displayed within the boundary of the capture area in response to the second adjustment input.

Operating in the selected capture mode in response to detecting the selection of the screen capture mode includes operating in the second capture mode and capturing a screen recording of video content output in a screen recording capture area of the display in response to detection of the selection of the second capture mode; detecting a selection of the first capture mode while operating in the second capture mode; and triggering operation in the first capture mode and capturing a screenshot while capturing the screen recording. Detecting selection of the first capture mode while operating in the second capture mode may include detecting selection of a custom capture function defining a screenshot capture area within the screen recording, including detecting a point input identifying an initial capture point of the screenshot capture area; detecting a drag input from the point input; detecting a release of the point input identifying a final capture point of the screenshot capture area; and defining the screenshot capture area based on a trajectory and a distance between the initial capture point and the final capture point.

In some implementations, triggering operation in the first mode further includes detecting selection of screen capture area; capturing a screenshot of screen content included in the detected screen capture area in response to detecting the selection of the screen capture area; and storing the captured screenshot in a clipboard storage area of the computing device. In some implementations, triggering operation in the second capture mode comprises accessing a GPU of the computing device and recording one or more frames of the moving image content from the GPU.

In another general aspect, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed by at least one processor, are configured to cause a computing system to at least generate a screen capture user interface; detect selection of a screen capture mode; and operate in the selected capture mode in response to detecting the selection of the screen capture mode, including trigger operation in a first capture mode in response to detection of selection of the first capture mode, the first capture mode being a screenshot mode for capture of still image content output by a display of a computing device; and trigger operation in a second capture mode in response to detection of selection of the second capture mode, the second capture mode being a screen recording mode for capture of moving image content output by the display. In some examples, the instructions, when executed by the at least one processor, cause an operating system of the computing system to perform the above operations. Optionally, causing a computing system to operate in the selected capture mode in response to detecting the selection of the screen capture mode comprises causing the operating system of the computing system to operate a content manager of the operating system in the selected screen capture mode in response to detecting the selection of the screen capture mode.

In some implementations, in the second capture mode, the instructions may cause the computing system to trigger screen recording of video content including moving image content and audio content associated with the video content. In the second capture mode, the instructions may cause the computing system to trigger screen recording of video content including recording at least a portion of the moving image content output by the display and audio content associated with the moving image content output by the display. The instructions cause the computing device to generate the screen capture user interface including a first section of the screen capture user interface including user interface elements providing for selection of the first capture mode or the second capture mode; and a second section of the screen capture user interface including user interface elements providing for selection of a screen capture area, optionally including a full screen capture area, a custom screen capture area, or a window capture area.

In some implementations, the instructions cause the computing device to detect selection of the custom screen capture area, including detect a point input identifying an initial capture point of the screen capture area; detecting a drag input from the point input; detecting a release of the point input identifying a final capture point of the screen capture area; and defining the screen capture area based on a trajectory and a distance between the initial capture point and the final capture point. The instructions may cause the computing device to further define the screen capture area, including detect a first adjustment input at at least one of a corner of a boundary defining the screen capture area or a side of the boundary of the screen capture area; and adjust at least one of a size or a shape of the screen capture area in response to the detected first adjustment input. The instructions may cause the computing device to further define screen content captured within the screen capture area, including detect a second adjustment input within the boundary of the screen capture area; and adjust screen content displayed within the boundary of the screen capture area in response to the second adjustment input.

The instructions cause the computing device to operate in the selected capture mode, including operate in the second capture mode and capture a screen recording of video content output in a screen recording capture area of the display in response to detection of the selection of the second capture mode; detect a selection of the first capture mode while operating in the second capture mode; and trigger operation in the first capture mode and capture a screenshot while operating in the second capture mode and capturing the screen recording.

In some implementations, the instructions cause the computing device to detect selection of the first capture mode while operating in the second capture mode including detecting selection of a custom screen capture function defining a screenshot capture area within the screen recording area, including detect a point input identifying an initial capture point of the screenshot capture area; detect a drag input from the point input; detect a release of the point input identifying a final capture point of the screenshot capture area; and define the screenshot capture area based on a trajectory and a distance between the initial capture point and the final capture point.

In some implementations, which may be implemented together with any suitable combination of the above-described features, a computer program product or a non-transitory computer-readable storage medium may comprise instructions that, when executed by at least one processor, are configured to cause a computing system to perform the method described herein.

This document describes screen capture functionality of a computing device in which a screen content capture mode may be invoked, and in which a screen capture area may be designated. The screen capture functionality allows for selection of a screenshot capture mode in which still image content is captured, or a screen recording capture mode in which moving image content, or video content, is captured. The screen capture functionality may allow for selection of a screen capture area, including selection of a full screen capture area, selection of a window or a tab for capture, or selection of a customized capture area. The screen capture functionality may allow for adjustment of boundaries defining the capture area and/or content captured within the boundaries of the capture area. In this way, a custom portion of the content displayed by the computing device may be captured by a user; this can reduce the computational resources needed to take the screen capture (as compared to a full screen capture, for example), as well as minimizing or eliminating the need for post-capture editing operations on the screen capture, which can further reduce the use of computational resources. The screen capture functionality may also provide for the preview, editing, sharing, saving, deleting and other such follow on actions in response to completion of the capture of screen content. The screen capture functionality may also allow for batch sharing, batch editing, batch deleting and/or batch saving of multiple instances of captured screen content. In some implementations, one or more user interfaces (UIs) may be generated and triggered for display to provide for the selection of the screen capture mode. In some implementations, one or more UIs may be generated and triggered for display to provide for the selection of the screen capture area. In some implementations, one or more UIs may be generated and triggered for display to facilitate the sharing, editing, storage, deleting and the like of captured screen content.

For example, in operation, the systems and methods described herein can generate one or more system level UIs, including, for example, a screen capture UI to provide ready access to screen capture functionality, and a quick action UI to provide ready access to tools for the management of captured screen content. These system level UIs may reduce a number of steps required to access and invoke screen capture functionality, and may expand features and functions of screen capture capability compared to that of conventional systems. These system level UIs may reduce a number of steps required to take action with respect to captured screen content, and may facilitate management of the captured screen content to preserve system resources. The use of a system level screen capture can also improve security as compared to performing screen capture using third party applications or programs, since the capture and storage of the screen capture is isolated within the system.

The systems and methods herein may solve the technical challenge(s) (e.g., problem(s)) of discoverability of the screen capture functionality of the computing device, and the management of captured screen content. A system is needed that provides users with easily discoverable and easily accessible screen capture functionality, so that the screen capture function can be quickly and easily located and launched by the user. Conventional systems often employ complicated combinations of button presses, shortcuts and the like that are not readily apparent and/or easily accessible to the user to invoke a screen capture function, relying on previous user knowledge and experience to invoke the screen capture function. Similarly, conventional systems often rely on the user's familiarity with numerous different system menus, functions and the like for the editing, sharing, storage and management of captured screen content.

The systems and methods described herein provide a technical solution to these technical problems by configuring the computer O/S to generate the screen capture UI and the quick action UI described herein. The OS may render the screen capture UI in response to a user invoking the screen capture functionality of the computing device to allow for quick selection of a screen capture mode and for selection of a screen content capture area. The OS may render the quick action UI in response to the completion of capture of screen content to allow for quick editing, sharing, storage, deleting and the like of captured screen content without having to access multiple different system menus, system applications, system storage areas and the like.

The technical solutions described herein may provide a technical effect of providing a screen capture UI and a quick action UI that may be widely adapted to a variety of devices including small devices such as wearables and mobile devices as well as larger devices including tablets, laptops, desktops, and the like. In some implementations, this type of screen capture UI and/or quick action UI may be generated at the operation system level. For such devices, the UIs described herein may provide an advantage of presenting the functionality and features available to the user at the point of screen content capture, and for follow on action after completion of screen content capture. The UIs described herein may provide for faster access to screen capture options and features without user intervention (i.e., automatically, in response to invoking the screen capture function and in response to completion of screen content capture) compared to conventional systems which typically rely on user access to numerous different system menus, applications, storage areas and the like to accomplish screen content capture and management. The time and number of user inputs needed to manage or use the screenshots and/or screen recordings may therefore be reduced, which can facilitate a reduction in computational resources. Moreover, the implementation at the operating system level can improve security as compared to performing screen capture using third party applications or programs, since the capture and storage of the screen capture is isolated within the system.

<FIG> is a schematic illustration of an example implementation of a screen capture UI <NUM> and a quick action UI <NUM>, in accordance with implementations described herein. The example schematic illustration shown in <FIG> includes a window and a number of browser tabs displayed within a full screen of a display. In response to invoking a screen content capture function, a system in accordance with implementations described herein is configured to render and display a screen capture UI <NUM> as shown in <FIG> to facilitate user access and use of functionality and features associated with the capture of screen content. The functionality and features accessible via the screen capture UI <NUM> may be applicable to screen content including still image content, moving image content (i.e., video content), audio content, and the like. The screen capture UI <NUM> provides for user selection of a screenshot mode (in which selected still image content is captured) or a screen recording mode (in which moving image content with or without audio content is captured). The screen capture UI <NUM> in this example provides for selection of a full screen capture, a selected window or tab capture, or a custom capture as shown in <FIG>. In some implementations, a custom capture area may provide for the capture of screen content displayed within a single window, multiple windows and/or tabs, a particular application, and the like. Selection of, for example, operation in the screenshot mode for a still image capture of a selected window as shown in <FIG> captures the screen content included within the boundaries of the selected window. In response to completion of the screenshot capture, the system renders a quick action UI <NUM> for display. The quick action UI <NUM> may include, for example, a preview of the captured screen content, and various options for editing, sharing, saving, deleting and the like of the captured screenshot. In response to completion of user activity with the captured screenshot content, the system may cause the display to return screen content to the arrangement prior to the screen capture activity, as in <FIG>.

<FIG> is a block diagram illustrating an example computing system <NUM> configured to generate a screen capture UI and a quick action UI, in accordance with implementations described herein. The system <NUM> may be used to configure computing devices such as, for example, a client computing device <NUM> and a server computing system <NUM>, and/or other devices (not shown in <FIG>) to generate the screen capture UI, to capture content in accordance with user input received via the screen capture UI, to generate the quick action UI, and to execute action with respect to the captured screen content in response to user input received via the quick action UI. In some examples, the client computing device <NUM> can communicate through a communication module <NUM> via a network <NUM>. In some examples, external resources <NUM> and/or external services <NUM> are available to the client computing device <NUM> and/or the server computing system <NUM> via the network <NUM>. The external resources may include, for example, online storage, access to content to be consumed by the user via on the computing device <NUM>, data access and the like. In some implementations, the external resources <NUM> may function to duplicate or replace functionality provided by the server computing system <NUM>.

The example computing device <NUM> shown in <FIG> includes an operating system (O/S) <NUM>. The O/S <NUM> may execute and/or otherwise manage applications <NUM>. The example computing device <NUM> includes one or more input devices <NUM> and one or more output devices <NUM>. The input devices <NUM> can include, for example, a keyboard, a touch input device, a microphone, a gesture recognition device, an interface for connection to an external device such as a controller or a mouse, and other such input devices. The output devices can include, for example, a display, an audio output device (such as speakers), a physical output device, an interface for connection to external output devices, and other such output devices. In some examples, the computing device <NUM> includes a sensing system <NUM> including one or more sensors such as, for example, an image sensor (such as a camera), an audio sensor (such as a microphone), a positional sensor (such as an inertial measurement unit including an accelerometer, a gyroscope, a magnetometer and the like). The O/S <NUM> may have access to a memory <NUM>, and CPU/GPU <NUM>. The O/S <NUM> may also include or have access to policies and permissions <NUM> and preferences <NUM> associated with the computing device <NUM>.

The O/S <NUM> may function to execute and/or control applications, UI interactions, accessed services, and/or device communications that are not shown. The applications <NUM> may be any type of computer program that can be executed/delivered by the computing device <NUM> (or server computing system <NUM> or via an external service). Applications <NUM> may provide a user interface (e.g., application window) to allow a user to interact with the functionalities of a respective application <NUM>. The application window of a particular application <NUM> may display application data along with any type of controls such as menu(s), icons, widgets, etc. The applications <NUM> may include or have access to app information <NUM> and session data <NUM>, both of which may be used to generate content and/or data and provide such content and/or data to the users and/or the O/S <NUM> via a device interface including one or more output devices <NUM> of the computing device <NUM>. The app information <NUM> may correspond with information being executed or otherwise accessed by a particular application <NUM>. For example, the app information <NUM> may include text, images, audio content, control signals associated with input, output, or interaction with the application <NUM>. In some implementations, the app information <NUM> may include information from a repository <NUM> associated with one or more content items <NUM>. In some implementations, the app information <NUM> may include data associated with a particular application <NUM> including, but not limited to metadata, table tags, image tags, timestamp data, URL data, and the like. In some implementations, the applications <NUM> may include a browser <NUM>. The applications <NUM> can be configured to support capture commands that allow users of the computing device <NUM> to capture screen content, for example in the form of a screenshot or a screen recording as described above. The applications <NUM> can be configured to support the editing, sharing, storage and the like of captured screen content.

In some implementations, a UI generator <NUM> may be operated by the O/S <NUM>. The UI generator may generate the screen capture UI <NUM> and the quick action UI <NUM> as described above. In some implementations, a content manager <NUM> may be operated by the O/S <NUM> to manage screen capture content, for example, in response to inputs received at one of the input devices <NUM> related to the screen capture UI <NUM> and/or the quick action UI <NUM>. In some implementations, the content manager <NUM> may prepare captured screen content for sharing, editing, storing and the like post-capture.

The computing device <NUM> may generate and/or distribute particular policies and permissions <NUM> and preferences <NUM>. The policies and permissions <NUM> and preferences <NUM> may be configured by a device manufacturer of computing device <NUM> and/or by the user accessing device <NUM>. Policies <NUM> and preferences <NUM> may include routines (i.e., a set of actions) that trigger based on a particular action or command such as, for example, detection of an event based command, such as the detection of the invoking of the screen capture mode and/or the completion of a screen capture as described above, an audio command, a visual command, a schedule-based command, or other configurable command. Other policies <NUM> and preferences <NUM> may be configured to modify and or control content associated with the computing device <NUM> configured with the policies and permissions <NUM> and/or preferences <NUM>. In this way, operation in the selected capture mode may be triggered in accordance with policies and permissions <NUM>. In some examples, triggering the operation in the selected capture mode may be in accordance with accessed, predefined, permission data. Optionally, triggering the operation in the selected capture mode in accordance with the permission data comprises triggering the operation of the content manger in the selected capture mode in accordance with the permission data.

The server computing system <NUM> may include any number of computing devices that take the form of a number of different devices, for example a standard server, a group of such servers, or a rack server system. In some examples, the server computing system <NUM> may be a single system sharing components such as processors, memory, communication modules and the like.

The network <NUM> may include the Internet and/or other types of data networks, such as a local area network (LAN), a wide area network (WAN), a cellular network, satellite network, or other types of data networks. The network <NUM> may also include any number of computing devices (e.g., computer, servers, routers, network switches, etc.) that are configured to receive and/or transmit data within the network <NUM> and/or external to the network <NUM>. The network <NUM> may include any number of wired and/or wireless connections.

<FIG> is a perspective view of an example computing device <NUM>, including the features of the computing device <NUM> described above with respect to <FIG> in a laptop configuration. <FIG> is a top view of the base portion <NUM> of the example computing device <NUM> shown in <FIG>. <FIG> are front views of the example computing device <NUM> in a tablet configuration. The example computing device <NUM> shown in <FIG> is provided simply for purposes of discussion and illustration. The principles to be described herein may be applied to other types of computing devices having screen capture functionality such as, for example, tablet computing devices, handheld computing devices such as smartphones, and other such computing devices having screen capture functionality.

The example computing device <NUM> includes a display portion <NUM> coupled to a base portion <NUM>. The display portion <NUM> includes an output device in the form of a display <NUM>. The base portion <NUM> includes input devices in the form of an example keyboard <NUM> and an example touchpad <NUM>. In some implementations, the display <NUM> can be a touchscreen display device such that the display <NUM> functions as both an input device and an output device. In the example device <NUM> shown in <FIG>, one or more audio output devices <NUM>, or speakers <NUM>, and one or more audio input devices <NUM>, or microphones <NUM> are provided in the base portion <NUM>. In the example device <NUM> shown in <FIG>, a power control button <NUM> and one or more volume control buttons <NUM> are also provided in the base portion <NUM>.

As noted above, discoverability of screen capture functionality in conventional systems can be difficult for a user to discover and access. For example, some systems may rely on a particular combination of keys and/or sequential entry of a combination of keys, defining a keyboard shortcut to invoke screen capture functionality of the device. For example, in the example computing device <NUM> in the laptop mode shown in <FIG>, this could include depression of the <Ctrl> or <Alt> key together with one of the function keys shown in <FIG> to invoke screen capture functionality, and/or another combination of keys to invoke a full screen capture mode vs a window capture mode and the like. These different combinations of keys can be easily forgotten, erroneously entered and the like. Similarly, in a tablet computing device or a smartphone, or in the tablet mode of the example computing device <NUM> as shown in <FIG>, this could involve pressing and holding multiple buttons such as, for example the power control button <NUM> together with one of the volume control buttons <NUM> at the same time. This may be awkward and difficult to accomplish, particularly while holding the device, and may lead to unintentional device shutdown, unintentional volume change and the like. The need to press multiple control buttons simultaneously may pose particular challenges for people with one or more physical disabilities.

A system and method, in accordance with implementations described herein, provides for improved user discovery and access (or accessibility) to screen capture functionality. In some implementations, this improved user discovery and access includes a dedicated physical key <NUM> as shown in <FIG> that can cause the computing device <NUM> to invoke screen capture functionality and/or enter the screen capture mode in response to a detected depression of the dedicated physical screen capture key <NUM> (rather than a combination of multi-purpose keys). In some implementations, this improved user discovery and access includes a persistent screen capture icon <NUM>, or a persistent screen capture UI element <NUM>, as shown in <FIG> that can cause the computing device <NUM> to invoke screen capture functionality and/or enter the screen capture mode in response to a detected selection of the screen capture UI element <NUM>. The location of the screen capture key <NUM> and/or the persistent screen capture icon <NUM> shown in <FIG> and <FIG> are just exemplary. The screen capture key <NUM> and/or the persistent screen capture icon <NUM> can be in different locations, can be arranged differently with other elements, can have a different appearance and the like. In some implementations, screen capture functionality can be invoked from expansion of a system menu <NUM> shown in <FIG> that provides for user selection of the screen capture UI element <NUM> as shown in <FIG>. In some implementations, screen capture functionality can be invoked via a launcher search initiated by selection of a launcher search icon <NUM> as shown in <FIG> that can map the user to the screen capture mode of the computing device <NUM>. In some implementations, screen capture functionality can be invoked through a press and hold of the power control button <NUM>. The press and hold of the power control button <NUM> can trigger display of a power menu that provides for user selection of the screen capture UI element <NUM> in addition to various power related options, as shown in <FIG>. Accessibility to functionality of the device may therefore be improved.

In response to invoking screen capture functionality in the example computing device <NUM> as described above, an operating system of the example computing device <NUM>, such as the O/S <NUM> described above, may function to generate a screen capture UI <NUM>, as shown in <FIG>. For example, the O/S <NUM> may control the UI generator <NUM> to generate the screen capture UI <NUM> in response to an action that invokes screen capture functionality of the computing device, to be displayed by the computing device <NUM> for user interaction.

<FIG> illustrates an example screen capture UI <NUM> that may be displayed when the computing device <NUM> is in the screen capture mode and screen capture functionality of the computing device <NUM> is active. The screen capture UI <NUM> is displayed on the display <NUM> of the computing device <NUM> such that the screen capture UI <NUM> floats on the display <NUM>, to provide easy access to the screen capture functionality of the computing device <NUM>. In some implementations, the screen capture UI <NUM> may be moved by the user to, for example, improve visibility of/access to content displayed on the display <NUM>. In some implementations, the screen capture UI <NUM> may be hidden by the user to improve visibility of/access to content displayed on the display <NUM>. In the example shown in <FIG>, a plurality of different tabs <NUM> are displayed on the display <NUM> of the computing device <NUM> and are available for selection by the user. An open window <NUM> displays content that is partially obscured by the plurality of tabs <NUM>. The window <NUM> and the tabs <NUM> occupy a portion of a full screen area of the display <NUM>. In the example shown in <FIG>, the screen capture UI <NUM> is displayed in response to the user having invoked the screen capture functionality of the computing device <NUM> as described above.

The example screen capture UI <NUM> includes a first section <NUM>, a second section <NUM>, a third section <NUM> and a fourth section <NUM>. In the example shown in <FIG>, the first section <NUM> includes selectable items or UI elements or icons that trigger a selected screen capture mode in response to selection of one of the icons or elements. In the example shown in <FIG>, the second section <NUM> includes selectable items or icons or UI elements to designate a particular screen capture area in response to selection of one of the icons. In the example shown in <FIG>, the third section <NUM> includes selectable items or icons or UI elements that allow for user selection of audio settings during screen capture (and in particular, during screen recording). In the example shown in <FIG>, the fourth section <NUM> provides for exit from the screen capture mode and/or closure of the screen capture functionality and/or termination of a screen recording session in response to user selection. The example screen capture UI <NUM> shown in <FIG> is just one example arrangement. In some implementations, the screen capture UI <NUM> can include more, or fewer sections, arranged differently than shown in <FIG>. In some implementations, the screen capture UI <NUM> can include more, or fewer, selectable items or icons or UI elements, grouped differently and/or arranged differently.

The first section <NUM> of the example screen capture UI <NUM> includes a first icon <NUM>, or a first UI element <NUM> providing for selection of a screen capture mode, or a screenshot mode (a first capture mode), in which still image content displayed on the display <NUM> of the computing device <NUM> may be captured, and a second icon <NUM>, or a second UI element providing for selection of a screen recording mode (a second capture mode) in which moving image content with or without audio content may be captured. In the screenshot mode, the O/S <NUM> functions to capture screenshots of still image content displayed on the display <NUM> of the computing device <NUM>. In the screen recording mode, the O/S <NUM> functions to capture screen recordings of moving image content, or video content displayed on the display <NUM> of the computing device <NUM>. In the screen recording mode, the O/S <NUM> is configured to function to capture screen recordings of video content displayed on the display <NUM> of the computing device <NUM>, without audio also capturing audio content based on, for example, an audio setting selected by a user. In the screen recording mode, the O/S <NUM> may also function to capture audio content associated with the video content during the screen recording session. In some implementations, the audio content may be captured during the screen capture recording session by the microphone <NUM> of the computing device <NUM>. Audio content captured for recording by the microphone <NUM> may include ambient audio content, or ambient sound, detected within a detection range of the microphone <NUM> and/or a proximity of the computing device <NUM>. This may include, for example audio content output by the speakers <NUM> of the computing device <NUM> and detected by the microphone <NUM>, vocal output detected by the microphone <NUM>, and other ambient sound detectable within the detection range of the microphone <NUM>. In some implementations, the O/S <NUM> may function to capture audio content embedded or inherent in the video content being captured during the screen recording session (audio content associated with the moving image content), with the microphone <NUM> in a standby mode. In some implementations, the O/S <NUM> may function to capture both audio content that is embedded or inherent in the video content and ambient audio content detected by the microphone <NUM>.

The second section <NUM> of the example screen capture UI <NUM> includes first, second and third icons <NUM>, <NUM>, <NUM>, or first, second and third UI elements <NUM>, <NUM>, <NUM> providing for selection of a screen capture area, the screen capture area including at least a portion of the content being displayed. The first UI element <NUM> provides for selection of a full screen capture area. The second UI element <NUM> provides for selection of a custom screen capture area. The third UI element <NUM> provides for selection of a window or tab capture area. By selecting a custom screen capture area or a window or tab capture area, a user can select a portion of the content being displayed by computing device <NUM> to capture.

The third section <NUM> of the example screen capture UI <NUM> includes an icon <NUM> or UI element <NUM> providing for user selection of audio settings associated with the capture of screen content, particularly in a screen recording mode. For example, selection of the UI element <NUM> may enable recording of audio content during screen recording (for example, audio content detected by the microphone of the computing device <NUM> and/or audio content embedded in or inherent in the video content). The fourth section <NUM> of the example screen capture UI <NUM> includes an icon <NUM> or UI element <NUM> that is selectable for termination of a screen recording session, and for exiting the screen capture mode, for example upon completion of the capture of screen content and/or the end of a screen capture session.

<FIG> illustrate an example screen recording session, in accordance with implementations described herein. The features to be described with respect to <FIG> may in most circumstances be similarly applied to an example in which the screen capture mode, or screenshot mode, is selected for the capture of still image content displayed on the display <NUM> of the computing device <NUM>.

In particular, <FIG> illustrates selection of the screen recording mode (the second capture mode) in response to detected selection of the second UI element <NUM> (or the screen recording UI element <NUM>) in the first section <NUM> of the screen capture UI <NUM>. The screen capture of moving image content may be triggered in response to selection of the second UI element <NUM>. In other words, selection of the second capture mode triggers operation in the second capture mode, or screen recording mode. In <FIG>, the second UI element <NUM> is selected, for selecting a custom screen capture area for the screen recording of image content displayed on the display <NUM>. In some implementations, the screen capture area <NUM> may be designated in response gesture inputs detected by a touch sensitive element of the display <NUM>. This may include, for example, a tap input, or a series of tap inputs, to indicate boundaries of the desired screen capture area <NUM>. In some implementations, the gesture inputs may include touch and drag type of inputs detected by the touch sensitive element of the display <NUM>. In some implementations, to designate a specific screen capture area <NUM> within the content displayed on the display <NUM>, the user may click or touch in the area of the display <NUM> to be captured, and then drag, for example, from a first point <NUM>, or an initial point <NUM>, to a second point <NUM>, or a terminal point <NUM>, to set the desired screen capture area <NUM>, as shown in <FIG>. A release of the touch or click at the second point <NUM> may set a boundary <NUM> of the screen capture area <NUM>. Thus, in this example, the custom screen capture area <NUM> may be defined by, for example, a trajectory and a distance between the first point <NUM> and the second point <NUM>. In this example, a custom screen capture area <NUM> is selected by the user. In some implementations, the full screen UI element <NUM> may be selected to enable screen recording of the full screen <NUM> of the display <NUM>, or the window/tab UI element <NUM> may be selected to enable screen recording of a selected window or tab.

In some implementations, the O/S <NUM> functions to determine and display dimensions of the dragged screen capture area <NUM> while the dragging is implemented. In some implementations, the dimension of the dragged screen capture area <NUM> is displayed as a pixel area. In some implementations, the O/S may function to display the dimension of the dragged screen capture area <NUM> in response to a detected hover over a particular portion of the screen capture area <NUM> such as, for example a corner of the screen capture area <NUM>. This may allow the user to designate an accurately sized screen capture area <NUM>. In some implementations, the O/S <NUM> functions to highlight the selected screen capture area <NUM>, and/or to adjust or diminish luminance of unselected areas of the display <NUM> as shown in <FIG>, to accentuate or highlight the selected screen capture area <NUM>. In some implementations, the O/S <NUM> functions to hide the screen capture UI <NUM> and/or adjust a display position of the screen capture UI <NUM> while the screen capture area <NUM> is set in this manner, so that the screen capture UI <NUM> does not obscure the screen capture area <NUM>. In some implementations, the O/S <NUM> may function to auto-hide the screen capture UI <NUM> as the screen capture area <NUM> is selected and adjusted. In some implementations, the O/S <NUM> may hide the screen capture UI <NUM> in response to a user selection or set user preference. In some implementations, the screen capture UI <NUM> may be draggable to different display positions on the display <NUM>, so that the screen capture UI <NUM> is available to the user, but does not obscure the screen capture area <NUM>.

In some implementations, after the screen capture area <NUM> is selected in the manner described above, the boundaries <NUM> of the screen capture area <NUM> may be adjusted by dragging on corners of the screen capture area <NUM> and/or edges of the screen capture area <NUM>, as illustrated by the arrows shown in <FIG>. In some implementations, the user may adjust the screen content that is captured within the boundaries <NUM> of the screen capture area <NUM> by clicking within the screen capture area <NUM> and dragging the displayed content, as shown by the arrow B in <FIG>.

Once the screen capture area <NUM> is finalized, the O/S <NUM> may function to present a capture UI element <NUM>, as shown in <FIG>. In this example, in which the screen recording mode has been selected, selection of the capture UI element <NUM> may initiate a screen recording session in which image content displayed within the boundaries <NUM> of the screen capture area <NUM> is recorded. As described above, in some implementations, audio content may also be recorded during the screen recording session. In other words, triggering screen recording of video content includes recording at least a portion of the moving image content output by the display and optionally recording audio content associated with the video content (audio content associated with the moving image content output by the display and/or audio content detected by the microphone, i.e. both types of audio content may be captured at the same time in some implementations). In some implementations, the O/S <NUM> may function to generate and display a countdown timer, to provide a visual indication to the user of an amount of time remaining until initiation of the screen recording. This may allow the user to further refine the screen capture area <NUM> and the like prior to the start of screen recording. Similarly, the O/S <NUM> may function to generate and display a run timer, to provide a visual indication to the user of an amount of elapsed recording time.

In some situations, the user may wish to capture a screenshot during a screen recording session. In the example shown in <FIG>, a screenshot may be captured during a screen recording session by, for example, selecting the first UI element <NUM> in the first section <NUM> of the screen capture UI <NUM> to enable the screenshot mode (the first capture mode), and designating a screen capture area <NUM> for the screenshot. The screen capture of content within the screen capture area <NUM> may be executed in response to selection of the capture UI element <NUM>. In other words, selection of the first capture mode triggers operation in the first capture mode, or screenshot mode.

In some situations, a user may wish to move content into and out of the screen capture area <NUM> as the screen recording is in progress. As the O/S <NUM> functions to record whatever is displayed within the boundaries <NUM> of the screen capture area <NUM>, the movement of content into the screen capture area <NUM> (for example, dragged into the screen capture area <NUM> by the user) will be captured in the screen recording as long as the dragged content remains within the screen capture area <NUM>. Content dragged into the screen capture area <NUM> for screen recording may include still image content, moving image content, windows, tabs, and other such content. Similarly, content removed from the screen capture area will no longer be included in the screen recording. Additionally, cursor movement, mouse movement, and other such movement of items within the screen capture area <NUM>, alteration of items within the screen capture area <NUM> and the like will be included in the screen recording.

The user may end the screen recording session by, for example, selecting the UI element <NUM> in in the screen capture UI <NUM>.

The screen capture UI <NUM> may allow the user to select and capture screen content in fewer steps, and with fewer user inputs, thus reducing power consumption and computational load. Even so, in some situations, recording of screen content as described above can be computationally intensive and memory intensive depending on, for example, a length of the screen recording, image quality, accompanying audio content and the like. In some implementations, in an effort to manage system level resources, an upper limit may be set for video frame cache, so that issues with memory overwrite are not encountered. In a system and method, in accordance with implementations described herein, screen recording logic is contained within its own independent process outside of the main O/S <NUM> and associated operating system user interface, browser rendering and the like, for example within the GPU <NUM>. Triggering operation in a second capture mode can thus comprise accessing a GPU of the computing device and recording one or more frames of the moving image content from the GPU. Isolation of the screen recording functionality from the main operating system utility may provide a more secure capture process than conventional systems, and may allow for a relatively high performance, memory efficient approach to recording video screen content.

The example screen content capture described above with respect to <FIG> is described with respect to the capture of a screen recording, in which content, for example, moving image content captured within the designated screen capture area is recorded, alone or together with audio content. However, the principles described above can also be applied to the capture of screen content in the screenshot mode, in which still image content within the designated screen content area is captured.

As discussed above, the principles described above with respect to the example capture of screen content in the screen recording mode shown in <FIG> may in general also be applied to the capture of screen content when operating in the screenshot mode (the first capture mode). <FIG> illustrate the example capture of screen content in the screenshot mode, in accordance with implementations described herein.

In this example, the first UI element <NUM> in the first section <NUM> of the screen capture UI <NUM> is selected at (<NUM>) to invoke operation in the screenshot mode, and the third UI element <NUM> in the second section <NUM> of the screen capture UI <NUM> is selected at (<NUM>) to select the Documents tab <NUM> from the plurality of tabs <NUM> displayed on the display <NUM> at (<NUM>). In response to the selection of the Documents tab <NUM> at (<NUM>) in <FIG>, the O/S <NUM> functions to enable capture of a screenshot of the (still image) content in the Documents tab <NUM>, as shown in <FIG>. In this example, only a portion of the Agenda document is visible in the screen capture area displayed in connection with the Documents tab <NUM>. In this situation, a user may wish to capture a screenshot of all of the content contained in the window or tab, or the entire document, even though not all of the content is visible on the display <NUM>. In some implementations, the O/S <NUM> may function to generate and display a full capture UI <NUM> in response to, for example, a cursor hover over the capture area defined by the window/tab/document, touching/holding in the capture area defined by the window/tab/document and the like.

In some implementations, the full capture UI <NUM> may allow the user to choose to capture long form content manually by selecting scrolling capture mode. The scrolling capture mode may allow the user to capture a scrolling screenshot that allows the user to scroll through a document and mark a portion of the document corresponding to an end of the content desired for capture in the screenshot. In some implementations, the full capture UI <NUM> may allow the user to capture long form content automatically by selecting a capture full page mode, in which the system marks all of the content of the identified document for capture in the screenshot. In the example shown in <FIG>, only a portion of the content is visible within the confines of the tab selected for capture. However, selection of the scrolling capture mode or the full page capture mode in this circumstance may yield a screenshot <NUM> including the full content of the document and/or page and/or including content not visible on the screen at the time of initial selection of the window/tab, as shown in <FIG>.

As noted above, captured screen content (including screen recordings and screenshots) are often intended for use shortly after capture, or immediately after capture. Management of captured screen content, including improved and ready access to captured screen content post capture may enhance the user experience and may improve system resource management. That is, system level resource management associated with the capture of screen content and the use and disposition of the captured screen content post-capture may improve computational efficiency of the computing device <NUM>. Accessibility to functionality of the computing device <NUM> may also be improved.

For example, in some implementations, the O/S <NUM> may function to automatically save captured content to a designated location. For example, a captured screenshot may be automatically saved to a clipboard utility for ready user access. A captured screen recording may be automatically saved to a designated storage utility depending on, for example, file size, system capabilities, user preferences, and the like. In some implementations, the O/S <NUM> may function to generate and display a quick action UI <NUM> in response to completion of the capture of screen content (a screenshot or a screen recording), as shown in <FIG>. In some implementations, the quick action UI <NUM> may provide a thumbnail or link to a preview of the captured screen content. In some implementations, the quick action UI <NUM> may include a link for storing the captured screen content. In some implementations, the quick action UI <NUM> may include a link for editing the captured screen content. In some implementations, the quick action UI <NUM> may include various options for sharing the captured content. In some implementations, the quick action UI <NUM> may provide for deleting the captured content. The quick action UI <NUM> may allow the user to take follow on action with the captured screen content in fewer steps, and with fewer user inputs, thus reducing power consumption and computational load. Moreover, this approach can help overcome one or more limitations of a small display size (e.g. on mobile computing devices) by minimizing the need to switch between windows and applications. Accessibility may therefore be improved, particularly for uses with one or more physical disabilities.

In some implementations, multiple content capture areas may be designated by the user. In some implementations, the designation of multiple content capture areas may provide for the substantially simultaneous and/or continuous capture of screen content. In some implementations, two or more moving image capture areas may be designated for capture of screen content, and/or two or more still image capture areas may be designated for capture of screen content, and/or two or more capture areas may be designated for capture of a combination of still images and moving images. In some implementations, one or more still image capture area(s) may be designated within one or more moving image capture area(s). In some implementations, one or more still image capture area(s) may be designated outside of one or more moving image capture area(s). In some implementations, screen content captured from multiple content capture areas in this manner may be saved separately, for viewing separately. In some implementations, screen content captured from multiple content capture areas in this manner may be saved in combination with each other, or in association with each other so that still and/or moving images may be stitched together and displayed and/or replayed in sequence or in parallel.

<FIG> is a flowchart of an example method <NUM> of capturing screen content, in accordance with implementations described herein. The example method <NUM> may be performed by a computing device, such as the example computing device <NUM> described above, that is capable of capturing screen content, including the screen capture still image screen content and moving image screen content (or video content) with or without audio content.

With the computing device <NUM> in an active state (block <NUM>), the screen capture functionality of the device <NUM> may be invoked. The screen capture functionality of the computing device <NUM> may be invoked by one of the example methods described above with respect to <FIG>. That is, in some implementations, the screen capture functionality of the computing device <NUM> may be invoked through user interaction with a dedicated key <NUM> of a keyboard (a physical keyboard or a virtual keyboard) of the computing device <NUM> and/or a persistent screen capture UI element <NUM> displayed by the computing device <NUM>. In some implementations, the screen capture functionality of the computing device <NUM> may be discoverable through a system menu <NUM> and/or through a launcher search <NUM> implemented by the computing device <NUM>. In some implementations, the screen capture functionality of the computing device <NUM> may be discoverable through a press and hold of a designated key or button such as, for example, the power control button <NUM>. In some implementations, the screen capture functionality of the computing device <NUM> may be discoverable and launched by other methods.

In response to detection by the O/S <NUM> that the screen capture functionality of the computing device <NUM> has been invoked (block <NUM>), the O/S <NUM> generates a screen capture UI, such as the example screen capture UI <NUM> shown in <FIG>, for display by the computing device (block <NUM>). A selection of a screen capture mode is then detected. The O/S <NUM> operates the computing device <NUM> in screenshot mode (first capture mode) in response to detection of selection of the screenshot mode, for example, via detected selection of a screenshot UI element of the screen capture UI (blocks <NUM>, <NUM>), or operates the computing device <NUM> in screen recording mode (second capture mode) in response to detected selection of a screen recording UI element of the screen capture UI (blocks <NUM>, <NUM>). The first capture mode is a screenshot mode for capture of still image content output by a display of a computing device, and the second capture mode is a screen recording mode for capture of moving image content output by the display.

Once the screen capture mode has been set, based on user selection (block <NUM>), a screen capture area may be set. The O/S <NUM> operates the computing device <NUM> to capture screen content included in a full screen capture area in response to detection of selection of full screen capture, for example via a detected selection of a full screen capture UI element of the screen capture UI (blocks <NUM>, <NUM>), or operates the computing device <NUM> to capture screen content included in a selected tab or window in response to detection of selection of tab/window capture, for example via detected selection of a tab/window UI element of the screen capture UI (blocks <NUM>, <NUM>), or operates the computing device to capture screen content included in a custom capture area in response to detection of selection of custom capture, for example via detected selection of a custom capture UI element of the screen capture UI (blocks <NUM>, <NUM>). The custom capture area may be set, for example, as described above with respect to <FIG>.

Once the screen capture mode and the screen capture area are set (blocks <NUM>, <NUM>), the O/S <NUM> operates the computing device <NUM> to record (or capture) screen content based on the set screen capture mode and the set screen capture area (block <NUM>). In response to detected completion of the screen capture (block <NUM>), the O/S <NUM> generates a quick action UI, such as the example quick action UI <NUM> shown in <FIG>, for display by the computing device <NUM> (block <NUM>). The O/S <NUM> functions to control the computing device <NUM> to take further action on the captured screen content in response to one or more inputs received via the quick action UI (block <NUM>). In some implementations, the further action can include, for example, previewing, editing, saving, sharing, deleting and the like of the captured screen content.

In a system and method, in accordance with implementations described herein, an operating system can be configured to provide access to screen capture functionality of a computing device via a system menu, a dedicated physical key or virtual user interface element, from a launcher search, and the like. In a smaller form factor device (such as a smartphone or a tablet), the operating system can be configured to provide access to the screen capture functionality in response to a press and hold on the power key of the computing device. The operating system can be configured to activate a screen capture mode and generate a screen capture user interface allowing the user to select a screenshot mode or a screen recording mode; to select a custom portion of the screen, or a particular window, or the full screen for capture; and to exit the screen capture mode. In response to selection of a custom portion of the screen for capture, the operating system is configured to provide the ability to select and edit/alter a portion of the screen to be captured, and to move content into/out of the portion of the screen that is designated for capture, and to allow for full page capture and/or scrolling during capture to provide for capture of content not currently visible on the screen. The operating system is configured to provide for screen recording of video content (including audio content) in addition to screenshotting of still image content, and to provide for screenshotting within the screen recording function. The operating system can provide access to screen capture content on a share sheet or a quick action user interface, including captured screen content and share, copy, edit and delete options. Performing screen recording by accessing a graphics processing unit of the computing device can provide a high performance, memory efficient, approach to capturing video content that is isolated within the operating system so as to be more secure than conventional content capture systems.

The systems and methods described herein may be widely adapted to a variety of user computing devices including small devices such as mobile devices, as well as larger devices including tablets, laptops, desktops and the like. For such devices, the operating system and user interfaces described herein may provide an advantage of making screen capture functionality easily discoverable and accessible, thus avoiding the need to establish/use shortcuts to invoke the screen capture functionality of the device. Fewer user inputs may be needed to access the screen capture functionality, which can reduce power consumption of the device. The operating system and user interfaces described herein make screen recording available on user devices, which is unavailable on conventional systems. The operating system and user interfaces described herein provide for selection of a screen area for capture, and expand pre- and post-capture editing, thus expanding functionality of the user devices. The operating system and user interfaces described herein provide for management of screen capture content and facilitate the storage and sharing of screen capture content, thereby shortening the number of steps required by conventional systems and avoiding time consuming management of the screen capture content.

<FIG> shows an example of a computer device <NUM> and a mobile computer device <NUM>, which may be used with the techniques described here. Computing device <NUM> is intended to represent various forms of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, smart devices, appliances, electronic sensor-based devices, televisions, servers, blade servers, mainframes, and other appropriate computing devices. Computing device <NUM> is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, and other similar computing devices.

The processor <NUM> can be a semiconductor-based processor. The memory <NUM> can be a semiconductor-based memory.

In general, the computer-readable medium may be a non-transitory computer-readable medium.

The computer program product may also contain instructions that, when executed, perform one or more methods and/or computer-implemented methods, such as those described above.

In addition, it may be implemented in a computer such as a laptop computer <NUM>.

Such communication may occur, for example, through radiofrequency transceiver <NUM>. In addition, short-range communication may occur, such as using a Bluetooth, Wi-Fi, or other such transceiver (not shown).

These computer programs (also known as modules, programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, or LED (light emitting diode)) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. For example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.

In some embodiments, the computing devices depicted in <FIG> can include sensors that interface with a virtual reality or headset (VR headset/AR headset/HMD device <NUM>). For example, one or more sensors included on computing device <NUM> or other computing device depicted in <FIG>, can provide input to AR/VR headset <NUM> or in general, provide input to an AR/VR space. The sensors can include, but are not limited to, a touchscreen, accelerometers, gyroscopes, pressure sensors, biometric sensors, temperature sensors, humidity sensors, and ambient light sensors. Computing device <NUM> can use the sensors to determine an absolute position and/or a detected rotation of the computing device in the AR/VR space that can then be used as input to the AR/VR space. For example, computing device <NUM> may be incorporated into the AR/VR space as a virtual object, such as a controller, a laser pointer, a keyboard, a weapon, etc. Positioning of the computing device/virtual object by the user when incorporated into the AR/VR space can allow the user to position the computing device to view the virtual object in certain manners in the AR/VR space.

In some embodiments, one or more input devices included on, or connect to, the computing device <NUM> can be used as input to the AR/VR space. The input devices can include, but are not limited to, a touchscreen, a keyboard, one or more buttons, a trackpad, a touchpad, a pointing device, a mouse, a trackball, a joystick, a camera, a microphone, earphones or buds with input functionality, a gaming controller, or other connectable input device. A user interacting with an input device included on the computing device <NUM> when the computing device is incorporated into the AR/VR space can cause a particular action to occur in the AR/VR space.

In some embodiments, one or more output devices included on the computing device <NUM> can provide output and/or feedback to a user of the AR/VR headset <NUM> in the AR/VR space. The output and feedback can be visual, tactical, or audio. The output and/or feedback can include, but is not limited to, rendering the AR/VR space or the virtual environment, vibrations, turning on and off or blinking and/or flashing of one or more lights or strobes, sounding an alarm, playing a chime, playing a song, and playing of an audio file. The output devices can include, but are not limited to, vibration motors, vibration coils, piezoelectric devices, electrostatic devices, light emitting diodes (LEDs), strobes, and speakers.

In some embodiments, computing device <NUM> can be placed within AR/VR headset <NUM> to create an AR/VR system. AR/VR headset <NUM> can include one or more positioning elements that allow for the placement of computing device <NUM>, such as smart phone <NUM>, in the appropriate position within AR/VR headset <NUM>. In such embodiments, the display of smart phone <NUM> can render stereoscopic images representing the AR/VR space or virtual environment.

In some embodiments, the computing device <NUM> may appear as another object in a computer-generated, 3D environment. Interactions by the user with the computing device <NUM> (e.g., rotating, shaking, touching a touchscreen, swiping a finger across a touch screen) can be interpreted as interactions with the object in the AR/VR space. As just one example, computing device can be a laser pointer. In such an example, computing device <NUM> appears as a virtual laser pointer in the computer-generated, 3D environment. As the user manipulates computing device <NUM>, the user in the AR/VR space sees movement of the laser pointer. The user receives feedback from interactions with the computing device <NUM> in the AR/VR environment on the computing device <NUM> or on the AR/VR headset <NUM>.

In some embodiments, a computing device <NUM> may include a touchscreen. For example, a user can interact with the touchscreen in a particular manner that can mimic what happens on the touchscreen with what happens in the AR/VR space. For example, a user may use a pinching-type motion to zoom content displayed on the touchscreen. This pinching-type motion on the touchscreen can cause information provided in the AR/VR space to be zoomed. In another example, the computing device may be rendered as a virtual book in a computer-generated, 3D environment. In the AR/VR space, the pages of the book can be displayed in the AR/VR space and the swiping of a finger of the user across the touchscreen can be interpreted as turning/flipping a page of the virtual book. As each page is turned/flipped, in addition to seeing the page contents change, the user may be provided with audio feedback, such as the sound of the turning of a page in a book.

In some embodiments, one or more input devices in addition to the computing device (e.g., a mouse, a keyboard) can be rendered in a computer-generated, 3D environment. The rendered input devices (e.g., the rendered mouse, the rendered keyboard) can be used as rendered in the AR/VR space to control objects in the AR/VR space.

A number of embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the scope of the invention.

In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems.

Further to the descriptions above, a user is provided with controls allowing the user to make an election as to both if and when systems, programs, devices, networks, or features described herein may enable collection of user information (e.g., information about a user's social network, social actions, or activities, profession, a user's preferences, or a user's current location), and if the user is sent content or communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that user information is removed. For example, a user's identity may be treated so that no user information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

The computer system (e.g., computing device) may be configured to wirelessly communicate with a network server over a network via a communication link established with the network server using any known wireless communications technologies and protocols including radio frequency (RF), microwave frequency (MWF), and/or infrared frequency (IRF) wireless communications technologies and protocols adapted for communication over the network.

In accordance with aspects of the disclosure, implementations of various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Implementations may be implemented as a computer program product (e.g., a computer program tangibly embodied in an information carrier, a machine-readable storage device, a computer-readable medium, a tangible computer-readable medium), for processing by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). In some implementations, a tangible computer-readable storage medium may be configured to store instructions that when executed cause a processor to perform a process. A computer program, such as the computer program(s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed to be processed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

Specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments. Example embodiments, however, may be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including," when used in this specification, specify the presence of the stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof.

It will be understood that when an element is referred to as being "coupled," "connected," or "responsive" to, or "on," another element, it can be directly coupled, connected, or responsive to, or on, the other element, or intervening elements may also be present. In contrast, when an element is referred to as being "directly coupled," "directly connected," or "directly responsive" to, or "directly on," another element, there are no intervening elements present. As used herein the term "and/or" includes any and all combinations of one or more of the associated listed items.

Spatially relative terms, such as "beneath," "below," "lower," "above," "upper," and the like, may be used herein for ease of description to describe one element or feature in relationship to another element(s) or feature(s) as illustrated in the figures. Thus, the term "below" can encompass both an orientation of above and below. The device may be otherwise oriented (rotated <NUM> degrees or at other orientations) and the spatially relative descriptors used herein may be interpreted accordingly.

Example embodiments of the concepts are described herein with reference to cross-sectional illustrations that are schematic illustrations of idealized embodiments (and intermediate structures) of example embodiments. Thus, example embodiments of the described concepts should not be construed as limited to the particular shapes of regions illustrated herein but are to include deviations in shapes that result, for example, from manufacturing. Accordingly, the regions illustrated in the figures are schematic in nature and their shapes are not intended to illustrate the actual shape of a region of a device and are not intended to limit the scope of example embodiments.

It will be understood that although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. Thus, a "first" element could be termed a "second" element without departing from the teachings of the present embodiments.

Unless otherwise defined, the terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which these concepts belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and/or the present specification and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art.

Claim 1:
A computer-implemented method, comprising:
generating (<NUM>) a screen capture user interface (<NUM>) for display on a display of a computing device, including:
generating a first section (<NUM>) of the screen capture user interface providing for selection of a first capture mode or a second capture mode, and
generating a second section (<NUM>) of the screen capture user interface providing for selection of a screen capture area;
displaying the screen capture user interface on the display of the computing device, such that the screen capture user interface floats on the display;
detecting selection of a screen capture mode on the screen capture user interface (<NUM>); and
operating (<NUM>) in the selected capture mode in response to detecting the selection of the screen capture mode, including:
triggering operation in the first capture mode in response to detection of selection (<NUM>) of the first capture mode, the first capture mode being a screenshot mode for capture of still image content output by the display of the computing device; and
triggering operation in the second capture mode in response to detection of selection (<NUM>) of the second capture mode, the second capture mode being a screen recording mode for capture of moving image content output by the display,
wherein operating in the selected screen capture mode in response to detecting the selection of the screen capture mode further includes:
operating in the second capture mode and capturing a screen recording of the moving image content output in a screen recording capture area of the display in response to detection of the selection of the second capture mode,
detecting a selection of the first capture mode on the screen capture user interface (<NUM>) while operating in the second capture mode, and
triggering operation in the first capture mode and capturing a screenshot while capturing the screen recording.