Notifying applications of screen recording

In some implementations, a computing device can notify a media application when a screen recording may be in progress. For example, the computing device can be configured to send media content provided by the media application for presentation on the display of the computing device to a second destination (e.g., a file, a remote playback device, a remote recording device, etc.) that may result in a recording of the display screen of the computing device. To allow the media application an opportunity to protect the media content from unauthorized recording, the computing device can notify the media application that the media content being sent to the second destination may be recorded. In response to receiving the notification, the media application can modify the presentation of the media content to prevent or downgrade any recording that may be performed.

TECHNICAL FIELD

The disclosure generally relates to managing screen recording on computing devices.

BACKGROUND

Screen recording functionality is common on many computing devices. A user simply has to invoke an operating system function, utility, or standalone application to cause the computing device to record images (e.g., graphical user interfaces, individual windows, everything, etc.) presented on a display of the computing device. Unfortunately, screen recording functionality can allow a user to bypass digital rights management protections built into media applications or media systems and recording the display screen of a computing device from a remote device can be cumbersome.

SUMMARY

In some implementations, a computing device can notify a media application when a screen recording may be in progress. For example, the computing device can be configured to send media content provided by the media application for presentation on the display of the computing device to a second destination (e.g., a file, a remote playback device, a remote recording device, etc.) that may result in a recording of the display screen of the computing device. To allow the media application an opportunity to protect the media content from unauthorized recording, the computing device can notify the media application that the media content being sent to the second destination may be recorded. In response to receiving the notification, the media application can modify the presentation of the media content to prevent or downgrade any recording that may be performed.

In some implementations, a first device can record a display screen of a second device remotely. For example, the first device (e.g., a remote computing device) can receive information identifying other computing devices on a network. The first device can receive a selection of one of the other computing devices (e.g., the second device) and send a message to the second device to initiate a recording of the display screen of the second device. In response to receiving the message, the second device can record video frames of images presented on the display of the second device. The second device can send the video frames to the first device for display and/or storage. While recording the images presented on the display screen, the second device can present indicia on the display screen of the second device indicating that a screen recording is in progress.

Particular implementations provide at least the following advantages. Media applications are provided an opportunity to enforce digital rights management policies when a user initiates a screen recording on a computing device. Media applications are provided an opportunity to enforce digital rights management policies when a user causes media content provided by the media application to be sent to remote computing devices (e.g., a remote playback device, a remote recording device, etc.) where digital rights management policies may not be enforced.

A remote computing device can initiate a screen recording of a second computing device wirelessly, over a network. Because the remote computing device can detect and present information identifying other computing devices on the network, a user of the remote computing device can quickly select between a number of computing device available on the network when selecting a second computing device for screen recording. The remote computing device and the remote computing device do not have to be physically, or directly connected by a wired connection. While the screen recording is in progress, the second computing device can present indicia indicating that a screen recording is in progress on the second computing device so that users of the second computing device are warned of the screen recording in progress.

DETAILED DESCRIPTION

Screen recording functionality is common on many computing devices. A user simply has to invoke an operating system function, utility, or standalone application to cause the computing device to record images (e.g., graphical user interfaces, individual windows, everything, etc.) presented on a display of the computing device.

Unfortunately, screen recording functionality can allow a user to bypass digital rights management protections built into media applications. For example, a media streaming software application may include digital rights management (DRM) protections that prevent the recording and/or sharing of digital media (e.g., movies, videos, music, etc.) by unauthorized users who have not licensed the digital media. However, when the software application sends the DRM protected media to the operating system of a computing device to be presented on the screen (e.g., display) of the computing device, a user may initiate a screen recording and circumvent the DRM protections implemented by the software application. In other instances, the user may cause the computing device to send the digital media to another computing device (e.g., playback device) for presentation to the user. The user may initiate a screen recording on the playback device and circumvent the DRM protections implemented by the software application on the first device. Thus, a mechanism for allowing software applications to enforce DRM protections is needed when media content sent to a display of a computing device is being sent to or presented at a second destination (e.g., recorded to a file, sent to a remote playback device, sent to a remote recording device, etc.) other than the display of the computing device.

Some computing devices (e.g., a first computing device) are configured to allow a second computing device (e.g., remote computing device) to record images presented on a display of the first computing device. However, the second computing device often must be connected to the first computing device through a direct wired connection before the second device is able to record images presented on the screen of the first computing device. A more flexible, dynamic, and less burdensome approach to remotely recording the screen of the first device is needed.

FIG. 1is a block diagram of an example system100for notifying an application of local screen recording in progress. For example, system100can be configured to notify a media application (e.g., application112) that is providing media content for presentation on a display of a computing device that the display screen of the computing device is being recorded by the operating system of the computing device. To give the media application an opportunity to prevent unauthorized copying or recording of the media content by the computing device, the computing device can notify the application that the media content may be recorded by the computing device. The media application can be automatically notified that screen recording has begun by the operating system of the computing device when the screen recording is initiated. The media application can be notified that screen recording is in progress by the operating system in response to a query from the media application.

In some implementations, system100can include computing device110. For example, computing device110can be a laptop computer, a tablet computer, a smartphone, a wearable device (e.g., smartwatch, smart eyeglasses, etc.), a handheld device, an in-vehicle system, and/or a set-top-box.

In some implementations, computing device110can include application112. For example, application112can be a software application that provides media content for presentation on a display screen of computing device110. Application112can be a first party application built by the same vendor that built computing device110and/or the operating system of computing device110. Application112can be a third-party application built by a vendor different than the vendor that built computing device110and/or the operating system of computing device110. When application112is a third-party application, application112may not have access to all of the first-party features and/or private application programming interfaces available to a first party application. Thus, application112may not have direct access to data stored on computing device110or first party notification systems built into computing device110that may indicate when a screen recording is in progress.

In some implementations, a user of computing device110can interact with application112to request that application112present media content on a display (e.g., local display122) of computing device110. The media content can include music, movies, videos, webpages, and/or other media content. Access to the media content can be protected by digital rights management (DRM) technologies built into the media content and/or application112. Access to the media content can be protected by proprietary access control technologies created by the vendor of application112. Application112can control which users and/or computing devices have access to the media content provided by application112using these access control technologies.

In some implementations, application112can send media content to local display122for presentation. For example, when application112determines that computing device110and/or the user of computing device110should be allowed access to media content provided by application112, application112can send the media content114to an operating system (OS) application programming interface (API)116of computing device110. For example, application112can send media content114to rendering server118through a public display API116of the operating system of computing device110.

In some implementations, rendering server118can process the media content114for presentation on the local display (e.g., local display122) of computing device110and store the processed media content in local display buffer120. For example, rendering server118can render image frames (e.g., video frames) corresponding to media content114that are suitable for presentation on local display122(e.g., based on the capabilities of local display122). Local display122can then read the rendered frames from local display buffer120and present the rendered frames on local display122.

In some implementations, computing device110can include local recording user interface124. For example, local recording user interface124can be a graphical user interface of the operating system of computing device110. Local recording user interface can present a graphical object (e.g., button, slider, etc.) on local display122that a user can manipulate to initiate a local screen recording on computing device110. For example, in response to receiving user input to initiate a local screen recording, local recording user interface124can send a local screen recording command to session manager126(e.g., an operating system component) to initiate a screen recording session.

In some implementations, in response to receiving the screen recording command from local recording user interface124, session manager126can create recording session128. For example, recording session128can be a data structure for managing the attributes and data associated with a screen recording session. Recording session128can store screen recording attributes that identify what system or process initiated the screen recording, when the screen recording was initiated, what part of the screen should be recorded (e.g., a particular window, windows of a particular application, the entire display screen, etc.), and a destination (e.g., a second destination other than the local display122) where to send the frames captured or recorded as part of the screen recording session. For example, the destination can be a local file or media object, a remote playback device, a remote recording device, or other destination, as described herein. In the case of a local recording, the destination can be a file or media object stored on computing device110. In some implementations, media content provided by application112for presentation on display122can be sent to display122, the second destination, or both display122and the second destination.

In some implementations, session manager126can send a notification to application112indicating that a screen recording session has been initiated or requested. For example, in response to receiving the screen recording command from local recording UI, session manager126can determine that application112is currently presenting content on local display122and send application112message130notifying application112that a screen recording session is about to commence.

In some implementations, session manager126can send a notification to application112indicating that a screen recording session is in progress. For example, before application112starts to send media content to local display122, application112can send a message (e.g., query) to session manager126through API116to determine the current screen recording status of computing device110. Session manager126can determine if a screen recording session is in progress (e.g., determine if recording session128exists) and send message130to application112indicating whether a screen recording session is in progress. If application112receives a message indicating that a screen recording session is not in progress, application112can send media content114to local display122, as described above.

In response to receiving a message130indicating that a screen recording is about to commence or that a screen recording is in progress, application112can modify the media content presented by application112on local display122. For example, when a screen recording session is initiated or in progress, session manager126can copy frames of video from local display buffer120and store the frames in media file132(e.g., a media object, a package of media files, etc.). To prevent computing device110from making copies of access protected (e.g., DRM protected) media, application112can send modified media content to local display122when a screen recording has been initiated or is in progress so that the modified media content is copied or recorded instead of the protected media content. For example, instead of sending high-quality media content to display122, application112can send media content that presents a warning instead of the requested media content, watermarked media content, low quality or low-resolution media content, or other types of modified media content in response to receiving the notification that a screen recording has been initiated or is in progress.

FIG. 2is a block diagram of an example system200for notifying an application of a potential screen recording by a remote playback device. For example, system200can correspond to system100described above. System200can be configured to notify a media application that is providing media content for presentation on a display of a computing device that the media content may be recorded by a remote playback device. For example, computing device110can be configured to stream the media content presented on local display122to a remote playback device. To give the media application an opportunity to prevent unauthorized copying or recording of the media content by the remote playback device, computing device110can notify application112of the possibility that the media content may be recorded by the remote playback device. In one aspect, the media application can be automatically notified that the media content may be recorded by a remote playback device when the media content is streamed to the remote playback device. In another aspect, the media application can be notified that the media content may be recorded by a remote playback device in response to a query from the media application.

In some implementations, system200can include computing device110. For example, computing device110can be a computing device, such as a laptop computer, a tablet computer, a smartphone, a wearable device (e.g., smartwatch, smart eyeglasses, etc.), an in-vehicle system, and/or a set-top-box.

In some implementations, computing device110can include application112. For example, application112can be a software application that provides media content for presentation on a display screen of computing device110. For example, a user of computing device110can interact with application112to request that application112present media content on a display (e.g., local display122) of computing device110. The media content can include music, movies, videos, webpages, and/or other media content. Access to the media content can be protected by digital rights management (DRM) technologies built into the media content and/or application112. Access to the media content can be protected by proprietary access control technologies created by the vendor of the software application. Thus, application112can control which users and/or computing devices have access to the media content provided by application112.

In some implementations, application112can send media content to local display122for presentation. For example, when application112determines that computing device110and/or the user of computing device110should be allowed access to media content provided by application112, application112can send the media content114to an operating system (OS) application programming interface (API)116of computing device110. For example, application112can send media content114to rendering server118through a display API116of the operating system of computing device110.

In some implementations, rendering server118can process the media content114for presentation on the local display (e.g., local display122) of computing device110and store the processed media content in local display buffer120. For example, rendering server118can render image frames (e.g., video frames) corresponding to media content114that are suitable for presentation on local display122. A display driver (not shown) for local display122can then read the rendered frames from local display buffer120and present the rendered frames on local display122.

In some implementations, computing device110can include remote display user interface202. For example, remote display user interface202can be a graphical user interface of the operating system of computing device110. Remote display user interface202can present a graphical object (e.g., button, slider, etc.) on local display122that a user can manipulate to cause computing device110to stream the media content intended for local display122to a remote playback device (e.g., playback device230). For example, in response to receiving user input to stream local display122to playback device230, remote display user interface202can send a screen recording command to session manager126(e.g., an operating system component) to send the media content (e.g., display content) intended for local display122to playback device230.

In some implementations, in response to receiving the screen recording command from remote display user interface202, session manager126can create recording session204. For example, recording session204can be a data structure for managing the attributes and data associated with a screen recording session. Recording session204can store screen recording attributes that identify what system or process initiated the screen recording, when the screen recording was initiated, what part of the screen should be recorded (e.g., a particular window, windows of a particular application, the entire display screen, etc.), and a destination (e.g., a second destination other than the local display122) where to send the frames captured or recorded as part of the screen recording session. For example, the destination can be a local file or media object, a remote playback device, or a remote recording device, as described herein. When the local display screen is recorded for the purposes of presenting the media content intended for the local display screen122on a remote playback device230, the destination can be remote playback device230. For example, the media content rendered into video frames and stored in local display buffer120can be recorded (e.g., copied) and streamed (e.g., sent) to playback device230for presentation on a display of playback device230.

In some implementations, session manager126can send a notification to application112indicating that a screen recording session (e.g., a remote display session) has been initiated. For example, in response to receiving the screen recording command from local recording UI, session manager126can determine that application112is currently presenting content on local display122and send application112message130notifying application112that a screen recording session is about to commence.

In some implementations, session manager126can send a notification to application112indicating that a screen recording session (e.g., a remote display session) is in progress. For example, before application112starts to send media content to local display122, application112can send a message (e.g., query) to session manager126through API116to determine the current screen recording status of computing device110. Session manager126can determine if a screen recording session is in progress (e.g., determine if recording session204exists) and send message130to application112indicating whether a screen recording session is in progress. If application112receives a message indicating that a screen recording session is not in progress, application112can send media content114to local display122, as described above.

In response to receiving a message130indicating that a screen recording is about to commence or that a screen recording is in progress, application112can modify the media content presented by application112on local display122. For example, when a screen recording session is initiated or in progress, session manager126can copy frames of video from local display buffer120and store the frames in remote display buffer206(e.g., a memory location). Streaming module208can then read the video frames from remote display buffer206and send the video frames to playback device230, which may be configured similarly to computing device110, through network220(e.g., local area network, Wi-Fi network, etc.). Streaming module232on playback device230can receive the video frames sent from computing device110and present the video frames on remote display234of playback device230. For example, remote display234can be an internal component of playback device230. Remote display234can be a display (e.g., television, monitor, etc.) external to playback device230.

To prevent computing device110from making copies of access protected (e.g., DRM protected) media, application112can send modified media content to local display122when a screen recording has been initiated or is in progress so that the modified media content is copied or recorded instead of the protected media content. For example, application112can send media content that presents a warning instead of the requested media content, watermarked media content, low quality or low-resolution media content, or other types of modified media content in response to receiving the notification that a screen recording has been initiated or is in progress.

FIG. 3is a block diagram of an example system300for screen recording from a remote recording device. For example, system300can correspond to system100and/or system200described above. System300can be configured to initiate and perform a screen recording of media content intended for presentation on local display122of computing device110from a remote computing device (e.g., recording device302). For example, media content intended for presentation on local display122can be streamed to recording device302. Recording device302can then present and/or record the streamed media content.

System300can be configured to notify a media application that is providing media content for presentation on a display of a computing device that the media content may be recorded by a remote recording device302. For example, computing device110can be configured to stream the media content presented on (or intended for presentation on) local display122to a remote recording device. To give the media application an opportunity to prevent unauthorized copying or recording of the media content by the remote recording device302, computing device110can notify application112of the possibility that the media content may be recorded by the remote recording device. The media application can be automatically notified that the media content may be recorded by a remote recording device when the media content is streamed to the remote recording device. The media application can be notified that the media content may be recorded by a remote recording device in response to a query from the media application.

In some implementations, system300can include computing device110. For example, computing device110can be a computing device, such as a laptop computer, a tablet computer, a smartphone, a wearable device (e.g., smartwatch, smart eyeglasses, etc.), an in-vehicle system, and/or a set-top-box.

In some implementations, computing device110can include application112. For example, application112can be a software application that provides media content for presentation on a display screen of computing device110. For example, a user of computing device110can interact with application112to request that application112present media content on a display (e.g., local display122) of computing device110. The media content can include music, movies, videos, webpages, and/or other media content. Access to the media content can be protected by digital rights management (DRM) technologies built into the media content and/or application112. Access to the media content can be protected by proprietary access control technologies created by the vendor of the software application. Thus, application112can control which users and/or computing devices have access to the media content provided by application112.

In some implementations, application112can send media content to local display122for presentation. For example, when application112determines that computing device110and/or the user of computing device110should be allowed access to media content provided by application112, application112can send the media content114to an operating system (OS) application programming interface (API)116of computing device110. For example, application112can send media content114to rendering server118through a display API116of the operating system of computing device110.

In some implementations, rendering server118can process the media content114for presentation on the local display (e.g., local display122) of computing device110and store the processed media content in local display buffer120. For example, rendering server118can render image frames (e.g., video frames) corresponding to media content114that are suitable for presentation on local display122. A display driver (not shown) for local display122can then read the rendered frames from local display buffer120and present the rendered frames on local display122.

In some implementations, system300can include recording device302. For example, recording device302can be configured similarly to computing device110. In a particular example, recording device302could be a laptop computer that a user uses to record content presented by computing device110(e.g., a smartphone, tablet computer, set-top-box, etc.).

When recording device302is connected to the same network220as other computing devices (e.g., computing device110and/or other similar devices), recording device302can receive advertisement messages from the other computing devices indicating the availability, features, and/or capabilities of the other computing devices on network220. Media editing application304can present a graphical user interface that identifies computing device110and other computing devices from which advertisement messages have been received.

A user of media editing application304can select one of the advertising computing devices (e.g., computing device110) and initiate a recording of the display screen of computing device110from media editing application304on recording device302. For example, media editing application304can use information in the advertisement message received from computing device110to establish a network connection (e.g., through a local area network, Wi-Fi network, etc.) to computing device110. When establishing the network connection, recording device302and computing device110can perform an access control procedure, as described below with reference toFIG. 4. For example, the access control procedure can include presenting a pairing code on computing device110and receiving the pairing code as user input to recording device302.

Media editing application304can send a message through the established network connection (e.g., through network220) to computing device110to initiate a screen recording session on computing device110. For example, media editing application304can send a request to streaming module306to initiate a screen recording session on computing device110. Streaming module306can communicate with streaming module208on computing device110to request a screen recording session on computing device110. Streaming module208can then send a command to initiate a screen recording session to session manager126. Session manager126can cause computing device110to present a prompt requesting user input to allow the screen recording session on computing device110to proceed, as illustrated byFIG. 5.

In some implementations, in response to receiving user input to allow the screen recording, session manager126can create recording session304. For example, recording session304can be a data structure for managing the attributes and data associated with a screen recording session. Recording session304can store screen recording attributes that identify what system or process initiated the screen recording, when the screen recording was initiated, what part of the screen should be recorded (e.g., a particular window, windows of a particular application, the entire display screen, etc.), and a destination (e.g., a second destination other than the local display122) where to send the frames captured or recorded as part of the screen recording session. For example, the destination can be a local file or media object, a remote playback device, or a remote recording device, as described herein. When media editing application304on recording device302initiates the screen recording on computing device110, the destination can be remote recording device302and/or media editing application304. For example, the media content rendered into video frames and stored in local display buffer120can be recorded (e.g., copied) and streamed (e.g., sent) to recording device302for presentation and/or storage on recording device302.

In some implementations, session manager126can send a notification to application112indicating that a screen recording session (e.g., a remote recording session) has been initiated. For example, in response to receiving the screen recording command from streaming module208(e.g., originated by media editing application304), session manager126can automatically determine that application112is currently presenting content on local display122and send application112message130notifying application112that a screen recording session is about to commence.

In some implementations, session manager126can send a notification to application112indicating that a screen recording session (e.g., a remote display session) is in progress. For example, before application112starts to send media content to local display122, application112can send a message (e.g., query) to session manager126through API116to determine the current screen recording status of computing device110. Session manager126can determine if a screen recording session is in progress (e.g., determine if recording session304exists) and send message130to application112indicating whether a screen recording session is in progress. If application112receives a message indicating that a screen recording session is not in progress, application112can send media content114to local display122, as described above.

In response to receiving a message130indicating that a screen recording is about to commence or that a screen recording is in progress, application112can modify the media content presented by application112on local display122. For example, when a screen recording session is initiated or in progress, session manager126can copy frames of video from local display buffer120and send the frames to streaming module208. Streaming module208can then send the video frames to recording device302, which may be configured similarly to computing device110, through network220(e.g., local area network, Wi-Fi network, etc.). Streaming module306on recording device302can receive the video frames sent from computing device110and send the video frames to media editing application304. Media editing application304can then present the video frames on a display of recording device302and/or store the video frames in a media object (e.g., video file) on recording device302.

To prevent computing device110from making copies of access protected (e.g., DRM protected) media, application112can send modified media content to local display122when a screen recording has been initiated or is in progress so that the modified media content is copied or recorded instead of the protected media content. For example, application112can send media content that presents a warning instead of the requested media content, watermarked media content, low quality or low-resolution media content, or other types of modified media content in response to receiving the notification that a screen recording has been initiated or is in progress.

In some implementations, computing device110can present indicia indicating that screen recording is in progress on computing device110. For example, when a screen recording is in progress on computing device110, session manager126can cause a graphical element (e.g., indicia, warning, etc.) to be presented on local display122indicating that a screen recording session is in progress, as illustrated byFIG. 6. While local display122may present the graphical element, indicia, warning, etc., during the screen recording session, the graphical element, indicia, warning, etc., may be excluded from the screen recording. For example, the media file created as a result of the screen recording and/or the video frames sent to the remote devices may not include the graphical element, indicia, warning, etc., warning users of computing device110of the screen recording in progress.

FIG. 4illustrates an example graphical user interface400for pairing a remote recording device with a computing device for the purposes of remote screen recording. For example, when recording device302attempts to establish a network connection with computing device110, computing device110can present graphical user interface (GUI)400as part of an access control procedure (e.g., pairing process) to determine whether recording device302should be allowed to connect to and/or interoperate with computing device110. In some implementations, GUI400can present a textual prompt that instructs a viewing user to provide a pairing code402(e.g., sequence of characters or numbers) as input to recording device302. When recording device302receives input specifying the pairing code, recording device302can send the pairing code to computing device110through network220. If the pairing code received from recording device302matches the pairing code presented by computing device110, then computing device110can allow recording device302to access various features or functionality of computing device110, including the screen recording features described above.

FIG. 5illustrates an example graphical user interface500for presenting a prompt for allowing screen recording on computing device110. For example, when session manager126receives a screen recording command (e.g., as part of initiating a screen recording from a remote device), session manager126can cause computing device110to present GUI500, including a textual prompt asking a user of computing device110whether the user would like to begin a screen recording session. If the user selects graphical element504, then session manager126can prevent the screen recording session from starting. If the user selects graphical element506, then session manager126can proceed with the screen recording session on computing device110.

FIG. 6illustrates an example graphical user interface600for presenting indicia of a screen recording in progress. For example, GUI600can be presented on a display of computing device110when a screen recording session (e.g., as part of a screen recording initiated from a remote device) is in progress on computing device110. GUI600can include media content display area602for presenting media content provided by application112. Media content display area602can be inset in a colored (e.g., red, yellow, etc.) frame that indicates that a screen recording session is in progress. GUI600can include a textual warning606or graphical warning (not shown) that indicates that a screen recording is in progress. These indicia of a screen recording in progress can inform a potential user of computing device110that any content presented on a display of computing device110may be recorded by computing device110or sent to another computing device (e.g., playback device230, recording device302, etc.) and may be recorded. In some implementations, the indicia of a screen recording in progress is not recorded, streamed, or stored as part of the screen recording.

Example Processes

To enable the reader to obtain a clear understanding of the technological concepts described herein, the following processes describe specific steps performed in a specific order. However, one or more of the steps of a particular process may be rearranged and/or omitted while remaining within the contemplated scope of the technology disclosed herein. Moreover, different processes, and/or steps thereof, may be combined, recombined, rearranged, omitted, and/or executed in parallel to create different process flows that are also within the contemplated scope of the technology disclosed herein. Additionally, while the processes below may omit or briefly summarize some of the details of the technologies disclosed herein for clarity, the details described in the paragraphs above may be combined with the process steps described below to get a more complete and comprehensive understanding of these processes and the technologies disclosed herein.

FIG. 7is flow diagram of an example process700for automatically notifying an application that media content sent to a display screen may be recorded. For example, process700can be performed by computing device110to notify (e.g., warn) a media content presenting application that media content presented by the application may be recorded. For example, when computing device110records screen content to a file or sends the screen content to a remote device that may record the screen content, the application is automatically notified (e.g., without the application making a query) that a screen recording has been initiated or in progress so that the application can take steps to avoid unauthorized duplication of protected or proprietary content.

At step702, computing device110can receive media content from application112for presentation by computing device110. For example, a user of application112can request that application112present media content on a display of computing device110. In response to receiving the request, application112can provide media content (e.g., movies, video, electronic books, etc.) to computing device110for presentation on a display of computing device110.

At step704, computing device110can present the media content on a display of computing device110. For example, the media content can be rendered into a display buffer and the rendered content can be presented on a display of computing device110.

At step706, computing device110can receive a request to send the presented media content to a second destination. For example, when a local screen recording has been requested, the second destination can be a media file stored on computing device110. When a user provides input requesting that the media content be presented on a remote playback device (e.g., remote display), then the second destination can be the remote playback device. When computing device110receives a request from a remote recording device to initiate a screen recording, then the second destination can be the remote recording device.

At step708, computing device110can send a message to application112indicating that the media content provided by application112will be recorded. For example, based on one or more factors (e.g., an identifier of the remote device, an identifier of an application that is displaying the media content at the remote device, a type of application that is displaying the media content at the remote device, an indication from the remote device that a recording has been or will be initiated, etc.), computing device110can estimate that remote devices may be recording the display screen content received by the remote devices. In some instances, computing device110can estimate that the remote devices may be recording the display screen content based on the fact that the display screen content is being sent to a remote device. For example, computing device cannot control what remote devices (e.g., remote playback devices, remote recording devices, etc.) may do with the display screen content sent to the remote devices, thus as a precaution computing device110can estimate that remote devices may be recording the display screen content received by the remote devices based simply on the fact that the display screen content is being sent to a remote device. Thus, regardless of whether the second destination is a media file or remote device, computing device110can send a message to application112indicating that the display screen of computing device110is being recorded. This may be helpful particularly when computing device110cannot control what remote devices (e.g., remote playback devices, remote recording devices, etc.) may do with the display screen content sent to the remote devices.

At step710, computing device110can store recording session data indicating media content presented on the display of computing device is being sent to a second destination. For example, the recording session data can include, among other things, information indicating where the display screen recording (e.g., image frames, video frames, etc.) is being sent (e.g., to a file, to a remote device, etc.). The existence of the recording session data can also be used by computing device110to determine whether a recording session is in progress when a subsequent query is made by an application on computing device110, as described herein.

At step712, computing device110can receive modified media content from application112. For example, in response to receiving a message indicating that media content presented on a display of computing device110is being or is about to be recorded, application112can modify the media content provided by application112to avoid the unauthorized duplication of access control (e.g., DRM) protected media content. For example, application112can be configured to modify the media content provided by application112so that a blank screen, a warning, or low-quality media content is presented and/or recorded instead of a higher quality version of the requested media content.

FIG. 8is flow diagram of an example process800for notifying an application that media content sent to a display screen may be recorded in response to receiving a query from the application. For example, process700can be performed by computing device110to notify (e.g., warn) a media content presenting application that media content presented by the application may be recorded. For example, whether computing device110is recording screen content to a file or sending the screen content to a remote device that may record the screen content, the application is notified that a screen recording has been initiated or in progress so that the application can take steps to avoid unauthorized duplication of protected or proprietary content.

At step802, computing device110can receive a request to send media content presented on a display of computing device110to a second destination. For example, when a local screen recording has been requested, the second destination can be a media file stored on computing device110. As another example, when a user provides input requesting that the media content be presented on a remote playback device (e.g., remote display), then the second destination can be the remote playback device. As yet another example, when computing device110receives a request from a remote recording device to initiate a screen recording, then the second destination can be the remote recording device.

At step804, computing device110can store recording session data indicating media content presented on the display of computing device is being sent to a second destination. For example, the recording session data can include, among other things, information indicating where the display screen recording (e.g., image frames, video frames, etc.) is being sent (e.g., to a file, to a remote device, etc.). The existence of the recording session data can also be used by computing device110to determine whether a recording session is in progress when a subsequent query is made by an application on computing device110, as described herein.

At step806, computing device110can receive a request from application112for the screen recording status of computing device110. For example, application112can send query to an API of computing device110requesting the current display screen recording status of computing device110.

At step808, computing device110can send a message to application112indicating that media content presented on the display of computing device110is being recorded. For example, computing device110can determine that a screen recording session is in progress on computing device110based on the recording session data stored on computing device110.

At step810, computing device110can receive modified media content from application112. For example, after receiving the message indicating that a screen recording is in progress, application112can determine whether to present content on the display of computing device110and/or what media content to send to the display of computing device110. For example, application112can modify the media content provided by application112to avoid the unauthorized duplication of access control (e.g., DRM) protected media content. For example, application112can be configured to modify the media content provided by application112so that a blank screen, a warning, or low-quality media content is presented and/or recorded instead of a higher quality version of the requested media content.

FIG. 9is an example process900for initiating a screen recording from a remote device. For example, process900can be performed by remote recording device302ofFIG. 3to remotely record content presented on a display screen of computing device110.

At step902, recording device302can determine the computing devices that are available on the network to which recording device302is connected. For example, each computing device connected to the same network as recording device302can broadcast their existence and/or availability on the network. Recording device302can receive the broadcast messages and use the information in the broadcast messages to determine which computing devices are on the network and how to connect to the broadcast computing devices (e.g., computing device110).

At step904, recording device302can receive a selection of computing device110. For example, recording device302can include a media editing application. The media editing application can include functionality for initiating a screen recording on another computing device. The media editing application can present a graphical user interface that identifies other computing devices (e.g., computing device110) that are available on the same network as recording device302. A user can select one of the identified computing devices and provide input to the media editing application to initiate a recording of a display screen of the selected computing device. For example, the user can provide input to specify that the media editing application should initiate a screen recording of the display of computing device110.

At step906, recording device302can send a message to the selected computing device to initiate a screen recording of the display of the computing device. For example, recording device302can send a message to computing device110to initiate a screen recording of the media content presented on the display of computing device110.

At step910, recording device302can cooperate with computing device110perform an access control procedure. For example, before computing device110allows recording device302to initiate the screen recording, computing device110can present a pairing code on the display of computing device110. The user of computing device110and recording device302can enter the pairing code into recording device302and recording device302can send the pairing code to computing device110over the network. If the code entered at recording device302matches the code presented by computing device110, then recording device302can be allowed to access the functionality of computing device110and initiate the screen recording on computing device110, as described above.

At step910, recording device302can receive video frames corresponding to the display screen recorded on computing device110. For example, as computing device110copies the frames of video being presented on the display of computing device110, computing device110can send the frames of video to the media editing application on recording device302over the network.

At step912, recording device302can present the received video frames on the display of recording device302. For example, the media editing application can present the received frames on the display of recording device302in a graphical user interface generated by the media editing application.

At step914, recording device302can store the video frames in a media object stored on recording device302. For example, the media editing application can store the received video frames in a video file stored on recording device302.

At step916, recording device302can send a message to computing device110to terminate the screen recording. For example, when the user of recording device302is finished recording the media content presented on the display of computing device110, the user can provide input to the media editing application to termination the screen recording session. In response to the input, the media editing application can send a message to computing device110to terminate the remote screen recording.

FIG. 10is an example process1000for sending a screen recording to a remote recording device. For example, process1000can be performed by computing device110to send or stream content presented on a display screen of computing device110to remote recording device302so that the displayed content can be recorded by recording device302.

At step1002, computing device110can receive a message from recording device302requesting a screen recording. For example, computing device110can broadcast a message indicating the availability of computing device110on a network (e.g., local area network, Wi-Fi network, etc.). The broadcast message can include an identifier for computing device110and/or other information indicating how to contact and/or interact with computing device110. For example, the broadcast message can indicate that computing device110is configured to provide remote screen recording capabilities. Computing device110can receive the screen recording request message from a remote recording device302that received the broadcast message.

At step1004, computing device110can perform an access control procedure in cooperation with recording device302. For example, before computing device110allows recording device302to receive a screen recording (e.g., image frames, video frames, etc.), computing device110will determine whether recording device302is authorized to access the features or functionality of computing device110. To do so, computing device110can present a pairing code on a display (e.g., internal display, external display, television, etc.) of computing device110. After presenting the pairing code, computing device110can receive a pairing code from recording device302. If the received pairing code matches (e.g., is identical to) the received pairing code, then computing device110can allow recording device302to obtain a recording of the display screen of computing device110. Stated differently, if the received pairing code matches (e.g., is identical to) the received pairing code, then computing device110may send a recording of the display screen of computing device110to recording device302.

At step1006, computing device110can present a prompt to initiate a screen recording of a display screen of computing device110. For example, computing device110can present a prompt (e.g., the prompt ofFIG. 5) to obtain the user's authorization to proceed with the screen recording on computing device110. When the user provides input indicating that the screen recording should not proceed, computing device110can prevent the screen recording from proceeding. When the user provides input indicating that the screen recording should proceed, computing device110can initiate the screen recording on computing device110.

At step1008, computing device110can initiate a screen recording. For example, computing device110can create a screen recording session. Before or during the screen recording session, computing device110can notify media content providing applications that a screen recording session is in progress, as described above.

At step1010, computing device110can present indicia of the screen recording in progress. For example, computing device110can present on a display of computing device110textual or graphical warnings, colored borders or frames, inset media content, and/or other indicia that a screen recording is in progress so that users of computing device110will be warned that any content the user's cause to be presented on the display of computing device110may be recorded.

At step1012, computing device110can obtain video frames corresponding to the content presented on the display of computing device110. For example, computing device110can record, capture, copy, etc., rendered frames of video intended for presentation on a display of computing device110from a display buffer, as described above.

At step1014, computing device110can send the frames of video to recording device302. For example, computing device110can send the rendered frames of video obtained from the display buffer to recording device302through network220.

FIG. 11is an example process1100for terminating a remote screen recording at a computing device. For example, process1100can be performed by computing device110to terminate a remotely initiated recording of the display screen of computing device110.

At step1102, computing device110can receive a message from recording device302to terminate the screen recording session. For example, recording device302may send the termination message in response to the user providing input to a media editing application indicating that the user wishes to terminate the screen recording session. In some implementations, computing device110can generate an internal message or command to terminate the screen recording session when computing device110detects that the connection to recording device302has been terminated or when computing device110receives user input indicating that the screen recording session should be terminated.

At step1104, computing device110can terminate the screen recording session. For example, computing device110can stop copying rendered frames of video from the display buffer associated with the display of computing device110.

At step1106, computing device110can hide the indicia of screen recording in progress. For example, since the media content displayed on computing device110is no longer being recorded, the user of computing device110no longer needs to be warned of the screen recording. Thus, computing device110can hide the indicia of screen recording in progress and present content on the display of computing device110using the full display screen (e.g., not inset).

Graphical User Interfaces

This disclosure above describes various Graphical User Interfaces (GUIs) for implementing various features, processes or workflows. These GUIs can be presented on a variety of electronic devices including but not limited to laptop computers, desktop computers, computer terminals, television systems, tablet computers, e-book readers and smart phones. One or more of these electronic devices can include a touch-sensitive surface. The touch-sensitive surface can process multiple simultaneous points of input, including processing data related to the pressure, degree or position of each point of input. Such processing can facilitate gestures with multiple fingers, including pinching and swiping.

When the disclosure refers to “select” or “selecting” user interface elements in a GUI, these terms are understood to include clicking or “hovering” with a mouse or other input device over a user interface element, or touching, tapping or gesturing with one or more fingers or stylus on a user interface element. User interface elements can be virtual buttons, menus, selectors, switches, sliders, scrubbers, knobs, thumbnails, links, icons, radio buttons, checkboxes and any other mechanism for receiving input from, or providing feedback to a user.

Privacy

As described above, one aspect of the present technology provides for recording content presented on a display screen of a computing device. The present disclosure contemplates that in some instances, this recorded data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data can include demographic data, location-based data, telephone numbers, email addresses, twitter ID's, home addresses, data or records relating to a user's health or level of fitness (e.g., vital signs measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

The present disclosure contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. Such policies should be easily accessible by users, and should be updated as the collection and/or use of data changes. Personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection/sharing should occur after receiving the informed consent of the users.

Additionally, such entities should consider taking any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices. In addition, policies and practices should be adapted for the particular types of personal information data being collected and/or accessed and adapted to applicable laws and standards, including jurisdiction-specific considerations. For instance, in the US, collection of or access to certain health data may be governed by federal and/or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); whereas health data in other countries may be subject to other regulations and policies and should be handled accordingly. Hence different privacy practices should be maintained for different personal data types in each country.

Despite the foregoing, the present disclosure also contemplates embodiments in which users selectively block the use of, recording or collection of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to or recording of such personal information data. For example, in the case of screen recordings, the present technology can be configured to allow users to select to “opt in” or “opt out” of participation in the screen recording. For example, a computing device that is the subject or target of a screen recording can be configured to warn users of a screen recording in progress or to allow users of the computing device to disable or disallow screen recordings on the computing device. In addition to providing “opt in” and “opt out” options, the present disclosure contemplates providing notifications relating to the access or use of personal information. For instance, a user may be notified upon initiation of a screen recording or during a screen recording that screen recording is about to be commenced or is in progress.

Therefore, although the present disclosure broadly covers the possibility of recording of personal information data while recording display screen content, the present disclosure also contemplates that the various embodiments can be implemented or executed without recording such personal information data.

Example System Architecture

FIG. 12is a block diagram of an example computing device1200that can implement the features and processes ofFIGS. 1-11. The computing device1200can include a memory interface1202, one or more data processors, image processors and/or central processing units1204, and a peripherals interface1206. The memory interface1202, the one or more processors1204and/or the peripherals interface1206can be separate components or can be integrated in one or more integrated circuits. The various components in the computing device1200can be coupled by one or more communication buses or signal lines.

Sensors, devices, and subsystems can be coupled to the peripherals interface1206to facilitate multiple functionalities. For example, a motion sensor1210, a light sensor1212, and a proximity sensor1214can be coupled to the peripherals interface1206to facilitate orientation, lighting, and proximity functions. Other sensors1216can also be connected to the peripherals interface1206, such as a global navigation satellite system (GNSS) (e.g., GPS receiver), a temperature sensor, a biometric sensor, magnetometer or other sensing device, to facilitate related functionalities.

A camera subsystem1220and an optical sensor1222, e.g., a charged coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) optical sensor, can be utilized to facilitate camera functions, such as recording photographs and video clips. The camera subsystem1220and the optical sensor1222can be used to collect images of a user to be used during authentication of a user, e.g., by performing facial recognition analysis.

Communication functions can be facilitated through one or more wireless communication subsystems1224, which can include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem1224can depend on the communication network(s) over which the computing device1200is intended to operate. For example, the computing device1200can include communication subsystems1224designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth™ network. In particular, the wireless communication subsystems1224can include hosting protocols such that the device100can be configured as a base station for other wireless devices.

An audio subsystem1226can be coupled to a speaker1228and a microphone1230to facilitate voice-enabled functions, such as speaker recognition, voice replication, digital recording, and telephony functions. The audio subsystem1226can be configured to facilitate processing voice commands, voiceprinting and voice authentication, for example.

The I/O subsystem1240can include a touch-surface controller1242and/or other input controller(s)1244. The touch-surface controller1242can be coupled to a touch surface1246. The touch surface1246and touch-surface controller1242can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface1246.

The other input controller(s)1244can be coupled to other input/control devices1248, such as one or more buttons, rocker switches, thumb-wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) can include an up/down button for volume control of the speaker1228and/or the microphone1230.

In one implementation, a pressing of the button for a first duration can disengage a lock of the touch surface1246; and a pressing of the button for a second duration that is longer than the first duration can turn power to the computing device1200on or off. Pressing the button for a third duration can activate a voice control, or voice command, module that enables the user to speak commands into the microphone1230to cause the device to execute the spoken command. The user can customize a functionality of one or more of the buttons. The touch surface1246can, for example, also be used to implement virtual or soft buttons and/or a keyboard.

In some implementations, the computing device1200can present recorded audio and/or video files, such as MP3, AAC, and MPEG files. In some implementations, the computing device1200can include the functionality of an MP3 player, such as an iPod™.

The memory interface1202can be coupled to memory1250. The memory1250can include high-speed random-access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and/or flash memory (e.g., NAND, NOR). The memory1250can store an operating system1252, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks.

The operating system1252can include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, the operating system1252can be a kernel (e.g., UNIX kernel). In some implementations, the operating system1252can include instructions for performing voice authentication. For example, operating system1252can implement the screen recording features as described with reference toFIGS. 1-11.

The memory1250can also store communication instructions1254to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers. The memory1250can include graphical user interface instructions1256to facilitate graphic user interface processing; sensor processing instructions1258to facilitate sensor-related processing and functions; phone instructions1260to facilitate phone-related processes and functions; electronic messaging instructions1262to facilitate electronic-messaging related processes and functions; web browsing instructions1264to facilitate web browsing-related processes and functions; media processing instructions1266to facilitate media processing-related processes and functions; GNSS/Navigation instructions1268to facilitate GNSS and navigation-related processes and instructions; and/or camera instructions1270to facilitate camera-related processes and functions.

The memory1250can store software instructions1272to facilitate other processes and functions, such as the screen recording processes and functions as described with reference toFIGS. 1-11.

The memory1250can also store other software instructions1274, such as web video instructions to facilitate web video-related processes and functions; and/or web shopping instructions to facilitate web shopping-related processes and functions. In some implementations, the media processing instructions1266are divided into audio processing instructions and video processing instructions to facilitate audio processing-related processes and functions and video processing-related processes and functions, respectively.