SCREEN SHARING METHOD, RELATED ELECTRONIC DEVICE, AND SYSTEM

This application discloses a screen sharing method, a related electronic device, and a system, to support a first terminal and a second terminal to be in a multi-screen collaboration function, so that the second terminal shares screen interface content with a video call application on the first terminal, and when a third terminal has a video call with the first terminal, the third terminal may view, by using the video call application, the interface content shared by the second terminal.

TECHNICAL FIELD

This application relates to the field of communication technologies, and in particular, to a cross-device collaborative screen sharing method, a related electronic device, and a system.

BACKGROUND

Current video conferencing software has been able to support a plurality of participants to perform screen sharing. However, a screen sharing function depends on the conferencing software and a terminal joining a conference. In other words, the terminal needs to install the conferencing software, and join the conference based on a conference identity document (identity document, ID). The conferencing software needs to provide a screen sharing function based on a screen recording. Requirements of users who do not install or cannot install the conferencing software on their terminals but still want to join the conference and perform screen sharing are not met.

SUMMARY

This application provides a screen sharing method, a related electronic device, and a system, to support a first terminal and a second terminal to be in a multi-screen collaboration function, so that the second terminal shares screen interface content with a video call application on the first terminal, and when a third terminal has a video call with the first terminal, the third terminal may view, by using the video call application, the interface content shared by the second terminal.

The foregoing objective and another objective are achieved based on features in the independent claims. Further implementations are provided in the dependent claims, the specification, and the accompanying drawings.

According to a first aspect, an embodiment of this application provides a screen sharing method. The method may include: A first terminal establishes a first connection to a second terminal. The first terminal obtains a first media stream of the second terminal. The first media stream includes data of audio and a video generated by the second terminal, the audio includes a sound collected by a microphone of the second terminal or audio corresponding to a media file played by the second terminal, and the video includes a screen picture recorded by the second terminal. The first terminal generates a second media stream based on the first media stream. The second media stream includes the screen picture recorded by the second terminal. The first terminal sends the second media stream to a third terminal through a second connection. The third terminal plays the second media stream received from the first terminal. The first terminal plays a third media stream received from the third terminal. The third media stream includes a picture captured by a camera of the third terminal or a sound collected by a microphone of the third terminal.

According to the method in the first aspect, the second terminal may quickly share screen interface content with the first terminal and the third terminal. This step is simple, fast, and efficient, to further improve office efficiency. For example, the first terminal and the second terminal may be supported to be in a multi-screen collaboration function, so that the second terminal shares the screen interface content with a video call application on the first terminal, and when the third terminal has a video call with the first terminal, the third terminal may view, by using the video call application, the interface content shared by the second terminal.

Display content of a media stream (including an audio stream and a video stream) sent by the first terminal to the third terminal includes but is not limited to the following three cases:

1. The Display Content is Edited by the First Terminal.

After the first terminal receives a video stream captured by the second terminal, a user may perform some control operations on a first terminal side, to edit, set, or adjust the video stream, generate a new video stream, and send the new video stream to the third terminal. For example, the user may make some comments in a video picture of the first terminal. The comments and the original video picture are combined into a new video picture and sent to the third terminal. For another example, the user may set that a video picture sent by the first terminal to the third terminal not only includes a screen recording picture of the second terminal, but also includes a portrait that is of the user and that is captured by a camera of the first terminal. In this way, a third terminal side not only displays the screen recording picture from the second terminal, but also displays the portrait of the user of the first terminal. Therefore, the user has better video call experience. In addition, a picture parameter may be further adjusted in a video call screen of the first terminal. The picture parameter may include one or more of the following: an image cropping size, a color temperature of an image, a beauty/bodybuilding type, a filter type, a sticker option, and the like.

Therefore, the first terminal may adjust, based on an editing capability of the first terminal, an image captured by the second terminal, to generate a new display picture and send the new display picture to the third terminal. This brings use convenience to the user of the first terminal.

2. The Display Content is Edited by the Second Terminal.

After the second terminal generates a screen recording picture, a user may perform some control operations on a second terminal side, to edit, set, or adjust the video stream, generate a new video stream, and send the new video stream to the first terminal.

For example, the user may make some comments on the screen recording picture of the second terminal. The comments and the original screen recording picture are combined into a new video picture and sent to the first terminal. The first terminal sends the new video picture to the third terminal. For another example, the user may set that a video picture sent by the second terminal to the first terminal not only includes a screen recording picture of the second terminal, but also includes a portrait that is of the user and that is captured by a camera of the second terminal. In this way, a first terminal side and a third terminal side not only display the screen recording picture from the second terminal, but also display the portrait of the user of the second terminal. Therefore, the user has better video call experience. In addition, a picture parameter may be further adjusted in a user interface of the second terminal. The picture parameter may include one or more of the following: an image cropping size, a color temperature of an image, a beauty/bodybuilding type, a filter type, a sticker option, and the like.

Therefore, the second terminal may adjust, based on an editing capability of the second terminal, an image captured by the second terminal, to generate a new display picture and send the new display picture to the first terminal. This brings use convenience to the user of the second terminal.

3. The Display Content is Collaboratively Edited by the First Terminal and the Second Terminal.

With reference to the descriptions of the foregoing two aspects, in an implementation, the first terminal and the second terminal each may have a video picture editing function. In this way, the first terminal and the second terminal may collaboratively complete editing of the video picture, to be close to display effect that the user wants to achieve as much as possible. This brings convenience to users of both the first terminal and the second terminal, and avoids a case in which editing display effect cannot satisfy both users when the first terminal or the second terminal separately edits a display picture.

For example, after the second terminal edits a screen recording picture, the first terminal may edit the edited picture again. After the first terminal or the second terminal edits a screen recording picture, a display picture is synchronized to display interfaces of the first terminal and the second terminal.

Further, after the first terminal sends a display effect adjustment instruction to the second terminal, the second terminal may further perform compatibility processing on the instruction, so that the second terminal can effectively complete adjustment of corresponding display effect in response to the processed instruction. Because the first terminal and the second terminal may have different device types and device models, in response to a user operation that the user adjusts the display effect, the first terminal may generate some instructions that are not supported or cannot be identified by the second terminal, and the second terminal may not directly adjust the interface display effect in response to these instructions. The second terminal performs compatibility processing on the instruction, so that the second terminal can respond to an operation performed by the user on the first terminal, and the first terminal can control the second terminal to adjust the interface display effect.

It may be learned from the foregoing situation that, in addition to a case in which the second terminal may have a capability of editing a screen recording picture or another video stream, for a screen recording picture or another video stream from the second terminal, the first terminal may have a capability of editing the screen recording picture or the another video stream. This enhances effect of controlling the display interface of the second terminal by the first terminal, and improves use experience of the user.

In this embodiment of this application, that the first terminal displays an image of the second terminal may be that the first terminal transmits an image from the second terminal through a data stream channel collected by the camera of the first terminal. In other words, the first terminal originally transmits, through the data stream channel, data of an image captured by the camera of the first terminal. However, in a scenario of this embodiment of this application, the data stream channel is used to transmit a screen recording data stream or another video stream from the second terminal. Therefore, an upper-layer application of the first terminal does not need to perceive whether a displayed image comes from the first terminal or the second terminal. The video stream captured by the second terminal is equivalent to a video stream captured by a remote camera of the first terminal. The first terminal can send the obtained image of the second terminal to the upper-layer application, without a need to modify a manner in which a third-party application invokes a system interface, in other words, without requiring a third-party application to adapt, to expand a use scenario of screen sharing between the first terminal and the second terminal.

With reference to the first aspect, in an implementation, before the first terminal sends the second media stream to a third terminal through a second connection, the first terminal establishes the second connection to the third terminal.

With reference to the first aspect, in an implementation, before the first terminal establishes the first connection to the second terminal, the first terminal establishes the second connection to the third terminal. The first terminal plays the third media stream received from the third terminal. The third media stream includes the picture captured by the camera of the third terminal or the sound collected by the microphone of the third terminal. The third terminal plays a fourth media stream received from the first terminal. The fourth media stream includes a picture captured by a camera of the first terminal or a sound collected by a microphone of the first terminal.

With reference to the first aspect, in an implementation, before the first terminal obtains the first media stream of the second terminal, the first terminal detects a first user operation. The first user operation indicates the second terminal to start recording the screen picture.

With reference to the first aspect, in an implementation, the second media stream is the same as the first media stream.

With reference to the first aspect, in an implementation, the second media stream further includes the picture captured by the camera of the first terminal.

With reference to the first aspect, in an implementation, the second media stream further includes an editing trace of the first terminal in the screen picture recorded by the second terminal.

With reference to the first aspect, in an implementation, the second media stream further includes the sound collected by the microphone of the first terminal.

With reference to the first aspect, in an implementation, the screen picture recorded by the second terminal includes an entire screen picture recorded by the second terminal, a screen picture that corresponds to a partial area and that is recorded by the second terminal, or a screen picture that corresponds to a specified file and that is recorded by the second terminal.

With reference to the first aspect, in an implementation, the first terminal transmits a video stream included in the second media stream to an image stream channel of an image captured by the camera of the first terminal; the first terminal transmits an audio stream included in the second media stream to an audio stream channel of the sound collected by the microphone of the first terminal; and the first terminal sends a media stream in the image stream channel and the audio stream channel to the third terminal through the second connection.

With reference to the first aspect, in an implementation, after the first terminal establishes the first connection to the second terminal, the first terminal negotiates a first parameter with the second terminal. The first parameter includes one or more of the following: a bitrate of a screen recording or a sound recording, a resolution of the screen recording, a sound source of the sound recording, a cache path, an audio or video encoding/decoding manner, a specified screen recording area, a specified recording file, and a video window rotation policy, and the sound source of the sound recording includes audio corresponding to a played media file of the second terminal or audio collected by the microphone of the second terminal.

With reference to the first aspect, in an implementation, a first user interface of the first terminal includes a first window and a second window, and a second user interface of the third terminal includes a third window and a fourth window.

With reference to the first aspect, in an implementation, after the first terminal sends the second media stream to the third terminal through the second connection, a picture displayed in the first window is the picture captured by the camera of the third terminal, a picture displayed in the second window is the screen picture recorded by the second terminal, a picture displayed in the third window is the screen picture recorded by the second terminal, and a picture displayed in the fourth window is the picture captured by the camera of the third terminal.

With reference to the first aspect, in an implementation, after the first terminal sends the second media stream to the third terminal through the second connection, a picture displayed in the first window is the screen picture recorded by the second terminal, a picture displayed in the second window is the picture captured by the camera of the third terminal, a picture displayed in the third window is the screen picture recorded by the second terminal, and a picture displayed in the fourth window is the picture captured by the camera of the third terminal.

With reference to the first aspect, in an implementation, the first user interface of the first terminal further includes a fifth window, and the second user interface of the third terminal further includes a sixth window. A picture displayed in the fifth window is the picture captured by the camera of the first terminal, and a picture displayed in the sixth window is the picture captured by the camera of the first terminal.

With reference to the first aspect, in an implementation, when the first terminal plays the third media stream received from the third terminal, the picture displayed in the first window is the picture captured by the camera of the third terminal, and the picture displayed in the second window is the picture captured by the camera of the first terminal. When the third terminal plays the fourth media stream received from the first terminal, the picture displayed in the third window is the picture captured by the camera of the first terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

According to a second aspect, this application provides a screen sharing method. The method may include: A first terminal establishes a first connection to a second terminal. The first terminal obtains a first media stream of the second terminal. The first media stream includes data of audio and a video generated by the second terminal, the audio includes a sound collected by a microphone of the second terminal or audio corresponding to a media file played by the second terminal, and the video includes a screen picture recorded by the second terminal. The first terminal generates a second media stream based on the first media stream. The second media stream includes the screen picture recorded by the second terminal. The first terminal sends the second media stream to a third terminal through a second connection. The screen picture that is recorded by the second terminal and that is included in the second media stream is displayed on a screen of the third terminal. The first terminal plays a third media stream received from the third terminal. The third media stream includes a picture captured by a camera of the third terminal or a sound collected by a microphone of the third terminal.

According to the method in the second aspect, the second terminal may quickly share screen interface content with the first terminal and the third terminal. This step is simple, fast, and efficient, to further improve office efficiency. For example, the first terminal and the second terminal may be supported to be in a multi-screen collaboration function, so that the second terminal shares the screen interface content with a video call application on the first terminal, and when the third terminal has a video call with the first terminal, the third terminal may view, by using the video call application, the interface content shared by the second terminal.

With reference to the second aspect, in an implementation, before the first terminal sends the second media stream to a third terminal through a second connection, the first terminal establishes the second connection to the third terminal.

With reference to the second aspect, in an implementation, before the first terminal establishes the first connection to the second terminal, the first terminal establishes the second connection to the third terminal. The first terminal sends a fourth media stream to the third terminal through the second connection. The fourth media stream includes a picture captured by a camera of the first terminal or a sound collected by a microphone of the first terminal, and the picture captured by the camera of the first terminal is displayed on the screen of the third terminal. The first terminal receives, through the second connection, the third media stream sent by the third terminal. The first terminal plays the third media stream received from the third terminal. The third media stream includes the picture captured by the camera of the third terminal or the sound collected by the microphone of the third terminal.

With reference to the second aspect, in an implementation, before the first terminal obtains the first media stream of the second terminal, the first terminal detects a first user operation. The first user operation indicates the second terminal to start recording the screen picture.

With reference to the second aspect, in an implementation, the second media stream is the same as the first media stream.

With reference to the second aspect, in an implementation, the second media stream further includes the picture captured by the camera of the first terminal.

With reference to the second aspect, in an implementation, the second media stream further includes an editing trace of the first terminal in the screen picture recorded by the second terminal.

With reference to the second aspect, in an implementation, the second media stream further includes the sound collected by the microphone of the first terminal.

With reference to the second aspect, in an implementation, the screen picture recorded by the second terminal includes an entire screen picture recorded by the second terminal, a screen picture that corresponds to a partial area and that is recorded by the second terminal, or a screen picture that corresponds to a specified file and that is recorded by the second terminal.

With reference to the second aspect, in an implementation, the first terminal transmit a video stream included in the second media stream to an image stream channel of an image captured by the camera of the first terminal. The first terminal transmits an audio stream included in the second media stream to an audio stream channel of the sound collected by the microphone of the first terminal. The first terminal sends a media stream in the image stream channel and the audio stream channel to the third terminal through the second connection.

With reference to the second aspect, in an implementation, after the first terminal establishes the first connection to the second terminal, the first terminal negotiates a first parameter with the second terminal. The first parameter includes one or more of the following: a bitrate of a screen recording or a sound recording, a resolution of the screen recording, a sound source of the sound recording, a cache path, an audio or video encoding/decoding manner, a specified screen recording area, a specified recording file, and a video window rotation policy, and the sound source of the sound recording includes audio corresponding to a played media file of the second terminal or audio collected by the microphone of the second terminal.

With reference to the second aspect, in an implementation, a first user interface of the first terminal includes a first window and a second window, and a second user interface of the third terminal includes a third window and a fourth window.

With reference to the second aspect, in an implementation, after the first terminal sends the second media stream to the third terminal through the second connection, a picture displayed in the first window is the picture captured by the camera of the third terminal, a picture displayed in the second window is the screen picture recorded by the second terminal, a picture displayed in the third window is the screen picture recorded by the second terminal, and a picture displayed in the fourth window is the picture captured by the camera of the third terminal.

With reference to the second aspect, in an implementation, after the first terminal sends the second media stream to the third terminal through the second connection, a picture displayed in the first window is the screen picture recorded by the second terminal, a picture displayed in the second window is the picture captured by the camera of the third terminal, a picture displayed in the third window is the screen picture recorded by the second terminal, and a picture displayed in the fourth window is the picture captured by the camera of the third terminal.

With reference to the second aspect, in an implementation, the first user interface of the first terminal further includes a fifth window, and the second user interface of the third terminal further includes a sixth window. A picture displayed in the fifth window is the picture captured by the camera of the first terminal, and a picture displayed in the sixth window is the picture captured by the camera of the first terminal.

With reference to the second aspect, in an implementation, after the first terminal sends the fourth media stream to the third terminal through the second connection, the picture displayed in the third window is the picture captured by the camera of the first terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal. After the first terminal receives, through the second connection, the third media stream sent by the third terminal, the picture displayed in the first window is the picture captured by the camera of the third terminal, and the picture displayed in the second window is the picture captured by the camera of the first terminal.

According to a third aspect, this application provides an electronic device. The electronic device may include a communication apparatus, a display, a memory, a processor coupled to the memory, a plurality of applications, and one or more programs, the display is configured to display a picture, the memory stores computer-executable instructions, and when the processor executes the instructions, the electronic device is enabled to implement any function of the first terminal in the second aspect.

According to a fourth aspect, this application provides a communication system. The communication system includes a first terminal, a second terminal, and a third terminal, and the first terminal may implement any function of the first terminal in the second aspect.

According to a fifth aspect, an embodiment of this application provides a computer storage medium. The storage medium stores a computer program, the computer program includes executable instructions, and when the executable instructions are executed by a processor, the processor is enabled to perform an operation corresponding to the method provided in the second aspect.

According to a sixth aspect, an embodiment of this application provides a computer program product. When the computer program product runs on an electronic device, the electronic device is enabled to perform any one of the possible implementations of the second aspect.

According to a seventh aspect, an embodiment of this application provides a chip system. The chip system may be applied to an electronic device, the chip includes one or more processors, and the processor is configured to invoke computer instructions, so that the electronic device implements any one of the possible implementations of the second aspect.

According to the method provided in this application, the second terminal may quickly share the screen interface content with the first terminal and the third terminal. This step is simple, fast, and efficient, to further improve office efficiency.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The following clearly describes the technical solutions in embodiments of this application in detail with reference to the accompanying drawings. In the descriptions of embodiments of this application, “/” means “or” unless otherwise specified. For example, A/B may represent A or B. In this specification, “and/or” describes only an association relationship for describing associated objects and represents that three relationships may exist. For example, A and/or B may represent the following three cases: Only A exists, both A and B exist, and only B exists.

A term “user interface (user interface, UI)” in the following embodiments of this application is a medium interface for interaction and information exchange between a user and an application (application, APP) or an operating system (operating system, OS), and implements conversion between an internal form of information and a form acceptable to the user. The user interface is source code written in a specific computer language such as java or an extensible markup language (extensible markup language, XML). The source code of the interface is parsed and rendered on an electronic device, and finally is presented as user-recognizable content. A common representation form of the user interface is a graphical user interface (graphical user interface, GUI), and is a user interface that is displayed in a graphical manner and that is related to a computer operation. The user interface may be visual interface element such as a text, an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, or a widget displayed on a display of the electronic device.

In a current mainstream multi-person video conference scenario, a terminal needs to install conferencing software, and joins the conference by using a conference ID, to join a video conference. When the terminal joins the video conference, the conferencing software provides a screen sharing function based on a screen recording, so that attendees view conference content in a sharing manner.

In a video conference scenario shown inFIG.1, both a mobile phone A and a mobile phone B install conferencing software A, and may enter a conference by logging in to a same conference ID.

InFIG.1, the mobile phone A may share a home screen or a file of the mobile phone A based on a screen recording function, and then a screen recording video stream of the mobile phone A may be transmitted to the mobile phone B through the Internet. The mobile phone B may play the received screen recording video stream of the mobile phone A. In this way, the mobile phone A, the mobile phone B, and another terminal connected to join a conference may perform a video conference, to perform conference communication instantly and online.

Due to factors such as a development capability and a development progress, some conferencing software do not support all types of terminals or all types of operating systems.

For example, the conferencing software A supports only a mobile phone client, and does not support a personal computer (personal computer, PC) client. Therefore, the computer A cannot install the conferencing software A, and cannot join a conference to participate in screen sharing. If a conference demonstration material is stored on a computer A, but the computer A cannot join the conference to participate in screen sharing, the computer A needs to send the conference demonstration material to a terminal that may participate in a screen sharing function, for example, the mobile phone A. This is very inconvenient for a user.

For a terminal that does not install or cannot install conferencing software, if the terminal wants to join the conference to participate in screen sharing, a browser application may be used. The user may enter a conference link and a login account corresponding to the conference ID in the browser application, to enter the conference. A process is cumbersome. Essence of joining the conference by using the conferencing software or the browser application is the same, and is performing the video conference by using a medium that may carry a conferencing application on the terminal.

For another example, two persons are discussing work by using video call software (for example, MeeTime or WeChat), and need to demonstrate a material on a PC. A solution may be to send a material to a peer end on a platform such as Email or communication software, or reschedule a network conference, to wait both parties to join the conference and share the material. However, the solutions have cumbersome steps and low sharing efficiency for the user.

This application provides a screen sharing method. The method may be applied to the related electronic device and system provided in this application. The electronic device may include terminal devices such as a mobile phone, a tablet device (portable Android device, PAD), a personal computer (personal computer, PC), an intelligent television, and an electronic conference whiteboard. A specific type of the electronic device is not limited in this application.

A multi-screen collaboration function is a distributed communication technology. The multi-screen collaboration function may implement collaborative work across operating systems or across devices. For example, after the mobile phone is connected to the PC, the mobile phone and the PC may share a resource and perform a collaborative operation, to improve event processing efficiency.

In the screen sharing method provided in this application, a first terminal and a second terminal may be supported to be in the multi-screen collaboration function, so that the second terminal shares screen interface content (for example, a screen recording picture) with a video call app on the first terminal, and when a third terminal has a video call with the first terminal, the third terminal may view, by using the video call app, the interface content shared by the second terminal, without a need to use a conferencing application.

Display content of a media stream (including an audio stream and a video stream) sent by the first terminal to the third terminal includes but is not limited to the following three cases:

1. The Display Content is Edited by the First Terminal.

After the first terminal receives a video stream captured by the second terminal, a user may perform some control operations on a first terminal side, to edit, set, or adjust the video stream, generate a new video stream, and send the new video stream to the third terminal.

For example, the user may make some comments in a video picture of the first terminal. The comments and the original video picture are combined into a new video picture and sent to the third terminal.

For another example, the user may set that a video picture sent by the first terminal to the third terminal not only includes a screen recording picture of the second terminal, but also includes a portrait that is of the user and that is captured by a camera of the first terminal.

In this way, a third terminal side not only displays the screen recording picture from the second terminal, but also displays the portrait of the user of the first terminal. Therefore, the user has better video call experience.

In addition, a picture parameter may be further adjusted in a video call screen of the first terminal. The picture parameter may include one or more of the following: an image cropping size, a color temperature of an image, a beauty/bodybuilding type, a filter type, a sticker option, and the like. The image cropping size may be 3:3, 3:4, 9:16, or the like. A color temperature calibration mode of the image may be a sunlight, fluorescence, incandescent bulb, shadow, or cloudy day calibration mode, or the like. The beauty/bodybuilding type may be Head, Slim, Smooth, White, Eye, Acne, or the like. The filter type may be Japanese, Texture, Bright, Soft, Cyberpunk, or the like. A sticker may be Expression, Animal, Landscape, Illustration, or the like.

Therefore, the first terminal may adjust, based on an editing capability of the first terminal, an image captured by the second terminal, to generate a new display picture and send the new display picture to the third terminal. This brings use convenience to the user of the first terminal.

2. The Display Content is Edited by the Second Terminal.

After the second terminal generates a screen recording picture, a user may perform some control operations on a second terminal side, to edit, set, or adjust the video stream, generate a new video stream, and send the new video stream to the first terminal.

For example, the user may make some comments on the screen recording picture of the second terminal. The comments and the original screen recording picture are combined into a new video picture and sent to the first terminal. The first terminal sends the new video picture to the third terminal.

For another example, the user may set that a video picture sent by the second terminal to the first terminal not only includes a screen recording picture of the second terminal, but also includes a portrait that is of the user and that is captured by a camera of the second terminal. In this way, a first terminal side and a third terminal side not only display the screen recording picture from the second terminal, but also display the portrait of the user of the second terminal. Therefore, the user has better video call experience.

In addition, a picture parameter may be further adjusted in a user interface of the second terminal. The picture parameter may include one or more of the following: an image cropping size, a color temperature of an image, a beauty/bodybuilding type, a filter type, a sticker option, and the like.

Therefore, the second terminal may adjust, based on an editing capability of the second terminal, an image captured by the second terminal, to generate a new display picture and send the new display picture to the first terminal. This brings use convenience to the user of the second terminal.

3. The Display Content is Collaboratively Edited by the First Terminal and the Second Terminal.

With reference to the descriptions of the foregoing two aspects, in an implementation, the first terminal and the second terminal each may have a video picture editing function. In this way, the first terminal and the second terminal may collaboratively complete editing of the video picture, to be close to display effect that the user wants to achieve as much as possible. This brings convenience to users of both the first terminal and the second terminal, and avoids a case in which editing display effect cannot satisfy both users when the first terminal or the second terminal separately edits a display picture.

For example, after the second terminal edits a screen recording picture, the first terminal may edit the edited picture again. After the first terminal or the second terminal edits a screen recording picture, a display picture is synchronized to display interfaces of the first terminal and the second terminal.

Further, after the first terminal sends a display effect adjustment instruction to the second terminal, the second terminal may further perform compatibility processing on the instruction, so that the second terminal can effectively complete adjustment of corresponding display effect in response to the processed instruction.

Because the first terminal and the second terminal may have different device types and device models, in response to a user operation that the user adjusts the display effect, the first terminal may generate some instructions that are not supported or cannot be identified by the second terminal, and the second terminal may not directly adjust the interface display effect in response to these instructions.

The second terminal performs compatibility processing on the instruction, so that the second terminal can respond to an operation performed by the user on the first terminal, and the first terminal can control the second terminal to adjust the interface display effect.

It may be learned from the foregoing situation that, in addition to a case in which the second terminal may have a capability of editing a screen recording picture or another video stream, for a screen recording picture or another video stream from the second terminal, the first terminal may have a capability of editing the screen recording picture or the another video stream. This enhances effect of controlling the display interface of the second terminal by the first terminal, and improves use experience of the user.

In this embodiment of this application, that the first terminal displays an image of the second terminal may be that the first terminal transmits an image from the second terminal through a data stream channel collected by the camera of the first terminal. In other words, the first terminal originally transmits, through the data stream channel, data of an image captured by the camera of the first terminal. However, in a scenario of this embodiment of this application, the data stream channel is used to transmit a screen recording data stream or another video stream from the second terminal. Therefore, an upper-layer application of the first terminal does not need to perceive whether a displayed image comes from the first terminal or the second terminal. The video stream captured by the second terminal is equivalent to a video stream captured by a remote camera of the first terminal. The first terminal can send the obtained image of the second terminal to the upper-layer application, without a need to modify a manner in which a third-party application invokes a system interface, in other words, without requiring a third-party application to adapt, to expand a use scenario of screen sharing between the first terminal and the second terminal.

According to the method provided in this application, the second terminal may quickly share the screen interface content with the first terminal and the third terminal. This step is simple, fast, and efficient, to further improve office efficiency.

For example, in the mainstream multi-person video conference scenario, participation in the video conference depends on installation of the conferencing software on the terminal, and the conferencing software further needs to provide a screen sharing function based on a screen recording.

In the technical solution provided in this application, assuming that the mobile phone A and the mobile phone B are in a conference by using video conferencing software or the video call software, a conference material that needs to be demonstrated is on the computer A. Therefore, the computer A does not need to install the video conferencing software or the video call software, and the mobile phone A does not need to support the screen sharing function based on a screen recording. When the mobile phone A is connected to the computer A, based on the multi-screen collaboration function, the computer A may inject demonstration content of the screen recording into a camera of the mobile phone A, and the mobile phone B views, by using a received video stream of a camera of the mobile phone A, the conference material from screen sharing of the computer A.

In the technical solution provided in this application, the user can implement the cross-device screen sharing function on an app that supports only a video call, and it is unnecessary for the terminal to install the conferencing software.

First, a communication system provided in an embodiment of this application is described.

FIG.2shows a communication system10according to an embodiment of this application.

The communication system10may include a plurality of intelligent terminal devices, and a communication connection is established between the plurality of terminal devices. The terminal device may be a smartphone, a notebook computer, a home screen computer, a tablet computer, an electronic conference whiteboard, a smart screen (intelligent television), or another type of electronic device, for example, may further include a home screen computer, a laptop computer, a handheld computer, an augmented reality (augmented reality, AR) device, a virtual reality (virtual reality, VR) device, an artificial intelligence (artificial intelligence, AI) device, a vehicle, a game console, or another intelligent wearable device. This is not limited in this application. In this embodiment, the terminal device may also be briefly referred to as a terminal. The terminal device is usually an intelligent electronic device that may provide a user interface, may interact with a user, and provides a service function for the user.

Each terminal device in the communication system10may be provided with an Android® system, a Windows® system, iOS®, macOS®, a Linux® system, a HarmonyOS® system (HarmonyOS, HOS), or another type of operating system. Operating systems of terminal devices in the communication system10may be the same or different. This is not limited in this application. In some embodiments, a plurality of terminals are all provided with the HarmonyOS® system. A system including the plurality of terminals may be referred to as a HarmonyOS® super virtual terminal (super virtual device), or may be referred to as a HarmonyOS® super terminal. In the system, the capabilities of the plurality of terminals are integrated by using a distributed technology, and are stored in a virtual hardware resource pool. The capabilities of the terminals are uniformly managed, scheduled, and integrated based on a service requirement, to externally provide a service, so that a fast connection, capability complementation, and resource sharing are implemented between different terminals.

As shown inFIG.2, the communication system10includes a first terminal, a second terminal, and a third terminal. Each terminal may be provided with a camera.

A first connection is established between the first terminal and the second terminal, and a second connection is established between the first terminal and the third terminal. In this embodiment of this application, a type of the first connection or a type of the second connection is not limited. The first connection or the second connection may be a wireless connection, or may be a wired connection.

In some embodiments, the second connection may be an Internet connection, and the first terminal and the third terminal implement a connection and communication through the Internet, for example, perform a video call. The first terminal and the second terminal may access the Internet through a wireless communication network (for example, a wireless fidelity (wireless fidelity, Wi-Fi) network) or a mobile communication network (for example, a mobile communication 2G/3G/4G/5G network).

The second terminal may record a home screen, and then send a recorded video stream to the first terminal through the first connection. The first connection may include a wired connection, for example, a universal serial bus (universal serial bus, USB) connection; or a wireless connection, for example, a Bluetooth (Bluetooth, BT) connection, a wireless fidelity (wireless fidelity, Wi-Fi) connection, or a hotspot connection, to implement communication between the first terminal and the second terminal in a case of a same account, no account, or different accounts.

In some embodiments, the plurality of terminals may log in to a same account, to implement a connection and communication through the Internet. The plurality of terminals may alternatively log in to different accounts, but are connected in a binding manner. For example, the first terminal and the second terminal may log in to different accounts. In a device management application, the first terminal binds the second terminal to the first terminal, and then is connected to the second terminal by using the device management application.

A type of the communication connection is not limited in this embodiment of this application. The terminals may transmit and exchange data based on various communication connection types. In addition, the terminals may alternatively perform a connection and communication with each other in any several manners in the foregoing manners. This is not limited in this embodiment of this application.

The terminal device may be provided with a mobile communication module and a wireless communication module.

The mobile communication module may provide a wireless communication solution that is applied to the terminal and that includes 2G/3G/4G/5G.

The wireless communication module may include a Bluetooth (Bluetooth, BT) module and/or a wireless local area network (wireless local area network, WLAN) module.

The Bluetooth module may provide one or more Bluetooth communication solutions including classic Bluetooth (Bluetooth 2.1) or Bluetooth low energy (Bluetooth low energy, BLE), and the WLAN module may provide one or more WLAN communication solutions including wireless fidelity peer-to-peer (wireless fidelity peer-to-peer, Wi-Fi P2P), a wireless fidelity local area network (wireless fidelity local area network, Wi-Fi LAN), or a wireless fidelity software access point (wireless fidelity software access point, Wi-Fi softAP).

In some embodiments, Wi-Fi P2P is that devices in a wireless network are allowed to be connected to each other in a peer-to-peer manner without a need to use a wireless router, and may also be referred to as wireless fidelity direct (Wi-Fi direct) in the Android® system. Devices between which a Wi-Fi P2P connection is established may directly exchange data through Wi-Fi (which needs to be in a same frequency band) without connecting to a network or a hotspot, to implement peer-to-peer communication, for example, transmit data such as a file, a picture, and a video. Compared with Bluetooth, Wi-Fi P2P has advantages such as faster search speed, a faster transmission speed, and a farther transmission distance.

The first terminal may replace the recorded video stream from the second terminal with a photographed video stream from a camera of the first terminal. In this way, when the first terminal performs a video call with the third terminal, an interface of the third terminal displays the recorded video stream from the second terminal rather than a picture photographed by the camera of the first terminal. In this way, the second terminal can share a screen and participate in a video conference, without a cumbersome step.

It should be noted that the communication system10shown inFIG.2is merely used to assist in describing the technical solutions provided in embodiments of this application, and constitutes no limitation in embodiments of this application. In an actual service scenario, the communication system10may include more or fewer terminal devices. A quantity of terminals, a connection manner, and the like are not limited in this application.

The following provides example descriptions of a structure of the electronic device provided in this embodiment of this application.

The electronic device provided in this embodiment of this application may be any electronic device in the communication system10shown inFIG.2, and may be the first terminal, the second terminal, the third terminal, or another electronic device described in this embodiment of this application. This is not limited herein. An electronic device100is used as an example herein for description.

In this embodiment of this application, the electronic device100may include but is not limited to a mobile phone, a tablet computer, a smart screen (smart television), an electronic conference whiteboard, a home screen computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR) device, a virtual reality (virtual reality, VR) device, an artificial intelligence (artificial intelligence, AI) device, a wearable device, a vehicle-mounted device, a smart home device, and/or a smart city device. A specific type of the electronic device100is not limited in this embodiment of this application.

FIG.3is a schematic diagram of a hardware structure of an electronic device100according to an embodiment of this application.

The controller may generate an operation control signal based on an instruction operation code and a time sequence signal, to complete control of instruction reading and instruction execution.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be configured to connect the processor110and the camera193, the display194, the wireless communication module160, the audio module170, the sensor module180, or the like. The GPIO interface may alternatively be configured as an I2C interface, an I2S interface, a UART interface, an MIPI interface, or the like.

The USB port130is a port that conforms to a USB standard specification, and may be specifically a mini USB port, a micro USB port, a USB Type C port, or the like. The USB port130may be configured to be connected to the charger to charge the electronic device100, or may be configured to transmit data between the electronic device100and a peripheral device, or may be configured to be connected to a headset, to play audio by using the headset. The interface may be further configured to be connected to another electronic device, for example, an AR device.

It can be understood that an interface connection relationship between modules illustrated in this embodiment of the present invention is merely an example for description, and does not constitute a limitation on the structure of the electronic device100. In some other embodiments of this application, different interface connection manners in the foregoing embodiments or a combination of a plurality of interface connection manners may alternatively be used for the electronic device100.

A wireless communication function of the electronic device100may be implemented through the antenna1, the antenna2, the mobile communication module150, the wireless communication module160, the modem processor, the baseband processor, and the like.

The electronic device100may implement a photographing function by using the ISP, the camera193, the video codec, the GPU, the display194, the application processor, and the like.

The NPU is a neural-network (neural-network, NN) computing processor. The NPU quickly processes input information with reference to a structure of a biological neural network, for example, a transfer mode between human brain neurons, and may further continuously perform self-learning. The NPU may be used to implement applications such as intelligent cognition of the electronic device100, for example, image recognition, facial recognition, voice recognition, and text understanding.

The internal memory121may include one or more random access memories (random access memory, RAM) and one or more nonvolatile memories (nonvolatile memory, NVM).

The random access memory has characteristics such as a fast read/write speed and a volatile characteristic. The volatile characteristic means that once power is cut off, data stored in the RAM disappears. Usually, static power consumption of the random access memory is very low, and running power consumption is large. Data in the RAM is memory data, and may be read at any time. Data disappears once power is cut off.

The nonvolatile memory has a volatile characteristic and stable storage data. A nonvolatile characteristic means that data stored in the nonvolatile memory does not disappear after power is cut off, and the nonvolatile memory may store data for a long period of time after power is cut off. A data packet in the NVM includes application data, and may be stably stored in the NVM for a long period of time. The application data is content written in a running process of an application or a service process, for example, a photo or a video obtained by a photographing application or a text edited by a user in a document application.

The random access memory may include a static random access memory (static random access memory, SRAM), a dynamic random access memory (dynamic random access memory, DRAM), a synchronous dynamic random access memory (synchronous dynamic random access memory, SDRAM), a double data rate synchronous dynamic random access memory (double data rate synchronous dynamic random access memory, DDR SDRAM, where for example, a 5th generation DDR SDRAM is usually referred to as a DDR5 SDRAM), and the like.

The nonvolatile memory may include a magnetic disk storage (magnetic disk storage) device, a flash memory (flash memory), and the like.

The magnetic disk storage device is a memory in which a disk serves as a storage medium, and is featured by a large storage capacity, a high data transmission rate, and long-term storage of stored data.

The flash memory may include a NOR flash, a NAND flash, a 3D NAND flash, and the like based on an operating principle; may include a single-level cell (single-level cell, SLC), a multi-level cell (multi-level cell, MLC), a triple-level cell (triple-level cell, TLC), a quad-level cell (quad-level cell, QLC), and the like based on an order of a level of a storage unit; and may include a universal flash storage (English: universal flash storage, UFS), an embedded multimedia card (embedded multimedia Card, eMMC), and the like based on a storage specification.

The random access memory may be directly read and written by the processor110, and may be configured to store an executable program (for example, a machine instruction) of an operating system or another running program, and may be further configured to store data of a user and data of an application.

The nonvolatile memory may store an executable program, data of a user and an application, and the like, and may be loaded into the random access memory in advance, so that the processor110directly performs reading and writing.

The external memory interface120may be configured to be connected to an external nonvolatile memory, to expand a storage capacity of the electronic device100. The external nonvolatile memory communicates with the processor110through the external memory interface120, to implement a data storage function. For example, a file such as music or a video is stored in the external nonvolatile memory.

The electronic device100may implement an audio function, for example, music playing or sound recording, by using the audio module170, the speaker170A, the receiver170B, the microphone170C, the headset jack170D, the application processor, and the like.

The audio module170is configured to convert digital audio information into an analog audio signal for output, and is also configured to convert analog audio input into a digital audio signal. The audio module170may be further configured to code and decode an audio signal. In some embodiments, the audio module170may be disposed in the processor110, or some functional modules in the audio module170are disposed in the processor110.

The speaker170A, also referred to as a “horn”, is configured to convert an audio electrical signal into a sound signal. The electronic device100may be configured to listen to music or answer a call in a hands-free mode by using the speaker170A.

The pressure sensor180A is configured to sense a pressure signal, and may convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor180A may be disposed on the display194. There are a plurality of types of pressure sensors180A such as a resistive pressure sensor, an inductive pressure sensor, and a capacitive pressure sensor. The capacitive pressure sensor may include at least two parallel plates made of conductive materials. When force is applied to the pressure sensor180A, capacitance between electrodes changes. The electronic device100determines pressure intensity based on the change in the capacitance. When a touch operation is performed on the display194, the electronic device100detects intensity of the touch operation through the pressure sensor180A. The electronic device100may also calculate a touch location based on a detection signal of the pressure sensor180A. In some embodiments, touch operations that are performed at a same touch location but have different touch operation intensity may correspond to different operation instructions. For example, when a touch operation whose touch operation intensity is less than a first pressure threshold is performed a Messaging application icon, an instruction for viewing an SMS message is executed. When a touch operation whose touch operation intensity is greater than or equal to the first pressure threshold is performed on the Messaging application icon, an instruction for newly creating an SMS message is executed.

The gyro sensor180B may be configured to determine a moving posture of the electronic device100. In some embodiments, an angular velocity of the electronic device100around three axes (that is, axes x, y, and z) may be determined by using the gyro sensor180B. The gyro sensor180B may be configured to implement image stabilization during photographing. For example, when the shutter is pressed, the gyro sensor180B detects an angle at which the electronic device100jitters, calculates, based on the angle, a distance for which a lens module needs to compensate, and allows the lens to cancel the jitter of the electronic device100through reverse motion, to implement image stabilization. The gyro sensor180B may be further used in a navigation scenario and a motion sensing game scenario.

The barometric pressure sensor180C is configured to measure barometric pressure. In some embodiments, the electronic device100calculates an altitude by using a barometric pressure value measured by the barometric pressure sensor180C, to assist in positioning and navigation.

The magnetic sensor180D includes a Hall sensor. The electronic device100may detect opening and closing of a flip leather case by using the magnetic sensor180D. In some embodiments, when the electronic device100is a flip phone, the electronic device100may detect opening and closing of a flip cover based on the magnetic sensor180D, and Further, a feature such as automatic unlocking upon opening of the flip leather case is set based on a detected opening or closing state of the flip leather case.

The acceleration sensor180E may detect accelerations in various directions (usually on three axes) of the electronic device100. When the electronic device100is still, a magnitude and a direction of gravity may be detected. The acceleration sensor180E may be further configured to recognize a posture of the electronic device, and is applied to applications such as switching between a landscape mode and a portrait mode and a pedometer.

The distance sensor180F is configured to measure a distance. The electronic device100may measure the distance in an infrared manner or a laser manner. In some embodiments, in a photographing scenario, the electronic device100may measure a distance by using the distance sensor180F, to implement quick focusing.

The optical proximity sensor180G may include, for example, a light emitting diode (LED) and an optical detector, for example, a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device100emits infrared light by using the light-emitting diode. The electronic device100detects infrared reflected light from a nearby object by using the photodiode. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device100. When insufficient reflected light is detected, the electronic device100may determine that there is no object near the electronic device100. The electronic device100may detect, by using the optical proximity sensor180G, that the user holds the electronic device100close to an ear to make a call, to automatically perform screen-off for power saving. The optical proximity sensor180G may also be used in a smart cover mode or a pocket mode to automatically perform screen unlocking or locking.

The fingerprint sensor180H is configured to collect a fingerprint. The electronic device100may use a feature of the collected fingerprint to implement fingerprint-based unlocking, application lock access, fingerprint-based photographing, fingerprint-based call answering, and the like.

The temperature sensor180J is configured to detect a temperature. In some embodiments, the electronic device100executes a temperature processing policy based on the temperature detected by the temperature sensor180J. For example, when the temperature reported by the temperature sensor180J exceeds a threshold, the electronic device100lowers performance of a processor near the temperature sensor180J, to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is less than another threshold, the electronic device100heats the battery142to prevent the electronic device100from being shut down abnormally due to a low temperature. In some other embodiments, when the temperature is less than still another threshold, the electronic device100boosts an output voltage of the battery142to avoid abnormal shutdown due to a low temperature.

The touch sensor180K is also referred to as a “touch device”. The touch sensor180K may be disposed on the display194. The touch sensor180K and the display194form a touchscreen, which is also referred to as a “touchscreen”. The touch sensor180K is configured to detect a touch operation performed on or near the touch sensor180K. The touch sensor may transfer the detected touch operation to the application processor, to determine a touch event type. The display194may be configured to provide a visual output related to the touch operation. In some other embodiments, the touch sensor180K may also be disposed on a surface of the electronic device100at a location different from that of the display194.

The SIM card interface195is configured to be connected to a SIM card. The SIM card may be inserted into the SIM card interface195or removed from the SIM card interface195, to implement contact with or separation from the electronic device100. The electronic device100may support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface195may support a nano-SIM card, a micro-SIM card, a SIM card, and the like. A plurality of cards may be inserted into a same SIM card interface195together. The plurality of cards may be of a same type or different types. The SIM card interface195is compatible with different types of SIM cards. The SIM card interface195may also be compatible with an external memory card. The electronic device100interacts with a network by using the SIM card, to implement functions such as conversation and data communication. In some embodiments, an eSIM, namely, an embedded SIM card, is used for the electronic device100. The eSIM card may be embedded into the electronic device100, and cannot be separated from the electronic device100.

A screen sharing method provided in embodiments of this application may be applied to a plurality of service scenarios, including but not limited to:

(1) Video Conference Scenario

For example, a first terminal and a third terminal perform a video conference by using conferencing software. A conference material that needs to be demonstrated by the first terminal is on a second terminal. The first terminal may be connected to the second terminal. The second terminal sends, to the first terminal, a screen video stream recorded by the second terminal, that is, the conference material that needs to be demonstrated, and then the first terminal sends the demonstrated conference material to the third terminal. In this way, when the third terminal does not establish a connection to the second terminal, and the second terminal does not access the video conference by using the conferencing software, the third terminal may display demonstration content recorded by the second terminal.

(2) Video Call Scenario

For example, a first terminal and a third terminal are performing video chatting by using video call software. A topic is discussed in a chatting process, and a material related to the topic is stored on the second terminal, for example, a working document and a child photo.

The first terminal may be connected to the second terminal, to facilitate display to a peer party, so that the peer party views the material. The second terminal sends, to the first terminal, the screen video stream recorded by the second terminal, that is, a picture or a document that needs to be displayed, and then the first terminal sends, to the third terminal, the picture or the document that needs to be displayed.

In this way, when the third terminal does not establish a connection to the second terminal, the third terminal may display content recorded by the second terminal. When the first terminal keeps connected to the third terminal, the first terminal may quickly and conveniently display content of the second terminal to the third terminal, without a cumbersome operation, to bring a user more convenient video call experience.

In the live scenario, a first terminal is connected to a second terminal, and the second terminal stores some demonstration materials. A streamer may select, as current livestreaming content, an image photographed by the first terminal, or switch the current livestreaming content to a demonstration material on the second terminal. The second terminal may send a video stream to the first terminal, and the first terminal may upload the video stream to a livestreaming server, and the livestreaming server distributes the video stream to more users for viewing.

In this process, the streamer may control switching of whether current display content is a video stream photographed by a camera of the first terminal or a video stream recorded by the second terminal.

The foregoing described scenarios are merely examples for description, and constitute no limitation on another embodiment of this application. This is not limited to the foregoing scenarios. The screen sharing method provided in this embodiment of this application may be applied to another social scenario, an office scenario, a shopping scenario, or the like.

With reference to the descriptions of the communication system10,FIG.4toFIG.8B-1toFIG.8B-3show a video conferencing service scenario and a related user interface.

As shown inFIG.4, a user A and a user B are performing a video conference. The user A uses a mobile phone A, the user B uses a mobile phone B, and the mobile phone A and the mobile phone B transmit video data through the Internet by using a video call application.

As shown inFIG.4, an image of a peer party, namely, the user B is displayed in a main window401of a video call user interface of the mobile phone A, that is, an image of the user B. The image of the user B is an image captured by a camera of the mobile phone B. The main window401is a window filling a screen, and the main window401may display a video stream that is of the mobile phone B and that is received by the mobile phone A, namely, the image of the user B.

The main window401further displays a voice on/off control402, a camera on/off control403, an “Exit” control404, a camera switching control405, a floating window406, and the like. The floating window406may display an image that is of the user A and that is captured by a local camera of the mobile phone A.

When the user taps the camera switching control405, the mobile phone A may switch a camera that currently captures an image, for example, may be switched from a front-facing camera to a rear-facing camera to capture the image.

Images displayed in the main window401and the floating window406may be exchanged. In other words, the main window401may display the image of the user A, and the floating window406displays the image of the user B. An image exchange method may be a method for switching a view size. To be specific, a window size of the floating window406and a window size of the main window401are exchanged. In addition, a method for switching a read video stream may alternatively be used. To be specific, a video stream obtained by the main window401and a video stream obtained by the floating window406are exchanged.

In this embodiment of this application, the main window and the floating window are distinguished based on a window nature (instead of a main body of the obtained video stream). To be specific, a window that displays a video image and that fills the screen is referred to as a main window, and a window that is superposed on the main window is referred to as a floating window. Usually, the floating window covers the main window and is movable, and the floating window has a small size, and occupies only a part of a display area of the main window. The user may manually drag a display location of the floating window.

In an implementation, when the user taps an area of the floating window406, displayed images of the main window401and the floating window406may be exchanged. To be specific, a case in which the floating window406originally displays the image of the user A is switched to a case in which the main window401displays the image of the user A, and a case in which the main window401originally displays the image of the user B is switched to a case in which the floating window406displays the image of the user B.

A main window407of a video call user interface of the peer mobile phone B displays the image that is of the user A and that is received through the Internet, and the image of the user A is captured by a camera of the mobile phone A.

Similarly, a voice on/off control, a camera on/off control, an “Exit” control, a camera switching control, a floating window408, and the like exist in the main window407, and the floating window408displays an image that is of the user B and that is captured by a local camera of the mobile phone B.

Similarly, the floating window408covers the main window407and is movable. In an implementation, when the user taps an area of the floating window408, displayed images of the main window407and the floating window408may be exchanged. To be specific, a case in which the floating window408originally displays the image of the user B is switched to a case in which the main window407displays the image of the user B, and a case in which the main window407originally displays the image of the user A is switched to a case in which the floating window408displays the image of the user A.

In some embodiments, as shown inFIG.5AtoFIG.5C, a conference material that needs to be demonstrated by a user A is on a computer A, and a mobile phone A may establish a Wi-Fi P2P connection to the computer A. A connection is not limited to the Wi-Fi P2P connection. The connection between the mobile phone A and the computer A may alternatively be another type of wireless connection or a wired connection (for example, a USB connection). Examples are not listed one by one herein. The conference material is displayed in an interface501of the computer A. The computer A may record an interface501, and then send a recorded media stream (including an audio stream, a video stream, and the like) to the mobile phone A through the connection. The mobile phone A may send the received recorded media stream from the computer A to a mobile phone B by using a video call application.

As shown inFIG.5AtoFIG.5C, a main window502of a user interface of the mobile phone A displays an image that is of a user B, that is captured by the mobile phone B, and that is received by using the video call application, and a floating window503displays a received picture in the recording interface501of the computer A. The mobile phone B receives the media stream from the mobile phone A by using a video call application through a connection. A main window504of a user interface of the mobile phone B displays the picture in the recording interface501of the computer A instead of a picture that is of the user A and that is captured by the mobile phone A. The floating window505displays an image that is of the user B and that is captured by a camera of the mobile phone B.

In some embodiments, for convenience of viewing by the user, when the user taps an area of the floating window503in an interface of the mobile phone A, displayed images of the main window502and the floating window503may be exchanged.

As shown inFIG.6AtoFIG.6C, a conference material is displayed in an interface601of a computer A. The computer A may send a media stream (including an audio stream, a video stream, and the like) of a recording interface601to a mobile phone A. The mobile phone A may send the received recorded media stream from the computer A to the mobile phone B by using a video call application.

A main window602of a user interface of the mobile phone A displays a received picture in the recording interface601of the computer A. A floating window603displays an image that is of a user B, that is captured by the mobile phone B, and that is received by using a video call application.

The mobile phone B receives the media stream from the mobile phone A by using a video call application through a connection. A main window604of a user interface of the mobile phone B displays the picture in the recording interface601of the computer A. The floating window605displays an image that is of the user B and that is captured by a camera of the mobile phone B. That is, the main window602of the mobile phone A and the main window604of the mobile phone B display the conference material, and a display area of the main window is large, to facilitate viewing by the user.

In some embodiments, the mobile phone A may not only send a received recorded picture of the computer A to the mobile phone B, but also send, to the mobile phone B at the same time, an image captured by a camera of the mobile phone A. In this way, a user of the mobile phone A and a user of the mobile phone B not only can view the conference material of the computer A, but also can see a portrait of a peer user. This is more convenient for visual communication between the user A and the user B.

As shown inFIG.7A-1toFIG.7A-3, a conference material is displayed in an interface701of a computer A. The computer A may send a media stream (including an audio stream, a video stream, and the like) of a recording interface701to a mobile phone A. The mobile phone A may send the received recorded media stream from the computer A to the mobile phone B by using a video call application, and the mobile phone A may further send, to the mobile phone B, an image that is of the user A and that is captured by a camera of the mobile phone A.

A main window702of a user interface of the mobile phone A displays a received picture in the recording interface701of the computer A. The user interface of the mobile phone A may include two floating windows. A floating window703displays an image that is of a user B, that is captured by the mobile phone B, and that is received by using a video call application. A floating window704displays an image that is of a user A and that is captured by a camera of the mobile phone A.

The mobile phone B receives the media stream from the mobile phone A by using a video call application through a connection. A main window705of a user interface of the mobile phone B displays the picture in the recording interface701of the computer A. The user interface of the mobile phone B may include two floating windows. A floating window706displays the image that is of the user A, that is captured by the mobile phone A, and that is received by using the video call application. A floating window707displays the image that is of the user B and that is captured by a camera of the mobile phone B.

In other embodiments, the mobile phone A or the mobile phone B may hide a floating window in which an image captured by a camera of the mobile phone A or the mobile phone B is located, to save screen display space and spare more screen areas to display a main window interface.

Compared withFIG.7A-1toFIG.7A-3, as shown inFIG.7B-1toFIG.7B-3, only a floating window703is displayed in a user interface of a mobile phone A. The floating window703displays an image that is of a user B, that is captured by a mobile phone B, and that is received by using a video call application, and a floating window704that displays an image captured by a camera of the mobile phone A is hidden.

Only a floating window706is displayed in a user interface of the mobile phone B. The floating window706displays an image that is of a user A, that is captured by the mobile phone A, and that is received by using a video call application, and a floating window707that displays an image captured by a camera of the mobile phone B is hidden.

In some embodiments, the user may further add an editing operation such as scrawling, highlighting, and commenting to a display interface of the computer A or the mobile phone A. These editing operations may be combined with an original video picture, to generate a new media stream and then transmit the new media stream to the mobile phone B for display.

As shown inFIG.8A-1toFIG.8A-3, in some embodiments, a mobile phone A may add an editing operation to a display picture.

As shown inFIG.8A-1toFIG.8A-3, a conference material is displayed in an interface801of a computer A. The computer A may send a media stream (including an audio stream, a video stream, and the like) of a recording interface801to a mobile phone A. The mobile phone A displays the received recording interface801of the computer A in a main window802. A user may further touch a display of the mobile phone A with a finger, and makes a mark, for example, a check mark803, on a screen of the mobile phone A. The mobile phone A may combine the check mark803on the screen and an original media stream on the recording interface into a new media stream, and then send the media stream with the check mark803to the mobile phone B by using a video call application. A main window804of the mobile phone B may display the received media stream of the mobile phone A, which is consistent with display content in the main window802of the mobile phone A. In other words, content displayed in the main window804of the mobile phone B not only includes a conference material, but also includes a check mark805.

As shown inFIG.8B-1toFIG.8B-3, in some other embodiments, a computer A may also add an editing operation to a display picture.

As shown inFIG.8B-1toFIG.8B-3, a conference material is displayed in an interface810of the computer A. A user may operate an input device such as a mouse, a keyboard, or a stylus, to make a mark, for example, a check mark811in the interface810. A recording interface of the computer A is marked with the check mark811. Then, the computer A sends a recorded media stream (including an audio stream and a video stream) to a mobile phone A. The mobile phone A displays the received recording interface810of the computer A in the main window812. It may be learned that content displayed in the main window812of the mobile phone A also includes a check mark813. Then, the mobile phone A sends the media stream with the check mark813to the mobile phone B by using a video call application. A main window814of the mobile phone B may display the received media stream of the mobile phone A, which is consistent with display content in the main window812of the mobile phone A. In other words, content displayed in the main window814of the mobile phone B not only includes a conference material, but also includes a check mark815.

In other embodiments, the computer A and the mobile phone A may simultaneously edit a display interface, combine a new media stream, and transmit the new media stream to the mobile phone B for display. In this way, the computer A and the mobile phone A may collaboratively complete editing of the display picture, to be close to display effect that the user wants to achieve as much as possible, thereby improving a human-computer interaction function of the computer A and the mobile phone A, bringing convenience to users of the computer A and the mobile phone A, and avoiding a case in which editing display effect cannot satisfy both users when the computer A or the mobile phone A separately edits the display picture.

In some other embodiments, the mobile phone B may also edit the display interface, and synchronize the edited display interface to a display interface of the mobile phone A. In this way, a human-computer interaction function of the mobile phone B can also be improved, and convenience is also brought to a user of the mobile phone B.

It should be noted that, service scenarios and user interfaces shown inFIG.4,FIG.5AtoFIG.5C,FIG.6AtoFIG.6C,FIG.7A-1toFIG.7A-3,FIG.7B-1toFIG.7B-3,FIG.8A-1toFIG.8A-3, andFIG.8B-1toFIG.8B-3are merely used to assist in describing the technical solutions of embodiments of this application. In an actual service scenario, a type of an included terminal device is not only a mobile phone or a computer, but also may be a smart screen, an electronic conference whiteboard, a laptop computer, or the like. More terminal devices may participate a video conference, and displayed user interfaces may be different. Examples shown inFIG.4,FIG.5AtoFIG.5C,FIG.6AtoFIG.6C,FIG.7A-1toFIG.7A-3,FIG.7B-1toFIG.7B-3,FIG.8A-1toFIG.8A-3, andFIG.8B-1toFIG.8B-3sets no limitation on this application.

In scenarios of the video conferencing service that are shown inFIG.4,FIG.5AtoFIG.5C,FIG.6AtoFIG.6C,FIG.7A-1toFIG.7A-3,FIG.7B-1toFIG.7B-3,FIG.8A-1toFIG.8A-3, andFIG.8B-1toFIG.8B-3, the computer A may quickly share screen interface content (for example, a conference material, a photo, and a video) to the mobile phone A and the mobile phone B for viewing. This step is simple, fast, and efficient. Software and hardware capabilities of different devices are integrated, to further improve office efficiency, and implement intelligent office scenario experience.

With reference to the foregoing embodiments, the following describes schematic diagrams of some user interfaces in embodiments shown inFIG.9A-1andFIG.9A-2,FIG.9B-1andFIG.9B-2,FIG.10A-1andFIG.10A-2, andFIG.10B-1andFIG.10B-2.

A mobile phone A and a computer A may perform device matching and establish a connection by using a short-range communication technology such as Bluetooth, Wi-Fi, and NFC, or may establish a connection by using a long-range communication technology. This is not limited in this application.

In examples ofFIG.9A-1andFIG.9A-2,FIG.9B-1andFIG.9B-2,FIG.10A-1andFIG.10A-2, andFIG.10B-1andFIG.10B-2, the mobile phone A and the computer A establish a communication connection through Wi-Fi P2P.

If the mobile phone A and the computer A log in to a same system account or have been matched, the peer party is identified as a trusted device. After a connection invitation sent by a peer device is received, there are the following cases:

If the peer party is identified as a trusted device, the connection may be automatically allowed, without a cumbersome authentication process.

If the peer device is identified as an untrusted device, authentication needs to be performed to ensure device security. The untrusted device may be a device to which no communication connection is established or that logs in to a system account different from that of the device. An authentication manner may be a manner such as scanning a two-dimensional code, or entering a verification code.

In some embodiments, a user may control a screen sharing process by using an application “Huawei multi-screen collaboration” on the mobile phone A and the computer A.

After a connection is successfully established between the mobile phone A and the computer A, as shown inFIG.9A-1andFIG.9A-2, after the mobile phone A detects, for example, a user operation of sliding downward from a top status bar, the mobile phone A displays a user interface901in response to the user operation. The user interface901may include the top status bar and a notification bar902.

The top status bar may include a mobile signal indicator, a wireless network signal indicator, a power indicator, a time indicator, and the like. The notification bar902may display a notification item, a shortcut setting item, and the like of “Huawei multi-screen collaboration”. For example, the notification bar902displays a case in which the mobile phone A is currently connected to the computer A. Certainly, a displayed name of a connected device may be customized.

The notification bar902may further display an “Enable screen sharing” option903. After the user taps the “Enable screen sharing” option903, the computer A starts screen recording, and sends a recorded audio/video stream to the mobile phone A. A local audio/video channel of the mobile phone A is switched to access the audio/video stream sent by the computer A.

As shown inFIG.9A-1andFIG.9A-2, a user interface904of the computer A displays an interface905of the application “Huawei multi-screen collaboration”. The interface905displays a case in which the computer A is currently connected to the mobile phone A, and further displays some function options such as an “Enable screen sharing” option906, a “Select a to-be-shared file” option907, and an “Add a comment box” option908.

After the user taps the “Enable screen sharing” option906, the computer A starts to record a screen and sends a recorded video stream to the mobile phone A.

The user may alternatively share a specified file. For example, if the user taps the “Select a to-be-shared file” option907, and then selects a file A, the computer A records only an interface corresponding to the file A, and another application interface does not enter a recording interface. An “Add a comment box” function is to help the user comment content at any time during screen sharing. The recording interface includes comment content, and then sends a recorded audio/video stream to the mobile phone A. The mobile phone A may display the recording interface with the comment content.

If the user selects a to-be-shared file on the mobile phone A or the computer A, for example, a “Conference PPT” file, as shown inFIG.9B-1andFIG.9B-2, a user interface910of the computer A displays a case in which the selected to-be-shared file “Conference PPT” is being recorded. A user interface911of the mobile phone A displays a notification bar912, and the notification bar912displays an indication word913of “Displaying and controlling a screen of the computer A”. The notification bar912may further include a “Pause screen sharing” option914, to help the user operate a screen sharing pause instruction.

After the user performs a sliding operation to close the notification bar912, for a display interface and a scenario, refer toFIG.5AtoFIG.5C. The mobile phone A sends, to the mobile phone B, a screen recording media stream (including an audio stream, a video stream, and the like) received from the computer A, and a user interface504of the mobile phone B displays a screen recording video picture of the computer A.

In some embodiments, some options may be set on the mobile phone A, to help the user control the computer A on a side of the mobile phone A, and give a more diversified function to screen sharing. For an example interface, refer toFIG.10A-1andFIG.10A-2andFIG.10B-1andFIG.10B-2.

As shown inFIG.10A-1andFIG.10A-2, the computer A and the mobile phone A have established a connection, and are sharing a screen. The computer A records the interface1001that is being displayed, and sends the recorded media stream to the mobile phone A. A recording interface of the computer A is displayed in a main window1002of a display interface of the mobile phone A. The display interface of the mobile phone A may further include a floating window1003and a floating window1004. The floating window1003may be used to display an image transmitted by a video call connection peer end, namely, the mobile phone B (not shown), and the floating window1004may be used to display an image that is of a user and that is captured by a camera of the mobile phone A.

The display interface of the mobile phone A may further include an option control1005. When it is detected that the user taps the option control1005, an option bar1006may be expanded in the display interface of the mobile phone A. As shown inFIG.10B-1andFIG.10B-2, the option bar1006may include options such as “Select a to-be-shared file”, “Add a comment box”, “End screen sharing”, “More function options”, and “Help and feedback”. The user may select an option, to implement a more diversified control function.

It should be noted that, service scenarios and user interfaces shown inFIG.9A-1andFIG.9A-2,FIG.9B-1andFIG.9B-2,FIG.10A-1andFIG.10A-2, andFIG.10B-1andFIG.10B-2are merely examples, are used to assist in describing the technical solutions of embodiments of this application, and constitute no limitation on this application. In an actual service scenario, there may be different displayed user interfaces.

With reference to the embodiments shown inFIG.2toFIG.10B-1andFIG.10B-2, the following describes a screen sharing method provided in embodiments of this application.

In embodiments, a communication system including a first terminal, a second terminal, and a third terminal is used as an example for description. A type of each terminal is not limited in this embodiment of this application, and may be a mobile phone, a PC, a smart screen, or the like. In some examples, the first terminal may be the foregoing mobile phone A, the second terminal may be the foregoing computer A, and the third terminal may be the foregoing mobile phone B. It can be understood that, the method provided in embodiments is merely an example, and constitutes no limitation on another embodiment of this application. In an actual service scenario, more or fewer terminal devices may be included.

FIG.11is a flowchart of a screen sharing method according to Embodiment 1 of this application. In Embodiment 1, a first terminal establishes a second connection to a third terminal, and then the first terminal establishes a first connection to a second terminal. Specifically, the following steps are included:

S101: The first terminal establishes the second connection to the third terminal.

In some embodiments, the second connection established between the first terminal and the third terminal may be a video call connection, or may be a video conference connection. This is not limited in this embodiment of this application. The second connection may be a communication connection for transferring data through the Internet or a local area network based on a video call application platform such as a chatting app (for example, MeeTime, WeChat, DingTalk, or WeCom) or a conference app (for example, ZOOM, welink, or cloudlink).

The first terminal or the third terminal may initiate and establish the second connection to a peer end in response to a received user operation, for example, a user operation of initiating a video call connection, or a user operation of receiving a video call invitation.

S102: The first terminal plays a third media stream received from the third terminal, where the third media stream includes a picture captured by a camera of the third terminal.

S103: The third terminal plays a fourth media stream received from the first terminal, where the fourth media stream includes a picture captured by a camera of the first terminal.

After the first terminal establishes the second connection to the third terminal, the first terminal and the third terminal may send a data stream to each other. The data stream is a set of data sequences that arrive sequentially, massively, fast, and continuously. Usually, the data stream may be considered as a dynamic data set that infinitely increases over time.

The data stream in this embodiment is a media stream, the media stream includes an audio stream, a video stream, and the like, and the video stream is a plurality of pieces of video data, and includes an image stream including frames of images. Usually, an audio stream, that is, audio data corresponding to a video picture, is transmitted together with the video stream.

In this embodiment, the first terminal sends the fourth media stream to the third terminal. The fourth media stream is a data stream generated by encoding a plurality of pictures and corresponding sounds that are captured by the camera and a microphone of the first terminal. After the fourth media stream is received, decoding is performed, and the third terminal plays audio and a video that correspond to the fourth media stream received from the first terminal, displays a corresponding video picture, and plays corresponding audio.

In addition, the third terminal sends the third media stream to the first terminal. The third media stream is a data stream generated by encoding a plurality of pictures and corresponding sounds that are captured by the camera and a microphone of the third terminal. After the third media stream is received, decoding is performed, and the first terminal plays audio and a video that correspond to the third media stream received from the third terminal, displays a corresponding video picture, and plays corresponding audio.

In some embodiments, a display interface of the first terminal or the third terminal includes a plurality of windows.

As shown inFIG.4, a video call display interface of the first terminal or the third terminal may include a main window and a floating window. The main window may be used to display a video picture sent by a peer device, and the floating window may be used to display a video picture captured by a local camera. For example, a first user interface of the first terminal may include a main window and a floating window. The main window displays a video picture that corresponds to the third media stream and that is sent by the third terminal, and the floating window displays a video picture that corresponds to the fourth media stream and that is captured by the camera of the first terminal. Similarly, a second user interface of the third terminal may also include a main window and a floating window. The main window displays a video picture that corresponds to the fourth media stream and that is sent by the first terminal, and the floating window displays a video picture that corresponds to the third media stream and that is captured by the camera of the third terminal.

In some embodiments, the first user interface of the first terminal includes a first window and a second window, the first window is a main window, the second window is a floating window, the second user interface of the third terminal includes a third window and a fourth window, the third window is a main window, and the fourth window is a floating window.

In an example, as shown inFIG.4, after the first terminal establishes the second connection to the third terminal, a picture displayed in the first window is a picture captured by the camera of the third terminal, a picture displayed in the second window is a picture captured by the camera of the first terminal, a picture displayed in the third window is a picture captured by the camera of the first terminal, and a picture displayed in the fourth window is a picture captured by the camera of the third terminal.

S104: The first terminal establishes the first connection to the second terminal, and negotiates a screen sharing parameter.

In this embodiment of this application, the first connection established between the first terminal and the second terminal may be a wired connection or a wireless connection. In an implementation, the first terminal may establish a communication connection to the second terminal based on Wi-Fi near field networking communication, for example, a Wi-Fi P2P connection.

In some embodiments, the first terminal may find the second terminal, and then send a second connection establishment request to the second terminal. After receiving feedback information that is sent by a second terminal selected by a user and that indicates that establishment of the communication connection is allowed, the first terminal successfully establishes the second connection to the second terminal.

Certainly, a sender of the second connection establishment request may also be the second terminal. After receiving a connection establishment invitation sent by the second terminal, the first terminal sends, to the second terminal, a feedback indicating that establishment of the communication connection is allowed, and then the first terminal successfully establishes the second connection to the second terminal. The examples herein constitute no limitation on other embodiments.

When the first terminal establishes the communication connection to the second terminal, there are but not limited to the following two cases:

(1) The Second Terminal is a Trusted Device.

When the second terminal is a trusted device for the first terminal, after receiving a communication connection establishment request sent by the first terminal, the second terminal may directly send, to the first terminal, feedback information indicating that establishment of the communication connection is allowed, to complete establishment of the communication connection. In this way, an operation of the user can be simplified, and control of the second terminal or the first terminal by the user is facilitated.

(2) The Second Terminal is an Untrusted Device.

When the second terminal is an untrusted device, and the first terminal sends the communication connection establishment request to the second terminal, the first terminal and the second terminal enter an authentication process, and the first terminal verifies account information of the second terminal. When the first terminal and the second terminal complete the authentication process, establishment of the communication connection is completed. In this way, security of communication between devices can be ensured.

In some embodiments, authentication between the first terminal and the second terminal may be code (for example, a two-dimensional code or a barcode) scanning verification.

In some other embodiments, authentication between the first terminal and the second terminal may alternatively be PIN code verification. Specifically, when the first terminal initiates the communication connection establishment request to the second terminal, the second terminal displays, in a user interface of the second terminal, a PIN code used for authentication. The first terminal may provide an input interface used for entering the PIN code. When the user enters, on the first terminal, the PIN code displayed by the second terminal, the authentication process between the first terminal and the second terminal may be completed.

It can be learned that, an operation that the user establishes the communication connection may be optimized by setting the trusted device. For a device that has not established a communication connection to the first terminal, the communication connection needs to be established through authentication. When the user reselects a device that once establishes a communication connection, the user may not need to perform authentication again; or when the user selects another electronic device that has a same account as the first terminal to be connected to the first terminal, an authentication process may be omitted. In this way, an operation is simplified, and communication security of the device is ensured.

It can be understood that, in another embodiment of this application, the trusted device or the untrusted device may not be distinguished. All devices found by the first terminal may be processed in a connection manner of the trusted device, or all devices may be processed in a connection manner of the untrusted device.

This is not limited to the foregoing manners. The first terminal and the second terminal may alternatively establish the communication connection in another manner. For example, verification is performed by entering a verification code, and a “Tap” operation based on an NFC technology is performed. This is not limited in this embodiment of this application.

After establishing the first connection, the first terminal and the second terminal may negotiate the screen sharing parameter, including but not limited to a screen recording, a bitrate of a sound recording module, a recording format, a cache path, a bitrate, a frame rate, a resolution of a screen recording, a source of a sound recording (audio corresponding to a media file or audio collected by a microphone), an audio/video encoding/decoding manner, a screen recording area, a specified recording file, a video window rotation policy (a direction of a recorded video is a vertical direction or a horizontal direction), a size of a video recording window, a policy of filling a black border for an area that does fills a screen, or the like. This is not limited in this embodiment.

For example, a resolution of a video stream recorded and transmitted by the second terminal may be negotiated. Affected by a rate of a transmission channel, to ensure that interface display is smooth during screen sharing, the first terminal and the second terminal may negotiate a resolution of a recorded video stream based on a transmission speed of a current connection. Alternatively, because a screen of a mobile phone is small and a resolution does not need to be particularly high, when a computer serves as the second terminal to record a video stream, while it is ensured that display is smooth and a specific definition is ensured, the resolution of the recorded video stream may be properly reduced, sharing efficiency may be improved, and a bandwidth can be saved.

After the first terminal establishes the first connection to the second terminal, the first terminal or the second terminal may initiate a screen recording start instruction. Step S105A is that the first terminal initiates the screen recording start instruction, and step S105B is that the second terminal initiates the screen recording start instruction. Therefore, the screen recording start instruction may be initiated in any manner in step S105A or step S105B.

S105A: In response to a first user operation, the first terminal sends the screen recording start instruction to the second terminal.

In an example, as shown inFIG.9A-1andFIG.9A-2, after the user taps an “Enable screen sharing” option903in an interface of a mobile phone A, the first terminal sends the screen recording start instruction to the second terminal in response to a first user operation of tapping the “Enable screen sharing” option903.

S105B: In response to a second user operation, the second terminal detects the screen recording start instruction.

In an example, as shown inFIG.9A-1andFIG.9A-2, the user may operate a mouse pointer to tap an “Enable screen sharing” option906, and in response to a second user operation of tapping the “Enable screen sharing” option906, the second terminal detects the screen recording start instruction.

S106: The second terminal records a screen picture, and generates a first media stream.

After the screen sharing parameter is negotiated, the second terminal may start screen recording in response to the screen recording start instruction in step S105A or step S105B, and generate the first media stream.

The first media stream may include a display picture of a screen of the second terminal, and audio collected by a microphone of the second terminal or a sound corresponding to a media file played by the second terminal.

In some embodiments, after the first connection is established between the first terminal and the second terminal, the screen picture may automatically start to be recorded, and the user does not need to send the screen recording start instruction.

A picture recorded by the second terminal may be a full-screen interface of the second terminal. Alternatively, the user may specify recording content, to protect privacy. For example, the user may specify a screen area, or the user may specify one or more documents or pictures.

In some embodiments, the second terminal may further enable a local camera, to combine a picture captured by the local camera and a screen recording picture, to generate a first media stream and send the first media stream to the first terminal together. In this way, not only screen recording content can be viewed, but also a picture captured by the camera of the second terminal, for example, an image of a user who uses the second terminal can be viewed.

In some embodiments, the second terminal may further combine a real-time comment of the user on the second terminal and the screen recording picture, to generate a first media stream and send the first media stream to the first terminal, so as to improve screen sharing efficiency.

S107: The second terminal sends the first media stream to the first terminal.

The second terminal sends the first media stream to the first terminal through the first connection.

S108: The first terminal generates a second media stream based on the obtained first media stream.

In some embodiments, after obtaining the first media stream (including an audio data stream and a video data stream) from the second terminal, the first terminal forwards the first media stream to the third terminal. In other words, the second media stream is equal to the first media stream.

In some embodiments, after obtaining the first media stream from the second terminal, the first terminal may perform new editing on the first media stream, to generate the second media stream and send the second media stream to the third terminal.

For example, the first terminal may package a first media stream of a screen recording of the second terminal and an image that is of the user and that is captured by the camera of the first terminal, to generate the second media stream, and send the second media stream to the third terminal. As shown inFIG.7A-1toFIG.7A-3, a mobile phone B not only can view a conference material picture shared by the computer A, but also can view an image that is of the user and that is captured by a camera of the mobile phone A.

For another example, the first terminal may perform an operation such as scrawling and commenting on a recorded screen picture corresponding to the first media stream, and then generate a second media stream. A video picture in the second media stream has an editing trace such as scrawling and commenting. The first terminal sends the second media stream to the third terminal, and the video picture corresponding to the second media stream displayed by the third terminal displays a picture with an editing trace. As shown inFIG.8A-1toFIG.8A-3, the mobile phone B not only can view a conference material shared by the computer A, but also can view a newly generated scrawling and commenting trace on the mobile phone A.

In some embodiments, audio data transmitted by the second terminal, for example, a sound collected by the microphone of the second terminal or audio corresponding to the media file played by the second terminal, may be selected as audio data of the second media stream.

In other embodiments, audio data of the second media stream, for example, a voice that is of the user and that is collected by the microphone of the first terminal or an audio corresponding to a media file on the first terminal, may be selected from the first terminal.

S109: The first terminal sends a second media stream to the third terminal.

The first terminal sends the second media stream to the third terminal by using a video call app.

Specifically, the first terminal may enable audio and image data of the second media stream to use a transmission channel that is of the local camera and the microphone and that used to transmit audio and a video, and then transmit the second media stream to upper-layer video call software. An upper-layer video call application does not perceive a case in which a video stream changes. The first terminal can send, to the third terminal, a screen picture captured by the second terminal, without a need to modify a system interface invoked by a third-party application, in other words, without a need to adaptively modify the third-party application.

S110: The first terminal plays the third media stream received from the third terminal, where the third media stream includes the picture captured by the camera of the third terminal.

S111: The third terminal plays the second media stream received from the first terminal, where the second media stream includes the screen picture captured by the second terminal through screen recording.

After the foregoing steps, the picture displayed by the third terminal includes the screen picture captured by the second terminal through screen recording.

In some embodiments, the display interface of the first terminal or the third terminal includes a plurality of windows.

As shown inFIG.5AtoFIG.5C, the video call display interface of the first terminal or the third terminal may include the main window and the floating window. The main window may be used to display the video picture sent by the peer device, and the floating window may be used to display the video picture captured by the local camera. For example, the second user interface of the first terminal may include a main window and a floating window. The main window displays a video picture corresponding to the third media stream sent by the third terminal, and the floating window displays a screen recording picture from the second terminal. Similarly, a fourth user interface of the third terminal may also include a main window and a floating window. The main window displays the video picture corresponding to the second media stream sent by the first terminal, including the screen recording picture of the second terminal, and the floating window displays the video picture corresponding to the third media stream collected by the local camera of the third terminal.

In some other embodiments, the third terminal not only displays the screen recording picture from the second terminal, but also displays an image that is of the user and that is captured by the camera of the first terminal. As shown inFIG.7A-1toFIG.7A-3, a user interface of the mobile phone B displays a conference material recording picture of the computer A, and a floating window706further displays an image of the user A of the mobile phone A.

In some other embodiments, the third terminal not only displays the screen recording picture of the second terminal, but also may display an editing trace such as a scrawling and commenting trace made by the first terminal. As shown inFIG.8A-1toFIG.8A-3, a user interface of the mobile phone B not only displays a conference material recording picture of the computer A, but also synchronously displays a check mark805made by the user on a side of the mobile phone A.

In an example, as shown inFIG.5AtoFIG.5C, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the screen picture recorded by the second terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

In an example, as shown inFIG.6AtoFIG.6C, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the screen picture recorded by the second terminal, the picture displayed in the second window is the picture captured by the camera of the third terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

In an example, as shown inFIG.7A-1toFIG.7A-3, the first user interface of the first terminal further includes a fifth window, the second user interface of the third terminal further includes a sixth window, and the fifth window and the sixth window are floating windows. A picture displayed in the fifth window is the picture captured by the camera of the first terminal, and a picture displayed in the sixth window is the picture captured by the camera of the first terminal.

FIG.12is a flowchart of a screen sharing method according to Embodiment 2 of this application. In Embodiment 2, a first terminal establishes a first connection to a second terminal, and then the first terminal establishes a second connection to a third terminal. In other words, the first terminal establishes a screen sharing connection to the second terminal, and then establishes a video call connection to the third terminal. In this way, an interface of the third terminal may directly receive, from the first terminal for display, a picture captured by the second terminal through screen recording. A camera of the first terminal does not need to be enabled, to further improve screen sharing efficiency.

Specifically, the following steps are included:

S201: The first terminal establishes the first connection to the second terminal, and negotiates a screen sharing parameter.

In this embodiment of this application, the first connection established between the first terminal and the second terminal may be a wired connection or a wireless connection. In an implementation, the first terminal may establish a communication connection to the second terminal based on Wi-Fi near field networking communication, for example, a Wi-Fi P2P connection.

For more descriptions, refer to step S104. Details are not described herein again.

After establishing the first connection, the first terminal and the second terminal may negotiate the screen sharing parameter, including but not limited to a screen recording, a bitrate of a sound recording module, a recording format, a cache path, a bitrate, a frame rate, a resolution of a screen recording, a source of a sound recording (audio corresponding to a media file or audio collected by a microphone), an audio/video encoding/decoding manner, a screen recording area, a specified recording file, a video window rotation policy (a direction of a recorded video is a vertical direction or a horizontal direction), a size of a video recording window, a policy of filling a black border for an area that does fills a screen, or the like. This is not limited in this embodiment.

For example, a resolution of a video stream recorded and transmitted by the second terminal may be negotiated. Affected by a rate of a transmission channel, to ensure that interface display is smooth during screen sharing, the first terminal and the second terminal may negotiate a resolution of a recorded video stream based on a transmission speed of a current connection. Alternatively, because a screen of a mobile phone is small and a resolution does not need to be particularly high, when a computer serves as the second terminal to record a video stream, while it is ensured that display is smooth and a specific definition is ensured, the resolution of the recorded video stream may be properly reduced, sharing efficiency may be improved, and a bandwidth can be saved.

After the first terminal establishes the first connection to the second terminal, the first terminal or the second terminal may initiate a screen recording start instruction. Step S202A is that the first terminal initiates the screen recording start instruction, and step S202B is that the second terminal initiates the screen recording start instruction. Therefore, the screen recording start instruction may be initiated in any manner in step S202A or step S202B.

S202A: In response to a first user operation, the first terminal sends the screen recording start instruction to the second terminal.

In an example, as shown inFIG.9A-1andFIG.9A-2, after the user taps an “Enable screen sharing” option903in an interface of a mobile phone A, the first terminal sends the screen recording start instruction to the second terminal in response to the first user operation of tapping the “Enable screen sharing” option903.

S202B: In response to a second user operation, the second terminal detects the screen recording start instruction.

In an example, as shown inFIG.9A-1andFIG.9A-2, the user may operate a mouse pointer to tap an “Enable screen sharing” option906, and in response to the second user operation of tapping the “Enable screen sharing” option906, the second terminal detects the screen recording start instruction.

S203: The second terminal records a screen picture, and generates a first media stream.

After the screen sharing parameter is negotiated, the second terminal may start screen recording in response to the screen recording start instruction in step S202A or step S202B, and generate the first media stream.

The first media stream may include a display picture of a screen of the second terminal, and audio collected by a microphone of the second terminal or a sound corresponding to a media file played by the second terminal.

In some embodiments, after the first connection is established between the first terminal and the second terminal, the screen picture may automatically start to be recorded, and the user does not need to send the screen recording start instruction.

A picture recorded by the second terminal may be a full-screen interface of the second terminal. Alternatively, the user may specify recording content, to protect privacy. For example, the user may specify a screen area, or the user may specify one or more documents or pictures.

In some embodiments, the second terminal may further enable a local camera, to combine a picture captured by the local camera and a screen recording picture, to generate a first media stream and send the first media stream to the first terminal together. In this way, not only screen recording content can be viewed, but also a picture captured by the camera of the second terminal, for example, an image of a user who uses the second terminal can be viewed.

In some embodiments, the second terminal may further combine a real-time comment of the user on the second terminal and the screen recording picture, to generate a first media stream and send the first media stream to the first terminal, so as to improve screen sharing efficiency.

S204: The second terminal sends the first media stream to the first terminal.

The second terminal sends the first media stream to the first terminal through the first connection.

S205: The first terminal establishes the second connection to the third terminal.

In some embodiments, the second connection established between the first terminal and the third terminal may be a video call connection, or may be a video conference connection. This is not limited in this embodiment of this application. The second connection may be a communication connection for transferring data through the Internet or a local area network based on a video call application platform such as a chatting app (for example, MeeTime, WeChat, DingTalk, or WeCom) or a conference app (for example, ZOOM, welink, or cloudlink).

The first terminal or the third terminal may initiate and establish the second connection to a peer end in response to a received user operation, for example, a user operation of initiating a video call connection, or a user operation of receiving a video call invitation.

S206: The first terminal generates a second media stream based on the obtained first media stream.

In some embodiments, after obtaining the first media stream from the second terminal, the first terminal forwards the first media stream to the third terminal. In other words, the second media stream is equal to the first media stream.

In some embodiments, after obtaining the first media stream from the second terminal, the first terminal may perform new editing on the first media stream, to generate the second media stream and send the second media stream to the third terminal.

For example, the first terminal may package a first media stream of a screen recording of the second terminal and an image that is of the user and that is captured by the camera of the first terminal, to generate the second media stream, and send the second media stream to the third terminal. As shown inFIG.7A-1toFIG.7A-3, a mobile phone B not only can view a conference material picture shared by the computer A, but also can view an image that is of the user and that is captured by a camera of the mobile phone A.

For another example, the first terminal may perform an operation such as scrawling and commenting on a screen recording picture corresponding to the first media stream, and then generate a second media stream. A video picture in the second media stream has an editing trace such as scrawling and commenting. The first terminal sends the second media stream to the third terminal, and the video picture corresponding to the second media stream displayed by the third terminal displays a picture with an editing trace. As shown inFIG.8A-1toFIG.8A-3, the mobile phone B not only can view a conference material shared by the computer A, but also can view a newly generated scrawling and commenting trace on the mobile phone A.

S207: The first terminal sends a second media stream to the third terminal.

The first terminal sends the second media stream to the third terminal by using a video call app.

Specifically, the first terminal may enable audio and image data of the second media stream to use a transmission channel that is of the local camera and the microphone and that used to transmit audio and a video, and then transmit the second media stream to upper-layer video call software. An upper-layer video call application does not perceive a case in which a video stream changes. The first terminal can send, to the third terminal, a screen picture captured by the second terminal, without a need to modify a system interface invoked by a third-party application, in other words, without a need to adaptively modify the third-party application.

S208: The first terminal plays the third media stream received from the third terminal, where the third media stream includes the picture captured by the camera of the third terminal.

S209: The third terminal plays the second media stream received from the first terminal, where the second media stream includes the screen picture captured by the second terminal through screen recording.

After the foregoing steps, the picture displayed by the third terminal includes the screen picture captured by the second terminal through screen recording.

In some embodiments, the display interface of the first terminal or the third terminal includes a plurality of windows.

As shown inFIG.5AtoFIG.5C, the video call display interface of the first terminal or the third terminal may include the main window and the floating window. The main window may be used to display the video picture sent by the peer device, and the floating window may be used to display the video picture captured by the local camera. For example, the second user interface of the first terminal may include a main window and a floating window. The main window displays a video picture corresponding to the third media stream sent by the third terminal, and the floating window displays a screen recording picture from the second terminal. Similarly, a fourth user interface of the third terminal may also include a main window and a floating window. The main window displays the video picture corresponding to the second media stream sent by the first terminal, including the screen recording picture of the second terminal, and the floating window displays the video picture corresponding to the third media stream collected by the local camera of the third terminal.

In some other embodiments, the third terminal not only displays the screen recording picture from the second terminal, but also displays an image that is of the user and that is captured by the camera of the first terminal. As shown inFIG.7A-1toFIG.7A-3, a user interface of the mobile phone B displays a conference material recording picture of the computer A, and a floating window706further displays an image of the user A of the mobile phone A.

In some other embodiments, the third terminal not only displays the screen recording picture of the second terminal, but also may display an editing trace such as a scrawling and commenting trace made by the first terminal. As shown inFIG.8A-1toFIG.8A-3, a user interface of the mobile phone B not only displays a conference material recording picture of the computer A, but also synchronously displays a check mark805made by the user on a side of the mobile phone A.

In some embodiments, the first user interface of the first terminal includes a first window and a second window, the first window is a main window, the second window is a floating window, the second user interface of the third terminal includes a third window and a fourth window, the third window is a main window, and the fourth window is a floating window.

In an example, as shown inFIG.5AtoFIG.5C, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the screen picture recorded by the second terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

In an example, as shown inFIG.6AtoFIG.6C, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the screen picture recorded by the second terminal, the picture displayed in the second window is the picture captured by the camera of the third terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

In an example, as shown inFIG.7A-1toFIG.7A-3, the first user interface of the first terminal further includes a fifth window, the second user interface of the third terminal further includes a sixth window, and the fifth window and the sixth window are floating windows. A picture displayed in the fifth window is the picture captured by the camera of the first terminal, and a picture displayed in the sixth window is the picture captured by the camera of the first terminal.

According to the method provided in this embodiment, the second terminal can quickly share screen interface content (for example, a conference material, a photo, and a video) with the first terminal and the third terminal without a need to install video call software. This step is simple, fast, and efficient, and communication efficiency and user communication experience are further improved.

With reference to the foregoing embodiments, the following describes a virtual apparatus architecture of a communication system80according to an embodiment of this application.

FIG.13is a schematic diagram of a software module of a communication system80according to an embodiment of this application. The communication system80includes a first terminal and a second terminal. A device type and an operating system of the first terminal and the second terminal are not limited in this embodiment. A software system of the first terminal or the second terminal includes but is not limited to Harmony®, iOS®, Android®, Windows®, Linux®, or another operating system.

As shown inFIG.13, the first terminal establishes a first connection to the second terminal. The first terminal may include a communication module810, a video call application811, a display module812, an audio/video switching module813, a policy management module814, a camera management module815, a microphone management module816, and the like. The second terminal may include a display module801, a communication module802, a screen recording module803, a sound recording module804, a policy management module805, and the like. Modules of a same terminal may communicate through an application programming interface (application programming interface, API).

On a side of the first terminal, the communication module810is responsible for receiving an audio/video stream sent by the second terminal.

The video call application811is configured to establish a video call connection to another terminal device. As an upper-layer application, the video call application811simply invokes a local microphone and a camera, and does not perceive a case in which a lower-layer audio/video stream is replaced with a screen recording of the second terminal.

The display module812is responsible for displaying an image.

The audio/video switching module is configured to separately inject received audio/video content into a camera channel and a microphone channel of the first terminal.

The policy management module814is responsible for negotiating a screen sharing parameter with the second terminal, for example, information such as an audio/video bitrate, a video resolution, a screen sharing area, and a video window rotation policy.

The camera management module815and the microphone management module816are responsible for decoding the audio/video stream sent by the second terminal, and then delivering the audio/video stream to the video call application811.

On a side of the second terminal, the display module801is responsible for displaying an image.

The communication module802is responsible for transmitting recorded screen content to the first terminal.

The screen recording module803is responsible for recording screen content.

The sound recording module804is responsible for recording output audio of a screen or input audio of a microphone.

The policy management module805is responsible for managing all policy parameters such as a screen recording, a bitrate of the sound recording module, a screen resolution, a sound recording source (screen audio or audio collected by the microphone), an audio/video encoding/decoding manner, a screen recording area or a specified recording file, and a video window rotation policy.

According to the shown embodiment of the communication system80, the second terminal may quickly share screen interface content with the first terminal and the third terminal. This step is simple, fast, and efficient, to further improve office efficiency. For example, the first terminal and the second terminal may be supported to be in a multi-screen collaboration function, so that the second terminal shares the screen interface content with a video call application on the first terminal, and when the third terminal has a video call with the first terminal, the third terminal may view, by using the video call application, the interface content shared by the second terminal.

Display content of a media stream (including an audio stream and a video stream) sent by the first terminal to the third terminal includes but is not limited to the following three cases:

1. The Display Content is Edited by the First Terminal.

After the first terminal receives a video stream captured by the second terminal, a user may perform some control operations on a first terminal side, to edit, set, or adjust the video stream, generate a new video stream, and send the new video stream to the third terminal.

For example, the user may make some comments in a video picture of the first terminal. The comments and the original video picture are combined into a new video picture and sent to the third terminal. For another example, the user may set that a video picture sent by the first terminal to the third terminal not only includes a screen recording picture of the second terminal, but also includes a portrait that is of the user and that is captured by a camera of the first terminal. In this way, a third terminal side not only displays the screen recording picture from the second terminal, but also displays the portrait of the user of the first terminal. Therefore, the user has better video call experience. In addition, a picture parameter may be further adjusted in a video call screen of the first terminal. The picture parameter may include one or more of the following: an image cropping size, a color temperature of an image, a beauty/bodybuilding type, a filter type, a sticker option, and the like.

Therefore, the first terminal may adjust, based on an editing capability of the first terminal, an image captured by the second terminal, to generate a new display picture and send the new display picture to the third terminal. This brings use convenience to the user of the first terminal.

2. The Display Content is Edited by the Second Terminal.

After the second terminal generates a screen recording picture, a user may perform some control operations on a second terminal side, to edit, set, or adjust the video stream, generate a new video stream, and send the new video stream to the first terminal.

For example, the user may make some comments on the screen recording picture of the second terminal. The comments and the original screen recording picture are combined into a new video picture and sent to the first terminal. The first terminal sends the new video picture to the third terminal. For another example, the user may set that a video picture sent by the second terminal to the first terminal not only includes a screen recording picture of the second terminal, but also includes a portrait that is of the user and that is captured by a camera of the second terminal. In this way, a first terminal side and a third terminal side not only display the screen recording picture from the second terminal, but also display the portrait of the user of the second terminal. Therefore, the user has better video call experience. In addition, a picture parameter may be further adjusted in a user interface of the second terminal. The picture parameter may include one or more of the following: an image cropping size, a color temperature of an image, a beauty/bodybuilding type, a filter type, a sticker option, and the like.

Therefore, the second terminal may adjust, based on an editing capability of the second terminal, an image captured by the second terminal, to generate a new display picture and send the new display picture to the first terminal. This brings use convenience to the user of the second terminal.

3. The Display Content is Collaboratively Edited by the First Terminal and the Second Terminal.

With reference to the descriptions of the foregoing two aspects, in an implementation, the first terminal and the second terminal each may have a video picture editing function. In this way, the first terminal and the second terminal may collaboratively complete editing of the video picture, to be close to display effect that the user wants to achieve as much as possible. This brings convenience to users of both the first terminal and the second terminal, and avoids a case in which editing display effect cannot satisfy both users when the first terminal or the second terminal separately edits a display picture.

For example, after the second terminal edits a screen recording picture, the first terminal may edit the edited picture again. After the first terminal or the second terminal edits a screen recording picture, a display picture is synchronized to display interfaces of the first terminal and the second terminal.

Further, after the first terminal sends a display effect adjustment instruction to the second terminal, the second terminal may further perform compatibility processing on the instruction, so that the second terminal can effectively complete adjustment of corresponding display effect in response to the processed instruction. Because the first terminal and the second terminal may have different device types and device models, in response to a user operation that the user adjusts the display effect, the first terminal may generate some instructions that are not supported or cannot be identified by the second terminal, and the second terminal may not directly adjust the interface display effect in response to these instructions. The second terminal performs compatibility processing on the instruction, so that the second terminal can respond to an operation performed by the user on the first terminal, and the first terminal can control the second terminal to adjust the interface display effect.

It may be learned from the foregoing situation that, in addition to a case in which the second terminal may have a capability of editing a screen recording picture or another video stream, for a screen recording picture or another video stream from the second terminal, the first terminal may have a capability of editing the screen recording picture or the another video stream. This enhances effect of controlling the display interface of the second terminal by the first terminal, and improves use experience of the user.

In this embodiment of this application, that the first terminal displays an image of the second terminal may be that the first terminal transmits an image from the second terminal through a data stream channel collected by the camera of the first terminal. In other words, the first terminal originally transmits, through the data stream channel, data of an image captured by the camera of the first terminal. However, in a scenario of this embodiment of this application, the data stream channel is used to transmit a screen recording data stream or another video stream from the second terminal. Therefore, an upper-layer application of the first terminal does not need to perceive whether a displayed image comes from the first terminal or the second terminal. The video stream captured by the second terminal is equivalent to a video stream captured by a remote camera of the first terminal. The first terminal can send the obtained image of the second terminal to the upper-layer application, without a need to modify a manner in which a third-party application invokes a system interface, in other words, without requiring a third-party application to adapt, to expand a use scenario of screen sharing between the first terminal and the second terminal.

The foregoing descriptions of the virtual apparatus architecture of the communication system80are merely an example. It can be understood that a software architecture shown in embodiments of the present invention does not constitute a specific limitation on this application. In some other embodiments of this application, a software architecture of the communication system80may include more or fewer modules than shown in the figure, or combine some modules, or split some modules, or have different architecture arrangements. The components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.

With reference to a schematic diagram of the software module of the communication system80shown inFIG.13, the following describes a procedure of an embodiment from a perspective of a module function.

FIG.14AtoFIG.14Cshow steps of a screen sharing procedure.

S301: A user initiates a video call in a video call application of a first terminal.

The video call application is not limited in this embodiment. The video call application may be a social application with a video chatting function, for example, WeChat, QQ, MeeTime, or DingTalk, or may be a video conference application, for example, ZOOM, welink, or cloudlink.

S302: The video call application invokes a local camera of the first terminal.

S303: The video call application invokes a local microphone of the first terminal.

When a user uses the video call application to make a video call, the video call application may invoke the local camera and the microphone to collect audio and image data, and send the audio and image data to a peer end of the video call.

S304: The user initiates a screen sharing function on the first terminal.

As shown in the embodiment shown inFIG.9A-1andFIG.9A-2, the user may select an “Enable screen sharing” option903after the first terminal establishes a connection to the second terminal, and then the first terminal sends a screen recording start instruction to the second terminal.

The screen sharing function may enable the first terminal to obtain a screen picture of the second terminal, and then the first terminal displays the screen picture of the second terminal. After obtaining the screen picture of the second terminal, the first terminal may send the screen picture of the second terminal to a third terminal, and then the third terminal may display the screen picture of the second terminal. In some embodiments, the first terminal may implement control of a screen of the second terminal.

S305: An audio/video switching module and a policy management module of the first terminal negotiate a screen sharing parameter.

After the audio/video switching module receives a screen sharing start instruction initiated by the user, the audio/video switching module of the first terminal may negotiate the screen sharing parameter with the policy management module, including but not limited to a screen recording, a bitrate of a sound recording module, a resolution of a screen recording, a source of a sound recording (screen audio or audio collected by a microphone), an audio/video encoding/decoding manner, a screen recording area or a specified recording file, a video window rotation policy (a direction of a recorded video is a vertical direction or a horizontal direction), a size of a video recording window, filling a black border for an area that does fills a screen, or the like.

S306: The policy management module of the first terminal delivers a screen sharing audio/video parameter to a policy management module of the second terminal.

S307: The policy management module of the second terminal delivers a screen recording instruction to a screen recording module.

S308: The policy management module of the second terminal delivers an audio recording instruction to a sound recording module.

The policy management module of the second terminal delivers an audio/video stream recording instruction to the screen recording module and the sound recording module based on the negotiated screen sharing audio/video parameter.

S309: The screen recording module of the second terminal uploads a screen recording video stream to the policy management module.

S310: The sound recording module of the second terminal uploads a screen recording audio stream to the policy management module.

S311: The policy management module of the second terminal encodes and combines a screen recording audio-video stream.

After receiving the screen recording video stream and the screen recording audio stream, the policy management module of the second terminal combines and integrates the screen recording video stream and the screen recording audio stream, to generate the screen recording audio-video stream. For example, the second terminal may combine a screen recording video picture and an image captured by a camera of the second terminal into one picture, and integrate the video picture and the image into one audio-video stream.

S312: The policy management module of the second terminal sends the screen recording audio-video stream to a communication module.

S313: The communication module of the second terminal sends the screen recording audio-video stream to a communication module of the first terminal.

S314: The communication module of the first terminal reports the screen recording audio-video stream to the audio/video switching module.

S315: The audio/video switching module of the first terminal delivers an audio switching instruction to the microphone.

S316: The audio/video switching module of the first terminal delivers a video switching instruction to the camera.

The audio/video switching module may separately inject received audio/video content into a camera channel and a microphone channel of the first terminal, and replace, with the audio-video stream received from the second terminal, the audio-video stream captured by the local camera and the microphone of the first terminal.

S317: A microphone management module of the first terminal uploads the audio stream to the video call application.

S318: A camera management module of the first terminal uploads the video stream to the video call application.

As an upper-layer application, the video call application simply invokes the local microphone and the camera, and does not perceive a case in which a lower-layer audio/video stream is replaced with the screen recording audio-video stream of the second terminal.

According to the embodiments shown inFIG.13andFIG.14AtoFIG.14C, the second terminal may quickly share screen interface content (for example, a conference material, a photo, and a video) with the first terminal without a need to install video call application software. The first terminal may transmit the screen recording audio/video of the second terminal to another terminal for viewing. This step is simple, fast, and efficient, and communication efficiency and user communication experience are further improved.

With reference to the foregoing embodiments, the following describes an embodiment of screen sharing between a mobile phone A and a computer A.FIG.15shows a screen sharing process between a mobile phone A and a computer A described in this embodiment.

Step {circle around (1)}: A video call application of the mobile phone A initiates a video call, and establishes a video call channel.

The video call application is not limited in this embodiment. The video call application may be a social application with a video chatting function, for example, WeChat, QQ, MeeTime, or DingTalk, or may be a video conference application, for example, ZOOM, welink, or cloudlink.

In a video call process, the video call application may send received video data to a streaming media engine, and the streaming media engine performs processing and then displays the video data on a display. The streaming media engine may include an encoding function, a decoding function, an image rendering function, an image capturing function, a streaming media protocol, a quality of service (quality of service, QoS) technology, and the like.

Step {circle around (2)}: The mobile phone A establishes a multi-screen collaboration connection to the computer A.

A multi-screen collaboration function is a distributed communication technology. The multi-screen collaboration function may implement collaborative work across operating systems or across devices. For example, after the mobile phone A establishes the multi-screen collaboration connection to the computer A, a function such as screen sharing, resource sharing, and a collaborative operation may be implemented between the mobile phone A and the computer A, to improve data processing efficiency. For example, in this embodiment, after the multi-screen collaboration connection is established, the computer A may share a screen picture of the computer A with the mobile phone A, and the mobile phone A may display the screen picture of the computer A.

Step {circle around (3)}: A user selects a screen sharing media stream on the mobile phone A.

In some embodiments, the user may deliver a screen sharing start instruction on the mobile phone A. The user may further select the screen sharing media stream, that is, a video stream and/or an audio stream of the computer A. For example, the video stream may be an image stream captured by a camera of the computer A, or a screen recording video stream of the computer A, and the audio stream may be a sound collected by a microphone of the computer A, or audio corresponding to a media file played by the computer A.

After the user selects to start screen sharing, the mobile phone A and the computer A may negotiate an audio/video recording parameter. The parameter includes but is not limited to a screen recording, a bitrate of a sound recording module, a recording format, a cache path, a bitrate, a frame rate, a resolution of a screen recording, a source of a sound recording (audio corresponding to a media file or audio collected by a microphone), an audio/video encoding/decoding manner, a screen recording area, a specified recording file, a video window rotation policy (a direction of a recorded video is a vertical direction or a horizontal direction), a size of a video recording window, a policy of filling a black border for an area that does fills a screen, or the like. This is not limited in this embodiment.

Step {circle around (4)}: The user selects, on the computer A, an audio/video stream that needs to be shared.

In some embodiments, the user selects a to-be-shared audio/video stream on the computer A. For example, the user may choose to share a recording video stream of a PC home screen, or may share a specified file, or may choose to combine, into a new audio-video stream for sharing, the picture captured by the camera of the computer A and the recording video stream of the home screen.

Step {circle around (5)}: A codec of the computer A may perform encoding/decoding processing on the audio-video stream, and perform compression.

A media stream encoding and decoding manner is not limited in this embodiment.

Step {circle around (6)}: The computer A sends the audio-video stream to the display transmission module.

Step {circle around (7)}: The computer A transmits the recorded audio-video stream to the mobile phone A.

The computer A is a projection source end, and the mobile phone A is a projection receiving end. The computer A may send screen data of the computer A to the mobile phone A, and the mobile phone A may view a screen interface of the computer A on a screen.

Step {circle around (8)}: After receiving the recorded audio and video of the computer A, the mobile phone A sends the virtual video stream to the video call application.

Specifically, the mobile phone A may enable received audio and video data of the computer A to separately use a camera channel and a microphone channel of the mobile phone A, and then transmit the audio and the video to the upper-layer video call application. As an upper-layer application, the video call application simply invokes a local microphone and a camera, and does not perceive a case in which a lower-layer audio-video stream has changed from an audio-video stream captured by the camera and the microphone of the mobile phone A to the screen recording audio-video stream from a computer A.

When the mobile phone A and the mobile phone B make a video call by using the video call application, the mobile phone A may transmit the received screen picture of the computer A to the mobile phone B by using the video call application through a connection. The audio-video stream transmitted by the mobile phone A to the video call application is media stream data from the local camera channel and the microphone channel. The mobile phone A enables a virtual audio-video stream from the computer A to occupy the local camera channel and the microphone channel. Therefore, a picture and a sound that are from the mobile phone A and that are viewed on the mobile phone B should be an image captured by the camera of the mobile phone A and a sound collected by the microphone, but the picture and the sound are replaced with the audio-video stream from the computer A.

In addition, the mobile phone A may further edit the received screen picture recorded by the computer A, to help the user change display effect of the screen picture on a side of the mobile phone A, for example, perform an operation such as scrawling and commenting on the screen picture. The user may alternatively select, on the side of the mobile phone A, a source of an audio stream to be transmitted to the mobile phone B. For example, the audio stream may be the sound collected by the microphone of the computer A, or the sound collected by the microphone of the mobile phone A, or audio corresponding to a media file on the computer A, or audio corresponding to a media file on the mobile phone A. For details, refer to the foregoing embodiments. Details are not described herein again.

According to the embodiment shown inFIG.15, the computer A may quickly share screen interface content (for example, a conference material, a photo, and a video) with the mobile phone A without a need to install video call application software. The mobile phone A may transfer, to another terminal for viewing by using the video call application software, the audio and video recorded by the computer A. This step is simple, fast, and efficient, and communication efficiency and user communication experience are further improved.

Implementations described in the foregoing embodiments are merely example descriptions, and constitute no limitation on another embodiment of this application. A specific internal implementation may vary with different types of electronic devices, different used operating systems, different used programs, and different invoked interfaces. Embodiments of this application constitute no limitation, and a feature function described in embodiments of this application may be implemented.

As used in embodiments, depending on the context, the term “when” can be interpreted as “if . . . ”, “after . . . ”, “in response to determining that . . . ”, or “in response to detecting that . . . ”. Similarly, depending on the context, the phrase “when it is determined that . . . ” or “if it is detected that . . . (a stated condition or event)” can be interpreted as “if it is determined that . . . ”, “in response to determining that . . . ”, “when it is detected that . . . (the stated condition or event)”, or “in response to detecting that . . . (the stated condition or event)”.

All or some of the foregoing embodiments may be implemented by using software, hardware, firmware, or any combination thereof. When software is used to implement the embodiments, all or a part of the embodiments may be implemented in a form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the procedure or functions according to embodiments of this application are all or partially generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable apparatuses.