Systems and methods for implementing privacy filters with variable obfuscation for video communications

In one embodiment, a method includes maintaining a video communication between two or more client devices with each client device comprising cameras and being associated with a respective video stream in the video communication, determining scene data within a field of view in a real-world environment captured by one or more of the cameras of a first client device of the two or more client devices, determining a privacy filter to apply to a first video stream associated with the first client device based on the scene data, and providing instructions to apply the privacy filter to the first video stream in the video communication.

TECHNICAL FIELD

This disclosure relates generally to database and file management within network environments, and in particular relates to video communications.

BACKGROUND

Standard video call systems are commonplace. They have a fixed start time and end time, and during this period they transmit video and audio between participants (either which can be enabled or disabled). They are used frequently between friends, family members, business calls (one to one), business meetings (group), and presentations (one to many). There are many different platforms with different features tailored to the use case, e.g., replacing a person's video with an avatar on a friend-focused platform, applying noise suppression in a gaming focused platform, or applying a virtual background to enhance privacy in a business focused platform. Some new always-on video communication systems are emerging that aim to avoid the standard video call start and end structure. These systems are primarily aimed at collaborative workplaces with the goal of reducing barriers between coworkers communicating.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Client System Overview

FIG.1illustrates an example electronic device100. In particular embodiments, the electronic device100may include, for example, any of various personal electronic devices102, such as a mobile phone electronic device, a tablet computer electronic device, a laptop computer electronic device, and so forth. In particular embodiments, as further depicted byFIG.1, the personal electronic device102may include, among other things, one or more processor(s)104, memory106, sensors108, cameras110, a display112, input structures114, network interfaces116, a power source118, and an input/output (I/O) interface120. It should be noted thatFIG.1is merely one example of a particular implementation and is intended to illustrate the types of components that may be included as part of the electronic device100.

In particular embodiments, the one or more processor(s)104may be operably coupled with the memory106to perform various algorithms, processes, or functions. Such programs or instructions executed by the processor(s)104may be stored in any suitable article of manufacture that includes one or more tangible, computer-readable media at least collectively storing the instructions or routines, such as the memory106. The memory106may include any suitable articles of manufacture for storing data and executable instructions, such as random-access memory (RAM), read-only memory (ROM), rewritable flash memory, hard drives, and so forth. Also, programs (e.g., an operating system) encoded on such a computer program product may also include instructions that may be executed by the processor(s)104to enable the electronic device100to provide various functionalities.

In particular embodiments, the sensors108may include, for example, one or more cameras (e.g., depth cameras), touch sensors, microphones, motion detection sensors, thermal detection sensors, light detection sensors, time of flight (ToF) sensors, ultrasonic sensors, infrared sensors, or other similar sensors that may be utilized to detect various user inputs (e.g., user voice inputs, user gesture inputs, user touch inputs, user instrument inputs, user motion inputs, and so forth). The cameras110may include any number of cameras (e.g., wide cameras, narrow cameras, telephoto cameras, ultra-wide cameras, depth cameras, and so forth) that may be utilized to capture various 2D and 3D images. The display112may include any display architecture (e.g., AMLCD, AMOLED, micro-LED, and so forth), which may provide further means by which users may interact and engage with the electronic device100. In particular embodiments, as further illustrated byFIG.1, one more of the cameras110may be disposed behind, underneath, or alongside the display112(e.g., one or more of the cameras110may be partially or completely concealed by the display112), and thus the display112may include a transparent pixel region and/or semi-transparent pixel region through which the one or more concealed cameras110may detect light, and, by extension, capture images. It should be appreciated that the one more of the cameras110may be disposed anywhere behind or underneath the display110, such as at a center area behind the display110, at an upper area behind the display110, or at a lower area behind the display110.

In particular embodiments, the input structures114may include any physical structures utilized to control one or more global functions of the electronic device100(e.g., pressing a button to power “ON” or power “OFF” the electronic device100). The network interface116may include, for example, any number of network interfaces suitable for allowing the electronic device100to access and receive data over one or more cloud-based networks (e.g., a cloud-based service that may service hundreds or thousands of the electronic device100and the associated users corresponding thereto) and/or distributed networks. The power source118may include any suitable source of power, such as a rechargeable lithium polymer (Li-poly) battery and/or an alternating current (AC) power converter that may be utilized to power and/or charge the electronic device100for operation. Similarly, the I/O interface120may be provided to allow the electronic device100to interface with various other electronic or computing devices, such as one or more auxiliary electronic devices.

Implementing Privacy Filters with Variable Obfuscation for Video Communications

Existing video call systems (including always-on systems) focus primarily on full-engagement calls and may not provide suitable features for calls where participants may be less engaged. As a result, the participants may have limited privacy options. These options may be usually limited to merely “on” and “off” for the video and the audio. Existing systems may implement timers, fixed schedules, or accessible controls to specify when video and audio are “on” and “off”. These features may not effectively address the privacy issues associated with low engagement video calls, such as users feeling as though they are being watched when at their desks but not interacting with anyone, inadvertently oversharing due to forgetting about the video call, etc. To address the aforementioned issues of existing systems, the embodiments disclosed herein may enable much greater flexibility in the level of information transmitted (and hence privacy) in video communications systems. The embodiments disclosed herein may be employed to significantly improve the always-on video call experience, or any other video communications where variable privacy is desired. In particular embodiments, a video communication system disclosed herein may include physical camera privacy filters that allow for variable obfuscation of the camera's view by physically moving one or more elements. The video communication system may also include physical camera privacy filters that allow for variable obfuscation of the camera's view with an element that can change transparency in response to electrical signals, wherein at least four different electrical signals may be applied to produce at least four corresponding levels of transparency. The video communication system may additionally provide presence visualizations, which may comprise visually appealing animations that convey basic activity or presence information between connected users without sharing video. In particular embodiments, the video communication system may also use a range of software-based video and audio obfuscation methods to further enhance the privacy for users.

In particular embodiments, the video communication system may maintain a video communication between two or more client devices. Each client device may comprise one or more cameras and each client device may be associated with a respective video stream in the video communication. In particular embodiments, the video communication system may determine scene data within a field of view in a real-world environment captured by one or more of the cameras of a first client device of the two or more client devices. The video communication system may then determine, based on the scene data, a privacy filter to apply to a first video stream associated with the first client device. The video communication system may further provide instructions to apply the privacy filter to the first video stream in the video communication.

Certain technical challenges exist for implementing privacy filters. One technical challenge may include using appropriate presence visualization as privacy filters. The solution presented by the embodiments disclosed herein to address this challenge may be determining privacy filters based on various data fields capturing participants' engagement with a video communication as such data fields are effective in evaluating how much privacy the participants may need during the video communication. Another technical challenge may include effectively determining activity level of users in a video communication. The solution presented by the embodiments disclosed herein to address this challenge may be detecting people within the field of view based on a person detection model, determining their poses based on a pose detection model, and determining the level of activity based on the poses as such progressing approach detects discriminating information at different stages that are helpful for determining the level of activity.

Certain embodiments disclosed herein may provide one or more technical advantages. A technical advantage of the embodiments may include significantly improving the always-on video call experience, or any other video communications where variable privacy is desired by using various techniques including physical camera privacy filters that allow for variable obfuscation, presence visualizations, and software-based obfuscation methods. Another technical advantage of the embodiments may include clear visibility of the level of obfuscation to users by using physical camera privacy filters as the users can see particularly element (e.g., partially opaque element) in front of the camera to expect that camera's view may not be clear. Another technical advantage of the embodiments may include interpretable user engagement even with privacy filters as the presence visualization may have different patterns indicating different level of engagement. Certain embodiments disclosed herein may provide none, some, or all of the above technical advantages. One or more other technical advantages may be readily apparent to one skilled in the art in view of the figures, descriptions, and claims of the present disclosure.

In particular embodiments, the privacy filter may be a physical camera privacy filter comprising one or more of a movable element with transparency gradient, a semi-transparent layer, a movable patterned element with transparency gradient, a displacement of lenses associated with a camera, or two or more sheets with polarizing filters. The physical camera privacy filters may be elements that physically move within the camera's optical path, which may be used to adjust the clarity of the camera's view and hence provide privacy.FIG.2illustrates example locations200of the physical camera privacy filters. The physical camera privacy filters may be located in front of the camera (e.g., front glass210), in front of the image sensor230, or anywhere in between (e.g., lenses220).

In particular embodiments, the physical camera privacy filter may comprise a moving element with transparency gradient.FIG.3illustrates example moving elements310/320with transparency gradient. The element310/320may be transparent at one end, opaque at the other end. The element310/320may have a transparency gradient between the two ends. In particular embodiments, the element320with the transparency gradient may be textured or patterned in order to apply a desired visual appearance to the camera's view. The element310/320may physically move in front of the camera to obscure the view to some degree. The element310/320may be moved by hand, spring, electromechanical actuator, or other method.

In particular embodiments, a video communication device associated with the video communication system may have multiple cameras, with one camera being permanently obfuscated to some degree. The method of obfuscated may be a semi-transparent layer in front of the lens or lenses that result in the camera being out of focus. The non-obfuscated camera may be covered entirely when privacy is desired, such that the device may only see through the obfuscated camera.FIG.4illustrates an example permanent privacy filter. The camera410may have a semi-transparent layer420that may result in the obfuscation430.

In particular embodiments, one or more camera lenses may be displaced out of the usual range to force the camera view out of focus, such that the camera is unable to refocus. This may result in a blurring and/or warping of the image. The lens(es) may be moved by hand, spring, electromechanical actuator, or other method. The displacement direction may be along the optical axis, or normal to the optical axis.FIG.5illustrates an example displacement and a corresponding image. The lenses510of the camera520may displace along the direction530between the front glass540and the sensor550. When displacement occurs, the corresponding image560may be out of focus.

In particular embodiments, two sheets with polarizing filters may be mounted in front of the camera. One of the sheets may be rotated to control the transparency of the camera's view. When the polarizing direction of the two sheets are aligned, the camera's view is clear. As the polarizing direction of the two sheets approaches 90 degrees, the filters may approach 0% transparency.FIG.6illustrates example polarizing filters. When the polarizing direction of sheet610and that of sheet620is 90 degrees, the filters may have no transparency630. When the polarizing direction of sheet610and that of sheet620is aligned, the filters may have complete transparency640. By using physical camera privacy filters, the embodiments disclosed herein may have a technical advantage of clear visibility of the level of obfuscation to users as the users can see particularly element (e.g., partially opaque element) in front of the camera to expect that camera's view may not be clear.

In particular embodiments, the privacy filter may be a physical element placed in an optical path associated with a camera. The physical element may be operable to change its transparency to at least four different transparency levels. The privacy filter may be an electrically controlled, variable transparency camera privacy filter. An element could be placed in the camera's optical path that is able to change its transparency in response to an electrical signal. The obfuscation of the camera's view could then be altered as desired via electronic control. The control will allow for a minimum of four different steps of transparency levels. The elements may be located in front of the camera, in front of the image sensor, or anywhere in between. In particular embodiments, the element may be based on one or more of polymer dispersed liquid crystal (PDLC), polymer stabilized liquid crystal (PSLC), electrochromic, electrophoresis, electrowetting, or smectic A (SmA) liquid crystal. Polymer dispersed liquid crystal may comprise liquid crystal (LC) droplets surrounded by a polymer mixture where the concentration of both is about equal between two pieces of conducting glass or plastic. An applied electric field may align the LC to create transparent region on command. Absence of electric field may result in random LC orientation opaque appearance. For polymer stabilized liquid crystal, the concentration of polymer may be less than 10% of the liquid crystals. Adding the polymers to a liquid crystal to may form a phase-separated PSLC mixture that creates differently oriented domains of the liquid crystal, and it may scatter light from those domains, and the size of those domains determines the intensity of scattering. Electrochromic material may change its opacity between a translucent state (usually blue) and a transparent state due to the electrochemical redox reactions that takes place in electrochromic materials in response to voltage and thus allowing control over the amount of light passing through. Electrowetting may comprise modification of the wetting properties of a surface with an applied electric field, which may allow an opaque material to selectively block light passing. The bistable smectic a liquid crystal may work by creating or erasing a dense field of light-scattering focal conic domains. Applying a low frequency electric field, the LC may convert to a turbulent light-scattering state. Applying high frequency electric field, homeotropic (HT) texture may be generated because of the dielectric re-orientation of the smectic A LC.

FIGS.7A,7B,7C and7Dillustrate example PDLC with four levels and their corresponding levels of obfuscation to a camera.FIG.7Aillustrates an example PDLC with a first level710and a corresponding level of obfuscation to the camera715.FIG.7Billustrates an example PDLC with a second level720and a corresponding level of obfuscation to the camera725.FIG.7Cillustrates an example PDLC with a third level730and a corresponding level of obfuscation to the camera735.FIG.7Dillustrates an example PDLC with a fourth level740and a corresponding level of obfuscation to the camera745.

In particular embodiments, the privacy filter may be a presence visualization based on one or more of a shape, a motion, a shade, or an animation. Presence visualization may have elements that may change in response to certain data fields of the users who are in the video communication. However, presence visualizations may not present the camera stream. Instead, they may be intended to convey a sense of presence between users when high levels of privacy are desired, in a visually appealing manner. In particular embodiments, the scene data may comprise one or more of a level of activity, a sound in the real-world environment, a presence of a person within the field of view, an absence of a person within the field of view, a location of the real-world environment, an engagement of a person with respect to the video communication, a facial expression of a person within the field of view, a gesture of a person within the field of view, a pose of a person within the field of view, a time at the real-world environment, or weather at the real-world environment. Determining privacy filters based on various data fields capturing participants' engagement with a video communication may be an effective solution to address the technical challenge of using appropriate presence visualization as privacy filters as such data fields are effective in evaluating how much privacy the participants may need during the video communication. In particular embodiments, the aforementioned data fields may be obtained from a range of sensors, which may include but are not limited to camera(s), microphone(s), lidar sensor which uses lasers to perform many distance measurements at a rapid rate to provide a point cloud (usually too sparse to resolve any fine details) in the three-dimensional (3D) space, radar (uses much longer wavelengths than lidar) which emits and receives radar waves to produce depth maps with very low levels of detail, ultrasound sensor which uses acoustic waves to measure the distance to objects, event camera (a.k.a. dynamic vision sensor) which is an optical camera that reports only changes in brightness on a per-pixel basis but cannot capture images, infrared sensors which passively sense infrared energy as people are usually higher temperature than the environment and can be detected by their increase in infrared energy. In particular embodiments, non-camera sensors may be advantageous as they may continue to function if a user has physically covered the camera.

In particular embodiments, presence visualizations may be generated using one or more methods within the fields of two-dimensional (2D) graphics, 3D graphics, VFX (i.e., parallax effect), and others.FIG.8illustrates example presence visualizations. Visualizations similar to the aurora borealis are generated. The color tone, detail and locations of the lights may change to represent the presence of a person. A certain area and a certain color may correspond to each user. If activity or movement is detected, the animation may become more dynamic and attention-grabbing. To be more specific, visualizations810,820,830, and840may indicate that persons or activity are not detected whereas visualizations815,825,835, and845may indicate that persons or activity are detected.FIGS.9A-9Cillustrate another example presence visualization. The presence visualization may represent each user as a circle.FIG.9Aillustrates presence visualization of three users910-930.FIG.9Billustrates presence visualization of four users910-940. The color, size and glowing effect may change in response to the presence and activity level of each user.FIG.9Cillustrates presence visualization based on activity levels. For example, users920and940may be both passive.

In particular embodiments, the presence visualization may be a passive visualization. As an example and not by way of limitation, the presence visualization may be similar to the appearance of curtains. The presence visualization may comprise one section to represent each connected client device.FIG.10illustrates example presence visualizations for three users. Presence visualizations1010-1030may represent three client devices, respectively. From each connected device, the video communication system may receive data about the locations of any detected persons as well as their activity level. This data may be then used to animate the visualization. When a connected device reports a person location, a corresponding area of the visualization may be shaded in a dynamic and abstract shape. When the reported person location moves, the shaded area may move correspondingly. The person's activity level may determine how prevalent, dynamic, and attention-grabbing the shaded area is.FIG.11illustrates example behavior for a presence visualization for one user. Presence visualization1110may display no shaded area as the received data indicates that no persons are detected. Presence visualization1120may display a subtle animation as the received data indicates unchanging person location. Presence visualization1130may display more dynamic motion as the received data indicates that the person is active. Presence visualization1140may display shaded area moves as the received data indicates that the person location is moving across the frame. The embodiments disclosed herein may have another technical advantage of interpretable user engagement even with privacy filters as the presence visualization may have different patterns indicating different level of engagement.

FIG.12illustrates an example block diagram1200for presence visualization. As an example implementation, assume that multiple users (users A1210, B1220, and C1230) are connected via the video communication system who have all indicated that they want a high level of privacy. At step1212, user A's1210camera may capture a video stream. At step1214, the video communication system may analyze the captured video stream to determine a small number of outputs, e.g., the number of people in view, their approximate locations, and their activity level. The same analysis may be performed for user B1220through step1222to step1224and for user C1230through step1232to step1234. The analysis outputs may be shared between all connected client devices. At step1216, step1226, or step1236, the video communication system may generate a visualization that represents some or all of the analysis outputs that were generated. At step1218, step1228, or step1238, the video communication system may display the visualization to each user. In the example block diagram1200, the analysis and visualization generation may be performed locally on each client device.FIG.13illustrates another example block diagram1300for presence visualization. Different from the block diagram1200, in this block diagram1300, analysis1315and visualization generation1320are performed on a server1310. An advantage of this approach may be that less processing power is required at each client device.

In particular embodiments, the video communication system may analyze the captured video stream (e.g., steps1214/1224/1234in the block diagram1200and step1315in the block diagram1300) by processing a given video stream to extract data fields. As an example and not by way of limitation, such data fields may include, but are not limited to, person activity level, presence or absence of persons, locations of persons, poses of persons, and gestures. In particular embodiments, the video communication system may detect, based on a person detection model, one or more people within the field of view. The video communication system may then determine, based on a pose detection model, one or more poses of one or more of the detected people. The video communication system may further determine the scene data based on the determined poses of one or more of the detected people.FIG.14illustrates an example block diagram1400for analyzing video streams. At step1405, the camera may capture a video stream. At step1410, the video communication system may extract an image or frame. At step1415, the video communication system may run a person detection model. At step1420a, the video communication system may determine the person count. At step1420b, the video communication system may determine the person location data. At step1425, the video communication system may run a pose detection model on persons. At step1430, the video communication system may generate person pose landmark data. At step1435a, the video communication system may calculate total movement of pose. At step1440, the video communication system may determine the activity level. At step1435b, the video communication system may detect gestures from poses. At step1445, the video communication system may obtain the detected gestures. At step1450, the video communication system may generate the output data which may comprise the image or frame, the person count, the person location data, the activity level, and the detected gestures. Detecting people within the field of view based on a person detection model, determining their poses based on a pose detection model, and determining the level of activity based on the poses may be an effective solution for addressing the technical challenge of effectively determining activity level of users in a video communication as such progressing approach detects discriminating information at different stages that are helpful for determining the level of activity.

As previously described, the video communication system may run a person detection model. Human detection is a common task in computer vision applications. Many models exist to achieve this, most of which may be based on neural networks and output a rectangle around each detected person.FIG.15Aillustrates an example detection of a person. A rectangle1510is around the person1520. When running the pose detection model, the video communication system may determine the locations of the person's limbs and joints, usually as a collection of landmarks1530in either 2D or 3D coordinates.FIG.15Billustrates example locations of the person's limbs and joints as landmarks in 3D coordinates1540. In particular embodiments, the person detection model and the pose detection model may be combined in a single model. After the human poses are known, the video communication system may detect gestures by analyzing the limb and joint positions. As an example and not by way of limitation, a raised arm may be detected if the angle from the shoulder to the elbow is within 20 degrees of vertically “up” and the elbow joint is within 20 degrees of “straight”.

In alternative embodiments, the video communication system may additionally or alternatively use one or more software-based video obfuscation methods to provide flexible control over the amount of information transmitted via the video stream. In particular embodiments, applying the privacy filter results in an obfuscation of the field of view based on one or more of a transformation of the field of view, an overlay to the field of view, a manipulation of the one or more of the cameras to obfuscate the field of view, or a partial or complete replacement of the first video stream with one or more virtual elements. In one embodiment, the video communication system may apply a filter or transformation and/or overlay to the video stream. Examples may include, but not limited to, gaussian blur, vertical/horizontal blur, motion blur, mosaic effects, pixilation, hue change, saturation change, etc. Any filter, transformation or artistic effect may be applied.FIG.16Aillustrates an example flow diagram for applying a filter or transformation and/or overlay to a video stream. The video communication system may use camera hardware1610to get source image from camera1620. The video communication system may then apply a function1630to generate the new image output1640.FIG.16Billustrates example applications of filters. Image1650may be the original image. Image1660a-1660emay indicate the applications of different example filters. In another embodiment, the video communication system may manipulate the camera to intentionally obfuscate the view.FIG.17Aillustrates an example flow diagram for using the camera's focus system to intentionally defocus the camera. The video communication system may use camera hardware1710with a first focus distance to generate a source image output1720. The video communication system may use camera hardware with a second focus distance1730to generate another source image output1740.FIG.17Billustrates example defocused images. Image1750may be the original image. Image1760may indicate the application of a first focus distance. Image1770may indicate the application of a second focus distance.FIG.18Aillustrates an example flow diagram for panning, tilting, and zooming a video stream. The video communication may pan and/or crop the video stream to an unimportant view (which may intentionally avoid the users). The video communication system may use camera hardware1810with a first pan, tilt and zoom to generate a source image output1820at the target area. The video communication system may use camera hardware with a second pan, tilt and zoom1830to generate another source image output1840at the target area.FIG.18Billustrates example panned, tilted, or zoomed images. Image1850may be the original image. Image1860may indicate the application of a first pan, tilt and zoom. Image1870may indicate the application of a second pan, tilt and zoom. In yet another embodiment, the video communication system may replace some or all video stream with virtual elements, such as an avatar.FIG.19Aillustrates an example flow diagram for replacing a user with an avatar. The video communication system may use camera hardware1910to get source image from camera1920. The video communication system may then apply a layer1930to generate the new image output1940.FIG.19Billustrates an example image with a replacement of an avatar. Image1950may be the original image. Image1960may be the image where the subject is removed. Image1970may be the image where the avatar is added to replace the subject.FIG.20Aillustrates an example flow diagram for replacing the entire video with a virtual representation. The video communication system may use camera hardware2010to get source image from camera2020. The video communication system may then match the data fields and data style2030to generate the new image output2040.FIG.20Billustrates an example image with a virtual replacement. Image2050may be the original image. Image2060may be the image where there is a virtual replacement for the entire image.

In alternative embodiments, the video communication system may additionally or alternatively use audio obfuscation methods. Always-on video calls may also benefit from flexible control over the information transmitted between devices via the audio. Existing system may only allow for muting and volume control. The video communication system disclosed herein may have the ability to obfuscate the audio such that it conveys some audio activity yet does not transmit comprehensible voice. This mode may maintain some connection between users (as desired by always-on video), without inadvertently transmitting private conversations or audio that may be a nuisance to the recipient. As an example and not by way of limitation, modifications that may be applied to the audio to achieve this effect may be, but not limited to, reducing the amplitude (volume), applying a low-pass filter, applying reverberation which adds an echo effect to the audio and may both obfuscate the audio and convey a sense of distance.

FIG.21illustrates example applications of privacy filters. The privacy filters described herein may be applied on any device that supports video communication. Such devices may have a range of form factors. Options may include, but not limited to, software on a mobile device2110, software on a television unit2120optionally with an attached camera unit, software on a television-connected platform2130optionally with an attached camera unit, an all-in-one dedicated device which connects to a TV2140, a dedicated device with an attached camera unit2150, etc. The embodiments disclosed herein may have a technical advantage of significantly improving the always-on video call experience, or any other video communications where variable privacy is desired by using various techniques including physical camera privacy filters that allow for variable obfuscation, presence visualizations, and software-based obfuscation methods.

FIG.22illustrates is a flow diagram of a method2200for implementing privacy filters, in accordance with the presently disclosed embodiments. The method2200may be performed utilizing one or more processing devices (e.g., the electronic device100) that may include hardware (e.g., a general purpose processor, a graphic processing unit (GPU), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a microcontroller, a field-programmable gate array (FPGA), a central processing unit (CPU), an application processor (AP), a visual processing unit (VPU), a neural processing unit (NPU), a neural decision processor (NDP), or any other processing device(s) that may be suitable for processing 2D and 3D image data, software (e.g., instructions running/executing on one or more processors), firmware (e.g., microcode), or some combination thereof.

The method2200may begin at step2210with the one or more processing devices (e.g., the electronic device100). For example, in particular embodiments, the electronic device100may maintain a video communication between two or more client devices, wherein each client device comprises one or more cameras, and wherein each client device is associated with a respective video stream in the video communication. The method2200may then continue at step2220with the one or more processing devices (e.g., the electronic device100). For example, in particular embodiments, the electronic device100may determine scene data within a field of view in a real-world environment captured by one or more of the cameras of a first client device of the two or more client devices, wherein the determination comprises detecting one or more people within the field of view based on a person detection model, determining one or more poses of one or more of the detected people based on a pose detection model, and determining the scene data based on the determined poses of one or more of the detected people. The method2200may then continue at step2230with the one or more processing devices (e.g., the electronic device100). For example, in particular embodiments, the electronic device100may determine a privacy filter to apply to a first video stream associated with the first client device based on the scene data comprising one or more of a level of activity, a sound in the real-world environment, a presence of a person within the field of view, an absence of a person within the field of view, a location of the real-world environment, an engagement of a person with respect to the video communication, a facial expression of a person within the field of view, a gesture of a person within the field of view, a pose of a person within the field of view, a time at the real-world environment, or weather at the real-world environment, wherein the privacy filter is a physical camera privacy filter comprising one or more of a movable element with transparency gradient, a semi-transparent layer, a movable patterned element with transparency gradient, a displacement of lenses associated with a camera, or two or more sheets with polarizing filters, wherein the privacy filter is a physical element placed in an optical path associated with a camera, and wherein the physical element is operable to change its transparency to at least four different transparency levels, wherein the privacy filter is a presence visualization based on one or more of a shape, a motion, a shade, or an animation. The method2200may then continue at block2240with the one or more processing devices (e.g., the electronic device100). For example, in particular embodiments, the electronic device100may provide instructions to apply the privacy filter to the first video stream in the video communication, wherein applying the privacy filter results in an obfuscation of the field of view based on one or more of a transformation of the field of view, an overlay to the field of view, a manipulation of the one or more of the cameras to obfuscate the field of view, or a partial or complete replacement of the first video stream with one or more virtual elements. Particular embodiments may repeat one or more steps of the method ofFIG.22, where appropriate. Although this disclosure describes and illustrates particular steps of the method ofFIG.22as occurring in a particular order, this disclosure contemplates any suitable steps of the method ofFIG.22occurring in any suitable order. Moreover, although this disclosure describes and illustrates an example method for implementing privacy filters including the particular steps of the method ofFIG.22, this disclosure contemplates any suitable method for implementing privacy filters including any suitable steps, which may include all, some, or none of the steps of the method ofFIG.22, where appropriate. Furthermore, although this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method ofFIG.22, this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method ofFIG.22.

Systems and Methods

FIG.23illustrates an example computer system2300that may be utilized to perform implementing privacy filters, in accordance with the presently disclosed embodiments. In particular embodiments, one or more computer systems2300perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems2300provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems2300performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems2300. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems2300. This disclosure contemplates computer system2300taking any suitable physical form. As example and not by way of limitation, computer system2300may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (e.g., a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, computer system2300may include one or more computer systems2300; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks.

Where appropriate, one or more computer systems2300may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example, and not by way of limitation, one or more computer systems2300may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems2300may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

In particular embodiments, computer system2300includes a processor2302, memory2304, storage2306, an input/output (I/O) interface2308, a communication interface2310, and a bus2312. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement. In particular embodiments, processor2302includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor2302may retrieve (or fetch) the instructions from an internal register, an internal cache, memory2304, or storage2306; decode and execute them; and then write one or more results to an internal register, an internal cache, memory2304, or storage2306. In particular embodiments, processor2302may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor2302including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor2302may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory2304or storage2306, and the instruction caches may speed up retrieval of those instructions by processor2302.

Data in the data caches may be copies of data in memory2304or storage2306for instructions executing at processor2302to operate on; the results of previous instructions executed at processor2302for access by subsequent instructions executing at processor2302or for writing to memory2304or storage2306; or other suitable data. The data caches may speed up read or write operations by processor2302. The TLBs may speed up virtual-address translation for processor2302. In particular embodiments, processor2302may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor2302including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor2302may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors2302. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

In particular embodiments, memory2304includes main memory for storing instructions for processor2302to execute or data for processor2302to operate on. As an example, and not by way of limitation, computer system2300may load instructions from storage2306or another source (such as, for example, another computer system2300) to memory2304. Processor2302may then load the instructions from memory2304to an internal register or internal cache. To execute the instructions, processor2302may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor2302may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor2302may then write one or more of those results to memory2304. In particular embodiments, processor2302executes only instructions in one or more internal registers or internal caches or in memory2304(as opposed to storage2306or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory2304(as opposed to storage2306or elsewhere).

One or more memory buses (which may each include an address bus and a data bus) may couple processor2302to memory2304. Bus2312may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor2302and memory2304and facilitate accesses to memory2304requested by processor2302. In particular embodiments, memory2304includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory2304may include one or more memory devices2304, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

In particular embodiments, storage2306includes mass storage for data or instructions. As an example, and not by way of limitation, storage2306may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage2306may include removable or non-removable (or fixed) media, where appropriate. Storage2306may be internal or external to computer system2300, where appropriate. In particular embodiments, storage2306is non-volatile, solid-state memory. In particular embodiments, storage2306includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage2306taking any suitable physical form. Storage2306may include one or more storage control units facilitating communication between processor2302and storage2306, where appropriate. Where appropriate, storage2306may include one or more storages2306. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

In particular embodiments, I/O interface2308includes hardware, software, or both, providing one or more interfaces for communication between computer system2300and one or more I/O devices. Computer system2300may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system2300. As an example, and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces2306for them. Where appropriate, I/O interface2308may include one or more device or software drivers enabling processor2302to drive one or more of these I/O devices. I/O interface2308may include one or more I/O interfaces2306, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.

In particular embodiments, communication interface2310includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system2300and one or more other computer systems2300or one or more networks. As an example, and not by way of limitation, communication interface2310may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface2310for it.

Miscellaneous

Herein, “automatically” and its derivatives means “without human intervention,” unless expressly indicated otherwise or indicated otherwise by context.