Patent Description:
<CIT> describes a method which includes receiving, by an asset privacy system, an input video feed that includes images of one or more assets located within a data center, identifying, by the asset privacy system, one or more corresponding assets for the images of the one or more assets, determining, by the asset privacy system, an asset of the one or more assets a customer is not authorized to view, and generating, from the input video feed, by the asset privacy system based at least on the determining, an output video feed that does not include images of the asset the customer is not authorized to view and that does include images of at least one other asset of the one or more assets.

The present invention is defined by the features disclosed in the independent claims. Additional embodiments are described in the dependent claims.

Content may be streamed from a source device to a destination device. For example, the source device may cause live video data, of an environment associated with the source device (e.g., a room in which the source device is located) to be streamed to the destination device. In some instances, the live video data may include materials and/or objects (e.g., located in the environment) that are not intended to be displayed publicly. For example, the materials and/or objects may include confidential and/or proprietary materials and/or objects that may be visible within the live video data. The live video data may enable unauthorized access to the confidential and/or proprietary materials and/or objects to a user associated with the destination device and/or to a user that uses a device to intercept the live video data. Unauthorized access to the confidential and/or proprietary materials and/or objects creates data security concerns and causes an invasion of privacy.

In various implementations, a source device with spatial location technology (e.g., AR/VR device and/or a smartphone outfitted with such capabilities) identifies, within video data, a target indicator (e.g., a QR code or other type of unique or preconfigured object). The target indicator may indicate a location of a target zone of a physical environment that may be rendered on a display. For example, a visual aid or object may be configured to be positioned within the target zone to permit all or a portion of the target zone to be rendered (e.g., depending on the location of the user). Based on the detection of the target indicator and a determined spatial position of the source device relative to the target indicator, display data is provided to a destination device that causes a rendering of the physical environment on a display of the destination device to include a masking associated with the target zone.

In this way, security of a physical environment may be maintained to prevent fraud (thereby conserving resources associated with addressing fraudulent activity that might otherwise occur), prevent the need for a user to go to a dedicated space to show a particular object (e.g., the object may be on a manufacturing line or in an area that is inherently surrounded by confidential information and the confidential materials or objects cannot easily be moved), and/or maintain privacy associated with a portion of a physical environment, among other examples. Further, masking or obfuscating content of the video data can reduce an amount of data that is to be processed to depict a physical environment that otherwise may be irrelevant to the video stream.

<FIG> and <FIG> are diagrams of an example implementation <NUM> and <NUM>, respectively, associated with obfuscation of areas of a physical environment depicted in a video. As shown, example implementations <NUM> and <NUM> include a source device having a camera, a content management system, and a destination device. In some implementations, the content management system may be implemented using one or more server devices that are remote from (e.g., physically separate from) the source device. In some implementations, the content management system may be implemented on the source device and/or one or more server devices. These devices are described in more detail below in connection with <FIG> and <FIG>.

As shown in <FIG> by reference number <NUM>, video data is captured using the camera. The video data includes a captured scene that depicts a physical environment. The physical environment may include any type of physical environment such as, for example, a room (e.g., a product testing room, an office, a manufacturing floor), a hallway, an outdoor region, and/or a region within a vehicle, among other examples. As shown by reference number <NUM>, the captured video data may be provided to the content management system. The video data may be associated with a video stream that is being streamed from the source device to the destination device. Streaming media (e.g., video streaming) involves delivering content and continually presenting the content to an end-user. A media stream can include a live stream, a look-up stream, and/or the like. In a live stream, content is provided to a destination device without saving the content (e.g., using memory resources). In a look-up stream, content can be saved and/or buffered (e.g., temporarily stored) in a look-up storage prior to being provided to a destination device.

As shown by reference number <NUM>, the content management system detects a target indicator associated with a target zone of the physical environment. A target zone may include an area or volume in the physical environment that surrounds a target object. For example, the target zone may include a volume defined by a shape that is configured to correspond to the target object. In some implementations, for example, one or more target zones may correspond to one or more target objects. In some implementations, the target indicator may include a barcode or other object placed in the physical environment and that includes a code that may be read by a computer. For example, the target indicator may include a quick response (QR) code and/or a parallel line code, among other examples.

In some implementations, the target indicator may include a type of object that is detectable within the physical environment by an image processing model that is configured to analyze the video data to detect that type of object. For example, the type of object may include confidential objects (objects that are not to be displayed as part of the streamed video), the target object, and/or other unique, identifiable objects. In some implementations, prior to detecting the target indicator, the content management system may process the video data according to an image processing model that may be configured to detect one or more target indicators. Each target indicator may be associated with a respective target zone of the physical environment.

As shown by reference number <NUM>, the content management system determines a target zone configuration. A target zone configuration may include any number of different characteristics of a target zone. The characteristics may include, for example, a position of the target zone relative to the target indicator. The position may include, for example, a center position of the target zone, one or more coordinates of the target zone, a position of an edge or corner of the target zone, and/or a position of a side of the target zone, among other examples. The characteristics may additionally or alternatively include at least one of dimensions of the target zone relative to the target indicator; dimensions of the target zone relative to the position of the target zone; a shape of the target zone relative to the target indicator; or a shape of the target zone relative to the position of the target zone, among other examples.

In some implementations, the content management system may determine the zone configuration of the target zone based at least in part on decoding a barcode target identifier to obtain the zone configuration of the target zone. For example, configurations of target zones may be maintained in a database and may be accessed based at least in part on the detected target indicator. The database may include an index that maps the target indicator to the target zone. In some implementations, the target indicator identifies the zone configuration of the target zone based on spatial coordinates of the boundary of the target zone.

As shown by reference number <NUM>, the content management system may generate an initial masking content for the target zone. In some aspects, for example, the content management system may be configured to generate, based on a position of the source device (e.g., as indicated based at least in part on output from a spatial locator of the source device), an initial masking content that is associated with a boundary of the target zone that is defined by the zone configuration. For example, in some implementations, prior to generating the initial masking content, the content management system may determine, based on the spatial locator of the source device, whether the position of the source device is within the target zone. The content management system may generate the initial masking content using a spatial mapping model based on the video data and the target indicator. The content management system may generate a final masking content using chroma keying based on the initial masking content.

In some aspects, the source device may transmit positioning messages that indicate positioning information associated with the source device. Positioning information is information that the content management system may use to determine a position of the source device. The positioning messages may be sent, for example, via hyper text transport protocol (HTTP) to a server device that implements the content management system. In other aspects, positioning information may be sent along with the video data (e.g., embedded in the video data), via a control channel associated with the video data and/or through a dedicated positioning communication link, among other examples.

According to implementations, the content management system may generate initial and/or final masking content in different ways based at least in part on the position of the source device. In particular, a level of transparency of the final masking content is determined based on whether the position of the source device is within the target zone or outside of the target zone. For example, the content management system may determine that the position of the source device is within the target zone and may generate the final masking content as opaque content based at least in part on determining that the position of the source device is within the target zone. In this way, the video data may render a representation of a captured scene within the target zone, while a portion of the physical environment that is outside of the target zone is rendered as opaque to obfuscate that portion of the physical environment. The content management system may determine that the position of the source device is outside of the target zone and may generate the final masking content to be translucent based at least in part on determining that the position of the source device is outside of the target zone. In this way, the masking partially depicts a portion of the physical environment that is within the target zone while obfuscating the video data that depicts a portion of the physical environment that is outside of the target zone.

In some implementations, prior to generating the final masking content, the content management system may determine the position of the source device relative to the target zone; and select, based on the position of the source device, a color of the final masking content. The final masking content may be generated based on the color. For example, the final masking content may be generated using chroma keying so that other content may be conveniently superimposed thereon. In some implementations, the initial masking content may include a translucent virtual container, and the final masking content may include an opaque virtual container.

As shown by reference number <NUM>, the content management system generates and/or provides display data to the destination device. The display data includes the video data with the final masking content. In some implementations, the display data may include a video stream within which is integrated the final masking content. In some implementations, the display data may include a video stream and the final masking content separately, and the destination device may be configured to superimpose the final masking content on the rendered video data.

<FIG> illustrates another example implementation <NUM>. In example implementation <NUM>, similar operations to those described with respect to <FIG> may be performed. For example, as shown by reference number <NUM>, video data is captured by the camera. As shown by reference number <NUM>, the source device detects the target indicator. As shown by reference number <NUM>, the source device determines the target zone configuration (based at least in part on the target indicator). The source device inserts data associated with a representation of the target zone into the video data so that the representation of the target zone may be displayed as surrounding the target object. The representation of the target zone may include, for example, a three-dimensional shape configured to be rendered in a single color (e.g., a green rectangular prism) or multiple colors (e.g., a rectangular prism having two different colors in a checkered pattern).

As shown by reference number <NUM>, the source device may provide the video data with the target zone representation to the content management system. In some implementations, the source device may provide two streams of video data. A first stream may include a "raw" video stream and a second data stream (e.g., a data stream that includes the target zone representation). In this way, the content management system generates mask content for the target zone based at least in part on the target zone representation included in the second data stream. As show by reference number <NUM>, the content management system generates the masking content for the target zone. As shown by reference number <NUM>, the content management system generates and/or provides display data to the destination device.

In implementations described above, using content masking for security of a physical environment may be maintained to prevent fraud (thereby conserving resources associated with addressing fraudulent activity that might otherwise occur), prevent the need for a user to go to a dedicated space to show a particular object (e.g., the object may be on a manufacturing line or in an area that is inherently surrounded by confidential information and the confidential materials or objects cannot easily be moved), and/or maintain privacy associated with a portion of a physical environment, among other examples. Further, masking or obfuscating content of the video data can reduce an amount of data that is to be processed to depict a physical environment that otherwise may be irrelevant to the video stream.

<FIG> is a diagram of an example environment <NUM> in which systems and/or methods described herein may be implemented. As shown in <FIG>, environment <NUM> may include a source device <NUM>, a destination device <NUM>, a content management system <NUM>, and a network <NUM>. Devices of environment <NUM> may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections.

The source device <NUM> includes one or more devices capable of receiving, generating, storing, processing, and/or providing video data and/or positioning information associated with the source device <NUM>, as described elsewhere herein. The source device <NUM> may include a communication device and/or a computing device. For example, the source device <NUM> may include a smartphone, an augmented reality headset, a set of augmented reality glasses, wireless communication device, a user equipment, a laptop computer, a tablet computer, a desktop computer, a gaming console, a set-top box, a wearable communication device (e.g., a smart wristwatch, a pair of smart eyeglasses, a head mounted display, or a virtual reality headset), among other examples. The source device <NUM> may communicate with one or more other devices of environment <NUM> (e.g., via network <NUM>), as described elsewhere herein.

The destination device <NUM> includes one or more devices capable of receiving, generating, storing, processing, and/or providing video data and/or masking content, as described elsewhere herein. The destination device <NUM> may include a communication device and/or a computing device. For example, the destination device <NUM> may include a wireless communication device, a mobile phone, a user equipment, a laptop computer, a tablet computer, a desktop computer, a gaming console, a set-top box, a wearable communication device (e.g., a smart wristwatch, a pair of smart eyeglasses, a head mounted display, or a virtual reality headset), or a similar type of device.

The content management system <NUM> may be implemented on one or more server devices. The one or more server devices include one or more devices capable of receiving, generating, storing, processing, providing, and/or routing information associated with video data and/or obfuscating areas of a physical environment in the video data, as described elsewhere herein. The one or more server devices may include a communication device and/or a computing device. For example, the one or more server devices may include a server, such as an application server, a client server, a web server, a database server, a host server, a proxy server, a virtual server (e.g., executing on computing hardware), or a server in a cloud computing system. In some implementations, the one or more server devices include computing hardware used in a cloud computing environment.

The network <NUM> includes one or more wired and/or wireless networks. For example, the network <NUM> may include a cellular network, a public land mobile network, a local area network, a wide area network, a metropolitan area network, a telephone network, a private network, the Internet, and/or a combination of these or other types of networks. The network <NUM> enables communication among the devices of environment <NUM>. In some implementations, the network <NUM> may be multiple networks.

<FIG> is a diagram of example components of a device <NUM>, which may correspond to a source device (e.g., source device <NUM>), a destination device (e.g., destination device <NUM>), and/or a content management system (e.g., content management system <NUM>). In some implementations, the source device, the destination device, and/or the content management system may include one or more devices <NUM> and/or one or more components of device <NUM>. As shown in <FIG>, device <NUM> may include a bus <NUM>, a processor <NUM>, a memory <NUM>, a storage component <NUM>, an input component <NUM>, an output component <NUM>, and a communication component <NUM>.

Bus <NUM> includes a component that enables wired and/or wireless communication among the components of device <NUM>. Processor <NUM> includes a central processing unit, a graphics processing unit, a microprocessor, a controller, a microcontroller, a digital signal processor, a field-programmable gate array, an application-specific integrated circuit, and/or another type of processing component. Processor <NUM> is implemented in hardware, firmware, or a combination of hardware and software. In some implementations, processor <NUM> includes one or more processors capable of being programmed to perform a function. Memory <NUM> includes a random access memory, a read only memory, and/or another type of memory (e.g., a flash memory, a magnetic memory, and/or an optical memory).

Storage component <NUM> stores information and/or software related to the operation of device <NUM>. For example, storage component <NUM> may include a hard disk drive, a magnetic disk drive, an optical disk drive, a solid state disk drive, a compact disc, a digital versatile disc, and/or another type of non-transitory computer-readable medium. Input component <NUM> enables device <NUM> to receive input, such as user input and/or sensed inputs. For example, input component <NUM> may include a touch screen, a keyboard, a keypad, a mouse, a button, a microphone, a switch, a sensor, a global positioning system component, an accelerometer, a gyroscope, and/or an actuator. Output component <NUM> enables device <NUM> to provide output, such as via a display, a speaker, and/or one or more light-emitting diodes. Communication component <NUM> enables device <NUM> to communicate with other devices, such as via a wired connection and/or a wireless connection. For example, communication component <NUM> may include a receiver, a transmitter, a transceiver, a modem, a network interface card, and/or an antenna.

Device <NUM> may perform one or more processes described herein. For example, a non-transitory computer-readable medium (e.g., memory <NUM> and/or storage component <NUM>) may store a set of instructions (e.g., one or more instructions, code, software code, and/or program code) for execution by processor <NUM>. Processor <NUM> may execute the set of instructions to perform one or more processes described herein. In some implementations, execution of the set of instructions, by one or more processors <NUM>, causes the one or more processors <NUM> and/or the device <NUM> to perform one or more processes described herein. In some implementations, hardwired circuitry may be used instead of or in combination with the instructions to perform one or more processes described herein.

<FIG> is a flowchart of an example process <NUM> associated with obfuscating of confidential areas of a physical environment depicted in a video. In some implementations, one or more process blocks of <FIG> may be performed by a content management system (e.g., content management system <NUM>). In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including the content management system, such as a source device (e.g., source device <NUM>), and/or a destination device (e.g., destination device <NUM>). Additionally, or alternatively, one or more process blocks of <FIG> may be performed by one or more components of device <NUM>, such as processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, and/or communication component <NUM>.

As shown in <FIG>, process <NUM> includes receiving, from a camera of a source device, video data depicting a physical environment (block <NUM>). For example, the content management system may receive, from a camera of a source device, video data depicting a physical environment, as described above.

As further shown in <FIG>, process <NUM> includes detecting a target indicator associated with a target zone of the physical environment (block <NUM>). For example, the content management system may detect, within the video data, a target indicator associated with a target zone of the physical environment, as described above.

As further shown in <FIG>, process <NUM> includes determining, based on the target indicator, a zone configuration of the target zone relative to the target indicator (block <NUM>). For example, the content management system may determine, based on the target indicator, a zone configuration of the target zone relative to the target indicator, as described above.

As further shown in <FIG>, process <NUM> may include generating, based on a position of the source device, an initial masking content that is associated with a boundary of the target zone that is defined by the zone configuration using a spatial mapping model based on the video data and the target indicator (block <NUM>). For example, the content management system may generate, based on a position of the source device, an initial masking content that is associated with a boundary of the target zone that is defined by the zone configuration using a spatial mapping model based on the video data and the target indicator, as described above.

As further shown in <FIG>, process <NUM> includes generating display data associated with the masking content and the video data (block <NUM>). For example, the content management system may generate display data associated with the masking content and the video data, as described above.

As further shown in <FIG>, process <NUM> includes providing the display data to a destination device, wherein the display data causes a rendering of the physical environment on a display of the destination device to include a masking associated with the boundary of the target zone (block <NUM>). For example, the content management system may provide the display data to a destination device, wherein the display data causes a rendering of the physical environment on a display of the destination device to include a masking of the boundary of the target zone, as described above. The display data causes a rendering of the physical environment on a display of the destination device to include a masking of the boundary of the target zone.

The process <NUM> further includes prior to detecting the target indicator, processing the video data according to an image processing model.

In a first implementation, the image processing model is configured to detect a plurality of target indicators, each target indicator associated with a respective target zone of the physical environment.

In a second implementation, the target indicator comprises a barcode, and wherein determining the zone configuration of the target zone comprises decoding the barcode to obtain the zone configuration of the target zone.

In a third implementation, process <NUM> includes, prior to generating the masking content, determining, based on a spatial locator of the source device, that the position of the source device is within the target zone, and generating the masking content as opaque content, wherein the masking obfuscates a portion of the physical environment that is outside of the target zone.

In a fourth implementation, process <NUM> includes, prior to generating the masking content, determining that the position of the source device is outside of the target zone, generating the masking content to be translucent, wherein the masking partially depicts a portion of the physical environment that is within the target zone, and obfuscating the video data that depicts a portion of the physical environment that is outside of the target zone.

In a fifth implementation, process <NUM> includes, prior to generating the masking content, determining the position of the source device relative to the target zone, and selecting, based on the position of the source device, a color of the masking content, wherein the masking content is generated based on the color.

In a sixth implementation, the video data is associated with a video stream that is being streamed from the source device to the destination device.

In a seventh implementation, the initial masking content comprises a translucent virtual container, and the final masking content comprises an opaque virtual container.

Modifications may be made in light of the above disclosure or may be acquired from practice of the implementations.

Although particular combinations of features are disclosed in the specification, these combinations are not intended to limit the disclosure of various implementations. In fact, many of these features may be combined in ways not specifically disclosed in the specification. As an example, "at least one of: a, b, or c" is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiple of the same item.

Claim 1:
A method, comprising:
receiving, by a device and from a camera of a source device, video data depicting a physical environment;
detecting, by the device, a target indicator depicted within the physical environment by processing the video data according to an image processing model;
determining, by the device and based on the target indicator, a zone configuration of a target zone of the physical environment, said target zone being associated with the target indicator;
generating, by the device, based on a position of the source device and the zone configuration, masking content that is associated with a boundary of the target zone that is defined by the zone configuration,
wherein a level of transparency of the masking content is determined based on whether the position of the source device is within the target zone or outside of the target zone;
generating, by the device, display data by combining the masking content and the video data; and
providing, by the device, the display data to a destination device,
wherein the display data causes a rendering of the physical environment on a display of the destination device to include a masking associated with the boundary of the target zone, and
wherein the zone configuration of the target zone is associated with at least one of:
a position of the target zone relative to the target indicator;
dimensions of the target zone relative to the target indicator;
dimensions of the target zone relative to the position of the target zone;
a shape of the target zone relative to the target indicator; or
a shape of the target zone relative to the position of the target zone.