Patent ID: 12243121

DETAILED DESCRIPTION

Overview

In order to edit a particular portion of a person that is depicted in multiple digital images using conventional systems, a user interacts with an input device (e.g., a touchscreen, a microphone, a mouse, a stylus, etc.) relative to a user interface of an application for editing digital content to modify the particular portion of the person for each of the multiple digital images that depict the person. Individually editing each of the multiple digital images in this manner is time consuming and tedious for the user which is a shortcoming of the conventional systems. In order to overcome the limitations of conventional systems, techniques and systems for generating and propagating personal masking edits are described.

In one example, a computing device implements a mask system to detect a face of a person depicted in a digital image displayed in a user interface of an application for editing digital content. For instance, the application for editing digital content includes or has access to a first machine learning model trained on training data to detect faces of people depicted in digital images and the mask system leverages the trained first machine learning model to detect the face of the person depicted in the digital image. A segment mask is generated for the person using a second machine learning model included in or available to the application for editing digital content that is trained on training data to generate segment masks for person objects depicted in digital images. For example, the segment mask is usable to select or segment a person object depicted in the digital image that corresponds to the person.

In an example, the mask system associates the face of the person with the person object by computing an overlap score based on a number of pixels of the face of the person that overlap with the segment mask and a total number of the pixels of the face of the person. In this manner, the mask system determines that the face of the person is associated with the person object and not associated with an additional person object depicted in the digital image. For instance, the mask system extracts the person from the digital image as a separate digital image using the segment mask for the person.

For example, the mask system uses the separate digital image to determine types of masks of the application for editing digital content that are usable to select and segment all of the person (e.g., all of the person object) or portions of the person depicted in the digital image. Examples of masks usable to select and segment portions of the person include facial masks, facial skin masks, beard masks, hair masks, clothes masks, teeth masks, eye masks, and so forth. For instance, the mask system displays indications of the determined types of masks in the user interface of the application for editing digital content.

The user interacts with the input device relative to the user interface of the application for editing digital content to interact with a displayed indication of a mask, e.g., a hair mask. The interaction with the displayed indication of the hair mask causes selection or segmentation of the person's hair as depicted in the digital image. For example, after selecting the person's hair using the hair mask, the user further interacts with the input device relative to the user interface to specify properties of an editing operation that edits the digital image by modifying a color of the person's hair from blonde to green.

The mask system receives edit data describing the hair mask and the properties of the editing operation, and the mask system processes the edit data to generate a personal masking edit for the person depicted in the digital image. For example, the personal masking edit is usable to edit additional digital image depicting the person by coloring the person's hair green in the additional digital images. In one example, the mask system updates a database of person data that describes associations between identifiers of persons depicted in digital images of a set of digital images and personal masking edits for the persons depicted in the digital images. In this example, the person data also describes associations between the identifiers of the persons and identifiers of faces of the persons depicted in the digital images. Continuing this example, the mask system updates the database of the person data by associating an identifier of the person depicted in the digital image with the personal masking edit for the person that is usable to color the person's hair green in the additional digital images.

For example, the mask system identifies the additional digital images of the set of digital images as depicting the person using the identifier of the person. In one example, the mask system automatically edits the additional digital images identified as depicting the person using the personal masking edit for the person by selecting the person's hair in the additional digital images using the hair mask and coloring the person's hair green in the additional digital images using the properties of the editing operation. In another example, the mask system displays indications of the additional digital images in the user interface of the application for editing digital content. In this example, the user interacts with the input device relative to the user interface to specify particular digital images of the additional digital images.

For instance, the mask system receives the edit data as describing the particular digital images and the mask system process the edit data to edit the particular digital images using the personal masking edit. In an example, the mask system selects the hair of the person as depicted in the particular digital images using the hair mask and colors the person's hair green using the properties of the editing operation. Consider an example in which the person data describes additional personal masking edits associated with the identifier of the person. In this example, after the mask system identifies the additional digital images of the set of digital images as depicting the person using the identifier of the person, the mask system displays indications of the additional personal masking edits in the user interface of the application for editing digital content.

Continuing the previous example, the user interacts with the input device relative to the user interface to specify that a particular personal masking edit is to be applied (e.g., by interacting with an indication of the particular personal masking edit). The user continues to interact with the input device relative to the user interface to specify that a particular digital image of the of the set of digital images that depict the person is to be edited using the particular personal masking edit (e.g., by interacting with an indication of the particular digital image). For example, the mask system receives and processes the edit data as describing the particular personal masking edit and the particular digital image, and the mask system edits the particular digital image using a type of machine learning based mask and properties of an editing operation described by the particular personal masking edit.

By leveraging the database of person data that describes the associations between the identifier of the person and the personal mask edits for the person, the described systems are capable of simultaneously modifying all or a portion of the person as depicted in multiple digital images. This is not possible using conventional systems that are limited to editing portions of digital images manually one digital image at a time. The described systems are also capable of identifying objects other than person objects depicted in digital images and automatically editing the digital images locally by applying editing presets to the identified objects which is also not possible using the conventional systems.

In the following discussion, an example environment is first described that employs examples of techniques described herein. Example procedures are also described which are performable in the example environment and other environments. Consequently, performance of the example procedures is not limited to the example environment and the example environment is not limited to performance of the example procedures.

Example Environment

FIG.1is an illustration of an environment100in an example implementation that is operable to employ digital systems and techniques as described herein. The illustrated environment100includes a computing device102connected to a network104. The computing device102is configurable as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld configuration such as a tablet or mobile phone), and so forth. Thus, the computing device102is capable of ranging from a full resource device with substantial memory and processor resources (e.g., personal computers, game consoles) to a low-resource device with limited memory and/or processing resources (e.g., mobile devices). In some examples, the computing device102is representative of a plurality of different devices such as multiple servers utilized to perform operations “over the cloud.”

The illustrated environment100also includes a display device106that is communicatively coupled to the computing device102via a wired or a wireless connection. A variety of device configurations are usable to implement the computing device102and/or the display device106. The computing device102includes a storage device108and a mask module110.

The storage device108is illustrated to include person data112that describes associations between identifiers of persons depicted in digital images and corresponding identifiers of faces of the persons depicted in the digital images. In an example, the person data112also describes personal or person-specific masking edits corresponding to each of the identifiers of the persons. For example, the personal masking edits describe properties of editing operations performed on digital images depicting a particular person that change a visual appearance of at least a portion of the particular person as depicted in the digital images. In this example, the personal masking edits are represented as extensible metadata platform documents; however, in other examples, the personal masking edits are represented in other formats and/or using different types of documents.

For instance, each unique person identified and described by the person data112has a corresponding identifier that is also unique, and each identifier of a person is associated with every identifier of the person's face described by the person data112. Consider an example in which the particular person of the previous example is depicted in three different digital images. In this example, the person data112describes a unique identifier for the particular person (e.g., Person-ID1) which is associated with an identifier of the particular person's face as depicted in each of the three digital images (e.g., Face-ID1, Face-ID2, and Face-ID3).

In an example, personal masking edits for the particular person described by the person data112are also associated with the identifier for the particular person (e.g., Person-ID1). In this example, two of the three digital images have been edited to modify a visual appearance of a portion of the particular person, and the person data112describes the identifier of the particular person (e.g., Person-ID1) as being associated with identifiers of personal masking edits (e.g., Edit-ID1and Edit-ID2) that describe properties of editing operations performed to modify the visual appearance of the portion of the particular person depicted in the two digital images. Accordingly, the identifier for the particular person (e.g., Person-ID1) is usable to determine a set of digital images that depict the particular person's face and also to determine properties of editing operations performed to modify a visual appearance of a portion of the particular person.

The mask module110is illustrated as having, receiving, and/or transmitting image data114and the mask module110is also illustrated as having, receiving, and/or transmitting edit data116. For instance, the image data114describes a digital image118which depicts nine people that are posing for a group picture. A person120is included among the nine people posing for the group picture. For example, the person data112describes an identifier for the person120as well as an identifier for each of the eight other people depicted in the digital image118. In this example, the person data112also describes an identifier for a face of the person120and an identifier for each of faces of the other eight people depicted in the digital image118. Similarly, the person data112describes identifiers of personal masking edits that describe properties of editing operations performed on additional digital images depicting the person120which modify a visual appearance of a portion of the person120as depicted in the additional digital images.

Consider an example in which the mask module110leverages the person data112and the edit data116to modify the digital image118which depicts the person120with hair colored pink and a naturally colored beard before the digital image118is modified. In this example, the edit data116describes properties of an editing operation and a type of mask usable to modify a particular portion of the person120depicted in the digital image118. For instance, a user interacts with an input device (e.g., a mouse, a stylus, a touchscreen, a microphone, a keyboard, etc.) relative to a user interface of an application for editing digital content which is displaying the digital image118to generate the edit data116.

In one example, the user interacts with the input device relative to the user interface of the application for editing digital content to implement a machine learning based mask to select or segment the particular portion of the person120depicted in the digital image118. Continuing this example, the application for editing digital content selects or segments the particular portion of the person120using the machine learning based mask which is part of functionality available to the application for editing digital content via a machine learning model. As used herein, the term “machine learning model” refers to a computer representation that is tunable (e.g., trainable) based on inputs to approximate unknown functions. By way of example, the term “machine learning model” includes a model that utilizes algorithms to learn from, and make predictions on, known data by analyzing the known data to learn to generate outputs that reflect patterns and attributes of the known data. According to various implementations, such a machine learning model uses supervised learning, semi-supervised learning, unsupervised learning, reinforcement learning, and/or transfer learning. For example, the machine learning model is capable of including, but is not limited to, clustering, decision trees, support vector machines, linear regression, logistic regression, Bayesian networks, random forest learning, dimensionality reduction algorithms, boosting algorithms, artificial neural networks (e.g., fully-connected neural networks, deep convolutional neural networks, or recurrent neural networks), deep learning, etc. By way of example, a machine learning model makes high-level abstractions in data by generating data-driven predictions or decisions from the known input data.

Consider an example in which the particular portion of the person120described by the edit data116is the person's120hair and the properties of the editing operation modify the color of the person's120hair from pink as depicted in the digital image118to green as depicted in a modified digital image122which is displayed in a user interface124of the display device106. For example, the mask module110generates a personal masking edit for the person120based on the edit data116that associates the person120with the editing operation that colored the person's120hair green. In this example, the mask module110generates additional person data112as describing the personal masking edit for the person120.

For instance, the personal masking edit for the person120is usable (e.g., by the mask module110) to propagate the editing operation to additional digital images that depict the person120such that the additional digital images depict the person's120hair colored green as in the modified digital image122. In one example, the mask module110identifies the additional images as depicting the person based on the identifier for the person120described by the person data112. In this example, the mask module110segments or selects the person's120hair in the additional digital images using a hair machine learning based mask or a hair mask in order to color the person's120hair green in the additional digital images.

Consider another example in which the mask module110leverages the person data112to edit the digital image118. In this example, the mask module110uses the identifier for the person120to identify personal masking edits for the person120described by the person data112. In one example, the mask module110displays indications of identified personal masking edits for the person120in the user interface (e.g., the user interface124) of the application for editing digital content. For example, the mask module110displays an indication of a personal masking edit for the person120that was generated by coloring the person's120beard pink in another digital image.

Continuing the previous example, the user interacts with the input device relative to the user interface (e.g., the user interface124) of the application for editing digital content to generate the edit data116as describing a selection of the indication of the personal masking edit for the person120. The mask module110receives and processes the edit data116to color the person's120beard pink which is illustrated in the modified digital image122. In a first example, the mask module110colors the person's120beard pink automatically and without user intervention in response to receiving and processing the edit data116.

In a second example, the mask module110selects or segments the person's120beard using a beard machine learning based mask or a beard mask in response to receiving and processing the edit data116. In this second example, the user interacts with the input device relative to the user interface124to specify the color pink for coloring the person's120beard. In a third example, the mask module110suggests the color pink for coloring the person's120beard, e.g., by coloring the indication pink, based on the hair color of the person120as depicted in the digital image118. In this third example, the user interacts with the input device relative to the user interface124of the application for editing digital content to accept the suggestion and causes the person's120beard to be colored pink. For instance, the mask module110colors the person's120beard pink in response to receiving the edit data116describing the accepted suggestion.

Although the mask module110is described as generating and propagating personal masking edits, it is to be appreciated that in some examples the mask module110is also capable of generating and propagating presets for editing the digital image118. For example, the presets are also represented as extensible metadata platform documents and the presets are usable to edit the digital image118globally and/or locally. For instance, the mask module110is capable of detecting and classifying objects depicted in the digital image118in addition to detecting faces of people depicted in the digital image118.

In one example, the mask module110leverages label vectors of detected objects and compares the label vectors to preset labels to identify relevant presets for modifying the detected objects. In this example, the mask module110selects or segments a detected object depicted in the digital image118using a type of machine learning based mask generated by a machine learning model of the application for editing digital content. For example, the mask module110selects a subject object using a subject machine learning based mask, a sky object using a sky machine learning based mask, and so forth. The mask module110then applies a relevant preset to the selected or segmented object to edit the object locally within the digital image118. The mask module110is also capable of suggesting and applying the relevant preset to objects similar to the selected or segmented object identified in additional digital images.

FIG.2depicts a system200in an example implementation showing operation of a mask module110. The mask module110is illustrated to include a face module202, a person module204, an interface module206, and a display module208. In an example, the mask module110receives the image data114and the edit data116as inputs. For example, the face module202receives and processes the image data114to generate face data210.

FIGS.3A,3B,3C, and3Dillustrate an example of generating and propagating personal masking edits.FIG.3Aillustrates a representation300of detecting faces depicted in a digital image.FIG.3Billustrates a representation302of extracting a person depicted in the digital image using a segment mask for the person.FIG.3Cillustrates a representation304of different segment masks for selecting different particular portions of the person depicted in the digital image.FIG.3Dillustrates a representation306of input images depicting the person and output images modified using personal masking edits associated with an identifier of the person.

With reference toFIGS.2and3A, the face module202receives the image data114as describing a digital image308. For instance, the digital image308depicts the person120and eight other people who are posing as a group for a photograph. The face module202processes the image data114to detect faces of persons depicted in the digital image308.

To do so in one example, the face module202leverages a machine learning model trained on training data to detect distinct faces depicted in digital images and to classify the distinct faces as unique persons. For example, the face module202assigns a unique identifier to each identified unique person and also assigns a unique identifier to each instance of the unique person's face as depicted in digital images of a set of digital images. In this manner, the face module202establishes a one-to-many relationship between an identifier of a particular person and identifiers of instances in which the particular person's face is depicted in the digital images of the set of digital images.

In addition to detecting the faces of persons depicted in the digital image308, the face module202detects objects depicted in the digital image308and classifies detected objects as person objects or detected persons. For example, the face module202leverages a machine learning model such as a convolutional neural network trained on training data to detect person objects depicted in digital images and to segment detected person objects within the digital images. In one example the face module202leverages a convolutional neural network such as EfficientNet-BO as a backbone network to detect person objects depicted in digital images and to segment detected person objects within the digital images.

For example, by detecting faces of persons depicted in the digital image308and by detecting person objects depicted in the digital image308, the face module202is capable of generating a processed digital image310. As shown, the processed digital image310depicts a bounding box312for a face of the person120as well as detected person objects314-326which each correspond to one of the eight other people who are posing as the group depicted in the digital image308. In the illustrated example, the face module202generates the face data210as describing the processed digital image310.

As shown inFIG.2, the person module204receives the face data210and the person data112, and the person module204processes the face data210and/or the person data112to generate association data212. For instance, the person module204is illustrated as having, receiving, and/or transmitting the person data112which describes associations between identifiers of persons depicted in digital images of the set of digital images and corresponding identifiers of faces of the persons depicted in the digital images. In an example, the person data112is included in a database that associates the digital images of the set of digital images with identifiers of persons depicted in the digital images. The person module204processes the face data210to identify a face identifier and a person identifier for each of the detected person objects314-326, and then updates the person data112to include associations between the person identifiers and the corresponding face identifiers of the detected person objects314-326.

In some examples, the person module204updates the person data112to include additional information such as information specified by a user interacting with an input device (e.g., a microphone, a stylus, a mouse, a keyboard, a touchscreen, etc.) relative to the user interface124which is displaying the digital image308. In one example, the user interacts with the input device to generate the edit data116as describing a name of the person120. In this example, the face module202receives and processes the edit data116to generate the face data210as describing the name of the person120and the person module204receives and processes the face data210to update the person data112by associating the name of the person120with an identifier of the person120.

For example, the person data112also describes personal or person-specific masking edits corresponding to each of the identifiers of the persons described by the person data112. In one example, the personal masking edits are represented as extensible metadata platform documents. In some examples, the personal masking edits are represented in other formats and/or using different types of documents. For instance, the personal masking edits are usable to automatically propagate editing operations to the digital image308and other digital images included in the set of digital images.

Consider an example in which the person data112describes a particular personal masking edit that is associated with an identifier that uniquely identifies a person detected as the person object316in the processed digital image310. As shown in the digital image308, the person detected as the person object316is wearing a multi-colored plaid shirt. For instance, the particular masking edit describes a clothes machine learning based mask or a clothes mask and a single solid color. In this example, the particular masking edit is usable to automatically modify the multi-colored plaid shirt such that the shirt depicts the single solid color in the digital image308.

Consider another example in which the person data112does not describe the particular personal masking edit. In this example, the user interacts with the input device relative to the user interface124to generate the edit data116as describing the clothes machine learning based mask and the single solid color. The face module202receives and processes the edit data116to generate the face data210as describing the single solid color applied to the shirt worn by the person detected as the person object316. The person module204receives and processes the face data210to update the person data112to include the particular personal masking edit which is now usable in examples such as the previous example.

As part of processing the face data210, the person module204generates a segment mask328for the person120(e.g., an object identified as the person120). As illustrated inFIG.3B, the person module204computes an overlap score for the person120and also for each of the detected person objects314-326. For example, the person module204computes the overlap score for the person120based on an area of overlap between the bounding box312for the face of the person120and the segment mask328for the person120as illustrated in a digital image330.

In another example, the person module204computes the overlap score for the person120based on a number of pixels of the face of the person120that are overlapping a full person mask332for the person120and a total number of pixels of the face of the person120. In this example, the person module204computes an overlap score for each of the detected person objects314-326based on a number of pixels of the face of the person120that are overlapping a full person mask for each of the detected person objects314-326and the total number of pixels of the face of the person120. In one example, this is representable as:

overlap⁢score=number⁢of⁢face⁢pixels⁢overlapping⁢with⁢mask⁢regiontotal⁢number⁢of⁢face⁢pixels

In an example, a person object having a highest overlap score is determined to be the person object corresponding to the bounding box312of the face of the person120. For example, the person module204determines that the person120has a higher overlap score than each of the detected person objects314-326. Based on this determination, the person module204uses the segment mask328to segment the person120from the digital image330by generating an extracted segment mask image334. For example, the extracted segment mask image334is a separate digital image from the digital image308.

The person module204generates the association data212as describing the extracted segment mask image334in one example. For instance, the person module204updates the person data112to indicate that an identifier for the person120corresponds to the bounding box312for the face of the person120. The interface module206receives the association data212, the edit data116, and the person data112, and the interface module206processes the association data212, the edit data116, and/or the person data112to generate modified image data214.

In an example workflow, the interface module206processes the association data212to identify types of machine learning based masks of the application for editing digital content that are usable to select a particular portion of the person120based on the extracted segment mask image334. When selected using a machine learning based mask, the particular portion of the person120is editable locally, e.g., without modifying a remaining portion of the digital image308. For example, the interface module206identifies a facial mask, a facial skin mask, a teeth mask, a clothes mask, a hair mask, and a beard mask as the types of machine learning based masks of the application for editing digital content that are usable to select various particular portions of the person120.

In this example, indications of the identified types of machine learning based masks are displayed in a user interface124of the application as illustrated in the representation304illustrated inFIG.3C. As shown, the representation304includes an indication336of the facial mask, an indication338of the facial skin mask, an indication340of the teeth mask, an indication342of the clothes mask, an indication344of the hair mask, and an indication346of the beard mask. Although the particular masks corresponding to the indications336-346are usable to segment or select the various particular portions of the person120, it is to be appreciated that the mask module110is not limited to the particular masks. For instance, in other examples, the mask module110is capable of generating and/or implementing eye masks, ear masks, eyebrow masks, mustache masks, accessory masks, handbag masks, jewelry masks, hat masks, glove masks, shoe masks, and so forth.

For example, the user interacts with the input device relative to the user interface124of the application for editing digital content to select one of the types of machine learning based masks for editing the person120depicted in the digital image308by selecting a corresponding one of the indications336-346. In this example, the user interacts with the input device to generate the edit data116as describing a selection of the indication344of the hair mask and the color green. The interface module206receives and processes the edit data116and the association data212to generate the modified image data214as describing the hair mask and the color green.

In one example, the interface module206generates the modified image data214as describing a modification to input images348-352that are included in the representation306ofFIG.3D. As shown, each of the input images348-352depicts the person120having hair that is colored pink. For instance, the display module208receives and processes the modified image data214to display output images354-358in the user interface124.

For example, output image354is generated by editing input image348to change the person's120hair color from pink to green; output image356is generated by editing input image350to change the person's120hair color from pink to green; and output image358is generated by editing input image352to change the person's120hair color from pink to green. Because the input images348-352are edited using the hair mask in one example, the color green is only applied to the person's120hair in the output images354-358. As shown, all portions of the output images354,365,358other than the person's120hair are identical to corresponding portions of the input images348,350,352, respectively. For example, the output digital images354-358are generated automatically and without user intervention.

In general, functionality, features, and concepts described in relation to the examples above and below are employed in the context of the example procedures described in this section. Further, functionality, features, and concepts described in relation to different figures and examples in this document are interchangeable among one another and are not limited to implementation in the context of a particular figure or procedure. Moreover, blocks associated with different representative procedures and corresponding figures herein are applicable individually, together, and/or combined in different ways. Thus, individual functionality, features, and concepts described in relation to different example environments, devices, components, figures, and procedures herein are usable in any suitable combinations and are not limited to the particular combinations represented by the enumerated examples in this description.

Example Procedures

The following discussion describes techniques which are implementable utilizing the previously described systems and devices. Aspects of each of the procedures are implementable in hardware, firmware, software, or a combination thereof. The procedures are shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks. In portions of the following discussion, reference is made toFIGS.1-3D.FIG.4is a flow diagram depicting a procedure400in an example implementation in which edit data is received describing properties of an editing operation used to modify a particular portion of a person depicted in a digital image and an additional digital image is edited using the properties of the editing operation to modify the particular portion of the person as depicted in the additional digital image.

A face of a person depicted in a digital image displayed in a user interface of an application for editing digital content is detected (block402). For example, the computing device102implements the mask module110to detect the face of the person depicted in the digital image. An identifier of the person is determine based on an identifier of the face (block404). In an example, the mask module110determines the identifier of person.

Edit data describing properties of an editing operation and a type of mask used to modify a particular portion of the person depicted in the digital image is received (block406). In one example, the computing device102implements the mask module110to receive the edit data. An additional digital image identified based on the identifier of the person is edited using the type of mask and the properties of the editing operation to modify the particular portion of the person as depicted in the additional digital image (block408). For example, the mask module110edits the additional digital image using the type of mask and the properties of the editing operation.

FIG.5illustrates a representation500of example extensible metadata platform documents. The representation500includes an extensible metadata platform document502for representing an identifier of a face of a person depicted in a digital image of a set of digital images; an extensible metadata platform document504for representing an identifier of a person depicted in a digital image or in multiple digital images of the set of digital images; and an extensible metadata platform document506for representing a personal masking edit that is usable to modify a portion of a person depicted in a digital image or in multiple digital images of the set of digital images. For example, the person data112describes the extensible metadata platform documents502-506.

Consider an example in which an identifier of a person “8b2422142da96b9a957a264ff206f837” included in the extensible metadata platform document506corresponds to a mask group for the person. In this example, the mask module110leverages information available in the person data112to generate the mask group for the person using a personalized name of the person described by the person data112. For example, the identifier of the person “8b2422142da96b9a957a264ff206f837” uniquely identifies the person120and the user interacts with the input device relative to the user interface124to specify a personalized name of the person120as “John Walker.” In the illustrated example, the extensible metadata platform document506includes the personalized name in a “CorrectionName” field.

In response to the user interacting with the input device to select the indication344of the hair mask, the mask module110includes the hair mask in a mask group named “John's Hair” or generates a new mask group named “John's Hair” and includes the hair mask in the new mask group. The mask module110uses the mask group “John's Hair” to organize personal masking edits that use the hair mask to segment and modify the person's120hair in digital images of the set of digital images. For example, the mask group “John's Hair” includes a personal masking edit that colors the person's120hair green, a personal masking edit that colors the person's120hair pink, and so forth.

FIG.6illustrates a representation600of an example user interface for generating and propagating personal masking edits. The representation600includes a first example602of the user interface124and a second example604of the user interface124. In the first example602, a digital image depicting the person120is displayed in the user interface124in addition to user interface elements606,608. For example, user interface element606is usable to segment or select the person120in the digital image and user interface element608is usable to segment or select a different person610that is also depicted in the digital image displayed in the user interface124.

For example, a user interacts with an input device (e.g., a microphone, a finger/touchscreen, a stylus, a mouse, etc.) relative to the user interface124to select the user interface element606. In this example, the mask module110receives the edit data116describing the selection of the user interface element606and the mask module110processes the edit data116to segment or select the person120as shown in the second example604. For instance, in the second example604the user interface124displays user interface elements612-620which are selectable to segment or select different portions of the person120depicted in the digital image.

In some examples, user interface element612is usable to apply a whole person mask to the digital image to select/segment an entirety of the person120; user interface element614is usable to apply a clothes mask to the digital image to select/segment a shirt of the person120; user interface element618is usable to apply a facial mask to the digital image to select/segment a face of the person120; and user interface element620is usable to apply a hair mask to the digital image to select/segment the person's120hair. The user interacts with the input device relative to the user interface124to select one of the user interface elements612-620to segment or select a portion of the person120, and the user further interacts with the input device relative to the user interface124to modify a visual appearance of the selected portion of the person120. For example, the mask module110receives the edit data116describing properties of an editing operation performed to modify the visual appearance of the selected portion of the person120, and the mask module110generates a personal masking edit for the person120that is usable to select and edit the portion of the person120as depicted in additional digital images.

FIG.7illustrates a representation700of an example user interface for propagating edits applied to a person depicted in a digital image to additional digital images that depict the person. In a first example702, the user interacts with the input device relative to the user interface124to modify portions of the person120in a digital image displayed in the user interface124. For example, the user interacts with the input device to segment or select a beard of the person120using a beard mask. After segmenting or selecting the beard of the person120using the beard mask, the user interacts with the input device relative to the user interface124to color the beard of the person120green. In an example, the mask module110generates a first personal masking edit for the person120that is usable to color the person's120beard green in additional digital images that depict the person120. In this example, the mask module110updates the person data112to include the first personal masking edit for the person120as being associated with an identifier for the person120.

Continuing the first example702, the user interacts with the input device relative to the user interface124to segment or select hair of the person120using a hair mask. After segmenting or selecting the hair of the person120using the hair mask, the user interacts with the input device relative to the user interface124to color the hair of the person120blue. In one example, the mask module110generates a second personal masking edit for the person120that is usable to color the person's120hair blue in additional digital images that depict the person120. In this example, the mask module110updates the person data120to include the second personal masking edit for the person120as being associated with the identifier for the person120.

For instance, after coloring the person's120beard green and the person's120hair blue, the user interacts with the input device relative to the user interface124to select a user interface element704. The mask module110receives the edit data116describing the selection of the user interface element704, and the mask module110processes the person data112to load the first personal masking edit for the person120and the second personal masking edit for the person120into a memory or a register for applying the color green to the beard and the color blue to the hair of the person120in additional digital images that depict the person120.

In a second example706, the user interacts with the input device relative to the user interface124to specify five additional digital images that depict the person120to be modified using the first personal masking edit and the second personal masking edit described by the person data112and/or by the memory or the register. In an example, the mask module110leverages the identifier for the person120described by the person data112to identify a set of digital images that depict the person120. In this example, the user interacts with the input device relative to the user interface124to specify a subset (e.g., the five additional digital images) of the set of digital images that depict the person120to be modified.

The user then interacts with the input device relative to the user interface124to select a user interface element708. For example, the mask module110receives the edit data116describing the selection of the user interface element708, and the mask module110processes the edit data116, the person data112, and/or data in the memory or the register to edit each of the five additional digital images such that each of the five edited digital images depicts the beard of the person120colored green and the hair of the person120colored blue. To do so in one example, the mask module110uses the beard mask to first select the beard of the person120in each of the five additional digital images and then colors the beard of the person120green in each of the five additional digital images. Similarly, the mask module110uses the hair mask to first select the hair of the person120in each of the five additional digital images and then colors the hair of the person120blue in each of the five additional digital images.

In one example, the mask module110edits the five additional digital images automatically and without additional intervention by the user upon receiving the edit data116describing the selection of the user interface element708. In other examples, the mask module110is capable of generating edited versions of the five additional digital images depicting the person120having a green colored beard and blue colored hair as thumbnail images or full resolution images for display in the user interface124, and the user interacts with the input device relative to the user interface124to accept the edits applied to each of the five additional digital images. In a third example710, the edited digital images are displayed in the user interface124. As shown, each of the five additional digital images depicts the person120with the person's120beard colored green and with the person's120hair colored blue. A user interface element712indicates “Changes applied to 6 photos” which includes the digital image of the first example702that the user edited to color the person's120beard green and hair blue as well as the five additional digital images that are edited automatically using the beard mask and the hair mask (and properties of editing operations).

Example System and Device

FIG.8illustrates an example system800that includes an example computing device that is representative of one or more computing systems and/or devices that are usable to implement the various techniques described herein. This is illustrated through inclusion of the mask module110. The computing device802includes, for example, a server of a service provider, a device associated with a client (e.g., a client device), an on-chip system, and/or any other suitable computing device or computing system.

The example computing device802as illustrated includes a processing system804, one or more computer-readable media806, and one or more I/O interfaces808that are communicatively coupled, one to another. Although not shown, the computing device802further includes a system bus or other data and command transfer system that couples the various components, one to another. For example, a system bus includes any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures. A variety of other examples are also contemplated, such as control and data lines.

The processing system804is representative of functionality to perform one or more operations using hardware. Accordingly, the processing system804is illustrated as including hardware elements810that are configured as processors, functional blocks, and so forth. This includes example implementations in hardware as an application specific integrated circuit or other logic device formed using one or more semiconductors. The hardware elements810are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, processors are comprised of semiconductor(s) and/or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions are, for example, electronically-executable instructions.

The computer-readable media806is illustrated as including memory/storage812. The memory/storage812represents memory/storage capacity associated with one or more computer-readable media. In one example, the memory/storage812includes volatile media (such as random access memory (RAM)) and/or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth). In another example, the memory/storage812includes fixed media (e.g., RAM, ROM, a fixed hard drive, and so on) as well as removable media (e.g., Flash memory, a removable hard drive, an optical disc, and so forth). The computer-readable media806is configurable in a variety of other ways as further described below.

Input/output interface(s)808are representative of functionality to allow a user to enter commands and information to computing device802, and also allow information to be presented to the user and/or other components or devices using various input/output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., capacitive or other sensors that are configured to detect physical touch), a camera (e.g., which employs visible or non-visible wavelengths such as infrared frequencies to recognize movement as gestures that do not involve touch), and so forth. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a network card, tactile-response device, and so forth. Thus, the computing device802is configurable in a variety of ways as further described below to support user interaction.

Various techniques are described herein in the general context of software, hardware elements, or program modules. Generally, such modules include routines, programs, objects, elements, components, data structures, and so forth that perform particular tasks or implement particular abstract data types. The terms “module,” “functionality,” and “component” as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques are implementable on a variety of commercial computing platforms having a variety of processors.

Implementations of the described modules and techniques are storable on or transmitted across some form of computer-readable media. For example, the computer-readable media includes a variety of media that is accessible to the computing device802. By way of example, and not limitation, computer-readable media includes “computer-readable storage media” and “computer-readable signal media.”

“Computer-readable storage media” refers to media and/or devices that enable persistent and/or non-transitory storage of information in contrast to mere signal transmission, carrier waves, or signals per se. Thus, computer-readable storage media refers to non-signal bearing media. The computer-readable storage media includes hardware such as volatile and non-volatile, removable and non-removable media and/or storage devices implemented in a method or technology suitable for storage of information such as computer readable instructions, data structures, program modules, logic elements/circuits, or other data. Examples of computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, hard disks, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage device, tangible media, or article of manufacture suitable to store the desired information and which are accessible to a computer.

“Computer-readable signal media” refers to a signal-bearing medium that is configured to transmit instructions to the hardware of the computing device802, such as via a network. Signal media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves, data signals, or other transport mechanism. Signal media also include any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

As previously described, hardware elements810and computer-readable media806are representative of modules, programmable device logic and/or fixed device logic implemented in a hardware form that is employable in some embodiments to implement at least some aspects of the techniques described herein, such as to perform one or more instructions. Hardware includes components of an integrated circuit or on-chip system, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), and other implementations in silicon or other hardware. In this context, hardware operates as a processing device that performs program tasks defined by instructions and/or logic embodied by the hardware as well as a hardware utilized to store instructions for execution, e.g., the computer-readable storage media described previously.

Combinations of the foregoing are also employable to implement various techniques described herein. Accordingly, software, hardware, or executable modules are implementable as one or more instructions and/or logic embodied on some form of computer-readable storage media and/or by one or more hardware elements810. For example, the computing device802is configured to implement particular instructions and/or functions corresponding to the software and/or hardware modules. Accordingly, implementation of a module that is executable by the computing device802as software is achieved at least partially in hardware, e.g., through use of computer-readable storage media and/or hardware elements810of the processing system804. The instructions and/or functions are executable/operable by one or more articles of manufacture (for example, one or more computing devices802and/or processing systems804) to implement techniques, modules, and examples described herein.

The techniques described herein are supportable by various configurations of the computing device802and are not limited to the specific examples of the techniques described herein. This functionality is also implementable entirely or partially through use of a distributed system, such as over a “cloud”814as described below.

The cloud814includes and/or is representative of a platform816for resources818. The platform816abstracts underlying functionality of hardware (e.g., servers) and software resources of the cloud814. For example, the resources818include applications and/or data that are utilized while computer processing is executed on servers that are remote from the computing device802. In some examples, the resources818also include services provided over the Internet and/or through a subscriber network, such as a cellular or Wi-Fi network.

The platform816abstracts the resources818and functions to connect the computing device802with other computing devices. In some examples, the platform816also serves to abstract scaling of resources to provide a corresponding level of scale to encountered demand for the resources that are implemented via the platform. Accordingly, in an interconnected device embodiment, implementation of functionality described herein is distributable throughout the system800. For example, the functionality is implementable in part on the computing device802as well as via the platform816that abstracts the functionality of the cloud814.

CONCLUSION

Although implementations of systems for generating and propagating personal masking edits have been described in language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations of systems for generating and propagating personal masking edits, and other equivalent features and methods are intended to be within the scope of the appended claims. Further, various different examples are described and it is to be appreciated that each described example is implementable independently or in connection with one or more other described examples.