Patent Description:
Consumer shopping networks and websites form a big sector of the retail market. They strive to provide convenience to consumers; however, they often fall short in providing a true visual representation of customized products. For example, sometimes they are unable to realistically depict various patterns, textures or finishes on digitally customized products such as wearable apparel, toys, furniture, stationaries, and the like.

To increase the realism of customized products displayed digitally, some companies implement markups applied to digital images of the customized products. When a markup is well-described and can be clearly seen on a digitally displayed product, the markup may be easily identified, and thus additional customization of a digital image of the product may be automated. However, when only a small portion of the markup is visible on the displayed product, or the markup is deformed or folded in a complex manner, the markup may be difficult to identify and processing of the markup using automated approaches may be challenging. <CIT> describes a method for producing custom-made garments. Try-on garments are presented to customers. The pieces of the base patterns are marked and modified according to the body contour and the fit preferences of a customer and connected as sample garment for try-on. The marked pieces are recorded and sent to a cutting machine as digital data. Multiple try-on garments can be combined to form new sample garments. <CIT> describes a method for allowing a user to design on a 3D model of an apparel or accessory and automatically generating the manufacturing patterns for the apparel or accessory with the corresponding design. <CIT> describes a three-dimensional modeling apparatus comprising: a stereo processing unit that receives images from a plural number of stereo cameras, and produces a plurality of distance images of an object; a voxel processing unit that receives the plurality of distance images, and selects voxels in which the surface of the object exists; and a modeling unit for producing a three-dimensional model of the object, based on coordinates of the selected voxels. <CIT> describes a method of generating a pseudo-3D user-adapted avatar by using artificial intelligence based on a method of converting a 2D image to the pseudo-3D image having visual and partial functions of a 3D model, and a method of performing a coordination simulation by applying the pseudo-3D coordination image generated by applying total clothes coordination information derived from the artificial intelligence to a user-adapted avatar.

The invention is defined by independent claims <NUM> and <NUM>.

In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present approach.

Embodiments and implementations are described herein according to the following outline:.

In an implementation, techniques are described for generating digital markups and using the digital markups to generate digital representations of customized products. A digital markup may be for example, a digital image that has a unique pattern or a unique arrangement of digital elements.

The techniques may include applying a digital markup to a digital image, using the marked-up digital image to manufacture a physical reference product, and obtaining a reference image representing a photograph of the manufactured reference product and having the imprinted digital markup. The techniques may also include recognizing surfaces of the digital markup depicted in the reference image, and applying a customized pattern to the surfaces recognized in the reference image. The techniques may be used to process digital images of wearable apparel, toys, and other objects that have components or elements covered with patterns or textures.

The techniques may be implemented in computer-based shopping websites, advertising websites, and other computer-based applications that digitally visualize various products offered in various patterns, textures or finishes. An example implementation may include a website featuring a particular piece of clothing that is available in three different versions: a version with a polka-dotted pattern, a version with a plaid pattern, and a version with a zig-zag pattern. Traditionally, to realistically represent the three version on digital media, a website developer would take three pictures of different mannequins, each mannequin wearing a different version of the particular piece of clothing, and post the three pictures on the website for users to see. Each of the pictures would realistically represent the folds of the clothing and the folds of the patterns on the clothing. However, obtaining three (or more) different pieces of the clothing for each piece of clothing that the website features, and taking three (or more) different pictures for each and every featured piece may be time consuming and expensive. But, the presented techniques allow require obtaining only one piece of clothing and only one picture of the piece. The additional pictures may be generated by applying different patterns to the same picture in such a way that the same deformations of the patterns are depicted on each of the customized images.

In an implementation, this is accomplished by using digital markups to cut-out forms used by a manufacturer. For example, a digital markup may be applied to a cut-out form used by a manufacturer to produce a particular piece of clothing. Upon receiving a digital image of a picture of a mannequin wearing the marked-up piece of clothing, the digital image may be modified by replacing the markup depicted on the digital image with a customized pattern. The replacement may be performed according to the digital markup depicted on the digital image. Therefore, instead of generating separate pictures of each individual variation of the piece of clothing, only one physical piece and one picture of the piece is needed. The additional pictures showing the same piece of clothing, but having different patterns, textures or finishes, may be generated automatically.

Since it is important to generate a realistic representation of the piece of clothing as it was featured on a mannequin or a person, a digital representation of the piece needs to capture the fact that the piece may be deformed and wrapped around the model's body. In an implementation, the presented approach allows the capture the deformations and the wrapping of the piece, and allows to realistically reproduce the corresponding deformations of the customized patterns depicted on the customized image of the piece.

In an implementation, a computer-implemented method comprises receiving, at an electronic device, a digital markup image. The digital markup image has one or more reference markup regions, and is associated with a markup identifier. The digital markup image may represent a digital markup that has one or more unique patterns arranged in a unique way or a composition.

In an implementation, a computer-implemented method further comprises receiving a digital layout image that represents a cut-out used to manufacture a reference product. Examples of reference products may include a piece of clothing, a soft toy, a piece of furniture having a cloth upholstery, and the like.

Based on the digital markup image and the digital layout image, a digital markup layout is generated by overlaying the digital markup image over the digital layout image. This may be performed using an image processing tool that allows projecting the digital markup image onto the digital layout image in such a way so that both images can be seen in the resulting digital markup layout.

In an implementation, based on, at least in part, the digital markup layout, one or more manufacturing files are generated. The manufacturing files comprise digital data usable in manufacturing a physical product, also referred to as a reference product. The manufacturing files may be represented in any format that a computer system of the manufacturer can accept and use to control a cutting of a fabric or cloth to form physical pieces of the reference product. Once the reference product is manufactured, a picture is taken of the reference product as it is worn by a model, a mannequin, or used by customers. A digital form of the picture is referred to as a digital reference image.

In an implementation, a digital reference image of the reference product is received. The digital reference image may be for example, a digital photograph of a model wearing the reference product s manufactured based on the manufacturing files.

Since the reference product, such as a pair of leggings, may appear on the digital reference image as wrapped around the model's body, the digital markup may appear on the digital reference image deformed or cropped. Furthermore, the digital reference image of the reference product may include only a portion of the digital markup image. Therefore, recognizing the digital markup just by visually inspecting the digital reference image may be difficult and not straightforward.

In an implementation, a digital reference image of a reference product is digitally analyzed, and one or more markup regions, also referred to as found markup regions, are identified in the digital reference image. The found markup regions are the regions that are suspected to correspond to some regions of a digital markup image. Identifying the found markup regions may be performed using various approaches such as image processing and shape recognition techniques described in detail below.

In an implementation, one or more reference markup regions of a digital markup image used to generate the digital markup layout are retrieved. The one or more reference markup regions are used to determine a region mapping that maps one or more reference markup regions of the digital markup image to one or more found markup regions identified in the digital reference image.

A region mapping is a mapping that maps some of the reference markup regions of the digital markup image onto some of the found markup regions identified in the digital reference image. As described above, since the reference product, such as a pair of leggings, may appear on the digital reference image deformed or cropped, it is possible that only some regions of a digital markup image may be found.

In an implementation, based on, at least in part, a region mapping, a geometry map is generated. A geometry map comprises a plurality of polygons generated based on found markup regions. Each of the polygons corresponds to a different portion of the digital markup image, and each of the polygons captures a deformation of the portion of the digital markup.

In an implementation, an interactive asset image is generated from a digital reference image. This is performed by removing markings from the digital reference image and obtaining an unmarked version of the digital reference image. The unmarked version of the digital reference image may be then customized using customized patterns, such as a user pattern, a pattern designed by a manufacturer, or a pattern designed by a web developer. This may include retrieving the customized pattern from a data storage device or a server, and applying the customized pattern to the unmarked version of the digital reference image to generate a customized product image. The customized pattern may be applied in such a way that the customized pattern appears on to be deformed in the same fashion as the markup regions were deformed on the digital reference image.

In an implementation, a customized product image is displayed on a display device or other display media. For example, the customized product image may be sent to a customer device to cause displaying the customized product image on a display of the customer device. According to another example, the customized product image may be used to generate a new webpage of a website, and the new webpage may be posted on the website. According to other example, the customized product image may be sent to a smartphone operated by a user, and the customized product image may be displayed on a display of the smartphone.

In an implementation, techniques for using digital markups on digital images is implemented using a series of ordered process steps. The steps are intended to illustrate example algorithms that may be implemented using one or more computer.

In an implementation, techniques for using digital markups on digital images for purposes of recognizing surfaces and applying customized patterns to deformable objects such as wearing apparel are implemented using any type of electronic devices, such as a computer, a workstation, a laptop, a server, a smartphone and other electronic device configured to receive, process and transmit digital images.

In an implementation, the steps describe an algorithm, process or other outline for how the functions are implemented in programming instructions for a computer. Any suitable programming language or development environment may be used such as JAVA, OBJECTIVE-C, C++, scripting languages, and the like. In practice, an implementation or embodiment will include program instructions for many steps other than those listed herein, but the specific listings of steps nevertheless indicate the algorithmic information sufficient to communicate to a skilled programmer or software engineer how to implement the specified functions in complete working code.

In an implementation, techniques are implemented in a digital marker computer processing ("DMCP") system. The DMCP may generate digital markers that have certain characteristics that provide information needed to solve a task of mapping markup on a variety of manufactured surface, such as apparel. In the DMCP, the pattern of the digital markup itself can provide or improve the marker coordinate frame, and the pattern itself may characterize a local surface deformation or curvature.

In an implementation, the steps to build, apply and recognize the DMCP comprise defining digital markups, defining digital markup identifiers, applying a digital markup to a digital layout and generating one or more manufacturing files containing digital markups. Once a reference product is manufactured based on the manufacturing files, a region mapping and a geometry mapping are generated. The mappings are used to generate an interactive asset image, which is then used to generate a customized product image by applying a customized pattern to the interactive asset image.

<FIG> illustrates an example image processing computing system in which an implementation may be implemented. In an implementation, an example image processing computing system <NUM> is implemented in a computing environment <NUM>. Image processing computing system <NUM> may be configured to generate digital markups and use the digital markups to generate digital representations of customized products. Image processing computing system <NUM> may generate a plurality of digital markup images, each digital markup image having a unique pattern or a unique arrangement of digital elements.

In an implementation, image processing computing system <NUM> is configured to apply a digital markup to a digital image, use the marked-up digital image to manufacture a physical reference product, obtain a reference image representing a photograph of the manufactured reference product and having the imprinted digital markup, recognize surfaces of the digital markup depicted in the reference image, and apply a customized pattern to the surfaces recognized in the reference image.

In an implementation, image processing computing system <NUM> comprises one or more computer processors <NUM> configured to execute program instructions stored in one or more memory units <NUM>. Image processing computing system <NUM> may also include one or more of: a digital markup generator <NUM>, a layout manager <NUM>, a digital markup applicator <NUM>, manufacturing file generator <NUM>, a reference product manager <NUM>, one or more memory units <NUM>, a markup segmenting unit <NUM>, a markup regions generator <NUM>, a markup recognizer <NUM>, a geometry map builder <NUM>, or an interactive asset builder <NUM>.

In an implementation, digital markup generator <NUM> is programmed or configured to generate a plurality of digital markup images. A digital markup image may be any type of a digital image that has a unique pattern or a unique arrangement of digital elements.

In an implementation, digital markup generator <NUM> is programmed or configured to determine a checksum for a digital markup image. Digital markup generator <NUM> may also be configured to generate a plurality of digital pattern images that have the checksum embedded in each digital pattern image. A checksum may be a count of rows (and columns) in an individual digital pattern of a digital markup image. Based on the plurality of digital pattern images, digital markup generator <NUM> may generate a plurality of rotated digital pattern images, select a subset of the plurality of rotated digital pattern images, and generate the digital markup image based on the subset of the plurality of rotated digital pattern images.

Digital markup generator <NUM> may also be programmed or configured to generate a markup identifier for a digital markup image. A markup identifier may be generated based on contents of the digital markup image. A markup identified may be stored along with the digital markup image in a markup index data structure.

In an implementation, layout manager <NUM> is programmed or configured to receive a digital markup image that has one or more reference markup regions, and is associated with a markup identifier. Layout manager <NUM> is also configured to receive a digital layout image that represents a form of a product used to manufacture a reference product.

In an implementation, digital markup applicator <NUM> is programmed or configured to generate a digital markup layout by overlaying the digital markup image over the digital layout image. This may be performed using an image processing tool that allows overlaying the digital markup image over the digital layout image so that both images can be seen in the resulting digital markup layout.

In an implementation, manufacturing file generator <NUM> is programmed or configured to generate one or more manufacturing files based on, at least in part, a digital markup layout. The one or more manufacturing files comprise digital data usable in manufacturing the reference product. For example, manufacturing file generator <NUM> may be programmed or configured to scan a digital markup layout to generate a scanned file, and modify the scanned file by adding program instructing for using the scanned digital markup layout to generate physical components of the physical reference product.

A scanned file may be sent to a manufacturer and downloaded on a computer system operated by the manufacturer. Upon receiving the scanned file, the computer system operated by the manufacturer may initiate generating one or more physical components of a physical reference product. The component then may be assembled into the reference product.

In an implementation, reference product manager <NUM> is programmed or configured to receive a digital reference image of a physical reference product that has been manufactured based on, at least in part, one or more manufacturing files.

In an implementation, markup segmenting unit <NUM> is programmed or configured to determine a plurality of distinct regions in a digital layout image, and determine whether any of the distinct regions of the plurality of distinct regions contains at least a portion of a digital markup image.

Markup segmenting unit <NUM> may also be programmed or configured to segment a digital reference image into one or more found markup regions by performing one or more of: applying a color shed approach to the digital reference image, determining distance values between color values in areas of interest in the digital reference image and color values in a reference photograph, generating a spatial frequency histogram for the digital reference image, or generating a bandpass histogram for the digital reference image.

In an implementation, markup regions generator <NUM> is programmed or configured to identify, based on a plurality of distinct regions, one or more found markup regions in a digital reference image that potentially include a digital markup image.

In an implementation, markup recognizer <NUM> is programmed or configured to determine a region mapping that maps one or more reference markup regions to one or more found markup regions.

In an implementation, geometry map builder <NUM> is programmed or configured to generate a geometry map based on, at least in part, a region mapping.

In an implementation, interactive asset builder <NUM> is programmed or configured to generate an interactive asset image by removing marking in one or more found markup regions of a digital reference image.

Interactive asset builder <NUM> is also programmed or configured to generate a customized product image based on, at least in part, a region mapping and a geometry map. A customized product image may be generated by applying a user pattern to an interactive asset image. Interactive asset builder <NUM> is also configured to cause the customized product image to be displayed on a display device.

Besides image processing computing system <NUM>, computing environment <NUM> may also include one or more storage devices <NUM> accessible to the image processing system <NUM>.

Computing environment <NUM> may also include a digital camera <NUM> configured to capture digital images of reference products and provide the digital images to image processing computer system <NUM>.

Computing environment <NUM> may also include one or more user computing systems <NUM> that may communicate with image processing computing system <NUM> via a communication computer network <NUM>. A user computing system <NUM> may be configured to store images representing patterns, textures and finishes in internal storages <NUM> and/or external storage devices, such as a storage device <NUM>.

Computing environment <NUM> may also include one or more manufacturer computers that may have one or more manufacturing file processors <NUM>. Manufacturing file processors <NUM> may be configured to receive manufacturing files from image processing computing system <NUM>, and use the manufacturing files to control manufacturing equipment to manufacture physical parts of a physical reference product <NUM> and to assembly the parts into reference product <NUM>. The manufacturing equipment may be maintained by a manufacturer <NUM>.

Computing environment <NUM> may also include shipping services <NUM> that allow receiving reference product <NUM> from manufacturer <NUM>, and ship, or otherwise deliver, reference product <NUM> to image processing computing system <NUM>. Upon receiving reference product <NUM>, a camera <NUM> may be used to take a picture of reference product <NUM>, and a digital reference image of the reference product may be provided to reference product manager <NUM>.

Digital markup generator <NUM> is configured to generate digital markups. Digital markups are generally applicable to any kind of printable deformable surface, including all kinds of wearing apparel, shoes, textiles such as bedding, plush toys, and the like.

For purposes of illustrating an example of a digital marker, certain embodiments and implementations are explained with reference to wearing apparel, such as a pair of leggings; however, the disclosure is not limited to implementations for the wearing apparel. A pair of leggings is a type of apparel worn on the legs and hips. Due to its structure and purpose, the leggings wrap around the legs and hips. The wrapping may be represented or captured using a deformable surface. In an implementation, a digital marker may be placed on any type of a surface, including a deformable surface. Using the presented approach, the coordinates and deformation parameters of the marker may be found for any type of the surface.

A digital markup may be applied to cut-outs or layouts of clothing materials used to assembly a piece of clothing. Once the piece of clothing is assembled, the piece of clothing may have the digital markup imprinted on it. The piece of clothing may be put on a mannequin or a model, and photographed using a camera. The photograph may be then processed to recognize the digital markup on the photograph, and the recognized markup may be used to determine a geometry map of the surface on which the markup is present. The geometry may be used to generate an unmarked digital image, and a custom pattern may be applied to the unmarked digital image to generate an interactive asset depicted the piece of clothing having the customized pattern.

In an implementation, a digital markup is a digital markup image that has certain properties that distinguish the digital markup from other digital markups. A digital markup may comprise one or more digital patterns, also referred to as elements. Each digital patterns may be encoded as an array of bits in a regular pattern that has even intervals between elements. For example, a digital pattern may be encoded as a set of 4x4 dots. If a digital pattern is encoded using a set of 4x4 dots, then the total number of possible patterns is <NUM>.

<FIG> illustrates an example marker element of a digital markup image. In the depicted example, a marker element has four rows and four columns storing dots that are either large dots or small dots. For example, dots <NUM> and <NUM> are small dots, while a dot <NUM> is a large dot. The large dots may correspond to a bit set to 'true,' while the small dots may correspond to a bit set to 'false.

In an implementation, digital pattern or element may be represented by a sequence of bits corresponding to dots depicted in the digital pattern. A large dot may be represented by 'one' or 'true. ' A small dot may be represented by 'zero' or `false. " Hence, a bit may be set to 'true' by making the dot larger, or 'false' by making the dot smaller. The size change may be determined by the best recognizable separation between elements, and the best recognizable scale for determining a 'true' state or a 'false' state.

In an implementation, a digital pattern may be represented by a value of a vector that represents the pattern. The value of an encoded binary number in the markup is limited by <NUM>.

In an implementation, a count of elements in a digital pattern set to 'true' may be limited to n/<NUM> to embed a checksum in the markup.

Binary numbers encoded in a markup that match a number with a lower value if its array of elements is rotated by <NUM> degrees are used to embed rotation in the markup, rather than as separate markup numbers.

<FIG> illustrates an example digital markup image. The depicted digital markup image <NUM> includes a plurality of digital patterns arranged in a table having a certain number of rows and columns. The digital patterns are selected in such a way that non two digital patterns from the plurality of digital patterns are identical. Since no two digital patterns in the plurality of digital patterns in the digital markup image are identical, the arrangement of the plurality of digital patterns in the digital markup image is unique.

In an implementation, a digital markup image has a checksum. A digital checksum is the number of large dots in a 4x4 pattern. For instance, a checksum of <NUM> would mean that the digital markup is that subset of 4x4 patterns that have <NUM> large dots and <NUM> small dots.

In an implementation, a checksum is determined before a digital markup is generated and it is used as a parameter for generating the digital markup image. A checksum need not be actually represented by a number included in the digital markup image. Instead, a checksum is known for the digital markup image before the image is generated.

In an implementation, a digital markup image is created in such a way that a markup pattern is distributed across individual markers.

<FIG> illustrates an example digital markup image having a markup arrangement of dots that is unique for rotations by <NUM>, <NUM>, <NUM>, <NUM> degrees. In the depicted example, a digital markup image comprises a plurality of digital markup patterns, wherein each digital markup pattern is a 4x4 element that has a unique arrangement of dots for each of rotations of <NUM>, <NUM>, <NUM>, and <NUM> degrees. The pattern is designed so that no 4x4 sub-area is used in any other 4x4 sub-area of the design in any of the <NUM> rotations. Therefore, a design of 24x24 elements may use 21x21 or <NUM> unique overlapping 4x4 sub-designs. Using such arrangements reduces the image resolution required to encode the design. Since the ability to generate a design having unique elements in each of rotations of <NUM>, <NUM>, <NUM> and <NUM> degrees, the size limit for the design with this kind of 4x4 uniqueness is 127x127.

In an implementation, a markup has an associated markup identifier, which is referred to as a MarkupID herein for convenience, although other embodiments or implementations may use other designations. A markup identifier may be generated based on a binary number corresponding to a value of the vector obtained from a markup pattern. One approach for generating a markup identifier is to use a binary vector built based on the content of the markup. For example, a digital markup pattern may be represented as a binary string, the binary string may be converted to a binary value, the binary value may be used to generate a vector, and the vector may be used as a MarkupID of the digital markup pattern.

In an implementation, markup identifiers are generated based on ordinal values. The ordinal values may be generated and used to index a markup index, also referred to as a MarkupIndex herein for convenience, although other embodiments or implementations may use other designations. For example, a MarkupID for a particular digital markup may be assigned a next available ordinal value in a MarkupIndex.

The manner in which markup identifiers are created and used to index a MarkupIndex may be encoded as software code such as C++ code. For example, a C++ object may be created that includes instructions which, when executed, cause generating MarkupIDs and a MarkupIndex.

A process of creating digital markup identifiers for digital markups and digital markup index may include digitally creating and storing, in computer storage, a list of MarkupIDs based on the properties of the digital markups. For example, a particular MarkupID for a particular digital markup may be created based on either a value of the digital markup vector or a next available ordinal value in a MarkupIndex.

The process may also include building a mapping from a MarkupIndex to MarkupIDs. This process may be performed in either direction. For example, if the MarkupIDs are created based on the markups' vectors, then the MarkupIDs may be used to generate the MarkupIndex, and then the mapping. However, if a sequence of ordinary numbers is used to generate a MarkupIndex, then the ordinary numbers may be used to generate the MarkupIndex, and the ordinary numbers may be assigned to digital markups as the MarkupIDs.

In an implementation, a MarkupID associated with a markup includes information about rotations applicable to the markup image. For example, if a digital markup image is generated in such a way that its elements are unique in a rotation by for example, <NUM> degrees, then the MarkupID generated for the digital markup image may include not only the information about a markup pattern of the digital markup, but also the information about the rotation. Alternatively, the rotation information may be stored separately from the MarkupID. For example, the rotation information may be associated with the MarkupID.

In an implementation, if a rotation value is included in a MarkupID for a particular digital markup image, then a process of creating digital markup identifiers includes a method for returning the rotation value for the particular digital markup. If the rotation value is a MarkupID, then returning value for the particular digital markup would include returning the rotation value. However, if the rotation value is not included in the MarkupID, then two separate queries may be issued: one to request a rotation value and another to request the MarkupID.

In an implementation, digital markup images may be stored in a storage device, such as device <NUM> depicted in <FIG>. The storage device may store the digital markup images using any type of data organization. For example, the digital markup images may be stored in association with a MarkupIndex and in association with any additional information if such as available or provided.

<FIG> illustrates an example process for using digital markups on digital images for the purpose of recognizing markup regions in digital reference images and applying customized patterns to deformable objects.

In step <NUM>, one or more digital markups are defined. Digital markups may be defined in advance. If two or more digital markups are defined, then the digital markups are defined in such a way that no two, of the two or more digital markups, are identical. Examples of digital markups are described in <FIG> and <FIG>.

In an implementation, a digital markup is defined by generating a unique arrangement of dots of different sizes. The unique arrangement may be represented in a digital markup image. A digital markup image may be stored using any data format, including the formats such as mpeg, jpeg, pix, and the like.

In step <NUM>, markup identifiers are assigned to digital markups. Assigning a markup identifier to a digital markup image may include generating a MarkupID based on a value of the vector representing the digital markup image, and assigning the value as the MarkupID of the digital markup image.

Referring again to <FIG>, in step <NUM>, a digital markup image is applied to a digital layout image. Applying a digital markup image to a digital layout image may be implemented using an automated approach that automatically fits the digital markup image to the digital layout image, or overlays the digital markup image over the digital layout image. As described above, a digital markup image may be a digital image that represents a unique combination of dots, or squares, that have different sizes, and each of the dots, or squares, represents either the binary ` <NUM>' or the binary `<NUM>.

A digital layout image may be a digital image representing a physical cut-out or a physical form that a manufacturer may use to cut physical pieces of for example, a cloth, a piece of garment, or a toy, that later may be assembled into a physical product. Examples of digital layout images are described in <FIG>.

<FIG> illustrates example digital layout images. The depicted example digital layout images include two images: a digital layout image <NUM> and a digital layout image <NUM>. Both images represent physical cut-out forms that may be used by a manufacturer, or a seamstress, to cut out pieces of fabric to assembly for example, a pair of leggings.

A cut-out form usually has certain markings imprinted on it. The certain markings may include several different types of lines, and a count of the different lines depicted in the cut-out form depends on the preferences implemented by a manufacturer. For example, the markings may represent leading lines for cutting the fabric, leading lines for stitching the fabric, and/or leading lines beyond which neither cutting nor stitching could be performed.

In the example depicted in <FIG>, a digital layout image <NUM> represents a cut-out form that has three types of markings. One marking, depicted using a broken line <NUM>, indicates a cut-out line. This line is to be followed to cut a fabric. Another marking, depicted using a solid line <NUM>, indicates a stitch line which is to be followed by a sewing machine to assembly the garment. Other marking, depicted using a dashed line <NUM>, indicates a safe line beyond which neither cutting nor stitching should be performed.

In an implementation, a digital markup image is applied to a digital layout image provided by a manufacturer or a seamstress. Applying a digital markup image to a digital layout image may be performed by overlaying the digital markup image over the digital layout image in such a way that an outline of the digital markup image follows an outline of the digital layout image as much as possible. By overlaying a digital markup image over a digital layout image, a digital markup layout is generated.

Since a digital markup image has usually a rectangular shape and a digital layout image rarely has a rectangular shape, overlaying the digital markup image over the digital layout image in such a way that the corresponding outlines follow each other as much as possible may involve stretching some areas of the digital markup image. The overlaying and the stretching may be performed using various techniques. One technique may involve dividing the digital overlay image into a grid, and mapping the elements of the digital markup image onto the grid identified in the digital overlay image.

In an implementation, a digital layout image is divided into a grid comprising a plurality of grid elements. A grid element may have any type of shape, not necessarily a rectangular shape. A size of the grid may be determined based on a size of the digital markup image, a resolution of the manufacturing process, a size of the digital layout image, and/or a size of the physical reference product that will be assembled based on a cut-out form represented by the digital layout image. Elements of the grid are also referred to as distinct regions of the digital layout image.

In an implementation, for each separate and distinct region in a digital layout image, a size of a markup unit is set. Also, a size of an enclosing boundary or a grid element may be set based on a resolution of a manufacturing process and a size of the resulting physical product. For example, if the resulting product is a small product, such as a case of a cell phone, a grid size may be set to <NUM>; however, if the resulting product is a large product, such as a shirt, then a grid size may be set to <NUM>, or so. The size of the grid may be adjusted or modified if the initial size of the grid is too small or too large.

Furthermore, for each separate and distinct region in a digital layout image, an offset or border of a markup is set within a grid element.

In an implementation, for each separate and distinct region in a digital layout image, a color of the color of the markup. A color may be set based on the instructions provided by a manufacturer or based on a technique used to detect the markup during the manufacturing process and/or by a system that will be used to process a photograph of a resulting physical product. The color may be also set based on the capabilities of the processing system so that the markup can be removed from the photograph of the resulting physical product. In an implementation, the color is set to yellow, and the markup is printed on the resulting physical product at <NUM>% yellow for a color segmentation.

In an implementation, a width and a height of the grid is set. The width and the height may be set in such a way so that the grid can over the entire region of a digital layout image to be marked using a digital markup image.

In an implementation, a grid is generated according to the setting described above. The grid may be placed, or overlaid over a digital layout image. The grid may be simply projected onto a digital layout image, or a rotated or wrapped in such a way that there is a continuity of the grid across the corresponding seams joining the respective regions. An example of a digital marker image overlaid over a digital layout image is described in <FIG>.

<FIG> illustrates an example of overlaying a digital markup image over a digital layout image. The depicted example depicts two digital layout images: a digital layout image <NUM> and a digital layout image <NUM>. Both images represent physical cut-out forms that may be used by a manufacturer, or a seamstress, to cut out pieces of fabric to assembly for example, a pair of leggings.

Both digital layout image <NUM> and digital layout image <NUM> have three types of markings. One marking, depicted using a broken line <NUM>, indicates a cut-out line. This line is to be followed to cut a fabric. Another marking, depicted using a solid line <NUM>, indicates a stitch line which is to be followed by a sewing machine to assembly the garment. Other marking, depicted using a dashed line <NUM>, indicates a safe line beyond which neither cutting nor stitching should be performed.

In <FIG>, digital layout image <NUM> is used to illustrate overlying <NUM> a digital markup image over a layout image. As shown in <FIG>, a grid is placed onto digital layout image <NUM>, and a digital markup image is projected, or overlaid, onto the grid of digital layout image <NUM>. In this example, the grid is stretched along the widest portion of digital layout image <NUM>. In other examples, the grid may be rotated, wrapped, or otherwise deformed to allow for continuity across seams that would be used to join individual pieces of fabric to assembly a physical product.

In the example depicted in <FIG>, a digital markup image does not extend to the entire perimeter of a digital layout image. This may be recommended to preserve a scale of the design. For example, if the digital layout image represents a cut-out form of a pair of leggings, then a scale of the markup pattern in for example, the vertical directions of the leggings may be preserved. If the design layout image represents a cut-out form of a fitted shirt, then a scale of the design in for example, a horizontal direction of the shirt may be preserved.

A grid-based approach for overlaying a digital marker image over a digital layout image while preserving a scale of the design in at least one direction may be appropriate for various products in which depicting the design in the correct scale matters. For this products, this approach may be more appropriate than for example, fitting the pattern to the exact edge. If a uniform scale is more important than the ability to tile a pattern across the edge, then fitting the pattern to the exact edge may be undesirable. Referring again to the example of a pair of leggings, it may be desirable to map a digital markup image onto a digital layout image in such a way so that at least some patterns will appear continuously across both legs.

<FIG> illustrates an example design area that shows a digital markup image overlaid over a digital layout image and that is displayed to a user in a graphical user interface. The example depicts the design area for a pair of leggings. To illustrate an example, a lower portion of the legging has been cropped.

In the depicted example, a digital layout image represents a cut-out form that has three types of markings. One marking, depicted using a broken line <NUM>, indicates a cut-out line. This line is to be followed to cut a fabric. Another marking, depicted using a solid line <NUM>, indicates a stitch line which is to be followed by a sewing machine to assembly the garment. Other marking, depicted using a dashed line <NUM>, indicates a safe line beyond which neither cutting nor stitching should be performed.

As it may be gleaned from <FIG>, a digital markup image is overlaid over a digital layout image in such a way that the pattern of the digital markup overlaps itself in the center of the example design area. When the pattern is printed on the fabric, it is masked by the cut lines.

In step <NUM>, one or more manufacturing files are generated based on a digital markup layout. A digital markup layout may be a digital image obtained by overlaying a digital markup image over a digital layout image. If the digital markup layout is obtained by overlaying the digital markup image over the digital layout image, then the digital markup layout include at least a portion of the digital markup imprinted on the digital layout.

Manufacturing files are digital files that provide at least a digital markup layout. Manufacturing files may also include instructions that specify the details for cutting physical components out of fabric, or other material, and the details for processing the components to assembly the components into a physical reference product. Manufacturing files may be represented in any format that a computer system of a manufacturer can accept and process.

Manufacturing files may be generated based on a digital markup layout and based on settings used by a software application used to overlay a digital markup image over a digital layout image. This may be performed automatically by selecting one or more settings from a graphical user interface displayed by a software application used to generate a digital markup layout. This may also be performed manually by having a user select a digital markup layout and embedding instructions for processing the digital markup layout. For example, using a graphical user interface, a user may select a digital markup layout, and request generating a manufacturing file based on the digital markup layout.

In an implementation, one or more manufacturing files are transmitted to a computer system of a manufacturer. The files may be transmitted via any type of communications connection and any type of communications network.

Upon receiving one or more manufacturing files, an employee of a manufacturer may print out a digital markup layout on a substrate. For example, if the digital markup layout is intended for a manufacturing a piece of clothing, then the digital markup layout may be printed on a cloth or a fabric.

Once a digital markup layout is printed on a substrate, the substrate may be cut into pieces that are to be used to assembly a physical reference product. The cutting may be performed manually or automatically. For example, an employee may use the digital markup layout printed on a cloth, and cut the cloth along the cutting lines, such as line <NUM> in <FIG>.

If instructions for cutting a substrate are included in a manufacturing file provided to a manufacturer, then the instructions may be used by either an employee or by a computer system of the manufacturer to perform the cutting to obtain one or more components of a physical reference product.

Once one or more components of a physical reference product are cut, or otherwise obtained, the components are assembled into the physical reference product. For example, the pieces may be sewn together to form a pair of leggings. The pair of legging will have a digital markup imprinted on the leggings.

In step <NUM>, a test is performed to determine whether a physical reference product has been received from a manufacturer or a shipper. If the physical reference product has been received, then step <NUM> is performed. Otherwise, the test in step <NUM> is repeated.

In some situations, a physical reference product is not received from a manufacturer because one or more manufacturing files generated based on a digital markup layout were corrupted or had some errors. In such situations, steps <NUM>-<NUM> may be repeated so that new manufacturing files are generated, and a physical reference product is created and provided.

In an implementation, a reference product is received from a manufacturer or a shipper facilitating delivery of the reference product. A reference product is typically a physical product manufactured based on manufacturing files provided to a manufacturer.

In step <NUM>, a digital reference image is generated based on a reference product. A digital reference image may be a photograph of a reference product. Depending on a nature of the reference product, the product may be featured on a model, a mannequin, a stand, or in any other arrangements appropriate for demonstrating the reference product. For example, if a reference product is a pair of leggings, then the leggings may be put on a mannequin, and a photograph of the mannequin wearing the leggings may be taken. If a reference product is a piece of furniture with a cloth upholstery, then the piece of upholstered furniture may be features in a showroom or in a private house.

In an implementation, a reference product may be provided with instructions specifying how the reference product is to be featured, lit, illuminated, or otherwise processed. The instructions may by encoded in a code provided along with the reference product. The instructions may also be obtained from a server, a cloud system or other computing system.

Instructions that accompany a reference product may specify a rig to be used to photograph the reference product. The instructions may also specify the method that was used to process digital process markups imprinted in the reference product. Furthermore, the instructions may include suggestions for placing, illuminating and arranging the reference product to product a high-quality photograph of the reference product. The instructions may also specify a position of the camera, a position of the light sources illuminating the reference product, and locations of other elements that may enhance the quality of the photograph of the reference product.

In an implementation, a reference product is placed in an automated photographic system. The manner in which the reference product is placed in the system may be specified by instructions associated with the reference product, or by a photographer or a designer. For example, the reference product may be placed on a stand, a fixture, or a mounting configured to support the reference product.

In an implementation, a reference product placed in an automated photographic system is illuminated using one or more light sources or any type of lighting. The type of lighting and the positioning of the lighting may be specified in the instructions that accompany the reference product, by a photographer or by a designer.

In an implementation, one or more photographs are taken of a reference product. A photograph may be taken using any type of camera. For example, the photograph may be taken using a digital camera which is configured to capture digital images. The photograph may also be taken using an analog camera. In this case, a photograph taken using an analog camera may be digitally scanned, and a digital image may be generated from the photograph. The obtained digital images are referred to as digital reference images, and may be stored in a storage device.

<FIG> illustrates an example digital reference image. The example depicts a digital reference image of a pair of leggings <NUM> featured on a model <NUM>. Pair of leggings <NUM> is depicted in such a way that the front and the back of the leggings wrap around model <NUM>, and a leg of the leggings wraps around a leg of model <NUM>, as depicted using element <NUM>.

In step <NUM>, a digital reference image is digitally processed to segment any markup in the digital reference image. Segmenting a markup in an image is also referred to as recognizing the markup in the digital reference image.

In an implementation, a digital reference image is analyzed using an electronic device to recognize markups depicted in the image. Recognizing markups in a digital reference image presents many challenges. For example, the markups may not be clearly visible in a digital reference image. Furthermore, it may be difficult to distinguish an edge or an element of a markup in a region in which seams of the reference product are depicted, or in a region in which the reference product wraps around a mannequin and thus it is only partially visible on the digital reference image.

Also, it may be difficult to recognize a markup in a digital reference image if the image has a low contrast between the markup imprinted in the digital reference image and regions that do not include markups. Moreover, it may be difficult to recognize a markup in a digital reference image if the image has a poor quality, a low resolution, a low contrast, or a low brightness. In addition, a digital reference image may include so many different elements that recognizing a markup in the image may be quite difficult. Furthermore, a digital reference image may include not one, but a plurality of markups. Therefore, the process of recognizing a markup in a digital reference image may be sometimes repeated several times until one or more markups are segmented in the image.

<FIG> illustrates an example digital reference image to be processed using a color filtering approach to determine a digital markup imprinted in the image. The example depicts a digital reference image of a pair of leggings <NUM> featured on a model <NUM>. Pair of leggings <NUM> is depicted in such a way that the front and the back of the leggings wrap around model <NUM>, and each leg of the leggings wraps around the model's leg. The example depicts that the leggings wraps around a right leg of model <NUM>, as depicted using an element <NUM>.

In an implementation, a digital reference image is processed to determine a plurality of distinct regions in the digital layout image. The distinct regions may be determined in the digital layout image by analyzing color values for the digital layout image and determining the distinct regions that include the color values that are different than the color values of other regions. For example, if a digital reference image depicts a pair of white leggings <NUM> with black (or yellow) printing on it, then the digital reference image may be processed to identify a plurality of distinct regions that include the back (or yellow) printing, and not the white regions.

In an implementation, a digital reference image is processed to determine a plurality of distinct regions in the digital layout image based on color and color differences in the digital reference image. Using this approach, a product image may be partitioned into different regions based in part on color differences. For example, each location of the product markup image input image may be assigned a color difference value indicating how different the location is from neighboring locations in terms of color. Each location may be assigned an image region of the plurality of image regions, and locations may be considered for region assignments according to an order that is based, at least in part, on the color difference value assigned to the pixel. For example, locations with low associated color difference values may be assigned regions before locations with high associated color difference values. This approach may be referred to as a color shed approach.

In an implementation, region information determined based in visualization of a customizable product in a digital reference image allows determining a position of particular markup portions in the digital reference image. Based on the determined positions, an image processing system may determine instructions for rendering computer-generated visualization of the customized products. The region information may also be utilized to verify the quality of a customized product after the customized product has been manufactured.

In an implementation, a digital reference image may be partitioned into a plurality of image regions. The partitioning processes may result in partitioning at least the markup portion of the digital reference image into a set of regions, where each region of the set of regions represents a portion of the markup.

In an implementation, an image partitioning process may result in determining regions information that identifies, for example, for each found region, the image locations that belong to the region. The information may also include descriptions of the area of each region, and/or the representative color of the region. In some implementations, each image location corresponds to a separate pixel. In other implementations, each image location is a group of pixels, or a markup element.

In an implementation, the resulting region information is utilized in the automatic visualization of a custom product, which may be customized according to customer-provided parameters.

<FIG> illustrates an example process for identifying markups in digital reference images using a color filtering approach.

In step <NUM>, a digital image comprising a markup is received. The digital image may correspond to a digital reference image described above.

In step <NUM>, one or more locations in the digital image are identified. A location identified in the digital image may be a group of pixels or just a pixel. Hence, a location may correspond to either a group of pixels or an individual pixel.

In step <NUM>, one or more neighboring locations that neighbor to a particular location in a digital image are identified. If locations include groups of pixels, then for a particular location, one or more neighboring locations adjacent to the particular location include groups of pixels. If locations include individual pixels, then for a particular location, one or more neighboring pixels adjacent to the particular location are just individual pixels.

In step <NUM>, a color difference value for each of the locations is determined. For example, for a particular location, a particular color value for the particular location and color values for the neighboring locations are determined, and an average value of differences between the particular color values and the neighboring color values is determined. The color difference value may also be determined using other approaches.

The process of determining a color difference value for each location is repeated for each location identified in the digital image.

If in step <NUM>, if is determined that the process has been repeated for all locations identified in the digital image, then step <NUM> is performed. Otherwise, steps <NUM>-<NUM> are repeated for the remaining locations.

In step <NUM>, the locations identified in the digital image are ordered according to the color difference values determined for the locations.

In step <NUM>, the digital image is partitioned based on the ordered locations into one or more regions. The partitioning includes assigning, to each location, an image region of the plurality of image regions. The assignment between the locations and regions may be performed according to an order that is based, at least in part, on the color difference value assigned to the pixel. For example, locations with low associated color difference values may be assigned regions before locations with high associated color difference values.

By assigning regions to locations based on respective color difference values determined for the locations, the locations with low color difference values (i.e., the locations that are very similar to their surrounding locations) may be more likely to be at the center of the determined regions or in other non-border locations of the regions. Such an image partitioning approach may cause the resulting partitioned image to be partitioned into regions that are more uniformly-colored than regions determined according to other approaches.

In step <NUM>, based on the partitioning, data is identified that represents at least a portion of a markup in the digital image.

It is possible that only a portion of the markup, not the entire markup, is identified in a digital reference image. This may happen because some markups may not be clearly visible in the digital reference image. Furthermore, this may happen because it may be difficult to distinguish an edge or an element of a markup in a region in which seams of the reference product are depicted. This may also happen in a region in which for example, the reference product wraps around a mannequin and thus it is only partially visible on the digital reference image.

In step <NUM>, a particular markup is identified based on the data representing at least a portion of the markup. Different approaches may be implemented to identify a particular markup based on the data representing at least a portion of the markup using the data representing at least a portion of the markup. One approach includes generating a mapping between found markup regions and regions identified of a digital markup image.

Other approaches for identifying at least a portion of the markup imprinted on a digital reference image may include determining a distance value between locations. A distance value may be determined for example, as a difference between the color of a pixel in a digital reference image and the color of a digital markup image in a particular color space. The distance values may be computed for each pixels in a digital reference image, and a region may be created based on a group of pixels for which the respective color differences are the smallest.

Another approach for determining a markup region in a digital reference image may use a color spatial frequency approach. This approach may include computing a bandpass determined based on the dot sized in a digital markup image and comparing the bandpass with the characteristics of the pixels of the digital reference image.

Referring again to <FIG>, in step <NUM>, one or more found markup regions are identified in a digital reference image. The found markup regions are the regions that most likely, or with some certainty, include at least a portion of a markup.

In an implementation, one or more found markup regions are used to build a list of MarkupIDs of markups potentially included in the found markup regions.

In an implementation, for each found markup region, of one or more found markup regions identified in a digital reference image, a list of MarkupIDs is generated. A list of MarkupIDs for a particular found markup region identified in a digital reference image is a list that includes identifiers of markups that are included in the particular found markup region of the digital reference image, or may be included in the particular found markup region of the digital reference image. For example, a particular found markup region may include one or more markups, one or more portions of one or more markups, one or more portions of one markup and a depiction of entire another markup, and so forth.

A list of MarkupIDs generated for a particular found markup region may be any type of data structure, such as a data table, a data index, a data space region indexed using pointers, and so forth.

In an implementation, a list generated for a particular found markup region comprises a MarkupID of a particular digital markup image, a position information of the particular found markup region within a digital reference image, and any additional metadata that may be used to identify the particular digital markup image. A list for a particular found markup region may include more than one MarkupID if more than one digital markup image may be included in the particular found markup region.

A list comprising MarkupIDs may be generated for a particular found markup region if the particular found markup region meets certain conditions. For example, a test may be performed to determine whether a particular found markup region is a valid region. The test may include checking a size of the dots included in the found markup region, a size of the areas between the dots, and the like.

A particular MarkupID may be included in a list for a particular found markup region if the particular MarkupID meets certain criteria. For example, a test may be performed to determine whether a checksum of the particular MarkupID is valid. As described below, a checksum of a markup may be defined by a count of dots in the vertical direction and/or a count of dots in the horizontal directions. For example, if the markup is expected to have five dots horizontally and five dots vertically, then a checksum of the markup is 5x5. Testing the checksum of the markup may involve testing whether the markup has the checksum of 5x5.

A particular MarkupID may be included in a list for a particular found markup region if the particular MarkupID corresponds to a markup that has acceptable extents defined in terms of dots in a markup pattern of the markup. This may include rotations of the markup elements within the markup pattern. Based on extents, a markup transformation for the markup may be generated and used to determine and verify a map of the markup dots.

A particular MarkupID may be included in a list for a particular found markup region if a digital signature of a corresponding markup is acceptable or correct. A digital signature of a markup may be defined as a count of regions distinguishable in the markup. For example, if a markup includes <NUM> tiles of markup elements, then a digital signature of that markup may be <NUM>. Testing a digital signature of a markup may include determining whether the markup indeed includes <NUM> tiles.

Generating a list of MarkupIDs for a particular found markup region may include determining a rotation for a markup associated with a MarkupID. Some markups may be expressed using one or more rotation parameters. An example of such a markup is described in <FIG>.

In an implementation, additional information about a markup may be obtained or generated before a MarkupID of the markup is included in a list of MarkupIDs for a found markup region. For example, a rotation information of the marker may be obtained and included in the list. Furthermore, surface derivatives may be computed based on a spacing of dot centers within the found markup region may be calculated. Moreover, the corrected markup region geometry may be calculated using the information about the markup transformation and the information about the surface derivatives, and included in the list.

In step <NUM>, one or more reference markup regions identified in a digital markup image are mapped onto one or more found markup regions identified in a digital reference image. The resulting mapping may be referred to as a region mapping.

In an implementation, one or more reference markup regions are identified in a digital markup image. This may be performed before a digital markup layout is sent to a manufacturer, or when a digital markup image is generated and stored.

In an implementation, one or more reference markup regions are identified in a digital markup image by dividing the digital markup image into one or more areas corresponding to individual markup elements. For example, if a markup includes four markup elements repeated horizontally and four markup elements repeated vertically, then the markup may be divided into <NUM> areas, which may be equated to one or more reference markup regions.

For each reference markup region, of one or more reference regions, a position of the reference markup region within a digital reference image and extents of the reference markup region may be identified.

In an implementation, a list of reference markup regions for a digital markup image is created. A list of reference markup regions may include information about the regions identified within the digital markup image. For each reference markup region identified in a digital markup image, a list may include a MarkupID of the markup, information about the position and extents of the markup region.

Once lists of reference markup regions for a digital markup image is created, the regions of the markups may be mapped onto one or more found markup regions identified in a digital reference image. The mapping may be performed by traversing the lists, and trying to match the characteristics of the reference markup region with the characteristics of found markup regions. The traversing and matching may be accelerated by using for example, data hashing techniques.

In step <NUM>, a geometry map is generated. A geometry map may be generated based on a mapping between reference markup regions of markups and one or more found reference regions identified in a digital reference image.

In an implementation, a geometry map is generated by building a set of polygons. A set of polygons may be determined based on reference markup regions identified in a markup and mapping the polygons onto one or more found reference regions identified in a digital reference image. Each vertex of a polygon may have an associated reference point in a reference markup region and a found marker point in a found reference region. A set of polygons may be build using the information in a list of reference markup regions for the markups, extents information for the markups, and adjacency information for the reference markup regions.

In an implementation, a set of polygons determined for a geometry map includes information about the polygons that are adjacent to each other. The adjacency information for two polygons may be determined based on a distance between the centers of the polygons and based on a distance tolerance value. The geometry map may also include information about an average contributions of adjacent vertices of the neighboring polygons to each of the polygons.

In an implementation, once a geometry map for a digital reference image is created, markups, or portions of the markups, may be removed from the digital reference image. Since the purpose of depicting a markup in a digital reference image is to register a reference points of the markup in the digital reference image, once a geometry map is created for the digital reference markup, a further processing of the digital reference markup may be performed on an image in which the markups have been removed.

In step <NUM>, an interactive asset is rendered from an unmarked digital image. This may include processing a digital reference image by removing any marking. An example of the process is described in <FIG>.

<FIG> illustrates an example image having markups removed. The example depicts a pair of leggings <NUM> featured on a model <NUM>. In contrast to a pair of leggings depicted in <FIG>, the leggings in <FIG> do not have any imprinted marking. For example, if the leggings in <FIG> were white with a yellow imprinted marking, then the leggings in in <FIG> show a white pair of leggings with no yellow marking.

In <FIG>, a pair of leggings <NUM> is depicted in such a way that the front and the back of the leggings wrap around model <NUM>, and each leg of the leggings wraps around the model's leg. The example depicts that the leggings wraps around a right leg of model <NUM>, as depicted using an element <NUM>.

In an implementation, an unmarked digital image is used as an interactive asset image. The interactive asset image may have the same shape and size characteristics and the unmarked digital image. Furthermore, the interactive asset image may have the same background characteristics and the unmarked digital image.

In an implementation, an interactive asset image is further processed. The further processing may include applying a user pattern to the interactive asset image to generate a customized product image. For example, if an interactive asset image depicts a pair of leggings, the interactive asset image may be interactively customized by allowing a user to apply a custom pattern to the leggings and generate an image of the customized product. The customization may allow for deforming the customized pattern in the customized product image in the same fashion as a digital markup image was deformed on some surfaces of the digital reference image described above.

In step <NUM>, a user pattern is mapped onto an interactive asset according to a geometry mapping to generate a customized product image. A user pattern may be for example, a customized pattern generated by a customer who designs the interactive asset using a website application. A user pattern may also be a customized pattern generated by a web developer who designs different customization options that may be available to customers who access the website.

In an implementation, a customization of an interactive asset image using a user pattern allows for deforming the user pattern in a customized product image in the same fashion as a digital markup image was deformed on some surfaces of the digital reference image.

In an implementation, a customization of an interactive asset image using a user pattern includes encoding means for rendering an unmarked digital image into an interactive asset, and encoding means for mapping a user pattern or image in a photo-real manner using a geometry map onto an interactive asset image.

<FIG> illustrates an example customized product image that is rendered by applying a user pattern to an interactive asset image. The figure depicts a pair of leggings <NUM> modelled by a model <NUM>. Pair of leggings <NUM> has a user-defined pattern imprinted on the garment. Noticeably, the user pattern is deformed in the customized product image in the same fashion as a digital markup image was deformed on the surfaces of the digital reference image depicted in <FIG>. In particular, pair of leggings <NUM> is depicted in such a way that the front and the back of the leggings wrap around model <NUM>, and a leg of the leggings wraps around a leg of model <NUM>, as depicted using element <NUM>.

In an implementation, an approach for customizing digital images provides an automated method for realistically visualizing customized image. The approach includes encoding a digital signature in markups used to manufacture reference products.

In an implementation, an approach includes processes that allow imprinting markups on a product manufactured with specified colors and patterns. The methods allow detecting specific marked-up regions in a digital reference image, find the position and orientation of these regions, apply photo-realistic visualization of customized patterns to the product, and display digital images of the customized product.

The processes work well when at least a portion of the markups is visible in a digital reference image. For example, the software algorithm used for finding the markup in a digital reference image may be fully automated when the markup is well-visible in a digital reference image, or when only a small portion of the markup is visible.

In an implementation, techniques for detecting and mapping digital markers onto digital reference image may be implemented in specialized program application. The application may be encoded using open source computer vision projects, such as the openCV.

In an implementation, an approach for using digital markups to customize products is implemented in an Augmented Reality system. The system may be calibrated using a set of digital markup images and other calibration techniques. For example, a digital marker boundary frame maybe used to provide a coordinate frame that allows identifying the marker in an image. Markers used to calibrate the Augmented Reality system are usually presented on a flat or non-deformed surface to allow an efficient identification and mapping of the markup. In an implementation, digital markers is encoded using various techniques, including QR codes.

According to one implementation, the techniques described herein are implemented by one or more special-purpose computing devices.

For example, <FIG> is a block diagram that illustrates a computer system <NUM> upon which an implementation of the invention may be implemented.

According to one implementation, the techniques herein are performed by computer system <NUM> in response to processor <NUM> executing one or more sequences of one or more instructions contained in main memory <NUM>. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions.

ISP <NUM> in turn provides data communication services through the world-wide packet data communication network now commonly referred to as the "Internet" <NUM>.

Claim 1:
A computer implemented method comprising:
receiving, at an electronic device, a digital markup image that has one or more reference markup regions, and is associated with a markup identifier;
receiving, at the electronic device, a digital layout image that represents a form of a product used to manufacture a reference product;
generating, using the electronic device, a digital markup layout by overlaying the digital markup image over the digital layout image;
based on, at least in part, the digital markup layout, generating (<NUM>) one or more manufacturing files comprising digital data usable in manufacturing the reference product;
receiving, (<NUM>) at the electronic device, a digital reference image of the reference product that has been manufactured based on, at least in part, the one or more manufacturing files;
identifying (<NUM>) one or more found markup regions in the digital reference image;
determining a region mapping that maps (<NUM>) one or more of the one or more reference markup regions to the one or more found markup regions;
based on, at least in part, the region mapping, generating (<NUM>) a geometry map, wherein the geometry map comprises a plurality of polygons generated based on the one or more found markup regions of the digital markup image;
each polygon corresponds to a portion of the digital markup image, and each of the polygons captures a deformation of said portion of the digital markup image;
generating (<NUM>) an interactive asset image by removing the one or more found markup regions from the digital reference image;
based on, at least in part, the region mapping and the geometry map, generating (<NUM>) a customized product image by applying a user pattern to the interactive asset image;
causing the customized product image to be displayed on a display device.