Patent Publication Number: US-8532340-B2

Title: Projecting patterns for high resolution texture extraction

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application claims the benefit of International Patent Application Serial No. PCT/US10/50863 filed on Sep. 30, 2010. The disclosures of the International Patent Application are hereby incorporated by reference for all purposes. 
     BACKGROUND 
     Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section. 
     A variety of applications ranging from visual inspection to autonomous navigation recognize objects in an image and compute their poses in a three dimensional scene. Many such applications employ a pre-stored model of an object for accurate detection. When objects are to be detected without prior information, the task becomes more complex. Objects with textured surfaces render the task of object recognition even more complicated. 
     Augmented reality (AR) refers to a view of a physical (real) world environment whose elements are augmented by virtual, typically computer-generated, imagery, thereby creating a mixed reality. The augmentation may be conventionally in real time and in context with environmental elements, such a sporting event, a training exercise, a game, etc. AR technology enables the information about surrounding real world of a person to become interactive and digitally usable by adding object recognition and image generation. Artificial information about the environment and the objects may be stored and retrieved as an information layer separate from a real world view layer. 
     Texture extraction may be employed for AR to allow more realistic integration or recognition of real world objects. For texture extraction, projected patterns are sometimes used in order to determine the three dimensional shape of objects. These patterns are often called “structured light” and can help extract the three dimensional shape of features on objects such as faces, castings, etc. Conventional systems typically turn off or otherwise subtract out any projected image(s), sometimes removing (or cancelling) a complete three dimensional grid before texture processing. The texture is then calculated by space/frequency analysis on the pixels in the image of the subject as it would be without the projected patterns. 
     The present disclosure appreciates that there are numerous limitations with texture extraction in AR systems. For example, textures below a resolution capability of a detection device (e.g., a camera) may not be extracted using conventional techniques. 
     SUMMARY 
     The present disclosure describes an Augmented Reality (AR) scene capture system capable of high resolution texture extraction. According to some examples, the system may include a projector for projecting one or more fine-lined patterns on a three dimensional subject to be captured, a camera for capturing an image of the subject with the projected patterns, and a processor. The processor may determine a textured portion of the subject, record local Moiré pattern pitches on the textured portion of the subject, and adjust one or more properties of the projected patterns such that the projected patterns substantially match the textured portion of the subject. 
     The present disclosure further provides a method for high resolution texture extraction in an Augmented Reality (AR) scene capture system. According to some examples, the method may include determining a textured portion of a three dimensional subject to be captured, extracting a three dimensional shape for the textured portion of the subject, and projecting two fine-lined patterns on the textured portion of the subject. According to other examples, the method may further include recording local Moiré pattern images on the textured portion of the subject, selecting two or more Moiré patterns with pitches that are clear in the recorded images, and determining a pattern angle and a spatial frequency. 
     The present disclosure also describes a computer-readable storage medium having instructions stored thereon for high resolution texture extraction in an Augmented Reality (AR) scene capture system. According to some examples, the instructions may include determining a textured portion of a three dimensional subject to be captured, extracting a three dimensional shape for the textured portion of the subject, projecting two fine-lined patterns on the textured portion of the subject, recording local Moiré pattern images on the textured portion of the subject, and selecting two or more Moiré patterns with pitches that are clear in the recorded images. According to other examples, the instructions may further include determining a pattern angle and a spatial frequency, rotating the projected two fine-lined patterns over each other, and iteratively generating new projected patterns that match a texture of the textured portion of the subject in finer detail and determining a new pattern angle and spatial frequency at each iteration until a predefined pattern knowledge is achieved. 
     The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The foregoing and other features of this disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several embodiments in accordance with the disclosure and are, therefore, not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings, in which: 
         FIG. 1  illustrates an example Augmented Reality (AR) system, where projected patterns may be employed for high resolution texture extraction; 
         FIG. 2  illustrates an example pattern projection and capture system for high resolution texture extraction; 
         FIG. 3  illustrates another example pattern projection and capture system for high resolution texture extraction; 
         FIG. 4  illustrates how patterns below the resolution of a camera may be discerned employing two sets of similar lines at an angle and rotating the line sets; 
         FIG. 5  illustrates a geometric approach to high resolution texture extraction employing two superimposed patterns of parallel and equidistant lines; 
         FIG. 6  illustrates a general purpose computing device, which may be used to extract high resolution texture information employing projected patterns; 
         FIG. 7  illustrates a processor, which may be used to extract high resolution texture information employing projected patterns; 
         FIG. 8  is a flow diagram illustrating an example method for texture extraction that may be performed by a computing device, such as computer  600  in  FIG. 6  or special purpose processor  710  in  FIG. 7 ; and 
         FIG. 9  illustrates a block diagram of an example computer program product; 
       all arranged in accordance with at least some embodiments described herein. 
     
    
    
     DETAILED DESCRIPTION 
     In the following detailed description, reference is made to the accompanying drawings, which form a part hereof In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein. 
     This disclosure is generally drawn, inter alia, to methods, apparatus, systems, devices, and/or computer program products related to enhancing texture extraction for augmented reality through manipulation of projected patterns. 
     Briefly stated, camera-based texture extraction in AR systems may be enhanced by manipulating projected patterns. According to some examples, a fine line pattern may be projected, a Moiré interference pattern measured, and different properties of the projected pattern adjusted until the Moiré interference pattern measurements indicate that a similar texture pattern to that of the three dimensional target is being projected. Thereby, the target texture may be more closely matched even as sub-pixel resolutions. 
       FIG. 1  illustrates an example Augmented Reality (AR) system, where projected patterns may be employed for high resolution texture extraction arranged in accordance with at least some embodiments described herein. AR explores the application of computer-generated imagery in live video streams to expand the real world presentation. Example AR systems arranged in accordance with the present disclosure may be in controlled environments containing a number of sensors and actuators, may include one or more computing device adapted to process real and computer-generated imagery, and may include visualization systems such as head-mounted displays, virtual retinal displays, monitor or similar regular displays, and comparable devices. 
     Example AR system  100  includes image sensors  104 - 1  for capturing live images of real scene (objects)  102 , as well as tracking sensors  104 - 2  for tracking a position and/or a motion of the objects. The image sensors  104 - 1  may be digital cameras, webcams, or some other image capturing devices. The tracking sensors  104 - 2  may include a number of receiving devices arranged in a passive sensing network to enhance tracking performance through frequency, bandwidth, and spatial diversity of the network. The receiving devices (e.g., one or more RF receivers) may be adapted to utilize communication signals (e.g., electromagnetic waves such as RF signals) from nearby signal sources such as communication towers (e.g., cellular telephony communication towers) or communication base stations. The tracking sensors  104 - 2  may be located in different positions and may be communicatively coupled to a centralized or distributed computing system to form the collaborative network. 
     The captured image(s) may be provided to an image processing sub-system  106 , which may be adapted to perform one or more of digitization of images into digital images, receipt of digital images, and/or processing digital images. Processing of digital images may include one or more of determining locations of feature points in the images, computation of affine projections, tracking of edges, filtering, and/or similar operations. Image processing sub-system  106  may be configured to provide projection information, such as one or more of the results of the above described operations, to reality engine  110 . The tracking sensors  104 - 2  may be configured to provide position and/or motion information associated with objects of interest in real scene  102  to the reality engine  110 . The reality engine  110  may be adapted to execute a graphics process to render scenes based on the captured images that incorporates position and/or motion information from the tracking sensors  104 - 2 . 
     Image generator  108  may be adapted to receive reference image(s) from the image sensors  104 - 1  as well as image data associated with virtual object(s), and may be adapted to overlay the captured real scene images with the image data associated with the virtual object(s) to provide an augmented scene  114 . Display  112  is one example of a visualization mechanism that may be utilized in the AR system  100 . A number of other display types, such as projectors, wearable displays, etc. may also be used to present the AR imagery to a user. 
     Processing for at least some of the components of the AR system  100  such as the image processing sub-system  106 , the reality engine  110 , the image generator  108 , and/or the display  112  may be performed by separate applications, one or more integrated applications, one or more centralized services, or one or more distributed services on one or more computing devices. Each computing device may be either a general purpose computing devices or a special purpose computing device that may be a standalone computer, a networked computer system, a general purpose processing unit (e.g., a micro-processor, a micro-controller, a digital signal processor or DSP, etc.), or a special purpose processing unit. If executed on different computing devices, various components of the AR system  100  may be adapted to communicate over one or more networks. 
       FIG. 2  illustrates an example pattern projection and capture system for high resolution texture extraction arranged in accordance with at least some embodiments described herein. Three dimensional feature extraction systems typically include a camera  226 , a projector  224 , and image processing capabilities (pattern recognition program  228 ) as shown in diagram  200 . 
     A system according to at least some embodiments may employ these components to project a pattern onto a fine textured surface  222  of an object  220  to enhance the visibility of textures by manipulating projected patterns. The projector  224  may project a fine line pattern onto the textured surface  222 . The camera  226  may capture an image of the textured surface  222  with the projected pattern. Pattern recognition program  228  may measure a Moiré interference pattern and adjust different properties of the projected pattern until the Moiré interference pattern measurements indicate that a similar texture pattern to that of the three dimensional target (object  220 ) is being projected. 
     According to at least some embodiments, a mobile device or similar mechanism may be utilized to move the projector  224  for adjusting features of the projected pattern such as an angle of orientation, a width of projected lines, a compensation for surface coloration or three dimensional features, and similar ones. The resulting changes in the Moiré interference pattern image may be used to extract the pitch and orientation of the sub-pixel texture pattern enhancing obtained texture information. By modifying the patterns, a spatially varying texture may also be sequentially extracted. Beyond sub-resolution patterns, a system according to embodiments may also be employed to analyze textures under variable lighting conditions or complicated geometries. In some example implementations, the projector  224  and the camera  226  may be integrated or installed on the same platform (e.g., a handheld device, a vehicle-mount device, etc.) as shown in diagram  200 . 
       FIG. 3  illustrates another example pattern projection and capture system for high resolution texture extraction arranged in accordance with at least some embodiments described herein. Major components of a system for high resolution texture extraction for an AR environment depicted in diagram  300  may be similarly configured and perform likewise tasks as similarly numbered components in diagram  200  of  FIG. 2 . Differently from  FIG. 2 , projector  324  and camera  326  are independent in the system of diagram  300 . Thus, the camera  326  may capture images of the textured surface  222  from a fixed perspective, while the projector  324  is moved to adjust properties of the projected pattern such as orientation of the lines. Pattern recognition program  328  may control the movements of the projector  324  automatically according to some example implementations. 
     Texture extraction by a system according to embodiments may begin with extracting a rough estimate of the three dimensional texture from a three dimensional feature computation. According to other examples, the surface may be assumed to be relatively planar. Alternatively, the projection may be three dimensionally corrected so that the computations may be performed at camera perspective as if the surface is nearly planar. 
     Projection of lines with a different pitch than the texture results in a high frequency visual component, which may be sub-pixel resolution, and a low frequency visual component, which is at a spatial frequency of half the difference between the spatial frequencies of the two patterns. For example, if a corduroy fabric surface has about 2 mm pitch (spatial frequency of about 5 cm −1 ) and a projected pattern has lines with a pitch of about 3 mm (spatial frequency of about 3.3 cm −1 ), a virtual pattern in the aligned state may emerge at about 0.85 cm −1  or about 1.17 cm pitch—a size easier to see with a camera. This virtual pattern may then change with rotation in a way that provides more equations than unknowns so that true texture pitch and angle can be extracted. 
       FIG. 4  illustrates how patterns below the resolution of a camera may be discerned employing two sets of similar lines at an angle, or one set of lines plus the texture of a material, and rotating the line sets arranged in accordance with at least some embodiments described herein. A typical Moiré pattern is made of an array of parallel lines, at approximately pixel width. The pattern interacts with the texture elements at a sub-pixel level, creating a Moiré interference pattern. By tilting the pattern (both in displacement as well as in rotation), the alignment, pitch, and/or shape of the sub-resolution texture can be determined. 
     Diagram  400  includes an example, where a projected first pattern  430  of parallel lines is rotated into a second pattern  432  compared against a pattern on the object texture to enhance texture extraction. Both patterns have the distance between lines p. The second pattern  432  is rotated by an angle α. From a distance, dark and pale lines ( 435 ,  434 ) can be seen. The pale lines  434  correspond to the lines of nodes, that is, lines passing through the intersections of the two patterns  430  and  432 . If each cell  440  of the “net” formed by the intersecting patterns is a parallelogram with the four sides equal to d=p/sin α. Each cell includes a right triangle with hypotenuse d and side p opposing the angle α. It should be noted that this type of pattern generation is in addition to the effect of varying pitch and both approaches together can be used to generate a pattern revealing smaller patterns as discussed below. 
     Sets of similar lines at an angle create a virtual pattern with larger features than either pattern. If one pattern is the surface texture and the other is a projected pattern, the projection enables the discerning of patterns that may be below the resolution of the camera. The virtual pattern changes during rotation of the projection so that the true surface pattern may be extracted. If the true surface pattern is not formed by straight lines, the patterns may change in different areas at different times and this may be used to extract the orientation and pitch of the pattern in each area. 
     The pale lines  434  correspond to the small diagonal of the rhombus. As the diagonals are the bisectors of the neighboring sides, the pale lines have an angle equal to α/2 with the perpendicular of the lines of each pattern. Additionally, the spacing between two pale lines is D, the half of the big diagonal. The big diagonal 2D is the hypotenuse of a right triangle and the sides of the right angle are d·(1+cos α) and p. Using the Pythagorean Theorem:
 
(2 D ) 2   =d   2 ·(1+cos α) 2   +p   2 ,  [1]
 
which can be transformed into:
 
(2 D ) 2 =2 p   2 ·(1+cos α)/sin 2 α.  [2]
 
When α is very small (α&lt;p/6), two approximations may be made: sin α≈α and cos α≈1. Thus:
 
 D=p/α,   [3]
 
where α is in radians.
 
     As α becomes smaller (patterns less rotated), the pale lines  434  become farther apart. When both patterns  430  and  432  are parallel (i.e., α=0), the spacing between the pale lines becomes infinite (i.e., there is no pale line). A camera with lower resolution than the texture can have lines (dark and light) to recognize even though the actual texture and Moiré pattern may be irresolvable grey. α may be determined using two different techniques: by the orientation of the pale lines  434  and by their spacing (α=p/D). 
     If the angle is measured, the final error may be proportional to the measurement error. If the spacing is measured, the final error may be proportional to the inverse of the spacing. Thus, for smaller angles, measuring the spacing may yield more accurate results. Moreover, multiple sets of virtual patterns may be obtained by varying the frequency and/or angle of the projected lines. Two patterns are required to extract the underlying texture size with certainty as there are high and low frequency virtual patterns as discussed above. 
       FIG. 5  illustrates a geometric approach to high resolution texture extraction employing two superimposed patterns of parallel and equidistant lines arranged in accordance with at least some embodiments described herein. According to some embodiments, a geometric approach may be employed in texture extraction. Diagram  500  illustrates two patterns  542  and  544  formed by parallel and equidistant lines. The lines of the first pattern  544  are separated by p and the lines of the second pattern  542  are separated by p+δp, where δ has a value between 0 and 1. 
     If the lines of the patterns are superimposed at the left portion of the figure, the shift between the lines increases when going to the right. After a number of lines, the patterns are opposed (i.e., the lines of the second pattern  542  are between the lines of the first pattern  544 ). From a far distance, pale zones become visible when the lines are superimposed (white area between the lines) and dark zones when the lines are “opposed”. The middle of the first dark zone appears when the shift is equal to p/2. The nth line of the second pattern  542  is shifted by n·δp compared to the nth line of the first pattern. The middle of the first dark zone thus corresponds to:
 
 n·δp=p/ 2,  [4]
 
from which n can be expressed as:
 
 n=p /(2δ p ).  [5]
 
     The distance d between the middle of a pale zone and a dark zone can then be expressed as:
 
 d=n·p=p   2 /(2δ p ).  [6]
 
Thus, the distance between the middle of two dark zones, which is also the distance between two pale zones, can be expressed as:
 
2 d=p   2 /(δ p ).  [7]
 
     From formula [7], one can deduce that the bigger the separation between the lines, the bigger the distance between the pale and dark zones and the bigger the discrepancy δ p , the closer the dark and pale zones. A great spacing between dark and pale zones means that the patterns have very close line distances. When p=p/2, a uniformly grey figure is obtained with no contrast. 
     According to other embodiments, an interferometric approach may also be employed. For the interferometric approach, two transparent patterns with a contrast I that varies with a sinusoidal law may be considered:
 
 I   1 ( x )= I   0 ·sin(2π k   1   x )  [8]
 
 I   2 ( x )= I   0 ·sin(2π k   2   x ),  [9]
 
where
 
the distances between the lines of the patterns are respectively p 1 =l/k 1  and p 2 =l/k 2 . When the patterns are superimposed, the resulting intensity (or interference) may be expressed as:
 
 I ( x )= I   0 ·[sin(2π k   1   x )+sin(2π k   1   x )].  [10]
 
Using Euler&#39;s formula, the interference may be expressed as:
 
 I ( x )= I   0 ·2 cos(2π x ( k   1   −k   2 )/2)·sin(2π x ( k   1   +k   2 )/2.  [11]
 
     The resulting intensity includes a sinusoid of a high “spatial frequency” (wave number), which is the average of the spatial frequencies of the two patterns, and a sinusoid of a low spatial frequency, which is the half of the difference between the spatial frequencies of the two patterns. The second component is an “envelope” for the first sinusoid. The wavelength of this component is the inverse of the spatial frequency:
 
 l /λ=( k   1   −k   2 )/2·( l/p   1   −l/p   2 )/2.  [12]
 
If p 1  and p 2  are written in δp form as p 1 =p and p 2 =p+δp, the wavelength can be expressed as:
 
λ=2· p   1   p   2 /( p   2   −p   1 )≈ p   2 /2δ p.   [13]
 
The distance between the zeros of this envelope is spaced by l/2, and the maxima of amplitude are also spaced by l/2. Thus, the same results as the geometric approach may be obtained with a discrepancy of p/2, which is the uncertainty linked to the reference that is considered first pattern or second pattern. The discrepancy becomes negligible when δp&lt;&lt;p. Thus, angle data is not necessarily needed for the three different approaches (e.g. it is known that the target has vertical lines). The projected image may be employed as one of the patterns.
 
       FIG. 6  illustrates a general purpose computing device, which may be used to extract high resolution texture information employing projected patterns arranged in accordance with at least some embodiments described herein. 
     Computer  600  includes a processor  610 , a memory  620 , and one or more drives  630 . The drives  630  and their associated computer storage media such as removable storage media  634  (e.g., CD-ROM, DVD-ROM) and non-removable storage media  632  (e.g., a hard drive disk), may provide storage of computer readable instructions, data structures, program modules and other data for the computer  600 . The drives  630  may include an operating system  640 , application programs  650 , program modules  660 , and database  680 . The computer  600  further may include user input devices  690  through which a user may enter commands and data. The input devices  690  may include an electronic digitizer, a microphone  696 , a keyboard  694 , and a pointing device such as a mouse device  692 , trackball device or touch pad device. Other input devices may include a joystick device, game pad device, satellite dish, scanner device, or the like. 
     The application programs  650  may include a projection application  652  and a pattern recognition application  654 . The projection application  652  may generate, modify, and/or control projection of pattern sets on textured surfaces to extract texture from three dimensional objects in AR applications. The pattern recognition application  654  may process captured images compute Moiré interference patterns and provide feedback to the projection application  652  in order to adjust the patterns and enhance the obtained information iteratively. 
     The above described and other input devices may be coupled to a processor  610  through a user input interface that is coupled to a system bus  605 , but may be coupled by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). Computers such as the computer  600  may also include other peripheral output devices such as speakers  676 , printer  674 , display  672 , and communication module  678 , which may be coupled through an output peripheral interface  670  or the like. The communication module  678  may be used to communicate with a projector and/or a mobile platform controlling the projector to adjust different properties of the projected patterns. 
     The memory  620 , removable storage devices  634  and non-removable storage devices  632  are examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information and which may be accessed by the computer  600 . Any such computer storage media may be part of the computer  600 . 
     The computer  600  may operate in a networked environment using logical connections to one or more computers, such as a remote computer connected to network interface  606 . The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and can include many or all of the elements described above relative to the computer  600 . Networking environments are commonplace in offices, enterprise-wide area networks (WAN), local area networks (LAN), intranets and world-wide networks such as the Internet. For example, in the subject matter of the present application, the computer  600  may comprise the controller machine from which data is being migrated to multilayer circuit board manufacturing systems such as automatic drill systems, etching systems, etc., and the remote computer may comprise controllers of the systems. It should be noted, however, that source and destination machines need not be coupled together by a network(s)  608  or any other means, but instead, data may be migrated via any media capable of being written by the source platform and read by the destination platform or platforms. When used in a LAN or WLAN networking environment, computer  600  may be coupled to the LAN through the network interface  606  or an adapter. 
     The network(s) may comprise any topology employing servers, clients, switches, routers, modems, Internet service providers (ISPs), and any appropriate communication media (e.g., wired or wireless communications). A system according to some embodiments may have a static or dynamic network topology. The network(s) may include a secure network such as an enterprise network (e.g., a LAN, WAN, or WLAN), an unsecure network such as a wireless open network (e.g., IEEE 802.11 wireless networks), or a world-wide network such (e.g., the Internet). The network(s) may also comprise a plurality of distinct networks that are adapted to operate together. The network(s) are adapted to provide communication between the nodes described herein. By way of example, and not limitation, the network(s) may include wireless media such as acoustic, RF, infrared and other wireless media. 
     The network communication link may be one example of a communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A “modulated data signal” may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein may include both storage media and communication media. 
     The computer  600  may be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a portable computing device, a mobile computing device, an application specific device, or a hybrid device that include any of the above functions. The computer  600  may also be implemented as a personal computer including both laptop computer and non-laptop computer configurations. Moreover, the computer  600  may be implemented as a networked system or as part of a general purpose or specialized server. 
       FIG. 7  illustrates a processor, which may be used to extract high resolution texture information employing projected patterns arranged in accordance with at least some embodiments described herein. Processor  710  of diagram  700  may be part of a computing device communicatively coupled to one or more modules responsible for projecting patterns onto a textured surface of an object to be recognized, extracting texture(s) from the projected surfaces, adjusting projection parameters, etc. 
     The processor  710  may include a number of modules for controlling different aspects of projecting patterns for high resolution texture extraction. The modules may include a projection control module  730  for projecting fine line patterns with adjustable parameters onto a textured surface, an image capture module  740  for capturing images of the textured surface with the projected patterns, and a pattern recognition module  750  for processing captured images, measuring Moiré interference patterns, and adjusting projected patterns for finer detection. 
     Memory  720  may be configured to store instructions for the control modules of the processor  710 , which may be implemented as hardware, software, or combination of hardware and software. The processor  710  may communicate through direct electrical coupling or through networked communications (e.g., network(s)  790 ) with other computing devices and/or data stores such as storage facility  780 . 
     Example embodiments may also include methods. These methods can be implemented in any number of ways, including the structures described herein. One such way is by machine operations, of devices of the type described in the present disclosure. Another optional way is for one or more of the individual operations of the methods to be performed in conjunction with one or more human operators performing some of the operations while other operations are performed by machines. These human operators need not be collocated with each other, but each can be only with a machine that performs a portion of the program. In other examples, the human interaction can be automated such as by pre-selected criteria that are machine automated. 
       FIG. 8  is a flow diagram illustrating an example method for texture extraction that may be performed by a computing device, such as the computer  600  in  FIG. 6  or the processor  710  in  FIG. 7  in accordance with at least some embodiments. The operations described in blocks  822  through  836  may be stored as computer-executable instructions in a computer-readable medium such as the drives  640  of the computer  600  or the memory  720  of the processor  710 . 
     A process of high resolution texture extraction by projecting patterns may begin with operation  822 , “DETERMINE FINE TEXTURE SUBJECT.” At operation  822 , a three dimensional object to be modeled for an AR system may be determined along with fine textured surfaces of the object. A camera such as the camera  226  of  FIG. 2  may be used to detect the object. The textured surface of the object may have multiple orientations and pitches. Furthermore, the system may be positioned such that a projector (e.g., the projector  224 ) of the system can project predefined and adjustable patterns on the fine textured surfaces. 
     At optional operation  824 , “EXTRACT 3D SHAPE AND GENERATE CORRECTIVE MATRIX”, the three dimensional shape of the object may be extracted, possibly using three dimensional structured light techniques incompatible with conventional texture extractions. The three dimensional shape may be used to generate a correction matrix for the following operations to compensate for three dimensional shape. 
     At operation  826 , “PROJECT FINE LINES AND ROTATE”, following operation  824 , projector  224  may project a fine lined pattern such as pattern  430  and rotate it. The fine lines may be straight, bent, or curved lines depending on a surface shape of the object (e.g., curved lines for curved objects). At subsequent operation  828 , “RECORD LOCAL MOIRÉ PATTERN PITCHES”, the Moiré pitches may be recorded. As discussed previously, a typical Moiré pattern  500  is made of an array of parallel lines, at approximately pixel width. The pattern interacts with the texture elements at a sub-pixel level, creating a Moiré interference pattern. By tilting the pattern (both in displacement as well as in rotation), the alignment, pitch, and shape of the sub-resolution texture can be determined. 
     At operation  830 , “FOR EACH SUB-AREA OF TEXTURE CHOOSE TWO OR MORE MOIRÉ PATTERNS WITH CLEAR PITCHES”, following operation  828 , the system may choose at least two images for each sub-area that contain clear Moiré patterns. The selection may be performed sequentially or with different colors as long as they are separable. At next operation  832 , “DETERMINE PATTERN ANGLE AND FREQUENCY”, the system may compute spatial frequency and pattern orientation for each sub area as discussed above. Optionally, the resulting information on pattern pitch and angle may be used to build a new set of fine lines that more closely match the texture  222  of the object  220  at operation  834 , “GENERATE NEW FINE LINE PROJECTION BASED ON KNOWN ANGLE AND FREQUENCY.” The new pattern may be projected by projector  224  onto the textured surface for a new set of measurements by the camera  226  and the pattern recognition program  228  in an iterative manner as shown by operation  836 , “REPEAT UNTIL DESIRED DEGREE ACHIEVED.” This iterative cycle of projecting the pattern and re-measuring may evolve to closely match complicated curving or changing patterns such that even complex texture patterns at sub-resolution sizes such as engraving or intaglio may be extracted. 
     According to other embodiments, additional modifications may be performed. Such modifications may include, but are not limited to, adjusting the width of the projected lines, adjusting the linear shape of the lines to account for curvature in the feature which may be detected by shifts in the virtual line patterns, curving the pattern to account for their expected distortion given the known three dimensional contour of the surface having the texture of interest, and/or curving the pattern to match detected curvature in the texture. The captured Moiré interference pattern may then be converted to a sub-resolution imputed texture map, which can be further analyzed using a variety of texture classification techniques. 
     The operations included in the above described process are for illustration purposes. High resolution texture extraction in AR systems using pattern projection and capture may be implemented by similar processes with fewer or additional operations. In some examples, the operations may be performed in a different order. In some other examples, various operations may be eliminated. In still other examples, various operations may be divided into additional operations, or combined together into fewer operations. 
       FIG. 9  illustrates a block diagram of an example computer program product arranged in accordance with at least some embodiments described herein. In some examples, as shown in  FIG. 9 , computer program product  900  may include a signal bearing medium  902  that may also include machine readable instructions  904  that, when executed by, for example, a processor, may provide the functionality described above with respect to  FIG. 6  and  FIG. 7 . Thus, for example, referring to the processor  710 , the modules  730 ,  740 , and  750  may undertake one or more of the tasks shown in  FIG. 9  in response to instructions  904  conveyed to the processor  710  by the medium  902  to perform actions associated with controlling high resolution texture extraction through projected patterns as described herein. Some of those instructions may include projecting fine lines and rotating, determining pattern angle and frequency from clear or dark pitches, and iteratively refining the projected lines. 
     In some implementations, the signal bearing medium  902  depicted in  FIG. 9  may encompass a computer-readable medium  906 , such as, but not limited to, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, memory, etc. In some implementations, signal bearing medium  902  may encompass a recordable medium  908 , such as, but not limited to, memory, read/write (R/W) CDs, R/W DVDs, etc. In some implementations, the signal bearing medium  902  may encompass a communications medium  910 , such as, but not limited to, a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.). Thus, for example, program product  900  may be conveyed to one or more modules of the processor  910  by an RF signal bearing medium, where the signal bearing medium  902  is conveyed by a wireless communications medium  910  (e.g., a wireless communications medium conforming with the IEEE 802.11 standard). 
     The present disclosure presents an augmented Reality (AR) scene capture system capable of high resolution texture extraction. According to some embodiments, the system may include a projector  224  configured to project at least one fine-lined pattern  430  on a three dimensional subject to be captured, a camera  226  configured to capture an image of the subject with the projected patterns, and a processor  910 . The processor may determine a textured portion of the subject  222 , record local Moiré pattern pitches on the textured portion of the subject  828 , and adjust one or more properties of the projected patterns such that the at least one projected patterns  430  substantially match the textured portion of the subject  222 . 
     According to other embodiments, the properties of the at least one projected pattern  430  may include an angle of orientation, a width, a compensation for surface coloration, and/or a three dimensional feature. The processor  910  may extract a pitch and an orientation of a texture pattern of the textured portion  222  of the subject from changes in the Moiré patterns. Furthermore, the camera  226  and the projector  224  may be integrated into a mobile device whose motion is controlled by the processor  910 . 
     According to further embodiments, the processor may vary the at least one projected fine-lined pattern  430  for sequential extraction of spatially varying texture on the textured portion  222  of the subject, and adjust the properties of the projected pattern  430  in order to capture the textured portion  222  of the subject under varying lighting conditions. The projected pattern  430  may include parallel lines, where the processor adjusts a linear shape of the lines to account for curvature in a feature of the textured portion  222  of the subject. The processor may also curve the projected pattern  430  to account for an expected distortion based on a three dimensional contour of a surface of the textured portion  222  of the subject. Moreover, the processor may curve the projected patterns to match a detected curvature in a texture of the textured portion  222  of the subject. 
     The present disclosure also presents a method for high resolution texture extraction in an Augmented Reality (AR) scene capture system  100 . The method may include determining a textured portion of a three dimensional subject to be captured  822 , extracting a three dimensional shape for the textured portion of the subject  824 , projecting a fine-lined pattern on the textured portion of the subject  826 , recording local Moiré pattern images on the textured portion of the subject  828 , selecting two or more Moiré patterns with pitches that are clear in the recorded images  830 , and determining a pattern angle and a spatial frequency  832 . 
     According to further examples, the method may further include generating a new projected pattern that matches a texture of the textured portion of the subject in finer detail employing the pattern angle and the spatial frequency  834  and iteratively generating new projected patterns that match a texture of the textured portion of the subject in finer detail and determining a new pattern angle and spatial frequency at each iteration until a predefined pattern knowledge is achieved  836 . 
     According to yet other examples, the method may also include generating a corrective matrix based on the extracted three dimensional shape for the textured portion of the subject  824  and rotating the projected fine-lined pattern over the textured portion. The projected fine-lined pattern  430  may have a different pitch than that of a texture of the textured portion  222  of the subject providing a high frequency visual component and a low frequency visual component. The high frequency visual component may provide a sub-pixel resolution and the low frequency visual component may provide a spatial frequency of about half a difference between spatial frequencies of the projected patterns  430 ,  432  and the texture of the textured portion  222  of the subject. 
     The present disclosure further presents a computer-readable storage medium  906  having instructions  904  stored thereon for high resolution texture extraction in an Augmented Reality (AR) scene capture system  100 . According to some examples, the instructions may include determining a textured portion of a three dimensional subject to be captured  822 , extracting a three dimensional shape for the textured portion of the subject  824 , projecting two fine-lined patterns on the textured portion of the subject  826 , recording local Moiré pattern images on the textured portion of the subject  828 , selecting two or more Moiré patterns with pitches that are clear in the recorded images  830 , and determining a pattern angle and a spatial frequency  832 . According to other examples, the projected patterns may be projected sequentially and the instructions may also include rotating the projected two fine-lined patterns over each other  826  and iteratively generating new projected patterns that match a texture of the textured portion of the subject in finer detail and determining a new pattern angle and spatial frequency at each iteration until a predefined pattern knowledge is achieved  834 ,  836 . 
     According to further examples, the instructions may include modifying one or more of an angle of orientation, a width, a compensation for surface coloration, and/or a three dimensional feature of the projected patterns  430 ,  432 . A system according to example embodiments may also perform one or more of adjusting a width of lines in the projected patterns  430 ,  432 ; adjusting a linear shape of the lines of the projected patterns  430 ,  432  to account for curvature in a feature of the textured portion  222  of the subject; curving the projected patterns  430 ,  432  to account for an expected distortion based on a three dimensional contour of a surface of the textured portion  222  of the subject; and/or curving the projected patterns  430 ,  432  to match a detected curvature in a texture of the textured portion  222  of the subject. According to yet other examples, the instructions may include converting the Moiré patterns to a sub-resolution imputed texture map and analyzing the texture map employing a texture classification approach. 
     There is little distinction left between hardware and software implementations of aspects of systems; the use of hardware or software is generally (but not always, in that in certain contexts the choice between hardware and software may become significant) a design choice representing cost vs. efficiency tradeoffs. There are various vehicles by which processes and/or systems and/or other technologies described herein may be effected (e.g., hardware, software, and/or firmware), and that the preferred vehicle will vary with the context in which the processes and/or systems and/or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may opt for a mainly hardware and/or firmware vehicle; if flexibility is paramount, the implementer may opt for a mainly software implementation; or, yet again alternatively, the implementer may opt for some combination of hardware, software, and/or firmware. 
     The foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples. Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such block diagrams, flowcharts, or examples may be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, may be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. 
     The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It is to be understood that this disclosure is not limited to particular methods, materials, and configurations, which can, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. 
     In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, a computer memory, etc.; and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.). 
     Those skilled in the art will recognize that it is common within the art to describe devices and/or processes in the fashion set forth herein, and thereafter use engineering practices to integrate such described devices and/or processes into data processing systems. That is, at least a portion of the devices and/or processes described herein may be integrated into a data processing system via a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, a memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and/or control systems including feedback loops and control modules (e.g., adjusting pattern and image capture parameters of the system). 
     A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data computing/communication and/or network computing/communication systems. The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely exemplary, and that in fact many other architectures may be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively “associated” such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality may be seen as “associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermediate components. Likewise, any two components so associated may also be viewed as being “operably connected”, or “operably coupled”, to each other to achieve the desired functionality, and any two components capable of being so associated may also be viewed as being “operably couplable”, to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically connectable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components. 
     With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity. 
     It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, means at least two recitations, or two or more recitations). 
     Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to “at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.” 
     In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group. 
     As will be understood by one skilled in the art, for any and all purposes, such as in terms of providing a written description, all ranges disclosed herein also encompass any and all possible subranges and combinations of subranges thereof Any listed range can be easily recognized as sufficiently describing and enabling the same range being broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily broken down into a lower third, middle third and upper third, etc. As will also be understood by one skilled in the art all language such as “up to,” “at least,” “greater than,” “less than,” and the like include the number recited and refer to ranges which can be subsequently broken down into subranges as discussed above. Finally, as will be understood by one skilled in the art, a range includes each individual member. Thus, for example, a group having 1-3 cells refers to groups having 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers to groups having 1, 2, 3, 4, or 5 cells, and so forth. 
     While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.