Source: https://patents.google.com/patent/US9107697B2/en
Timestamp: 2019-04-23 12:12:38+00:00

Document:
FIG. 7 illustrates additional filtering criterion that could be used in a selection methodology according to the present invention. Sometimes, images of the objects in two different views (for example, from left and right cameras where stereo cameras are used) do not appear the same. One example of such situation is shown in FIG. 7 where two separate FUs can be easily identified in one image of the stereo pair (in this case the left image), while in the corresponding other stereo image the two segmented FU images have morphed into a single object. This scenario is typically encountered in high-density areas of the scalp where the hairs might be longer than 2-3 μm, and the combination of the tool vantage point (and by extension the camera axis) and imaging noise prevent the FUs from being imaged as completely separate in both left and right stereo views. While generally more advanced imaging systems are capable of “virtually splitting” the FUs where they have morphed together; the tracking of such hairs can be more burdensome and problematic due to the asymmetric views from the left and right cameras. As a result, to simplify and speed up the process of hair harvesting, one may choose to exclude these FUs all together and prefer to harvest the easier ones. As will be appreciated by those skilled in the art, many other filtering criteria may be implemented with the method of the present invention. These various criteria may be used individually or in numerous combinations depending on the specific needs and conditions in each harvesting case. As was previously mentioned, any of the filtering criteria may be turned “on” or “off” as desired.
wherein the one or more policy, or the one of more additional policy is based on at least one of 1) appearance of follicular units in one or more images of a body surface in a predetermined preceding time interval, 2) location of follicular units within a certain distance to an item comprising one or more of another follicular unit, an edge of the field of view of the image, or a previous harvesting location, 3) follicular unit type, 4) follicular unit distance to a harvesting tool, 5) follicular unit orientation relative to an orientation of the harvesting tool, 6) follicular unit emergence angle from a body surface, wherein those follicular units that have less than a predetermined emergence angle from the body surface are rejected, or 7) a shape of follicular unit.
2. The method of claim 1, wherein automatically selecting the particular follicular unit is based on a combination of a plurality of filters and policies.
3. The method of claim 1, further comprising identifying the pool of candidate follicular units for harvesting in the one or more images of the body surface containing follicular units; and wherein automatically sorting comprises keeping only those follicular units that have consistently appeared in the one or more images in the predetermined preceding time interval.
4. The method of claim 1, further comprising harvesting the selected follicular unit.
5. The method of claim 1, wherein the steps of automatically sorting and selecting are continuously repeated.
6. The method of claim 1, wherein the one or more policy, or the one or more additional policy comprises 1) discarding follicular units that are located within a certain minimum distance to a location containing blood; or 2) keeping follicular units that are located outside a certain minimum distance to a location containing blood.
7. The method of claim 1, wherein the location containing blood is determined using a variance filter.
8. The method of claim 1, wherein the one or more policy, or one or more additional policy comprises 1) discarding follicular units that are less than a predetermined width, or 2) keeping follicular units that are equal or more than a predetermined width.
9. The method of claim 1, wherein the one or more policy, or one or more additional policy comprises 1) discarding follicular units that do not meet a predetermined criteria based on a number of hairs in the follicular units; or 2) keeping follicular units that meet a predetermined criteria based on a number of hairs in the follicular units.
10. The method of claim 1, wherein the one or more policy, or the one or more additional policy comprises discarding follicular units possessing a complex shape or keeping follicular units possessing a simple shape.
11. The method of claim 1, wherein automatically selecting follicular units comprises scoring follicular units based on a weighted sum of variables associated with the one or more additional policy.
12. The method of claim 1, wherein automatically sorting follicular units comprises skipping every Nth follicular unit in the pool of candidate follicular units.
13. The method of claim 11, wherein scoring of follicular units based on the weighted sum of variables occurs after discarding one or more follicular units from the pool of candidate follicular units.
14. The method of claim 1, further comprising maneuvering a robotic arm of a robotic system to place a hair harvesting tool operatively connected to the robotic arm next to the selected follicular unit to be harvested.
15. The method of claim 12, wherein the skipping pattern comprises a multidirectional template of selecting every Nth follicle in a patch or cluster of follicular units.
wherein automatically selecting comprises scoring follicular units based on a weighted sum of variables associated with two or more of 1) appearance of follicular units in one or more images of a body surface in a predetermined preceding time interval, 2) location of follicular units within a certain distance to an item comprising one or more of another follicular unit, an edge of the field of view of the image, or a previous harvesting location, 3) follicular unit type, 4) follicular unit distance to a harvesting tool, 5) follicular unit orientation relative to an orientation of the harvesting tool, 6) follicular unit emergence angle from a body surface, or 7) a shape of follicular unit.
17. The method of claim 16, further comprising automatically filtering follicular units from the pool of candidate follicular units based on a predetermined criterion.
18. The method of claim 17, wherein a combination of the filtering criterion and one or more policy, or additional policy, comprises proximity to blood on a surface, a preference of the follicular units comprising one or more of follicular units located within a minimum distance to a hair harvesting tool, and follicular units whose orientation matches the orientation of a follicular unit that has been harvested immediately prior to the current selection.
20. The system of claim 19, wherein automatically selecting follicular units comprises scoring follicular units based on a weighted sum of variables comprising one or more of follicular unit type, follicular unit distance to a harvesting tool, follicular unit orientation relative to an orientation of a harvesting tool, or follicular unit emergence angle from a body surface.
21. The system of claim 19, further comprising a memory operatively associated with the processor.
22. The system of claim 19, further comprising an image acquisition device wherein the image acquisition device comprises at least one camera.
23. The system of claim 19, wherein the image processor is a personal computer.
24. The system of claim 19, wherein the system is a robotic system and it further comprises a robotic arm.
wherein the one or more policy or the one or more additional policy comprises keeping only those follicular units that have consistently appeared in the one or more images in a predetermined preceding time interval or rejecting those follicular units that have not consistently appeared in the one or more images in the predetermined preceding time interval.
English Translation of Korean Office Action mailed Aug. 19, 2014, in connection with commonly assigned Korean Patent Application No. 2013-7015877, Restoration Robotics, Inc. (4 pages).
English Translation of Office Action mailed Apr. 21, 2015, in relation to commonly assigned Chinese Patent Application No. 200980117701.0, Apr. 21, 2015, 8 pages.
English Translation of Office Action mailed Apr. 23, 2013, in relation to commonly assigned Chinese Patent Application No. 200980117701.0, (6 pages).
English Translation of Office Action mailed Oct. 29, 2013 in connection with commonly assigned Chinese Patent Application No. 200980117701.0. Applicant Restoration Robotics, Inc. (9 pages).
Mohammed Alhaddab, MD; Thomas Kohn, MD; and Mark Sidloi, BSc. Effect of Graft Size, Angle, and Intergraft Distance on Dense Packing in Hair Transplant. Dermatol Surg 2005; 31: pp. 650-654. Published by BC Decker Inc.
Office Action mailed Feb. 10, 2014 in connection with commonly assigned European Patent Application No. 09759083.0. Applicant: Restoration Robotics, Inc., (5 pages).
Rolf Hoffmann and Dominique van Neste. "Recent Findings with Computerized Methods for Scalp Hair Growth Measurements". J Investig Deramtoal Symp Proc 10:285-288, 2005.
Translation of Examiner's Office Action in connection with commonly assigned Japanese Patent Application No. 2011-512533; Applicant: Restoration Robotics, Inc., dated Sep. 18, 2012 (3 pages).

References: Application No. 2013
 Application No. 200980117701
 Application No. 200980117701
 Application No. 200980117701
 Application No. 09759083
 Application No. 2011