SYSTEMS AND METHODS FOR NOSE-BASED PET IDENTIFICATION

A method for identifying a pet via an image of a nose of the pet includes receiving at least one image comprising the nose of the pet, identifying the nose of the pet and nostrils of the nose in the at least one image, comparing a degree of alignment of the nostrils of the nose with a first threshold value, in accordance with the degree of alignment of the nostrils meeting the threshold value, determining a quality metric for at least a nose portion of the image, and in accordance with the quality meeting a second threshold value, determining a representation of features of the nose of the pet in the at least one image.

FIELD OF THE DISCLOSURE

This disclosure related generally to pet identification, and in particular to pet identification via image recognition.

BACKGROUND OF THE DISCLOSURE

Current method for lost dog identification include the cumbersome and inconvenient methods of microchipping or dog tags. Both of these methods include significant barriers to register and identify a pet. Microchipping requires trips to a vet and dog tags often require the involvement of a government agency. Further, there can be some concern regarding safety for the pet involved in microchipping and dog tags can become lost.

SUMMARY OF THE DISCLOSURE

According to various embodiments, users can identify a pet using images of the pet's nose. A user can capture an image of the nose of a pet and the image can be analyzed to determine features of the nose that are distinctive for the pet. A database of registered pets that store representations of the features of the noses of the registered pets can be queried to determine the identity of a pet. According to various embodiments, a user need only a mobile device running a nose identification App to register a pet, in the case of an owner, or identify a pet, in the case of a pet service provide or lost pet finder. Thus, pet identification can be achieved with significantly fewer barriers than conventional microchipping and dog tagging.

According to various embodiments, a method for identifying a pet via an image of a nose of the pet includes receiving at least one image comprising the nose of the pet, identifying the nose of the pet and nostrils of the nose in the at least one image, comparing a degree of alignment of the nostrils of the nose with a first threshold value, in accordance with the degree of alignment of the nostrils meeting the threshold value, determining a quality metric for at least a nose portion of the image, and in accordance with the quality meeting a second threshold value, determining a representation of features of the nose of the pet in the at least one image.

In any of these embodiments, identifying the nose of the pet and nostrils of the nose in the at least one image can include using a machine learning algorithm.

In any of these embodiments, comparing a degree of alignment of the nostrils of the nose with a first threshold value can include determining a horizontal alignment of the nostrils by determining a vertical offset between the nostrils.

In any of these embodiments, the first threshold value can be 30 degrees.

In any of these embodiments, the method can further include, prior to determining the representation of features of the nose of the pet in the at least one image, comparing a size of one nostril to the other.

In any of these embodiments, the quality metric can be a blur metric. Optionally, the blur metric is determined using singular value decomposition.

In any of these embodiments, the representation of features of the nose of the pet can be determined via a local binary patterns histogram algorithm.

In any of these embodiments, the method may further include comparing the representation of features of the nose of the pet with a plurality of stored representations of features of noses of pets corresponding to stored pet identities to identify the pet.

According to various embodiments, a system includes one or more processors, memory, and one or more programs stored in the memory and comprising instructions for execution by the one or more processors for performing any of the above methods.

According to various embodiments, a non-transitory computer readable storage medium stores one or more programs for execution by one or more processors for performing any of the above methods.

DETAILED DESCRIPTION OF THE DISCLOSURE

Systems and methods, according to various embodiments, enable the identification of pets via image recognition of on images of pet noses. According to various embodiments, an image of at least a portion of a face of a pet, such as a dog or cat, is captured and analyzed to detect the nose of the pet in the image. The nose portion is analyzed and converted to a representation of features of at least a portion of the nose is stored in a database along with identifying information for the pet. Subsequently captured images of the pet can be analyzed in similar fashion and compared to the stored representation to identify the pet.

According to various embodiments, a mobile application on a user mobile device may enable a user to capture an image of a face of a pet and upload the image or a representation of the image to a server that stores the representation of the image in a pet identification database. A pet owner may provide identifying information along with an image of their pet, such as during a registration process. A finder of a lost pet may use the mobile application to capture an image of a lost pet for identifying the lost pet. According to some embodiments, upon matching an image captured by a finder of a lost pet with a representation of an image in the pet identification databased, the server may provide an alert to a pet owner that their pet has been found. The systems and methods described herein enable lost pets to be reunited with their owners without requiring veterinary visits and surgery, as required with microchipping, and without requiring tags that can be damaged or mistakenly left off of the pet.

In some embodiments, the systems and methods described herein enable storing of pet related activities, such as veterinary visits, in associated with pet identities for tracking the health or other aspects of the pet over the pet's lifetime. A veterinary office, groomer, or other pet service provider, can capture an image of a pet when checking the pet in for an appointment. The pet can be automatically identified and records generated during a previous appointment can be accessed for reference by the provider and/or records generated during the appointment can be associated with the pet for future access by the provider or a different provider.

Certain aspects of the present disclosure include process steps and instructions described herein in the form of an algorithm. It should be noted that the process steps and instructions of the present disclosure could be embodied in software, firmware, or hardware and, when embodied in software, could be downloaded to reside on and be operated from different platforms used by a variety of operating systems. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that, throughout the description, discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” “generating” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission, or display devices.

FIG.1illustrates a method100for pet identification using image recognition performed on an image of distinctive anatomy of the pet. According to various embodiments, method100can be used to identify a pet, such as a dog or cat, from an image of the nose of the pet. The description below refers to identification of a dog, but it is to be understood that the principles described below could be applied to identification of other animals, such as a cat, and that such application is within the scope of this disclosure.

At step102, an image of a face dog is captured, such as using the camera of a mobile device. The image can be captured using an App running on the mobile device, which may guide a user to capture at least one image the nose of the dog. In some embodiments, the App controls the mobile device to ensure that the flash is on during image capture so that the dog nose is adequately illuminated.

At step104, the image is analyzed to detect the nose in the image and to detect the nostrils in the image. In some embodiment, the nose and/or nostrils are detected using a machine learning algorithm, such as the Google™ AutoML Object Detection model. In some embodiments, the machine learning algorithm is trained on images of dog noses and/or nostrils. In some embodiments, the machine learning algorithm is trained on images of dog noses and/or nostrils that are labeled and include bounding boxes for the nose and/or nostrils. In some embodiments, the machine learning algorithm determines a bounding box of the nose. According to various embodiments, if the nose and/or nostrils cannot be detected, then method100may return to step102for the capture of another image. The user may be prompted, for example, to capture another image.

At step106, if the nose and nostrils have been detected, then the nostrils are analyzed to determine the alignment of the nostrils in the image. In some embodiments, a vertical distance between top edges or centers of the detected nostrils may be measured and used along with a horizontal spacing between the measurement points to determine the alignment of the nostrils. An example of this is shown inFIG.8, which includes a schematic representation of an exemplary nose802having nostrils804and806. A machine learning model has determined a bounding box808for the nose802, a bounding box810for nostril804, and a bounding box812for nostril806. A vertical measurement814may be taken between the tops of the bounding boxes810,812, which may correspond with the tops of the nostrils804,806, and a horizontal measurement816may be taken between the horizontal centers of the bounding boxes810,812. One or both of these measurements may be used to determine alignment of the nostrils. In some embodiments, the determined alignment may be compared to an alignment threshold value to determine whether the nostrils are adequately aligned in the image. Examples of alignment thresholds include 30 degrees or less, 25 degrees or less, 20 degrees or less, 15 degrees or less, 5 degrees or less, 2 degrees or less, and 1 degree or less, with the angle taken between a line connecting an estimate of the nostril top edges or centers and a horizontal line across the image.FIG.8illustrates an exemplary angular measurement818that may be determined from the horizontal and vertical spacing of the nostrils, as discussed above. If the nostrils are not adequately aligned, then the method returns to step102for the capture of a new image. In some embodiments, the alignment of the nostrils may be used to rotate the image instead of or in addition to accepting or rejecting the image. For example, if the determined alignment is 5 degrees from horizontal, then the image may be rotated by 5 degrees.

In some embodiments, the sizes of the nostrils may be compared and a measure of the difference between the sizes of the nostrils may be compared to a threshold value. Nostrils that are too dissimilar in size may indicate that the nose is turned to far to one side in the image or that the light comes too strongly from one side, either of which may provide poor performance. For example, with respect toFIG.8, the size of nostril804may be determined to be less than the size of the nostril806based on the sizes of the respective bounding boxes810,812. Images having nostrils that are too dissimilar in size (for example, failing to meet a predetermined threshold size difference) may be rejected and the method may return to step102for the capture of a new image. In some embodiments, guidance may be provided to the user regarding how to improve the next image capture.

At step108, if the nostrils are adequately aligned and/or after rotating the image according to the alignment, the image may be cropped to include primarily the nose-portion. At least some of any non-nose portion of the image may be removed, such as by removing portions outside of the bounding box of the nose determined at step104. In some embodiments, the image is down-sampled to reduce the amount of data and processing time. In some embodiments, the image is converted to a gray-scale image (in some embodiments, the image is converted to a gray-scale image prior to or during any one of steps104-106).

At step110, the cropped image is analyzed to determine an image quality metric for the image. In some embodiments, the image quality metric is a degree of blur. The image blur can be determined using any suitable blur analysis technique or combination of techniques, such as Variance of Laplacian, Cumulative Probability of Blur Detection (CPBD), Structural Similarity (SSIM), and Haar Wavelet. In some embodiments, a singular value decomposition (SVD) method is used to calculate a blur degree number, which is then compared to a threshold. According to various embodiments, if the blur degree number (or other image quality metric) fails to meet the threshold requirement, the image is rejected and the method returns to step102for capturing a new image.

At step112, if the image meets the image quality threshold (e.g., blur degree) requirement of step110, then the image (or cropped portion of the image) is analyzed to extract the distinctive features of the nose of the pet and generate a representation of the features of the nose in the image that will be relatively invariant from one image of the nose to the next such that two images of the same nose taken at different times and in different conditions will produce the same or substantially the same representation. In some embodiments, a Local Binary Patterns Histogram (LBPH) technique is used to generate the representation of the features of the nose in the image. According to some embodiments, the LBPH technique includes dividing the cropped image into a grid (e.g., with 8×8 cells), calculating the LBP for each cell, and then calculating the histogram of that LBP.FIGS.2A and2Billustrate the LBPH technique for extracting the features of the nose of a pet, according to various embodiments.FIG.2Aillustrates the generation of an LBP representation204of the cropped image202.FIG.2Billustrates the calculation of a histogram206for each cell of a grid208of the LBP representation204. The LBPH used in step112is merely an example of a method for generating a representation of the features of the nose. Another example is perceptual hashing.

In some embodiments, a user may be guided to capture a plurality of images of the nose of the pet and method100may be performed on the images after all have been captured. In some embodiments, a subsequent image is captured only if an image fails to meet one or more image quality thresholds of method100. In some embodiments, method100is performed on each of a plurality of images and the image that has the best quality metrics may be selected for generating the representation of the features of the nose in step112.

FIG.3illustrates a method300for registering a pet using a nose-recognition based pet identification method, such as method100ofFIG.1. At step302, one or more images of a nose of a pet are received and used to generate a representation of features of the nose, such as using method100ofFIG.1. The images may be captured by a mobile device, such as the mobile device of an owner of the pet, and received by an App running on the mobile device and/or may be received by a server that is communicatively connected to a mobile device that captures the image(s). In some embodiments, the App may guide the user through an image capture process. One or more steps of method100for generating the representation of the features of the nose of the pet may be performed on a server connected to the mobile device. For example, an image captured by the mobile device may be uploaded to the server and the server may perform steps104-112. In some embodiments, one or more steps of method100are performed locally on the mobile device. In some embodiments, all steps of method100are performed on the device and the representation of the features of the nose of the pet generated by the mobile device (e.g., App running on the mobile device) is uploaded to a server. The representation of the features of the nose are stored in a pet identification database on the server.

At step304, identifying information for the pet is received, such as at the mobile device of the pet owner and/or at a server. Examples of identifying information are: name of the pet, breed of the pet, sex of the pet, physical characteristics of the pet (weight, height, color, etc.), name of the pet owner, phone number of the pet owner, address of pet owner, etc. The identifying information may be entered into an App running on the mobile device via one or more fields of a graphical user interface. The identifying information is uploaded to the server and stored in the database on the server in association with the representation of the features of the nose of the pet.

FIG.4illustrates a method for identifying a pet via the nose recognition method100ofFIG.1. At step402, one or more images of a nose of a pet are captured at a mobile device. The mobile device may belong to someone who found a pet that appeared to be lost and the mobile device owner may use a lost pet identification feature of an App running on their mobile device to try to identify the pet, or the mobile device could be a mobile device of a veterinary practice or other pet service that is checking in the pet for an appointment. The one or more images of the nose of the pet are analyzed according to method100ofFIG.1and a representation of the features of the nose is generated. The representation of the features of the nose may be uploaded to a server communicatively connected to the mobile device or the server may generate the representation of the features of the nose from images captured by the mobile device.

At step404, the representation of the features of the nose generated at step402are compared with a database of stored representations corresponding to registered pets to determine whether the representation matches any stored representations. In some embodiments, matching can be determined by calculating a similarity score between the representation of the features of the nose generated at step402and a respective representation in the database, and the similarity score can be compared to a threshold value. Various techniques for comparing the LBPH of step112of method100include Euclidean distance, chi-square, and absolute value. Upon achieving a match, the identifying information for the stored representation is retrieved from the database.

At step406, upon achieving a successful match in step404, an indication of the identification of the pet may be provided. In some embodiments, the indication is provided to the mobile device that initiated the identification process, such as the lost pet finder or the pet service provider. In some embodiments, an indication that a lost pet has been found may be provided to the mobile device of the pet owner along with some indication of the location of the pet, such as GPS or map location of the pet based on the location of the mobile device of the pet finder or location information provided by the pet finder (e.g., manually entered by the pet finder into the App). In some embodiments, information regarding the lost pet finder may be provided to the pet owner, such as a phone number. In some embodiments, the App on the mobile device of the pet owner and pet finder may enable communication between the pet owner and pet finder.

FIG.5Aillustrates a graphical user interface500for capturing one or more images of a nose of a dog for use in method100ofFIG.1. A target box502may be provided to guide the user regarding how to capture an image of the nose of the dog.FIG.5Billustrates pet identifying information that may be provided via the user interface500.

FIG.6illustrates a system600for pet identification, according to various embodiments. One or more mobile devices602can be used to capture images of a nose of a pet. The images or results of one or more steps of image processing performed by the mobile device602can be transmitted to a server604connected to the mobile device602via one or more networks606. The server604is connected to a database608for storing a representation of the features of a nose of a pet along with identifying information. The server604may query the database608to search for a match to a representation of features of a nose of a pet for identifying the pet, according to the principles discussed above.

FIG.7illustrates an example of a computing system700, in accordance with some embodiments, that can be used for one or more of components of system600ofFIG.6, such mobile device602or server604. System600can be a computer connected to a network, such as a local area network, a wide area network, a cellular network, etc. System600can be a client or a server. As shown inFIG.7, system700can be any suitable type of processor-based system, such as a personal computer, workstation, server, handheld computing device (portable electronic device) such as a phone or tablet, or dedicated device. The system700can include, for example, one or more of input device720, output device730, one or more processors710, storage740, and communication device760. Input device720and output device730can generally correspond to those described above and can either be connectable or integrated with the computer.

Input device720can be any suitable device that provides input, such as a touch screen, keyboard or keypad, mouse, gesture recognition component of a virtual/augmented reality system, or voice-recognition device. Output device730can be or include any suitable device that provides output, such as a display, touch screen, haptics device, virtual/augmented reality display, or speaker.

Storage740can be any suitable device that provides storage, such as an electrical, magnetic, or optical memory including a RAM, cache, hard drive, removable storage disk, or other non-transitory computer readable medium. Communication device760can include any suitable device capable of transmitting and receiving signals over a network, such as a network interface chip or device. The components of the computing system700can be connected in any suitable manner, such as via a physical bus or wirelessly.

Processor(s)710can be any suitable processor or combination of processors, including any of, or any combination of, a central processing unit (CPU), field programmable gate array (FPGA), and application-specific integrated circuit (ASIC). Software750, which can be stored in storage740and executed by one or more processors710, can include, for example, the programming that embodies the functionality or portions of the functionality of the present disclosure (e.g., as embodied in the devices as described above). For example, software750can include one or more programs for performing one or more of the steps of method400, method800, and/or method1000.