Patent Description:
<NPL>, describes an approach for classifying characters by identifying a rotation angle of an image using a convolutional neural network and performing classification on the rotated image. <NPL>, describes estimating the rotation angle of a rotated image using a feature map extracted by a CNN for classification trained using non-rotated images, correcting the angle of the image by the estimated angle and inputting it to the network again to perform classification for the rotated image.

<NPL>, describes a convolutional neural network for classifying and identifying different classes of targets. A SBR-based fast imaging scheme is applied to acquire scattering data that is used to train the CNN.

<NPL>, describes an approach combining a CNN and a convolutional autoencoder for aircraft target recognition on Inverse Synthetic Aperture Radar images reconstructed using Inverse Fast Fourier Transform.

The invention is defined by appended claim <NUM>.

The foregoing features may be more fully understood from the following description of the drawings in which:.

<FIG> is a diagram of a computing system <NUM>, according to aspects of the disclosure. As illustrated, the system <NUM> may include a radar system <NUM> and an image processing system <NUM>. According to the present example, the radar system includes an Inverse Synthetic Aperture Radar (ISAR) system. However, alternative implementations are possible in which the radar system <NUM> includes a side-looking airborne radar (SLAR), a synthetic aperture radar (SAR), a perimeter surveillance radar, a counter-battery radar, and/or any other suitable type of radar system. Stated succinctly the present disclosure is not limited to any specific type of radar system being used in conjunction with the image processing system <NUM>.

The image processing system <NUM> may include any suitable type of computing system that is configured to classify data that is provided by the radar system <NUM>. In some implementations, the image processing system <NUM> may be configured to classify targets that are acquired by the radar system. Although the radar system <NUM> and the image processing system <NUM> are depicted as separate systems, it will be understood that in some implementations they can be integrated together into a single system. Although in the example of <FIG> the image processing system <NUM> is used in conjunction with a radar system, it will be understood that alternative implementations are possible in which the image processing system <NUM> is used in conjunction with another type of imaging system. For example, the image processing system <NUM> may be used in conjunction with a medical image system (e.g., a CAT-scan or an MRI system), a SONAR system, a visible-range camera, an infrared camera, and/or any other suitable type of imaging system.

<FIG> depicts the image processing system <NUM> in further detail. As illustrated, the image processing system <NUM> includes an orientation classifier <NUM>, an image rotator <NUM>, and an image classifier <NUM>. In operation, the orientation classifier <NUM> receives an input image <NUM> and generates an output vector <NUM> that identifies an angle by which the image is rotated from a "base" orientation. The image rotator <NUM> rotates the input image <NUM> by the angle that is identified in the output vector <NUM> to produce a rotated image <NUM>. The image classifier <NUM> classifies the rotated image <NUM> and outputs an object class identifier <NUM> that identifies a class of the image. According to the present example, the input image <NUM> is an image of an aircraft and the object class identifier includes a string that identifies a type of the aircraft (e.g., B747, F16, Eurofighter, etc.).

According to the present example, the input image <NUM> is an Inverse Synthetic-Aperture Radar (ISAR) image. The input image <NUM> may be obtained by (i) receiving an original image from the radar system <NUM>, (ii) filtering the original image (e.g., with a low-pass filter) to produce a filtered image, and (iii) and down-sampling the filtered image to produce the input image <NUM>. According to the present example, the filtered image is down-sampled to a resolution of 28x28 pixels. However, the present disclosure is not limited to any specific resolution.

The orientation classifier <NUM> includes a first convolutional neural network and the image classifier <NUM> may include a second convolutional network that is separate from the first convolutional network. In some implementations, the orientation classifier <NUM> and the image classifier <NUM> may be executed as separate processes or threads. Although in the present example, each of the orientation classifier <NUM>, the image rotator <NUM>, and the image classifier <NUM> are implemented in software. Alternative implementations are possible in which any of the orientation classifier <NUM>, the image rotator <NUM>, and the image classifier <NUM> is implemented in hardware or as a combination of hardware or software.

The orientation classifier <NUM>, as noted above, employs a first convolutional neural network to determine a rotational offset of the input image <NUM> (i.e., a rotational offset of an object depicted in the input image <NUM>), relative to a base orientation. As illustrated in <FIG>, the base orientation of the image can be a "portrait" orientation. However, it will be understood that the base orientation may be any orientation that is the same as the orientation of images that are used to train the image classifier <NUM>.

The image rotator <NUM> is configured to rotate (or de-rotate) the input image <NUM> by the rotational offset that is determined by the orientation classifier <NUM>. Rotating the input image <NUM> may include generating a new image (e.g., the rotated image <NUM>) which depicts the same object as the input image <NUM>, and in which the object is rotated by the rotational angle determined by the orientation classifier <NUM>. In some implementations, both the input image <NUM> and the rotated image <NUM> may have the same resolution (e.g., 28x28 pixels). Additionally or alternatively, in some implementations, both the input image <NUM> and the rotated image <NUM> may have the same aspect ratio. In some implementations, the rotated image <NUM> may be generated, at least in part, by applying a rotational transformation to the input image <NUM>.

The image classifier <NUM>, as noted above, may employ a second convolutional neural network to classify the rotated image <NUM>. As is known in the art, features learned by convolutional neural networks are not rotationally invariant. In this regard, rotating the input image <NUM> such that it is placed in an upright orientation for example, before providing it to the image classifier <NUM>, may help improve the accuracy of the classification. As can be readily appreciated, the image classifier <NUM> may be trained with images that are rotated at random angles. However, this approach may require a larger amount of training data than the present approach.

As noted above, the image processor <NUM> uses a two-fold approach in which a rotation angle is identified for the image <NUM>, and the image <NUM> is rotated (or transformed) based on the rotation angle. In general, a neural network can be used to transform (or rotate) an image without having to find a rotation angle first. However, such a method may permit nonlinear manipulation of the image, which could sacrifice the structure of the image. By contrast, the image rotator <NUM> may support only rotational transformation, and, as such, it might not alter the structure of the image in the same way the neural network approach would. Accordingly, it will be understood that using the image rotator <NUM> to generate the rotated image <NUM> (which is subsequently input into the convolutional neural network <NUM>) is advantageous because does not sacrifice the structure of the input image <NUM>.

According to the example of <FIG>, the input image <NUM> is an ISAR image, however alternative implementations are possible in which the input image <NUM> is another type of image, such as a photographic image (e.g., an image captured with a digital camera), a graphics image, a sonar image, a SONAR image, an X-ray image, an MRI image, and/or any other suitable type of image. In this regard, it will be understood that the concepts and ideas described throughout the specification are not limited to the processing of any specific type of image.

<FIG> shows the orientation classifier <NUM> in further detail. As illustrated, the orientation classifier <NUM> includes a convolutional neural network <NUM>. The convolutional neural network <NUM> may include: a convolutional and max-pooling layer <NUM>, a convolutional and max-pooling layer <NUM>, a flattening layer <NUM>, a fully connected layer <NUM>, and a fully-connected layer <NUM>. The convolutional neural network <NUM> may receive the input image <NUM> as input and output the output vector <NUM>. It will be understood that <FIG> is provided as an example only. Those of ordinary skill in the art will readily recognize, after reading this disclosure, that alternative configurations are possible for the convolutional neural network <NUM>, which may include a different sequence of hidden layers, a different number of convolutional layers, and/or a different number fully-connected layers. In this regard, it will be understood that the present disclosure is not limited to any specific implementation of the convolutional neural network <NUM>.

<FIG> shows an example of the output vector <NUM> in further detail. As illustrated, the output vector <NUM> may include a plurality of elements. Each element in the output vector <NUM> may correspond to a different classification category(or angle bin in the present case). The value of each element in the output vector <NUM> may include the probability of the input image <NUM> belonging to the element's category. According to the example of <FIG>, the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with an <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the element with index <NUM> has a value of <NUM> and is associated with an <NUM>-degree angle; and the element with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle. The output vector <NUM> indicates that the input image <NUM> is classified in the <NUM>-degree angle category (because the element at index <NUM> has the largest value).

<FIG> further shows a one-hot representation <NUM> of the output vector <NUM>. As illustrated, the one-hot representation <NUM> may include a different bit for each element of the output vector <NUM>. The one-hot representation may be generated by: (i) identifying the largest element in the output vector <NUM>, (ii) identifying the index of the largest element in the output vector <NUM>, (iii) setting the bit in the hot-representation <NUM> that has the same index as the largest element in the output vector <NUM> to '<NUM>', and (iv) setting all other bits to '<NUM>'. According to the example of <FIG>, the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle; and the bit with index <NUM> has a value of <NUM> and is associated with a <NUM>-degree angle. The one hot representation <NUM> has the bit at index <NUM> set to '<NUM>' because the input image <NUM> has been classified in the <NUM>-degree angle category by the convolutional neural network <NUM>. As used throughout the disclosure, the phrase "index of a one-hot representation of an output vector (or label)" shall refer to the index of the bit in the one-hot representation that is set to '<NUM>'. Under this definition, the index of the one-hot representation <NUM> is <NUM>.

As noted above, the convolutional neural network <NUM> may implement a classification method that labels discrete "bins" of orientation angles as unique categories. The width of each bin is <NUM> degrees. According to the present disclosure, it has been observed that the accuracy of the classifier deteriorates if bins larger than <NUM> degrees are used. However, it will be understood that the present disclosure is not limited to any specific bin width. Furthermore, it will be further understood that the number of bins used by the convolutional neural network <NUM> may vary. In general, the number of bins in the neural network <NUM> would be the same as the number of bits in the output vector <NUM>.

<FIG> is a diagram of an example of a training data set <NUM> that can be used to train the convolutional neural network <NUM>. According to the present example, the training data set includes a plurality of entries <NUM>. Each entry <NUM> may include a training image and a label corresponding to the training image. The label corresponding to the training image may be the same as an output vector, which the convolutional neural network <NUM> would generate, when classifying the training image, if the training image were classified correctly. According to the example of <FIG>, entry 370A may include a first training image that is associated with a first label (i.e., a first output vector); entry 370B may include a second training image that is associated with a second label (i.e., a second output vector); and entry 370C may include a third training image that is associated with a third label (i.e., a third output vector). Although in the example of <FIG>, the training data set <NUM> includes three training images only, it will be understood that in practice the training data set <NUM> would have a much larger number of images.

An example of one method for generating training images is now discussed in further detail. According to the example, the training images are generated by Inverse Synthetic Aperture Radar (ISAR) and/or by using a signal processing technique that is performed by ISAR. Although the signal processing technique is discussed in the context of generating training images, it will be understood that the input image <NUM> may also be generated, at least in part, by using the same or similar signal processing.

In signal processing, a matched filter is obtained by correlating a known delayed signal, with an unknown signal to detect the presence of the known signal in the unknown signal. This is equivalent to convolving the unknown signal with a conjugated time-reversed version of the known signal. Matched filters are commonly used in radar, in which a known signal is sent out, and the reflected signal is examined for common elements of the outgoing signal. Pulse compression is an example of matched filtering where the impulse response is matched to input pulse signals. Pulse compression is used to increase the range resolution as well as the signal to noise ratio. This is achieved by modulating the transmitted pulse and then correlating the received signal with the transmitted pulse. The cross-correlation is computed by convolving the received signal with a conjugated and time-reversed version of the transmitted signal.

In some implementations, linear frequency modulation (LFM) waveforms may be incorporated with a point backscattering model to generate range-Doppler image of a moving target. Suppose T<NUM> is the single chirp pulse duration, fc is the center frequency of the chirp, and the slope of the chirp pulse is defined as K = B/T<NUM> in which B is the frequency bandwidth. The LFM chirp pulse can be written as: <MAT>.

The returned signal may be re-written by adding time delay term ti = 2Ri/c ( Ri is the relative range w. the reference point for i-th scatterer, and c is the light speed) to equation (<NUM>): <MAT>.

The matched filter response function may be expressed as the conjugated and time-reversed transmitted signal shown in the following form: <MAT>.

Therefore, the pulse compression response is represented as a convolution of the returned signal as in equation (<NUM>) with the matched filter response function as in equation (<NUM>): <MAT>.

The response may be discretized into M samples with the range resolution being <MAT>, and N pulses with cross-range resolution being <MAT>, where λ is the wavelength, Ω is related to the rotation angle, and T is the total dwell time T = N/PRF.

An example for generating training images is now discussed in further detail. According to the example, 3D airplane CAD models may be used to generate a representative application-based dataset. For example, four airplane CAD models may be used: Boeing <NUM>, Euro-fighter EF2000, F16 and F35. These models may be limited to providing geometrical information only. The 3D airplane models may be run through the physical optics program POFACETS to generate target Radar Cross-Section (RCS) estimates for each target aspect angle. The target mesh nodes may be down-selected and used as representative radar scatterer points. Randomization of scatter locations and amplitudes may be used throughout to increase the amount of variation within each class. In some implementations, the values of fc, B, and T<NUM> may be set as follows: fc = <NUM>, B = <NUM> GHz, T<NUM> = <NUM>.

Furthermore, to simulate real-world effects in the ISAR images object shadowing and simulated delayed returns may be imparted on training images that are generated based on CAD models. Shadowing occurs when scatterers on an object are occluded by the hull of that object. The shadowing may be based on aspect angle for given radar line-of-sight. Delayed returns occur when radar energy bounces off multiple surfaces on the object. For example, when radar energy enters into an open cavity on an object. In some implementations, delayed returns may be randomly generated to simulate representative multi-bounce characteristics.

According to the example of <FIG>, the training images that are used to train the convolutional neural network <NUM> are ISAR images, however alternative implementations are possible in which another type of images is used to train the convolutional neural network <NUM>, such as a photographic image (e.g., an image captured with a digital camera), a graphics image, a sonar image, an X-ray image, an MRI image, and/or any other suitable type of image. In this regard, it will be understood that the concepts and ideas described throughout the specification are not limited to using any specific type of image to train the convolutional neural network <NUM>.

An example of a method for training the convolutional neural network <NUM> is now discussed in further detail. According to the method, the convolutional neural network <NUM> may be trained by using a gradient descent algorithm that is designed to minimize the value of the loss function Lcat. The loss function Lcat may be defined as follows: <MAT> <MAT> <MAT> where i is an index of an element in an output vector that is generated for a training image in the training data set <NUM>; yi is the i-th element of the output vector that is generated for the training image; ŷi is the i-th element in a label that is associated with the training image; ξy is the index of the largest element in the output vector (e.g. <MAT>), ξŷ is the index of the one-hot representation of the label y̌ (e.g. <MAT>) (i.e., the index of the bit in the one-hot representation of ŷ that is set to '<NUM>' - it will be recalled that ŷ may have the same format as an output vector); and K is a count of classes that are associated with the convolutional neural network <NUM> (e.g., a count of bins that are implemented by the network). It will be understood that the definition of a categorical distance is not limited to Equation <NUM>.

The term lH represents a standard cross-entropy loss. The term lθ represents a categorical distance. The difference between the entropy loss term lH and the categorical distance lθ is now discussed with respect to <FIG> shows a plot <NUM> of an output vector <NUM> that is generated by a <NUM>-class neural network in response to a first input image. The neural network is configured to classify any image that is input into it in one of three classes - i.e., class C<NUM>, class C<NUM>, and class C<NUM>. <FIG> further shows a plot <NUM> of an output vector <NUM> that is generated by the same neural network in response to a second input image. The correct classification of the first input image and the second input image is class C<NUM>. Both output vectors have a highest value for class C<NUM> and will therefore be classified, incorrectly, in class C<NUM>. Because both input images are classified incorrectly in the same class (i.e., class C<NUM>), the value of lθ for each of the input images will be the same. However, in the example of <FIG>, the first output vector <NUM> has a much higher value for the correct class (C<NUM>) than the second output vector <NUM>. For this reason, the value of lH for the first input image will be lower than the value of lH for the second input image. Put differently, lH is a measure of how different is the probability mass distribution of a label corresponding to a training image from the output vector that is produced by the convolutional network for the training image. By contrast, lθ is a categorical distance - namely distance between the correct category of a training image and the category that has been assigned to the training image by the convolutional neural network <NUM>. The value of lH would normally depend on all probabilities in the output vector that is produced by the convolutional neural network <NUM> (i.e., on all elements of the output vector), whereas the value of lθ would depend on the index of the largest probability in the output vector (i.e., on the index of the largest element of the output vector).

The term lθ is, in essence, a circular absolute error term. It incurs no extra error when the predicted class matches the correct class, but it penalizes incorrect classifications further from the desired angle bin. This loss strategy is very granular; it does not take into account the specific values of each prediction vector y, only which simplex vertex it is closest to and penalizes accordingly. The latter behavior is compensated for by the standard categorical cross-entropy loss term lH.

In some respects, the addition of the term lθ helps the convolutional neural network <NUM> maintain high accuracy with a comparatively low number of neurons. Keeping the number of neurons in the convolutional neural network <NUM> low is advantageous, because it makes the convolutional neural network <NUM> less susceptible to jamming and/or deceptive tactics that aim to alter the radar signature of targets.

As noted above, training the convolutional neural network <NUM> may entail finding coefficient and bias values for neurons in the convolutional neural network <NUM> for which the loss function Lcat is minimized. In this regard, training the convolutional neural network <NUM> with the loss function Lcat causes the convolutional neural network <NUM> to include a set of the coefficient and bias values that are specific to (and/or derivable from) the loss function Lcat (and the training data set). By contrast, if the convolutional neural network <NUM> were to be trained with another loss function (and the same training data set), the convolutional neural network <NUM> may include a different set of coefficient and bias values (when all other characteristics of the method for training the two neural networks are held equal). In other words, training the convolutional neural network <NUM> with the loss function Lcat may impart, at least in part, a specific configuration on the convolutional neural network <NUM> that is specific (and/or directly related) to the loss function Lcat. Or put differently, the loss function Lcat encourages the convolutional neural network <NUM> to cluster classification attempts around the correct category. According to the present disclosure, it has been observed that using the categorical distance lθ to train the convolutional network <NUM> may improve the accuracy of the convolutional neural network <NUM> with respect to images that contain noise due to delayed return or noise due to shadowing (such as ISAR images or other types of radar images).

<FIG> shows a plot <NUM>, which compares the performance of the image processing system <NUM> to that of a conventional image classifier. The conventional image classifier is a convolutional neural network that includes the following sequence of layers: (i) a convolutional layer, (ii) a max-pool layer, (iii) a convolutional layer, (iv) a max-pool layer, (v) a flattening layer, a (vi) a dense layer, (vii) a drop-out layer (viii) a dense layer, and (ix) a dense layer. The conventional image classifier is set to include a similar number of tunable parameters as the convolutional neural network <NUM>. The conventional neural network is trained by using randomly rotated images. Unlike the image classifier <NUM>, the conventional image classifier classifies images that are not rotated beforehand. In the example of <FIG>, the X-axis of the plot <NUM> corresponds to the size of a training data set that is used to train the image classifier <NUM> and the conventional image classifier, while the Y-axis of the plot <NUM> corresponds to accuracy. Shown in the plot <NUM> are a curve <NUM> and a curve <NUM>. The curve <NUM> shows the accuracy of the image classifier <NUM> and the curve <NUM> shows the accuracy of the conventional image classifier. Together the curve <NUM> and the curve <NUM> illustrate that the introduction of the orientation classifier <NUM> (which incorporates the convolutional neural network <NUM>) results in the image processing system <NUM> having a higher accuracy than the conventional image classifier, when a small training data set is used. In various specialized applications, such as recognition of aircraft, large training data sets may be unavailable. In such applications, using the image processing system <NUM> to perform image classification may be advantageous because it could result in higher accuracy.

<FIG> is a flowchart of an example of a process <NUM>, according to aspects of the disclosure. At step <NUM>, the image processing system <NUM> obtains an image from the radar system <NUM>. At step <NUM>, the image processing system <NUM> filters the obtained image with a low-pass filter. At step <NUM>, the image processing system <NUM>, down-samples the image to predetermined resolution (e.g., 28x28 pixels, etc.) to produce the input image <NUM>. At step <NUM>, the image processing system <NUM> identifies a rotational angle for the input image <NUM> by using the orientation classifier <NUM>. At step <NUM>, the image processing system <NUM> uses the image rotator <NUM> to rotate the input image <NUM> by the identified angle to produce the rotated image <NUM>. Ideally, in some implementations, if the rotational angle is determined correctly, the rotation would substantially align (e.g., within <NUM> degrees) the orientation of an object that is depicted in the image (e.g., an aircraft) with the orientation of similar objects in training images that are used to the train the convolutional neural network. At step <NUM>, the image processing system <NUM> classifies the rotated image by using image classifier <NUM>. At step <NUM>, the image processing system outputs an indication of an outcome of the classification. According to the present example, the image processing system <NUM> outputs an indication of a class identifier that is assigned to the input image by the image classifier <NUM>.

Referring to <FIG>, computing device <NUM> that can be used to implement, at least in part, the image processing system <NUM>. As illustrated, the computing device <NUM> may include processor <NUM>, volatile memory <NUM> (e.g., RAM), non-volatile memory <NUM> (e.g., a hard disk drive, a solid-state drive such as a flash drive, a hybrid magnetic and solid-state drive, etc.), graphical user interface (GUI) <NUM> (e.g., a touchscreen, a display, and so forth) and input/output (I/O) device <NUM> (e.g., a mouse, a keyboard, etc.). Non-volatile memory <NUM> stores computer instructions <NUM>, an operating system <NUM> and data <NUM> such that, for example, the computer instructions <NUM> are executed by the processor <NUM> out of volatile memory <NUM>. Program code may be applied to data entered using an input device of GUI <NUM> or received from I/O device <NUM>.

Processor <NUM> may be implemented by one or more programmable processors executing one or more computer programs to perform the functions of the system. As used herein, the term "processor" describes an electronic circuit that performs a function, an operation, or a sequence of operations. The function, operation, or sequence of operations may be hard-coded into the electronic circuit or soft coded by way of instructions held in a memory device. A "processor" may perform the function, operation, or sequence of operations using digital values or using analog signals. In some embodiments, the "processor" can be embodied in an application-specific integrated circuit (ASIC). In some embodiments, the "processor" may be embodied in a microprocessor with associated program memory. In some embodiments, the "processor" may be embodied in a discrete electronic circuit. The "processor" may be analog, digital or mixed-signal. In some embodiments, the "processor" may be one or more physical processors or one or more "virtual" (e.g., remotely located or "cloud") processors.

The processes described herein are not limited to use with hardware and software of <FIG>; they may find applicability in any computing or processing environment and with any type of machine or set of machines that is capable of running a computer program. The processes described herein may be implemented in hardware, software, or a combination of the two. The processes described herein may be implemented in computer programs executed on programmable computers/machines that each includes a processor, a non-transitory machine-readable medium or another article of manufacture that is readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code may be applied to data entered using an input device to perform any of the processes described herein and to generate output information.

The system may be implemented, at least in part, via a computer program product, (e.g., in a non-transitory machine-readable storage medium such as, for example, a non-transitory computer-readable medium), for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). Each such program may be implemented in a high-level procedural or object-oriented programming language to work with the rest of the computer-based system. However, the programs may be implemented in assembly, machine language, or Hardware Description Language. The language may be a compiled or an interpreted language, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or another unit suitable for use in a computing environment. A computer program may be deployed to be executed on one computer or multiple computers at one site or distributed across multiple sites and interconnected by a communication network. A computer program may be stored on a non-transitory machine-readable medium that is readable by a general or special purpose programmable computer for configuring and operating the computer when the non-transitory machine-readable medium is read by the computer to perform the processes described herein. For example, the processes described herein may also be implemented as a non-transitory machine-readable storage medium, configured with a computer program, where upon execution, instructions in the computer program cause the computer to operate in accordance with the processes. A non-transitory machine-readable medium may include but is not limited to a hard drive, compact disc, flash memory, non-volatile memory, volatile memory, magnetic diskette and so forth but does not include a transitory signal per se.

Claim 1:
A method comprising:
obtaining an image (<NUM>);
identifying a rotation angle for the image by processing the image with a first neural network, wherein the first neural network is a convolutional neural network that determines a rotational offset of an object depicted in the image relative to a base orientation;
rotating the image by the identified rotation angle to generate a rotated image (<NUM>); and
classifying the rotated image with a second neural network; and
outputting an indication of an outcome of the classification,
wherein the first neural network is trained, at least in part, based on a sum of a categorical cross-entropy term that is associated with an output of the first neural network and a categorical distance between training data and the output that is produced by the first neural network,
wherein the categorical distance is defined as follows: <MAT>
where ξy is an index of a one-hot representation of an output vector that is generated by the first neural network for a training image, ξŷ is an index of a one-hot representation of a label that is associated with the training image, and K is a count of classes that are associated with the first neural network.