Patent Description:
Currently, on many live fire and virtual fire ranges, the instructor must pay close attention to each trainee in order to identify whether the trainee is making fundamental shooting errors. Typically, firearms instructors focus on addressing the four fundamentals of marksmanship: <NUM>) establishing a steady body position in order to observe the target, <NUM>) aiming the weapon correctly and consistently to align the sights with the target, <NUM>) controlling breathing during firing of the weapon, and <NUM>) properly squeezing the trigger. The instructor analyzes these aspects of a shooter to determine whether proper techniques are being applied. For example, the instructor may be trying to detect if a shooters poor performance is due to improper breathing, body position, canting the weapon, improper trigger squeeze, suboptimal point of aim, unsteady position or some combination thereof. This can prove difficult given the high numbers of trainees using the system simultaneously, and is even more difficult on a live range where many of diagnostic sensors are not instrumented on the weapon. To this end, it would be desirable to have a system and method to allow a shooter to provide more consistent and frequent feedback about what mistakes he or she may be making that results in a suboptimal shot. The following documents disclose relevant features of the prior art: <CIT>, <CIT>, <CIT>, <CIT>, <CIT>, <CIT>.

In view of the foregoing, an object according to one aspect of the present patent document is to provide systems and methods for automated coaching of a shooter. Preferably, the methods and systems address, or at least ameliorate, one or more of the problems described above.

The present invention provides a method as set out in claim <NUM> and a system as set out claim <NUM>.

In some embodiments, the machine learning algorithm may be an Artificial Neural Network. The machine learning algorithm may use techniques including Kernel Discriminant Analysis and the like.

Although many different kinds of sensors may be used, in a preferred embodiment, the shot dispersion data is obtained by a camera. In other systems other methods may be used to obtain shot data including acoustic analysis. Once the shot data is obtained, it is preferably analyzed using image analysis.

The sensors may also be used to measure or track any important quantity. In a preferred embodiment, the sensor measures any one of the group consisting of trigger squeeze pressure or displacement, weapon cant, butt pressure, aim trace, trigger break (the moment the trigger breaks typically causing the weapon's hammer to fall, initiating the shot), aim trace relative to the ideal aim-point on the target, shooter pose (e.g. standing, kneeling, prone), shot group size, and shot group mean point of impact relative to ideal impact.

In preferred embodiments, a predicted cause from a plurality of trainees is displayed on a single screen. This may be the screen of an instructor, trainee, or both. In some embodiments, select information is provided to the instructor while other information is provided to the trainee. This may be an appropriate subset of the information shown to the instructor.

Further aspects, objects, desirable features, and advantages of the systems and methods disclosed herein will be better understood from the detailed description and drawings that follow in which various embodiments are illustrated by way of example. It is to be expressly understood, however, that the drawings are for the purpose of illustration only and are not intended as a definition of the limits of the claimed invention.

The following detailed description includes representative examples utilizing numerous features and teachings, both separately and in combination, and describes numerous embodiments in further detail with reference to the attached drawings. This detailed description is merely intended to teach a person of skill in the art further details for practicing preferred aspects of the present teachings and is not intended to limit the scope of the claims. Therefore, combinations of features disclosed in the following detailed description may not be necessary to practice the teachings in the broadest sense, and are instead taught merely to describe particularly representative examples of the present teachings.

Some portions of the detailed descriptions that follow are presented in terms of algorithms and sequences of operations which are performed within a computer memory. An algorithm or sequence of operations is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing," "computing," "calculating," "determining," "displaying" or the like, refer to the action and processes of a computer system, or similar electronic device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the electronic device's memory or registers or other such information storage, transmission or display devices.

The embodiments disclosed also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose processor selectively activated or reconfigured by a computer program stored in the electronic device. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, Flash memory, magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.

The algorithms presented herein are not inherently related to any particular electronic device or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct a more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear from the description below.

Moreover, the various features of the representative examples and the dependent claims may be combined in ways that are not specifically and explicitly enumerated in order to provide additional useful embodiments of the present teachings. It is also expressly noted that all value ranges or indications of groups of entities disclose every possible intermediate value or intermediate entity for the purpose of original disclosure, as well as for the purpose of restricting the claimed subject matter. It is also expressly noted that the dimensions and the shapes of the components shown in the figures are designed to help understand how the present teachings are practiced, but not intended to limit the dimensions and the shapes shown in the examples.

In the present patent document, methods and systems for automating coaching of a shooter are provided. In such methods and systems, computerized algorithms are used to analyze the shot dispersions of a shooter and predict the cause for any suboptimal performance.

In a preferred embodiment, the embodiments of the present patent document may enhance a small arms training simulator to assist the shooting instructor in quickly discovering the most common shooting mistakes made by trainees. For example, the methods and systems herein may automatically analyze and highlight the most prevalent poor shooting habits. Accordingly, greater training throughput may be achieved because the system may automatically recognize the errors that an instructor would typically have to find.

The embodiments disclosed herein include novel techniques for enabling a computer system to automatically flag potential shooting mistakes. Traditionally, shooting analysis has been entirely the domain of a qualified shooting instructor who observes the shooter's physical interaction (e.g. body position, breathing patterns, sight address), hit results (i.e. where the round(s) hit the target), and environmental factors (e.g. wind, heat, visibility) in order to determine the shooter's mistakes and recommend improvements. Small arms simulators complement an instructor by providing additional data related to the shooter's performance by adding sensors to the weapon to provide details that are difficult to discern by mere observance (e.g. trigger squeeze pressure, weapon cant, butt pressure, aim trace).

In preferred embodiments, shot dispersion data and other sensor data streams are fed into an algorithmic analysis to provide advice to a shooter automatically. Preferably the advice relates to a shooter's mistakes without needing an expert shooting instructor to make the preliminary assessment. In some embodiments, an instructor can use the preliminary analysis to confirm the system's supposition. These systems and methods benefit the instructor since it can speed up the throughput of trainees on the system and reduce the burden on the instructor when training multiple trainees simultaneously. In at least one embodiment, the results are displayed to an instructor via a computer display.

In embodiments where an instructor remains in the loop, the instructor may be unobtrusively alerted. This allows the instructor to focus on trainees having performance issues while minimizing the delay for the other trainees undergoing training.

Machine learning is utilized to suggest diagnoses of shooting mistakes. In some embodiments, only common mistakes may be recognized. However, in other embodiments, more complicated problems may be detected. For example, problems related to complex combinations of problems may be detected.

In some embodiments, the suggestions are communicated directly to the shooter or trainee. This may be done through a screen display or an application ("app") located on a mobile device. In addition to the suggested causes of any suboptimal performance, the trainee may be able to see all the stats about their performance including but not limited to dispersion patterns, sensor responses, number of shots, distance between shots, time between shots and many other statistics.

In other embodiments, the potential causes for suboptimal performance may be communicated to an instructor. The instructor may evaluate the suggestions and/or predictions made by the automated system and then provide advice to a shooter or trainee. In some embodiments, data from multiple trainees or users may be gathered, and displayed to an instructor at a firing range. In other embodiments, both the trainee and the instructor may receive all the feedback available or any appropriate distribution of information based on the trainee instructor relationship.

In preferred embodiments the system may make use of previously stored shot patterns and shot dispersion patterns and their related causes to help make a prediction of a trainee's problems. <FIG> illustrates examples of shot dispersion patterns and their associated cause <NUM>. In preferred embodiments, systems may include entire databases <NUM> of shot patterns and or dispersion patterns and their associated causes <NUM>.

The data used to analyze the shooter or trainee may be gathered prior to a training exercise and/or in real time. The data may also be typical data related to all shooters or may be custom data specifically related to the trainee in question.

<FIG> illustrates a graphical representation of a method for automated coaching of a shooter <NUM>. As may be seen in <FIG>, both reference data <NUM> and trainee data <NUM> are fed into an algorithm <NUM> to produce a suggested cause of suboptimal performance <NUM>.

In a preferred embodiment, the reference data <NUM> is gathered as a set of training examples based on either doctrinal reference material or from expert shooting instructors. In other embodiments, reference data <NUM> can be gathered from other reference material or other types of instructors, such as the internet, surveys, or compiled over time. In some embodiments, this reference data <NUM> can include both the shot dispersion data <NUM> (e.g. where the bullets impacted the target) as well as biometric and weapon sensor data <NUM> if available. In preferred embodiments, for each type of collected data, there may be corresponding reference data <NUM> in a database for query.

The reference data <NUM> is used to determine a classification for the trainee data. For example, the reference data may be used to classify the trainee data into any number of sub-optimal performance results like bad aim, bad breathing or other mistakes.

In addition to dispersion data <NUM>, the reference data <NUM> includes sensor data <NUM>. Weapon sensor data <NUM> may include but is not limited to breathing data, weapon point of aim trace, weapon cant, weapon trigger pull sensor data, weapon butt pressure sensor data.

As the trainee or shooter practices, trainee data <NUM> is collected. Similar to the reference data <NUM>, trainee data may be made up of both dispersion data <NUM> and weapon sensor data <NUM>. One or more of these data points may be compared to the reference data <NUM> in order to determine a cause of suboptimal performance.

In some embodiments, cameras may be focused on the targets to obtain a trainee's shot dispersion data <NUM> in real time. An image analysis program maybe used to locate the shots and determine locations of each and distances between shots and distances from each shot to the aimed at target.

A machine learning algorithm <NUM> is used to compare the trainee data <NUM> with the reference data <NUM>. In some embodiments, each entry of reference data (termed an input vector) is paired with a desired output value (termed a supervisory signal). Accordingly, an algorithm <NUM> may analyze the reference data <NUM> in order to create an inferred function known as a classifier or regression function (depending upon the type of data). In some embodiments, this algorithm <NUM> essentially generalizes from the reference data <NUM> in order to reasonably handle similar (but not necessarily exact) data in order to make appropriate predictions <NUM>. In some embodiments, the data may be stored on a computer and in particular, in a database. The algorithm may also be run on a computer.

The algorithm <NUM> uses machine learning to allow the predictions <NUM> to become more accurate over time. Machine learning is a branch of artificial intelligence, primarily focused on how machines analyze and learn from data. By separating the problem into data representations (the sample training data <NUM>) and then generalizing based on the reference data <NUM> (to allow for new data that was not gathered), machine learning algorithms are created that can then return reasonable inferred predictions of the output solution.

In some embodiments, supervised learning, which is a specific type of machine learning may be employed. In supervised learning, the shooting instructors may rely on significant historical data to determine a likely diagnosis of the trainee's shooting issues and the machine may learn from this prediction. In some embodiments, the shooting instructor may confirm or deny a predicted cause of suboptimal shooting such that the algorithm learns over time.

In some embodiments, the machine learning algorithm <NUM> uses the reference data <NUM> to "train" the classifier resulting in associated relative weighting values being applied to the internal algorithm parameters. These "weights" allow the classifier to adequately classify the reference data <NUM> and, subsequently, are used to correctly classify the trainee data <NUM>. This may be thought of as an Artificial Neural Network. In some embodiments, the machine learning algorithm <NUM> can be a linear function (also known as linear predictor function) where weights are assigned to each possible category of classification (based on the reference data <NUM>)and then used (via a dot product) to combine with a particular instance (i.e. trainee data <NUM>) in order to be classified. In some embodiments, a linear function known as Kernel Discriminant Analysis (KDA) may be used. In particular, KDA may be used in conjunction with trigger squeeze data to classify any sub-optimal performance.

The weights are derived through a process whereby successive iterations of refinement are performed using instances of reference data <NUM> to calculate an error value as part of training the machine learning algorithm <NUM>. The error value is derived from the difference between the desired response (e.g. a known "Good" or "Bad") of the machine learning algorithm <NUM> and the actual response. The error value is then back-propagated through the algorithm's internal structure and used for successive iterations until the error is below a given threshold. Once this is achieved, the machine learning algorithm <NUM> is sufficiently "trained" and the weights can then be used to classify trainee data <NUM>.

<FIG> and <FIG> each show a representative trigger squeeze sensor feed. <FIG> illustrates a "good" trigger squeeze while <FIG> illustrates a "bad" trigger squeeze. In preferred embodiments, the machine learning algorithm <NUM> may be fed a number of "good" and "bad" trigger feeds along with the their respective classifications as "good" and "bad" to train the algorithm what a "good" feed looks like. Numerous data samples may be used. Once trained, the machine learning algorithm <NUM> can correctly classify the trigger squeeze sensor shown in <FIG> as "Good", which would be available for presentation to the instructor on a display.

As may be seen in <FIG>, in one aspect of the present patent document an automated coaching system <NUM> is provided. In some embodiments, the system comprises: at least one sensor <NUM> for obtaining training data of a trainee's shooting habits; a computer <NUM> containing an algorithm that evaluates the training data; a display <NUM> for displaying to an instructor at least one trainee's at least one mistake. In some embodiments, the system provides suggested corrections to fix the trainee's mistake.

As may be seen in <FIG>, sensors <NUM> may be located and sensing a plurality of different items. Sensors may be located on a trainee's weapon <NUM>, on the trainee themselves, on the target <NUM> or any other place that's desirable to acquire data. All of the various components of the system <NUM> may be connected via a network <NUM>. The network may be wired or wireless. A display <NUM> may be provided to the shooting coach and/or trainee to provide feedback about performance and potential correctable errors the trainee is making. A computer <NUM> runs a software program that includes a machine learning algorithm <NUM> that classifies the input from the various sensors <NUM> into shooting errors made by the trainee. In preferred embodiments, the machine learning algorithm uses a database <NUM> to compare the sensor outputs <NUM> with reference data. By comparing the trainee data against reference data, the machine learning algorithm <NUM> provides automated feedback to a shooter and/or coach about potential shooting errors that are occurring.

Claim 1:
A method for automatically predicting a suboptimal shooting mistake comprising:
providing a plurality of good example reference data examples (<NUM>) of shot dispersion patterns and examples from a sensor mounted on a weapon (<NUM>) that do not indicate the suboptimal shooting mistake to a machine learning algorithm (<NUM>);
providing a plurality of bad example reference data examples (<NUM>) of shot dispersion patterns and examples from the sensor mounted on the weapon (<NUM>) that do indicate the suboptimal shooting mistake to the machine learning algorithm (<NUM>);
calculating an error value for the machine learning algorithm (<NUM>) based on differences between a desired response of the machine learning algorithm (<NUM>) and an actual response of the machine learning algorithm (<NUM>) to the plurality of good example reference data examples (<NUM>) and to the plurality of bad example reference data examples (<NUM>);
adjusting weights in the machine learning algorithm (<NUM>) and back propagating the error value until the error value is below a given threshold value;
obtaining training data (<NUM>) of a trainee's shot dispersion data (<NUM>) and from at least one sensor mounted on a trainee's weapon, wherein the at least one sensor is identical to the sensor (<NUM>) used in examples from the reference data;
using the machine learning algorithm (<NUM>) including the weights to classify the training data (<NUM>) into a classification of either including the suboptimal shooting mistake or not including the suboptimal shooting mistake; and,
displaying feedback on a screen about the suboptimal shooting mistake based on the classification