Patent Description:
Shooting incidents, involving active shooters shooting victims within buildings such as schools or malls, are increasingly a concern. Identifying and reacting quickly to such incidents can reduce loss of life. For example, first responders need to know if gunshots have actually been fired and the location and number of the fired shots.

In confined areas, such as in a school or a private or public building, detecting and locating the source of gunshots is a complicated problem. A gunshot typically generates several sounds including the gunshot itself, the bullet's bow shockwave, noise from bullet impacts and noise of reflected gunshot sounds. In addition, numerous noises are generated in buildings that may be mistakenly identified as gunshots.

The broad concept of detecting gunshots utilizing acoustics is known. More specifically, it is known to provide a gunshot detection system including an array of acoustic sensors positioned in a pattern which enables signals from the sensors to be employed to not only detect the firing of a gunshot but to also locate the origin of the shot. One main requirement of such a system is the need to accurately distinguish between the sound produced from a gunshot and a host of other ambient sounds. In at least one known arrangement, a microphone is used to detect each sound, which is then amplified, converted to an electrical signal and then the electrical signal is compared with a threshold value above which a gunshot sound is expected to exceed.

Examples of gunshot detection systems are described in <CIT> and <CIT>.

Recently, gunshot detection systems with improved accuracy, dependability, and effectiveness have been described. One such system is described in International Publication Number <CIT> and entitled "System and Method for Acoustically Identifying Gunshots Fired Indoors. " This system provides for low false alarms or false positives and high detection rates by employing two microelectromechanical microphones (MEMs) having different sensitivity levels. Acoustic signals from a first microphone with lower sensitivity (for example, making the anomaly detection microphone essentially deaf to routine sounds) are first analyzed for a peak amplitude level large enough to be a potential gunshot. Then acoustic signals from a second microphone having a higher sensitivity are then analyzed further to confirm that the sound was a gunshot.

Gunshot detection methods have also been proposed that can count the number of gunshots fired, particularly from an automatic or fast acting weapon. One such method is described in International Publication Number <CIT> and entitled "Method for Acoustically Counting Gunshots Fired Indoors. " In this method, an acoustic signature of captured noise is analyzed to accurately count how many shots are fired. The method can be employed to identify that the gun is an automatic or rapid fire weapon, which information can be provided to emergency personnel.

Additionally, gunshot detection system that can accurately determine where sensed events are located have been proposed. One such system is described in International Publication Number <CIT> and entitled "System and Method for Identifying and Locating Sensed Events. " Here, a sensor network is employed to detect an event in the form of an audible signal. The event is time stamped and sent to a controller, which evaluates the event as either unique or a multiple detection using the sensor's time of alarm to determine which sensor activated first and to suppress subsequent alarms for the same event. This process is known as de-confliction.

Features can be added to these gunshot detection systems to make them more useful in a wider range of contexts. For example, it would be desirable to extend acoustic recognition capabilities of these systems, for example, to include other sounds that might be indicative of dangerous situations.

The presently disclosed gunshot detection system continually or periodically monitors ambient noise throughout a building, possibly learning the building's acoustic patterns, and alerts users when sounds are found to be unusual or possibly indicative of a dangerous situation. Gunshot sensor units positioned throughout the building include microphones. During a learning period, these microphones capture ambient noise during normal conditions. The gunshot detection system generates ambient noise models representing the ambient noise during normal conditions. The gunshot detection system then processes currently captured audio data depicting, for example, potential acoustic anomalies against the ambient noise models to detect and/or identify the acoustic anomalies and to determine whether to generate an alert. In this way, the system can alert users such as an operator of a control panel to any unusual sounds occurring throughout the building. The operator might then investigate the unusual sounds to make sure there is no emergency.

For example, if sensors were deployed in hotel rooms it could monitor the usual sounds generated therein and generate the ambient noise model. If the system detected a loud thump or screech, it could notify the front desk to initiate an inquiry by, for example, sending someone to knock on the door of the room. In another example, if the system is expecting a certain level of ambient noise based on the ambient noise model but the amplitude is suddenly diminished, that could be a sign that someone has blocked the microphone and an inquiry is warranted.

Further enhancements to the gunshot detection system could also include not only detecting the acoustic anomalies but identifying them as well. For example, the system could use preconfigured danger indicators, which represent acoustic patterns indicative of different dangerous situations, for example, including acoustic patterns associated with breaking glass, human screams, unexpected crashing noises, or even obstruction of the microphones of the gunshot sensor units. By processing the current audio data against the danger indicators (e.g. by recognizing the acoustic patterns of the danger indicators in the current audio data), the system could identify the acoustic anomalies and include information about the recognized danger indicator in the alerts sent to the users of the system.

The gunshot detection system can further include privacy features for ensuring that communications between occupants of the building are not monitored using the.

The gunshot detection system can further include privacy features for ensuring that communications between occupants of the building are not monitored using the gunshot detection system. For example, the ambient noise models could be configured to ensure that speech sounds are not detected as acoustic anomalies. In another example the current audio data generated depicting the ambient noise could be filtered to remove speech sounds matching certain criteria (e.g. at a normal expected volume). In another example, the gunshot detection system automatically deletes the current audio data upon confirmation that no acoustic anomalies were detected. In yet another layer of protection, the current audio data could even be deleted if detected acoustic anomalies are not then identified based on the preconfigured danger indicators.

In general, according to one aspect, the invention features a system for detecting gunshots as defined in claim <NUM>.

In embodiments, controllers of the gunshot sensor units could generate the ambient noise models and process the current audio data to detect the acoustic anomalies, or the gunshot sensor units could stream the current audio data to a control panel of the gunshot detection system, which would generate the ambient noise models and process the current audio data. In either case, the ambient noise models could then be stored in nonvolatile storage of the gunshot sensor units and the control panels respectively. The acoustic anomalies could be identified by processing the current audio data against danger indicators, which represent acoustic patterns indicative of dangerous situations such as gunshot sounds, screaming sounds, unusually loud sounds and/or unusually low amplitude sounds (e.g. indicating obstruction of the microphones of the gunshot sensor units). The current audio data could be automatically deleted in response to detecting no acoustic anomalies. On the other hand, when acoustic anomalies are detected, alerts could be generated based on the detected acoustic anomalies.

In general, according to another aspect, the invention features a method for detecting gunshots as set out in claim <NUM>.

The above and other features of the invention including various novel details of construction and combinations of parts, and other advantages, will now be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular method and device embodying the invention are shown by way of illustration and not as a limitation of the invention. The principles and features of this invention may be employed in various and numerous embodiments without departing from the scope of the invention.

<FIG> is a schematic diagram of an exemplary gunshot detection system <NUM>.

In general, the gunshot detection system <NUM> monitors, detects and reports the occurrence of gunshots or other emergencies within a premises <NUM> such as a building (e.g. office, hospital, warehouse, retail establishment, shopping mall, school, multi-unit dwelling, government building).

In the illustrated example, the premises <NUM> is a simplified floor example of a building with three areas <NUM>, a lecture hall <NUM>-<NUM>, classroom A <NUM>-<NUM>, and classroom B <NUM>-<NUM>.

The gunshot detection system <NUM> includes gunshot sensor units <NUM> and a control panel <NUM>, which communicate with each other over a communication network <NUM>.

The gunshot sensor units <NUM> are distributed throughout the premises <NUM>, for example, in areas <NUM> of the premises such as rooms, hallways, lobbies or stairways, to name a few examples. In the illustrated example, two gunshot sensor units <NUM>-<NUM>, <NUM>-<NUM> are located in the lecture hall <NUM>-<NUM>, while one gunshot sensor unit <NUM>-<NUM> is located in classroom A <NUM>-<NUM>, and one gunshot sensor unit <NUM>-<NUM> is located in classroom B <NUM>-<NUM>.

In general, according to a baseline gunshot detection functionality of the gunshot detection system <NUM>, the gunshot sensor units <NUM> detect conditions indicative of the gunshots or other emergencies and alert the control panel <NUM>, which takes one or more responsive actions such as alerting building personnel, law enforcement, and/or a monitoring center, or collecting and presenting data pertaining to the detected gunshots to an operator of the control panel <NUM>.

More specifically, the gunshot sensor units <NUM> detect acoustic anomalies indicating potential gunshots and generate audio data depicting the acoustic anomalies. The gunshot sensor units <NUM> also generate event data based on and descriptive of the acoustic anomalies and locally store and/or send the event data to the control panel <NUM>.

The event data often includes: audio data (e.g. digitized audio clips) depicting the acoustic anomalies and metadata. This metadata includes, for example, time information indicating when the acoustic anomalies started and/or stopped, duration information for the acoustic anomalies and/or the audio data depicting the acoustic anomalies, file information, and identification information for the gunshot sensor unit <NUM> that detected the anomaly; and other sensor data generated by the gunshot sensor unit.

The control panel <NUM> directs the overall functionality of the gunshot detection system <NUM> by sending instructions (e.g. control messages) to be executed by the gunshot sensor units <NUM>, receiving the event data from the gunshot sensor units <NUM> and taking the responsive actions based on the event data. The control panel <NUM> might receive preliminary event data (e.g. metadata indicating time and date information) from multiple gunshot sensor units <NUM> and perform a de-confliction process in order to determine which event data from the different sensor units <NUM> pertains to the same detected acoustic anomaly and which of the gunshot sensor units <NUM> is closest to the source of the acoustic anomaly based on, for example, which of the units first detected the acoustic anomaly. The control panel <NUM> might then send instructions to the gunshot sensor unit <NUM> closest to the source to send full event data (e.g. including a full audio data sample, environmental data, and other sensor data) to the control panel <NUM> for further processing and/or to be presented to the operator.

In addition, according to an enhanced anomaly detection and identification functionality of the gunshot detection system <NUM>, the gunshot sensor units <NUM> capture ambient noise (e.g. sounds, in aggregate, emitted by people, objects, animals, devices, natural/environmental forces and, in general, any other source of sounds that are within range of the gunshot sensor units <NUM>) and generate audio data depicting the captured ambient noise. In general, the gunshot detection system <NUM> continually monitors the captured ambient noise throughout premises <NUM>, learns the premises' <NUM> auditory patterns (e.g. during a learning mode), and alerts users when sounds are found to be unusual or generally indicative of a dangerous situation.

<FIG> is a block diagram showing an exemplary gunshot sensor unit <NUM>.

The gunshot sensor unit <NUM> includes a controller <NUM>, local nonvolatile storage <NUM>, a wired and/or wireless network interface <NUM>, an anomaly detection microphone <NUM>, and an audio capture microphone <NUM>.

The wired and/or wireless network interface <NUM> provides connectivity with the gunshot detection system control panel <NUM> and possibly other devices via the gunshot detection communication network <NUM>. In addition, the network also provides power to the devices, in many examples. Direct current (DC) is superimposed upon the data that is transmitted between the devices and other nodes on the network.

In general, the controller <NUM> executes firmware/operating system instructions and generally directs the functionality of the gunshot sensor unit <NUM>. In one example, the controller <NUM> is small single-board computer. In other examples, the controller is a microcontroller unit or a system on a chip (SoC), including one or more processor cores along with memory and programmable input/output peripherals such as analog to digital converts and digital to analog converters.

According to the baseline gunshot detection functionality of the gunshot detection system <NUM>, the anomaly detection microphone <NUM> detects certain acoustic anomalies, and the audio capture microphone <NUM> captures the acoustic anomalies and generates the audio data depicting the acoustic anomalies. In one embodiment, both microphones <NUM>, <NUM> are micro electro-mechanical system (MEMS) microphones having different sensitivity levels, and the controller <NUM> is configured to sample the microphones <NUM>, <NUM> such that outputs from the microphones can be continuously analyzed in near real time for an acoustic signature. The anomaly detection microphone <NUM> has the lower sensitivity level and a high clipping level, while the audio capture microphone <NUM> has the higher sensitivity level. The audio capture microphone <NUM> continuously captures ambient sound, which is stored in a <NUM> second (for example) loop in a ring buffer of the controller <NUM>. At the same time, incoming acoustic signals from the anomaly detection microphone <NUM> are continuously analyzed to detect acoustic anomalies, particularly by searching the incoming acoustic signal for a peak amplitude level large enough to be at least preliminarily identified as a gunshot.

Once an indication of a possible gunshot has been triggered utilizing the anomaly detection microphone <NUM>, further processing may be performed by the controller <NUM>. The controller <NUM> analyzes the sound stored in the loop to confirm that the acoustic anomaly is a gunshot. If confirmed as gunshot, the controller stores the captured sound stored in the loop buffer, which would include the acoustic anomaly and the previously captured sound (up to <NUM> seconds, in this example) as audio data <NUM> in the local nonvolatile storage <NUM> associated with different event files <NUM> or instances of event data for different gunshot detection events, along with the metadata <NUM>, which includes the time and/or date information for the events. In embodiments, the local nonvolatile storage <NUM> could be fixed storage such as flash memory, or removable storage such as an SD card, among other examples.

According to the enhanced anomaly detection/identification functionality of the gunshot detection system <NUM>, the audio capture microphone <NUM> also captures ambient noise and generates audio data depicting the ambient noise. The ambient noise might be captured during normal conditions (e.g. during the learning mode, during times when it is known that no dangerous situations are present, during times when the areas <NUM> where the gunshot sensor units <NUM> are located are being used in typical ways by occupants of the premises <NUM>). The ambient noise might also be captured on a continual basis in order to detect and/or identify acoustic anomalies within the ambient noise. For this purpose, the gunshot sensor units <NUM> might generate current audio data depicting the captured ambient noise in real time, possibly including acoustic anomalies. The current audio data might be an ongoing series of audio clips of different predetermined lengths and might be processed (e.g. by the controller <NUM>) to recognize and remove speech sounds to provide privacy to the occupants of the premises <NUM>.

In the illustrated embodiment, the controller <NUM> is configured to stream audio data to the control panel <NUM> in response to instructions from the control panel <NUM>, for example, upon initiation of the learning mode or upon detection of an acoustic anomaly by the anomaly detection microphone <NUM>. The controller <NUM> is also configured to preserve or delete stored, current and/or buffered audio data in response to instructions from the control panel <NUM>.

<FIG> is a block diagram showing an exemplary control panel <NUM>.

The control panel <NUM> includes a central processing unit (CPU) <NUM>, nonvolatile memory <NUM>, a wired and/or wireless network interface <NUM>, a display <NUM>, and speakers <NUM>.

Similar to analogous components on the gunshot sensor units <NUM>, the wired and/or wireless network interface <NUM> provides connectivity with the gunshot sensor units <NUM> and possibly other devices via the communication network <NUM>. In some examples, the control panel may also supply power to the units.

The speakers <NUM> provide audio playback of audio data from the gunshot sensor units <NUM>.

The CPU <NUM> executes firmware instructions and an operating system (OS) <NUM> and generally directs the functionality of the control panel <NUM>. The OS <NUM> interfaces with the hardware components of the control panel <NUM> for access by a command and control application <NUM>, which is a software process executing on top of the OS <NUM>.

The command and control application <NUM>, in general, controls the gunshot sensor units <NUM>, maintains the gunshot event database <NUM>, and provides a graphical user interface (GUI) <NUM> that is rendered on the display <NUM> (e.g. touchscreen display) of the control panel <NUM>. In one example, the GUI <NUM> might present gunshot sensor unit information to an operator of the control panel <NUM> and receive input indicating selections of various options for controlling the gunshot sensor units <NUM> such as presenting the audio data (e.g. via the speakers <NUM>), specifying various configuration settings and/or preserving or deleting locally stored audio data.

In the illustrated embodiment, the control panel <NUM> receives the streamed audio data from the gunshot sensor units <NUM> and processes the audio data to detect and/or identify the acoustic anomalies.

Thus, an audio analytics module <NUM> also executes on the CPU <NUM>. In general, the audio analytics module <NUM> might be part of the command and control application <NUM> or perform operations in conjunction and/or communication with the command and control application <NUM>.

The audio analytics module <NUM> generates ambient noise models <NUM> based on audio data generated by the gunshot sensor units <NUM> depicting the ambient noise captured by the audio capture microphones <NUM> of the units during normal conditions. These ambient noise models <NUM> could be generated and defined, as described, during a finite period wherein the system is placed in a learning mode. On the other hand, the ambient noise models <NUM> might also be initially generated and then continuously updated during normal operation of the gunshot detection system <NUM> after the initial learning mode period or even without a learning mode period entirely.

The ambient noise models <NUM>, in general, are representations of the ambient noise captured by the audio capture microphones <NUM> of the gunshot sensor units <NUM> during the normal conditions. Each gunshot sensor unit <NUM> has its own ambient noise model <NUM> representing the typical ambient noise at the particular gunshot sensor unit's <NUM> location. The ambient noise models <NUM> are further preferably generated based on variable factors such as current time of day, day of week, time of year, semester, month and/or season when the ambient noise was captured, known schedule information for the premises <NUM> and/or areas <NUM> of the premises indicating times when the ambient noise would be known to include more or less sounds, among other examples. The gunshot sensor units <NUM> could have multiple ambient noise models <NUM> pertaining to the different variable factors. During the generation process, the ambient noise models <NUM> could be configured to ensure that normal speech sounds are not detected as acoustic anomalies, for example, by including additional layers representing the normal speech sounds independent of whether these normal speech sounds were captured in the ambient noise. The ambient noise models <NUM> might include or be generated based on one or more condensed digital summaries (e.g. acoustic fingerprints) of the captured ambient noise, spectrograms for an audio signal for the ambient noise (e.g. generated via a spectrum analyzer), quantitative representations of properties of isolated sounds and/or the ambient noise in aggregate, including frequency, duration, loudness, timbre, sonic texture (e.g. number of different sources of sound and the interaction between them), and/or spatial location, among other examples.

The audio analytics module <NUM> receives the current audio data (e.g. captured in real time by the audio capture microphones <NUM> of the gunshot sensor units <NUM>) detects acoustic anomalies based on the current audio data by processing the current audio data against the ambient noise models <NUM> for the gunshot sensor units <NUM>. In general, the processing to detect the acoustic anomalies might include generating a current model representing the current ambient noise based on the current audio data using the same process for generating the ambient noise models <NUM>. The audio analytics module <NUM> would then compare the current model with the ambient noise model <NUM> in order to generate a deviation value representing a difference between the ambient noise during the normal conditions and the current ambient noise. In this example, acoustic anomalies could be detected based on whether the deviation value is above a predetermined threshold.

Additionally, the audio analytics module <NUM> identifies the acoustic anomalies (e.g. infers a source of the acoustic anomaly and/or classifies the acoustic anomaly as possibly associated with one or more known dangerous situations) by processing the current audio data against danger indicators <NUM>.

The danger indicators <NUM> are preconfigured representations of acoustic patterns associated with and/or indicative of dangerous situations. Each danger indicator might include identification information for the situation (e.g. a unique identifier that can be interpreted by the gunshot detection system <NUM>), descriptive text for the situation (e.g. a text string that can be interpreted by the users, which is included in the event data and/or alert messages sent to the users), and acoustic information for one or more acoustic patterns associated with the dangerous situation. For example, the acoustic patterns for the different danger indicators might be acoustic signatures of target sounds such as those associated with gunshots, screaming, breaking glass, or any unusually loud sounds, and/or unusually low amplitude sounds (e.g. indicating obstruction of the microphones <NUM>, <NUM> of the gunshot sensor units <NUM>).

In general, the audio analytics module <NUM> identifies the acoustic anomalies by recognizing acoustic patterns indicative of danger within the current ambient noise depicted by the current audio data. This might include isolating audio signals for discrete sounds (e.g. the acoustic anomalies) within the current ambient noise and determining whether properties of the isolated sounds match the acoustic signatures for one or more of the danger indicators. In one example, the acoustic information of the danger indicators would include danger models generated using the same processes for generating the ambient noise models <NUM> and the current noise models (e.g. based on a representative set of sample audio clips depicting the sounds indicative of the dangerous situations), and these danger models would be compared to similar models for the isolated sounds. In another example, the acoustic information could be different combinations of criteria for isolating within the current ambient noise the acoustic patterns associated with the danger indicators <NUM>.

In one embodiment, the audio analytics module <NUM> might employ machine learning mechanisms and/or a neural network architecture. Generally, neural networks include processes organized into virtual nodes with interconnected inputs and outputs loosely based on neurons of the human brain. Numerical data corresponding to real-world phenomena propagate through the nodes based on different properties (e.g. functions performed by the nodes, variously weighted connections between nodes), producing an output which may or may not represent a meaningful result. The output might be evaluated against a target outcome and/or other information known to the networks, and the properties of the nodes and connections increasingly adjusted to make targeted and/or meaningful outcomes increasingly frequent. In this way, neural networks learn to process complex data sets and are used, for example, to recognize patterns and/or objects based on similarities and even classify objects based on known (e.g. labeled) data sets.

The audio analytics module <NUM> could include a neural network configured to detect and classify acoustic anomalies in the current audio data captured by the gunshot sensor units <NUM> based on recognized acoustic patterns in the audio data and a relationship between the patterns and reference data labeled to indicate a context for the data such as normal conditions and/or dangerous situations. For example, during a learning process, the audio analytics module <NUM> is provided with the labeled audio data and breaks the data into frames. An acoustic fingerprint could then be generated for each frame, the fingerprint including information such as energy levels for different frequency bands. Fourier transform methods could be used to further divide the frames into discrete sound waves with further values representing the energy of the sound waves. This numerical data is then analyzed by the audio analytics module <NUM>, and meaningful associations are determined between, for example, recognized patterns in the audio data and whatever information is already known to the audio analytics module <NUM>. Given an audio data sample of sufficient length depicting the ambient noise for a gunshot sensor unit <NUM> during normal conditions, the audio analytics module <NUM> might recognize patterns associated with discrete sound events and generate an ambient noise model <NUM> indicating acoustic properties of the sound events that occurred most frequently (or with a frequency above a predetermined threshold). In subsequently processed data (e.g. current audio data captured to monitor for anomalies), recognized sound events conforming to the ambient noise model <NUM> could be ignored, while those that deviate from the ambient noise model <NUM> might be flagged as detected acoustic anomalies. In another example, given a sufficiently large set of discrete audio samples depicting examples of sound events of known types (e.g. different examples of screams, explosions or gunshots), the audio analytics module <NUM> might recognize acoustic patterns in the audio data and build a learned association between the recognized acoustic patterns and the type of sounds based on which patterns occurred most frequently across the set of samples for each type. Subsequently processed audio data with recognized acoustic patterns matching those frequently recognized in the sample data could be classified based on the matching type and meaningful information about the nature of the detected acoustic anomaly can be provided, for example to the operator <NUM> of the control panel <NUM>. The ambient noise models <NUM> for each of the gunshot sensor units <NUM> and the danger indicators <NUM> are stored in the nonvolatile memory <NUM> along with a gunshot event database <NUM>.

The gunshot event database <NUM> stores information about the gunshot detection system <NUM>. This information includes information about the devices of the gunshot detection system <NUM> including identification information and location information for each of the gunshot sensor units <NUM>. The gunshot event database <NUM> also stores event data, audio data, any generated audio, event and/or environmental data analytics information, such as forensic data generated as a result of processing the data received from the gunshot sensor units <NUM>, or order information indicating the order in which the gunshot sensor units <NUM> detected the gunshots. In the illustrated example, the gunshot event database <NUM> is stored in the nonvolatile memory <NUM> of the control panel <NUM>. However, in other examples, the gunshot event database <NUM> might be part of a cloud management system and/or connected services system, which is accessible only over public networks, or even on the local nonvolatile storage <NUM> of the gunshot sensor units <NUM> themselves. In general, the audio analytics module.

Based on the acoustic anomalies detected and/or identified by the audio analytics module <NUM>, the command and control application <NUM> generates alerts notifying users of the acoustic anomalies. These alerts might include information about the location of the acoustic anomalies based on information retrieved from the database <NUM> and/or the descriptive text for one or more matching danger indicators <NUM> identifying the acoustic anomalies.

<FIG> is a sequence diagram illustrating a process by which the control panel <NUM> generates the ambient noise models <NUM>, detects and identifies acoustic anomalies, and generates alerts, based on audio data streamed from the gunshot sensor units <NUM>.

First, in step <NUM>, the control panel <NUM> sends instructions to the gunshot sensor units <NUM> to initiate the learning mode in which the gunshot detection system <NUM> learns the auditory patterns of the premises <NUM>. These instructions might also indicate that the gunshot sensor units <NUM> should stream captured audio data to the control panel <NUM>.

In step <NUM>, the gunshot sensor units capture the ambient noise during the normal conditions of the premises <NUM>, generate audio data depicting the ambient noise, and stream the audio data to the control panel <NUM>.

In step <NUM>, the control panel <NUM> generates the ambient noise model <NUM> for each gunshot sensor unit <NUM> based on the streamed audio data from the gunshot sensor units <NUM> and based on other variable factors such as current time of day, day of week, time of year, semester, month and/or season when the ambient noise was captured, known schedule information for the premises <NUM> and/or areas <NUM> of the premises <NUM> stored in the database <NUM> indicating times when the ambient noise would be known to include more or less sounds, among other examples. The ambient noise models <NUM> are stored in the nonvolatile memory <NUM>, for example, associated with identification information for each of the gunshot sensor units <NUM>.

In step <NUM>, the gunshot sensor unit <NUM>-<NUM> detects a potential acoustic anomaly (e.g. via the anomaly detection microphone <NUM> and begins recording audio data depicting the acoustic anomaly. The gunshot sensor units <NUM>-<NUM> also alerts the control panel <NUM> of the potential anomaly and sends event data generated based on the potential anomaly to the control panel <NUM> in step <NUM>.

In step <NUM>, the control panel <NUM> sends instructions to the gunshot sensor unit <NUM>-<NUM> to stream the recorded audio data depicting the acoustic anomaly and current audio data depicting ambient noise captured in real time. The gunshot sensor unit <NUM>-<NUM> streams the audio data to the control panel <NUM> in step <NUM>.

In one example, the control panel <NUM> is a a hybrid fire alarm system control panel and gunshot detection system control panel and might also instruct and receive streaming audio data from fire notification devices equipped with microphones for self-testing purposes. During normal operation, these microphones of the fire notification devices might be used to capture audio data depicting the sounds emitted by the fire notification devices in order to test that they are properly functioning. However, the hybrid control panel, or even the gunshot detection control panel <NUM> with connectivity with a fire alarm control panel, might send instructions to the fire notification devices to capture ambient noise and stream audio data depicting ambient noise to the control panel <NUM>. The control panel <NUM> would generate an ambient noise model for the fire notification device. Against this ambient noise model, the control panel <NUM> would analyze ambient noise captured by the fire notification devices for acoustic anomalies.

In step <NUM>, the control panel <NUM>, via the audio analytics module <NUM>, detects and/or identifies acoustic anomalies depicted in the streamed audio data based on the ambient noise model <NUM> for the gunshot sensor unit <NUM>-<NUM> and based on the danger indicators <NUM>. Based on the detected/identified acoustic anomalies, in step <NUM>, the control panel <NUM> generates and sends alerts to users such as an operator <NUM> of the control panel <NUM>, a security guard station, or a cloud monitoring service, among other examples. The alerts might simply be presented to the operator via the GUI <NUM>, or they could be messages sent to the users via email, text, and/or push notifications, among other examples.

In step <NUM>, the control panel <NUM> also sends instructions to the gunshot sensor unit <NUM>-<NUM> to preserve or delete the recorded audio data based on whether any acoustic anomalies were detected and/or identified in the streamed audio data (e.g. based on the results of step <NUM>).

<FIG> is a flow diagram illustrating the process by which the control panel <NUM> detects and/or identifies acoustic anomalies in audio data streamed from the gunshot sensor units <NUM> based on the ambient noise model <NUM> for the gunshot sensor unit <NUM> and based on the danger indicators <NUM>. This process generally corresponds to the steps <NUM> through <NUM>, which were previously described with respect to <FIG>. Now, however, more detail is provided.

First, in step <NUM>, the audio analytics module <NUM> executing on the control panel <NUM> receives the current audio data streamed from the gunshot sensor unit <NUM> and generates a deviation value based on a comparison between the current audio data stream and ambient noise model <NUM> for the gunshot sensor unit <NUM>. The deviation value represents the difference between the ambient noise during the normal conditions, as represented by the ambient noise model <NUM>, and the currently captured ambient noise, as represented by the current audio data and/or any models generated based on the current audio data.

In step <NUM>, the audio analytics module <NUM> determines whether the deviation value is above a predetermined threshold. If not, in step <NUM>, the control panel <NUM> sends instructions to the gunshot sensor unit <NUM> to delete the current audio data and/or to stop recording the audio data in real time.

However, if the deviation value is above the predetermined threshold, in step <NUM>, the audio analytics module <NUM> determines whether characteristics of the current audio data match the acoustic patterns of any of the preconfigured danger indicators <NUM> such as those representing screaming, gunshots, unusual loud noises, or unusually low amplitude indicating obstruction of the microphones <NUM>, <NUM>, among other examples. If so, in step <NUM>, information about the matching danger indicators are added to the event data (which, for, example, could be stored in the database <NUM> and/or provided with alert messages). In one example, this added information might include the descriptive text for the danger indicators <NUM> such as "broken glass" or "screams".

Whether or not any matching danger indicators <NUM> were found in the current audio data, in step <NUM>, the control panel <NUM> generates and sends the alerts to users such as the operator <NUM>. The alerts might include the event data and information about the recognized danger indicators, as previously described.

In step <NUM>, the control panel <NUM> also sends instructions to the gunshot sensor unit <NUM> to preserve the recorded audio data associated with the acoustic anomaly, for example, by storing the audio data in the local nonvolatile storage <NUM> and/or sending the audio data to be stored by the control panel <NUM>.

<FIG> is a block diagram of an exemplary gunshot sensor unit <NUM> according to an embodiment in which the gunshot sensor units <NUM> generate their own ambient noise models <NUM> and autonomously detect and identify the acoustic anomalies based on the ambient noise models <NUM>. The gunshot sensor unit <NUM> is largely the same as the unit illustrated in and described with respect to <FIG>. Now, however, the audio analytics module <NUM> executes on the controller <NUM> of the gunshot sensor unit <NUM>. Additionally, the ambient noise model <NUM> and the danger indicators <NUM> are stored in the local nonvolatile memory <NUM>.

<FIG> is a flow diagram illustrating a process by which the gunshot sensor units <NUM> generate the ambient noise models <NUM> and detect and/or identify acoustic anomalies based on the ambient noise model <NUM> and the danger indicators <NUM>.

First, in step <NUM>, the captures via the audio capture microphone <NUM> ambient noise during normal conditions (e.g. during the learning mode) and generates audio data depicting the ambient noise during normal conditions.

In step <NUM>, the audio analytics module <NUM> executing on the gunshot sensor unit <NUM> generates the ambient noise model <NUM> based on the audio data depicting the ambient noise during normal conditions and stores the ambient noise model <NUM> for the gunshot sensor unit <NUM> in local nonvolatile storage <NUM>.

In step <NUM>, on a continuous basis, the gunshot sensor unit <NUM> captures ambient noise via the audio capture microphone <NUM> and generates current audio data depicting the ambient noise captured in real time.

In step <NUM>, the audio analytics module <NUM> generates the deviation value based on a comparison of the current audio data and the ambient noise model <NUM> for the gunshot sensor unit <NUM>. As previously described, the deviation value represents differences between the current ambient noise and the ambient noise captured during normal conditions.

In step <NUM>, the audio analytics module <NUM> determines whether the deviation value is above the predetermined threshold. If not, in step <NUM>, the gunshot sensor unit <NUM> deletes the current audio data and returns to step <NUM>, as the process of monitoring the ambient noise repeats indefinitely.

However, if the deviation value was above the predetermined threshold, in step <NUM>, the audio analytics module <NUM> determines whether the characteristics of the current ambient noise depicted in the current audio data match the acoustic patterns of any of the predetermined danger indicators <NUM>. If so, in step <NUM>, the gunshot sensor unit <NUM> adds the information about the matching danger indicators (e.g. the descriptive text) to the event data, which might be stored in the local nonvolatile storage <NUM> and/or send to the control panel <NUM>.

In either case, in step <NUM>, the gunshot sensor unit <NUM> generates and sends an alert message indicating that an acoustic anomaly was detected/identified. The alert message might include the audio data depicting the acoustic anomaly and/or the information for the matching danger indicators <NUM> that might have been added to the event data in step <NUM>. The alert message might be sent to the control panel <NUM> to be further processed by the control panel <NUM> (including sending alerts on to the users), or the alert could be sent directly to the users.

The process then returns to step <NUM>, as the ambient noise is continuously monitored by the gunshot sensor unit <NUM>.

Additionally, it should be noted that the gunshot detection system <NUM> could be configured to perform the previously described processes for the baseline gunshot detection (e. g using the anomaly detection microphone <NUM>) and the enhanced anomaly detection/identification (e.g. using the ambient noise models <NUM> and the audio analytics module <NUM>) in a number of different ways. For example, the enhanced functionality might be used to expand the system's detection capability by detecting and identifying anomalies that were not detected via the baseline gunshot detection functionality. The gunshot detection system <NUM> could also be configured to perform both the enhanced and the baseline detection processes in parallel. In another example, the ambient noise models could be used to confirm and/or identify potential acoustic anomalies that were initially detected via the baseline gunshot detection functionality. In yet another example, the gunshot detection system <NUM> could be configured to perform only the enhanced detection process using the ambient noise models <NUM>.

Claim 1:
A system for detecting gunshots, the system comprising:
gunshot sensor units (<NUM>) for detecting gunshots, each of the gunshot sensor units (<NUM>) comprising one or more microphones (<NUM>) for generating audio data depicting ambient noise; and
ambient noise models for the gunshot sensor units (<NUM>) that represent ambient noise captured by the gunshot sensor units (<NUM>) during normal conditions,
wherein each gunshot sensor unit (<NUM>) has a respective ambient noise model representing the typical ambient noise at the location of the gunshot unit, and
wherein the system is configured to detect acoustic anomalies by processing current audio data against the ambient noise models.