Patent Description:
An unmanned vehicle, which may also be referred to as an autonomous vehicle, is a vehicle capable of travel without a physically-present human operator. An unmanned vehicle may operate in a remote-control mode, in an autonomous mode, or in a partially autonomous mode.

When an unmanned vehicle operates in a remote-control mode, a pilot or driver that is at a remote location can control the unmanned vehicle via commands that are sent to the unmanned vehicle via a wireless link. When the unmanned vehicle operates in autonomous mode, the unmanned vehicle typically moves based on pre-programmed navigation waypoints, dynamic automation systems, or a combination of these. Further, some unmanned vehicles can operate in both a remote-control mode and an autonomous mode, and in some instances may do so simultaneously. For instance, a remote pilot or driver may wish to leave navigation to an autonomous system while manually performing another task, such as operating a mechanical system for picking up objects, as an example.

Various types of unmanned vehicles exist for various different environments. For instance, unmanned vehicles exist for operation in the air, on the ground, underwater, and in space. Propulsion of unmanned vehicles may occur using, for example, electric motors. Electric motors may also be used in connection with payload handling by the unmanned vehicles. Designs that improve reliability of unmanned vehicle propulsion and/or payload handling will expand their mission capabilities.

<CIT> discloses a device and a method for the early detection and prediction of damage to assemblies in machine plants, in particular, mobile machine plants. For this purpose, the structure-borne sound of the machine system is sensed by a sensor and output as an acceleration signal and analyzed in a digital signal processor. In this context, in order to avoid negative influences of ambient vibrations or structure-borne sounds which are not associated with the state of the machine plant, the acceleration signal is firstly transformed into the frequency domain by means of a fast-Fourier transformation, and the data obtained in this way is then transformed again into the time domain by means of cepstrum analysis so that resonance data relating to individual shock pulses (a cepstrum) is obtained in the time domain. This cepstrum is then compared with a comparison cepstrum which is available in accordance with load signals and rotational speed signals for the present operating state in a new machine plant in a storage device. When limiting values are exceeded, the diagnostic signal, in particular information relating to the assembly which is diagnosed as damaged and its predicted remaining service life, are displayed for the user and an emergency operating mode is initiated.

<CIT> discloses a method of monitoring electric motors, so that when an electric motor develops an abnormality, the abnormality is discovered immediately and a maintenance engineer is promptly instructed to repair or replace the electric motor. To this end, vibration sensors are mounted on the electric motors. A fast Fourier transform (FFT) is used to convert the vibration signals; then, using the vibration signals in the frequency domain, a possible type of electric motor downgrade can be conjectured, and the remaining useful life thereof can be predicted.

<CIT> discloses a multi-rotor vehicle system comprising an inertial sensor coupled to a DC-powered motor and a flight controller which implements a health monitor system that detects a failure or impending failure based on at least a reading of the sensor to detect abnormal vibration as a precursor to an impending failure to at least one of the rotor/propeller/ motor.

In order to address the shortcomings of the prior art, the current invention provides a method according to claim <NUM>, and an apparatus according to claim <NUM>. Advantageous embodiments are provided in the dependent claims.

Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified. Not all instances of an element are necessarily labeled so as not to clutter the drawings where appropriate. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles being described.

Embodiments of a system, apparatus, and method for automatic detection of operational states of electric motors included in unmanned aerial vehicles are described herein. In the following description numerous specific details are set forth to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

Embodiments described herein include an unmanned aerial vehicle (UAV), which refers to any autonomous or semi-autonomous vehicle that is capable of performing some functions without a physically present human pilot. A UAV can take various forms. For example, a UAV may take the form of a fixed-wing aircraft, a glider aircraft, a tail-sitter aircraft, a jet aircraft, a ducted fan aircraft, a lighter-than-air dirigible such as a blimp or steerable balloon, a rotorcraft such as a helicopter or multicopter, and/or an ornithopter, among other possibilities. Further, the terms "drone," "unmanned aerial vehicle system" (UAVS), or "unmanned aerial system" (UAS) may also be used to refer to a UAV.

<FIG> and <FIG> illustrate an aerial vehicle or UAV <NUM>, in accordance with an embodiment of the present disclosure. The illustrated embodiment of UAV <NUM> is a vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) that includes separate propulsion units <NUM> and <NUM> for providing horizontal and vertical propulsion, respectively. UAV <NUM> is a fixed-wing aerial vehicle, which as the name implies, has a wing assembly <NUM> that can generate lift based on the wing shape and the vehicle's forward airspeed when propelled horizontally by propulsion units <NUM>. <FIG> is a perspective top view illustration of UAV <NUM> while <FIG> is a bottom side plan view illustration of UAV <NUM>.

The illustrated embodiment of UAV <NUM> includes a fuselage <NUM>. In one embodiment, fuselage <NUM> is modular and includes a battery module, an avionics module, and a mission payload module. These modules are detachable from each other and mechanically securable to each other to contiguously form at least a portion of the fuselage or UAV main body.

The battery module includes a cavity for housing one or more batteries for powering UAV <NUM>. The avionics module houses flight control circuitry of UAV <NUM>, which may include a processor and memory, communication electronics and antennas (e.g., cellular transceiver, Wi-Fi transceiver, etc.), and various sensors (e.g., global positioning sensor, an inertial measurement unit (IMU), a magnetic compass, etc.). The mission payload module houses equipment associated with a mission of UAV <NUM>. For example, the mission payload module may include a payload actuator for holding and releasing an externally attached payload. In another embodiment, the mission payload module may include a camera/sensor equipment holder for carrying camera/sensor equipment (e.g., camera, lenses, radar, LIDAR, pollution monitoring sensors, weather monitoring sensors, etc.).

The illustrated embodiment of UAV <NUM> further includes horizontal propulsion units <NUM> positioned on wing assembly <NUM>, which can each include a motor, shaft, motor mount, and propeller, for propelling UAV <NUM>. The illustrated embodiment of UAV <NUM> includes two boom assemblies <NUM> that secure to wing assembly <NUM>. In one embodiment, wing assembly <NUM> includes a wing spar <NUM> (see <FIG>) disposed within a wing foil of wing assembly <NUM>. Wing spar <NUM> may be a hollow structural member (e.g., tubular rod) extending along the internal length of the wing foil and provides a main structural member that connects wing assembly <NUM> to fuselage <NUM> and to which boom assemblies <NUM> mount.

The illustrated embodiments of boom assemblies <NUM> each include a boom housing <NUM> in which a boom <NUM> (see <FIG>) is disposed, vertical propulsion units <NUM>, printed circuit boards <NUM>, and stabilizers <NUM>. In the illustrated embodiment, booms <NUM> are also hollow structural members (e.g., tubular rods) that provide the main structural support to which wing spar <NUM> and vertical propulsion units <NUM> are mounted. Booms <NUM> are also referred to as "boom carriers" since they carry the load forces on boom assemblies <NUM>. Vertical propulsion units <NUM> can each include a motor, shaft, motor mounts, and propeller, for providing vertical propulsion. Vertical propulsion units <NUM> may be used during a hover mode where UAV <NUM> is descending (e.g., to a delivery location) or ascending (e.g., following a delivery). Stabilizers <NUM> (or fins) may be included with UAV <NUM> to stabilize the UAV's yaw (left or right turns) during flight. In some embodiments, UAV <NUM> may be configured to function as a glider. To do so, UAV <NUM> may power off its propulsion units and glide for a period of time.

During flight, UAV <NUM> may control the direction and/or speed of its movement by controlling its pitch, roll, yaw, and/or altitude. For example, the stabilizers <NUM> may include one or more rudders 108a for controlling the UAV's yaw, and wing assembly <NUM> may include elevators for controlling the UAV's pitch and/or ailerons 102a for controlling the UAV's roll. As another example, increasing or decreasing the speed of all the propellers simultaneously can result in UAV <NUM> increasing or decreasing its altitude, respectively.

Many variations on the illustrated fixed-wing aerial vehicle are possible. For instance, aerial vehicles with more wings (e.g., an "x-wing" configuration with four wings), are also possible. Although <FIG> and <FIG> illustrate one wing assembly <NUM>, two boom assemblies <NUM>, two horizontal propulsion units <NUM>, and six vertical propulsion units <NUM> per boom assembly <NUM>, it should be appreciated that other variants of UAV <NUM> may be implemented with more or less of these components.

It should be understood that references herein to an "unmanned" aerial vehicle or UAV can apply equally to autonomous and semi-autonomous aerial vehicles. In a fully autonomous implementation, all functionality of the aerial vehicle is automated; e.g., pre-programmed or controlled via real-time computer functionality that responds to input from various sensors and/or pre-determined information. In a semi-autonomous implementation, some functions of an aerial vehicle may be controlled by a human operator, while other functions are carried out autonomously. Further, in some embodiments, a UAV may be configured to allow a remote operator to take over functions that can otherwise be controlled autonomously by the UAV. Yet further, a given type of function may be controlled remotely at one level of abstraction and performed autonomously at another level of abstraction. For example, a remote operator may control high level navigation decisions for a UAV, such as specifying that the UAV should travel from one location to another (e.g., from a warehouse in a suburban area to a delivery address in a nearby city), while the UAV's navigation system autonomously controls more fine-grained navigation decisions, such as the specific route to take between the two locations, specific flight controls to achieve the route and avoid obstacles while navigating the route, and so on.

<FIG> is a perspective view illustration of a structural airframe <NUM> (also referred to as a "structural H-frame" or an "H-frame") of UAV <NUM>, in accordance with an embodiment of the present disclosure. H-frame <NUM> includes wing spar <NUM> and booms <NUM>. In some embodiments wing spar <NUM> and booms <NUM> may be made of carbon fiber, hard plastic, aluminum, light metal alloys, or otherwise. Wing spar <NUM> and booms <NUM> are mechanically connected with spar-boom joiners <NUM>. Spar-boom joiners <NUM> are mechanical joiners that clamp wing spar <NUM> to booms <NUM> with a "frangible" design. Wing spar <NUM> may include pre-drilled holes <NUM> for mounting horizontal propulsion units <NUM>, and boom carriers <NUM> may include pre-drilled holes (hidden by motor mounts <NUM>) for mounting vertical propulsion units <NUM>. In the illustrated embodiment, fuselage <NUM> is removably attached to the H-frame with a spar joiner <NUM> that clamps onto wing spar <NUM>.

Mechanical joiners <NUM> and/or <NUM> may be frangible structures designed to break apart to decouple the kinetic energy between linked structures in the event of a catastrophic impact or crash landing. This controlled failure mode improves safety and reduces property damage in the event of crash landings of UAV <NUM> by diverting impact energy way from booms <NUM> and/or wing spar <NUM>.

In an embodiment, fuselage <NUM> comprises a modular fuselage including a battery module <NUM> for housing a battery to power a UAV, a mission payload module <NUM> that houses equipment associated with a mission of the UAV, and an avionics module <NUM> for housing flight control circuitry of the UAV. Battery module <NUM>, mission payload module <NUM>, and avionics module <NUM> are shaped to secure to each other to form a contiguous and operational fuselage separate from being mechanically secured to wing assembly <NUM> or boom assemblies <NUM>. This enables modular fuselage <NUM> to be assembled and operationally tested in isolation to the aerodynamic structures and systems on wing assembly <NUM> and boom assemblies <NUM>. The modularity further enables the easy replacement of worn out or damaged modules, swapping modules (e.g., mission payload module) for a given UAV mission or flight, or updating particular modules without having to replace the entire UAV.

<FIG> is a perspective view illustration of a motor <NUM> of UAV <NUM>, in accordance with an embodiment of the present disclosure. <FIG> is a partial, exploded perspective view illustration of the motor <NUM>, in accordance with an embodiment of the present disclosure. Motor <NUM> can be a motor included in the propulsion unit <NUM>, propulsion unit <NUM>, mission payload module <NUM>, and/or the like of UAV <NUM>. Motor <NUM> can comprise an electric motor, a brushless motor, a direct current (DC) motor, an alternating current (AC) motor, a brushed motor, or the like. Motors included in the UAV <NUM> can be the same or different from each other.

In an embodiment, a shaft <NUM> extends both above and below a first side (e.g., the top side) of the motor <NUM>. A first portion of the shaft <NUM> above the first side is configured to physically couple with a propeller or other structure of UAV <NUM> to which mechanical power is to be applied or provided. A second portion of the shaft <NUM>, opposite the first portion, is located below the first side within a rotor <NUM>. A second side opposite the first side (e.g., the bottom side) of the motor <NUM> includes a base <NUM>. Motor <NUM> further includes the rotor <NUM> and a stator <NUM>. Rotor <NUM> is configured to be circumferential with, supported by, and cause to be rotated by the stator <NUM>.

As shown in <FIG>, the second portion of the shaft <NUM> extends from a center of the rotor <NUM> and is configured to insert into a corresponding shaft receiving space <NUM> of the stator <NUM>. Disposed between the rotor <NUM> and stator <NUM> are a plurality of bearings (not shown) located on a surface <NUM> of the stator <NUM>. The inner sides of the rotor <NUM> (e.g., sides parallel to the axis of the shaft <NUM>) include a plurality of magnets <NUM>. The perimeter of the stator <NUM> (e.g., the sides parallel to the axis of the shaft <NUM>) includes a plurality of copper windings <NUM>. The plurality of magnets <NUM> is configured to be circumferential to and separated by a small gap from the plurality of copper windings <NUM>.

Magnetic field generated by the plurality of copper windings <NUM> of the stator <NUM> causes the plurality of magnets <NUM> of the rotor <NUM> to rotate about the plurality of copper windings <NUM>. Such rotation, in turn, rotates the rotor <NUM> overall as well as the shaft <NUM>, thereby providing mechanical power to the structure physically coupled to the first portion of the shaft <NUM>.

<FIG> illustrates an example storage facility for UAVs, in accordance with an embodiment of the present disclosure. One of skill in the art will appreciate that one or more components/features depicted in <FIG> can be omitted in accordance with various embodiments of the present disclosure.

<FIG> illustrates an enclosed storage facility <NUM> with separate ingress and egress points (e.g., windows that may open and close) on opposite sides of the structure. Also depicted are parts of a control system for the UAVs including a network <NUM>, storage <NUM>, controller or compute device <NUM> (e.g., servers in a distributed system, local computer, a combination thereof, or the like), and communication system <NUM> (e.g., radio frequency (RF) transceiver, Wi-Fi transceiver, Bluetooth, or the like). Charging pads <NUM>-<NUM> and staging area <NUM> are also depicted.

In an embodiment, charging pads <NUM> comprise unoccupied charging and/or landing pads that are available for UAVs to respectively be located. Charging pads <NUM>, <NUM>, and <NUM> comprise charging/landing pad locations at which UAVs in first, second, and third states, respectively, are located. UAVs located in the staging area <NUM> comprise UAVs designated for a particular action or activity. As an example, without limitation, UAVs in the first state comprise UAVs designated to be out of service, to be serviced, flagged for impending motor failure, and/or the like, as will be described in detail herein. UAVs in the second state comprise UAVs that are partially charged and which are undergoing battery charging (or other power replenishment) at charging pads <NUM>. UAVs in the third state comprise UAVs that are fully charged. Staging area <NUM> can comprise UAVs that are ready for deployment for their intended purpose, UAVs selected for testing, and/or the like.

One or both of audio capture devices <NUM>, <NUM> can also be included with the storage facility <NUM>. Audio capture devices <NUM>, <NUM> are configured to capture or receive sounds emanating from the UAVs entering, exiting, and/or within the storage facility <NUM>. Sounds captured by audio capture devices <NUM>, <NUM> comprise audio data that are provided to compute device <NUM> via communication system <NUM>. Each of audio capture devices <NUM>, <NUM> can comprise one or more audio capture devices located at one or more locations. Audio capture devices <NUM> and/or <NUM> can comprise part of a larger system such as a security system, audio visual system, and/or the like. Audio capture device <NUM> is located proximate the ingress point, egress point, and/or the like to capture sounds emanating from UAVs as they enter or exit the storage facility <NUM>. Audio capture device <NUM> is located in proximity to the charging pads <NUM>-<NUM> to capture sounds emanating from UAVs within the storage facility <NUM>. In some embodiments, if one of audio capture devices <NUM> or <NUM> is implemented, then the other of the audio capture devices <NUM> or <NUM> can be optional.

In the illustrated embodiment, the control system for the UAVs receives, via a receiver included in communication system <NUM>, a status update, audio data, or other information from one or more of the UAVs. The control system may calculate with controller <NUM> moving instructions for one or more of the UAVs based on the received information. The control system may then send, using communication system <NUM>, the movement instructions to the one or more UAVs, and the movement instructions include directions to move particular ones of the UAVs from a first location to a second location within storage facility <NUM>.

For example, if UAV <NUM> located within the storage facility <NUM> includes one or more audio capture devices (e.g., microphones) that captures sounds or audio emanating from onboard motors of the UAV <NUM>, the captured audio data can be provided to the control system. Movement instructions in response to captured audio data from UAV <NUM> can include, for instance, an instruction for UAV <NUM> to move to an area of the storage facility <NUM> designated for UAV servicing, UAV maintenance, UAVs designated not to be used for deliveries, and/or the like (e.g., to charging pads <NUM>).

Movement instructions may be provided to rearrange the UAVs for the reasons described above. In some embodiments, audio data is analyzed and the control system may determine whether one or more motors of particular UAVs are experiencing impending failure, operating outside of normal parameters, number of operational cycles to motor failure, and/or the like. Actual motor failure may cause the UAVs to fail, UAVs to be unduly damaged, and/or otherwise adversely impact the UAVs' mission. Thus, moving instructions can be formulated that are proactive in nature, such as designating certain UAVs to be serviced or not to be used for normal operations and causing such designated UAVs to locate to charging pads <NUM> or other particular area(s) of the storage facility <NUM>.

Although storage facility <NUM> is depicted with ingress and egress points located at different sides, it is contemplated that both the ingress and egress points can be located on the same side and/or comprise the same opening. Storage facility <NUM> can comprise an open area or a structure without a roof rather than the depicted enclosed structure. In which case ingress and egress points may be omitted.

<FIG> illustrates a block diagram of devices implemented in connection with automatic motor failure detection and analysis in accordance with an embodiment of the present disclosure. An audio capture device <NUM> is in direct communication or indirect communication, via a network <NUM>, with a compute device <NUM>.

In an embodiment, audio capture device <NUM> is configured to capture, receive, and/or sense sounds emanating from or generated by one or more motors during operation (e.g., active state) of the one or more motors. The monitored motor(s) comprise a motor such as the motor <NUM>, motor included in propulsion unit <NUM> of UAV <NUM>, motor included in propulsion unit <NUM> of UAV <NUM>, motor included in the payload module of UAV <NUM>, and/or other motors of UAV <NUM>.

Audio capture device <NUM> comprises one or more devices. Audio capture device <NUM> can comprise part of the UAV <NUM> and/or be external to the UAV <NUM>. For example, without limitation, audio capture device <NUM> can comprise audio capture device <NUM> and/or <NUM> included on the exterior of the UAV <NUM> (see <FIG>), audio capture device <NUM> and/or <NUM> associated with storage facility <NUM>, and/or the like. Audio capture device <NUM> can comprise a dedicated audio sensing device or be part of a system. For instance, audio capture device <NUM> can comprise a (dedicated) microphone, part of a security system, part of a security camera, part of an audio/visual system, part of an Internet of Things (IoT) device, and/or the like. In an embodiment, a given one of the audio capture device <NUM> is configured to capture audio from a particular one of the motor(s) of interest. In other embodiments, a given one of the audio capture device <NUM> is configured to capture audio from a plurality of motors, with audio associated with each respective motor of the plurality of motors from the total audio data collected to be identified by the compute device <NUM>.

Compute device <NUM> is configured to receive audio data from the audio capture device <NUM> and perform processing, analysis, and/or determinations associated with detection of impending motor failure, as described in detail below in connection with <FIG>. In an embodiment, motor failure detection logic is included in compute device <NUM> to perform such processing, analysis, and/or determinations. Motor failure detection logic can be implemented as software comprising one or more instructions to be executed by one or more processors included in compute device <NUM> (and/or a remotely located server/compute device if compute device <NUM> is resource constrained or processing is to be performed remotely or by a central processing unit). In alternative embodiments, motor failure detection logic (or a portion thereof) may be implemented as firmware or hardware such as, but not limited, to, an application specific integrated circuit (ASIC), programmable array logic (PAL), field programmable gate array (FPGA), and the like included in the compute device <NUM> (and/or remotely located server/compute device).

Compute device <NUM> comprises one or more devices, and is located proximate or distal from the audio capture device <NUM> from which audio data is to be received. Compute device <NUM> can comprise part of the UAV <NUM> and/or be separate from the UAV <NUM>. For example, without limitation, compute device <NUM> comprises a processor included in the avionics module of the UAV <NUM>, a processor included in the UAV <NUM>, the controller or compute device <NUM>, a central remote processor, and/or the like. Compute device <NUM> comprises one or more computers, workstations, servers, laptops, processors, smartphones, tablets, and/or the like. Compute device <NUM> can comprise a device dedicated for detection of motor failure and associated functionality, or a device that is a part of a system and/or configured to perform motor failure detection as well as other processing functions.

In an embodiment, audio capture device <NUM> and compute device <NUM> may be the same or different devices from each other. In an embodiment, each audio capture device <NUM> is associated with a particular compute device <NUM>. In an embodiment, a single compute device <NUM> is configured to receive audio data from more than one of the audio capture devices <NUM>. In other words, the number of audio capture devices <NUM> to the number of compute devices <NUM> for the UAV <NUM> can be one to one, one to many, or many to one. The number of audio capture devices <NUM> to the number of compute devices <NUM> for a plurality of UAVs, such as may be handled by the storage facility <NUM>, is likewise one to one, one to many, or many to one.

Network <NUM> comprises one or more switches, routers, firewalls, gateways, relays, repeaters, interconnects, network management controllers, servers, memory, processors, and/or other components configured to interconnect and/or facilitate communication between audio capture device <NUM> and compute device <NUM>. The network <NUM> may also be referred to as a fabric, compute fabric, or cloud.

<FIG> illustrates an example process <NUM> implemented by compute device <NUM> to perform impending motor failure detection and associated activities in connection with the detected state of one or more motors included in the UAV <NUM>, in accordance with an embodiment of the present disclosure.

At a block <NUM>, compute device <NUM> is configured to receive audio data associated with one or more motors detected by the audio capture device <NUM>. Audio data comprises audio or sound emanating from a particular motor or a particular plurality of motors during operation of the motor(s), which was detected and stored by the audio capture device <NUM>. Audio data comprises a continuous or discrete audio sample of a certain time duration. Audio data comprises data in the time domain and may include related information such as a time stamp, audio capture device identifier, motor(s) identifier, and/or the like.

As described in connection with <FIG>, in an embodiment, both the audio capture device <NUM> and compute device <NUM> can be included in the UAV <NUM> and UAV <NUM> can perform self-diagnosis regarding its motor(s). Or the audio capture device <NUM> can be included in the UAV <NUM> and compute device <NUM> is located external to the UAV <NUM>, with audio data from audio capture device <NUM> provided to the compute device <NUM> via the network <NUM> or other communication mechanisms. Alternatively, both the audio capture device <NUM> and compute device <NUM> are located remotely from the UAV <NUM> to detect failure of motor(s) of the UAV <NUM>.

In response to receipt of audio data, compute device <NUM> is configured to process the received audio data, as needed, at a block <NUM>. Example processing includes, without limitation, filtering, de-noising, converting to a different audio format, and/or the like. If the audio data comprises data for more than one motor, compute device <NUM> is configured to determine and separate audio data for each of the respective motors. For example, compute device <NUM> may be able to disambiguate audio data among different motors based on different directionality information associated with respective portions of the audio data, different audio profile for different types of motors, location information of respective motors relative to the audio capture device <NUM>, and/or the like. As another example, a controller included in the UAV <NUM> can be configured to cause pulsing or surging of each motor to temporarily increase (or otherwise change) the frequency or sound level of each motor at a different time from each other; a plurality of audio capture devices <NUM> can be used to facilitate triangulation of motor sounds and by extension disambiguation of audio data among different motors; UAV <NUM> can be moved relative to a single audio capture device <NUM> to permit identification of a degrading motor, in which each motor of the UAV <NUM> is placed near the single audio capture device <NUM> sequentially in time; and/or the like. In some embodiments, block <NUM> may be optional if processing of audio data is not required.

Next, at a block <NUM>, compute device <NUM> is configured to select or extract a particular portion of the audio data associated with a motor for analysis (or a particular portion of the audio data for each motor if the audio data is associated with a plurality of motors). The selected portion of the audio data can be a pre-set/nominal time interval of the audio data or a time interval that is different or shifted from the pre-set/nominal time interval. For example, audio data may comprise <NUM> minutes of audio data and the pre-set/nominal time interval is a <NUM> second portion of the audio data at time points <NUM>:<NUM>-<NUM>:<NUM> of the audio sample/clip. If the data at time points <NUM>:<NUM>-<NUM>:<NUM> is deemed to be undesirable because of noise, wind, interference, low data quality, and/or the like, then a different time point can be selected such as at times <NUM>:<NUM>-<NUM>:<NUM> of the audio sample/clip. Any of a variety of time points and/or time intervals can be selected from the audio sample/clip. As another example, audio data (or portions of the audio data) when the motor(s) are operating at a consistent or repeatable rotation rate are recorded and analyzed in order to facilitate comparisons over time and between motors and/or UAVs.

The compute device <NUM> converts the selected portion of the audio data from the time domain to the frequency domain to generate audio data in the frequency domain, at a block <NUM>. Discrete Fourier transform, and in particular, fast Fourier transform (FFT), is applied to the selected portion of the audio data to generate the audio data in the frequency domain.

<FIG> depicts a graph illustrating example plots of audio data in the frequency domain in accordance with an embodiment of the present disclosure. Plots <NUM>-<NUM> represent audio data in the frequency domain for the same motor. Plots <NUM>-<NUM> represents audio data in the frequency domain at respective different life cycle points of the motor, at respective different cumulative operational cycles of the motor, at respective different cumulative operational times of the motor, or at respective different operational points in time of the motor. The plot of amplitude vs. frequency for the current audio data in the frequency domain (from block <NUM>) may be any one of the plots <NUM>-<NUM>.

At a block <NUM>, compute device <NUM> is configured to analyze the audio data in the frequency domain generated at block <NUM> to determine one or more characteristics associated with motor failure. The analysis can include analyzing the audio data in the frequency domain in conjunction with one or more other audio data in the frequency domain for the same motor (e.g., from previous audio data for the same motor). In an embodiment, for a given motor, particular changes over time of the audio data in the frequency domain indicate impending motor failure (e.g., whether the motor is in an abnormal state, soon-to-fail state, or failure is imminent) and prediction of how close the motor is to (actually) failing relative to the current audio data in the frequency domain. In an embodiment, frequency analysis for each audio data in the frequency domain (e.g., the data corresponding to a selected portion or interval of the audio data from block <NUM>) is performed using a Hamming window technique and <NUM> bins.

As an example, compute device <NUM> can analyze a plurality of audio data in the frequency domain for the same motor such as shown in <FIG>. In <FIG>, plots <NUM>-<NUM> are associated with the same motor at <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> operational cycle remaining before failure, respectively. In an embodiment, the amplitude of plots centered at two frequencies increases inverse to the number of operational cycles remaining before failure of the motor. New peaks <NUM> and <NUM> are present or start to be present at frequencies between peaks <NUM> and <NUM>, which are consistently present through the motor's life cycle (see the existence of peaks <NUM> and <NUM> in each of plots <NUM>-<NUM>). Peaks <NUM> and <NUM> appear at approximately within the last <NUM> % of the motor's life or if the motor has approximately <NUM> to <NUM> operational cycles remaining before (actual) failure.

In <FIG>, it can be seen that peaks <NUM> and <NUM> of appreciable amplitude are present in plots <NUM> (e.g., associated with <NUM> operational cycles remaining before failure), plot <NUM> (e.g., associated with <NUM> operational cycles remaining before failure), and plot <NUM> (e.g., associated with <NUM> operational cycle remaining before failure). The amplitudes of peaks <NUM> and <NUM> also increase in time as the motor gets closer to failing (e.g., peaks <NUM> and <NUM> have greater amplitude in plot <NUM> than in plot <NUM>). The center frequencies of peaks <NUM> and <NUM> are at approximately <NUM> and <NUM> Hertz (Hz), respectively. It is understood that for a different motor or different operational profile (e.g., different rotation rate), the center frequency values can be different than <NUM> and <NUM>.

A region <NUM> of plots <NUM>-<NUM> is shown as a plot <NUM> of maximum to minimum amplitude ratios as a function of operational cycles remaining before failure or operational time remaining before failure in <FIG>, in accordance with an embodiment of the present disclosure. In an embodiment, region <NUM> is associated with the frequencies of the peak <NUM> or for a frequency range of approximately <NUM> to <NUM>. Alternatively, region <NUM> may be the frequency value of the peak <NUM>.

As shown in <FIG>, plot <NUM> is substantially horizontal for most of the motor's life (e.g., has a consistent maximum/minimum amplitude ratio in the approximate range of <NUM>-<NUM>) but then rapidly increases in ratio value as the motor approaches failure. A portion <NUM> of plot <NUM> associated with the rapidly increasing amplitude ratio occurs when the motor is within approximately <NUM> operational cycles before failure (or within approximately <NUM> hours of operation before failure). Portion <NUM> of plot <NUM> starts at an approximate ratio of <NUM> and ends at an approximate ratio of <NUM>. The closer the motor is to failing, the greater the amplitude ratio value. Plots, data points, or other data equivalents similar to plot <NUM> can exist for each different motor type, model, or configuration to be later accessed for each motor to be monitored.

Since the audio data in frequency domain is of a particular motor at a particular operational cycle of the particular motor, compute device <NUM> can perform a look up based on the particular motor's identifier information to select a particular plot (or data equivalent) of maximum/minimum amplitude ratio to operational cycle/time remaining to failure matching the particular motor's type, model, or configuration. Then using such particular plot, compute device <NUM> predicts the current number of operational cycles and/or time remaining before failure for the particular motor based on the current maximum/minimum amplitude ratio value for the particular motor.

Based on the current maximum/minimum amplitude ratio value from the current audio data in the frequency domain, compute device <NUM> can determine whether the motor is operating in an abnormal or failure imminent state; if in the abnormal/failure imminent state, how close to failing (e.g., number of operational cycles remaining before failure, number of operational time remaining before failure); and/or the like. In an embodiment, motor failure may be due to lateral movement of the bearings between the rotor <NUM> and stator <NUM>, which causes physical contact and rubbing between the magnets <NUM> of rotor <NUM> and the windings <NUM> of stator <NUM>. The physical contact and rubbing progressively increases over time until motor failure occurs.

In other embodiments, compute device <NUM> is configured to determine whether the motor is in an abnormal/failure imminent state and/or how close to failing based on analysis of just the current audio data in the frequency domain. For example, if peaks, such as peaks <NUM> and <NUM>, are present in the current audio data in the frequency domain and such peaks have amplitudes above a pre-set threshold associated with the particular type or model of the motor of interest, then compute device <NUM> can use such information to make failure determinations about the motor.

In another embodiment, propeller damage, degradation, or fouling may also be detected through similar audio analysis as described herein.

With the frequency analysis complete at block <NUM>, if an abnormal/failure imminent state is not detected (no branch of block <NUM>), then process <NUM> proceeds to block <NUM> to store the analysis results and to wait to receive the next audio data for the same motor(s) at block <NUM>. Then process <NUM> proceeds to return to block <NUM> to continue monitoring the motor(s) for upcoming failure. If an abnormal/failure imminent state is detected (yes branch of block <NUM>), then process <NUM> proceeds to block <NUM>. For example, if the current maximum/minimum amplitude ratio value for the motor is above a threshold value (e.g., greater than <NUM>), then the motor is deemed to be in an abnormal/failure imminent state.

At block <NUM>, compute device <NUM> is configured to determine whether the current state of the motor is such that the motor, and by extension UAV <NUM>, should be flagged for one or more action or restriction. In an embodiment, the predicted number of operational cycles or operational time remaining before failure for the motor based on the current maximum/minimum amplitude ratio value can be compared against a pre-set threshold at block <NUM>. The pre-set threshold value can be the same among the different motors; be different based on different motor types, models, or configurations; can be selected depending on what actions are to be taken to the motor, and by extension UAV <NUM>, if flagged; and/or the like. For example, if the goal is to be conservative about servicing or maintenance so that actual failures do not occur, then the pre-set threshold may be selected to flag motors well before actual failure is likely to occur, perhaps as soon as the first indication of motor failure is detected, such as a pre-set threshold of <NUM> operational cycles. As another example, if the goal is to maximize the operation of motors without prematurely retiring them, then the pre-set threshold may be set closer to when the motor operational cycle remaining will reach zero, such as a pre-set threshold equal to <NUM> operational cycles.

If the predicted number of operational cycles remaining before failure is greater than the pre-set threshold (no branch of block <NUM>), then process <NUM> proceeds to block <NUM> to continue monitoring. If the predicted number of operational cycles remaining before failure is equal to or less than the pre-set threshold (yes branch of block <NUM>), then process <NUM> proceeds to block <NUM>.

At block <NUM>, compute device <NUM> is configured to flag, identify, classify, designate, or otherwise generate an indication that the particular motor, and by extension UAV <NUM> that includes the particular motor, is to be subject to appreciate action(s) different from normal operations. With the ability to detect and anticipate motor failure, such flagged UAV <NUM> may be retired to reduce the chance of a hazard that may occur if allowed to continue to operate, restricted or removed from normal use for preventive servicing or maintenance, or designated for short distance missions only or other non-standard use. Safety increases as well as reduced costs associated with premature retiring of the UAV, retrieval of the UAV at unexpected emergency landing locations, damage caused by UAV due to motor failure, and/or the like.

<FIG> depicts an example device that may be implemented in the UAV <NUM>, compute device <NUM> or <NUM>, storage <NUM>, and/or audio capture devices <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> of the present disclosure, according to some embodiments. The device of <FIG> may comprise at least a portion of any of UAV <NUM>, compute device <NUM> or <NUM>, storage <NUM>, and/or audio capture devices <NUM>, <NUM>, <NUM>, <NUM>, or <NUM>. Platform <NUM> as illustrated includes bus or other internal communication means <NUM> for communicating information, and processor <NUM> coupled to bus <NUM> for processing information. The platform further comprises random access memory (RAM) or other volatile storage device <NUM> (alternatively referred to herein as main memory), coupled to bus <NUM> for storing information and instructions to be executed by processor <NUM>. Main memory <NUM> also may be used for storing temporary variables or other intermediate information during execution of instructions by processor <NUM>. Platform <NUM> also comprises read only memory (ROM) and/or static storage device <NUM> coupled to bus <NUM> for storing static information and instructions for processor <NUM>, and data storage device <NUM> such as a magnetic disk, optical disk and its corresponding disk drive, or a portable storage device (e.g., a universal serial bus (USB) flash drive, a Secure Digital (SD) card). Data storage device <NUM> is coupled to bus <NUM> for storing information and instructions.

Platform <NUM> may further be coupled to display device <NUM>, such as a liquid crystal display (LCD) or light emitting diode (LED) display coupled to bus <NUM> through bus <NUM> for displaying information to a user. Alphanumeric input device <NUM>, including alphanumeric and other keys, may also be coupled to bus <NUM> through bus <NUM> (e.g., via infrared (IR) or radio frequency (RF) signals) for communicating information and command selections to processor <NUM>. An additional user input device is cursor control device <NUM>, such as a mouse, a trackball, stylus, or cursor direction keys coupled to bus <NUM> through bus <NUM> for communicating direction information and command selections to processor <NUM>, and for controlling cursor movement on display device <NUM>. In embodiments utilizing a touch-screen interface, it is understood that display <NUM>, input device <NUM>, and cursor control device <NUM> may all be integrated into a touch-screen unit.

Another component, which may optionally be coupled to platform <NUM>, is a communication device <NUM> for accessing other nodes of a distributed system via a network. Communication device <NUM> may include any of a number of commercially available networking peripheral devices such as those used for coupling to an Ethernet, token ring, Internet, or wide area network. Communication device <NUM> may further be a null-modem connection, or any other mechanism that provides connectivity between platform <NUM> and the outside world. Note that any or all of the components of this system illustrated in <FIG> and associated hardware may be used in various embodiments of the disclosure.

Additionally, the processes may be embodied within hardware, such as an application specific integrated circuit (ASIC) or otherwise.

A tangible machine-readable storage medium includes any mechanism that provides (e.g., stores) information in a non-transitory form accessible by a machine.

(e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.). For example, a machine-readable storage medium includes recordable/non-recordable media (e.g., read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

Claim 1:
A method comprising:
receiving sounds emanating from one or more motors (<NUM>) included in an unmanned aerial vehicle, UAV, (<NUM>) during operation of the one or more motors, wherein the one or more motors comprises a vertical (<NUM>) or horizontal (<NUM>) propulsion motor of the UAV;
characterised by:
predicting a number of operational cycles remaining before the one or more motors is to fail based on analysis of the sounds; and
based on the determination of the number of operational cycles remaining, restricting the UAV from normal use;
wherein predicting the number of operational cycles remaining before the one or more motors is to fail comprises:
converting the sounds to audio data in a frequency domain;
detecting an increase in amplitude at a pre-determined frequency range of the audio data in the frequency domain; and
calculating a maximum to minimum amplitude ratio at the pre-determined frequency range.