Patent Publication Number: US-2022238133-A1

Title: Method of specifying generation point of abnormal sound and application program

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims priority to Japanese Patent Application No. 2021-009401 filed on Jan. 25, 2021, incorporated herein by reference in its entirety. 
     BACKGROUND 
     1. Technical Field 
     The present disclosure relates to a method of specifying a generation point of abnormal sound and an application program. 
     2. Description of Related Art 
     For example, Japanese Unexamined Patent Application Publication No. 9-196825 discloses a device that detects abnormality based on output when image data is input into a neural network and output when waveform data obtained by analyzing the frequency of an acoustic signal is input into the neural network. 
     SUMMARY 
     In the case of objects that generate sound at normal times, such as vehicles and the like, the sound pressure level of an abnormal sound generated when abnormality occurs does not necessarily exceed the sound pressure level of the sound generated at the normal times. Therefore, in order to make it possible to specify the generation point of abnormality by using a neural network as described above, the neural network has much information to identify, which often makes it hard for the neural network to learn. Furthermore, it is also necessary to identify the relationship between an image represented by image data and sound represented by an acoustic signal. However, it can be difficult for a person to identify the generation point of sound relating to abnormality. Therefore, to specify the relationship, a high level information processing ability is necessary. 
     Hereinafter, measures to solve the stated problems and the operation effects thereof will be described. 
     1. A method of specifying a generation point of abnormal sound includes: a reproduction step; an indication reception step; an association step; a selection step; and a specification step. The reproduction step is for reproducing sound of each of a plurality of frequency components by a reproduction device. The indication reception step is for receiving, after the reproduction step, an indication indicating which component, out of the frequency components, is close to a component of an abnormal sound of which a generation point is to be specified, via a human interface. The association step is for associating sound data with image data and storing the sound data and the image data in a storage device, the sound data being recorded by a recording device that is arranged in proximity to each of a plurality of regions that are obtained by dividing a target object, the image data being a part of image data on the target object imaged by an imaging device, the part being related to the target object that is in proximity to the recording device at a time of recording. The selection step is for selecting, out of the sound data associated with the image data in the association step, the image data associated with the data that is highest in sound pressure level of an indication frequency component that is the frequency component indicated in the indication reception step. The specification step is for specifying the generation point of the abnormal sound based on the image data selected in the selection step. 
     The method enables a person to indicate the frequency component that is characteristic of a sound that the person feels abnormal by reproducing sound of each of a plurality of frequency components. Therefore, regardless of whether the sound pressure level of the sound that the person feels abnormal is the maximum sound pressure level of the sound generated from the target object, the abnormal sound can properly be grasped. The sound data recorded by the recording device that is arranged in proximity to each of a plurality of regions that are obtained by dividing a target object is stored in association with a part of the image data related to the target object that is in proximity to the recording device. By selecting the image data associated with the data highest in sound pressure level of the indication frequency component, the selected image data is highly likely to represent the image data close to the generation point of the abnormal sound. Therefore, it becomes possible to specify the generation point of the abnormal sound detected by the person. 
     2. The method according to the first aspect may include: a recording step of recording the sound generated from the target object with the recording device and storing the sound as sound data in the storage device; and an extraction step of extracting the frequency components of the sound indicated by the sound data. In the method according to the first aspect, the reproduction step may be a step of reproducing sound of each of the frequency components extracted in the extraction step. 
     In the method, the frequency components of the sound data are extracted and reproduced. This enables a person to specify the component that the person feels abnormal based on the sound that the person actually hear. 
     3. In the method according to the second aspect, the target object may include a rotary machine, and the frequency components may be predetermined components that are proportional to a rotational frequency of the rotary machine. 
     When the target object includes a rotary machine, the rotary machine or a rotor coupled to the rotary machine may generate abnormal sound with rotation. In that case, the frequency of the abnormal sound is proportional to the rotation frequency of the rotary machine. In the above method, the frequency components to be extracted are the predetermined components that are proportional to the rotational frequency of the rotary machine. This makes it possible to accurately extract the components of the abnormal sound generated with rotation of the rotary machine. 
     4. The method according to the third aspect may include a measurement result acquisition step of acquiring a measurement result by a device that measures a variable value indicating rotation speed of the rotary machine. The extraction step may have a step of extracting the frequency components based on the rotational frequency indicated by the measurement result. 
     In the method, frequency components are extracted based on the rotational frequency indicated by the measurement result. Accordingly, target components can accurately be extracted without having to fix the rotation speed of the rotary machine to predetermined values. 
     5. The method according to the third or fourth aspect may include an inquiry step of inquiring whether or not the component of the abnormal sound of which the generation point is to be specified is a component dependent on the rotational frequency of the rotary machine. The extraction step may include: a step of extracting a plurality of predetermined components independently of the rotational frequency of the rotary machine when a response as a result of the inquiry is that the component is not dependent, and a step of extracting predetermined components proportional to the rotational frequency of the rotary machine when the response as the result of the inquiry is that the component is dependent. 
     In the method, it is possible to obtain useful information to determine the type of abnormal sound by inquiring whether the component of the abnormal sound is dependent on the rotation frequency of the rotary machine. 
     6. In the method according to any one of the first to fifth aspects, the specification step may include a first probability calculation step of using the image data as input to calculate probability that one or more candidates of the generation point of the abnormal sound are points indicated by the image that is indicated by the above image data, a second probability calculation step of calculating probability that the one or more candidates of the generation point of the abnormal sound are actual generation points of the abnormal sound based on the sound data associated with the image data selected in the selection step, and a comprehensive probability calculation step of calculating comprehensive probability that the one or more candidates of the generation point of the abnormal sound are actual generation points of the abnormal sound based on a result of probability calculation in the first probability calculation step and a result of probability calculation in the second probability calculation. 
     In the above method, the generation point of the abnormal sound is specified based on the position information on the target object indicated by the image data when the sound pressure level of the specified frequency component is the highest, as well as on the sound data at that time. This makes it possible to specify the generation point of the abnormal sound with high accuracy as compared with the case of specifying based on only the position information. 
     7. In the method according to the sixth aspect, the second probability calculation step may be a step of calculating probability corresponding to one or more candidates of the generation point of the abnormal sound, based on similarity between the sound data associated with the image data selected in the selection step and waveform pattern data representing relationship between frequencies that are characteristic when the candidates of the generation point of the abnormal sound generate sound and sound pressure levels. 
     In the above method, the probability corresponding to candidates of the generation point is calculated based on the similarity between the sound data on the candidates of the generation point and waveform pattern data on the sound pressure levels relative to the frequencies. This makes it possible to specify the generation point of the abnormal sound by using specific frequency that is characteristic of the abnormal sound, as well as the sound pressure waveform when the abnormal sound is generated. 
     8. In the method according to any one of the first to seventh aspects, the first probability calculation step may be a step of using the image data as input to calculate probability corresponding to the candidates of the generation point indicated by the image data. The second probability calculation step may be a step of calculating probability corresponding to the candidates of the generation point of the abnormal sound, based on similarity between the sound data associated with the image data selected in the selection step and waveform pattern data representing relationship between frequencies that are characteristic when the candidates of the generation point generate sound and sound pressure levels. 
     In the method, the recording device as well as the imaging device are arranged in proximity to each of a plurality of regions obtained by dividing a target object. This makes it easy to associate the recorded sound with the position information on the target object that faces the recording device. 
     9. An application program that causes a computer to execute the reproduction step, the indication reception step, the association step, and the selection step in the method of specifying a generation point of abnormal sound according to any one of the first to eighth aspects. 
     10. The application program according to the ninth aspect may cause the computer to execute: a transmission step of transmitting the image data selected in the selection step and the sound data associated with the image data; and a reception step of receiving a signal indicating information regarding the generation point of the abnormal sound specified in the specification step based on the image data and the sound data transmitted in the transmission step. 
     The above method allows the computer to execute the transmission step and reception step. Accordingly, another computer other than the computer can execute the specification step. Therefore, a computational load of the computer executing the reproduction step and other steps can be reduced as compared with the case where the computer also executes the specification step. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Features, advantages, and technical and industrial significance of exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings, in which like signs denote like elements, and wherein: 
         FIG. 1  shows the configuration of an abnormal sound generation point specification system according to an embodiment; 
         FIG. 2  is a flowchart showing processing procedures executed by a mobile terminal according to the embodiment; 
         FIG. 3A  shows processing executed by the mobile terminal and steps executed by a person; 
         FIG. 3B  shows processing executed by the mobile terminal and steps executed by a person; 
         FIG. 3C  shows processing executed by the mobile terminal and steps executed by a person; 
         FIG. 4  shows data prescribing the frequency components of abnormal sound in rotational synchronization according to the embodiment; 
         FIG. 5A  shows processing procedures executed by the mobile terminal and an analyzer; 
         FIG. 5B  shows processing procedures executed by the mobile terminal and an analyzer; 
         FIG. 6  illustrates waveform data according to the embodiment; and 
         FIG. 7  is a plan view showing a display example of the mobile terminal according to the embodiment. 
     
    
    
     DETAILED DESCRIPTION OF EMBODIMENTS 
     Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. 
       FIG. 1  shows a system for specifying a generation point of abnormal sound according to the embodiment. 
     A vehicle  10  is a vehicle brought into a repair shop by a user who detects abnormal sound. The vehicle  10  is mounted with an internal combustion engine  12  and a control device  14  of the internal combustion engine  12 . The repair shop may also be a dealer shop. 
     A mobile terminal  20  is a portable terminal possessed by an operator at the repair shop. The mobile terminal  20  includes a CPU  22 , a storage device  24 , a display  26 , a touchpanel  28 , a speaker  30 , a microphone  32 , a camera  34  and a communication device  36 , which can communicate with each other via a communication line  38 . Here, the storage device  24  is an electrically rewritable, non-volatile memory. The display  26  is constituted of, for example, an LCD or an LED. The touchpanel  28  is arranged so as to be superimposed on the display  26 . The communication device  36  can communicate with the control device  14  of the vehicle  10 . The communication device  36  can also communicate with an analyzer  50  possessed by a vehicle manufacturer via a network  40 . 
     The analyzer  50  includes a CPU  52 , a storage device  54 , and a communication device  56 , which can communicate via a communication line  58 . The storage device  54  is an electrically rewritable, non-volatile memory. 
     The storage device  24  of the mobile terminal  20  stores an application program  24   a . The storage device  54  of the analyzer  50  stores a specification program  54   a . The CPU  22  executes instructions defined by the application program  24   a , and the CPU  52  executes instructions defined by the specification program  54   a  so as to perform processing to specify abnormal sound generated in the vehicle  10 . Hereinafter, the processing will be described in detail. 
       FIGS. 2 and 5A  show the processing procedures defined by the application program  24   a . The processing shown in  FIGS. 2 and 5A  are implemented when the CPU  22  executes the processing whenever a prescribed condition is established, for example.  FIG. 5B  shows the processing procedures defined by the specification program  54   a . The processing shown in  FIG. 5B  is implemented when the CPU  52  executes the processing whenever a prescribed condition is established, for example. Hereinafter, the step number of each process will be expressed by numerals prepended with “S”. 
     In a series of processing shown in  FIG. 2 , the CPU  22  first photographs a video image of a target object in the vicinity of the point where an abnormal sound is generated (S 10 ). Specifically, the CPU  22  uses the camera  34  to photograph the target object, while using the microphone  32  to record surrounding sound. The processing may be achieved by, for example, outputting guidance information to urge photographing a video image of the target object when the application program  24   a  for specifying the generation point of the abnormal sound is started. Here, the guidance information may be output as visual information by operating the display  26 , or as audible information by operating the speaker  30 . 
       FIG. 3A  shows the case where the target object is an object housed in an engine compartment  16 . In this case, in accordance with an instruction given by the mobile terminal  20 , the operator opens the hood  15  and starts to photograph a video image of the entire object housed in the engine compartment  16 . In other words, the operator starts to photograph a video image of the entire target object. 
     With reference to  FIG. 2  again, the CPU  22  operates the display  26  to inquire whether the abnormal sound is the sound that synchronizes with rotation speed of the internal combustion engine  12  (S 12 ). In this case, the CPU  22  may display a supplementary explanation on the display  26  stating that when the frequency of the abnormal sound changes as the rotation speed of the internal combustion engine  12  changes due to operation of an accelerator or the like, then the sound is caused by synchronization. 
     The CPU  22  then determines whether input operation performed on the touchpanel  28  is input operation indicating the rotational synchronization or input operation indicating no synchronization (S 14 ). When it is determined that the input operation indicates synchronization (S 14 : YES), the CPU  22  communicates with the control device  14  to receive data about rotation speed NE of a crank shaft that is a rotation shaft of the internal combustion engine  12  (S 16 ). The CPU  22  then sets the frequency band of n-th order rotation as a prescribed frequency component that is characteristic as a candidate of the abnormal sound, based on the rotation speed NE (S 18 ). In this process, the CPU  22  refers to frequency component data  24   b  stored in the storage device  24  shown in  FIG. 1 . 
       FIG. 4  illustrates the frequency component data  24   b.    
     As shown in  FIG. 4 , the frequency component data  24   b  is data that defines the relationship between the rotation speed NE and the frequency of the abnormal sound for each of a plurality of candidates of the generation point of the abnormal sound. In  FIG. 4 , line segments f1, f2, . . . define the relationship of the frequency with the rotation speed NE. The line segments f1, f2, . . . each indicate relationship between the rotation speed NE relating to the candidates of the generation point of the abnormal sound and the frequency of generated sound. These candidates may include, for example, an oil pump that circulates lubricating oil of the internal combustion engine  12 , and may include, for example, a chain that transmits the power of the crank shaft to a cam shaft. 
     The process of S 18  is to set frequency bands that are defined based on the line segments f1, f2, . . . and the rotation speed NE. In other words, the number of the frequency bands set as prescribed frequency bands here is equal to the number of the line segments f1, f2, . . . . 
     With reference to  FIG. 2  again, when it is determined that the input operation indicates no synchronization (S 14 : NO), the CPU  22  sets as the prescribed frequency bands, a plurality of frequency bands characteristic of the abnormal sound independent of the rotation speed NE (S 20 ). This process can be achieved by setting a characteristic frequency band for each candidate of the generation point of sound not synchronized with the rotation speed NE in the frequency component data  24   b.    
     When the processes of S 18  and S 20  are completed, the CPU  22  operates the speaker  30  to reproduce sounds of a plurality of frequency bands set in the process of S 18  or the process of S 20  (S 22 ). This process can be achieved by band-pass filter processing applied to the sounds recorded in the process of S 10  and thereby generating data on sounds in the frequency bands set by the process of S 18  or process of S 20 . 
     The CPU  22  then operates the display  26  to display visual information for urging a user to indicate the sound closest to the abnormal sound among the reproduced sounds (S 24 ). The CPU  22  continues the processes of S 22  and S 24  until the frequency band closest to the abnormal sound is indicated (S 26 : NO) by input operation performed on the touchpanel  28 . Specifically, the processes of S 22  to S 26  may be performed by the CPU  22  to display the number of buttons, corresponding to the number of the frequency bands to be reproduced, on the display  26 , with the button currently selected for reproduction being displayed in color different from other buttons. In other words, when a specific button is indicated via the touchpanel  28 , the CPU  22  may determine that the sound closest to the abnormal sound is indicated. 
     When it is determined that there is input operation indicating the sound closest to the abnormal sound (S 26 : YES), the CPU  22  determines the indicated band as the indication frequency band (S 28 ). 
     Then, the CPU  22  operates the display  26  as shown in  FIG. 5A  to make an instruction to photograph a video image by arranging the mobile terminal  20  to face each of a plurality of regions obtained by dividing the target object (S 30 ). Specifically, the CPU  22  displays on the display  26  a moving path of the mobile terminal  20  that is superimposed on the entire image of the target object photographed by the process of S 10 . 
       FIG. 3B  shows a display example of the display  26 .  FIG. 3B  shows an example of the moving path of the mobile terminal  20  that is moved three times from left side to right side while the position of the mobile terminal  20  being shifted in a vehicle front-rear direction. Specifically,  FIG. 3B  shows an example of the moving path in which the mobile terminal  20  is moved within the engine compartment  16  such as from a rear left side to the rear right side of the vehicle and then to the front side of the vehicle  10 . 
     For proximity photographing of a moving image, it is desirable that a distance between the target object and the mobile terminal  20 , and the speed of moving the mobile terminal  20  are set in advance and the operator performs the processing based on the set distance and speed. This can be achieved by, for example, giving an instruction regarding the distance and speed by operating the display  26  or the speaker  30  in the process of S 30  shown in  FIG. 5A . This process can be substituted for giving instructions in an operation manual. 
     When proximity photographing of the moving image is started by operation of the operator, the CPU  22  stores image data from the camera  34  and sound data from the microphone  32  in the storage device  24  for each region obtained by dividing the target object (S 32 ). 
       FIG. 3C  shows an example of divided regions. In the example shown in  FIG. 3C , the target object is divided into six parts in the front-rear direction of the vehicle  10  and 12 parts in the right-left direction. In the example shown in  FIG. 3C , the target object is divided into a plurality of regions A11, A12, . . . , A21, A22, . . . . 
     Dividing the target object into regions can be achieved when, for example, the CPU  22  determines the position of the image indicated by the proximity video image in an overall image stored by the process of S 10 . Instead of determining the position, the CPU  22  may display on the display  26  the current position on the moving path shown in  FIG. 3B , and specify the regions based on the assumption that the mobile terminal  20  is arranged in proximity to the displayed position. Here, for example, the region A11 is a region of the target object that faces the mobile terminal  20  that is arranged in proximity to the target object. Therefore, the process of S 32  stores image data about a part of the target object that is in proximity to the mobile terminal  20  in association with the sound data at that time. 
     With reference to  FIG. 5A  again, the CPU  22  extracts an indication frequency component from the sound recorded by the microphone  32  when the mobile terminal  20  is arranged in proximity (S 34 ). The CPU  22  then assigns the sound pressure level of the indication frequency component to a sound pressure level L(i) identified by a label variable i indicating the current region (S 36 ). 
     The CPU  22  performs processes of S 32  to S 36  for all the regions of the target object. In other words, when it is determined that processing for all the regions are not yet complete (S 38 : NO), the CPU  22  updates the label variable i (S 40 ) and returns to the process of S 32 . Contrary to this, when it is determined that processing for all the regions are complete (S 38 : YES), the CPU  22  selects the image data and the sound data corresponding to the maximum sound pressure level L(i), among the image data and the sound data stored by the process of S 32  (S 42 ). The CPU  22  then operates the communication device  36  to transmit the selected image data and sound data to the analyzer  50  (S 44 ). 
     In this connection, as shown in  FIG. 5B , the CPU  52  of the analyzer  50  receives the transmitted image data and sound data (S 50 ). The CPU  52  then inputs the image data into mapping defined by mapping data  54   b  stored in the storage device  54  shown in  FIG. 1  to calculate a first probability a(j) regarding each of candidates j of the generation point of the abnormal sound (S 52 ). Here, the mapping is a convolutional neural network (CNN) using Softmax function as an activation function in an output layer. The number of dimensions of the output layer is the number of candidates of the generation point of abnormal sound. The value of each node in the output layer indicates the probability that the corresponding candidate is the generation point of the abnormal sound. 
     The mapping defined by the mapping data  54   b  is a learned model that is learned using the image data and training data that sets the value of a node corresponding to the object indicated by the image data to “1” and the value of other nodes to “0”. 
     The CPU  52  then calculates a second probability b(j) regarding the candidates j of the generation point of the abnormal sound based on the waveform of the sound pressure level corresponding to the frequency indicated by the received sound data (S 54 ). The process is based on the similarity between a waveform pattern indicated by waveform pattern data  54   c  for each of the candidates of the generation point of the abnormal sound stored in the storage device  54  shown in  FIG. 1 , and the waveform of the sound pressure level corresponding to the frequency indicated by the received sound data. 
       FIG. 6  shows the waveform pattern data  54   c . As shown in  FIG. 6 , the waveform pattern data  54   c  indicates, for each of the candidates of the generation point of abnormal sound, the waveform pattern representing the relationship between the frequency of the sound generated from the entire target object and the sound pressure level when the abnormal sound is generated. Here, the frequency in the waveform pattern of one candidate is not limited to the frequency that is characteristic of the abnormal sound generated from the candidate. Rather, the frequency is in a wide frequency band incorporating the characteristic frequency. This is the process of identifying the generation point of the abnormal sound based on sound information that is typical when the abnormal sound is generated. Accordingly, even when a plurality of generation points of an abnormal sound generate the abnormal sound having the same order frequency as the rotation frequency of the crank shaft, it becomes possible to identify which generation point generates the abnormal sound. 
     When, among the waveform patterns indicated by the waveform pattern data  54   c , some waveform patterns are below a prescribed similarity to the waveform of the sound pressure level corresponding to the frequency indicated by the received sound data, the CPU  52  may set the second probability b that the candidates corresponding to the patterns are the generation point of the abnormal sound to “0”. 
     With reference to  FIG. 5B  again, the CPU  52  calculates the comprehensive probability for each of the candidates based on the first probability a(j) and the second probability b(j) (S 56 ). Specifically, the CPU  52  defines, for each of the candidates, a product of the first probability a and the second probability b as the comprehensive provability. 
     The CPU  52  then operates the communication device  56  to transmit the result of specifying the generation point of the abnormal sound to a transmission source of the data received by the process of S 50  (S 58 ). Here, the CPU  52  transmits information about three candidates of the generation point higher in the comprehensive probability. The CPU  52  temporarily ends a series of the processes shown in  FIG. 5B , when the process of S 58  is completed. 
     In contrast, as shown in  FIG. 5A , upon reception of the specification result (S 46 : YES), the CPU  22  of the mobile terminal  20  operates the display  26  to display the received information as visual information (S 48 ). 
       FIG. 7  shows a display example displayed on the display  26  of the mobile terminal  20 . As shown in  FIG. 7 , parts A, B, C are listed in descending order of the comprehensive probability. 
     The CPU  22  temporarily ends a series of processing shown in  FIGS. 2 and 5A , when the process of S 48  is completed. 
     Now, the functions and effects of the present embodiment will be described. 
     When an abnormal sound of which generation point is expected to be specified, is proportional to the rotation frequency of the crank shaft, the CPU  22  reproduces sounds of frequencies corresponding to candidates of the generation point, which are component parts that can generate proportional abnormal sounds. When the operator indicates the frequency closest to the abnormal sound, out of the reproduced sounds, the CPU  22  sets the frequency as an indication frequency. The CPU  22  then photographs images and records sounds when mobile terminal  20  is arranged in proximity to a plurality of regions obtained by dividing the target object. Then, the CPU  22  calculates the sound pressure level of the indication frequency component among the recorded sounds, and stores the calculated sound pressure level. The CPU  22  selects the region, out of the regions, highest in recorded sound pressure level, and transmits the image data and sound data on the region to the analyzer  50 . 
     The CPU  52  of the analyzer  50  calculates the first probability a that the object, indicated by the transmitted image data, is each of the candidates of the generation point. The CPU  52  also calculates the second probability b that each of the candidates of the generation point is actually the generation point of the abnormal sound, based on the similarity between the waveform indicated by the transmitted sound data and the waveform indicated by the waveform pattern data  54   c . Then, the CPU  52  multiplies values of the first probability a and the second probability b of the same candidate to obtain the comprehensive probability of the candidate, and transmits the obtained comprehensive probability to the CPU  22 . The CPU  22  displays the transmitted result of specifying the generation point of the abnormal sound. 
     This makes it possible to calculate the generation point of the abnormal sound with high accuracy. Specifically, for example, since the frequency of inverse number of the time interval at a compression top dead center of the internal combustion engine  12  is the frequency of inverse number of the period of a combustion phase, generated sound has a frequency with a high sound pressure level. However, the sound sensed as abnormal sound is not necessarily the sound of this frequency. Therefore, it is difficult to grasp which sound the user recognizes as abnormal sound based only on the frequency analysis of the sound data. On the contrary, in the present embodiment, sounds of frequencies that are characteristic as the candidates of the generation point of abnormal sound are reproduced, and a person indicates which sound of frequency is close to the abnormal sound. This makes it possible to correctly understand the abnormal sound and then specify the generation point thereof. As a result, it is possible to grasp the generation point of the abnormal sound with high accuracy. 
     According to the present embodiment described in the foregoing, the functions and effects as described below are further achieved. 
     (1) An indication frequency component is extracted from the sounds recorded when the mobile terminal  20  is arranged in proximity to each of a plurality of regions obtained by dividing a target object, and the point with a highest sound pressure level is specified. Accordingly, even in the case where an operator who heard abnormal sound has an incorrect preconception regarding the generation point of the abnormal sound, it is possible to specify the place highly likely to generate the sound of the indication frequency regardless of the preconception. 
     (2) In the case of abnormal sound proportional to the rotation frequency of the crank shaft, the rotation speed NE is obtained from the control device  14 . Based on the rotation speed NE, frequency components, characteristic of the sound generated by each of the candidates of the generation point of the abnormal sound, are extracted. Accordingly, a target component can accurately be extracted without having to fix the rotation speed NE of the crank shaft to predetermined values. 
     (3) An inquiry is made to a person regarding whether the abnormal sound is proportional to the rotation frequency of the crank shaft. As a result, it is possible to obtain useful information to determine the type of abnormal sound. 
     (4) The CPU  52  calculates the first probability a that each of the candidates is the generation point, from the image data when the indication frequency is at its maximum, and calculates the second probability b that each of the candidates is the generation point, based on the similarity between the waveform indicated by the sound data with the frequency at its maximum and the waveform indicated by the waveform pattern data  54   c . This makes it possible to specify the generation point of the abnormal sound by using the indication frequency that is a specific frequency characteristic of the abnormal sound as well as the sound pressure waveform when the abnormal sound is generated. Accordingly, even when the frequencies generated by a plurality of generation candidates are the same-order frequencies as the rotation frequency of the crank shaft, it becomes possible to specify which of these candidate is the generation point. 
     (5) The analyzer  50  executes the processes of S 52  and S 54 . This makes it possible to reduce the computational load on the mobile terminal  20  as compared with the case where the processes of S 52  and S 54  are executed by the mobile terminal  20 . 
     Correspondence Relation 
     The correspondence relation between the matters in the embodiments and the matters described in the column “Solution to Problem” is as follows. In the followings, the correspondence relation is shown for every number of the solutions stated in the column “Solution to Problem”. [1, 8] The target object corresponds to the object housed in the engine compartment  16  shown in  FIG. 3A . The storage device corresponds to the storage device  24 . The reproduction device corresponds to the speaker  30 . The reproduction step corresponds to the process of S 22 . The indication reception step corresponds to the process of S 24  and S 26 . The association step corresponds to the process of S 32 . The selection step corresponds to the processes of S 34  to S 42 . The specification step corresponds to the processes of S 52  to S 56 . [2] The recording step corresponds to the process of S 10 . The extraction step corresponds to the processes of S 18  and S 20 . [3] The rotary machine corresponds to the internal combustion engine  12 . [4] The device that measures corresponds to the control device  14 . The measurement result acquisition step corresponds to the process of S 16 . [5] The inquiry step corresponds to the process of S 12 . [6] The first probability calculation step corresponds to the process of S 52 . The second probability calculation step corresponds to the process of S 54 . The comprehensive probability calculation step corresponds to the process of S 56 . [7] The pattern data corresponds to the waveform pattern data  54   c.  [9] The application program corresponds to the application program  24   a.  [10] The transmission step corresponds to the step of executing the process of S 44 . The reception step corresponds to the step of executing the process of S 46 . 
     Other Embodiments 
     The present embodiment can be implemented with modifications as shown below. The present embodiment and following modifications can be implemented in combination with each other without departing from the range of technically consistency. 
     About Rotary Machine
         The rotary machine included in the vehicle is not limited to an internal combustion engine, and may be a motor generator, for example.   The rotary machine is not limited to those included in the vehicle.       

     About Measurement Result Acquisition Step
         In the above embodiment, the rotation speed NE is transmitted in real time from the control device  14  during recording of sound. However, the present disclosure is not limited to the configuration. For example, the rotation speed NE acquired each time may be associated with the time of measurement and stored in the control device  14 . Upon request from the mobile terminal  20 , the control device  14  may collectively transmit time series data on the rotation speed NE and the time information.   For example, when the rotation speed of the rotary machine, such as the internal combustion engine  12 , is controlled to a predetermined value at the time of recording, reception of data regarding the rotation speed NE may be omitted.       

     About Inquiry Step
         In the embodiment, the inquiry step is executed after the start of recording. However, without being limited to this, the recording may be started after the inquiry process is completed.   For example, the application program  24   a  may be formed as a program specialized in specifying abnormal sound that is synchronized with rotation, and the processes of S 12 , S 14 , S 20  may be removed. In other words, in this case, the inquiry process can be omitted.       

     About Reproduction Step and Indication Reception Step
         In the embodiment, in the process of S 22 , reproduction data is created by band-pass filter processing applied to the sound data recorded in the process of S 10 . However, the present disclosure is not limited to the configuration. For example, sample sound source data may be included in the frequency component data  24   b , and be reproduced.   In the embodiment, sounds of a plurality of frequency components are sequentially reproduced on the side of the mobile terminal  20 , and then a person is urged to indicate the sound of which generation point is expected to be specified. However, the present disclosure is not limited to the configuration. For example, buttons assigned with the frequency components may be displayed on the display  26 , and the sound of the frequency components corresponding to the pressed button may be reproduced. In this case, for example, long-press of the button may be defined as an operation to indicate the sound of which generation point is expected to be specified.   The human interface operated by a person to indicate the sound of which generation point is expected to be specified is not limited to the touchpanel  28 . For example, a microphone  32  may be employed as the human interface, and voice input operation may be used to indicate the sound of which generation point is expected to be specified.       

     About Association Step
         In the embodiment, image data and sound data on a part of the target object that faces the mobile terminal  20  are associated by photographing video image and recording sound while the mobile terminal  20  is arranged in proximity to the target object. However, the present disclosure is not limited to the configuration. For example, each time the mobile terminal  20  is slightly relocated, a still image of the part of the target object facing the mobile terminal  20  may be photographed, and the sound recorded while the mobile terminal  20  is fixed may be associated with the corresponding still image data.   It is not essential to photograph images of the target object when the mobile terminal  20  is arranged in proximity to the target object. For example, after photographing a video image or a still image of the entire target object, a portion of the entire image of the target object may be marked and displayed as the point where the mobile terminal  20  is to be arranged in proximity. In this case, the sound data recorded by the mobile terminal  20  when the marked point is displayed may be regarded as the sound data recorded while the mobile terminal  20  is arranged in proximity to the marked point. This makes it possible to associate the image data on the marked point with the sound data recorded in proximity to the marked point.       

     About First Probability Calculation Step
         In the embodiment, dimensionality of output variables of mapping is equal to the number of candidates of the generation point. However, the dimensionality is not limited to this. For example, the dimensionality may be the number of the candidates of the generation point plus “1”, to add an output variable indicating the probability that no candidate is the generation point.       

     About Second Probability Calculation Step
         In the embodiment, the probability of the generation point is calculated based on the similarity between the waveform of the sound pressure level corresponding to the frequency of sound data when the sound pressure level of the indication frequency component is at its maximum and the pattern of waveform data for each candidate of the generation point of abnormal sound. However, the calculation of the probability is not limited to this. For example, the probability may be calculated with a learned model that outputs the probability of each candidate of the generation point by using as the input variable the sound pressure level of each frequency indicated by the sound data when the sound pressure level of the indication frequency component is at its maximum. Here, the learned model may be a neural network, for example. However, without being limited thereto, the learned model may be, for example, an identification model using a support vector machine. It is also possible to use a decision tree, for example. The learning model is also not limited to those outputting the probability of each candidate of the generation point in a single input. It is also possible to use a recurrent neural network, such as LSTM, in which input variables are input for a plurality of times to provide precise output values.       

     About Comprehensive Probability
         In the embodiment, the comprehensive probability is calculated as a product of the first probability a and the second probability b. However, the comprehensive probability is not limited to this. For example, the comprehensive probability may be a sum of the first probability a and the second probability b.       

     About Display of Specification Result
         In the embodiment, the three candidates higher in probability of being an actual generation point are displayed. However, the number of the candidates to be displayed is not limited to three. For example, the top candidate may be displayed, or higher five candidates may be displayed.   In the embodiment, only the candidates of the generation point are displayed. However, the probability of the candidates being an actual generation point may also be displayed at the same time.       

     About Computer that Executes Processing to Specify Generation Point
         In the embodiment, the processing to specify the generation point of abnormal sound is executed in cooperation with the CPU  22  of the mobile terminal  20  and the CPU  52  of the analyzer  50 . However, the processing configuration is not limited to this. For example, the CPU  22  of the mobile terminal  20  may also execute the processes of S 52  to S 56 . For example, a computer with a built-in scanning tool provided in a repair shop may execute the processing shown in  FIG. 5B  in place of the analyzer  50  provided in the manufacturer. For example, a computer that processes big data, independent of the manufacturer, may execute the processing shown in  FIG. 5B . For example, the computer that executes the processing shown in  FIGS. 2 and 5A  may be built into the user&#39;s mobile terminal, and the computer that executes the processing shown in  FIG. 5B  may be built into the mobile terminal possessed by the operator at the repair shop.       

     About Computer 
     The computer is not limited to those executing software processing. The computer may include a dedicated hardware circuit for hardware processing, such as ASICs. The computer may further be a combination of a software execution device that executes programs and the dedicated hardware processing circuit. Specifically, the computer may have any one of the configurations (a) to (c) below: (a) including a software execution device that executes all the processing based on programs; (b) including a software execution device that executes some of the processing based on programs, and a dedicated hardware circuit that executes the remaining processing; and (c) including a dedicated hardware circuit that executes all the processing. Here, the number of the software execution devices, or the number of the dedicated hardware circuits may be two or more. 
     About Target Object as Specification Target of Generation Point of Abnormal Sound
         In the embodiment, an object housed in the engine compartment is illustrated as the target object. However, the target object is not limited to this. For example, processing may be performed to specify the generation point of abnormal sound in an instrument panel in the vehicle cabin. Furthermore, the abnormal sound is not limited to sound generated in the vehicle.