Unmanned aircraft, information processing method, and recording medium

An unmanned aircraft includes: a sensor that includes at least a microphone that generates sound data; and a processor. The processor determines the quality of a target sound by use of the sound data generated by the microphone, identifies a sound source direction from the unmanned aircraft to the sound source of the target sound by use of data generated by the sensor, and controls an unmanned aircraft state that is a state of the unmanned aircraft such that a direction of a sound pickup area is aligned with the sound source direction, in accordance with the determined quality. The sound pickup area is a range in which sound pickup quality of the microphone is higher than that of another area.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority of Japanese Patent Application Number 2018-135175 filed on Jul. 18, 2018, the entire content of which is hereby incorporated by reference.

BACKGROUND

1. Technical Field

The present disclosure relates to an unmanned aircraft, an information processing method, and a recording medium.

2. Description of the Related Art

Japanese Unexamined Patent Application Publication No. 2017-502568 (Patent Literature 1: PTL 1) discloses an unmanned aerial vehicle that performs a process of removing background noise generated by the unmanned aerial vehicle from sound data picked up by a microphone in order to isolate a desired sound signal.

SUMMARY

When the background noise is louder than the other sounds, the technology disclosed in PTL 1 may reduce the resulting quality of the desired sounds obtained by the process of removing the background noise.

In view of the above, the present disclosure aims to provide an unmanned aircraft, an information processing method, and a recording medium that are capable of enhancing the quality of the target sound.

The unmanned aircraft according to the present disclosure includes: a sensor that includes at least a microphone that generates sound data; and a processor. In this unmanned aircraft, the processor determines the quality of a target sound by use of the sound data generated by the microphone, identifies the sound source direction from the unmanned aircraft to the sound source of the target sound by use of data generated by the sensor, and controls an unmanned aircraft state that is a state of the unmanned aircraft such that a direction of a sound pickup area is aligned with the sound source direction, in accordance with the determined quality. The sound pickup area is a range in which sound pickup quality of the microphone is higher than that of another area.

Note that these general or specific aspects may be implemented as a system, a method, an integrated circuit, a computer program, or a computer readable recording medium such as a CD-ROM, or may be implemented as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

The unmanned aircraft, the information processing method, and the recording medium according to the present disclosure allow high quality sound to be recorded by the unmanned aircraft.

DETAILED DESCRIPTION OF THE EMBODIMENT

Findings that Forms Basis of the Present Disclosure

The unmanned aerial vehicle disclosed in PTL 1 fails to consider a relative positional relationship between the unmanned aerial vehicle and the sound source from which sound data is to be picked up. This may cause the sound source not to be included in a sound pickup area of a microphone included in the unmanned aerial vehicle. When the sound source is not included in the sound pickup area of the microphone as in this case, the microphone cannot effectively pick up the target sound, and consequently picks up a relatively larger amount of background noise. This results in a relatively larger noise component of the sound data picked up by the microphone, and thus in a smaller signal-to-noise (SN) ratio. For this reason, it is difficult to obtain high-quality sound data even by processing the obtained sound data to remove the background noise.

In order to solve the above concerns, the unmanned aircraft according to one aspect of the present disclosure is an unmanned aircraft, including: a sensor that includes at least a microphone that generates sound data; and a processor. In the unmanned aircraft, the processor determines the quality of a target sound by use of the sound data generated by the microphone, identifies the sound source direction from the unmanned aircraft to the sound source of the target sound by use of data generated by the sensor, and controls an unmanned aircraft state that is a state of the unmanned aircraft such that a direction of a sound pickup area is aligned with the sound source direction, in accordance with the determined quality. Unmanned aircraft state includes, but is not limited to, position or altitude. The sound pickup area is a range in which sound pickup quality of the microphone is higher than that of another area.

In this configuration, the processor controls the unmanned aircraft state in order to include the sound source direction within the sound pickup area, such that the sound quality of the target sound recorded by microphone is higher than that of another sound pickup area. This configuration thus achieves a relatively larger component of the target sound of the sound data generated by the microphone, and enhances the quality of the target sound.

The processor may control the unmanned aircraft state such that the change in the unmanned aircraft states is minimized. Changes to the aircraft state that require the increase in rotation speed of at least one propeller result in increased background noise levels. The processor may also vary the level of change in the unmanned aircraft state in order to perform the sound recording.

This configuration controls the unmanned aircraft state with the least possible amount of change, and thus reduces the noise from the unmanned aircraft caused by the control of the aircraft state. For example, when the change in the unmanned aircraft state is minimized, such as performing the shortest possible maneuver, the above configuration minimizes the length of time required to change the rotational speeds of the rotor blades included in the unmanned aircraft, or the amount of change required to change the rotational speeds of the rotor blades. This configuration is thus capable of reducing the noise that is produced until when the sound source is included within the sound pickup area in which the sound pickup quality of the microphone is higher than that of another area. Also, a minimized change in the rotational speeds achieves a smaller change in noise characteristics. Consequently, the accuracy of noise removal processing is improved. That the noise is hard to be picked up and easy to be removed results in an enhanced quality of the target sound.

In one configuration, the processor may also control the unmanned aircraft state such that noise from the unmanned aircraft due to the change is minimized.

This configuration changes the aircraft state of the unmanned aircraft to reduce the noise, and thus reduces the noise that is produced until when the sound source is included within the sound pickup area in which the sound pickup quality of the microphone is higher than that of another area. This configuration thus enhances the quality of the target sound.

The processor may also identify the sound source direction by use of the sound data generated by the microphone.

This configuration enables the unmanned aircraft to identify the sound source direction only by being equipped with a microphone as a sensor.

The sensor may further include an image sensor that generates image data, and the processor may identify the sound source direction by use of the image data generated by the image sensor.

This configuration enables the unmanned aircraft to identify the sound source direction without being affected by the surrounding noise.

The processor may modify the unmanned aircraft state when the determined quality is lower than a threshold.

This configuration improves the quality of the target sound when the quality of the obtained target sound is low.

The processor may maintain the unmanned aircraft state when the determined quality is higher than a threshold.

In this configuration, the processor does not change the aircraft state of the unmanned aircraft when the quality of the target sound is sufficiently high, and thus prevents the reduction in the quality of the target sound caused by the noise that is produced due to the change in the unmanned aircraft state or by the change in the noise characteristics.

The unmanned aircraft according to another aspect of the present disclosure is an unmanned aircraft, including: a sensor that includes at least a microphone that generates sound data, and an image sensor; and a processor. In the unmanned aircraft, the processor may determine the quality of a target sound by use of the sound data generated by the microphone, and control an unmanned aircraft state that is a state of the unmanned aircraft such that a direction of a sound pickup area is aligned with an imaging direction of the image sensor, in accordance with the determined quality. Unmanned aircraft state includes, but is not limited to, position or altitude. The sound pickup area is a range in which sound pickup quality of the microphone is higher than that of another area.

In this configuration, the processor controls the aircraft state of the unmanned aircraft in order to include the imaging direction within the sound pickup area in which the sound pickup quality of the microphone is higher than that of another area. Here, in many cases, the imaging direction is in the direction in which the sound source of the target sound is located. This configuration thus achieves a relatively larger component of the target sound of the sound data generated by the microphone, and enhances the quality of the target sound.

Note that these general or specific aspects may be implemented as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be implemented as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

The following specifically describes the unmanned aircraft according to one aspect of the present disclosure with reference to the drawings.

Note that the following embodiment is a specific example of the present disclosure. The numerical values, shapes, materials, structural components, the arrangement and connection of the structural components, steps, the processing order of the steps, etc. shown in the following embodiment are mere examples, and thus are not intended to limit the present disclosure. Of the structural components described in the following embodiment, structural components not recited in any one of the independent claims that indicate the broadest concepts of the present disclosure will be described as optional structural components.

Embodiment

The following describes the embodiment with reference toFIG. 1throughFIG. 12B.

FIG. 1shows external views of an unmanned aircraft and a controller according to the embodiment.FIG. 2is a plan view of the unmanned aircraft according to the embodiment in a top view.

As shown inFIG. 1, unmanned aircraft100receives from controller200an operation signal that is in accordance with a user operation to controller200, and flies in accordance with the received operation signal. Unmanned aircraft100may perform imaging mid-flight by use of camera107included in unmanned aircraft100in accordance with the received operation signal. The image data captured by camera107may be sent to mobile terminal300to be described later.

Controller200accepts an operation from the user, and sends to unmanned aircraft100an operation signal that is in accordance with the accepted operation. Controller200may hold mobile terminal300such as a smartphone with a display.

Mobile terminal300receives from unmanned aircraft100the image data captured by camera107of unmanned aircraft100, and displays, for example, the received image data in real time.

This enables the user to operate controller200while checking on mobile terminal300the image data captured by camera107of unmanned aircraft100in real time, thereby changing the aircraft state of unmanned aircraft100that is at least one of the in-flight position and attitude of unmanned aircraft100. The user thus can freely change the range of imaging by camera107of unmanned aircraft100.

Each of four generators110generates thrust to fly unmanned aircraft100. More specifically, each of four generators110produces an airflow to generate thrust to fly unmanned aircraft100. Each of four generators110includes rotor blade111that rotates to produce an airflow, and actuator112that rotates rotor blade111. Each rotor blade111and actuator112include an axis of rotation that is substantially parallel in the vertical direction to produce an airflow that flows downward from above. This configuration enables four generators110to produce thrust that levitates unmanned aircraft100upward, allowing unmanned aircraft100to fly. Each actuator112is, for example, a motor.

In a top view, four generators110are arranged around main body140at 90 degree intervals. Stated differently, four generators110are arranged in a circular form to surround main body140.

Note that rotor blade111included in each of four generators110is illustrated to be formed by a single propeller as a non-limited example, and thus rotor blade111may be formed by a counter-rotating propeller that includes two propellers that rotate in counter directions about the same axis of rotation. The propeller may also have more than two blades.

FIG. 3is a cross-sectional view of the unmanned aircraft taken at line III-III inFIG. 2. Stated differently,FIG. 3is a cross-sectional view of a single generator110and its corresponding duct130cut along a plane that runs through the axis of rotation of rotor blade111.

One duct130is provided to each generator110, with the axis of rotation of the generator substantially parallel and in-line with the axis of duct130internal open volume. Each of four ducts130is arranged to laterally cover the corresponding generator110. Stated differently, each of four ducts130is arranged to cover the corresponding generator110in a direction that is substantially orthogonal to the direction of the axis of rotation of rotor blade111of the corresponding generator110. For example, each of four ducts130laterally covers the corresponding generator110along the length in the direction of the axis of rotation of such generator110. Stated differently, each of four ducts130has space131in which the corresponding generator110is arranged, and which has the shape of a circular cylinder that vertically passes through duct130. Each of four ducts130has a shape that tapers in thickness toward the downstream side of an airflow produced by the corresponding generator110. More specifically, each of four ducts130has a shape in which the outer surface of duct130is nearer to the inner surface of such duct130in a circular cylindrical shape toward the downstream side of an airflow produced by the corresponding generator110. Stated differently, each of four ducts130has a pointed shape at its downstream side of an airflow produced by the corresponding generator110. Also, the inner surface of each duct130has a rounded end portion at its upstream side of an airflow. More specifically, such end portion has a shape in which the inner diameter of duct130tapers in the flow direction of an airflow. This shape facilitates the flow of the air into duct130, and thus improves the flight performance. This also achieves the weight reduction of ducts130, and further the lightening of unmanned aircraft100. Note that such end portion may have a linear shape that extends along the flow direction of an airflow.

An example of main body140is a boxy member in a circular cylindrical shape, i.e., a cabinet. Electrical components such as a processor, a memory, a battery, and various sensors are arranged inside main body140. Note that the shape of the member of main body140is not limited to a circular cylindrical shape, and thus may be another boxy shape such as a quadrangular prism. Main body140also includes four microphones105, gimbal106, and camera107on the outer surface. Each of four microphones105is arranged, for example, at a position on main body140in between two adjacent generators110of four generators110. Stated differently, four microphones105are arranged about main body140at positions that are 45 degrees off from each other with respect to the direction that faces four generators110.

Four arms141are members that connect the respective four ducts130with main body140. Each of four arms141has one end fixed to main body140, and the other end fixed to the corresponding one of four ducts130.

FIG. 4is a block diagram of the configuration of the unmanned aircraft according to the embodiment. More specifically,FIG. 4is a block diagram that illustrates the function of processor101that is implemented by use of the hardware configuration of unmanned aircraft100.

Processor101obtains results such as: detection results from various sensors such as acceleration sensor103, gyroscope sensor104, four microphones105, an image sensor of camera107, and distance sensor108; a reception result from GPS receiver102or communication IF109; and others. Processor101executes various processes on the obtained detection results or the reception result by executing a predetermined program stored in a non-illustrated memory or storage, thereby controlling at least one of: four generators110; gimbal106; and camera107.

GPS receiver102receives information indicating the position of GPS receiver102from satellites including a GPS satellite. Stated differently, GPS receiver102detects the current position of unmanned aircraft100.

Acceleration sensor103is a sensor that detects accelerations in three different directions of unmanned aircraft100.

Gyroscope sensor104is a sensor that detects an angular rate of rotation about each of the three axes in the three different directions of unmanned aircraft100.

Each of four microphones105is a microphone, and an example of the sensors. Each of four microphones105has directionality that enables the pickup of sounds, in a sound pickup area of microphone105, having higher sound quality than the quality of sounds in an angular range other than such sound pickup area. Here, the sound pickup area is a predetermined angular range that is defined with respect to a specified direction. The predetermined angular range is, for example, an angular range of 90 degrees or less, and a three-dimensional angular range that expands with respect to the position of each microphone105. Each of four microphones105may be a microphone array having a plurality of microphone elements. Each of four microphones105picks up sound to generate sound data, and outputs the generated sound data.

Gimbal106is a device for maintaining a constant attitude of camera107in the three-axis directions. Stated differently, gimbal106is a device for maintaining a desired attitude of camera107relative to the terrestrial coordinate system, for example, even when the attitude of unmanned aircraft100changes. Here, the desired attitude is an attitude that is defined by an imaging direction of camera107indicated by an operation signal received from controller200.

Camera107, which is an example of the sensors, is a device having an image sensor, and an optical system such as a lens.

Distance sensor108is a sensor that detects the distance from distance sensor108to an object around. Examples of distance sensor108include an ultrasonic sensor, a time of flight (TOF) camera, and a light detection and ranging (LIDAR).

Communication IF109is intended for communication with controller200or mobile terminal300. Communication IF109includes, for example, a communication interface for receiving a transmission signal from controller200. Communication IF109may also be a communication interface for wireless communication with mobile terminal300. Stated differently, communication IF109may be a wireless local area network (LAN) interface compliant with, for example, the IEEE802.11a,b,g,n, and ac standards.

Four generators110have been described above, and thus will not be described here.

Functional components included in processor101are sound pickup processing unit101a, quality determiner101b, sound source identifier101c, position detector101d, flight controller101e, video controller101f, and obstacle detector101g.

Sound pickup processing unit101aobtains the sound data generated by each of four microphones105by picking up sounds. Sound pickup processing unit101amay perform signal conversion to convert digital microphone signal into an analog waveform. Sound pickup processing unit101amay perform on the obtained sound data a predetermined sound process of filtering a sound component in a predetermined frequency range to reduce noise included in the sound data. The sound component in the predetermined frequency range is, for example, a sound component in a frequency range of noise produced by the rotation of each rotor blade111of the corresponding generator110.

Quality determiner101buses the sound data generated by each of four microphones105to determine the quality of the target sound included in the sound data. More specifically, quality determiner101bdetermines the SN ratio of the target sound to determine the quality of such target sound. For example, quality determiner101bdetermines whether the SN ratio, which is an example quality, is greater than a threshold. When the SN ratio is greater than the threshold, quality determiner101bdetermines that the quality is high, whereas when the SN ratio is less than the threshold, quality determiner101bdetermines that the quality is low. The SN ratio is calculated, for example, by first determining the sound level in decibels of the microphone signal with only the unmanned aircraft sound prior to noise reduction sound filtering, calculating the sound level of a target sound recorded by the microphone after noise reduction, also in decibels, and then calculating the difference between the two as the SN ratio.

Sound source identifier101cuses the sound data generated by each of four microphones105to identify the sound source direction, which is a direction to the sound source of the target sound from unmanned aircraft100. Sound source identifier101cmay compare the four pieces of sound data obtained from four microphones105to identify as the sound source direction a direction in which the sound pressure of the target sound is estimated to be high. Sound source identifier101cmay also compare the time of arrival at the four microphones and the physical arrangement of the four microphones to identify the sound source direction. Sound source identifier101cmay also compare pieces of data, which are included in each piece of the sound data obtained from each of four microphones105and which are obtained from a plurality of microphone elements included in each of four microphones105. Through this process, sound source identifier101cidentifies as the sound source direction a direction in which the sound pressure of the target sound is estimated to be high.

Alternatively, sound source identifier101cmay use the image data generated by the image sensor of camera107to identify the sound source direction, which is a direction to the sound source of the target sound from unmanned aircraft100. In this case, sound source identifier101cmay recognize, through an image process on the image data, the color, shape, type, or other feature of the sound source that have been previously determined, thereby identifying the sound source direction, or estimating the distance to the sound source. When the sound source direction has been identified, sound source identifier101cmay use distance sensor108to detect the distance to the object in the sound source direction, thereby estimating the distance to the sound source. Sound source identifier101cmay obtain the volume of the target sound emitted from the sound source to compare the sound pressure of the target sound included in the sound data generated by each of four microphones105with the obtained volume of the target sound, thereby estimating the distance to the sound source. In this case, the volume of the target sound emitted from the sound source may be a predetermined volume. Alternatively, sound source identifier101cmay obtain position information on the sound source position from the sound source to identify the sound source direction or the distance to the sound source.

The sound source may be, for example, a person, a speaker, or a vehicle.

Position detector101dobtains the detection result from GPS receiver102to detect the current position of unmanned aircraft100. The GPS receiver102may optionally include a compass sensor.

Flight controller101econtrols the rotational speed of actuator112of each generator110, thereby controlling the unmanned aircraft100state in accordance with: the current position of unmanned aircraft100detected by position detector101d; the flight speed and flight attitude of unmanned aircraft100obtained from the detection results of acceleration sensor103and gyroscope sensor104; and the operation signal from controller200received by communication IF109. Stated differently, flight controller101eperforms normal control of controlling the aircraft state of unmanned aircraft100in accordance with a user operation to controller200.

In addition to the normal control, flight controller101emay perform sound recording control of controlling the aircraft state of unmanned aircraft100in order to include the sound source direction identified by sound source identifier101cwithin the sound pickup area of at least one of four microphones105, in accordance with the determination result of quality determiner101b. When quality determiner101bdetermines that the quality is low, for example, flight controller101echanges the aircraft state of unmanned aircraft100in the control of the aircraft state in the sound recording control. When quality determiner101bdetermines that the quality is high, for example, flight controller101emaintains the aircraft state of unmanned aircraft100in the control of the aircraft state in the sound recording control.

When changing the aircraft state of unmanned aircraft100in the sound recording control, flight controller101emay control the aircraft state of unmanned aircraft100such that the change in unmanned aircraft states is minimized. In this case, flight controller101emay control the aircraft state of unmanned aircraft100such that the noise from the unmanned aircraft due to the change is minimized.

More specifically, flight controller101eperforms control of rotating unmanned aircraft100about main body140of unmanned aircraft100serving as the central axis in order to include the identified sound source direction within the sound pickup area closest to such sound source direction among the four sound pickup areas of four microphones105. Stated differently, the above sound recording control changes the attitude of unmanned aircraft100by rotating unmanned aircraft100with the minimum amount of change, thereby minimizing the period of time required to change the rotational speeds of rotor blades111of unmanned aircraft100or the amount of change required to change the rotational speeds of rotor blades111. This thus reduces the noise from generators110that is produced until when the inclusion of the sound source direction within the sound pickup area of the corresponding microphone105completes.

Note that information indicating the positions of the sound pickup areas of the respective four microphones105relative to unmanned aircraft100is previously stored in a non-illustrated memory included in unmanned aircraft100. This enables quality determiner101b, which determines an amount of change by which flight controller101erotates unmanned aircraft100to change the attitude of unmanned aircraft100, to determine an amount of change indicating the extent required to rotate unmanned aircraft100to include the sound source direction within the sound pickup area, on the basis of the information on the sound pickup areas read from the memory, and the attitude of unmanned aircraft100obtained, for example, from the various sensors such as acceleration sensor103and gyroscope sensor104.

Note that flight controller101emay perform the sound recording control in the case where four microphones105record the target sound. For example, flight controller101emay stop the normal control to start the sound recording control when four microphones105start recording the target sound, and may stop the sound recording control to start the normal control when four microphones105finish recording the target sound.

The sound recording control may be performed in the case where four microphones105record the target sound. Stated differently, the sound recording control may be control of recording only the target sound or may be control of recording the target sound together with capturing images by camera107.

Video controller101fcontrols gimbal106in accordance with the operation signal received by communication IF109in order to orient the imaging direction of camera107to face the direction indicated by the operation signal, thereby controlling the attitude of camera107. Video controller101fmay also perform a predetermined image process on the image data captured by camera107. Video controller101fmay transmit the image data obtained from camera107or image data that has undergone the predetermined image process to mobile terminal300via communication IF109.

Obstacle detector101gdetects an obstacle around unmanned aircraft100in accordance with the distance, detected by distance sensor108, from unmanned aircraft100to the object. Obstacle detector101gmay exchange information with flight controller101e, thereby detecting an obstacle located at a destination to which unmanned aircraft100is to travel. When detecting an object at the destination to which unmanned aircraft100is to travel, obstacle detector101gmay instruct flight controller101eto cause unmanned aircraft100to avoid the obstacle to travel.

The following describes the operation performed by unmanned aircraft100according to the embodiment.

FIG. 5is a flowchart of a first exemplary operation in sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 6Ais a diagram that illustrates a first scene in the first exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 6Bis a diagram that illustrates a second scene in the first exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 6Cis a diagram that illustrates a third scene in the first exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment. Note thatFIG. 6AtoFIG. 6Care diagrams that illustrate operations performed by unmanned aircraft100in a top view.

As shown inFIG. 5, quality determiner101bof unmanned aircraft100determines the quality of the target sound included in each of the four pieces of sound data generated by four microphones105when sound pickup processing unit101astarts the sound recording control (S1). Sound pickup processing unit101astarts sound recording when, for example, an operation signal received from controller200includes a signal indicating that sound recording is to start.

Next, quality determiner101bdetermines whether to change the attitude of unmanned aircraft100, in accordance with the determined quality of the target sounds (S2). More specifically, when determining that the quality of all of the target sounds is low, quality determiner101bdetermines that the attitude of unmanned aircraft100is to be changed (Yes in step S2), and goes on to step S3. When determining that the quality of any one of the target sounds is high, quality determiner101bdetermines that the attitude of unmanned aircraft100is to be maintained without change (No in step S2), and goes on to step S7.

For example, when none of sound pickup areas A1through A4of four microphones105of unmanned aircraft100faces the direction toward sound source400, as shown inFIG. 6A, the ratio of noise picked up from generators110of unmanned aircraft100becomes large, and thus the quality of all the target sounds are determined as being low.

Sound source identifier101cidentifies the sound source direction, which is a direction from unnamed aircraft100to the sound source of the target sound, by use of the sound data generated by each of four microphones105(S3). Through this process, the sound source direction is identified as, for example, shown by a hollow arrow illustrated inFIG. 6B.

Subsequently, quality determiner101bcompares the sound source direction identified by sound source identifier101cwith four sound pickup areas A1through A4of four microphones105(S4). Through this process, quality determiner101bidentifies, for example, that the identified sound source direction shown inFIG. 6Bis closest to sound pickup area A1among four sound pickup areas A1through A4.

Next, quality determiner101bdetermines an amount of change required to change the attitude of unmanned aircraft100in order to include the sound source direction identified by sound source identifier101cwithin identified sound pickup area A1(S5). In so doing, quality determiner101bdetermines a direction of rotation in which the attitude of unmanned aircraft100is to be changed, together with the amount of change.

Then, flight controller101erotates unmanned aircraft100in accordance with the amount of change and direction of rotation determined by quality determiner101b, thereby changing the attitude of unmanned aircraft100(56). More specifically, flight controller101echanges the rotational speeds of actuators112of four generators110to change the attitude of unmanned aircraft100in accordance with the determined amount of change and direction of rotation. Through this process, as shown inFIG. 6C, for example, the aircraft state of unmanned aircraft100is controlled in order to include the sound source direction within sound pickup area A1, as a result of which unmanned aircraft100rotates clockwise. Note that flight controller101eis simply required to rotate unmanned aircraft100at the minimum amount of rotation that enables the inclusion of sound pickup area A1within the sound source direction, and thus not required to align the sound source direction at the center of sound pickup area A1. Stated differently, flight controller101eis simply required to rotate unmanned aircraft100so that an end part of sound pickup area A1at the sound source direction side is oriented to face the sound source direction.

Sound pickup processing unit101adetermines whether to stop the sound recording (S7). When determining that the sound recording is to be stopped (Yes in S7), sound pickup processing unit101astops the sound recording control. When sound pickup processing unit101adetermines that the sound recording is not to be stopped (No in S7), quality determiner101bmakes the same determination made in step S1again.

Sound pickup processing unit101adetermines that the sound recording is to be stopped when, for example, the operation signal received from controller200includes a signal indicating that the sound recording is to stop, and determines that the sound recording is to continue without stopping when, for example, the operation signal does not include a signal indicating that the sound recording is to stop.

Instead of the first exemplary operation, the second exemplary operation described below may be performed in the sound recording control of the unmanned aircraft.

FIG. 7is a flowchart of the second exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 8Ais a diagram that illustrates a first scene in the second exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 8Bis a diagram that illustrates a second scene in the second exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 8Cis a diagram that illustrates a third scene in the second exemplary operation in the sound recording control performed by the unmanned aircraft according to the embodiment. Note thatFIG. 8AtoFIG. 8Care diagrams that illustrate operations performed by unmanned aircraft100in a top view.

As shown inFIG. 7, quality determiner101bof unmanned aircraft100determines the quality of the target sound included in each of the four pieces of sound data generated by four microphones105when sound pickup processing unit101astarts the sound recording control (S11). Sound pickup processing unit101astarts sound recording when, for example, an operation signal received from controller200includes a signal indicating that sound recording is to start.

Next, quality determiner101bdetermines whether to change the position of unmanned aircraft100, in accordance with the determined quality of the target sounds (S12). More specifically, when determining that the quality of all of the target sounds is low, quality determiner101bdetermines that the position of unmanned aircraft100is to be changed (Yes in step S12), and goes on to step S13. When determining that the quality of any one of the target sounds is high, quality determiner101bdetermines that the position of unmanned aircraft100is to be maintained without change (No in step S12), and goes on to step S21.

Sound source identifier101cidentifies a relative position that includes the sound source direction, which is a direction from unnamed aircraft100to the sound source of the target sound, and the distance to the sound source, by use of the sound data generated by each of four microphones105, or the image data generated by the image sensor of camera107(S13). Through this process, for example, the sound source direction is identified as shown by a hollow arrow illustrated inFIG. 8A, and the distance to sound source400is identified as shown by a broken line arrow illustrated inFIG. 8B. Note that the distance to sound source400is a distance on a three-dimensional space.

Subsequently, quality determiner101bcompares the relative position, identified by sound source identifier101c, including the sound source direction and the distance to sound source400with four sound pickup areas A1through A4of four microphones105(S14). Through this process, quality determiner101bdetermines, for example, that the identified sound source direction shown inFIG. 8Bis included in sound pickup area A1among four sound pickup areas A1through A4, and that unmanned aircraft100is at the position that is away from sound source400by the distance that exceeds a predetermined distance range. Note that the predetermined distance range may be set in accordance with the extent to which four sound pickup areas A1through A4are distanced from unmanned aircraft100.

Next, obstacle detector101gdetermines whether any obstacle exists in the sound source direction when unmanned aircraft100travels by the distance to the identified sound source400(S15). More specifically, obstacle detector101gdetermines whether any obstacle exists between unmanned aircraft100and sound source400, in accordance with the distance, detected by distance sensor108, from unmanned aircraft100to the object.

When determining that an obstacle exists (Yes in S15), obstacle detector101gdetermines whether the route of unmanned aircraft100to sound source400is changeable (S16).

When obstacle detector101gdetermines that no obstacle exists (No in S15) or determines that the route is changeable (Yes in S16), quality determiner101bdetermines an amount of change required to change the position of unmanned aircraft100in order to include sound source400within sound pickup area A1in the shortest travel distance, in accordance with the route of unmanned aircraft100(S17).

Then, flight controller101ecauses unnamed aircraft100to travel in accordance with the determined amount of change required to change the position, thereby changing the position of unmanned aircraft100(S18). Through these processes, for example, unmanned aircraft100is moved to the position at which sound pickup area A1includes sound source400, as shown inFIG. 8C.

When obstacle detector101gdetermines that the route is unchangeable (No in S16), quality determiner101bdetermines an amount of change required to change the attitude of unmanned aircraft100(S19). As in the case shown inFIG. 8A, for example, the sound source direction is included in sound pickup area A1, and thus the amount of change required to change the attitude is determined to be zero. Note that when the sound source direction is included in none of sound pickup areas A1through A4as shown inFIG. 6A, for example, an amount of change required to change the attitude of unmanned aircraft100and a direction of rotation in which the attitude of unmanned aircraft100is to be changed are determined as in step S5in the first exemplary operation.

Then, flight controller101erotates unmanned aircraft100in accordance with the determined amount of change and direction of rotation required to change the attitude of unmanned aircraft100, thereby changing the attitude of unmanned aircraft100(S20).

Sound pickup processing unit101adetermines whether to stop the sound recording (S21). When determining that the sound recording is to be stopped (Yes in S21), sound pickup processing unit101astops the sound recording control. When sound pickup processing unit101adetermines that the sound recording is not to be stopped (No in S21), quality determiner101bmakes the same determination made in step S11again.

FIG. 9is a flowchart of a first exemplary operation of control of adjusting the imaging direction of the camera in the sound recording control performed by the unmanned aircraft according to the embodiment. The control of adjusting the imaging direction of camera107starts upon the start of the sound recording control.

As shown inFIG. 9, video controller101fof unmanned aircraft100determines whether the attitude of unmanned aircraft100is to be changed (S31).

When determining that the attitude of unmanned aircraft100is to be changed (Yes in S31), video controller101fcontrols the orientation of gimbal106in accordance with the change in the attitude of unmanned aircraft100(S32). More specifically, video controller101frotates gimbal106in the direction that is opposite to the direction in which unmanned aircraft100rotates, at the rotational speed at which unmanned aircraft100rotates. Through this process, video controller101fmaintains the imaging direction of camera107to a constant orientation even before and after the change in the attitude of unmanned aircraft100.

Video controller101fdetermines whether to stop the sound recording (S33). When determining that the sound recording is to stop (Yes in S33), video controller101fstops controlling gimbal106. When determining that the attitude of unmanned aircraft100is not to be changed (No in S31) or that the sound recording is not to stop (No in S33), video controller101fmakes that same determination made in step S31again.

The control in a second exemplary operation described below may be performed as the control of adjusting the imaging direction of camera107, together with the control in the first exemplary operation.

FIG. 10is a flowchart of a second exemplary operation of control of adjusting the imaging direction of the camera in the sound recording control performed by the unmanned aircraft according to the embodiment.FIG. 11Ais a diagram that illustrates a first scene in the second exemplary operation of control of adjusting the imaging direction of the camera included in the unmanned aircraft according to the embodiment.FIG. 11Bis a diagram that illustrates a second scene in the second exemplary operation of control of adjusting the imaging direction of the camera included in the unmanned aircraft according to the embodiment.FIG. 12Ais a diagram that illustrates the first scene in the second exemplary operation of control of adjusting the imaging direction of the camera included in the unmanned aircraft according to the embodiment.FIG. 12Bis a diagram that illustrates the second scene in the second exemplary operation of control of adjusting the imaging direction of the camera included in the unmanned aircraft according to the embodiment. Note thatFIG. 11A,FIG. 11B,FIG. 12A, andFIG. 12Bare diagrams that illustrate operations performed by unmanned aircraft100in a top view.

As shown inFIG. 10, video controller101fof unmanned aircraft100obtains the orientation of gimbal106(S41). Through this process, video controller101fobtains the imaging direction of camera107that is based on the orientation of gimbal106.

Video controller101fthen determines whether the operation signal received by communication IF109includes an operation signal indicating rotation instruction to rotate the imaging direction of camera107(S42).

When determining that the operation signal indicating rotation instruction to rotate the imaging direction of camera107is included (Yes in S42), quality determiner101bcompares the obtained orientation of gimbal106with sound pickup areas A1through A4of four microphones105(S43).

Quality determiner101bdetermines whether the rotation of the imaging direction of camera107in accordance with the rotation instruction is a rotation within a sound pickup area that includes the current imaging direction of camera107(S44).

When the rotation of the imaging direction of camera107in accordance with the rotation instruction is determined as being a rotation within the sound pickup area that includes the current imaging direction of camera107(Yes in S44), video controller101fchanges the orientation of gimmal106in accordance with the direction of rotation and the rotational speed in accordance with the rotation instruction, thereby changing the imaging angle of camera107(S45). When this is done, flight controller101edoes not perform control of changing the attitude of unmanned aircraft100. For example, as shown inFIG. 11A, when the imaging direction of camera107is within sound pickup area A1and the instructed direction of rotation in accordance with the rotation instruction is within sound pickup area A1, unmanned aircraft100changes the imaging direction of camera107by changing the orientation of gimbal106without changing the attitude of unmanned aircraft100as shown inFIG. 11B.

Note that video controller101fmay change the imaging direction of camera107by changing the orientation of gimbal106without changing the attitude of unmanned aircraft100in accordance with the direction of rotation and the rotational speed indicated by the rotation instruction as described above, when none of sound pickup areas A1through A4includes the imaging direction of camera107that is based on the orientation of gimbal106obtained in step S41, but when the rotation of the imaging direction of camera107in accordance with the rotation instruction is a rotation to within a sound pickup area that includes the current imaging direction of camera107.

Meanwhile, when the rotation of the imaging direction of camera107in accordance with the rotation instruction is determined as a rotation to outside of the sound pickup area that includes the current imaging direction of camera107(No in S44), flight controller101erotates unmanned aircraft100in accordance with the direction of rotation and the rotational speed in accordance with the rotation instruction, thereby changing the attitude of unmanned aircraft100(S46). When this is done, video controller101fdoes not perform control of changing the orientation of gimbal106. For example, as shown inFIG. 12A, when the imaging direction of camera107is within sound pickup area A1and the instructed direction of rotation in accordance with the rotation instruction is outside of sound pickup area A1, the attitude of unmanned aircraft100is changed with the orientation of gimbal106unchanged as shown inFIG. 12B.

Note that flight controller101eand video controller101fmay rotate unmanned aircraft100in the direction of rotation in accordance with the rotation instruction in order to include the sound source direction within one of the four sound pickup areas A1thorough A4that is closest to the sound source direction and may rotate at the same time the orientation of gimbal106in the same direction as the direction of rotation of unmanned aircraft100, when none of sound pickup areas A1through A4includes the imaging direction of camera107that is based on the orientation of gimbal106obtained in step S41, but when the rotation of the imaging direction of camera107in accordance with the rotation instruction is a rotation to outside of the sound pickup area that includes the current imaging direction of camera107. More specifically, flight controller101eand video controller101fchange the attitude of unmanned aircraft100and the orientation of gimbal106so that the total of the rotational speed required to rotate unmanned aircraft100and the rotational speed required to rotate the orientation of gimbal106amounts to the rotational speed that is in accordance with the rotation instruction.

Video controller101fdetermines whether to stop the sound recording after step S45or step S46(S47). When determining that the sound recording is to stop (Yes in S47), video controller101fstops controlling gimbal106. When determining that an operation signal indicating a rotation instruction to rotate the imaging direction of camera107is not included (No in S42), or when determining that the sound recording is not to stop (No in S47), video controller101fperforms the same process performed in step S41again.

In unmanned aircraft100according to the present embodiment, processor101determines the quality of a target sound by use of the sound data generated by microphone105. Processor101identifies a sound source direction from unmanned aircraft100to the sound source of the target sound by use of the sound data generated by microphone105or image data generated by the image sensor of camera107. Processor101controls an unmanned aircraft state that is a state of the unmanned aircraft such that a direction of a sound pickup area is aligned with the sound source direction, in accordance with the determined quality. Unmanned aircraft state includes, but is not limited to, position or altitude. The sound pickup area is a range in which sound pickup quality of microphone105is higher than that of another area.

In this configuration, processor101controls the aircraft state of unmanned aircraft100in order to include the sound source direction within the sound pickup area of microphone105. This configuration thus achieves a relatively larger component of the target sound of the sound data generated by microphone105, and enhances the quality of the target sound.

In unmanned aircraft100, processor101varies the level of changes in the unmanned aircraft state in order to perform the sound recording. Stated differently, processor101changes the aircraft state of the unmanned aircraft with the least possible amount of change, and thus reduces the noise from the unmanned aircraft caused by the control of the aircraft state. For example, since the aircraft state of unmanned aircraft100is changed with the minimum amount of change, the above configuration minimizes the length of time required to change the rotational speeds of rotor blades111included in unmanned aircraft100, or the amount of change required to change the rotational speeds of rotor blades111. This configuration is thus capable of reducing the noise that is produced until when the sound source direction is included within the sound pickup area in which the sound pickup quality of the microphone is higher than that of another area. Also, a minimized change in the rotational speeds achieves a smaller change in noise characteristics. Consequently, the accuracy of noise removal processing is improved. That the noise is hard to be picked up and easy to be removed results in an enhanced quality of the target sound.

In unmanned aircraft100, processor101identifies the sound source direction by use of the sound data generated by microphone105. This configuration enables unmanned aircraft100to identify the sound source direction only by being equipped with microphone105as a sensor.

In unmanned aircraft100, processor101identifies the sound source direction by use of the image data generated by the image sensor. This configuration enables unmanned aircraft100to identify the sound source direction without being affected by the surrounding noise.

In unmanned aircraft100, processor101modifies the unmanned aircraft state when the determined quality is lower than a threshold. This configuration improves the quality of the target sound when the quality of the obtained target sound is low.

In unmanned aircraft100, processor101maintains the unmanned aircraft state when the determined quality is higher than a threshold. Stated differently, processor101does not change the aircraft state of unmanned aircraft100when the quality of the target sound is sufficiently high. This configuration thus prevents the reduction in the quality of the target sound caused by the noise that is produced due to the change in the aircraft state of unmanned aircraft100or by the change in the noise characteristics.

In the above description, processor101of unmanned aircraft100according to the embodiment identifies the sound source direction, and controls the aircraft state of unmanned aircraft100in order to include such sound source direction within a sound pickup area of microphone105, but the present disclosure is not limited to this example. Processor101may alternatively control the aircraft state of unmanned aircraft100in order to, for example, include the imaging direction of the image sensor of camera107within sound pickup area of microphone105.

FIG. 13is a flowchart of an exemplary operation in sound recording control performed by the unmanned aircraft according to Variation 1.

The sound recording control according to Variation 1 is different from the first exemplary operation in the sound recording control performed by unmanned aircraft100according to the embodiment in that step S3aand step S4aare performed instead of step S3and step S4of the first exemplary operation. The following thus describes step S3aand step S4a, and omits the description of the other processes.

When the determination result is “Yes” in step S2, video controller101fof unmanned aircraft100obtains the orientation of gimbal106(S3a). Through this process, video controller101fobtains the imaging direction of camera107that is based on the orientation of gimbal106.

Next, quality determiner101bcompares the imaging direction of camera107obtained by video controller101fwith four sound pickup areas A1through A4of four microphones105(S4a).

Subsequently, step S5through step S7in the first exemplary operation in the sound recording control of unmanned aircraft100according to the embodiment will be performed.

Processor101of unmanned aircraft100according to Variation 1 estimates that the sound source of the target sound is located in the imaging direction of camera107, and controls the aircraft state of unmanned aircraft100accordingly, thereby including the imaging direction within a sound pickup area of microphone105. In many cases, the imaging direction is in the direction in which the sound source of the target sound is located. This achieves a relatively greater component of the target sound of the sound data generated by microphones105, and thus enhanced quality of the target sound.

Unmanned aircraft100according to the above-described embodiment includes four generators110, but the number of generators included in unmanned aircraft100is not limited to four, and thus may be one to three, or five or more.

Unmanned aircraft100according to the above-described embodiment includes four ducts130and main body140that are connected by four arms141as a non-limited example, and thus four ducts130or four arms141may not be included so long as four generators140are connected to main body140. Stated differently, in the unmanned aircraft, main body140may be directly connected to four generators110, or may be directly connected to four ducts140. Also, the unmanned aircraft may not include four ducts130, or more specifically, the lateral sides of four generators110may not be covered.

Unmanned aircraft100according to the above-described embodiment includes four microphones105, but the number of microphones included in unmanned aircraft100is not limited to four, and thus may be one to three, or five or more. When unmanned aircraft100includes a small number of microphones105, the sound source direction may be estimated by rotating the attitude of unmanned aircraft100to obtain plural pieces of sound data at different timings and compare such pieces of sound data. Microphones105are simply required to be arranged on the outer side of unmanned aircraft100, or more specifically, exposed to an exterior space, and thus may be arranged on a side of arms141other than on a side of main body140. Alternatively, microphones105may be arranged away from main body140. For example, microphones105may be arranged at ends or in the middle of stick-like arms, lines like metallic wires, or ropes like threads that are provided to main body140in addition to arms141and that extend in a direction away from main body140.

Unmanned aircraft100according to the above-described embodiment is rotated at the minimum rotation angle or moved in the minimum distance of travel in order to include a previously identified sound source direction within one of sound pickup areas A1through A4. The present disclosure, however, is not limited to this example, and thus unmanned aircraft100may be rotated in order to include the sound source direction within one of sound pickup areas A1through A4by changing the attitude of unmanned aircraft100, with the quality determined by quality determiner101bbeing fed back until such quality exceeds the threshold.

Unmanned aircraft100according to the above-described embodiment starts sound recording when an operation signal received from controller200includes a signal indicating that sound recording is to start as a non-limited example. Unmanned aircraft100may thus start sound recording, for example, when: the sound data obtained by sound pickup processing unit101aincludes a sound recording command indicating that sound recording is to start; when a user's gesture indicating that sound recording should start has been recognized from the analysis of the image data obtained by camera107; or when a user's lip movement to utter words, such as a keyword, indicating that sound recording should start has been recognized.

Also, unmanned aircraft100may autonomously fly in accordance with a previously set program without being operated by controller200.

Controller200operates unmanned aircraft100in accordance with a previously set program without having an interface for operating unmanned aircraft100.

Moreover, in the above embodiment and variations, the structural components may be implemented as dedicated hardware or may be implemented by executing a software program suited to such structural components. Alternatively, the structural components may be implemented by a program executor such as a CPU and a processor reading out and executing the software program recorded in a recording medium such as a hard disk and a semiconductor memory. Here, the software program that enables unmanned aircraft100and the information processing method, etc. according to the embodiment is as described below.

Stated differently, such program causes a computer to execute an information processing method performed by a processor in an unmanned aircraft that includes the processor and a sensor including at least a microphone that generates sound data. Such information processing method includes: determining quality of a target sound by use of the sound data generated by the microphone; identifying a sound source direction from the unmanned aircraft to a sound source of the target sound by use of data generated by the sensor; and controlling an aircraft state that is a state of the unmanned aircraft such that a direction of a sound pickup area is aligned with the sound source direction, in accordance with the determined quality. Unmanned aircraft state includes, but is not limited to, position or altitude. The sound pickup area is a range in which sound pickup quality of the microphone is higher than that of another area.

The unmanned aircraft, the information processing method, and the recording medium according to one or more aspects of the present disclosure have been described on the basis of the embodiment, but the present disclosure is not limited to the such embodiment. The range of one or more aspects of the present disclosure may include variations achieved by making various modifications and alternations to the present disclosure that can be conceived by those skilled in the art without departing from the essence of the present disclosure, and an embodiment achieved by any combination of structural components described in the present specification.

Note that machine learning may be utilized for processes performed by quality determiner101b, sound source identifier101c, obstacle detector101g, and flight controller101e, and for image recognition processing and sound recognition processing. Examples of machine learning include: supervised learning in which an input-output relationship is studied by use of teaching data, which is input information labeled with output information; unsupervised learning in which data structure is built up only from an unlabeled input; semi-supervised learning in which both labeled and unlabeled data are utilized; and reinforcement learning in which feedback (reward) is obtained to an action selected from the result of state observation to study successive actions that enable the obtainment the maximum amount of reward. More specific techniques of machine learning include neural-network learning (including deep learning that utilizes multi-layered neural network), genetic programming, decision tree learning, Bayesian network learning, and support vector machine (SVM) learning. The present disclosure uses one of these example techniques.

Although only an exemplary embodiment of the present disclosure has been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiment without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of the present disclosure.

INDUSTRIAL APPLICABILITY

The present disclosure is applicable for use as an unmanned aircraft, an information processing method, a recording medium, etc. capable of enhancing the quality of the target sound.