Patent Description:
PTL <NUM> discloses a technique for capturing an image of a plurality of moving objects using an autonomous flight robot.

However, the technique of PTL <NUM> is to capture an image of a large number of unspecified intruders or moving objects such as vehicles having intruded into a monitored space using a camera of an autonomous flight robot for the purpose of crime prevention.

When an image of a subject registered in advance is captured by an unmanned aerial vehicle, it may be necessary to capture an image by using information of the subject registered in advance.

An object of the present invention is to provide a technology capable of capturing an image of a subject registered in advance by an unmanned aerial vehicle.

<CIT> discloses an imaging system for dispatching a drone to a user according to a request from the user, to take an aerial photograph of the user. The imaging system includes: a drone <NUM> which has a camera <NUM> loaded thereon; and an imaging control device <NUM> which controls the drone <NUM> according to operation information from a user terminal <NUM>. On receiving from the user terminal <NUM> the position information of the user terminal <NUM> and the face image of the user, the imaging control device <NUM> moves the drone <NUM> to a place corresponding to the position information, and collates the image captured by the camera <NUM> with the face image to identify a user U from among the captured image.

<CIT> discloses a logistics system related to package delivery, which associates authentication information on a user who is to leave or receive a package with the package and records the associated authentication information into storage means and which instructs an unmanned aerial vehicle into which the package is to be loaded to move to a location, based on location information on a location of a collection destination of the package or a location of a delivery destination of the package.

An image capturing system according to a first aspect of the present invention is an image capturing system including: an unmanned aerial vehicle including an image capturing unit capable of capturing an image of a subject in a flight state; a plurality of vehicles for a plurality of users constituting a subject to travel on, each vehicle of the plurality of vehicles including a communication device; and a control device capable of communicating via a network with a terminal of the subject, the unmanned aerial vehicle and the communication devices, wherein
the control device includes:.

The image capturing system according to a second aspect of the present invention is characterized in that,.

The image capturing system according to a third aspect of the present invention is characterized in that.

The image capturing system according to a fourth aspect of the present invention is characterized in that,.

The image capturing system according to a fifth aspect of the present invention is characterized in that,.

The image capturing system according to a sixth aspect of the present invention is characterized in that
the control device further includes:.

The image capturing system according to a seventh aspect of the present invention is characterized in that,.

The image capturing system according to an eighth aspect of the present invention is characterized in that.

The image capturing system according to a ninth aspect of the present invention is characterized in that.

The image capturing system according to a tenth aspect of the present invention is characterized in that
the plurality of vehicles each further includes:.

The image capturing system according to an eleventh aspect of the present invention is characterized in that
the control device further includes:.

The image capturing system according to a twelfth aspect of the present invention is characterized in that,.

A control method according to a thirteenth aspect of the present invention is a control method in an image capturing system comprising a control device capable of communicating via a network with an unmanned aerial vehicle including an image capturing unit capable of capturing an image of a subject in a flight state, the control device further capable of communicating via a network with a terminal of the subject and communication devices of a plurality of vehicles for a plurality of users constituting the subject to travel on, the method comprising:.

A program according to a fourteenth aspect of the present invention is a program causing a computer to execute each step of a control method in an image capturing system comprising a control device capable of communicating via a network with an unmanned aerial vehicle including an image capturing unit capable of capturing an image of a subject in a flight state, the control device further capable of communicating via a network with a terminal of the subject and communication devices of a plurality of vehicles for a plurality of users constituting the subject to travel on, wherein
the control method includes:.

A storage medium according to a fifteenth aspect of the present invention is a computer-readable storage medium storing a program causing a computer to execute each step of a control method in an image capturing system comprising a control device capable of communicating via a network with an unmanned aerial vehicle including an image capturing unit capable of capturing an image of a subject in a flight state, the control device further capable of communicating via a network with a terminal of the subject and communication devices of a plurality of vehicles for a plurality of users constituting the subject to travel on, wherein
the control method includes:.

A first aspect of the invention relates to an imaging capturing apparatus as defined by claim <NUM>. Other aspects of the invention relate to a control method as defined by claim <NUM>, a program as defined by claim <NUM> and a computer-readable storage medium as defined by claim <NUM>.

Advantageous embodiments of the first aspect are further defined by dependent claims <NUM>-<NUM>.

<FIG> is a diagram illustrating an example of the configuration of an image capturing system STM according to a first embodiment. The image capturing system STM includes an unmanned aerial vehicle DRN and a control device CNT (control server) capable of communicating with a terminal of a subject and the unmanned aerial vehicle DRN. The unmanned aerial vehicle DRN includes an image capturing unit <NUM> (camera) capable of capturing images of a plurality of vehicles in a flight state. The subject includes users of a plurality of vehicles. In the first embodiment, an example in which users of a plurality of vehicles constituting the subject is described as a group of targets to be captured will be described.

The control device CNT can remotely communicate with a plurality of vehicles 1A and 1B and the unmanned aerial vehicle DRN via a network NT, and can output a signal for controlling the unmanned aerial vehicle DRN via the network NT.

An information processing apparatus <NUM> is an external terminal that manages vehicle renting (vehicle use service). When a vehicle is rented to a user, user information (e.g., including terminal information of the user) for identifying the user and vehicle information for identifying the rented vehicle are transmitted to the control device CNT via the network NT. The information processing apparatus <NUM> (external terminal) can be installed, for example, in an external base (agency) such as a hotel, a rental-car vendor, or a dealer that provides sales or maintenance service of vehicles.

The external base (agency) can provide service of capturing an image of, by using the unmanned aerial vehicle DRN, users A and B traveling in the vehicles 1A and 1B using the plurality of users A and B who receive vehicle rental as a group GR.

When acquiring information (user information (including terminal information of the user) and vehicle information) transmitted from the information processing apparatus <NUM> (external terminal) via a communication interface unit <NUM> (communication I/F), the control device CNT registers, in a database DB of a storage unit <NUM>, subject information in which a subject is set as a target to be captured. When the users of the plurality of vehicles constituting the subject are set as the group of targets to be captured, the control device CNT registers, in the database DB of the storage unit <NUM>, group information in which the users A and B of the plurality of vehicles 1A and 1B are set as one group GR. Then, when a plurality of vehicles set as the group are traveling in a predetermined image capturing area, a processing unit <NUM> of the control device CNT generates a control signal for controlling the image capturing unit <NUM> of the unmanned aerial vehicle DRN, and transmits the control signal to the unmanned aerial vehicle DRN via the network NT.

The image capturing unit <NUM> of the unmanned aerial vehicle DRN can capture an image on the basis of a control signal transmitted from the control device CNT. The image captured by the image capturing unit <NUM> is transmitted to the control device CNT via the network NT and stored in the database DB of the storage unit <NUM>. The captured image can be confirmed (viewed) by using preview display on the information processing apparatus <NUM> (external terminal) or terminals of the plurality of users A and B, for example, mobile terminals SP (e.g., smartphones) at the time of returning the rented vehicle. If the user A or B likes the captured image, the image data can be purchased. When the image data is purchased, the image data may be downloaded to the respective mobile terminals SP (smartphones) of the plurality of users A and B, or the image data can be stored in a storage medium such as a compact disc-read only memory (CD-ROM) or a digital versatile disc (DVD) to provide to the user. Specific functional configurations of the processing unit <NUM> of the control device CNT and the unmanned aerial vehicle DRN will be described in detail later.

In the embodiment, as an example of the plurality of vehicles, an electric motorcycle such as a straddle type vehicle can be used for the vehicles 1A and 1B, for example. Note that the straddle type vehicle refers to a type of vehicle in which a driver rides on a vehicle body, and the concept thereof includes a scooter type motorcycle and the like.

The vehicle A and the vehicle B have similar configurations, and the configuration of the vehicle 1A will be described as a representative in the following description. Although <FIG> illustrates an example of two vehicles 1A and 1B as an example of a plurality of vehicles, the present invention is not limited to this example, and three or more vehicles can form a group.

The vehicle 1A includes a power source <NUM>, a battery <NUM> (power supply device) that supplies electric power to the vehicle, an operation mechanism <NUM>, a vehicle control device <NUM> that controls the vehicle, and a communication device <NUM>. In the embodiment, the power source <NUM> is an electric motor, and the battery <NUM> can supply electric power to the power source <NUM> and each element constituting the vehicle <NUM>. A rechargeable secondary battery is used as the battery <NUM>, for example. Examples thereof include a lead storage battery, a lithium-ion battery, and a nickel-hydrogen battery. The battery <NUM> can be charged by being coupled to a power supply capable of supplying a predetermined voltage via a cable. Alternatively, the battery <NUM> may be replaced with a charged battery at a battery-exchange station provided in the middle of the travel route, and the charged battery <NUM> may be mounted on the vehicle.

The operation mechanism <NUM> is configured to be able to input operation for controlling the power source <NUM>, and outputs a predetermined signal to the vehicle control device <NUM> described later on the basis of, for example, an operation input by a user. Examples of the operation input to the operation mechanism <NUM> include rotational operation using a predetermined key corresponding to the vehicle (ignition key, remote key, or the like), pressing operation using a pressing type switch (a start switch or the like).

The vehicle control device <NUM> is an electronic control unit (ECU) capable of controlling the operation of the entire vehicle 1A, and can transmit and receive signals to and from each component of the vehicle 1A via a predetermined signal line, for example. As an example, the vehicle control device <NUM> can receive a signal in accordance with an operation input to the operation mechanism <NUM> and control such that the power source <NUM> is started.

The function of the vehicle control device <NUM> can be implemented by any of software or hardware. For example, the function of the vehicle control device <NUM> may be implemented by a central processing unit (CPU) executing a predetermined program using a memory. Alternatively, the function of the vehicle control device <NUM> may be implemented by a publicly known semiconductor device such as a programmable logic device (PLD) or an application specific integrated circuit (ASIC). In addition, here, the vehicle control device <NUM> is described as a single element, but the vehicle control device <NUM> may be divided into two or more elements as necessary.

The communication device <NUM> includes an antenna for implementing communication with the control device CNT via the network NT. Furthermore, the communication device <NUM> includes a telematics control unit (TCU) or the like that performs signal processing for implementing communication with the control device CNT via the network NT.

The TCU can acquire voltage information indicating a voltage value of the battery <NUM> from the battery <NUM>, and the TCU can acquire control information indicating a control state of the vehicle <NUM> from the vehicle control device <NUM> (ECU). The TCU transmits, to the control device CNT via the network NT, the acquired voltage information of the battery <NUM> and control information of the vehicle control device <NUM> (ECU). In addition, the TCU can intervene in vehicle control in the vehicle control device <NUM> on the basis of information received from the control device CNT.

The communication device <NUM> can perform inter-vehicle communication between a plurality of vehicles constituting the group GR, and the communication device <NUM> of the vehicle 1A can perform wireless communication with the other vehicle 1B constituting the group GR and exchange information between the vehicles. The vehicle control device <NUM> can perform control such that, within the group GR, the speed and the vehicle distance during traveling in the image capturing area are adjusted by inter-vehicle communication between the vehicles.

A detection device <NUM> includes various sensors that detect various states of the vehicle 1A, and examples thereof includes a gyro sensor, a global positioning system (GPS) sensor, and a vehicle speed sensor that detects vehicle speed information. The vehicle control device <NUM> can control the vehicle 1A on the basis of the information detected by the detection device <NUM>, and the communication device <NUM> can transmit the detection result of the detection device <NUM> to the control device CNT via the network NT.

The gyro sensor detects a rotational motion of the vehicle 1A. The vehicle control device <NUM> can determine the course of the vehicle 1A by the detection result of the gyro sensor, the vehicle speed sensor, and the like. The GPS sensor detects the current location of the vehicle 1A. In addition, the communication device <NUM> can communicate wirelessly with a server device that provides map information and traffic information, and acquire information of the current location of the vehicle 1A.

In the embodiment, the communication device <NUM> and the detection device <NUM> function as an acquisition unit that acquires location information of the vehicle, and the communication device <NUM> functions as a vehicle communication unit that transmits the location information of the vehicle via the network NT.

A display device <NUM> is configured to be able to display the remaining battery level of the battery <NUM> and the notification information received from the control device CNT, together with the speedometer and the tachometer. When the notification information regarding the vehicle speed and the inter-vehicle distance between the vehicles in the group GR is received from the control device CNT during traveling in the image capturing area, the display device <NUM> can display the notification information to the user to prompt the user to adjust the vehicle speed and the inter-vehicle distance. This makes it possible to travel in the image capturing area in a state where the vehicle speed and the inter-vehicle distance are adjusted in the group GR as preparation for image capturing when capturing an image.

The control device CNT includes the processing unit <NUM>, the storage unit <NUM>, and the communication interface unit <NUM> (communication I/F), and is installed, for example, in a management company that provides vehicle use service. The processing unit <NUM> includes a processor including a CPU and a memory, and the storage unit <NUM> includes a random-access memory (RAM) serving as a processing area of a program, a ROM that stores various programs and data, and a hard disk drive (HDD) having relatively large capacity. Further, they may be distributed on a cloud.

For example, the processing unit <NUM> can communicate with the vehicles 1A and 1B and the unmanned aerial vehicle DRN via the network NT by the communication interface unit <NUM>, store information on the vehicles 1A and 1B and the unmanned aerial vehicle DRN in the storage unit <NUM>, or read information on the vehicles 1A and 1B and the unmanned aerial vehicle DRN from the storage unit <NUM>. In addition, image data captured by the image capturing unit <NUM> of the unmanned aerial vehicle DRN can be stored in the storage unit <NUM>.

When a vehicle is rented to a user, user information for identifying the user and vehicle information for identifying the rented vehicle are transmitted to the control device CNT via the network NT. The storage unit <NUM> registers subject information in which a subject is set as a target to be captured. Here, the subject information includes terminal information of the user and vehicle information of the vehicle used by the user. Since users of a plurality of vehicles are subjects, the storage unit <NUM> can register group information in which the users of the plurality of vehicles are set as one group GR. In the example of <FIG>, the group GR includes the user A who uses the vehicle 1A and the user B who uses the vehicle 1B. Although the group GR is one group in the example of <FIG>, the storage unit <NUM> can register group information related to a plurality of groups.

Next, a functional configuration of the unmanned aerial vehicle DRN will be described. ST21 in <FIG> is a block diagram illustrating a functional configuration of the unmanned aerial vehicle DRN. The image capturing unit <NUM> is a camera mounted on the unmanned aerial vehicle DRN, and the image capturing unit <NUM> is configured to be able to capture an image of a plurality of vehicles by the unmanned aerial vehicle DRN in a flight state. The image capturing unit <NUM> of the unmanned aerial vehicle DRN can capture a still image or a moving image.

A communication interface unit <NUM> (communication I/F) can communicate with the vehicles 1A and 1B and the control device CNT via the network NT. The communication interface unit <NUM> transmits the image data captured by the image capturing unit <NUM> to the control device CNT.

An identification unit <NUM> could identify the subject on the basis of the subject information distributed from the terminal of the subject (mobile terminal SP) and the subject information transmitted from the control device CNT (communication control unit <NUM>). In the described embodiment it is possible to identify the plurality of vehicles constituting the group GR on the basis of the vehicle information distributed from the plurality of vehicles 1A and 1B and the vehicle information included in the subject information (group information) transmitted from the control device CNT.

When the unmanned aerial vehicle DRN captures an image, group information including vehicle information for identifying the vehicle is transmitted from the communication interface unit <NUM> of the control device CNT to the unmanned aerial vehicle DRN as information for identifying the group GR. In addition, the plurality of vehicles 1A and 1B distribute vehicle information for identifying the vehicles from the communication device <NUM> during traveling, and the identification unit <NUM> can identify the plurality of vehicles constituting the group GR on the basis of the group information transmitted from the communication interface unit <NUM> of the control device CNT by checking the vehicle information.

An image capturing control unit <NUM> controls the image capturing unit <NUM> on the basis of the control signal to control image capturing of the plurality of vehicles identified by the identification unit <NUM>. When receiving the control signal (parameter control signal) transmitted from the control device CNT, the image capturing control unit <NUM> can perform, on the basis of the control signal, control to move the angle of view of the image capturing unit <NUM> in the horizontal direction (pan control), control to move the angle of view of the image capturing unit <NUM> in the vertical direction (tilt control), and control to enlarge (zoom up) or reduce (zoom out) the angle of view and capture an image.

A rotor <NUM> rotates using a motor <NUM> as a drive source, and generates propulsive force of the unmanned aerial object DRN. In order to control the attitude in the flight state, the unmanned aerial vehicle DRN is provided with at least four rotors <NUM> and motors <NUM>, and a flight control unit <NUM> can control the output of each motor <NUM>. The flight control unit <NUM> can turn to change the flight position or change the flight altitude on the basis of the control signal transmitted from the control device CNT.

A sensor <NUM> is, for example, a distance sensor, and detects a distance between the plurality of vehicles identified by the identification unit <NUM> and the unmanned aerial vehicle DRN.

Next, a specific functional configuration of the processing unit <NUM> of the control device CNT will be described. ST22 in <FIG> is a block diagram illustrating a functional configuration of the processing unit <NUM>. <FIG> is a diagram schematically illustrating processing of a determination unit <NUM>.

The determination unit <NUM> can perform various types of determination processing, and determines whether the subject is present in the predetermined image capturing area on the basis of the location information of the subject acquired by communication with the terminal of the subject and the map information. For example, when users of a plurality of vehicles are subjects, the determination unit <NUM> determines whether the plurality of vehicles 1Aand 1B are traveling in the predetermined image capturing area on the basis of the map information and the location information of the plurality of vehicles 1A and 1B set as the group GR. The determination unit <NUM> can access the database of the map information constructed in the storage unit <NUM>, and the determination unit <NUM> compares the location information of the plurality of vehicles with the map information to determine whether or not the vehicle is traveling in the set image capturing area.

For example, as illustrated in ST31 of <FIG>, in a case where the determination unit <NUM> determines that at least one (vehicle 1A) of the plurality of vehicles 1A and 1B set as the group GR has entered the image capturing area, for example, when the image capturing area is smaller (has a smaller distance) than the reference area, or when the inter-vehicle distance between the vehicles 1A and 1B in the group GR is large, the determination unit <NUM> determines that the plurality of vehicles 1A and 1B are traveling in the image capturing area.

Further, as illustrated in ST32 of <FIG>, in a case where all of the plurality of vehicles 1A and 1B set as the group GR enter the image capturing area, for example, when the image capturing area is larger (has a greater distance) than the reference area, the determination unit <NUM> determines that the plurality of vehicles is traveling in the image capturing area.

A signal generation unit <NUM> can generate various signals on the basis of the determination of the determination unit <NUM>, and for example, generates a control signal for controlling the image capturing unit <NUM> of the unmanned aerial vehicle DRN on the basis of the determination of the determination unit <NUM>.

When the determination unit <NUM> determines that the plurality of vehicles is traveling in the image capturing area in a state where image capturing is not started, the signal generation unit <NUM> generates a control signal for instructing the start of image capturing.

In addition, when the determination unit <NUM> determines that at least one of the plurality of vehicles is traveling in the image capturing area after image capturing is started, the signal generation unit <NUM> generates a control signal for instructing continuation of image capturing.

Then, as illustrated in ST33 of <FIG>, when the determination unit <NUM> determines that all of the plurality of vehicles 1A and 1B have left the image capturing area, the signal generation unit <NUM> generates a control signal for instructing end of image capturing.

The communication control unit <NUM> can transmit the signal generated by the signal generation unit <NUM> via the communication interface unit <NUM>, and for example, transmits the control signal generated by the signal generation unit <NUM> to the unmanned aerial vehicle DRN via the communication interface unit <NUM>.

When image capturing is started, the communication control unit <NUM> transmits subject information and the control signal to the unmanned aerial vehicle DRN. When users of a plurality of vehicles are subjects, the storage unit <NUM> registers, as part of the subject information, group information in which users of the plurality of vehicles constituting the subjects are set as a group of targets to be captured. In this case, the communication control unit <NUM> transmits subject information (including group information) and the control signal to the unmanned aerial vehicle DRN. In this way, the identification unit <NUM> of the unmanned aerial vehicle DRN can identify the plurality of vehicles constituting the group GR on the basis of the group information, and images of the plurality of identified vehicles can be captured by the image capturing unit <NUM>.

Note that, when the determination unit <NUM> determines that at least one of the plurality of vehicles has entered the image capturing area or the predetermined preparation area set before the image capturing area, the signal generation unit <NUM> can generate an area notification signal for notifying the user that the vehicle has entered the image capturing area or the preparation area, and the communication control unit <NUM> can transmit the area notification signal to the plurality of vehicles.

Traveling in preparation for image capturing can be performed, such as aligning a platoon of vehicles in the group GR, by notifying the user of the area notification signal before image capturing.

When continuing image capturing or ending image capturing, the communication control unit <NUM> transmits, to the unmanned aerial vehicle DRN, a control signal for instructing continuation of image capturing or a control signal for instructing end of image capturing, both of which have been generated by the signal generation unit <NUM>.

The image capturing control unit <NUM> of the unmanned aerial vehicle DRN controls the image capturing unit <NUM> on the basis of a control signal for instructing continuation of image capturing to continue image capturing. In addition, the image capturing control unit <NUM> of the unmanned aerial vehicle DRN controls the image capturing unit <NUM> on the basis of a control signal for instructing end of image capturing to end image capturing.

An image processing unit <NUM> can perform image processing of extracting a face of a user from image data captured by the image capturing unit <NUM> of the unmanned aerial vehicle DRN. When the image data is a moving image, the image processing unit <NUM> performs image processing on an image of each frame. Furthermore, the image processing unit <NUM> can also perform image processing on an image sampled at a predetermined frame rate.

On the basis of the result of the image processing and the group information, an image determination unit <NUM> determines whether or not images of the faces of the users have been captured equivalent to the number of persons set as the group GR. For example, when the faces of the two users A and B set as the group GR are extracted by the image processing, that is, when the set number (N1) of the group GR is equal to the number of extraction (N2) extracted by the image processing (N1 = N2), the image determination unit <NUM> determines that images of the faces of all the users constituting the group GR have been captured.

On the other hand, when the number of faces of the user extracted in the result of the image processing is one, that is, when the set number (N1 = <NUM>) of the groups GR is not equal to the number of extraction (N2 = <NUM>) extracted by the image processing (in this case, N1 > N2), the image determination unit <NUM> determines that images of the faces of the users has not been captured equivalent to the number of persons set as the group GR.

A backlight determination unit <NUM> determines whether or not it is in a backlight condition on the basis of the image data captured by the image capturing unit <NUM>. For example, when the image data captured by the image capturing unit <NUM> includes a region where the pixel value locally exceeds the reference pixel value, the backlight determination unit <NUM> can determine that such an image has been captured in the backlight image capturing condition. In this case, the signal generation unit <NUM> generates a flight control signal for instructing a change in the flight position of the unmanned aerial vehicle DRN so as to avoid backlight, or generates a control signal for changing the angle of view of the image capturing unit <NUM>. A specific processing will be described in the additional processing regarding the backlight determination after step S570 in <FIG>.

<FIG> is a diagram for explaining the flow of processing of the storage unit <NUM> and the processing unit <NUM> (determination unit <NUM>, signal generation unit <NUM>, and communication control unit <NUM>).

In step S400, the storage unit <NUM> registers information of the group GR. When the control device CNT acquires information (user information and vehicle information) transmitted from the information processing apparatus <NUM> (external terminal) via the communication interface unit <NUM>, the storage unit <NUM> registers, in the database DB, group information in which the users A and B of the plurality of vehicles 1A and 1B are set as one group GR.

In step S405, the determination unit <NUM> acquires map information from the database of map information constructed in the storage unit <NUM>.

In step S410, the determination unit <NUM> acquires the location information of the plurality of vehicles. In step S415, the determination unit <NUM> determines whether the plurality of vehicles 1A and 1B are traveling in a predetermined image capturing area on the basis of the location information of the plurality of vehicles 1A and 1B set as the group GR and the map information.

When the vehicles are not traveling in the image capturing area in the determination in step S415 (NO in S415), the determination unit <NUM> returns the processing to step S410 and repeats the same processing.

On the other hand, when the vehicle is traveling in the image capturing area in the determination in step S415 (YES in S415), the process proceeds to step S420.

In step S420, the signal generation unit <NUM> generates a control signal for instructing start of image capturing, as a control signal for controlling the image capturing unit <NUM> of the unmanned aerial vehicle DRN on the basis of the determination of the determination unit <NUM>.

Then, in step S425, the communication control unit <NUM> transmits the information of the registered group GR and the generated control signal to the unmanned aerial vehicle DRN.

The identification unit <NUM> of the unmanned aerial vehicle DRN identifies the plurality of vehicles constituting the group GR on the basis of the group information, and images of the plurality of identified vehicles are captured by the image capturing unit <NUM>.

In step S430, the determination unit <NUM> determines whether all of the plurality of vehicles 1A and 1B have left the image capturing area. When all of the vehicles have not left the image capturing area (NO in S430), that is, when the determination unit <NUM> determines that at least one of the plurality of vehicles is traveling in the image capturing area, the processing proceeds to step S435.

In step S435, the signal generation unit <NUM> generates a control signal for instructing continuation of image capturing. In step S440, the communication control unit <NUM> transmits, to the unmanned aerial vehicle DRN, a control signal for instructing continuation of image capturing, which has been generated by the signal generation unit <NUM>. When receiving the control signal for instructing the continuation of image capturing, the image capturing control unit <NUM> of the unmanned aerial vehicle DRN controls the image capturing unit <NUM> on the basis of a control signal for instructing continuation of image capturing to continue image capturing.

On the other hand, when the determination unit <NUM> determines that all of the plurality of vehicles 1A and 1B have left the image capturing area in the determination of step S430 (YES in S430), the signal generation unit <NUM> generates, in step S445, a control signal for instructing end of image capturing.

Then, in step S450, the communication control unit <NUM> transmits, to the unmanned aerial vehicle DRN, the control signal for instructing end of image capturing, which has been generated by the signal generation unit <NUM>. When receiving the control signal for instructing the end of image capturing, the image capturing control unit <NUM> of the unmanned aerial vehicle DRN controls the image capturing unit <NUM> on the basis of a control signal for instructing end of image capturing to end image capturing.

<FIG> is a diagram describing a flow of processing of the image processing unit <NUM> and the image determination unit <NUM>. In step S500, the image processing unit <NUM> acquires image data captured by the image capturing unit <NUM> of the unmanned aerial vehicle DRN.

In step S510, the image processing unit <NUM> performs image processing of extracting the face of the user from the image data.

In step S520, on the basis of the result of the image processing acquired in step S510 and the group information registered in advance, the image determination unit <NUM> performs image determination on whether or not images of the faces of the users have been captured equivalent to the number of persons set as the group GR.

When images of the faces of the users have been captured equivalent to the number of persons set as the group in the determination in step S520 (YES in S520), the process proceeds to step S530.

Then, in step S530, the storage unit <NUM> stores the captured image data in the database, and ends the processing.

The image data stored in the database can be provided to the preview display on the information processing apparatus <NUM> (external terminal) or the mobile terminals SP of the plurality of users A and B when the rented vehicle is returned. When the user A or B likes the captured image, such a user can purchase the image data. In this case, such a user can download the image data to the respective mobile terminals SP (smartphones) of the plurality of users A and B. It is also possible to store image data in a storage medium and provide the image data to the user.

On the other hand, when images of the faces of the users have not been captured equivalent to the number of persons set as the group in the determination in step S520 (NO in S520), the process proceeds to step S540.

In step S540, the signal generation unit <NUM> generates a parameter control signal for controlling the image capturing parameter. When images of the faces of the users have not been captured equivalent to the number of persons set as the group, the signal generation unit <NUM> generates a parameter control signal for controlling the image capturing parameters of the image capturing unit <NUM> such that images of the faces of the users (all the users) can be captured equivalent to the number of persons set as the group. The signal generation unit <NUM> can generate a parameter control signal for moving the angle of view of the image capturing unit <NUM> in the horizontal direction as the image capturing parameter for the pan control, or generate a parameter control signal for moving the angle of view of the image capturing unit <NUM> in the vertical direction as the image capturing parameter for the tilt control. Furthermore, a parameter control signal for capturing an image by enlarging (zooming up) or reducing (zooming out) the angle of view can be generated.

In step S550, the communication control unit <NUM> transmits the parameter control signal generated by the signal generation unit <NUM> to the unmanned aerial vehicle DRN. When receiving the parameter control signal, the image capturing control unit <NUM> of the unmanned aerial vehicle DRN controls the image capturing unit <NUM> on the basis of the parameter control signal to capture an image while moving using the unmanned aerial vehicle DRN. Images of the faces of all members in the group can be captured by controlling the angle of view of the image capturing unit <NUM> on the basis of the parameter control signal.

In step S560, the signal generation unit <NUM> generates an image capturing guide signal for the redoing of image capturing. When images of the faces of the users A and B have not been captured equivalent to the set number of persons in the determination of the image determination unit <NUM>, the signal generation unit <NUM> generates an image capturing guide signal for guiding the users A and B to redo image capturing.

Then, in step S570, the communication control unit <NUM> transmits the image capturing guide signal to the plurality of vehicles 1A and 1B. When receiving the image capturing guide signal, the display device <NUM> of each vehicle presents a display based on the image capturing guide signal to the user, and guides the user to capture an image again. In this way, even when images of the faces of all the users could not be captured, an image can be re-captured immediately in the image capturing area by generating the image capturing guide signal and transmitting the image capturing guide signal to the plurality of vehicles.

After step S570, the processing returns to step S500, and thereafter, similar processing is repeatedly executed.

Note that, after step S570, additional processing regarding backlight determination can be performed. The backlight determination unit <NUM> determines whether or not the image capturing condition is backlight condition on the basis of the image data captured by the image capturing unit <NUM>. For example, when the image data captured by the image capturing unit <NUM> includes a region where the pixel value locally exceeds the reference pixel value, the backlight determination unit <NUM> determines that such an image has been captured in the backlight condition.

When the image capturing condition is determined to be backlight condition, the signal generation unit <NUM> generates a flight control signal for instructing a change in the flight position of the unmanned aerial vehicle DRN so as to avoid backlight. For example, the signal generation unit <NUM> generates a flight control signal for instructing the unmanned aerial vehicle DRN to turn such that the sun does not come within the range of the viewing angle of the image capturing unit.

The communication control unit <NUM> transmits a flight control signal to the unmanned aerial vehicle DRN. When receiving the flight control signal, the flight control unit <NUM> of the unmanned aerial vehicle changes the flight position on the basis of the flight control signal.

Furthermore, when the image capturing condition is determined to be backlight condition, the signal generation unit <NUM> can also generate a control signal (parameter control signal) for performing control to move the angle of view of the image capturing unit <NUM> in the horizontal direction (pan control), or control to move the angle of view of the image capturing unit <NUM> in the vertical direction (tilt control).

The communication control unit <NUM> transmits a parameter control signal to the unmanned aerial vehicle DRN. When receiving the parameter control signal, the image capturing control unit <NUM> of the unmanned aerial vehicle DRN changes the angle of view of the image capturing unit <NUM> on the basis of the parameter control signal.

When the image capturing condition is determined to be backlight condition on the basis of the captured image data, an image can be captured while avoiding backlight by changing the flight position of the unmanned aerial vehicle or changing the angle of view of the image capturing unit <NUM>.

The control device CNT checks the inter-vehicle distance at the time when the plurality of vehicles 1A and 1B travel in the image capturing area or the predetermined preparation area set before the image capturing area, and transmits the notification information to the plurality of vehicles so as to ensure the inter-vehicle distance suitable for image capturing (distance falling within a predetermined reference distance range).

For example, when the inter-vehicle distance between the vehicles exceeds a distance of the upper limit of the predetermined reference distance range, a distance notification signal for notifying that the inter-vehicle distance is too large is transmitted to the plurality of vehicles 1A and 1B to notify the user of each vehicle.

When the inter-vehicle distance between the vehicles is equal to or less than a distance of the lower limit of the predetermined reference distance range, an approach notification signal for notifying that the inter-vehicle distance is too small is transmitted to the plurality of vehicles 1A and 1B to notify the user of each vehicle.

<FIG> is a diagram describing a flow of processing of adjusting an inter-vehicle distance between a plurality of vehicles. In step S600, the determination unit <NUM> acquires location information of the plurality of vehicles 1A and 1B traveling in the image capturing area or the predetermined preparation area set before the image capturing area. In step S610, the determination unit <NUM> acquires the inter-vehicle distance of the plurality of vehicles 1A and 1B on the basis of the location information. For example, the determination unit <NUM> can acquire the inter-vehicle distance on the basis of the difference in the location information.

In step S620, when determining that the acquired inter-vehicle distance exceeds the distance of the upper limit of the predetermined reference distance range (YES in S620), the determination unit <NUM> advances the processing to step S630.

In step S630, the signal generation unit <NUM> generates a distance notification signal for notifying the user that the inter-vehicle distance exceeds the distance of the upper limit of the reference distance range. Then, in step S640, the communication control unit <NUM> transmits the distance notification signal to the plurality of vehicles 1A and 1B. When the inter-vehicle distance is too large, there is a possibility that images of a plurality of users cannot be simultaneously captured during traveling in the image capturing area. Accordingly, it is possible to prompt the user to reduce the inter-vehicle distance by transmitting, to a plurality of vehicles, a distance notification signal for notifying that the inter-vehicle distance exceeds a distance of the upper limit of the reference distance range to notify the users.

In step S620, when the acquired inter-vehicle distance does not exceed the distance of the upper limit of the predetermined reference distance range (NO in S620), the determination unit <NUM> advances the processing to step S650.

In step S650, when the determination unit <NUM> determined that the acquired inter-vehicle distance is not equal to or less than the distance of the lower limit of the reference distance range (NO in S650), the determination unit <NUM> returns the processing to step S600 and repeatedly executes the same processing. In this case, the inter-vehicle distance between the plurality of vehicles 1A and 1B is an inter-vehicle distance suitable for image capturing (a distance falling within a predetermined reference distance range), and check processing of the inter-vehicle distance is continuously executed without generating the notification signals (distance notification signal, approach notification signal).

On the other hand, when the acquired inter-vehicle distance is equal to or less than the distance of the lower limit of the predetermined reference distance range in determination in step S650 (YES in S650), the determination unit <NUM> advances the processing to step S660.

In step S660, the signal generation unit <NUM> generates an approach notification signal for notifying the user that the inter-vehicle distance is equal to or less than the distance of the lower limit of the reference distance range.

Then, in step S670, the communication control unit <NUM> transmits the approach notification signal to the plurality of vehicles. When the inter-vehicle distance is too small, there is a possibility that images of a plurality of users cannot be simultaneously captured due to overlap of users during traveling in the image capturing area. Accordingly, it is possible to prompt the user to increase the inter-vehicle distance by transmitting, to a plurality of vehicles, an approach notification signal for notifying that the inter-vehicle distance is equal to or less than a distance of the lower limit of the reference distance range to notify the users.

The control device CNT checks the speed difference between the vehicles at the time when the plurality of vehicles 1A and 1B travel in the image capturing area or the predetermined preparation area set before the image capturing area, and transmits the speed notification signal to the plurality of vehicles so as to ensure the speed difference suitable for image capturing (equal to or less than predetermined reference speed).

<FIG> is a diagram describing a flow of processing of adjusting speed difference between a plurality of vehicles. In step S700, the determination unit <NUM> acquires speed information of the plurality of vehicles 1A and 1B traveling in the image capturing area or a predetermined preparation area set before the image capturing area. In step S710, the determination unit <NUM> acquires the speed difference between the plurality of vehicles 1A and 1B on the basis of the speed information of the plurality of vehicles 1A and 1B. For example, the determination unit <NUM> can acquire the speed difference between the vehicles on the basis of the difference in the speed information.

In step S720, when the acquired speed difference does not exceed the reference speed (NO in S720), the determination unit <NUM> returns the processing to step S700 and repeatedly executes the same processing. In this case, the speed difference between the plurality of vehicles 1A and 1B is a speed difference suitable for image capturing (equal to or less than predetermined reference speed), and check processing of the speed difference is continuously executed without generating the speed notification signal.

On the other hand, when the speed difference exceeds the predetermined reference speed in determination in step S720 (YES in S720), the determination unit <NUM> advances the processing to step S730.

In step S730, the signal generation unit <NUM> generates a speed notification signal for notifying the user that the speed difference exceeds the reference speed.

Then, in step S740, the communication control unit <NUM> transmits the speed notification signal to the plurality of vehicles 1A and 1B.

Claim 1:
An image capturing system comprising: an unmanned aerial vehicle (DRN) including an image capturing unit (<NUM>) capable of capturing an image of a subject in a flight state; a plurality of vehicles (1A, 1B) for a plurality of users (A, B) constituting a subject to travel on, each vehicle of the plurality of vehicles including a communication device (<NUM>); and a control device (CNT) capable of communicating via a network (NT) with a terminal of the subject, the unmanned aerial vehicle and the communication devices (<NUM>), wherein
the control device includes:
a storage unit (<NUM>) configured to register, as subject information, group information in which users (A, B) of the plurality of vehicles (1A, 1B) constituting the subject are set as a group (GR) of targets to be captured, wherein the group information includes vehicle information of the plurality of vehicles (1A, 1B);
a determination unit (<NUM>) configured to determine whether the subject is travelling in a predetermined image capturing area, on the basis of location information of the subject acquired by communicating with the terminal and map information;
a signal generation unit (<NUM>) configured to generate a control signal that controls the image capturing unit (<NUM>) on the basis of determination of the determination unit (<NUM>); and
a communication control unit (<NUM>) configured to transmit the subject information and the control signal to the unmanned aerial vehicle, and
the unmanned aerial vehicle further includes:
an identification unit (<NUM>) configured to identify the plurality of vehicles used by the group (GR) on the basis of checking the vehicle information distributed from the communication devices (<NUM>) of the plurality of vehicles and the vehicle information included in the group information transmitted from the communication control unit (<NUM>); and
an image capturing control unit (<NUM>) configured to control the image capturing unit (<NUM>) on the basis of the control signal to control image capturing of the subject being users (A,B) of the plurality of vehicles identified by the identification unit (<NUM>).