Imaging plan generation device, imaging plan generation method, and program

Disclosed are an imaging plan generation device, an imaging plan generation method, and a program that generate an imaging plan for efficiently imaging captured images for use in inspection of a bridge without omission. An imaging plan generation device (400) includes a space information acquisition unit (401) that acquires space information of a panel, a first decision unit (403) that decides a plurality of deck slab imaging positions and postures of a camera based on the space information of the panel, a second decision unit (405) that decides, based on the space information of the panel, a plurality of steel member imaging positions and postures of the camera in imaging a plurality of joint portions of steel members with the camera, and an imaging plan generation unit (407) that generates an imaging plan of a camera-equipped mobile robot based on the plurality of deck slab imaging positions and postures decided by the first decision unit and the plurality of steel member imaging positions and postures decided by the second decision unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging plan generation device, an imaging plan generation method, and a program, and in particular, to an imaging plan generation device, an imaging plan generation method, and a program that generate an imaging plan of a camera-equipped mobile robot.

2. Description of the Related Art

In the related art, a technique that images a structure, such as a bridge, with a robot device (camera-equipped mobile robot) with a camera and performs inspection of the structure using the captured image has been suggested.

For example, JP2016-079615A discloses a camera-equipped movable robot that, in a case where a lower surface of a deck slab of a bridge or a steel plate girder is inspected, is attached to the lower surface of the bridge by a suspended carriage and rails.

In the related art, a technique relating to generation of a movement plan of a mobile robot has been suggested.

For example, JP2003-266345A discloses a technique relating to generation of a moving route plan in a case where a robot device is moved to a destination. In the technique described in JP2003-266345A, the robot device is made to observe an unobserved region on the moving route under a given condition, thereby updating an obstacle map for use in generating a moving route plan to generate the moving route plan.

SUMMARY OF THE INVENTION

In general, in a case where a bridge is inspected, inspection is performed on a deck slab and steel members, such as a main girder, constituting the bridge. Then, a camera-equipped mobile robot that performs the inspection of the bridge should acquire captured images for inspection on the deck slab and the steel members constituting the bridge without omission. Since the camera-equipped mobile robot that performs the inspection of the bridge needs to acquire many captured images, the camera-equipped mobile robot needs to efficiently acquire the captured images for the inspection of the deck slab and the steel members.

However, in JP2016-079615A and JP2003-266345A described above, there is no description of an imaging plan relating to the acquisition of the captured images for the inspection of the bridge. That is, in JP2016-079615A and JP2003-266345A described above, there is no description of the imaging plan of the captured images for the inspection of the deck slab and the steel members of the bridge.

The invention has been accomplished in consideration of such a situation, and an object of the invention is to provide an imaging plan generation device, an imaging plan generation method, and a program that generate an imaging plan for efficiently imaging captured images for use in inspection of a bridge without omission.

In order to achieve the above-described object, an aspect of the invention provides an imaging plan generation device that generates an imaging plan of a camera-equipped mobile robot moving a camera within a three-dimensional space and changing a posture of the camera in imaging a deck slab and steel members including a main girder and a cross beam or a cross frame of a bridge as an inspection target. The imaging plan generation device comprises a space information acquisition unit that acquires space information of one panel divided by two facing main girders and two facing cross beams or cross frames, a first decision unit that decides, based on the space information of the panel, a plurality of deck slab imaging positions and postures of the camera in dividing the entire deck slab corresponding to the one panel into a plurality of pieces and imaging the plurality of pieces with the camera, a second decision unit that decides, based on the space information of the panel, a plurality of steel member imaging positions and postures of the camera in imaging at least a plurality of joint portions of the steel members among the steel members corresponding to the one panel with the camera, and an imaging plan generation unit that generates the imaging plan of the camera-equipped mobile robot based on the plurality of deck slab imaging positions and postures decided by the first decision unit and the plurality of steel member imaging positions and postures decided by the second decision unit.

According to the aspect, the plurality of deck slab imaging positions and postures of the camera in dividing the entire deck slab corresponding to the one panel into a plurality of pieces and imaging the plurality of pieces with the camera are decided by the first decision unit based on the space information of the panel. According to the aspect, the plurality of steel member imaging positions and postures of the camera in imaging at least the plurality of joint portions of the steel members among the steel members corresponding to the one panel with the camera are decided by the second decision unit based on the space information of the panel. Then, according to the aspect, since the imaging plan is generated based on the imaging positions and postures decided by the first decision unit and the second decision unit, it is possible to generate an imaging plan capable of efficiently imaging captured images of the deck slab and the steel members constituting the bridge without omission.

Preferably, the space information acquisition unit acquires the space information based on CAD drawing data of the bridge.

According to the aspect, since the space information acquisition unit acquires the space information based on the CAD drawing data of the bridge, the space information acquisition unit can acquire accurate space information.

Preferably, the space information acquisition unit acquires a first distance to the two facing main girders, a second distance to the two facing cross beams or cross frames, and a third distance to the deck slab based on a robot initial position within the three-dimensional space measured by a distance measurement unit, and acquires the space information based on the acquired first distance, second distance, and third distance.

According to the aspect, the first distance to the two facing main girders, the second distance to the two facing cross beams or cross frames, and the third distance to the deck slab based on the robot initial position within the three-dimensional space are measured by the distance measurement unit. Then, according to the aspect, the space information acquisition unit acquires the space information based on the first distance, the second distance, and the third distance acquired by the distance measurement unit. With this, in the aspect, even in a case where the CAD drawing data of the bridge is absent, since the space information is acquired based on distance information measured by the distance measurement unit, it is possible to generate an imaging plan.

Preferably, the imaging plan generation device further comprises a member information acquisition unit that acquires member information as information relating to a member of the panel, the first decision unit decides the deck slab imaging positions and postures based on the space information of the panel and the member information, and the second decision unit decides the steel member imaging positions and postures based on the space information of the panel and the member information.

According to the aspect, the member information as information relating to the member of the panel is acquired by the member information acquisition unit, and the first decision unit and the second decision unit decide the imaging positions and postures using the acquired member information and the space information. With this, in the aspect, even in a case where the members are provided in the bridge, it is possible to generate an imaging plan capable of efficiently imaging captured images for inspection without omission.

Preferably, the imaging plan generation device further comprises a member information acquisition unit that acquires member information as information relating to a member of the panel, the member information being information relating to a fourth distance to a lateral frame based on the robot initial position, a width of the lateral frame, and a shape of the lateral frame, the first decision unit decides the deck slab imaging positions and postures based on the space information of the panel and the member information, and the second decision unit decides the steel member imaging positions and postures based on the space information of the panel and the member information.

According to the aspect, information relating to the fourth distance to the lateral frame based on the robot initial position, the width of the lateral frame, and the shape of the lateral frame is acquired by the member information acquisition unit. With this, in the aspect, even in a case where the members are provided in the bridge, it is possible to efficiently image captured images for inspection without omission.

Preferably, the first decision unit calculates a horizontal plane imaging range as an imaging range in a horizontal plane of the lateral frame using the fourth distance, a focal length of the camera, and a size of an imaging element of the camera, and decides the deck slab imaging positions and postures based on the space information of the panel, the member information, and the horizontal plane imaging range.

According to the aspect, the horizontal plane imaging range as the imaging range in the horizontal plane of the lateral frame is calculated by the first decision unit, and the deck slab imaging positions and postures are decided based on the horizontal plane imaging range, the space information of the panel, and the member information. With this, in the aspect, since an imaging plan taking into consideration the imaging range in the horizontal plane of the lateral frame is generated, it is possible to generate an imaging plan capable of efficiently imaging the deck slab without omission.

Preferably, the first decision unit calculates a deck slab imaging range using the third distance, a focal length of the camera, and a size of an imaging element of the camera, and decides the deck slab imaging positions based on the deck slab imaging range and the space information of the panel.

According to the aspect, since the deck slab imaging range is calculated by the first decision unit, and the deck slab imaging positions are decided based on the calculated imaging range, it is possible to generate an imaging plan capable of efficiently imaging the captured images of the deck slab without omission.

Preferably, the imaging plan generation device further comprises an imaging plan database in which a plurality of existing imaging plans are stored, the first decision unit selects the existing imaging plan from the imaging plan database based on the space information of the panel and decides the deck slab imaging positions based on the selected existing imaging plan, and the second decision unit selects the existing imaging plan from the imaging plan database based on the space information of the panel and decides the steel member imaging positions and postures based on the selected existing imaging plan.

According to the aspect, the imaging plan database in which the plurality of existing imaging plans are stored is provided, and the first decision unit and the second decision unit select the existing imaging plan from the imaging plan database based on the space information and decide the imaging positions and postures based on the selected existing imaging plan. With this, in the aspect, it is possible to efficiently generate an imaging plan based on the existing imaging plan.

Preferably, the imaging plan generation device further comprises a member information acquisition unit that acquires member information as information relating to a member of the panel, and the imaging plan generation unit corrects the deck slab imaging positions and postures decided by the first decision unit and the steel member imaging positions and postures decided by the second decision unit based on the difference between the space information acquired by the space information acquisition unit and space information of the selected imaging plan or the difference between the member information acquired by the space information acquisition unit and member information of the selected imaging plan.

According to the aspect, the imaging positions and postures decided by the first decision unit or the second decision unit are corrected based on the difference between the space information acquired by the space information acquisition unit and the space information of the selected imaging plan or the difference between the member information acquired by the space information acquisition unit and the member information of the selected imaging plan. With this, in the aspect, since the existing imaging plan is corrected, it is possible to generate an efficient imaging plan without omission.

Preferably, the imaging plan generation device further comprises a storage control unit that makes the imaging plan database store the imaging plan generated by the imaging plan generation unit.

According to the aspect, since the generated imaging plan is stored in the imaging plan database by the storage control unit, it is possible to effectively use the generated imaging plan.

Preferably, the imaging plan generation device further comprises an imaging plan adjustment unit that adjusts the deck slab imaging positions and postures or the steel member imaging positions and postures in the imaging plan generated by the imaging plan generation unit based on an adjustment command of the deck slab imaging positions and postures or the steel member imaging positions and postures.

According to the aspect, the deck slab imaging positions and postures or the steel member imaging positions and postures in the imaging plan are adjusted by the imaging plan adjustment unit based on the adjustment command of the deck slab imaging positions and postures or the steel member imaging positions and postures.

Preferably, the imaging plan generation device further comprises an imaging plan addition unit that adds the deck slab imaging positions or the steel member imaging positions to the imaging plan generated by the imaging plan generation unit based on an addition command of the deck slab imaging positions or the steel member imaging positions.

According to the aspect, deck slab imaging positions or the steel member imaging positions are added to the imaging plan generated by the imaging plan generation unit by the imaging plan addition unit based on the addition command. With this, in the aspect, the deck slab imaging positions or the steel member imaging positions are added to the imaging plan based on the addition command.

Preferably, the first decision unit decides the deck slab imaging positions in a case where the camera is made to face the deck slab.

According to the aspect, since the deck slab imaging positions in a case where the camera is made to face the deck slab are decided by the first decision unit, captured images are efficiently acquired without omission.

Another aspect of the invention provides an imaging plan generation method that generates an imaging plan of a camera-equipped mobile robot moving a camera within a three-dimensional space and changing a posture of the camera in imaging a deck slab and steel members including a main girder and a cross beam or a cross frame of a bridge as an inspection target. The imaging plan generation method comprises a space information acquisition step of acquiring space information of one panel divided by two facing main girders and two facing cross beams or cross frames, a first decision step of deciding, based on the space information of the panel, a plurality of deck slab imaging positions and postures of the camera in dividing the entire deck slab corresponding to the one panel into a plurality of pieces and imaging the plurality of pieces with the camera, a second decision step of deciding, based on the space information of the panel, a plurality of steel member imaging positions and postures of the camera in imaging at least a plurality of joint portions of the steel members among the steel members corresponding to the one panel with the camera, and an imaging plan generation step of generating the imaging plan of the camera-equipped mobile robot based on the plurality of deck slab imaging positions and postures decided in the first decision step and the plurality of steel member imaging positions and postures decided in the second decision step.

A further aspect of the invention provides a program that causes a computer to execute an imaging plan generation method of generating an imaging plan of a camera-equipped mobile robot moving a camera within a three-dimensional space and changing a posture of the camera in imaging a deck slab and steel members including a main girder and a cross beam or a cross frame of a bridge as an inspection target. The program causes the computer to execute a space information acquisition step of acquiring space information of one panel divided by two facing main girders and two facing cross beams or cross frames, a first decision step of deciding, based on the space information of the panel, a plurality of deck slab imaging positions and postures of the camera in dividing the entire deck slab corresponding to the one panel into a plurality of pieces and imaging the plurality of pieces with the camera, a second decision step of deciding, based on the space information of the panel, a plurality of steel member imaging positions and postures of the camera in imaging at least a plurality of joint portions of the steel members among the steel members corresponding to the one panel with the camera, and an imaging plan generation step of generating the imaging plan of the camera-equipped mobile robot based on the plurality of deck slab imaging positions and postures decided in the first decision step and the plurality of steel member imaging positions and postures decided in the second decision step.

According to the invention, a plurality of deck slab imaging positions and postures of the camera in dividing the entire deck slab corresponding to one panel into a plurality of pieces and imaging a plurality of pieces with the camera are decided by the first decision unit based on the space information of the panel, a plurality of steel member imaging positions and postures of the camera in imaging at least a plurality of joint portions of the steel members among the steel members corresponding to one panel with the camera are decided by the second decision unit based on the space information of the panel, and the imaging plan is generated based on the imaging positions and postures decided by the first decision unit and the second decision unit. For this reason, it is possible to generate an imaging plan capable of efficiently acquiring the captured images of the deck slab and the steel members constituting the bridge without omission.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, a preferred embodiment of an imaging plan generation device, an imaging plan generation method, and a program according to the invention will be described referring to the accompanying drawings.

FIG. 1is a perspective view showing a structure of a bridge1that is one of structures as an inspection target, and is a perspective view of the bridge1when viewed from the below.

The bridge1shown inFIG. 1has main girders2, cross beams3, cross frames4, and lateral frames5, and the main girders2, the cross beams3, the cross frames4, and the lateral frames5are connected by bolts, rivets or welding. Deck slabs6on which vehicles and the like travel are placed on the main girders2and the like. The deck slabs6are generally made of reinforced concrete.

The main girder2is a member that is laid between the abutments or the bridge piers and supports the load of the vehicles and the like on the deck slab6. The cross beam3is a member that connects the main girders2to support the load by a plurality of main girders2. The cross frame4and the lateral frame5are members that connect the main girders2to resist a lateral load of wind and earthquake, respectively. A panel is a space that is formed by dividing the deck slab6by two facing main girders2and two facing cross beams3or cross frames4, and in a case where inspection of the bridge1is performed, inspection may be performed in units of panels.

FIG. 2is a perspective view showing an appearance of a robot device100including a twin-lens camera202according to an embodiment of an imaging device200, and shows a state in which the robot device100is provided between the main girders2of the bridge1.FIG. 3is a cross-sectional view showing a main part of the robot device100shown inFIG. 2. The robot device100is an example of a camera-equipped mobile robot, and a robot for an inspection work of the bridge1.

As shown inFIGS. 2 and 3, the robot device100comprises the imaging device200, and controls a position (imaging position) of the imaging device200within a three-dimensional space and controls an imaging direction of the imaging device200to capture an image of any member to be inspected of the bridge1having a plurality of members at the time of inspection of the bridge1.

Though details will be described below, the robot device100comprises a main frame102, a vertical telescopic arm104, a housing106where a drive unit, various control units, and the like of the vertical telescopic arm104are provided, an X-direction drive unit108(FIG. 5) that moves the housing106in a longitudinal direction of the main frame102(a direction perpendicular to a longitudinal direction of the main girder2) (X direction), a Y-direction drive unit110(FIG. 5) that moves the entire robot device100in the longitudinal direction of the main girder2(Y direction), and a Z-direction drive unit112(FIG. 5) that makes the vertical telescopic arm104expand and contract in a vertical direction (Z direction).

The X-direction drive unit108has a ball screw108A that is provided in the longitudinal direction of the main frame102(X direction), a ball nut108B that is provided in the housing106, and a motor108C that rotates the ball screw108A, and rotates the ball screw108A in a normal direction or a reverse direction by the motor108C to move the housing106in the X direction.

The Y-direction drive unit110has tires110A and110B that are provided at both ends of the main frame102, and motors (not shown) that are provided in the tires110A and110B, and drives the tires110A and110B by the motors to move the entire robot device100in the Y direction.

The robot device100is provided in an aspect in which the tires110A and110B at both ends of the main frame102are disposed on lower flanges of the two main girders2and are disposed such that the main girders2are sandwiched between the tires110A and110B. With this, the robot device100can move (be self-propelled) along the main girders2while being suspended from the lower flanges of the main girders2. Though not shown, the main frame102is configured such that the length of the main frame102can be adjusted according to an interval between the main girders2.

The vertical telescopic arm104is provided in the housing106of the robot device100and moves in the X direction and the Y direction along with the housing106. The vertical telescopic arm104expands and contracts in the Z direction by the Z-direction drive unit112(FIG. 5) provided in the housing106.

FIG. 4is an appearance perspective view of the twin-lens camera and a pan/tilt mechanism120. As shown inFIG. 4, a camera mounting portion104A is provided at a distal end of the vertical telescopic arm104, and the twin-lens camera202that can be rotated in a pan direction and a tilt direction by the pan/tilt mechanism120is provided in the camera mounting portion104A.

The twin-lens camera202has a first imaging unit202A and a second imaging unit202B that capture two parallax images (stereoscopic image) with different parallax, acquires space information of a structure (bridge1) corresponding to an imaging range of the twin-lens camera202that is space information of the bridge1in a local coordinate system (camera coordinate system) based on the twin-lens camera202, and acquires at least one image of two images to be captured as an “inspection image” to be attached to an inspection report.

The twin-lens camera202is rotated around a pan axis P coaxial with the vertical telescopic arm104or is rotated around a tilt axis T in a horizontal direction by the pan/tilt mechanism120to which a driving force is applied from a pan/tilt drive unit206(FIG. 5). With this, the twin-lens camera202can capture images in any posture (capture images in any imaging direction).

An optical axis L1of the first imaging unit202A and an optical axis L2of the second imaging unit202B of the twin-lens camera202of the example are parallel to each other. The pan axis P is perpendicular to the tilt axis T A base line of the twin-lens camera202(that is, an interval at which the first imaging unit202A and the second imaging unit202B are provided) is known.

The camera coordinate system based on the twin-lens camera202has, for example, a cross point of the pan axis P and the tilt axis T as an origin Or, a direction of the tilt axis T as an x-axis direction, a direction of the pan axis P as a z-axis direction, and a direction perpendicular to the x axis and the y axis as a y-axis direction.

A position of the twin-lens camera202(a position of the origin Or of the camera coordinate system) that is a position (hereinafter, referred to as an “imaging position”) in a global coordinate system (bridge coordinate system) is detected by a global positioning system (GPS) (hereinafter, referred to as a “GPS device”), and can be detected using moving distances of the robot device100in the X direction and the Y direction with respect to the origin of the bridge coordinate system and a moving distance of the vertical telescopic arm104in the Z direction. The imaging direction of the twin-lens camera202is detected by a pan angle α and a tilt angle β of the pan/tilt mechanism120, and can be detected by an azimuth sensor (not shown) mounted in the twin-lens camera202.

FIG. 5is a block diagram showing a functional configuration example of an inspection system10.

As shown inFIG. 5, the inspection system10has a robot control unit130, the X-direction drive unit108, the Y-direction drive unit110, and the Z-direction drive unit112on the robot device100side, the twin-lens camera202, an imaging control unit204, a pan/tilt control unit210, and the pan/tilt drive unit206on the imaging device200side, a robot-side communication unit230, and a terminal device300.

The robot-side communication unit230performs bidirectional wireless communication with a terminal-side communication unit310, receives various commands, such as a movement command for controlling the movement of the robot device100, a pan/tilt command for controlling the pan/tilt mechanism120, and an imaging command for controlling the twin-lens camera202, transmitted from the terminal-side communication unit310, and outputs the received commands to the corresponding control units.

The robot control unit130controls the X-direction drive unit108, the Y-direction drive unit110, and the Z-direction drive unit112based on the movement command input from the robot-side communication unit230, moves the robot device100in the X direction and the Y direction, and makes the vertical telescopic arm104expand and contract in the Z direction (seeFIG. 2).

The pan/tilt control unit210operates the pan/tilt mechanism120in the pan direction and the tilt direction through the pan/tilt drive unit206based on the pan/tilt command input from the robot-side communication unit230and makes the twin-lens camera202pan and tilt in a desired direction (seeFIG. 4). That is, the pan/tilt control unit210changes the posture of the camera (imaging device200) by the pan/tilt mechanism120and appropriately acquires captured images.

The imaging control unit204makes the first imaging unit202A and the second imaging unit202B of the twin-lens camera202capture a live view image or an inspection image based on the imaging command input from the robot-side communication unit230.

Image data indicating a first image ILand a second image IRwith different parallax captured by the first imaging unit202A and the second imaging unit202B of the twin-lens camera202at the time of inspection of the bridge1and information indicating an imaging position (the position of the origin Or of the camera coordinate system in the bridge coordinate system) and the imaging direction (in the example, a pan angle α and a tilt angle β) of the twin-lens camera202are transmitted to the terminal-side communication unit310through the robot-side communication unit230.

The terminal device300is operated by an inspector who operates the inspection system10, and primarily has the terminal-side communication unit310, a terminal control unit320, an input unit330that functions as an operating unit, a display unit340, and a recording unit350. For example, a tablet terminal can be applied to the terminal device300.

The terminal-side communication unit310performs bidirectional wireless communication with the robot-side communication unit230, receives various kinds of information that are input from the robot-side communication unit230(image data indicating the live view image captured by the first imaging unit202A and the second imaging unit202B, the first image IL, and the second image IR, and information indicating the imaging position and the imaging direction of the twin-lens camera202), and transmits various commands according to operations on the input unit330input through the terminal control unit320to the robot-side communication unit230.

The terminal control unit320outputs image data indicating the live view image received through the terminal-side communication unit310to the display unit340, and makes the display unit340display the live view image on the screen of the display unit340. The input unit330has a robot operation input unit, a pan/tilt operation input unit, and an imaging operation input unit, the robot operation input unit outputs the movement command for moving the robot device100(twin-lens camera202) in the X direction, the Y direction, and the Z direction, the pan/tilt operation input unit outputs the pan/tilt command for rotating the pan/tilt mechanism120(twin-lens camera202) in the pan direction and the tilt direction, and the imaging operation input unit outputs the imaging command for instructing the twin-lens camera202to capture the inspection image. The inspector manually operates the input unit330while viewing the live view image displayed on the display unit340, and the input unit330outputs various commands, such as the movement command of the twin-lens camera202in the X direction, the Y direction, and the Z direction, the pan/tilt command, and the imaging command, to the terminal control unit320according to the operations of the inspector. The terminal control unit320transmits various commands input from the input unit330to the robot-side communication unit230through the terminal-side communication unit310.

The terminal control unit320has an imaging plan generation device400. An imaging plan generated by the imaging plan generation device400is transmitted to the robot device100. The robot control unit130, the pan/tilt mechanism120, and the imaging control unit204are controlled based on the received imaging plan. Hereinafter, the imaging plan generation device400will be described.

First Embodiment

First, an imaging plan generation device400according to a first embodiment of the invention will be described.

FIG. 6is a block diagram showing a functional configuration example of the imaging plan generation device400of the embodiment. The imaging plan generation device400shown inFIG. 6comprises a space information acquisition unit401, a deck slab imaging decision unit (first decision unit)403, a steel member imaging decision unit (second decision unit)405, and an imaging plan generation unit407.

The space information acquisition unit401acquires space information of one panel divided by two facing main girders2and two facing cross beams3or cross frames4. The space information refers to information indicating the size or position relating to the space of the panel. For example, the space information refers to the three-dimensional coordinates of the panel. The space information acquisition unit401can acquire the space information of the panel in various aspects. For example, the space information acquisition unit401may acquire the space information based on CAD information (CAD drawing data)411of the bridge1or may acquire the space information based on distance information measured by a distance measurement unit409. In a case where the CAD information411of the bridge1is incorrect or insufficient, the space information acquisition unit401may acquire the CAD information411of the bridge1and may acquire the distance information from the distance measurement unit409to acquire the space information.

The deck slab imaging decision unit403decides, based on the space information of the panel, a plurality of deck slab imaging positions and deck slab imaging postures of the camera in dividing the entire deck slab6corresponding to one panel into a plurality of pieces and imaging a plurality of pieces with the camera. A plurality of deck slab imaging positions are decided to cover the deck slab6inside the panel. A plurality of divided captured images may be subjected to panorama composition for each panel, and in this case, the deck slab imaging positions are decided with the captured images including a composition overlap width of panorama composition. The deck slab imaging postures may face the deck slab6or may be inclined with respect to the deck slab6, and are not particularly limited within a range in which imaging is performed to cover the deck slab6of the panel. The captured images captured as inclined with respect to the deck slab6are made to face the deck slab6through image processing. The deck slab imaging positions and the deck slab imaging postures decided by the deck slab imaging decision unit403are transmitted to the imaging plan generation unit407.

The steel member imaging decision unit405decides, based on the space information of the panel, a plurality of steel member imaging positions and steel member imaging postures of the camera in imaging at least a plurality of joint portions of the steel members among the steel members corresponding to one panel with the camera. The steel members are members including the main girders2, the cross beams3, and the cross frames4. The steel member imaging decision unit405primarily decides the steel member imaging positions and the steel member imaging postures for acquiring the captured images of the joint portions of the steel members. The joint portions of the steel member are connected by nuts or welding, and in the inspection, inspection is performed on the joint portions. The steel member imaging positions and the steel member imaging postures decided by the steel member imaging decision unit405are transmitted to the imaging plan generation unit407.

The imaging plan generation unit407generates an imaging plan of the camera-equipped mobile robot based on a plurality of deck slab imaging positions and deck slab imaging postures decided by the deck slab imaging decision unit403and a plurality of steel member imaging positions and steel member imaging postures decided by the steel member imaging decision unit405. The imaging plan generation unit407generates an imaging plan such that the robot device100can efficiently move and accurately acquire the captured images in one panel. For example, the imaging plan generation unit407makes the robot device100acquire the captured images of the deck slab6using the deck slab imaging positions and the deck slab imaging postures decided by the deck slab imaging decision unit403in an outward path, and makes the robot device100acquire the captured images of the steel members using the steel member imaging positions and the steel member imaging postures decided by the steel member imaging decision unit405in a return path. The imaging positions or the imaging postures inside the panel may be graphically displayed based on the generated imaging plan.

Next, the distance measurement unit409will be described. The distance measurement unit409measures a first distance to the two facing main girders2, a second distance to the two facing cross beams3or cross frames4, and a third distance to the deck slab6based on a robot initial position S within the three-dimensional space. Then, the space information acquisition unit401acquires the space information based on the measured first distance, second distance, and third distance.

FIG. 7is a diagram illustrating an aspect of distance measurement of the distance measurement unit409. (A) ofFIG. 7shows designation of a distance measurement position is shown, and (B) ofFIG. 7shows measurement of a distance of a panel7within the three-dimensional space divided by the facing main girders2, the cross beam3, and the cross frame4. The imaging device200of the robot device100described above comprises the twin-lens camera202, can perform distance measurement, and functions as the distance measurement unit409. In the example shown inFIG. 7, although a case where distance measurement is performed using a stereo image acquired by the twin-lens camera202will be been described, the invention is not limited thereto. For example, a distance may be measured by a laser distance sensor.

In (A) ofFIG. 7, a live view image within a panel space displayed on a display unit340(FIG. 5) is shown. The panel space is imaged by the twin-lens camera202of the imaging device200of the robot device100, and the inspector (user) confirms the live view image of the panel space on the display unit340(FIG. 5). Then, the inspector manually inputs and designates the distance measurement position, for example, through the input unit330. The distance measurement position may be automatically designated through image recognition processing on an image acquired by the twin-lens camera202in the deck slab imaging decision unit403. In (A) ofFIG. 7, the distance measurement position in measuring the distance (the (X+) direction of (B) ofFIG. 7) between the robot initial position S and the cross frame4is shown. In a case where the distance measurement position is manually input and designated, the inspector designates the distance measurement position in the (X−), (Y+), (Y−), and (Z) directions of (B) ofFIG. 7similarly. In a case where the entire captured image is the same plane, the designation of the distance measurement position is not needed.

The robot device100measures, based on the designated distance measurement position, the distance (first distance) between the robot initial position S and the cross frame4in a case where the twin-lens camera202is turned to the (X+) direction, the distance (first distance) between the robot initial position S and the cross beam3in a case where the twin-lens camera202is turned to the (X−) direction, the distance (second distance) between the robot initial position S and the main girder2in a case where the twin-lens camera202is turned to the (Y+) and (Y−) directions, and the distance (third distance) between the robot initial position S and the deck slab6in a case where the twin-lens camera202is turned to the (Z) direction. The robot device100transmits the measured distances to the space information acquisition unit401, and the space information acquisition unit401generates and acquire the three-dimensional coordinates of the panel7based on received distance information.

Next, the deck slab imaging decision unit403will be described in connection with a specific example. The deck slab imaging decision unit403acquires space information of the panel7from the space information acquisition unit401and decides the deck slab imaging positions and the deck slab imaging postures based on the acquired space information.

First, a deck slab imaging range that is calculated by the deck slab imaging decision unit403will be described. The deck slab imaging decision unit403calculates the deck slab imaging range using the distance between the imaging device200and the deck slab6, a focal length of the twin-lens camera202, and the size of an imaging element250of the camera, and can decide the deck slab imaging positions based on the deck slab imaging range and the space information of the panel7.

FIG. 8is a diagram illustrating the calculation of the deck slab imaging range.

The distance between the imaging device200and the deck slab6is referred to as D, the focal length of the lens of the first imaging unit202A is referred to as F, and the size of the imaging element250of the imaging device200is referred to as Sx×Sy. In this case, the imaging range for the deck slab6corresponding to the deck slab images can be calculated as follows. The deck slab images are divided images of the deck slab6captured at the deck slab imaging positions.

Imaging range (Ax) of deck slab on X axis=D×Sx/F

Imaging range (Ay) of deck slab on Y axis=D×Sy/F

Description will be provided below using a specific example.

It is assumed that the distance between the imaging device200and the deck slab6: D=2100 mm, the focal length: F=27 mm, the imaging element250(image sensor (Advanced Photo System (APS)-C)): Sx=23.6 mm, and Sy=15.6 mm.

The deck slab imaging range is as follows under the above-described conditions.

Imaging range (Ax) of deck slab on X axis=1835.6 mm

Imaging range (Ay) of deck slab on Y axis=1213.3 mm

The deck slab imaging decision unit403calculates the deck slab imaging range as described above and compares the area of the deck slab6inside the panel7in the space information of the panel7with the deck slab imaging range to decide the deck slab imaging positions as the positions whether the deck slab6can be imaged comprehensively.

FIGS. 9 and 10are diagrams illustrating the deck slab imaging positions that are decided by the deck slab imaging decision unit403. In the cases shown inFIGS. 9 and 10, a case where the imaging device200is made to face the deck slab6to perform deck slab imaging is shown.

FIG. 9is a diagram showing the deck slab imaging positions for the deck slab6in the panel7. In the case shown inFIG. 9, deck slab images are decided to be captured at deck slab imaging positions A to L on the deck slab6divided by the panel7.

In the case shown inFIG. 9, the deck slab imaging positions for one panel7include the six deck slab imaging positions A to F and the six deck slab imaging positions G to L arranged along on the Y axis, and the deck slab imaging positions arranged two by two along the X axis. The panel7has, for example, the size of 5 m in the X-axis direction and 3 m in the Y-axis direction.

FIG. 10is a diagram illustrating the deck slab imaging positions in a case where the captured deck slab images are subjected to panorama composition. Although the deck slab imaging decision unit403decides the deck slab imaging positions such that the entire deck slab6can be imaged comprehensively, in a case where the captured deck slab images are subjected to panorama composition, the deck slab imaging decision unit403decides the deck slab imaging positions such that a composition overlap width (overlapping portion) with adjacent deck slab images is secured. The composition overlap width may be, for example, 30% at both ends of the deck slab images. Since a minimum amount of the composition overlap width to be secured depends on a composition algorithm, it is desirable that the minimum amount of the composition overlap width to be secured can be changed.

Next, the steel member imaging decision unit405will be described in connection with a specific example. The steel member imaging decision unit405acquires the space information of the panel7from the space information acquisition unit401and decides the deck slab imaging positions and the deck slab imaging postures based on the acquired space information.

FIGS. 11 and 12are diagrams illustrating the steel member imaging decision unit405. InFIGS. 11 and 12, a stereoscopic conceptual diagram of the panel7is shown in (A), and a planar conceptual diagram of the panel7is shown in (B).

FIG. 11is a diagram illustrating steel member imaging positions at a lower end (in (A) ofFIG. 11, shown in a dot pattern) of the panel7as an imaging target in the Z-axis direction. In (B) ofFIG. 11, steel member imaging positions M to U are shown by arrows. In this way, the steel member imaging positions are primarily imaging positions where joint portions of the steel members are imaged. Specifically, a joint portion of the main girder2and the cross frame4is imaged at the steel member imaging position M, and a joint portion of the main girder2and the cross beam3is imaged at the steel member imaging position Q. A joint portion of the cross frame4and the lateral frame5is imaged at the steel member imaging position N, and a joint portion of the main girder2and the lateral frame5is imaged at the steel member imaging position O. A joint portion of the cross beam3and the lateral frame5is imaged at the steel member imaging position R, and a joint portion of the main girder2and the lateral frame5is imaged at the steel member imaging position S. A joint portion of the lateral frames5is imaged at the steel member imaging position P. and joint portions of the lateral frames5and the main girder2are imaged at the steel member imaging positions T and U. The steel member imaging positions T and U may not be set depending on inspection and the type of the bridge1. The example shown inFIG. 11describes the steel member imaging positions in a case where the lateral frame5is provided at the lower end of the panel7in the Z-axis direction. In a case where the lateral frame5is not provided at the lower end of the panel7in the Z-axis direction, the steel member imaging positions are the same as steel member imaging positions ofFIG. 12described below.

FIG. 12is a diagram illustrating steel member imaging positions at an upper end of the panel7as an imaging tart in the Z-axis direction. InFIG. 12, steel member imaging positions V to Y are shown by arrows. Specifically, joint portions of the main girders2and the cross frame4are imaged at the steel member imaging positions V and W, and joint portions of the main girders2and the cross beam3are imaged at the steel member imaging positions X and Y.

The steel member imaging decision unit405also decides steel member imaging postures at the steel member imaging positions. In regards to the steel member imaging postures, parameters of the pan/tilt mechanism120are decided such that the joint portions of the steel members are imaged.

FIG. 13is a flowchart showing an operation of the imaging plan generation device400.

First, the space information acquisition unit401determines whether or not the space information is input based on the CAD information411(Step S10). In a case where the space information based on the CAD information411is input, the deck slab imaging positions and postures are decided by the deck slab imaging decision unit403based on the input space information (Step S15).

In a case where the space information is not input to the space information acquisition unit401, the distance between the facing cross beams3forming the panel7and the initial position is measured by the distance measurement unit409(Step S11), the distance between the facing main girders2forming the panel7and the initial position is measured (Step S12), and the distance between the initial position and the deck slab6is measured (Step S13).

Thereafter, the space information acquisition unit401acquires the distance to the cross beams3, the distance to the main girders2, and the distance to the deck slab6from the distance measurement unit409. Thereafter, the space information acquisition unit401acquires the space information based on the acquired distance information (space information acquisition step: Step S14).

Thereafter, the deck slab imaging decision unit403decides the deck slab imaging positions and postures based on the acquired space information (first decision step: Step S15). Thereafter, the steel member imaging decision unit405decides the steel member imaging positions and postures (second decision step: Step S16). Then, the imaging plan generation unit407generates the imaging plan based on the decision of the deck slab imaging decision unit403and the decision of the steel member imaging decision unit405(imaging plan generation step: Step S17).

The above-described configurations and functions can be appropriately implemented by any hardware, software, or combinations of hardware and software. For example, the invention can be applied to a program that causes a computer to execute the above-described processing steps (processing procedure), a computer-readable recording medium (non-transitory recording medium) having the program recorded thereon, or a computer on which the program can be installed.

Second Embodiment

Next, a second embodiment of the invention will be described.FIG. 14is a block diagram showing a functional configuration example of an imaging plan generation device400of the embodiment. The imaging plan generation device400shown inFIG. 14comprises a space information acquisition unit401, a deck slab imaging decision unit403, a steel member imaging decision unit405, an imaging plan generation unit407, and a member information acquisition unit413. The parts already described referring toFIG. 6are represented by the same reference numerals, and description thereof will not be repeated.

The member information acquisition unit413acquires member information as information relating to a member of the panel7. The member information acquired by the member information acquisition unit413is transmitted to the deck slab imaging decision unit403and the steel member imaging decision unit405. Then, the deck slab imaging decision unit403decides the imaging positions and postures for the deck slab6based on the space information of the panel7and the member information, and the steel member imaging decision unit405decides the imaging positions and postures for the steel members based on the space information of the panel7and the member information.

The member information is, for example, information415relating to the lateral frame5. Information415relating to the lateral frame5is, for example, information relating to the distance (fourth distance) between the robot initial position S and the lateral frame5, the width of the lateral frame5, and the shape of the lateral frame5. The member information may be input by the user or may be measured by the twin-lens camera202of the imaging device200of the robot device100similarly to when the space information inside the panel7is acquired. The shape of the lateral frame5is, for example, a “left angle bracket” shape, a “right angle bracket” shape, or the like. The deck slab imaging decision unit403and the steel member imaging decision unit405decide the imaging positions and the imaging postures based on the space information. The member information includes not only information of the lateral frame5but also information relating to other members. For example, information relating to a pipe inside the panel7is input as the member information.

Next, the decision of the deck slab imaging positions in the deck slab imaging decision unit403in a case where the lateral frame5as a specific example of the member is present will be described.FIGS. 15 and 16are diagrams illustrating the decision of the deck slab imaging positions in a case where the lateral frame5having a “left angle bracket” shape is present.FIG. 15is a diagram of the bridge1when viewed from the X-Y direction, and shows the arrangement of the deck slab6, the lateral frame5, and the first imaging unit202A in the Z-axis direction (vertical direction).FIG. 16is a diagram of the panel7of the bridge1when viewed from the Z-axis direction, and shows a case where the lateral frame5having a “left angle bracket” shape is present inside the panel7. InFIG. 16, a horizontal plane imaging range434and a lateral frame imaged region432of the lateral frame5are shown.FIG. 15is a diagram of a cross-section430shown inFIG. 16.

The deck slab imaging decision unit403calculates the horizontal plane imaging range434as an imaging range in the horizontal plane of the lateral frame5using the distance431to the lateral frame5, the focal length of the camera, and the size of the imaging element250of the camera, and decides the imaging positions and postures for the deck slab6based on the space information of the panel7, the member information, and the horizontal plane imaging range434. Hereinafter, the deck slab imaging decision unit403of the example will be specifically described.

First, the deck slab imaging decision unit403calculates the deck slab imaging range as described referring toFIG. 8. Thereafter, as shown inFIG. 15, the deck slab imaging decision unit403calculates the horizontal plane imaging range434as the imaging range in the horizontal plane of the lateral frame5using the distance between the robot initial position S and the lateral frame5, the focal length of the twin-lens camera202, and the size of the imaging element250of the twin-lens camera202by the same method as the deck slab imaging range describe referring toFIG. 8. Thereafter, the deck slab imaging decision unit403calculates the lateral frame imaged region432. The lateral frame imaged region432is decided based on the imaging position in the lateral frame horizontal plane where the lateral frame5is shown (FIG. 15). InFIG. 15, a case where the twin-lens camera202faces the deck slab6and the lateral frame5is shown.

As shown inFIG. 16, a given region along the lateral frame5becomes the lateral frame imaged region432. Then, the deck slab imaging decision unit403decides the deck slab imaging positions within a range outside the lateral frame imaged region432to allow the entire deck slab to be comprehensively imaged without being obstructed by the lateral frame5. The deck slab imaging decision unit403decides the deck slab imaging positions where the composition overlap width in a case where panorama composition is performed can be secured as needed.

FIG. 17is a diagram showing deck slab imaging positions AA to MM in a case where the lateral frame5is present. Since the deck slab imaging positions AA to MM are not positioned within the lateral frame imaged region432, the deck slab6is imaged at the deck slab imaging positions AA to MM, whereby it is possible to image the deck slab6comprehensively while restraining the lateral frame5from being shown in the deck slab images. In comparison of the deck slab imaging positions in a case where the lateral frame5is absent described referring toFIG. 10with the deck slab imaging positions in a case where the lateral frame5is present described referring toFIG. 17, while the number of deck slab imaging positions is 12 in a case where the lateral frame5is absent, the number of deck slab imaging positions is 13 in a case where the lateral frame5is present.

The deck slab imaging decision unit403adds an imaging point for supplementing an unimaged portion as needed. In this case, in a case where the entire deck slab cannot be covered only with the movement of the imaging device200in the horizontal direction (X-Y direction), an imaging point to which the imaging device200is moved in the vertical direction (Z direction) is added.

Third Embodiment

Next, a third embodiment of the invention will be described.FIG. 18is a diagram showing a functional configuration example of an imaging plan generation device400of the embodiment. The imaging plan generation device400shown inFIG. 18comprises a space information acquisition unit401, a deck slab imaging decision unit403, a steel member imaging decision unit405, an imaging plan generation unit407, and an imaging plan correction unit417.

The imaging plan correction unit417corrects the imaging plan generated by the imaging plan generation unit407. For example, the imaging plan correction unit417optimizes an imaging sequence such that a total imaging time or a total moving distance becomes the shortest. The imaging plan correction unit417includes an imaging plan adjustment unit419and an imaging plan addition unit421.

The imaging plan adjustment unit419adjusts the deck slab imaging positions or the steel member imaging positions and postures in the imaging plan generated by the imaging plan generation unit407based on an adjustment command. The adjustment command is received by, for example, an adjustment command reception unit (not shown), and is a command for adjusting the deck slab imaging positions, the deck slab imaging postures, the steel member imaging positions, or the steel member imaging postures. The adjustment command reception unit is implemented by, for example, the input unit330.

The imaging plan addition unit421receives an addition command of the deck slab imaging positions or the steel member imaging positions. The addition command is received by, for example, an addition command reception unit (not shown), and the deck slab imaging positions or the steel member imaging positions are added to the imaging plan generated by the imaging plan generation unit407based on the addition command. The addition command reception unit is implemented by, for example, the input unit330.

FIG. 19is a flowchart showing the operations of the robot device100and the imaging plan generation device400of the embodiment in a case where the imaging plan is corrected.FIG. 20is a diagram showing the imaging plan described referring toFIG. 19.

First, the robot device100receives the imaging plan generated by the imaging plan generation device400through the robot-side communication unit230(Step S20), and operates the robot control unit130, the pan/tilt control unit210, and the imaging control unit204according to the received imaging plan. In the received imaging plan, first, since steel members are imaged according to an imaging position (a), the robot device100is moved from the robot initial position S to the imaging position (a) (Step S21) (seeFIG. 20). Next, the imaging plan correction unit417determines whether or not the adjustment command of the steel member imaging positions and/or the steel member imaging postures is input (Step S22). In a case where the imaging plan correction unit417determines that the adjustment command is not input, the robot control unit130, the pan/tilt control unit210, and the imaging control unit204make the robot device100be moved to an imaging position (b) (Step S25).

In a case where the imaging plan correction unit417determines that the adjustment command is input, the steel member imaging positions and/or the steel member imaging postures are adjusted through the input unit330(Step S23). That is, the user confirms the steel members imaged by the twin-lens camera202with the live view image on the display unit340when the robot device100X) is moved to the imaging position (a), and inputs the adjustment command of the steel member imaging positions and/or the steel member imaging postures through the input unit330. Then, the steel member imaging positions and the steel member imaging postures after the adjustment are established, and the imaging plan correction unit417makes the adjustment be reflected in the imaging plan (Step S24). Thereafter, the robot device100is moved to the imaging position (b) according to the imaging plan (Step S25).

As described above, while the imaging positions and the imaging postures are adjusted, the steel member images are sequentially captured at the imaging positions (a), (b), (c), and (d) according to the received imaging plan as shown inFIG. 20.

Fourth Embodiment

Next, a fourth embodiment will be described.FIG. 21is a diagram showing a functional configuration example of an imaging plan generation device400of the embodiment. The imaging plan generation device400shown inFIG. 21comprises a space information acquisition unit401, a deck slab imaging decision unit403, a steel member imaging decision unit405, an imaging plan generation unit407, a member information acquisition unit413, an imaging plan database427, and a storage control unit429. The parts already described referring toFIGS. 6 and 14are represented by the same reference numerals, and description thereof will not be repeated.

The imaging plan database427stores a plurality of existing imaging plans. The imaging plan database427stores previously generated imaging plans or templates of the imaging plans. The imaging plan database427stores the imaging plan generated by the imaging plan generation unit407. The imaging plan database427may store a bridge name, a panel number, a panel size, member information, and the like in association with the imaging plan.

The storage control unit429makes the imaging plan database427store the imaging plan generated by the imaging plan generation unit407. As described above, in a case where the imaging plan database427stores the imaging plan generated by the imaging plan generation unit407, the generated imaging plan is stored in the imaging plan database427under the control of the storage control unit429.

The deck slab imaging decision unit403of the embodiment selects the existing imaging plan from the imaging plan database427based on the space information of the panel7and decides the deck slab imaging positions based on the selected existing imaging plan.

The steel member imaging decision unit405of the embodiment selects the existing imaging plan from the imaging plan database427based on the space information of the panel7and decides the steel member imaging positions and postures based on the selected existing imaging plan.

The imaging plan generation unit407corrects the deck slab imaging positions decided by the deck slab imaging decision unit403and the steel member imaging position and postures decided by the steel member imaging decision unit405based on the difference between the space information acquired by the space information acquisition unit401and the space information of the selected imaging plan or the difference between the member information acquired by the space information acquisition unit401and the member information of the selected imaging plan. That is, the imaging plan generation unit407corrects the existing imaging plan stored in the imaging plan database427based on the space information or the member information. In this case, in the existing imaging plan stored in the imaging plan database427, the space information and the member information on the imaging plan are stored in association with the imaging plan.

<Robot Devices of Other Examples>

In the above description, although the robot device100(FIG. 2) has been described as an example of the camera-equipped mobile robot, the camera-equipped mobile robot of the invention is not limited thereto.

FIG. 22is a diagram showing the appearance of an unmanned flying object (drone) as another example of a camera-equipped mobile robot. An unmanned flying object500comprises the imaging device200(FIG. 4). The unmanned flying object500images an inspection target based on the imaging plan generated by the imaging plan generation device400described above. The imaging device200in the unmanned flying object500is moved within the three-dimensional space by the unmanned flying object500, and the imaging posture of the imaging device200is changed by the pan/tilt mechanism120. Even in the unmanned flying object500, the inspection system10described referring toFIG. 5is applied.

Although the example of the invention has been described above, the invention is not limited to the above-described embodiments, and may have various modifications without departing from the spirit of the invention.

EXPLANATION OF REFERENCES