Patent Description:
In known techniques, the efficiency of a crane operation is conventionally improved by confirming a suspended load and a status around the suspended load based on guide information displayed on a monitor. Such a technique is disclosed in, for example, PTL <NUM>.

PTL <NUM> discloses a technique of a height information notification system that provides a notification of an object height near a suspended load. In the height information notification system for an object near a suspended load according to PTL <NUM>, a distance from the distal end of a boom to the vicinity of the suspended load is measured by distance measuring sections such as a laser distance sensor, a microwave distance sensor, and a stereo camera. The system is configured so as to detect the position of an object (ground object) near a suspended load by using a distance measurement result, calculate the height of the object, generate a processed image (guide information) in which the position and height of the object near the suspended load are associated with an image taken by a camera, and then display the guide information on a monitor.

In a guide information display device (height information notification system) described in PTL <NUM>, a mark (warning display) or the like is displayed on a monitor when a suspended load approaches an object near the suspended load, thereby suppressing the contact of the suspended load with the object near the suspended load.

<NPL> discloses a framework for enabling real-time safety assistance for mobile crane lifting operations, and a quantitative method to validate the impact of such assistance system on lift performance and the operator's SA. Based on the framework, a practical system architecture is created featuring three major components: real-time crane motion capturing, as-is site condition modeling and updating, and hazard analysis and real-time visualization. First, crane poses are reconstructed in real-time based on the critical motions of crane parts captured by a hybrid sensor system. Second, as-is lift site conditions are modeled based on point cloud data and updated using a point cloud-vision hybrid approach. Lastly, the risk of colliding the crane parts and lifted load into nearby obstructions is pro-actively analyzed and warnings are provided to the operator through a graphical user interface. A prototype assistance system implementing the system architecture is disclosed. A series of field tests in realistic lift scenarios reveal that this system was able to accurately capture and visualize crane motion with minimal delay. Based on crane motion and as-is site conditions, the system was able to identify the potential collision hazards and provide timely warnings to the operator to mitigate the risk. The effectiveness of the assistance system is quantitatively validated by the improvement in lift performance and SA, where lift performance is quantified by five key performance indexes (KPIs) and SA is measured by an online query-based technique. Disclosed are a technical framework and a practical system architecture toward providing effective safety assistance to crane operators during lifting operations.

<NPL> discloses that Crane Lifting Path Planning (CLPP) is an important task, especially in congested construction sites. This activity becomes extremely complex since, as the project advances, new (and sometimes unpredicted) constraints may occur. These constraints force lift engineers to explore the possibility of alternative paths for the objects still to be moved. Under severe time constraints, manual analysis of such paths is practically impossible, which makes it necessary to rely on computer implementations in order to avoid project delays. From a mathematical standpoint, if the crane payload trajectory is to be defined analytically, the polar coordinate system, (r, θ), is naturally the most suitable. An algorithm for CLPP is disclosed in which the path is represented as a piece-wise continuous function where each portion is defined either by constant radius (rotation of the load) or a constant angle (translation of the load). In other words, rather than forcing a crane to adapt to un-natural rectilinear trajectories as obtained by traditional path searching procedures, it is the trajectory that is adapted to the crane motion. In fact, even though (mathematically) a precise balance between the rates of rotation and translation of the jib (or boom) will make the payload follow any continuous path (regardless of its complexity), the coupling between the rotations and translations increases the difficulty of the lifting activity. However, a crane lifting path in which these motions are uncoupled will lead to less stringent requirements in terms of controlling the balance between rotations and translations.

<CIT> discloses an anti-collision protection system for machines that operate in a raw material stockpile yard. The machines are moveable in the yard in defined paths, with each machine including a material delivery and/or material recovery boom that can be rotated about a vertical axis. The system includes a means for defining an envelope around each boom that moves with the boom. Each envelope forms a boundary of an exclusion zone for the boom. The system also includes a means for detecting an intersection of the boundaries of the exclusion zones and a means responsive to a detected boundary intersection to prevent collision of the machines.

However, if position information on a suspended load and an object near the suspended load is obtained by using local measurement data acquired by a distance measuring section as in the related art described in PTL <NUM>, it is difficult to accurately acquire the shapes of the suspended load and a ground object near the suspended load and the height of a ground surface, leading to difficulty in outputting accurate warning display when the suspended load approaches the ground object.

The present invention has been devised to solve the problem. An object of the present invention is to provide a guide information display device that can accurately obtain the shapes of a suspended load and a ground object near the suspended load and the height of a ground surface and can provide accurate warning display if the suspended load approaches the ground object, a crane equipped with the same, and a guide information display method.

The technical problem of the present invention was described above. Solutions to the problem will be described below.

According to a first aspect, the present invention provides a guide information display apparatus in accordance with independent claim <NUM>. According to a second aspect, the present invention provides a work vehicle in accordance with claim <NUM> Further aspects are set forth in the dependent claims, the drawings and the following description.

More specifically, a guide information display device according to the present disclosure includes: a data acquisition section including a camera and a laser scanner, the camera being configured to capture a part of a work area of a crane, the laser being configured to acquire point cloud data from above a suspended load in the work area; a data processing section that estimates a top surface of the suspended load, a ground surface of the work area, and a top surface of a ground object in the work area based on the point cloud data acquired by the laser scanner, generates a guide frame image of a guide frame surrounding the top surface of the suspended load and a guide frame surrounding the top surface of the ground object and an elevation value image of elevation values of the suspended load and the ground object, calculates a distance between the suspended load and the ground object based on the estimated top surfaces of the suspended load and the ground object, and outputs warning display when the distance is equal to or smaller than a threshold value; and a data display section that displays guide information including the guide frame image and the elevation value image that are generated by the data processing section, the warning display, and the image captured by the camera such that the images, the warning display, and the image are superimposed on one another.

In the guide information display device according to the present disclosure, the warning display is outputted when the distance between the suspended load and the ground object in a horizontal direction is equal to or smaller than a threshold value in the horizontal direction and the distance between the suspended load and the ground object in a vertical direction is equal to or smaller than a threshold value in the vertical direction.

In the guide information display device according to the present disclosure, the warning display is display that outputs the guide frame image and the elevation value image in any colors.

In the guide information display device according to the present disclosure, the warning display is display that outputs the guide frame image and the elevation value image in a blinking manner.

A work vehicle according to the present disclosure includes the guide information display device.

A non-claimed guide information display method according to the present disclosure is a method in a guide information display device comprising: a data acquisition section including a camera and a laser scanner, the camera being configured to capture a part of a work area of a crane, the laser being configured to acquire point cloud data from above a suspended load in the work area; a data processing section that estimates a top surface of the suspended load, a ground surface of the work area, and a top surface of a ground object in the work area based on the point cloud data acquired by the laser scanner, generates a guide frame image of a guide frame surrounding the top surface of the suspended load and a guide frame surrounding the top surface of the ground object and an elevation value image of elevation values of the suspended load and the ground object, calculates a distance between the suspended load and the ground object based on the estimated top surfaces of the suspended load and the ground object, and outputs warning display when the distance is equal to or smaller than a threshold value in a vertical direction and a threshold value in a horizontal direction; and a data display section that displays guide information including the guide frame image and the elevation value image that are generated by the data processing section, the warning display, and the image captured by the camera such that the images, the warning display, and the image are superimposed on one another, the display method comprising: a vertical-approach determination step of determining whether a difference in elevation value between the suspended load and the ground object is equal to or smaller than a threshold value in the vertical direction; a visible-outline extension step of generating a visible outline of the top surface of the suspended load and generating an extended visible outline that is a visible outline formed by horizontally moving sides of the visible outline in a direction that extends a range surrounded by the visible outline, by a distance as large as the threshold value in the horizontal direction; a non-overlap determination step of, in a coordinate system where coordinate axes are a lateral line direction of laser radiation by the laser scanner and a horizontal direction perpendicular to the lateral line direction, determining that the extended visible outline and a visible outline of the ground object do not overlap each other when a minimum value of a coordinate in the lateral line direction at all vertexes of the visible outline of the ground object is larger than a maximum value of a coordinate in the lateral line direction at all vertexes of the extended visible outline, when a minimum value of a coordinate in the horizontal direction at all the vertexes of the visible outline of the ground object is larger than a maximum value of a coordinate in the horizontal direction at all the vertexes of the extended visible outline, when a minimum value of a coordinate in the lateral line direction at all the vertexes of the extended visible outline is larger than a maximum value of a coordinate in the lateral line direction at all the vertexes of the visible outline of the ground object, or when a minimum value of a coordinate in the horizontal direction at all the vertexes of the extended visible outline is larger than a maximum value of a coordinate in the horizontal direction at all the vertexes of the visible outline of the ground object; a line-segment overlap determination step of determining whether a line segment of the extended visible outline and a line segment of the visible outline of the ground object overlap each other; and an inclusion determination step of determining whether one of the extended visible outline and the visible outline of the ground object includes the other, in which in the vertical-approach determination step, the data processing section performs the visible-outline extension step and the non-overlap determination step when a difference in elevation value between the suspended load and the ground object is equal to or smaller than the threshold value in the vertical direction, when it is not determined that the extended visible outline and the visible outline of the ground object do not overlap each other in the non-overlap determination step, the line-segment overlap determination step is performed, when it is determined that the line segment of the extended visible outline and the line segment of the visible outline of the ground object do not overlap each other in the line-segment overlap determination step, the inclusion determination step is performed, and when it is determined that the extended visible outline and the visible outline of the ground object overlap each other in the line-segment overlap determination step or when it is determined that one of the extended visible outline and the visible outline of the ground object includes the other, the warning display is outputted.

The guide information display method according to the present disclosure, further includes: a vertex-to-vertex distance calculation step of calculating distances between all vertexes of the extended visible outline and the visible outline of the ground object in the horizontal direction; and a vertex-to-line segment distance calculation step of calculating distances between all the vertexes of the extended visible outline and all line segments of the visible outline of the ground object in the horizontal direction, in which when it is determined that the extended visible outline and the visible outline of the ground object do not overlap each other in the non-overlap determination step or when it is determined that one of the extended visible outline and the visible outline of the ground object does not include the other in the inclusion determination step, the data processing section performs the vertex-to-vertex distance calculation step, when it is determined that all of the distances between the vertexes of the visible outline of the suspended load and the visible outline of the ground object in the horizontal direction are larger than a threshold value in the vertex-to-vertex distance calculation step, the data processing section performs the vertex-to-line segment distance calculation step, and when at least one of the calculated distances in the horizontal direction is equal to or smaller than the threshold value in the horizontal direction in the vertex-to-vertex distance calculation step or when at least one of the calculated distances in the horizontal direction is equal to or smaller than the threshold value in the horizontal direction in the vertex-to-line segment distance calculation step, the data processing section outputs the warning display.

The effect of the present invention will be described below.

The present invention can accurately obtain the shapes of a suspended load and a ground object near the suspended load and the height of a ground surface and provide accurate warning display if the suspended load approaches the ground object.

An embodiment of the invention will be described below.

As illustrated in <FIG>, crane <NUM> is an example of a crane to which a guide information display device according to the embodiment of the present invention is applied. Crane <NUM> is a mobile crane that can move to a desired location.

Crane <NUM> includes travelling vehicle <NUM> and crane apparatus <NUM>.

Travelling vehicle <NUM> transports crane apparatus <NUM>, has a plurality of (four in the present embodiment) wheels <NUM>, and runs with an engine (not illustrated) acting as a power source.

Outriggers <NUM> are provided at the four corners of travelling vehicle <NUM>. Outrigger <NUM> includes overhanging beam 12a extendable to both sides in the width direction of travelling vehicle <NUM> by a hydraulic pressure and hydraulic jack cylinder 12b extendable vertically relative to the ground. Travelling vehicle <NUM> can place crane <NUM> into an operable state by grounding jack cylinder 12b and extend the operable range (operating radius) of crane <NUM> by increasing the extended length of overhanging beam 12a.

Crane apparatus <NUM> hoists suspended load W with a wire rope and includes swivel base <NUM>, extendable/retractable boom <NUM>, main hook block <NUM>, sub hook block <NUM>, luffing cylinder <NUM>, main winch <NUM>, main wire rope <NUM>, sub winch <NUM>, sub wire rope <NUM>, and cabin <NUM>.

Swivel base <NUM> is configured so as to swivel crane apparatus <NUM> and is mounted on the frame of travelling vehicle <NUM> via an annular bearing. The annular bearing is disposed with the rotation center perpendicular to the installation surface of travelling vehicle <NUM>. Swivel base <NUM> is configured so as to rotate about the center of the annular bearing in one direction and the other direction. Swivel base <NUM> is rotated by a hydraulic motor (not illustrated).

Extendable/retractable boom <NUM> supports the wire rope so as to hoist suspended load W. Extendable/retractable boom <NUM> includes base boom members 22a that are a plurality of boom members, second boom member 22b, third boom member 22c, fourth boom member 22d, fifth boom member 22e, and top boom member 22f. The boom members are telescopically inserted in decreasing order of cross-sectional areas. Extendable/retractable boom <NUM> is configured to axially extend and retract by moving the boom members with an extendable/retractable cylinder, which is not illustrated. Extendable/retractable boom <NUM> is mounted such that the base end of base boom member 22a can swivel on swivel base <NUM>. Thus, extendable/retractable boom <NUM> is configured so as to horizontally rotate and swivel on the frame of travelling vehicle <NUM>.

Main hook block <NUM> is configured to catch and hang suspended load W and includes a plurality of hook sheaves where main wire rope <NUM> is wound and main hook <NUM> for hanging suspended load W.

Crane apparatus <NUM> further includes sub hook block <NUM> for catching and hanging suspended load W in addition to main hook block <NUM>. Sub hook block <NUM> has sub hook <NUM> for hanging suspended load W.

Luffing cylinder <NUM> luffs up and down extendable/retractable boom <NUM> and keeps the attitude of extendable/retractable boom <NUM>. Luffing cylinder <NUM> includes a hydraulic cylinder having a cylinder part and a rod part.

Main winch <NUM> reels in (winds up) and out (winds down) main wire rope <NUM> and includes a hydraulic winch in the present embodiment.

Main winch <NUM> is configured such that a main drum having wound main wire rope <NUM> is rotated by a main hydraulic motor. Main winch <NUM> is configured such that main wire rope <NUM> wound around the main drum is reeled out by feeding hydraulic oil so as to rotate the main hydraulic motor in one direction and main wire rope <NUM> is reeled in and wound around the main drum by feeding hydraulic oil so as to rotate the main hydraulic motor in the other direction.

Sub winch <NUM> reels in and out sub wire rope <NUM> and includes a hydraulic winch in the present embodiment.

Cabin <NUM> covers operator's seat <NUM> for a seated operator and is provided on a side of extendable/retractable boom <NUM> on swivel base <NUM>.

Crane <NUM> configured thus can move crane apparatus <NUM> to any position by running travelling vehicle <NUM> and extend extendable/retractable boom <NUM> to any extended/retracted boom length by raising extendable/retractable boom <NUM> to any luffing angle by means of luffing cylinder <NUM>.

Moreover, crane <NUM> includes controller <NUM> that controls the operations of, for example, swivel base <NUM>, extendable/retractable boom <NUM>, and luffing cylinder <NUM> (that is, the operations of crane <NUM>). Controller <NUM> can output information on operating states of, for example, swivel base <NUM>, extendable/retractable boom <NUM>, and luffing cylinder <NUM>, information on performance unique to crane <NUM>, and the weight of suspended load W and the like.

In the present explanation, an XYZ coordinate system is defined as illustrated in <FIG> with respect to the axial direction of the luffing fulcrum of extendable/retractable boom <NUM> (this holds true also in the following explanation).

The X-axis direction (also referred to as a lateral line direction) is a horizontal direction parallel to the axial direction of the luffing fulcrum of extendable/retractable boom <NUM>. The Y-axis direction (also referred to as a elevation direction) is a vertical direction. Furthermore, the Z-axis direction (also referred to as a depth direction) is a horizontal direction perpendicular to the axial direction of the luffing fulcrum of extendable/retractable boom <NUM>. In other words, as illustrated in <FIG>, the XYZ coordinate system is defined as a local coordinate system with reference to extendable/retractable boom <NUM>.

The guide information display device according to the embodiment of the present invention will be described below.

Crane <NUM> includes guide information display device <NUM> as illustrated in <FIG>.

Guide information display device <NUM> is an example of the guide information display device according to the present invention. The guide information display device is a device for displaying an image of information (hereinafter, will be referred to as guide information) on an area including suspended load W (hereinafter, will be referred to as suspended load area WA) and indicating the information to an operator in order to efficiently and safely operate crane <NUM> illustrated in <FIG>.

As illustrated in <FIG> and <FIG>, "suspended load area WA" is set as an area including suspended load W when viewed in the Y-axis direction in work area SA of crane <NUM>. The work area is an area where "guide information" is generated.

"Suspended load area WA" is set as an area including an area directly under top boom member 22f of extendable/retractable boom <NUM> of crane <NUM>. Suspended load W, ground surface F, and ground object C in suspended load area WA are to be measured by guide information display device <NUM>. "Suspended load area WA" changes in response to a swiveling operation, a luffing operation, and extending and retracting operations of extendable/retractable boom <NUM>.

Moreover, "guide information" assists the determination of an operator who evaluates the length, the swiveling position, and the luffing angle of extendable/retractable boom <NUM> and the feeding amount of the wire rope when transporting suspended load W by crane <NUM>. The information includes image information on suspended load area WA, information on the shapes of suspended load W and ground object C, height information on suspended load W, height information on ground object C, and information on the path of suspended load W.

As illustrated in <FIG> and <FIG>, guide information display device <NUM> includes data acquisition section <NUM>, data processing section <NUM>, data display section <NUM>, and data input section <NUM>.

Data acquisition section <NUM> is a section that acquires data necessary for generating guide information in suspended load area WA. As illustrated in <FIG>, data acquisition section <NUM> includes camera <NUM>, laser scanner <NUM>, and inertial measurement unit (IMU) <NUM>.

As illustrated in <FIG>, data acquisition section <NUM> is attached to top boom member 22f positioned at the distal end of extendable/retractable boom <NUM> of crane <NUM> and is placed so as to capture a state immediately under data acquisition section <NUM> from the boom distal end immediately above suspended load W. "Immediately above" means a concept including a position vertically above suspended load W and a position in a certain range (for example, the range of the top surface of suspended load W) defined with reference to the position vertically above suspended load W.

Data acquisition section <NUM> is attached to top boom member 22f of the distal end part of extendable/retractable boom <NUM> via gimbal <NUM> (see <FIG>). Data acquisition section <NUM> is configured so as to keep a substantially fixed attitude (directed in the Y-axis direction) when extendable/retractable boom <NUM> performs a luffing operation, a swiveling operation, and extending and retracting operations. Thus, camera <NUM> and laser scanner <NUM> can be always directed to suspended load W. With this configuration, data acquisition section <NUM> can always acquire data from suspended load W and ground surface F (that is, suspended load area WA) under suspended load W by means of camera <NUM> and laser scanner <NUM>. If ground object C is present in suspended load area WA, data on ground object C can be acquired by camera <NUM> and laser scanner <NUM>.

As illustrated in <FIG>, camera <NUM> is a digital video camera for capturing video of suspended load area WA. Camera <NUM> has the function of outputting taken video to the outside in real time. Camera <NUM> has angles of view (horizontal angle of view θh and vertical angle of view θv) as illustrated in <FIG>. Furthermore, camera <NUM> has the number of pixels, a frame rate, and an image transmission rate in consideration of a data amount necessary for generating suitable guide information.

As illustrated in <FIG>, laser scanner <NUM> is an apparatus that radiates laser light to a measuring object and receives reflected light from the measuring object of the laser light so as to acquire information on a reflecting point and acquire point cloud data on the measuring object. The measuring object of laser scanner <NUM> is suspended load W, ground object C, or ground surface F. Moreover, first GNSS receiver <NUM> for acquiring a measurement time is connected to laser scanner <NUM>.

In guide information display device <NUM>, laser scanner <NUM> two-dimensionally acquires three-dimensional point cloud data in real time.

As illustrated in <FIG>, laser scanner <NUM> includes <NUM> laser transmitting/receiving sensors in total. Laser scanner <NUM> can simultaneously radiates <NUM> laser beams to the measuring object and acquire point cloud data on the measuring object. The <NUM> laser transmitting/receiving sensors of laser scanner <NUM> are each disposed at a radiation angle of <NUM>° difference in the Z-axis direction and are configured so as to radiate laser light to the measuring object with a radiation angle of <NUM>°. Moreover, the laser transmitting/receiving sensors of laser scanner <NUM> are configured so as to rotate <NUM>° (at each azimuth angle) about the Z axis. In the following explanation, trails drawn by laser light radiated to suspended load area WA will be referred to as laser lateral lines. The laser lateral lines are parallel to the X-axis direction. Laser scanner <NUM> draws <NUM> laser lateral lines at the same time.

Laser scanner <NUM> is disposed such that the laser lateral lines are parallel to the X-axis direction. Moreover, laser scanner <NUM> has a reference axis for changing the radiation angle of laser light, the reference axis being parallel to the Z-axis direction.

As illustrated in <FIG>, inertial measurement unit (hereinafter, will be referred to as IMU) <NUM> is an apparatus for acquiring attitude data on camera <NUM> and laser scanner <NUM> during data acquisition. IMU <NUM> can measure an attitude angle in real time and has measurement accuracy usable for correcting point cloud data acquired by laser scanner <NUM>. Moreover, second GNSS receiver <NUM> for acquiring a measurement time is connected to IMU <NUM>.

As illustrated in <FIG>, data acquisition section <NUM> is an integrated sensor unit of camera <NUM>, laser scanner <NUM>, and inertial measurement unit (IMU) <NUM> that are fixed to frame unit <NUM>.

Frame unit <NUM> is an object that is substantially shaped like a rectangular parallelepiped and includes five combined plates. Frame unit <NUM> is configured such that the four plates form the four sides of the rectangular parallelepiped, the other plate forms the top surface of the rectangular parallelepiped and an opening is formed at the bottom of the rectangular parallelepiped. In data acquisition section <NUM>, camera <NUM> and laser scanner <NUM> are provided inside the sides of frame unit <NUM> and IMU <NUM> is provided on the top surface of frame unit <NUM>. As illustrated in <FIG>, the center position of an image pickup device of camera <NUM> and the laser center position of laser scanner <NUM> are separated from each other by distance Δzh in the Z-axis direction when viewed in the Y-axis direction. The laser center position is the rotation center of laser in laser scanner <NUM>. The laser center position is located on the Z axis.

As illustrated in <FIG>, the center position of the image pickup device of camera <NUM> and the laser center position of laser scanner <NUM> are separated from each other by distance Δyv in the Y-axis direction when viewed in the X-axis direction.

Data acquisition section <NUM> is disposed such that the four sides of frame unit <NUM> are opposed to each other in pairs, one of the paired sides is perpendicular to the Z axis, and the other side is perpendicular to the X axis. Furthermore, data acquisition section <NUM> is disposed such that the top surface of frame unit <NUM> is perpendicular to the Y axis.

A method of converting the X coordinate of point (x, y) in the XYZ coordinate system into an Xc coordinate in a camera-space coordinate system will be described below.

In guide information display device <NUM>, coordinate values are converted between the XYZ coordinate system and the camera-space coordinate system in order to display guide information GD, which will be described later, on data display section <NUM> while guide information GD is superimposed on image M captured by camera <NUM>. In guide information display device <NUM>, three-dimensional camera-space coordinate system Xc·Yc·Zc is defined in the image space of camera <NUM>.

As illustrated in <FIG>, dh is a distance from a perpendicular line extending from the lens center of camera <NUM> to point (x, y) in the X-axis direction and wh is the maximum screen width of camera <NUM> in the horizontal direction. Furthermore, x of point (x, y) is a position from the center of the screen in the X-axis direction. At this point, the Xc coordinate of point (x, y) in the camera space is expressed by following Expressions (<NUM>) and (<NUM>).

In the following expressions, Δzh is a difference between the position of an image pickup device of camera <NUM> and the laser center position in the horizontal direction (see <FIG>), wh is the breadth of a camera image, θh is the horizontal angle of view of camera <NUM>, and tmp1 is a temporary variable. <MAT> <MAT>.

A method of converting the Z coordinate of point (y, z) in the XYZ coordinate system into a Zc coordinate in the camera-space coordinate system will be described below.

As illustrated in <FIG>, dv is a distance from point (y, z) to the lens center in the Z-axis direction and wv is the maximum screen width of camera <NUM> in the horizontal direction. Furthermore, z of point (y, z) is a position from the center of the screen in the Z-axis direction. At this point, the Zc coordinate of point (y, z) in the camera space is expressed by following Expressions (<NUM>) and (<NUM>).

In the following expressions, Δyv is a difference between the position of an image pickup device of camera <NUM> and the laser center position of laser scanner <NUM> in the vertical direction (see <FIG>), wv is the length of a camera image, θv is the vertical angle of view of camera <NUM>, and tmp2 is a temporary variable. <MAT> <MAT>.

In guide information display device <NUM>, the coordinates of point cloud data acquired by laser scanner <NUM> or the like in the XYZ coordinate system are converted to the coordinates of the camera-space coordinate system by using Expressions (<NUM>) to (<NUM>), thereby displaying guide information GD positioned on image M captured by camera <NUM>.

As laser scanner <NUM>, an apparatus capable of measuring the three-dimensional shape of a measuring object from the maximum height (for example, about <NUM>) of extendable/retractable boom <NUM> is selected in consideration of the maximum height. Furthermore, as laser scanner <NUM>, an apparatus having predetermined performance for specifications such as a measurement speed, the number of measurement points, and the accuracy of measurement is selected in consideration of a data amount and data accuracy that are necessary for generating proper guide information.

In the present embodiment, laser scanner <NUM> including <NUM> laser transmitting/receiving sensors in total is used. The guide information display device according to the present invention is not limited depending upon the number of laser transmitting/receiving sensors constituting the laser scanner. In other words, in the guide information display device according to the present invention, any laser scanner with optimum specifications is selected according to, for example, the maximum height of the boom (jib) of the crane.

Data acquired in suspended load area WA by data acquisition section <NUM> includes image data acquired by capturing suspended load W, ground surface F under suspended load W, and ground object C around suspended load W by means of camera <NUM>. Moreover, data acquired in suspended load area WA by data acquisition section <NUM> includes point cloud data acquired by scanning suspended load W, ground surface F, and ground object C by means of laser scanner <NUM>. In this case, ground surface F widely includes a sender of suspended load W and a surface at the receiver and also includes the rooftop floor of a building and a roof surface or the like as well as a ground surface.

As illustrated in <FIG>, data processing section <NUM> is a section for processing data acquired by data acquisition section <NUM> and generating guide information GD to be presented to an operator. In the present embodiment, data processing section <NUM> includes a general-purpose personal computer having a predetermined data processing program installed on the computer.

Moreover, data processing section <NUM> is electrically connected to controller <NUM> of crane <NUM> and receives "crane information" outputted from controller <NUM>.

Data display section <NUM> is a section for displaying guide information GD to be presented to an operator. Data display section <NUM> includes a display device connected to data processing section <NUM>.

As illustrated in <FIG>, image M of suspended load area WA captured by camera <NUM> is displayed in real time by data display section <NUM>.

As illustrated in <FIG>, guide information GD includes guide frame GD1 that is a guide frame image illustrating the outside shapes of suspended load W and ground object C in the Y-axis direction, height information GD2 that is an elevation value image of the underside of suspended load W, height information GD3 that is an elevation value image of ground object C, operating radius information GD4 indicating the path of suspended load W, axis information GD5 indicating the axial direction of extendable/retractable boom <NUM> and the like.

Furthermore, data display section <NUM> displays guide information GD generated by data processing section <NUM> and image M while superimposing guide information GD and image M.

As illustrated in <FIG>, data input section <NUM> is a section for inputting a set value or the like to data processing section <NUM> and includes a touch panel, a mouse, a keyboard and the like.

As illustrated in <FIG>, in guide information display device <NUM>, data processing section <NUM>, data display section <NUM>, and data input section <NUM> are preferably integrated by a general-purpose tablet personal computer (hereinafter, will be also referred to as a tablet PC). As illustrated in <FIG>, in guide information display device <NUM>, data display section <NUM> and data input section <NUM> may be integrated by a touch-panel display device and data processing section <NUM> acting as a general-purpose PC may be connected to the touch-panel display device.

As illustrated in <FIG>, data display section <NUM> and data input section <NUM> are disposed so as to be viewable to an operator at the front of operator's seat <NUM> in cabin <NUM>. Data processing section <NUM> is preferably disposed near data acquisition section <NUM>. If data processing section <NUM>, data display section <NUM>, and data input section <NUM> are integrated by a tablet PC, data processing section <NUM> may be disposed in cabin <NUM>.

Data is preferably transmitted between data acquisition section <NUM> and data processing section <NUM> via a wired LAN. Alternatively, data may be transmitted between data acquisition section <NUM> and data processing section <NUM> via a wireless LAN or power line communications.

A status of data acquisition by data acquisition section <NUM> will be described below.

In data acquisition section <NUM>, suspended load area WA is successively captured by camera <NUM> and image M of suspended load area WA is acquired.

As illustrated in <FIG>, in data acquisition section <NUM>, suspended load area WA is successively scanned by laser scanner <NUM> and point cloud data on a measuring object in suspended load area WA is acquired. Hereinafter, point cloud data acquired by laser scanner <NUM> will be referred to as point cloud data P. Point cloud data P is a set of point data p. Point data p indicates points on ground surface F, suspended load W, and the top surface of ground object C in suspended load area WA. As illustrated in <FIG>, point data p includes information on distance a from a measuring object (for example, ground object C) to laser scanner <NUM> and radiation angle b of laser scanner <NUM> when point data p is acquired.

As illustrated in <FIG>, first GNSS receiver <NUM> is connected to laser scanner <NUM>. First GNSS receiver <NUM> receives time information from a plurality of positioning satellites while acquiring point cloud data P. Data processing section <NUM> adds information on the acquisition time of point data p to point data p. In other words, information on point data p includes acquisition time tp in addition to distance a and radiation angle b.

In data acquisition section <NUM>, laser scanner <NUM> acquires point cloud data P; meanwhile, IMU <NUM> acquires attitude data Q on laser scanner <NUM> with a predetermined period. Attitude data Q includes information on angles with respect to the X-axis, Y-axis, and Z-axis directions of laser scanner <NUM> and accelerations. The acquisition period of attitude data Q acquired by IMU <NUM> is set shorter than the acquisition period of point cloud data P acquired by laser scanner <NUM>. Attitude data Q is a set of individual attitude data q that is measured in each measuring period.

Second GNSS receiver <NUM> is connected to IMU <NUM>. Second GNSS receiver <NUM> receives time information from a plurality of positioning satellites while acquiring attitude data Q. Data processing section <NUM> adds acquisition time tq as information on the acquisition time of individual attitude data q to individual attitude data q. In other words, information on individual attitude data q includes acquisition time tq.

A status of data processing by data processing section <NUM> will be described below.

As indicated in <FIG>, "frame extraction" is first performed in data processing by data processing section <NUM> (STEP-<NUM>).

In data processing on point cloud data P by data processing section <NUM>, point cloud data P for one frame is extracted and outputted from the stream data of point cloud data P. Point cloud data P of one frame is a set of point data p that is acquired while the direction of laser radiation from laser scanner <NUM> rotates once about the Z axis.

As illustrated in <FIG>, "synchronous processing on point cloud data and attitude data" is subsequently performed in data processing by data processing section <NUM> (STEP-<NUM>).

Data processing section <NUM> synchronizes point data p included in point cloud data P of one frame with attitude data Q acquired by IMU <NUM>.

Specifically, point data p is synchronized by searching for acquisition time tq of individual attitude data q closest to acquisition time tp of individual point data p and associating individual attitude data q in acquisition time tq with point data p.

Thereafter, data processing section <NUM> outputs point data p synchronized with individual attitude data q.

As illustrated in <FIG>, data processing section <NUM> calculates distance h from the laser center position of laser scanner <NUM> to point data p based on distance a and radiation angle b. "Distance h" is a distance from the laser center position of laser scanner <NUM> to a horizontal surface having point data p.

When distance h of point data p is calculated in data processing section <NUM>, the distance is corrected by using individual attitude data q corresponding to point data p. This can eliminate an error caused by the attitude of laser scanner <NUM>, thereby calculating distance h of point data p with higher accuracy.

Specifically, in guide information display device <NUM>, data acquisition section <NUM> includes IMU <NUM> that acquires attitude data Q of laser scanner <NUM> and data processing section <NUM> corrects point cloud data P based on attitude data Q of laser scanner <NUM> after attitude data Q is acquired by IMU <NUM>.

With this configuration, guide information display device <NUM> can present more accurate guide information GD to an operator.

<FIG> is a plot of point cloud data P of one frame in the XYZ coordinate system. <FIG> indicates point cloud data P (a set of point data p) viewed in the Z-axis direction.

As indicated in <FIG>, "ground surface estimation" is then performed in data processing by data processing section <NUM> (STEP-<NUM>). Data processing section <NUM> estimates ground surface F.

The estimation of ground surface F with reference to a specific position on an image will be first described below. In this case, a specific position on an image is manually specified by an operator. Data processing section <NUM> may automatically determine and specify a specific position on an image.

In guide information display device <NUM>, data display section <NUM> and data input section <NUM> specify the position of the ground surface, thereby determining ground surface F as a reference.

In a manual operation, as illustrated in the upper drawing of <FIG>, the operator first specifies a position to be located on the ground surface on an image displayed on data display section <NUM>. Thereafter, as illustrated in the middle drawing of <FIG>, data processing section <NUM> generates a reference circle having a predetermined radius with respect to the specified position (point). Subsequently, as illustrated in the lower drawing of <FIG>, data processing section <NUM> detects an overlap with point data p on laser lateral lines and selects the segments of point data p contained in the reference circle.

Data processing section <NUM> first extracts, as illustrated in <FIG>, point data p at distance h maximized to distance hmax from the selected segments of point data p. Point data p at maximum distance hmax is assumed to be point data p at the closest position. Data processing section <NUM> then extracts point data p where deviation D of distance h falls within a certain range (<NUM> or less in the present embodiment) with reference to maximum distance hmax, and calculates the mean value of distances h of extracted point data p. Data processing section <NUM> estimates that the calculated mean value is distance h to ground surface F and determines the height of ground surface F (hereinafter, will be referred to as reference height H0) based on the mean value.

Subsequently, data processing section <NUM> calculates elevation value H of point data p from distance h and reference height H0. As illustrated in <FIG>, elevation value H is a height from reference height H0 of point data p.

Guide information display device <NUM> is configured to generate guide information GD based on reference height H0 of ground surface F after reference height H0 is accurately obtained by the above process. Thus, in guide information display device <NUM>, the shapes of suspended load W and ground object C around suspended load W can be accurately calculated based on the height of ground surface F.

Automatic estimation of ground surface F will be described below.

In the foregoing configuration, the operator specifies ground surface F. Data processing section <NUM> may automatically estimate ground surface F in guide information display device <NUM>.

If data processing section <NUM> automatically estimates ground surface F, as illustrated in <FIG>, data processing section <NUM> divides suspended load area WA into small equal areas S (<NUM> areas in the present embodiment).

Data processing section <NUM> then extracts point data p at maximum distance h (distance h is maximized to distance hmax) in each small area S and as illustrated in <FIG>, data processing section <NUM> extracts point data p where deviation D of distance h falls within a certain range (deviation D is <NUM> or less in the present embodiment) with reference to maximum distance hmax.

Subsequently, data processing section <NUM> calculates the mean value of distances h of extracted point data p in each small area S. Data processing section <NUM> automatically estimates reference height H0 of ground surface F in each small area S from the calculated mean value of distances h.

Alternatively, data processing section <NUM> further averages the mean values of distances h calculated in small areas S with all small area S and automatically estimates reference height H0 of ground surface F of suspended load area WA from the mean value. In this case, with reference to the maximum value of the average values of distances h in small areas S, data processing section <NUM> calculates reference height H0 by using only small area S where deviation D from the maximum value falls within a predetermined threshold value.

As indicated in <FIG>, "plane estimation" is then performed in data processing by data processing section <NUM> (STEP-<NUM>). Data processing section <NUM> estimates the top surfaces of suspended load W, which is a measuring object in suspended load area WA, and ground object C according to the following top-surface estimation method.

<FIG> is a plot of point cloud data P of one frame on suspended load area WA indicated in the XYZ coordinate system. Point cloud data P in suspended load area WA is schematically illustrated in the upper drawing of <FIG>.

Data processing section <NUM> divides point cloud data P, which is acquired in suspended load area WA as illustrated in the upper drawing of <FIG>, into layers having predetermined thickness d in the Y-axis direction as illustrated in the middle drawing of <FIG>, and allocates point cloud data P to a plurality of groups (see <FIG>).

At this point, data processing section <NUM> allocates group IDs (in this case, ID: <NUM> to <NUM>) to the respective groups and associates point data p with the group IDs.

Subsequently, data processing section <NUM> estimates a plane in each group by using segments of point data p included each group. In this case, "plane" is a plane on top of suspended load W and ground object C, that is, "top surfaces" of suspended load W and ground object C.

Specifically, as illustrated in <FIG> and the upper drawing of <FIG>, data processing section <NUM> first selects two segments of point data p from the segments of point data p included in the same group (the step of selecting two points: STEP-<NUM>).

As illustrated in <FIG> and the lower drawing of <FIG>, data processing section <NUM> calculates distance L1 between the selected two segments of point data p (the step of calculating a distance between points: STEP-<NUM>).

Subsequently, as illustrated in <FIG> and the upper drawing of <FIG>, data processing section <NUM> assumes that the two segments (the two segments of point data p indicated by dotted lines) are placed on the same plane (the step of estimating a two-point plane: STEP-<NUM>) if distance L1 between the two segments is equal to or smaller than predetermined threshold value r1 (STEP-<NUM>). Data processing section <NUM> then calculates center of gravity G1 of the segments (in this case, selected two points) assumed to be located on the same plane as illustrated in <FIG> and the lower drawing of <FIG> (the step of calculating the center of gravity: STEP-<NUM>). If "no" is determined in (STEP-<NUM>), the process returns to (STEP-<NUM>) to select another two points.

As illustrated in <FIG> and the upper drawing of <FIG>, data processing section <NUM> then searches for point data p serving as a point near calculated center of gravity G1 (the step of searching for a nearby point: STEP-<NUM>). In this case, "nearby point" is a point where a point-to-point distance with respect to center of gravity G1 is equal to or smaller than threshold value r1.

Subsequently, as illustrated in <FIG> and the lower drawing of <FIG>, data processing section <NUM> assumes that point data p as the nearby point is also located on the same plane as the selected two segments of point data p (the step of estimating a nearby-point plane: STEP-<NUM>) when point data p on the nearby point is found (STEP-<NUM>).

As illustrated in <FIG> and the upper drawing of <FIG>, data processing section <NUM> then returns to (STEP-<NUM>) and calculates another center of gravity G2 from the points assumed to be located on the same plane (the three segments of point data p indicated by dotted lines).

Data processing section <NUM> advances to (STEP-<NUM>) and further searches for point data p as a point near center of gravity G2. Subsequently, as illustrated in <FIG> and the lower drawing of <FIG>, data processing section <NUM> assumes that point data p as the nearby point is also located on the same plane as the selected points (STEP-<NUM>) when point data p on the nearby point is further found (STEP-<NUM>).

Data processing section <NUM> then searches for a nearby point while calculating another center of gravity. Data processing section <NUM> repeatedly performs processing from (STEP-<NUM>) to (STEP-<NUM>) in the order mentioned until no point data p is detected as a nearby point.

As indicated in <FIG>, if another nearby point is not found, data processing section <NUM> determines "no" in (STEP-<NUM>) and clusters subsets (clusters) of point data p assumed to be located on the same plane as illustrated in <FIG>, thereby estimating a plane (STEP-<NUM>). In this case, "clustering" is processing for dividing point cloud data P, which is a set of point data p, into clusters such that point data p included in each cluster is located on the same plane as a common feature.

Data processing section <NUM> divides point cloud data P into point data p assumed to be located on the same plane and sets plane cluster CL1 (see the lower drawing of <FIG>). According to point data p belonging to plane cluster CL1, a plane (that is, "top surfaces" of suspended load W and ground object C) can be defined. Additionally, a plurality of plane clusters CL1 may be present in a group having the same group ID.

Furthermore, data processing section <NUM> estimates "width" of the plane from the maximum value and the minimum value of an X coordinate of point data p belonging to plain cluster CL1 and estimates "depth" of the plane according to the maximum value and the minimum value of a Z coordinate of point data p. In this way, data processing section <NUM> defines the plane where plane cluster CL1 is formed. The defined plane may be a polygon other than rectangles.

In the method of estimating a top surface, the top surfaces of suspended load W and ground object C can be estimated only based on point cloud data P corresponding to a top surface acquired by laser scanner <NUM>. Thus, in the method of estimating a top surface according to the present embodiment, the top surfaces of suspended load W and ground object C can be estimated in a short time based on point cloud data P acquired by laser scanner <NUM>, leading to real-time estimation of the top surfaces of suspended load W and ground object C.

Moreover, in the method of estimating a top surface, the top surfaces of suspended load W and ground object C can be estimated without using a statistical method, thereby reducing computational complexity required for estimating the top surfaces of suspended load W and ground object C as compared with a statistical method. Hence, in the method of estimating a top surface according to the present embodiment, the top surfaces of suspended load W and ground object C can be estimated in a shorter time based on point cloud data P acquired by laser scanner <NUM>.

In the method of estimating the top surfaces of suspended load W and ground object C according to the present embodiment, crane <NUM> includes data acquisition section <NUM> provided on top boom member 22f of extendable/retractable boom <NUM> and laser scanner <NUM> acquires point cloud data P on suspended load W, ground object C, and ground surface F vertically from above suspended load W. The method of estimating the top surface of a measuring object according to the present invention is not limited to application for measuring a suspended load of the crane and an object around the suspended load.

In other words, the method of estimating a top surface according to the present embodiment is widely applicable, for example, in the case where a laser scanner is provided on the boom distal end or the drone of a work vehicle (for example, an aerial work platform) including a boom and point cloud data on a measuring objects vertically disposed below the laser scanner is acquired from above, and the top surface of the measuring object is estimated based on the acquired point cloud data.

Subsequently, data processing section <NUM> combines estimated plane clusters CL1 (top surfaces).

As illustrated in <FIG> and the upper drawing of <FIG>, data processing section <NUM> selects two plane clusters CL1 having different group IDs from among estimated plane clusters CL1 and calculates difference dH between height values H of plane clusters CL1 (STEP-<NUM>: the step of calculating a height value difference).

At this point, data processing section <NUM> searches for a combination having difference dH not larger than threshold value r2 (STEP-<NUM>). Elevation value H of plane cluster CL1 is the mean value of elevation values H of point data p belonging to plane cluster CL1.

Subsequently, when data processing section <NUM> detects a combination of plane clusters CL1 where difference dH between elevation values H is equal to or smaller than threshold value r2 as illustrated in <FIG> and the middle drawing of <FIG>, data processing section <NUM> detects an overlap dW of plane clusters CL1 in the X-axis direction (STEP-<NUM>: the step of detecting an overlap). In this case, "overlap" is a degree of overlapping and a degree of separation in the X-axis direction of a plane defined by plane clusters CL1. As illustrated in <FIG> and <FIG>, if an overlapping amount dW1 of "width" is detected (dW1 > <NUM>) or if a separation amount dW2 is equal to or smaller than predetermined threshold value r3 (<NUM> ≤ dW2 ≤ r3), "overlap" is detected.

As illustrated in <FIG> and <FIG>, if "overlap" is detected (STEP-<NUM>), data processing section <NUM> assumes that point data p belonging to plane clusters CL1 is present on the same plane, combines two plane clusters CL1, and updates the clusters as another plane cluster CL1 (STEP-<NUM>: the step of combining planes). At this point, another elevation value H is calculated from point data p belonging to another plane cluster CL1.

As indicated in <FIG>, data processing section <NUM> repeatedly performs the processing until a combination of plane clusters CL1 satisfying the conditions becomes undetected (STEP-<NUM>), thereby estimating a plane over multiple groups.

Data processing section <NUM> then outputs the combined planes (that is, plane clusters CL1).

The plane defined by plane clusters CL1 is a plane on top of suspended load W and ground object C, that is, the top surfaces of suspended load W and ground object C.

In the method of estimating a plane, a plane can be estimated without using the normal vector of point cloud data P. This reduces computational complexity as compared with the estimation of a plane by using the normal vector of point cloud data P.

Furthermore, in the method of estimating a plane, the top surfaces of suspended load W and ground object C are estimated, thereby recognizing the three-dimensional shapes of suspended load W and ground object C without acquiring point data p on the sides of suspended load W and ground object C.

As indicated in <FIG>, "clustering of the same area" is then performed in data processing by data processing section <NUM> (STEP-<NUM>). In this case, "clustering" is processing for dividing point cloud data P, which is a set of point data p, into clusters such that point data p included in each cluster is located in "the same area" as a common feature.

"Clustering of the same area" is processing for clustering generated plane clusters CL1 (planes) from a different viewpoint, that is, whether the plane clusters are present in "the same area" or not regardless of whether the plane clusters constitute the same plane or not.

Specifically, as illustrated in the upper drawing of <FIG>, data processing section <NUM> extracts plane cluster CL1 including point data p having elevation value H maximized to value Hh and another plane cluster CL1 that is not combined with plane cluster CL1. Data processing section <NUM> then calculates difference ΔH between elevation values H of extracted plane clusters CL1 and advances to subsequent determination if difference ΔH is equal to or smaller than a predetermined threshold value.

In the subsequent determination, as illustrated in the middle drawing of <FIG>, data processing section <NUM> confirms an overlap of two plane clusters CL1 in the Y-axis direction with difference ΔH not larger than the predetermined threshold value.

If two plane clusters CL1 overlap each other in the Y-axis direction, as illustrated in the lower drawing of <FIG>, data processing section <NUM> assumes that plane clusters CL1 are located in "the same area" and forms same area cluster CL2 including plane clusters CL1.

Subsequently, data processing section <NUM> further searches for plane cluster CL1 including point data p having elevation value H maximized to value Hh and another plane cluster CL1 that is not combined with plane cluster CL1. If uncombined plane cluster CL1 is extracted, data processing section <NUM> makes a determination according to difference ΔH and confirms an overlap in the Y-axis direction. If plane cluster CL1 matching the conditions is found, plane cluster is added to same area cluster CL2.

Data processing section <NUM> repeatedly performs the processing until no plane cluster CL1 uncombined with plane cluster CL1 is detected, plane cluster CL1 including point data p having elevation value H maximized to value Hh. Through the processing, data processing section <NUM> forms same area cluster CL2.

Thereafter, point data p belonging to same area cluster CL2 formed thus is processed as a single shape in the display of guide information GD, which will be described later, and guide frame GD1 is displayed around same area cluster CL2.

"Clustering of the same area" is preferably configured as hierarchical clustering using a tree structure based on elevation values as illustrated in <FIG>. Data processing section <NUM> creates a tree structure using elevation value H for each ground object C in "clustering of the same area. " <FIG> illustrates a first example of ground object C in hierarchical clustering using a tree structure. <FIG> illustrates a second example of ground object C in hierarchical clustering using a tree structure.

In hierarchical clustering using a tree structure based on elevation values, plane cluster CL1 having a minimum mean value of elevation values H is set as "root" by data processing section <NUM>. If plane cluster CL1 overlaps plane cluster CL1 constituting "root" in the Y-axis direction, data processing section <NUM> extends "branch" from "root" and adds overlapping plane cluster CL1 to the end of "branch. " Subsequently, plane cluster CL1 having a maximum mean value of elevation values H is set as "slave" by data processing section <NUM>.

A method of generating guide frame GD1 will be described below.

Data processing section <NUM> acquires the three structure of ground object C, the tree structure being created in "clustering of the same area. " Moreover, data processing section <NUM> acquires point data p included in plane clusters CL1 constituting the tree structure.

As illustrated in the upper drawing of <FIG>, data processing section <NUM> acquires point data p on a laser lateral line placed at a maximum depth in the Z-axis direction, from point data p on "slave" plane clusters CL1. Furthermore, data processing section <NUM> creates a rectangle that is separated from an adjacent laser lateral line by a half of a distance from the adjacent laser lateral line in the Z-axis direction and has a width in the X-axis direction so as to surround point data p.

Subsequently, if point data p is present on the laser lateral line adjacent to the created rectangle, data processing section <NUM> deforms the rectangle so as to contain entire point data p on the corresponding laser lateral line, forming a visible outline as illustrated in the lower drawing of <FIG>.

Data processing section <NUM> then searches for point data p on the adjacent laser lateral line and repeats the processing until no point data p on the target laser lateral line is detected.

Finally, data processing section <NUM> creates a visible outline containing all plane clusters CL1 included in the selected tree structure.

As guide frame GD1, data processing section <NUM> outputs only a visible outline matching a condition from among created visible outlines.

As a condition for outputting a visible outline as guide frame GD1, for example, a condition for displaying only a visible outline serving as the overall frame of ground object C can be selected as illustrated in <FIG>. If the condition is selected, single guide frame GD1 surrounding overall ground object C is displayed for the ground object C on data display section <NUM>.

As a condition for outputting a visible outline as guide frame GD1, for example, a condition can be selected so as to display a visible outline for plane cluster CL1 having maximum elevation value H at each branch as illustrated in <FIG> from among visible outlines (sub frames) where a difference (difference ΔH) between elevation values H relative to "root" is not smaller than a threshold value. The visible outline is displayed in addition to the visible outline serving as the overall frame of ground object C. If the condition is selected, data display section <NUM> displays first guide frame GD1 surrounding overall ground object C and second guide frame GD1 included in first guide frame GD1, more specifically displaying guide information GD in consideration of the three-dimensional shape of ground object C.

As a condition for outputting a visible outline as guide frame GD1, for example, a condition can be selected so as to display all visible outlines (sub frames) where a difference (difference ΔH) between elevation values H relative to "root" is not smaller than a threshold value as illustrated in <FIG>. The visible outline is displayed in addition to the visible outline serving as the overall frame of ground object C. Also in the case where the condition is selected, data display section <NUM> displays first guide frame GD1 surrounding overall ground object C and second guide frame GD1 included in first guide frame GD1, more specifically displaying guide information GD in consideration of the three-dimensional shape of ground object C.

Such display conditions can be achieved also by adjusting the threshold value of difference ΔH. An operator can optionally select the display conditions of guide frame GD <NUM> so as to display guide information GD with greater ease of viewing.

In other words, in guide information display device <NUM>, guide frame GD1 is created based on same area cluster CL2, so that guide frame GD1 can be created so as to more specifically express ground object C in consideration of the three-dimensional shape of ground object C. Moreover, in guide information display device <NUM>, guide frame GD1 can be created around plane clusters CL1 present in the same area. In other words, guide information display device <NUM> can more specifically present guide information GD with ease of viewing.

As illustrated in <FIG>, "synchronous processing on point cloud data and camera video" is subsequently performed in data processing by data processing section <NUM> (STEP-<NUM>).

In this processing, as illustrated in <FIG>, point cloud data P acquired in the XYZ coordinate system is converted into the coordinate values of the camera-space coordinate system, is synchronized (positioned) with image M captured by camera <NUM>, and then is outputted to data display section <NUM>.

As indicated in <FIG>, "guide display" is then performed in data processing by data processing section <NUM> (STEP-<NUM>).

Data processing section <NUM> generates guide information GD based on information on generated same area clusters CL2 and outputs the information to data display section <NUM>.

For "guide display", "crane information" outputted from controller <NUM> of crane <NUM> is used. "Crane information" to be used includes information on the length and luffing angle of extendable/retractable boom <NUM>, the operating radius of crane <NUM>, the weight of suspended load W, and the like.

The series of data processing by data processing section <NUM> was described above. With this configuration, the three-dimensional shapes of suspended load W and ground object C can be accurately recognized and guide information GD can be generated with low computational complexity without the need for acquiring point data p on the sides of a measuring object. This configuration only requires low data computational complexity and thus is suitable for recognizing the shapes of suspended load W and ground object C in real time, allowing the use of data processing section <NUM> with a simple hardware configuration.

The contents of guide information GD will be described below.

In guide information display device <NUM>, data display section <NUM> displays guide information GD. Guide information GD displayed on data display section <NUM> includes information on a position specified on ground surface F by an operator as illustrated in <FIG>.

Moreover, suspended load W can be specified in guide information display device <NUM>. Suspended load W is specified on the screen as in the case where the operator specifies ground surface F, so that a plane (top surface) at the specified position is set to indicate the top surface of suspended load W. After suspended load W is specified, guide frame GD1 for suspended load W and guide frame GD1 for ground object C are preferably displayed with distinct line colors and thicknesses.

Information on the specified positions of ground surface F and suspended load W is displayed by markers shaped like circles.

Moreover, guide information GD displayed by data display section <NUM> includes guide frame GD1 created by data processing section <NUM>.

Data processing section <NUM> outputs guide frame GD1 based on set same area cluster CL2. Data processing section <NUM> can provide guide frame GD1 of suspended load W with allowance for reliably avoiding a collision and output a frame line as guide frame GD1, the frame line being offset from the visible outline of suspended load W to the outside by a predetermined distance. Guide frame GD1 is displayed as a frame including line segments surrounding top surfaces (plane clusters CL1) estimated on suspended load W and ground object C.

Moreover, guide information GD displayed by data display section <NUM> includes height information GD2 from reference height H0 to the underside of suspended load W and height information GD3 from reference height H0 to the top surface of ground object C.

Height information GD2 of suspended load W is preferably displayed in an independent area at a position easily viewed on the screen of data display section <NUM>.

Guide information display device <NUM> configured thus distinguishes between height information GD2 on suspended load W and height information GD3 on ground object C without fail.

Data processing section <NUM> calculates height information GD2 by subtracting the height of suspended load W from the height of the top surface of plane cluster CL1 that is assumed to be the top surface of suspended load W.

In guide information display device <NUM>, the operator inputs information on suspended load W (hereinafter, will be referred to as "suspended load information") in advance to data processing section <NUM>. "Suspended load information" is inputted from data input section <NUM> by the operator. Subsequently, data processing section <NUM> acquires the height of suspended load W by using "suspended load information.

Guide information display device <NUM> is configured such that height information GD3 on ground object C is displayed inside guide frame GD1 surrounding ground object C. Alternatively, in the case of small guide frame GD1, guide information display device <NUM> displays the height information partially superimposed on guide frame GD1.

Guide information display device <NUM> configured thus clarifies correspondence between ground object C and height information GD3.

Furthermore, guide information display device <NUM> is configured such that data processing section <NUM> changes the line color of guide frame GD1 according to elevation value H of plane cluster CL1 corresponding to guide frame GD1.

With this configuration, guide information display device <NUM> allows an operator to roughly perceive the elevation values (heights) of suspended load W and ground object C when viewing guide frame GD1. Thus, in guide information display device <NUM>, the heights of suspended load W and ground object C can be presented with higher accuracy.

Furthermore, guide information display device <NUM> is configured such that data processing section <NUM> changes the font color of height information GD2 according to elevation value H of plane cluster CL1 corresponding to guide frame GD <NUM>.

With this configuration, guide information display device <NUM> allows an operator to roughly perceive the elevation values (heights) of suspended load W and ground object C when viewing height information GD2. Thus, in guide information display device <NUM>, the heights of suspended load W and ground object C can be presented with higher accuracy.

Moreover, the display of guide information GD by guide information display device <NUM> includes path information on suspended load W. The path information on suspended load W includes operating radius information GD4 on suspended load W and axis information GD5 on extendable/retractable boom <NUM> of crane <NUM>.

Operating radius information GD4 is a guide for the path of suspended load W when extendable/retractable boom <NUM> is swiveled from a current status. Suspended load W moves along a circle indicated as operating radius information GD4.

Axis information GD5 is a guide for the path of suspended load W when extendable/retractable boom <NUM> is luffed up and extended or retracted from a current status. Suspended load W moves along a straight line indicated as operating radius information GD4.

In guide information display device <NUM>, operating radius information GD4 on suspended load W and axis information GD5 on extendable/retractable boom <NUM> are generated based on "crane information.

Data processing section <NUM> calculates the operating radius of crane <NUM> based on "crane information", generates a circle indicating the operating radius, and outputs the circle as operating radius information GD4.

Moreover, data processing section <NUM> calculates the axial direction of extendable/retractable boom <NUM> based on "crane information", generates a straight line indicating the axial direction, and outputs the straight line as axis information GD5.

Furthermore, guide information display device <NUM> is configured to display operating radius information GD4 and axis information GD5 as broken lines and display the lengths and intervals of the broken lines as guide lengths (hereinafter, will be referred to as a reference length). For example, in the case of a reference length of <NUM>, the lengths and intervals of the broken lines on the display of operating radius information GD4 and axis information GD5 are changed according to the size of suspended load area WA displayed on data display section <NUM>. The lengths and intervals are equivalent to <NUM> on ground surface F at a current scale.

Guide information display device <NUM> is configured to display the lengths and intervals of the broken lines with respect to the reference length (for example, <NUM>), allowing the operator to perceive the scale of suspended load W and ground object C based on guide information GD.

Moreover, data processing section <NUM> calculates the height of data acquisition section <NUM> based on "crane information", calculates the size of suspended load area WA and the size of the display range of data display section <NUM>, and changes the scale of the broken lines (the scale of the broken lines and the intervals) displayed as operating radius information GD4 and axis information GD5 according to the calculation result.

The display of guide information GD by guide information display device <NUM> includes warning display for preventing contact between suspended load W and ground object C.

Data processing section <NUM> determines that contact may occur if a horizontal distance is equal to or smaller than a predetermined threshold value (for example, <NUM>) or a vertical distance is equal to or smaller than a predetermined distance (for example, <NUM>) when suspended load W and ground object C are projected onto a horizontal plane.

Referring to <FIG>, the processing of the warning display for preventing contact between suspended load W and ground object C will be specifically described below. The threshold values serving as references in the warning display in the horizontal direction and the vertical direction are set at any values in order to prevent suspended load W and ground object C from coming into contact with each other due to vibrations of suspended load W or measurement errors or the like of the shapes of suspended load W and ground object C.

As indicated in <FIG>, the guide information display method includes vertical-approach determination step K101 of determining whether suspended load W and ground object C are approaching each other in the vertical direction depending on a difference in elevation value H between suspended load W and ground object C, visible-outline extension step K102 of horizontally moving the visible outline of suspended load W, non-overlap determination step K103 of determining whether a non-overlapping part appears between an extended visible outline and the visible outline of ground object C, line-segment overlap determination step K104 of determining whether the line segment of the extended visible outline and the line segment of the visible outline of ground object C overlap each other, inclusion determination step K105 of determining whether one of the extended visible outline and the visible outline of ground object C includes the other, vertex-to-vertex distance calculation step K106 of calculating distances between all the vertexes of the extended visible outline and the visible outline of ground object C in the horizontal direction, and vertex-to-line segment distance calculation step K107 of calculating distances between all the vertexes of the extended visible outline and all the line segments of the visible outline of ground object C in the horizontal direction. The visible outlines of suspended load W and ground object C are visible outlines surrounding overall suspended load W and ground object C, the visible outlines being formed in "clustering of the same area" (STEP-<NUM>) (see <FIG>).

As illustrated in <FIG>, data processing section <NUM> first calculates a difference in elevation value H between suspended load W and ground object C (vertical-approach determination step K101: STEP-<NUM>). Elevation value H of suspended load W is the elevation value of the underside of suspended load W and is calculated as height information GD2. Moreover, elevation value H of ground object C is elevation value H of the top surface of ground object C and is calculated as height information GD3.

Data processing section <NUM> determines whether a difference in elevation value H between suspended load W and ground object C is equal to or smaller than a threshold value in the vertical direction (vertical-approach determination step K101: STEP-<NUM>).

If it is determined that a difference in elevation value H between suspended load W and ground object C is larger than the threshold value in the vertical direction in STEP-<NUM>, data processing section <NUM> terminates the processing of warning display.

If it is determined that a difference in elevation value H between suspended load W and ground object C is equal to or smaller than the threshold value in the vertical direction in STEP-<NUM>, data processing section <NUM> starts overlap determination A in the horizontal direction in order to determine an overlap of the visible outline of suspended load W and the visible outline of ground object C on an XZ plane (STEP-<NUM>).

As indicated in <FIG> and <FIG>, in overlap determination A in the horizontal direction, data processing section <NUM> first generates an extended visible outline that is a visible outline formed by horizontally moving the sides of the visible outline of suspended load W in a direction that extends a range surrounded by the visible outline, by a distance as large as the threshold value in the horizontal direction (visible-outline extension step K102: STEP-<NUM>).

As indicated in <FIG>, data processing section <NUM> determines whether the sum of point data p included in the extended visible outline and the visible outline of ground object C is at least two points (STEP-<NUM>).

If it is determined that the sum of point data p included in the extended visible outline and the visible outline of ground object C is not at least two points in STEP-<NUM>, data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C do not overlap each other on the XZ plane (STEP-<NUM>).

If it is determined that the sum of point data p included in the extended visible outline and the visible outline of ground object C is at least two points in STEP-<NUM>, data processing section <NUM> determines whether a non-overlapping part appears between the extended visible outline and the visible outline of ground object C on the XZ plane (non-overlap determination step K103: STEP-<NUM>).

Whether a non-overlapping part appears between the extended visible outline and the visible outline of ground object C on the XZ plane is determined based on the maximum value and the minimum value of the vertexes of the extended visible outline and the visible outline of ground object C on the XZ plane in a coordinate system where coordinate axes are the lateral line direction (X-axis direction) of laser radiation by laser scanner <NUM> and the horizontal direction (Z-axis direction) perpendicular to the lateral line direction. Specifically, on an XZ plane at all the vertexes of the extended visible outline, MaxX1 is the maximum value of an X coordinate, MinX1 is the minimum value of an X coordinate, MaxZ1 is the maximum value of a Z coordinate, and MinZ1 is the minimum value of a Z coordinate. On an XZ plane at all the vertexes of the visible outline of ground object C, MaxX2 is the maximum value of an X coordinate, MinX2 is the minimum value of an X coordinate, MaxZ2 is the maximum value of a Z coordinate, and MinZ2 is the minimum value of a Z coordinate. If any one of the conditions of following Expressions (<NUM>) to (<NUM>) is satisfied, data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C do not overlap each other on the XZ plane (STEP-<NUM>). <MAT> <MAT> <MAT> <MAT>.

As illustrated in <FIG>, the condition of Expression (<NUM>) is satisfied if minimum value MinX2 of the visible outline of ground object C is larger than maximum value MaxX1 of the extended visible outline (see <FIG>), the condition of Expression (<NUM>) is satisfied if minimum value MinZ2 of the visible outline of ground object C is larger than maximum value MaxZ1 of the extended visible outline (see <FIG>), the condition of Expression (<NUM>) is satisfied if minimum value MinX1 of the extended visible outline is larger than maximum value MaxX2 of the visible outline of ground object C (see <FIG>), and the condition of Expression (<NUM>) is satisfied if minimum value MinZ1 of the extended visible outline is larger than maximum value MaxZ2 of the visible outline of ground object C (see <FIG>).

As illustrated in <FIG>, if any one of the conditions of Expressions (<NUM>) to (<NUM>) is not satisfied, a line segment of the extended visible outline and a line segment of ground object C overlap each other (see <FIG>), one of the extended visible outline and the visible outline of ground object C includes the other (see <FIG>), or the extended visible outline and the visible outline of ground object C do not overlap each other while one of the extended visible outline and the visible outline of ground object C does not contain the other (see <FIG>).

As indicated in <FIG>, if any one of the conditions of Expressions (<NUM>) to (<NUM>) is not satisfied in STEP-<NUM>, data processing section <NUM> then determines whether a line segment of the extended visible outline and a line segment of the visible outline of ground object C overlap each other on the XZ plane in determination according to outer products (line-segment overlap determination step K104: STEP-<NUM>).

In the determination according to outer products in STEP-<NUM>, the outer products of two line segments are calculated. Specifically, the outer products of line segment ab of the extended visible outline and line segment cd of the visible outline of ground object C are calculated by Expressions (<NUM>) to (<NUM>) below, in which ax is the X coordinate of endpoint a of line segment ab, az is the Z coordinate of endpoint a, bx is the X coordinate of endpoint b, bz is the Z coordinate of endpoint b, cx is the X coordinate of endpoint c of linear segment cd, cz is the Z coordinate of endpoint c, dx is the X coordinate of endpoint d, and dz is the Z coordinate of endpoint d. The outer products of the two line segments are the outer products of one line segment and both endpoints of the other line segment. Values for processing the calculation results of the outer products are expressed as ta, tb, tc, and td. ta is the outer product of line segment cd and endpoint a, tb is the outer product of line segment cd and endpoint b, tc is the outer product of line segment ab and endpoint c, and tb is the outer product of line segment ab and endpoint d. <MAT> <MAT> <MAT> <MAT>.

If the calculation result of the outer products of the two line segments are tc × td ≤ <NUM> and ta × tb ≤ <NUM>, data processing section <NUM> determines that line segment ab and line segment cd overlap each other in STEP-<NUM>. For all the line segments of the extended visible outline and all the line segments of the visible outline of ground object C, data processing section <NUM> makes determinations according to the outer products in STEP-<NUM>.

Data processing section <NUM> makes the determinations of all the line segments of the extended visible outline and all the line segments of the visible outline of ground object C in STEP-<NUM>. If it is determined that at least one of the line segments overlaps in the determination (see <FIG>), data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C overlap each other on the XZ plane (STEP-<NUM>).

Data processing section <NUM> makes the determinations of all the line segments of the extended visible outline and all the line segments of the visible outline of ground object C in STEP-<NUM>. If it is determined that the line segments do not overlap at all, data processing section <NUM> determines whether the extended visible outline is included in the visible outline of ground object C (inclusion determination step K105: STEP-<NUM>).

In the determination of whether the extended visible outline is included in the visible outline of ground object C, it is first determined whether overlaps are made between a half line having an endpoint at any vertex of the extended visible outline and all the line segments of the visible outline of ground object C, according to the same calculation method as the determination according to outer products in STEP-<NUM>. The number of overlaps is then calculated. If the number of overlaps is an odd number, data processing section <NUM> determines that the extended visible outline is included in the visible outline of ground object C. In the determination according to outer products, a virtual endpoint is provided on a half line separated from another endpoint so as to determine an overlap, thereby calculating the outer products of the line segments.

If it is determined that the extended visible outline is included in the visible outline of ground object C in STEP-<NUM> (see <FIG>), data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C overlap each other on the XZ plane (STEP-<NUM>).

If it is determined that the extended visible outline is not included in the visible outline of ground object C in STEP-<NUM>, data processing section <NUM> determines whether the visible outline of ground object C is included in the extended visible outline (inclusion determination step K105: STEP-<NUM>).

The determination of whether the visible outline of ground object C is included in the extended visible outline is processing reversed from the determination of whether the extended visible outline is included in the visible outline of ground object C. In other words, data processing section <NUM> determines whether overlaps are made between a half line having an endpoint at any vertex of the visible outline of ground object C and all the line segments of the extended visible outline, according to the same calculation method as the determination according to outer products in STEP-<NUM>. The number of overlaps is then calculated. If the number of overlaps is an odd number, data processing section <NUM> determines that the visible outline of ground object C is included in the extended visible outline. In the determination according to outer products, a virtual endpoint is provided on a half line separated from another endpoint so as to determine an overlap, thereby calculating the outer products of the line segments.

If it is determined that the visible outline of ground object C is included in the extended visible outline in STEP-<NUM>, data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C overlap each other on the XZ plane (STEP-<NUM>).

If it is determined that the visible outline of ground object C is not included in the extended visible outline in STEP-<NUM> (see <FIG>), data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C do not overlap each other on the XZ plane (STEP-<NUM>).

In STEP-<NUM>, data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C overlap each other on the XZ plane and terminates overlap determination A in the horizontal direction. In STEP-<NUM>, data processing section <NUM> determines that the extended visible outline and the visible outline of ground object C do not overlap each other on the XZ plane and terminates overlap determination A in the horizontal direction.

As indicated in <FIG>, data processing section <NUM> determines whether the extended visible outline and the visible outline of ground object C overlap each other on the XZ plane, based on the determination result in overlap determination A in the horizontal direction (STEP-<NUM>). If it is determined that the extended visible outline and the visible outline of ground object C overlap each other on the XZ plane, a warning is displayed on data display section <NUM> (STEP-<NUM>).

In STEP-<NUM>, if it is determined that the extended visible outline and the visible outline of ground object C do not overlap each other on the XZ plane, data processing section <NUM> performs horizontal-approach determination B (STEP-<NUM>).

As indicated in <FIG>, in horizontal-approach determination B, data processing section <NUM> first calculates distances between the visible outline of suspended load W and the visible outline of ground object C and all the vertexes in the horizontal direction (vertex-to-vertex distance calculation step K106: STEP-<NUM>).

Data processing section <NUM> then determines whether at least one of the calculated distances between the vertexes in the horizontal direction is equal to or smaller than the threshold value of warning display in the horizontal direction (STEP-<NUM>).

In STEP-<NUM>, if it is determined that at least one of the calculated distances between the vertexes in the horizontal direction is equal to or smaller than the threshold value in the horizontal direction, data processing section <NUM> determines that warning display is necessary (STEP-<NUM>).

If it is determined that all of the calculated distances between the vertexes in the horizontal direction are larger than the threshold value in the horizontal direction in STEP-<NUM>, data processing section <NUM> calculates distances between all the vertexes of the visual outline of suspended load W and all the line segments of the visual outline of ground object C in the horizontal direction (vertex-to-line segment distance calculation step K107: STEP-<NUM>). As a distance between a vertex of the visual outline of suspended load W and a line segment of the visual outline of ground object C in the horizontal direction, data processing section <NUM> calculates a distance from a vertex of the visual outline of suspended load W to the nearest point on a line segment of the visual outline of ground object C.

Subsequently, data processing section <NUM> determines whether at least one of the calculated distances between the vertexes and the line segments in the horizontal direction is warning display in the horizontal direction (STEP-<NUM>).

In STEP-<NUM>, if it is determined that at least one of the calculated distances between the vertexes and the line segments in the horizontal direction is equal to or smaller than the threshold value of warning display in the horizontal direction, data processing section <NUM> determines that warning display is necessary (STEP-<NUM>).

In STEP-<NUM>, if it is determined that the calculated distances between the vertexes and the line segments in the horizontal direction are larger than the threshold value of warning display in the horizontal direction, data processing section <NUM> determines that warning display is not necessary (STEP-<NUM>).

In STEP-<NUM>, data processing section <NUM> determines that warning display is necessary and terminates horizontal-approach determination B. In STEP-<NUM>, data processing section <NUM> determines that warning display is not necessary and terminates horizontal-approach determination B.

As indicated in <FIG>, data processing section <NUM> determines whether warning display is necessary or not based on the determination result in horizontal-approach determination B (STEP-<NUM>).

If it is determined that warning display is necessary in STEP-<NUM>, data processing section <NUM> displays a warning on data display section <NUM> (STEP-<NUM>) and then terminates the processing of warning display.

If it is determined that warning display is not necessary in STEP-<NUM>, data processing section <NUM> terminates the processing of warning display.

As illustrated in <FIG>, guide information GD and image M that are generated by data processing section <NUM> are displayed so as to be superimposed on each other on data display section <NUM>. If a distance between suspended load W and ground object C in the horizontal direction is equal to or smaller than a threshold value in the horizontal direction and a distance in the vertical direction is equal to or smaller than a threshold value in the vertical direction, guide frame GD1 and height information GD2 on ground object C that may come into contact with suspended load W are outputted from data processing section <NUM> so as to blink in any colors. Alternatively, guide frame GD1 and height information GD2 on ground object C are outputted with thicker lines and larger font sizes from data processing section <NUM>. In guide information display device <NUM>, guide frame GD1 and height information GD2 on ground object C serving as warning display are outputted from data processing section <NUM> and is displayed on data display section <NUM>, thereby calling attention to an operator. With this configuration, guide information display device <NUM> can accurately obtain the shapes of suspended load W and ground object C near the suspended load and the height of ground surface F and provide accurate warning display if suspended load W approaches ground object C.

When a distance between suspended load W and ground object C in the horizontal direction is equal to or smaller than a threshold value, data processing section <NUM> may change the colors or the intervals of blinking of guide frame GD1 and height information GD2 on ground object C according to a change of the distance so as to display a warning that the possibility of contact increases or decreases. Alternatively, data processing section <NUM> may display, as guide frame GD1, an extended visible outline and a visible outline before being extended. The output of a warning is not limited to warning display on data display section <NUM>. Warning sound or voice may be outputted instead through a speaker.

Moreover, in guide information display device <NUM>, an excluded area JA is set between suspended load W and top boom member 22f as illustrated in <FIG> when guide information GD is generated by data processing section <NUM>. Data processing section <NUM> then excludes point data p acquired in excluded area JA, from data processing.

Main wire rope <NUM> passes through excluded area JA. In guide information display device <NUM>, main wire rope <NUM> is not included in the generation (measuring object) of guide information GD, thereby presenting more accurate guide information GD with ease of viewing. The lower end of excluded area JA has a height set at a position separated from the top surface of suspended load W by a predetermined distance so as not to affect the generation of guide frame GD1 of suspended load W.

Guide information display device <NUM> configured thus can accurately present guide information GD to an operator of crane <NUM>, guide information GD including guide frame GD1 on the shapes of suspended load W and ground object C near suspended load W and height information GD2 and GD3 on the heights of suspended load W and ground object C. For example, even if an operator cannot directly recognize suspended load W, guide information display device <NUM> configured thus allows the operator to efficiently and safely operate crane <NUM> based on guide information GD indicated by guide information display device <NUM>.

Claim 1:
A guide information display device (<NUM>), comprising:
a data acquisition section (<NUM>) including a camera (<NUM>) and a laser scanner (<NUM>), the camera being configured to capture a part of a work area of a crane, the laser scanner (<NUM>) being configured to acquire point cloud data from above a suspended load (W) in the work area; and
a data processing section (<NUM>) that is configured to
estimate a top surface of the suspended load (W), a ground surface (F) of the work area, and a top surface of a ground object (C) in the work area based on the point cloud data acquired by the laser scanner (<NUM>),
calculate elevation values (H) of point data which is a height from a reference height (H0) defined as the height of the ground surface (F),
generate a guide frame image of a guide frame surrounding the top surface of the suspended load (W) and a guide frame surrounding the top surface of the ground object (C) and an elevation value image of elevation values (H) of the suspended load (W) and the ground object (C),
calculate a distance between the suspended load (W) and the ground object (C) based on the estimated top surface of the suspended load (W) and the ground object (C), wherein the distance comprises the difference in elevation value (H) between the suspended load (W) and the ground object (C), such that the elevation value (H) of the suspended load (W) is an elevation value of the underside of suspended load (W) and the elevation value (H) of the ground object (C) is an elevation value (H) of the top surface of the ground object (C), and
output warning display in a case where the distance is equal to or smaller than a threshold value; and
a data display section that is configured to display guide information generated by superimposing the guide frame image, the elevation value image and the warning display that are generated by the data processing section and the image captured by the camera with one another.