Patent Description:
Crane information display systems that display information about the cranes have been known (e.g., refer to PTL <NUM>).

PTL <NUM> discloses a configuration that displays an operation state of a crane to an operator, and displays an operation state of a crane to a person outside the crane with a mobile terminal. Consequently, a person outside the crane can grasp an operation state of the crane.

PLT <NUM> discloses a crane information display system comprising: a terminal device that mounts a camera to obtain a camera image by imaging, with a camera, an information processing unit configured to convert the image information about the crane, into image information that corresponds to the position and the posture of the crane; and an image display unit configured to overlay and display, onto the camera image to which the processing by the information processing unit is configured to convert the image information has been applied.

PLT <NUM> discloses an operation assist apparatus of a construction machine, at which at least operating procedures corresponding to an operation of the construction machine are displayed with text and illustrations.

PLT <NUM> discloses a system for determining the position of an offshore structure in a fixed reference frame by observations of a plurality of markers located at defined positions on the structure by an imaging device in a moving reference frame.

PLT <NUM> discloses a remote operation terminal for remotely operating a work machine displaying a virtual object indicating a part of the work machine or an object to be moved, and simulating operation of the virtual object for sending, to the work machine, operation information input from a operation unit during a simulated operation.

However, the configuration described in Patent Literature <NUM> simply displays an operation state of an actual crane, and has a problem that a work plan of the crane cannot be made without disposing the crane in a place in which the crane is going to be installed.

It is therefore an object of the present invention to provide a crane information display system that can make a work plan of a crane without disposing the crane in a place in which the crane is going to be installed.

A main present disclosure that solves the problems described above is directed to a crane information display system according to claim <NUM>.

The crane information display system according to the present invention configured as described above can make a work plan of the crane without disposing the crane in a place in which the crane is going to be installed.

Hereinafter, an embodiment that realizes a crane information display system according to the present invention will be described on the basis of a first embodiment illustrated in the drawings.

<FIG> is a drawing that illustrates a crane information display system according to a first embodiment. <FIG> is a side view that illustrates a crane according to the first embodiment. Hereinafter, the configuration of the crane information display system according to the first embodiment will be described. Note that an actual crane is a crane <NUM>, and a virtual crane is a virtual crane 1A. Further, forward and backward directions of the crane <NUM> are forward and backward directions D.

As illustrated in <FIG>, a crane information display system <NUM> according to the first embodiment will be described about an example in which a worker M images, with a tablet terminal <NUM> as a user terminal that includes a camera <NUM>, a work site in which the crane <NUM> is going to be installed.

As illustrated in <FIG>, the crane <NUM> includes a travelling body <NUM>, a rotating body <NUM>, and a boom <NUM>.

The travelling body <NUM> includes a vehicle body frame <NUM>, outriggers <NUM>, a travelling device for travelling by itself a road and a work site, and the like.

The outriggers <NUM> include rear outriggers <NUM> attached to rear side surfaces of the vehicle body frame <NUM> of the travelling body <NUM>, and front outriggers <NUM> attached to front side surfaces of the vehicle body frame <NUM>. The outriggers <NUM> are accommodated in the vehicle body frame <NUM> at a time of travelling. On the other hand, the outriggers <NUM> protrude in a horizontal direction and a perpendicular direction at a time of work to lift the whole vehicle body and stabilize the posture.

The rotating body <NUM> is provided over the travelling body <NUM> and can rotate on a vertical axis C1 relative to the travelling body <NUM>. The rotating body <NUM> includes a cabin <NUM>. The cabin <NUM> includes an operation unit for controlling travel of the travelling body <NUM> (e.g., a steering wheel, a shift lever, an accelerator pedal, a brake pedal, and the like). Further, the cabin <NUM> includes an operation unit for operating the rotating body <NUM>, the boom <NUM>, a winch, and the like. A worker who has boarded the cabin <NUM> operates the operation unit to rotate the rotating body <NUM>, raise and lower and extend and contract the boom <NUM>, and turn the winch to perform work.

The base end side of the boom <NUM> is supported by the rotating body <NUM>. The boom <NUM> is attached to the rotating body <NUM> in such a manner that the boom <NUM> can be raised and lowered relative to the rotating body <NUM>. The boom <NUM> is raised and lowered by a raising and lowering cylinder <NUM> provided for the rotating body <NUM>, and is extended and contracted by an extending and contracting cylinder (not illustrated).

The boom <NUM> includes intermediate booms <NUM> to <NUM> between a base end boom <NUM> on the base end side and a front end boom <NUM> on the front end side. The intermediate booms <NUM> to <NUM> and the front end boom <NUM> are telescopic to be accommodated sequentially in the base end boom <NUM>.

A sheave <NUM> is disposed at a boom head 36a provided at the front end of the front end boom <NUM>. A wire rope <NUM> for a hung load is wound around the winch of the rotating body <NUM> provided near the base end of the boom <NUM>. The wire rope <NUM> is disposed along an axis direction of the boom <NUM> from the winch to the sheave <NUM>. The wire rope <NUM> wound around the sheave <NUM> is hung downward in a vertical direction from the sheave <NUM>. A hook <NUM> is provided at a lowest portion of the wire rope <NUM>.

A load is hung on the hook <NUM>, and the wire rope <NUM> wound around the winch is let out to lower the hook <NUM>. The wire rope <NUM> is wound up to raise the hook <NUM>.

The winch lets out and winds up the wire rope <NUM>, the boom <NUM> is raised, lowered, extended, and contracted, and the rotating body <NUM> is rotated so that the crane <NUM> configured as described above moves a load hung from the hook <NUM> to a predetermined position.

As illustrated in <FIG>, the tablet terminal <NUM> includes the camera <NUM>, an image display unit <NUM>, and an input unit <NUM>.

An image imaged with the camera <NUM> is displayed on the image display unit <NUM>. The image display unit <NUM> is also configured as a touch panel as the input unit <NUM>.

As illustrated in <FIG>, a first information display unit <NUM> (that is to say, a marker) is attached to a support member <NUM>, such as a standing signboard. The support member <NUM> to which the first information display unit <NUM> is attached is installed in a place in which the crane <NUM> is going to be installed.

The first information display unit <NUM> may be an AprilTag. In the first information display unit <NUM>, information about the machine type of the crane <NUM>, the shape and size of the first information display unit <NUM>, and information about a predetermined position of the crane <NUM> are stored. The information about the predetermined position of the crane <NUM> may be the center of the left side surface of the vehicle body frame <NUM> of the travelling body <NUM>.

As illustrated in <FIG>, a second information display unit <NUM> (that is to say, a marker) is installed at a position onto which the crane <NUM> is going to unload a hung load, e.g., a rooftop of a building B. The second information display unit <NUM> may be an AprilTag. In the second information display unit <NUM>, the shape and size of the second information display unit <NUM> are stored. The first information display unit <NUM> and the second information display unit <NUM> have different display information (that is to say, code information) due to, for example, characters, marks, signs, patterns, or the like provided for the surfaces of the first information display unit <NUM> and the second information display unit <NUM>.

The display information (that is to say, code information) that the first information display unit <NUM> and the second information display unit <NUM> have respectively can be read from a camera image generated by the camera <NUM>, using a decipherment program preliminarily stored in a control unit <NUM> (described below), or the like. Further, the first information display unit <NUM> and the second information display unit <NUM> can be identified from the display information (that is to say, code information) that the first information display unit <NUM> and the second information display unit <NUM> have respectively. Note that the shapes and sizes stored in the first information display unit <NUM> and the second information display unit <NUM> are, for example, the shapes and sizes obtained when the first information display unit <NUM> and the second information display unit <NUM> are imaged from the fronts at a predetermined distance (hereinafter referred to as "reference shapes" and "reference sizes").

<FIG> is a block diagram that illustrates a functional configuration of the crane information display system <NUM> according to the first embodiment. <FIG> is a drawing that illustrates an image displayed on the image display unit <NUM> according to the first embodiment. Hereinafter, the functional configuration of the crane information display system <NUM> according to the first embodiment will be described.

As illustrated in <FIG>, the crane information display system <NUM> is such that an image of the first information display unit <NUM> imaged with the camera <NUM>, an image of the second information display unit <NUM> imaged with the camera <NUM>, and input information input into the input unit <NUM> are input into the control unit <NUM> (in the present embodiment, a control unit incorporated in the tablet terminal <NUM>), and information controlled by the control unit <NUM> is output on the image display unit <NUM>.

The camera <NUM> may be, for example, the camera <NUM> that the tablet terminal <NUM> that is common includes. The camera <NUM> can image a work site in which the crane <NUM> is going to be installed, the first information display unit <NUM>, and the second information display unit <NUM>.

A protrusion amount by which the outriggers <NUM> protrude in a horizontal direction, a hung load weight, the length of the boom <NUM>, and the like can be input into the input unit <NUM>. The length of the boom <NUM> is the length of the boom <NUM> in a state in which the front end boom <NUM> and the intermediate booms <NUM> to <NUM> are accommodated in the base end boom <NUM> (a completely contracted state), the length of the boom <NUM> in a state in which the front end boom <NUM> extends, the length of the boom <NUM> in a state in which the front end boom <NUM> and the intermediate booms <NUM> to <NUM> extend (a completely extending state), or the like.

The control unit <NUM> includes a storage unit <NUM>, a crane information acquisition unit <NUM>, a virtual crane generation unit <NUM>, a position/posture calculation unit <NUM>, a position calculation unit <NUM>, and an information processing unit <NUM>. Note that the control unit <NUM> is, for example, a widely-known microcomputer that includes a central processing unit (CPU), random access memory (RAM), read only memory (ROM), and the like. The functions of the control unit <NUM> (the crane information acquisition unit <NUM>, the virtual crane generation unit <NUM>, the position/posture calculation unit <NUM>, and the position calculation unit <NUM>, and the information processing unit <NUM>) are realized by, for example, the CPU referring to control programs and various data stored in the storage unit <NUM> (e.g., a hard disk drive (HDD), the ROM, or the RAM).

The storage unit <NUM> stores shape information about various types of cranes, and performance information about various types of cranes. The performance information includes, for example, information regarding the shapes of the outriggers <NUM>, information regarding possible work area in a horizontal direction and information regarding possible work area in a height direction that correspond to the protrusion amount of the outriggers <NUM> and the length of the boom <NUM>, information regarding a range of angles by which the boom is raised or lowered, information regarding load ratios, information regarding a tail swing area, and the like.

On the basis of the first information display unit <NUM> imaged with the camera <NUM>, the crane information acquisition unit <NUM> acquires information about the machine type of the crane <NUM>, and acquires, from the storage unit <NUM>, shape information and performance information about the acquired machine type of the crane <NUM>. That is to say, the crane information acquisition unit <NUM> reads into the first information display unit <NUM> imaged with the camera <NUM> (that is to say, reads display information of the first information display unit <NUM> from the camera image), acquires the machine type of the crane <NUM>, and acquires, from the storage unit <NUM>, shape information and performance information about the acquired machine type.

The crane information acquisition unit <NUM> includes a virtual outrigger generation unit <NUM>, and a possible work area calculation unit <NUM>.

On the basis of information regarding the shapes of the outriggers <NUM> stored in the storage unit <NUM> (that is to say, the shapes of the outriggers <NUM> that correspond to the machine type of the crane <NUM> specified from display information of an information display unit <NUM>), and a protrusion amount by which the outriggers <NUM> protrude in a horizontal direction that has been input into the input unit <NUM>, the virtual outrigger generation unit <NUM> generates three-dimensional data on virtual outriggers 80A, as information about the crane <NUM>.

The possible work area calculation unit <NUM> refers to information about the machine type of the crane <NUM> specified from the display information of the information display unit <NUM>, and calculates possible work area of the crane <NUM>, as information about the crane <NUM>, on the basis of a hung load weight and the length of the boom <NUM> input into the input unit <NUM>. The possible work area calculation unit <NUM> can calculate possible work area for a plurality of load ratios. In the first embodiment, the possible work area calculation unit <NUM> calculates a possible work area for a load ratio of <NUM>%, and a possible work area for a load ratio of <NUM>%. Note that the possible work area is an area in which the crane <NUM> can work in a horizontal direction, in a plane in which the crane <NUM> is installed.

The virtual crane generation unit <NUM> generates three-dimensional data on a virtual crane 1A, on the basis of the information about the crane <NUM> acquired by the crane information acquisition unit <NUM>.

The position/posture calculation unit <NUM> calculates the position and posture of the virtual crane 1A (that is to say, the position at which the virtual crane 1A is to be disposed and the posture to be taken by the virtual crane 1A in the image of the camera <NUM>) on the basis of the first information display unit <NUM> imaged with the camera <NUM>.

More specifically, the position/posture calculation unit <NUM> acquires information about the posture of the virtual crane 1A, on the basis of the shape of the first information display unit <NUM> imaged with the camera <NUM> (hereinafter referred to as an "outline shape of the information display unit <NUM>"), and the reference shape of the first information display unit <NUM> stored in the first information display unit <NUM>. That is to say, the position/posture calculation unit <NUM> reads, from an image of the camera <NUM>, display information (that is to say, code information) of the first information display unit <NUM> to acquire information regarding the reference shape of the first information display unit <NUM>, and extracts, from the image of the camera <NUM>, an outline shape of the first information display unit <NUM> that appears in the camera <NUM> by publicly-known pattern matching, or the like. Then the position/posture calculation unit <NUM> compares the outline shape of the first information display unit <NUM> that appears in the image of the camera <NUM>, with the reference shape of the first information display unit <NUM> to calculate the position at which the virtual crane 1A is to be disposed and the posture to be taken by the virtual crane 1A in the image of the camera <NUM>.

For example, if the shape of the first information display unit <NUM> imaged with the camera <NUM> is a rectangular shape stored in the first information display unit <NUM> (that is to say, if the outline shape of the first information display unit <NUM> that appears in the image of the camera <NUM> and the reference shape of the first information display unit <NUM> specified from the display information of the first information display unit <NUM> are both a rectangular shape), the posture is such that the left side surface of the virtual crane 1A is right in front of the camera <NUM>. For example, if the shape of the first information display unit <NUM> imaged with the camera <NUM> is not a rectangular shape stored in the first information display unit <NUM> but is a trapezoid such that the left edge of the information display unit <NUM> is longer (higher) than the right edge (that is to say, if the outline shape of the first information display unit <NUM> that appears in the image of the camera <NUM> is a trapezoid that has a left edge longer than the right edge, and the reference shape of the first information display unit <NUM> specified from the display information of the first information display unit <NUM> is a rectangular shape), the posture is such that the left side surface of the virtual crane 1A slightly faces left from right in front of the camera <NUM>. If the first information display unit <NUM> imaged with the camera <NUM> does not have a rectangular shape stored in the first information display unit <NUM> but is trapezoidal such that the right edge of the first information display unit <NUM> is longer than the left edge (that is to say, if the outline shape of the first information display unit <NUM> that appears in the image of the camera <NUM> is a trapezoid that has a right edge longer than the left edge, and the reference shape of the first information display unit <NUM> specified from the display information of the first information display unit <NUM> is a rectangular shape), the posture is such that the left side surface of the virtual crane 1A slightly faces right from right in front of the camera <NUM>.

That is to say, the position/posture calculation unit <NUM> compares the outline shape of the first information display unit <NUM> imaged with the camera <NUM>, with the reference shape of the first information display unit <NUM> stored in the first information display unit <NUM> to acquire information about the posture of the virtual crane 1A.

Further, the position/posture calculation unit <NUM> acquires information about the position of the virtual crane 1A, on the basis of the size of the outline shape of the first information display unit <NUM> imaged with the camera <NUM>, and the reference size of the first information display unit <NUM> stored in the first information display unit <NUM>. More specifically, the position/posture calculation unit <NUM> compares the reference size of the first information display unit <NUM> stored in the first information display unit <NUM> with the size of the outline shape of the first information display unit <NUM> imaged with the camera <NUM> to calculate the distance from the camera <NUM> to the first information display unit <NUM>.

The position calculation unit <NUM> calculates information about a relative position of the second information display unit <NUM> from the first information display unit <NUM>, on the basis of the second information display unit <NUM> imaged with the camera <NUM>. More specifically, the position calculation unit <NUM> reads into the second information display unit <NUM> imaged with the camera <NUM> (that is to say, reads, from the image of the camera <NUM>, display information of the second information display unit <NUM>), and calculates the distance and direction from the first information display unit <NUM>, with the position of the first information display unit <NUM> as the reference. That is to say, the position calculation unit <NUM> calculates information about the distance of the second information display unit <NUM> in a horizontal direction, and information about the distance of the second information display unit <NUM> in a height direction, with the first information display unit <NUM> as the reference.

More specifically, the position calculation unit <NUM> functions in a state in which the first information display unit <NUM> and the second information display unit <NUM> both appear in an image of the camera <NUM>. For example, the position calculation unit <NUM> acquires the reference shape of the second information display unit <NUM>, extracts an outline shape of the second information display unit <NUM> that appears in an image of the camera <NUM>, and specifies the position of the second information display unit <NUM> in the image of the camera <NUM>, on the basis of information about the reference shape of the second information display unit <NUM>, and the outline shape of the second information display unit <NUM>. Then the position calculation unit <NUM> refers to the position of the first information display unit <NUM> in the image of the camera <NUM> calculated by the position/posture calculation unit <NUM>, and the position of the second information display unit <NUM> in the image of the camera <NUM> calculated by the position calculation unit <NUM> to calculate information about the relative position of the second information display unit <NUM> from the first information display unit <NUM>. However, if the positional relationship between an image of the camera <NUM> in which a first information display unit <NUM> appears and an image of the camera <NUM> in which a second information display unit <NUM> appears is specified by simultaneous localization and mapping (SLAM) technology or the like, the function of the position calculation unit <NUM> can be realized even if the first information display unit <NUM> and the second information display unit <NUM> both do not appear in one image.

The information processing unit <NUM> processes the virtual crane 1A generated by the virtual crane generation unit <NUM>, into information that corresponds to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>. In other words, the information processing unit <NUM> converts image information about the virtual crane 1A generated by the virtual crane generation unit <NUM>, into image information that corresponds to the position at which the virtual crane 1A is to be disposed and the posture to be taken by the virtual crane 1A in an image of the camera <NUM> calculated by the position/posture calculation unit <NUM>. For example, the information processing unit <NUM> converts image information about the virtual crane 1A in such a manner that when a position at which the crane is going to be installed (that is to say, a position at which the first information display unit <NUM> is installed) is seen from the imaging position of the camera <NUM>, a three-dimensional image of the virtual crane 1A becomes an image that simulates a state in which the crane <NUM> actually exists. Note that such image processing by the information processing unit <NUM> is realized by publicly-known coordinate conversion processing or the like.

Further, the information processing unit <NUM> processes the performance information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outriggers 80A generated by the virtual outrigger generation unit <NUM>, and the possible work area of the crane <NUM> calculated by the possible work area calculation unit <NUM>, into information that corresponds to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>. That is to say, the information processing unit <NUM> processes the information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, into information that corresponds to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>.

Further, on the basis of the position information calculated by the position calculation unit <NUM>, the information processing unit <NUM> processes the information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, into information that corresponds to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>, with the second information display unit <NUM> as the reference. In other words, the information processing unit <NUM> processes the information about the crane <NUM> that has been made to correspond to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>, into information with the second information display unit <NUM> as the reference. That is to say, on the basis of the relative position of the second information display unit <NUM> from the first information display unit <NUM> calculated by the position calculation unit <NUM>, the information processing unit <NUM> converts the information about the crane <NUM> acquired by the crane information acquisition unit <NUM> (for example, possible work area <NUM> and <NUM> of the crane <NUM>), into three-dimensional image information in which the information about the crane <NUM> acquired by the crane information acquisition unit <NUM> (for example, the possible work area <NUM> and <NUM> of the crane <NUM>) corresponds to a work position at a time when the crane <NUM> unloads a hung load.

The image display unit <NUM> overlays and displays the information processed by the information processing unit <NUM> onto an image of the camera <NUM>. More specifically, as illustrated in <FIG>, the image display unit <NUM> superimposes and displays, on an image of a work site imaged with the camera <NUM> in which the crane <NUM> is going to be installed, the virtual crane 1A, a tail swing area <NUM> of the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outriggers 80A generated by the virtual outrigger generation unit <NUM>, possible work area <NUM> and <NUM> in a plane in which the crane <NUM> is installed that has been calculated by the possible work area calculation unit <NUM>, and possible work area <NUM> and <NUM> of the crane <NUM> at the height at which the second information display unit <NUM> is disposed. Further, the image display unit <NUM> displays a hung load weight input into the input unit <NUM>.

The virtual outriggers 80A include virtual front outriggers 82A and virtual rear outriggers 81A. The possible work area in a plane in which the crane <NUM> is installed include the possible work area <NUM> for a load ratio of <NUM>%, and the possible work area <NUM> for a load ratio of <NUM>%. The possible work area at the height at which the second information display unit <NUM> is disposed include the possible work area <NUM> for a load ratio of <NUM>%, and the possible work area <NUM> for a load ratio of <NUM>%.

That is to say, to allow a three-dimensional examination on how each portion of the crane <NUM> affects a site environment when a user actually operates the crane <NUM> in the site, the image display unit <NUM> displays the virtual crane 1A, the possible work area <NUM> and <NUM> (<NUM> and <NUM>) of the crane <NUM>, the virtual outriggers 80A, and the tail swing area <NUM> of the crane <NUM> in a three-dimensional image in such manner that the virtual crane 1A, the possible work area <NUM> and <NUM> (<NUM> and <NUM>) of the crane <NUM>, the virtual outriggers 80A, and the tail swing area <NUM> of the crane <NUM> are overlaid onto a surrounding environment that appears in an image of the camera <NUM>. Note that the image display unit <NUM> displays the virtual outriggers 80A at, for example, the positions of outriggers of the virtual crane 1A in an image of the camera <NUM>. Further, the image display unit <NUM> displays the possible work area <NUM> and <NUM> of the crane <NUM> around the virtual crane 1A as the center, in an image of the camera <NUM>, for example. Further, the image display unit <NUM> displays the tail swing area <NUM> of the crane <NUM> around a rotation mount of the virtual crane 1A as the center, in an image of the camera <NUM>, for example.

<FIG> is a flowchart that illustrates a process of processing by the control unit <NUM> of the crane information display system <NUM> according to the first embodiment. Hereinafter, the process of processing by the control unit <NUM> of the crane information display system <NUM> according to the first embodiment will be described.

When a worker M images, with the camera <NUM> attached to the tablet terminal <NUM>, the first information display unit <NUM> attached to the support member <NUM> installed in a work site in which the crane <NUM> is going to be installed, and a site environment around the first information display unit <NUM>, the crane information acquisition unit <NUM> reads into the first information display unit <NUM> imaged with the camera <NUM>, as illustrated in <FIG> (step S101).

Next, the crane information acquisition unit <NUM> acquires, from the storage unit <NUM>, shape information and performance information about the machine type of the crane <NUM> that has been acquired (step S102).

Next, the virtual crane generation unit <NUM> generates three-dimensional data on virtual outriggers, on the basis of information about the crane <NUM> acquired by the crane information acquisition unit <NUM> (step S103).

Next, the position/posture calculation unit <NUM> calculates the position and posture of a virtual crane 1A, on the basis of the first information display unit <NUM> imaged with the camera <NUM> (step S104).

Next, the control unit <NUM> determines whether or not a protrusion amount of the outriggers <NUM> has been input into the input unit <NUM> (step S105). If it is determined that a protrusion amount of the outriggers <NUM> has been input into the input unit <NUM> (YES in step S105), the virtual outrigger generation unit <NUM> generates virtual outriggers 80A (step S106), and the processing proceeds to step S107. On the other hand, if a protrusion amount of the outriggers <NUM> has not been input into the input unit <NUM>, (NO in step S105), the processing proceeds to step S107.

Next, the control unit <NUM> determines whether or not a hung load weight and the length of the boom <NUM> have been input into the input unit <NUM> (step S107). If it is determined that a hung load weight and the length of the boom <NUM> have been input into the input unit <NUM> (YES in step S107), the possible work area calculation unit <NUM> calculates possible work area <NUM> and <NUM> (step S108), and the processing proceeds to step <NUM>. On the other hand, if it is determined that a hung load weight and the length of the boom <NUM> have not been input into the input unit <NUM> (NO in step S108), the processing proceeds to step S109.

Further, the information processing unit <NUM> processes the virtual outriggers 80A generated by the virtual outrigger generation unit <NUM>, and the information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, into information that corresponds to the position and posture of the crane <NUM> calculated by the position/posture calculation unit <NUM> (step S109).

Next, the position calculation unit <NUM> reads into the second information display unit <NUM> imaged with the camera <NUM> (step S110).

Next, the position calculation unit <NUM> calculates position information about the second information display unit <NUM> from the first information display unit <NUM> (step S111).

Next, on the basis of the position information calculated by the position calculation unit <NUM>, the information processing unit <NUM> processes the information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, into information that corresponds to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>, with the second information display unit <NUM> as the reference (step S112).

Next, the image display unit <NUM> superimposes and displays, on an image of a work site imaged with the camera <NUM> in which the crane <NUM> is going to be installed, the virtual crane 1A, a tail swing area <NUM> of the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outriggers 80A generated by the virtual outrigger generation unit <NUM>, possible work area <NUM> and <NUM> in a plane in which the virtual crane 1A is installed, and possible work area <NUM> and <NUM> at the height at which the second information display unit <NUM> is disposed (step S113), and the processing is ended.

If an installation place of the crane <NUM> is examined, and the installation place of the crane <NUM> is determined in this way, the crane <NUM> is carried and installed in such a manner that a predetermined position of the actual crane <NUM> (e.g., the center of the left side surface of the vehicle body frame <NUM> of the travelling body <NUM>) is made to correspond to the first information display unit <NUM> attached to the support member <NUM> put in the work site.

Hereinafter, effects of the crane information display system <NUM> according to the first embodiment will be described.

The crane information display system <NUM> according to the first embodiment includes the crane information acquisition unit <NUM> that images, with the camera <NUM>, the first information display unit <NUM> that displays information about the crane <NUM> to acquire the information about the crane <NUM>, the virtual crane generation unit <NUM> that generates a virtual crane 1A of the crane <NUM> that is three-dimensional, on the basis of the information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, the position/posture calculation unit <NUM> that calculates the position and posture of the virtual crane 1A, on the basis of the first information display unit <NUM> imaged with the camera <NUM>, the information processing unit <NUM> that processes the virtual crane 1A generated by the virtual crane generation unit <NUM>, into information that corresponds to the position and posture of the virtual crane 1A calculated by the position/posture calculation unit <NUM>, and the image display unit <NUM> that overlays and displays the information processed by the information processing unit <NUM> onto the image imaged with the camera <NUM> (<FIG> and <FIG>).

Consequently, the virtual crane 1A can be disposed and displayed in a scenery image of a place in which the crane <NUM> is going to be installed. Therefore, disposition of the crane <NUM> can be examined in a three-dimensional image without actually disposing the crane <NUM> in a place in which the crane <NUM> is going to be installed. As a result, a work plan of the crane <NUM> can be easily examined.

In the crane information display system <NUM> according to the first embodiment, the information processing unit <NUM> processes information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, into information that corresponds to the position and posture of a virtual crane 1A calculated by the position/posture calculation unit <NUM> (<FIG> and <FIG>).

Consequently, the information about the crane <NUM> can be overlaid and displayed onto an image in which the virtual crane 1A is disposed. Therefore, the virtual crane 1A, a site environment around the virtual crane 1A, and the information about the crane <NUM> can be checked in a three-dimensional image in real time. As a result, a work plan of the crane <NUM> can be easily examined without disposing the actual crane <NUM> in a place in which the crane <NUM> is going to be installed.

In the crane information display system <NUM> according to the first embodiment, the second information display unit <NUM> is provided and disposed at a position onto which the crane <NUM> unloads a hung load, the second information display unit <NUM> is imaged with the camera <NUM>, the position calculation unit <NUM> calculates position information about the second information display unit <NUM> from the first information display unit <NUM>, and the information processing unit <NUM> processes, on the basis of the position information calculated by the position calculation unit <NUM>, information about the crane <NUM> acquired by the crane information acquisition unit <NUM>, into information that corresponds to the position and posture of a virtual crane 1A calculated by the position/posture calculation unit <NUM>, with the second information display unit <NUM> as the reference (<FIG> and <FIG>).

Consequently, possible work area <NUM> and <NUM> at the height at which the second information display unit <NUM> is installed can be displayed. Therefore, for example, if work that unloads a hung load is performed at a high position, such as a rooftop of a building B, as illustrated in <FIG>, possible work area <NUM> and <NUM> of the crane <NUM> at the height at which the second information display unit <NUM> is installed can be displayed. As a result, even if a position onto which a hung load is unloaded is at a height different from the height of a plane in which the crane <NUM> is installed, a work plan of the crane <NUM> can be easily examined.

In the crane information display system <NUM> according to the first embodiment, the input unit <NUM> into which a protrusion amount of the outriggers <NUM> of a virtual crane 1A is input is provided, and the crane information acquisition unit <NUM> includes the virtual outrigger generation unit <NUM> that generates three-dimensional virtual outriggers 80A, on the basis of the input value input into the input unit <NUM> (<FIG> and <FIG>).

Consequently, the virtual outriggers 80A of a protrusion amount that corresponds to the input value can be overlaid and displayed onto an image in which the virtual crane 1A disposed in a place in which the crane <NUM> is going to be installed, and the surroundings around the virtual crane 1A are imaged. Therefore, a protrusion amount of the outriggers <NUM> can be checked in a three-dimensional image in real time in the work site.

In the crane information display system <NUM> according to the first embodiment, the input unit <NUM> into which a hung load weight and the length of the boom <NUM> of the crane <NUM> are input is provided, and the crane information acquisition unit <NUM> includes the possible work area calculation unit <NUM> that calculates possible work area <NUM> and <NUM> of the crane <NUM>, on the basis of the input values input into the input unit <NUM> (<FIG> and <FIG>).

Consequently, the possible work area <NUM> and <NUM> for a predetermined hung load weight can be overlaid and displayed, on the basis of the length of the boom <NUM>, onto an image in which a virtual crane 1A disposed in a place in which the crane <NUM> is going to be installed and the surroundings around the virtual crane 1A are imaged. Therefore, the possible work area <NUM> and <NUM> can be checked in a three-dimensional image in real time in the work site.

In the crane information display system <NUM> according to the first embodiment, information about the crane <NUM> includes a tail swing area <NUM> of the crane <NUM> (<FIG> and <FIG>).

Consequently, the tail swing area <NUM> can be overlaid and displayed onto an image in which a virtual crane 1A disposed in a place in which the crane <NUM> is going to be installed and the surroundings around the virtual crane 1A are imaged. Therefore, the tail swing area <NUM> can be checked in a three-dimensional image in real time in the work site.

The crane information display system according to the present invention has been described above on the basis of the first embodiment. However, specific configurations are not limited to the embodiment, but design changes, addition, and the like are allowed unless the design changes, addition, and the like depart from the gist of the invention according to each of the claims.

In the first embodiment, an example is shown in which information about a predetermined position of the crane <NUM> stored in the first information display unit <NUM> is the center of the left side surface of the vehicle body frame <NUM> of the travelling body <NUM>. However, information about a predetermined position of the crane <NUM> stored in the first information display unit <NUM> is not limited to the center of the left side surface of the vehicle body frame <NUM>. Further, in addition to the left side surface of the vehicle body frame <NUM>, the front surface, the back surface, and the right side surface of the vehicle body frame <NUM> may be stored as information about predetermined positions of the crane <NUM> stored in the first information display unit <NUM>, and a worker M may appropriately perform the selection with the input unit <NUM>.

In the first embodiment, an example is shown in which information about the machine type of the one crane <NUM> is stored in the first information display unit <NUM>. However, information about a plurality of machine types of cranes may be stored in the first information display unit <NUM>, and a worker M may appropriately perform the selection with the input unit <NUM>.

In the first embodiment, an example is shown in which the information display unit <NUM> is an AprilTag. However, an information display unit is not limited to the aspect, but may be, for example, a two-dimensional code, such as a QR code (registered trademark). Further, an information display unit may be a crane itself, and information about the crane may be acquired by image recognition using deep learning.

In the first embodiment, an example is shown in which the possible work area <NUM> and <NUM> for a load ratio of <NUM>%, and the possible work area <NUM> and <NUM> for a load ratio of <NUM>% are displayed on the image display unit <NUM>. However, one possible work area may be displayed, or three or more possible work area may be displayed on an image display unit. Further, load ratios of possible work area are not limited to <NUM>% and <NUM>%.

In the first embodiment, an example is shown in which a user terminal is the tablet terminal <NUM> that includes the camera <NUM>, the input unit <NUM>, and the image display unit <NUM>. However, the user terminal may be a smartphone. Further, the user terminal may include a camera and an image display unit that are separate.

In the first embodiment, an example is shown in which shape information and performance information of the crane <NUM> are stored in the storage unit <NUM>. However, shape information and performance information of a crane may be stored in the first information display unit.

Claim 1:
A crane information display system (<NUM>) comprising:
a first information display unit (<NUM>) that is installed at the location where a crane (<NUM>) is scheduled to be installed and that is configured to display information about the crane (<NUM>), and
a terminal device (<NUM>) that mounts a camera (<NUM>) to obtain a camera image by imaging, with the camera (<NUM>), the first information display unit (<NUM>),
wherein
the terminal device (<NUM>) comprises:
a crane information acquisition unit (<NUM>) configured to read, from the camera image, display information of the first information display unit (<NUM>) to acquire information about the crane (<NUM>);
a virtual crane generation unit (<NUM>) configured to generate, on a basis of the information about the crane (<NUM>) acquired by the crane information acquisition unit (<NUM>), image information about a virtual crane (1A) that corresponds to a three-dimensional image of the crane (<NUM>);
a position/posture calculation unit (<NUM>) configured to read, from the camera image, the display information of the first information display unit (<NUM>) to acquire information about a reference shape of the first information display unit (<NUM>), and extract, from the camera image, information about an outline shape of the first information display unit (<NUM>) that appears in the camera image, and calculate, on a basis of the reference shape of the first information display unit (<NUM>) and the information about the outline shape of the first information display unit (<NUM>), a position at which the virtual crane (1A) is to be disposed and a posture to be taken by the virtual crane (1A) in the camera image;
an information processing unit (<NUM>) configured to convert the image information about the virtual crane (1A) generated by the virtual crane generation unit (<NUM>), into image information that corresponds to the position and the posture of the virtual crane (1A) calculated by the position/posture calculation unit (<NUM>); and
an image display unit (<NUM>) configured to overlay and display, onto the camera image, the virtual crane (1A) to which the processing by the information processing unit (<NUM>) that is configured to convert the image information has been applied.