Patent Description:
Conventionally, a crane information display system that displays information of a crane is known (for example, see PTL <NUM>).

PTL <NUM> discloses a configuration that visualizes information on an operating state of a crane and displays a display screen on a display unit of a mobile terminal. As a result, even a person outside the crane can see the operating state of the crane.

PTL <NUM> discloses a remote operation device that is capable of providing, to the operator, guidance of a work content to be performed by manual work or remote operation on the work machine in accordance with a state of the work machine. The remote operation device is configured to be capable of communicating with the work machine, and includes: an operation unit that generates an operation signal for performing remote operation of the work machine in accordance with operator's operation; a reception unit that receives detection signals of detection devices that detect the state of the work machine; and a display unit that displays a screen that indicates a work content to be performed by manual work or the remote operation on the work machine on the basis of the received state of the work machine.

Therefore, PTL <NUM> discloses a crane information display system comprising a terminal device that mounts a camera to capture an image of a crane with the camera and obtain a camera image, wherein the terminal device comprises: an information processing unit that is configured to convert the information about the crane and an image display unit that is configured to overlay the information about the crane converted by the information processing unit on the camera image and displays the information.

PTL <NUM> discloses an operation assist apparatus that includes a display device that is installed in an operator's cab of a construction machine, at which at least operating procedures for the construction machine are displayed with text and illustrations; an image processing device that generates images; and a control device that engages the image processing device to generate an image of an operating procedure corresponding to an operation of the construction machine and engages the display device to display the image generated by the image processing device.

PTL <NUM> discloses a system for determining the position of an offshore structure in a fixed reference frame by observations from a moving reference frame. The system includes a plurality of markers that are located at defined positions on the structure and an imaging device that is arranged to view the markers and generate image data representative of the relative positions of the markers in the moving reference frame. The imaging device is mounted aboard a vessel. A local positioning system is arranged to determine a position of the imaging device in the fixed reference frame and generate position data. A processing unit is arranged to receive the image data from the imaging device and the position data from the local positioning system and compute the position of the structure based on the defined positions of the markers.

However, in the configuration disclosed in PTL <NUM>, information on an operating state of a crane is visualized and displayed by drawing the crane from above or from the side. Therefore, in the configuration disclosed in PTL <NUM>, there is a problem in that it is necessary to examine it in two-dimensions.

Therefore, an object of the present invention is to provide a crane information display system capable of enabling information about a crane at a work site to be three-dimensionally examined.

According to a first aspect, the present invention provides a crane information display system in accordance with independent claim <NUM>. Further aspects are set forth in the dependent claims, the drawings and the following description.

In the crane information display system of the present invention configured as described above, information about a crane can be examined three-dimensionally at a work site.

Hereinafter, embodiments for achieving a crane information display system according to the present invention will be described with reference to Example <NUM> illustrated in the drawings.

<FIG> is a diagram illustrating a crane information display system of Example <NUM>. Hereinafter, a configuration of the crane information display system of Example <NUM> will be described.

As illustrated in <FIG>, an example in which, in the crane information display system <NUM> of Example <NUM>, a worker M captures an image of a crane <NUM> placed at a work site with a tablet terminal <NUM> as a user terminal provided with a camera <NUM> will be described. An example in which the crane information display system <NUM> of Example <NUM> captures an image of the crane <NUM> with the camera <NUM> in a state where an outrigger <NUM> does not protrude will be described. The front-rear direction of the crane <NUM> is defined as a front-rear direction D.

As illustrated in <FIG>, the crane <NUM> includes a traveling body <NUM>, a swirling body <NUM>, and a boom <NUM>.

The traveling body <NUM> includes a vehicle body frame <NUM>, the outrigger <NUM>, a traveling device for self-traveling on a road or a work site, and the like.

The outrigger <NUM> includes a rear outrigger <NUM> attached to a rear side surface of the vehicle body frame <NUM> of the traveling body <NUM> and a front outrigger <NUM> attached to a front side surface of the vehicle body frame <NUM>. The outrigger <NUM> is stored in the vehicle body frame <NUM> during travel. On the other hand, at the time of work, the outrigger <NUM> protrudes in the horizontal direction and the vertical direction, lifts the entire vehicle body, and stabilizes the orientation.

An April Tag (April Tag) is attached to the vehicle body frame <NUM> of the traveling body <NUM> as an information display unit <NUM> (that is, a marker). The information display unit <NUM> includes a first information display unit <NUM> attached to the front surface of the vehicle body frame <NUM>, a second information display unit <NUM> attached to the rear surface of the vehicle body frame <NUM>, a third information display unit <NUM> attached to the right side surface of the vehicle body frame <NUM>, and a fourth information display unit <NUM> attached to the left side surface of the vehicle body frame <NUM>. The first information display unit <NUM>, the second information display unit <NUM>, the third information display unit <NUM>, and the fourth information display unit <NUM> each have separate display information (that is, code information) of, for example, text, a code, a symbol, a pattern, or the like attached to the surface thereof. The display information (that is, the code information) held by each of the first information display unit <NUM>, the second information display unit <NUM>, the third information display unit <NUM>, and the fourth information display unit <NUM> can be read from the camera image generated by the camera <NUM> using a decoding program stored in advance in a control unit <NUM> (described later) or the like.

The first information display unit <NUM> records information about the model of the crane <NUM>, information about the position where the first information display unit <NUM> is attached in the crane <NUM>, and the shape and size of the first information display unit <NUM>. The second information display unit <NUM> records information about the model of the crane <NUM>, information about the position where the second information display unit <NUM> is attached in the crane <NUM>, and the shape and size of the second information display unit <NUM>. The third information display unit <NUM> records information about the model of the crane <NUM>, information about the position where the third information display unit <NUM> is attached in the crane <NUM>, and the shape and size of the third information display unit <NUM>. The fourth information display unit <NUM> records information about the model of the crane <NUM>, information about the position where the fourth information display unit <NUM> is attached in the crane <NUM>, and the shape and size of the fourth information display unit <NUM>. The shapes and sizes recorded in the first information display unit <NUM>, the second information display unit <NUM>, the third information display unit <NUM>, and the fourth information display unit <NUM> are, for example, shapes and sizes obtained from a predetermined distance in front (hereinafter referred to as a "reference shape" and a "reference size").

The first information display unit <NUM>, the second information display unit <NUM>, the third information display unit <NUM>, and the fourth information display unit <NUM> have the same outer shape (for example, a rectangle) and the same size. The first information display unit <NUM>, the second information display unit <NUM>, the third information display unit <NUM>, and the fourth information display unit <NUM> can be identified from the display information (that is, the code information) held by each.

The swirling body <NUM> is provided above the traveling body <NUM> and is rotatable about a vertical axis C1 with respect to the traveling body <NUM>. The swirling body <NUM> includes a cabin <NUM>. The cabin <NUM> includes an operation unit (for example, a steering wheel, a shift lever, an accelerator pedal, a brake pedal, and the like) for controlling traveling of the traveling body <NUM>. The cabin <NUM> includes an operation unit that operates the swirling body <NUM>, a boom <NUM>, a winch, or the like. The operator riding in the cabin <NUM> operates the operation unit to swing the swirling body <NUM>, raise and lower the boom <NUM>, extend and contract the boom <NUM>, and rotate the winch to perform the work.

The proximal end side of the boom <NUM> is supported by the swirling body <NUM>, and is attached to the swirling body <NUM> to be raised and lowered. The boom <NUM> is raised and lowered by a raisable/lowerable cylinder <NUM> provided in the swirling body <NUM>, and is expanded and contracted by a telescopic cylinder (not illustrated).

The boom <NUM> includes intermediate booms <NUM> to <NUM> between a proximal end boom <NUM> on the proximal side and a distal end boom <NUM> on the distal end side. The intermediate booms <NUM> to <NUM> and the distal end boom <NUM> are telescopically stored in the proximal end boom <NUM> in sequence.

A sheave <NUM> is disposed in a boom head 36a provided at the distal end of the distal end boom <NUM>. A wire rope <NUM> for a suspended load is wound around a winch provided near the proximal end of the boom <NUM> of the swirling body <NUM>. The wire rope <NUM> is disposed along the boom <NUM> from the winch to the sheave <NUM> in the axial direction, and the wire rope <NUM> wound around the sheave <NUM> is suspended downward in the vertical direction from the sheave <NUM>. A hook <NUM> is provided at the lowermost portion of the wire rope <NUM>.

A load is hung on the hook <NUM>, the wire rope <NUM> wound around the winch is unwound to lower the hook <NUM>, and the wire rope <NUM> is wound up to raise the hook <NUM>.

The crane <NUM> configured as described above moves the load hung on the hook <NUM> to a predetermined position by unwinding and winding the wire rope <NUM> by the winch, raising, lowering, extending, and contracting the boom <NUM>, and swinging the swirling body <NUM>.

As illustrated in <FIG>, the tablet terminal <NUM> includes the camera <NUM>, an image display unit <NUM>, and an input unit <NUM>.

An image captured with the camera <NUM> is displayed on the image display unit <NUM>. The image display unit <NUM> is also configured as a touch panel as the input unit <NUM>.

<FIG> is a block diagram illustrating a functional configuration of the crane information display system <NUM> of Example <NUM>. <FIG> is a diagram illustrating an image displayed on an image display unit of Example <NUM>. Hereinafter, a functional configuration of the crane information display system <NUM> of Example <NUM> will be described.

In the crane information display system <NUM>, an image of the information display unit <NUM> captured with the camera <NUM> and the input information input by the input unit <NUM> are input to the control unit <NUM> (in the present embodiment, a control unit built in the tablet terminal <NUM>), and the information controlled by the control unit <NUM> is output by the image display unit <NUM>.

The camera <NUM> can be, for example, a camera <NUM> provided in a general tablet terminal <NUM>. The camera <NUM> can capture an image of the crane <NUM>, a site environment around the crane <NUM>, and the information display unit <NUM>.

The input unit <NUM> can input an overhanging amount of the outrigger <NUM> in the horizontal direction, a suspended load, a length of the boom <NUM>, and the like. The length of the boom <NUM> is, for example, the length of the boom <NUM> in a state where the distal end boom <NUM> and the intermediate booms <NUM> to <NUM> are stored in the proximal end boom <NUM> (a fully contracted state), the length of the boom <NUM> in a state where the distal end boom <NUM> is extended, or the length in a state in which the distal end boom <NUM> and the intermediate booms <NUM> to <NUM> are extended (a fully extended state).

The control unit <NUM> includes a storage unit <NUM>, a crane information acquisition unit <NUM>, a position/orientation calculation unit <NUM>, and an information processing unit <NUM>. The control unit <NUM> is a known microcomputer including, for example, a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and the like, and functions of the control unit <NUM> (the crane information acquisition unit <NUM>, the position/orientation calculation unit <NUM>, and the information processing unit <NUM>) are achieved by, for example, the CPU referring to a control program or various data stored in the storage unit <NUM> (for example, an HDD), the ROM, or the RAM.

The storage unit <NUM> stores performance information about various cranes. The performance information includes, for example, information on the shape of the outrigger <NUM>, information about a workable area in the horizontal direction and information about a workable area in the height direction according to the overhanging amount of the outrigger <NUM> and the length of the boom <NUM>, information about a derricking angle range of the boom <NUM>, information on a load factor, information on a tail swing area, and the like.

The crane information acquisition unit <NUM> acquires information about the model of the crane <NUM> based on the image of the information display unit <NUM> captured with the camera <NUM>, and acquires performance information about the acquired model of the crane <NUM> from the storage unit <NUM>. That is, the crane information acquisition unit <NUM> reads the image of the information display unit <NUM> captured with the camera <NUM> (that is, the display information of the information display unit <NUM> is read from the camera image), acquires the model of the crane <NUM>, and acquires the performance information of the acquired model from the storage unit <NUM>.

The crane information acquisition unit <NUM> includes a virtual outrigger generation unit <NUM> and a workable area calculation unit <NUM>.

The virtual outrigger generation unit <NUM> generates three-dimensional data (that is, image information about the three-dimensional image of the outrigger <NUM>) of the virtual outrigger as the information of the crane <NUM> based on the information on the shape of the outrigger <NUM> stored in the storage unit <NUM> (that is, the shape of the outrigger <NUM> corresponding to the model of the crane <NUM> specified from the display information of the information display unit <NUM>) and the overhanging amount of the outrigger <NUM> in the horizontal direction input to the input unit <NUM>.

The workable area calculation unit <NUM> refers to the information about the model of the crane <NUM> specified from the display information of the information display unit <NUM>, and calculates the workable area of the crane <NUM> as the information about the crane <NUM> based on the suspended load input to the input unit <NUM> and the length of the boom <NUM>. The workable area calculation unit <NUM> can calculate workable areas with a plurality of load factors. In Example <NUM>, the workable area calculation unit <NUM> calculates a workable area with a load factor of <NUM>% and a workable area with a load factor of <NUM>%. A workable area is an area where work can be performed in the horizontal direction of the crane <NUM> on the installation surface of the crane <NUM>.

The position/orientation calculation unit <NUM> calculates the position and orientation of the crane <NUM> based on the image of the information display unit <NUM> captured with the camera <NUM>.

Specifically, the position/orientation calculation unit <NUM> reads the image of the information display unit <NUM> captured with the camera <NUM> (that is, the display information of the information display unit <NUM> is read from the image of the camera <NUM>), acquires information about the position (that is, the mounting position of the information display unit <NUM>) where the information display unit <NUM> is attached to the crane <NUM>, and calculates the orientation of the crane <NUM>.

For example, when the image of the information display unit <NUM> captured with the camera <NUM> shows the first information display unit <NUM>, since the first information display unit <NUM> is attached to the front surface of the vehicle body frame <NUM>, the camera <NUM> captures an image of the crane <NUM> facing forward. When the image of the information display unit <NUM> captured with the camera <NUM> shows the second information display unit <NUM>, since the second information display unit <NUM> is attached to the rear surface of the vehicle body frame <NUM>, the camera <NUM> captures an image of the crane <NUM> facing rearward. When the image of the image of the information display unit <NUM> captured with the camera <NUM> shows the third information display unit <NUM>, since the third information display unit <NUM> is attached to the right side surface of the vehicle body frame <NUM>, the camera <NUM> captures an image of the crane <NUM> facing rightward. When the image of the information display unit <NUM> captured with the camera <NUM> shows the fourth information display unit <NUM>, since the fourth information display unit <NUM> is attached to the left side surface of the vehicle body frame <NUM>, the camera <NUM> captures an image of the crane <NUM> facing leftward.

The position/orientation calculation unit <NUM> acquires information about the orientation of the crane <NUM> based on the shape (hereinafter, referred to as a "contour shape of the information display unit <NUM>") of the information display unit <NUM> captured with the camera <NUM> and the reference shape of the information display unit <NUM> recorded in the information display unit <NUM>. That is, the position/orientation calculation unit <NUM> reads the display information (that is, code information) of the information display unit <NUM> from the image of the camera <NUM> to acquire information related to the reference shape of the information display unit <NUM>, and extracts the contour shape of the information display unit <NUM> reflected in the camera <NUM> from the image of the camera <NUM> by known pattern matching or the like. Then, the position/orientation calculation unit <NUM> calculates information about the orientation of the crane <NUM> by comparing the contour shape of the information display unit <NUM> reflected in the image of the camera <NUM> with the reference shape of the information display unit <NUM>.

For example, in a case where the information display unit <NUM> captured with the camera <NUM> is the first information display unit <NUM> and has a rectangular shape recorded in the first information display unit <NUM> (that is, in a case where both the contour shape of the first information display unit <NUM> reflected in the image of the camera <NUM> and the reference shape of the first information display unit <NUM> specified from the display information of the first information display unit <NUM> are rectangular), the crane <NUM> is in an orientation facing straight forward with respect to the camera <NUM>. When the information display unit <NUM> captured with the camera <NUM> is the first information display unit <NUM>, is not the rectangular shape recorded in the first information display unit <NUM>, and is a trapezoid in which the left edge of the information display unit <NUM> is longer (higher) than the right edge (that is, the contour shape of the first information display unit <NUM> reflected in the image of the camera <NUM> is a trapezoid of which the left edge is longer than the right edge, and the reference shape of the first information display unit <NUM> specified from the display information of the first information display unit <NUM> is a rectangular shape), the crane <NUM> is in an orientation facing slightly left from the straight front with respect to the camera <NUM>. When the information display unit <NUM> captured with the camera <NUM> is the first information display unit <NUM>, is not the rectangular shape recorded in the first information display unit <NUM>, and is a trapezoid in which the right edge of the information display unit <NUM> is longer than the left edge (that is, the contour shape of the first information display unit <NUM> reflected in the image of the camera <NUM> is a trapezoid of which the right edge is longer than the left edge, and the reference shape of the first information display unit <NUM> specified from the display information of the first information display unit <NUM> is a rectangular shape), the crane <NUM> is in an orientation facing slightly right from the straight front with respect to the camera <NUM>.

That is, the position/orientation calculation unit <NUM> compares the shape of the information display unit <NUM> captured with the camera <NUM> with the shape of the information display unit <NUM> recorded in the information display unit <NUM>, and acquires information about the orientation of the crane <NUM>.

The position/orientation calculation unit <NUM> acquires information about the position of the crane <NUM> based on the size of the contour shape of the information display unit <NUM> captured with the camera <NUM> and the reference size of the information display unit <NUM> recorded in the information display unit <NUM>. Specifically, the position/orientation calculation unit <NUM> compares the reference size of the information display unit <NUM> recorded in the information display unit <NUM> with the size of the contour shape of the information display unit <NUM> captured with the camera <NUM>, and calculates the distance from the camera <NUM> to the information display unit <NUM>.

The information processing unit <NUM> processes the performance information of the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outrigger generated by the virtual outrigger generation unit <NUM>, and the workable area of the crane <NUM> calculated by the workable area calculation unit <NUM> into information corresponding to the position and the orientation of the crane <NUM> calculated by the position/orientation calculation unit <NUM>.

That is, the information processing unit <NUM> processes the information about the crane <NUM> acquired by the crane information acquisition unit <NUM> into information corresponding to the position and the orientation of the crane <NUM> calculated by the position/orientation calculation unit <NUM>. In other words, the information processing unit <NUM> converts the information about the crane <NUM> acquired by the crane information acquisition unit <NUM> into three-dimensional image information corresponding to the position and the orientation of the crane <NUM> calculated by the position/orientation calculation unit <NUM>. For example, when the crane <NUM> is viewed from the capturing position of the camera <NUM>, the information processing unit <NUM> converts the three-dimensional image of a virtual outrigger 80A so that the three-dimensional image of the virtual outrigger 80A becomes an image simulating a state where the outrigger <NUM> is actually overhung. Such image processing of the information processing unit <NUM> is achieved by known coordinate conversion processing or the like.

The image display unit <NUM> displays the information (that is, information about the crane <NUM> converted into image information to be displayed by the information processing unit <NUM>) processed by the information processing unit <NUM> to be superimposed on the image of the camera <NUM>. Specifically, as illustrated in <FIG>, the image display unit <NUM> superimposes and displays the tail swing area <NUM> of the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outrigger 80A generated by the virtual outrigger generation unit <NUM>, and workable areas <NUM> and <NUM> of the crane <NUM> calculated by the workable area calculation unit <NUM> on the image of the crane <NUM> and the site environment around the crane <NUM> captured with the camera <NUM>. The image display unit <NUM> displays the suspended load input to the input unit <NUM>.

That is, the image display unit <NUM> displays the workable areas <NUM> and <NUM> of the crane <NUM>, the virtual outrigger 80A of the crane <NUM>, and the tail swing area <NUM> of the crane <NUM> in a three-dimensional image so as to be superimposed on the crane <NUM> or the surrounding environment of the crane <NUM> reflected in the image of the camera <NUM> so that the user can three-dimensionally examine how each part of the crane <NUM> affects the site environment when the crane <NUM> is actually operated on site. The image display unit <NUM> displays the virtual outrigger 80A at the position of the outrigger of the crane <NUM> reflected in the image of the camera <NUM>, for example. In addition, the image display unit <NUM> also displays the workable areas <NUM> and <NUM> of the crane <NUM> around the crane <NUM> reflected in the image of the camera <NUM>, for example. In addition, the image display unit <NUM> displays the tail swing area <NUM> of the crane <NUM> around the swivel base of the crane <NUM> shown in the image of the camera <NUM>, for example.

The virtual outrigger 80A includes a virtual front outrigger 82A and a virtual rear outrigger 81A. The workable area of the crane <NUM> includes a workable area <NUM> with a load factor of <NUM>% and a workable area <NUM> with a load factor of <NUM>%.

<FIG> is a flowchart illustrating a flow of processing by the control unit <NUM> of the crane information display system <NUM> of Example <NUM>. Hereinafter, a flow of processing by the control unit <NUM> of the crane information display system <NUM> according to Example <NUM> will be described.

When the worker M captures an image of the crane <NUM> installed at the work site and the surrounding site environment with the camera <NUM> of the tablet terminal <NUM>, as illustrated in <FIG>, the crane information acquisition unit <NUM> acquires information about the model of the crane <NUM> based on the information display unit <NUM> captured with the camera <NUM> (step S101).

Next, the position/orientation calculation unit <NUM> calculates the position and orientation of the crane <NUM> based on the information display unit <NUM> captured with the camera <NUM> (step S102).

Next, the control unit <NUM> determines whether the overhanging amount of the outrigger <NUM> has been input to the input unit <NUM> (step S103). In a case where it is determined that the overhanging amount of the outrigger <NUM> is input to the input unit <NUM> (YES in step S103), the virtual outrigger generation unit <NUM> generates the virtual outrigger 80A (step S104), and the process proceeds to step S105. On the other hand, in a case where the overhanging amount of the outrigger <NUM> is not input to the input unit <NUM> (NO in step S103), the process proceeds to step S105.

Next, the control unit <NUM> determines whether the suspended load and the length of the boom <NUM> have been input to the input unit <NUM> (step S105). In a case where it is determined that the suspended load and the length of the boom <NUM> are input to the input unit <NUM> (YES in step S105), the workable area calculation unit <NUM> calculates the workable areas <NUM> and <NUM> (step S106), and the process proceeds to step <NUM>. On the other hand, in a case where it is determined that the suspended load and the length of the boom <NUM> are not input to the input unit <NUM> (NO in step S105), the process proceeds to step S107.

Next, the information processing unit <NUM> processes the information about the crane <NUM> acquired by the crane information acquisition unit <NUM> into information corresponding to the position and the orientation of the crane <NUM> calculated by the position/orientation calculation unit <NUM> (step S107). That is, the information processing unit <NUM> processes the performance information of the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outrigger 80A generated by the virtual outrigger generation unit <NUM>, and the workable areas <NUM> and <NUM> of the crane <NUM> calculated by the workable area calculation unit <NUM> into information corresponding to the position and the orientation of the crane <NUM> calculated by the position/orientation calculation unit <NUM>.

Next, the image display unit <NUM> superimposes and displays the tail swing area <NUM> of the crane <NUM> acquired by the crane information acquisition unit <NUM>, the virtual outrigger 80A generated by the virtual outrigger generation unit <NUM>, and workable areas <NUM> and <NUM> of the crane <NUM> calculated by the workable area calculation unit <NUM> on the image of the crane <NUM> and the site environment around the crane <NUM> captured with the camera <NUM> (step S108), and ends the processing.

Hereinafter, an operation of the crane information display system <NUM> of Example <NUM> will be described.

The crane information display system <NUM> according to Example <NUM> includes: the crane information acquisition unit <NUM> that acquires information about the crane <NUM> by capturing the information display unit <NUM> that displays information about the crane <NUM> by the camera <NUM>, the information display unit <NUM> being provided in the crane <NUM>; the position/orientation calculation unit <NUM> that calculates a position and an orientation of the crane <NUM> based on the information display unit <NUM> captured with the camera <NUM>; the information processing unit <NUM> that processes the information about the crane acquired by the crane information acquisition unit <NUM> into information corresponding to the position and the orientation of the crane <NUM> calculated by the position/orientation calculation unit <NUM>; and the image display unit <NUM> that displays the information processed by the information processing unit <NUM> to be superimposed on an image captured with the camera <NUM> (<FIG>).

As a result, the information about the crane <NUM> can be superimposed and displayed on the image obtained by capturing the crane <NUM> and the surroundings thereof. Therefore, at the work site, the crane <NUM>, the site environment around the crane <NUM>, and the information about the crane <NUM> can be confirmed by a three-dimensional image in real time. As a result, a work plan of the crane <NUM> can be examined in real time at the work site.

The crane information display system <NUM> of Example <NUM> includes the input unit <NUM> that inputs the overhanging amount of the outrigger <NUM> of the crane <NUM>, and the crane information acquisition unit <NUM> includes the virtual outrigger generation unit <NUM> that generates the three-dimensional virtual outrigger 80A based on the input value to the input unit <NUM> (<FIG>).

As a result, the virtual outrigger 80A having the overhanging amount corresponding to the input value can be superimposed and displayed on the image obtained by capturing the crane <NUM> and the surroundings thereof. Therefore, at the work site, the overhanging amount of the outrigger <NUM> can be confirmed on a three-dimensional image in real time.

The crane information display system <NUM> of Example <NUM> includes the input unit <NUM> that inputs the suspended load and the length of the boom <NUM> of the crane <NUM>, and the crane information acquisition unit <NUM> includes the workable area calculation unit <NUM> that calculates the workable areas <NUM> and <NUM> of the crane <NUM> based on the input value to the input unit <NUM> (<FIG>).

As a result, it is possible to display the workable areas <NUM> and <NUM> for a predetermined suspended load in an overlapping manner based on the length of the boom <NUM> on the image obtained by capturing the crane <NUM> and the surroundings thereof. Therefore, at the work site, the workable areas <NUM> and <NUM> can be confirmed by a three-dimensional image in real time.

In the crane information display system <NUM> of Example <NUM>, the information about the crane <NUM> includes the tail swing area <NUM> of the crane <NUM> (<FIG>).

As a result, the tail swing area <NUM> can be superimposed and displayed on the image obtained by capturing the crane <NUM> and the surroundings thereof. Therefore, at the work site, the tail swing area <NUM> can be confirmed by a three-dimensional image in real time.

The crane information display system of the present invention has been described above based on Example <NUM>. However, the specific configuration is not limited to this example, and design changes, additions, and the like are allowed without departing from the gist of the invention according to each claim of the claims.

In Example <NUM>, an example in which the information display unit <NUM> is an April Tag has been described. However, the information display unit is not limited to this mode, and may be a two-dimensional code such as a QR code (registered trademark). In addition, a crane itself may be used as the information display unit, and information of the crane may be acquired by image recognition using deep learning.

In Example <NUM>, an example in which one April Tag as the information display unit <NUM> is attached to each of the front surface, the rear surface, the right side surface, and the left side surface of the vehicle body frame <NUM> of the crane <NUM> has been described. However, two or more information display units <NUM> may be attached to each of the front surface, the rear surface, the right side surface, and the left side surface of the vehicle body frame <NUM> of the crane <NUM>.

In Example <NUM>, an example has been described in which the workable area <NUM> with a load factor of <NUM>% and the workable area <NUM> with a load factor of <NUM>% are displayed on the image display unit <NUM>. However, one workable area may be displayed on the image display unit, or three or more workable areas may be displayed on the image display unit. The load factor of the workable area is not limited to <NUM>% or <NUM>%.

In Example <NUM>, the user terminal is the tablet terminal <NUM> including the camera <NUM>, the input unit <NUM>, and the image display unit <NUM>. However, the user terminal may be a smartphone. Further, in the user terminal, the camera and the image display unit may be separate bodies.

Claim 1:
A crane information display system comprising:
an information display unit (<NUM>) that is mounted on a vehicle body frame (<NUM>) of a crane (<NUM>), and
a terminal device (<NUM>) that mounts a camera (<NUM>) to capture an image of the crane (<NUM>) with the camera (<NUM>) and obtain a camera image;
wherein the terminal device (<NUM>) comprises:
a crane information acquisition unit (<NUM>) that is configured to read display information of the information display unit (<NUM>) from the camera image and is configured to acquire information about the crane (<NUM>);
a position/orientation calculation unit (<NUM>) that is configured to read display information of the information display unit (<NUM>), is configured to acquire information about a reference shape of the information display unit (<NUM>) and a mounting position of the information display unit (<NUM>) on the crane (<NUM>) from the camera image, is configured to extract a contour shape of the information display unit (<NUM>) reflected in the camera image, and
is configured to calculate a position and an orientation of the crane (<NUM>) based on information about the reference shape of the information display unit (<NUM>), the mounting position of the information display unit (<NUM>), and the contour shape of the information display unit (<NUM>);
an information processing unit (<NUM>) that is configured to convert the information about the crane (<NUM>) acquired by the crane information acquisition unit (<NUM>) into three-dimensional image information corresponding to the position and orientation of the crane (<NUM>) calculated by the position/orientation calculation unit (<NUM>); and
an image display unit (<NUM>) that is configured to overlay the information about the crane (<NUM>) converted by the information processing unit (<NUM>) on the camera image and is configured to display the information.