Patent Description:
Conventionally, cranes are known to be typical working vehicles (see Patent Literature <NUM>). The crane is mainly composed of a vehicle and a crane device. The vehicle is provided with a plurality of wheels and is configured to be travelable. The crane equipment is equipped with wire ropes and hooks besides a boom, and is configured so that a load can be transported freely.

A crane operation support system capable of sharing information among a plurality of operators has been proposed (see Patent Literature <NUM>). In such a crane operation support system, a camera attached to the distal portion of the boom takes an image downward. The operator can input information to be displayed on the image taken by the camera. The image and input information are displayed on a display carried by other operators, and therefore the image of the work site and the input information can be shared. However, if the work range of the crane is wider than the imaging area of the camera, the entire work range is not displayed on the image. Thus, the crane operation support system could not display the information of an entire work range including the work range outside the imaging area of the camera. It also could not enable the information input in the entire work range to be shared among a plurality of operators. Accordingly, there has been a need for a crane and an information-sharing system capable of displaying the information of the entire work range including the work range outside the imaging area of the camera, and capable of sharing information input in the entire work range among a plurality of operators. <CIT> discloses an information-sharing system acquiring an image taken by a camera from a working vehicle including a working apparatus, the camera supported by the working apparatus, and a sensor for acquiring coordinates of the camera, comprising:a control apparatus;a display; wherein the control apparatus displays the map information and displays the image taken by the camera on an area on the map information.

A crane and an information-sharing system are provided, each capable of displaying information of an entire work range including a work range outside a imaging area of a camera, and capable of sharing information input in the entire work range among a plurality of operators.

According to a first aspect, the present invention provides an information-sharing system in accordance with independent claim <NUM>. Further aspects are set forth in the drawings and the following description.

An information-sharing system of the present invention acquires an image taken by a camera from a working vehicle including a working apparatus, the camera supported by the working apparatus, and a sensor for acquiring coordinates of the camera, the information-sharing system comprising:.

According to the information sharing-system of the present invention, it comprises: a control apparatus; a plurality of displays; and an information input section for inputting information to be displayed on the displays. The control apparatus displays the image taken by the camera on an area on the map information having the same size as an imaging area of the camera, the area being a coordinate position of the camera on the map information, when the scale of the map information or the distance ratio on the image with respect to the predetermined distance in the imaging area is changed in one display, displays the area of the map information and the imaging area of the camera in conjunction in all the display, and displays the information input into the one display by the information input section in the other display. According to such an information-sharing system, it is possible to display information of the entire work range including the work range outside the imaging area of the camera, and to share information input in the entire work range among a plurality of operators.

The technical idea disclosed in the present application can be applied to other cranes as well as the crane <NUM> described below.

First, for illustration purposes, crane <NUM> will be described with reference to <FIG>.

Crane <NUM> is mainly composed of vehicle <NUM> and crane device <NUM>.

Vehicle <NUM> is provided with a pair of left and right front wheels <NUM> and rear wheels <NUM>. Further, vehicle <NUM> is provided with outrigger <NUM> to stabilize by grounding when carrying work of load W. It should be noted that crane device <NUM> supported on vehicle <NUM> is swivelable by an actuator.

Crane device <NUM> is provided with boom <NUM> so as to protrude forward from the rear portion thereof. Boom <NUM> is swivelable by the actuator for swiveling crane device <NUM> (see arrow A). Further, the boom <NUM> is extendible and retractable by the actuator (see arrow B). Further, boom <NUM> is luffing-free by the actuator (see arrow C). In addition, wire rope <NUM> is stretched over boom <NUM>. On the proximal end side of boom <NUM>, a winch <NUM> around which wire rope <NUM> is wrapped is disposed, on the distal end side of boom <NUM>, hook <NUM> is suspended by wire rope <NUM>. Winch <NUM> is integrally configured with the actuator to allow winding and unwinding of wire rope <NUM>. Therefore, hook <NUM> is movable up and down by the actuator (see arrow D). It should be noted that the crane device <NUM> is provided with cabin <NUM> on the side of boom <NUM>. Inside cabin <NUM>, swivel manipulation tool <NUM>, extension and retraction manipulation tool <NUM>, luffing manipulation tool <NUM>, winding manipulation tool <NUM> are provided. Further, crane-side display terminal <NUM> is provided.

Next, a control configuration of crane <NUM> will be described with reference to <FIG> and <FIG>. Crane <NUM> includes control apparatus <NUM>. Hereinafter, an operator who performs an operation in crane <NUM> and an operator who performs an operation at the work site riding on crane <NUM> will be described as "operator X".

Various manipulating tools <NUM> to <NUM> are connected to control apparatus <NUM>. Further, various valves <NUM> to <NUM> are connected to control apparatus <NUM>.

Boom <NUM> is swivelable by the actuator (see arrow A in <FIG>). In the present application, such an actuator is defined as swivel hydraulic motor <NUM>. Swivel hydraulic motor <NUM> is appropriately operated by swivel valve <NUM> which is a directional control valve. In other words, swivel hydraulic motor <NUM> is appropriately operated by switching the flow direction of the hydraulic oil with swivel valve <NUM>. Swivel valve <NUM> is operated based on the operation of swivel manipulation tool <NUM> by operator X. Further, the swing angle of boom <NUM> is detected by a sensor which is not shown. Therefore, control apparatus <NUM> can recognize the swivel angle of boom <NUM>.

Further, boom <NUM> is extendible and retractable by the actuator (see arrow B in <FIG>). In the present application, such an actuator is defined as extension/retraction hydraulic cylinder <NUM>. Extension/retraction hydraulic cylinder <NUM> is appropriately operated by extension/retraction valve <NUM> which is a directional control valve. In other words, extension/retraction hydraulic cylinder <NUM> is appropriately operated by switching the flow direction of the hydraulic oil with extension/retraction valve <NUM>. Extension/retraction valve <NUM> is operated based on the operation of extension/retraction manipulation tool <NUM> by operator X. Further, the extension/retraction length of boom <NUM> is detected by a sensor which is not shown. Therefore, control apparatus <NUM> can recognize the extension/retraction length of boom <NUM>.

Further, boom <NUM> is luffing-free by the actuator (see arrow C in FIG). In the present application, such an actuator is defined as luffing hydraulic cylinder <NUM>. Luffing hydraulic cylinder <NUM> is appropriately operated by luffing valve <NUM> which is a directional control valve. In other words, luffing hydraulic cylinder <NUM> is appropriately operated by switching the flow direction of the hydraulic oil with luffing valve <NUM>. Luffing valve <NUM> is operated based on the operation of luffing manipulation tool <NUM> by operator X. Further, the luffing angle of boom <NUM> is detected by a sensor which is not shown. Therefore, control apparatus <NUM> can recognize the luffing angle of boom <NUM>.

Hook <NUM> is movable up and down by the actuator (see arrow D in <FIG>). In the present application, such an actuator is defined as winding hydraulic motor <NUM>. Winding hydraulic motor <NUM> is appropriately operated by winding valve <NUM> which is a directional control valve. In other words, winding hydraulic motor <NUM> is appropriately operated by switching the flow direction of the hydraulic oil with winding valve <NUM>. Winding valve <NUM> is operated based on the operation of winding manipulation tool <NUM> by operator X. Further, the slinging length of hook <NUM> is detected by a sensor which is not shown. Therefore, control apparatus <NUM> can recognize the slinging length of hook <NUM>.

Further, crane <NUM> includes camera <NUM>, zoom manipulation tool <NUM>, GNSS receiver <NUM>, azimuth sensor <NUM>, distance sensor <NUM>, map information receiver <NUM>, communication apparatus <NUM>, crane-side display terminal <NUM>, and display terminal <NUM>. In the following, though control apparatus <NUM> will be described as obtaining the coordinates of camera <NUM> using GNSS receiver <NUM>, control apparatus <NUM> may grasp the position of camera <NUM> by the deviation of the taken image by camera <NUM> which is obtained by aligning the position reference point in the image and the map information. Further, control apparatus <NUM> may grasp the position of camera <NUM> by grasping the position of communication apparatus <NUM> by using wireless communication of communication apparatus <NUM> attached in the vicinity of camera <NUM>.

Camera <NUM> is an apparatus for taking an image. Camera <NUM> is attached to the distal end portion of boom <NUM> to take an image of the work site from above (see <FIG>). The angle of view of camera <NUM> is stored in control apparatus <NUM> as information. Camera <NUM> is connected to communication apparatus <NUM>.

Zoom manipulation tool <NUM> is a manipulation tool for operating the zoom magnification of camera <NUM>. Zoom manipulation tool <NUM> is mounted inside cabin <NUM>. Zoom manipulation tool <NUM> can operate the optical zoom magnification and the digital zoom magnification of camera <NUM>. The digital zoom is a process of scaling (enlarging or reducing) an image by image processing. Therefore, the digital zoom may be performed using an image taken by camera <NUM> on control apparatus <NUM> side rather than on camera <NUM> side. It should be noted that camera <NUM> is connected to control apparatus <NUM>. Therefore, control apparatus <NUM> can recognize the operation signal of zoom manipulation tool <NUM>.

GNSS receiver <NUM> is a sensor for receiving ranging radio waves from satellites and calculates latitude, longitude, and elevation, which are coordinates. GNSS receiver <NUM> is attached to the distal end portion of boom <NUM> in the same manner as camera <NUM>, and is able to calculate the coordinates of camera <NUM>. It should be noted that GNSS receiver <NUM> is connected to communication apparatus <NUM>.

Azimuth sensor <NUM> is a sensor for detecting the orientation. Azimuth sensor <NUM> is attached to vehicle <NUM>. Azimuth sensor <NUM> detects the directions of the east, west, south, and north. It should be noted that azimuth sensor <NUM> is connected to control apparatus <NUM>.

Distance sensor <NUM> is a sensor for detecting the distance. Distance sensor <NUM> is attached to the distal end portion of boom <NUM> in the same manner as camera <NUM>. Distance sensor <NUM> is, for example, a laser distance sensor or a microwave distance sensor. Distance sensor <NUM> can detect the distance to the ground or the grounded object. It should be noted that distance sensor <NUM> is connected to communication apparatus <NUM>.

Map information receiver <NUM> is an apparatus for acquiring map information including the work range of crane <NUM>. The map information receiver <NUM> is attached to the distal end portion of the boom <NUM>. The map information receiver <NUM> can acquire the map information including the work range via network <NUM>. The work range (for example, within a radius of <NUM> from crane <NUM>) is determined based of the maximum length, the minimum length, the maximum luffing angle and the minimum luffing angle of boom <NUM>, and is stored in control apparatus <NUM> as information. Instead of acquiring the map information including the work range via the network <NUM>, the map information of the work site may be stored in advance in control apparatus <NUM>. Map information receiver <NUM> is connected to communication apparatus <NUM>.

Communication apparatus <NUM> is an apparatus for transmitting and receiving information converted into a radio signal. In communication apparatus <NUM>, in order to reduce the influence on the radio waves due to the grounded objects or the like, at least an antenna is attached to the distal end portion of boom <NUM>. Communication apparatus <NUM> is connected to display terminal <NUM> in addition to control apparatus <NUM>. Therefore, communication apparatus <NUM> can transmit information from control apparatus <NUM> to display terminal <NUM>. Further, communication apparatus <NUM> can transmit the work range of crane <NUM> from control apparatus <NUM> to map information receiver <NUM>. Further, communication apparatus <NUM> may transmit an operational signal of zoom manipulation tool <NUM> from control apparatus <NUM> to camera <NUM>. In addition, communication apparatus <NUM> can transmit information from display terminal <NUM> to control apparatus <NUM>. In addition, communication apparatus <NUM> may transmit map information including the image taken by camera <NUM>, the coordinates of camera <NUM>, the distance from camera <NUM> to the ground or the grounded object, and the work range of crane <NUM> to control apparatus <NUM>.

Crane-side display terminal <NUM> is a display for displaying various images. Crane-side display terminal <NUM> is mounted on the front side inside cabin <NUM> so that operator X can visually recognize various manipulation tools <NUM> to <NUM> while operating them. Crane-side display terminal <NUM> can display the map information and the image taken by camera <NUM>. Crane-side display terminal <NUM> is a display provided with a touch panel, and is also an information input section for inputting information. Therefore, crane-side display terminal <NUM> can perform change of the scale of the map information, change of the zoom magnification of camera <NUM>, input of information to be displayed on crane-side display terminal <NUM>, and the like. Instead of inputting information using a touch panel, information may be input using another input apparatus such as a mouse. Further, crane-side display terminal <NUM> is connected to control apparatus <NUM>. Therefore, control apparatus <NUM> can provide information to operator X via crane-side display terminal <NUM>. Crane-side display terminal <NUM> can transmit information input by operator X to control apparatus <NUM>.

Display terminal <NUM> is a display for displaying various images. Display terminal <NUM> can display the map information and the image taken by camera <NUM>. Display terminal <NUM> is a tablet terminal provided with a touch panel, and is also an information input section for inputting information. Therefore, display terminal <NUM> can perform change of the scale of the map information, change of the zoom magnification of camera <NUM>, input of information to be displayed on crane-side display terminal <NUM>, and the like. Instead of inputting information using the touch panel, information may be input using another input apparatus such as a mouse. Display terminal <NUM> is connected to control apparatus <NUM> described above via a radio signal. Therefore, control apparatus <NUM> can provide information to operator X via display terminal <NUM>. Display terminal <NUM> can transmit information input by operator X to control apparatus <NUM>.

Next, image I displayed on map information M will be described with reference to <FIG>. Here, the imaging area of camera <NUM> is the area of the work site in which camera <NUM> take an image as image I.

First, control apparatus <NUM> acquires the coordinates of camera <NUM> by GNSS receiver <NUM>, and acquires the orientation in which vehicle <NUM> is directed by azimuth sensor <NUM>. Further, the control apparatus <NUM> calculates the coordinates of the swivel center of boom <NUM> based on the coordinates of camera <NUM>, and the swivel angle, the extension/retraction length, the luffing angle of boom <NUM>. Then, control apparatus <NUM> acquires map information M including the work range of crane <NUM> by map information receiver <NUM> using the coordinates of the swivel center of boom <NUM> and the work range of crane <NUM>. Control apparatus <NUM> causes crane-side display terminal <NUM> and display terminal <NUM> to display the acquired map information M at a predetermined scale which is initially set. For example, in map information M to be displayed, building image Ma indicating a building and a road, building image Mb, building image Mc, and road image Md are displayed. Further, based on the coordinates of the swivel center of boom <NUM> and the orientation in which vehicle <NUM> is facing, crane image M1 indicating crane <NUM> is displayed. Further, based on the work range of crane <NUM>, boundary line L of the work range is displayed.

Further, Control apparatus <NUM>, based on the orientation in which vehicle <NUM> is facing and the swivel angle of boom <NUM>, calculates the orientation on image I. Further, control apparatus <NUM> detects the distance to the ground or the grounded object below by distance sensor <NUM>. Control apparatus <NUM>, based on the detected distance and the angle of view of camera <NUM>, calculates the imaging area of camera <NUM> on map information M. Then, control apparatus <NUM>, while matching the coordinate position of camera 51and the center of image I on map information M, matches the orientation on map information M and the orientation on image I, and superimposes image I in the area on map information M having the same size as the imaging area of camera <NUM> to display. At this time, control apparatus <NUM> displays image I by enlarging or reducing the size of image I, such that the image I is displayed in the area on map information M having the same size as the imaging area of camera <NUM>.

Further, control apparatus <NUM> displays boom image M7 indicating boom <NUM> so as to extend from the swivel center of boom <NUM> to the center of image I. Further, control apparatus <NUM> displays azimuth image Me indicating the orientation on map information M, and displays azimuth image Ia indicating the north on image I. In addition, a display terminal image Ib indicating the orientation of display terminal <NUM> with respect to the center of image I may be displayed by attaching a GNSS receiver to display terminal <NUM> and calculating the coordinate of display terminal <NUM>. When operator X performs a so-called swipe operation for moving the finger while touching the touch panel, control apparatus <NUM> scrolls map information M and image I to display any position of map information M and image I. For example, when a swipe operation is performed to display a predetermined position determined in the construction guideline, control apparatus <NUM> displays the predetermined position.

As described above, the present crane <NUM> includes control apparatus <NUM>, a display (crane-side display terminal <NUM>, display terminal <NUM>), a sensor for obtaining the coordinates of camera <NUM> (GNSS receiver <NUM>). Then, control apparatus <NUM> acquires map information M within the work range of crane <NUM>, and while displaying map information M on the display (<NUM>, <NUM>), displays image I taken by camera <NUM> in the coordinate position of camera <NUM> on map information M. According to such crane <NUM>, it is possible to display the information of the entire work range including the work range outside the imaging area of camera <NUM>.

Next, the relation between map information M and images I when the scale of map information M is changed will be described with reference to <FIG>. In the following, increasing the scale of map information M means enlarging map information M. Also, reducing the scale of map information M means reducing map information M. Further, increasing the ratio of the distance on image I with respect to the predetermined distance in the imaging area of camera <NUM> (hereinafter referred to as "distance ratio") means increasing the zoom magnification of camera <NUM>. In addition, reducing the distance ratio of image I means reducing the zoom magnification of camera <NUM>. Scale S which is the scale of map information M is displayed on map information M, and the scale of map information M is displayed on scale S.

When operator X performs a so-called pinch-out operation in which operator X detaches the two fingers from the touch panel of crane-side display terminal <NUM> or display terminal <NUM> with the fingers, control apparatus <NUM> increases the scale of map information M. Further, when the operator X performs a so-called pinch-in operation in which operator X approaches two fingers while touching the touch panel of the crane-side display terminal <NUM> or the display terminal <NUM> with the fingers, control apparatus <NUM> reduces the scale of map information M.

Here, it is assumed that the scale of map information M is a and the width of load W on image I is b prior to the change of the scale of map information M (refer to <FIG>). It is assumed that the scale of map information M after the scale of map information M has been changed by the pinch-out operation is c (refer to <FIG>). It is assumed that the actual lateral width of load W is e (not shown).

At this time, the change rate of the scale is c/a. Control apparatus <NUM> changes the zoom magnification of camera <NUM> at the c/a which is the change rate of scale to change the distance ratio of image I. The horizontal width of load W on image I to be displayed on map information M displayed in scale c shall be Y. The distance ratio before the scale of map information M is changed is b/e, and the distance ratio after the scale of map information M is changed is Y/e. The change rate of the distance ratio is Y/e·e/b=Y/b. Since the change rate of the scale c/a is equal to the change rate of the distance ratio Y/b, c/a=Y/b, that is, Y=b·c/a. Therefore, by displaying image I so that Y=b·c/a, control apparatus <NUM> can display image I in the area on the map information M having the same size as the imaging area of camera <NUM> even if the scale of map information M is changed.

Next, with reference to <FIG>, the display of map information M when the zoom magnification of camera <NUM> is changed will be described.

Operator X changes the zoom magnification of camera <NUM> by operating zoom manipulation tool <NUM> of crane <NUM>. At this time, control apparatus <NUM>, while changing the distance ratio of image I at the change rate of the zoom magnification, changes the scale of map information M. That is, control apparatus <NUM> changes the scale of map information M at the change rate of the distance ratio.

Here, it is assumed that the scale of map information M is a and the width of load W on image I is b before the zoom magnification of camera <NUM> is changed (see <FIG>). Assume that the width of the load W on image I is d after the zoom magnification of camera <NUM> is changed by the operation of the zoom manipulation tool <NUM> (see <FIG>). It is assumed that the actual lateral width of load W is e (not shown).

The distance ratio before the scale of map information M is changed is b/e, and the distance ratio after the scale of map information M is changed is d/e. At this time, the change rate of the distance ratio is d/e·e/b=d/b. Control apparatus <NUM> changes the scale of map information M at the change rate b of the distance ratio. The scale of map information M to be displayed together with image I displayed in the distance ratio d/b shall be set to Z. The change rate of the scale is Z/a. Since the change rate Z/a of the scale is equal to the change rate d/e of the distance ratio, Z/a=d/b, that is, Z=a·d/b. Therefore, by displaying map information M so that Z=a d/b, control apparatus <NUM>, even if the zoom magnification of camera <NUM> is changed, can display image I in the area on map information M having the same size as the imaging area of camera <NUM>.

Next, with reference to <FIG><FIG>, as a first illustrating example, the display mode of map information M and image I by the digital zoom of camera <NUM> will be described. First, a case will be described in which the operation of changing the zoom magnification of camera <NUM> is performed.

When the operation of increasing the zoom magnification of camera <NUM> is performed while image I is displayed on map information M (see <FIG>), control apparatus <NUM> increases the distance ratio of image I by the digital zoom of camera <NUM>, and increases the scale of map information M at the change rate of the distance ratio (see <FIG>). At this time, control apparatus <NUM> increases the size of image I in accordance with the distance ratio. When the operation of increasing the zooming ratio of cameras <NUM> is performed until a state in which image I is displayed on the entire display screen, control apparatus <NUM> displays only image I on crane-side display terminal <NUM> or display terminal <NUM> (refer to <FIG>). As a result, operator X can acquire the state of load W and the detailed information about its surroundings by image I. It should be noted that the center portion of image I may be cropped and displayed by image processing so that the size of image I becomes constant.

When the operation of reducing the zoom magnification of camera <NUM> is performed while displaying only image I (see <FIG>), control apparatus <NUM>, while reducing the distance ratio of image I by the digital zoom of camera <NUM>, reduces the scale of map information M at the change rate of the distance ratio (see <FIG>). At this time, control apparatus <NUM> reduces the size of image I in accordance with the distance ratio. As a result, control apparatus <NUM> displays map information M and image I on crane-side display terminal <NUM> or display terminal <NUM>. By this, operator X can acquire the state of load W and the information of the grounded objects around it by image I and map information M.

Next, the case where operations for changing the scale of map information M are performed will be described with reference to <FIG>.

When the pinch-out operation is performed while image I is displayed on map information M (refer to <FIG>), control apparatus <NUM> increases the scale of map information M and increases the distance ratio of image I by the digital zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). When the pinch-out operation is performed until image I is displayed on the entire display screen, control apparatus <NUM> displays only image I on the crane-side display terminal <NUM> or the display terminal <NUM> (refer to <FIG>).

When the pinch-in operation is performed while only image I is displayed (refer to <FIG>), control apparatus <NUM> reduces the scale of map information M and reduces the distance ratio of image I by the digital zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). As a result, control apparatus <NUM> displays map information M and image I on crane-side display terminal <NUM> or display terminal <NUM>.

Next, with reference to <FIG>, as a second illustrating example, the display mode of map information M and image I by the optical zoom of camera <NUM> will be described. First, the case where the operation of changing the optical zoom magnification of camera <NUM> is performed will be described.

When the operation of increasing the optical zoom magnification of camera <NUM> is performed while displaying image I on map information M (refer to <FIG>), control apparatus <NUM> increases the distance ratio of image I by the optical zoom of camera <NUM>, and increases the scale of map information M at the change rate of the distance ratio (refer to <FIG>). Since the scale of map information M becomes larger and the imaging range of image I becomes smaller at the same change rate as the change rate of the scale, the size of image I does not change. By this, operator X can clearly acquire the state of load W and the detailed information about its surroundings by image I.

If the operation of reducing the optical zoom magnification of camera <NUM> while displaying image I on map information M is performed (refer to <FIG>), control apparatus <NUM>, together with reducing the distance ratio of the image I by the optical zoom, the image I to reduce the scale of the map information M at the change rate of the distance ratio (refer to <FIG>). At this time, since the scale of map information M becomes smaller and the imaging range of image I becomes larger at the same change rate as the change rate of the scale, the size of image I does not change. By this, operator X can acquire the state of load W and the information of the grounded object around it by image I and map information M.

Next, the case where operations for changing the scale of map information M is performed will be described with reference to <FIG>.

When the pinch-out operation is performed while displaying image I on map information M (refer to <FIG>), control unit <NUM> increases the scale of map information M, and increases the distance ratio of image I by the optical zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). At this time, since the scale of map information M becomes larger and the imaging range of image I becomes smaller at the same change rate as the change rate of the scale, the size of image I does not change.

When the pinch-in operation is performed while image I is displayed on map information M (refer to <FIG>), control apparatus <NUM> reduces the scale of map information M and reduces the distance ratio of image I by the optical zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). At this time, since the scale of map information M becomes smaller, and the imaging range of image I becomes larger at the same change rate as the change rate of the scale, the size of image I does not change.

Next, with reference to <FIG>, as a third illustrating example, the display mode of map information M and image I by both the optical zoom and the digital zoom of camera <NUM> will be described. First, the case where the operation of changing the zoom magnification of camera <NUM> is performed will be described.

When the operation of increasing the zoom magnification of camera <NUM> is performed while displaying image I on map information M (refer to <FIG>), control apparatus <NUM> increases the distance ratio of image I at a magnification obtained by multiplying the digital zoom magnification of camera <NUM> by the optical zoom magnification of camera <NUM>, and increases the scale of map information M at the change rate of the distance ratio (refer to <FIG>). Specifically, controller <NUM> increases the distance ratio of image I by the digital zoom of camera <NUM>, and increases the scale of map information M at the change rate of the distance ratio. At this time, control apparatus <NUM> increases the size of image I in accordance with the distance ratio. At the same time, control apparatus <NUM> increases the distance ratio of image I by the optical zoom of camera <NUM>, and increases the scale of the map information M at the change rate of the distance ratio. Since the scale of map information M becomes larger and the imaging range of image I becomes smaller at the same change rate as the change rate of the scale, the size of image I does not change. Therefore, image I having the increased size at the change rate of the distance ratio by the digital zoom of camera <NUM> is displayed. By this, operator X can clearly acquire the state of load W and the detailed information on wide range of surroundings by image I. It should be noted that the ratio of the digital zoom magnification and the optical zoom magnification of camera <NUM> may be any ratio.

When the operation of reducing the zoom magnification of camera <NUM> is performed while displaying image I on map information M (refer to <FIG>), control apparatus <NUM> reduces the distance ratio of image I at the magnification obtained by multiplying the digital zoom magnification of camera <NUM> by the optical zoom magnification of camera <NUM>, and reduces the scale of map information M at the change ratio of the distance ratio (refer to <FIG>). Specifically, control apparatus <NUM>, while reducing the distance ratio of image I by the digital zoom of camera <NUM>, reduces the scale of map information M at the change rate of the distance ratio. At this time, control apparatus <NUM> reduces the size of image I in accordance with the distance ratio. At the same time, control apparatus <NUM>, while reducing the distance ratio of image I by the optical zoom of camera <NUM>, reduces the scale of map information M at the change rate of the distance ratio. Since the scale of map information M becomes smaller and the imaging range of image I becomes larger at the same change rate as the change rate of the scale, the size of image I does not change. Therefore, image I having the reduced size at the change rate of the distance ratio by the digital zoom of camera <NUM> is displayed. By this, operator X can acquire the state of load W and the information on the grounded objects around it by image I and map information M.

When the pinch-out operation is performed while image I is displayed on map information M (refer to <FIG>), control apparatus <NUM> increases the scale of map information M and increases the distance ratio of image I by the digital zoom and the optical zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). At this time, by control apparatus <NUM>, the change rate of the magnification and the scale obtained by multiplying the digital zoom magnification and the optical zoom magnification of camera <NUM> are controlled to be equal.

When the pinch-in operation is performed while image I is displayed on map information M (refer to <FIG>), control apparatus <NUM> reduces the scale of map information M and reduces the distance ratio of image I by the digital zoom and the optical zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). At this time, by the control apparatus <NUM>, the change rate of the magnification and the scale obtained by multiplying the digital zoom magnification and the optical zoom magnification of camera <NUM> are controlled to be equal.

Next, with reference to <FIG>, <FIG>, as a fourth illustrating example, the display mode of map information M and image I for switching the optical zoom and the digital zoom of camera <NUM> will be described. First, the case where the operation of changing the zoom magnification of camera <NUM> is performed will be described.

When the operation of increasing the zoom magnification of camera <NUM> is performed while displaying image I on map information M (refer to <FIG>), control apparatus <NUM> increases the distance ratio of image I by the optical zoom of camera <NUM>, and increases the scale of map information M at the change rate of the distance ratio (refer to <FIG>). Since the scale of map information M becomes larger and the imaging range of image I becomes smaller at the same change rate as the change rate of the scale, the size of image I does not change. In the case where the operation of increasing the zoom magnification of camera <NUM> is performed when the optical zoom magnification of camera <NUM> is the maximum magnification, control apparatus <NUM> increases the distance ratio of image I by the digital zoom of camera <NUM>, and increases the scale of map information M at the change rate of the distance ratio (refer to <FIG>). At this time, control apparatus <NUM> increases the size of image I in accordance with the distance ratio. By this, operator X can acquire the state of load W and the detailed information about its surroundings by image I even when the optical zoom of camera <NUM> is at the maximum magnification.

When the operation of reducing the zoom magnification of camera <NUM> is performed while displaying image I on map information M (refer to <FIG>), control apparatus <NUM> reduces the distance ratio of image I by the optical zoom of camera <NUM> together, and reduces the scale of map information M at a change rate of the distance ratio of image I (refer to <FIG>). Then, in the case where the operation of reducing the zoom magnification of camera <NUM> is performed when the optical zoom magnification of camera <NUM> is the minimum magnification, control apparatus <NUM> reduces the distance ratio of image I by the digital zoom of camera <NUM>, and reduces the scale of map information M at a change rate of the distance ratio (refer to <FIG>). At this time, control apparatus <NUM> reduces the size of image I in accordance with the distance ratio. Bu this, operator X can acquire the state of load W and the information of the grounded objects on a wide range of its surroundings by image I and map information M even when the optical zoom of camera <NUM> is at the minimum magnification.

Next, with reference to <FIG>, <FIG>, the case where operations for changing the scale of map information M is performed will be described.

When the pinch-out operation is performed while displaying image I on map information M (refer to <FIG>), control unit <NUM> increases the scale of map information M, and increases the distance ratio of image I by the optical zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). At this time, since the scale of map information M becomes larger and the imaging range of image I becomes smaller at the same change rate as the change rate of the scale, the size of image I does not change. In the case where the pinch-out operation is performed when the optical zoom magnification of camera <NUM> is the maximum magnification, control apparatus <NUM> increases the scale of map information M, and increases the distance ratio of image I by the digital zoom of camera <NUM> at a change rate of the scale (refer to <FIG>).

When the pinch-in operation is performed while image I is displayed on map information M (refer to <FIG>), control apparatus <NUM> reduces the scale of map information M, and reduces the distance ratio of image I by the optical zoom of camera <NUM> at the change rate of the scale (refer to <FIG>). At this time, since the scale of map information M becomes smaller and the imaging range of image I becomes larger at the same change rate as the change rate of the scale, the size of image I does not change. In the case where the pinch-in operation is performed when the optical zoom magnification of camera <NUM> is the minimum magnification, control apparatus <NUM> decreases the distance ratio of image I by the digital zoom of camera <NUM> at the change rate of the scale (refer to <FIG>).

As described above, in the present crane <NUM>, control apparatus <NUM> displays image I in an area on map information M having the same size as the imaging area of camera <NUM>, changes the ratio of the distance (distance ratio) on image I with respect to a predetermined distance in the imaging area at the change rate of the scale when changing the scale of map information M, and changes the scale of map information M at the change rate of the ratio when changing the ratio of the distance (distance ratio) on image I with respect to a predetermined distance in the imaging area. According to such crane <NUM>, even if the scale of map information M and the zoom magnification of camera <NUM> are changed, since the area on map information M and the imaging area of camera <NUM> are interlocked, the information corresponding to the scale and the zoom magnification can be displayed.

Next, with reference to <FIG>, as a fifth illustrating example, a display mode in which input information is shared will be described.

Control apparatus <NUM> displays image I in an area on map information M having the same size as the imaging area of camera <NUM>. Then, in one of crane-side display terminal <NUM> and display terminal <NUM>, operator X inputs information to be displayed on map information M and image I. The input of the information to be displayed is performed by handwriting input or arrangement of marks on the touch panel. For example, installation position image Mf1 and installation position image Mf2 which are the marks indicating the installation positions of load W are displayed on map information M (refer to <FIG>). A handwritten slinging position image Mg indicating a slinging position is displayed on map information M. Further, on map information M, a handwritten carriage route image Mh indicating the carriage route of load W is displayed. In addition, a handwritten entry prohibition instruction image Mi1 indicating an entry prohibition instruction and a mark entry prohibition instruction image Mi2 are displayed on map information M. Control apparatus <NUM> automatically recognizes the area of entry prohibition instruction image Mi1 overlapping with entry prohibition instruction image Mi2 as the entry prohibition area, and performs control so that load W and boom <NUM> do not enter the entry prohibition area.

Crane-side display terminal <NUM> or display terminal <NUM> to which the information has been input transmits the input information to control apparatus <NUM>. Then, control apparatus <NUM> transmits and displays the input information to crane-side display terminal <NUM> and display terminal <NUM> to which no information has been input (refer to <FIG>). For example, a scaffolder or a site supervisor who grasps the installation position of load W can share the information of the installation position of load W with an operator who performs an operation of carrying load W in cabin <NUM> by inputting the installation position of load W to display terminal <NUM>. Since the installation position of load W is displayed on map information M and image I, it is easier to understand than an instruction by voice using radio, and a difference in recognition of the installation position of load W is less likely to occur between the scaffolder or the site supervisor and the operator. In addition, since the situation of the work site can be shared between the scaffolder or the site supervisor and the operator, the safety and the work efficiency can be improved. Furthermore, since the direction on map information M can be used as the operation direction of load W in the operation method using the carriage direction of load W as a reference, control apparatus <NUM> easily cooperates with the operation method using the carriage direction of load W as a reference.

Next, a mode of hiding image I and sharing it will be described with reference to <FIG>.

Control apparatus <NUM> displays image I in an area on map information M having the same size as the imaging area of camera <NUM>. Then, in one of crane-side display terminal <NUM> and display terminal <NUM>, operator X performs an operation of hiding image I. For example, when operator X touches image I, image I is hidden (refer to <FIG>).

Crane-side display terminal <NUM> and display terminal <NUM> which have been operated transmit to control apparatus <NUM> that the operation of hiding image I has been performed. Then, the control apparatus <NUM> transmits an instruction to hide the image I to crane-side display terminal <NUM> and display terminal <NUM> which have not been operated, and hides image I (refer to <FIG>). Therefore, when map information M is difficult to see for operator X due to the display of image I, only map information M can be displayed by hiding image I. It should be noted that only crane-side display terminal <NUM> and display terminal <NUM> which have been operated may hide image I, and crane-side display terminal <NUM> and display terminal <NUM> which have not been operated may keep image I displayed.

Further, one of crane-side display terminal <NUM> and display terminal <NUM>, when the operation of changing the scale of map information M or the zoom magnification of camera <NUM> is performed, transmit to control apparatus <NUM> that the operation of changing the scale of map information M or the zoom magnification of camera <NUM> is performed. Then, control apparatus <NUM> changes and displays the zoom magnification of camera <NUM> and the scale of map information M of crane-side display terminal <NUM> and display terminal <NUM> which have not been operated.

Next, with reference to <FIG>, the sharing of map information M and image I when the operation is simultaneously performed on a plurality of crane-side display terminal <NUM> and display terminal <NUM> will be described.

When the operation of inputting the information, hiding image I, and changing the scale of map information M or the zoom magnification of camera <NUM> is performed simultaneously in a plurality of crane-side display terminals <NUM> and display terminals <NUM>, there is a possibility that confusion such as a change in the scale of map information M may occur during the input of the information. Therefore, map information M, image I, and the input information may not be simultaneously shared by a plurality of crane-side display terminals <NUM> and display terminals <NUM>. For example, in one of crane-side display terminal <NUM> and display terminal <NUM>, operator X inputs information (refer to <FIG>). At this time, when another operator X changes the scale of map information M in crane-side display terminal <NUM> or display terminal <NUM> to which the information is not input (see <FIG>), the scale of map information M in crane-side display terminal <NUM> or display terminal <NUM> to which the information is input is not changed.

Next, with reference to <FIG>, the rotation of map information M and image I will be described.

Map information M and image I may be rotated as desired by operator X so as to be individually displayed on crane-side display terminal <NUM> or display terminal <NUM>. For example, in one of crane-side display terminal <NUM> and display terminal <NUM>, operator X performs the rotation of map information M and image I by moving the two fingers to twist while touching the touch panel (refer to <FIG>). At this time, in crane-side display terminal <NUM> and display terminal <NUM> which are not operated, map information M and image I are not rotated (refer to <FIG>).

As described above, crane <NUM> includes an information input section (crane-side display terminal <NUM> and display terminal <NUM>) for inputting the information to be displayed on map information M and image I (installation position image Mf1, installation position image Mf2, slinging position image Mg, carriage route image Mh, approach prohibition instruction image Mi1, and approach prohibition instruction image Mi2). In the case where a plurality of displays (<NUM> and <NUM>) are provided, one display (<NUM> and <NUM>) displays map information M, image I, and the information (Mf1·Mf2·Mg·Mh·Mi1·Mi2) input by the information input section, and the other display (<NUM> and <NUM>) displays map information M, image I, and the information (Mf1·Mf2·Mg·Mh·Mi1·Mi2) in the same manner as the one display (<NUM> and <NUM>). According to such crane <NUM>, each operator X inputs the information (Mf1·Mf2·Mg·Mh·Mi1·Mi2) and displays the input information (Mf1·Mf2·Mg·Mh·Mi1·Mi2) on displays (<NUM>, <NUM>) carried by other operators X, thereby allowing each operator X to issue instructions at the discretion of each operator X and to communicate with each other.

Next, with reference to <FIG>, <FIG>, a display mode of map information M and image I when load W is moved by automatic control will be described. When load W is moved by automatic control, operator X indicates the installation position of load W, whereby control apparatus <NUM> automatically generates the carriage route of load W. Crane <NUM> automatically moves load W along the generated carriage route of load W.

As shown in <FIG><FIG>, in one of crane-side display terminal <NUM> and display terminal <NUM>, the installation position of load W is input by operator X, installation position image Mj of the mark indicating the installation position of load W is displayed (refer to <FIG>). Control apparatus <NUM> generates the carriage route of load W based on the current position and the installation position of load W, and displays carriage route image Mk. Therefore, control apparatus <NUM> can display the carriage route of load W so that operator X can confirm the carriage route on map information M. When load W is automatically moved along the generated carriage route, image I is displayed while moving to the position on map information M corresponding to the coordinates of camera <NUM> (refer to <FIG>). By this, control apparatus <NUM> can display the actual situation around load W being carried so that operator X can monitor and confirm the situation. In addition, control apparatus <NUM> can display the movement of load W so that operator X can easily recognize the movement intuitively. When starting to suspend load W or when load W falls within a predetermined range from the installation position, control apparatus <NUM> displays image I by increasing the distance ratio of image I with the digital zoom or the optical zoom of camera <NUM> (refer to <FIG>). By this, the control apparatus <NUM> can display such that operator X can easily monitor and confirm the surrounding situation of load W and the surrounding situation of the installation position, and adjust and designate the installation position and the like of load W.

As shown in <FIG>, image I may be displayed by fixing the display position of image I on screen of the crane-side display terminal <NUM> or display terminal <NUM>. More specifically, when load W is automatically carried along the generated carriage route, map information M is displayed such that the position of image I matches the position of map information M in accordance with the coordinates of camera <NUM> while the display position of image I on screen of the crane-side display terminal <NUM> or display terminal <NUM> is fixed (refer to <FIG>). At this time, map information M is scrolled and displayed on the screen of crane-side display terminal <NUM> or display terminal <NUM> in accordance with the coordinates of camera <NUM>. As a result, control apparatus <NUM> can display load W so that operator X can constantly monitor and confirm a certain range from load W. Except for fixing the display position of image I on the screen of crane-side display terminal <NUM> or display terminal <NUM>, the display mode is the same as that in the case of changing the display position of image I on the screen of crane-side display terminal <NUM> or display terminal <NUM> (refer to <FIG>). It should be noted that the display mode of map information M and image I when load W is moved has been described as automatically generating the carriage route of load W and automatically moving load W along the generated carriage route, but even when load W is moved by a manual operation, map information M and image I can be displayed in the same display mode.

Next, information-sharing system <NUM> will be described with reference to <FIG>. Here, crane <NUM> does not include map information receiver <NUM> and display terminal <NUM>, and differs from crane <NUM> in that it does not display map information M, image I and the input information.

The working vehicle in information-sharing system <NUM> is crane <NUM>, the working apparatus is crane device <NUM> (refer to <FIG>). The working vehicle can also be applied to other working vehicles that take images from above by a camera attached to the working apparatus, in addition to crane <NUM>. For example, the working vehicle can be applied to an aerial working vehicle.

System-side control apparatus <NUM> is a control apparatus of information-sharing system <NUM>. System side controller <NUM> is attached to the distal end portion of boom <NUM>. The system-side control apparatus <NUM> is connected to communication apparatus <NUM>. Therefore, system-side control apparatus <NUM> can acquire, from controller <NUM> via communication apparatus <NUM>, image I, the coordinates of camera <NUM>, the operation signal of zoom manipulation tool <NUM>, the orientation in which the vehicle <NUM> is facing, and the orientation on the image I.

System-side control apparatus <NUM> displays image I in an area on map information M having the same size as the imaging area of camera <NUM> in the plurality of display terminals <NUM> based on the information acquired from crane <NUM>. Then, system-side control apparatus <NUM> uses the same display mode as the above-described illustrating examples and embodiments when the scale of map information M is changed, when the zoom magnification of camera <NUM> is changed, when information is input, when the image I is hidden, and the like. Since display terminal <NUM> is a tablet terminal, it can be used by operator X by being disposed inside cabin <NUM>.

As described above, information sharing system <NUM> is connected to the control apparatus of the working vehicle via communication apparatus <NUM> and acquires necessary information from the working vehicle, thereby easily configures a system for displaying the same display mode as the above-described embodiments.

As described above, information-sharing system <NUM> includes a control apparatus (system-side control apparatus <NUM>), a plurality of displays (display terminal <NUM>), and an information input section for inputting information (installation position image Mf1, installation position image Mf2, slinging position image Mg, carriage route image Mh, entry prohibition instruction image Mi1, and entry prohibition instruction image Mi2) to be displayed on display (<NUM>). Then, control apparatus (<NUM>) displays image I taken by camera <NUM> in an area on map information M having the same size as the imaging area of camera <NUM> which is the coordinate position of camera <NUM> on map information M, in the case the scale of map information M or the distance ratio on image I with respect to the predetermined distance in the imaging area is changed in one display (<NUM>), displays the area of map information M and the imaging area of camera <NUM> in conjunction in all the displays (<NUM>), and displays the information (Mf1, Mf2, Mg, Mh, Mi1, Mi2) input into the one display (<NUM>) by the information input section in other display (or displays) (<NUM>). According to such information-sharing system <NUM>, it is possible to display the information of the entire work range including the work range outside the imaging area of camera <NUM>, and to share the information (Mf1, Mf2, Mg, Mh, Mi1, Mi2) input in the entire work range among the plurality of operators X.

However, the operation method described with reference to <FIG> is only an example of an applicable operation method, and other operation methods may be used to display any position of map information M and image I, to change the scale of map information M and the zoom magnification of camera <NUM>, to input information to be displayed on map information M and image I, to hide image I, and to rotate map information M and image I.

Although the tablet terminal has been described as an example of display terminal <NUM> in the above embodiment, a so-called head mounted display or a display provided in a remote operation terminal of crane <NUM> can also be applied to the present invention in the same manner. When performing an operation on map information M and image I displayed on the head mount display, operator X can perform an operation on map information M and image I by a gesture operation for recognizing the content of the operation from the movement of operator X, a pointer operation for recognizing the position on map information M and image I from the direction of the head and the direction of the line of sight of operator X, or the like.

Finally, the technical idea disclosed in the present application can be applied to images obtained by combining three-dimensional data and construction drawings of a work site, BIM (Building Information Model) construction data, aerial images taken using a drone, images taken by camera <NUM> from various positions by swivel of boom <NUM>, etc., in addition to map information M. Three-dimensional data is data of the three-dimensional shape of the work site detected by a laser scanner or the like. By cooperating with the BIM, various functions such as automatic notification of a construction procedure change can be provided.

Claim 1:
An information-sharing system (<NUM>) acquiring an image taken by a camera (<NUM>) from a working vehicle (<NUM>) including a working apparatus (<NUM>), the camera (<NUM>) supported by the working apparatus (<NUM>), and a sensor (<NUM>) for acquiring coordinates of the camera (<NUM>), comprising:
a control apparatus (<NUM>);
a plurality of displays (<NUM>); and
an information input section (<NUM>) for inputting information to be displayed on the displays (<NUM>),
wherein the control apparatus (<NUM>) acquires a work range of the working vehicle (<NUM>) from the working vehicle (<NUM>), acquires coordinates of the camera (<NUM>) from the sensor (<NUM>), displays the map information in the work range on all the displays (<NUM>), and displays the image taken by the camera (<NUM>) on an area on the map information having the same size as an imaging area of the camera (<NUM>), the area being a coordinate position of the camera (<NUM>) on the map information,
wherein when the scale of the map information in one of the displays (<NUM>) is changed, the control apparatus (<NUM>) changes a scale of the map information displayed on the other display (<NUM>) and a ratio of a distance on the image with respect to a predetermined distance in the imaging area on all the displays (<NUM>) at a change rate of the scale,
wherein when a ratio of a distance on the image with respect to a predetermined distance in the imaging area on one of the displays (<NUM>) is changed, the control apparatus (<NUM>) changes a scale of the map information displayed on all the displays (<NUM>) and a ratio of a distance on the image with respect to a predetermined distance in the imaging area on the other display (<NUM>) at a change rate of the scale,
wherein the control apparatus (<NUM>) displays information input into one of the displays (<NUM>) by the information input section (<NUM>) on the other display (<NUM>).