Patent Description:
A technique has been proposed in which an operator operates a working machine using a remote operation device while referring to a remote image representing a working status of the working machine (see Patent Literature <NUM>, for example).

Furthermore patent document <CIT> is as well a relevant piece of prior art.

However, when the operator causes an upper revolving body of the working machine to perform a revolving motion with respect to a lower traveling body, the feeling of the revolving motion that the operator perceives through the remote image is generally lower than when actually getting on the working machine. Therefore, the control accuracy of the position (orientation) of the upper revolving body may decrease, and the efficiency of work using the working machine may decrease at last.

Therefore, the object of the present invention to provide a device or the like capable of causing a result of an operator perceiving a motion aspect of a working machine when the operator remotely operates the working machine to approximate to a result of an operator perceiving a motion aspect of a working machine when the operator gets on the working machine.

A remote operation assistance device of the present invention comprises:.

The remote operation device with this configuration makes it possible to determine whether a relative motion of the first component and the second component that constitute the working machine is represented based on a result of detecting the motion state of the working machine or the operation aspect of the remote operation device.

Then, when the determination result is positive, control is performed on the individual sound output aspects of a plurality of sound output devices, which are arranged at different locations in the remote operation space for the operator who operates the remote operation device. This displaces a virtual point sound source along a specified position path (time series of positions), which is determined according to the relative motion aspect of the second component with respect to the first component.

This allows a result of an operator, in the remote operation space, perceiving the motion aspect of the working machine, specifically the relative motion aspect of the first component and the second component, to approximate to a result of the operator perceiving the relative motion aspect of the first component and the second component when the operator gets on the working machine (in other words, this allows the operator to perceive a motional feeling of the working machine).

A remote operation assistance system shown in <FIG> includes a remote operation assistance server <NUM> and a remote operation device <NUM> configured to remotely operate a working machine <NUM>. The remote operation assistance server <NUM>, the remote operation device <NUM>, and working machine <NUM> are configured to be communicable with each other through a network. An intercommunication network between the remote operation assistance server <NUM> and the remote operation device <NUM> may be the same as or different from an intercommunication network between the remote operation assistance server <NUM> and the working machine <NUM>.

The remote operation is a concept that means that an operator operates the working machine <NUM> from a position away from the working machine <NUM> without getting on the working machine <NUM>.

The operator is a concept that means a person who operates the remote operation device <NUM> to operate the working machine <NUM>.

The remote operation assistance server <NUM> comprises a database <NUM>, a first assistance processing element <NUM>, and a second assistance processing element <NUM>. The database <NUM> stores and holds captured image data and the like. The database <NUM> may be constituted of a database server separate from the remote operation assistance server <NUM>. Each assistance processing element includes an arithmetic processing unit (single-core processor, or multi-core processor or a processor core that constitutes this). The assistance processing element reads necessary data and software from a storage device such as a memory, and executes arithmetic processing, to be described below, on the data according to the software.

The remote operation device <NUM> comprises a remote control device <NUM>, a remote input interface <NUM>, and a remote output interface <NUM>. In this embodiment, the remote control device <NUM> constitutes a "remote operation assistance device" of the present invention. The remote control device <NUM> comprises a state detection element <NUM> and a point sound source control element <NUM>. The remote control device <NUM> and components thereof include an arithmetic processing unit (single-core processor, or multi-core processor or a processor core that constitutes this). The assistance processing element reads necessary data and software from a storage device such as a memory, and executes arithmetic processing on the data according to the software.

The remote input interface <NUM> comprises a remote operation mechanism <NUM>. The remote output interface <NUM> comprises an image output device <NUM>, a sound output device <NUM>, and a remote wireless communication device <NUM>.

The remote operation mechanism <NUM> includes a traveling operation device, a revolving operation device, a boom operation device, an arm operation device, and a bucket operation device. Each operating device has an operation lever that receives a turning operation. An operation lever of the traveling operation device (travel lever) is operated to move the lower traveling body <NUM> of the working machine <NUM>. The travel lever may also serve as a travel pedal. For example, a travel pedal may be provided that is fixed to the base or lower end of the travel lever. An operation lever of the revolving operation device (revolving lever) is operated to move a hydraulic revolving motor that constitutes the revolving mechanism <NUM> of the working machine <NUM>. An operation lever of the boom operation device (boom lever) is operated to move a boom cylinder <NUM> of the working machine <NUM>. An operation lever of the arm operation device (arm lever) is operated to move an arm cylinder <NUM> of the working machine <NUM>. An operation lever of the bucket operation device (bucket lever) is operated to move a bucket cylinder <NUM> of the working machine <NUM>.

For example, as shown in <FIG>, the individual operation levers that constitutes the remote operation mechanism <NUM> is arranged around the seat St on which the operator sits. The seat St is in a form of a high-back chair with armrests, but it may be in any form of seating on which the operator can sit, such as a form of low-back chair without a headrest, or a form of chair without a backrest.

A pair of left and right travel levers <NUM> corresponding to left and right crawlers are arranged side by side in front of the seat St. One operation lever may serve as a plurality of operation levers. For example, the left operation lever <NUM>, which is provided in front of a left frame of the seat St shown in <FIG>, may function as an arm lever when operated in the front-rear direction, and as a revolving lever when operated in the left-right direction. Similarly, the right operation lever <NUM>, which is provided in front of a right frame of the seat St shown in <FIG>, may function as a boom lever when operated in the front-rear direction, and as a bucket lever when operated in the left-right direction. The lever pattern may be changed by an operation instruction of the operator in any way.

For example, as shown in <FIG>, the image output device <NUM> includes a central image output device <NUM>, a left side image output device <NUM> and a right side image output device <NUM> each having a substantially rectangular screen, which are respectively arranged in front of the seat St, diagonally forward left thereof, and diagonally forward right thereof. The screens (image display areas) of the central image output device <NUM>, the left side image output device <NUM> and the right side image output device <NUM> may have a shape and size identical to each other, or different shapes and sizes.

As shown in <FIG>, the screen of the central image output device <NUM> and the screen of the left side image output device <NUM> form an inclination angle θ1 (for example, <NUM>° ≤ θ1 ≤ <NUM>°). As such, the right edge of the left side image output device <NUM> is adjacent to the left edge of the central image output device <NUM>. As shown in <FIG>, the screen of the central image output device <NUM> and the screen of the right side image output device <NUM> form an inclination angle θ2 (for example, <NUM>° ≤ θ2 ≤ <NUM>°). As such, the left edge of the right side image output device <NUM> is adj acent to the right edge of the central image output device <NUM>. The inclination angles θ1 and θ2 may be the same or may be different.

The individual screens of the central image output device <NUM>, the left side image output device <NUM>, and the right side image output device <NUM> may be parallel to the vertical direction or may be inclined with respect to the vertical direction. At least one of the central image output device <NUM>, the left side image output device <NUM>, and the right side image output device <NUM> may be constituted of a plurality of divided image output devices. For example, the central image output device <NUM> may be constituted of a pair of vertically adjacent image output devices each having a substantially rectangular screen.

The sound output device <NUM> includes one or a plurality of speakers. For example, as shown in <FIG>, the sound output device <NUM> includes a central sound output device <NUM>, a left side sound output device <NUM> and a right side sound output device <NUM>, which are respectively arranged in the rear side of the seat St, the rear part of the left armrest, and the rear part of the right armrest. The specifications of the central sound output device <NUM>, the left side sound output device <NUM>, and the right side sound output device <NUM> may be the same or may be different.

The working machine <NUM> comprises an actual machine control device <NUM>, an actual machine input interface <NUM>, an actual machine output interface <NUM>, and an actuating mechanism <NUM>. The actual machine control device <NUM> includes an arithmetic processing unit (single-core processor, or multi-core processor or a processor core that constitutes this). The actual machine control device <NUM> reads necessary data and software from a storage device such as a memory, and executes arithmetic processing on the data according to the software.

The working machine <NUM> is, for example, a crawler excavator (construction machine). As shown in <FIG>, the working machine <NUM> comprises a crawler-type lower traveling body <NUM> and an upper revolving body <NUM> mounted on the lower traveling body <NUM> so as to be revolvable via a revolving mechanism <NUM>. The front left part of the upper revolving body <NUM> is provided with a cab <NUM> (operation room). The front central part of the upper revolving body <NUM> is provided with a working mechanism <NUM>.

The actual machine input interface <NUM> comprises an actual machine operation mechanism <NUM>, an actual machine imaging device <NUM>, and an actual machine positioning device <NUM>. The actual machine operation mechanism <NUM> comprises a plurality of operation levers arranged around the seat arranged inside the cab <NUM> in the same manner as the remote operation mechanism <NUM>. The cab <NUM> is provided with a drive mechanism or a robot that receives a signal corresponding to an operation aspect of the remote operation lever and moves the actual machine operation lever based on the received signal. The actual machine imaging device <NUM> is installed inside the cab <NUM>, for example, and images an environment including at least part of the actuating mechanism <NUM> through a front window and a pair of left and right side windows. Some or all of the front window and side windows may be omitted. The actual machine positioning device <NUM> includes a GPS and, if necessary, a gyro sensor.

The actual machine output interface <NUM> comprises an actual machine wireless communication device <NUM>.

The working mechanism <NUM>, as the actuating mechanism, comprises a boom <NUM> installed onto the upper revolving body <NUM> so as to be able to rise and fall, an arm <NUM> turnably connected to the end of the boom <NUM>, and a bucket <NUM> turnably connected to the end of the arm <NUM>. The working mechanism <NUM> has a boom cylinder <NUM>, an arm cylinder <NUM>, and a bucket cylinder <NUM> installed thereon, each constituted of a hydraulic cylinder that can expand and contract.

The boom cylinder <NUM> is interposed between the boom <NUM> and the upper revolving body <NUM> so as to expand and contract when supplied with hydraulic oil to turn the boom <NUM> in the rising and falling direction. The arm cylinder <NUM> is interposed between the arm <NUM> and the boom <NUM> so as to expand and contract when supplied with hydraulic oil to turn the arm <NUM> around the horizontal axis with respect to the boom <NUM>. The bucket cylinder <NUM> is interposed between the bucket <NUM> and the arm <NUM> so as to expand and contract when supplied with hydraulic oil to turn the bucket <NUM> around the horizontal axis with respect to the arm <NUM>.

The following describes a first function of the remote control device <NUM>, as the remote operation assistance device, configured as described above, using a flowchart shown in <FIG>. In the flowchart, the block "C●" is used for simplification of the description, means that data is transmitted and/or received, and means a conditional branch in which the processing in the branch direction is executed on the condition that the data is transmitted and/or received.

The remote operation device <NUM> determines whether the operator has performed a specifying operation via the remote input interface <NUM> (<FIG>/STEP <NUM>). The "specifying operation" is, for example, an operation such as tapping on the remote input interface <NUM> for specifying the working machine <NUM> for which the operator intends remote operation. If the determination result is negative (<FIG>/STEP <NUM>: NO), the series of processing ends. On the other hand, if the determination result is positive (<FIG>/STEP <NUM>: YES), an environment confirmation request is transmitted to the remote operation assistance server <NUM> via the remote wireless communication device <NUM> (<FIG>/STEP <NUM>).

If the remote operation assistance server <NUM> receives an environment confirmation request, the first assistance processing element <NUM> transmits the environment confirmation request to the relevant working machine <NUM> (<FIG>/C10).

If the working machine <NUM> receives an environment confirmation request via the actual machine wireless communication device <NUM> (<FIG>/C40), the actual machine control device <NUM> acquires a captured image through the actual machine imaging device <NUM> (<FIG>/STEP <NUM>). The actual device control device <NUM> then transmits captured image data representing the captured image to the remote operation assistance server <NUM> via the actual machine wireless communication device <NUM> (<FIG>/STEP <NUM>).

In the remote operation assistance server <NUM>, when the first assistance processing element <NUM> receives captured image data (<FIG>/C11), the second assistance processing element <NUM> transmits environmental image data corresponding to the captured image to the remote operation device <NUM> (<FIG>/STEP <NUM>). The environmental image data are the captured image data itself and additionally image data representing a simulated environmental image generated based on the captured image.

When the remote operation device <NUM> receives environmental image data via the remote wireless communication device <NUM> (<FIG>/C21), the remote control device <NUM> outputs an environmental image corresponding to the environmental image data to the image output device <NUM>. (<FIG>/STEP <NUM>).

As a result, for example, as shown in <FIG>, an environmental image in which the boom <NUM>, arm <NUM> and bucket <NUM>, which are part of the working mechanism <NUM>, are reflected is output to the image output device <NUM>.

In the remote operation device <NUM>, the remote control device <NUM> recognizes the operation aspect of the remote operation mechanism <NUM> (<FIG>/STEP <NUM>), and transmits a remote operation command corresponding to the operation aspect to the remote operation assistance server <NUM> via the remote wireless communication device <NUM> (<FIG>/STEP <NUM>).

In the remote operation assistance server <NUM>, if the second assistance processing element <NUM> receives the remote operation command, the first assistance processing element <NUM> transmits the remote operation command to the working machine <NUM> (<FIG>/C12).

In the working machine <NUM>, if the actual machine control device <NUM> receives an operation command via the actual machine wireless communication device <NUM> (<FIG>/C41), control is performed on the motion of the working mechanism <NUM> and the like (<FIG>/STEP <NUM>). For example, an operation is executed in which soil in front of the working machine <NUM> is scooped with the bucket <NUM>, the upper revolving body <NUM> is revolved, and the soil is dropped from the bucket <NUM>.

The following describes the second function of the remote control device <NUM>, as the remote operation assistance device, configured as described above, using the flowchart shown in <FIG>.

In the remote control device <NUM>, the state detection element <NUM> recognizes the operation aspect of the remote operation mechanism <NUM> (<FIG>/STEP <NUM>). This is the same processing as <FIG>/STEP <NUM>. Alternatively, the state detection element <NUM> may recognize or detect the motion state of working machine <NUM> based on communication with working machine <NUM>.

The state detection element <NUM> determines whether the operation aspect is the operation aspect of revolving lever for revolving the upper revolving body <NUM> (second component) with respect to the lower traveling body <NUM> (first component) (<FIG>/STEP <NUM>). If the operation aspect is determined to be an operation aspect for revolving the upper revolving body <NUM> with respect to the lower traveling body <NUM> as shown in <FIG> (<FIG>/STEP <NUM>: YES), the point sound source control element <NUM> controls the sound output aspects of the sound output devices <NUM> so that the position of a virtual point sound source is displaced along a first specified position path (<FIG>/STEP <NUM>).

As a result, for example, as shown in <FIG>, a virtual point sound source SP1 is displaced along a first specified position path tr1 extending in a ring (for example, an annular or elliptical ring) surrounding a remote operation area (seat St) of the operator. The first specified position path tr1 may extend along a plane (cross section) parallel to the floor surface of the cab <NUM> or an inclined plane, or may extend along a curved surface such as a spherical surface, an ellipsoidal surface, or a wave surface. The height position of the plane or curved surface may be adjusted to fit within a range of normal operator ear height positions.

For example, when a revolving angular velocity (relative motion speed) of the upper revolving body <NUM> with respect to the lower traveling body <NUM> is zero, the position of the reference point of the virtual point sound source SP1 is adjusted so that it is positioned on the rear side of or directly behind the remote control area. The displacement direction of the virtual point sound source SP1 is controlled to be the same as the revolving direction of the upper revolving body <NUM> with respect to the lower traveling body <NUM>. In addition, the displacement angular velocity of the virtual point sound source SP1 is controlled to be the same as the revolving angular velocity of the upper revolving body <NUM>. This allows when the operation aspect of the remote operation mechanism <NUM> is an operation aspect in which the upper revolving body <NUM> is revolved counterclockwise with respect to the lower traveling body <NUM> (viewed from above), by an angle θ at an angular velocity ω(t) from time t=t0 to time t=t1, the virtual point sound source SP1 is displaced so as to revolve counterclockwise along the first specified position path tr1 by an angle θ at an angular velocity ω(t) from time t=t0 to time t=t1.

As shown in <FIG>, the first specified position path tr1 may extend so as to be biased to the rear side of the remote operation space of the operator. In this case, the virtual point sound source SP1 can be displaced along the specified position path, which is more biased to the rear side of the remote operation space than on the front side thereof. This allows the operator to perceive a motional feeling of the working machine <NUM> without losing his/her attention to the front.

On the other hand, if it is determined that the operation aspect is not an operation aspect for revolving the upper revolving body <NUM> with respect to the lower traveling body <NUM> (<FIG>/STEP <NUM>: NO), the state detection element <NUM> determines whether the operation aspect is an operation aspect of an arm lever or the like for displacing the bucket <NUM> (second component) as a working unit with respect to the upper revolving body <NUM> (first component) (<FIG>/STEP <NUM>). If the operation aspect is determined to be the operation aspect of the arm lever or the like for displacing the bucket <NUM> (second component) as a working unit with respect to the upper revolving body <NUM> (first component) (<FIG>/STEP <NUM>: YES), the point sound source control element <NUM> controls the sound output aspect of the sound output device <NUM> so that the position of the virtual point sound source is displaced along a second specified position path (<FIG>/STEP <NUM>). If the determination result is negative (<FIG>/STEP <NUM>: NO), the series of processing ends.

Thus, as shown in <FIG>, for example, the virtual point sound source SP1 is displaced along the second specified position path tr2 extending on the rear and upper sides of the operator's remote control area (seat St). The second specified position path tr2 may extend along a plane (sagittal plane) perpendicular to the floor surface of the cab <NUM> or an inclined plane, and may extend along a curved surface such as a spherical surface, an ellipsoidal surface, or a wave surface. The plane or curved surface may be adjusted to fit within a range of normal operator head positions.

For example, when the displacement speed (relative motion speed) of the bucket <NUM> with respect to the upper revolving body <NUM> is zero, the position of the reference point of the virtual point sound source SP1 is adjusted so that it is positioned on the rear side of or directly behind the remote control area. The displacement direction of the virtual point sound source SP1 is controlled in the same direction as the movement direction of the bucket <NUM> with respect to the upper revolving body <NUM>, and the displacement speed of the virtual point sound source SP1 is controlled to be proportional to the displacement speed of the bucket <NUM>. This allows if operation aspect of the operation mechanism <NUM> is an operation aspect for lowering the bucket <NUM> with respect to the upper revolving body <NUM> by a lowering amount Δ at a displacement speed v(t) from time t=t1 to time t=t2, the virtual point sound source SP1 is displaced along the second specified position path tr2 by an amount yΔ proportional to the lowering amount Δ at a displacement speed yv(t) from time t=t1 to time t=t2.

The remote control device <NUM> that constitutes the remote operation assistance device having this configuration makes it possible to determine whether there is represented the relative motion of the first component (lower traveling body <NUM>, upper revolving body <NUM>) and the second component (upper revolving body <NUM>, bucket <NUM> (working unit)), each constituting the working machine <NUM>, based on the result of detecting the operation aspect of the remote operation device <NUM> (see <FIG>/STEP <NUM> and STEP <NUM>).

Then, if the determination result is positive, control is performed on the individual sound output aspects of the plurality of sound output devices <NUM>, <NUM> and <NUM>, each arranged at different locations in the remote operation space for the operator who operates the remote operation device <NUM>. This displaces the virtual point sound source SP1 along the specified position paths (time series of positions) tr1 and tr2, which are determined according to the relative motion aspect of the second component with respect to the first component (<FIG>/STEP <NUM>: YES -> STEP <NUM>, STEP <NUM>: YES -> STEP <NUM>, see <FIG>).

This allows the result of the operator, in the remote operation space, perceiving the motion aspect of the working machine <NUM>, specifically the relative motion aspect of the first component and the second component, to approximate to a result of the operator perceiving the relative motion aspect of the first component and the second component when the operator gets on the working machine <NUM> (or in its cab <NUM>).

In displacing the virtual point sound source SP1, the position of the virtual point sound source SP1 is controlled to a position of the predetermined reference point when the relative motion speed of the second component with respect to the first component is zero. Since the position of the virtual point sound source SP1 does not vary when the relative motion speed of the second component with respect to the first component is zero, the operator can uniformly perceive the motional feeling of the working machine <NUM>. The reference point is positioned on the rear side of the operator in the remote operation space, so that the virtual point sound source SP1 is displaced from the reference point on the rear side of the operator. This allows the operator to perceive the motional feeling of the working machine <NUM> without losing his/her attention to the front.

In the above embodiment, the remote operation assistance device is constituted of the remote control device <NUM>, but another embodiment may be such that: the remote operation assistance device is constituted of the remote operation assistance server <NUM>; and the first assistance processing element <NUM> and/or the second assistance processing element <NUM> function as the state detection element <NUM> and/or the point sound source control element <NUM>.

The point sound source control element <NUM> may change at least one of a frequency distribution and a volume of a sound of the virtual point sound source SP1 when displacing the virtual point sound source SP1.

According to the remote operation assistance device with this configuration, the virtual point sound source SP1 is displaced along the specified position path determined according to the relative motion aspect of the second component with respect to the first component of the working machine <NUM>, and also the frequency distribution and/or the volume of the sound of the virtual point sound source changes. This allows the operator in the remote operation space to more clearly pay attention to the virtual point sound source and more clearly perceive the position and/or displacement aspect of the virtual point sound source. Then, this allows the result of the operator perceiving the relative motion aspect of the first component and the second component to approximate to the result of the operator perceiving the relative motion aspect of the first component and the second component when the operator gets on the working machine <NUM>.

As a displacement speed of the virtual point sound source SP1 is higher, the point sound source control element <NUM> may control a frequency distribution of a sound of the virtual point sound source SP1 to a higher frequency side, and additionally or alternatively may control a volume to be louder.

According to the remote operation assistance device with this configuration, the virtual point sound source SP1 is displaced along the specified position paths tr1 and tr2 determined according to the relative motion aspect of the second component with respect to the first component of the working machine <NUM>. In addition, the frequency distribution of the sound of the virtual point sound source SP1 is adjusted so as to shift to the higher frequency side or the lower frequency side and/or volume thereof is adjusted, according to the displacement speed. This allows the operator in the remote operation space to more clearly pay attention to the virtual point sound source, and to more clearly perceive particularly the displacement aspect of the virtual point sound source based on the difference in frequency or the like. Then, this allows the result of the operator perceiving the relative motion aspect of the first component and the second component to approximate to the result of the operator perceiving the relative motion aspect of the first component and the second component when the operator gets on the working machine <NUM>.

In the remote operation assistance device of the present invention, it is preferable that
the point sound source control element changes at least one of a frequency distribution and a volume of a sound of the virtual point sound source when displacing the virtual point sound source.

According to the remote operation assistance device with this configuration, the virtual point sound source is displaced along a specified position path determined according to the relative motion aspect of the second component with respect to the first component. In addition, the frequency distribution and/or volume of the sound of the virtual point sound source changes. This allows the operator in the remote operation space to more clearly pay attention to the virtual point sound source and more clearly perceive the position and/or displacement aspect of the virtual point sound source. Then, this allows the result of the operator perceiving the relative motion aspect of the first component and the second component to approximate to the result of the operator perceiving the relative motion aspect of the first component and the second component when the operator gets on the working machine.

In the remote operation assistance device of the present invention, it is preferable that as a displacement speed of the virtual point sound source is higher, the point sound source control element controls a frequency distribution of a sound of the virtual point sound source to a higher frequency side, and additionally or alternatively control a volume to be louder.

According to the remote operation assistance device with this configuration, the virtual point sound source is displaced along a specified position path determined according to the relative motion aspect of the second component with respect to the first component. In addition, the frequency distribution of the sound of the virtual point sound source is adjusted so as to shift to the higher frequency side or the lower frequency side and/or volume thereof is adjusted, according to the displacement speed. This allows the operator in the remote operation space to more clearly pay attention to the virtual point sound source, and to more clearly perceive particularly the displacement aspect of the virtual point sound source based on the difference in frequency or the like. Then, this allows the result of the operator perceiving the relative motion aspect of the first component and the second component to approximate to the result of the operator perceiving the relative motion aspect of the first component and the second component when the operator gets on the working machine.

In the remote operation assistance device of the present invention, it is preferable that the point sound source control element displaces a virtual point sound source along a specified position path when a motion state of the working machine or an operation aspect of the remote operation device represents a revolving motion of an upper revolving body, as the second component, with respect to a lower traveling body, as the first component, the specified position path being determined according to a revolving speed of the upper revolving body with respect to the lower traveling body, the specified position path extending in a curved shape so as to partially surround an operator in the remote operation space, the motion state of the working machine or the operation aspect of the remote operation device being detected by the state detection element.

The remote operation assistance device with this configuration allows the result of the operator, in the remote operation space, perceiving the revolving motion of the upper revolving body (second component) with respect to the lower traveling body (first component) to approximate to the result of the operator perceiving the revolving motion of the upper revolving body with respect to the lower traveling body when the operator gets on the working machine.

In the remote operation assistance device of the present invention, it is preferable that the point sound source control element displaces a virtual point sound source along a specified position path when a motion state of the working machine or an operation aspect of the remote operation device represents a relative motion of a working unit of a working mechanism, as the second component, with respect to an upper revolving body, as the first component, the specified position path being determined according to a relative motion aspect of the working unit with respect to the upper revolving body, the motion state of the working machine or the operation aspect of the remote operation device being detected by the state detection element.

The remote operation assistance device with this configuration allows the result of the operator, in the remote operation space, perceiving the relative motion aspect of the working unit (second component) with respect to the upper revolving body (first component) to approximate to the result of the operator perceiving the relative motion aspect of the working unit with respect to the upper revolving body when the operator gets on the working machine.

In the remote operation assistance device of the present invention, it is preferable that in a case in which the point sound source control element displaces the virtual point sound source, the point sound source control element positions the virtual point sound source at a predetermined reference point when a relative motion speed of the second component with respect to the first component is zero.

According to the remote operation assistance device with this configuration, the virtual point sound source is adjusted so that: it is positioned at the reference point when the relative motion speed of the second component with respect to the first component is <NUM> (zero); and it is displaced from the base point at a speed corresponding to the relative motion speed when the relative motion speed becomes greater than zero. The position of the virtual point sound source returns to the reference point when the relative motion speed of the second component with respect to the first component is zero, and the position of the virtual point sound source does not vary. Therefore, the operator can uniformly perceive the motional feeling of the working machine.

In the remote operation assistance device of the present invention, it is preferable that the point sound source control element positions the reference point on a rear side of an operator in the remote operation space.

According to the remote operation assistance device with this configuration, the virtual point sound source is displaced from the reference point on the rear side of the operator. This allows the operator to perceive the motional feeling of the working machine without losing his/her attention to the front.

In the remote operation assistance device of the present invention, it is preferable that the point sound source control element displaces the virtual point sound source along the specified position path biased to a rear side of an operator in the remote operation space.

According to the remote operation assistance device with this configuration, the virtual point sound source can be displaced along the specified position path, which is more biased to the rear side of the remote operation space than on the front side thereof. This allows the operator to perceive the motional feeling of the working machine without losing his/her attention to the front.

Claim 1:
A remote operation assistance device, (<NUM>) comprising:
a state detection element (<NUM>) configured to detect a motion state of a working machine (<NUM>) or an operation aspect of a remote operation device, (<NUM>) the remote operation device being configured to remotely control the working machine; and
a point sound source control element (<NUM>) configured to control individual sound output aspects of a plurality of sound output devices (<NUM>) to displace a virtual point sound source (SP1) a specified position path (tr1, tr2) when a motion state of the working machine or an operation aspect of the remote operation device represents a relative motion of a second component (<NUM>) with respect to a first component, (<NUM>) the plurality of sound output devices each being arranged at different locations in a remote operation space for an operator who operates the remote operation device, the first component and the second component constituting the working machine, the specified position path being determined according to a relative motion aspect of the second component with respect to the first component, the motion state of the working machine or the operation aspect of the remote operation device being detected by the state detection element.