Patent Description:
Patent Literature <NUM> discloses that images of a work region of a press brake are captured by a plurality of image capturing devices, and the captured images captured by the plurality of image capturing devices are displayed on display means. Patent Literature <NUM>, which forms the basis for the preambles of claims <NUM> and <NUM>,discloses a controller capable of further accurately reflecting an actual position of a worker in the control of the operation speed of a forging pressure machine. The controller obtains background image data which does not include images of a workpiece and a worker of the forging pressure machine, and current-state image data presenting a current state of a predetermined area when the forging pressure machine operates, from a sensor which senses a predetermined area including a machining area of the forging pressure machine to output image data for discriminating a distance in a height direction. The system controller calculates difference data between the background image data and the current-state image data, and determines an area of the worker on the basis of the difference data.

However, since the structure of the press brake is complicated, in addition to a gaze object that a user desires to gaze at, various other objects are present in the captured image captured by the image capturing device. In addition, the press brake has similar colors overall. Therefore, when the user confirms the display means, there is an inconvenience that it is difficult to gaze at the gaze object within the captured image in an efficient manner.

The object is achieved with a press brake according to claim <NUM>, and with an image output method according to claim <NUM>. One aspect of the present invention is a press brake including a press brake main body, an image capturing device, a distance measuring device, and an image processing device. The press brake main body is provided with an upper table configured to hold an upper tool and a lower table configured to hold a lower tool, and is configured to carry out a bending process to a plate-shaped workpiece when the upper table moves up and down relative to the lower table. The image capturing device captures an image of a work region in which the upper table and the lower table carry out the bending process in the press brake main body, and outputs a captured image. The distance measuring device detects a distance to an object present in the captured image and generates distance data in which the object and the distance are associated with each other. The image processing device generates, based on the distance data, a gaze image obtained by cutting out, from the captured image, a gaze object that is an object to be gazed at within the work region, and outputs the gaze image to a target device usable by a user.

According to the one aspect of the present invention, it is possible to recognize the distance to the object present in the captured image by generating the distance data. Since the distance data is data in which the object and the distance are associated with each other, by referring to the distance data, the image processing device can separate the gaze object and the other objects among the objects images of which are captured by the image capturing device. This makes it possible for the image processing device to generate the gaze image obtained by cutting out the gaze object from the captured image. Then, when the image processing device outputs the gaze image to the target device, the user can use the gaze image via the target device. Since the gaze image is an image obtained by cutting out the gaze object, the gaze object can be visually recognized more easily as compared with the case in which the captured image is visually recognized as it is.

According to the one aspect of the present invention, it is possible to visually recognize the gaze object in the captured image in an efficient manner.

A press brake according to the present embodiment will be described with reference to the drawings.

<FIG> is a front view schematically showing an overall configuration of a press brake according to a first embodiment. <FIG> is a side view schematically showing the overall configuration of the press brake according to the first embodiment. <FIG> is a block diagram showing a configuration of a control system of the press brake according to the first embodiment. The press brake according to the present embodiment is provided with an upper table <NUM> that holds a punch <NUM> and a lower table <NUM> that holds a die <NUM>, and includes a press brake main body <NUM> that carries out a bending process to a plate-shaped workpiece W when the upper table <NUM> moves up and down relative to the lower table <NUM>, a camera <NUM> that captures an image of a work region in which the upper table <NUM> and the lower table <NUM> carry out the bending process in the press brake main body <NUM> and outputs a captured image ID, a distance measuring device <NUM> that detects a distance to an object present in the captured image ID and generates distance data DD in which the object and the distance are associated with each other, and an image processing device <NUM> that generates, based on the distance data DD, a gaze image CD obtained by cutting out, from the captured image ID, a gaze object that is an object to be gazed at within the work region and outputs the gaze image CD to a target device usable by a user.

Hereinafter, the configuration of the press brake will be explained in detail. The press brake includes the press brake main body <NUM>, an NC device <NUM>, the camera <NUM>, the distance measuring device <NUM>, the image processing device <NUM>, and a display device <NUM>.

The configuration of the press brake main body <NUM> will be described. "FF", "FR", "L", "R", "U", and "D" shown in the drawings refer to a forward direction, a backward direction, a left direction, a right direction, upward direction and a downward direction, respectively.

The press brake main body <NUM> is a working machine that carries out the bending process to the plate-shaped workpiece (sheet metal) W by a pair of tools. The press brake main body <NUM> is provided with the lower table <NUM> and the upper table <NUM>.

The lower table <NUM> is provided at the lower part of the main body frame <NUM> and extends in the lateral direction. The lower table <NUM> holds a die <NUM> that is a lower tool. A lower tool holder <NUM> is attached on the upper end side of the lower table <NUM>, and a die <NUM> is mounted on the lower tool holder <NUM>.

The upper table <NUM> is provided at the upper part of the main body frame <NUM> and extends in the lateral direction. The upper table <NUM> is provided above the lower table <NUM> so as to face the lower table <NUM>. The upper table <NUM> holds a punch <NUM> that is an upper tool. An upper tool holder <NUM> is attached on the lower end side of the upper table <NUM>, and a punch <NUM> is mounted on the upper tool holder <NUM>.

The upper table <NUM> is configured to move up and down with respect to the lower table <NUM> when a pair of hydraulic cylinders <NUM> provided on the left and right are driven up and down, respectively. The individual hydraulic cylinders <NUM> are driven up and down when an actuator mainly composed of a pump and a motor is operated. The vertical position of the upper table <NUM> is detected by a position detection sensor such as an unillustrated linear encoder. Position information detected by the position detection sensor is supplied to the NC device <NUM>.

The press brake main body <NUM> may have a configuration in which the lower table <NUM> is moved up and down in lieu of the configuration in which the upper table <NUM> is moved up and down. In other words, the upper table <NUM> may be configured to move up and down relative to the lower table <NUM>.

An unillustrated table cover that covers the upper table <NUM> is fixedly attached to the main body frame <NUM>. Even when the upper table <NUM> moves up and down, the table cover does not move up and down and maintains a stationary state.

In the press brake main body <NUM>, the workpiece W is placed on, for example, the die <NUM>. When the upper table <NUM> is lowered, the workpiece W is sandwiched between the punch <NUM> and the die <NUM> to be bent.

A foot switch <NUM> on which an operator M carries out a stepping operation is installed in front of the lower table <NUM>. When the operator M carries out the stepping operation, the foot switch <NUM> outputs an activation signal. The activation signal is a signal for starting a lowering operation of the upper table <NUM>.

Behind the lower table <NUM>, a back gauge <NUM> for positioning the workpiece W in the front-rear direction with respect to the die <NUM> is provided. The back gauge <NUM> includes an abutting member <NUM> against which the end face of the workpiece W can be abutted. The abutting member <NUM> protrudes forward from the back gauge <NUM>. The position of the abutting member <NUM> in the front-rear direction is adjustable.

In the press brake main body <NUM>, a three-dimensional space, which includes the lower table <NUM> and the surroundings thereof, and the upper table <NUM> and the surroundings thereof, corresponds to the work region in which the lower table <NUM> and the upper table <NUM> carry out the bending process. A gaze region GR, which is to be gazed at within the work region, is defined in the work region.

The gaze region GR is an approximately cubic three-dimensional space that extends in the lateral direction, the up-down direction and the front-rear direction. As shown in <FIG> and <FIG>, an upper end plane Fa1 of the gaze region GR is set to a size smaller than a lower end plane Fa2 of the gaze region GR, and the gaze region GR has a shape narrowed upward. The reason that the upper end plane Fa1 of the gaze region GR is set to the size smaller than the lower end plane Fa2 of the gaze region GR is to correspond to angles of view of the camera <NUM> and the distance measuring device <NUM>.

For example, a range in the lateral direction in the gaze region GR is set to include the die <NUM> mounted on the lower tool holder <NUM> and the punch <NUM> mounted on the upper tool holder <NUM>. Further, the vertical range in the gaze region GR is set to include the upper end side of the lower table <NUM> and the lower end side of the upper table <NUM>. The range in the vertical direction is based on the state when the upper table <NUM> is in the most raised position (a fully open position).

Further, the range in the front-rear direction in the gaze region GR is set such that predetermined distances are ensured at the front and at the back centering on the lower table <NUM> and the upper table <NUM>, respectively. The predetermined distance is determined in consideration of the length of the workpiece W in the front-rear direction, the distance from the lower table <NUM> to the back gauge <NUM>, and the like.

The gaze region GR set in this manner includes the gaze object that is an object to be gazed at within the work region. The gaze object is, for example, the punch <NUM>, the die <NUM>, the back gauge <NUM>, the workpiece W placed on the press brake main body <NUM>, and a hand and an arm of the operator M.

The gaze region GR may be set to a certain range and position regardless of a size of the workpiece W, a layout of the punch <NUM> and the die <NUM>, and a position of the abutting member <NUM>. However, the gaze region GR may be variably set in a range and a position in accordance with the layout of the punch <NUM> and the die <NUM> and the position of the abutting member <NUM>.

The NC (Numerical Control) device <NUM> is a control device that controls the press brake main body <NUM>. The NC device <NUM> drives the pair of hydraulic cylinders <NUM> up and down to control the vertical movement of the upper table <NUM>. The NC device <NUM> controls the vertical position of the upper table <NUM> based on the position information detected by a position detection unit.

The camera <NUM> is an image capturing device that captures an image of the work region centering on the gaze region GR and outputs the captured image ID. The camera <NUM> is attached to the table cover of the upper table <NUM> and is arranged behind the upper table <NUM>. The camera <NUM> captures an image of the gaze region GR and a surrounding region thereof from above the gaze region GR. The camera <NUM> attached to the table cover does not move up and down even when the upper table <NUM> moves up and down, and maintains the same position.

When the operator M works, the operator M stands in front of the press brake main body <NUM> so as to face the press brake main body <NUM>. Since the line of sight of the operator M is obstructed by the upper table <NUM>, the punch <NUM>, the lower table <NUM>, the die <NUM>, and the like, visibility in the work region behind the upper table <NUM> and the lower table <NUM> is reduced. In the work region behind the upper table <NUM> and the lower table <NUM>, there are back sides of the punch <NUM> and the die <NUM>, a rear region of the workpiece W abutted against the back gauge <NUM>, and the like. By capturing the image of the work region from above and behind the upper table <NUM> with the camera <NUM>, the work region at the rear in which the visibility of the operator M is low can be covered with the imaging range of the camera <NUM>.

Here, in the situation in which the camera <NUM> captures the image of the work region, it is sufficient that the gaze region GR is included in the image capturing range, that is, the angle of view of the camera <NUM>. Therefore, when observed from the camera <NUM>, it may be a situation in which a part of the gaze region GR is obstructed by a structure of the press brake main body <NUM> such as the upper table <NUM>, the upper tool holder <NUM>, the punch <NUM>, the lower table <NUM>, the lower tool holder <NUM>, or the die <NUM>.

The camera <NUM> includes an image capturing element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). A wide-angle lens or a fish-eye lens may be attached to the camera <NUM> so as to be able to capture a wide range. The camera <NUM> captures the image of the work region in response to a control signal from the image processing device <NUM>, and acquires the captured image ID. The camera <NUM> outputs the acquired captured image ID to the image processing device <NUM>.

As shown in <FIG>, the distance measuring device <NUM> is a distance measuring sensor that detects a distance to an object included in an observation range and generates distance data DD in which the object and the distance are associated with each other. The distance measuring device <NUM> includes, for example, a beam projector such as an LED that projects a beam to the observation range, and an image capturing element. The distance measuring device <NUM> measures, for each pixel, a TOF (Time Of Flight) that is the time from when the beam projector emits the beam to when the beam is reflected by the object and received by the image capturing element. The distance measuring device <NUM> detects the distance to the object for each pixel based on the measurement result for each pixel. The distance data DD generated by the distance measuring device <NUM> corresponds to a two-dimensional distribution of the distance in the observation range.

As shown in <FIG>, the distance measuring device <NUM> is attached to the table cover of the upper table <NUM> in the same manner as the camera <NUM>, and is arranged behind the upper table <NUM>. The distance measuring device <NUM> is arranged at nearly the same position as the camera <NUM>, and the observation range of the distance measuring device <NUM> also nearly coincides with the imaging pick-up range of the camera <NUM>. As a result, the distance measuring device <NUM> detects, from above the gaze region GR, the distance to the object that exists in the gaze region GR and the surrounding region thereof. In other words, the distance measuring device <NUM> detects the distance to the object present in the captured image ID of the camera <NUM>, and generates the distance data DD.

In the same manner as the camera <NUM>, the distance measuring device <NUM> acquires the distance data DD in response to the control signal from the image processing device <NUM>, and outputs the acquired distance data DD to the image processing device <NUM>.

The resolution (the number of pixels) of the distance data DD generated by the distance measuring device <NUM> is the same as the resolution (the number of pixels) of the captured image ID output from the camera <NUM>. However, the resolution (the number of pixels) of the distance data DD may be different from the resolution (the number of pixels) of the captured image ID.

In <FIG>, the image processing device <NUM> displays the captured image ID, which is captured by the camera <NUM>, on the display device <NUM>. The image processing device <NUM> does not display the captured image ID, which captures the situation in which the press brake main body <NUM> carries out the bending process, as it is, but displays the gaze image CD obtained by cutting out the gaze object from the captured image ID.

The image processing device <NUM> outputs the control signals to the camera <NUM> and the distance measuring device <NUM> at a predetermined cycle, so as to periodically acquire the captured image ID from the camera <NUM> and periodically acquire the distance data DD from the image processing device <NUM>. The control signals output from the image processing device <NUM> to the camera <NUM> and the distance measuring device <NUM> are synchronized with each other. Therefore, the image capturing timing by the camera <NUM> and the distance measuring timing by the distance measuring device <NUM> are synchronized with each other.

The image processing device <NUM> is composed of a microcomputer that is mainly composed of a memory such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), as well as an I/O (Input/Output) interface. The CPU of the image processing device <NUM> reads, from the ROM or the like, various programs and data in accordance with the processing contents, expands them in the RAM, and executes the expanded various programs. Thereby, the microcomputer functions as a plurality of information processing circuits provided to the image processing device <NUM>. In the present embodiment, an example of realizing the plurality of information processing circuits provided to the image processing device <NUM> by way of software is shown, but dedicated hardware for executing the respective information processing circuits may be prepared.

The image processing device <NUM> includes a storage unit <NUM>, an object identification unit <NUM>, an image cutout unit <NUM>, and an image output unit <NUM> as the plurality of information processing circuits.

The storage unit <NUM> stores definition information that defines the range of the gaze region GR. The definition information defines the range of the gaze region GR in accordance with the distance from the distance measuring device <NUM>. As shown in <FIG> and <FIG>, an upper end plane Fa1 of the gaze region GR is a horizontal plane corresponding to the upper end of the gaze region GR. When the vertical direction is used as a reference, the distance between the distance measuring device <NUM> and the upper end plane Fa1 is a distance D1 at any point on the upper end plane Fa1. On the other hand, a lower end plane Fa2 of the gaze region GR is a horizontal plane corresponding to the lower end of the gaze region GR. When the vertical direction is used as a reference, the distance between the distance measuring device <NUM> and the lower end plane Fa2 is a distance D2 at any point on the lower end plane Fa2.

<FIG> is an explanatory diagram showing distances from the distance measuring device to the upper end plane and the lower end plane of the gaze region. A reference plate having the same size as the upper end plane Fa1 is arranged on the upper end plane Fa1, and the distance to the reference plate is measured by the distance measuring device <NUM>. The distance measuring device <NUM> receives a reflected beam reflected by an observation point OBij on the reference plate for each pixel constituting the distance data DD, and measures a linear distance Dij between the observation point OBij and the distance measuring device <NUM>. By carrying out such measurement, the distance measuring device <NUM> measures a distance distribution of the upper end plane Fa1. The distance distribution of the upper end plane Fa1 has a unique distance Dij for each point of the upper end plane Fa1, that is, for each observation point OBij. The same applies to a distance distribution of the lower end plane Fa2.

<FIG> is an explanatory diagram showing the distance distribution of the upper end plane and the distance distribution of the lower end plane. Information stored in the storage unit <NUM> includes the distance distribution of the upper end plane Fa1 and the distance distribution of the lower end plane Fa2. As shown in (a) of <FIG>, the distance distribution of the upper end plane Fa1 is information indicating a two-dimensional distance distribution when the upper end plane Fa1 of the gaze region GR is observed from the distance measuring device <NUM>, and the distance Dij is associated with each pixel. As shown in (b) of <FIG>, the distance distribution of the lower end plane Fa2 is information indicating a two-dimensional distance distribution when the lower end plane Fa2 of the gaze region GR is observed from the distance measuring device <NUM>, and the distance Dij is associated with each pixel.

<FIG> is an explanatory diagram of the definition information that defines the gaze region. From the information stored in the storage unit <NUM>, the distance to the upper end plane Fa1 of the gaze region GR and the distance to the lower end plane Fa2 of the gaze region GR can be recognized, respectively. Therefore, as shown in <FIG>, a range equal to or greater than the distance to the upper end plane Fa1 of the gaze region GR and equal to or smaller than the distance to the lower end plane Fa2 of the gaze region GR functions as the definition information indicating the range of the gaze region GR.

Note that except for the method in which the reference plates are respectively arranged on the upper end plane Fa1 and the lower end plane Fa2 so as to actually measure the distances to the reference plates, other methods may also be used to acquire the distance distribution of the upper end plane Fa1 and the distance distribution of the lower end plane Fa2. For example, the distance distribution of the upper end plane Fa1 and the distance distribution of the lower end plane Fa2 may be obtained by way of geometric calculations in consideration of the positional relationship between the gaze region GR and the distance measuring device <NUM>.

The object identification unit <NUM> identifies the gaze object based on the definition information and the distance data DD that is output from the distance measuring device <NUM>. The object identification unit <NUM> identifies, as the gaze object, an object to which the distance Dij is within the gaze region GR, from among the objects present in the captured image ID.

The image cutout unit <NUM> generates the gaze image CD obtained by cutting out the gaze object from the captured image ID. The gaze image CD generated by the image cutout unit <NUM> is output to the image output unit <NUM>.

The image output unit <NUM> outputs the gaze image CD to the display device <NUM>. Further, the image output unit <NUM> may output the gaze image CD to the image storage device <NUM>. The image storage device <NUM> can store a predetermined volume of gaze images CD. A user including the operator M can use the gaze image CD stored in the image storage device <NUM> via a computer.

The display device <NUM> includes a display panel <NUM> for displaying information. The display device <NUM> is arranged at a position visible to the operator M who operates the foot switch <NUM>. The gaze image CD output from the image output unit <NUM> is displayed on the display panel <NUM> of the display device <NUM>. Note that the display device <NUM> may display either the gaze image CD that is output from the image output unit <NUM> or the captured image ID, which is switched in accordance with the operation of the operator M. Further, on the display panel <NUM> of the display device <NUM>, information that is output from the NC device <NUM> or the like can be displayed in addition to the gaze image CD.

<FIG> is a flowchart showing an operation of the press brake according to the first embodiment. An image output method according to the present embodiment will be described. The processing shown in the present flowchart is executed by the image processing device <NUM> at a predetermined cycle.

In step S10, the image cutout unit <NUM> outputs the control signal to the camera <NUM> and acquires the captured image ID from the camera <NUM>.

In step S11, the object identification unit <NUM> outputs the control signal to the distance measuring device <NUM> and acquires the distance data DD from the distance measuring device <NUM>.

For convenience of description, the processing of step S10 and the processing of step S11 are separated, but it is desirable to execute the processing of step S10 and the processing of step S11 such that the capturing timing of the captured image ID and the distance measuring timing for the object are synchronized with each other.

<FIG> is an explanatory diagram showing a concept of alignment between the captured image and the distance data. The distance data DD that is output from the distance measuring device <NUM> is pre-processed in advance so as to correspond to the captured image ID. When the positions of the camera <NUM> and the distance measuring device <NUM> are offset, as shown in <FIG>, the position (the coordinates) of a pixel PAij of the captured image ID and the position (the coordinates) of the pixel PBij of the distance data DD, which correspond to the same object, may be out of alignment. The object identification unit <NUM> utilizes a publicly known image processing technique such as coordinate conversion so that the distance data DD is corrected. As a result, the coordinates of the pixel PBij of the distance data DD are corrected to be the coordinates of a pixel PCij corresponding to the coordinates of the pixel PAij of the captured image ID.

The correction process of the distance data DD is carried out not only to a specific pixel but also to all of the pixels constituting the distance data DD. As a result, the coordinates of the distance data DD and the coordinates of the captured image ID are matched for the same object. Therefore, the distance data DD is data in which the distance is associated with each pixel PCij corresponding to the captured image ID, in other words, is equivalent to data in which the object reflected in the pixel PCij and the distance are associated with each other. The correction process of the distance data DD does not have to be carried out by the distance measuring device <NUM>, and may be executed by the object identification unit <NUM> that has acquired the distance data DD.

In step S12, the object identification unit <NUM> extracts the target pixel PCij to be processed from among the pixels PCij that constitute the distance data DD. When the processing of step S12 is executed for the first time, the object identification unit <NUM> extracts, as the target pixel PCij, the pixel PCij that is set in advance such as the pixel PCij located at the upper left in the distance data DD. On the other hand, when the processing of step S12 is executed for the second time onwards, the target pixel PCij is extracted according to processing coordinate information updated in step S17 that will be described later.

In step S13, the object identification unit <NUM> refers to the definition information stored in the storage unit <NUM>, and identifies the range of the gaze region GR corresponding to the coordinates of the target pixel PCij.

In step S14, the object identification unit <NUM> determines whether or not the distance Dij of the target pixel PCij is within the gaze region GR. Specifically, the object identification unit <NUM> determines whether or not the distance Dij of the target pixel PCij is equal to or greater than the distance to the upper end plane Fa1 of the gaze region GR and equal to or smaller than the distance to the lower end plane Fa2 of the gaze region GR. When the distance of the target pixel PCij is within the gaze region GR, an affirmative determination is made in step S14 and the process proceeds to step S15. On the other hand, when the distance of the target pixel PCij is outside the gaze region GR, a negative determination is made in step S14 to skip the process of step S15, and the process proceeds to step S16.

In step S15, the object identification unit <NUM> identifies the target pixel PCij as the gaze object. The object identification unit <NUM> outputs coordinate information of the target pixel PCij to the image cutout unit <NUM> as the coordinate information of the gaze object.

In step S16, the object identification unit <NUM> determines whether or not the processing is completed. If not all the pixels PCij constituting the distance data DD are extracted as the target pixels PCij, the object identification unit <NUM> determines that the processing is not completed. In this case, since a negative determination is made in step S16, the process proceeds to step S17. On the other hand, if all of the pixels PCij constituting the distance data DD are extracted as the target pixels PCij, the object identification unit <NUM> determines that the processing is completed. In this case, since an affirmative determination is made in step S16, the process proceeds to step S18.

In step S17, the object identification unit <NUM> updates the processing coordinate information. The processing coordinate information is information that identifies the pixel PCij to be extracted as the target pixel PCij in the processing of step S12. The object identification unit <NUM> updates the processing coordinate information so that the pixel PCij that has not yet been extracted in the distance data DD becomes the target pixel PCij.

<FIG> is an explanatory diagram showing the captured image and the gaze image that is generated from the captured image by a cutout process. In step S18, as shown in <FIG>, the image cutout unit <NUM> carries out a process of cutting out a target to be gazed at from the captured image ID, and generates the cutout image as the gaze image CD. The image cutout unit <NUM> refers to the coordinate information of the gaze object identified by the object identification unit <NUM>, and extracts the pixel PAij corresponding to the information of the gaze object from the captured images ID. The image cutout unit <NUM> generates an image composed of the extracted pixels PAij as the gaze image CD. The image cutout unit <NUM> outputs the gaze image CD to the image output unit <NUM>.

In step S19, the image output unit <NUM> outputs the gaze image CD to the display device <NUM>. The display device <NUM> to which the gaze image CD is input displays the gaze image CD on the display panel <NUM> thereof.

As described above, the press brake according to the present embodiment includes the image processing device <NUM> that generates the gaze image CD obtained by cutting out the gaze object, which is the object to be gazed at within the work region, from the captured image ID based on the distance data DD, and outputs the gaze image CD to the target device usable by the user.

According to the present configuration, by generating the distance data DD, it is possible to recognize the distance to the object present in the captured image ID. Since the distance data DD is the data in which the object and the distance are associated with each other, by referring to the distance data DD, the image processing device <NUM> can separate the gaze object that exists in the gaze region GR and the object that exists outside the gaze region GR, from among the objects captured with the camera <NUM>. This makes it possible for the image processing device <NUM> to generate the gaze image CD obtained by cutting out the gaze object from the captured image ID. Then, when the image processing device <NUM> outputs the gaze image CD to the target device, the user can use the gaze image CD via the target device. Since the gaze image CD is the image obtained by cutting out the gaze object, the gaze object can be visually recognized more easily as compared with the case
in which the captured image ID is visually recognized as it is. As a result, when the image is observed, the gaze object can be visually recognized in an efficient manner.

In the present embodiment, the target device is the display device <NUM> that is visually recognized by the operator M of the press brake main body <NUM>. According to the present configuration, by using the display device <NUM>, the operator M can confirm the gaze image CD in real time. Then, when the image is observed, the gaze object can be visually recognized in an efficient manner.

However, the target device may be the image storage device <NUM> usable, via the computer, by the user such as the operator M. According to the present configuration, by using the image storage device <NUM> via the computer, the user can confirm the gaze image CD at a necessary timing. Then, when the image is observed, the gaze object can be visually recognized in an efficient manner.

In the present embodiment, the gaze region GR is the three-dimensional space that is set between the lower end side of the upper table <NUM> and the upper end side of the lower table <NUM>.

According to the present configuration, the object involved in the bending process is included in the gaze region GR. This makes it possible for the image processing device <NUM> to appropriately generate the gaze image CD in which the object involved in the bending process is cut out. Since the gaze image CD is the image obtained by cutting out the gaze object, the gaze object can be visually recognized more easily as compared with the case in which the captured image ID is visually recognized as it is. As a result, when the image is observed, the gaze object can be visually recognized in an efficient manner.

In the present embodiment, the gaze region GR is set to include the punch <NUM>, the die <NUM>, the back gauge <NUM> against which the workpiece W is abutted, and the workpiece W.

According to the present configuration, the image processing device <NUM> can appropriately generate the gaze image CD obtained by cutting out the gaze objects such as the punch <NUM>, the die <NUM>, the back gauge <NUM>, and the workpiece W. Since the gaze image CD is the image obtained by cutting out the gaze object, the gaze object can be visually recognized more easily as compared with the case in which the captured image ID is visually recognized as it is. As a result, when the image is observed, the gaze object can be visually recognized in an efficient manner.

The gaze region GR may be set to include the hand and the arm of the operator M that hold the workpiece W. According to the present configuration, the operator M can easily grasp the positional relationship between the body of the operator M himself/herself and the workpiece W by realizing the hand and arm displayed on the display device <NUM>. This makes it possible to improve workability.

The distance data DD is the data in which the distance is associated with each of the plurality of pixels PCij corresponding to the captured image ID. According to the present configuration, in the distance data DD, the distance is associated with each of the pixels PCij. The image processing device <NUM> can cut out the gaze object in a unit of the pixel by cutting out the captured image ID based on the distance data DD. This makes it possible for the image processing device <NUM> to cut out the gaze image CD corresponding to the gaze object by a simple process.

Note that in the present embodiment, the distance data DD is the data in which the distance is associated with each of the pixels PCij. However, the distance data DD may be data in which the distance is associated not only with a single pixel PCij but also with each pixel block composed of a plurality of adjacent pixels PCij. Further, the distance data DD may be data in which the distance is associated with each pixel block grouped for each object recognized by an image processing technique. In addition, the distance data DD does not have to correspond to all of the pixels PAij constituting the captured image ID. The distance data DD may be data in which the distance is associated only with a specific pixel PAij that is selected from all of the pixels PAij constituting the captured image ID.

In the present embodiment, the image processing device <NUM> identifies the gaze object among the objects present in the captured image ID based on the definition information and the distance data DD. According to the present configuration, the image processing device <NUM> can recognize, by referring to the definition information, the range of the gaze region GR by way of the distance from the distance measuring device <NUM>. Therefore, the image processing device <NUM> can identify the gaze object by comparing the distance to the object in consideration of the definition information and the distance data DD. This makes it possible to appropriately identify the gaze object.

In the present embodiment, the definition information includes the distance distribution of the upper end plane Fa1 of the gaze region GR and the distance distribution of the lower end plane Fa2 of the gaze region GR. According to the present configuration, the image processing device <NUM> can recognize, as the gaze object, an object that has a distance equal to or greater than the distance recognized from the distance distribution of the upper end plane Fa1 and equal to or smaller than the distance recognized from the distance distribution of the lower end plane Fa2. This makes it possible for the image processing device <NUM> to appropriately identify the gaze object.

<FIG> is an explanatory diagram showing another form of the gaze image. Note that in the present embodiment, the object existing in the gaze region GR is recognized as the gaze object and the gaze image CD corresponding to the gaze object is generated. However, the image cutout unit <NUM> may generate an image obtained by cutting out only a more characteristic part from the gaze image CD. For example, as shown in <FIG>, the image cutout unit <NUM> may generate cutout images CD1 and CD2 in which attention is paid to an abutting position at which the workpiece W is abutted against the abutting member <NUM> of the back gauge <NUM>, and a cutout image CD3 in which attention is paid to a contact position of the punch <NUM> with respect to the workpiece W. In this case, the image cutout unit <NUM> may hold information of the coordinates corresponding to the positions to which attention should be paid, so as to further cut out an image from the gaze image CD corresponding to the gaze object.

<FIG> is a side view schematically showing an overall configuration of a press brake according to a second embodiment. A press brake according to the second embodiment will be described. The press brake according to the second embodiment includes a camera <NUM> and a distance measuring device <NUM> arranged in front of the upper table <NUM>, in addition to the camera <NUM> and the distance measuring device <NUM> arranged behind the upper table <NUM>. Hereinafter, the camera <NUM> and the distance measuring device <NUM> arranged behind the upper table <NUM> are referred to as the first camera <NUM> and the first distance measuring device <NUM>, and the camera <NUM> and the distance measuring device <NUM> arranged in front of the upper table <NUM> are referred to as the second camera <NUM> and the second distance measuring device <NUM>.

The first camera <NUM> and the first distance measuring device <NUM> are arranged behind the upper table <NUM>. For this reason, in the gaze region GR, objects located in front of the lower table <NUM> and the upper table <NUM> may be blocked by the structures such as the upper table <NUM>, the punch <NUM>, the lower table <NUM>, and the die <NUM> and may not be captured by the first camera <NUM> and the first distance measuring device <NUM>. Further, depending on the angles of view of the first camera <NUM> and the first distance measuring device <NUM>, it may not be possible to cover the entire area of the gaze region GR. Therefore, the second camera <NUM> and the second distance measuring device <NUM> are arranged in front of the upper table <NUM> to cover the entire area of the gaze region GR.

<FIG> is an explanatory diagram showing the distance distribution of the upper end plane and the distance distribution of the lower end plane that correspond to the first distance measuring device, the distance distribution of the upper end plane and the distance distribution of the lower end plane that correspond to the second distance measuring device, and the definition information that defines the gaze region. As shown in (a) of <FIG>, by detecting the upper end plane Fa1 of the gaze region GR with the first distance measuring device <NUM>, the distance distribution of the upper end plane Fa1 corresponding to the observation range of the first distance measuring device <NUM> is acquired. As shown in (b) of <FIG>, by detecting the lower end plane Fa2 of the gaze region GR with the first distance measuring device <NUM>, the distance distribution of the lower end plane Fa2 corresponding to the observation range of the first distance measuring device <NUM> is generated. Then, as shown in (c) of <FIG>, first definition information, which is a range of the gaze region GR corresponding to the observation range of the first distance measuring device <NUM>, is defined from the two distance distributions.

In a similar manner, as shown in Figure (d) of <FIG>, by detecting the upper end plane Fa1 of the gaze region GR with the second distance measuring device <NUM>, the distance distribution of the upper end plane Fa1 corresponding to the observation range of the second distance measuring device <NUM> is acquired. As shown in (e) of <FIG>, by detecting the lower end plane Fa2 of the gaze region GR with the first distance measuring device <NUM>, the distance distribution of the lower end plane Fa2 corresponding to the observation range of the second distance measuring device <NUM> is generated. Then, as shown (f) of <FIG>, second definition information, which is a range of the gaze region GR corresponding to the observation range of the first distance measuring device <NUM>, is defined from the two distance distributions.

In the press brake having such a configuration, the object identification unit <NUM> of the image processing device <NUM> identifies the gaze object existing in the gaze region GR based on the first and second definition information and the distance data DD of the first distance measuring device <NUM> and the second distance measuring device <NUM>. The image cutout unit <NUM> of the image processing device <NUM> generates the gaze image CD obtained by cutting out the gaze object from the captured image ID.

A method for the process of identifying the gaze object and the process of generating the gaze image CD includes, for example, a method shown below. That is, the method is the one in which the process using the first camera <NUM> and the first distance measuring device <NUM> and the process using the second camera <NUM> and the second distance measuring device <NUM> are individually carried out, and the gaze image CDs cut out from the respective processes are combined at the end.

Specifically, the object identification unit <NUM> identifies the gaze object based on the first definition information and the distance data DD of the first distance measuring device <NUM>. Then, the image cutout unit <NUM> generates the gaze image CD that is obtained by cutting out the gaze object, which identified by the first distance measuring device <NUM>, from the captured image ID of the first camera <NUM>. In the same manner, the object identification unit <NUM> identifies the gaze object based on the second definition information and the distance data DD of the second distance measuring device <NUM>. Then, the image cutout unit <NUM> generates the gaze image CD that is obtained by cutting out the gaze object, which is identified by the second distance measuring device <NUM>, from the captured image ID of the second camera <NUM>. Finally, the image cutout unit <NUM> combines the gaze image CD cut out from the captured image ID of the first camera <NUM> and the gaze image CD cut out from the captured image ID of the second camera <NUM> so that the images overlap with each other in the regions in which the image capturing ranges of the first camera <NUM> and the second camera <NUM> overlap with each other. As a result, the image cutout unit <NUM> generates the gaze image CD.

According to the present configuration, by using the plurality of cameras <NUM> and <NUM> and the plurality of distance measuring devices <NUM> and <NUM> in combination, it is possible to efficiently cover the entire area of the gaze region GR.

<FIG> is an explanatory diagram showing a correction concept of the definition information. <FIG> is an explanatory diagram showing the correction concept of the definition information. The press brake according to a third embodiment will be described. In the press brake according to the third embodiment, the image processing device <NUM> corrects the range of the gaze region GR in accordance with an operation of the press brake main body <NUM>.

A configuration is considered in which the distance measuring device <NUM> is attached to a table cover <NUM> of the upper table <NUM> as shown in <FIG>. In this case, the distance measuring device <NUM> attached to the table cover <NUM> maintains the same position without moving up and down even when the upper table <NUM> moves up and down.

When the upper table <NUM> moves up and down, the region to be gazed at substantially changes. This is because the gaze region GR only need to include the gaze objects such as the punch <NUM>, the die <NUM>, the back gauge <NUM>, the workpiece W, and the hand of the operator M. Therefore, the object identification unit <NUM> corrects the definition information so as to correct the range of the gaze region GR.

When the distance between the distance measuring device <NUM> and the lower end side of the upper table <NUM> is "D1", the distance distribution of the upper end plane Fa1 held by the storage unit <NUM> is set based on the distance D1 when the upper table <NUM> is in a fully open position. Therefore, the object identification unit <NUM> corrects the distance distribution of the upper end plane Fa1, which is read from the storage unit <NUM>, in accordance with the vertical movement of the upper table <NUM>. Specifically, the object identification unit <NUM> corrects the distance distribution of the upper end plane Fa1 as the distance D1 changes in accordance with an amount of movement of the upper table <NUM> when the upper table <NUM> moves up and down, that is, the upper end plane Fa1 moves up and down.

Next, a configuration is considered in which the distance measuring device <NUM> is attached to the upper table <NUM> as shown in <FIG>. In this case, the distance measuring device <NUM> attached to the upper table <NUM> moves up and down in accordance with the vertical movement of the upper table <NUM>.

When the distance measuring device <NUM> moves up and down, the region to be gazed at substantially changes. This is because the gaze region GR only need to include the gaze objects such as the punch <NUM>, the die <NUM>, the back gauge <NUM>, the workpiece W, and the hand of the operator M. Therefore, the object identification unit <NUM> corrects the definition information so as to correct the range of the gaze region GR.

When the distance between the distance measuring device <NUM> and the upper end side of the lower table <NUM> is "D2", the distance distribution of the lower end plane Fa2 held by the storage unit <NUM> is set based on the distance D2 when the upper table <NUM> is in a fully open position. Therefore, the object identification unit <NUM> corrects the distance distribution of the lower end plane Fa2, which is read from the storage unit <NUM>, in accordance with the vertical movement of the distance measuring device <NUM>. Specifically, the object identification unit <NUM> corrects the distance distribution of the lower end plane Fa2 as the distance D2 changes in accordance with an amount of movement of the distance measuring device <NUM> when the distance measuring device <NUM> moves up and down, that is, the lower end plane Fa2 moves up and down.

In this manner, in the present embodiment, the image processing device <NUM> can correct the distance distribution of the lower end plane Fa2 in accordance with the vertical movement of the upper table <NUM>. According to the present configuration, the image processing device <NUM> corrects the distance distribution of the lower end plane Fa2 in accordance with the vertical movement of the distance measuring device <NUM> that is interlocked with the upper table <NUM>. This makes it possible to optimize the range of the gaze region GR in accordance with the vertical movement of the distance measuring device <NUM>. As a result, only the necessary gaze object can be cut out as the gaze image.

Further, in the present embodiment, the image processing device <NUM> can correct the distance distribution of the upper end plane Fa1 in accordance with the vertical movement of the upper table <NUM>. According to the present configuration, the image processing device <NUM> corrects the distance distribution of the upper end plane Fa1 in accordance with the vertical movement of the upper table <NUM>. This makes it possible to optimize the range of the gaze region GR in accordance with the vertical movement of the upper table <NUM>. As a result, only the necessary gaze object can be cut out as the gaze image.

<FIG> is a block diagram showing a configuration of a control system of the press brake in which the captured image and the distance data are structured as one unit of a sensor. Note that in the embodiment described above, the camera <NUM> and the distance measuring device <NUM> are configured as separate devices. As shown in <FIG>, the camera <NUM> and the distance measuring device <NUM> may also be configured as one unit of a sensor <NUM>.

For example, the sensor <NUM> includes an image capturing element 52a and a beam projector 52b. The sensor <NUM> includes an image output unit 52c that receives a luminance signal indicating an intensity of a reflected beam received by the image capturing element 52a and outputs the image capturing ID having the luminance indicating the intensity of the reflected signal based on the luminance signal. Further, the sensor <NUM> measures a delay time of a beam receiving timing with respect to the beam receiving timing for each pixel based on the luminance signal, and generates the distance data DD indicating the distance to the object.

According to the present configuration, since the camera <NUM> and the distance measuring device <NUM> can be integrated, the device can be simplified.

As described above, the embodiments of the present invention have been described, but the statements and drawings that form a part of the present disclosure should not be understood to limit the present invention. The present disclosure will reveal, to those skilled in the art, various alternative embodiments, examples, and operational techniques.

For example, the distance measuring device is not limited to the configuration using the image capturing element, and may have a configuration in which a two-dimensional distance distribution is generated by using a laser radar, an ultrasonic sensor, or the like. Further, the press brake main body <NUM> is configured such that the operator M places the workpiece W, but the workpiece W may be placed by a transfer robot.

Claim 1:
A press brake, comprising:
a press brake main body (<NUM>) provided with an upper table (<NUM>) configured to hold an upper tool (<NUM>) and a lower table (<NUM>) configured to hold a lower tool (<NUM>), the press brake main body (<NUM>) being configured to carry out a bending process to a plate-shaped workpiece when the upper table (<NUM>) moves up and down relative to the lower table (<NUM>);
an image capturing device (<NUM>) configured to capture an image of a work region in which the upper table (<NUM>) and the lower table (<NUM>) carry out the bending process in the press brake main body (<NUM>), and output a captured image (ID); and
a distance measuring device (<NUM>) configured to detect a distance to an object present in the captured image (ID) and generate distance data (DD) in which the object and the distance are associated with each other
characterized by
an image processing device (<NUM>) configured to generate, based on the distance data (DD), a gaze image (CD) obtained by cutting out, from the captured image (ID), a gaze object that is an object to be gazed at within the work region, and output the gaze image (CD) to a target device usable by a user.