Apparatus simulating operations between a robot and workpiece models

A robot simulation apparatus (10) capable of creating and executing a robot program includes a virtual space creating unit (31) for creating a virtual space (60), a workpiece model layout unit (32) for automatically arranging at least one workpiece model (40) in an appropriate posture at an appropriate position in a workpiece accommodation unit model (24) defined in the virtual space, a virtual camera unit (33) for acquiring a virtual image (52) of workpiece models existing in the range of a designated visual field as viewed from a designated place in the virtual space, a correcting unit (34) for correcting the teaching points in the robot program based on the virtual image, and a simulation unit (35) for simulating the operation of the robot handling the workpieces, and as a result, interference between the robot and the workpieces can be predicted while at the same time accurately determining the required workpiece handling time.

This application also claims priority under 35 U.S.C. §119 and/or §365 to Japanese Application No. 2006-157557, filed on Jun. 6, 2006, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a simulation apparatus for a robot system including an imaging means and a robot.

2. Description of the Related Art

A simulation apparatus is known wherein three-dimensional models such as a robot, workpiece and peripheral devices, etc. are arranged and displayed on the screen of a computer, and based on designated robot move commands, etc., the robot motion is simulated. This simulation apparatus is used for creating a robot motion program off line, and evaluating, correcting or editing the program created off line.

However, in a place, such as an office where a robot is not actually arranged, the conventional simulation apparatus cannot substantially perform a simulation relating to the imaging means in the case of creating off line, correcting or editing a program.

Japanese Unexamined Patent Publication No. 2005-135278 discloses a simulation apparatus in which an imaging means is arranged on a three-dimensional model and the visual field of the imaging means is displayed on the three-dimensional model, and as a result, can perform the simulation relating to an imaging means.

There is a case in which a robot picks up a plurality of workpieces having the same shape stacked in bulk in a container. The plurality of workpieces are arranged irregularly three-dimensionally. At the time of picking up the workpiece, the robot hand may interfere with workpieces other than a workpiece to be picked up or the container's wall. The simulation apparatus disclosed in Japanese Unexamined Patent Publication No. 2005-135278 does not presuppose the picking up of workpieces in bulk, and therefore, interference between the robot hand and workpieces or the container cannot be predicted.

In picking up a plurality of workpieces piled in bulk from a container, the position of the robot hand in the handling operation is varied depending on the workpiece to be taken out, and therefore, time required to pick up all of the workpieces from the container cannot be accurately calculated.

This invention has been achieved in view of this situation, and the object thereof is to provide a simulation apparatus wherein the time required to pick up all of the workpieces from a container accommodating the workpieces stacked in bulk therein, is accurately determined while predicting interference between the robot and workpieces and between the robot and the container.

SUMMARY OF THE INVENTION

In order to achieve the above described object, according to a first aspect of the invention, there is provided a robot simulation apparatus capable of creating and executing a robot program, comprising a virtual space creating means for creating a virtual space for three-dimensionally expressing the working space in which the robot carries out the handling operation of workpieces, a workpiece model layout means for automatically arranging one or a plurality of models of the workpieces in an appropriate posture at appropriate positions in a workpiece layout area model defined in the virtual space created by the virtual space creating means or a workpiece accommodation means model arranged in the virtual space, a virtual camera means for acquiring a virtual image of the workpiece models existing in the range of a designated visual field designated as viewed from a designated place in the virtual space, a correcting means for correcting the teaching points of the robot program based on the virtual image acquired by the virtual camera means, and a simulation means for simulating the robot operation of handling the workpieces based on the program in which the teaching points are corrected by the correcting means.

Specifically, in the first aspect, a plurality of workpiece models are arranged in appropriate postures at appropriate positions, and therefore, the workpieces stacked in bulk in the container can be expressed on a virtual space. In the case where a plurality of workpieces in bulk are picked up, interference between the robot and the workpieces and between the robot and the container can be predicted, thereby making it possible to accurately determine the time required to pick up all of the workpieces.

According to a second aspect, there is provided a robot simulation apparatus in the first aspect, wherein the workpiece model layout means arranges a designated number of the workpiece models in the virtual space.

Specifically, in the second aspect, interference and required time can be more accurately predicted based on the number of the workpiece models arranged in the container model.

According to a third aspect, there is provided a robot simulation apparatus in the first or second aspect, wherein the workpiece model layout means arranges a plurality of the workpiece models in such a manner that the workpiece models does not share the same space.

Specifically, in the third aspect, the more accurate simulation is made possible by making sure that a plurality of workpiece models do not overlap one another in the three-dimensional virtual space.

According to a fourth aspect, there is provided a robot simulation apparatus in any one of the first to third aspects, further comprising a workpiece attribute information storage means for storing workpiece attribute information, wherein the workpiece model layout means calculates the distortion amount of each workpiece based on the workpiece attribute information stored in the workpiece attribute information storage means and the information of the contact portions of a plurality of the workpieces, and arranges the workpiece models in a deformed state based on the distortion amount.

According to a fifth aspect, there is provided a robot simulation apparatus in the fourth aspect, wherein the workpiece attribute information includes the stiffness, weight and center of gravity of the workpieces.

Specifically, in the fourth and fifth aspects, the distortion amount of the workpieces stacked in bulk is taken into account, and therefore, a more accurate simulation is possible.

According to a sixth aspect, there is provided a robot simulation apparatus in any one of the first to fifth aspects, further comprising a robot motion storage means for monitoring and storing the actual motion of the robot, wherein the workpiece model layout means, based on the actual motion of the robot stored in the robot motion storage means, calculates the position and posture of the robot hand at the time of grasping each of the plurality of the workpieces and arranges the workpiece models based on the hand position and posture thus calculated.

Specifically, in the sixth aspect, the handling operation for picking up a plurality of workpieces stacked in bulk in the container by the robot can be reproduced in a virtual space.

These and other objects, features and advantages of the present invention will be more apparent in light of the detailed description of exemplary embodiments thereof as illustrated by the drawings.

DETAILED DESCRIPTION OF THE INVENTION

An embodiment of the invention is explained below with reference to the accompanying drawings. In the drawings, the same component members are designated by the same reference numerals, respectively, and to facilitate understanding, the scale of the drawings has been appropriately changed.

FIG. 1is a diagram showing a general layout of a robot simulation system according to the invention. A robot simulation apparatus10shown inFIG. 1is a digital computer, for example, and can create and execute a robot program. According to the embodiment shown inFIG. 1, the robot simulation apparatus10is connected to a robot controller29via a LAN29, such as an Ethernet (registered trade mark) and may alternatively be connected by another method.

InFIG. 1, a cage-like container24is arranged at a predetermined position. As shown, a plurality of workpieces20having the same shape are placed in bulk in the container24. The container24has an outer wall25whereby an opening26is defined. The opening26, though rectangular inFIG. 1, may have another shape. Also, each of the plurality of the workpieces, though shown parallelepiped, may alternatively have another shape.

The robot controller21is connected to a robot22whose arm has a hand23at the forward end of the arm. The robot controller21controls the robot22to pick up workpieces20sequentially from the container24.

An imaging means55such as a CCD camera is arranged at a predetermined position above the container24. InFIG. 1, the imaging means55is positioned by a stand (not shown). The imaging means55may alternatively be mounted on a part of the robot22.

FIG. 2is a block diagram showing the robot simulation apparatus according to the invention. As shown inFIG. 2, the robot simulation apparatus10includes a CPU11, a storage unit having a ROM12and RAM13, an input and output device14having a mouse and/or keyboard and a graphics control circuit15, which are interconnected by a bidirectional bus16.

The CPU11functions as a virtual space creating means31for creating a virtual space for three-dimensionally expressing the working space in which the robot22carries out the handling operation of the workpieces20, a workpiece model layout means32for automatically arranging workpiece models of the workpieces20in the model of the container24in the virtual space, a virtual camera means33for acquiring a virtual image of the workpiece models in the virtual space, a correcting means34for correcting the teaching points in the robot program based on the virtual image and a simulation means35for simulating the operation of the robot22to handle the workpieces20based on the corrected program.

The RAM13of the storage unit has stored therein data on the three-dimensional geometric models for the robot22, container24, workpieces20and peripheral devices thereof intended for simulation. The RAM13also has stored therein workpiece attribute information such as the stiffness of workpieces20, weight of the workpieces20and center of gravity of each workpiece20.

Further, operation programs100to400, described later for the robot simulation apparatus10and the robot program for operating the robot22are stored in the ROM12of the storage unit. Also, as shown, the graphics control circuit15is connected to a display unit19, such as a liquid crystal monitor or CRT.

FIG. 3is a flowchart showing the operation program of the robot simulation apparatus according to the invention. The operation of the robot simulation apparatus10according to the invention is explained below with reference to the operation program shown inFIG. 3.

First, a three-dimensional virtual space60for carrying out the simulation is created by a virtual space creating means31(step101). The three-dimensional virtual space60thus created is displayed on a display unit19via the graphics control circuit15.

FIG. 4is a diagram showing the three-dimensional virtual space displayed on the display unit19. InFIG. 4, the three-dimensional virtual space60is displayed on a first screen51on the display unit19. As shown, a container model44corresponding to the container24shown inFIG. 1is displayed in the three-dimensional virtual space60on the first screen51. This container model44is created based on the three-dimensional geometric data stored in the storage unit, and an outer wall45and an opening46corresponding to the outer wall25and the opening26, respectively, are also displayed.

Although the three-dimensional virtual space60is plotted as a plane inFIG. 4, the viewpoints of the three-dimensional virtual space60can be three-dimensionally changed by use of an input/output device, such as a mouse.

Next, the workpiece models40corresponding to the workpieces20are arranged in the three-dimensional virtual space60by the workpiece model layout means32. The workpiece models40are also created based on the three-dimensional geometric data of the workpieces20stored in the storage unit.

As shown, a plurality of the workpiece models40are automatically arranged at appropriate positions in appropriate postures in the container model44by the workpiece model layout means32. The method of arranging the plurality of the workpiece models40is explained below.

FIG. 5is a diagram explaining the method of arranging the plurality of the workpiece models40, and to facilitate explanation,FIG. 5shows the models two-dimensionally. As indicated by step201of the program200inFIG. 5, first, one workpiece model40is arranged in an appropriate posture at an appropriate position above the container model44by the workpiece model layout means32. The posture and position of each workpiece model40is determined in the workpiece model layout means32using a random number or the like.

Then, in step202, the workpiece model40is moved down without changing the posture thereof, so that the workpiece model40reaches the bottom surface47of the container model44.

After that, the process is returned to step201, in which another workpiece model40is arranged above the container model44and then moved down toward the bottom surface47. In the process, another workpiece model40is arranged in such a manner so as to not be superposed on the workpiece model40already arranged in the container model44. As shown, another workpiece model40may not be in contact with the bottom surface47or the outer wall45of the container model44. This process is repeated a predetermined number of times thereby to arrange a plurality of the workpiece models40in the container model44(step203).

According to this invention, the posture and the position of each workpiece model40are determined at random using a random number or the like, and a plurality of the workpiece models40are arranged in a manner not to be superposed on one another. Therefore, the postures, etc. of the workpiece models40are very similar to the postures, etc. of the actual workpieces20stacked in bulk in the container24. According to this invention, the operation of picking up a plurality of workpieces having the same shape stacked in bulk in the container can be simulated with higher accuracy.

Then, as shown inFIG. 4, a camera model55′ is arranged at a place designated in advance in the three-dimensional virtual space60of the first screen51. This place corresponds to the place of the actual imaging means55(FIG. 1). In the three-dimensional virtual space60shown inFIG. 4, the camera model55′ is arranged just above the container model44.

InFIG. 4, the dashed lines extending from the camera model55′ indicate a visual field56′ of the camera model55′. This visual field56′ corresponds to the visual field of the actual imaging means55(FIG. 1). As shown, the visual field56′ of the camera model55′ covers a plurality of the workpiece models40arranged in the container model44.

The camera model55′ can acquire the virtual image of the workpiece models40in the visual field56′ through a virtual camera means33. The virtual camera means33displays the acquired virtual image as a second screen52on the display unit19(see step103inFIG. 3). As shown, the first screen51and the second screen52are plotted as different windows. Instead of using the second screen52, however, the virtual image covered by the camera model55′ may be plotted at a corner of the first screen51.

Next, based on the virtual image displayed on the second screen52, the teaching points in the robot program are corrected by the correcting means34. The correcting means34first selects an appropriate workpiece model40such as a workpiece model40afrom the virtual image on the second screen52and calculates the posture and position thereof. The robot program describes the method by which the hand23of the robot22grasps a workpiece20in a predetermined posture at a predetermined position. The teaching points in the robot program are thus changed based on the calculated posture and position of the workpiece model40aso that the hand23of the robot22can grasp the workpiece model40a.

After that, the correcting means34sequentially selects other workpiece models40b,40c, . . . and calculating the posture and position thereof similarly, changes the teaching points so that the hand23can grasp the workpiece models40b,40c, . . . (step104).

As can be seen fromFIG. 4, the workpiece model40dis located under the workpiece model40a, and therefore, only a part of the workpiece model40dis shown in the second screen52. InFIG. 4, the workpiece model40aexisting on this side constitutes an obstacle, and therefore, the posture and position of the workpiece model40dare difficult to calculate. In such a case, the workpiece model40aconstituting an obstacle is provisionally deleted to calculate the posture and position of the workpiece model40dat a lower position. In this way, the teaching points of the robot program are changed in such a manner that all the workpiece models40can be grasped.

Next, the robot model22′ of the robot22is displayed in the three-dimensional virtual space60on the first screen51. Based on the robot program corrected by the correcting means34, the simulation means35operates the robot model22′ by simulation on the three-dimensional virtual space60(step105). The simulation operation by the simulation means35may be performed on other screen from the first screen51.

By this simulation, the hand model23′ of the robot model22′ performs the operation of sequentially picking up the workpiece models40arranged in the container model44. As a result, it is possible to predict on the simulation screen whether the hand23will be interrupted by the outer wall25of the container24and other workpieces other than the workpiece20to be grasped.

In the case where the interrupting of the hand23with the outer wall25is comparatively light, the dimensions of the container model44may be changed on the three-dimensional virtual space60, and the simulation may be repeated until the interruption is avoided, and as a result, the optimum size of the container24can be studied using the simulation.

Similarly, in the case where interrupting of the hand23with other workpieces20is comparatively light, the hand model23′ of the optimum size can be studied similarly by changing the size of the hand model23′ on the three-dimensional virtual space60.

Further, according to this invention, the teaching points of the robot program are changed for each of the workpiece models40corresponding to a plurality of the workpieces20. Specifically, the posture of the hand23in the handling operation is successively changed in accordance with the specific workpiece20to be grasped. Therefore, according to this invention, the time required to pick up all of the plurality of the workpieces20from the container24can be accurately calculated in advance.

From this, it is possible to construct a picking up system with a robot to pick up a plurality of workpieces having the same shape stacked in bulk in a container, the use of the robot simulation apparatus according to this invention can shorten the startup time of the picking up system.

The number of the workpiece models40arranged in the three-dimensional virtual space60can of course be easily changed by use of an input/output device14, such as a mouse and/or keyboard. In other words, the simulation can be carried out more accurately by matching the number of the workpiece models40with the number of the actual workpieces20.

As shown on the left side ofFIG. 6a, this is a case in which one workpiece model40emay be arranged over two other workpiece models40f,40gin the container model44. In other words, the workpiece model40eis supported at a supporting point p on the workpiece model40fand a supporting point q on the workpiece model40g. In such a case, the workpiece model40eis distorted between the supporting points p and q, and therefore, it is preferable to display the workpiece model40ewhile taking the distortion amount thereof into account.

FIG. 6bis a diagram showing the method of displaying a workpiece model based on the distortion amount of a workpiece. First, the distance L between the supporting points p and q is calculated based on the relative positions of the workpiece models40e,40f,40g. Then, the maximum displacement δ is calculated from Equation (1) below (step301).
δ=P·L3/(48·E·I)  (1)
where P is the weight of the workpiece20, and E and I the modulus of elasticity and the geometrical moment of inertia, respectively. The weight P, the modulus of elasticity E and the geometrical moment of inertia I are stored in the ROM12of the storage unit in advance.

Next, the intermediate face M of the workpiece model40eextending in parallel to the line segment connecting the supporting points p and q is set. Then, the distance r to each corner of the workpiece model40efrom the intermediate surface M is determined. The workpiece model40e, if parallelopipedal as shown, has eight corners Qi (=Q1to Q8), and therefore, eight distances ri (=r1to r8) are determined (step302). It should be noted thatFIG. 6bshows only two upper corners Q1, Q2, two lower corners Q5, Q6and the distances r1, r2, r5, r6to them.

Then, the distortion amount εi (=ε1to ε8) for each of the eight corners of the workpiece model40eis calculated from Equation (2) below (step303).
εi=ri/ρ(2)
where ρ is the radius of curvature of the intermediate face.

After that, as shown on the right side ofFIG. 6a, all the corners Qi of the workpiece model40eon the three-dimensional virtual space60are moved in accordance with the calculated distortion amounts εi and the workpiece model40eis displayed again.

Although the workpiece model40eis shown two-dimensionally inFIGS. 6a,6b, the actual workpiece20is three-dimensional. The four corners Q1to Q4located above the intermediate face M, therefore, are compressed toward the center of the workpiece model40e. The remaining four corners Q5to Q8located under the intermediate face M, on the other hand, are extended outward away from the center.

The distortion amount is of course similarly calculated for other workpiece models40and the corners thereof moved. In such a case, it is apparent that the three-dimensional virtual space60obtained is higher in accuracy, and therefore, a more accurate simulation is possible. Especially, the embodiment shown inFIG. 6is advantageously applicable in the case where the workpieces20piled in the container24are liable to be deformed with comparative ease.

FIGS. 6a,6bexplained above assume that the workpiece model40eis substantially horizontal. Nevertheless, it is apparent that the distortion amount can be similarly calculated and the corners of a workpiece model moved by a similar method also in the case where the particular workpiece model is inclined by other workpiece models.

In picking up a plurality of workpieces20stacked in bulk in the container24as shown inFIG. 1, the posture and position of the hand23grasping the workpieces20undergo a change each time. After picking up of all of the workpieces20in the container24, the manner in which the workpieces20have so far been arranged in the container24cannot be confirmed, and as a result that the handling operation of the robot22cannot be reproduced.

In contrast, according to the invention, the handling operation of the robot22for picking up a plurality of the workpieces20piled in bulk in the container24can be reproduced.FIG. 7is a diagram explaining the method of reproducing the actual operation of the robot in virtual space, and is explained below with reference toFIG. 7.

First, the robot22shown inFIG. 1is actually operated thereby to successively pick up a plurality of workpieces20stacked in bulk in the container24. In the process, the robot simulation apparatus10monitors the motion of the robot22at every sampling period, and stores the result of monitoring in the RAM13(step401). Thus, the result of monitoring contains the posture and position of the hand23when grasping each of a plurality of the workpieces20and the timing of opening/closing of the hand23. The black circles inFIG. 7designate teaching points.

Next, based on the posture, etc. of the hand23thus retrieved, the posture and position of the workpieces20grasped by the hand23are estimated (step402). This estimation is carried out for all of the workpieces20picked up from the container24.

After the container model44is created in the three-dimensional virtual space60on the first screen51, a plurality of the workpiece models40are arranged in the container model44by the workpiece model layout means32based on the estimated postures, etc. of the plurality of the workpieces20(step403), and as a result, the plurality of the workpieces20stacked in bulk in the actual container24can be reproduced in the three-dimensional virtual space60.

After that, the robot model22′ (not shown inFIG. 7) is displayed in the three-dimensional virtual space60, and the simulation operation of the robot model22′ is carried out using the simulation means35(step404). In the robot simulation apparatus10according to this invention, the handling operation of the robot22for picking up the plurality of the workpieces20stacked in bulk in the container24can be reproduced. In the case where an error occurs during the picking up operation of the plurality of the workpieces20in the container24by the actual robot22, the cause of the error can be verified by the reproducing operation described above.

Although a plurality of the workpieces20are arranged in the container24according to the embodiment explained above with reference to the drawings, this invention is also applicable to a case in which the plurality of the workpieces20are arranged in a predetermined workpiece stock area on the floor.

Although the invention has been shown and described with exemplary embodiments thereof, it should be understood by those skilled in the art that the foregoing and various other changes, omissions and additions may be made therein and thereto without departing from the spirit and scope of the invention.