Source: https://patents.google.com/patent/CN104057453B/en
Timestamp: 2020-01-19 19:08:27
Document Index: 691157654

Matched Legal Cases: ['art; 11', 'art; 13', 'art; 16', 'art) 6', 'art 9', 'art 12', 'art 9', 'art 14', 'art 14', 'art 14', 'art 14', 'art 12', 'art 12', 'art 12', 'art 12', 'art 12', 'art 14', 'art 12', 'art 12', 'art 12', 'art 12', 'art 9', 'art 9', 'art 9', 'art 9', 'art 9', 'art 14', 'art 12', 'art 9', 'art 9', 'art 9', 'art 9', 'art 9', 'art 12', 'art 9']

CN104057453B - The manufacture method of robot device and machined object - Google Patents
The manufacture method of robot device and machined object Download PDF
CN104057453B
CN104057453B CN201410040153.9A CN201410040153A CN104057453B CN 104057453 B CN104057453 B CN 104057453B CN 201410040153 A CN201410040153 A CN 201410040153A CN 104057453 B CN104057453 B CN 104057453B
CN201410040153.9A
CN104057453A (en
关山友之
神谷阳介
2013-03-18 Priority to JP2013-055238 priority
2014-01-27 Application filed by 株式会社安川电机 filed Critical 株式会社安川电机
2014-09-24 Publication of CN104057453A publication Critical patent/CN104057453A/en
2016-03-23 Publication of CN104057453B publication Critical patent/CN104057453B/en
The invention provides the manufacture method of robot device and machined object, this robot device can simplify the teaching operation to robot.Robot device (1) is the device for carrying out action teaching to robot (R2), and comprises: make robot (R2) do the robot controlling portion (11) of action based on operation program; Obtain the robot shoot part (12) of view data; Data processing division (13), it has Virtual Space data retention portion (18) and at least utilizes view data and Virtual Space data and generate the augmented reality spatial data generating unit (23) of augmented reality spatial data; And the display part (9) of the image of display augmented reality space (AR).Virtual Space packet contains: the information of the dummy object (VB) after being simulated in Virtual Space (VS) by the object existed in the actual job space (RS) of robot (R2).
The manufacture method of robot device and machined object
The present invention relates to the robot device for carrying out action teaching to robot and utilize the manufacture method of machined object of robot, described robot carries out action teaching by robot device.
There will be a known and a kind ofly the view data by Practical computer teaching to be synthesized in the photographed data shot by camera and to carry out that show, so-called augmented reality (AR) technology.Patent document 1 has disclosed the robot device utilizing AR technology.
Patent document 1: International Publication No. 2011/080882 publication
In addition, technical staff can carry out action teaching to robot.But, when implementing the teaching operation of robot in the operating environment in reality, robot may with robot around object interfere.Therefore, the situation of the teaching operation be difficult to carry out according to actual act can be there is.
Therefore, the object of the invention is, a kind of robot device that can simplify teaching operation to robot is provided.
The present invention is the robot device for carrying out action teaching to robot, described robot device comprises robot controlling portion, robot shoot part, data processing division and display part, described robot controlling portion has the program maintaining part kept the operation program of regulation robot motion, and makes robot do action based on operation program; Described robot shoot part is for obtaining the view data comprising robot; Described data processing division has: keep the Virtual Space data retention portion of Virtual Space data and at least utilize view data and Virtual Space data and generate the augmented reality spatial data generating unit of augmented reality spatial data; Described display part utilizes augmented reality spatial data to show the image in augmented reality space, and Virtual Space packet contains: the information of the dummy object after being simulated in Virtual Space by the object existed in the actual job space of robot.
According to the present invention, the teaching operation to robot can be simplified.
Fig. 1 represents the figure using robot device to carry out the situation of the teaching operation of robot.
Fig. 2 is the figure for illustration of one of robot example.
Fig. 3 is the block diagram of the structure for illustration of robot device.
Fig. 4 is the figure for illustration of Virtual Space data.
Fig. 5 is the figure for illustration of the first track data and the second track data.
Fig. 6 is the figure for illustration of the computer making robot device run.
Fig. 7 is the figure of the master operation for illustration of robot teaching method.
Fig. 8 is the figure of the master operation for illustration of robot teaching method.
Fig. 9 is the figure of one of image representing augmented reality space example.
Figure 10 is the figure of one of image representing augmented reality space example.
Figure 11 is the figure of one of image representing augmented reality space example.
Figure 12 is the figure of one of image representing augmented reality space example.
Figure 13 is the figure of one of image representing augmented reality space example.
Figure 14 is the figure of one of image representing augmented reality space example.
Figure 15 is the figure of one of image representing augmented reality space example.
Figure 16 calculates for illustration of in order to generate track data the figure sitting calibration method.
Figure 17 is the figure representing one of image of being taken by robot filming apparatus example.
Figure 18 calculates for illustration of in order to generate track data the figure sitting calibration method.
Figure 19 calculates for illustration of in order to generate track data the figure sitting calibration method.
Figure 20 is for illustration of the figure about the method for discrimination of interfering.
1... robot device; 6a ~ 6d... camera; 7... input unit; 8... technical staff; 9... display part; 11... robot controlling portion; 12... robot shoot part; 13... data processing division; 14... program maintaining part; 16... correction program portion; 17... position and attitude data generating section; 18... Virtual Space data retention portion; 19... the first track data generating unit; 21... the second track data generating unit; 22... interference data generating unit; 23... augmented reality spatial data generating unit; 24... data correction portion; A1, A2... can not interference regions; AR... augmented reality space; L1... the first track; L2... the second track; R2, V2... robot; R4a, R4b... operation post; RS... actual job space; C... robot coordinate system; VS... Virtual Space.
Referring to accompanying drawing, explain for implementing embodiments of the present invention.In the description of the drawings, the Reference numeral same to same element annotation, and the repetitive description thereof will be omitted.
Be described using the environment of robot device 1.Fig. 1 represents the figure using robot device 1 to carry out the situation of the teaching operation (teaching) of robot R2.As shown in Figure 1, actual job space RS is the space that robot R2 carries out regulation operation, such as, be configured with robot R2 and operation post R4a, R4b.
The inside of actual job space RS is taken by multiple camera (shoot part) 6a ~ 6d.On robot R2, be connected with robot device 1.Robot device 1 has: the function of the action of control R2 and carry out the function of teaching operation of robot R2.On robot device 1, be connected with the input unit 7 for inputting regulated procedure or data etc. to robot device 1.The robot device 1 of present embodiment is operated by technical staff 8.Technical staff 8 detects by an unaided eye the image of the such display part 9 of head mounted display that robot device 1 has, and input device 7 carries out the teaching of robot R2 simultaneously.
Fig. 2 is the figure for illustration of one of robot R2 example.As shown in Figure 2, robot R2 is the robot of the joint type with six degree of freedom.The end side of robot R2 is fixed on ground 3, and its another side is provided with robot 2d.Position in each several part of robot R2 and the anglec of rotation, with robot coordinate system C for reference coordinate represents.Robot coordinate system C is using the direction vertical relative to the ground 3 being configured with robot R2 as Z-direction, using the direction parallel with ground 3 as X-direction.Further, using the direction (with paper vertical direction) mutually orthogonal with Z-direction with X-direction as Y-direction.In addition, the initial point of robot coordinate system C is such as robot R2 is fixed on the point on ground 3 as fixing point P, and using the initial point of fixing point P as robot coordinate system C.
Robot R2 has multiple connecting rod becoming arm configuration body.Connecting rod K1 is fixed in the ground 3 being provided with robot R2.Connecting rod K2 can rotate around the rotating shaft A1 vertical relative to ground 3 thus be connected on connecting rod K1.Connecting rod K3 can rotate around the rotating shaft A2 vertical relative to rotating shaft A1 thus be connected on connecting rod K2.Connecting rod K4 can rotate around the rotating shaft A3 parallel relative to rotating shaft A2 thus be connected on connecting rod K3.Connecting rod K5 can rotate around the rotating shaft A4 vertical relative to rotating shaft A3 thus be connected on connecting rod K4.Connecting rod K6 can rotate around the rotating shaft A5 vertical relative to rotating shaft A4 thus be connected on connecting rod K5.Connecting rod K7 can rotate around the rotating shaft A6 vertical relative to rotating shaft A5 thus be connected on connecting rod K6.
In addition, so-called " parallel " and " vertically " are not only proper " parallel " and " vertically " here, and are comprise sensu lato meaning slightly devious with " parallel " and " vertical ".On each rotating shaft A1 ~ A6, be respectively arranged with servomotor (joint J1 ~ J6), each servomotor has: the angular transducer T1 ~ T6 detecting respective position of rotation.Each servomotor is connected on robot device 1, each servomotor based on robot device 1 control instruction and do action.
< robot device >
Control signal based on operation program is input to robot R2 thus makes robot R2 do action by robot device 1.And, based on the output valve of angular transducer T1 ~ T6 of each several part and the image of the robot R2 by camera 6a ~ 6d shooting that are configured in robot R2, generate the path of motion of the reality of robot R2.Technical staff 8 according to based on the difference between the path of motion of operation program and the path of motion of reality, use input unit 7 corrective action program with make robot R2 perform desired by action.
Fig. 3 is the block diagram of the structure for illustration of robot device 1.As shown in Figure 3, robot device 1 comprises: make robot R2 do the robot controlling portion 11 of action; Obtain the robot shoot part 12 of the view data comprising robot R2; Generate the data processing division 13 of augmented reality spatial data; And the display part 9 of the image in display augmented reality space.
< robot controlling portion >
Robot controlling portion 11 has: generate control signal and the function of driven machine people based on operation program.In addition, robot controlling portion 11 also has: the function carrying out corrective action program based on the data inputted from data processing division 13.Control signal is outputted to robot R2 by robot controlling portion 11, simultaneously from input unit 7 and data processing division 13 input signal.Robot controlling portion 11 has: the program maintaining part 14 keeping operation program; The correction program portion 16 of corrective action program; And generate the position and attitude data generating section 17 of position and attitude data.
Program maintaining part 14 has: the function kept the operation program of the action for specifying robot R2.Program maintaining part 14 is from input unit 7 input action program.In addition, operation program is revised by input unit 7 and correction program portion 16.
Correction program portion 16 has: the function carrying out corrective action program based on the information exported from data processing division 13.Correction program portion 16 is the data inputting regulation from data processing division 13.In addition, correction program portion 16 exports the data being used for corrective action program to program maintaining part 14.In addition, although the correction operation of operation program that correction program portion 16 implements for support technician 8, also can corrective action program independently.
Position and attitude data generating section 17 has: the function receiving the sensing data of the angular transducer T1 ~ T6 output had from robot R2.Position and attitude data generating section 17 is from angular transducer T1 ~ T6 input pickup data.In addition, position and attitude data generating section 17 is to data processing division 13 outgoing position attitude data.
< robot shoot part >
Robot shoot part 12 is made up of multiple camera 6a ~ 6d.Robot shoot part 12 has: obtain the function of view data and the function to data processing division 13 output image data.
As shown in Figure 1, robot shoot part 12 comprises: the camera 6a ~ 6d configured in the indoor setting actual job space RS.Camera 6a is for obtaining the image from the robot R2 in X-direction observation actual job space RS etc.; Camera 6b is for obtaining the image from the robot R2 in Z-direction observation actual job space RS etc.; Camera 6c is for obtaining the image from the robot R2 in Y direction observation actual job space RS etc.With robot coordinate system C for benchmark, these cameras 6a ~ 6c is fixed on respective position.The view data obtained by these cameras 6a ~ 6c is such as the image of the fixing sight line comprising the picture of robot R2 and the picture of operation post R4a, R4b.
In addition, robot shoot part 12 comprises: with robot coordinate system C for benchmark thus the camera 6d be configured on the Z axis of robot coordinate system C.The structure of this camera 6d is can zoom and panning, such as, this camera 6d can obtain follow the tracks of robot R2 robot 2d(with reference to Fig. 2) the image of movement.In addition, the position that camera 6d follows the tracks of is not limited to robot 2d, also can follow the tracks of moving thus obtaining image of other positions of robot R2.
< data processing division >
Data processing division 13 has: utilize the various data inputted from robot controlling portion 11 and robot shoot part 12 to generate the function of augmented reality spatial data.In addition, data processing division 13 also has: utilize augmented reality spatial data to revise the function of Virtual Space data.As shown in Figure 3, data processing division 13 has: the Virtual Space data retention portion 18 keeping Virtual Space data; Generate the first track data generating unit 19 of the first track data; Generate the second track data generating unit 21 of the second track data; Generate the interference data generating unit 22 of interference data; Generate the augmented reality spatial data generating unit 23 of augmented reality spatial data; And revise the data correction portion 24 of Virtual Space data.
< Virtual Space data retention portion >
Virtual Space data retention portion 18 has: the function keeping following Virtual Space data.Virtual Space data retention portion 18 inputs Virtual Space data from input unit 7.In addition, Virtual Space data retention portion 18 is the information inputted from data correction portion 24 for revising Virtual Space data.And Virtual Space data retention portion 18 exports Virtual Space data respectively to interference data generating unit 22 and augmented reality spatial data generating unit 23.
Fig. 4 is the figure for illustration of Virtual Space data.As shown in Figure 4, Virtual Space data comprise: the information relevant to the dummy object VB in the VS of Virtual Space.Virtual Space VS refers to, simulates the virtual space on the computer of actual job space RS.Dummy object VB simulates the shape of the object be present in the RS of actual job space or configuration.Be present in the object in the RS of actual job space, such as, have robot R2 and operation post R4a, R4b.And dummy object VB has virtual robot V2, virtual operation post V4a, V4b, and these dummy objects VB is set in the VS of Virtual Space.In addition, the position of these dummy objects VB and shape, although with robot coordinate system C for reference coordinate specifies, also can specify based on the coordinate system except robot coordinate system C.
< first track data generating unit >
First track data generating unit 19 has the function generating the first following track data.As shown in Figure 3, the first track data generating unit 19 is the program maintaining part 14 input action programs from robot controlling portion 11.In addition, the first track data generating unit 19 exports the first track data to augmented reality spatial data generating unit 23.Further, the first track data generating unit 19 exports the first track data to interference data generating unit 22.
< second track data generating unit >
Second track data generating unit 21 has the function generating the second following track data.Second track data generating unit 21 is from robot shoot part 12 input image data.Second track data generating unit 21 is from position and attitude data generating section 17 input position attitude data.In addition, the second track data generating unit 21 exports the second track data to interference data generating unit 22.Further, the second track data generating unit 21 exports the second track data to augmented reality spatial data generating unit 23.
Here, the first track data and the second track data are described.Fig. 5 is the figure for illustration of the first track data and the second track data.As shown in Figure 5, the first track data represents the first track L1 based on the control signal being imported into robot R2, might not represent the movement locus of the reality of robot R2.Therefore, the first track data generates based on operation program (with reference to Fig. 3) in the first track data generating unit 19.
And the second track data represents: the movement locus i.e. second track L2 of the reality of robot R2.Therefore, the second track data in the second track data generating unit 21, utilize at least one party in position and attitude data and view data to generate (with reference to Fig. 3).Here, when utilizing sensing data to generate the second track data, using the length from the angle-data of angular transducer T1 ~ T6 and each connecting rod K1 ~ K7 of formation robot R2 as variable, the matrix computations based on known positive kinematics can be utilized to obtain the second track data.
The method utilizing view data to generate track data is described in detail.The three-dimensional coordinate of 1 of robot 2d front end, such as, can obtain from two images fixed camera (robot shoot part 12).The method of the point of robot 2d is extracted out as using image, such as can to the round dot symbol of 1 of robot 2d front end mark and other part different colours, extracted out the point of robot 2d by color detection and the image procossing obtaining round dot symbol center, also in the front end of robot 2d, LED can be installed and the image procossing utilizing the threshold value of brightness to propose and extract the point of robot 2d out.In addition, if can senior image procossing be carried out, then can use following method: namely registered as threedimensional model by robot 2d in advance, and extract part consistent with this threedimensional model in the picture out.And, as long as extract the coordinate on two images out with the fixed cycle, and its Coordinate Conversion is become three-dimensional robot coordinate system C, just can generate track data.And when the attitude of computing machine staff 2d, at every turn three-dimensional measurement is carried out to the coordinate of 3 on robot 2d and after converting robot coordinate system C to, by calculating by the attitude of 3 planes formed, calculate the attitude of robot 2d thus.
As an example, Coordinate Conversion when carrying out three-dimensional measurement to using camera 6a and camera 6c, is described with reference to Figure 16 ~ Figure 19.Figure 16, Figure 18 ~ Figure 20 calculates for illustration of in order to generate track data the figure sitting calibration method.Figure 17 (a) be one of image representing camera 6a example figure, Figure 17 (b) be one of image representing camera 6c example figure.Be set to for the purpose of simplifying the description, the plane of delineation of camera 6a and the YZ plane parallel of robot coordinate system C.Equally, the plane of delineation of camera 6c and the XZ plane parallel of robot coordinate system C.In addition, when observing from robot coordinate system C, camera 6a is configured at coordinate ［ a x, a y, a zin, when robot coordinate system C observes, camera 6c is configured at coordinate ［ c x, c y, c zin.
First, by the image of camera 6a, obtain the some p of robot 2d when observing from image coordinate system C6a coordinate 6ap x, 6ap y(with reference to Figure 17 (a)), and convert value when observing from robot coordinate system C to.Conversion (numerical expression (1) with reference to following) between robot coordinate system C and image coordinate system C6a, uses homogeneous transform matrix ct 6a, described homogeneous transform matrix ct 6arepresent the position and attitude (numerical expression (2) with reference to following) of image coordinate system when observing from robot coordinate system C.In addition, " * " in numerical expression (1) represents uncertain value.
p X C p Y C p Z C 1 = T 6 a C &CenterDot; p x 6 a p y 6 a * 1 = * p x 6 a a y - p y 6 a + a z 1 . . . ( 1 )
Equally, by the image of camera 6c, obtain the some p of robot 2d when observing from image coordinate system C6c coordinate 6cp x, 6cp y(with reference to Figure 17 (b)), use homogeneous transform matrix ct 6c(numerical expression (4) with reference to following), converts the value (numerical expression (3) with reference to following) when observing from robot coordinate system C to.
p X C p Y C p Z C 1 = T 6 c C &CenterDot; p x 6 c p y 6 c * 1 = p x 6 c + c x * - p y + c z 6 c 1 . . . ( 3 )
Therefore, the coordinate of some p when observing from robot coordinate system C is represented by following numerical expression (5).
p X C p Y C p Z C = p x 6 a + c x p y 6 a + a y - p y 6 a + a z . . . ( 5 )
Next, the method for the attitude calculating robot 2d is described.First, by 3 on image procossing extraction robot 2d and as some P1, some P2, put P3(with reference to Figure 18), coordinate when using above-mentioned method migration to become to observe from robot coordinate system C to each P1 ~ P3.Secondly, use following numerical expression (6), the vector a(size calculated from a P1 towards a P2 direction is 1).In addition, use following numerical expression (7), calculate the vector b from a P1 towards a P3 '.
a = p 2 - p 1 | p 2 p 1 | = p 2 x - p 1 x p 2 y - p 1 y p 2 z - p 1 z ( p 2 x - p 1 x ) 2 + ( p 2 y - p 1 y ) 2 + ( p 2 z - p 1 z ) 2 . . . ( 6 )
b &prime; = p 3 - p 1 | p 3 p 1 | . . . ( 7 )
Here, due to vector a and vector b ' might not orthogonal (with reference to Figure 19 (a)), therefore calculate the component of the b ' orthogonal with vector a, and calculate size be 1 vector b (numerical expression (8) with reference to following) (with reference to Figure 19 (b)).
b = b &prime; - ( a &CenterDot; b &prime; ) a | b &prime; - ( a &CenterDot; b &prime; ) a | . . . ( 8 )
Then, from the vector product of vector a and vector b, vector C (numerical expression (9) with reference to following) is calculated
c＝a×b…(9)
When arranging these trivector a, vector b and vector C as follows, then calculate matrix ct h, described matrix ct hrepresent robot (instrument) the coordinate system H(reference Figure 18 of robot 2d when observing from robot coordinate system C) position and attitude (numerical expression (10) with reference to following).In addition, using the position of a P1 as robot 2d.
Further, obtain the coordinate of 3 with the fixed cycle, all perform above-mentioned calculating at every turn and obtain ct hand preserve, thus, except the position of robot 2d, can also track data be generated about attitude.
As mentioned above, utilize at least two visual point images exported from robot shoot part 12, second track data of the robot 2d of robot R2 can be generated.Further, these position and attitude data are combined and view data also can generate the second track data.Such as, the positional information of the robot 2d utilizing view data to obtain, can revise second track data of the robot 2d utilizing position and attitude data to obtain, thereby, it is possible to make the precision of the second track data improve.In addition, these first track datas and the second track data, although with robot coordinate system C for reference coordinate specifies, also can specify based on the coordinate system except robot coordinate system C.
< interference data generating unit >
Interference data generating unit 22 has the function generating following interference data.As shown in Figure 3, interference data generating unit 22 inputs the first track data from the first track data generating unit 19.In addition, interference data generating unit 22 inputs the second track data from the second track data generating unit 21.Further, interference data generating unit 22 inputs Virtual Space data from Virtual Space data retention portion 18.And interference data generating unit 22 exports interference data to augmented reality spatial data generating unit 23.
Interference data represents: relative to the interference state of the robot R2 of dummy object VB.Therefore, interference data utilize Virtual Space data, with the first track data or the second track data and generate, described Virtual Space data have the information of dummy object VB; Described first track data or the second track data are the information relevant to the action of the robot R2 of reality.Due to these Virtual Space data, the first track data and the second track data with share robot coordinate system C for reference coordinate, therefore, it is possible to confirm presence of interference.About interference state, first calculated by positive kinematics until the position of robot 2d and joint J1 ~ J6.Secondly, each position is converted to position when observing from object coordinates system (coordinate system that the object becoming the object of interference has), and judge whether in interference region.
Here, the method about confirming presence of interference is described in further detail.First, for the robot R2 shown in Fig. 2, the positive kinematics of robot is calculated and is described.The position and attitude of the first coordinate system (joint J1) when observing from robot coordinate system C ct 1, represented by following numerical expression (11).Here, θ 1 is the anglec of rotation of joint J1, L 1the length of connecting rod K1.
In addition, the position and attitude of the second coordinate system (joint J2) when observing from the first coordinate system (joint J1) 1t 2, represented by following numerical expression (12).
In addition, the position and attitude of the three-coordinate (joint J3) when observing from the second coordinate system (joint J2) 2t 3, represented by following numerical expression (13).
In addition, the position and attitude of the 4-coordinate system (joint J4) when observing from three-coordinate (joint J3) 3t 4, represented by following numerical expression (14).
In addition, the position and attitude of the Five Axis system (joint J5) when observing from 4-coordinate system (joint J4) 4t 5, represented by following numerical expression (15).
In addition, the position and attitude of the 6th coordinate system (joint J6) when observing from Five Axis system (joint J5) 5t 6, represented by following numerical expression (16).
In addition, robot coordinate system H(robot 2d when observing from the 6th coordinate system (joint J6)) position and attitude 6t h, represented by following numerical expression (17).
By using above-mentioned numerical expression (11) ~ (17), utilizing following Matrix Products, having obtained the position and attitude of robot coordinate system H when observing from robot coordinate system C ct h(numerical expression (18) with reference to following).
CT H＝ CT 1· 1T 2· 2T 3· 3T 4· 4T 5· 5T 6· 6T H…(18)
In addition, by using above-mentioned numerical expression (11) ~ (17), can also calculate until the position of midway.Such as, the position and attitude in the joint 5 when observing from robot coordinate system C ct 5, can be obtained by following numerical expression (19).
[numerical expression 19]
CT 5＝ CT 1· 1T 2· 2T 3· 3T 4· 4T 5…(19))
Further, such as, when to calculate the connecting rod K6(length that is connected with joint J6 joint J5 be L6) the coordinate of intermediate point M, can be obtained by following numerical expression (20).
Next, the method for discrimination of interfering is described.Figure 20 is for illustration of the figure about the method for discrimination of interfering.As shown in figure 20, interference region is defined as: as the scope of each change in coordinate axis direction of object coordinates system CA.In the present note, using the space of the inner side of object A as interference region.Therefore, interference region is the scope in the scope of each reference axis of object coordinates system CA, represented by following numerical expression (21) ~ (23).
Ixmin≤X≤Ixmax…(21)
Iymin≤Y≤Iymax…(22)
Izmin≤Z≤Izmax…(23)
The interference region whether the some P1 of regulation being present in object A judges.First, the coordinate of some P1 when observing from robot coordinate system C is calculated cp 1.Concrete some P1 is such as the position such as front end or elbow of the robot 2d of the robot R2 that will confirm presence of interference.Secondly, following numerical expression (24) is used, by coordinate cp 1convert coordinate when observing from the object coordinates system CA of object A to cAp 1.
[numerical expression 24] cAp1=( ct a) -1. cp 1(24)
Then, judge cAp 1each component (P1x, P1y, P1z) whether be present in interference region.If three conditions of following numerical expression (25) ~ (27) are all true, then because a P1 is present in the inner side of object A, be therefore judged as interfering with object A.
Ixmin≤Plx≤Ixmax…(25)
[numerical expression 26]
Iymin≤Ply≤Iymax…(26)
[numerical expression 27]
Izmin≤Plz≤Izmax…(27)
Here, can as required for the first track L1 and the second track L2 set can not interference region A1, A2(with reference to Figure 11 and Figure 13).First track L1 and the second track L2 is such as the track of the regulation point be set on the robot 2d of robot R2.On the other hand, due to the position of the robot R2 of reality can be there is around regulation point, even if therefore when the first track L1 and the second track L2 does not interfere itself, sometimes also can interfere in the robot R2 of reality.Therefore, when adding the outer shape etc. of robot R2, when object invades within the scope of this, as the region with the possibility interfered with the robot R2 of reality and set can not interference region A1, A2.In addition, using the region that sets for the first track L1 as can not interference region A1, and using the region that sets for the second track L2 as can not interference region A2.
According to the interference data utilizing the first track data and Virtual Space data to obtain, interference state when can confirm to make robot R2 to do action as specified in operation program.Namely the confirmation of operation program can, be carried out.On the other hand, according to the interference data utilizing the second track data and Virtual Space data to obtain, interference state when can confirm in fact to make robot R2 to do action.Namely the confirmation of the movement locus of the robot R2 of reality can, be carried out.
< augmented reality spatial data generating unit >
Augmented reality spatial data generating unit 23 has the function generating augmented reality spatial data.Augmented reality spatial data generating unit 23 inputs Virtual Space data from Virtual Space data retention portion 18.Augmented reality spatial data generating unit 23 inputs the first track data from the first track data generating unit 19.Augmented reality spatial data generating unit 23 inputs the second track data from the second track data generating unit 21.Augmented reality spatial data generating unit 23 is from robot shoot part 12 input image data.Augmented reality spatial data generating unit 23 inputs interference data from interference data generating unit 22.In addition, augmented reality spatial data generating unit 23 exports augmented reality spatial data respectively to display part 9, data correction portion 24 and correction program portion 16.
Augmented reality spatial data makes virtual robot V2 and dummy activity platform V4a, V4b overlapping with the image of the robot R2 that have taken reality, and utilize view data and Virtual Space data and generate.In this augmented reality spatial data, the first track L1 or the second track L2 can be made as required overlapping with the image of the robot R2 that have taken reality.In addition, robot R2 also can be made overlapping with the interference state of dummy activity platform V4a, V4b.
The position of the lime light in robot R2 is, by resolving at least two view data obtained from different viewpoints, obtains the robot coordinate system C as reference coordinate thus.And the data of the first track L1 overlapping with the image of robot R2, the second track L2 and interference state are for reference coordinate with robot coordinate system C.Therefore, it is possible to the virtual data making the first track L1, the second track L2 such are overlapping relative to the image of the robot R2 of reality.
< data correction portion >
Data correction portion 24 has: the function revising Virtual Space data based on augmented reality spatial data.Data correction portion 24 inputs augmented reality spatial data from augmented reality spatial data generating unit 23.In addition, data correction portion 24 exports data for revising Virtual Space data to Virtual Space data retention portion 18.
Data correction portion 24 is such as used to the correction of Virtual Space data.Such as, when making the virtual robot V2 of the simulation in the VS of Virtual Space and dummy activity platform V4a, V4b are overlapping with the robot R2 be configured in the RS of actual job space and operation post R4a, R4b, then there is the situation that the dummy object VB of the simulation in the VS of Virtual Space is inconsistent relative to the object in the RS of actual job space.The difference of dummy object VB relative to the object in the RS of actual job space is extracted in data correction portion 24 out, and makes the position of dummy object VB or shape close to the object in the RS of actual job space.In addition, as long as the correction of these Virtual Space data is implemented as required.In addition, when technical staff 8 revises Virtual Space data, also secondarily can use the correction of these Virtual Space data.
< display part >
Display part 9 has: the image in display augmented reality space also provides the function of information to technical staff 8.Display part 9 inputs augmented reality spatial data from augmented reality spatial data generating unit 23.This display part 9 can use known image display.Such as, head mounted display or display panels etc. can be used.
Fig. 6 is the figure for illustration of the computer making robot device 1 run.As shown in Figure 6, computer 100 is examples for the hardware of the robot device 1 forming present embodiment.Computer 100 has CPU, comprises and carries out the information processor such as the process of software or the personal computer of control.Computer 100 is made up of computer system, and it comprises the auxilary units such as the display parts such as the input units such as CPU101, main storage means and RAM102 and ROM103, keyboard, mouse and programmable device 7, display 9, hard disk 108 etc.Functional inscape shown in Fig. 3 is the computer software making the hardware such as CPU101, RAM102 read regulation, under the control of CPU101, make input unit 7, display part 9 does action, in RAM102 or auxilary unit 108, carry out reading and the write of data simultaneously.
< robot teaching operation >
Then, the robot teaching method employing robot device 1 is described.Fig. 7 and Fig. 8 is the figure of the master operation for illustration of robot teaching method.As shown in Figure 7, first, robot R2 and operation post R4a, R4b are configured in (operation S1) in the RS of actual job space (with reference to Fig. 1).Secondly, use input unit 7 that initial actuating program is input to program maintaining part 14(operation S2) (with reference to Fig. 3).Then, input unit 7 is used virtual reality space data to be input to Virtual Space data retention portion 18(operation S3) (with reference to Fig. 3).
Make before robot R2 does action, first to implement the correction of Virtual Space data.Fig. 9 is the figure of one of image representing augmented reality space AR example.Generate make the image of Virtual Space VS, with by the equitant augmented reality spatial data (operation S4) of the image of the actual job space RS of robot shoot part 12 gained, utilize augmented reality spatial data that the image of augmented reality space AR is presented at display part 9(operation S5).As shown in Figure 9, in display part 9, show Virtual Space data are comprised dummy object VB, with by the equitant image of image of camera 6c gained.
In the images, show operation post R4a, the R4b of the robot R2 of reality and reality, with virtual robot V2 and virtual operation post V4a, V4b.Because robot R2 and robot V2 can not misplace, therefore do not need to revise the data of the robot V2 in the data of Virtual Space.On the other hand, operation post V4a is different relative to the position of the X-direction of operation post R4a.Further, operation post V4b is different with shape relative to the position of the Z-direction of operation post R4b.Therefore, judge to need to revise virtual reality space data (operation S6: yes).Technical staff 8 can use input unit 7 to implement the correction of virtual reality space data.In addition, data processing division 13 can calculate correction based on the difference of pixel detection error bit and shape, thus revises Virtual Space data.When being judged as not needing revising relative to the position of operation post R4a, R4b and shape relative to the position of robot R2 and shape and operation post V4a, V4b robot V2, be transitioned into subsequent processing (operation S6: no).
As shown in Figure 8, after the correction of Virtual Space data terminates, remove the object (operation S8) except robot R2 be configured in the RS of actual job space.Then, in the RS of actual job space, only there is robot R2.Figure 10 is the figure of one of image representing augmented reality space AR example.As shown in Figure 10, the image of augmented reality space AR comprises robot R2, V2 and operation post V4a, V4b.Namely, in the image of augmented reality space AR, owing to eliminating operation post R4a, R4b of reality from the RS of actual job space, so do not comprise operation post R4a, R4b of reality.
Then, the interference detection of initial actuating program is implemented.When making the robot 2d of robot R2 move as the first track L1 by initial actuating procedure stipulation, be confirmed whether to interfere with virtual operation post V4a, V4b.In more detail, first, utilize view data, Virtual Space data and the first track data and generate augmented reality spatial data (operation S9).And, utilize augmented reality spatial data to demonstrate the image (operation S10) of augmented reality space AR.The image of augmented reality space AR comprises: the robot R2 based on view data, the robot V2 based on Virtual Space data and operation post V4a, V4b, the first track L1 and can not interference region A1.Figure 11 is the figure of one of image representing augmented reality space AR example.Figure 11 is the figure after amplifying a part of augmented reality space AR, and shows operation post V4a, V4b, the first track L1 and can not interference region A1.The first track L1 based on initial actuating program to reach home PE from initial point PS via impact point PO.Example according to Figure 11, the first track L1 has the part Eb interfered with operation post V4b.Further, the part Ea interfered with operation post V4b can not be had by interference region A1.Therefore, be judged as needing to revise initial actuating program (operation S11: yes).
Technical staff 8 uses input unit 7 corrective action program (operation S12).And, utilize corrected operation program, regeneration augmented reality spatial data (operation S9), the image of augmented reality space AR be presented at display part 9(operation S10).Figure 12 is the figure of one of image representing augmented reality space AR example.As shown in figure 12, corrected first track L1 has new intermediate point P1.Here, as change first track L1, also automatically changing can not interference region A1.This first track L1 can not interfere with operation post V4b, can not also can not interfere with operation post V4b by interference region A1.Therefore, be judged as not needing corrective action program, and be transitioned into subsequent processing (operation S11: no).
Implement interference detection when in fact making robot R2 do action.In more detail, first, use the operation program of generation first track L1 and make robot R2 do action (operation S13).And, generate augmented reality spatial data (operation S14), the image of augmented reality space AR be presented at display part 9(operation S15).Here, in operation S14, utilize view data, Virtual Space data, the first track data and the second track data and generate augmented reality spatial data.Figure 13 is the figure of one of image representing augmented reality space AR example.Figure 13 comprises operation post V4a, V4b, the first track L1, the second track L2 and can not interference region A2.In addition, this can not interference region A2 based on the second track L2.
Example according to Figure 13, the second track L2 has the part Ec interfered with operation post V4b.Further, the part Ed interfered with operation post V4b can not be had by interference region A2.
But, in the action of robot R2, the situation that dislocation occurs relative to the first track L1 based on operation program actual track i.e. the second track L2 can be there is.There is following situation in the action control of robot R2: using from primary importance to the time of next second place movement as top priority.In the present embodiment, the time moving to terminal PE from initial point PS is set as top priority.So, in the movement from initial point PS to terminal PE, even if complete movement at the appointed time, also can exist from initial point PS to the situation that the second track L2 of the reality of terminal PE departs from from the first track L1.Such as, example is as shown in Figure 13 known: the second track L2 does not arrive impact point PO.This phenomenon is called as inner ring phenomenon.Therefore, releasing second track L2 and can not interference region A2 to the interference of operation post V4b, and when strictly making it via impact point PO, need corrective action program (operation S16: yes).
Figure 14 is the figure of one of image representing augmented reality space AR example.In order to remove the second track L2 and can not interference region A2 to the interference of operation post V4b, the position of intermediate point P1 is revised (operation S17).And, in the mode of the second track L2 via impact point PO, set the position of intermediate point P2.The correction of these intermediate points P1, P2, can be implemented by technical staff 8, also can be implemented by correction program portion 16.In the correction of the intermediate point P1 implemented by correction program portion 16, calculate interfere with the region shared by operation post V4b can not the repeat length along each change in coordinate axis direction of coordinate system C of interference region A2, and make the amount of the position of intermediate point P2 only this length mobile.In addition, the setting of the intermediate point P2 implemented by correction program portion 16 is, calculate the second track L2 relative to impact point PO along each axial separating distance.And, based on separating distance, the position of intermediate point P1 is moved.
Then, utilize corrected operation program Zai Shi robot R2 to do action (operation S13), generate augmented reality spatial data (operation S14), the image of augmented reality space AR is presented at display part 9(operation S15).Figure 15 is the figure of one of image representing augmented reality space AR example.As shown in figure 15, according to corrected operation program, relieve the second track L2 and can not interference region A2 to the interference of operation post V4b, and the second track L2 is via impact point PO.Therefore, corrective action program (operation S16: no) is not needed.
By above operation S1 ~ S16, just complete the teaching operation of the robot R2 using robot device 1.
Robot device 1 according to the present embodiment, can obtain from robot shoot part 12 view data making the robot R2 of reality do action.In addition, robot device 1 has: the dummy object VB existed in the RS of actual job space is carried out in Virtual Space the Virtual Space data simulated.And, in augmented reality spatial data generating unit 23, utilize view data and Virtual Space data and generate augmented reality spatial data.According to which, in the Virtual Space being configured with dummy object VB, can carry out overlapping with the result making the robot R2 of reality do action.Thus, the teaching operation that can be implemented as follows: namely object such to operation post R4a, R4b of reality need not be configured in actual job space RS, with regard to Neng Shi robot R2 and do action.Therefore, the actual interference of robot R2 and periphery object can not be there is, just can implement the teaching operation of robot R2, so, can safely and easily implement teaching operation by the trial and error of operation program.
The robot device 1 of present embodiment generates the first track data and the second track data, and utilizes these data genaration augmented reality spatial datas, and these the first track L1 and the second track L2 are presented at display part 9.According to which, due to the first set track L1 can be confirmed by visually observing and makes robot R2 do the difference of result that is between the second track L2 of action, so easily and effectively can implement the correction of operation program.Therefore, it is possible to more easily carry out the teaching operation of robot R2.
Robot device 1 according to the present embodiment, can generate interference data, and described interference data represents the robot R2 of reality and the interference state of dummy object VB.Therefore, due to robot R2 and dummy object VB presence of interference can be confirmed by visually observing, so, can corrective action program in the mode that can not interfere with dummy object VB.Therefore, it is possible to easily carry out action teaching to robot R2, described action can not interfere with the periphery object of reality.
Above, the preferred embodiment of the present invention is illustrated, but the present invention might not be only limitted to above-mentioned embodiment, can various change be carried out in the scope not departing from its purport.Such as, robot R2 can be vertical tow-armed robot.In addition, the image of augmented reality space AR, except the image of the image of display device people R2, dummy object VB, the first track L1, the second track L2, can also show the various information being easy to carry out teaching operation.In addition, in the above-described embodiment, be illustrated for the situation of the track to robot R2 teaching two dimension, but robot device 1 also may be used for the situation of the track to robot R2 teaching three-dimensional.In addition, in the above-described embodiment, robot device 1 is used for the correction of Virtual Space data, the first track data takes industry really as and the second track data takes industry as really, but in order to implement an operation in these operations, also can use robot device 1.
In addition, can use and carry out the robot R2 after action teaching by above-mentioned robot device 1, manufacture desired product (machined object).
1. a robot device, for carrying out action teaching to robot, is characterized in that, described robot device comprises:
Robot controlling portion, it has the program maintaining part kept the operation program of the described robot motion of regulation, and makes described robot do action based on described operation program;
Robot shoot part, it obtains the view data comprising described robot;
Data processing division, it has and keeps the Virtual Space data retention portion of Virtual Space data and at least utilize described view data and described Virtual Space data and generate the augmented reality spatial data generating unit of augmented reality spatial data;
And display part, it utilizes described augmented reality spatial data to show the image in augmented reality space,
Described Virtual Space packet contains: the information of the dummy object after being simulated in described Virtual Space by the object existed in the actual job space of described robot,
Described data processing division has the first track data generating unit, and described first track data generating unit utilizes described operation program, generates the first track data represented based on the movement locus of the described robot of described operation program,
Described augmented reality spatial data generating unit also utilizes described first track data, generates described augmented reality spatial data.
2. robot device as claimed in claim 1, is characterized in that:
Described data processing division has the second track data generating unit, at least one party the sensing data that described second track data generating unit utilizes the sensor had from described robot to export and described view data, generating makes described robot do result i.e. second track data of action based on described operation program
Described augmented reality spatial data generating unit also utilizes described second track data, generates described augmented reality spatial data.
3. robot device as claimed in claim 2, is characterized in that:
Described data processing division has the interference data generating unit generating interference data, and described interference data represents the interference state of described robot relative to described dummy object,
Described interference data generating unit utilize in described first track data and described second track data at least one, with described Virtual Space data, generate described interference data,
Described augmented reality spatial data generating unit also utilizes described interference data, generates described augmented reality spatial data.
4. robot device as claimed any one in claims 1 to 3, is characterized in that:
Described data processing division has: the data correction portion revising described Virtual Space data.
5. robot device as claimed any one in claims 1 to 3, is characterized in that:
Described robot controlling portion has: the correction program portion revising described operation program.
6. robot device as claimed any one in claims 1 to 3, is characterized in that:
Described robot shoot part is set to, based on the robot coordinate system be set in described robot.
7. a manufacture method for machined object, is characterized in that:
By utilizing the robot device according to any one of claim 1 to 6 to carry out the described robot after action teaching, manufacture machined object.
CN201410040153.9A 2013-03-18 2014-01-27 The manufacture method of robot device and machined object CN104057453B (en)
JP2013-055238 2013-03-18
CN104057453A CN104057453A (en) 2014-09-24
CN104057453B true CN104057453B (en) 2016-03-23
JP5980873B2 (en) 2014-10-17 2016-08-31 ファナック株式会社 Robot interference area setting device
JP2016107379A (en) 2014-12-08 2016-06-20 ファナック株式会社 Robot system including augmented reality corresponding display
CN101085523A (en) * 2006-06-06 2007-12-12 发那科株式会社 Robot simulation apparatus
JP3606595B2 (en) * 1994-01-28 2005-01-05 三菱電機株式会社 Machine tool control device
JP4473849B2 (en) * 2003-06-02 2010-06-02 パナソニック株式会社 Article handling system and article handling server
JP5439062B2 (en) * 2009-07-03 2014-03-12 中村留精密工業株式会社 Method for preventing collision of machine tools
JP2014180707A (en) 2014-09-29
KR20040104578A (en) 2004-12-10 Camera corrector
JP2002229730A (en) 2002-08-16 Information processing unit, complex reality providing device and method thereof, and storage medium
Zou et al. 2012 Virtual manipulator-based binocular stereo vision positioning system and errors modelling
Termination date: 20170127