Where’s My Camera?
Chris Lewis
Jonathan Meyer
Southwest Research InstituteLibrary Original Goal
• Intrinsic and Extrinsic Calibration for Cameras
in industrial robot Installations,( no user input)
URDF

CamerasKillerROS
Application

Calibration

MoveIt!

Robot
Mutable JSWhy it was Awful?
• Virtually every flavor of intrinsic and extrinsic
calibration in one node
• Massive Effort to Configure– Launch File/URDF madness– Yaml File Madness• Define every scene– Triggers– What camera sees which targets– Cost function for each observation• Define list of cameras and targets– Camera triggers– Transform interfaces

Minimize Configuration• Divorce calibration from motion and triggers• Service Based Calibration

Stand Alone GUI

Dedicated Nodes• Intrinsic– Rail– On Robot• Camera on Wrist• Target on Wrist• Wrist– Camera on Wrist– Target on Wrist• Stereo– Cameras on Wrist– Target on Wrist

Load and Covariance• Load– Add more data to existing calibration– Run with Any Method for Data Collection– Adjust Grid Finding Parameters– Remove troublesome images• Custom Covariance matrix– Low Residual is no Guarantee– Enough Data
– Diverse DataGUI as an Rviz Plug inRobo-cylinder Calibration•2 Passes Necessary• Unknowns:• Pose to each Pass
• Axis of Motion• Intrinsics• Extruded grid
• Process forces
measured motion tomatch observed motion

On Robot I-Cal

Repeatability and Accuracy• Why do I get .25 per/pixel residual error, but
5-10 pixels variation in Cx, Cy, Fx and Fy?
• 1% σ in Fx corresponds to 1cm at 1meter• Camera accuracy– Circles found with .01 pixel accuracy– Position Repeatable to .0254mm at 1 meter• Is more data needed?
• Is the model redundant?

Covariance from Ceres• Focal length correlated to radial distortion
Fx,Fy  k1,k2,k3• Principal point correlated to de-centering
distortion Cx  p2 & Cy  p1
• k1  k2  k3

Ical w/data Everywhere• Residual is low, Covariance Still High

Covariance Results
Full Modelfx
0.017370.87709-0.01683
0.00282-0.80504
0.73464-0.65705
0.00479-0.01570fy
0.877090.017530.00157-0.00464-0.820910.75606-0.67664-0.00344
0.00295cx-0.01683
0.00157
0.02018-0.00058
0.00525-0.01022
0.01526-0.00060
0.92158cy
0.00282-0.00464-0.00058
0.01929-0.00174
0.00171-0.00166
0.88249-0.00052k1-0.80504-0.82091
0.00525-0.00174
0.00010-0.96252
0.90466-0.00536
0.00130cy-0.00433
0.000180.009100.01461-0.01172k1-0.69724
0.010220.014610.00004-0.93120k2
0.68697-0.00075-0.01172-0.93120
0.00001

Reduced Modelf0.01224-0.00063-0.00433-0.69724
0.68697cx-0.00063
0.007860.000180.01022-0.00075• Conclusion:– Probably shouldn’t use K3,P1&P2

k2
0.734640.75606-0.01022
0.00171-0.96252
0.00007-0.98257
0.00319-0.00781

k3-0.65705-0.67664
0.01526-0.00166
0.90466-0.98257
0.00002-0.00211
0.01221p1
0.00479-0.00344-0.00060
0.88249-0.00536
0.00319-0.00211
0.00002-0.00048p2-0.01570
0.00295
0.92158-0.00052
0.00130-0.00781
0.01221-0.00048
0.00002

Create and Execute aCalibration Using MoveIt!

C:\Users\clewis\roscon2018\wristTeachRunSave.mp4

Stereo Calibration

Camera on Wrist Calibration

Target on Wrist Calibration

Robot_Cal_Tools

Robot_Cal_Tools• Industrial_calibration works well but requires an ecosystem to support it• robot_cal_tools was made as a learning experience
with a focus on pure functions• Makes no assumptions about data collection or what
you do with the answer• Covers common calibrations– Intrinsic & extrinsic pinhole camera(s)– 3D (e.g. IFM) to 3D• Hopefully useful for integrating into custom systems or
as a basis for writing something custom

Robot Cal Tools (Contents)• Core dependency: only Ceres (and Eigen)• The API: struct Problem {// Specify camera data, correspondences};
struct Result {// Shows average cost, other convergence data};
Result optimize(const Problem& params);• Provides some tools for using OpenCV targetfinders• Emphasis on documentation

Where Next• Contributions welcome!
• https://github.com/Jmeyer1292/robot_cal_tools• https://github.com/ros-industrial/industrial_calibration.git– (look for branch with 500 commits)• WE ARE HIRING• Future plans:– Integrate robot_cal_tools into industrial_calibration– Provide experiments for robot kinematic calibration– Provide experiments for laser profile scanners– LiDAR?
– Documentation with examples for Service based CAL