Package: ikflow
IKFlowNormalizing flows for Inverse Kinematics. Open source implementation to the paper"IKFlow: Generating Diverse Inverse Kinematics Solutions"Setup, inference onlypipinstallgit+https://github.com/jstmn/ikflow# orgitclonehttps://github.com/jstmn/ikflow.git&&pipinstallikflow/Setup - training, inference, and visualizationThe following section outlines the setup procedures required to run the visualizer that this project uses. The only supported OS is Ubuntu. Visualization may work on Mac and Windows, I haven't tried it though. For Ubuntu, there are different system wide dependencies forUbuntu > 21andUbuntu < 21. For example,qt5-defaultis not in the apt repository for Ubuntu 21.0+ so can't be installed. Seehttps://askubuntu.com/questions/1335184/qt5-default-not-in-ubuntu-21-04.Ubuntu >= 21.04sudo apt-get install -y qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools libosmesa6 build-essential qtcreator
export PYOPENGL_PLATFORM=osmesa # this needs to be run every time you run a visualization script in a new terminal - annoying, I knowUbuntu <= 20.x.y(This includes 20.04 LTS, 18.04 LTS, ...)sudo apt-get install -y qt5-default build-essential qtcreatorLastly, install with pip:git clone https://github.com/jstmn/ikflow.git && cd ikflow
pip install -e ".[dev]"Getting started> Example 1: Use IKFlow to generate IK solutions for the Franka Panda armEvaluate the pretrained IKFlow model for the Franka Panda arm. Note that this was the same model whose performance was presented in the IKFlow paper. Note that the value formodel_name- in this casepanda_tpmshould match an entry inmodel_descriptions.yamlpython scripts/evaluate.py --testset_size=500 --model_name=panda_full_tpm> Example 2: Visualize the solutions returned by thepanda_tpmmodelRun the following:python scripts/visualize.py --model_name=panda_full_tpm --demo_name=oscillate_targetRun an interactive notebook:jupyter notebook notebooks/robot_visualizations.ipynb> Example 3: Run IKFlow yourselfExample code for how to run IKFlow is provided inexamples/example.py. A sample excerpt:ik_solver,_=get_ik_solver(args.model_name)target_pose=np.array([0.5,0.5,0.5,1,0,0,0])number_of_solutions=5solutions,l2_errors,angular_errors,joint_limits_exceeded,self_colliding,runtime=ik_solver.solve(target_pose,number_of_solutions,refine_solutions=False,return_detailed=True)NotesThis project uses thew,x,y,zformat for quaternions. That is all.Training new modelsThe training code usesPytorch Lightningto setup and perform the training andWeights and Biases('wandb') to track training runs and experiments. WandB isn't required for training but it's what this project is designed around. Changing the code to use Tensorboard should be straightforward (so feel free to put in a pull request for this if you want it :)).First, create a dataset for the robot:python scripts/build_dataset.py --robot_name=panda --training_set_size=25000000 --only_non_self_collidingThen start a training run:# Login to wandb account - Only needs to be run once
wandb login

# Set wandb project name and entity
export WANDB_PROJECT=ikflow 
export WANDB_ENTITY=<your wandb entity name>

python scripts/train.py \
    --robot_name=panda \
    --nb_nodes=12 \
    --batch_size=128 \
    --learning_rate=0.0005Common errorsGLUT font retrieval function when running a visualizer. Runexport PYOPENGL_PLATFORM=osmesaand then try again. Seehttps://bytemeta.vip/repo/MPI-IS/mesh/issues/66Traceback (most recent call last):
  File "visualize.py", line 4, in <module>
    from ikflow.visualizations import _3dDemo
  File "/home/jstm/Projects/ikflow/utils/visualizations.py", line 10, in <module>
    from klampt import vis
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/klampt/vis/__init__.py", line 3, in <module>
    from .glprogram import *
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/klampt/vis/glprogram.py", line 11, in <module>
    from .glviewport import GLViewport
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/klampt/vis/glviewport.py", line 8, in <module>
    from . import gldraw
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/klampt/vis/gldraw.py", line 10, in <module>
    from OpenGL import GLUT
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/OpenGL/GLUT/__init__.py", line 5, in <module>
    from OpenGL.GLUT.fonts import *
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/OpenGL/GLUT/fonts.py", line 20, in <module>
    p = platform.getGLUTFontPointer( name )
  File "/home/jstm/Projects/ikflow/venv/lib/python3.8/site-packages/OpenGL/platform/baseplatform.py", line 350, in getGLUTFontPointer
    raise NotImplementedError( 
NotImplementedError: Platform does not define a GLUT font retrieval functionIf you get this error:tkinter.TclError: no display name and no $DISPLAY environment variable, add the lines below to the top ofik_solvers.py(anywhere beforeimport matplotlib.pyplot as pltshould work).importmatplotlibmatplotlib.use("Agg")TODOAdd CPU versions of pretrained modelAdd 'light' pretrained models. These are smaller networks for faster inferenceInclude a batched jacobian optimizer to enable parallelized solution refinement.Note: This is in the works in a seperate repositoryCitation@ARTICLE{9793576,
  author={Ames, Barrett and Morgan, Jeremy and Konidaris, George},
  journal={IEEE Robotics and Automation Letters}, 
  title={IKFlow: Generating Diverse Inverse Kinematics Solutions}, 
  year={2022},
  volume={7},
  number={3},
  pages={7177-7184},
  doi={10.1109/LRA.2022.3181374}
}