thirumalaivasank/gx-knotix

GXKNOTIX — End-to-End NVIDIA Physical AI Flower-Knotting Workcell

A high-performance robotics automation system built on the complete NVIDIA Physical AI stack, combining NVIDIA Omniverse, Isaac Lab 3.0, Newton Physics, and NVIDIA TensorRT for sim-to-real robotic flower knotting.

• Interactive Web Demo: https://huggingface.co/spaces/thirumalaivasank/gx-knotix-demo
• Model & Policy Repository: https://huggingface.co/thirumalaivasank/gx-knotix

---

Architecture: Complete NVIDIA Physical AI Pipeline

 ┌────────────────────────────────────────────────────────┐
 │       1. SIMULATION & TRAINING (Isaac Lab + Newton)    │
 │  • Multiphysics: Franka arms + Newton Cosserat rod     │
 │  • Domain Randomization: thread elasticity, friction   │
 │  • Environment: simulation/gxknotix_knotting_rl_env.py │
 └───────────────────────────┬────────────────────────────┘
                             │
                             ▼ Export & Optimize
 ┌────────────────────────────────────────────────────────┐
 │       2. MODEL EXPORT & COMPILATION (TensorRT)         │
 │  • ONNX Export: simulation/export_policy_tensorrt.py   │
 │  • Output: gxknotix_policy.onnx -> policy.engine (FP16)│
 └───────────────────────────┬────────────────────────────┘
                             │
                             ▼ Deploy to Edge Hardware
 ┌────────────────────────────────────────────────────────┐
 │       3. REAL-WORLD EDGE EXECUTION (TensorRT / ROS)    │
 │  • Hardware Bus: deployment/hardware_interface.py      │
 │  • 60 Hz Closed-Loop: deployment/nvidia_physical_ai_runner.py │
 └────────────────────────────────────────────────────────┘

---

Traditional Knotting Process (14-Step Canonical Guide)

The workcell kinematics faithfully execute the 14-step opposite-direction jasmine knotting technique (reference: docs/final_guide.png):

1. Thread Preparation: Thread held under 2-3 N tension between tensioner and indexer.
2. Bud Pick: LeftPicker & RightPicker pick fresh Kundumalli buds from dedicated feed trays.
3. Opposite-Direction Nesting: Bud A placed tip-up (+Y), Bud B inverted 180° tip-down (-Y). Stems touching with zero gap; thread passes over the center.
4. Wrap: Spinner rotor carries thread around both stems together.
5. Loop: Billhook mechanism opens jaw, forming a locking loop around the stems.
6. Pull Through & Cinch: Jaw closes on thread, pulls through loop, tensioner cinches tight.
7. Slide & Index: Pitch indexer advances the garland +9 mm, accumulating a dense double-sided floral rope.

---

Quick Start: One-Click Launchers

You can start and stop the entire workcell directly using the provided Windows batch scripts:

• Launch: Double-click start.bat or run:

  start.bat
  

  (Presents a quick menu to launch with Omniverse Kit visualizer, headless verification, or edge runner).

• Stop: Double-click stop.bat or run:

  stop.bat
  

  (Terminates all running Omniverse Kit, Isaac Lab, and policy runner background processes).

---

1. Newton Physics Digital Twin Simulation

Run the photorealistic 3D workcell in Omniverse Kit:

Set-Location "C:\Users\thiru\gx-workspace\gx-knotix\IsaacLab"
uv run isaaclab -p ..\simulation\gxknotix_knotting_workcell.py --viz kit

Headless / verification mode:

uv run isaaclab -p ..\simulation\gxknotix_knotting_workcell.py --num_steps 100

---

2. Policy Export to NVIDIA TensorRT

Export the trained PyTorch policy to ONNX and compile to TensorRT:

Set-Location "C:\Users\thiru\gx-workspace\gx-knotix\IsaacLab"
uv run python ..\simulation\export_policy_tensorrt.py --output_onnx ..\simulation\gxknotix_policy.onnx

On your edge target (Jetson Orin or RTX edge GPU), build the FP16 TensorRT engine:

trtexec --onnx=gxknotix_policy.onnx --saveEngine=gxknotix_policy.engine --fp16

---

3. Real-World Edge Deployment

Execute the sub-2ms TensorRT policy runner driving the physical actuators at 60 Hz:

Set-Location "C:\Users\thiru\gx-workspace\gx-knotix"
python deployment\nvidia_physical_ai_runner.py --model simulation\gxknotix_policy.onnx --hz 60.0

---

Directory Structure

gx-knotix/
├── start.bat                                   # One-click interactive workcell launcher
├── stop.bat                                    # One-click process terminator
├── assets/
│   └── thread_spool.usd                        # Procedural OpenUSD thread spool asset
├── simulation/
│   ├── gxknotix_knotting_workcell.py           # Interactive Newton multiphysics workcell
│   ├── gxknotix_knotting_rl_env.py             # Isaac Lab RL task environment & MDP
│   ├── export_policy_tensorrt.py               # ONNX & TensorRT policy compiler
│   ├── gxknotix_policy.onnx                    # Exported neural policy
│   └── README.md
├── deployment/
│   ├── nvidia_physical_ai_runner.py            # Closed-loop TensorRT policy runner (60 Hz)
│   └── hardware_interface.py                   # Real-time hardware bus interface
├── docs/
│   └── final_guide.png                         # 14-step canonical opposite-direction knotting guide
├── .gitignore
├── README.md
└── LICENSE