Add files using upload-large-folder tool

README.md

@@ -0,0 +1,43 @@
+# VLAarchTests Bench
+
+This repository is a bundled benchmark workspace for future VLAarchTests model
+iterations, with emphasis on RLBench2 `take tray out of oven`.
+
+## Included
+
+- VLAarchTests benchmark code and generated public benchmark manifest
+- Patched AnyBimanual RLBench runtime used to execute the public oven benchmark
+- Official `katefgroup/3d_flowmatch_actor` PerAct2 checkpoint and public test data
+- Public AnyBimanual LF baseline weights and comparison logs
+- Verified benchmark reports:
+  - oven subset run: `9/10`
+  - oven full official run: `95/100 = 0.95`
+  - hybrid public benchmark smoke outputs
+- DexGarmentLab benchmark-related validation scripts and validation logs
+
+## Key Result
+
+The strongest public out-of-box checkpoint validated here is:
+
+- `models/3dfa_peract2/3dfa_peract2.pth`
+
+Official oven result artifacts:
+
+- `reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval_after_official_ttm.json`
+- `reports/3dfa_peract2_take_tray_out_of_oven_full100/eval.json`
+
+## Important Code Paths
+
+- `code/VLAarchtests4/code/VLAarchtests2_code/VLAarchtests/code/reveal_vla_bimanual/eval/public_benchmark_package.py`
+- `code/VLAarchtests4/code/VLAarchtests2_code/VLAarchtests/code/reveal_vla_bimanual/eval/run_rlbench_hybrid_smoke.py`
+- `third_party/AnyBimanual/third_party/RLBench/rlbench/bimanual_tasks/bimanual_take_tray_out_of_oven.py`
+- `third_party/AnyBimanual/third_party/RLBench/rlbench/task_ttms/bimanual_take_tray_out_of_oven.ttm`
+
+## External Dependencies Not Mirrored Here
+
+- CoppeliaSim v4.1.0 binary runtime
+- Local Python environments under `/workspace/envs`
+- Full IsaacSim installation
+- Full DexGarmentLab simulator assets beyond the benchmark-related scripts and logs
+
+See `docs/ENVIRONMENT_NOTES.md` for the runtime notes used in this workspace.

docs/BUNDLE_CONTENTS.txt

@@ -0,0 +1,10 @@
+# Bundle Contents
+
+2.0K	README.md
+19M	code
+979K	docs
+979K	scripts
+1.9G	third_party
+1.6G	models
+3.2G	data
+6.9M	reports

docs/ENVIRONMENT_NOTES.md

@@ -0,0 +1,47 @@
+# Environment Notes
+
+This benchmark bundle was validated on a Linux GPU machine with:
+
+- CoppeliaSim v4.1.0 available at `COPPELIASIM_ROOT`
+- `xvfb` and `xauth` installed
+- Qt xcb runtime libraries installed:
+  - `libxrender1`
+  - `libxkbcommon0`
+  - `libxkbcommon-x11-0`
+  - `libxcb-icccm4`
+  - `libxcb-image0`
+  - `libxcb-keysyms1`
+  - `libxcb-randr0`
+  - `libxcb-render-util0`
+  - `libxcb-shape0`
+  - `libxcb-shm0`
+  - `libxcb-sync1`
+  - `libxcb-xfixes0`
+  - `libxcb-xinerama0`
+  - `libxcb-xkb1`
+
+The successful RLBench2 oven evaluation was run with:
+
+```bash
+xvfb-run -a -s "-screen 0 1400x900x24" python \
+  online_evaluation_rlbench/evaluate_policy.py \
+  --checkpoint models/3dfa_peract2/3dfa_peract2.pth \
+  --task bimanual_take_tray_out_of_oven \
+  --data_dir data/3dfa/peract2_test \
+  --dataset Peract2_3dfront_3dwrist \
+  --image_size 256,256 \
+  --model_type denoise3d \
+  --bimanual true \
+  --prediction_len 1 \
+  --backbone clip \
+  --fps_subsampling_factor 4 \
+  --embedding_dim 120 \
+  --num_attn_heads 8 \
+  --num_vis_instr_attn_layers 3 \
+  --num_history 3 \
+  --num_shared_attn_layers 4 \
+  --relative_action false \
+  --rotation_format quat_xyzw \
+  --denoise_timesteps 5 \
+  --denoise_model rectified_flow
+```

models/anybimanual/peract_lf/config.yaml

@@ -0,0 +1,139 @@
+ddp:
+  master_addr: localhost
+  master_port: 13985
+  num_devices: 2
+rlbench:
+  task_name: perlf+ab
+  tasks:
+  - bimanual_pick_laptop
+  - bimanual_pick_plate
+  - bimanual_straighten_rope
+  - coordinated_lift_ball
+  - coordinated_lift_tray
+  - coordinated_push_box
+  - coordinated_put_bottle_in_fridge
+  - dual_push_buttons
+  - handover_item
+  - bimanual_sweep_to_dustpan
+  - coordinated_take_tray_out_of_oven
+  - handover_item_easy
+  demos: 100
+  demo_path: /mnt/disk_1/tengbo/bimanual_data/train
+  episode_length: 25
+  cameras:
+  - over_shoulder_right
+  - wrist_right
+  - front
+  - overhead
+  - over_shoulder_left
+  - wrist_left
+  camera_resolution:
+  - 256
+  - 256
+  scene_bounds:
+  - -0.3
+  - -0.5
+  - 0.6
+  - 0.7
+  - 0.5
+  - 1.6
+  include_lang_goal_in_obs: true
+  instructions: ''
+replay:
+  batch_size: 2
+  timesteps: 1
+  prioritisation: false
+  task_uniform: true
+  use_disk: true
+  path: /mnt/disk_2/tengbo/replay/
+  max_parallel_processes: 32
+  task_folder: multi
+framework:
+  log_freq: 1000
+  save_freq: 10000
+  train_envs: 1
+  replay_ratio: ${replay.batch_size}
+  transitions_before_train: 200
+  tensorboard_logging: true
+  csv_logging: true
+  training_iterations: 100001
+  gpu: 0
+  env_gpu: 0
+  logdir: /mnt/disk_1/tengbo/peract_bimanual/log
+  logging_level: 20
+  seeds: 1
+  start_seed: 0
+  load_existing_weights: false
+  num_weights_to_keep: 60
+  num_workers: 0
+  record_every_n: 5
+  checkpoint_name_prefix: checkpoint
+  use_wandb: true
+  wandb_project: ''
+  wandb_group: perlf+ab
+  seed: 0
+  wandb_name: perlf+ab
+  use_skill: true
+  use_pretrained: false
+  use_prefix: false
+  frozen: ''
+  anybimanual: true
+  augmentation_type: ab
+method:
+  name: PERACT_BC
+  agent_type: leader_follower
+  robot_name: bimanual
+  image_crop_size: 64
+  bounds_offset:
+  - 0.15
+  voxel_sizes:
+  - 100
+  include_prev_layer: false
+  num_latents: 2048
+  latent_dim: 512
+  transformer_depth: 6
+  transformer_iterations: 1
+  cross_heads: 1
+  cross_dim_head: 64
+  latent_heads: 8
+  latent_dim_head: 64
+  pos_encoding_with_lang: true
+  conv_downsample: true
+  lang_fusion_type: seq
+  voxel_patch_size: 5
+  voxel_patch_stride: 5
+  final_dim: 64
+  low_dim_size: 4
+  input_dropout: 0.1
+  attn_dropout: 0.1
+  decoder_dropout: 0.0
+  lr: 0.0005
+  lr_scheduler: false
+  num_warmup_steps: 3000
+  optimizer: lamb
+  lambda_weight_l2: 1.0e-06
+  trans_loss_weight: 1.0
+  rot_loss_weight: 1.0
+  grip_loss_weight: 1.0
+  collision_loss_weight: 1.0
+  rotation_resolution: 5
+  activation: lrelu
+  norm: None
+  crop_augmentation: true
+  transform_augmentation:
+    apply_se3: true
+    aug_xyz:
+    - 0.125
+    - 0.125
+    - 0.125
+    aug_rpy:
+    - 0.0
+    - 0.0
+    - 45.0
+    aug_rot_resolution: ${method.rotation_resolution}
+  demo_augmentation: true
+  demo_augmentation_every_n: 10
+  no_skip_connection: false
+  no_perceiver: false
+  no_language: false
+  keypoint_method: heuristic

models/anybimanual/peract_lf/eval_data.csv

@@ -0,0 +1,4 @@
+step,eval_envs/return/bimanual_pick_laptop,eval_envs/length/bimanual_pick_laptop,eval_envs/total_transitions/bimanual_pick_laptop,eval_envs/return/bimanual_pick_plate,eval_envs/length/bimanual_pick_plate,eval_envs/total_transitions/bimanual_pick_plate,eval_envs/return/bimanual_straighten_rope,eval_envs/length/bimanual_straighten_rope,eval_envs/total_transitions/bimanual_straighten_rope,eval_envs/return/coordinated_lift_ball,eval_envs/length/coordinated_lift_ball,eval_envs/total_transitions/coordinated_lift_ball,eval_envs/return/coordinated_lift_tray,eval_envs/length/coordinated_lift_tray,eval_envs/total_transitions/coordinated_lift_tray,eval_envs/return/coordinated_push_box,eval_envs/length/coordinated_push_box,eval_envs/total_transitions/coordinated_push_box,eval_envs/return/coordinated_put_bottle_in_fridge,eval_envs/length/coordinated_put_bottle_in_fridge,eval_envs/total_transitions/coordinated_put_bottle_in_fridge,eval_envs/return/dual_push_buttons,eval_envs/length/dual_push_buttons,eval_envs/total_transitions/dual_push_buttons,eval_envs/return/handover_item,eval_envs/length/handover_item,eval_envs/total_transitions/handover_item,eval_envs/return/bimanual_sweep_to_dustpan,eval_envs/length/bimanual_sweep_to_dustpan,eval_envs/total_transitions/bimanual_sweep_to_dustpan,eval_envs/return/coordinated_take_tray_out_of_oven,eval_envs/length/coordinated_take_tray_out_of_oven,eval_envs/total_transitions/coordinated_take_tray_out_of_oven,eval_envs/return/handover_item_easy,eval_envs/length/handover_item_easy,eval_envs/total_transitions/handover_item_easy
+50000,0.0,25.0,625,4.0,24.76,1244,0.0,25.0,1869,0.0,25.0,2494,0.0,25.0,3119,0.0,25.0,3744,0.0,25.0,4369,20.0,21.84,4915,0.0,25.0,5540,0.0,25.0,6165,0.0,25.0,6790,8.0,24.0,7390
+60000,8.0,25.0,625,4.0,24.48,1237,4.0,25.0,1862,8.0,23.24,2443,4.0,25.0,3068,20.0,25.0,3693,4.0,25.0,4318,24.0,21.56,4857,4.0,25.0,5482,24.0,25.0,6107,0.0,25.0,6732,12.0,24.6,7347
+70000,4.0,25.0,625,12.0,23.24,1206,4.0,25.0,1831,0.0,25.0,2456,0.0,25.0,3081,12.0,25.0,3706,0.0,25.0,4331,12.0,23.16,4910,0.0,25.0,5535,16.0,25.0,6228,0.0,25.0,6160,8.0,25.0,6785

reports/3dfa_peract2_take_tray_out_of_oven_full100/eval.json

@@ -0,0 +1,6 @@
+{
+    "bimanual_take_tray_out_of_oven": {
+        "0": 95,
+        "mean": 0.95
+    }
+}

reports/3dfa_peract2_take_tray_out_of_oven_full100/run.log

@@ -0,0 +1,242 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_full100/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+WARNING:root:not sure how _robot_shapes are used is used.
+
  0%|          | 0/1 [00:00<?, ?it/s]WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+
+bimanual_take_tray_out_of_oven Variation 0 Demo 0 Reward 1.00 max_reward 1.00 SR: 1/1 SR: 1.00/1 # valid demos 1
+bimanual_take_tray_out_of_oven Variation 0 Demo 1 Reward 1.00 max_reward 1.00 SR: 2/2 SR: 2.00/2 # valid demos 2
+bimanual_take_tray_out_of_oven Variation 0 Demo 2 Reward 1.00 max_reward 1.00 SR: 3/3 SR: 3.00/3 # valid demos 3
+bimanual_take_tray_out_of_oven Variation 0 Demo 3 Reward 1.00 max_reward 1.00 SR: 4/4 SR: 4.00/4 # valid demos 4
+bimanual_take_tray_out_of_oven Variation 0 Demo 4 Reward 1.00 max_reward 1.00 SR: 5/5 SR: 5.00/5 # valid demos 5
+bimanual_take_tray_out_of_oven Variation 0 Demo 5 Reward 1.00 max_reward 1.00 SR: 6/6 SR: 6.00/6 # valid demos 6
+bimanual_take_tray_out_of_oven Variation 0 Demo 6 Reward 0.00 max_reward 0.00 SR: 6/7 SR: 6.00/7 # valid demos 7
+bimanual_take_tray_out_of_oven Variation 0 Demo 7 Reward 1.00 max_reward 1.00 SR: 7/8 SR: 7.00/8 # valid demos 8
+bimanual_take_tray_out_of_oven Variation 0 Demo 8 Reward 1.00 max_reward 1.00 SR: 8/9 SR: 8.00/9 # valid demos 9
+bimanual_take_tray_out_of_oven Variation 0 Demo 9 Reward 1.00 max_reward 1.00 SR: 9/10 SR: 9.00/10 # valid demos 10
+bimanual_take_tray_out_of_oven Variation 0 Demo 10 Reward 1.00 max_reward 1.00 SR: 10/11 SR: 10.00/11 # valid demos 11
+bimanual_take_tray_out_of_oven Variation 0 Demo 11 Reward 1.00 max_reward 1.00 SR: 11/12 SR: 11.00/12 # valid demos 12
+bimanual_take_tray_out_of_oven Variation 0 Demo 12 Reward 1.00 max_reward 1.00 SR: 12/13 SR: 12.00/13 # valid demos 13
+bimanual_take_tray_out_of_oven Variation 0 Demo 13 Reward 1.00 max_reward 1.00 SR: 13/14 SR: 13.00/14 # valid demos 14
+bimanual_take_tray_out_of_oven Variation 0 Demo 14 Reward 1.00 max_reward 1.00 SR: 14/15 SR: 14.00/15 # valid demos 15
+bimanual_take_tray_out_of_oven Variation 0 Demo 15 Reward 1.00 max_reward 1.00 SR: 15/16 SR: 15.00/16 # valid demos 16
+bimanual_take_tray_out_of_oven Variation 0 Demo 16 Reward 1.00 max_reward 1.00 SR: 16/17 SR: 16.00/17 # valid demos 17
+bimanual_take_tray_out_of_oven Variation 0 Demo 17 Reward 0.00 max_reward 0.00 SR: 16/18 SR: 16.00/18 # valid demos 18
+bimanual_take_tray_out_of_oven Variation 0 Demo 18 Reward 1.00 max_reward 1.00 SR: 17/19 SR: 17.00/19 # valid demos 19
+bimanual_take_tray_out_of_oven Variation 0 Demo 19 Reward 1.00 max_reward 1.00 SR: 18/20 SR: 18.00/20 # valid demos 20
+bimanual_take_tray_out_of_oven Variation 0 Demo 20 Reward 1.00 max_reward 1.00 SR: 19/21 SR: 19.00/21 # valid demos 21
+bimanual_take_tray_out_of_oven Variation 0 Demo 21 Reward 1.00 max_reward 1.00 SR: 20/22 SR: 20.00/22 # valid demos 22
+bimanual_take_tray_out_of_oven Variation 0 Demo 22 Reward 1.00 max_reward 1.00 SR: 21/23 SR: 21.00/23 # valid demos 23
+bimanual_take_tray_out_of_oven Variation 0 Demo 23 Reward 1.00 max_reward 1.00 SR: 22/24 SR: 22.00/24 # valid demos 24
+bimanual_take_tray_out_of_oven Variation 0 Demo 24 Reward 1.00 max_reward 1.00 SR: 23/25 SR: 23.00/25 # valid demos 25
+bimanual_take_tray_out_of_oven Variation 0 Demo 25 Reward 1.00 max_reward 1.00 SR: 24/26 SR: 24.00/26 # valid demos 26
+bimanual_take_tray_out_of_oven Variation 0 Demo 26 Reward 0.00 max_reward 0.00 SR: 24/27 SR: 24.00/27 # valid demos 27
+bimanual_take_tray_out_of_oven Variation 0 Demo 27 Reward 1.00 max_reward 1.00 SR: 25/28 SR: 25.00/28 # valid demos 28
+bimanual_take_tray_out_of_oven Variation 0 Demo 28 Reward 1.00 max_reward 1.00 SR: 26/29 SR: 26.00/29 # valid demos 29
+bimanual_take_tray_out_of_oven Variation 0 Demo 29 Reward 1.00 max_reward 1.00 SR: 27/30 SR: 27.00/30 # valid demos 30
+bimanual_take_tray_out_of_oven Variation 0 Demo 30 Reward 1.00 max_reward 1.00 SR: 28/31 SR: 28.00/31 # valid demos 31
+bimanual_take_tray_out_of_oven Variation 0 Demo 31 Reward 1.00 max_reward 1.00 SR: 29/32 SR: 29.00/32 # valid demos 32
+bimanual_take_tray_out_of_oven Variation 0 Demo 32 Reward 1.00 max_reward 1.00 SR: 30/33 SR: 30.00/33 # valid demos 33
+bimanual_take_tray_out_of_oven Variation 0 Demo 33 Reward 1.00 max_reward 1.00 SR: 31/34 SR: 31.00/34 # valid demos 34
+bimanual_take_tray_out_of_oven Variation 0 Demo 34 Reward 1.00 max_reward 1.00 SR: 32/35 SR: 32.00/35 # valid demos 35
+bimanual_take_tray_out_of_oven Variation 0 Demo 35 Reward 1.00 max_reward 1.00 SR: 33/36 SR: 33.00/36 # valid demos 36
+bimanual_take_tray_out_of_oven Variation 0 Demo 36 Reward 1.00 max_reward 1.00 SR: 34/37 SR: 34.00/37 # valid demos 37
+bimanual_take_tray_out_of_oven Variation 0 Demo 37 Reward 1.00 max_reward 1.00 SR: 35/38 SR: 35.00/38 # valid demos 38
+bimanual_take_tray_out_of_oven Variation 0 Demo 38 Reward 1.00 max_reward 1.00 SR: 36/39 SR: 36.00/39 # valid demos 39
+bimanual_take_tray_out_of_oven Variation 0 Demo 39 Reward 1.00 max_reward 1.00 SR: 37/40 SR: 37.00/40 # valid demos 40
+bimanual_take_tray_out_of_oven Variation 0 Demo 40 Reward 1.00 max_reward 1.00 SR: 38/41 SR: 38.00/41 # valid demos 41
+bimanual_take_tray_out_of_oven Variation 0 Demo 41 Reward 1.00 max_reward 1.00 SR: 39/42 SR: 39.00/42 # valid demos 42
+bimanual_take_tray_out_of_oven Variation 0 Demo 42 Reward 1.00 max_reward 1.00 SR: 40/43 SR: 40.00/43 # valid demos 43
+bimanual_take_tray_out_of_oven Variation 0 Demo 43 Reward 1.00 max_reward 1.00 SR: 41/44 SR: 41.00/44 # valid demos 44
+bimanual_take_tray_out_of_oven Variation 0 Demo 44 Reward 1.00 max_reward 1.00 SR: 42/45 SR: 42.00/45 # valid demos 45
+bimanual_take_tray_out_of_oven Variation 0 Demo 45 Reward 1.00 max_reward 1.00 SR: 43/46 SR: 43.00/46 # valid demos 46
+bimanual_take_tray_out_of_oven Variation 0 Demo 46 Reward 1.00 max_reward 1.00 SR: 44/47 SR: 44.00/47 # valid demos 47
+bimanual_take_tray_out_of_oven Variation 0 Demo 47 Reward 1.00 max_reward 1.00 SR: 45/48 SR: 45.00/48 # valid demos 48
+bimanual_take_tray_out_of_oven Variation 0 Demo 48 Reward 1.00 max_reward 1.00 SR: 46/49 SR: 46.00/49 # valid demos 49
+bimanual_take_tray_out_of_oven Variation 0 Demo 49 Reward 1.00 max_reward 1.00 SR: 47/50 SR: 47.00/50 # valid demos 50
+bimanual_take_tray_out_of_oven Variation 0 Demo 50 Reward 1.00 max_reward 1.00 SR: 48/51 SR: 48.00/51 # valid demos 51
+bimanual_take_tray_out_of_oven Variation 0 Demo 51 Reward 1.00 max_reward 1.00 SR: 49/52 SR: 49.00/52 # valid demos 52
+bimanual_take_tray_out_of_oven Variation 0 Demo 52 Reward 1.00 max_reward 1.00 SR: 50/53 SR: 50.00/53 # valid demos 53
+bimanual_take_tray_out_of_oven Variation 0 Demo 53 Reward 1.00 max_reward 1.00 SR: 51/54 SR: 51.00/54 # valid demos 54
+bimanual_take_tray_out_of_oven Variation 0 Demo 54 Reward 1.00 max_reward 1.00 SR: 52/55 SR: 52.00/55 # valid demos 55
+bimanual_take_tray_out_of_oven Variation 0 Demo 55 Reward 1.00 max_reward 1.00 SR: 53/56 SR: 53.00/56 # valid demos 56
+bimanual_take_tray_out_of_oven Variation 0 Demo 56 Reward 1.00 max_reward 1.00 SR: 54/57 SR: 54.00/57 # valid demos 57
+bimanual_take_tray_out_of_oven Variation 0 Demo 57 Reward 1.00 max_reward 1.00 SR: 55/58 SR: 55.00/58 # valid demos 58
+bimanual_take_tray_out_of_oven Variation 0 Demo 58 Reward 1.00 max_reward 1.00 SR: 56/59 SR: 56.00/59 # valid demos 59
+bimanual_take_tray_out_of_oven Variation 0 Demo 59 Reward 1.00 max_reward 1.00 SR: 57/60 SR: 57.00/60 # valid demos 60
+bimanual_take_tray_out_of_oven Variation 0 Demo 60 Reward 1.00 max_reward 1.00 SR: 58/61 SR: 58.00/61 # valid demos 61
+bimanual_take_tray_out_of_oven Variation 0 Demo 61 Reward 1.00 max_reward 1.00 SR: 59/62 SR: 59.00/62 # valid demos 62
+bimanual_take_tray_out_of_oven Variation 0 Demo 62 Reward 1.00 max_reward 1.00 SR: 60/63 SR: 60.00/63 # valid demos 63
+bimanual_take_tray_out_of_oven Variation 0 Demo 63 Reward 1.00 max_reward 1.00 SR: 61/64 SR: 61.00/64 # valid demos 64
+bimanual_take_tray_out_of_oven Variation 0 Demo 64 Reward 1.00 max_reward 1.00 SR: 62/65 SR: 62.00/65 # valid demos 65
+bimanual_take_tray_out_of_oven Variation 0 Demo 65 Reward 1.00 max_reward 1.00 SR: 63/66 SR: 63.00/66 # valid demos 66
+bimanual_take_tray_out_of_oven Variation 0 Demo 66 Reward 1.00 max_reward 1.00 SR: 64/67 SR: 64.00/67 # valid demos 67
+bimanual_take_tray_out_of_oven Variation 0 Demo 67 Reward 1.00 max_reward 1.00 SR: 65/68 SR: 65.00/68 # valid demos 68
+bimanual_take_tray_out_of_oven Variation 0 Demo 68 Reward 1.00 max_reward 1.00 SR: 66/69 SR: 66.00/69 # valid demos 69
+bimanual_take_tray_out_of_oven Variation 0 Demo 69 Reward 1.00 max_reward 1.00 SR: 67/70 SR: 67.00/70 # valid demos 70
+bimanual_take_tray_out_of_oven Variation 0 Demo 70 Reward 1.00 max_reward 1.00 SR: 68/71 SR: 68.00/71 # valid demos 71
+bimanual_take_tray_out_of_oven Variation 0 Demo 71 Reward 1.00 max_reward 1.00 SR: 69/72 SR: 69.00/72 # valid demos 72
+bimanual_take_tray_out_of_oven Variation 0 Demo 72 Reward 1.00 max_reward 1.00 SR: 70/73 SR: 70.00/73 # valid demos 73
+bimanual_take_tray_out_of_oven Variation 0 Demo 73 Reward 1.00 max_reward 1.00 SR: 71/74 SR: 71.00/74 # valid demos 74
+bimanual_take_tray_out_of_oven Variation 0 Demo 74 Reward 1.00 max_reward 1.00 SR: 72/75 SR: 72.00/75 # valid demos 75
+bimanual_take_tray_out_of_oven Variation 0 Demo 75 Reward 1.00 max_reward 1.00 SR: 73/76 SR: 73.00/76 # valid demos 76
+bimanual_take_tray_out_of_oven Variation 0 Demo 76 Reward 1.00 max_reward 1.00 SR: 74/77 SR: 74.00/77 # valid demos 77
+bimanual_take_tray_out_of_oven Variation 0 Demo 77 Reward 1.00 max_reward 1.00 SR: 75/78 SR: 75.00/78 # valid demos 78
+bimanual_take_tray_out_of_oven Variation 0 Demo 78 Reward 1.00 max_reward 1.00 SR: 76/79 SR: 76.00/79 # valid demos 79
+bimanual_take_tray_out_of_oven Variation 0 Demo 79 Reward 1.00 max_reward 1.00 SR: 77/80 SR: 77.00/80 # valid demos 80
+bimanual_take_tray_out_of_oven Variation 0 Demo 80 Reward 1.00 max_reward 1.00 SR: 78/81 SR: 78.00/81 # valid demos 81
+bimanual_take_tray_out_of_oven Variation 0 Demo 81 Reward 1.00 max_reward 1.00 SR: 79/82 SR: 79.00/82 # valid demos 82
+bimanual_take_tray_out_of_oven Variation 0 Demo 82 Reward 1.00 max_reward 1.00 SR: 80/83 SR: 80.00/83 # valid demos 83
+bimanual_take_tray_out_of_oven Variation 0 Demo 83 Reward 1.00 max_reward 1.00 SR: 81/84 SR: 81.00/84 # valid demos 84
+bimanual_take_tray_out_of_oven Variation 0 Demo 84 Reward 1.00 max_reward 1.00 SR: 82/85 SR: 82.00/85 # valid demos 85
+bimanual_take_tray_out_of_oven Variation 0 Demo 85 Reward 0.00 max_reward 0.00 SR: 82/86 SR: 82.00/86 # valid demos 86
+bimanual_take_tray_out_of_oven Variation 0 Demo 86 Reward 1.00 max_reward 1.00 SR: 83/87 SR: 83.00/87 # valid demos 87
+bimanual_take_tray_out_of_oven Variation 0 Demo 87 Reward 1.00 max_reward 1.00 SR: 84/88 SR: 84.00/88 # valid demos 88
+bimanual_take_tray_out_of_oven Variation 0 Demo 88 Reward 1.00 max_reward 1.00 SR: 85/89 SR: 85.00/89 # valid demos 89
+bimanual_take_tray_out_of_oven Variation 0 Demo 89 Reward 0.00 max_reward 0.00 SR: 85/90 SR: 85.00/90 # valid demos 90
+bimanual_take_tray_out_of_oven Variation 0 Demo 90 Reward 1.00 max_reward 1.00 SR: 86/91 SR: 86.00/91 # valid demos 91
+bimanual_take_tray_out_of_oven Variation 0 Demo 91 Reward 1.00 max_reward 1.00 SR: 87/92 SR: 87.00/92 # valid demos 92
+bimanual_take_tray_out_of_oven Variation 0 Demo 92 Reward 1.00 max_reward 1.00 SR: 88/93 SR: 88.00/93 # valid demos 93
+bimanual_take_tray_out_of_oven Variation 0 Demo 93 Reward 1.00 max_reward 1.00 SR: 89/94 SR: 89.00/94 # valid demos 94
+bimanual_take_tray_out_of_oven Variation 0 Demo 94 Reward 1.00 max_reward 1.00 SR: 90/95 SR: 90.00/95 # valid demos 95
+bimanual_take_tray_out_of_oven Variation 0 Demo 95 Reward 1.00 max_reward 1.00 SR: 91/96 SR: 91.00/96 # valid demos 96
+bimanual_take_tray_out_of_oven Variation 0 Demo 96 Reward 1.00 max_reward 1.00 SR: 92/97 SR: 92.00/97 # valid demos 97
+bimanual_take_tray_out_of_oven Variation 0 Demo 97 Reward 1.00 max_reward 1.00 SR: 93/98 SR: 93.00/98 # valid demos 98
+bimanual_take_tray_out_of_oven Variation 0 Demo 98 Reward 1.00 max_reward 1.00 SR: 94/99 SR: 94.00/99 # valid demos 99
+bimanual_take_tray_out_of_oven Variation 0 Demo 99 Reward 1.00 max_reward 1.00 SR: 95/100 SR: 95.00/100 # valid demos 100
+
+bimanual_take_tray_out_of_oven variation success rates: {0: 95, 'mean': 0.95}
+bimanual_take_tray_out_of_oven mean success rate: 0.95
+[CoppeliaSim:loadinfo]   done.

reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json

@@ -0,0 +1,6 @@
+{
+    "bimanual_take_tray_out_of_oven": {
+        "0": 10,
+        "mean": 1.0
+    }
+}

reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval_after_official_ttm.json

@@ -0,0 +1,6 @@
+{
+    "bimanual_take_tray_out_of_oven": {
+        "0": 9,
+        "mean": 0.9
+    }
+}

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run.log

@@ -0,0 +1,11 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/workspace/assets/coppeliasim_v4_1_0" even though it was found.
+This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
+
+Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, webgl, xcb.
+

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_minimalegl.log

@@ -0,0 +1,11 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+qt.qpa.plugin: Could not load the Qt platform plugin "minimalegl" in "/workspace/assets/coppeliasim_v4_1_0" even though it was found.
+This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
+
+Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, webgl, xcb.
+

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_offscreen.log

@@ -0,0 +1,26 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
+Traceback (most recent call last):
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/evaluate_policy.py", line 143, in <module>
+    var_success_rates = env.evaluate_task_on_multiple_variations(
+                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/utils_with_bimanual_rlbench.py", line 196, in evaluate_task_on_multiple_variations
+    task = self.env.get_task(task_type)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/environment.py", line 178, in get_task
+    return TaskEnvironment(
+           ^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/task_environment.py", line 52, in __init__
+    self._scene.load(self._task)
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/backend/scene.py", line 112, in load
+    task.load()  # Load the task in to the scene
+    ^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/backend/task.py", line 314, in load
+    raise FileNotFoundError(
+FileNotFoundError: The following is not a valid task .ttm file: /workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/backend/../task_ttms/bimanual_take_tray_out_of_oven.ttm
+QMutex: destroying locked mutex

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_offscreen2.log

@@ -0,0 +1,31 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+Traceback (most recent call last):
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/evaluate_policy.py", line 143, in <module>
+    var_success_rates = env.evaluate_task_on_multiple_variations(
+                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/utils_with_bimanual_rlbench.py", line 196, in evaluate_task_on_multiple_variations
+    task = self.env.get_task(task_type)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/environment.py", line 178, in get_task
+    return TaskEnvironment(
+           ^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/task_environment.py", line 52, in __init__
+    self._scene.load(self._task)
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/backend/scene.py", line 115, in load
+    task.get_base().set_position(self._workspace.get_position())
+    ^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/RLBench/rlbench/backend/task.py", line 341, in get_base
+    self._base_object = Dummy(self.name)
+                        ^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/PyRep/pyrep/objects/object.py", line 24, in __init__
+    self._handle = sim.simGetObjectHandle(name_or_handle)
+                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/AnyBimanual/third_party/PyRep/pyrep/backend/sim.py", line 100, in simGetObjectHandle
+    raise RuntimeError('Handle %s does not exist.' % objectName)
+RuntimeError: Handle bimanual_take_tray_out_of_oven does not exist.
+QMutex: destroying locked mutex

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_offscreen3.log

@@ -0,0 +1,27 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+WARNING:root:not sure how _robot_shapes are used is used.
+
  0%|          | 0/1 [00:00<?, ?it/s]WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+This plugin does not support createPlatformOpenGLContext!
+
+
+Error: signal 11:
+
+/workspace/assets/coppeliasim_v4_1_0/libcoppeliaSim.so.1(_Z11_segHandleri+0x30)[0x73db22b0aae0]
+/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x73df84c74520]
+/workspace/assets/coppeliasim_v4_1_0/libQt5Gui.so.5(_ZNK14QOpenGLContext10shareGroupEv+0x0)[0x73db20d91060]
+/workspace/assets/coppeliasim_v4_1_0/libQt5Gui.so.5(_ZN16QOpenGLFunctions25initializeOpenGLFunctionsEv+0x4b)[0x73db2105da4b]
+/workspace/assets/coppeliasim_v4_1_0/libQt5Gui.so.5(_ZN24QOpenGLFramebufferObjectC1EiiNS_10AttachmentEjj+0xc8)[0x73db21061a18]
+/workspace/assets/coppeliasim_v4_1_0/libsimExtOpenGL3Renderer.so(_ZN18CFrameBufferObjectC2Eii+0x5a)[0x73de33e5424a]
+/workspace/assets/coppeliasim_v4_1_0/libsimExtOpenGL3Renderer.so(_ZN16COpenglOffscreenC1EiiiP14QOpenGLContext+0x72)[0x73de33e54602]
+/workspace/assets/coppeliasim_v4_1_0/libsimExtOpenGL3Renderer.so(_Z21executeRenderCommandsbiPv+0x2550)[0x73de33e52b90]
+/workspace/assets/coppeliasim_v4_1_0/libcoppeliaSim.so.1(_ZN16CPluginContainer11extRendererEiPv+0x19)[0x73db22cd4249]
+/workspace/assets/coppeliasim_v4_1_0/libcoppeliaSim.so.1(_ZN13CVisionSensor24_extRenderer_prepareViewEi+0x347)[0x73db229db107]
+QMutex: destroying locked mutex

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_xvfb.log

@@ -0,0 +1,12 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/workspace/assets/coppeliasim_v4_1_0" even though it was found.
+This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
+
+Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, webgl, xcb.
+
+Aborted (core dumped)

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_xvfb2.log

@@ -0,0 +1,29 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+WARNING:root:not sure how _robot_shapes are used is used.
+
  0%|          | 0/1 [00:00<?, ?it/s]WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+
  0%|          | 0/1 [00:08<?, ?it/s]
+Traceback (most recent call last):
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/evaluate_policy.py", line 143, in <module>
+    var_success_rates = env.evaluate_task_on_multiple_variations(
+                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/utils_with_bimanual_rlbench.py", line 211, in evaluate_task_on_multiple_variations
+    self._evaluate_task_on_one_variation(
+  File "/workspace/envs/anybi311/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
+    return func(*args, **kwargs)
+           ^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/utils_with_bimanual_rlbench.py", line 298, in _evaluate_task_on_one_variation
+    obs, reward, _ = move(action, collision_checking=False)
+                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/workspace/third_party/3d_flowmatch_actor/online_evaluation_rlbench/utils_with_bimanual_rlbench.py", line 59, in __call__
+    obs, reward, terminate = self._task.step(action_collision, ret_obs=True)
+                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+TypeError: TaskEnvironment.step() got an unexpected keyword argument 'ret_obs'
+QMutex: destroying locked mutex

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_xvfb3.log

@@ -0,0 +1,46 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+WARNING:root:not sure how _robot_shapes are used is used.
+
  0%|          | 0/1 [00:00<?, ?it/s]WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task coordinated_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+
+bimanual_take_tray_out_of_oven Variation 0 Demo 0 Reward 1.00 max_reward 1.00 SR: 1/1 SR: 1.00/1 # valid demos 1
+bimanual_take_tray_out_of_oven Variation 0 Demo 1 Reward 1.00 max_reward 1.00 SR: 2/2 SR: 2.00/2 # valid demos 2
+bimanual_take_tray_out_of_oven Variation 0 Demo 2 Reward 1.00 max_reward 1.00 SR: 3/3 SR: 3.00/3 # valid demos 3
+bimanual_take_tray_out_of_oven Variation 0 Demo 3 Reward 1.00 max_reward 1.00 SR: 4/4 SR: 4.00/4 # valid demos 4
+bimanual_take_tray_out_of_oven Variation 0 Demo 4 Reward 1.00 max_reward 1.00 SR: 5/5 SR: 5.00/5 # valid demos 5
+bimanual_take_tray_out_of_oven Variation 0 Demo 5 Reward 1.00 max_reward 1.00 SR: 6/6 SR: 6.00/6 # valid demos 6
+bimanual_take_tray_out_of_oven Variation 0 Demo 6 Reward 1.00 max_reward 1.00 SR: 7/7 SR: 7.00/7 # valid demos 7
+bimanual_take_tray_out_of_oven Variation 0 Demo 7 Reward 1.00 max_reward 1.00 SR: 8/8 SR: 8.00/8 # valid demos 8
+bimanual_take_tray_out_of_oven Variation 0 Demo 8 Reward 1.00 max_reward 1.00 SR: 9/9 SR: 9.00/9 # valid demos 9
+bimanual_take_tray_out_of_oven Variation 0 Demo 9 Reward 1.00 max_reward 1.00 SR: 10/10 SR: 10.00/10 # valid demos 10
+
+bimanual_take_tray_out_of_oven variation success rates: {0: 10, 'mean': 1.0}
+bimanual_take_tray_out_of_oven mean success rate: 1.0
+[CoppeliaSim:loadinfo]   done.

reports/3dfa_peract2_take_tray_out_of_oven_subset10/run_xvfb_official_ttm.log

@@ -0,0 +1,46 @@
+Arguments:
+Namespace(checkpoint='/workspace/models/3dfa_peract2/3dfa_peract2.pth', task='bimanual_take_tray_out_of_oven', max_tries=2, max_steps=25, headless=True, collision_checking=False, seed=0, data_dir=PosixPath('/workspace/data/3dfa/peract2_test_10'), dataset='Peract2_3dfront_3dwrist', image_size='256,256', output_file=PosixPath('/workspace/reports/3dfa_peract2_take_tray_out_of_oven_subset10/eval_after_official_ttm.json'), model_type='denoise3d', bimanual=True, prediction_len=1, backbone='clip', fps_subsampling_factor=4, embedding_dim=120, num_attn_heads=8, num_vis_instr_attn_layers=3, num_history=3, num_shared_attn_layers=4, relative_action=False, rotation_format='quat_xyzw', denoise_timesteps=5, denoise_model='rectified_flow')
+----------------------------------------------------------------------------------------------------
+Loading model from /workspace/models/3dfa_peract2/3dfa_peract2.pth
+/workspace/envs/anybi311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
+  warnings.warn(
+WARNING:root:not sure how _robot_shapes are used is used.
+
  0%|          | 0/1 [00:00<?, ?it/s]WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (True, '')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint3 - DualPanda') left=(True, '')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint3 - DualPanda'), (True, '')).
+WARNING:root:Waypoints are not reachable right=(False, 'waypoint0 - DualPanda') left=(False, 'waypoint5 - DualPanda')
+ERROR:root:Error when checking waypoints. Exception is: Error in task bimanual_take_tray_out_of_oven. Infeasible episode. Can't reach waypoint ((False, 'waypoint0 - DualPanda'), (False, 'waypoint5 - DualPanda')).
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+ERROR:root:robot is in collision
+ERROR:root:Error when checking waypoints. Exception is: 
+
+bimanual_take_tray_out_of_oven Variation 0 Demo 0 Reward 1.00 max_reward 1.00 SR: 1/1 SR: 1.00/1 # valid demos 1
+bimanual_take_tray_out_of_oven Variation 0 Demo 1 Reward 1.00 max_reward 1.00 SR: 2/2 SR: 2.00/2 # valid demos 2
+bimanual_take_tray_out_of_oven Variation 0 Demo 2 Reward 1.00 max_reward 1.00 SR: 3/3 SR: 3.00/3 # valid demos 3
+bimanual_take_tray_out_of_oven Variation 0 Demo 3 Reward 1.00 max_reward 1.00 SR: 4/4 SR: 4.00/4 # valid demos 4
+bimanual_take_tray_out_of_oven Variation 0 Demo 4 Reward 1.00 max_reward 1.00 SR: 5/5 SR: 5.00/5 # valid demos 5
+bimanual_take_tray_out_of_oven Variation 0 Demo 5 Reward 1.00 max_reward 1.00 SR: 6/6 SR: 6.00/6 # valid demos 6
+bimanual_take_tray_out_of_oven Variation 0 Demo 6 Reward 1.00 max_reward 1.00 SR: 7/7 SR: 7.00/7 # valid demos 7
+bimanual_take_tray_out_of_oven Variation 0 Demo 7 Reward 1.00 max_reward 1.00 SR: 8/8 SR: 8.00/8 # valid demos 8
+bimanual_take_tray_out_of_oven Variation 0 Demo 8 Reward 1.00 max_reward 1.00 SR: 9/9 SR: 9.00/9 # valid demos 9
+bimanual_take_tray_out_of_oven Variation 0 Demo 9 Reward 0.00 max_reward 0.00 SR: 9/10 SR: 9.00/10 # valid demos 10
+
+bimanual_take_tray_out_of_oven variation success rates: {0: 9, 'mean': 0.9}
+bimanual_take_tray_out_of_oven mean success rate: 0.9
+[CoppeliaSim:loadinfo]   done.

reports/anybimanual_eval_logs/bimanual_sweep_to_dustpan_ep1_demo.log

@@ -0,0 +1,178 @@
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'eval': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
+  warnings.warn(msg, UserWarning)
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package _group_'.
+See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information
+  deprecation_warning(
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
+See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
+  ret = run_job(
+[2026-04-02 22:20:49,219][root][INFO] - 
+method:
+  name: PERACT_BC
+  agent_type: leader_follower
+  robot_name: bimanual
+  image_crop_size: 64
+  bounds_offset:
+  - 0.15
+  voxel_sizes:
+  - 100
+  include_prev_layer: false
+  num_latents: 2048
+  latent_dim: 512
+  transformer_depth: 6
+  transformer_iterations: 1
+  cross_heads: 1
+  cross_dim_head: 64
+  latent_heads: 8
+  latent_dim_head: 64
+  pos_encoding_with_lang: true
+  conv_downsample: true
+  lang_fusion_type: seq
+  voxel_patch_size: 5
+  voxel_patch_stride: 5
+  final_dim: 64
+  low_dim_size: 4
+  input_dropout: 0.1
+  attn_dropout: 0.1
+  decoder_dropout: 0.0
+  lr: 0.0005
+  lr_scheduler: false
+  num_warmup_steps: 3000
+  optimizer: lamb
+  lambda_weight_l2: 1.0e-06
+  trans_loss_weight: 1.0
+  rot_loss_weight: 1.0
+  grip_loss_weight: 1.0
+  collision_loss_weight: 1.0
+  rotation_resolution: 5
+  activation: lrelu
+  norm: None
+  crop_augmentation: true
+  transform_augmentation:
+    apply_se3: true
+    aug_xyz:
+    - 0.125
+    - 0.125
+    - 0.125
+    aug_rpy:
+    - 0.0
+    - 0.0
+    - 45.0
+    aug_rot_resolution: ${method.rotation_resolution}
+  demo_augmentation: true
+  demo_augmentation_every_n: 10
+  no_skip_connection: false
+  no_perceiver: false
+  no_language: false
+  keypoint_method: heuristic
+rlbench:
+  task_name: per2+ab
+  tasks:
+  - bimanual_sweep_to_dustpan
+  demo_path: /workspace/data/rlbench2
+  episode_length: 25
+  cameras:
+  - over_shoulder_left
+  - over_shoulder_right
+  - overhead
+  - wrist_right
+  - wrist_left
+  - front
+  camera_resolution:
+  - 256
+  - 256
+  scene_bounds:
+  - -0.3
+  - -0.5
+  - 0.6
+  - 0.7
+  - 0.5
+  - 1.6
+  include_lang_goal_in_obs: true
+  time_in_state: true
+  headless: true
+  gripper_mode: BimanualDiscrete
+  arm_action_mode: BimanualEndEffectorPoseViaPlanning
+  action_mode: BimanualMoveArmThenGripper
+framework:
+  tensorboard_logging: true
+  csv_logging: true
+  gpu: 0
+  logdir: /workspace/runs/anybimanual/task_eval_sweep_demo1
+  start_seed: 0
+  record_every_n: 5
+  eval_envs: 1
+  eval_from_eps_number: 0
+  eval_episodes: 1
+  eval_type: 60000
+  eval_save_metrics: true
+cinematic_recorder:
+  enabled: false
+  camera_resolution:
+  - 1280
+  - 720
+  fps: 30
+  rotate_speed: 0.005
+  save_path: /tmp/videos/
+
+[2026-04-02 22:20:49,227][root][INFO] - Using env device cuda:0.
+[2026-04-02 22:20:49,231][root][INFO] - Evaluating seed 0.
+[2026-04-02 22:20:49,232][root][INFO] - Using method PERACT_BC with type leader_follower
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+Weight: [60000]
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 22:21:23] INFO     INFO:root:eval_env:  _independent_env_runner.py:130
+                             Launching env.                                     
+                    INFO     INFO:root:Agent      _independent_env_runner.py:133
+                             information:                                       
+                    INFO     INFO:root:<yarr.agen _independent_env_runner.py:134
+                             ts.agent.LeaderFollo                               
+                             werAgent object at                                 
+                             0x77e49d22f910>                                    
+QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
+[04/02/26 22:21:24] INFO     INFO:root:Using dual panda robot environment.py:119
+                    INFO     INFO:root:Setting control    arm_action_modes.py:79
+                             mode for both robots                               
+                    WARNING  WARNING:root:not sure how    task_environment.py:57
+                             _robot_shapes are used is                          
+                             used.                                              
+                    INFO     INFO:root:Evaluating _independent_env_runner.py:164
+                             weight 60000                                       
+loaded weights from /workspace/runs/anybimanual/task_eval_sweep_demo1/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_leader_layer_0.pt
+loaded weights from /workspace/runs/anybimanual/task_eval_sweep_demo1/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_follower_layer_0.pt
+
+
+                             Starting episode 0,                                
+                             seed 0.                                            
+eval_demo_seed:  0
+[04/02/26 22:22:14] INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+/workspace/third_party/AnyBimanual/third_party/YARR/yarr/utils/rollout_generator.py:94: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:254.)
+  prepped_data = {k: torch.tensor([v], device=self._env_device) for k, v in obs_history.items()}
+
+
+
+Evaluating bimanual_sweep_to_dustpan | Episode 0 | Score: 0.0 | Lang Goal: sweep dirt to dustpan
+Finished bimanual_sweep_to_dustpan | Final Score: unknown
+
+[04/02/26 22:23:20] INFO     INFO:root:Finished   _independent_env_runner.py:292
+                             evaluation.                                        
+[CoppeliaSim:loadinfo]   done.

reports/anybimanual_eval_logs/bimanual_sweep_to_dustpan_ep5.log

@@ -0,0 +1,205 @@
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'eval': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
+  warnings.warn(msg, UserWarning)
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package _group_'.
+See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information
+  deprecation_warning(
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
+See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
+  ret = run_job(
+[2026-04-02 21:00:56,975][root][INFO] - 
+method:
+  name: PERACT_BC
+  agent_type: leader_follower
+  robot_name: bimanual
+  image_crop_size: 64
+  bounds_offset:
+  - 0.15
+  voxel_sizes:
+  - 100
+  include_prev_layer: false
+  num_latents: 2048
+  latent_dim: 512
+  transformer_depth: 6
+  transformer_iterations: 1
+  cross_heads: 1
+  cross_dim_head: 64
+  latent_heads: 8
+  latent_dim_head: 64
+  pos_encoding_with_lang: true
+  conv_downsample: true
+  lang_fusion_type: seq
+  voxel_patch_size: 5
+  voxel_patch_stride: 5
+  final_dim: 64
+  low_dim_size: 4
+  input_dropout: 0.1
+  attn_dropout: 0.1
+  decoder_dropout: 0.0
+  lr: 0.0005
+  lr_scheduler: false
+  num_warmup_steps: 3000
+  optimizer: lamb
+  lambda_weight_l2: 1.0e-06
+  trans_loss_weight: 1.0
+  rot_loss_weight: 1.0
+  grip_loss_weight: 1.0
+  collision_loss_weight: 1.0
+  rotation_resolution: 5
+  activation: lrelu
+  norm: None
+  crop_augmentation: true
+  transform_augmentation:
+    apply_se3: true
+    aug_xyz:
+    - 0.125
+    - 0.125
+    - 0.125
+    aug_rpy:
+    - 0.0
+    - 0.0
+    - 45.0
+    aug_rot_resolution: ${method.rotation_resolution}
+  demo_augmentation: true
+  demo_augmentation_every_n: 10
+  no_skip_connection: false
+  no_perceiver: false
+  no_language: false
+  keypoint_method: heuristic
+rlbench:
+  task_name: per2+ab
+  tasks:
+  - bimanual_sweep_to_dustpan
+  demo_path: /workspace/nonexistent_demos
+  episode_length: 25
+  cameras:
+  - over_shoulder_left
+  - over_shoulder_right
+  - overhead
+  - wrist_right
+  - wrist_left
+  - front
+  camera_resolution:
+  - 256
+  - 256
+  scene_bounds:
+  - -0.3
+  - -0.5
+  - 0.6
+  - 0.7
+  - 0.5
+  - 1.6
+  include_lang_goal_in_obs: true
+  time_in_state: true
+  headless: true
+  gripper_mode: BimanualDiscrete
+  arm_action_mode: BimanualEndEffectorPoseViaPlanning
+  action_mode: BimanualMoveArmThenGripper
+framework:
+  tensorboard_logging: true
+  csv_logging: true
+  gpu: 0
+  logdir: /workspace/runs/anybimanual/peract_lf_eval
+  start_seed: 0
+  record_every_n: 5
+  eval_envs: 1
+  eval_from_eps_number: 0
+  eval_episodes: 5
+  eval_type: 60000
+  eval_save_metrics: true
+cinematic_recorder:
+  enabled: false
+  camera_resolution:
+  - 1280
+  - 720
+  fps: 30
+  rotate_speed: 0.005
+  save_path: /tmp/videos/
+
+[2026-04-02 21:00:56,984][root][INFO] - Using env device cuda:0.
+[2026-04-02 21:00:56,989][root][INFO] - Evaluating seed 0.
+[2026-04-02 21:00:56,989][root][INFO] - Using method PERACT_BC with type leader_follower
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+Weight: [60000]
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 21:01:22] WARNING  WARNING:root:Data set root does   environment.py:87
+                             not exist:                                         
+                             /workspace/nonexistent_demos.                      
+                             Continuing without demos.                          
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 21:01:35] INFO     INFO:root:eval_env:  _independent_env_runner.py:130
+                             Launching env.                                     
+                    INFO     INFO:root:Agent      _independent_env_runner.py:133
+                             information:                                       
+                    INFO     INFO:root:<yarr.agen _independent_env_runner.py:134
+                             ts.agent.LeaderFollo                               
+                             werAgent object at                                 
+                             0x7404edc39690>                                    
+QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
+[04/02/26 21:01:36] INFO     INFO:root:Using dual panda robot environment.py:119
+                    INFO     INFO:root:Setting control    arm_action_modes.py:79
+                             mode for both robots                               
+                    WARNING  WARNING:root:not sure how    task_environment.py:57
+                             _robot_shapes are used is                          
+                             used.                                              
+                    INFO     INFO:root:Evaluating _independent_env_runner.py:164
+                             weight 60000                                       
+loaded weights from /workspace/runs/anybimanual/peract_lf_eval/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_leader_layer_0.pt
+loaded weights from /workspace/runs/anybimanual/peract_lf_eval/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_follower_layer_0.pt
+
+
+                             Starting episode 0,                                
+                             seed 0.                                            
+eval_demo_seed:  0
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+/workspace/third_party/AnyBimanual/third_party/YARR/yarr/utils/rollout_generator.py:94: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:254.)
+  prepped_data = {k: torch.tensor([v], device=self._env_device) for k, v in obs_history.items()}
+
+
+[04/02/26 21:04:09] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 1,                                
+                             seed 1.                                            
+eval_demo_seed:  1
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+Terminated
+The X11 connection broke (error 1). Did the X11 server die?
+QMutex: destroying locked mutex
+      with name Dustpan_5 is                             
+                             already grasped by left                            
+                             robot                                              
+
+
+[04/02/26 21:03:50] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 2,                                
+                             seed 2.                                            
+eval_demo_seed:  2
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+[W402 21:04:37.404034653 CudaIPCTypes.cpp:16] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]

reports/anybimanual_eval_logs/bimanual_sweep_to_dustpan_ep5_clean.log

@@ -0,0 +1,242 @@
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'eval': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
+  warnings.warn(msg, UserWarning)
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package _group_'.
+See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information
+  deprecation_warning(
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
+See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
+  ret = run_job(
+[2026-04-02 21:05:04,333][root][INFO] - 
+method:
+  name: PERACT_BC
+  agent_type: leader_follower
+  robot_name: bimanual
+  image_crop_size: 64
+  bounds_offset:
+  - 0.15
+  voxel_sizes:
+  - 100
+  include_prev_layer: false
+  num_latents: 2048
+  latent_dim: 512
+  transformer_depth: 6
+  transformer_iterations: 1
+  cross_heads: 1
+  cross_dim_head: 64
+  latent_heads: 8
+  latent_dim_head: 64
+  pos_encoding_with_lang: true
+  conv_downsample: true
+  lang_fusion_type: seq
+  voxel_patch_size: 5
+  voxel_patch_stride: 5
+  final_dim: 64
+  low_dim_size: 4
+  input_dropout: 0.1
+  attn_dropout: 0.1
+  decoder_dropout: 0.0
+  lr: 0.0005
+  lr_scheduler: false
+  num_warmup_steps: 3000
+  optimizer: lamb
+  lambda_weight_l2: 1.0e-06
+  trans_loss_weight: 1.0
+  rot_loss_weight: 1.0
+  grip_loss_weight: 1.0
+  collision_loss_weight: 1.0
+  rotation_resolution: 5
+  activation: lrelu
+  norm: None
+  crop_augmentation: true
+  transform_augmentation:
+    apply_se3: true
+    aug_xyz:
+    - 0.125
+    - 0.125
+    - 0.125
+    aug_rpy:
+    - 0.0
+    - 0.0
+    - 45.0
+    aug_rot_resolution: ${method.rotation_resolution}
+  demo_augmentation: true
+  demo_augmentation_every_n: 10
+  no_skip_connection: false
+  no_perceiver: false
+  no_language: false
+  keypoint_method: heuristic
+rlbench:
+  task_name: per2+ab
+  tasks:
+  - bimanual_sweep_to_dustpan
+  demo_path: /workspace/nonexistent_demos
+  episode_length: 25
+  cameras:
+  - over_shoulder_left
+  - over_shoulder_right
+  - overhead
+  - wrist_right
+  - wrist_left
+  - front
+  camera_resolution:
+  - 256
+  - 256
+  scene_bounds:
+  - -0.3
+  - -0.5
+  - 0.6
+  - 0.7
+  - 0.5
+  - 1.6
+  include_lang_goal_in_obs: true
+  time_in_state: true
+  headless: true
+  gripper_mode: BimanualDiscrete
+  arm_action_mode: BimanualEndEffectorPoseViaPlanning
+  action_mode: BimanualMoveArmThenGripper
+framework:
+  tensorboard_logging: true
+  csv_logging: true
+  gpu: 0
+  logdir: /workspace/runs/anybimanual/peract_lf_eval
+  start_seed: 0
+  record_every_n: 5
+  eval_envs: 1
+  eval_from_eps_number: 0
+  eval_episodes: 5
+  eval_type: 60000
+  eval_save_metrics: true
+cinematic_recorder:
+  enabled: false
+  camera_resolution:
+  - 1280
+  - 720
+  fps: 30
+  rotate_speed: 0.005
+  save_path: /tmp/videos/
+
+[2026-04-02 21:05:04,341][root][INFO] - Using env device cuda:0.
+[2026-04-02 21:05:04,347][root][INFO] - Evaluating seed 0.
+[2026-04-02 21:05:04,347][root][INFO] - Using method PERACT_BC with type leader_follower
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+Weight: [60000]
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 21:05:25] WARNING  WARNING:root:Data set root does   environment.py:87
+                             not exist:                                         
+                             /workspace/nonexistent_demos.                      
+                             Continuing without demos.                          
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 21:05:36] INFO     INFO:root:eval_env:  _independent_env_runner.py:130
+                             Launching env.                                     
+                    INFO     INFO:root:Agent      _independent_env_runner.py:133
+                             information:                                       
+                    INFO     INFO:root:<yarr.agen _independent_env_runner.py:134
+                             ts.agent.LeaderFollo                               
+                             werAgent object at                                 
+                             0x7393c6a70f50>                                    
+QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
+[04/02/26 21:05:38] INFO     INFO:root:Using dual panda robot environment.py:119
+                    INFO     INFO:root:Setting control    arm_action_modes.py:79
+                             mode for both robots                               
+                    WARNING  WARNING:root:not sure how    task_environment.py:57
+                             _robot_shapes are used is                          
+                             used.                                              
+                    INFO     INFO:root:Evaluating _independent_env_runner.py:164
+                             weight 60000                                       
+loaded weights from /workspace/runs/anybimanual/peract_lf_eval/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_leader_layer_0.pt
+loaded weights from /workspace/runs/anybimanual/peract_lf_eval/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_follower_layer_0.pt
+
+
+                             Starting episode 0,                                
+                             seed 0.                                            
+eval_demo_seed:  0
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+[04/02/26 21:05:54] WARNING  WARNING:root:Object     gripper_action_modes.py:328
+                             with name broom is                                 
+                             already grasped by                                 
+                             right robot                                        
+[04/02/26 21:06:47] WARNING  WARNING:root:Object     gripper_action_modes.py:328
+                             with name broom is                                 
+                             already grasped by                                 
+                             right robot                                        
+[04/02/26 21:07:07] WARNING  WARNING:root:Object     gripper_action_modes.py:328
+                             with name broom is                                 
+                             already grasped by                                 
+                             right robot                                        
+/workspace/third_party/AnyBimanual/third_party/YARR/yarr/utils/rollout_generator.py:94: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:254.)
+  prepped_data = {k: torch.tensor([v], device=self._env_device) for k, v in obs_history.items()}
+
+
+[04/02/26 21:07:14] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 1,                                
+                             seed 1.                                            
+eval_demo_seed:  1
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+[04/02/26 21:08:28] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 2,                                
+                             seed 2.                                            
+eval_demo_seed:  2
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+[04/02/26 21:10:01] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 3,                                
+                             seed 3.                                            
+eval_demo_seed:  3
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+[04/02/26 21:11:04] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 4,                                
+                             seed 4.                                            
+eval_demo_seed:  4
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+
+Evaluating bimanual_sweep_to_dustpan | Episode 4 | Score: 0.0 | Lang Goal: sweep dirt to dustpan
+Finished bimanual_sweep_to_dustpan | Final Score: 0.0
+
+add_video needs package moviepy
+[04/02/26 21:12:19] INFO     INFO:root:Finished   _independent_env_runner.py:292
+                             evaluation.                                        
+[CoppeliaSim:loadinfo]   done.

reports/anybimanual_eval_logs/bimanual_sweep_to_dustpan_ep5_live.log

@@ -0,0 +1,226 @@
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'eval': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
+  warnings.warn(msg, UserWarning)
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package _group_'.
+See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information
+  deprecation_warning(
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
+See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
+  ret = run_job(
+[2026-04-02 22:12:16,880][root][INFO] - 
+method:
+  name: PERACT_BC
+  agent_type: leader_follower
+  robot_name: bimanual
+  image_crop_size: 64
+  bounds_offset:
+  - 0.15
+  voxel_sizes:
+  - 100
+  include_prev_layer: false
+  num_latents: 2048
+  latent_dim: 512
+  transformer_depth: 6
+  transformer_iterations: 1
+  cross_heads: 1
+  cross_dim_head: 64
+  latent_heads: 8
+  latent_dim_head: 64
+  pos_encoding_with_lang: true
+  conv_downsample: true
+  lang_fusion_type: seq
+  voxel_patch_size: 5
+  voxel_patch_stride: 5
+  final_dim: 64
+  low_dim_size: 4
+  input_dropout: 0.1
+  attn_dropout: 0.1
+  decoder_dropout: 0.0
+  lr: 0.0005
+  lr_scheduler: false
+  num_warmup_steps: 3000
+  optimizer: lamb
+  lambda_weight_l2: 1.0e-06
+  trans_loss_weight: 1.0
+  rot_loss_weight: 1.0
+  grip_loss_weight: 1.0
+  collision_loss_weight: 1.0
+  rotation_resolution: 5
+  activation: lrelu
+  norm: None
+  crop_augmentation: true
+  transform_augmentation:
+    apply_se3: true
+    aug_xyz:
+    - 0.125
+    - 0.125
+    - 0.125
+    aug_rpy:
+    - 0.0
+    - 0.0
+    - 45.0
+    aug_rot_resolution: ${method.rotation_resolution}
+  demo_augmentation: true
+  demo_augmentation_every_n: 10
+  no_skip_connection: false
+  no_perceiver: false
+  no_language: false
+  keypoint_method: heuristic
+rlbench:
+  task_name: per2+ab
+  tasks:
+  - bimanual_sweep_to_dustpan
+  demo_path: /workspace/data/rlbench2
+  episode_length: 25
+  cameras:
+  - over_shoulder_left
+  - over_shoulder_right
+  - overhead
+  - wrist_right
+  - wrist_left
+  - front
+  camera_resolution:
+  - 256
+  - 256
+  scene_bounds:
+  - -0.3
+  - -0.5
+  - 0.6
+  - 0.7
+  - 0.5
+  - 1.6
+  include_lang_goal_in_obs: true
+  time_in_state: true
+  headless: true
+  gripper_mode: BimanualDiscrete
+  arm_action_mode: BimanualEndEffectorPoseViaPlanning
+  action_mode: BimanualMoveArmThenGripper
+framework:
+  tensorboard_logging: true
+  csv_logging: true
+  gpu: 0
+  logdir: /workspace/runs/anybimanual/task_eval_sweep
+  start_seed: 0
+  record_every_n: 5
+  eval_envs: 1
+  eval_from_eps_number: 0
+  eval_episodes: 5
+  eval_type: 60000
+  eval_save_metrics: true
+cinematic_recorder:
+  enabled: false
+  camera_resolution:
+  - 1280
+  - 720
+  fps: 30
+  rotate_speed: 0.005
+  save_path: /tmp/videos/
+
+[2026-04-02 22:12:16,889][root][INFO] - Using env device cuda:0.
+[2026-04-02 22:12:16,896][root][INFO] - Evaluating seed 0.
+[2026-04-02 22:12:16,898][root][INFO] - Using method PERACT_BC with type leader_follower
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+Weight: [60000]
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 22:12:52] INFO     INFO:root:eval_env:  _independent_env_runner.py:130
+                             Launching env.                                     
+                    INFO     INFO:root:Agent      _independent_env_runner.py:133
+                             information:                                       
+                    INFO     INFO:root:<yarr.agen _independent_env_runner.py:134
+                             ts.agent.LeaderFollo                               
+                             werAgent object at                                 
+                             0x7d94a7c0bc50>                                    
+QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
+[04/02/26 22:12:53] INFO     INFO:root:Using dual panda robot environment.py:119
+                    INFO     INFO:root:Setting control    arm_action_modes.py:79
+                             mode for both robots                               
+                    WARNING  WARNING:root:not sure how    task_environment.py:57
+                             _robot_shapes are used is                          
+                             used.                                              
+                    INFO     INFO:root:Evaluating _independent_env_runner.py:164
+                             weight 60000                                       
+loaded weights from /workspace/runs/anybimanual/task_eval_sweep/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_leader_layer_0.pt
+loaded weights from /workspace/runs/anybimanual/task_eval_sweep/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_follower_layer_0.pt
+
+
+                             Starting episode 0,                                
+                             seed 0.                                            
+eval_demo_seed:  0
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+/workspace/third_party/AnyBimanual/third_party/YARR/yarr/utils/rollout_generator.py:94: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:254.)
+  prepped_data = {k: torch.tensor([v], device=self._env_device) for k, v in obs_history.items()}
+
+
+[04/02/26 22:14:21] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 1,                                
+                             seed 1.                                            
+eval_demo_seed:  1
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+[04/02/26 22:15:45] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 2,                                
+                             seed 2.                                            
+eval_demo_seed:  2
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+[04/02/26 22:16:21] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 3,                                
+                             seed 3.                                            
+eval_demo_seed:  3
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+[04/02/26 22:17:38] INFO     INFO:root:eval_env:  _independent_env_runner.py:191
+                             Starting episode 4,                                
+                             seed 4.                                            
+eval_demo_seed:  4
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 7, (right=2,  task.py:370
+                             left=5)                                            
+
+
+
+Evaluating bimanual_sweep_to_dustpan | Episode 4 | Score: 0.0 | Lang Goal: sweep dirt to dustpan
+Finished bimanual_sweep_to_dustpan | Final Score: 0.0
+
+add_video needs package moviepy
+[04/02/26 22:19:07] INFO     INFO:root:Finished   _independent_env_runner.py:292
+                             evaluation.                                        
+[CoppeliaSim:loadinfo]   done.

reports/anybimanual_eval_logs/coordinated_lift_tray_ep1_live.log

@@ -0,0 +1,181 @@
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'eval': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
+  warnings.warn(msg, UserWarning)
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/PERACT_BC': Usage of deprecated keyword in package header '# @package _group_'.
+See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/changes_to_package_header for more information
+  deprecation_warning(
+/workspace/envs/anybi311/lib/python3.11/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
+See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
+  ret = run_job(
+[2026-04-02 22:09:13,243][root][INFO] - 
+method:
+  name: PERACT_BC
+  agent_type: leader_follower
+  robot_name: bimanual
+  image_crop_size: 64
+  bounds_offset:
+  - 0.15
+  voxel_sizes:
+  - 100
+  include_prev_layer: false
+  num_latents: 2048
+  latent_dim: 512
+  transformer_depth: 6
+  transformer_iterations: 1
+  cross_heads: 1
+  cross_dim_head: 64
+  latent_heads: 8
+  latent_dim_head: 64
+  pos_encoding_with_lang: true
+  conv_downsample: true
+  lang_fusion_type: seq
+  voxel_patch_size: 5
+  voxel_patch_stride: 5
+  final_dim: 64
+  low_dim_size: 4
+  input_dropout: 0.1
+  attn_dropout: 0.1
+  decoder_dropout: 0.0
+  lr: 0.0005
+  lr_scheduler: false
+  num_warmup_steps: 3000
+  optimizer: lamb
+  lambda_weight_l2: 1.0e-06
+  trans_loss_weight: 1.0
+  rot_loss_weight: 1.0
+  grip_loss_weight: 1.0
+  collision_loss_weight: 1.0
+  rotation_resolution: 5
+  activation: lrelu
+  norm: None
+  crop_augmentation: true
+  transform_augmentation:
+    apply_se3: true
+    aug_xyz:
+    - 0.125
+    - 0.125
+    - 0.125
+    aug_rpy:
+    - 0.0
+    - 0.0
+    - 45.0
+    aug_rot_resolution: ${method.rotation_resolution}
+  demo_augmentation: true
+  demo_augmentation_every_n: 10
+  no_skip_connection: false
+  no_perceiver: false
+  no_language: false
+  keypoint_method: heuristic
+rlbench:
+  task_name: per2+ab
+  tasks:
+  - coordinated_lift_tray
+  demo_path: /workspace/data/rlbench2
+  episode_length: 25
+  cameras:
+  - over_shoulder_left
+  - over_shoulder_right
+  - overhead
+  - wrist_right
+  - wrist_left
+  - front
+  camera_resolution:
+  - 256
+  - 256
+  scene_bounds:
+  - -0.3
+  - -0.5
+  - 0.6
+  - 0.7
+  - 0.5
+  - 1.6
+  include_lang_goal_in_obs: true
+  time_in_state: true
+  headless: true
+  gripper_mode: BimanualDiscrete
+  arm_action_mode: BimanualEndEffectorPoseViaPlanning
+  action_mode: BimanualMoveArmThenGripper
+framework:
+  tensorboard_logging: true
+  csv_logging: true
+  gpu: 0
+  logdir: /workspace/runs/anybimanual/peract_lf_eval
+  start_seed: 0
+  record_every_n: 5
+  eval_envs: 1
+  eval_from_eps_number: 0
+  eval_episodes: 1
+  eval_type: 60000
+  eval_save_metrics: true
+cinematic_recorder:
+  enabled: false
+  camera_resolution:
+  - 1280
+  - 720
+  fps: 30
+  rotate_speed: 0.005
+  save_path: /tmp/videos/
+
+[2026-04-02 22:09:13,251][root][INFO] - Using env device cuda:0.
+[2026-04-02 22:09:13,261][root][INFO] - Evaluating seed 0.
+[2026-04-02 22:09:13,261][root][INFO] - Using method PERACT_BC with type leader_follower
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+Weight: [60000]
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+/workspace/third_party/AnyBimanual/eval.py:183: UserWarning: 
+The version_base parameter is not specified.
+Please specify a compatability version level, or None.
+Will assume defaults for version 1.1
+  @hydra.main(config_name="eval", config_path="conf")
+[04/02/26 22:09:50] INFO     INFO:root:eval_env:  _independent_env_runner.py:130
+                             Launching env.                                     
+                    INFO     INFO:root:Agent      _independent_env_runner.py:133
+                             information:                                       
+                    INFO     INFO:root:<yarr.agen _independent_env_runner.py:134
+                             ts.agent.LeaderFollo                               
+                             werAgent object at                                 
+                             0x7dcccb11fbd0>                                    
+QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
+[04/02/26 22:09:52] INFO     INFO:root:Using dual panda robot environment.py:119
+                    INFO     INFO:root:Setting control    arm_action_modes.py:79
+                             mode for both robots                               
+                    WARNING  WARNING:root:not sure how    task_environment.py:57
+                             _robot_shapes are used is                          
+                             used.                                              
+                    INFO     INFO:root:Evaluating _independent_env_runner.py:164
+                             weight 60000                                       
+loaded weights from /workspace/runs/anybimanual/peract_lf_eval/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_leader_layer_0.pt
+loaded weights from /workspace/runs/anybimanual/peract_lf_eval/per2+ab/PERACT_BC/seed0/weights/60000/checkpoint_peract_bc_follower_layer_0.pt
+
+
+                             Starting episode 0,                                
+                             seed 0.                                            
+eval_demo_seed:  0
+                    INFO     INFO:root:Dataset root      rollout_generator.py:38
+                             missing or unavailable ();                         
+                             using env.reset() for eval.                        
+                    INFO     INFO:root:total waypoints 6, (right=3,  task.py:370
+                             left=3)                                            
+/workspace/third_party/AnyBimanual/third_party/YARR/yarr/utils/rollout_generator.py:94: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /pytorch/torch/csrc/utils/tensor_new.cpp:254.)
+  prepped_data = {k: torch.tensor([v], device=self._env_device) for k, v in obs_history.items()}
+
+
+
+Evaluating coordinated_lift_tray | Episode 0 | Score: 0.0 | Lang Goal: Lift the tray
+Finished coordinated_lift_tray | Final Score: unknown
+
+[04/02/26 22:10:34] INFO     INFO:root:Finished   _independent_env_runner.py:292
+                             evaluation.                                        
+[CoppeliaSim:loadinfo]   done.

reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_fold_tops__trunk_only_ft_seed17.json

@@ -0,0 +1,65 @@
+{
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "Fold Tops",
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_ik_hand_state",
+    "benchmark_task": "Fold Tops",
+    "cameras": [
+      "env_camera",
+      "garment_camera",
+      "object_camera"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_pointcloud_3cam",
+    "resolution": 224,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "dexgarmentlab",
+    "track_id": "dexgarmentlab_fold_tops"
+  },
+  "nonzero_metric_count": 4,
+  "raw_metrics": {
+    "affordance_feature_norm": 28.461267471313477,
+    "final_point_count": 2048,
+    "fold_boundary_area": 0.38797936088666146,
+    "garment_point_count": 2048,
+    "success": true,
+    "task_name": "Fold Tops"
+  },
+  "runtime_sec": 62.63576006889343,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "dexgarmentlab",
+  "task_success": true,
+  "timed_out": false,
+  "track_id": "dexgarmentlab_fold_tops",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "Fold Tops",
+    "dataset_split_id": "dexgarmentlab_fold_tops_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "dexgarmentlab",
+    "track_id": "dexgarmentlab_fold_tops",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_hang_coat__trunk_only_ft_seed17.json

@@ -0,0 +1,65 @@
+{
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "Hang Coat",
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_ik_hand_state",
+    "benchmark_task": "Hang Coat",
+    "cameras": [
+      "env_camera",
+      "garment_camera",
+      "object_camera"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_pointcloud_3cam",
+    "resolution": 224,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "dexgarmentlab",
+    "track_id": "dexgarmentlab_hang_coat"
+  },
+  "nonzero_metric_count": 4,
+  "raw_metrics": {
+    "affordance_feature_norm": 36.793704986572266,
+    "garment_center_height": 0.7110686898231506,
+    "garment_point_count": 2048,
+    "object_point_count": 2048,
+    "success": true,
+    "task_name": "Hang Coat"
+  },
+  "runtime_sec": 50.74257040023804,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "dexgarmentlab",
+  "task_success": true,
+  "timed_out": false,
+  "track_id": "dexgarmentlab_hang_coat",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "Hang Coat",
+    "dataset_split_id": "dexgarmentlab_hang_coat_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "dexgarmentlab",
+    "track_id": "dexgarmentlab_hang_coat",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_hybrid_smoke_summary.json

@@ -0,0 +1,18 @@
+{
+  "adapter_mode": "trunk_only_ft",
+  "failures": [],
+  "result_files": [
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_store_tops__trunk_only_ft_seed17.json",
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_fold_tops__trunk_only_ft_seed17.json",
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_hang_coat__trunk_only_ft_seed17.json"
+  ],
+  "seed": 17,
+  "smoke_pass_count": 3,
+  "suite": "dexgarmentlab",
+  "task_success_count": 3,
+  "tracks": [
+    "dexgarmentlab_store_tops",
+    "dexgarmentlab_fold_tops",
+    "dexgarmentlab_hang_coat"
+  ]
+}

reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_store_tops__trunk_only_ft_seed17.json

@@ -0,0 +1,66 @@
+{
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "Store Tops",
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_ik_hand_state",
+    "benchmark_task": "Store Tops",
+    "cameras": [
+      "env_camera",
+      "garment_camera",
+      "object_camera"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_pointcloud_3cam",
+    "resolution": 224,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "dexgarmentlab",
+    "track_id": "dexgarmentlab_store_tops"
+  },
+  "nonzero_metric_count": 5,
+  "raw_metrics": {
+    "affordance_feature_norm": 26.555341720581055,
+    "distance_to_target_center": 0.0008265018479589052,
+    "garment_point_count": 2048,
+    "judge_point_count": 2048,
+    "object_point_count": 2048,
+    "success": true,
+    "task_name": "Store Tops"
+  },
+  "runtime_sec": 68.48123931884766,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "dexgarmentlab",
+  "task_success": true,
+  "timed_out": false,
+  "track_id": "dexgarmentlab_store_tops",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "Store Tops",
+    "dataset_split_id": "dexgarmentlab_store_tops_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "dexgarmentlab",
+    "track_id": "dexgarmentlab_store_tops",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/hybrid_public_benchmark_smoke_summary.json

@@ -0,0 +1,85 @@
+{
+  "adapter_mode": "trunk_only_ft",
+  "failures": [],
+  "seed": 17,
+  "suite_summaries": {
+    "dexgarmentlab": {
+      "returncode": 0,
+      "stderr_tail": [],
+      "stdout_tail": [
+        "{",
+        "  \"summary_path\": \"/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_hybrid_smoke_summary.json\",",
+        "  \"smoke_pass_count\": 3",
+        "}"
+      ],
+      "summary": {
+        "adapter_mode": "trunk_only_ft",
+        "failures": [],
+        "result_files": [
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_store_tops__trunk_only_ft_seed17.json",
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_fold_tops__trunk_only_ft_seed17.json",
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_hang_coat__trunk_only_ft_seed17.json"
+        ],
+        "seed": 17,
+        "smoke_pass_count": 3,
+        "suite": "dexgarmentlab",
+        "task_success_count": 3,
+        "tracks": [
+          "dexgarmentlab_store_tops",
+          "dexgarmentlab_fold_tops",
+          "dexgarmentlab_hang_coat"
+        ]
+      },
+      "summary_path": "/workspace/reports/public_hybrid_benchmark_smoke_v1/dexgarmentlab/dexgarmentlab_hybrid_smoke_summary.json"
+    },
+    "rlbench2": {
+      "returncode": 0,
+      "stderr_tail": [],
+      "stdout_tail": [
+        "WARNING: QApplication was not created in the main() thread.",
+        "WARNING:root:not sure how _robot_shapes are used is used.",
+        "ERROR:root:robot is in collision",
+        "ERROR:root:Error when checking waypoints. Exception is: ",
+        "WARNING: QApplication was not created in the main() thread.",
+        "WARNING:root:not sure how _robot_shapes are used is used.",
+        "WARNING: QApplication was not created in the main() thread.",
+        "WARNING:root:not sure how _robot_shapes are used is used.",
+        "WARNING: QApplication was not created in the main() thread.",
+        "WARNING:root:not sure how _robot_shapes are used is used.",
+        "[ 0.05       -0.27500001  0.75199997]",
+        "{",
+        "  \"summary_path\": \"/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench_hybrid_smoke_summary.json\",",
+        "  \"smoke_pass_count\": 5",
+        "}",
+        "[CoppeliaSim:loadinfo]   done.",
+        "[CoppeliaSim:loadinfo]   done.",
+        "[CoppeliaSim:loadinfo]   done.",
+        "[CoppeliaSim:loadinfo]   done.",
+        "[CoppeliaSim:loadinfo]   done."
+      ],
+      "summary": {
+        "adapter_mode": "trunk_only_ft",
+        "failures": [],
+        "result_files": [
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_put_bottle_in_fridge__trunk_only_ft_seed17.json",
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_take_out_tray__trunk_only_ft_seed17.json",
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_lift_tray__trunk_only_ft_seed17.json",
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_straighten_rope__trunk_only_ft_seed17.json",
+          "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_sweep_to_dustpan__trunk_only_ft_seed17.json"
+        ],
+        "seed": 17,
+        "smoke_pass_count": 5,
+        "suite": "rlbench2",
+        "task_success_count": 0,
+        "tracks": [
+          "rlbench2_put_bottle_in_fridge",
+          "rlbench2_take_out_tray",
+          "rlbench2_lift_tray",
+          "rlbench2_straighten_rope",
+          "rlbench2_sweep_to_dustpan"
+        ]
+      },
+      "summary_path": "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench_hybrid_smoke_summary.json"
+    }
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_lift_tray__trunk_only_ft_seed17.json

@@ -0,0 +1,61 @@
+{
+  "action_translation_norm": 2.146094799041748,
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "bimanual_lift_tray",
+  "description_count": 1,
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_pose_then_gripper",
+    "benchmark_task": "bimanual_lift_tray",
+    "cameras": [
+      "front",
+      "wrist_left",
+      "wrist_right"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_3cam",
+    "resolution": 256,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_lift_tray"
+  },
+  "left_pose_norm": 1.5036476850509644,
+  "reward": 0.0,
+  "right_pose_norm": 1.5118523836135864,
+  "runtime_sec": 1.2487857341766357,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "rlbench2",
+  "task_success": false,
+  "terminate": false,
+  "track_id": "rlbench2_lift_tray",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "bimanual_lift_tray",
+    "dataset_split_id": "rlbench2_lift_tray_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_lift_tray",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_put_bottle_in_fridge__trunk_only_ft_seed17.json

@@ -0,0 +1,61 @@
+{
+  "action_translation_norm": 2.146069288253784,
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "bimanual_put_bottle_in_fridge",
+  "description_count": 4,
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_pose_then_gripper",
+    "benchmark_task": "bimanual_put_bottle_in_fridge",
+    "cameras": [
+      "front",
+      "wrist_left",
+      "wrist_right"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_3cam",
+    "resolution": 256,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_put_bottle_in_fridge"
+  },
+  "left_pose_norm": 1.5036077499389648,
+  "reward": 0.0,
+  "right_pose_norm": 1.511855959892273,
+  "runtime_sec": 2.537095785140991,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "rlbench2",
+  "task_success": false,
+  "terminate": false,
+  "track_id": "rlbench2_put_bottle_in_fridge",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "bimanual_put_bottle_in_fridge",
+    "dataset_split_id": "rlbench2_put_bottle_in_fridge_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_put_bottle_in_fridge",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_straighten_rope__trunk_only_ft_seed17.json

@@ -0,0 +1,61 @@
+{
+  "action_translation_norm": 2.1460282802581787,
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "bimanual_straighten_rope",
+  "description_count": 6,
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_pose_then_gripper",
+    "benchmark_task": "bimanual_straighten_rope",
+    "cameras": [
+      "front",
+      "wrist_left",
+      "wrist_right"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_3cam",
+    "resolution": 256,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_straighten_rope"
+  },
+  "left_pose_norm": 1.5036031007766724,
+  "reward": 0.0,
+  "right_pose_norm": 1.5118026733398438,
+  "runtime_sec": 1.4220247268676758,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "rlbench2",
+  "task_success": false,
+  "terminate": false,
+  "track_id": "rlbench2_straighten_rope",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "bimanual_straighten_rope",
+    "dataset_split_id": "rlbench2_straighten_rope_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_straighten_rope",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_sweep_to_dustpan__trunk_only_ft_seed17.json

@@ -0,0 +1,61 @@
+{
+  "action_translation_norm": 2.1460604667663574,
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "bimanual_sweep_to_dustpan",
+  "description_count": 7,
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_pose_then_gripper",
+    "benchmark_task": "bimanual_sweep_to_dustpan",
+    "cameras": [
+      "front",
+      "wrist_left",
+      "wrist_right"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_3cam",
+    "resolution": 256,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_sweep_to_dustpan"
+  },
+  "left_pose_norm": 1.503666877746582,
+  "reward": 0.0,
+  "right_pose_norm": 1.5117847919464111,
+  "runtime_sec": 1.2797048091888428,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "rlbench2",
+  "task_success": false,
+  "terminate": false,
+  "track_id": "rlbench2_sweep_to_dustpan",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "bimanual_sweep_to_dustpan",
+    "dataset_split_id": "rlbench2_sweep_to_dustpan_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_sweep_to_dustpan",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_take_out_tray__trunk_only_ft_seed17.json

@@ -0,0 +1,61 @@
+{
+  "action_translation_norm": 2.1460111141204834,
+  "adapter_mode": "trunk_only_ft",
+  "benchmark_task": "bimanual_take_tray_out_of_oven",
+  "description_count": 7,
+  "episodes": 1,
+  "eval_protocol": {
+    "action_horizon": 8,
+    "action_space": "bimanual_pose_then_gripper",
+    "benchmark_task": "bimanual_take_tray_out_of_oven",
+    "cameras": [
+      "front",
+      "wrist_left",
+      "wrist_right"
+    ],
+    "episodes": 1,
+    "eval_mode": "trunk_only_ft",
+    "observation_stack": "rgbd_3cam",
+    "resolution": 256,
+    "role": "target",
+    "same_test_episodes": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_take_out_tray"
+  },
+  "left_pose_norm": 1.5035772323608398,
+  "reward": 0.0,
+  "right_pose_norm": 1.5118043422698975,
+  "runtime_sec": 1.6043481826782227,
+  "seed": 17,
+  "smoke_pass": true,
+  "success_rate": 1.0,
+  "successes": [
+    1
+  ],
+  "suite": "rlbench2",
+  "task_success": false,
+  "terminate": false,
+  "track_id": "rlbench2_take_out_tray",
+  "train_spec": {
+    "augmentations": "matched_rgbd_aug_v1",
+    "batch_size": 32,
+    "benchmark_task": "bimanual_take_tray_out_of_oven",
+    "dataset_split_id": "rlbench2_take_out_tray_shared_split_seed17",
+    "early_stopping_metric": "val_success",
+    "init_checkpoint_group": "shared_public_trunk",
+    "learning_rate": 0.0003,
+    "lr_schedule": "cosine",
+    "max_gradient_steps": 20000,
+    "model_variant": "trunk_only_ft",
+    "optimizer": "adamw",
+    "same_data_policy": true,
+    "same_init_policy": true,
+    "seed": 17,
+    "suite": "rlbench2",
+    "track_id": "rlbench2_take_out_tray",
+    "train_demos": 64,
+    "unfreeze_scope": "matched_trunk_scope",
+    "val_demos": 16
+  }
+}

reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench_hybrid_smoke_summary.json

@@ -0,0 +1,22 @@
+{
+  "adapter_mode": "trunk_only_ft",
+  "failures": [],
+  "result_files": [
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_put_bottle_in_fridge__trunk_only_ft_seed17.json",
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_take_out_tray__trunk_only_ft_seed17.json",
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_lift_tray__trunk_only_ft_seed17.json",
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_straighten_rope__trunk_only_ft_seed17.json",
+    "/workspace/reports/public_hybrid_benchmark_smoke_v1/rlbench2/rlbench2_sweep_to_dustpan__trunk_only_ft_seed17.json"
+  ],
+  "seed": 17,
+  "smoke_pass_count": 5,
+  "suite": "rlbench2",
+  "task_success_count": 0,
+  "tracks": [
+    "rlbench2_put_bottle_in_fridge",
+    "rlbench2_take_out_tray",
+    "rlbench2_lift_tray",
+    "rlbench2_straighten_rope",
+    "rlbench2_sweep_to_dustpan"
+  ]
+}

scripts/run_anybimanual_task_eval.sh

@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+TASK="${1:?usage: run_anybimanual_task_eval.sh <task_name> [episodes] [gpu]}"
+EPISODES="${2:-5}"
+GPU="${3:-0}"
+DEMO_PATH="${DEMO_PATH:-/workspace/data/rlbench2}"
+RUN_LOGDIR="${RUN_LOGDIR:-/workspace/runs/anybimanual/peract_lf_eval}"
+
+ROOT="/workspace/third_party/AnyBimanual"
+ENV_DIR="/workspace/envs/anybi311"
+export HOME="/workspace"
+export HF_HOME="/workspace/.hf"
+export TORCH_HOME="/workspace/.cache/torch"
+export COPPELIASIM_ROOT="/workspace/assets/coppeliasim_v4_1_0"
+export LD_LIBRARY_PATH="${LD_LIBRARY_PATH:-}:$COPPELIASIM_ROOT"
+export QT_QPA_PLATFORM_PLUGIN_PATH="$COPPELIASIM_ROOT"
+export MESA_GL_VERSION_OVERRIDE="4.1"
+export PYOPENGL_PLATFORM="egl"
+export PYTHONPATH="$ROOT:$ROOT/third_party/RLBench:$ROOT/third_party/YARR:$ROOT/third_party/PyRep:$ROOT/third_party/pytorch3d:${PYTHONPATH:-}"
+
+source "$ENV_DIR/bin/activate"
+cd "$ROOT"
+
+xvfb-run -a python eval.py \
+  method=PERACT_BC \
+  framework.logdir="$RUN_LOGDIR" \
+  rlbench.task_name='per2+ab' \
+  rlbench.demo_path="$DEMO_PATH" \
+  rlbench.gripper_mode='BimanualDiscrete' \
+  rlbench.arm_action_mode='BimanualEndEffectorPoseViaPlanning' \
+  rlbench.action_mode='BimanualMoveArmThenGripper' \
+  framework.start_seed=0 \
+  framework.eval_type=60000 \
+  framework.eval_episodes="$EPISODES" \
+  framework.eval_envs=1 \
+  framework.eval_save_metrics=true \
+  framework.gpu="$GPU" \
+  rlbench.headless=true \
+  cinematic_recorder.enabled=false \
+  "rlbench.tasks=[$TASK]"

third_party/DexGarmentLab/Data/Fold_Tops_Validation_HALO/validation_log.txt

@@ -0,0 +1,2 @@
+result:False  usd_path:Assets/Garment/Tops/NoCollar_Lsleeve_FrontClose/TNLC_Top441/TNLC_Top441_obj.usd  pos_x:-0.08444962764383653  pos_y:0.7054448732479126
+result:True  usd_path:Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_018/TCLC_018_obj.usd  pos_x:0.0  pos_y:0.8

third_party/DexGarmentLab/Data/Hang_Coat_Validation_HALO/validation_log.txt

@@ -0,0 +1,2 @@
+result:False  usd_path:Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Jacket131/TCLO_Jacket131_obj.usd  pos_x:0.0  pos_y:0.7  env_dx:0.0  env_dy:0.0 
+result:True  usd_path:Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Jacket140/TCLO_Jacket140_obj.usd  pos_x:-0.038650226833422834  pos_y:0.6444166676733603  env_dx:-0.24097593635609782  env_dy:-0.24736420450451033 

third_party/DexGarmentLab/Data/Store_Tops_Validation_HALO/validation_log.txt

@@ -0,0 +1 @@
+result:True  usd_path:Assets/Garment/Tops/NoCollar_noSleeve_FrontClose/TNNC_Top394/TNNC_Top394_obj.usd  pos_x:0.012405415000857324  pos_y:0.6967481309275767  env_dx:0.1354035858748196  env_dy:-0.01403608869312245 

third_party/DexGarmentLab/Env_StandAlone/Fold_Tops_Env.py

@@ -0,0 +1,440 @@
+import os
+import json
+from isaacsim import SimulationApp
+
+
+def _env_flag(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name)
+    if value is None:
+        return default
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
+HEADLESS = _env_flag("DEXGARMENTLAB_HEADLESS", False)
+EMIT_JSON = _env_flag("DEXGARMENTLAB_EMIT_JSON", False)
+RESULT_PREFIX = "DEXGARMENTLAB_RESULT="
+simulation_app = SimulationApp({"headless": HEADLESS})
+
+# load external package
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_record
+from Env_Config.Utils_Project.Position_Judge import judge_pcd
+from Env_Config.Room.Object_Tools import set_prim_visible_group, delete_prim_group
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+
+class FoldTops_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+    ):
+        # load BaseEnv
+        super().__init__()
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path=os.getcwd() + "/" + "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_018/TCLC_018_obj.usd" if usd_path is None else usd_path,
+            contact_offset=0.012,             
+            rest_offset=0.010,                
+            particle_contact_offset=0.012,    
+            fluid_rest_offset=0.010,
+            solid_rest_offset=0.010,
+        )
+        # Here are some example garments you can try:
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Jacket032/TCLC_Jacket032_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Jacket152/TCLC_Jacket152_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top566/TCLC_Top566_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top584/TCLC_Top584_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_top118/TCLC_top118_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top476/TCLC_Top476_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top030/TCLC_Top030_obj.usd",  
+
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.8, 0.0, 0.5]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.8, 0.0, 0.5]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([0.0, 1.0, 6.75]), 
+            camera_orientation=np.array([0, 90.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 4.0, 6.0]),
+            camera_orientation=np.array([0, 60, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.garment_pcd = None
+        self.points_affordance_feature = None
+        
+        # load GAM Model
+        self.model = GAM_Encapsulation(catogory="Tops_LongSleeve")   
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+        
+        # move garment to the target position
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.2]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+        
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment"
+            ]
+        )
+        
+        self.env_camera.initialize(depth_enable=True)
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+        
+        # open hand to be initial state
+        self.bimanual_dex.set_both_hand_state("open", "open")
+
+        # step world to make it ready
+        for i in range(100):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        
+        cprint("World Ready!", "green", "on_green")
+    
+    def record_callback(self, step_size):
+
+        if self.step_num % 5 == 0:
+        
+            joint_pos_L = self.bimanual_dex.dexleft.get_joint_positions()
+            
+            joint_pos_R = self.bimanual_dex.dexright.get_joint_positions()
+            
+            joint_state = np.array([*joint_pos_L, *joint_pos_R])
+
+            rgb = self.env_camera.get_rgb_graph(save_or_not=False)
+
+            point_cloud = self.env_camera.get_pointcloud_from_depth(
+                show_original_pc_online=False,
+                show_downsample_pc_online=False,
+            )
+            
+            self.saving_data.append({ 
+                "joint_state": joint_state,
+                "image": rgb,
+                "env_point_cloud": point_cloud,
+                "garment_point_cloud":self.garment_pcd,
+                "points_affordance_feature": self.points_affordance_feature,
+            })
+        
+        self.step_num += 1
+
+
+def FoldTops(pos, ori, usd_path, ground_material_usd, data_collection_flag, record_video_flag):
+    
+    env = FoldTops_Env(pos, ori, usd_path, ground_material_usd, record_video_flag)
+    
+    if record_video_flag:
+        env.thread_record.start()
+    
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+        
+    pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        real_time_watch=False,
+    )
+    env.garment_pcd=pcd
+    
+    # unhide
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=pcd, index_list=[957, 501, 1902, 448, 1196, 422])
+
+    manipulation_points[0:4, 2] = 0.02
+    manipulation_points[4:, 2] = 0.0
+    
+    # ---------------------- left hand ---------------------- #
+    
+    env.points_affordance_feature = normalize_columns(np.concatenate([points_similarity[0:1], points_similarity[0:1]], axis=0).T)
+        
+    env.bimanual_dex.dexleft.dense_step_action(target_pos=manipulation_points[0], target_ori=np.array([0.579, -0.579, -0.406, 0.406]), angular_type="quat")
+    
+    if data_collection_flag:
+        for i in range(20):
+            env.step()
+        env.record(task_name="Fold_Tops", stage_index=1)
+    
+    env.bimanual_dex.set_both_hand_state(left_hand_state="close", right_hand_state="None")
+    
+    left_sleeve_height = min(np.linalg.norm(manipulation_points[0][:2] - manipulation_points[3][:2]), 0.3)
+    
+    # print("left_sleeve_height: ", left_sleeve_height)
+
+    lift_point_1 = np.array([manipulation_points[0][0], manipulation_points[0][1], left_sleeve_height])
+
+    env.bimanual_dex.dexleft.dense_step_action(target_pos=lift_point_1, target_ori=np.array([0.579, -0.579, -0.406, 0.406]), angular_type="quat")
+    
+    lift_point_2 = np.array([manipulation_points[1][0], manipulation_points[1][1], left_sleeve_height])
+    
+    env.bimanual_dex.dexleft.dense_step_action(target_pos=lift_point_2, target_ori=np.array([0.579, -0.579, -0.406, 0.406]), angular_type="quat")
+
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="None")
+    
+    if data_collection_flag:
+        env.stop_record()
+    
+    env.garment.particle_material.set_gravity_scale(10.0)
+    for i in range(200):
+        env.step()
+    env.garment.particle_material.set_gravity_scale(1.0) 
+    
+    
+    
+    env.bimanual_dex.dexleft.dense_step_action(target_pos=np.array([-0.6, 0.8, 0.5]), target_ori=np.array([0.579, -0.579, -0.406, 0.406]), angular_type="quat")
+
+    
+    # --------------------- right hand --------------------- #
+
+    env.points_affordance_feature = normalize_columns(np.concatenate([points_similarity[2:3], points_similarity[2:3]], axis=0).T)
+            
+    env.bimanual_dex.dexright.dense_step_action(target_pos=manipulation_points[2], target_ori=np.array([0.406, -0.406, -0.579, 0.579]), angular_type="quat")
+
+    if data_collection_flag:
+        for i in range(20):
+            env.step()
+        env.record(task_name="Fold_Tops", stage_index=2)
+            
+    env.bimanual_dex.set_both_hand_state(left_hand_state="None", right_hand_state="close")
+    
+    right_sleeve_height = min(np.linalg.norm(manipulation_points[2][:2] - manipulation_points[1][:2]), 0.3)
+    
+    # print("right_sleeve_height: ", right_sleeve_height)
+    
+    lift_point_1 = np.array([manipulation_points[2][0], manipulation_points[2][1], right_sleeve_height])
+    
+    env.bimanual_dex.dexright.dense_step_action(target_pos=lift_point_1, target_ori=np.array([0.406, -0.406, -0.579, 0.579]), angular_type="quat")
+    
+    lift_point_2 = np.array([manipulation_points[3][0], manipulation_points[3][1], right_sleeve_height])
+    
+    env.bimanual_dex.dexright.dense_step_action(target_pos=lift_point_2, target_ori=np.array([0.406, -0.406, -0.579, 0.579]), angular_type="quat")
+
+    env.bimanual_dex.set_both_hand_state(left_hand_state="None", right_hand_state="open")
+    
+    if data_collection_flag:
+        env.stop_record()
+    
+    env.garment.particle_material.set_gravity_scale(10.0)
+    for i in range(200):
+        env.step()
+    env.garment.particle_material.set_gravity_scale(1.0) 
+    
+    
+    
+    env.bimanual_dex.dexright.dense_step_action(target_pos=np.array([0.6, 0.8, 0.5]), target_ori=np.array([0.406, -0.406, -0.579, 0.579]), angular_type="quat")
+    
+    # --------------------- bottom-top --------------------- #    
+    
+    env.points_affordance_feature = normalize_columns(points_similarity[4:6].T)   
+   
+    env.bimanual_dex.dense_move_both_ik(
+        left_pos=manipulation_points[4], 
+        left_ori=np.array([0.579, -0.579, -0.406, 0.406]),
+        right_pos=manipulation_points[5],
+        right_ori=np.array([0.406, -0.406, -0.579, 0.579]),
+    )
+    
+    if data_collection_flag:
+        for i in range(20):
+            env.step()
+        env.record(task_name="Fold_Tops", stage_index=3)
+    
+    env.bimanual_dex.set_both_hand_state(left_hand_state="close", right_hand_state="close")
+    
+    lift_height = manipulation_points[3][1] - manipulation_points[4][1]
+    
+    # print("lift_height: ", lift_height)
+    
+    lift_point_1 = np.array([manipulation_points[4][0], manipulation_points[4][1], lift_height/2])
+    lift_point_2 = np.array([manipulation_points[5][0], manipulation_points[5][1], lift_height/2])
+    
+    env.bimanual_dex.dense_move_both_ik(
+        left_pos=lift_point_1,
+        left_ori=np.array([0.579, -0.579, -0.406, 0.406]),
+        right_pos=lift_point_2,
+        right_ori=np.array([0.406, -0.406, -0.579, 0.579]),
+    )
+    
+    push_point_1 = np.array([manipulation_points[3][0], manipulation_points[3][1]+0.1, min(lift_height/2, 0.2)])
+    push_point_2 = np.array([manipulation_points[1][0], manipulation_points[1][1]+0.1, min(lift_height/2, 0.2)])
+    
+    env.bimanual_dex.dense_move_both_ik(
+        left_pos=push_point_1,
+        left_ori=np.array([0.579, -0.579, -0.406, 0.406]),
+        right_pos=push_point_2,
+        right_ori=np.array([0.406, -0.406, -0.579, 0.579]),
+    )
+    
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="open")
+    
+    if data_collection_flag:
+        env.stop_record()
+    
+    env.garment.particle_material.set_gravity_scale(10.0)
+    for i in range(100):
+        env.step()
+    env.garment.particle_material.set_gravity_scale(1.0)
+    
+    dexleft_prim = prims_utils.get_prim_at_path("/World/DexLeft")
+    dexright_prim = prims_utils.get_prim_at_path("/World/DexRight")
+    set_prim_visibility(dexleft_prim, False)
+    set_prim_visibility(dexright_prim, False)
+    
+    for i in range(50):
+        env.step()   
+        
+    success=True
+    points,*_=env.model.get_manipulation_points(pcd,[554,1540,1014,1385])
+    boundary=[points[0][0]-0.05,points[1][0]+0.05,points[3][1]-0.1,points[2][1]+0.1]
+    pcd_end,_=env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        real_time_watch=False,
+    )
+    success=judge_pcd(pcd_end,boundary,threshold=0.12)
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+    
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag and success:
+        if not os.path.exists("Data/Fold_Tops/video"):
+            os.makedirs("Data/Fold_Tops/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Fold_Tops/video/video", ".mp4"))
+
+    if data_collection_flag:
+        # write into .log file
+        with open("Data/Fold_Tops/data_collection_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}\n")
+        if success:
+            env.record_to_npz()
+            if not os.path.exists("Data/Fold_Tops/final_state_pic"):
+                os.makedirs("Data/Fold_Tops/final_state_pic")
+            env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Fold_Tops/final_state_pic/img",".png"))
+    result = {
+        "task_name": "Fold Tops",
+        "success": bool(success),
+        "garment_point_count": int(env.garment_pcd.shape[0]) if env.garment_pcd is not None else 0,
+        "final_point_count": int(pcd_end.shape[0]),
+        "affordance_feature_norm": float(np.linalg.norm(env.points_affordance_feature)) if env.points_affordance_feature is not None else 0.0,
+        "fold_boundary_area": float(max(boundary[1] - boundary[0], 0.0) * max(boundary[3] - boundary[2], 0.0)),
+    }
+    return result
+
+   
+if __name__=="__main__":
+    
+    args=parse_args_record()
+    
+    # initial setting
+    pos = np.array([0.0, 0.8, 0.2])
+    ori = np.array([0.0, 0.0, 0.0])
+    usd_path = None
+    
+    if args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        x = np.random.uniform(-0.1, 0.1) # changeable
+        y = np.random.uniform(0.7, 0.9) # changeable
+        pos = np.array([x,y,0.0])
+        ori = np.array([0.0, 0.0, 0.0])
+        Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+        assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Tops_LongSleeve/assets_training_list.txt")
+        assets_list = []
+        with open(assets_lists,"r",encoding='utf-8') as f:
+            for line in f:
+                clean_line = line.rstrip('\n')
+                assets_list.append(clean_line)
+        usd_path=os.getcwd() + "/" + np.random.choice(assets_list)
+    
+    result = FoldTops(pos, ori, usd_path, args.ground_material_usd, args.data_collection_flag, args.record_video_flag)
+    if EMIT_JSON:
+        print(f"{RESULT_PREFIX}{json.dumps(result, sort_keys=True)}")
+
+    if args.data_collection_flag:
+        simulation_app.close()
+    elif HEADLESS:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+    
+simulation_app.close()

third_party/DexGarmentLab/Env_StandAlone/Hang_Coat_Env.py

@@ -0,0 +1,417 @@
+import os
+import json
+from isaacsim import SimulationApp
+
+
+def _env_flag(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name)
+    if value is None:
+        return default
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
+HEADLESS = _env_flag("DEXGARMENTLAB_HEADLESS", False)
+EMIT_JSON = _env_flag("DEXGARMENTLAB_EMIT_JSON", False)
+RESULT_PREFIX = "DEXGARMENTLAB_RESULT="
+simulation_app = SimulationApp({"headless": HEADLESS})
+
+# load external package
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import pothook_load, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns, plot_column_distributions
+from Env_Config.Utils_Project.Parse import parse_args_record
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+
+class HangCoat_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+    ):
+        # load BaseEnv
+        super().__init__()
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path=os.getcwd() + "/" + "Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Jacket131/TCLO_Jacket131_obj.usd" if usd_path is None else usd_path,
+            friction=25.0,
+            contact_offset=0.015,             
+            rest_offset=0.012,                
+            particle_contact_offset=0.015,    
+            fluid_rest_offset=0.012,
+            solid_rest_offset=0.012,
+        )
+        # Here are some example garments you can try:
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Shirt025/TCLO_Shirt025_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Jacket037/TCLO_Jacket037_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Jacket140/TCLO_Jacket140_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontOpen/TCLO_Jacket131/TCLO_Jacket131_obj.usd"
+        # "Assets/Garment/Tops/Hooded_Lsleeve_FrontOpen/THLO_Jacket065/THLO_Jacket065_obj.usd"
+        
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.8, 0.0, 0.6]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.8, 0.0, 0.6]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+        
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([0.0, -3.0, 6.75]), 
+            camera_orientation=np.array([0, 60.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 6.65, 4.0]),
+            camera_orientation=np.array([0, 30.0, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+        
+        self.garment_pcd = None
+        self.object_pcd = None
+        self.points_affordance_feature = None
+        
+        # load GAM Model
+        self.model = GAM_Encapsulation(catogory="Tops_FrontOpen")   
+
+        # load hanger
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.pothook_center = pothook_load(self.scene, env_dx, env_dy)
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+        
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        # initialize gif camera to obtain rgb with the aim of creating gif
+        self.env_camera.initialize(
+            depth_enable=True,
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/pothook1",
+                "/World/pothook2",
+                "/World/pothook3",
+            ]
+        )
+
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+
+        # move garment to the target position
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.2]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+        
+        # open hand to be initial state
+        self.bimanual_dex.set_both_hand_state("open", "open")
+
+        # step world to make it ready
+        for i in range(200):
+            self.step()
+            
+        cprint("[CONFIG]----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"[CONFIG]usd_path: {usd_path}", "magenta")
+        cprint(f"[CONFIG]pos_x: {pos[0]}", "magenta")
+        cprint(f"[CONFIG]pos_y: {pos[1]}", "magenta")
+        cprint(f"[CONFIG]env_dx: {env_dx}", "magenta")
+        cprint(f"[CONFIG]env_dy: {env_dy}", "magenta")
+        cprint("[CONFIG]----------- World Configuration -----------", color="magenta", attrs=["bold"])
+
+        cprint("World Ready!", "green", "on_green")
+    
+    def record_callback(self, step_size):
+
+        if self.step_num % 5 == 0:
+        
+            joint_pos_L = self.bimanual_dex.dexleft.get_joint_positions()
+            
+            joint_pos_R = self.bimanual_dex.dexright.get_joint_positions()
+            
+            joint_state = np.array([*joint_pos_L, *joint_pos_R])
+
+            rgb = self.env_camera.get_rgb_graph(save_or_not=False)
+
+            point_cloud = self.env_camera.get_pointcloud_from_depth(
+                show_original_pc_online=False,
+                show_downsample_pc_online=False,
+            )
+            
+            self.saving_data.append({ 
+                "joint_state": joint_state,
+                "image": rgb,
+                "env_point_cloud": point_cloud,
+                "garment_point_cloud": self.garment_pcd,
+                "object_point_cloud": self.object_pcd,
+                "points_affordance_feature": self.points_affordance_feature,
+            })
+        
+        self.step_num += 1
+     
+def HangCoat(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, data_collection_flag, record_video_flag):
+    
+    env = HangCoat_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag)
+    
+    env.garment.particle_material.set_gravity_scale(0.7)
+    
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        real_time_watch=False,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pothook1", "/World/pothook2", "/World/pothook3"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+        
+    pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        real_time_watch=False,
+    )
+    env.garment_pcd=pcd
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pothook1", "/World/pothook2", "/World/pothook3"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    pcd_rotate = rotate_point_cloud(pcd, euler_angles=np.array([0, 0, 180]), center_point=env.garment.get_garment_center_pos())     
+
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=pcd_rotate, index_list=[793, 1805])
+    manipulation_points=pcd[indices]
+    manipulation_points[:, 2] = 0.0
+    
+    env.points_affordance_feature = normalize_columns(points_similarity.T)    
+    
+    garment_boundary_points, boundary_indices, _ = env.model.get_manipulation_points(input_pcd=pcd_rotate, index_list=[561, 1776])
+    garment_boundary_points = pcd[boundary_indices]
+    garment_length = abs(garment_boundary_points[0][1] - garment_boundary_points[1][1])
+    lift_height = garment_length * 0.35 + env.pothook_center[2]
+
+    if record_video_flag:
+        env.thread_record.start()
+        
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[1], left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=manipulation_points[0], right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+    
+    if data_collection_flag:
+        for i in range(20):
+            env.step()
+        env.record(task_name="Hang_Coat", stage_index=1)
+
+    env.bimanual_dex.set_both_hand_state(left_hand_state="close", right_hand_state="close")
+    manipulation_points[1][0]-=0.15
+    manipulation_points[0][0]+=0.15
+    manipulation_points[1][1]+=0.2
+    manipulation_points[0][1]+=0.2
+    manipulation_points[1][2]+=0.2
+    manipulation_points[0][2]+=0.2
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[1], left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=manipulation_points[0], right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    center_pos = env.pothook_center
+
+    left_pos_f = np.array([center_pos[0]-0.2, center_pos[1]-0.3, lift_height])
+    right_pos_f = np.array([center_pos[0]+0.2, center_pos[1]-0.3, lift_height])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_pos_f, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_pos_f, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+    
+    left_pos_f = np.array([center_pos[0]-0.2, center_pos[1]+0.15, lift_height])
+    right_pos_f = np.array([center_pos[0]+0.2, center_pos[1]+0.15, lift_height])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_pos_f, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_pos_f, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    left_pos_f = np.array([center_pos[0]-0.2, center_pos[1]+0.175, center_pos[2]])
+    right_pos_f = np.array([center_pos[0]+0.2, center_pos[1]+0.175, center_pos[2]])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_pos_f, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_pos_f, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="open")
+    
+    if data_collection_flag:
+       env.stop_record()
+
+    env.garment.particle_material.set_gravity_scale(2.0)
+    
+    for i in range(100):
+        env.step()
+        
+    env.garment.particle_material.set_gravity_scale(0.7)   
+        
+    # make prim visible
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+        
+    success=True
+    
+    cprint("[INFO]----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"[INFO]garment_center_height: {env.garment.get_garment_center_pos()[2]}", "blue")
+    cprint("[INFO]----------- Judge End -----------", "blue", attrs=["bold"])
+    success=env.garment.get_garment_center_pos()[2]>0.50 and env.garment.get_garment_center_pos()[2]<2.0
+    cprint(f"[INFO]final result: {success}", color="green", on_color="on_green")
+    
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag and success:
+        if not os.path.exists("Data/Hang_Coat/video"):
+            os.makedirs("Data/Hang_Coat/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Hang_Coat/video/video", ".mp4"))
+    
+    if data_collection_flag:
+        # write into .log file
+        with open("Data/Hang_Coat/data_collection_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+        
+    if data_collection_flag and success:
+        env.record_to_npz(env_change=True)
+        if not os.path.exists("Data/Hang_Coat/final_state_pic"):
+            os.makedirs("Data/Hang_Coat/final_state_pic")
+            env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Hang_Coat/final_state_pic/img",".png"))
+    result = {
+        "task_name": "Hang Coat",
+        "success": bool(success),
+        "garment_point_count": int(env.garment_pcd.shape[0]) if env.garment_pcd is not None else 0,
+        "object_point_count": int(env.object_pcd.shape[0]) if env.object_pcd is not None else 0,
+        "affordance_feature_norm": float(np.linalg.norm(env.points_affordance_feature)) if env.points_affordance_feature is not None else 0.0,
+        "garment_center_height": float(env.garment.get_garment_center_pos()[2]),
+    }
+    return result
+
+
+if __name__=="__main__":
+    
+    args = parse_args_record()
+    
+    # initial setting
+    pos = np.array([0, 0.7, 0.2])
+    ori = np.array([0.0, 0.0, 180.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0 
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.25, 0.25) # changeable
+            env_dy = np.random.uniform(-0.3, -0.05) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.1, 0.1) # changeable
+            y = np.random.uniform(0.5, 0.7) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([0.0, 0.0, 180.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Tops_FrontOpen/assets_training_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=os.getcwd() + "/" + np.random.choice(assets_list)
+            print(usd_path)
+
+    result = HangCoat(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.data_collection_flag, args.record_video_flag)
+    if EMIT_JSON:
+        print(f"{RESULT_PREFIX}{json.dumps(result, sort_keys=True)}")
+
+    if args.data_collection_flag:
+        simulation_app.close()
+    elif HEADLESS:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+            
+    
+simulation_app.close()

third_party/DexGarmentLab/Env_StandAlone/Store_Tops_Env.py

@@ -0,0 +1,511 @@
+import os
+import json
+from isaacsim import SimulationApp
+
+
+def _env_flag(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name)
+    if value is None:
+        return default
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
+HEADLESS = _env_flag("DEXGARMENTLAB_HEADLESS", False)
+EMIT_JSON = _env_flag("DEXGARMENTLAB_EMIT_JSON", False)
+RESULT_PREFIX = "DEXGARMENTLAB_RESULT="
+simulation_app = SimulationApp({"headless": HEADLESS})
+
+# load external package
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import pusher_loader, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_record
+from Env_Config.Utils_Project.Collision_Group import CollisionGroup
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+
+class StoreTops_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+    ):
+        # load BaseEnv
+        super().__init__()
+
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        
+        # add ground
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0, 0, 0]),            
+            usd_path=os.getcwd() + "/" + "Assets/Garment/Tops/Collar_noSleeve_FrontClose/TCNC_Top338/TCNC_Top338_obj.usd" if usd_path is None else usd_path,
+        )
+        
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.9, 0.0, 0.5]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.9, 0.0, 0.5]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([pos[0], pos[1], 6.75]), 
+            camera_orientation=np.array([0, 90.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 5.22, 8.11]),
+            camera_orientation=np.array([0, 60, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.judge_camera = Recording_Camera(
+            camera_position=np.array([0.0+env_dx,1.25+env_dy,6.75]),
+            camera_orientation=np.array([0, 90, -90.0]),
+            prim_path="/World/judge_camera",
+        )
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.pusher = pusher_loader(self.scene)
+        
+        # load GAM Model
+        self.model = GAM_Encapsulation(catogory="Tops_NoSleeve")  
+        
+        # define collision group - helper path
+        self.helper_path=['/World/defaultGroundPlane/GroundPlane', '/World/pusher']
+        self.collisiongroup = CollisionGroup(
+            self.world,
+            helper_path=self.helper_path,
+            garment=True,
+            collide_with_garment=True,
+            collide_with_robot=False,
+        )
+        
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+
+        self.garment_pcd = None
+        self.object_pcd = None     
+        self.points_affordance_feature = None    
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+        
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.20]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+                
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        # initialize gif camera to obtain rgb with the aim of creating gif
+        self.env_camera.initialize(depth_enable=True)
+        
+        self.judge_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/pusher",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+
+        # step world to make it ready
+        for i in range(100):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+
+        
+        cprint("World Ready!", "green", "on_green")
+        
+    def record_callback(self, step_size):
+        
+        if self.step_num % 5 == 0:
+        
+            joint_pos_L = self.bimanual_dex.dexleft.get_joint_positions()
+            
+            joint_pos_R = self.bimanual_dex.dexright.get_joint_positions()
+            
+            joint_state = np.array([*joint_pos_L, *joint_pos_R])
+
+            rgb = self.env_camera.get_rgb_graph(save_or_not=False)
+
+            point_cloud = self.env_camera.get_pointcloud_from_depth(
+                show_original_pc_online=False,
+                show_downsample_pc_online=False,
+            )
+            
+            self.saving_data.append({ 
+                "joint_state": joint_state,
+                "image": rgb,
+                "env_point_cloud": point_cloud,
+                "garment_point_cloud":self.garment_pcd,
+                "object_point_cloud":self.object_pcd,
+                "points_affordance_feature": self.points_affordance_feature,
+            })
+        
+        self.step_num += 1
+
+
+def StoreTops(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, data_collection_flag, record_video_flag):
+    
+    env = StoreTops_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag)
+      
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+         
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    if record_video_flag:
+        env.thread_record.start()
+            
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[1954, 1832, 528, 587]) 
+    manipulation_points[:, 2] = 0.025  # set z-axis to 0.005 to make sure dexhand can grasp the garment
+    
+    env.points_affordance_feature = normalize_columns(points_similarity[2:4].T)
+    
+    # move both dexhand to the manipulation points
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0], left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=manipulation_points[1], right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="open")
+    
+    env.bimanual_dex.set_both_hand_state(left_hand_state="close", right_hand_state="close")
+    
+    for i in range(20):
+        env.step()
+    
+    left_dis=np.sqrt((manipulation_points[0][0]-manipulation_points[2][0])**2+(manipulation_points[0][1]-manipulation_points[2][1])**2)
+    right_dis=np.sqrt((manipulation_points[1][0]-manipulation_points[3][0])**2+(manipulation_points[1][1]-manipulation_points[3][1])**2)
+    distance=(left_dis+right_dis)/4
+    # get lift points
+    y_off=0.01
+    z_off=0.005
+    left_lift_points,right_lift_points=np.array([manipulation_points[0][0], manipulation_points[0][1]-distance+y_off, distance+z_off]), np.array([manipulation_points[1][0], manipulation_points[1][1]-distance+y_off, distance+z_off]) 
+    
+    # move both dexhand to the lift points
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_lift_points, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    left_lift_points,right_lift_points=np.array([manipulation_points[0][0], manipulation_points[2][1]+0.06, distance+z_off]), np.array([manipulation_points[1][0], manipulation_points[3][1]+0.06, distance+z_off])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_lift_points, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    # release the garment
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="open")
+        
+    env.garment.particle_material.set_gravity_scale(10.0)
+    
+    for i in range(200):
+        env.step()
+    
+    env.garment.particle_material.set_gravity_scale(1.0)
+    
+    cprint("Store World Fold Procedure Finish! Store Procedure Begins!", "green", "on_green")
+    
+    pusher_center = np.array([0.0+env_dx, 1.10+env_dy, 0.0])
+    env.pusher.set_world_pose(position=pusher_center)
+
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    # # hide prim to get garment point cloud
+    # set_prim_visible_group(
+    #     prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+    #     visible=False,
+    # )
+    # for i in range(50):
+    #     env.step()
+         
+    # env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+    #     save_or_not=False,
+    #     save_path=get_unique_filename("data", extension=".ply"),
+    # )
+    
+    # set_prim_visible_group(
+    #     prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+    #     visible=True,
+    # )
+    # for i in range(50):
+    #     env.step()
+    
+    left_ori=np.array([0.7010574,0.5609855, 0.4304593, 0.092296])
+    right_ori=np.array([ 0.4304593, 0.092296, 0.7010574,0.5609855])    
+    left_lift_points,right_lift_points=np.array([-0.5, 0.6, 0.65]), np.array([0.5, 0.6, 0.65])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=left_ori, right_pos=right_lift_points, right_ori=right_ori)
+    # env.bimanual_dex.set_both_hand_state(left_hand_state="smooth", right_hand_state="smooth")
+
+    for i in range(50):
+        env.step()        
+    
+    garment_fold_length = np.max([
+        abs(manipulation_points[0][1] - manipulation_points[2][1]),
+        abs(manipulation_points[1][1] - manipulation_points[3][1])
+    ]) / 2
+
+    garment_fold_width = np.max([
+        abs(manipulation_points[0][0] - manipulation_points[1][0]),
+        abs(manipulation_points[2][0] - manipulation_points[3][0])
+    ])
+
+    
+    manipulation_points=manipulation_points[2:]
+    left_off0=np.array([-0.08,-0.0,0.005])
+    right_off0=np.array([0.08,-0.0,0.005])
+    left_off1=np.array([-0.05,-0.0,0.002])
+    right_off1=np.array([0.05,-0.0,0.002])
+    left_off2=np.array([0.005,0.05,-0.0])
+    right_off2=np.array([-0.005,0.05,-0.0])
+    manipulation_points[:,2]=0.00
+    
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off0, left_ori=left_ori, right_pos=manipulation_points[1]+right_off0, right_ori=right_ori)
+    for i in range(20):
+        env.step()
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off1, left_ori=left_ori, right_pos=manipulation_points[1]+right_off1, right_ori=right_ori)
+    for i in range(20):
+        env.step()
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off2, left_ori=left_ori, right_pos=manipulation_points[1]+right_off2, right_ori=right_ori)
+    for i in range(20):
+        env.step()
+    # env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off3, left_ori=left_ori, right_pos=manipulation_points[1]+right_off3, right_ori=right_ori)
+    # for i in range(20):
+    #     env.step()
+    
+    if data_collection_flag:
+        env.record(task_name="Store_Tops",stage_index=1)    
+        
+    env.bimanual_dex.set_both_hand_state(left_hand_state="close", right_hand_state="close")
+    
+    left_point = np.array([pusher_center[0]-garment_fold_width/1.75, manipulation_points[0][1]+0.35, np.max([garment_fold_length+0.2,0.5])])
+    right_point = np.array([pusher_center[0]+garment_fold_width/1.75, manipulation_points[0][1]+0.35, np.max([garment_fold_length+0.2,0.5])])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_point, left_ori=left_ori, right_pos=right_point, right_ori=right_ori)
+    
+    left_point = np.array([pusher_center[0]-garment_fold_width/1.75, pusher_center[1], 0.5])
+    right_point = np.array([pusher_center[0]+garment_fold_width/1.75, pusher_center[1], 0.5])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_point, left_ori=left_ori, right_pos=right_point, right_ori=right_ori)
+    
+    left_point = np.array([pusher_center[0]-garment_fold_width/1.75, pusher_center[1], pusher_center[2]+0.10])
+    right_point = np.array([pusher_center[0]+garment_fold_width/1.75, pusher_center[1], pusher_center[2]+0.10])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_point, left_ori=left_ori, right_pos=right_point, right_ori=right_ori)
+    
+    # left_point = np.array([pusher_center[0]-garment_fold_width/1.35, pusher_center[1]-garment_fold_length/1.2, pusher_center[2]+0.05])
+    # right_point = np.array([pusher_center[0]+garment_fold_width/1.35, pusher_center[1]-garment_fold_length/1.2, pusher_center[2]+0.05])    
+    # env.bimanual_dex.dense_move_both_ik(left_pos=left_point, left_ori=left_ori, right_pos=right_point, right_ori=right_ori)
+    
+    env.bimanual_dex.set_both_hand_state(
+        left_hand_state='open', 
+        right_hand_state='open'
+    )    
+
+    if data_collection_flag:
+        env.stop_record()
+    
+    delete_prim("/World/DexLeft")
+    delete_prim("/World/DexRight")    
+    for i in range(50):
+        env.step()
+        
+    success=True
+    store_state, color = env.judge_camera.get_point_cloud_data_from_segment(save_or_not=False)
+    # get max_x min_x max_y min_y
+    max_x = np.max(store_state[:, 0])
+    min_x = np.min(store_state[:, 0])
+    max_y = np.max(store_state[:, 1])
+    min_y = np.min(store_state[:, 1])
+    # get the center of the point cloud
+    center_x = (max_x + min_x) / 2
+    center_y = (max_y + min_y) / 2
+    # get the distance between the center and the pusher
+    distance = np.sqrt((center_x - pusher_center[0]) ** 2 + (center_y - pusher_center[1]) ** 2)
+    # if the distance is less than 0.05, it is considered as success
+    if distance < 0.1:
+        success=True
+    else:
+        success=False
+        
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"garment_center: {center_x}, {center_y}", "blue")
+    cprint(f"pusher_center: {pusher_center[0]}, {pusher_center[1]}", "blue")
+    cprint(f"distance: {distance}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+    
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag and success:
+        if not os.path.exists("Data/Store_Tops/video"):
+            os.makedirs("Data/Store_Tops/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Store_Tops/video/video", ".mp4"))
+
+    if data_collection_flag:
+        # write into .log file
+        with open("Data/Store_Tops/data_collection_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+
+    if data_collection_flag:
+        if success:
+            env.record_to_npz(env_change=True)
+            if not os.path.exists("Data/Store_Tops/final_state_pic"):
+                os.makedirs("Data/Store_Tops/final_state_pic")
+            env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Store_Tops/final_state_pic/img",".png"))
+    result = {
+        "task_name": "Store Tops",
+        "success": bool(success),
+        "garment_point_count": int(env.garment_pcd.shape[0]) if env.garment_pcd is not None else 0,
+        "object_point_count": int(env.object_pcd.shape[0]) if env.object_pcd is not None else 0,
+        "judge_point_count": int(store_state.shape[0]),
+        "affordance_feature_norm": float(np.linalg.norm(env.points_affordance_feature)) if env.points_affordance_feature is not None else 0.0,
+        "distance_to_target_center": float(distance),
+    }
+    return result
+
+     
+if __name__=="__main__":
+    
+    args = parse_args_record()
+    
+    # initial setting
+    pos = np.array([0.0, 0.7, 0.20])
+    ori = np.array([0.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.3, 0.3) # changeable
+            env_dy = np.random.uniform(-0.1, 0.1) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.05, 0.05) # changeable
+            y = np.random.uniform(0.65, 0.75) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([0.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Tops_NoSleeve/assets_training_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=os.getcwd() + "/" + np.random.choice(assets_list)
+    result = StoreTops(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.data_collection_flag, args.record_video_flag)
+    if EMIT_JSON:
+        print(f"{RESULT_PREFIX}{json.dumps(result, sort_keys=True)}")
+
+    if args.data_collection_flag:
+        simulation_app.close()
+    elif HEADLESS:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+
+simulation_app.close()

third_party/DexGarmentLab/Env_Validation/Hang_Tops_HALO.py

@@ -0,0 +1,405 @@
+from isaacsim import SimulationApp
+simulation_app = SimulationApp({"headless": True})
+
+# load external package
+import os
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from isaacsim.core.utils.types import ArticulationAction, ArticulationActions
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import hanger_load, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_val
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+from Model_HALO.SADP.SADP import SADP
+
+
+class HangTops_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+        training_data_num:int=100,
+        stage_1_checkpoint_num:int=1500, 
+        stage_2_checkpoint_num:int=1500, 
+        stage_3_checkpoint_num:int=1500, 
+    ):
+        # load BaseEnv
+        super().__init__()
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path="Assets/Garment/Tops/NoCollar_Lsleeve_FrontClose/TNLC_Top603/TNLC_Top603_obj.usd" if usd_path is None else usd_path,
+        )
+        # Here are some example garments you can try:
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Jacket032/TCLC_Jacket032_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Jacket152/TCLC_Jacket152_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top566/TCLC_Top566_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top584/TCLC_Top584_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_top118/TCLC_top118_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top476/TCLC_Top476_obj.usd",
+        # "Assets/Garment/Tops/Collar_Lsleeve_FrontClose/TCLC_Top030/TCLC_Top030_obj.usd",  
+
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.8, 0.0, 0.6]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.8, 0.0, 0.6]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([0.0, -3.0, 6.75]), 
+            camera_orientation=np.array([0, 60.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 6.65, 4.0]),
+            camera_orientation=np.array([0, 30.0, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+        
+        self.garment_pcd = None
+        self.object_pcd = None
+        self.points_affordance_feature = None
+                
+        # load GAM Model
+        self.model = GAM_Encapsulation(catogory="Tops_LongSleeve")        
+        
+        # load hanger
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.hanger_center = hanger_load(self.scene, env_dx, env_dy)
+        
+        self.judge_camera = Recording_Camera(
+            camera_position=np.array([self.hanger_center[0], 6.0, 0.5]),
+            camera_orientation=np.array([0, 0.0, -90.0]),
+            prim_path="/World/judge_camera",
+        )    
+        
+        self.sadp = SADP(task_name="Hang_Tops_stage_1", data_num=training_data_num, checkpoint_num=stage_1_checkpoint_num)
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        # initialize gif camera to obtain rgb with the aim of creating gif
+        self.env_camera.initialize(
+            depth_enable=True,
+        )
+        
+        self.judge_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/hanger1",
+                "/World/hanger2",
+                "/World/hanger3",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+            
+        # move garment to the target position
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.2]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+                
+        # open hand to be initial state
+        self.bimanual_dex.set_both_hand_state("open", "open")
+
+        # step world to make it ready
+        for i in range(200):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+
+        cprint("World Ready!", "green", "on_green")
+        
+        
+# if __name__=="__main__":
+def HangTops(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, validation_flag, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num):
+    
+    env = HangTops_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num)
+    
+    env.garment.particle_material.set_gravity_scale(0.45)
+    
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/hanger1", "/World/hanger2", "/World/hanger3"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+            
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+
+    # make prim visible
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/hanger1", "/World/hanger2", "/World/hanger3"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    if record_video_flag:
+        env.thread_record.start()
+
+    # get manipulation points from GAM Model
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[838,179])
+    
+    env.points_affordance_feature = normalize_columns(points_similarity.T)
+        
+    manipulation_points[:, 2] = 0.00  # set z-axis to 0.025 to make sure dexhand can grasp the garment
+
+    # move both dexhand to the manipulation points
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0], left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=manipulation_points[1], right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+    
+    for i in range(20):
+        env.step()
+    
+    for i in range(9):
+        
+        print(f"Stage_1_Step: {i}")
+
+        joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+        joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+        joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+        
+        obs = dict()
+        obs['agent_pos']=joint_state
+        obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+        obs['garment_point_cloud']=env.garment_pcd
+        obs['object_point_cloud']=env.object_pcd
+        obs['points_affordance_feature']=env.points_affordance_feature
+
+        action=env.sadp.get_action(obs)
+        
+        print("action_shape:",action.shape)
+        
+        for j in range(4):
+            
+            action_L = ArticulationAction(joint_positions=action[j][:30])
+            action_R = ArticulationAction(joint_positions=action[j][30:])
+
+            env.bimanual_dex.dexleft.apply_action(action_L)
+            env.bimanual_dex.dexright.apply_action(action_R)
+            
+            for _ in range(5):    
+                env.step()
+                
+            joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+            joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+            joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+            
+            obs = dict()
+            obs['agent_pos']=joint_state
+            obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+            obs['garment_point_cloud']=env.garment_pcd
+            obs['object_point_cloud']=env.object_pcd
+            obs['points_affordance_feature']=env.points_affordance_feature
+            
+            env.sadp.update_obs(obs)
+        
+    env.garment.particle_material.set_gravity_scale(2.0)
+    
+    for i in range(100):
+        env.step()
+    
+    env.garment.particle_material.set_gravity_scale(0.45)
+    
+    # make prim visible
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+        
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag:
+        if not os.path.exists("Data/Hang_Tops_Validation_HALO/video"):
+            os.makedirs("Data/Hang_Tops_Validation_HALO/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Hang_Tops_Validation_HALO/video/video", ".mp4"))
+   
+        
+        
+    success=True
+    
+    pcd_judge, _ = env.judge_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True
+    )
+    z_values = pcd_judge[:, 2]  # 假设 pcd_judge 的形状是 (N, 3)
+    points_below_threshold = np.sum(z_values < 0.01)
+    if points_below_threshold > 15:
+        success=False
+    elif env.garment.get_garment_center_pos()[0] > env.hanger_center[0]+0.035 or env.garment.get_garment_center_pos()[0] < env.hanger_center[0]-0.035:
+        success=False
+    else:
+        success=True
+
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"points_below_threshold: {points_below_threshold}", "blue")
+    cprint(f"garment_center_pos_x: {env.garment.get_garment_center_pos()[0]}", "blue")
+    cprint(f"hanger_center_x: {env.hanger_center[0]}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+
+    if validation_flag:
+        if not os.path.exists("Data/Hang_Tops_Validation_HALO"):
+            os.makedirs("Data/Hang_Tops_Validation_HALO")
+        # write into .log file
+        with open("Data/Hang_Tops_Validation_HALO/validation_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+        if not os.path.exists("Data/Hang_Tops_Validation_HALO/final_state_pic"):
+            os.makedirs("Data/Hang_Tops_Validation_HALO/final_state_pic")
+        env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Hang_Tops_Validation_HALO/final_state_pic/img",".png"))
+
+
+
+
+if __name__=="__main__":
+    
+    args = parse_args_val()
+    
+    # initial setting
+    pos = np.array([0, 0.7, 0.2])
+    ori = np.array([0.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0 
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.15, 0.15) # changeable
+            env_dy = np.random.uniform(-0.05, 0.05) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.1, 0.1) # changeable
+            y = np.random.uniform(0.5, 0.8) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([0.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Tops_LongSleeve/assets_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=np.random.choice(assets_list)
+
+    HangTops(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.validation_flag, args.record_video_flag, args.training_data_num, args.stage_1_checkpoint_num, args.stage_2_checkpoint_num, args.stage_3_checkpoint_num)
+
+    if args.validation_flag:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+    
+simulation_app.close()

third_party/DexGarmentLab/Env_Validation/Hang_Trousers_HALO.py

@@ -0,0 +1,409 @@
+from isaacsim import SimulationApp
+simulation_app = SimulationApp({"headless": True})
+
+# load external package
+import os
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from isaacsim.core.utils.types import ArticulationAction, ArticulationActions
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import hanger_load, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_val
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+from Model_HALO.SADP.SADP import SADP
+
+class HangTrousers_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+        training_data_num:int=100,
+        stage_1_checkpoint_num:int=1500, 
+        stage_2_checkpoint_num:int=1500, 
+        stage_3_checkpoint_num:int=1500, 
+    ):
+        # load BaseEnv
+        super().__init__()
+
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path="Assets/Garment/Trousers/Long/PL_074/PL_074_obj.usd" if usd_path is None else usd_path,
+            # contact_offset=0.012,             
+            # rest_offset=0.010,                
+            # particle_contact_offset=0.012,    
+            # fluid_rest_offset=0.010,
+            # solid_rest_offset=0.010,
+        )
+        # Here are some example garments you can try:
+        # "Assets/Garment/Trousers/Long/PL_Pants051/PL_Pants051_obj.usd",
+        # "Assets/Garment/Trousers/Long/PL_085/PL_085_obj.usd",
+        # "Assets/Garment/Trousers/Long/PL_Pants091/PL_Pants091_obj.usd",
+        # "Assets/Garment/Trousers/Long/PL_LongPants024/PL_LongPants024_obj.usd",
+        # "Assets/Garment/Trousers/Long/PL_M1_074/PL_M1_074_obj.usd",
+        # "Assets/Garment/Trousers/Long/PL_M2_015/PL_M2_015_obj.usd",            
+        # "Assets/Garment/Trousers/Long/PL_041/PL_041_obj.usd",
+
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.8, 0.0, 0.6]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.8, 0.0, 0.6]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+        
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([0.0, -3.0, 6.75]), 
+            camera_orientation=np.array([0, 60.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 6.65, 4.0]),
+            camera_orientation=np.array([0, 30.0, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+
+        self.garment_pcd = None
+        self.object_pcd = None
+        self.points_affordance_feature = None
+        
+        # load GAM Model
+        self.model = GAM_Encapsulation(catogory="Trousers")        
+        
+        # load hanger
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.hanger_center = hanger_load(self.scene, env_dx, env_dy)
+        
+        self.judge_camera = Recording_Camera(
+            camera_position=np.array([self.hanger_center[0], 6.0, 0.5]),
+            camera_orientation=np.array([0, 0.0, -90.0]),
+            prim_path="/World/judge_camera",
+        )
+
+        self.sadp = SADP(task_name="Hang_Trousers_stage_1", data_num=training_data_num, checkpoint_num=stage_1_checkpoint_num)
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        # initialize gif camera to obtain rgb with the aim of creating gif
+        self.env_camera.initialize(
+            depth_enable=True,
+        )
+        
+        self.judge_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/hanger1",
+                "/World/hanger2",
+                "/World/hanger3",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+            
+        # move garment to the target position
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.2]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+                
+        # open hand to be initial state
+        self.bimanual_dex.set_both_hand_state("open", "open")
+
+        # step world to make it ready
+        for i in range(200):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+
+
+        cprint("World Ready!", "green", "on_green")
+
+        
+# if __name__=="__main__":
+def HangTrousers(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, validation_flag, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num):
+    
+    env = HangTrousers_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num)
+    
+    env.garment.particle_material.set_gravity_scale(0.45)
+    
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/hanger1", "/World/hanger2", "/World/hanger3"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+            
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+
+    # make prim visible
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/hanger1", "/World/hanger2", "/World/hanger3"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    if record_video_flag:
+        env.thread_record.start()
+
+    # get manipulation points from GAM Model
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[1083,219])
+    
+    env.points_affordance_feature = normalize_columns(points_similarity.T)
+        
+    manipulation_points[:, 2] = 0.00  # set z-axis to 0.025 to make sure dexhand can grasp the garment
+
+    # move both dexhand to the manipulation points
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0], left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=manipulation_points[1], right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+    
+    for i in range(20):
+        env.step()
+    
+    for i in range(10):
+        
+        print(f"Stage_1_Step: {i}")
+
+        joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+        joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+        joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+        
+        obs = dict()
+        obs['agent_pos']=joint_state
+        obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+        obs['garment_point_cloud']=env.garment_pcd
+        obs['object_point_cloud']=env.object_pcd
+        obs['points_affordance_feature']=env.points_affordance_feature
+
+        action=env.sadp.get_action(obs)
+        
+        print("action_shape:",action.shape)
+        
+        for j in range(4):
+            
+            action_L = ArticulationAction(joint_positions=action[j][:30])
+            action_R = ArticulationAction(joint_positions=action[j][30:])
+
+            env.bimanual_dex.dexleft.apply_action(action_L)
+            env.bimanual_dex.dexright.apply_action(action_R)
+            
+            for _ in range(5):    
+                env.step()
+                
+            joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+            joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+            joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+            
+            obs = dict()
+            obs['agent_pos']=joint_state
+            obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+            obs['garment_point_cloud']=env.garment_pcd
+            obs['object_point_cloud']=env.object_pcd
+            obs['points_affordance_feature']=env.points_affordance_feature
+            
+            env.sadp.update_obs(obs)
+        
+    env.garment.particle_material.set_gravity_scale(2.0)
+    
+    for i in range(200):
+        env.step()
+    
+    env.garment.particle_material.set_gravity_scale(0.45)
+    
+    # make prim visible
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+        
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag:
+        if not os.path.exists("Data/Hang_Trousers_Validation_HALO/video"):
+            os.makedirs("Data/Hang_Trousers_Validation_HALO/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Hang_Trousers_Validation_HALO/video/video", ".mp4"))
+   
+        
+        
+    success=True
+    
+    pcd_judge, _ = env.judge_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True
+    )
+    z_values = pcd_judge[:, 2]  # 假设 pcd_judge 的形状是 (N, 3)
+    points_below_threshold = np.sum(z_values < 0.01)
+    if points_below_threshold > 15:
+        success=False
+    elif env.garment.get_garment_center_pos()[0] > env.hanger_center[0]+0.05 or env.garment.get_garment_center_pos()[0] < env.hanger_center[0]-0.05:
+        success=False
+    else:
+        success=True
+
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"points_below_threshold: {points_below_threshold}", "blue")
+    cprint(f"garment_center_pos_x: {env.garment.get_garment_center_pos()[0]}", "blue")
+    cprint(f"hanger_center_x: {env.hanger_center[0]}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+
+    if validation_flag:
+        if not os.path.exists("Data/Hang_Trousers_Validation_HALO"):
+            os.makedirs("Data/Hang_Trousers_Validation_HALO")
+        # write into .log file
+        with open("Data/Hang_Trousers_Validation_HALO/validation_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+        if not os.path.exists("Data/Hang_Trousers_Validation_HALO/final_state_pic"):
+            os.makedirs("Data/Hang_Trousers_Validation_HALO/final_state_pic")
+        env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Hang_Trousers_Validation_HALO/final_state_pic/img",".png"))
+
+
+if __name__=="__main__":
+    
+    args = parse_args_val()
+    
+    # initial setting
+    pos = np.array([0, 0.7, 0.2])
+    ori = np.array([0.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0 
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.15, 0.15) # changeable
+            env_dy = np.random.uniform(-0.05, 0.05) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.1, 0.1) # changeable
+            y = np.random.uniform(0.5, 0.8) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([0.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Trousers/assets_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=np.random.choice(assets_list)
+
+    HangTrousers(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.validation_flag, args.record_video_flag, args.training_data_num, args.stage_1_checkpoint_num, args.stage_2_checkpoint_num, args.stage_3_checkpoint_num)
+
+    if args.validation_flag:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+    
+simulation_app.close()

third_party/DexGarmentLab/Env_Validation/Store_Tops_HALO.py

@@ -0,0 +1,547 @@
+import os
+import json
+from isaacsim import SimulationApp
+
+
+def _env_flag(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name)
+    if value is None:
+        return default
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
+def _env_float(name: str):
+    value = os.environ.get(name)
+    if value is None or value == "":
+        return None
+    return float(value)
+
+
+def _env_text(name: str):
+    value = os.environ.get(name)
+    if value is None or value == "":
+        return None
+    return value
+
+
+HEADLESS = _env_flag("DEXGARMENTLAB_HEADLESS", True)
+EMIT_JSON = _env_flag("DEXGARMENTLAB_EMIT_JSON", False)
+RESULT_PREFIX = "DEXGARMENTLAB_RESULT="
+simulation_app = SimulationApp({"headless": HEADLESS})
+
+# load external package
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from isaacsim.core.utils.types import ArticulationAction, ArticulationActions
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import pusher_loader, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_val
+from Env_Config.Utils_Project.Collision_Group import CollisionGroup
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+from Model_HALO.SADP.SADP import SADP
+
+class StoreTops_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+        training_data_num:int=100,
+        stage_1_checkpoint_num:int=1500, 
+        stage_2_checkpoint_num:int=1500, 
+        stage_3_checkpoint_num:int=1500, 
+    ):
+        # load BaseEnv
+        super().__init__()
+
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        
+        # add ground
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0, 0, 0]),            
+            usd_path="Assets/Garment/Tops/Collar_noSleeve_FrontClose/TCNC_Top338/TCNC_Top338_obj.usd" if usd_path is None else usd_path,
+        )
+        
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.9, 0.0, 0.5]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.9, 0.0, 0.5]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([pos[0], pos[1], 6.75]), 
+            camera_orientation=np.array([0, 90.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 5.22, 8.11]),
+            camera_orientation=np.array([0, 60, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.judge_camera = Recording_Camera(
+            camera_position=np.array([0.0+env_dx,1.25+env_dy,6.75]),
+            camera_orientation=np.array([0, 90, -90.0]),
+            prim_path="/World/judge_camera",
+        )
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.pusher = pusher_loader(self.scene)
+        
+        # load GAM Model
+        self.model = GAM_Encapsulation(catogory="Tops_NoSleeve")  
+        
+        # define collision group - helper path
+        self.helper_path=['/World/defaultGroundPlane/GroundPlane', '/World/pusher']
+        self.collisiongroup = CollisionGroup(
+            self.world,
+            helper_path=self.helper_path,
+            garment=True,
+            collide_with_garment=True,
+            collide_with_robot=False,
+        )
+        
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+
+        self.garment_pcd = None
+        self.object_pcd = None       
+        self.points_affordance_feature = None
+        
+        self.sadp = SADP(task_name="Store_Tops_stage_1", data_num=training_data_num, checkpoint_num=stage_1_checkpoint_num)  
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+        
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.20]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+                
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        # initialize gif camera to obtain rgb with the aim of creating gif
+        self.env_camera.initialize(depth_enable=True)
+        
+        self.judge_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/pusher",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+
+        # step world to make it ready
+        for i in range(100):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+
+        
+        cprint("World Ready!", "green", "on_green")
+
+
+def StoreTops(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, validation_flag, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num):
+    
+    env = StoreTops_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num)
+      
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+         
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    if record_video_flag:
+        env.thread_record.start()
+            
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[1954, 1832, 528, 587]) 
+    manipulation_points[:, 2] = 0.025  # set z-axis to 0.005 to make sure dexhand can grasp the garment
+    
+    env.points_affordance_feature = normalize_columns(points_similarity[2:4].T)
+        
+    # move both dexhand to the manipulation points
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0], left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=manipulation_points[1], right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="open")
+    
+    env.bimanual_dex.set_both_hand_state(left_hand_state="close", right_hand_state="close")
+    
+    for i in range(20):
+        env.step()
+    
+    left_dis=np.sqrt((manipulation_points[0][0]-manipulation_points[2][0])**2+(manipulation_points[0][1]-manipulation_points[2][1])**2)
+    right_dis=np.sqrt((manipulation_points[1][0]-manipulation_points[3][0])**2+(manipulation_points[1][1]-manipulation_points[3][1])**2)
+    distance=(left_dis+right_dis)/4
+    # get lift points
+    y_off=0.01
+    z_off=0.005
+    left_lift_points,right_lift_points=np.array([manipulation_points[0][0], manipulation_points[0][1]-distance+y_off, distance+z_off]), np.array([manipulation_points[1][0], manipulation_points[1][1]-distance+y_off, distance+z_off]) 
+    
+    # move both dexhand to the lift points
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_lift_points, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    left_lift_points,right_lift_points=np.array([manipulation_points[0][0], manipulation_points[2][1]+0.06, distance+z_off]), np.array([manipulation_points[1][0], manipulation_points[3][1]+0.06, distance+z_off])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=np.array([0.579, -0.579, -0.406, 0.406]), right_pos=right_lift_points, right_ori=np.array([0.406, -0.406, -0.579, 0.579]))
+
+    # release the garment
+    env.bimanual_dex.set_both_hand_state(left_hand_state="open", right_hand_state="open")
+        
+    env.garment.particle_material.set_gravity_scale(10.0)
+    
+    for i in range(200):
+        env.step()
+    
+    env.garment.particle_material.set_gravity_scale(1.0)
+    
+    cprint("Store World Fold Procedure Finish! Store Procedure Begins!", "green", "on_green")
+    
+    pusher_center = np.array([0.0+env_dx, 1.10+env_dy, 0.0])
+    env.pusher.set_world_pose(position=pusher_center)
+
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    # # hide prim to get garment point cloud
+    # set_prim_visible_group(
+    #     prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+    #     visible=False,
+    # )
+    # for i in range(50):
+    #     env.step()
+         
+    # env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+    #     save_or_not=False,
+    #     save_path=get_unique_filename("data", extension=".ply"),
+    # )
+    
+    # set_prim_visible_group(
+    #     prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/pusher"],
+    #     visible=True,
+    # )
+    # for i in range(50):
+    #     env.step()
+    
+    left_ori=np.array([0.7010574,0.5609855, 0.4304593, 0.092296])
+    right_ori=np.array([ 0.4304593, 0.092296, 0.7010574,0.5609855])    
+    left_lift_points,right_lift_points=np.array([-0.5, 0.6, 0.65]), np.array([0.5, 0.6, 0.65])
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=left_ori, right_pos=right_lift_points, right_ori=right_ori)
+    # env.bimanual_dex.set_both_hand_state(left_hand_state="smooth", right_hand_state="smooth")
+
+    for i in range(50):
+        env.step()        
+    
+    garment_fold_length = np.max([
+        abs(manipulation_points[0][1] - manipulation_points[2][1]),
+        abs(manipulation_points[1][1] - manipulation_points[3][1])
+    ]) / 2
+
+    garment_fold_width = np.max([
+        abs(manipulation_points[0][0] - manipulation_points[1][0]),
+        abs(manipulation_points[2][0] - manipulation_points[3][0])
+    ])
+
+    
+    manipulation_points=manipulation_points[2:]
+    left_off0=np.array([-0.08,-0.0,0.005])
+    right_off0=np.array([0.08,-0.0,0.005])
+    left_off1=np.array([-0.05,-0.0,0.002])
+    right_off1=np.array([0.05,-0.0,0.002])
+    left_off2=np.array([0.005,0.05,-0.0])
+    right_off2=np.array([-0.005,0.05,-0.0])
+    manipulation_points[:,2]=0.00
+    
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off0, left_ori=left_ori, right_pos=manipulation_points[1]+right_off0, right_ori=right_ori)
+    for i in range(20):
+        env.step()
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off1, left_ori=left_ori, right_pos=manipulation_points[1]+right_off1, right_ori=right_ori)
+    for i in range(20):
+        env.step()
+    env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off2, left_ori=left_ori, right_pos=manipulation_points[1]+right_off2, right_ori=right_ori)
+    for i in range(20):
+        env.step()
+    # env.bimanual_dex.dense_move_both_ik(left_pos=manipulation_points[0]+left_off3, left_ori=left_ori, right_pos=manipulation_points[1]+right_off3, right_ori=right_ori)
+    # for i in range(20):
+    #     env.step()
+    
+    for i in range(20):
+        env.step()
+
+    for i in range(10):
+        
+        print(f"Stage_1_Step: {i}")
+
+        joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+        joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+        joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+        
+        obs = dict()
+        obs['agent_pos']=joint_state
+        obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+        obs['garment_point_cloud']=env.garment_pcd
+        obs['object_point_cloud']=env.object_pcd
+        obs['points_affordance_feature']=env.points_affordance_feature
+
+        action=env.sadp.get_action(obs)
+        
+        print("action_shape:",action.shape)
+        
+        for j in range(4):
+            
+            action_L = ArticulationAction(joint_positions=action[j][:30])
+            action_R = ArticulationAction(joint_positions=action[j][30:])
+
+            env.bimanual_dex.dexleft.apply_action(action_L)
+            env.bimanual_dex.dexright.apply_action(action_R)
+            
+            for _ in range(5):    
+                env.step()
+                
+            joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+            joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+            joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+            
+            obs = dict()
+            obs['agent_pos']=joint_state
+            obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+            obs['garment_point_cloud']=env.garment_pcd
+            obs['object_point_cloud']=env.object_pcd
+            obs['points_affordance_feature']=env.points_affordance_feature
+            
+            env.sadp.update_obs(obs)
+        
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag:
+        if not os.path.exists("Data/Store_Tops_Validation_HALO/video"):
+            os.makedirs("Data/Store_Tops_Validation_HALO/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Store_Tops_Validation_HALO/video/video", ".mp4"))
+    
+    delete_prim("/World/DexLeft")
+    delete_prim("/World/DexRight")    
+    for i in range(50):
+        env.step()
+        
+    success=True
+    store_state, color = env.judge_camera.get_point_cloud_data_from_segment(save_or_not=False)
+    # get max_x min_x max_y min_y
+    max_x = np.max(store_state[:, 0])
+    min_x = np.min(store_state[:, 0])
+    max_y = np.max(store_state[:, 1])
+    min_y = np.min(store_state[:, 1])
+    # get the center of the point cloud
+    center_x = (max_x + min_x) / 2
+    center_y = (max_y + min_y) / 2
+    # get the distance between the center and the pusher
+    distance = np.sqrt((center_x - pusher_center[0]) ** 2 + (center_y - pusher_center[1]) ** 2)
+    # if the distance is less than 0.05, it is considered as success
+    if distance < 0.1:
+        success=True
+    else:
+        success=False
+        
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"garment_center: {center_x}, {center_y}", "blue")
+    cprint(f"pusher_center: {pusher_center[0]}, {pusher_center[1]}", "blue")
+    cprint(f"distance: {distance}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+
+    if validation_flag:
+        if not os.path.exists("Data/Store_Tops_Validation_HALO"):
+            os.makedirs("Data/Store_Tops_Validation_HALO")
+        # write into .log file
+        with open("Data/Store_Tops_Validation_HALO/validation_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+
+        if not os.path.exists("Data/Store_Tops_Validation_HALO/final_state_pic"):
+            os.makedirs("Data/Store_Tops_Validation_HALO/final_state_pic")
+        env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Store_Tops_Validation_HALO/final_state_pic/img",".png"))
+
+    result = {
+        "task_name": "Store Tops HALO",
+        "success": bool(success),
+        "garment_point_count": int(env.garment_pcd.shape[0]) if env.garment_pcd is not None else 0,
+        "object_point_count": int(env.object_pcd.shape[0]) if env.object_pcd is not None else 0,
+        "judge_point_count": int(store_state.shape[0]),
+        "affordance_feature_norm": float(np.linalg.norm(env.points_affordance_feature)) if env.points_affordance_feature is not None else 0.0,
+        "distance_to_target_center": float(distance),
+    }
+    return result
+
+        
+
+     
+if __name__=="__main__":
+    
+    args = parse_args_val()
+    
+    # initial setting
+    pos = np.array([0.0, 0.7, 0.20])
+    ori = np.array([0.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0
+
+    if args.env_random_flag or args.garment_random_flag:
+        rng_seed = _env_text("DEXGARMENTLAB_RNG_SEED")
+        np.random.seed(int(rng_seed) if rng_seed is not None else int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.3, 0.3) # changeable
+            env_dy = np.random.uniform(-0.1, 0.1) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.05, 0.05) # changeable
+            y = np.random.uniform(0.65, 0.75) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([0.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Tops_NoSleeve/assets_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=np.random.choice(assets_list)
+
+    forced_usd_path = _env_text("DEXGARMENTLAB_USD_PATH")
+    if forced_usd_path is not None:
+        usd_path = forced_usd_path
+    forced_pos_x = _env_float("DEXGARMENTLAB_POS_X")
+    forced_pos_y = _env_float("DEXGARMENTLAB_POS_Y")
+    if forced_pos_x is not None or forced_pos_y is not None:
+        pos = np.array([
+            pos[0] if forced_pos_x is None else forced_pos_x,
+            pos[1] if forced_pos_y is None else forced_pos_y,
+            pos[2],
+        ])
+    forced_ori_z = _env_float("DEXGARMENTLAB_ORI_Z")
+    if forced_ori_z is not None:
+        ori = np.array([ori[0], ori[1], forced_ori_z])
+    forced_env_dx = _env_float("DEXGARMENTLAB_ENV_DX")
+    if forced_env_dx is not None:
+        env_dx = forced_env_dx
+    forced_env_dy = _env_float("DEXGARMENTLAB_ENV_DY")
+    if forced_env_dy is not None:
+        env_dy = forced_env_dy
+
+    result = StoreTops(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.validation_flag, args.record_video_flag, args.training_data_num, args.stage_1_checkpoint_num, args.stage_2_checkpoint_num, args.stage_3_checkpoint_num)
+    if EMIT_JSON:
+        print(f"{RESULT_PREFIX}{json.dumps(result, sort_keys=True)}")
+
+    if args.validation_flag:
+        simulation_app.close()
+    elif HEADLESS:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+
+simulation_app.close()

third_party/DexGarmentLab/Env_Validation/Wear_Baseballcap_HALO.py

@@ -0,0 +1,403 @@
+from isaacsim import SimulationApp
+simulation_app = SimulationApp({"headless": False})
+
+# load external package
+import os
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from isaacsim.core.utils.types import ArticulationAction, ArticulationActions
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import hat_helper_load, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_val
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+from Env_Config.Utils_Project.Collision_Group import CollisionGroup
+from Env_Config.Human.Human import Human
+from Env_Config.Utils_Project.Attachment_Block import attach_fixedblock
+from Model_HALO.SADP.SADP import SADP
+
+class WearBaseballcap_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+        training_data_num:int=100,
+        stage_1_checkpoint_num:int=1500, 
+        stage_2_checkpoint_num:int=1500, 
+        stage_3_checkpoint_num:int=1500, 
+    ):
+        # load BaseEnv
+        super().__init__()
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+
+        # load human
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.human = Human(
+            path="Assets/Human/human_model.usd",
+            position=[0.0+env_dx,1.15+env_dy,0.0], 
+            scale=np.array([0.6, 0.6, 0.6]),
+        )
+        
+        # load garment
+        self.garment=Deformable_Garment(
+            self.world,
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path="Assets/Garment/Hat/HA_Hat016/HA_Hat016_obj.usd" if usd_path is None else usd_path,
+            solver_position_iteration_count=8,
+            simulation_hexahedral_resolution=16
+        )
+
+        self.target_put_pos = hat_helper_load(self.scene, pos[0], pos[1], self.env_dx, self.env_dy)
+
+        self.helper_path=["/World/hanger_helper", "/World/hanger", "/World/Human"]
+
+        self.head_helper_path=["/World/head_helper"]
+
+
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.6, 0.0, 0.5]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.6, 0.0, 0.5]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([pos[0], pos[1], 6.75]), 
+            camera_orientation=np.array([0, 90.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+
+                
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0, -3.45, 4.17]),
+            camera_orientation=np.array([0, 40, 90]),
+            prim_path="/World/env_camera",
+        )
+        
+        
+        # load UniGarmentManip Model
+        self.model = GAM_Encapsulation(catogory="Baseball_Cap")   
+
+        # import Collision Group
+        self.collisiongroup=CollisionGroup(
+            self.world,
+            helper_path=self.helper_path,
+            garment=False,
+            collide_with_garment=True,
+            collide_with_robot=False
+        )     
+        self.collisiongroup.add_collision(group_path="head",target=self.head_helper_path)
+
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+
+        self.garment_pcd = None
+        self.object_pcd = None
+        self.points_affordance_feature = None
+        
+        self.sadp = SADP(task_name="Wear_Baseballcap_stage_1", data_num=training_data_num, checkpoint_num=stage_1_checkpoint_num)
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+        
+        self.garment.set_garment_pose(pos=np.array([pos[0], pos[1], 0.65]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+        
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Deformable/deformable",
+            ]
+        )
+        
+        self.env_camera.initialize(
+            depth_enable=True,
+        )
+
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Human",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+
+        # step world to make it ready
+        for i in range(100):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        
+        cprint("World Ready!", "green", "on_green")
+
+        
+def WearBaseballcap(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, validation_flag, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num):
+    
+    env = WearBaseballcap_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num)
+    
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Deformable/deformable", "/World/hanger"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Deformable/deformable", "/World/hanger"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Human", "/World/hanger"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+         
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+    )
+    
+    # get garment width
+    garment_width = np.max(env.garment_pcd[:, 0]) - np.min(env.garment_pcd[:, 0])
+    garment_center_x = (np.max(env.garment_pcd[:, 0]) - np.min(env.garment_pcd[:, 0])) / 2 + np.min(env.garment_pcd[:, 0])
+    pick_x = garment_center_x + garment_width * 0.15
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Human", "/World/hanger"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    if record_video_flag:
+        env.thread_record.start()
+
+    # get manipulation points from UniGarmentManip Model
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[1110])
+    
+    env.points_affordance_feature = normalize_columns(np.concatenate([points_similarity, points_similarity], axis=0).T)
+        
+    hat_length = np.max(env.garment_pcd[:, 1]) - np.min(env.garment_pcd[:, 1])  
+
+    left_lift_points = np.array([pick_x, manipulation_points[0][1]-0.025, 0.4])
+    # env.bimanual_dex.dexright.dense_step_action(target_pos=left_lift_points, target_ori=np.array([0.4062,0.4062,0.5774,0.5774]), angular_type="quat")
+    env.bimanual_dex.dexright.dense_step_action(target_pos=left_lift_points, target_ori=np.array([0.5,0.5,0.5,0.5]), angular_type="quat")
+    
+
+    for i in range(50):
+        env.step()
+    
+    manipulation_points[:, 2] = 0.59  # set z-axis to 0.025 to make sure dexhand can grasp the garment
+
+    # move both dexhand to the manipulation points
+    # env.bimanual_dex.dexright.dense_step_action(target_pos=np.array([manipulation_points[0][0]+0.02, manipulation_points[0][1]-0.015, manipulation_points[0][2]]), target_ori=np.array([0.4062,0.4062,0.5774,0.5774]), angular_type="quat", dense_sample_scale=0.005)
+    env.bimanual_dex.dexright.dense_step_action(target_pos=np.array([pick_x, manipulation_points[0][1]-0.025, manipulation_points[0][2]]), target_ori=np.array([0.5, 0.5, 0.5, 0.5]), angular_type="quat", dense_sample_scale=0.005)
+
+    env.garment.set_mass(0.02)
+    
+    for i in range(20):
+        env.step()
+
+    for i in range(9):
+        
+        print(f"Stage_1_Step: {i}")
+
+        joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+        joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+        joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+        
+        obs = dict()
+        obs['agent_pos']=joint_state
+        obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+        obs['garment_point_cloud']=env.garment_pcd
+        obs['object_point_cloud']=env.object_pcd
+        obs['points_affordance_feature']=env.points_affordance_feature
+
+        action=env.sadp.get_action(obs)
+        
+        print("action_shape:",action.shape)
+        
+        for j in range(4):
+            
+            action_L = ArticulationAction(joint_positions=action[j][:30])
+            action_R = ArticulationAction(joint_positions=action[j][30:])
+
+            env.bimanual_dex.dexleft.apply_action(action_L)
+            env.bimanual_dex.dexright.apply_action(action_R)
+            
+            for _ in range(5):    
+                env.step()
+                
+            joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+            joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+            joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+            
+            obs = dict()
+            obs['agent_pos']=joint_state
+            obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+            obs['garment_point_cloud']=env.garment_pcd
+            obs['object_point_cloud']=env.object_pcd
+            obs['points_affordance_feature']=env.points_affordance_feature
+            
+            env.sadp.update_obs(obs)
+        
+    for i in range(50):
+        env.step()
+        
+    attach_fixedblock(env.stage,env.garment.deformable_prim_path+"/mesh/attachment",env.garment.deformable_prim_path,"/World/head_helper")
+ 
+    delete_prim_group(["/World/DexLeft", "/World/DexRight"])
+        
+    for i in range(50):
+        env.step()
+
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag:
+        if not os.path.exists("Data/Wear_Baseballcap_Validation_HALO/video"):
+            os.makedirs("Data/Wear_Baseballcap_Validation_HALO/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Wear_Baseballcap_Validation_HALO/video/video", ".mp4"))
+
+    success=True
+    cur_pos=env.garment.get_garment_center_pos()
+    print(cur_pos)
+    distance = np.linalg.norm(cur_pos-np.array([env.target_put_pos[0], env.target_put_pos[1]+0.11, 1.06]))
+    if distance<0.10:
+        success=True
+    else:
+        success=False
+    
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"hat_cur_pos: {env.garment.get_garment_center_pos()}", "blue")
+    cprint(f"judge_pos: {np.array([env.target_put_pos[0],env.target_put_pos[1]+0.07,1.06])}", "blue")
+    cprint(f"distance between garment and head: {distance}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+
+
+    if validation_flag:
+        if not os.path.exists("Data/Wear_Baseballcap_Validation_HALO"):
+            os.makedirs("Data/Wear_Baseballcap_Validation_HALO")
+        # write into .log file
+        with open("Data/Wear_Baseballcap_Validation_HALO/validation_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+        if not os.path.exists("Data/Wear_Baseballcap_Validation_HALO/final_state_pic"):
+            os.makedirs("Data/Wear_Baseballcap_Validation_HALO/final_state_pic")
+        env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Wear_Baseballcap_Validation_HALO/final_state_pic/img",".png"))
+
+
+if __name__=="__main__":
+    
+    args = parse_args_val()
+    
+    # initial setting
+    pos = np.array([0.0, 0.8, 0.65])
+    ori = np.array([90.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0 
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.15, 0.15) # changeable
+            env_dy = np.random.uniform(-0.05, 0.05) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.1, 0.1) # changeable
+            y = np.random.uniform(0.7, 0.9) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([90.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Baseball_Cap/assets_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=np.random.choice(assets_list)
+
+    WearBaseballcap(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.validation_flag, args.record_video_flag, args.training_data_num, args.stage_1_checkpoint_num, args.stage_2_checkpoint_num, args.stage_3_checkpoint_num)
+
+    if args.validation_flag:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+
+simulation_app.close()

third_party/DexGarmentLab/Env_Validation/Wear_Bowlhat_HALO.py

@@ -0,0 +1,410 @@
+from isaacsim import SimulationApp
+simulation_app = SimulationApp({"headless": False})
+
+# load external package
+import os
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from isaacsim.core.utils.types import ArticulationAction, ArticulationActions
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import hat_helper_load, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_val
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud, compute_similarity
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+from Env_Config.Utils_Project.Collision_Group import CollisionGroup
+from Env_Config.Human.Human import Human
+from Env_Config.Utils_Project.Attachment_Block import attach_fixedblock
+from Model_HALO.SADP.SADP import SADP
+
+
+class WearBowlhat_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+        training_data_num:int=100,
+        stage_1_checkpoint_num:int=1500, 
+        stage_2_checkpoint_num:int=1500, 
+        stage_3_checkpoint_num:int=1500, 
+    ):
+        # load BaseEnv
+        super().__init__()
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load human
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.human = Human(
+            path="Assets/Human/human_model.usd",
+            position=[0.0+env_dx,1.15+env_dy,0.0], 
+            scale=np.array([0.6, 0.6, 0.6]),
+        )
+
+        # load garment
+        self.garment=Deformable_Garment(
+            self.world,
+            pos=np.array([0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path="Assets/Garment/Hat/HA_Hat007/HA_Hat007_obj.usd" if usd_path is None else usd_path,
+            scale=np.array([0.0075,0.0075,0.0075]),
+            youngs_modulus=1e5,
+            dynamic_friction=25.0
+        )
+
+        self.target_put_pos = hat_helper_load(self.scene, pos[0], pos[1], self.env_dx, self.env_dy)
+
+        self.helper_path=["/World/hanger_helper", "/World/hanger", "/World/Human"]
+
+        self.head_helper_path=["/World/head_helper"]
+
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-0.6, 0.0, 0.5]),
+            dexleft_ori=np.array([0.0, 0.0, 0.0]),
+            dexright_pos=np.array([0.6, 0.0, 0.5]),
+            dexright_ori=np.array([0.0, 0.0, 0.0]),
+        )
+        
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([0.0, 0.5, 6.75]), 
+            camera_orientation=np.array([0, 90.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0, -3.45, 4.17]),
+            camera_orientation=np.array([0, 40, 90]),
+            prim_path="/World/env_camera",
+        )
+        
+        # import Collision Group
+        self.collisiongroup=CollisionGroup(
+            self.world,
+            helper_path=self.helper_path,
+            garment=False,
+            collide_with_garment=True,
+            collide_with_robot=False
+        )     
+        self.collisiongroup.add_collision(group_path="head",target=self.head_helper_path)
+
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, -6.6, 4.9]),
+            camera_orientation=np.array([0, 30.0, 90.0]),
+            prim_path="/World/object_camera",
+        )
+
+        self.garment_pcd = None
+        self.object_pcd = None
+        self.points_affordance_feature = None
+        
+        self.sadp = SADP(task_name="Wear_Bowlhat_stage_1", data_num=training_data_num, checkpoint_num=stage_1_checkpoint_num)
+
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+
+        self.garment.set_garment_pose(pos=np.array([pos[0], pos[1], 0.65]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+                
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Deformable/deformable",
+            ]
+        )
+
+        self.env_camera.initialize(
+            depth_enable=True,
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Human",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+
+
+        # step world to make it ready
+        for i in range(100):
+            self.step()
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        
+        cprint("World Ready!", "green", "on_green")        
+        
+
+def WearBowlhat(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, validation_flag, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num):
+    
+    env = WearBowlhat_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num)
+    
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Deformable/deformable", "/World/hanger"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Deformable/deformable", "/World/hanger"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+    
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Human", "/World/hanger"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Human", "/World/hanger"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    if record_video_flag:
+        env.thread_record.start()
+    
+    center_y = env.garment.get_garment_center_pos()[1]
+    mask = np.abs(env.garment_pcd[:, 1] - center_y) < 0.02
+    garment_subset = env.garment_pcd[mask]
+    
+    if len(garment_subset) > 0:
+        point_x_max = garment_subset[np.argmax(garment_subset[:, 0])]
+        point_x_min = garment_subset[np.argmin(garment_subset[:, 0])]
+        cprint(f"left_manipulation_points: {point_x_min}", "green")
+        cprint(f"right_manipulation_points: {point_x_max}", "green")
+    else:
+        cprint("error!", "red", "on_red")
+        simulation_app.close()
+        
+    left_similarity = compute_similarity(env.garment_pcd, point_x_min, sigma=0.05)
+    right_similarity = compute_similarity(env.garment_pcd, point_x_max, sigma=0.05)
+    
+    env.points_affordance_feature = normalize_columns(np.concatenate([left_similarity, right_similarity], axis=1))
+        
+    left_lift_points = np.array([point_x_min[0]-0.05, point_x_min[1], 0.4])
+    right_lift_points = np.array([point_x_max[0]+0.05, point_x_max[1], 0.4])
+
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=np.array([0.653,0.653,0.271,0.271]), right_pos=right_lift_points, right_ori=np.array([0.271,0.271,0.653,0.653]), dense_sample_scale=0.005)
+    
+    for i in range(50):
+        env.step()
+    
+    left_lift_points = np.array([point_x_min[0], point_x_min[1], 0.56])
+    right_lift_points = np.array([point_x_max[0], point_x_max[1], 0.56])   
+    
+    env.bimanual_dex.dense_move_both_ik(left_pos=left_lift_points, left_ori=np.array([0.653,0.653,0.271,0.271]), right_pos=right_lift_points, right_ori=np.array([0.271,0.271,0.653,0.653]), dense_sample_scale=0.005)
+
+    for i in range(50):
+        env.step() 
+        
+    env.garment.set_mass(0.075)
+
+    for i in range(20):
+        env.step()
+    
+    for i in range(9):
+        
+        print(f"Stage_1_Step: {i}")
+
+        joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+        joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+        joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+        
+        obs = dict()
+        obs['agent_pos']=joint_state
+        obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+        obs['garment_point_cloud']=env.garment_pcd
+        obs['object_point_cloud']=env.object_pcd
+        obs['points_affordance_feature']=env.points_affordance_feature
+
+        action=env.sadp.get_action(obs)
+        
+        print("action_shape:",action.shape)
+        
+        for j in range(4):
+            
+            action_L = ArticulationAction(joint_positions=action[j][:30])
+            action_R = ArticulationAction(joint_positions=action[j][30:])
+
+            env.bimanual_dex.dexleft.apply_action(action_L)
+            env.bimanual_dex.dexright.apply_action(action_R)
+            
+            for _ in range(5):    
+                env.step()
+                
+            joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+            joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+            joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+            
+            obs = dict()
+            obs['agent_pos']=joint_state
+            obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+            obs['garment_point_cloud']=env.garment_pcd
+            obs['object_point_cloud']=env.object_pcd
+            obs['points_affordance_feature']=env.points_affordance_feature
+            
+            env.sadp.update_obs(obs)
+        
+    for i in range(50):
+        env.step()
+        
+    attach_fixedblock(env.stage,env.garment.deformable_prim_path+"/mesh/attachment",env.garment.deformable_prim_path,"/World/head_helper")
+ 
+    delete_prim_group(["/World/DexLeft", "/World/DexRight"])
+        
+    for i in range(50):
+        env.step()
+        
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag:
+        if not os.path.exists("Data/Wear_Bowlhat_Validation_HALO/video"):
+            os.makedirs("Data/Wear_Bowlhat_Validation_HALO/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Wear_Bowlhat_Validation_HALO/video/video", ".mp4"))
+
+    
+    success=True
+    # judge successful or not
+    cur_pos=env.garment.get_garment_center_pos()
+    print(cur_pos)
+    distance = np.linalg.norm(cur_pos-np.array([env.target_put_pos[0],env.target_put_pos[1]+0.11,1.06]))
+    if distance<0.05:
+        success=True
+    else:
+        success=False
+        
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"hat_cur_pos: {env.garment.get_garment_center_pos()}", "blue")
+    cprint(f"judge_pos: {np.array([env.target_put_pos[0],env.target_put_pos[1]+0.07,1.06])}", "blue")
+    cprint(f"distance between garment and head: {distance}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+
+
+    if validation_flag:
+        if not os.path.exists("Data/Wear_Bowlhat_Validation_HALO"):
+            os.makedirs("Data/Wear_Bowlhat_Validation_HALO")
+        # write into .log file
+        with open("Data/Wear_Bowlhat_Validation_HALO/validation_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+        if not os.path.exists("Data/Wear_Bowlhat_Validation_HALO/final_state_pic"):
+            os.makedirs("Data/Wear_Bowlhat_Validation_HALO/final_state_pic")
+        env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Wear_Bowlhat_Validation_HALO/final_state_pic/img",".png"))
+
+
+if __name__=="__main__":
+    
+    args = parse_args_val()
+    
+    # initial setting
+    pos = np.array([0.0, 0.8, 0.65])
+    ori = np.array([90.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0 
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.15, 0.15) # changeable
+            env_dy = np.random.uniform(-0.05, 0.05) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.1, 0.1) # changeable
+            y = np.random.uniform(0.7, 0.9) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([90.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Bowl_Hat/assets_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=np.random.choice(assets_list)
+
+    WearBowlhat(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.validation_flag, args.record_video_flag, args.training_data_num, args.stage_1_checkpoint_num, args.stage_2_checkpoint_num, args.stage_3_checkpoint_num)
+
+    if args.validation_flag:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+
+simulation_app.close()

third_party/DexGarmentLab/Env_Validation/Wear_Scarf_HALO.py

@@ -0,0 +1,470 @@
+from isaacsim import SimulationApp
+simulation_app = SimulationApp({"headless": True})
+
+# load external package
+import os
+import sys
+import time
+import numpy as np
+import open3d as o3d
+from termcolor import cprint
+import threading
+
+# load isaac-relevant package
+import omni.replicator.core as rep
+import isaacsim.core.utils.prims as prims_utils
+from pxr import UsdGeom,UsdPhysics,PhysxSchema, Gf
+from isaacsim.core.api import World
+from isaacsim.core.api import SimulationContext
+from isaacsim.core.api.objects import DynamicCuboid, FixedCuboid, VisualCuboid
+from isaacsim.core.utils.prims import is_prim_path_valid, set_prim_visibility, delete_prim
+from isaacsim.core.utils.string import find_unique_string_name
+from isaacsim.core.utils.viewports import set_camera_view
+from isaacsim.core.utils.stage import add_reference_to_stage, is_stage_loading
+from isaacsim.core.prims import SingleXFormPrim, SingleClothPrim, SingleRigidPrim, SingleGeometryPrim, SingleParticleSystem, SingleDeformablePrim
+from isaacsim.core.prims import XFormPrim, ClothPrim, RigidPrim, GeometryPrim, ParticleSystem
+from isaacsim.core.utils.rotations import euler_angles_to_quat
+from isaacsim.core.utils.types import ArticulationAction, ArticulationActions
+from omni.physx.scripts import deformableUtils,particleUtils,physicsUtils
+
+# load custom package
+sys.path.append(os.getcwd())
+from Env_StandAlone.BaseEnv import BaseEnv
+from Env_Config.Garment.Particle_Garment import Particle_Garment
+from Env_Config.Garment.Deformable_Garment import Deformable_Garment
+from Env_Config.Robot.BimanualDex_Ur10e import Bimanual_Ur10e
+from Env_Config.Camera.Recording_Camera import Recording_Camera
+from Env_Config.Room.Real_Ground import Real_Ground
+from Env_Config.Room.Object_Tools import hat_helper_load, set_prim_visible_group, delete_prim_group
+from Env_Config.Utils_Project.Code_Tools import get_unique_filename, normalize_columns
+from Env_Config.Utils_Project.Parse import parse_args_val
+from Env_Config.Utils_Project.Point_Cloud_Manip import rotate_point_cloud
+from Model_HALO.GAM.GAM_Encapsulation import GAM_Encapsulation
+from Env_Config.Utils_Project.Collision_Group import CollisionGroup
+from Env_Config.Human.Human import Human
+from Env_Config.Utils_Project.Attachment_Block import attach_fixedblock
+from Model_HALO.SADP.SADP import SADP
+
+class WearScarf_Env(BaseEnv):
+    def __init__(
+        self, 
+        pos:np.ndarray=None, 
+        ori:np.ndarray=None, 
+        usd_path:str=None, 
+        env_dx:float=0.0,
+        env_dy:float=0.0,
+        ground_material_usd:str=None,
+        record_video_flag:bool=False, 
+        training_data_num:int=100,
+        stage_1_checkpoint_num:int=1500, 
+        stage_2_checkpoint_num:int=1500, 
+        stage_3_checkpoint_num:int=1500, 
+    ):
+        # load BaseEnv
+        super().__init__()
+        
+        # ------------------------------------ #
+        # ---        Add Env Assets        --- #
+        # ------------------------------------ #
+        self.ground = Real_Ground(
+            self.scene, 
+            visual_material_usd = ground_material_usd,
+            # you can use materials in 'Assets/Material/Floor' to change the texture of ground.
+        )
+        
+        # load garment
+        self.garment = Particle_Garment(
+            self.world, 
+            # pos=np.array([0.0, 0.25, 0.15]),
+            pos=np.array([0.0, 3.0, 0.6]),
+            ori=np.array([0.0, 0.0, 0.0]),
+            usd_path="Assets/Flatten_Scarf/flatten_scarf_0.4.usd" if usd_path is None else usd_path,
+            friction=1.0,
+            particle_adhesion_scale=1.0,
+            particle_friction_scale=1.0,
+            contact_offset=0.015,         
+            rest_offset=0.01,            
+            particle_contact_offset=0.015,
+            fluid_rest_offset=0.01,
+            solid_rest_offset=0.01,
+        )
+        
+        # load human
+        self.env_dx = env_dx
+        self.env_dy = env_dy
+        self.human = Human(
+            path="Assets/Human/human_model.usd",
+            position=[0.0+env_dx, 0.9+env_dy, -0.7],
+            orientation=[90.0, 0.0, 180.0]
+        )
+        self.human_center = np.array([0.0+env_dx, 0.9+env_dy, 0.7])
+
+        # load bimanual_dex
+        self.bimanual_dex = Bimanual_Ur10e(
+            self.world,
+            dexleft_pos=np.array([-1.05, 1.15, 0.45]),
+            dexleft_ori=np.array([0.0, 0.0, -90.0]),
+            dexright_pos=np.array([1.25, 1.15, 0.45]),
+            dexright_ori=np.array([0.0, 0.0, 90.0]),
+        )
+        
+        # load camera
+        self.garment_camera = Recording_Camera(
+            camera_position=np.array([0.0, 0.75, 8.0]), 
+            camera_orientation=np.array([0, 90.0, 90.0]),
+            prim_path="/World/garment_camera",
+        )
+        
+        self.env_camera = Recording_Camera(
+            camera_position=np.array([0.0, 5.4, 5.85]),
+            camera_orientation=np.array([0, 50.0, -90.0]),
+            prim_path="/World/env_camera",
+        )
+        
+        self.judge_front_camera = Recording_Camera(
+            camera_position=np.array([self.human_center[0], 6.32, 0.42]),
+            camera_orientation=np.array([0, 0.0, -90.0]),
+            prim_path="/World/judge_front_camera",
+        )
+        
+        self.judge_back_camera = Recording_Camera(
+            camera_position=np.array([self.human_center[0], -4.2, 0.36]),
+            camera_orientation=np.array([0.0, 0.0, 90.0]),
+            prim_path="/World/judge_back_camera",
+        )
+        
+        # load UniGarmentManip Model
+        self.model = GAM_Encapsulation(catogory="Scarf")        
+
+        
+        # helper for seperating scarf
+        self.helper_1 = FixedCuboid(
+            prim_path = "/World/helper/helper_1",
+            color=np.array([0.0, 0.0, 1.0]),
+            name = "helper_1",
+            position = [0.0, 0.8, 0.0],
+            scale=[0.3, 1.0, 0.3],
+            orientation=[0.924, 0.0, 0.383, 0.0],
+            size=1.0,
+            visible = False,
+        )
+        # self.helper_2 = FixedCuboid(
+        #     prim_path = "/World/helper/helper_2",
+        #     color=np.array([0.0, 0.0, 1.0]),
+        #     name = "helper_2",
+        #     position = [0.45, 0.8, 0.0],
+        #     scale=[0.3, 1.0, 0.3],
+        #     orientation=[0.924, 0.0, 0.383, 0.0],
+        #     size=1.0,
+        #     visible = False,
+        # )
+        # self.helper_3 = FixedCuboid(
+        #     prim_path = "/World/helper/helper_3",
+        #     color=np.array([0.0, 0.0, 1.0]),
+        #     name = "helper_3",
+        #     position = [-0.45, 0.8, 0.0],
+        #     scale=[0.3, 1.0, 0.3],
+        #     orientation=[0.924, 0.0, 0.383, 0.0],
+        #     size=1.0,
+        #     visible = False,
+        # )
+
+
+        
+        # define collision group - helper path
+        self.helper_path=['/World/defaultGroundPlane/GroundPlane', '/World/Human']
+        self.collisiongroup = CollisionGroup(
+            self.world,
+            helper_path=self.helper_path,
+            garment=True,
+            collide_with_garment=True,
+            collide_with_robot=False,
+        )
+        
+        self.object_camera = Recording_Camera(
+            camera_position=np.array([0.0, 5.4, 5.85]),
+            camera_orientation=np.array([0, 50.0, -90.0]),
+            prim_path="/World/object_camera",
+        )
+
+        self.garment_pcd = None
+        self.object_pcd = None
+        self.points_affordance_feature = None
+        
+        self.sadp = SADP(task_name="Wear_Scarf_stage_1", data_num=training_data_num, checkpoint_num=stage_1_checkpoint_num)
+        
+        # ------------------------------------ #
+        # --- Initialize World to be Ready --- #
+        # ------------------------------------ #
+        # initialize world
+        self.reset()
+        
+        self.garment.set_pose(pos=np.array([pos[0], pos[1], 0.35]), ori=ori)
+        self.position = [pos[0], pos[1], 0.2]
+        self.orientation = ori
+                
+        # initialize recording camera to obtain point cloud data of garment
+        self.garment_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        # initialize gif camera to obtain rgb with the aim of creating gif
+        self.env_camera.initialize(depth_enable=True)
+        
+        self.judge_front_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        self.judge_back_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Garment/garment",
+            ]
+        )
+        
+        self.object_camera.initialize(
+            segment_pc_enable=True, 
+            segment_prim_path_list=[
+                "/World/Human",
+            ]
+        )
+        
+        # add thread and record gif Asynchronously(use to collect rgb data for generating gif)
+        if record_video_flag:
+            self.thread_record = threading.Thread(target=self.env_camera.collect_rgb_graph_for_video)
+            self.thread_record.daemon = True
+
+        # step world to make it ready
+                
+        for i in range(100):
+            self.step()
+        
+        delete_prim_group(["/World/helper/helper_1"])
+            
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        cprint(f"usd_path: {usd_path}", "magenta")
+        cprint(f"pos_x: {pos[0]}", "magenta")
+        cprint(f"pos_y: {pos[1]}", "magenta")
+        cprint(f"env_dx: {env_dx}", "magenta")
+        cprint(f"env_dy: {env_dy}", "magenta")
+        cprint("----------- World Configuration -----------", color="magenta", attrs=["bold"])
+        
+        cprint("World Ready!", "green", "on_green")
+
+def WearScarf(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, validation_flag, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num):
+    
+    env = WearScarf_Env(pos, ori, usd_path, env_dx, env_dy, ground_material_usd, record_video_flag, training_data_num, stage_1_checkpoint_num, stage_2_checkpoint_num, stage_3_checkpoint_num)
+
+    # hide prim to get object point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+    
+    env.object_pcd, color = env.object_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+        # real_time_watch=True,
+    )
+    
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Garment/garment"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    # hide prim to get garment point cloud
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Human"],
+        visible=False,
+    )
+    for i in range(50):
+        env.step()
+
+    env.garment_pcd, color = env.garment_camera.get_point_cloud_data_from_segment(
+        save_or_not=False,
+        save_path=get_unique_filename("data", extension=".ply"),
+    )
+
+    set_prim_visible_group(
+        prim_path_list=["/World/DexLeft", "/World/DexRight", "/World/Human"],
+        visible=True,
+    )
+    for i in range(50):
+        env.step()
+        
+    if record_video_flag:
+        env.thread_record.start()
+    
+    # manipulation_points, indices, max_values = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[205, 908])
+    manipulation_points, indices, points_similarity = env.model.get_manipulation_points(input_pcd=env.garment_pcd, index_list=[205, 1600])
+    
+    env.points_affordance_feature = normalize_columns(points_similarity.T)
+        
+    manipulation_points[:, 2] = -0.02
+    manipulation_points[:, 1] -= 0.05
+
+    # move to initial position
+    env.bimanual_dex.dense_move_both_ik(
+        left_pos=np.array([-0.5, 0.5, 0.5]),
+        left_ori=np.array([0.707, -0.707, 0.0, 0.0]),
+        right_pos=np.array([0.5, 0.5, 0.5]),
+        right_ori=np.array([0.0, 0.0, -0.707, 0.707]),
+    )
+    
+    env.bimanual_dex.set_both_hand_state(
+        left_hand_state='open', 
+        right_hand_state='open'
+    )
+
+    # move to grasp point
+    env.bimanual_dex.dense_move_both_ik(
+        left_pos=manipulation_points[0],
+        left_ori=np.array([0.707, -0.707, 0.0, 0.0]),
+        right_pos=manipulation_points[1],
+        right_ori=np.array([0.0, 0.0, -0.707, 0.707]),
+    )
+    
+    env.garment.particle_material.set_gravity_scale(0.5) 
+    
+    for i in range(20):
+        env.step()
+    
+    for i in range(24):
+        
+        print(f"Stage_1_Step: {i}")
+
+        joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+        joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+        joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+        
+        obs = dict()
+        obs['agent_pos']=joint_state
+        obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+        obs['garment_point_cloud']=env.garment_pcd
+        obs['object_point_cloud']=env.object_pcd
+        obs['points_affordance_feature']=env.points_affordance_feature
+
+        action=env.sadp.get_action(obs)
+        
+        print("action_shape:",action.shape)
+        
+        for j in range(4):
+            
+            action_L = ArticulationAction(joint_positions=action[j][:30])
+            action_R = ArticulationAction(joint_positions=action[j][30:])
+
+            env.bimanual_dex.dexleft.apply_action(action_L)
+            env.bimanual_dex.dexright.apply_action(action_R)
+            
+            for _ in range(5):    
+                env.step()
+                
+            joint_pos_L = env.bimanual_dex.dexleft.get_joint_positions()
+            joint_pos_R = env.bimanual_dex.dexright.get_joint_positions()
+            joint_state = np.concatenate([joint_pos_L, joint_pos_R])
+            
+            obs = dict()
+            obs['agent_pos']=joint_state
+            obs['environment_point_cloud']=env.env_camera.get_pointcloud_from_depth()
+            obs['garment_point_cloud']=env.garment_pcd
+            obs['object_point_cloud']=env.object_pcd
+            obs['points_affordance_feature']=env.points_affordance_feature
+            
+            env.sadp.update_obs(obs)
+            
+    for i in range(50):
+        env.step()
+        
+    # if you wanna create gif, use this code. Need Cooperation with thread.
+    if record_video_flag:
+        if not os.path.exists("Data/Wear_Scarf_Validation_HALO/video"):
+            os.makedirs("Data/Wear_Scarf_Validation_HALO/video")
+        env.env_camera.create_mp4(get_unique_filename("Data/Wear_Scarf_Validation_HALO/video/video", ".mp4"))
+        
+    set_prim_visible_group(["/World/DexLeft", "/World/DexRight", "/World/Human"], visible=False)
+        
+    for i in range(50):
+        env.step()
+        
+    success=True
+    scarf_front, color = env.judge_front_camera.get_point_cloud_data_from_segment(save_or_not=False)
+    scarf_back, color = env.judge_back_camera.get_point_cloud_data_from_segment(save_or_not=False)
+    
+    # calculate sum
+    front_points_below_threshold = np.sum(scarf_front[:, 2] < 0.02)
+    back_points_below_threshold = np.sum(scarf_back[:, 2] < 0.02)
+    
+    if front_points_below_threshold < 20 and back_points_below_threshold < 20:
+        success = True
+    else:
+        success = False
+        
+    set_prim_visible_group(["/World/Human"], visible=True)
+    for i in range(50):
+        env.step()
+    
+    cprint("----------- Judge Begin -----------", "blue", attrs=["bold"])
+    cprint(f"front_points_below_threshold: {front_points_below_threshold}", "blue")
+    cprint(f"back_points_below_threshold: {back_points_below_threshold}", "blue")
+    cprint("----------- Judge End -----------", "blue", attrs=["bold"])
+    cprint(f"final result: {success}", color="green", on_color="on_green")
+       
+
+    if validation_flag:
+        if not os.path.exists("Data/Wear_Scarf_Validation_HALO"):
+            os.makedirs("Data/Wear_Scarf_Validation_HALO")
+        # write into .log file
+        with open("Data/Wear_Scarf_Validation_HALO/validation_log.txt", "a") as f:
+            f.write(f"result:{success}  usd_path:{env.garment.usd_path}  pos_x:{pos[0]}  pos_y:{pos[1]}  env_dx:{env_dx}  env_dy:{env_dy} \n")
+        if not os.path.exists("Data/Wear_Scarf_Validation_HALO/final_state_pic"):
+            os.makedirs("Data/Wear_Scarf_Validation_HALO/final_state_pic")
+        env.env_camera.get_rgb_graph(save_or_not=True,save_path=get_unique_filename("Data/Wear_Scarf_Validation_HALO/final_state_pic/img",".png"))
+      
+if __name__=="__main__":
+    
+    args = parse_args_val()
+    
+    # initial setting
+    pos = np.array([0.0, 0.30, 0.65])
+    ori = np.array([90.0, 0.0, 0.0])
+    usd_path = None
+    env_dx = 0.0
+    env_dy = 0.0
+
+    if args.env_random_flag or args.garment_random_flag:
+        np.random.seed(int(time.time()))
+        if args.env_random_flag:
+            env_dx = np.random.uniform(-0.05, 0.1) # changeable
+            env_dy = np.random.uniform(-0.05, 0.05) # changeable
+        if args.garment_random_flag:
+            x = np.random.uniform(-0.05, 0.05) # changeable
+            y = np.random.uniform(0.30, 0.40) # changeable
+            pos = np.array([x,y,0.0])
+            ori = np.array([90.0, 0.0, 0.0])
+            Base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+            assets_lists = os.path.join(Base_dir,"Model_HALO/GAM/checkpoints/Scarf/assets_list.txt")
+            assets_list = []
+            with open(assets_lists,"r",encoding='utf-8') as f:
+                for line in f:
+                    clean_line = line.rstrip('\n')
+                    assets_list.append(clean_line)
+            usd_path=np.random.choice(assets_list)
+
+    WearScarf(pos, ori, usd_path, env_dx, env_dy, args.ground_material_usd, args.validation_flag, args.record_video_flag, args.training_data_num, args.stage_1_checkpoint_num, args.stage_2_checkpoint_num, args.stage_3_checkpoint_num)
+
+    if args.validation_flag:
+        simulation_app.close()
+    else:
+        while simulation_app.is_running():
+            simulation_app.update()
+
+simulation_app.close()