zhiminy/EvalEng huggingface dataset:
Repository: stanford-futuredata/ARES
Issue: Errors in tutorials and missing module for PPI evaluation
Root cause: Unimplemented Feature Gap

Hello,

I encountered issues in each of the tutorials, requiring modifications to the code to make them work.

Now, during the evaluation with PPI, I get the following error:
ModuleNotFoundError: No module named 'ares.LLM_as_a_Judge_Adaptation.Late_Chunking_Classifier'

This indicates that the module "Late_Chunking_Classifier" does not exist, which is clearly not normal.
Additionally, just before encountering this error, I had to manually add the parameter "use_late_chunking", which seems to be required but is not defined anywhere in the code!

Could you please provide clarification or a fix for these issues?

Thank you!
Repository: stanford-futuredata/ARES
Issue: RAG systems evaluation
Root cause: Documentation Deficiency

Hi, 

I am trying to reproduce **Figure 2: RAG Systems Evaluation on NQ - Context Relevance** and figure 3 as shown in your paper. I have not found any code file which will help me to generate figure 2 ad figure 3. Could you please suggest where to find this code file or I am missing something? 

Figures 2 and 3 are basically where you have used different retrievers like BM25, openAI Ada and Colbert with different LLMs for generation. 

<img width="439" alt="Image" src="https://github.com/user-attachments/assets/eb9372fe-8aee-4575-8ad3-3ad5c874d071" />
Repository: stanford-futuredata/ARES
Issue: What is short for UES/IDP?

I cannot find any related information in the paper or this repository.

Hi @zhimin-z,

Thanks for your question. UES/IDP stands for "Unlabeled Evaluation Set / In-Domain Prompts". This refers to the pipeline that uses in-domain prompts (few-shot prompting) with models such as GPT-4o, Claude, etc. to evaluate and label our unlabeled evaluation set.

The UES component represents the datasets we need to evaluate that don't have pre-existing labels, while the IDP component describes the methodology of using prompts from the same domain to help the models generate appropriate evaluations/labels for these samples.

Let me know if you have any further questions!
Repository: stanford-futuredata/ARES
Issue: Cannot run the tutorial example
Root cause: Validation Gap

Check the tutorial and run the following code:
```
from ares import ARES

ues_idp_config = {
    # Dataset for in-domain prompts
    "in_domain_prompts_dataset": "/content/nq_few_shot_prompt_for_judge_scoring.tsv",

    # Dataset for unlabeled evaluation
    "unlabeled_evaluation_set": "/content/nq_unlabeled_output.tsv",

    # Model: GPT-3.5
    "model_choice" : "gpt-3.5-turbo-0125"
}

# Optional: Provide an alternative model of your choice below.
# Here are some models you can choose from:
# - mistralai/Mistral-7B-Instruct-v0.2
# - mistralai/Mixtral-8x7B-Instruct-v0.1
# - gpt-4-turbo-preview
# - microsoft/deberta-v3-large
# - openlm-research/open_llama_7b_v2
# - mosaicml/mpt-7b-instruct
```
However, it throws the following error:
<img width="797" alt="Image" src="https://github.com/user-attachments/assets/4016fb32-2b87-4f64-80cc-353cc0207407" />
Repository: stanford-futuredata/ARES
Issue: Unable to replicate results from the tutorials
Root cause: Documentation Deficiency

The tutorials here don't work readily, with a few errors arising from how the main functions are created.

- `NameError: name 'use_late_chunking' is not defined` It seems like `late_chunking` is to be defined, but it's not.
- `ValueError: Missing required parameter 'azure_openai_config' for ppi.` Though the parameter is optional, it is not possible to configure ppi without azure creds.
- Azure creds are required even when a custom model is being specified.
Repository: stanford-futuredata/ARES
Issue: Is the project still supported?
Root cause: Environment Incompatibility

There are no commits after April 2025. Python versions 3.12+ are not supported.
There is a necessity to use too old versions of vllm (0.4.0), torch (1.13 - 2.2), etc. Even if I install the env correctly, there are errors during runtime in the tutorial  examples.
The FLAN-T5 XXL model reported in the paper is a bit obsolete, as for December 2025. 
Is the repo still supported?
Did anyone test the possibility to apply modern models (GPT 4+, Deepseek, etc.) for synthetic dataset generation?

I am checking other options like ragas and deepeval
Repository: stanford-futuredata/ARES
Issue: String "None" defaults cause TypeError in UES/IDP evaluation
Root cause: Validation Gap

Title: TypeError: string indices must be integers when using default OpenAI path in UES/IDP

Description:

When running UES/IDP evaluation with the standard OpenAI API (not Azure, not vLLM), the evaluation fails with "TypeError: string indices must be integers, not 'str'" if azure_openai_config is not explicitly provided in the config.

Root Cause:

In ares/ares.py, several optional parameters use the string "None" as the default value instead of Python None:

    "documents": (int, "None"),           # Line 32
    "azure_openai_config": (dict, "None"), # Line 36  
    "host_url": (str, "None")              # Line 37

Since the string "None" is truthy, the condition "if azure_openai_config:" in ues_idp.py evaluates to True, causing the code to incorrectly enter the Azure OpenAI branch. The Azure branch then attempts dictionary access on the string "None", which raises the TypeError.

Steps to Reproduce:

    from ares import ARES

    config = {
        'in_domain_prompts_dataset': 'few_shot.tsv',
        'unlabeled_evaluation_set': 'unlabeled.tsv',
        'model_choice': 'gpt-3.5-turbo-0125',
        'documents': 1,
    }

    ares = ARES(ues_idp=config)
    results = ares.ues_idp()  # Raises TypeError

Error Output:

    TypeError: string indices must be integers, not 'str'

Suggested Fix:

Replace string "None" defaults with falsy values that work correctly with truthiness checks:

    "documents": (int, 0),
    "azure_openai_config": (dict, {}),
    "host_url": (str, "")

And update the corresponding checks in ues_idp.py from string comparisons to truthiness checks.

Environment:

    ARES version: 0.6.6
    Python: 3.11
    OpenAI SDK: 2.14.0

Workaround:

Explicitly pass azure_openai_config=None in the config dictionary.
Repository: stepjam/RLBench
Issue: How can I replace the default with another robotic arm?
Root cause: Documentation Deficiency

If I want to replace the default arm with another robotic arm in the lab, how can make it?

Hi,

You can edit the V-REP file `rlbench/task_design.ttt` and import the robot that you need. You will then need to modify [this line](https://github.com/stepjam/RLBench/blob/master/rlbench/environment.py#L81). This relies on the fact that the robot is already in PyRep. I plan to include better documentation for this. Expect this in about a weeks time :)

Keep in mind a few things:
- The tasks have only been tested on the panda.
- Any sim-only benchmarking should be done on the panda.

Following this question, how can we add multiple arms into rlbench environment?
Repository: stepjam/RLBench
Issue: Multiple arms support? 
Root cause: Unimplemented Feature Gap

Hi Stephen, 

Thanks for open sourcing your work! Is there a way to add more arms into the environment and design tasks that use multiple arms? 

Thanks!

Hi,
Sorry for the delay in getting back to you (I've been away).

Multi arm support would be very nice! It shouldn't be too difficult to add in using a fork, but unfortunately I am far too swamped to add this in at the moment. If you want to give a stab at this, you will need to change how scene.py/task.py does the waypoint grabbing for the demos. At the moment, they look for dummies named 'waypoint0', 'waypoint1', etc. You might want to change this to 'arm0_waypoint0', 'arm1_waypoint0', etc. You will also need to add the arms (which hopefully are already implemented in PyRep) to the scene file (.ttt).

Best,
Stephen
Repository: stepjam/RLBench
Issue: typo in https://github.com/stepjam/RLBench/blob/master/rlbench/backend/scene.py

There is a typo in https://github.com/stepjam/RLBench/blob/master/rlbench/backend/scene.py
Should we replace `self._inital_task_state`  with `self._initial_task_state`?

Thanks for pointing out. Will fix this in my next push.
Repository: stepjam/RLBench
Issue: Fake Grasping Problem
Root cause: Architectural Constraint

As can be seen, there is a problem with fake grasping. The agent learns a bad policy (it seems to be easier to learn fake grasping).
![Problem_Fake_Grasping2](https://user-images.githubusercontent.com/69359729/105926609-a5c21480-6042-11eb-8354-138ffc8aca9e.gif)

I have read this #97, but setting `attach_grasped_objects=False` seems to turn off the rewards also. The latter wouldn't make sense in my eyes. Is there a possibility to set `attach_grasped_objects=False` but keep the reward?

Yes, fake grasping will indeed make the task easier, as it makes it easy for the policy to exploit this.
Indeed, setting `attach_grasped_objects=False` should not disable the reward; I'll take a look and report back.

After taking a quick look, It's not obvious to me what the reward would be disabled...
What exactly do you mean by the reward being disabled? How are you checking this?
have you shaped the reward to learn this policy? Or is it learned from the given sparse reward?

Thanks for the quick reply!
Sorry for my inaccurate description, it was late already.. 
So what I meant was that this particular task (derivate of "pick and lift") uses the `GraspedCondition` as a part if its reward. The latter checks for grasped objects at the gripper. However, the gripper will never grasp an object if `attach_grasped_objects=False`. So the `GraspedCondition`  won't be me. Or am I wrong?

This policy is learned by shaped rewards, including rewards given by (or based on) the `GraspedCondition`. 


Sorry for delay!! Since it’s been so long I’ll assume this is no longer relevant, but please open new issue if needed.
Repository: stepjam/RLBench
Issue: Training time for RLBench gym environments
Root cause: Documentation Deficiency

Hi, I am trying to train a Hierarchical Reinforcement Learning (HRL) agent to solve the tasks (1) open-box, (2) block-pyramid and (3) place_shape_in_shape_sorter using just the wrist-rgb camera view.

I am using option-critic algorithm to learn, and the problem with the environments is the sparse rewards. With environment like the block pyramid and shape sorter, I don't think it is feasible to ever encounter the reward through random exploration, thus never giving a learning signal to the agent.

How long does it take an average for an agent to learn these tasks? I have set the training to be 2e7 timesteps, and it is taking an absurdly long time to complete training (~3000 hours). 

Below are my episode return curves for the environments. The agent seems to be able to get the positive reward often enough through random exploration for the open-box environment, but totally seem to be stuck with 0 returns for the block-pyramid and shape sorter environment.

Open-box:
![image](https://user-images.githubusercontent.com/27609953/106363467-aff65280-6363-11eb-800b-352434830146.png)

Block-pyramid:
![image](https://user-images.githubusercontent.com/27609953/106363522-05cafa80-6364-11eb-8215-a4ade986ac89.png)

Place_Shape_in_shape_sorter:
![image](https://user-images.githubusercontent.com/27609953/106363524-082d5480-6364-11eb-98fb-14848e8a267e.png)

What methods do most people use to successfully learn these environments?

Hi, thanks for your question.

RLBench can be seen as a sort of grand challenge for robotics. The idea is that in the real world we don't have shaped rewards, and must find algorithms that can learn with sparse rewards; potentially by making use of demonstrations instead. The paper outlines this best: https://arxiv.org/abs/1909.12271
You can't expect to take a method off the shelf and expect it to work well on RLBench, this is not what the benchmark was intended to do. 

So to answer your question:
`What methods do most people use to successfully learn these environments?`
The short answer is that there currently doesn't exist an exploration-only based RL method that can work on all tasks.

All the best,
Stephen
Repository: stepjam/RLBench
Issue: is gym-env equivalent?

I am learning close-drawer, and looks that the gym version i.e. (gym.make("close_drawer-state-v0")), looks much easier to solve.
My question is: are the two task equivalent?

Hi. Thanks for your question.

RLBench allows you to customise what observations you want through the [ObservationConfig](https://github.com/stepjam/RLBench/blob/master/rlbench/gym/rlbench_env.py#L26). By default, not all of the observations are turned on. However, when you use the Gym wrapper, it turns on all the low dimensional observations ([see here](https://github.com/stepjam/RLBench/blob/master/rlbench/gym/rlbench_env.py#L26)). That is the only difference between the gym version and the 'vanilla' version.

Closing due to inactivity. I'll assume the response above answered the question.
Repository: stepjam/RLBench
Issue: Missing step in task creation tutorial.
Root cause: Documentation Deficiency

I might be missing something, but it looks like an important step in creating the task in https://github.com/stepjam/RLBench/blob/master/tutorials/simple_task.md is to make the box not collidable. Otherwise the path will be planned around the box as opposed to through it.

Hi,
Sorry for the delay in getting back to you (I've been away).
Thanks for pointing out! Will fix this soon!
Best,
Stephen

I thought the missing step is to add `ignore_collision()` in _waypoint1_, and I created a pull request #80 

Thanks. Merged into master.
Repository: stepjam/RLBench
Issue: Question: How to add Robotiq Hand-E Gripper or Other Custom Grippers?

I'm interested in using the Robotiq Hand-E gripper and possibly a custom designed gripper with RLBench.  I understand that these would needs to be added to PyRep first - presumably following a process similar to the [adding robots](https://github.com/stepjam/PyRep/blob/master/tutorials/adding_robots.md) tutorial.

What extra steps are required to make sure the grippers are comparable with RLBench?  Are there plans to add a guide on how to do this?

Hi. Once the gripper is in PyRep then it's easy for RLBench; basically just need to attach it to an arm of your choice, save the model into the repo, and the add the model to the list of available configurations [here](https://github.com/stepjam/RLBench/blob/master/rlbench/environment.py#L35).

I recommend taking a look at some of the existing grippers in PyRep and going from there :)
Repository: stepjam/RLBench
Issue: Running one of the examples in headless mode
Root cause: Environment Incompatibility

Hi, I'm just trying to use one of the examples to make sure that I install all the components correctly.
The command I was using is
`xvfb-run python3 few_shot_rl.py`.
And in the python file I adjusted the env variable into
`Environment(action_mode, obs_config=obs_config, headless=True)`.
However, I still get an error going as the attachment file.
![捕获](https://user-images.githubusercontent.com/71317990/110201763-cd0ccc80-7e9f-11eb-9449-b91d00702f77.PNG)
I'm sure that I installed PyRep correctly, since I could use the same command to run one of examples there.
Could you please help me firgure where the problem is and how to fix it?


Hi,
When running headless, could you try running Xvfb like this instead:
```
Xvfb :99 -screen 0 1024x768x24 +extension GLX +render -noreset &
export DISPLAY=:99
python3 few_shot_rl.py
```

Also, I assume you do not get this error when **not** running headless?

@Bowen-He Did this resolve your issue?

@stepjam Yes, kind of. Now I can use the environment by using Gym. But still got the problem with other modes. I think there might be some conflicts with my system, I'll check it out and try to correct them. Thanks for your help! And I think we can close the issue now!

Feel free to reopen if you have any other queries

You can now do GPU rendering: https://github.com/stepjam/RLBench/blob/master/README.md#running-headless
Repository: stepjam/RLBench
Issue: Multiple "Dummy" created
Root cause: Resource Mishandling

Hello, thank you very much for this nice library! 
I am currently trying to train on multiple environments, so I often need to do `env.get_task(....)`. The problem is, every time I do it a new "Dummy" is created in the simulation gui. Because of this, after a while I even get an error in terminal: `OSError: [Errno 28] No space left on device:`. Do you know how to solve this?
The multiple dummies can simply be reproduced by running:
```
from rlbench.environment import Environment
from rlbench.action_modes import ArmActionMode, ActionMode
from rlbench.observation_config import ObservationConfig
from rlbench.tasks import MT30_V1
import numpy as np


class Agent(object):

    def __init__(self, action_size):
        self.action_size = action_size

    def act(self, obs):
        arm = np.random.normal(0.0, 0.1, size=(self.action_size - 1,))
        gripper = [1.0]  # Always open
        return np.concatenate([arm, gripper], axis=-1)


obs_config = ObservationConfig()
obs_config.set_all(True)

action_mode = ActionMode(ArmActionMode.ABS_JOINT_VELOCITY)
env = Environment(
    action_mode, obs_config=obs_config, headless=False)
env.launch()

agent = Agent(env.action_size)

train_tasks = MT30_V1['train']

training_cycles_per_task = 30
training_steps_per_task = 80
episode_length = 40

for _ in range(training_cycles_per_task):

    task_to_train = np.random.choice(train_tasks, 1)[0]
    task = env.get_task(task_to_train)
    task.sample_variation()  # random variation

    for i in range(training_steps_per_task):
        if i % episode_length == 0:
            print('Reset Episode')
            descriptions, obs = task.reset()
            print(descriptions)
        action = agent.act(obs)
        obs, reward, terminate = task.step(action)

print('Done')
env.shutdown()
```

which is the example you provided, only with more iteration of environments. The result with multiple dummies looks like this
![Screenshot from 2021-03-08 18-29-50](https://user-images.githubusercontent.com/43339851/110358214-712b7a80-803c-11eb-9f48-8e2548719db1.png)

Thank you again for all the help :)

Thanks for reporting. I'll take a look today and report back 👍 

This has now been fixed in version 1.1.0. 
I'm expecting to have 1.1.0 merged into main in the next few days.

Thank you very much for solving this and for the library in general, it's really nice. Keep up with the good work :)
Repository: stepjam/RLBench
Issue: how to change robot's initial pose in the init_episode
Root cause: Documentation Deficiency

Hi, thank you for sharing your work first. It's really good.

I am going to change the initial pose of robots in the _def init_episode_ function. I have been attempted in a way shown in _def _set_to_init_pose(self):_, but it seems don't work.

Hope for your reply！
Here is my code.
```
class PegInHoleDirect(Task):
   def init_episode(self, index: int) -> List[str]:
        self._set_to_init_pose()
        hole_boundary = SpawnBoundary([self.hole_boundary])
        hole_boundary.sample(self.hole,max_rotation=[0,0,0],min_rotation=[0,0,0])
        self.waypoint0 = Dummy('waypoint0')
        self.waypoint1 = Dummy('waypoint1')
        self.waypoint0.set_pose([0,0,-0.2,0,0,0,1],relative_to=self.waypoint1)
        return ['peg in hole task']

  def _set_to_init_pose(self):
      current_tip_pose = self.robot.arm.get_tip().get_pose()
      target_tip_pose = copy.deepcopy(current_tip_pose)

      target_tip_pose[:3] = [2.8e-1,0,1.2]
      target_tip_pose[3:] = [0,1,0,0]
      path = self.robot.arm.get_path(position=target_tip_pose[:3],quaternion=target_tip_pose[3:])
      done = False
      while not done:
          done = path.step()
          self.pyrep.step()
 .....
```


Hi, thanks for the question.
To set the initial pose of the arm, you can simply call:
`self.robot.arm.set_joint_positions(my_initial_joint_values)`

If you want a more visual way of setting the pose, you could also edit the task_design.ttt file in CoppeliaSim and set the joint there (fi you want them to be consistent across all tasks)

> Hi, thanks for the question.
> To set the initial pose of the arm, you can simply call:
> `self.robot.arm.set_joint_positions(my_initial_joint_values)`
> 
> If you want a more visual way of setting the pose, you could also edit the task_design.ttt file in CoppeliaSim and set the joint there (fi you want them to be consistent across all tasks)

Thanks for your reply.
Actually, I want to modify the initial pose of the arm in every different episode.
I study the source code and find it can be solved by adding `self.robot.arm.set_control_loop_enabled(True)` before `self.robot.arm.get_path()` of  `def _set_to_init_pose(self)` function
Repository: stepjam/RLBench
Issue: Customizing the reward as dense function
Root cause: Documentation Deficiency

I first appreciate the contributors for maintaining this awesome and handy API. It really helps to accelerate my research on robot learning.

I am working on the customized RL environment that generates the dense reward signal (e.g. reward proportional to negative euclidean distance between the end-effector and the  target object).

I somehow can modify the low-level modules like `step()` method in `task_environment,py `to generate a dense reward signal.  
However, I find it a little hacky in that I should modify the module that is shared by multiple environments.

AFAIK, RLBench is designed to learn the sparse reward tasks, so I assume the manual modification of low-level modules is inevitable for my case. 

If not, I would be grateful for the help regarding this issue.

Thanks in advance!

Hi,
RLBench does actually support the addition of user specified rewards. All you have to do is override the [`reward()`](https://github.com/stepjam/RLBench/blob/master/rlbench/backend/task.py#L115) function in the desired task. E.g. Adding a reward() function to reach_target.py.

@stepjam Oh, why haven't I thought about that! Thank you for the quick response!!!


@mch5048 
Did you succeed? I added the reward function in open_drawer.py but the reward in code: 'obs, reward, terminate = task.step(action)' is always 0