semvec/pypi-packages huggingface dataset:
browsergym-webarena 0.14.3
WebArena benchmark for BrowserGym
License: Apache-2.0
Requires Python: >3.7

# WebArena benchmark for BrowserGym

This package provides `browsergym.webarena`, which is an unofficial port of the [WebArena](https://webarena.dev/) benchmark for BrowserGym.

Note: the original WebArena codebase has been slightly adapted to ensure compatibility.

## Server installation

You have two options to setup your webarena instance:
 - option 1: follow the official [webarena README](https://github.com/web-arena-x/webarena/blob/main/environment_docker/README.md)
 - option 2: use our [unofficial setup scripts](https://github.com/gasse/webarena-setup/tree/main/webarena)

We recommend **option 2** as it allows you to easily customize the ports of each webarena domain, and offers a reset functionality that allows browsergym to trigger a full instance reset remotely.

## Setup

1. Install the package
```sh
pip install browsergym-webarena
```

2. Download tokenizer resources
```sh
python -c "import nltk; nltk.download('punkt_tab')"
```

3. Setup the URLs as environment variables. The ports for each domain here should correspond to those you used when setting up your webarena instance. Note also the `WA_` prefix which is specific to browsergym.
```sh
BASE_URL=<YOUR_SERVER_URL_HERE>  # example: "http://myazuremachine.eastus.cloudapp.azure.com"

# webarena environment variables (change ports as needed)
export WA_SHOPPING="$BASE_URL:8082/"
export WA_SHOPPING_ADMIN="$BASE_URL:8083/admin"
export WA_REDDIT="$BASE_URL:8080"
export WA_GITLAB="$BASE_URL:9001"
export WA_WIKIPEDIA="$BASE_URL:8081/wikipedia_en_all_maxi_2022-05/A/User:The_other_Kiwix_guy/Landing"
export WA_MAP="$BASE_URL:443"
export WA_HOMEPAGE="$BASE_URL:80"

# if your webarena instance offers the FULL_RESET feature (optional)
export WA_FULL_RESET="$BASE_URL:7565"

# otherwise, be sure to NOT set WA_FULL_RESET, or set it to an empty string
export WA_FULL_RESET=""
```

4. Setup an OpenAI API key

```sh
export OPENAI_API_KEY=...
```

> **_NOTE:_**  be mindful of costs, as WebArena will call GPT4 for certain evaluations ([llm_fuzzy_match](https://github.com/web-arena-x/webarena/blob/1469b7c9d8eaec3177855b3131569751f43a40d6/evaluation_harness/helper_functions.py#L146C5-L146C20)).
browsergym-visualwebarena 0.14.3
VisualWebArena benchmark for BrowserGym
License: Apache-2.0
Requires Python: >3.7

# VisualWebArena benchmark for BrowserGym

This package provides `browsergym.visualwebarena`, which is an unofficial port of the [VisualWebArena](https://jykoh.com/vwa) benchmark for BrowserGym.

Note: the original VisualWebArena codebase has been slightly adapted to ensure compatibility.


## Server installation

You have two options to setup your webarena instance:
 - option 1: follow the official [visualwebarena README](https://github.com/web-arena-x/visualwebarena/blob/main/environment_docker/README.md)
 - option 2: use our [unofficial setup scripts](https://github.com/gasse/webarena-setup/tree/main/visualwebarena)

We recommend **option 2** as it allows you to easily customize the ports of each webarena domain, and offers a reset functionality that allwos browsergym to trigger a full instance reset remotely.

## Setup

1. Install the package
```sh
pip install browsergym-visualwebarena
```

2. Download tokenizer resources
```sh
python -c "import nltk; nltk.download('punkt_tab')"
```

3. Setup the URLs as environment variables. The ports for each domain here should correspond to those you used when setting up your webarena instance. Note also the `VWA_` prefix which is specific to browsergym.
```sh
BASE_URL=<YOUR_SERVER_URL_HERE>  # example: "http://myazuremachine.eastus.cloudapp.azure.com"

# visualwebarena environment variables (change ports as needed)
export VWA_CLASSIFIEDS="$BASE_URL:8083"
export VWA_CLASSIFIEDS_RESET_TOKEN="4b61655535e7ed388f0d40a93600254c"
export VWA_SHOPPING="$BASE_URL:8082"
export VWA_REDDIT="$BASE_URL:8080"
export VWA_WIKIPEDIA="$BASE_URL:8081"
export VWA_HOMEPAGE="$BASE_URL:80"

# if your webarena instance offers the FULL_RESET feature (optional)
export VWA_FULL_RESET="$BASE_URL:7565"

# otherwise, be sure to NOT set VWA_FULL_RESET, or set it to an empty string
export VWA_FULL_RESET=""
```

4. Setup an OpenAI API key

```sh
export OPENAI_API_KEY=...
```
browsergym-miniwob 0.14.3
MiniWoB++ benchmark for BrowserGym
License: Apache-2.0
Requires Python: >3.7

# Miniwob benchmark for BrowserGym

This package provides `browsergym.miniwob`, which is an unofficial port of the [MiniWoB++](https://miniwob.farama.org/) benchmark for BrowserGym.

## Setup

### Option 1: Automated setup (Recommended)

If you're working from the BrowserGym root directory, you can use the Makefile for automated setup:

```sh
make setup-miniwob
```

This will:

- Clone the MiniWoB++ repository
- Reset to the specific commit for reproducibility  
- Add the `MINIWOB_URL` to your `.env` file

Then load the environment variables:

```sh
source .env
```

### Option 2: Manual setup

1. Install the package

```sh
pip install browsergym-miniwob
```

1. Clone miniwob (use a specific frozen commit for reproducibility)

```sh
git clone git@github.com:Farama-Foundation/miniwob-plusplus.git
git -C "./miniwob-plusplus" reset --hard 7fd85d71a4b60325c6585396ec4f48377d049838
```

1. Setup Miniwob URL (change `PATH_TO_MINIWOB_CLONED_REPO` here to the absolute path to your `miniwob-plusplus` folder)

```sh
export MINIWOB_URL="file://<PATH_TO_MINIWOB_CLONED_REPO>/miniwob/html/miniwob/"
```

Alternatively, one can [setup a simple HTTP server](https://miniwob.farama.org/content/viewing/) and use a proper URL.
browsergym-experiments 0.14.3
Experimentation tools for BrowserGym
License: Apache-2.0
Requires Python: >3.7

# BrowserGym experiments

This package provides `browsergym.experiments`, a suite of experimentation tools for [BrowserGym](https://github.com/ServiceNow/BrowserGym).

As a convenience namespace, it also provides `bgym`.

## Setup

1. Install the package
```sh
pip install browsergym-experiments
```
browsergym-core 0.14.3
BrowserGym: a gym environment for web task automation in the Chromium browser
License: Apache-2.0
Requires Python: >3.9

# BrowserGym core

This package provides `browsergym.core`, which provides the core functionalities of [BrowserGym](https://github.com/ServiceNow/BrowserGym).

## Setup

1. Install the package
```sh
pip install browsergym-core
```
browsergym-assistantbench 0.14.3
AssistantBench benchmark for BrowserGym
License: Apache-2.0
Requires Python: >3.7

# AssistantBench <> BrowserGym

This package provides an implementation for using the [AssistantBench](https://assistantbench.github.io/) benchmark in BrowserGym.

Because AssistantBench includes open-ended tasks, setup is extremely easy and simply requires installing the package.

Please note that AssistantBench has a hidden test set, so test set predictions will need to be uploaded to the official [leaderboard](https://huggingface.co/spaces/AssistantBench/leaderboard).

## Setting up

- Install the package (this is still a wip)
```
pip install browsergym-assistantbench
```

- Run inference, e.g., run the following commands for demo on a simple toy task
```
python demo_agent/run_demo.py --task_name assistantbench.validation.3
```

- Test set predictions will be saved to `./assistantbench-predictions-test.jsonl`. To evaluate on the official test set, upload these predictions to the official [leaderboard](https://huggingface.co/spaces/AssistantBench/leaderboard).