Repository: microsoft/promptbench
Issue: Try to run the basic sample
Root cause: External Dependency Breakage

I am trying to run basic sample and the error has been reported:
(.venv) PS C:\Users\allan\Documents\promptbench> & c:/Users/allan/Documents/promptbench/.venv/Scripts/python.exe c:/Users/allan/Documents/promptbench/examples/basic.py
Traceback (most recent call last):
  File "c:\Users\allan\Documents\promptbench\examples\basic.py", line 5, in <module>
    import promptbench as pb
  File "C:\Users\allan\Documents\promptbench\promptbench\__init__.py", line 6, in <module>
    from .dataload import DatasetLoader, SUPPORTED_DATASETS, SUPPORTED_DATASETS_VLM
  File "C:\Users\allan\Documents\promptbench\promptbench\dataload\__init__.py", line 4, in <module>
    from .dataset import *
  File "C:\Users\allan\Documents\promptbench\promptbench\dataload\dataset.py", line 13, in <module>
    from datasets import load_dataset
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\datasets\__init__.py", line 18, in <module>
    from .arrow_dataset import Dataset
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\datasets\arrow_dataset.py", line 76, in <module>
    from .arrow_reader import ArrowReader
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\datasets\arrow_reader.py", line 32, in <module>
    from .download.download_config import DownloadConfig
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\datasets\download\__init__.py", line 9, in <module>
    from .download_manager import DownloadManager, DownloadMode
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\datasets\download\download_manager.py", line 35, in <module>
    from ..utils.file_utils import (
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\datasets\utils\file_utils.py", line 37, in <module>
    from aiohttp.client_exceptions import ClientError
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\aiohttp\__init__.py", line 6, in <module>
    from .client import (
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\aiohttp\client.py", line 85, in <module>
    from .connector import (
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\aiohttp\connector.py", line 34, in <module>
    import aiohappyeyeballs
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\aiohappyeyeballs\__init__.py", line 3, in <module>
    from .impl import start_connection
  File "C:\Users\allan\Documents\promptbench\.venv\lib\site-packages\aiohappyeyeballs\impl.py", line 22, in <module>
    socket_factory: Optional[SocketFactoryType] = None,
  File "C:\Python39\lib\typing.py", line 243, in inner
    return func(*args, **kwds)
  File "C:\Python39\lib\typing.py", line 316, in __getitem__
    return self._getitem(self, parameters)
  File "C:\Python39\lib\typing.py", line 433, in Optional
    return Union[arg, type(None)]
  File "C:\Python39\lib\typing.py", line 243, in inner
    return func(*args, **kwds)
  File "C:\Python39\lib\typing.py", line 316, in __getitem__
    return self._getitem(self, parameters)
  File "C:\Python39\lib\typing.py", line 421, in Union
    parameters = _remove_dups_flatten(parameters)
  File "C:\Python39\lib\typing.py", line 215, in _remove_dups_flatten
    all_params = set(params)
TypeError: unhashable type: 'list'

Any help about that? Thanks in advance.

Hi, can you share the code you are running with me?

I tested the following code, it worked well.
```
import promptbench as pb

dataset = pb.DatasetLoader.load_dataset("sst2")
print(dataset)
```

BTW, can you please check if the `datasets` package has been successfully installed?



The code is a python source copied from the notebook "basic.ipynb" and is
following attached.
Thanks.

Em seg., 17 de mar. de 2025 às 17:03, Kaijie Zhu ***@***.***>
escreveu:

> Hi, can you share the code you are running with me?
>
> I tested the following code, it worked well.
>
> import promptbench as pb
>
> dataset = pb.DatasetLoader.load_dataset("sst2")
> print(dataset)
>
> BTW, can you please check if the datasets package has been successfully
> installed?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/microsoft/promptbench/issues/94#issuecomment-2730707219>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
> [image: Immortalise]*Immortalise* left a comment
> (microsoft/promptbench#94)
> <https://github.com/microsoft/promptbench/issues/94#issuecomment-2730707219>
>
> Hi, can you share the code you are running with me?
>
> I tested the following code, it worked well.
>
> import promptbench as pb
>
> dataset = pb.DatasetLoader.load_dataset("sst2")
> print(dataset)
>
> BTW, can you please check if the datasets package has been successfully
> installed?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/microsoft/promptbench/issues/94#issuecomment-2730707219>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


May I ask for more help?

I am trying to load a pre-trained model locally, it is a derivation of
llama 3.1, can I load this model locally and make some attacks against it?

BTW!

Em qui., 20 de mar. de 2025 às 12:41, Allan Chamon Figueiredo <
***@***.***> escreveu:

> The code is a python source copied from the notebook "basic.ipynb" and is
> following attached.
> Thanks.
>
> Em seg., 17 de mar. de 2025 às 17:03, Kaijie Zhu ***@***.***>
> escreveu:
>
>> Hi, can you share the code you are running with me?
>>
>> I tested the following code, it worked well.
>>
>> import promptbench as pb
>>
>> dataset = pb.DatasetLoader.load_dataset("sst2")
>> print(dataset)
>>
>> BTW, can you please check if the datasets package has been successfully
>> installed?
>>
>> —
>> Reply to this email directly, view it on GitHub
>> <https://github.com/microsoft/promptbench/issues/94#issuecomment-2730707219>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE>
>> .
>> You are receiving this because you authored the thread.Message ID:
>> ***@***.***>
>> [image: Immortalise]*Immortalise* left a comment
>> (microsoft/promptbench#94)
>> <https://github.com/microsoft/promptbench/issues/94#issuecomment-2730707219>
>>
>> Hi, can you share the code you are running with me?
>>
>> I tested the following code, it worked well.
>>
>> import promptbench as pb
>>
>> dataset = pb.DatasetLoader.load_dataset("sst2")
>> print(dataset)
>>
>> BTW, can you please check if the datasets package has been successfully
>> installed?
>>
>> —
>> Reply to this email directly, view it on GitHub
>> <https://github.com/microsoft/promptbench/issues/94#issuecomment-2730707219>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE>
>> .
>> You are receiving this because you authored the thread.Message ID:
>> ***@***.***>
>>
>


I am using datasets package version 2.15.0, I´ve tried to use the 2.19.2 as well, the same erros happens when I try to execute the code snippet you suggested:

`import promptbench as pb

dataset = pb.DatasetLoader.load_dataset("sst2")
print(dataset)`

TypeError: unhashable type: 'list'

Solved! I have changed the version of packages below in order to create the conda enviroment correctly:
sentencepiece==0.1.99->0.2.0
tokenizers==0.15.0->0.15.1

I have the packages below so that I could use CUDA support:
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia



Stale issue message