Repository: stanford-crfm/helm
Issue: Weird behavior when installing from source
Root cause: Configuration Error

I know the official instructions say to download the stable version using `pip install crfm-helm`


But I've tried cloning and installing from source to allow file editing using `pip install -e .` from the cloned repo.

When installing dependent packages, it seems unable to find a valid version of `boto3` and goes on searching all possible versions for a solution. Issue might be unrelated to this repo but thought I would flag for awareness. At the beginning it prints

```
INFO: pip is looking at multiple versions of boto3 to determine which version is compatible with other requirements. This coul
d take a while.
```

Then it goes on for 1000s of lines trying to find a compatible version:


```
Collecting boto3
  Downloading boto3-1.23.6-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 44.2 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.6
  Downloading botocore-1.26.6-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.8 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.5-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 32.6 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.5
  Downloading botocore-1.26.5-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.4 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.4-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 39.2 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.4
  Downloading botocore-1.26.4-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 55.8 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.3-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 44.1 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.3
  Downloading botocore-1.26.3-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.7 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.2-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 41.4 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.2
  Downloading botocore-1.26.2-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.5 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.1-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 41.7 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.1
  Downloading botocore-1.26.1-py3-none-any.whl (8.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/8.7 MB 69.3 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.0-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 42.8 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.0
  Downloading botocore-1.26.0-py3-none-any.whl (8.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/8.7 MB 53.0 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.22.13-py3-none-any.whl (132 kB)
```



Looks like this is a [common issue in pip](https://pip.pypa.io/en/stable/topics/dependency-resolution/#possible-ways-to-reduce-backtracking), resolved this by manually installing boto3 before installing helm.

Root cause seems to be [coming from here](https://pypi.org/project/pytorch-pretrained-bert/) 

When installing from source, you can do `pip install -r requirements-freeze.txt`. This will install the [exact package versions](https://github.com/stanford-crfm/helm/blob/main/requirements-freeze.txt) that the GitHub Actions CI uses for installing from source and testing. This should allow you to avoid the pip dependency resolution issues.

Did this fix your issue?

I'm planning to make Boto an optional dependency as part of #1327, which should fix this issue.

boto3 is now an optional dependency.