Repository: Unbabel/COMET
Issue: COMET not working with python 3.8
Root cause: External Dependency Breakage

## 🐛 Bug

COMET works with python 3.6 but not with python 3.8

### Environment
OS: MacOS
Packaging: pip
Version 1.0.0rc6

### To Reproduce 

I pip installed comet on my python 3.8.12 virtual environment and then tested the example provided in the readme **Scoring with Python**

```python
seg_scores, sys_score = model.predict(data, batch_size=8, gpus=0)
```
But I get the following error:

```
/usr/local/Cellar/python@3.8/3.8.12/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py in _launch(self, process_obj)
     45         try:
     46             reduction.dump(prep_data, fp)
---> 47             reduction.dump(process_obj, fp)
     48         finally:
     49             set_spawning_popen(None)

/usr/local/Cellar/python@3.8/3.8.12/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py in dump(obj, file, protocol)
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)
     61 
     62 #

AttributeError: Can't pickle local object 'CometModel.predict.<locals>.<lambda>'
```

I tested the same code on python 3.6 and it did work, so thanks a lot :)

I expected comet to work with >=python3.5
Is there any plan to make it work for python 3.8?

Thanks again.

_Is there any plan to make it work for python 3.8?_

Yes, I already tested it before in python 3.8 and 3.9. It should support python 3.6, 3.7, 3.8, and 3.9. I'll take a look at it tomorrow. It is probably just a reqs/dependency problem...

Btw does the command: `comet-score -s src.de -t hyp.en -r ref.en` work?


Hi, I had the same issue. I switched to Python 3.7 and pandas==1.1.5 to make it work for me. (Installed from pip & running locally- no GPU)

> Btw does the command: `comet-score -s src.de -t hyp.en -r ref.en` work?

It does not work either. I get:
```python
File "/usr/local/Cellar/python@3.8/3.8.12/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'CometModel.predict.<locals>.<lambda>'
```

@joncamgui  I got the same error but using 3.7 and pandas==1.1.5 resolved it. 

I found the error. 

I believe during Inference Pytorch Lightning will create model copies and it will complain about the fact that there is a lambda function defines locally: https://github.com/Unbabel/COMET/blob/master/comet/models/base.py#L445 

After fixing that error it will complain about the local definition of PredictProgressBar.

I am just wondering why it works for some python versions and not for others.

I also have an issue with Python 3.8:

```bash
$ python3 -V
3.8.10
$ pip list | grep comet
unbabel-comet          0.1.0
$ comet score --model wmt-large-da-estimator-1719 [...]
Traceback (most recent call last):
  File "/usr/local/bin/comet", line 8, in <module>
    sys.exit(comet())
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/comet/cli.py", line 133, in score
    model = load_checkpoint(model) if os.path.exists(model) else download_model(model)
  File "/usr/local/lib/python3.8/dist-packages/comet/models/__init__.py", line 98, in download_model
    return load_checkpoint(checkpoint_path)
  File "/usr/local/lib/python3.8/dist-packages/comet/models/__init__.py", line 131, in load_checkpoint
    model = str2model[hparams["model"]].load_from_checkpoint(
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/core/saving.py", line 157, in load_from_checkpoint
    model = cls._load_model_state(checkpoint, strict=strict, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/core/saving.py", line 199, in _load_model_state
    model = cls(**_cls_kwargs)
  File "/usr/local/lib/python3.8/dist-packages/comet/models/estimators/comet_estimator.py", line 35, in __init__
    super().__init__(hparams)
  File "/usr/local/lib/python3.8/dist-packages/comet/models/estimators/estimator_base.py", line 70, in __init__
    super().__init__(hparams)
  File "/usr/local/lib/python3.8/dist-packages/comet/models/model_base.py", line 134, in __init__
    self.hparams = Namespace(**hparams)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 819, in __setattr__
    object.__setattr__(self, name, value)
AttributeError: can't set attribute
```

Hi @mjpost can you test it using the pre-release version?

`pip install unbabel-comet==1.0.0rc8`

I fixed some of the problems reported on this thread already. 


@joncamgui please test it now: `pip install unbabel-comet==1.0.0rc8`
I fixed the error you reported already

Yes, it works (with a few command-line parameter changes). Thank you!

Thanks a lot @ricardorei . It works perfectly!

Hi @ricardorei 
Any plan to make the official 1.0 available in standard 3.8 pip:
```
pip3.8 install unbabel-comet==1.0.0
ERROR: Could not find a version that satisfies the requirement unbabel-comet==1.0.0 (from versions: 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.6, 0.0.6.post1, 0.0.6.post2, 0.0.7, 0.1.0, 1.0.0rc1, 1.0.0rc2, 1.0.0rc3, 1.0.0rc4, 1.0.0rc5, 1.0.0rc6, 1.0.0rc7, 1.0.0rc8, 1.0.0rc9)
ERROR: No matching distribution found for unbabel-comet==1.0.0
```
Best
W.

@WilliamTambellini I release version 1.0 a few hours ago and I tested it on python 3.8. [As you can see](https://pypi.org/project/unbabel-comet/#history) your pip should return you 1.0.0 and 1.0.1 versions 

The install test for python 3.8 can be seen [here](https://github.com/Unbabel/COMET/runs/4267774614?check_suite_focus=true).

have you tried to upgrade pip?

I am reopening this until its fixed.

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/tmp/ipykernel_33/376669550.py in <module>
----> 1 from comet import download_model, load_from_checkpoint
      2 import os
      3 import sys
      4 #from comet.models import download_model
      5 #Load COMET Models before hand

ImportError: cannot import name 'download_model' from 'comet' (/opt/conda/lib/python3.7/site-packages/comet/__init__.py)

@AnanyaCoder 
the same error occours in my python3.7 env, have you figured it out?

It still doesn't work for Python 3.8...

I am getting the same error with Python 3.11.8:
```
Traceback (most recent call last):
  File "/home/souto/Repos/msoutopico/python-comet/code/main.py", line 2, in <module>
    from comet import download_model, load_from_checkpoint
ImportError: cannot import name 'download_model' from 'comet' (/home/souto/.cache/pypoetry/virtualenvs/python-comet-nDgw5fZj-py3.11/lib/python3.11/site-packages/comet/__init__.py)
```

> I am getting the same error with Python 3.11.8:
> 
> ```
> Traceback (most recent call last):
>   File "/home/souto/Repos/msoutopico/python-comet/code/main.py", line 2, in <module>
>     from comet import download_model, load_from_checkpoint
> ImportError: cannot import name 'download_model' from 'comet' (/home/souto/.cache/pypoetry/virtualenvs/python-comet-nDgw5fZj-py3.11/lib/python3.11/site-packages/comet/__init__.py)
> ```

I got this error 2 days ago and realized the PyPi package I had installed was https://pypi.org/project/Comet/ and not https://pypi.org/project/unbabel-comet/. Uninstalling the Comet and installing the unbabel-comet package solved it in my case, hope it does for yours as well !

> I got this error 2 days ago and realized the PyPi package I had installed was https://pypi.org/project/Comet/ and not https://pypi.org/project/unbabel-comet/. Uninstalling the Comet and installing the unbabel-comet package solved it in my case, hope it does for yours as well !

Thanks a lot @OrianeN, your feedback is very useful. 
I got the same error again today, all these packages (`comet`, `comet_ml`, `unbabel-comet`) are very confusing.