Repository: deepset-ai/haystack

Issue:
Default optimizer not working when training DensePassageRetriever
**Describe the bug**
Training a DensePassageRetriever model with the default optimizer `TransformersAdamW` results in an error that the optimizer cannot be found.

**Error message**
```
Traceback (most recent call last):
  File "/Users/bogdan/Documents/deepset/haystack/haystack/modeling/model/optimization.py", line 201, in _get_optim
    optim_constructor = getattr(import_module('torch.optim'), optimizer_name)
AttributeError: module 'torch.optim' has no attribute 'TransformersAdamW'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bogdan/Documents/deepset/haystack/haystack/modeling/model/optimization.py", line 204, in _get_optim
    optim_constructor = getattr(import_module('transformers.optimization'), optimizer_name)
AttributeError: module 'transformers.optimization' has no attribute 'TransformersAdamW'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bogdan/Documents/deepset/haystack/haystack/modeling/model/optimization.py", line 207, in _get_optim
    optim_constructor = getattr(import_module('apex.optimizers'), optimizer_name)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'apex'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bogdan/Documents/deepset/haystack/haystack/modeling/model/optimization.py", line 213, in _get_optim
    optim_constructor = getattr(sys.modules[__name__], optimizer_name)
AttributeError: module 'haystack.modeling.model.optimization' has no attribute 'TransformersAdamW'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bogdan/Documents/deepset/haystack/bogdan_scripts/try_dpr_training.py", line 11, in <module>
    retriever.train(data_dir="../test/samples/dpr", train_filename="sample.json", n_epochs=1)
  File "/Users/bogdan/Documents/deepset/haystack/haystack/retriever/dense.py", line 391, in train
    use_amp=use_amp
  File "/Users/bogdan/Documents/deepset/haystack/haystack/modeling/model/optimization.py", line 152, in initialize_optimizer
    optimizer = _get_optim(model, optimizer_opts)
  File "/Users/bogdan/Documents/deepset/haystack/haystack/modeling/model/optimization.py", line 215, in _get_optim
    raise AttributeError(f"Optimizer '{optimizer_name}' not found in 'torch', 'transformers', 'apex' or 'local imports")
AttributeError: Optimizer 'TransformersAdamW' not found in 'torch', 'transformers', 'apex' or 'local imports
```

**To Reproduce**
```python
from haystack.document_store import ElasticsearchDocumentStore
from haystack.retriever import DensePassageRetriever

document_store = ElasticsearchDocumentStore()
retriever = DensePassageRetriever(
    document_store=document_store,
    query_embedding_model="prajjwal1/bert-tiny",
    passage_embedding_model="prajjwal1/bert-tiny"
)

retriever.train(data_dir="test/samples/dpr", train_filename="sample.json", n_epochs=1)
```




Discussion:
Seems to be an import issue (there was an import workaround in that area of the code). I _might_ be able to fix it quickly, let's see

Resolving patch:
diff --git a/haystack/retriever/dense.py b/haystack/retriever/dense.py
--- a/haystack/retriever/dense.py
+++ b/haystack/retriever/dense.py
@@ -322,7 +322,7 @@ def train(self,
               num_warmup_steps: int = 100,
               grad_acc_steps: int = 1,
               use_amp: str = None,
-              optimizer_name: str = "TransformersAdamW",
+              optimizer_name: str = "AdamW",
               optimizer_correct_bias: bool = True,
               save_dir: str = "../saved_models/dpr",
               query_encoder_save_dir: str = "query_encoder",
@@ -355,7 +355,7 @@ def train(self,
                     "O2" (Almost FP16)
                     "O3" (Pure FP16).
                     For more information, refer to: https://nvidia.github.io/apex/amp.html
-        :param optimizer_name: what optimizer to use (default: TransformersAdamW)
+        :param optimizer_name: what optimizer to use (default: AdamW)
         :param num_warmup_steps: number of warmup steps
         :param optimizer_correct_bias: Whether to correct bias in optimizer
         :param save_dir: directory where models are saved


Test patch: