wandb metric argument is weird
Author: edmcman
Pull request: no

### System Info

- `transformers` version: 4.30.2
- Platform: Linux-5.7.19-050719-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <yes>
- Using distributed or parallel set-up in script?: <no>

### Who can help?

@AyushExel

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

There are a few bugs/weirdness when specifying the `metric` in `wandb` hyperparameter search.

1. If a `metric` is specified via the `hp_space` argument but there is not a `metric` argument to `hyperparameter_search`, the metric in the `hp_space` is ignored and changed to `eval/loss`.  See: https://github.com/huggingface/transformers/blame/6ce6d62b6f20040129ec9831e7c4f6576402ea42/src/transformers/integrations.py#L497
2. If a custom `hp_space` is provided that does not define `metric` at all, and the `metric` argument is specified to `hyperparameter_search`, this code throws an exception: https://github.com/huggingface/transformers/blame/6ce6d62b6f20040129ec9831e7c4f6576402ea42/src/transformers/integrations.py#L500 

### Expected behavior

1. If a `hp_space` defines a metric, use that instead of overwriting it.
2. Don't throw an exception is the `hp_space` lacks a `metric` key.

Comments:
Thanks @edmcman for the ping.

@ayulockin @morganmcg1 in case this is relevant to the integrations team. tbh, I don't remember much about this integration but I implemented it so happy to help in case there's a blocker.

Thanks for the detailed issue @edmcman; we are taking a look at this. Will connect with you @AyushExel in case of blocker.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Ping

On Mon, Jul 31, 2023, 11:03 AM github-actions[bot] -
***@***.*** <github.edmcman.99c9f1b9d0.notifications#
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1658554442>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZPH3FQDLIUC7DSYYRLXS7CKXANCNFSM6AAAAAAZPGOQMA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Ping

On Fri, Aug 25, 2023, 4:03 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1692940561>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZPWTBMM362D7HGYQGTXXBL3JANCNFSM6AAAAAAZPGOQMA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Ping

On Tue, Sep 19, 2023, 4:03 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1725018773>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZKLSBIWBWDK3UV6WGDX3FGVJANCNFSM6AAAAAAZPGOQMA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Hey @edmcman, apologies for the delay. I am taking a look at this. Will let you know soon.

No worries, I was just keeping the issue open :)

Boop

On Sat, Oct 14, 2023, 4:06 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1762708379>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZNRWB7DQT2CAGH3BZ3X7JBZVANCNFSM6AAAAAAZPGOQMA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Boop

On Fri, Nov 10, 2023, 3:07 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1805270715>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZLZLJX6TNBZJ2LOJG3YDXOFZAVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBVGI3TANZRGU>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Boop

On Tue, Dec 5, 2023, 3:06 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1840213033>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZJANX7W6GOKOVQQOWLYH3IZZAVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBQGIYTGMBTGM>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Boop

On Sat, Dec 30, 2023, 3:06 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1872479380>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZMEV7QFVIKKLL7VA3TYL7DP7AVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSGQ3TSMZYGA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Bump

On Wed, Jan 24, 2024, 3:07 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1907600067>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZJIJOVQKD7V5UOB2GDYQC6NPAVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBXGYYDAMBWG4>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Boop

On Sun, Feb 18, 2024, 3:07 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-1950997682>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZKVITBHJR65I2F5KTLYUGZCFAVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQHE4TONRYGI>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @AyushExel - any update on this?

@amyeroberts hey, I'm no longer associated with W&B. @ayulockin should be able to help. Thanks!

@ayulockin can you help?

On Mon, Apr 8, 2024, 4:09 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-2042121322>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZIUIH7S7OX4CA2SP6DY4JGD5AVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBSGEZDCMZSGI>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


.

On Fri, May 3, 2024, 4:09 AM github-actions[bot] - ***@***.***
***@***.***> wrote:

> This issue has been automatically marked as stale because it has not had
> recent activity. If you think this still needs to be addressed please
> comment on this thread.
>
> Please note that issues that do not follow the contributing guidelines
> <https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md>
> are likely to be ignored.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/24409#issuecomment-2092520316>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZK63QRSGSNWJ6OHLDTZANA2JAVCNFSM6AAAAAAZPGOQMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGUZDAMZRGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


cc @parambharat as they've been doing loads of great integration work between HF & W&B recently 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.