Repository: kubernetes
Issue: HPA ExternalMetricSource selectors shouldn't be bound to Kubernetes standards
Labels: sig/autoscaling,kind/feature,sig/instrumentation,needs-triage

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!

If the matter is security related, please disclose it privately via https://kubernetes.io/security/
-->


#### What happened:

By implementing the ExternalMetrics API in prometheus-adapter, we've got a couple of reports from our users that the HPA selector for ExternalMetrics prevented them to select some label values. For instance, in their cases the `#` and `/` characters were forbidden by Kubernetes so they couldn't autoscale their application as they wanted.
- https://github.com/kubernetes-sigs/prometheus-adapter/issues/323
- https://github.com/kubernetes-sigs/prometheus-adapter/issues/364

In one of these cases, the poster got the following warning in the HPA:
```
invalid label value: "/company/security/v1/account/login": at key: "uri": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')
``` 

There is also a PR against promtheus-adapter to work around this issue, but we would prefer having it fixed here. https://github.com/kubernetes-sigs/prometheus-adapter/pull/381

#### What you expected to happen:

I would have expected ExternalMetrics not to abide to the Kubernetes standards as ExternalMetrics are not tied to any Kubernetes resources.
I would also expect any validation on the selector to be done by the implementer of the ExternalMetrics API since it has all the knowledge about the metrics backend.

#### How to reproduce it (as minimally and precisely as possible):

- deploy prometheus-adapter
- create an HPA with an ExternalMetricsSource that has an invalid value in the metric selector. For example:
```
matchExpressions:
  - key: job
    operator: In
    values:
      - #InvalidValue#
 ```
- look for validation errors in the HPA 

#### Anything else we need to know?:

The root cause of the problem seems to be in the ExternalMetricSource structure of the HPA: https://github.com/kubernetes/kubernetes/blob/v1.21.0/pkg/apis/autoscaling/types.go#L306-L321. In the same fashion as PodMetricSource and ObjectMetricSource, it has a MetricIdentifier that allows identifying a particular metrics based on a selector. From the comment above the selector:
```
// Selector is the selector for the given metric
// it is the string-encoded form of a standard kubernetes label selector
```
This is very valid for pods and objects as they refer to the CustomMetrics API that is tied to a Kubernetes resource, but this is not the case for ExternalMetrics. Thus, I would expect ExternalMetricSource to have a custom selector that doesn't abide to Kubernetes label selector standards.

#### Environment:
- Kubernetes version (use `kubectl version`):
```
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"archive", BuildDate:"2021-04-09T16:47:30Z", GoVersion:"go1.16.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:25:06Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}
```

/cc @carsonoid

/sig instrumentation
/sig autoscaling