Repository: open-telemetry/opentelemetry-python

Issue:
Setting any value to OTEL_EXPORTER_OTLP_PROTOCOL has no effect.
**Describe your environment** 
```
$ uname -srvmpio
Linux 4.19.128-microsoft-standard #1 SMP Tue Jun 23 12:58:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ python -VV
Python 3.9.9 (main, Dec 19 2021, 23:35:26)
[GCC 9.3.0]
$ grep ^opentelemetry pyproject.toml
opentelemetry-sdk = "^1.10.0"
opentelemetry-exporter-otlp-proto-http = "^1.10.0"
opentelemetry-exporter-otlp-proto-grpc = "^1.10.0"
opentelemetry-sdk-extension-aws = "^2.0.1"
opentelemetry-distro = "^0.29b0"
opentelemetry-instrumentation-django = "^0.29b0"
```

**Steps to reproduce**
I am using `opentelemetry-instrument` to wire up the components.

**What is the expected behavior?**
According to [the documentation](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md), one may choose the transport protocol for the OTLP exporter through `OTEL_EXPORTER_OTLP_PROTOCOL`, but it doesn't seem to apply to OTel Python, as the variable is referenced from nowhere.

**What is the actual behavior?**
Instead I had to set `OTEL_*_EXPORTER` to either `otlp_proto_grpc` or `otlp_proto_http` to change the protocol.

**Additional context**
The current SDK's architecture for the implementation lookup is not great IMO.  This needs to be addressed.



Discussion:
Hey, I would like to take this PR.

Some follow-up questions:
1. Do the options `otlp_proto_grpc` and `otlp_proto_http` should be removed for `OTEL_*_EXPORTER`? As far as I understand, this is behavior in [Java SDK autoconfigure](https://github.com/open-telemetry/opentelemetry-java/blob/5def2f7fb23e9a3deabf7a1f13a6c6446e76d435/sdk-extensions/autoconfigure/README.md). It also makes more sense, otherwise users could define conflicting config by also using `OTEL_EXPORTER_OTLP_PROTOCOL`. 

2. Does building the logs exporter should be based on `OTEL_EXPORTER_OTLP_PROTOCOL` and `OTEL_EXPORTER_OTLP_LOGS_PROTOCOL`? As it's not defined in the [specs](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md).

Thanks!
The env would have more sense when we had one package. We decided to split the single package to multiple packages based on protocol + encoding so we have separate packages for gRPC, HTTP. I do not think this can easily be addressed without any breaking change.
The SDK already supports initializing the configured exporter from the relevant package (in this [module](https://github.com/open-telemetry/opentelemetry-python/blob/edb1391a3d29d844141b15e486cd1107e163ee0b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py)).
So I think this issue can be solved without changing the exporters packages structure.

But I do think that the breaking change required here is to remove the support for `otlp_proto_grpc` and `otlp_proto_http` for `OTEL_*_EXPORTER`.
After some discussions, adding an overview of my proposed changes (all are related to traces, metrics & logs):

1. Updating [SDK configurator](https://github.com/open-telemetry/opentelemetry-python/blob/edb1391a3d29d844141b15e486cd1107e163ee0b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py) to support the env vars `OTEL_EXPORTER_OTLP_*_PROTOCOL` defined by the [specs](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md#specify-protocol). These envs will take effect only when `OTEL_*_EXPORTER` is set to `otlp`.

2. In case `OTEL_*_EXPORTER` will be set to `otlp_proto_grpc`/`otlp_proto_http` and a conflicting value will be set to `OTEL_EXPORTER_OTLP_*_PROTOCOL`, the protocol will be set according to `OTEL_*_EXPORTER` and a warning will be printed.

3. Updating the default value for `OTEL_*_EXPORTER` to `otlp` in [opentelemetry-instrumentation](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/ebe6d1804bccff184edb379d7780469762c7b854/opentelemetry-instrumentation) package and in the SDK (also according to [specs](https://opentelemetry.io/docs/reference/specification/sdk-environment-variables/#exporter-selection)), as an actual value and not just an alias. The exporter used in this case will still be gRPC, to avoid a breaking change. This way, `OTEL_EXPORTER_OTLP_*_PROTOCOL` could be configured without specifically setting `OTEL_*_EXPORTER=otlp`.

------

Future *breaking* changes I propose is to remove the options `otlp_proto_grpc`/`otlp_proto_http`, since they are not defined in the [specs](https://opentelemetry.io/docs/reference/specification/sdk-environment-variables/#exporter-selection), and they introduce this redundant complexity. Also, I didn't find that these are used in other runtimes.

Note - it does not require changing package entry points (such [that](https://github.com/open-telemetry/opentelemetry-python/blob/main/exporter/opentelemetry-exporter-otlp-proto-http/setup.cfg#L59)), to avoid this unnecessary breaking change.
Sounds good to me. It looks like the env are already defined here https://github.com/open-telemetry/opentelemetry-python/blob/edb1391a3d29d844141b15e486cd1107e163ee0b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py. If this requires only changes in contrib repo, I will transfer this issue there.
It requires changes in both repos, so I think it can remain here.
Ah correct, the configuration part was also originally part of `opentelemetry-distro` package but later it was moved to SDK. Since it requires changes in both, You will have to update [SHA](https://github.com/open-telemetry/opentelemetry-python/blob/main/.github/workflows/test.yml#L13) in test.yml when you open PRs for tests to pass; more details here https://github.com/open-telemetry/opentelemetry-python/blob/main/CONTRIBUTING.md#pull-requests

Resolving patch:
diff --git a/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py
--- a/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py
+++ b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py
@@ -24,6 +24,7 @@
 from typing import Dict, Optional, Sequence, Tuple, Type
 
 from pkg_resources import iter_entry_points
+from typing_extensions import Literal
 
 from opentelemetry.environment_variables import (
     OTEL_LOGS_EXPORTER,
@@ -40,6 +41,10 @@
 from opentelemetry.sdk._logs.export import BatchLogProcessor, LogExporter
 from opentelemetry.sdk.environment_variables import (
     _OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED,
+    OTEL_EXPORTER_OTLP_LOGS_PROTOCOL,
+    OTEL_EXPORTER_OTLP_METRICS_PROTOCOL,
+    OTEL_EXPORTER_OTLP_PROTOCOL,
+    OTEL_EXPORTER_OTLP_TRACES_PROTOCOL,
 )
 from opentelemetry.sdk.metrics import MeterProvider
 from opentelemetry.sdk.metrics.export import (
@@ -55,26 +60,91 @@
 
 _EXPORTER_OTLP = "otlp"
 _EXPORTER_OTLP_PROTO_GRPC = "otlp_proto_grpc"
+_EXPORTER_OTLP_PROTO_HTTP = "otlp_proto_http"
+
+_EXPORTER_BY_OTLP_PROTOCOL = {
+    "grpc": _EXPORTER_OTLP_PROTO_GRPC,
+    "http/protobuf": _EXPORTER_OTLP_PROTO_HTTP,
+}
+
+_EXPORTER_ENV_BY_SIGNAL_TYPE = {
+    "traces": OTEL_TRACES_EXPORTER,
+    "metrics": OTEL_METRICS_EXPORTER,
+    "logs": OTEL_LOGS_EXPORTER,
+}
+
+_PROTOCOL_ENV_BY_SIGNAL_TYPE = {
+    "traces": OTEL_EXPORTER_OTLP_TRACES_PROTOCOL,
+    "metrics": OTEL_EXPORTER_OTLP_METRICS_PROTOCOL,
+    "logs": OTEL_EXPORTER_OTLP_LOGS_PROTOCOL,
+}
 
 _RANDOM_ID_GENERATOR = "random"
 _DEFAULT_ID_GENERATOR = _RANDOM_ID_GENERATOR
 
+_logger = logging.getLogger(__name__)
+
 
 def _get_id_generator() -> str:
     return environ.get(OTEL_PYTHON_ID_GENERATOR, _DEFAULT_ID_GENERATOR)
 
 
-def _get_exporter_names(names: str) -> Sequence[str]:
-    exporters = set()
+def _get_exporter_entry_point(
+    exporter_name: str, signal_type: Literal["traces", "metrics", "logs"]
+):
+    if exporter_name not in (
+        _EXPORTER_OTLP,
+        _EXPORTER_OTLP_PROTO_GRPC,
+        _EXPORTER_OTLP_PROTO_HTTP,
+    ):
+        return exporter_name
+
+    # Checking env vars for OTLP protocol (grpc/http).
+    otlp_protocol = environ.get(
+        _PROTOCOL_ENV_BY_SIGNAL_TYPE[signal_type]
+    ) or environ.get(OTEL_EXPORTER_OTLP_PROTOCOL)
+
+    if not otlp_protocol:
+        if exporter_name == _EXPORTER_OTLP:
+            return _EXPORTER_OTLP_PROTO_GRPC
+        return exporter_name
+
+    otlp_protocol = otlp_protocol.strip()
+
+    if exporter_name == _EXPORTER_OTLP:
+        if otlp_protocol not in _EXPORTER_BY_OTLP_PROTOCOL:
+            # Invalid value was set by the env var
+            raise RuntimeError(
+                f"Unsupported OTLP protocol '{otlp_protocol}' is configured"
+            )
+
+        return _EXPORTER_BY_OTLP_PROTOCOL[otlp_protocol]
+
+    # grpc/http already specified by exporter_name, only add a warning in case
+    # of a conflict.
+    exporter_name_by_env = _EXPORTER_BY_OTLP_PROTOCOL.get(otlp_protocol)
+    if exporter_name_by_env and exporter_name != exporter_name_by_env:
+        _logger.warning(
+            "Conflicting values for %s OTLP exporter protocol, using '%s'",
+            signal_type,
+            exporter_name,
+        )
+
+    return exporter_name
+
 
-    if names and names.lower().strip() != "none":
-        exporters.update({_exporter.strip() for _exporter in names.split(",")})
+def _get_exporter_names(
+    signal_type: Literal["traces", "metrics", "logs"]
+) -> Sequence[str]:
+    names = environ.get(_EXPORTER_ENV_BY_SIGNAL_TYPE.get(signal_type, ""))
 
-    if _EXPORTER_OTLP in exporters:
-        exporters.remove(_EXPORTER_OTLP)
-        exporters.add(_EXPORTER_OTLP_PROTO_GRPC)
+    if not names or names.lower().strip() == "none":
+        return []
 
-    return list(exporters)
+    return [
+        _get_exporter_entry_point(_exporter.strip(), signal_type)
+        for _exporter in names.split(",")
+    ]
 
 
 def _init_tracing(
@@ -232,9 +302,9 @@ def _import_id_generator(id_generator_name: str) -> IdGenerator:
 
 def _initialize_components(auto_instrumentation_version):
     trace_exporters, metric_exporters, log_exporters = _import_exporters(
-        _get_exporter_names(environ.get(OTEL_TRACES_EXPORTER)),
-        _get_exporter_names(environ.get(OTEL_METRICS_EXPORTER)),
-        _get_exporter_names(environ.get(OTEL_LOGS_EXPORTER)),
+        _get_exporter_names("traces"),
+        _get_exporter_names("metrics"),
+        _get_exporter_names("logs"),
     )
     id_generator_name = _get_id_generator()
     id_generator = _import_id_generator(id_generator_name)
diff --git a/opentelemetry-sdk/src/opentelemetry/sdk/environment_variables.py b/opentelemetry-sdk/src/opentelemetry/sdk/environment_variables.py
--- a/opentelemetry-sdk/src/opentelemetry/sdk/environment_variables.py
+++ b/opentelemetry-sdk/src/opentelemetry/sdk/environment_variables.py
@@ -234,6 +234,27 @@
 OTLP exporter.
 """
 
+OTEL_EXPORTER_OTLP_TRACES_PROTOCOL = "OTEL_EXPORTER_OTLP_TRACES_PROTOCOL"
+"""
+.. envvar:: OTEL_EXPORTER_OTLP_TRACES_PROTOCOL
+
+The :envvar:`OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` represents the the transport protocol for spans.
+"""
+
+OTEL_EXPORTER_OTLP_METRICS_PROTOCOL = "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL"
+"""
+.. envvar:: OTEL_EXPORTER_OTLP_METRICS_PROTOCOL
+
+The :envvar:`OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` represents the the transport protocol for metrics.
+"""
+
+OTEL_EXPORTER_OTLP_LOGS_PROTOCOL = "OTEL_EXPORTER_OTLP_LOGS_PROTOCOL"
+"""
+.. envvar:: OTEL_EXPORTER_OTLP_LOGS_PROTOCOL
+
+The :envvar:`OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` represents the the transport protocol for logs.
+"""
+
 OTEL_EXPORTER_OTLP_CERTIFICATE = "OTEL_EXPORTER_OTLP_CERTIFICATE"
 """
 .. envvar:: OTEL_EXPORTER_OTLP_CERTIFICATE
@@ -314,13 +335,6 @@
 A scheme of https indicates a secure connection and takes precedence over this configuration setting.
 """
 
-OTEL_EXPORTER_OTLP_TRACES_PROTOCOL = "OTEL_EXPORTER_OTLP_TRACES_PROTOCOL"
-"""
-.. envvar:: OTEL_EXPORTER_OTLP_TRACES_PROTOCOL
-
-The :envvar:`OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` represents the the transport protocol for spans.
-"""
-
 OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE = "OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE"
 """
 .. envvar:: OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE


Test patch:
diff --git a/opentelemetry-sdk/tests/test_configurator.py b/opentelemetry-sdk/tests/test_configurator.py
--- a/opentelemetry-sdk/tests/test_configurator.py
+++ b/opentelemetry-sdk/tests/test_configurator.py
@@ -25,6 +25,7 @@
 from opentelemetry.sdk._configuration import (
     _EXPORTER_OTLP,
     _EXPORTER_OTLP_PROTO_GRPC,
+    _EXPORTER_OTLP_PROTO_HTTP,
     _get_exporter_names,
     _get_id_generator,
     _import_exporters,
@@ -413,25 +414,81 @@ def test_metrics_init_exporter(self):
 
 
 class TestExporterNames(TestCase):
-    def test_otlp_exporter_overwrite(self):
-        for exporter in [_EXPORTER_OTLP, _EXPORTER_OTLP_PROTO_GRPC]:
+    @patch.dict(
+        environ,
+        {
+            "OTEL_TRACES_EXPORTER": _EXPORTER_OTLP,
+            "OTEL_METRICS_EXPORTER": _EXPORTER_OTLP_PROTO_GRPC,
+            "OTEL_LOGS_EXPORTER": _EXPORTER_OTLP_PROTO_HTTP,
+        },
+    )
+    def test_otlp_exporter(self):
+        self.assertEqual(
+            _get_exporter_names("traces"), [_EXPORTER_OTLP_PROTO_GRPC]
+        )
+        self.assertEqual(
+            _get_exporter_names("metrics"), [_EXPORTER_OTLP_PROTO_GRPC]
+        )
+        self.assertEqual(
+            _get_exporter_names("logs"), [_EXPORTER_OTLP_PROTO_HTTP]
+        )
+
+    @patch.dict(
+        environ,
+        {
+            "OTEL_TRACES_EXPORTER": _EXPORTER_OTLP,
+            "OTEL_METRICS_EXPORTER": _EXPORTER_OTLP,
+            "OTEL_EXPORTER_OTLP_PROTOCOL": "http/protobuf",
+            "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL": "grpc",
+        },
+    )
+    def test_otlp_custom_exporter(self):
+        self.assertEqual(
+            _get_exporter_names("traces"), [_EXPORTER_OTLP_PROTO_HTTP]
+        )
+        self.assertEqual(
+            _get_exporter_names("metrics"), [_EXPORTER_OTLP_PROTO_GRPC]
+        )
+
+    @patch.dict(
+        environ,
+        {
+            "OTEL_TRACES_EXPORTER": _EXPORTER_OTLP_PROTO_HTTP,
+            "OTEL_METRICS_EXPORTER": _EXPORTER_OTLP_PROTO_GRPC,
+            "OTEL_EXPORTER_OTLP_PROTOCOL": "grpc",
+            "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL": "http/protobuf",
+        },
+    )
+    def test_otlp_exporter_conflict(self):
+        # Verify that OTEL_*_EXPORTER is used, and a warning is logged
+        with self.assertLogs(level="WARNING") as logs_context:
+            self.assertEqual(
+                _get_exporter_names("traces"), [_EXPORTER_OTLP_PROTO_HTTP]
+            )
+        assert len(logs_context.output) == 1
+
+        with self.assertLogs(level="WARNING") as logs_context:
             self.assertEqual(
-                _get_exporter_names(exporter), [_EXPORTER_OTLP_PROTO_GRPC]
+                _get_exporter_names("metrics"), [_EXPORTER_OTLP_PROTO_GRPC]
             )
+        assert len(logs_context.output) == 1
 
+    @patch.dict(environ, {"OTEL_TRACES_EXPORTER": "jaeger,zipkin"})
     def test_multiple_exporters(self):
         self.assertEqual(
-            sorted(_get_exporter_names("jaeger,zipkin")), ["jaeger", "zipkin"]
+            sorted(_get_exporter_names("traces")), ["jaeger", "zipkin"]
         )
 
+    @patch.dict(environ, {"OTEL_TRACES_EXPORTER": "none"})
     def test_none_exporters(self):
-        self.assertEqual(sorted(_get_exporter_names("none")), [])
+        self.assertEqual(sorted(_get_exporter_names("traces")), [])
 
     def test_no_exporters(self):
-        self.assertEqual(sorted(_get_exporter_names(None)), [])
+        self.assertEqual(sorted(_get_exporter_names("traces")), [])
 
+    @patch.dict(environ, {"OTEL_TRACES_EXPORTER": ""})
     def test_empty_exporters(self):
-        self.assertEqual(sorted(_get_exporter_names("")), [])
+        self.assertEqual(sorted(_get_exporter_names("traces")), [])
 
 
 class TestImportExporters(TestCase):