Repository: open-telemetry/opentelemetry-python

Issue:
Make `opentelemetry_metrics_exporter` entrypoint support pull exporters
The `opentelemetry_metrics_exporter` is currently the option for configuring metric exporters for auto-instrumentation. Unfortunately, this doesn't work for pull exporters (e.g. prometheus) which are implemented as MetricReader subclasses, nor does it allow the entrypoint impl to set configure the MetricReader. Solution was discussed in https://github.com/open-telemetry/opentelemetry-python/pull/2843#issuecomment-1198417591

> If I could make a suggestion, the entrypoints for `OTEL_METRIC_EXPORTERS` should provide a `MetricReader` factory function. For example, for otlp that could be something like
> 
> ```py
> def provide_reader() -> MetricReader:
>   preferred_temporality=environ.get(
>                     OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE,
>                     "CUMULATIVE",
>                 )
>                 .upper()
>                 .strip()
> 
>   return PeriodicExportingMetricReader(
>     exporter=OTLPExporter(),
>     preferred_temporality=preferred_temporality,
>     ...,
>   )
> ```
> 
> then update https://github.com/open-telemetry/opentelemetry-python/blob/b9a6358583c0676482a043279c01b252dc6c308e/exporter/opentelemetry-exporter-otlp-proto-grpc/setup.cfg#L62
> 
> to
> ```cfg
>  otlp_proto_grpc = opentelemetry.exporter.otlp.proto.grpc.metric_exporter:provide_reader
> ```
> 
> one downside of this approach is some code duplication across implementations of `provide_reader()` for other exporters if we have common environment variables to check. However it would simplify things for using MetricReaders like Prometheus which act as a "pull exporter", as they can easily implement this method as well. Right now im not sure how we would implement `OTEL_METRICS_EXPORTER=prometheus`

and I think we are all on board with this approach (see https://github.com/open-telemetry/opentelemetry-python/issues/2864#issuecomment-1387311895).


Discussion:
Recap of the discussion from the SIG:

1. This is a breaking change, e.g. Azure implements this entrypoint and would have to modify it
2. It's inconsistent with Logging and Tracing where the entrypoints are expected to return an exporter and not a Span/Log processor
3. It's probably an oversight that the logging and tracing entrypoints are implemented to return exporters instead of processors

Possible paths forward:
1. Go ahead with the approach outline above, implemented in #3412 and leave the tracing/logging entrypoints inconsistent
1. Have two entrypoints for each signal, one for the exporter and one for the processor. Implementors of an exporter may provide one of those two entrypoints
   * pros: most flexible, no breaking changes, can do the work incrementally
   * cons: adds complexity (6 entrypoints instead of 3), more confusing for implementors of the entrypoint
1. Use the existing entrypoints but lossen them to allow either returning an exporter or a processor/reader `Callable[[], MetricReader] | Callable[[], MetricExporter]`, `Callable[[], SpanProcessor] | Callable[[], SpanExporter]`, etc.. In the `_init_metrics()` etc. methods, we can handle the two cases
   * basically the same pros and cons as 2. but without adding more entrypoints. May be simpler for implementors.

Please add your thoughts to move this forward.
I think we should also look at the config proposed by Configuration WG https://github.com/open-telemetry/opentelemetry-configuration and make sure it aligns with what we do.
I'd vote for exploring option 2 (did you mean reader instead of processor?) -- it's backwards compatible, is type safe/consistent, solves the problem for Prometheus users, and appears to be only a (minor) inconvenience for authors of pull-based exporters (of which, to my knowledge, there is only one).
Here's an [example snippet](https://github.com/open-telemetry/opentelemetry-configuration/blob/979a4817216eee572d8b2689130f4421887eb571/examples/kitchen-sink.yaml#L101-L136) from OTel configuration yaml as proposed:

```yaml
meter_provider:
  # Configure metric readers.
  readers:
    # Configure a pull-based metric reader.
    - pull:
        exporter:
          # Configure exporter to be prometheus.
          prometheus:
            # ...
    # Configure a periodic metric reader.
    - periodic:
        interval: 5000
        timeout: 30000
        exporter:
          otlp:
            # ...

tracer_provider:
  # Configure span processors.
  processors:
    - batch:
        schedule_delay: 5000
        export_timeout: 30000
        exporter:
          otlp:
            # ...
```

This is more hierarchical than what we current support with `opentelemetry-instrument`, I think we will need to make pretty invasive changes to our existing entrypoints when we implement this. It's probably best to just do a temporary workaround then to make a two sets of big breaking changes.
Discussed in SIG today. Decided because our current entrypoint structure is so different from what the Configuration WG has proposed, there will probably be breaking changes. Until then, we should do a minimal workaround to avoid churn but still support Prometheus.

I will send a PR that instead just checks that if the entrypoint points to a `MetricReader` it will use that directly. The existing behavior of using PeriodicExportingMetricReader with a push exporter will remain the same. This is basically option 3 above but only scoping it to metrics for now.

Resolving patch:
diff --git a/opentelemetry-api/src/opentelemetry/environment_variables.py b/opentelemetry-api/src/opentelemetry/environment_variables.py
--- a/opentelemetry-api/src/opentelemetry/environment_variables.py
+++ b/opentelemetry-api/src/opentelemetry/environment_variables.py
@@ -22,6 +22,29 @@
 """
 .. envvar:: OTEL_METRICS_EXPORTER
 
+Specifies which exporter is used for metrics. See `General SDK Configuration
+<https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_metrics_exporter>`_.
+
+**Default value:** ``"otlp"``
+
+**Example:**
+
+``export OTEL_METRICS_EXPORTER="prometheus"``
+
+Accepted values for ``OTEL_METRICS_EXPORTER`` are:
+
+- ``"otlp"``
+- ``"prometheus"``
+- ``"none"``: No automatically configured exporter for metrics.
+
+.. note::
+
+    Exporter packages may add entry points for group ``opentelemetry_metrics_exporter`` which
+    can then be used with this environment variable by name. The entry point should point to
+    either a `opentelemetry.sdk.metrics.export.MetricExporter` (push exporter) or
+    `opentelemetry.sdk.metrics.export.MetricReader` (pull exporter) subclass; it must be
+    constructable without any required arguments. This mechanism is considered experimental and
+    may change in subsequent releases.
 """
 
 OTEL_PROPAGATORS = "OTEL_PROPAGATORS"
diff --git a/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py
--- a/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py
+++ b/opentelemetry-sdk/src/opentelemetry/sdk/_configuration/__init__.py
@@ -21,7 +21,7 @@
 import os
 from abc import ABC, abstractmethod
 from os import environ
-from typing import Callable, Dict, List, Optional, Sequence, Tuple, Type
+from typing import Callable, Dict, List, Optional, Sequence, Tuple, Type, Union
 
 from typing_extensions import Literal
 
@@ -47,6 +47,7 @@
 from opentelemetry.sdk.metrics import MeterProvider
 from opentelemetry.sdk.metrics.export import (
     MetricExporter,
+    MetricReader,
     PeriodicExportingMetricReader,
 )
 from opentelemetry.sdk.resources import Resource
@@ -210,16 +211,24 @@ def _init_tracing(
 
 
 def _init_metrics(
-    exporters: Dict[str, Type[MetricExporter]],
+    exporters_or_readers: Dict[
+        str, Union[Type[MetricExporter], Type[MetricReader]]
+    ],
     resource: Resource = None,
 ):
     metric_readers = []
 
-    for _, exporter_class in exporters.items():
+    for _, exporter_or_reader_class in exporters_or_readers.items():
         exporter_args = {}
-        metric_readers.append(
-            PeriodicExportingMetricReader(exporter_class(**exporter_args))
-        )
+
+        if issubclass(exporter_or_reader_class, MetricReader):
+            metric_readers.append(exporter_or_reader_class(**exporter_args))
+        else:
+            metric_readers.append(
+                PeriodicExportingMetricReader(
+                    exporter_or_reader_class(**exporter_args)
+                )
+            )
 
     provider = MeterProvider(resource=resource, metric_readers=metric_readers)
     set_meter_provider(provider)
@@ -249,7 +258,7 @@ def _import_exporters(
     log_exporter_names: Sequence[str],
 ) -> Tuple[
     Dict[str, Type[SpanExporter]],
-    Dict[str, Type[MetricExporter]],
+    Dict[str, Union[Type[MetricExporter], Type[MetricReader]]],
     Dict[str, Type[LogExporter]],
 ]:
     trace_exporters = {}
@@ -267,7 +276,9 @@ def _import_exporters(
     for (exporter_name, exporter_impl,) in _import_config_components(
         metric_exporter_names, "opentelemetry_metrics_exporter"
     ):
-        if issubclass(exporter_impl, MetricExporter):
+        # The metric exporter components may be push MetricExporter or pull exporters which
+        # subclass MetricReader directly
+        if issubclass(exporter_impl, (MetricExporter, MetricReader)):
             metric_exporters[exporter_name] = exporter_impl
         else:
             raise RuntimeError(f"{exporter_name} is not a metric exporter")


Test patch:
diff --git a/opentelemetry-sdk/tests/test_configurator.py b/opentelemetry-sdk/tests/test_configurator.py
--- a/opentelemetry-sdk/tests/test_configurator.py
+++ b/opentelemetry-sdk/tests/test_configurator.py
@@ -18,7 +18,7 @@
 from os import environ
 from typing import Dict, Iterable, Optional, Sequence
 from unittest import TestCase
-from unittest.mock import patch
+from unittest.mock import Mock, patch
 
 from pytest import raises
 
@@ -158,6 +158,20 @@ def shutdown(self, timeout_millis: float = 30_000, **kwargs) -> None:
         return True
 
 
+# MetricReader that can be configured as a pull exporter
+class DummyMetricReaderPullExporter(MetricReader):
+    def _receive_metrics(
+        self,
+        metrics: Iterable[Metric],
+        timeout_millis: float = 10_000,
+        **kwargs,
+    ) -> None:
+        pass
+
+    def shutdown(self, timeout_millis: float = 30_000, **kwargs) -> None:
+        return True
+
+
 class DummyOTLPMetricExporter:
     def __init__(self, *args, **kwargs):
         self.export_called = False
@@ -309,7 +323,6 @@ def tearDown(self):
         environ, {"OTEL_RESOURCE_ATTRIBUTES": "service.name=my-test-service"}
     )
     def test_trace_init_default(self):
-
         auto_resource = Resource.create(
             {
                 "telemetry.auto.version": "test-version",
@@ -740,6 +753,18 @@ def test_metrics_init_exporter(self):
         self.assertIsInstance(reader, DummyMetricReader)
         self.assertIsInstance(reader.exporter, DummyOTLPMetricExporter)
 
+    def test_metrics_init_pull_exporter(self):
+        resource = Resource.create({})
+        _init_metrics(
+            {"dummy_metric_reader": DummyMetricReaderPullExporter},
+            resource=resource,
+        )
+        self.assertEqual(self.set_provider_mock.call_count, 1)
+        provider = self.set_provider_mock.call_args[0][0]
+        self.assertIsInstance(provider, DummyMeterProvider)
+        reader = provider._sdk_config.metric_readers[0]
+        self.assertIsInstance(reader, DummyMetricReaderPullExporter)
+
 
 class TestExporterNames(TestCase):
     @patch.dict(
@@ -835,6 +860,28 @@ def test_console_exporters(self):
             ConsoleMetricExporter.__class__,
         )
 
+    @patch(
+        "opentelemetry.sdk._configuration.entry_points",
+    )
+    def test_metric_pull_exporter(self, mock_entry_points: Mock):
+        def mock_entry_points_impl(group, name):
+            if name == "dummy_pull_exporter":
+                return [
+                    IterEntryPoint(
+                        name=name, class_type=DummyMetricReaderPullExporter
+                    )
+                ]
+            return []
+
+        mock_entry_points.side_effect = mock_entry_points_impl
+        _, metric_exporters, _ = _import_exporters(
+            [], ["dummy_pull_exporter"], []
+        )
+        self.assertIs(
+            metric_exporters["dummy_pull_exporter"],
+            DummyMetricReaderPullExporter,
+        )
+
 
 class TestImportConfigComponents(TestCase):
     @patch(