Repository: DataDog/dd-trace-py

Issue:
Parsing and usage of boolean environment variables doesn't work as expected
We cannot specify "falsy" values via environment variables, as they will be overridden by `default` in `get_env`.

Also some integrations (e.g. `botocore` for setting `distributed_tracing`) don't parse the result of `get_env` correctly, and thus cannot be changed via environment variables.

I would like to set `config.botocore["distributing_tracing"]` to `False` via the `DD_BOTOCORE_DISTRIBUTED_TRACING` environment variable, but because the default is `True`, there is no way to change the semantics of the setting to disable the behaviour.

There are two issues, and if either of them was fixed, the problem would be fixed.

I would be happy to provide a fix for both of them, or be directed to a way to fix it, without having to add extra code to my project.

## Empty environment variables are assumed to be missing

If you do `DD_BOTOCORE_DISTRIBUTED_TRACING=` (an empty string), [the relevant code](https://github.com/DataDog/dd-trace-py/blob/e7a73336ab7c4c1479d504f86730c4dfaeb2e5f7/ddtrace/utils/formats.py#L47-L58) will replace it with `default`, in which case it would be `True`:

```python
def get_env(*parts, **kwargs):
    ...

    # env = "DD_BOTOCORE_DISTRIBUTED_TRACING"
    value = os.getenv(env) # value = ""
    ...

    # legacy = None
    value = value or legacy # value = None
    # default = True
    return value if value else default # return True
```

The fix would be to check for `None`, instead for "falsy":

```python
if value is None:
    if legacy is not None:
        value = legacy
    else:
        value = default
```

## Usage of `get_env` for booleans should convert text values to boolean

When the `botocore` integration [uses](https://github.com/DataDog/dd-trace-py/blob/e7a73336ab7c4c1479d504f86730c4dfaeb2e5f7/ddtrace/contrib/botocore/patch.py#L39) `get_env` to define a boolean option, it should convert common boolean text values to boolean, and reject others:

```python
config._add(
    "botocore",
    {
        "distributed_tracing": get_bool_env("botocore", "distributed_tracing", default=True),
        ...,
    },
)
```
And
```python
TRUE_STRINGS = [True, "true", "yes", "y", "enable", "enabled", "1"]
FALSE_STRINGS = [False, "false", "no", "n", "disable", "disabled", "0"]


def to_boolean(value: str) -> bool:
    if value.lower() in TRUE_STRINGS:
        return True
    if value.lower() in FALSE_STRINGS:
        return False
    raise ValueError(f"Unknown boolean value '{value}'")



def get_bool_env(*parts, **kwargs):
    return to_boolean(get_env(*parts, **kwargs))
```

### Which version of dd-trace-py are you using?

`ddtrace==0.55.3`

### Which version of pip are you using?

`pip 21.2.4 (python 3.8)`

### Which version of the libraries are you using?

Not relevant

### How can we reproduce your problem?

First issue:

Before running the script:

```shell
DD_BOTOCORE_DISTRIBUTED_TRACING=
```

Run the script:

```python3
from ddtrace import config
from ddtrace.contrib import botocore
print(config.botocore["distributed_tracing"])
print(bool(config.botocore["distributed_tracing"]))
```

Outputs:

```
True
True
```

Second issue:

Before running the script:

```shell
DD_BOTOCORE_DISTRIBUTED_TRACING=False
```

Run the script:

```python3
from ddtrace import config
from ddtrace.contrib import botocore
print(config.botocore["distributed_tracing"])
print(bool(config.botocore["distributed_tracing"]))
```

Outputs:

```
False
True
```


### What is the result that you get?

I cannot meaningfully change the value of `config.botocore["distributed_tracing"]`

### What is the result that you expected?

I want to be able to meaningfully change `config.botocore["distributed_tracing"]` via the environment variable (i.e. `DD_BOTOCORE_DISTRIBUTED_TRACING`)



Discussion:
@costas-basdekis thanks for the detailed report + repro. We already have a helper to parse env variables as boolean, but we were not calling it for `botocore` 🤦🏻 

I have opened up #2980 which should fix the problem, at least for `DD_BOTOCORE_DISTRIBUTED_TRACING=false`, I don't think we have ever clearly defined what to do when we get `DD_BOTOCORE_DISTRIBUTED_TRACING=` as an empty string (for any of our env variables), so I am hesitant to try and address that right now, but something for us to think about (also assuming that fixing the `=false` problem works for you for now?)

Resolving patch:
diff --git a/ddtrace/contrib/botocore/patch.py b/ddtrace/contrib/botocore/patch.py
--- a/ddtrace/contrib/botocore/patch.py
+++ b/ddtrace/contrib/botocore/patch.py
@@ -20,6 +20,7 @@
 from ...pin import Pin
 from ...propagation.http import HTTPPropagator
 from ...utils import get_argument_value
+from ...utils.formats import asbool
 from ...utils.formats import deep_getattr
 from ...utils.formats import get_env
 from ...utils.wrappers import unwrap
@@ -37,8 +38,8 @@
 config._add(
     "botocore",
     {
-        "distributed_tracing": get_env("botocore", "distributed_tracing", default=True),
-        "invoke_with_legacy_context": get_env("botocore", "invoke_with_legacy_context", default=False),
+        "distributed_tracing": asbool(get_env("botocore", "distributed_tracing", default=True)),
+        "invoke_with_legacy_context": asbool(get_env("botocore", "invoke_with_legacy_context", default=False)),
     },
 )
 


Test patch:
diff --git a/tests/contrib/botocore/test.py b/tests/contrib/botocore/test.py
--- a/tests/contrib/botocore/test.py
+++ b/tests/contrib/botocore/test.py
@@ -1,4 +1,3 @@
-# 3p
 import base64
 import datetime
 import io
@@ -15,6 +14,7 @@
 from moto import mock_sqs
 
 from ddtrace import Pin
+from ddtrace import config
 from ddtrace.constants import ANALYTICS_SAMPLE_RATE_KEY
 from ddtrace.contrib.botocore.patch import patch
 from ddtrace.contrib.botocore.patch import unpatch
@@ -805,3 +805,15 @@ def test_firehose_no_records_arg(self):
         assert delivery_stream_span.get_tag("aws.operation") == "CreateDeliveryStream"
         assert put_record_batch_span.get_tag("aws.operation") == "PutRecordBatch"
         assert put_record_batch_span.get_tag("params.Records") is None
+
+    @TracerTestCase.run_in_subprocess(env_overrides=dict(DD_BOTOCORE_DISTRIBUTED_TRACING="true"))
+    def test_distributed_tracing_env_override(self):
+        assert config.botocore.distributed_tracing is True
+
+    @TracerTestCase.run_in_subprocess(env_overrides=dict(DD_BOTOCORE_DISTRIBUTED_TRACING="false"))
+    def test_distributed_tracing_env_override_false(self):
+        assert config.botocore.distributed_tracing is False
+
+    @TracerTestCase.run_in_subprocess(env_overrides=dict(DD_BOTOCORE_INVOKE_WITH_LEGACY_CONTEXT="true"))
+    def test_invoke_legacy_context_env_override(self):
+        assert config.botocore.invoke_with_legacy_context is True