Repository: open-telemetry/opentelemetry-python-contrib

Issue:
opentelemetry-instrumentation-aws-lambda | Invalid type NoneType for attribute 'http.user_agent' value.
Environment:

I'm using `opentelemetry-instrumentation-aws-lambda` to instrument the Lambda function behind the API Gateway (V2).

Python==3.9
AWS-OTEL Lmabda Layer == arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-amd64-ver-1-17-0:1
fastapi==0.73.0
opentelemetry-sdk==1.17.0

Environment variables:

```bash
AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-instrument
OPENTELEMETRY_COLLECTOR_CONFIG_FILE=/var/task/src/otel-config.yaml
OTEL_METRICS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OPENTELEMETRY_EXTENSION_LOG_LEVEL=error
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=fastapi
```

OTEL Config yaml:

```yaml
receivers:
  otlp:
    protocols:
      http:

exporters:
  otlp:
    endpoint: https://<ANOTHER_SERVICE>:4317

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [otlp]

```

**Steps to reproduce**
1. Instrument your Lambda with as stated here: https://aws-otel.github.io/docs/getting-started/lambda/lambda-python
2. Lambda should be behind the API Gateway
3. Send a request

**What is the expected behavior?**
There shouldn't be any Warning logs in the Lambda logs.

**What is the actual behavior?**
I'm seeing couple of Warning in the lambda logs.


<img width="1171" alt="Screenshot 2023-05-03 at 9 30 27 AM" src="https://user-images.githubusercontent.com/16422720/235990288-c0f8cef8-b85e-47ac-a717-744571945654.png">

```bash
[WARNING]	2023-05-02T23:36:45.395Z	ec784472-c5bb-4f74-867d-e45d39e0fcca	Invalid type NoneType for attribute 'http.user_agent' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
```

**Additional context**
I'm interested to open a PR if possible to add some logic here preventing None values being set as attributes: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/46e4b1da44c534fed8e1002899e9e41e6d668018/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py#LL232C3-L232C3

Also, if there is a way to disable Tracing/Warning Logs  it would be helpful, I couldn't find a way to do that.

Another related discussion: https://github.com/open-telemetry/opentelemetry-python/discussions/3293

Thanks.





Discussion:
>I'm interested to open a PR if possible to add some logic here preventing None values being set as attributes

Feel free to send a pull request. 

>Also, if there is a way to disable Tracing/Warning Logs it would be helpful, I couldn't find a way to do that.

See https://github.com/open-telemetry/opentelemetry-python/issues/1059
@srikanthccv I need some insights about the best way here.
After further investigation, I realized that my Lambda behind the APIGW is working with no problem. The problem is for Lambda behind the ALB.
The logic itself is checking if the version is equal to `2.0` set V2 attributes, otherwise set V1 attributes:

```python
if lambda_event.get(“version”) == “2.0":
    _set_api_gateway_v2_proxy_attributes(lambda_event, span)
else:
    _set_api_gateway_v1_proxy_attributes(lambda_event, span)
```

Do you think it’s best that I keep this behavior and only add some checking inside the `_set_api_gateway_v1_proxy_attributes`

Or should I change the behavior by changing `else` to `elif lambda_event.get(“version”) == “1.0"`

In addition, I also can add an `else` statement and provide some kind of default behavior, but I don’t think I could be able to make it so generic.

The problem (The second way) is the current Lambda functions behind the ALB will not have any attributes, even attributes like `X-Forwarded-Proto` will be removed.

The first way is more like a stable way, but I need to add a couple of `if` statements to the logic.

Let me know what you think.

Thanks.
You mentioned a lot of lambda and AWS-specific detail I do not understand. What I could tell is that warning originates from the package, and it's because of `.get()`. I could suggest to the point that there should be a `None` check.

Resolving patch:
diff --git a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py
--- a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py
+++ b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py
@@ -201,30 +201,35 @@ def _set_api_gateway_v1_proxy_attributes(
     span.set_attribute(
         SpanAttributes.HTTP_METHOD, lambda_event.get("httpMethod")
     )
-    span.set_attribute(SpanAttributes.HTTP_ROUTE, lambda_event.get("resource"))
 
     if lambda_event.get("headers"):
-        span.set_attribute(
-            SpanAttributes.HTTP_USER_AGENT,
-            lambda_event["headers"].get("User-Agent"),
-        )
-        span.set_attribute(
-            SpanAttributes.HTTP_SCHEME,
-            lambda_event["headers"].get("X-Forwarded-Proto"),
-        )
-        span.set_attribute(
-            SpanAttributes.NET_HOST_NAME, lambda_event["headers"].get("Host")
-        )
+        if "User-Agent" in lambda_event["headers"]:
+            span.set_attribute(
+                SpanAttributes.HTTP_USER_AGENT,
+                lambda_event["headers"]["User-Agent"],
+            )
+        if "X-Forwarded-Proto" in lambda_event["headers"]:
+            span.set_attribute(
+                SpanAttributes.HTTP_SCHEME,
+                lambda_event["headers"]["X-Forwarded-Proto"],
+            )
+        if "Host" in lambda_event["headers"]:
+            span.set_attribute(
+                SpanAttributes.NET_HOST_NAME,
+                lambda_event["headers"]["Host"],
+            )
+    if "resource" in lambda_event:
+        span.set_attribute(SpanAttributes.HTTP_ROUTE, lambda_event["resource"])
 
-    if lambda_event.get("queryStringParameters"):
-        span.set_attribute(
-            SpanAttributes.HTTP_TARGET,
-            f"{lambda_event.get('resource')}?{urlencode(lambda_event.get('queryStringParameters'))}",
-        )
-    else:
-        span.set_attribute(
-            SpanAttributes.HTTP_TARGET, lambda_event.get("resource")
-        )
+        if lambda_event.get("queryStringParameters"):
+            span.set_attribute(
+                SpanAttributes.HTTP_TARGET,
+                f"{lambda_event['resource']}?{urlencode(lambda_event['queryStringParameters'])}",
+            )
+        else:
+            span.set_attribute(
+                SpanAttributes.HTTP_TARGET, lambda_event["resource"]
+            )
 
     return span
 
@@ -237,35 +242,38 @@ def _set_api_gateway_v2_proxy_attributes(
     More info:
     https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html
     """
-    span.set_attribute(
-        SpanAttributes.NET_HOST_NAME,
-        lambda_event["requestContext"].get("domainName"),
-    )
-
-    if lambda_event["requestContext"].get("http"):
+    if "domainName" in lambda_event["requestContext"]:
         span.set_attribute(
-            SpanAttributes.HTTP_METHOD,
-            lambda_event["requestContext"]["http"].get("method"),
-        )
-        span.set_attribute(
-            SpanAttributes.HTTP_USER_AGENT,
-            lambda_event["requestContext"]["http"].get("userAgent"),
-        )
-        span.set_attribute(
-            SpanAttributes.HTTP_ROUTE,
-            lambda_event["requestContext"]["http"].get("path"),
+            SpanAttributes.NET_HOST_NAME,
+            lambda_event["requestContext"]["domainName"],
         )
 
-        if lambda_event.get("rawQueryString"):
+    if lambda_event["requestContext"].get("http"):
+        if "method" in lambda_event["requestContext"]["http"]:
             span.set_attribute(
-                SpanAttributes.HTTP_TARGET,
-                f"{lambda_event['requestContext']['http'].get('path')}?{lambda_event.get('rawQueryString')}",
+                SpanAttributes.HTTP_METHOD,
+                lambda_event["requestContext"]["http"]["method"],
             )
-        else:
+        if "userAgent" in lambda_event["requestContext"]["http"]:
             span.set_attribute(
-                SpanAttributes.HTTP_TARGET,
-                lambda_event["requestContext"]["http"].get("path"),
+                SpanAttributes.HTTP_USER_AGENT,
+                lambda_event["requestContext"]["http"]["userAgent"],
+            )
+        if "path" in lambda_event["requestContext"]["http"]:
+            span.set_attribute(
+                SpanAttributes.HTTP_ROUTE,
+                lambda_event["requestContext"]["http"]["path"],
             )
+            if lambda_event.get("rawQueryString"):
+                span.set_attribute(
+                    SpanAttributes.HTTP_TARGET,
+                    f"{lambda_event['requestContext']['http']['path']}?{lambda_event['rawQueryString']}",
+                )
+            else:
+                span.set_attribute(
+                    SpanAttributes.HTTP_TARGET,
+                    lambda_event["requestContext"]["http"]["path"],
+                )
 
     return span
 


Test patch: