Repository: DataDog/dd-trace-py

Issue:
httplib patch doesn't seem to support a default pin
I'm trying to set up `httplib` patching in my application and I'm struggling to get it to send spans with a `service` set. I am not certain, but I think that the documentation and code for the `httplib` patch in `ddtrace.contrib.httplib` is broken, and doesn't support setting up a default service to send spans as.

Here's my reproduction case:

```python
import logging
logging.basicConfig(level=logging.DEBUG)

from ddtrace import patch
patch(httplib=True)

# Code taken from the ddtrace-py docs for httplib on python 3
from ddtrace import Pin
import http.client
import urllib.request

# Use a Pin to specify metadata for all http requests
Pin.override(http.client, service='httplib')
resp = urllib.request.urlopen('https://www.datadoghq.com/')
```

If I run that locally on the latest master, I get messages like these from the `dd-trace-agent` running on my machine:

```
2018-11-16 14:51:47 ERROR (api.go:249) - dropping trace reason: invalid span (SpanID:11098683504101424356): span.normalize: empty `Service` (debug for more info), [name:"http.client.request" resource:"http.client.request" traceID:2003532794568525569 spanID:11098683504101424356 start:1542397906651150080 duration:274516105 meta:<key:"http.method" value:"GET" > meta:<key:"http.status_code" value:"200" > meta:<key:"http.url" value:"https://www.datadoghq.com/" > meta:<key:"system.pid" value:"29250" > type:"http" ]
```

I did some debugging locally, and it looks like the default Pin added to the `HTTPSConnection` instance in the wrapped constructor (https://github.com/DataDog/dd-trace-py/blob/v0.16.0/ddtrace/contrib/httplib/patch.py#L19) is the one that ends up feeding the span's their service value. The pin added in the example code (as suggested by the docs) isn't having any effect. Unless there's something I'm missing, this seems like it makes sense, because the pin the docs suggest adding is on the `http.client` module, not on the classes inside nor on instances of those classes. How was that supposed to work @palazzem? 


Discussion:
@airhorns thank you for reporting.

It does seem the documentation for this integration is incorrect. You are right that the Pin is being set on an instance of `HTTPConnection` rather than on the module itself.

The way this integration is currently configured you would need to do something like the following:

```python
import http.client

from ddtrace import Pin

conn = http.client.HTTPConnection('www.datadoghq.com')
Pin.override(conn, service='httplib')

# TODO: Call the following:
#   conn.request()
#   conn.getresponse()
```

However, in your (our) example you are using `urllib.request.urlopen` and therefore will not have access to the `HTTPConnection` (or `HTTPSConnection`) instance in order to override the Pin.

There are a few immediate options:

- Use `http.client` library directly without the aide if `urllib.request.urlopen`

- Add a parent span wrapping your request setting a service:

```python
import logging
logging.basicConfig(level=logging.DEBUG)

from ddtrace import patch
patch(httplib=True)

# Code taken from the ddtrace-py docs for httplib on python 3
from ddtrace import tracer
import http.client
import urllib.request

# Use a Pin to specify metadata for all http requests
with tracer.trace('http.request', service='httplib'):
    resp = urllib.request.urlopen('https://www.datadoghq.com/')
```

Neither of these are likely ideal for you, but might be able to help in the meantime while we discuss as a team how to approach this issue.

Okey dokey! I also discovered another issue in the meantime :) Even if there is a pin on the `HTTPConnection`, it's `.service` isn't used to create the span right now, oops. I did a hacky fix for both these issues here: https://github.com/DataDog/dd-trace-py/compare/master...fellowinsights:httplib-fixes?expand=1
I don't think we originally ever intended `httplib` to be a service on it's own. I think we always assumed it we be a child as a part of another service. This is the point I think we need to discuss as a team to make sure that is a change we want to make.

The documentation issue is one that definitely needs to get fixed.

We will update our documentation so as to reflect the state of the `httplib` integration. I'll close this issue due its age. Feel free to re-open if you see this as an unresolved feature we should add to the integration.

Resolving patch:
diff --git a/ddtrace/contrib/httplib/__init__.py b/ddtrace/contrib/httplib/__init__.py
--- a/ddtrace/contrib/httplib/__init__.py
+++ b/ddtrace/contrib/httplib/__init__.py
@@ -1,5 +1,5 @@
 """
-Patch the built-in httplib/http.client libraries to trace all HTTP calls.
+Patch the built-in ``httplib``/``http.client`` libraries to trace all HTTP calls.
 
 
 Usage::
@@ -9,23 +9,24 @@
     patch(httplib=True)
 
     # Python 2
-    from ddtrace import Pin
     import httplib
     import urllib
 
-    # Use a Pin to specify metadata for all http requests
-    Pin.override(httplib, service='httplib')
     resp = urllib.urlopen('http://www.datadog.com/')
 
     # Python 3
-    from ddtrace import Pin
     import http.client
     import urllib.request
 
-    # Use a Pin to specify metadata for all http requests
-    Pin.override(http.client, service='httplib')
     resp = urllib.request.urlopen('http://www.datadog.com/')
 
+``httplib`` spans do not include a default service name. Before HTTP calls are
+made, ensure a parent span has been started with a service name to be used for
+spans generated from those calls::
+
+    with tracer.trace('main', service='my-httplib-operation'):
+        resp = urllib.request.urlopen('http://www.datadog.com/')
+
 :ref:`Headers tracing <http-headers-tracing>` is supported for this integration.
 """
 from .patch import patch, unpatch


Test patch: