Add profiling support to glanceclinet

To be able to create profiling traces for Glance, client should be
able to send special HTTP header that contains trace info.
This patch is as well important to be able to make cross project
traces. (Typical case nova calls glance via python client, if
profiler is initialized in nova, glance client will add extra
header, that will be parsed by special osprofiler middleware in glance
api)
Don't worry no security issue here, trace information is signed by HMAC
key that is setted in api-paste.ini. So only person that knows HMAC key
is able to send proper header.

Change-Id: Ib13084fbe9b33c2f3dee165f7d6c778546cce6ca
Resolving the performance issue for image listing of v2 API

According to my diagnosis [1], currently v2 API client consumed a lot
of CPU times by JSON schema validation on image model initialization
stage for image listing case. Because each image entry will triggers
one validation operation which spent most CPU clocks. (My test env has
1002 image entries now, so it's O(N) logic obviously).

This fix changed the original logic, it doesn't validate each image
entry but do it only on first image entry for each page. This approach
is a trade-off, it balanced the data validity and performance.

As comparison [3], under the fixed new logic, the execution time of
image listing is about four times as fast as old logic.

[1]
Clock type: CPU
Ordered by: totaltime, desc
name                                  ncall ttot
..nt/v2/images.py:34 Controller.list  1003  12.08480
../warlock/core.py:30 image.__init__  1002  11.91074
..warlock/model.py:28 image.__init__  1002  11.90463
..arlock/model.py:130 image.validate  1002  11.73838
..nschema/validators.py:388 validate  1002  11.73682

[2]
name                                  ncall ttot
..nt/v2/images.py:35 Controller.list  1003  1.100710
..nceclient/v2/images.py:45 paginate  256.. 1.099965
../warlock/core.py:30 image.__init__  1002  0.944762
..warlock/model.py:28 image.__init__  1002  0.939725
..arlock/model.py:130 image.validate  51    0.779653
..nschema/validators.py:388 validate  51    0.779431

[3]
$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real	0m16.606s
user	0m10.629s
sys	0m2.540s

$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real	0m4.151s
user	0m1.080s
sys	0m0.092s

Change-Id: Ia6598d3c06a5ff516277053c2b6fa5db744fe9cf
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Merge "Add profiling support to glanceclinet"
Enable F841

F841 detects local variable is assigned to but never used.
This commit fixes the violations and enables F841 in gate.

Change-Id: Ic4dcac2733dfe334009327ac17aa3952cafaa63a
Replace old httpclient with requests

This review implements blueprint python-request and replaces the old
http client implementation in favor of a new one based on
python-requests.

Major changes:
* raw_request and json_request removed since everything is now being
  handled by the same method "_request"
* New methods that match HTTP's methods were added:
    - get
    - put
    - post
    - head
    - patch
    - delete
* Content-Type is now being "inferred" based on the data being sent:
    - if it is file-like object it chunks the request
    - if it is a python type not instance of basestring then it'll try
      to serialize it to json
    - Every other case will keep the incoming content-type and will send
      the data as is.
* Glanceclient's HTTPSConnection implementation will be used if
  no-compression flag is set to True.

Co-Author:  Flavio Percoco<flaper87@gmail.com>
Change-Id: I09f70eee3e2777f52ce040296015d41649c2586a
Use a correctly formatted example location in help

Change-Id: Iea1ebc13979a61d7bed397fb79e2b2a85f00c400
Closes-bug: 1342753
Don't stream non-binary requests

Setting stream=True with requests can lead to issues
with not closing the connection so the urllib3 connection
pool is not freed up, so only set stream=True if making
a request with application/octet-stream content-type.

See the body-content-workflow and keep-alive sections
in the requests docs here for more information:

http://docs.python-requests.org/en/latest/user/advanced/

Note that commit dbb242b changed the response body_iter
code to potentially return a six.StringIO object rather
than the old ResponseBodyIterator class and since the
images client code is not converting the body_iter into
a dict using json.loads, we have to do that directly
in the _request method where the body_iter is
constructed.

Co-authored-by: Flavio Percoco <fpercoco@redhat.com>

Change-Id: I86572b69c4511f933c9676108190271874346302
Partial-Bug: #1341777