opened
mgoffin
username_0
:warning: DO NOT ACCEPT YET!

This adds the ability to submit batch requests to the Graph.

There is a new `Batch` class which has a `.submit()` classmethod. This
takes an arbitrary number of dictionaries as arguments. Each dictionary
should contain the method, URL (full or relative), and potentially a
body. For kwargs it has detailed in the docstrings that it takes similar
named arguments as methods like `.objects()` insofar as modifying how
the requests library submits your requests (modifying headers, proxy,
verify, retries). It also takes an `include_headers` named argument
which allows you to enable headers in the response.

Added a feature to common class methods like `.objects()` where you can
now pass a `request_dict` argument (boolean). If you do, instead of
actually submitting a request to the Graph, it will instead return a
dictionary which is formatted specifically for passing as an argument to
the `Batch.submit()` method. Example:

    from pytx import Batch, ThreatDescriptor

    print Batch.submit(ThreatDescriptor.objects(text='foo',
                                                request_dict=True),
                       ThreatDescriptor.objects(text='bar',
                                                request_dict=True))

Fixed a bug where "include_expired" was not converting to a string so
when it got encoded by requests it was stripping some of the value.

Bumped copyright year.

Bumped version to 0.4.0 as this is a larger addition to the library.

Related to #111
Add batch request capability.
issue
created
mgoffin
username_0
@username_1: I'll need your help testing this one. I want to make sure that this catches edge cases properly and it adheres to the request formats for submitting batch requests to the Graph. Would you be able to give it a test drive and see if it works properly?
comment
created
mgoffin
username_0
I thought that due to the responses being in a list and loop able by the client that they could acquire the successful information and handle the error responses individually. It would allow them to resubmit if they wish, or at least report back to whatever interface so the user knows what succeeded and what failed.

Or did I misinterpret how the API works? :)
comment
created
motiwarifacebook
username_1
Good point - your design decision makes more sense. Test driving now.
comment
created
motiwarifacebook
username_1
Other than the comments I just made, everything looks good. Test driving now.
comment
created
motiwarifacebook
username_1
Oh and I see your point that its only for the initial POST, not for the individual batch requests
comment
created
motiwarifacebook
username_1
The following code gives an error (at the bottom): http://pastebin.com/pa9pMTBD. Do we support POSTs in Batch.submit? What about mixed GETs and POSTs?

1. I am confused as to the difference between send() and save() in common.py. Am I correct in that save() only functions on ThreatExchange OBJECTS, whereas send() applies to URLs we might want to send parameters to? But send() can also "save" if we POST certain params via send(), right?

2. How do I access a single node (like https://graph.facebook.com/v2.5/955977237771521) and create a ThreatDescriptor object out of it using pytx?
comment
created
mgoffin
username_0
When I read the Batch request API documentation (https://developers.facebook.com/docs/graph-api/making-multiple-requests) it looked like each batch request was its own dictionary. That dictionary had to contain a method (GET, POST, etc.), a URL (relative to https://graph.facebook.com), and an optional body (for POST parameters). So one batch request would look something like:

```python
{
    'type': 'GET',
    'url': 'threat_descriptors/?access_token=foo|bar&text=foo'
}
```

So there are two (bad example since we're not doing multiple requests) ways of doing this in pytx:

```python
from pytx import Batch, ThreatDescriptor

# using the classes (note the request_dict argument being set to True to create the dict)
print Batch.submit(ThreatDescriptor.objects(text='foo', request_dict=True))

# building your own custom dictionary
print Batch.submit({'type': 'GET', 'url': 'threat_descriptors/?access_token=foo|bar&text=foo'})
```

The former seemed a lot easier on the developer to leverage, but the latter allows some more general flexibility if you need to do something specific.


`send()` allows you to send parameters to the appropriate URL. By using the appropriate object (ThreatDescriptor, etc.) we just ensure the parameters provided are going to the appropriate Graph URL with your access_token appended. So if you use `ThreatDescriptor.send(type_='GET')` we will ensure it GETs from https://graph.facebook.com/threat_descriptors/, whereas if you use `ThreatDescriptor(id=123).send(type_='POST')` we will ensure it POSTs to https://graph.facebook.com/123/.

`save()` is used to update an existing ThreatExchange Object. Sure you can do the same thing with `send()` but instead of having to specify the type_, it will know it's a POST. It also requires an instantiated class object with a defined ID (to generate the `self._DETAILS` url). `.save()` will also try to unintelligently determine what's changed on the Object since it was instantiated and attempt to submit those changes as `params` if you don't provide `params` as an argument.

Accessing a single node can be done like this:

```python
from pytx import ThreatDescriptor

# fields argument can be omitted if you just want the default fields)
t = ThreatDescriptor(id='955977237771521').details(fields=ThreatDescriptor._fields)
```

Hope that answers everything! :)
comment
created
mgoffin
username_0
I made some code changes to allow for named arguments to `.submit()`. Example:

```python
Batch.submit(foo=ThreatDescriptor.objects(text='foo', request_dict=True)
```

When doing this, we will add `'name': 'foo'` to the request. This is noted in the FB documentation as being useful for chaining batch requests where one request relies on the results of another in the same Batch.

This was already able to be done by manually adding the `name` key to a custom dictionary, but this makes it a bit easier for the above example.

I also added support for `omit_response_on_success`.
comment
created
motiwarifacebook
username_1
This all looks great. After extensive testing, I am happy to approve this! Thanks for your patience @username_0, glad we could resolve all the (non-)issues ;)
comment