parse changes, moving files around for starcluster bulk

parse is the script Gabe provided
parse_2 is the modified script (plan is to integrate this as parse)
 * note SQLite appears to perform faster than MySQL
 * commit code in parse is not performing correctly
 * generator function in parse has issues

==============================

#### parse

```
def parse_files(filelist):
    if not filelist:
        return []
    parsed = itertools.imap(extract_xml_strings, filelist)
    return itertools.chain.from_iterable(parsed)

def parse_patents(xmltuples):
    if not xmltuples:
        return []
    return map(parse_patent, xmltuples)

def database_commit(patobjects, commit_frequency=commit_frequency):
    num_objs = len(patobjects)  # compute this once
    for i, patobj in enumerate(patobjects):
        alchemy.add(patobj)
        if commit_frequency and ((i+1) % commit_frequency == 0 or i == num_objs):
            alchemy.commit()
            print " *", (i+1), datetime.datetime.now()
    if not commit_frequency:
        alchemy.commit()
```

the above 3 functions are consolidated to parse_2 per below
   reason being, the generator function ends up with memory issues 
   (see Killed statements when executing against full dataset)

#### parse_2 

```
def parse_files(filelist):
    if not filelist:
        return []
    for filename in filelist:
        print filename
        for i, xmltuple in enumerate(extract_xml_strings(filename)):
            patobj = parse_patent(xmltuple)
            alchemy.add(patobj)
            if commit_frequency and ((i+1) % commit_frequency == 0):
                alchemy.commit()
                print " *", (i+1), datetime.datetime.now()
        alchemy.commit()
```


#### BENCHMARKING 

I am processing two datasets: 
2012/01/03 and 2013/04/16
  * full datasets
  * smaller datasets (random 25 record sample)

===========================

parse -- MySQL -- Full dataset
Mon Aug 19 22:19:50 CDT 2013
Killed
Mon Aug 19 22:26:37 CDT 2013

The job does not finish and runs out of memory, essentially "killing itself". This takes approximately 7 minutes to reach this conclusion.

------------------------------------------------------

parse_2 -- MySQL -- Full dataset
Mon Aug 19 22:46:15 CDT 2013
...
...
Mon Aug 19 23:33:19 CDT 2013

The job takes approximate 45 minutes with two files. 10,000 records or so.

------------------------------------------------------

parse -- sqlite - Full dataset
Mon Aug 19 23:54:03 CDT 2013
Killed
Tue Aug 20 00:00:22 CDT 2013

Similar to MySQL, the job does not complete. Takes a bit less time @ 6mins.

------------------------------------------------------

parse_2 -- sqlite -- Full dataset
Tue Aug 20 00:03:51 CDT 2013
...
...
Tue Aug 20 00:42:18 CDT 2013

The job takes a bit less than 40 minutes with two files. 10,000 records. Slight improvement over MySQL

------------------------------------------------------

parse -- MySQL -- Small dataset
Tue Aug 20 00:24:59 CDT 2013
Tue Aug 20 00:25:16 CDT 2013

Takes 17 seconds --- however this has commit issues (it doesn't properly commit)

------------------------------------------------------

parse_2 -- MySQL -- Small dataset
Tue Aug 20 00:26:25 CDT 2013
XML/out/ipg120103.xml
XML/out/ipg130416.xml
Tue Aug 20 00:26:39 CDT 2013

Takes 14 seconds --- with proper committing

------------------------------------------------------

parse -- sqlite - Small dataset
Tue Aug 20 00:23:07 CDT 2013
Tue Aug 20 00:23:17 CDT 2013

Smaller dataset takes 10 seconds
However, it appears the commit doesn't properly commit --
  * there is an error with this parse/commit method

------------------------------------------------------

parse_2 -- sqlite -- Small dataset
Tue Aug 20 00:21:39 CDT 2013
XML/out/ipg120103.xml
XML/out/ipg130416.xml
Tue Aug 20 00:21:51 CDT 2013

Smaller dataset takes 12 seconds