Package: fastpat
FastpatFetch and parse patent application, grant, assignment, and maintenance info fromUSPTO Bulk Data. This handles all patent formats and outputs to pure CSV. Clusters patents by firm name, first filtering using locality-sensitive hashing, then finding components induced by a Levenshtein distance threshhold.RequirementsIn general, you'll need thefirelibrary. For parsing, you'll need:numpy,pandas, andlxml. For firm clustering, you'll additionally need:xxhash,editdistance,networkx, andCython. All of these are available through bothpipandconda. You can install all the requirements withpipby running:pip install -r requirements.txt.UsageMost common tasks can be executed through thefastpatcommand. For more advanced usage, you can also directly call the functions in the library itself. When usingfastpatyou have to specify the data directory. You can either do this by passing the--datadirflag directly or by setting the environment variableFASTPAT_DATADIR. If you've cloned the repository locally, you have to runpython3 -m fastpatinstead offastpat.Downloading DataThe following USPTO data sources are supportedgrant: patent grantsapply: patent applicationsassign: patent resassignmentsmaint: patent maintenance eventstmapply: trademark applications (preliminary)To download the files for data sourceSOURCE, run the commandfastpatfetchSOURCEThis library ships with a list of source files for each type, however this will become out of date over time. As such, you can also specify your own metadata path containing these files. You can do this by passing the--metadirflag directly or by setting theFASTPAT_METADIRenvironment variable. If you've cloned this repository locally, you can also update the files infastpat/meta.Parsing DataParsing works similarly to fetching. Simply runfastpatparseSOURCEfor one of the sources listed above.Firm ClusteringThis step is a bit more bespoke, and you may want to change things to suit your needs. But in general, there are four subcommands you can pass tofastpat firms:assignwhich eliminates duplicate or redundant patent transfers from the reassignment data,clusterwhich groups firm names into common entities using locality sensitive matching and Levenshtein distance,citeswhich aggregates citation data to the patent level, andmergewhich brings it all together into a firm-year panel. The simplest thing is to simply run these subcommands in order.ExampleSuppose you just want to parse patent grants. To do this, you would go through the following steps:Set up the environment withexport FASTPAT_DATADIR=dataFetch the grant data withfastpat fetch grantParse the grant data withfastpat parse grantCluster firm names withfastpat firms cluster --sources grantProcess citations withfastpat firms citesIf you want to work with applications, grants, reassignment, and maintenance, you can run the followingSet up the environment withexport FASTPAT_DATADIR=dataFetch all the data withfastpat fetch SOURCEfor each ofSOURCEinapply,grant,assign,maint(four separate commands)Parse all the data withfastpat parse SOURCEfor each ofSOURCEinapply,grant,assign,maint(four separate commands)Prune the resassignment data withfastpat firms assignCluster firm names withfastpat firms cluster --sources apply,grant,assign,maintProcess citations withfastpat firms citesMerge into firm-year panel withfastpat firms mergeData UpdatesContinual data updating works very well for applications and grants. Only new files will be downloaded and unzipped. The way the patent office constructs the assignment data means that you'll have to delete it and re-download it roughly once a year. Similarly, maintenance information is stored in a single file, so to update that, you'll need to delete the data fileraw/maint/MaintFeeEvents.zipand rerun the fetch command.The parsing code will also only parse new files. If you wish to rerun the parsing step for a given file, either delete its outputs (in theparseddata directory) or pass the--overwriteflag (this works for the fetching step too). The clustering and merging steps must be run for any update to propagate the changes throughout. These will take about the same amount of time even for small updates, as they are undertaking global computations. Every command is idempotent, meaning it can be rerun without breaking anything.MigrationIf you've been using older versions of this repository, the new data layout is slightly different. To avoid having to re-download everything, you can move the contents of yourdatadirectly todata/rawand usedataas the data directory path that you pass tofastpat. It's probably best to then re-parse everything and remove theparsedandtablesdirectories.