Package: fastfec
FastFECA C program to stream and parseFederal Election Commission(FEC) filings, writing output to CSV.InstallationDownload thelatest releaseand place it on your path, or if you haveHomebrewand are on Mac or Linux, you can install via:brew install fastfecYou can also build a binary yourself following the development instructions below.UsageOnce FastFEC has been installed, you can run the program by callingfastfecin your terminal:Usage: fastfec [flags] <id or file> [output directory=output] [override id][flags]: optional flags which must come before other args; see below<file or id>is eithera file, in which case the filing is read from disk at the specified local patha numeric ID (only works with--print-url): prints the possible URLs the filing lives on the FEC docquery website[output directory]is the folder in which CSV files will be written. By default, it isoutput/.[override id]is an ID to use as the filing ID. If not specified, this ID is pulled out of the first parameter as a numeric component that can be found at the end of the path.The CLI will read the specified filing from disk and then write output CSVs for each form type in the output directory. The paths of the outputted files are:{output directory}/{filing id}/{form type}.csvYou can also pipe the output of another command in by following this usage:[some command] | fastfec [flags] <id> [output directory=output]FlagsThe CLI supports the following flags:--include-filing-id/-i: if this flag is passed, then the generated output will include a column at the beginning of every generated file calledfiling_idthat gets passed the filing ID. This can be useful for bulk uploading CSVs into a database--silent/-s: suppress all non-error output messages--warn/-w: show warning messages (e.g. for rows with unexpected numbers of fields or field types that don't match exactly)--no-stdin/-x: disable receiving piped input from other programs (stdin)--print-url/-p: print URLs from docquery.fec.gov (cannot be specified with other flags)The short form of flags can be combined, e.g.-iswould include filing IDs and suppress output.ExamplesParsing a local filingfastfec -s 13360.fec fastfec_output/This will run FastFEC in silent mode, parse the local filing 13360.fec, and store the output in CSV files atfastfec_output/13360/.Downloading and parsing a filingGet the FEC filing URL needed:fastfec-p13360If you havecurlinstalled, you can then run this command:curlhttps://docquery.fec.gov/dcdev/posted/13360.fec|fastfec13360This will download the filing with ID 13360 from the FEC's servers and stream/parse it, storing the output in CSV files atoutput/13360/If you don't have curl installed, you can also download the filing from the URL (https://docquery.fec.gov/dcdev/posted/13360.fec), save the file, and run (is equivalent to the above):fastfec13360.fecBenchmarksThe following was performed on an M1 Macbook Air:FilingSizeTimeMemory usageCPU usage1464847.fec8.4gb1m 42s1.7mb98%Local developmentBuild systemZigis used to build and compile the project. Download and install the latest version of Zig (>=0.9.1) by following the instructions on the website (you can verify it's working by typingzigin the terminal and seeing help commands).DependenciesFastFEC has no external C dependencies.PCREis bundled with the library to ensure compatibility with Zig's build system and cross-platform compilation.BuildingFrom the root directory of the repo, run:zigbuildThe above commands will output a binary atzig-out/bin/fastfecand a shared library file in thezig-out/lib/directoryIf you want to only build the library, you can pass-Dlib-only=trueas a build option followingzig buildYou can also compile for other operating systems via-Dtarget=x86_64-windows(seeherefor additional targets)TestingCurrently, there's C tests for specific parsing/buffer/write/CLI functionality and Python integration tests.Running the C tests:zig build testRunning the Python tests:cdpython
pipinstall-rrequirements-dev.txt
tox-epySee theGitHub test workflowfor more infoScriptspython scripts/generate_mappings.py: A Python script to auto-generate C header files containing column header and type mappings