Package: cdptools
cdptoolsMaking City Council data more accessible and actions taken by city council members more discoverable and trackable.AboutWe wondered why it was so hard to find out what was being discussed in Seattle City Council about a specific topic, so
we set out to solve that. The first step to this is basic data processing: automated transcript creation for city
council events, indexing those transcripts, and finally making them available on the web via our website and database.
We also wanted the entire system to aim to be low cost, modular, and open access, so that it would be relatively easy
for other CDP instances to be created and maintained. For us that means, databases and file stores are open access to
read from, the websites that users can interact with the data can be run on free hosting services such as GitHub Pages,
and computation choices should be flexible so that cost isn't a barrier issue.The first CDP instance to be deployed was for Seattle and an example of the data that is produced and available from
these systems can be seen on ourSeattle instance website. The
repository and code for the instance website can be foundhere.This repository and Python package is a collection of tools, pipelines, and processing functions, that are used by
servers to retrieve, package, store, and process data required by CouncilDataProject instances.While this package is primarily targeted towards developers of the CDP instances and backend services, a main mission
of CDP was to make city council data easier to access in all forms, on the web and programmatically, so included in
this package are objects to do just that; connect and request data from CDP instance databases and file stores
(examples below).User FeaturesPlain text query for events or minutes itemsDatabase schemaallows for simple querying of:events (meetings)voting history of a city council or city council memberbodies (committees)membersminutes itemsevent transcriptsFile stores and databases can be used in combination to download audio or the entire transcript of a meetingQuickstartSearch for events using plain text:fromcdptoolsimportCDPInstance,configsseattle=CDPInstance(configs.SEATTLE)matching_events=seattle.database.search_events("bicycle infrastructure, pedestrian mobility")# Returns list of Match objects sorted most to least relevant# [Match, Match, ...]# Use the `Match.data` attribute to view the match's datamatching_events[0].data# {#   'event_id': '05258417-9ad3-4d42-be1d-95eafcfa03c5',#   'legistar_event_id': 4053,#   'event_datetime': datetime.datetime(2019, 8, 5, 9, 30),#   ...# }Search for bills, appointments, 'minutes items' using plain text:fromcdptoolsimportCDPInstance,configsseattle=CDPInstance(configs.SEATTLE)matching_minutes_items=seattle.database.search_minutes_items("bicycle infrastructure")# Returns list of Match objects sorted most to least relevant# [Match, Match, ...]Get all data from a table:fromcdptoolsimportCDPInstance,configsseattle=CDPInstance(configs.SEATTLE)all_events=seattle.database.select_rows_as_list("event")# Returns list of dictionaries with event information# [{"event_id": "0123", ...}, ...]Download the highest confidence transcript for each event:fromcdptoolsimportCDPInstance,configsseattle=CDPInstance(configs.SEATTLE)event_corpus_map=seattle.download_transcripts()# Returns a dictionary mapping event id to a local path of the transcript# {"0123abc...": "~/4567def..."}Please view theexamplesdirectory which contains Jupyter notebooks with more examples on how to use CDP
databases and file stores.InstallationStable Release:pip install cdptoolsDevelopment Head:pip install git+https://github.com/CouncilDataProject/cdptools.gitAll City Installation:pip install cdptools[all]Individual City Installation:Seattle:pip install cdptools[seattle]Developer FeaturesModular system for gathering city council events, transcribing, and indexing them to make searchable.Data pipelines are highly customizable to fit your cities needs.Deploy and run pipelines using Docker to ensure your system has everything it needs.For additional information on system design please refer to ourdocumentation.DocumentationFor full package documentation please visitCouncilDataProject.github.io/cdptools.DevelopmentSeeCONTRIBUTING.mdfor information related to developing the code.Free software: BSD-3-Clause licenseThis package was created withCookiecutter.