AndreaBozzo/ceres-open-data-index huggingface dataset:
SFEC Form 3410B - Quarterly Report for Permit Consultants Filings

Effective January 1, 2015, an individual who qualifies as a permit consultant under San Francisco law must register and file quarterly reports with the Ethics Commission. A permit consultant is an individual who receives or is promised compensation to provide permit consulting services on a "major project" or a "minor project", including any employee who receives compensation for time spent on permit consulting services. 

"Permit consulting services" means any contact with the Department of Building Inspection, the Entertainment Commission, the Planning Department, or the Department of Public Works to help a permit applicant obtain a permit. "Permit consulting services" do not include simple requests for information which do not otherwise include attempts to help obtain a permit. 

A "major project" is a real estate development project located in the City and County of San Francisco with estimated construction costs exceeding $1,000,000 and which requires a permit issued by the Department of Building Inspection or the Planning Department. Estimated constructions costs are to be calculated in the same manner used to determine building permit fees under the Building Code. A "minor project" is a project located in the City and County of San Francisco which requires a permit issued by the Entertainment Commission.

The following individuals are not permit consultants and thus are not required to register or report: 
-a licensed architect or engineer of record for construction activity allowed or contemplated by the permit, or an employee of that architect or engineer; 
-a contractor who will be responsible for all construction activity associated with the requested permit, or an employee of that contractor; or 
-an employee or agent of an organization with tax exempt status under 26 United States Code Section 501(c)(3) communicating on behalf of that organization regarding the development of a project for that organization. 

Each permit consultant must register with the Ethics Commission no later than five business days after providing permit consulting services. Quarterly disclosure reports must subsequently be filed with the Ethics Commission

Tags: permit, permits
Organization: City of San Francisco
Stormwater inlets, drains and catch basins

The catch basin data is collected by SFDPW Hydraulics Engineering and SFPUC Wastewater Enterprise, Collection System Division engineers, drafters and operators.  The data is collected from sewer reference and construction documents and field surveys. You can learn more about how the City's wastewater collection system works here: http://www.sfwater.org/index.aspx?page=399

Tags: catch-basin, catchment-basin, storm-drain, stormwater
Organization: City of San Francisco
SFEC Form 107 - Training for Candidates and Treasurers - Filings

Candidates for City elective office and their treasurer/assistant treasurer must complete a training developed by Ethics Commission staff prior to each election in which the candidate will appear on the ballot. An individual who serves as the treasurer for more than one committee is not required to complete a training if that individual has completed such a training within the previous 12 months. This dataset lists candidates and treasurers that have completed the requirement and filed the required SFEC Form 107 since February 26, 2020.

Organization: City of San Francisco
Muni Stops

<strong>A. SUMMARY </strong>
All transit stops in SFMTA system. 

<strong>B. METHODOLOGY  </strong>
To provide location and feature data of all transit stops.

<strong>C. UPDATE FREQUENCY  </strong>
Stop data is exported from the Trapeze scheduling system and represents the most up to date location of transit stops. 

<strong>D. OTHER CRITICAL INFO </strong>
Updated by “Sign-up”, when transit system changes are put into effect. Updates range from every 3-6 months.

Tags: bus-stop, cable-car-stops, lrv-stops, rail-stops, shelter, stop-attributes, stop-features, stop-location, stops
Organization: City of San Francisco
San Francisco Mid-Block Points

<strong>A. SUMMARY</strong>
This is a dataset containing points representing the midpoint of each block in the City of San Francisco. This dataset is used for geomasking (e.g. removing an exact address and replacing it with a nearby anonymized point) and other anonymization tasks. 

<strong>B. HOW THE DATASET IS CREATED</strong>
This dataset was created by calculating the midpoint of each the street segment in the city (see <u><a href="https://data.sfgov.org/Geographic-Locations-and-Boundaries/Streets-Active-and-Retired/3psu-pn9h/about_data">Streets - Active and Retired</a></u>) then returning the latitude and longitude of that midpoint. A street segment is a line between two intersections. Street midpoints are calculated for each side of the block. 

<strong>C. UPDATE PROCESS</strong>
This dataset is updated weekly.

<strong>D. HOW TO USE THIS DATASET</strong>
Use this dataset for geomasking and other anonymization workflows.

<strong>E. RELATED DATASETS</strong>
- <u><a href="https://data.sfgov.org/Geographic-Locations-and-Boundaries/Streets-Active-and-Retired/3psu-pn9h/about_data">Streets - Active and Retired</a></u>

Tags: block-midpoint
Organization: City of San Francisco
Form 700 Schedule E - Travel Payments, Advances, and Reimbursements

This dataset is updated weekly. For more up-to-date filings, refer to the "View Filed Statements" or "Search Within Statements" links on the <a href="https://sfethics.org/disclosures/city-officer-disclosure/statement-of-economic-interests-city-officer-disclosure">Ethics Commission webpage</a>.

Travel payments from third parties
Link to Form 700 overview: http://www.fppc.ca.gov/Form700.html

In this dataset, the "address" fields is redacted.

This dataset contains records from Schedule E across electronic Form 700 filings with the SFEC since January 1, 2014.  To see all the data from a specific filing, you should search all the datasets listed
<a href="https://sfethics.org/disclosures/city-officer-disclosure/statement-of-economic-interests-city-officer-disclosure/data-statement-of-economic-interests">on this page</a> and filter by the filingId of the specific filing.

Prior to the start of e-filing for all in 2022, elected officials, department heads, and members of boards and Commissions filed the Form 700 with the Ethics Commission; designated employees filed the Form 700 on paper with their department's designated filing officer.

Tags: ethics, form-700, schedules, sei, statement-of-economic-interests
Organization: City of San Francisco
Organizations serving Children, Youth, and their Families

<strong>A. SUMMARY</strong>
San Francisco offers numerous events and activities tailored for children, youth, and families. However, finding and navigating the disparate sources of information can be a major challenge. Our415.org seeks to simplify this by consolidating all relevant details, ensuring that families can easily find what they need, when they need it. It also encourages discovery of new interests and things to do. This dataset includes data on organizations providing services in San Francisco for children, youth, and their families. 

<strong>B. HOW THE DATASET IS CREATED</strong>
This dataset includes data collected from various sources including DCYF’s Contract Management System, Jotform, and data shares from various departments in the City. 

<strong>C. UPDATE PROCESS</strong>
The dataset will be updated on a daily basis, reflecting changes to the source data.

<strong>D. HOW TO USE THIS DATASET</strong>
This dataset can be joined to the Services dataset (CHF-0013) using the ID field from the Organizations dataset and Organization ID field in the Services dataset. 

<strong>E. RELATED DATASETS</strong>
 <u><a href="https://data.sfgov.org/Economy-and-Community/Services-for-Children-Youth-and-their-Families/ga45-xynq/about_data">Services for Children, Youth, and their Families</a></u>

Tags: our415
Organization: City of San Francisco
District Attorney Cases Prosecuted

<strong>A. SUMMARY</strong>

<strong><i>Please note that the "Data Last Updated" date on this page denotes the most recent DataSF update and does not reflect the most recent update to this dataset. To confirm the completeness of this dataset please contact the District Attorney's office at <u><a href=”mailto:districtattorney@sfgov.org”>districtattorney@sfgov.org</a></u>.</strong></i>

This dataset includes information on all cases presented to the District Attorney’s Office in which the office has taken action to prosecute a case, either by filing new criminal charges or by filing a motion to revoke probation or parole (MTR). This does not include cases in which the San Francisco Adult Probation Department or the state Division of Adult Parole Operations files a motion to revoke.

For cases in which SFDA files a new criminal charge, the most serious offense type is categorized as the most serious offense for which SFDA prosecuted the defendant. For cases in which SFDA files an MTR, the most serious offense is categorized as the most serious offense for which the person was initially prosecuted. This is because a filing of an MTR means that new criminal charges were not filed for the new arresting offense, but rather that the District Attorney’s Office is seeking a new sanction as part of the sentence of a prior criminal conviction. For MTRs, the filing date is based on the date when SFDA filed the MTR for the new arrest.

More information about this dataset can be found under the “Cases Prosecuted” section on the <u><a href="https://sfdistrictattorney.org/policy/data-dashboards/#cases-prosecuted">Data Dashboards page</a></u>

<strong><i>Disclaimer: The San Francisco District Attorney's Office does not guarantee the accuracy, completeness, or timeliness of the information as the data is subject to change as modifications and updates are completed.</strong></i>

<strong>B. HOW THE DATASET IS CREATED</strong>

When District Attorney’s Office takes action to prosecute a case, relevant data is manually entered into the District Attorney Office's case management system. Data reports are pulled from this system on a semi-regular basis, cleaned, anonymized, and added to Open Data.

<strong>C. UPDATE PROCESS</strong>

We strive to update this dataset at the beginning of every week. However, the creation of this dataset requires a manual pull from the Office's case management system and is dependent on staff availability.

<strong>D. HOW TO USE THIS DATASET</strong>

Please review the “Cases Prosecuted” section on the <u><a href="https://sfdistrictattorney.org/policy/data-dashboards/#cases-prosecuted">Data Dashboards page</a></u> for more information about this dataset.

<strong>E. Related DATASETS</strong>

<u><a href="https://data.sfgov.org/d/czsm-3ei3">District Attorney Actions Taken on Arrests Presented</a></u> 
<u><a href="https://data.sfgov.org/d/ynfy-z5kt">District Attorney Case Resolutions</a></u>

Organization: City of San Francisco
SFMTA Parking Meter Detailed Revenue Transactions

Parking meter transaction records where each row equals a single transaction by a single customer at a single meter. Subsequent transactions by a customer to extend time, for example, are captured as new transaction record and indicated with a METER_EVENT_TYPE of AT (Additional Time).

POST_ID is a join key that refers to the identifier for a meter. You can find that inventory including location of the meter here: https://data.sfgov.org/d/8vzz-qzz9

Organization: City of San Francisco
Campaign Finance - Public Funds Approved

This dataset lists candidates who have been certified by the Ethics Commission as eligible to receive public funds. 

Each row represents a qualifying or matching request by a candidate that was approved by the Ethics Commission and the amount of funds to be issued.

More information about San Francisco’s public financing program for qualified City candidates, visit the <a href="https://sfethics.org/guidance/campaigns/candidates/public-financing-program">Ethics Commission’s Public Financing Program</a> page.

Tags: campaign-finance, candidates, election, ethics, public-financing
Organization: City of San Francisco
Planning Department Project Application Review metrics

<strong>A. SUMMARY</strong>
This dataset provides review time metrics for the San Francisco Planning Department’s application review process. The following metrics are provided: total days to Planning approval, days to finish completeness review, days to first check plan letter, and days to complete resubmission review. Targets for each metric and outcomes relative to these targets are also included. These metrics allow for ongoing tracking for individual planning projects and for the calculation of summary statistics for Planning review timelines. There are both Project level metrics and project event level metrics in this table. 

<u><a href="https://www.sf.gov/permit-performance-metrics">You can see a dashboard which shows the City's current permit processing performance on sf.gov.</a></u> 

<strong>B. HOW THE DATASET IS CREATED</strong>
Planning application review is tracked within Planning’s Project and Permit Tracking System (PPTS). Planners enter review period start and end dates in PPTS when review milestones are reached. Review timeline data is extracted from PPTS and review timelines and outcomes are calculated and consolidated within this dataset. The dataset is generated by a data model that pulls from multiple raw Accela sources and joins them together.

<strong>C. UPDATE PROCESS</strong>
This dataset is updated daily overnight.

<strong>D. HOW TO USE THIS DATASET</strong>
Use this dataset to analyze project level timelines for planning projects or to calculate summary metrics related to the planning review and approval processes. The review metric type is defined in the ‘project stage’ column. Note that multiple rounds of completeness check review and resubmission review may occur for a single Planning project. The ‘potential error’ column flags records where data entry errors are likely present. Filter out rows where a value is entered in this column before building summary statistics.  

<strong>E. RELATED DATASETS</strong>
<li>Planning Department Project Events (coming soon)
<li>Planning Department Projects (coming soon)
<u><a href="https://data.sfgov.org/d/i98e-djp9">Building Permits</a></u> 
<u><a href="https://data.sfgov.org/d/gzxm-jz5j">Building Permit Application Issuance Metrics</a></u> 
<u><a href="https://data.sfgov.org/d/abh5-gwaq">Building Permit Completeness Check Review Metrics</a></u> 
<u><a href="https://data.sfgov.org/d/XXXX-XXXX">Building Permit Application Review Metrics</a></u> 
<u><a href="https://data.sfgov.org/d/d4jk-jw33">Planning Department Project Application Review Metrics</a></u>

Tags: permitsf
Organization: City of San Francisco
District Attorney Actions Taken on Arrests Presented

<strong>A. SUMMARY</strong>

<strong><i>Please note that the "Data Last Updated" date on this page denotes the most recent DataSF update and does not reflect the most recent update to this dataset. To confirm the completeness of this dataset please contact the District Attorney's office at <u><a href=”mailto:districtattorney@sfgov.org”>districtattorney@sfgov.org</a></u>.</strong></i>

This dataset includes information on every arrest that has been presented to the SFDA since 2011 and the “action” decision made by the office on each, based on data from the SFDA’s internal case management system.

After law enforcement has made an arrest for suspected criminal activity, the arresting agency presents its evidence to the District Attorney’s Office to determine what, if any, charges can be proven beyond a reasonable doubt. This prosecutorial decision to file or to reject arrest charges represents the first, and one of the most important, decisions of the District Attorney’s Office. In addition to either filing new charges or declining to file charges (“discharging” the arrest), the prosecutor may also take other actions, such as initiating a motion to revoke probation or parole, referring the case back to the arresting agency for further investigation, or referring the case to another criminal justice agency.

More information about this dataset can be found under the “District Attorney Actions on Arrests Presented” section on the <u><a href="https://sfdistrictattorney.org/policy/data-dashboards/#sfda-actions-on-arrests-presented">Data Dashboards page</a></u>

<strong><i>Disclaimer: The San Francisco District Attorney's Office does not guarantee the accuracy, completeness, or timeliness of the information as the data is subject to change as modifications and updates are completed.</strong></i>

<strong>B. HOW THE DATASET IS CREATED</strong>

When an arrest is presented to the District Attorney’s office, relevant data is manually entered into the District Attorney Office's case management system. Data reports are pulled from this system on a semi-regular basis, cleaned, anonymized, and added to Open Data.

<strong>C. UPDATE PROCESS</strong>

We strive to update this dataset at the beginning of every week. However, the creation of this dataset requires a manual pull from the Office's case management system and is dependent on staff availability.

<strong>D. HOW TO USE THIS DATASET</strong>

Please review the “District Attorney Actions on Arrests Presented” section on the <u><a href="https://sfdistrictattorney.org/policy/data-dashboards/#sfda-actions-on-arrests-presented">Data Dashboards page</a></u> for more information about this dataset. 

<strong>E. RELATED DATASETS</strong>

<u><a href="https://data.sfgov.org/d/dcjk-vw8q">District Attorney Cases Prosecuted</a></u> 
<u><a href="https://data.sfgov.org/d/ynfy-z5kt">District Attorney Case Resolutions</a></u>

Organization: City of San Francisco
Department of Public Works Internal & External Projects (via coordinate application)

<strong>A. SUMMARY</strong>
This dataset contains Public Works coordinated projects with other city agencies and utilities companies. Each row contains project names, phases, status, contacts, and locations. It is replacing the <u><a href="https://data.sfgov.org/City-Infrastructure/-DEPRECATED-Envista-Projects-for-Utility-Excavatio/sf93-6dmr/about_data">Envista project datasets</a></u>.

<strong>B. HOW THE DATASET IS CREATED</strong>
This dataset is extracted from the DotMaps system via API. 
 
<strong>C. UPDATE PROCESS</strong>
This datasets is refreshed daily, though the underlying data generally updates on a monthly basis. 

<strong>D. HOW TO USE THIS DATASET</strong>
You can use this dataset to map out coordinated projects between Public Works, other city agencies, and utilities companies around the city of San Francisco.

Tags: construction, excavation, noi, paving, utilities, utility
Organization: City of San Francisco
Spending and Revenue

The San Francisco Controller's Office maintains a database of spending and revenue data sourced from it's citywide financial system. This data is presented on the Spending and Revenue report hosted at http://openbook.sfgov.org, and is also available in this dataset in CSV format. New data is added on a weekly basis, and is available from fiscal year 2000 forward.

Tags: amount, open-book, openbook, revenue, sfopenbook, spending
Organization: City of San Francisco
Active and upcoming street surface permits

<strong>A. SUMMARY</strong>
This table includes all active and upcoming "street use" permits issued by the Department of Public Works for activities on the street surface (not underground). It lists permits that are currently active or have a start date within the next 14 days.

<strong>B. HOW THE DATASET IS CREATED</strong>
This dataset is the union of two other permit tables on the open data portal: 
<li>  <u><a href="https://data.sfgov.org/City-Infrastructure/Street-Use-Permits/b6tj-gt35/about_data">Street Use Permits</a></u>
<li> <u><a href="https://data.sfgov.org/Economy-and-Community/Street-Vending-Permits/34ws-kyf6/about_data">Street Vending Permits</a></u> 

Only the following permits types are included from street use permits: 
        'Display',
        'Parklet',
        'FoodFac',
        'minorenc',
        'Referral',
        'NightNoise',
        'StrtImprov',
        'MajorEnc',
        'Excavation',
        'AddlStSpac',
        'StreetSpace',
        'StorCont',
        'MinorEnc',
        'ExcStreet',
        'TableChair',
        'TempOccup',
        'FreeSample'

<strong>C. UPDATE PROCESS</strong>
This dataset refreshes daily. The underlying data updates when a new permit is issued.

<strong>D. HOW TO USE THIS DATASET</strong>
This dataset will be used to create a map of active and upcoming street surface permits.

<strong>E. RELATED DATASETS</strong>
<li>  <u><a href="https://data.sfgov.org/City-Infrastructure/Street-Use-Permits/b6tj-gt35/about_data">Street Use Permits</a></u>
<li> <u><a href="https://data.sfgov.org/Economy-and-Community/Street-Vending-Permits/34ws-kyf6/about_data">Street Vending Permits</a></u>

Organization: City of San Francisco
Lobbyist Directory

<strong>A. SUMMARY</strong>

This is a directory of lobbyists registered with the Ethics Commission. Learn more
about the lobbyist program at the <a href="https://sfethics.org/guidance/lobbyists">SF
Ethics Commission</a> website. Contact and Expenditure Lobbyists both show up in this 
dataset. Lobbyists active only prior to 2018 were not required to file a "Registration"
type statement, and will not appear in this dataset. Those lobbyist's filings and 
activities will still appear in the corresponding lobbyst datasets.


<strong>B. HOW THE DATASET IS CREATED</strong>

Lobbyists are required to register and file regular statements with the Ethics Commission.
This dataset is updated with the lobbyist filings. The <a href="https://netfile.com/lobbyistpub/#sfo">live
lobbyist system</a> can be accessed as well.


<strong>C. UPDATE PROCESS</strong>

This dataset is updated with a sync script that checks nightly for new filings. Lobbyists are required to register and file monthly statements.


<strong>D. HOW TO USE THIS DATASET</strong>

Only the most recent firm for a lobbyist is displayed.

Tags: ethics, lobbyist
Organization: City of San Francisco