AndreaBozzo/ceres-open-data-index huggingface dataset:
PermitSF Permitting Data

<strong>A. SUMMARY</strong>
This dataset includes details on select permit applications filed with the City and County of San Francisco. See Section D for the permit types currently available.

<strong>B. HOW THE DATASET IS CREATED</strong>
This dataset is an extract from the City's OpenGov permitting system.

<strong>C. UPDATE PROCESS</strong>
This dataset updates four times a day.

<strong>D. HOW TO USE THIS DATASET</strong>
This dataset contains the following permit types:
- Doors, siding, windows permit
- Fire alarm permit
- Fire sprinkler permit
- Special event permit pre-application form

<strong>E. RELATED DATASETS</strong>
<b>Department of Building Inspections Permits Data</b>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Building-Permits/i98e-djp9">Building Permits (Unique)</a></u>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Department-of-Building-Inspection-Permit-Addenda-w/87xy-gk8d">Building Permit Addenda with Routing</a></u>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Building-Permits-Contacts/3pee-9qhc">Building Permits Contacts</a></u>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Electrical-Permits/ftty-kx6y">Electrical Permits</a></u>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Plumbing-Permits/a6aw-rudh">Plumbing Permits</a></u>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Boiler-Permits/5dp4-gtxk">Boiler Permits</a></u>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Dwelling-Unit-Completion-Counts-by-Building-Permit/j67f-aayr">Dwelling Unit Completion Counts by Building Permit</a></u>

<b>Planning Department Permits</b>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/SF-Planning-Permitting-Data/kncr-c6jw">Planning Department Permits</a></u>

<b>Public Works Construction Permits</b>
<u><a href="https://data.sfgov.org/City-Infrastructure/Street-Use-Permits/b6tj-gt35">Department of Public Works Street Use Permits</a></u>
<u><a href="https://data.sfgov.org/City-Infrastructure/Large-Utility-Excavation-Permits/i926-ujnc">Department of Public Works Large Utility Excavation Permits</a></u>

<b>Fire Department Permits</b>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Fire-Permits/893e-xam6">Fire Department Permits</a></u>

<b>Other housing/ construction related datasets related to building permits</b>
<u><a href="https://data.sfgov.org/browse?q=SF%20Development%20Pipeline&sortBy=newest&utf8=%E2%9C%93">Planning Department Housing Development Pipeline Quarterly datasets</a></u>

<b>Mayor’s Office of Housing and Community Development: Affordable Housing Pipeline</b>
<u><a href="https://data.sfgov.org/Housing-and-Buildings/Affordable-Housing-Pipeline/aaxw-2cb8">Affordable Housing Pipeline</a></u>

For even more permit datasets, <u><a href="https://datasf.org/opendata/">click this link</a></u> and search “Permits.”

Organization: City of San Francisco
Form 700 Schedule C - Income, Loans, & Business Positions

This dataset is updated weekly. For more up-to-date filings, refer to the "View Filed Statements" or "Search Within Statements" links on the <a href="https://sfethics.org/disclosures/city-officer-disclosure/statement-of-economic-interests-city-officer-disclosure">Ethics Commission webpage</a>.

Income, Loans, and Business positions (other than gifts and travel payments) Non-governmental salaries of public official and spouse/registered domestic partnerships
Link to Form 700 overview: http://www.fppc.ca.gov/Form700.html
In this dataset, the fields "incomeAddress" and "loanAddress" are redacted.
This dataset contains records from Schedule C across electronic Form 700 filings with the SFEC since January 1, 2014.  To see all the data from a specific filing, you should search all the datasets listed
<a href="https://sfethics.org/disclosures/city-officer-disclosure/statement-of-economic-interests-city-officer-disclosure/data-statement-of-economic-interests">on this page</a> and filter by the filingId of the specific filing.

Prior to the start of e-filing for all in 2022, elected officials, department heads, and members of boards and Commissions filed the Form 700 with the Ethics Commission; designated employees filed the Form 700 on paper with their department's designated filing officer.

Tags: ethics, form-700, schedules, sei, statement-of-economic-interests
Organization: City of San Francisco
Requests for Information Regarding Protected Status Related to Owner Move-In Evictions

This dataset includes requests for information filed with the San Francisco Rent Board under SF Admin. Code 37.9(i) or (j). Under the Code, residents receiving an eviction notice may claim protected status either due to age and/or disability and length of tenancy or based on length of tenancy and occupancy of a child under the age of 18 during the school year. They need not be filed as estoppels. However, it has become common practice to add the request pursuant to (I) or (j) to estoppels, which is a legal term for limiting a legal action that could normally be taken, e.g. evicting. Data are available starting in January 1999.

Tags: estoppel-notices, eviction, protected-status, rent-control
Organization: City of San Francisco
Taxi A-Card Information

<strong>A. SUMMARY</strong>
A Taxi A-Card is a permit issued to taxi drivers operating in San Francisco. This dataset contains DMV, SFO and SFMTA status information for drivers’ A-cards. Taxi data providers can integrate this data to verify A-Card validity and ensure compliance.

<strong>B. HOW THE DATASET IS CREATED</strong>
This data is pulled from SFMTA Salesforce, which is integrated with SFO Ground Transportation Management System (GTMS). 

<strong>C. UPDATE PROCESS</strong>
Full update of all driver records daily with the current status in Salesforce.

<strong>D. HOW TO USE THIS DATASET</strong>
When the A‑card expiration field is blank, it indicates that the driver’s A‑card has already expired.

Organization: City of San Francisco
Mental Health (MH) and Substance Use Disorder (SUD) Provider Directory - Site List

<STRONG>A. SUMMARY</STRONG>
This dataset contains information of Mental Health (MH) and Substance Use Disorder (SUD) programs / sites. It serves as provider directory for the general public to know and search for MH and SUD programs / sites contracted with the City and County of San Francisco. 

<STRONG>B. HOW THE DATASET IS CREATED</STRONG>
The dataset is pulled from our electronic systems. 

<STRONG>C. UPDATE PROCESS</STRONG>
The dataset is updated nightly through sf.gov through Application Programming Interface (API) whenever there are changes. There is monthly review and tracking. 

<STRONG>D. HOW TO USE THIS DATASET</STRONG>
There are filters for some data fields for quick search. For example, if we want to search all MH and SUD programs that serve women, we would use the Gender filter and select “Women” and “All”.

<STRONG>E. RELATED DATASETS</STRONG>

Organization: City of San Francisco
Beach Water Quality Monitoring

Working with the San Francisco Department of Public Health (SFDPH), we jointly administer the beach water quality monitoring program in San Francisco. Both agencies participate in sample collection; the San Francisco Water, Power and Sewer Microbiology Laboratory performs bacteriological analyses.

More information: 
https://sfwater.org/index.aspx?page=87
https://sfwater.org/sapps/beachesandbay.html

Organization: City of San Francisco
Maher

Projects that are located on sites with known or suspected soil and/or groundwater contamination are subject to the provisions of Health Code Article 22A, which is administered by the Department of Public Health (DPH). Submittal of the Maher Application to and coordination with DPH may be required. Applicants may contact DPH for assistance.

For more information, please see:
<a href="http://www.sfdph.org/dph/eh/HazWaste/hazWasteSiteMitigation.asp" target="_blank">http://www.sfdph.org/dph/eh/HazWaste/hazWasteSiteMitigation.asp</a>

Organization: City of San Francisco
San Francisco Municipal Natural Gas Equipment Inventory

<strong>A. SUMMARY</strong>
The Municipal Natural Gas Equipment Inventory serves to catalog natural gas-fueled equipment used in municipally owned buildings. 
This inventory, implemented by the SF Environment Department, aims to establish an understanding of the scope of work needed to electrify municipal buildings and inform an effective and collaborative planning process.
This effort was identified as an action in Section BO-2.4 of the  <u><a href="https://www.sfenvironment.org/files/events/2021_climate_action_plan.pdf">2021 Climate Action Plan</a></u> and is included in the <u><a href="https://codelibrary.amlegal.com/codes/san_francisco/latest/sf_environment/0-0-0-577">Environment Code Chapter 7</a></u> (Municipal Green Building Requirements). 

<strong>B. HOW THE DATASET IS CREATED</strong>
The list of buildings required to report data for the Municipal Natural Gas Equipment Inventory was compiled by cross-referencing the City’s  <u><a href="https://data.sfgov.org/City-Infrastructure/City-Facilities/nc68-ngbr/about_datax">Facility Systems of Record</a></u> and the  <u><a href="https://sfpuc.org/about-us/reports/municipal-buildings-energy-benchmarking">2020 municipal benchmarking report</a></u> to identify all city-owned buildings with non-zero carbon emissions. Numerous municipal buildings are exempt from these reporting requirements, including facilities of the Port of San Francisco and buildings with a primary purpose of providing collection, storage, treatment, delivery, distribution, and/or transmission of water, wastewater, and/or power utilities. 
Each department received an inventory template, provided by the Environment Department, to submit high level building data and detailed information on each piece of natural gas equipment in use in these buildings. Departments were asked to self-report the required building and equipment data over the course of a 6-month data collection period in 2023 and are asked to keep this inventory up to date in the following years as equipment is replaced. 

<strong>C. UPDATE PROCESS</strong>
The inventory will be regularly updated by department representatives via the inventory PowerApp. When a gas-powered equipment item is retired or replaced, departments are asked to mark it as no longer in use and provide information on any electric replacement equipment, if applicable. While departments have the flexibility to update the inventory at any time, they are encouraged to do so at 6 month intervals at the minimum. 

Updated inventory data will be automatically reflected in this dataset. 

<strong>D. HOW TO USE THIS DATASET</strong>
It is important to note that this dataset does not include facilities of the Port of San Francisco and buildings with a primary purpose of providing collection, storage, treatment, delivery, distribution, and/or transmission of water, wastewater, and/or power utilities, in accordance with Environment Code Chapter 7 exemptions.

Tags: environment, environmental-health, greenhouse-gas-emissions, natural-gas
Organization: City of San Francisco
NPL Superfund Site Feature Areas (EPA)

This GIS dataset contains polygons depicting U.S. EPA Superfund features. This dataset is reserved for important Superfund site features that are best captured as polygon features in geospatial datasets. Superfund features are managed by regional teams of geospatial professionals and remedial program managers (RPMs), and SEGS harvests regional data on a weekly basis to refresh the national dataset and feature services.

Tags: 020:108, cleanup, contaminant, environment, fac, facilities, health, human, impact, management, monitoring, regulatory, remediation, segs, sites, toxics, united states
Organization: U.S. Environmental Protection Agency
Mpox Cases Over Time

<strong>A. SUMMARY</strong>
This dataset represents probable or confirmed mpox cases among San Francisco residents. Cases are included by the episode date, defined as either the date when self-reported symptoms began or, if that date is not available, the date the positive test was collected (i.e., specimen collection date). 

Data are lagged by 5 days, meaning that the most recent date of self-reported symptom or specimen collection included is 1 day prior to today. All data update daily as more information becomes available. 

<strong>B. HOW THE DATASET IS CREATED</strong>
Case information is based on confirmed or probable positive laboratory tests reported to the San Francisco Department of Public Health. Processes to provide quality assurance and other data verification are completed before the data are reported. 

This data may not be immediately available for recently reported cases because of the time needed to process tests and validate cases. 

Data are continually updated to maximize completeness of information and reporting on San Francisco residents with confirmed or probable mpox.

<strong>C. UPDATE PROCESS</strong>
Updates to the data are made through automated and manual processes once each day, Monday through Friday.  

<strong>D. HOW TO USE THIS DATASET</strong>
This dataset shows new cases and cumulative cases (probable and confirmed) by episode date. Episode date is the date of self-reported symptom onset, or, if not available, date of specimen collection (i.e., the date the person was tested). 

New cases are the count of cases with a symptom onset or specimen collection date on the given episode date. Cumulative cases are the running total of all San Francisco cases through the episode date listed.

<strong>D. CHANGE LOG</strong>
6/15/2023 - added "data_updated_at" column to reference the last date the data was compiled by scripts.

Tags: monkeypox
Organization: City of San Francisco
311 Cases

<strong>SUMMARY</strong>
Dataset contains SF311 cases created since 7/1/2008 with location information. 

<strong>UPDATE PROCESS</strong>
This dataset updates nightly. New data is added around 6am pacific time. 

<strong>HOW TO USE THIS DATASET</strong>
Download data using 'export' button to the right. For more information on this dataset, visit the  <u><a href="https://sfdigitalservices.gitbook.io/dataset-explainers/311-cases">FAQ page</a></u>.

<strong>Want to map the data in your neighborhood?</strong> 
Check out https://sf.connect.socrata.com/

For more information about Open311, see http://www.open311.org/.

Tags: 311, calls, case, cases, customer-service
Organization: City of San Francisco
Vendor Payments (Purchase Order Summary)

The San Francisco Controller's Office maintains a database of payments made to vendors from fiscal year 2007 forward. This data is presented on the Vendor Payments report hosted at http://openbook.sfgov.org, and is also available in this dataset in CSV format, which represents summary data by purchase order. We have removed sensitive information from this data – this is intended to show payments made to entities providing goods and services to the City and County and to protect individuals. For example, we have removed payments to employees (reimbursements, garnishments) and jury members, revenue refunds, payments for judgments and claims, witnesses, relocation and rehousing, and a variety of human services payments. New data is added on a weekly basis.

Supplier payments represent payments to City contractors and vendors that provide goods and/or services to the City. Certain other non-supplier payee payments, which are made to parties other than traditional City contractors and vendors, are also included in this dataset, These include payments made for tax and fee refunds, rebates, settlements, etc.

Tags: amount, open-book, openbook, procurement-contract, purchase-order, sfopenbook, spending, vendor
Organization: City of San Francisco
HSH Shelter Waitlist

<strong>A. SUMMARY</strong>
Source of data for the Shelter Reservation Waitlist. Please view the below website for more information on the shelter reservation system:
-  <u><a href="https://hsh.sfgov.org/services/how-to-get-services/accessing-temporary-shelter/adult-temporary-shelter/shelter-reservation-waitlist/">Shelter Reservation Website</a></u>

For more information on Housing Assistance, please view the website below:
-   <u><a href="https://hsh.sfgov.org/services/how-to-get-services/referrals-and-housing-assistance/">Referrals and Housing Assistance</a></u>

<strong>B. HOW TO USE THIS DATASET</strong>
Your waitlist ID is your eight-digit birthdate (MMDDYYYY) and your initials. For example, if your name is Michael Jackson and you were born on August 29, 1958, your waitlist ID is 08291958MJ.

It may take 24 hours for your name to show up on the list after you register.

If you are at the top of the waitlist there will be a note in the row with your Waitlist ID. Since some beds are gender-specific, it might be your turn even if your name does not appear in the first few rows. Make sure you search your waitlist ID to see if there are instructions to call in. Learn how to claim your bed on the FAQs page.

If you haven’t already registered, get instructions about how to sign up online or by phone. You can only have one entry in the waitlist at one time.

Organization: City of San Francisco
Police Department Incident Reports: 2018 to Present

<i>Note: All incident locations are mapped to nearby intersections to ensure anonymity. The locations of incidents reported before April 24, 2024 are currently mapped to a slightly different set of intersections than those reported on or after April 24. This may result in slight reporting irregularities for analyses that span this date. We will harmonize all historical data and resolve this as soon as possible.</i>

<strong>A. SUMMARY</strong>
Read the <u><a href="https://sfdigitalservices.gitbook.io/dataset-explainers/sfpd-incident-report-2018-to-present">detailed overview of this dataset</a></u>.

This dataset includes incident reports that have been filed as of January 1, 2018. These reports are filed by officers or self-reported by members of the public using SFPD’s online reporting system.  The reports are categorized into the following groups based on how the report was received and the type of incident:

<strong>Initial Reports</strong>: the first report filed for an incident 
<strong>Coplogic Reports</strong>: incident reports filed by members of the public using SFPD’s online reporting system
<strong>Vehicle Reports</strong>: any incident reports related to stolen and/or recovered vehicles

Disclaimer: The San Francisco Police Department does not guarantee the accuracy, completeness, timeliness or correct sequencing of the information as the data is subject to change as modifications and updates are completed.

<strong>B. HOW THE DATASET IS CREATED</strong>
Data is added to open data once incident reports have been reviewed and approved by a supervising Sergeant or Lieutenant. Incident reports may be removed from the dataset if in compliance with court orders to seal records or for administrative purposes such as active internal affair investigations and/or criminal investigations.

<a href="https://support.datasf.org/help/police-department-incident-reports-2018-to-present-overview">Read more about how incident reports are created and approved in the detailed overview of this dataset</a>.

<strong>C. UPDATE PROCESS</strong>
Updated automatically daily by 10:00 Pacific

<strong>D. HOW TO USE THIS DATASET</strong>
<u><a href="https://sfdigitalservices.gitbook.io/dataset-explainers/sfpd-incident-report-2018-to-present">Read more about</a> </u>how to appropriately use identifiers, interpret different kinds of records, and limitations of analysis related to active privacy controls.

<strong>E. RELATED DATASETS</strong>
<li> <u><a href="https://data.sfgov.org/Public-Safety/Reference-Police-Department-Incident-Code-Crosswal/ci9u-8awy/about_data">Reference: Police Department Incident Code Crosswalk</a></u>

Tags: crime, crime-reports, police-department, sfpd
Organization: City of San Francisco
Fire Incidents

<strong>Note 3/27/2024: This data pipeline was recently updated to improve data quality and efficiency. Incidents outside of SF city boundaries will no longer be assigned a supervisor_district and neighborhood_district. </strong>

Fire Incidents includes a summary of each (non-medical) incident to which the SF Fire Department responded. Each incident record includes the call number, incident number, address, number and type of each unit responding, call type (as determined by dispatch), prime situation (field observation), actions taken, and property loss.

Organization: City of San Francisco
SFEC Departmental Gifts Disclosure List of Gifts

<strong>A. SUMMARY</strong>
Section 3.217 of the Campaign and Governmental Conduct Code requires each Department Head to disclose payments that their department receives from a source that is not a federal, state, or local government and for which the department does not provide equal or greater consideration.

Each department accepting gifts will disclose all gifts received during the calendar month from a source that is not a federal, state, or local government. Departments will be expected to submit one disclosure form covering all gifts received in the preceding calendar month, when applicable.

This dataset provides detailed information about individual gifts reported by departments, such as the gift donor name, date of gift receipt, and gift value.

For more information, visit the Ethics Commission’s webpage on <a href="https://sfethics.org/guidance/city-officers/gifts-and-travel-city-officers/gifts-to-city-departments">Gifts to city departments</a>.

<strong>B. HOW THE DATASET IS CREATED</strong>
Data for this dataset is created when departments file the gifts disclosure form provided by the Ethics Commission, starting in December 2024. This form is due within 15 calendar days after the end of the month the department received the gift(s). If a department has not received any gifts in a given month, no filing is necessary.

<strong>C. UPDATE PROCESS</strong>
The dataset is updated via the Socrata Publisher API as needed when form filings are submitted and signed by city departments.

<strong>D. HOW TO USE THIS DATASET</strong>
The unique row identifier for this dataset is “Entry Id”, which is obtained by adding an index to the “Envelope Id”. The “Filing Series” column can be used to identify the filing period (year and month) and department of each gift entry. For full context, this dataset should be joined with the Cover Page dataset by the Envelope Id column.

<strong>E. RELATED DATASETS</strong>
For an overview of each department filing and any amendment filings, visit the associated dataset <a href="https://data.sfgov.org/City-Management-and-Ethics/SFEC-Departmental-Gifts-Disclosure-Cover-Page/8m6p-t372/about_data">SFEC Departmental Gifts Disclosure Cover Page</a>.

A combined view of both datasets is also available: <a href="https://data.sfgov.org/City-Management-and-Ethics/SFEC-Departmental-Gifts-Disclosure-Joined-View/uac5-pxnx/about_data">SFEC Departmental Gifts Disclosure Joined View</a>.

Organization: City of San Francisco
Planning Department Records - Non-Projects

A. SUMMARY
Records from the Planning Department's Accela database, excluding project (PRJ) records. This dataset includes non-project records, but it also include child records related to parent PRJ records. Child records associated with a parent PRJ record often represent the supplemental applications required by the Planning Department to review a development project.  The dataset of PRJ records can be found at <a href="https://data.sfgov.org/Economy-and-Community/Planning-Department-Projects/qvu5-m3a2/about_data">https://data.sfgov.org/Economy-and-Community/Planning-Department-Projects/qvu5-m3a2/about_data</a>.

B. HOW THE DATASET IS CREATED
Records exported from the Planning Department's Accela permitting system.

C. UPDATE PROCESS
Updated nightly

D. HOW TO USE THIS DATASET
For further information on permits:
<a href-"https://sfplanning.org/permits">https://sfplanning.org/permits</a>
Record types are listed here: <a href="https://sfplanning.org/resource/project-suffixes-and-descriptions-case-numbers">https://sfplanning.org/resource/project-suffixes-and-descriptions-case-numbers</a>

Tags: non-prj, planning
Organization: City of San Francisco