AndreaBozzo/ceres-open-data-index huggingface dataset:
CPMS - Customer Profile Management System BET-Biometric Enrollment Tool

Biometrics Enrollment Tool (BET) is custom-built software in CPMS boundary that is used for the collection of biometric data, including photos, signatures and fingerprint images; these collection artifacts are used for record checks and identification purposes, as well as for USCIS card production. BET is used both domestically and abroad for biometric collections. The information captured is sent to the Office of Biometric Identify Management's (OBIM) Homeland Advance Recognition Technology System (HART)  to confirm and/or validate the identity of the benefit requestor.

Tags: none
Organization: Department of Homeland Security
CPMS - Customer Profile Management System

The Customer Profile Management System (CPMS) is the repository of biometric & background check data for USCIS. CPMS provides the capability to store and reuse biometric images and biographic information about applicants, and initiates background check requests about those applicants. CPMS records biometric and biographic data collected by other USCIS systems, and then uses a combination of biometric and biographic identifiers to determine an applicant’s identity in CPMS.  CPMS performs biometric vetting to determine which biometrics can be reused based on that identity, and verifies the identity of qualified applicants by sending newly captured fingerprints to DHS-IDENT for verification against previously captured fingerprint images. Once an identity is verified, CPMS requests background checks from FBI and, depending on the form type, DoD or a foreign partner country. Knowledge of both biometric identification and biographic identifiers at each stage of the process provides a person-centric view of applicants’ interactions with USCIS throughout the immigration process. CPMS provides USCIS along with ICE and CBP the capability to search for biometric and biographic information on immigrant applicants. Based on encounters at an Application Service Center (ASC), other locations around the globe and from benefit cards issued by USCIS systems, CPMS contains over 64 million identities, of which 27 million are derived from benefit cards, 4 million are derived from encounters without fingerprints, and the remainder are derived from encounters with full biometric data collection. Encounters with biometric matches are reported to IDENT to be shared among DHS components.

Tags: master
Organization: Department of Homeland Security
Spatiotemporal Assessment of the TEMPO Formaldehyde Column Retrieval using the Pandonia Global Network

Abstract copied from manuscript July 21, 2025

Launched in April 2023, the Tropospheric Emissions: Monitoring of Pollution (TEMPO), instrument provides for the first time
hourly measurements of atmospheric pollutants over most of North America at high spatial resolution (˜ 2 × 4.75 km2). This
evaluation of TEMPO’s first year demonstrates the capability of total formaldehyde column retrievals (ΩHCHO, version 3) at
different locations, seasons, and meteorological conditions. The ΩHCHO product is assessed using 36 ground-based Pandora
direct-sun measurements from Pandonia Global Network (PGN) as a reference dataset. The 36 PGN sites were chosen for
consistency in direct-sun and sky-scan measurement modes. In the first year of operation, TEMPO ΩHCHO exhibit moderate
to strong agreement at PGN sites in both measurement modes (R2 = 0.63 to 0.85). TEMPO shows a small bias of -2 ± 20% at
lower ΩHCHO (< 1.0×1016 molecule cm-2) and a larger underestimation of -22 ± 5% at higher ΩHCHO (> 1.5×1016 molecule
cm-2). TEMPO clearly captures the seasonal variability of ΩHCHO, with summer values exceeding those in winter, spring,
and fall by 62%, 45%, and 29%, respectively. TEMPO shows no consistent bias at any time of day with excellent agreement
for different meteorological conditions. For all hourly differences between TEMPO and Pandora, 96% fall within 1 × 1016
molecules cm-2. TEMPO provides almost 50% more days with at least one observation compared to observations only at 1 pm.
These findings confirm the high quality of TEMPO’s ΩHCHO measurements under a wide variety of conditions and show great
promise for future scientific applications. Portions of this dataset are inaccessible because: All publicly available. They can be accessed through the following means: Via links provided. Format: Data Availability Statement. All data are publicly available. The Pandora data is available at
PGN website (https://www.pandonia-global-network.org). Pandora data was accessed through the
EPA’s RSIG tool (https://www.epa.gov/hesc/remote-sensing-information-gateway; last access
April 2025). The TEMPO Level 2, version 3 data is available at NASA Earthdata ASDC website
545 (https://search.earthdata.nasa.gov/). The meteorological data is available at NOAA ISD Lite
database (ftp://ftp.ncdc.noaa.gov/pub/data/noaa/isd-lite/). The AERONET AOD data is available
via NASA GFSC website (https://aeronet.gsfc.nasa.gov/). 

This dataset is associated with the following publication:
Rawat, P., K. Travis, B. Henderson, J. Crawford, L.M. Judd, M.A. Demetillo, T. Lee, D. Flittner, J. Szykman, L. Valin, A. Whitehill, E. Baumann, T. Hanisco, A. Pandey, G. Abad, C. Nowlan, X. Liu, and K. Chance. Spatiotemporal Assessment of the TEMPO Formaldehyde Column Retrieval Using the Pandonia Global Network.   JOURNAL OF GEOPHYSICAL RESEARCH: ATMOSPHERES. American Geophysical Union, Washington, DC, USA, 131(2): e2025JD044788, (2026).

Tags: atmospheric-chemistry, column-formaldehyde, geostationary-satellite, pandora
Organization: U.S. Environmental Protection Agency
Source Attribution of PM2.5 Health Benefits over Northern Hemisphere Using Adjoint of Hemispheric CMAQ

Abstract copied from manuscript April 25:

The adjoint of the U.S. EPA’s Community Multiscale Air Quality (CMAQ) model is extended for hemispheric scale applications and is used to estimate location-specific health impacts from primary PM2.5, and PM2.5 precursor emissions (NH3, NOX and SO2). We estimate the monetized health burden due to mortality caused by chronic PM2.5 exposure among adults living in the northern hemisphere, using a generalized concentration-response function. The health impact sensitivities show large spatial variability over the northern hemisphere and exhibit a great deal of seasonal variability, especially for inorganic precursor emissions. The largest marginal impacts are seen for NH3 and primary PM2.5. The estimated health impacts for a 10% reduction in emissions reveal a hemispheric burden of 513,700 avoided mortality (0.9% of global mortality) and monetized health benefits at above 1.2 trillion USD2016. The largest regional contribution to hemispheric mortality is found to be in East and South Asia, particularly China and India. Monetized health burdens are estimated to be highest in China and Europe while it was relatively similar in India as in Canada and the United States. Sectoral source contribution analysis demonstrates that the sectoral emissions of agriculture and residential are the largest contributors to the northern hemispheric scale health burden, however, regional differences exist in the results. Examining location- and sector-specific health impacts can inform more effective regulatory measures. This dataset is not publicly accessible because: These files are too large for ScienceHub. It can be accessed through the following means: These are available from the corresponding author upon request. Format: CMAQ outputs formatted as IOAPI with attribution results. These are available from the corresponding author upon request. 

This dataset is associated with the following publication:
Oztaner, Y.B., S. Zhao, B. Henderson, R. Mathur, and A. Hakami. Source Attribution of PM2.5 Health Benefits Over Northern Hemisphere Using Adjoint of Hemispheric CMAQ.   GeoHealth. American Geophysical Union, Washington, DC, USA, 10(1): e2025GH001533, (2026).

Tags: aerosols-particulates, air-quality-health-impacts, attributable-risk
Organization: U.S. Environmental Protection Agency
[DEPRECATED] San Francisco Municipal Energy Benchmarking

<strong>As of March 20, 2026, this dataset will no longer update. To access new and historical data going forward, navigate to the <u><a href="https://data.sfgov.org/d/bfhx-j6n5/">dataset here</a></u>.</strong>

<strong>A. SUMMARY</strong>
San Francisco’s Existing Buildings Energy Performance Ordinance requires owners of non-residential buildings over 10,000 square feet to annually benchmark and disclose energy performance. On behalf of City agencies, the San Francisco Public Utilities Commission (SFPUC) benchmarks and reports energy use for a portfolio of approximately 500 public facilities buildings. The performance of public facilities can be examined in an interactive report at bit.ly/SFMunicipalBenchmarking, and annual reports from 2011-present are available there as well.
 
This dataset presents the energy performance and basic characteristics for public facilities that is visualized by the SFPUC’s interactive report.  
 
In addition, energy performance data for non-municipal buildings (i.e. commercial buildings of 10,000 square feet or larger, and multifamily & mixed-use buildings of 50,000 square feet or larger) is available at: bit.ly/ExistingBuildingsReport

<strong>B. HOW THE DATASET IS CREATED</strong>
In compliance with California Energy Benchmarking Regulations (CA Public Resources Code Section 25402.10 and CCR Title 20 Section 1680), and San Francisco Existing Buildings Energy Ordinance (Environment Code Chapter 20), the San Francisco Public Utilities Commission provides energy benchmarking services on behalf of municipal facilities. Details for public facilities are compiled from city records, and energy usage is compiled from utility records; related metrics such as energy use intensity are calculated from the combination of such records. Data is subjected to quality assurance validation prior to publication. For additional information regarding data sources and assumptions, please review the "Data Sources and Assumptions" page of the Municipal Facilities Energy Benchmarking dashboard: https://bit.ly/SFMunicipalBenchmarking.

<strong>C. UPDATE PROCESS</strong>
Updated Annually.

Tags: climate-change, energy, environment, sustainability
Organization: City of San Francisco
Preliminary Unintentional Drug Overdose Deaths

<strong>A. SUMMARY</strong> 
 This dataset includes data from the <u><a href=" https://sf.gov/resource/2020/ocme-accidental-overdose-reports">Office of the Chief Medical Examiner</a></u> on the number of preliminary unintentional fatal drug overdoses per month.  

<strong>B. HOW THE DATASET IS CREATED</strong> 
The Office of the Chief Medical Examiner releases a monthly report containing the previous month’s preliminary count of unintentional fatal drug overdoses. This dataset is manually updated based on that report. 

The San Francisco Office of the Chief Medical Examiner (OCME) investigates any unknown cause of death for deaths that occur in San Francisco. OCME uses drug testing, death scene investigation, autopsy, medical record, and informant information to determine the cause of death. Preliminary determinations are generally based on drug testing and death scene investigations. 

Preliminary deaths reported by the medical examiner consist of two categories: (a) cases that are still under investigation and involve suspected acute toxicity from opioids, cocaine, or methamphetamine; and (b) cases that have been finalized and were attributed to acute toxicity from any substance (including prescribed medication and over-the-counter medication).  

<strong>C. UPDATE PROCESS</strong> 
This dataset is updated monthly following the release of the monthly accidental fatal drug overdose report from the Office of the Chief Medical Examiner. Department of Public Health staff manually copy data from the Office of the Chief Medical Examiner’s report to update this dataset.  

<strong>D. HOW TO USE THIS DATASET</strong> 
This dataset is updated each month to include the most recent month’s preliminary accidental fatal drug overdose count. Counts from previous months are often also updated as it can take more than a month for the Office of the Chief Medical Examiner to finish reviewing cases.  

<strong>E. RELATED DATASETS</strong> 
<u><a href="https://data.sfgov.org/d/ubf6-e57x">San Francisco Department of Public Health Substance Use Services</a></u> 
<u><a href="https://data.sfgov.org/d/ed3a-sn39">Overdose-Related 911 Responses by Emergency Medical Services (EMS)</a></u> 
<u><a href="https://data.sfgov.org/d/k4g8-b3sf">Unintentional Drug Overdose Death Rate by Race/Ethnicity</a></u>

Organization: City of San Francisco
Police Department Investigated Hate Crimes

<strong>A. SUMMARY</strong>
These data represent hate crimes reported by the SFPD to the California Department of Justice. <u><a href="https://sfdigitalservices.gitbook.io/dataset-explainers/police-department-investigated-hate-crimes">Read the detailed overview of this dataset here.</a></u>

What is a Hate Crime? 
A hate crime is a crime against a person, group, or property motivated by the victim's real or perceived protected social group. An individual may be the victim of a hate crime if they have been targeted because of their actual or perceived: (1) disability, (2) gender, (3) nationality, (4) race or ethnicity, (5) religion, (6) sexual orientation, and/or (7) association with a person or group with one or more of these actual or perceived characteristics. Hate crimes are serious crimes that may result in imprisonment or jail time.  

<strong>B. HOW THE DATASET IS CREATED</strong>
How is a Hate Crime Processed?  
Not all prejudice incidents including the utterance of hate speech rise to the level of a hate crime.  The U.S. Constitution allows hate speech if it does not interfere with the civil rights of others. While these acts are certainly hurtful, they do not rise to the level of criminal violations and thus may not be prosecuted. When a prejudice incident is reported, the reporting officer conducts a preliminary investigation and writes a crime or incident report. Bigotry must be the central motivation for an incident to be determined to be a hate crime. In that report, all facts such as verbatims or statements that occurred before or after the incident and characteristics such as the race, ethnicity, sex, religion, or sexual orientations of the victim and suspect (if known) are included. To classify a prejudice incident, the San Francisco Police Department’s Hate Crimes Unit of the Special Investigations Division conducts an analysis of the incident report to determine if the incident falls under the definition of a “hate crime” as defined by state law. 
<u><a href="https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=PEN&sectionNum=422.55.">California Penal Code 422.55 - Hate Crime Definition</a></u>

<strong>C. UPDATE PROCESS</strong>
These data are updated monthly. 

<strong>D. HOW TO USE THIS DATASET</strong>
This dataset includes the following information about each incident: the hate crime offense, bias type, location/time, and the number of hate crime victims and suspects. The data presented mirrors data published by the California Department of Justice, albeit at a higher frequency. The publishing of these data meet requirements set forth in PC 13023.

<strong>E. RELATED DATASETS</strong>
<u><a href="https://oag.ca.gov/hatecrimes">California Department of Justice - Hate Crimes Info</a></u>
<u><a href="https://openjustice.doj.ca.gov/data">California Department of Justice - Hate Crimes Data</a></u>

Tags: hate, hate-crime, police, police-department, sfpd
Organization: City of San Francisco
San Francisco Municipal Energy Benchmarking

<strong>This dataset will replace the <u><a href="https://data.sfgov.org/d/k3fc-45qw/">previous version of this dataset</a></u>, which will no longer be updated.</strong>

<strong>A. SUMMARY</strong>
San Francisco’s Existing Buildings Energy Performance Ordinance requires owners of non-residential buildings over 10,000 square feet to annually benchmark and disclose energy performance. On behalf of City agencies, the San Francisco Public Utilities Commission (SFPUC) benchmarks and reports energy use for a portfolio of approximately 500 public facilities buildings. The performance of public facilities can be examined in an interactive report at https://bit.ly/SFMunicipalBenchmarking, and annual reports from 2011-present are available there as well.

This dataset presents the energy performance and basic characteristics for public facilities that is visualized by the SFPUC’s interactive report.

In addition, energy performance data for non-municipal buildings (i.e. commercial buildings of 10,000 square feet or larger, and multifamily & mixed-use buildings of 50,000 square feet or larger) is available at: https://bit.ly/ExistingBuildingsReport

<strong>B. HOW THE DATASET IS CREATED</strong>
In compliance with California Energy Benchmarking Regulations (CA Public Resources Code Section 25402.10 and CCR Title 20 Section 1680), and San Francisco Existing Buildings Energy Ordinance (Environment Code Chapter 20), the San Francisco Public Utilities Commission provides energy benchmarking services on behalf of municipal facilities. Details for public facilities are compiled from city records, and energy usage is compiled from utility records; related metrics such as energy use intensity are calculated from the combination of such records. Data is subjected to quality assurance validation prior to publication. For additional information regarding data sources and assumptions, please review the "Data Sources and Assumptions" page of the Municipal Facilities Energy Benchmarking dashboard: https://bit.ly/SFMunicipalBenchmarking.

<strong>C. UPDATE PROCESS</strong>
Updated Annually, but any adjustments may be made throughout the year.

<strong>E. RELATED DATASETS</strong>
<u><a href="https://data.sfgov.org/d/96ck-qcfe/">Existing Buildings Energy Performance Ordinance Report</a></u>

Organization: City of San Francisco
Dataset for development and evaluation of a qPCR method modification that mitigates DNA recovery losses for analyses of E. coli fecal indicator bacteria in Great Lakes region recreational waters

Results previously reported by EPA/ORD on the modified method have led to its adoption into a Standard Operating Procedure for labs performing qPCR analyses for E. coli at recreational beaches in the state of Michigan. This dataset provides further documentation of the performance and applicability of the modified method for recreational beach water testing including data from an extensive survey that can be used to determine when and where unacceptable levels of DNA losses may occur with and without this method modification in the Great Lakes region.

Please see the attached dataset description document for additional information. 

Tags: e-coli, great-lakes-region, interference, michigan, qpcr-methods-comparisons
Organization: U.S. Environmental Protection Agency
SFEC Campaign Consultant Report - Political Contributions

<strong>A. SUMMARY</strong>
This dataset contains information about the political contributions section of the Campaign Consultant Report.

For more information, visit the Ethics Commission’s webpage on <a href="https://sfethics.org/guidance/campaign-consultants">Campaign Consultants</a>.

<strong>B. HOW THE DATASET IS CREATED</strong>
Data for this dataset is created when campaign consultants electronically file the following reports with political contribution information: 

Form 1 – Registration Report
Form 3 – Quarterly Disclosure Report for Campaign Consultants
Form 6 – Campaign Consultant Termination Statement

This dataset contains reports filings starting from December 9, 2024, when electronic filing of campaign consultant reports will be required.

<strong>C. UPDATE PROCESS</strong>
The dataset is updated via the Socrata Publisher API as needed when form filings are submitted and signed by campaign consultants.

<strong>D. HOW TO USE THIS DATASET</strong>
The unique row identifier for a contribution entry is “Entry Id”. This dataset should be joined with the SFEC Campaign Consultant Report dataset by “Envelope Id”.

<strong>E. RELATED DATASETS</strong>
For detail reporting on campaign consultant activities, visit <a href="https://data.sfgov.org/City-Management-and-Ethics/SFEC-Campaign-Consultant-Report/iv34-5p9x/about_data">SFEC Campaign Consultant Report</a>.

Organization: City of San Francisco
Stop Signs

A. SUMMARY 
Stop signs located at each intersection node.

B. METHODOLOGY  
Stop sign legislation information is taken from MTAB legislation. Stop sign implementation information is taken from SSD Shops Reports and then parsed via python code. Implementation for stop signs will specify where signs are installed or removed, and the work order that the sign installation/removal was spec'd in. 

C. UPDATE FREQUENCY  
Updated four times a year at the end of each quarter.

Tags: intersections, signage, stop-signs, traffic-control
Organization: City of San Francisco
311 Service Requests - Street Lights - All Out - Historical

As described in https://data.cityofchicago.org/stories/s/311-Dataset-Changes-12-11-2018/d7nq-5g7t, the function of this dataset was replaced by https://data.cityofchicago.org/d/v6vf-nfxy. This dataset is historical-only.

All open reports of "Street Lights - All Out" (an outage of 3 or more lights) made to 311 and all requests completed since January 1, 2011.The Chicago Department of Transportation (CDOT) oversees approximately 250,000 street lights that illuminate arterial and residential streets in Chicago. CDOT performs repairs and bulb replacements in response to residents’ reports of street light outages. 
 
Whenever CDOT receives a report of an “All Out” the electrician assigned to make the repair looks at all the lights in that circuit (each circuit has 8-16 lights) to make sure that they are all working properly. If a second request of lights out in the same circuit is made within four calendar days of the original request, the newest request is automatically given the status of “Duplicate (Open).” Since CDOT's electrician will be looking at all the lights in a circuit to verify that they are all working, any “Duplicate (Open)” address will automatically be observed and repaired. 
 
Once the street lights are repaired, the status in CSR will read “Completed” for the original request and “Duplicate (Closed)” for any duplicate requests.  A service request also receives the status of “Completed” when the reported lights are inspected but found to be in good repair and functioning; when the service request is for a non-existent address; or when the lights are maintained by a contractor. Data is updated daily.

Tags: 311, historical, lights, service-requests, streets
Organization: City of Chicago
Taxi Trips (2013-2023)

<b>This dataset ends with 2023. Please see the Featured Content link below for the dataset that starts in 2024.</b>

Taxi trips from 2013 to 2023 reported to the City of Chicago in its role as a regulatory agency.  To protect privacy but allow for aggregate analyses, the Taxi ID is consistent for any given taxi medallion number but does not show the number, Census Tracts are suppressed in some cases, and times are rounded to the nearest 15 minutes.

Due to the data reporting process, not all trips are reported but the City believes that most are.

Tags: historical, taxis, transportation
Organization: City of Chicago