2_use_case_name: FAQ for SMaRT
3_agency: Department of Commerce
3_abr: DOC
4_bureau: CENSUS - U.S. Census Bureau
8_topic_area: Government Services (includes Benefits and Service Delivery)
10_commercial_ai: Searching for information using AI.
11_purpose_benefits: The purpose of the FAQ search tool is to find the correct Census Bureau-provided answers to common questions from the public about the 2025 Census Test. The 2025 Census Test is a proposed test as part of the Small-Scale Response Test (SmaRT) program in decennial. The purpose of this program is to perform experiments using different methods to improve self-response rates and data collection. The SmaRT program accomplishes this using iterative tests with relatively small sample sizes. Four installments of the SmaRT program were fielded before the 2020 Census. Three small-scale tests have occurred so far this decade with the fourth test planned for 2025. There is an online questionnaire for each of the tests, with a census.gov landing page to access the online questionnaire. In the first three tests this decade, there were approximately 15 FAQs on the landing page that the user could read. For the 2025 Census Test, we would like to offer over 50 FAQs to answer the public’s questions via a search tool.
Current public-facing search tools at the Census Bureau use exact match only. Depending on what users type to search, the correct FAQ might not appear because users do not always use the same terminology as the Census Bureau. This FAQ search tool built with machine-learning models will use semantic similarity (i.e., synonyms or misspellings) to find the correct answers. The answers it provides back to the public are census-approved answers, and not generative AI answers. If the 2025 Census Test is approved, this search feature will be on the landing page for the survey. Implementing this search tool is considered a proof-of-concept for the use of such a search feature. If it is successful, the search tool could be used in the 2026 Census Test on its landing page, or the 2028 Dress Rehearsal. Ultimately, if it is successful, the public will benefit by finding answers to common questions more easily and more accurately.
12_outputs: Description of what the AI system outputs, whether it’s a prediction, recommendation, decision, etc.
Utilizing transfer learning from Large Language Models (LLMs) we utilize them to encode our Census approved FAQs to then compare to a user query. This user query is then encoded as an embedding from the same LLM model and then we take a distance calculation using cosine similarity to rank our reference materials. We then showcase the top 3 results depending if a threshold is met of .25 cosine similarity or above. It is a semantic search tool that ranks reference material.
16_dev_stage: Acquisition and/or Development
17_impact_type: Neither
18_date_initiated: 7/8/2024
19_date_acq_dev_began: 9/9/2024
20_date_implemented: 7/11/2025
22_dev_method: Developed in-house.
24_hisp_support: No
33_agency_data: We did not fine tune the LLM because it was just used for transfer learning. We created a question-and-answer pairing dataset to estimate search accuracy. We also selected FAQs that had a high use rate in the 2020 Census to populate the 2025 FAQ list.
34_data_docs: Documentation has been partially completed: Some documentation exists (detailing the composition and any statistical bias or measurement skew for training and evaluation purposes), but documentation took place within this use case’s development.
37_custom_code: Yes
38_code_access: Yes – agency has access to source code, but it is not public.
40_has_ato: No
42_dev_tools_wait: Less than 6 months
43_infra_provisioned: Yes
45_compute_request: Yes
47_timely_resources: Yes
49_existing_reuse: Use of existing data platforms: This use case is being developed on existing enterprise data and analytics platforms within the agency rather than procuring additional platforms or SaaS to operate.
50_internal_review: Limited documentation for review: Some information regarding the model’s performance on specific benchmarks are available or have been shared with specific stakeholders within the agency.