Gathering data sources to coordinate census outreach activities
By: Riccardo Pressiani
Tags: Case Study, Web Development, Software Development, Data Science, Census

What is the Census and why is it important?

Every 10 years, the US Government is required by the Constitution to count every single resident in the United States. This exercise has happened since the founding of the country and determines both political representation and monetary resources to every single community in the country. The State of Illinois allocated $29mn for FY2020 and an additional $14.5 were recommended for FY2021 to support a coalition of municipalities, government agencies, and nonprofit organizations to reach out to the communities they serve to ensure all residents fill out their census forms. Billions of dollars and at least 2 Congressional representative seats are at stake for the State of Illinois.

The Illinois Department of Human Services (IDHS) and the University of Illinois at Chicago College of Urban Planning and Public Administration (CUPPA) have an inter-governmental partnership to coordinate the efforts of over 360 organizations involved in community education, engagement, and responses to the 2020 Census.

The State of Illinois recognizes the challenges presented by the need to coordinate across so many different organizations and geographies. Due to the impact of COVID-19 on field efforts to engage residents, many organizations are conducting virtual townhalls and other online meetings to reach their constituents. IDHS and CUPPA’s mission is to support this large ecosystem of organizations and provide them with the tools to maximize the results of their outreach efforts. An emphasis of the project was to enhance peer-to-peer learning and collaboration opportunities among all of the organizations that are conducting field activities.

Challenge

Wepo started working with IDHS and CUPPA a few months before the beginning of the 2020 Census. IDHS and CUPPA needed quick deliveries of prototypes to validate them with the large community of organizations involved in the outreach efforts.

First, IDHS and CUPPA asked the developers to design and build a solution with the goal of aggregating social media activity and the events promoted by the organizations involved. Due to the sudden revisions of the outreach plans because of COVID-19 and the large number of entities involved, it was important for them to have a one-stop solution to monitor outreach activities.

Second, the coordinators of the program needed to be up-to-speed with the latest data on response rates. The outreach efforts needed to be particularly focused on hard-to-count and low-responding communities.

Solution

Wepo worked together with IDHS and CUPPA to understand how to best address their needs. After gathering all the requirements and specifications of the project, our engineers proposed the development of a web platform to aggregate social media activities of the organizations involved. In addition, Wepo suggested the use of the official U.S. Census Bureau to create a historical dataset of the response rates at different levels of geographic aggregation.

Particularly due to the shift to online activities because of the COVID-19 pandemic, it was important for all the entities involved in the project to have an online platform where they could see each other’s activities, get new ideas, and coordinate the promoted events. The developed platform collects data using the APIs of the social media platforms most commonly used by the organizations involved.

In addition, Wepo developed a Machine Learning model to classify the content retrieved from the organizations’ social media profiles with the goal of identifying content promoting various events, including webinars, online meetings, and virtual townhalls.

Outcome

The high number of organizations involved and the need for continuous test and validation of the new features introduced was addressed by identifying clear milestones that could be met with quick cycles of design, prototype, validation, development, and delivery. Every component of the system was designed as a containerized microservice, loosely coupled with the others, giving the development team the possibility to daily release new features in a completely transparent way.

IDHS and CUPPA now have a platform that provides them with a clear picture of the activities of the organizations involved. Moreover, our data science team worked closely with IDHS and CUPPA to reshape the response rates data collected from the U.S. Census Bureau’s APIs in a useful and clear format.

IDHS and CUPPA have provided the Census outreach organizations cohort with a tool that allows them to track each other’s activities and organically share content and ideas without any distraction. The organizations can share among themselves the approaches used to maximize the results of their efforts while drawing on best practices based on their experiences. The public has a census events calendar that they can consult to choose the most appropriate event to gather knowledge about the census.

Tools

Data Science & Machine Learning: Python, Spacy, TensorFlow

Web Development: Node.js, React

Databases: MongoDB, Postgres

Deployment: Docker, Kubernetes, Nginx, Seldon

Cloud Provider: Google Cloud Platform

Conclusions

Our team worked with excitement throughout the whole project. We consolidated our best practices in the whole pipeline of design, development, validation, and deployment. We drew out a plan with the teams from IDHS and CUPPA and implemented the data collection processes for all the data sources needed. We extracted useful information from the different sources of data used in the project. Then, we successfully used Machine Learning and Natural Language Processing techniques to speed up the process of identifying events from retrieved social media content. Lastly, we created all the interfaces needed to serve the content and information generated by the developed tools, including a web platform and automated reports.