The Implementation of Dashboards in Governments Division 

Surveys

Terri L. Craig and Carma R. Hogue 

U.S. Census Bureau 

4600 Silver Hill Road, Washington DC 20233 

terri.l.craig@census.gov, carma.ray.hogue@census.gov

Abstract 

Dashboards have gained popularity as a quick way to monitor metrics in manufacturing, business, healthcare, and 

even in election polling. After facing challenges with low unit response rates and Total Quantity Response Rates 

(TQRRs) for the 2007 Census of Governments: Finance Component, we decided to develop a dashboard that would 

monitor response in the very early stages of processing. The initial dashboard developed for the Annual Survey of 

Local Government Finances attempts to yield compliant unit response rates for each state by type of government, 

compliant TQRRs for each state by key item by type of government, and an adequate number of responses and 

acceptable response rate in each imputation cell. This dashboard has been used as a prototype for monitoring 

response for other surveys of governmental units. Initial dashboards have been developed for the Annual Survey 

of Local Government Finances, Annual Survey of Public Employment and Payroll, and Quarterly Survey of 

Property Tax Collections. This paper presents the conceptual and operational challenges that we faced in

constructing each dashboard. 

Background 

In response to the 21 recommendations issued in 2007 by the Committee on National Statistics in its report, State 

and Local Government Statistics at a Crossroads, the U.S. Census Bureau’s Governments Division (GOVS) 

launched a massive modernization and re-engineering effort. As a part of its re-engineering efforts, GOVS 

implemented the use of dashboards to guide non-response follow-up and to help ensure that the Census Bureau 

Statistical Quality Standards on response rates are met. 

Responsive design has been widely used in household surveys. Real-time paradata provide tools that survey 

analysts can use to monitor survey processes to yield better quality statistical estimates. Groves and Heeringa 

(2006) presented examples of using paradata to monitor survey processes to increase response. Laflamme and Mohl 

(2007) and Laflamme, et al (2008) discussed changes in the field collection procedures for Statistics Canada surveys 

to actively manage survey quality. Axinn, Link, and Groves (2009) offered a suite of tools for the survey 

methodologists to use to improve the representative qualities of surveys. At the same time dashboards have gained 

popularity as a quick way to monitor metrics in manufacturing, business, and healthcare. It is only natural to look 

towards dashboards as a way to monitor survey processes paradata to actively manage survey quality. Although 

there is not a vast literature on using paradata and responsive design in establishment surveys, it is thought that such 

an approach should certainly be researched as a way to improve the quality of the estimates from our surveys of 

governments. To monitor the process, we developed nonresponse follow-up dashboards to guide analysts in their 

nonresponse follow-up. 

In this paper, the description of the implementation and development of our initial dashboard for monitoring nonresponse follow-up is in Section II. Section III describes the alterations to the initial dashboard to fit other GOVS 

surveys. Section IV provides some results from our use of dashboards to monitor survey processes. Section V 

describes how we plan to expand the use of dashboards for other facets of the survey process for GOVS surveys. 

Dashboard Implementation in Governments Division 

This report is released to inform interested parties of research and to encourage discussion of work in progress. 

The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. 
The Census Bureau Statistical Quality Standards set requirements on levels of response using two basic response 

rates: the Unit Response Rate (URR) which measures the percentage of units responding to the survey and the Total 

Quantity Response Rate (TQRR) which yields a rate that reflects the portion of the estimate that is from the 

respondents. During the preparations for the release of the 2007 Census of Governments: Finance Component 

(CoG-F), the review of the Total Quantity Response Rates and the Unit Response Rates, by state, revealed two 

things. First, the URR was low in two states (below the required 60 percent specified in the Census Bureau’s 

Statistical Quality Standards). Second, the low URR in those states was not a critical factor because the TQRRs 

were well above the statistical standards requirement for TQRRs of 70 percent. Looking at the TQRRs in the other 

states where the URR met the statistical standards showed about six states were below 70 percent. In a plan to 

increase future response rates Governments Division offered as a mitigation strategy a dashboard that would guide 

the analysts through nonresponse follow-up (NRFU) to decrease the number of noncompliant states in the future. 

Governments Division conducts the Census of Governments every five years in years ending in 2 and 7. This 

Census has three components: Organization, Finance, and Employment. The Organization component provides 

counts of the number of general purpose and special purpose governments. General purpose governments are 

counties, municipalities, and townships, which cover multiple governmental functions (police, fire, education, 

administrative, etc.) Special purpose governments generally cover a single or very limited number of purposes. For 

example, school districts provide data on the education function. Special districts (airport authorities, regional 

libraries, housing authorities, drainage ditch districts, etc.) provide data on their single function. The CoG-F 

provides data on revenue, expenditures, debt, and assets from state and local governments and pension systems. The 

Employment component provides data on full-time and part-time employees and payroll for state and local 

governments by function. Between censuses, various annual and quarterly surveys provide more current estimates 

for our data users. The annual sample surveys and the censuses have historically used the same questionnaires. 

The first dashboard developed was for the Annual Survey of Local Government Finances (ALFIN). For this survey, 

it was decided to monitor check-in rate as a surrogate for the URR, a preliminary TQRR for key estimates, and an 

estimated response rate for each imputation cell. In order to raise the TQRRs, the most efficient method would be to 

follow-up large units that would contribute the most to the TQRRs. Monitoring solely on the URR could increase 

the sample’s representativeness if the effort did not focus on following up only the large units. Because most 

nonrespondents tend to be small- and medium-sized units, increasing response from those units by emphasizing 

imputation cell response rates should improve the representativity. The monitoring of all three sets of response 

rates, (URR, TQRR, and Imputation Cell), should yield a responsive design that will be representative and will 

satisfy the Census Bureau Statistical Quality Standards. 

Check-in Rate 

The check-in rate is used to give us a predicted response rate. This rate is not an accurate reflection of the URR 

because it does not account for the quality of the response. After editing, the unit may be found to lack sufficient 

reported data to be deemed a unit respondent. Consequently, in building the dashboard to monitor when the Quality 

Standards URR level has been obtained, some allowance, ‘padding’, was added in, that is, rather than striving to 

obtain a URR of 60 percent, the dashboard is set to obtain a rate higher than 60 percent. The amount of padding 

depends on several factors, such as, historical patterns of check-in quality. To calculate the check-in rate, the unit’s 

status code is used to determine if it should be included in both the numerator and denominator (as a response) or 

just in the denominator (as a nonresponse). A check-in response rate is calculated for the whole state and by type of 

government (county, city, township, special district, independent school district). 

The state check-in response rate is displayed as a bar chart, as shown in Figure 1. If the check-in response rate is 

less than 70 percent the bar is red. If the check-in response rate is equal to or greater than 70 percent, the bar is 

green. For the ALFIN, there is also a measure indicator behind the bar. The measure indicator is set up like a traffic 

light with red indicating that the response rate is in the 0 percent to 70 percent range (which is unacceptable), yellow 

from 70 percent to 75 percent, and green from 75 percent to 100 percent. The yellow area is called ‘padding.’ 

When a unit falls in this area it indicates that the state could fall below 70 percent once the data are reviewed. 

Therefore, if time permits the analyst should follow-up with this state to attempt to get the check-in response rate 

above 75 percent. This will give the state a better chance of achieving a final response rate greater than 60 percent, 

thus meeting the Quality Standards. Below the state check-in rate bar there is a state summary that lists the number 

of school districts and major local county and city governments. This will give the analyst a summary of everything 

2 
that is needed for that state and what is counted in the check-in rate. It also lists the check-in rate, the number of 

units checked in, and the number of units in the survey. The analyst also has the option to view all local government 

units in the state to identify specific units that have or have not been checked in. Reported data for a unit can be 

viewed if the unit has been checked in with data. 

 

 Figure 1-SOURCE: U.S. Census Bureau, 2007 Census of Governments – Finance Component 

The check-in response rates by type of government are displayed as dials. The red section of the dial is 0 percent to 

60 percent. The yellow section of the dial is 60 percent to 70 percent. The green section of the dial is 70 percent to 

100 percent. If the type of government has a check-in rate less than 70 percent, the summary section (showing 

number in survey, number checked in, check-in rate and the additional number of responses needed to meet the 

goal) is written with a red background; otherwise it has a grey background, allowing the analyst to easily see which 

types of governments should be followed-up. Figure 2 below shows the type of governments’ response rates for the 

same state as Figure 1. Therefore, it is evident that follow-up is needed for special districts in this state. 

Figure 2-SOURCE: U.S. Census Bureau, 2007 Census of Governments – Finance Component 

TQRR 

For the ALFIN, TQRRs are tracked for four key estimates: Total Revenue, Total Expenditures, Outstanding Debt, 

and Total Assets. For each variable, an estimate of the current year’s total forms the denominator. This estimate is 

the sum of each government’s weighted current year response or prior year response if the current year is not yet 

available. The weighted values for the current year respondents form the numerator. The top of the TQRR 

dashboard page also contains a bar chart like the check-in page. There is one bar for each key estimate. The bottom 

of the page contains the dials for each type of government. The first key estimate shown is Total Revenue. There is 

a drop down menu from which the analyst may select the key estimate to be displayed. The analyst quickly see if 

the TQRR for the state might meet Quality Standards, as well as which key estimates might not meet Quality 

Standards and to help determine for which states non-response follow-up should occur. The TQRR tab also has an 

option for the analyst to see individual units. Figure 3 shows the TQRR bars for the same state that the other two 

figures. It shows that although the check-in rate does not meet the Quality Standards, the TQRRs do. The lowest 

TQRR is 95.0% for Total Assets. Figure 4 shows the TQRR for Total Assets by Type of Government. The Special 

Districts dial shows the lowest TQRR. Because it is over 70%, the focus in this state should be to raise the URR for 

Special Districts. 

 

3 
 

 

 Figure 3-SOURCE: U.S. Census Bureau, 2007 Census of Governments – Finance Component 

Figure 4-SOURCE: U.S. Census Bureau, 2007 Census of Governments – Finance Component 

Imputation Cells 

The imputation cells are defined by state, type of government, and sometimes population. Independent school 

districts are excluded from the imputation cell tab because school district data are collected in a different survey. 

Likewise, special districts are excluded because a program was independently developed to get imputation cells with 

at least 15 respondents and at least a 50 percent response rate. The methodology begins at the state level, and then 

will go to the Census Division level, then the Census Region level, and lastly nationally. Prior to developing the 

dashboard, a general purpose response file would be reviewed to target states that had imputation cells that did not 

meet the response criteria. The non-responding units in those imputation cells would be targeted for non-response 

follow-up. By having this information in the dashboard, this step is eliminated, and when it is time for imputation, 

the cells most likely will meet the response requirements needed. The dashboard page displays a dial for each 

imputation cell. As in the other tabs, we allowed ‘padding’ because the quality of the response and the prior year 

response is not known using the check-in information. The red section on the dial is 0 to 60 percent, the yellow 

section is 60 to 70 percent, and the green section is 70 to 100 percent. Beneath the dial the analyst can see the 

imputation cell response rate, the number of responses, the total number of units in that imputation cell, the 

definition of the imputation cell, and the number of additional units needed to get the dial into the green section. 

The background color of the information written under the dial is red if more units are needed, gray otherwise. 

Once the ALFIN dashboard was developed, we began working on other survey dashboards that would require minor 

adjustments to the basic structure set forth in the ALFIN dashboard. 

4 
Customization of Dashboards for the Governments Division Surveys 

In addition to the dashboard developed for ALFIN, dashboards have been developed for the Annual Survey of 

Public Employment and Payroll (ASPEP), the Quarterly Survey of Property Tax Collection (F-71), and the Medical 

Expenditure Panel Survey - Public (MEPS).

Annual Survey of Public Employment and Payroll 

The ASPEP is an annual survey that collects full- and part-time data on public employment and payroll from state 

and local governments. For most economic surveys, the TQRRs are very high if the large units respond, but the 

URRs are generally low because small units, which are numerous but contribute little to the TQRRs, are not 

followed up. For ASPEP, data on number of employees is not as skewed as most economic data. Therefore, great 

effort is placed on URRs. Although the national URR is usually above 85.0 percent, the TQRRs on number of 

employees and payroll may not pass the Quality Standards requirement of 70 percent. Obtaining major city and 

county data may not be enough to bring the TQRRs up to an acceptable level. Many more medium-sized 

governments must also report. 

For the ASPEP the data are processed differently. Special districts are imputed first, then dependent and 

independent schools, and lastly general purpose governments. For the check-in tab, a dial for schools and state 

agencies was added to the dashboard. 

Because special districts typically have a low response rate, require more follow-up, and are the first group of data to 

go through imputation, the monitoring of them is important. A tab for special districts showing the check-in rate by 

special district function was added. There is also a button to see more information on each special district function; 

a table is displayed which lists the check-in rate, the number of units comprising the numerator and the number of 

units comprising the denominator. 

There are two TQRR tabs: overall TQRR and by type of government TQRR. The key estimates displayed on the 

TQRR tabs are full-time employees, full-time pay, part-time employees, and part-time pay. There is the option of 

scrolling down to see all key estimates on one screen or to select one key estimate to be displayed on the screen. 

The description background color at the bottom of the dial corresponds to the area to which the dial is pointing. 

This makes it easier for the analyst to quickly see which states and types of governments need to be followed-up

with. 

On the imputation cell tab, using the drop down menu the general purpose, dependent schools, or independent 

schools imputation cells are displayed. The information at the bottom of the dial is similar to what is displayed on 

the ALFIN dials. As in ALFIN, this helps to eliminate the step of getting a file shortly before non-response followup, looking at the response/non-response information and sending a list of where the analyst should focus their nonresponse follow-up. 

Quarterly Survey of Property Tax Collection (F-71)

 

The F-71 is mailed to property tax collectors each quarter. This survey has a compressed processing schedule, 

which makes the use of a dashboard extremely helpful in monitoring response. 

The first tab of the dashboard, ‘home page’, displays a dial for the overall TQRR and URR for the current quarter 

being processed. The colors and percentages for the dial are red if the response rate is less than 60 percent, yellow if 

it is between 60 - 70 percent, and green if it is >70 percent. This page gives the analyst a quick synopsis of how 

response is progressing for the current quarter. 

The TQRR/URR tabs show the TQRR/URR for the current quarter and the previous quarters. Up to 11 previous 

quarters can be shown. The analyst can use this page to see collection is compared with previous quarters. The dial 

has a red, yellow, and green color scheme, which is the same as on the home page. 

The imputation cell tab shows the imputation cells at the highest level as a horizontal bar chart. The bar is red if 

more units are needed in the imputation cell. It is green otherwise. 

5 
The metrics included in the MEPS dashboard that differ from the finance dashboard are listed below: 

a pie diagram showing check-in rate by type of submission (mail vs. web), 

a pie chart showing unit edit status, 

a line graph showing the response rate over time, and 

a map of the United States showing when states have been received, the number of responses received to the 

number expected, and the percent received. The map is color coded as shown in the table below. 

Medical Expenditure Panel Survey - Public (MEPS) 

MEPS is a reimbursable survey done on behalf of the Agency for Healthcare Research and Quality in the U.S. 

Department of Health and Human Services. Governments Division collects, processes, and reports data for the 

public sector. Another division handles the private sector. The MEPS dashboard was developed using a SAS 

program. The SAS program was given to the analysts so they can obtain the metrics whenever needed. They also 

had the option to examine the units in their workload or all the units in the survey. The output from the SAS 

program is saved as a .pdf file and therefore can become a static report. 

State Color Response Percentage 

 < 20% 

20 – 40%

40-60%

60-80%

≥ 80%

Results 

The implementation of dashboards has enabled staff to monitor the check-in response rates and determine the focus 

of follow-up. In the ALFIN, it has given staff the opportunity to see where there are problems and attempt to 

mitigate them prior to the scheduled data release, thus allowing Census to release estimates that meet the Census 

Bureau’s Quality Statistical Standards. The number of states failing the Quality Standards at the time of preliminary 

data release fell from 6 states in 2007 to 2 states in subsequent years. Table 1 shows the number of states failing the 

Quality Standards at the time of preliminary data release in 2007, 2008, and 2009. 

Table 1: Number of Noncompliant States in ALFIN: 2007 - 2009 

2007 2008 2009

Number of Noncompliant States for 2 0 0 

URR 

Number of Noncompliant States for TQRR 

 Revenue 0 0 0 

 Expenditure 1 0 0 

 Debt 5 1 1 

 Assets 4 2 2 

SOURCE: U.S. Census Bureau, 2007 Census of Government: Finance Component, 2008 

and 2009 Annual Survey of Local Government Finances. 

6 
Future Dashboards in Governments Division Surveys 

Dashboard Evaluations 

The dashboard evaluation will see how the use of the check-in rate estimates the response rate for the state, state by 

type of government, and imputation cells. It will also see how using prior year data for the non-respondents in the 

total quantity response rates compares with the final total quantity response rates. The results will indicate whether 

the padding level is sufficient and whether another way to calculate the nonrespondents portion of the total quantity 

response rate is needed. 

Prioritized Unit Follow-up Listing 

Currently, for some dashboards it is possible to see the individual units. This listing would be more useful if it 

strategically prioritized listing units by those that are expected to have the most impact on the estimate or for chronic 

respondents where their size is not know. Each unit will be given a score based on information unique to the survey, 

such as the last time the unit reported, the value of certain key variables, and their survey weight. Eventually, it is 

hoped that this dashboard listing will be incorporated into the nonresponse tracking application and then integrated 

into the Governments Master Address File. 

Integrated Review Dashboard 

The planned integrated review dashboard is one that will work to monitor the processes for the Finance surveys. 

The Annual Summary of State and Local Government Finances is an aggregation of several different data 

collections: web collection, paper collection, and compilations from administrative records for local governments, 

school district compilations, pension plan compilations, and data from the Integrated Post-secondary Education Data 

System from the National Center for Education Statistics. These data must all be available concurrently to release 

the Annual Summary. A dashboard can monitor the timing, but also insure that the quality of the data is compliant 

with the Census Bureau’s Statistical Quality Standards. 

Macro Review Dashboard 

Prior to data release, a dashboard to guide the analyst through the final macro review of the estimates in a survey 

year or aggregates in a census year would be beneficial. The macro review will examine trends of important 

aggregates over time. Influential values will be flagged for further investigation with indications of the most 

important detail item weighted contributors to the outlier. Coefficients of variation that are noncompliant with 

standards and those that have displayed substantial changes since the prior year will also be identified. Quality 

checks of summations, counts, etc. and HB ratio edits on final macro level estimates will be put into the dashboard. 

Conclusion 

 

The massive modernization and re-engineering effort has begun. We continue to enhance the current dashboards 

and expand the use of dashboards for other Governments Division Surveys. The current dashboards in conjunction 

with our future dashboard plans will ensure that the Census Bureau’s Statistical Quality Standards are met and that 

the Census Bureau continues to release quality data products to its users. 

References 

Axinn, William, Cynthia F. Link, and Robert M. Groves (2009). “Responsive Survey Design, Demographic Data 

Collection, and Models of Demographic Behavior.” NSFG Paper No. 09-005. October 2009. 

Groves, Robert and Steven Heeringa (2006). “Responsive Design for Household Surveys: Tools for Actively 

Controlling Survey Errors and Costs.” Journal of the Royal Statistical Society, Series A, Volume 169, Part 3. pp. 

439-457.

Laflamme, Francois, Mike Maydan, and Andrew Miller (2008). “Using Paradata to Actively Manage Data 

Collection Survey Process.” Joint Statistical Meetings Proceedings of the Section on Survey Research Methods. 

7 
Laflamme, Francois and Chris Mohl (2007). “Research and Responsive Design Options for Survey Data Collection 

at Statistics Canada.” Joint Statistical Meetings Proceedings of the Section on Survey Research Methods.

National Research Council, Committee on National Statistics, State and Local Government Statistics at a 

Crossroads, September 2007. 

8