Searching for the number of COVID-19 cases in NYC schools in my neighborhood shouldn’t be this…
By: Jonathan G. Mogil
Tags: Covid 19, New York City, Data Science, New York, Data

This article and the new dashboard I have made available for the public focuses on three of the categories listed above: Public, Charter and Private schools within New York State, effectively capturing the 3.3 million students (Pre-K through 12), teachers and staff at the roughly 6,300+ schools across the state.

As of writing, over 28,780¹ or 0.86%² of New York State’s school population had tested positive for COVID-19 since September 8th, 2020, when the data started being tracked. These figures include students, teachers, and other staff who have been tested both on-site and off-site. This compares to an incremental increase for all of New York State’s population of approximately 1.81%³ over the same period (i.e., from September 8th to December 14th).

Reported COVID-19 cases in Manhattan as of December 11th, 2020. An interactive map can be found at moges.io. Underlying data sourced from the New York State COVID-19 Report Card.

The New York City School District, the largest school system in the United States, has over 1.1 million students taught in more than 1,800 separate schools and includes public and charter schools. Adding in the number of private schools (independent, religious, etc.) brings the total to over 2,600 with a total population of over 1.3 million students. Within this school district, over 7,050¹ cases have been reported since the beginning of the school year, an infection rate of 0.53%². This compares to the incremental number of cases in all of New York City during the same period of 1.56% ³

Underlying data sourced from the New York State COVID-19 Report Card.

While both NYS and NYC school figures compare favorably to the overall state statistics, the school population's current rate of change continues to rise, as the following chart illustrates. Note that the week ending November 27th included only 3 days of data as a result of the Thanksgiving holiday.

Underlying data sourced from the New York State COVID-19 Report Card.

Over the past seven days alone, New York State cases within the school system have increased by 5,981, implying an increase of 26%.

Steps to solving the data problem

First off, it should be noted that I am not a programmer by trade. As you can see from my LinkedIn profile, I have spent most of my career working in the investment management industry as a fundamental equity analyst and portfolio manager. These roles require a significant amount of time to analyze data. And while the spreadsheet has been the primary tool used by the investment community for well over 30 years, the role of Data Science and tools like Python and R are becoming a more critical part of the process.

Creating the dashboard

The underlying data for each of the 6,300+ schools in New York State can be found on the state’s website (i.e., the Report Card). To analyze the aggregate data, I first needed to (legally) “scrape” or extract the available data from the site for all the individual public, charter, and public schools in the state using Python. The data was then cleaned, processed, matched up with geo-coordinates, organized, and aggregated daily to explore the different regions’ trends. As the last step, the data was exported into a curated dashboard I created (using R Studio’s Shiny), so the public can now easily access and visualize the relevant information.

The underlying data from New York State is usually published each school day morning with data from the end of the prior school day, and my own dashboard is updated shortly after.

A more detailed description of the process and the underlying Python code can be found on my GitHub repository.