Package: pypeds
pypedsThis is a python package aiming to analyze IPEDS and other education datasets using the data science tools available within python.  Dare I say, enrollment science tools?WARNING:  This is a very young project with exploration around the API and the toolkits included.Install and Test NotesInstallation ofpypedsis as simple aspip install pypeds.It is highly recommended that you leverage the use of environments when coding in python; at least that is my opinion anyway.  I prefer to use conda for my environment management.  Assuming that you have conda setup properly, this could be as simple as:conda create -n pypeds python=3.6
conda activate pypeds
pip install pypedsAlternatively, if you want to use the development version of this package, you can reviewthis Google Colab Notebookwhich provides an installation overview to install from this repository.  You can replicate this process anytime you want code/analyze data using Google's fantastic notebook coding environment.Last but not least, it's worth stating that you could just as easily installpypedsvia collab with:!pip install pypedsBasic UsageTheyearsargument is based on the survey year shown on the IPEDS website.  When the datasets are built, this package adds both the survey year and the fall year (of the academic year) to the datasets.  For example, the survey SFA (Student Financial Aid) for the survey year2017is reporting on data for the 1617 academic year, so I includefall_yearwith a value of 2016 in this case.Below is a simple session to get the last 2 years of the directory information surveys:from pypeds import ipeds

# generate a list of survey years -- this must be a python list
# YEARS = list(range(2014, 2018, 1))  ## another way to create a range of years
YEARS = [2019,2020]

# instantiate the survey of interest
hd = ipeds.HD(years=YEARS)

#  extract, or download the surveys
hd.extract()

# load the surveys as an explicit pandas dataframe in your session
df = hd.load()

# confirm that we have the data we expected
df.survey_year.value_counts()

2019.0    6559
2020.0    6440This package aims to be a go-to resource for those of us who analyze data in higher education, and perhaps more specifically, enrollment management.  As such, I have chosen an API scheme that hopefully will make your transition to python easier, especially as you transition into machine learning with scikit-learn.  The use of classes and methods is heavily inspired by that toolkit.Moreover, this package attempts to remove the friction of data prep as much as possible.  For that reason, and also borrowing from the tidyverse use of verbs for data munging, the API is built around the idea of ETL (Extract, Transform, Load) as well as common Exploratory Data Analysis (EDA) concepts.  Below is an outline of the toolsInstantiate the survey of interest withipeds.IC()oripeds.HD().  The years argument defaults to the currentsurvey(2017) year but can setup with alistof years.  For example,ipeds.HD(years=[2016,2017])as well.Next, we.extractthe data, which collects 1 or more years of full survey datasets from IPEDS and keeps the data within our survey object.Optionally, you can.transformthe data.  This will supply functionality to modify the survey data by filtering rows, selecting columns, or deriving new columns usingcommonly accepted logicin order to establish standard definitions.  Again, the aim is to make things easier.Lastly,.loadpulls out the survey datasets as a pandas dataframe, at which point you can analyze, visualize the survey dataset using the full suite of data analytics tools found within python.This package is under heavy development and as noted at the top, is subject to breaking changes within the API.  However, beyond the ETL verbs, this package will also include various methods for exploration, competitive benchmarking, and data visualization.  While all of this work can be done by the analyst after the.loadmethod, the aim is facilitate learning and insight by extracting away the "how" for basic and common questions in the enrollment management space.Surveys currently supported:HD: Directory Info [HD]IC: Institutional Characteristics, which also merges on the ADM survey which started in 2014 [IC]SFA: Student Financial Aid [SFA]EF_C: Residence and Migration of First-Time Freshmen [EFC]EF_D: Total entering class, retention rates, and student-to-faculty ratio [EFD]IC_AY: Charges for Academic Year Programs [ICAY]C_A: Awards/degrees conferred by program (6-digit CIP code), award level, race/ethnicity, and gender [C_A]OM: Award and enrollment data at four, six and eight years of entering degree/certificate-seeking undergraduate cohorts [OM]FF1: Finance, Public institutions - GASB [FF1]FF2: Private not-for-profit institutions or Public institutions using FASB [FF2]The class names are in the brackets.  For example, theEF_Csurvey can be instantiated usingipeds.EFC().Additional datasetsThis package will also periodically add datasets that one might use when studying and analyzing data in higher education and enrollment management.Most notably, once importing thedatasetsmodule,pypedsattempts to provide simple access to core education-related datasets.  For example:wichefor data, and projections, of high school graduates in the US by state.In addition to above, there are other datasets to help with mapping code and geographies for easier reporting and visualization.For example, let's play around with the WICHE projections below.WICHE High School ProjectionsFor example, you might want to look at theWICHE projections for High School Graduates.  The data are currently included in this package via the datasets module.# import the datasets module
from pypeds import datasets

# extract the wiche dataset
wiche = datasets.wiche()   

# first few rows
wiche.head(1).T

                                                    0
stabbr                                             _M
schoolyear                                    2000-01
raceethnicity           American Indian/Alaska Native
gradelevel                                    Grade 1
schoolsector                           Public Schools
projection                            Reported/Actual
students                                         5106
statename                                     Midwest
region_wiche                                  Midwest
region_census_division                            NaN
region_census                                     NaNYou will notice that the data are not wide, but long.  This is by design, and it allows us to reshape and aggregate as desired.For example, plot theactualandprojected(the projection column) total high school graduates (gradelevel and raceethnicity columns) in the dataset.# isolate total grads, but still for each state and year
all_grads = wiche.loc[(wiche.gradelevel=='Graduates') & (wiche.raceethnicity=='Total') & (wiche.statename == 'United States') & (wiche.schoolsector == 'Grand Total Public & Private'), : ]

# you will notice that we have to filter the dataset to get total US high school grads, but it highlights the resolution of the data!

# generate the line plot
all_grads.plot(x="schoolyear", y="students", kind="line")Future workinclude tooling to help analysts connect to Salesforce or Data Warehouses/Databasessubmit via pypedsadditional higher education datasetsPull in Wikipedia links and pageview trendsNotesIt appears that the mission statement data is not part of the full surveys,butis included in the MS Access version.  Obviously it's less than ideal that the data collected is not included within the full file option, which this package highly leverages.Resources:IPEDS survey info