An introduction to quantile regression | by Peter Flom | Towards Data Science

Ordinary least square regression is one of the most widely used statistical methods. However, it is a parametric model and relies on assumptions that are often not met. Quantile regression makes no assumptions about the distribution of the residuals. It also lets you explore different aspects of the relationship between the dependent variable and the independent variables.
There are at least two motivations for quantile regression: Suppose our dependent variable is bimodal or multimodal that is, it has multiple humps. If we knew what caused the multimodality, we could separate on that variable and do stratified analysis, but if we don’t know that, quantile regression might be good. OLS regression will, here, be as misleading as relying on the mean as a measure of centrality for a bimodal distribution.
If our DV is highly skewed as, for example, income is in many countries we might be interested in what predicts the median (which is the 50th percentile) or some other quantile; just as we usually report median income rather than mean income.
One more example is where our substantive interest is in people at the highest or lowest quantiles. For example, if studying the spread of sexually transmitted diseases, we might record number of sexual partners that a person had in a given time period. And we might be most interested in what predicts people with a great many partners, since they will be key parts of spreading the disease.
For those interested in business, it is likely that the amount of money spent in a store or website is skewed. You, as the store’s owner, may be more interested in figuring out what predicts the top quantiles rather than the mean.
Predicting low birth weight is important because babies born at low weight are much more likely to have health complications than babies of more typical weight. The usual approaches to this are either to model the mean birth weight as a function of various factors using OLS regression, or to dichotomize or otherwise categorize birth weight and then use some form of logistic regression (either ‘normal’ or ordinal). Both these are inadequate.
Modeling the mean is inadequate because, as we shall see, different factors are important in modeling the mean and the lower quantiles. We are often interested in predicting which mothers are likely to have the lowest weight babies, not the average birth weight of a particular group of mothers.
Categorizing the dependent variable is rarely a good idea, principally because it throws away useful information and treats people within categories as the same. A typical cutoff value for low birth weight is 2.5 kg. Yet this implies that a baby born at 2.49 kg is the same as a baby born at 1.0 kg, while one born at 2.51 kg is the same as one who is 4 kg. This is clearly not the case.
One model of birth weight (provided by SAS and adapted from Koenker) includes the child’s sex, the mother’s marital status, mother’s race, the mother’s age (as a quadratic), her educational status, whether she had prenatal care, and, if so, in which trimester, whether she smokes, and, if so, how many cigarettes a day, and her weight gain (as a quadratic).
Mother’s marital status was coded as married vs. not married; race was either Black or White (it is not clear if mothers of other races were simply excluded), mother’s education was coded as either less than high school (the reference category), high school graduate, some college, or college graduate. Prenatal care was coded as none, first trimester (the reference category), secondtrimester or third trimester. Mother’s weight gain and age were centered on the means.
We can now look at the effects of each of these variables at each quantile. The best way to do this is probably graphically. The output from the program is 9 times as long as for a regular regression (since we have 9 quantiles) and is laborious to read. However, it can be printed out. The SAS code shown below generates four panels of graphics. The first panel:
shows the effect of the intercept, the mother being Black, the mother being married and the child being a boy. The intercept is the mean birth weight for each quantile for a baby girl born to a unmarried White woman who has less than high school education, does not smoke, is the average age and gains the average amount of weight. Just about 5% of these babies weigh less than the usual cut-off weight of 2,500 grams. Babies born to Black women are lighter than those born to White women, and this effect is greater at the low end than elsewhere — the difference is about 280 grams at the 5%tile, 180 grams atthe median, and 160 grams at the 95%tile. Babies whose mothers were married weigh more than those whose mothers were not, and the effect is relatively constant across quantiles. Boys weigh more than girls, and this effect is larger at the high end: At the 5%tile boys weigh about 50 grams more than girls, but at the 95%tile the difference is over 100 grams.
The next panel:
shows the effects of prenatal care, and the first part of education, and can be read in a similar way as the first panel.
The third panel:
shows the other education effects and the effects of smoking. The 4th and final panel:
shows the effects of maternal age and weight gain. These last two are somewhat harder to interpret, as is always the case with quadratic effects compared to linear effects. One way to ameliorate this confusion is to plotthe predicted birth weight of babies for different maternal ages or weight gain, holding other variables constant at their means or most common values. SAS code to do this is below. The result for maternal weight gain:
This is a fascinating graph! Note that the extreme quantiles are the ones where the quadratic effect is prominent. Further note that mothers who either lose weight or gain a great deal of weight have much higher chances of having low birth weight babies than women who gain a moderate amount. In addition, women who gain a great deal have higher chances of having extremely large babies. This sort of finding confirms medical opinion, but is not something we could find with ordinary least squares regression.
We can do the same thing for maternal age but the effect of age is not that huge, and the quadratic effect is so small that we might consider simplifying the model by eliminating it. On the other hand, if the literature says that there should be strong quadratic effects of maternal age, then either there is something odd about this data set or we have evidence counter to thatclaim. One thing to note is that this data set spans a limited range of ages — all mothers were 18 to 45 years old. There might be strong effects that occur at younger and older ages.
proc quantreg ci=sparsity/iid algorithm=interior(tolerance=1.e-4)data=new;class visit MomEdLevel;model weight = black married boy visit MomEdLevel MomSmokecigsperday MomAge MomAge*MomAgeMomWtGain MomWtGain*MomWtGain/quantile= 0.05 to 0.95 by 0.05plot=quantplot;run;
Get predicted values:
proc quantreg ci=sparsity/iid algorithm=interior(tolerance=1.e-4)data=new;class visit MomEdLevel;model weight = black married boy visit MomEdLevel MomSmokecigsperday MomAge MomAge*MomAgeMomWtGain MomWtGain*MomWtGain/quantile= 0.05 to 0.95 by 0.05;output out = predictquant p = predquant;run;
then we subset this to get only the cases where the other values are their means or modes. First, for maternal age:
data mwtgaingraph;set predictquant;where black = 0 and married = 1 and boy = 1 and MomAge = 0 and MomSmoke = 0 and visit = 3 and MomEdLevel = 3;run;
Then sort it:
proc sort data = mwtgaingraph;by MomWtGain;run;
Then graph it.
proc sgplot data = mwtgaingraph;title ’Quantile fit plot for maternal weight gain’;yaxis label = “Predicted birth weight”;series x = MomWtGain y = predquant1 /curvelabel = “5 %tile”;series x = MomWtGain y = predquant2/curvelabel = “10 %tile”;series x = MomWtGain y = predquant5/curvelabel = “25 %tile”;series x = MomWtGain y = predquant10/curvelabel = “50 %tile”;series x = MomWtGain y = predquant15/curvelabel = “75 %tile”;series x = MomWtGain y = predquant18/curvelabel = “90 %tile”;series x = MomWtGain y = predquant19/curvelabel = “95 %tile”;run;
I use SAS by choice. However, R offers the quantreg package, Python has quantile regression in the statsmodels package and STATA has qreg. Other software may also offer it.
Quantile regression is a valuable tool for cases where the assumptions of OLS regression are not met and for cases where interest is in the quantiles.