Prediction Intervals in Linear Regression | by Nathan Maton | Towards Data Science

This post covers how to calculate prediction intervals for Linear Regression. Normally when modeling, we get a single value from a regression model. But what if that value is used to plan or make important decisions?
Then a single value may overstate our confidence when we’d like to know our uncertainty or error margin. Sure, you can look at a general error score for all of your predictions like RMSE, but what about for a single prediction? Prediction intervals give you a range for the prediction that accounts for any threshold of modeling error that matters to you. Similar to confidence intervals you can pick a threshold like 95%, where you want the actual value to fall into a range 95% of the time. This is extremely nice when planning, as you can use the upper and lower bounds in your estimation process.
I’ve created a small method (with some input from here) to predict a range for a certain confidence threshold that matters to you or your project. I’ll illustrate a prediction interval with the Boston Housing dataset, predicting the median value of homes in different regions. Here’s the whole notebook if you prefer to read the code on GitHub.
Otherwise, here’s a description of the dataset:
from sklearn import datasetsprint(datasets.load_boston()['DESCR'])Boston house prices dataset---------------------------**Data Set Characteristics:**      :Number of Instances: 506     :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.    :Attribute Information (in order):        - CRIM     per capita crime rate by town        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.        - INDUS    proportion of non-retail business acres per town        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)        - NOX      nitric oxides concentration (parts per 10 million)        - RM       average number of rooms per dwelling        - AGE      proportion of owner-occupied units built prior to 1940        - DIS      weighted distances to five Boston employment centres        - RAD      index of accessibility to radial highways        - TAX      full-value property-tax rate per $10,000        - PTRATIO  pupil-teacher ratio by town        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town        - LSTAT    % lower status of the population        - MEDV     Median value of owner-occupied homes in $1000's
We’ll preprocess the data, model it using the Linear Regression package from sklearn.
#preprocess datadf = pd.DataFrame(datasets.load_boston()['data'])df.columns = datasets.load_boston()['feature_names']df['target'] = datasets.load_boston()['target'] #split dataX_train, X_test, y_train, y_test = train_test_split(    df.drop('target',axis=1), df.target, test_size=0.2, random_state=42)#do modelingmodel = LinearRegression()fit_model = model.fit(X_train, y_train)predictions = fit_model.predict(X_test)
What if you want to understand the model error on a single prediction level? Here’s where I built a simple method to get this interval.
def get_prediction_interval(prediction, y_test, test_predictions, pi=.95):    '''    Get a prediction interval for a linear regression.        INPUTS:         - Single prediction,         - y_test        - All test set predictions,        - Prediction interval threshold (default = .95)     OUTPUT:         - Prediction interval for single prediction    '''        #get standard deviation of y_test    sum_errs = np.sum((y_test - test_predictions)**2)    stdev = np.sqrt(1 / (len(y_test) - 2) * sum_errs)#get interval from standard deviation    one_minus_pi = 1 - pi    ppf_lookup = 1 - (one_minus_pi / 2)    z_score = stats.norm.ppf(ppf_lookup)    interval = z_score * stdev#generate prediction interval lower and upper bound    lower, upper = prediction - interval, prediction + interval    return lower, prediction, upperget_prediction_interval(predictions[0], y_test, predictions)OUTPUT: (19.24072024369257, 28.996723619824934, 38.752726995957296)
So what should you take away from this post? There are tools you can use to calculate uncertainty called a prediction interval and for Linear Regression you can use the code above in your project. This has helped me calculate uncertainty for very critical business processes and is a useful technique in your tool belt.
As you continue to progress in your data science career you’ll want more ways to quantify risk and uncertainty as that will help inform key decisions. Hope you enjoyed this short tutorial!