Package: scikit-bonus
scikit-bonusThis is a package withscikit-learn- compatible transformers,
classifiers and regressors that I find useful.Give it a try with a simplepip install scikit-bonus! You can find the documentation onhttps://garve.github.io/scikit-bonus.Special linear regressorsThis module contains linear regressors for special use-cases, for exampletheLADRegressionwhich optimizes for theabsolutemean deviation (error) instead of the meansquarederror andtheImbalancedLinearRegressionthat lets you punish over and under estimations of the model differently.Since a picture says more than a thousand words:In the left picture, you can see how the LAD regression line is not affected by outliers.In the right picture, an imbalanced linear regression was trained to punish overestimations by the model 5 times as high as under estimations.
That's why it makes sense that the imbalanced regression line is lower: the model is more afraid of overestimate the true labels.The time moduleThe time module contains transformers for dealing with time series in
a simple-to-understand way.
Most of them assume the input to be apandasdataframe with aDatetimeIndexwith a fixed frequency, for example created viapd.date_range(start="2020-12-12", periods=5, freq="d").Let us see two examples of what this module provides for you.SimpleTimeFeaturesVery often, you don't need to use complicated models such as anRNNorARIMA.
It is often sufficient to just extract some simple date features from the
DatetimeIndex and run a simple, even linear regression. These features can
be the hour, the day of the week, the week of the year, or the month, among others.Let us look at an example:importpandasaspdfromskbonus.pandas.timeimportSimpleTimeFeaturesdata=pd.DataFrame({"X_1":range(10)},index=pd.date_range(start="2020-01-01",periods=10,freq="d"))tfa=SimpleTimeFeatures(day_of_month=True,month=True,year=True)tfa.fit(data)print(tfa.transform(data))------------------------------------------OUTPUT:X_1day_of_monthmonthyear2020-01-0101120202020-01-0212120202020-01-0323120202020-01-0434120202020-01-0545120202020-01-0656120202020-01-0767120202020-01-0878120202020-01-0989120202020-01-1091012020The SimpleTimeFeatures has many more built-in features. The full list
issecondminutehourday_of_weekday_of_monthday_of_yearweek_of_monthweek_of_yearmonthyearYou can work with these features directly, but
of course, you can utilize scikit-bonus to further process these features,
such as with aOneHotEncoderWithNames, a thin wrapper around
scikit-learn'sOneHotEncoder.
Another way to post-process these features is using theCyclicalEncoderof
scikit-bonus.CyclicalEncoderIn short, this encoder takes a single cyclic feature such
as most time features (except foryear) and converts it
into atwo-dimensionalfeature with the property that close points
in time are close in the encoded space.Consider hours for example: A model working with hours in the form
0 - 23 cannot know that 0 and 23 are actually close together in time.
The CyclicalEncoder arranges the hours in the way you know it from a
normal, analog, round clock.In code:importpandasaspdfromskbonus.preprocessing.timeimportCyclicalEncoderdata=pd.DataFrame({"hour":range(24)})ce=CyclicalEncoder(cycles=[(0,23)])ce.fit(data)print(ce.transform(data.query("hour in [22, 23, 0, 1, 2]")))------------------------------------------OUTPUT:[[1.0.][0.965925830.25881905][0.86602540.5][0.8660254-0.5][0.96592583-0.25881905]]Note that the hours 0 and 23 are as close together as 0 and 1.Air PassengersLet us check out the famous air passenger example:fromsktime.datasetsimportload_airlinedata_raw=load_airline()print(data_raw)This yields the pandas seriesPeriod1949-01112.01949-02118.01949-03132.01949-04129.01949-05121.0...1960-08606.01960-09508.01960-10461.01960-11390.01960-12432.0Freq:M,Name:Numberofairlinepassengers,Length:144,dtype:float64While the index of this series looks fine, it's actually aPeriodIndexwhich scikit-bonus cannot deal with so far. Also, it's a series and not a dataframe.We can address both problems viadata=(data_raw.to_frame()# convert to dataframe.set_index(data_raw.index.to_timestamp())# overwrite index with a DatetimeIndex)Let us create the dataset in the scikit-learn-typicalX,yform and split
it into a train and test set. Note that we do not do a random split
since we deal with a time series.If there is a date from the test set between two dates
that are in the training set, it might leak information
about the label of the test set.So, let us use the first 100 dates as the training set, and the remaining 44 are the
test set.X=data.drop(columns=["Number of airline passengers"])y=data["Number of airline passengers"]X_train=X.iloc[:100]X_test=X.iloc[100:]y_train=y[:100]y_test=y[100:]Before we continue, let us take a look at the time series.We can see that it hasan increasing amplitudea seasonal patternan upward trendWe want to cover these properties with the following approach:apply the logarithm on the dataextract the month from the DatetimeIndex using scikit-bonus' SimpleTimeFeaturesAdd a linear trend using scikit-bonus' PowerTrendPutting everything together in a pipeline looks like this:fromskbonus.pandas.timeimportSimpleTimeFeatures,PowerTrendfromskbonus.pandas.preprocessingimportOneHotEncoderWithNamesfromsklearn.linear_modelimportLinearRegressionfromsklearn.composeimportTransformedTargetRegressor,make_column_transformerfromsklearn.pipelineimportmake_pipelinect=make_column_transformer((OneHotEncoderWithNames(),["month"]),remainder="passthrough")pre=make_pipeline(SimpleTimeFeatures(month=True),PowerTrend(power=0.92727),# via hyper parameter optimizationct,TransformedTargetRegressor(regressor=LinearRegression(),func=np.log,# log the datainverse_func=np.exp# transform it back))pre.fit(X_train,y_train)pre.score(X_test,y_test)---------------------OUTPUT:0.6843610111874758Quite decent for such a simple model! Our predictions look like this: