Using FunctionTransformer and Pipeline in SkLearn to Predict Chardonnay Ratings | by Eric Kleppen | Towards Data Science

One of the best things about learning to code is the endless number of rabbit holes to fall into as you discover new concepts. An excercise I worked through on DataCamp exposed me to Pipeline in Scikit-Learn, and it blew my mind! Pipelines allow you to sequentially transform your data and implement fit(). As soon as I was done with the excercise, I started applying what I had learned and crafted a pipeline to clean the wine review dataset and predict the rating of Chardonnay. This is an in-depth review of the process I went through while creating my pipeline and predictive model. I discuss creating features, applying category encoders, constructing a pipeline and generating predictions. The complete code and a link to the github repo can be found at the bottom of the article:
The data originates from the Wine Review dataset available on Kaggle. I’ve written several articles covering exploring the dataset and have already stored it in a SQLite database. The data is only partially cleaned, and it needs to be transformed so it can be used in the machine learning model.
import numpy as npimport pandas as pdimport sqlite3import category_encoders as ceimport refrom sklearn.feature_selection import chi2, SelectKBestfrom sklearn.model_selection import train_test_splitfrom sklearn.feature_extraction.text import HashingVectorizerfrom sklearn.pipeline import Pipeline, FeatureUnionfrom sklearn.impute import SimpleImputerfrom sklearn import ensemblefrom sklearn.preprocessing import MaxAbsScalerfrom sklearn.preprocessing import FunctionTransformerconn = sqlite3.connect('db/wine_data.sqlite')c = conn.cursor#create Dataframedf = pd.read_sql("select country  \                 ,description     \                 ,rating          \                 ,price           \                 ,province        \                 ,title           \                 ,winery from wine_data where variety = 'Chardonnay'", conn)df.head(2)
Notice how many columns contain text data.
In order to use them in the machine learning model, text will have to be transformed using encoding. There are several strategies for encoding the categorical data like winery, country, province, and title, and I’ll cover them in the Category Encoders section. Before I encode them, I need to create a few features that might help the predictive model.
Feature engineering can be done to extract potentially predictive qualities within the data, and add it to the dataset as a column. I have created a set of funtions that take in a dataframe and output the transformed dataframe. Using Scikit Learn’s Function Transformer, I can use the functions in the pipeline to transform the dataframe. I tried to make the functions dynamic using global variables.
One of the features I want to add is the wine’s year found within the title column. The function extract_year takes in a dataframe and returns a dataframe with a year column added.
def extract_year(dataframe):#set the column name containing the year using a global variable    global year_column    years = dataframe[year_column]    #years.reset_index(inplace=False)    #years.fillna("", inplace=True)    l = []    i = 0 #use for loop to extract the year from each title row    for year in range(len(dataframe)):        temp = re.findall(r'\d+', years[i])         res = list(map(int, temp))         try:             if len(str(res[0])) == 4:                l.append(res[0])            elif len(str(res[0])) != 4:                l.append(0)        except:            l.append(0)        #print(res[0])        i+=1    dataframe['year'] = lreturn dataframe
While exploring the dataset, I noticed wines with shorter reviews tended to have lower ratings. Because of that, I’ll add the word count of the description to the model to see if it is a predictor. I use a function that relies on a global variable named word_count_column to indicate which column of the data frame to use. It takes in a data frame and returns a data frame with a word_count column added.
def word_count(dataframe):    global word_count_column    dataframe['word_count'] = dataframe[text].apply(lambda word: len(str(word).split(" ")))    return dataframe
We can see the two functions work and produce the desired data frame.
After adding the two numeric features, it is time to consider encoding the categorical data in columns country, province, title, and winery.
Since the machine learning model takes numbers as input, the text must be transformed. The strategy used to encode the data can have a big impact on the accuracy of the model. To easily try a variety of coding strategies, I recommend using the Category Encoder package that works with Scikit Learn.
The category encoders available in the package are compatible with Pipeline since they are transformers.
pip install category_encodersORconda install -c conda-forge category_encoders
There are 15 or so encoders included in the package. I recommend trying these with the model pipeline to see how the different strategies impact the accuracy of the model.
James-Stein Encoder
Leave One Out
M-estimate
Ordinal
Target Encoder
Although they are compatible with Pipeline, I created a function so I could pass in additional logic. I pass in a data frame and two global variables to control which category and target columns are used in the transformation.
# encoder = ce.JamesSteinEncoder(cols=[...]) --maybe # encoder = ce.LeaveOneOutEncoder(cols=[...]) --maybe# encoder = ce.MEstimateEncoder(cols=[...]) --maybe# encoder = ce.OrdinalEncoder(cols=[...]) --maybe# encoder = ce.TargetEncoder(cols=[...]) --maybedef category_encode(dataframe):    global category_columns    global category_target    x = dataframe[category_columns]    y = dataframe[target]    ce_ord = ce.OrdinalEncoder(cols=category_columns)    dataframe[category_columns] = ce_ord.fit_transform(x, y)    return dataframe
Notice the categorical text columns have been transformed into numeric columns.
The description column can be used in the model, but it needs to go through a different transformation method. Instead, I’m going to use a function that selects the numeric columns from the data fram and ignores the description column.
get_numeric_data = FunctionTransformer(lambda x: x[numeric], validate=False)
Pipelines are great because they enforce order during the transformation process making the workflow compact and easy to understand. This also can make the work easier to reproduce.
Remember that pipeline uses transformers, so we need to use the FunctionTransformer on our functions to make them compatible.
Using FunctionTransformer, it is easy to make the functions used in the feature engineering and column selection process compatible with the pipeline.
get_year = FunctionTransformer(extract_year, validate=False)get_word_count = FunctionTransformer(word_count, validate=False)get_encoded_text = FunctionTransformer(category_encode, validate=False)get_numeric_data = FunctionTransformer(lambda x: x[numeric], validate=False)
Because some of functions rely on an index value, I need to make a function that resets the index so the pipeline works properly after splitting the data into train and test sets.
def reset_index(dataframe):    dataframe = dataframe.reset_index(inplace = False)    return dataframeget_reset_index = FunctionTransformer(reset_index, validate=False)
The functions I created use global variables instead of hard-coded values so they are easier to reuse. Because of this, all of the global variables need to be set:
year_column = 'title'word_count_column = 'description'category_columns = ['country','province','title','winery']target = 'price'numeric= ['price', 'year', 'word_count', 'country', 'province', 'title', 'winery']
The pipeline can be used to select the model you want to use. To learn more about setting up a pipeline for model selection, I recommend checking out this Medium article by Rebecca Vickery. I am going to keep things simple and just use the Gradient Boost Regressor in Scikit Learn.
#create Gradient Boosting Regressor model model = ensemble.GradientBoostingRegressor(    n_estimators = 100, #how many decision trees to build    learning_rate = 0.5, #controls rate at which additional decision trees influes overall prediction    max_depth = 6,     min_samples_split = 21,    min_samples_leaf = 19,     max_features = 0.9,    loss = 'huber')
Once the model has been selected, the global variables have been set, and all the functions have been transformed and are compatible with the pipeline, it is time to put the pieces together:
pl = Pipeline(memory=None,    steps=[        ('reset_index', get_reset_index),        ('year', get_year),        ('word_count', get_word_count),        ('encode', get_encoded_text),        ('selector', get_numeric_data),        ('model', model)    ], verbose=True)
Notice I set Verbose = True. Doing this prints the step has been complete. It makes the process more transparent for debugging.
To test the pipeline, use the Train Test Split to split the data and run it through the pipline:
features = df.drop(['rating'], axis=1)X = featuresy = df['rating']X_train, X_test, y_train, y_test = train_test_split(X, y                                                   , test_size = .3                                                   #, stratify=y                                                   )pl.fit(X_train, y_train)pl.score(X_test, y_test)
import numpy as npimport pandas as pdimport sqlite3import category_encoders as ceimport refrom sklearn.feature_selection import chi2, SelectKBestfrom sklearn.model_selection import train_test_splitfrom sklearn.feature_extraction.text import HashingVectorizerfrom sklearn.pipeline import Pipeline, FeatureUnionfrom sklearn.impute import SimpleImputerfrom sklearn.linear_model import LogisticRegressionfrom sklearn.preprocessing import MaxAbsScalerfrom sklearn.preprocessing import FunctionTransformerconn = sqlite3.connect('db/wine_data.sqlite')c = conn.cursor#create Dataframedf = pd.read_sql("select country  \                 ,description     \                 ,rating          \                 ,price           \                 ,province        \                 ,title           \                 ,winery from wine_data where variety = 'Chardonnay'", conn)#df.head(2)def extract_year(dataframe):    global year_column    years = dataframe[year_column]    #years.reset_index(inplace=False)    #years.fillna("", inplace=True)    l = []    i = 0     for year in range(len(dataframe)):        temp = re.findall(r'\d+', years[i])         res = list(map(int, temp))         try:             if len(str(res[0])) == 4:                l.append(res[0])            elif len(str(res[0])) != 4:                l.append(0)        except:            l.append(0)        #print(res[0])        i+=1    dataframe['year'] = lreturn dataframe#df = extract_year(df)def word_count(dataframe):    global word_count_column    dataframe['word_count'] = dataframe[word_count_column].apply(lambda word: len(str(word).split(" ")))    return dataframe# df = word_count(df)# df.head(3)# encoder = ce.JamesSteinEncoder(cols=[...]) --maybe (best score)# encoder = ce.LeaveOneOutEncoder(cols=[...]) --maybe# encoder = ce.MEstimateEncoder(cols=[...]) --maybe (good)# encoder = ce.OrdinalEncoder(cols=[...]) --maybe# encoder = ce.TargetEncoder(cols=[...]) --maybeyear_column = 'title'word_count_column = 'description'category_columns = ['country','province','title','winery']target = 'price'combine_text = ['country','province','title','winery', 'description']numeric= ['price', 'year', 'word_count','country','province','title','winery']def category_encode(dataframe):    global category_columns    global category_target    x = dataframe[category_columns]    y = dataframe[target]    ce_ord = ce.OrdinalEncoder(cols=category_columns)    dataframe[category_columns] = ce_ord.fit_transform(x, y)    return dataframe# df = category_encode(df)# df.head()get_year = FunctionTransformer(extract_year, validate=False)get_word_count = FunctionTransformer(word_count, validate=False)get_encoded_text = FunctionTransformer(category_encode, validate=False)get_numeric_data = FunctionTransformer(lambda x: x[numeric], validate=False)def reset_index(dataframe):    dataframe = dataframe.reset_index(inplace = False)    return dataframeget_reset_index = FunctionTransformer(reset_index, validate=False)from sklearn import ensemble model = ensemble.GradientBoostingRegressor(    n_estimators = 100, #how many decision trees to build    learning_rate = 0.5, #controls rate at which additional decision trees influes overall prediction    max_depth = 6,     min_samples_split = 21,    min_samples_leaf = 19,     max_features = 0.9,    loss = 'huber')pl = Pipeline(memory=None,    steps=[        ('reset_index', get_reset_index),        ('year', get_year),        ('word_count', get_word_count),        ('encode', get_encoded_text),        ('selector', get_numeric_data),        ('model', model)    ], verbose=False)features = df.drop(['rating'], axis=1)X = featuresy = df['rating']X_train, X_test, y_train, y_test = train_test_split(X, y                                                   , test_size = .3                                                   #, stratify=y                                                   )pl.fit(X_train, y_train)pl.score(X_test, y_test)
github.com
If you enjoyed this, follow me on Medium for more
Get FULL ACCESS and help support my content by subscribing
Let’s connect on LinkedIn
Analyze Data using Python? Check out my website
— Eric Kleppen