Spark MultiLayer Perceptron Classifier for POI Classification | by Pranav Thaenraj | Towards Data Science

Note: This article is the third in a series of articles regarding the classification of transportation POI data. The first article looked to use various Machine Learning models to classify records as Airports, Bus stops, and Train stations. The second article was centered around the use of feature reduction algorithms as means of tuning the models from the first article to provide better accuracy. Check out these articles to see how this project has evolved in the search for the best way to classify POI data.
This article will use the Spark MLlib package- specifically the MultiLayer Perceptron Classifier properly classifying the POI records from SafeGraph using foot traffic patterns. SafeGraph is a data provider that provides POI data for hundreds of businesses and categories. It provides data for free to academics. For this project, I have chosen to use SafeGraph patterns data to classify records as various POI’s. The schema for the patterns data can be found here: Schema Info
Apache Spark is an open-source framework developed as means to handle Big Data. The framework's call to fame is its ability to massively increase performance and reduce runtime through the use of parallel (or distributed) computing.
Why Spark MLlib?
Supports many of the popular Machine Learning Algorithms supported by packages such as Statsmodels and Scikit-LearnAllows for the construction of complex Machine Learning pipelinesAllows for the saving and reuse of algorithms and ML pipelinesSupports distributed computing techniques that vastly reduce the run-time and improve performance of all algorithms by splitting tasks into chunks and assigning them to multiple nodes to perform in parallel
Supports many of the popular Machine Learning Algorithms supported by packages such as Statsmodels and Scikit-Learn
Allows for the construction of complex Machine Learning pipelines
Allows for the saving and reuse of algorithms and ML pipelines
Supports distributed computing techniques that vastly reduce the run-time and improve performance of all algorithms by splitting tasks into chunks and assigning them to multiple nodes to perform in parallel
Spark MLlib is truly a very interesting and straightforward package to have an understanding of and can provide some very interesting insights in a very short amount of time, providing the best of both worlds in terms of runtime and performance.
Spark Deep Learning
The MLlib package has very few accommodations for Deep Learning. As of Spark 3.0, the package does not support many Deep Learning models and is more focused on the notions of Regression, Classification, and Clustering. The one exception to this shortcoming is the Multilayer Perceptron Classifier.
Why the MultiLayer Perceptron Classifier?
Each Classification model in the MLlib package has its advantages and disadvantages and different data may call for the use of a different model for maximum efficiency. The MLPC is superior to the other supervised classification algorithms that are available in the Spark MLlib package because it has the capability of finding correlations between features on its own. furthermore, the classifier works well for multi-class classifications and requires not nearly as much preprocessing and pipeline creation as traditional algorithms such as the SVM algorithm do in the Spark interface. Lastly, with a large enough and diverse enough dataset, the MLPC algorithm can perform better than most classifiers.
Given this intro let’s see if the Spark MLPC classifier can perform better than our classifiers from part 1 of this series
Before we can start with the Deep Learning aspects we must first load the Data. This particular step has been covered in both the first article of the series as well as the second one. The basic steps behind the data loading and preprocessing to fit our needs for this article are:
Before we take our first steps into the notions of feature reduction, we must first load the data that we will use for this project: This process of loading the data can be found in the notebook and has been explained in detail in the first part of the series. For our purposes all that is required is a brief outline of the steps taken and the resulting dataframe:
Dropping unnecesscay columns- [‘parent_safegraph_place_id’,’placekey’,’safegraph_place_id’,’parent_placekey’,’parent_placekey’,’safegraph_brand_ids’,’brands’, ‘poi_cbg’]Creating ground truth column that establishes each record as either Airport, Bus station, Airport, or UnkownDropping Unknown records to clear out records that cannot be identifiedHorizontally exploding columns of JSON strings using pysparkHorizontally exploding columns of arraysUsing Sklearn LabelEncoder package to transform class column
Dropping unnecesscay columns- [‘parent_safegraph_place_id’,’placekey’,’safegraph_place_id’,’parent_placekey’,’parent_placekey’,’safegraph_brand_ids’,’brands’, ‘poi_cbg’]
Creating ground truth column that establishes each record as either Airport, Bus station, Airport, or Unkown
Dropping Unknown records to clear out records that cannot be identified
Horizontally exploding columns of JSON strings using pyspark
Horizontally exploding columns of arrays
Using Sklearn LabelEncoder package to transform class column
As a result of these transformations the outputted data looks like this and has the following columns:
Raw_visit_counts: Number of visits in our panel to this POI during the date range.
Raw_visitor_counts: Number of unique visitors from our panel to this POI during the date range.
Distance_from_home: Median distance from home traveled by visitors (of visitors whose home we have identified) in meters.
Median_dwell: Median minimum dwell time in minutes.
Bucketed Dwells (Exploded to <5, 5–10,11–20,21–60,61–120,121–240): Key is the range of minutes and value is the number of visits that were within that duration
Popularity_by_day(Exploded to Monday-Sunday): A mapping of the day of the week to the number of visits on each day (local time) in the course of the date range
Popularity_by_hour(Exploded to popularity_1-popularity_24): A mapping of the hour of the day to the number of visits in each hour over the course of the date range in local time. The first element in the array corresponds to the hour of midnight to 1 am
Device_type(Exploded to IOS and Android): The number of visitors to the POI that is using android vs. ios. Only device_type with at least 2 devices are shown and any category with less than 5 devices are reported as
Now that the data is ready to go, we can begin on the Deep Learning aspects.
The Spark MLlib package performs ML functions in a manner that is slightly different from the traditional code snippets written with packages such as Sci-kit Learn. The package requires all of the features used for prediction to be compiled into one singular column of vectors using the Vector Assembler transformer. This requires the Label column to be separated from the feature columns beforehand, the code for this step of preprocessing looks like this:
Step 1: separating of label column from features column and conversion of Pandas DF to Spark DF
from pyspark.ml.classification import MultilayerPerceptronClassifierfrom pyspark.ml.evaluation import MulticlassClassificationEvaluatorlabel = transportation_df[‘Class’]transportation_df = spark.createDataFrame(transportation_df)transportation_df.show(5)
Step 2: Using Vector Assembler to create features column
from pyspark.ml.feature import VectorAssemblertransportation_df = transportation_df.drop(‘Class’)assembler = VectorAssembler().setInputCols(transportation_df.columns[1:]).setOutputCol(“features”)transportation_df = assembler.transform(transportation_df)transportation_df.show(10)
The end result of this should have all of the features compiled into one column as such:
From here the next step is to combine this data back with the label column. Since the other columns are no longer necessary (They have been compiled into the features column), all we need to run our model is the features and the label columns
transportation_df = transportation_df.toPandas()features = transportation_df[‘features’]features_df = pd.DataFrame(data = {‘features’: features})features_df = features_df.loc[~features_df.index.duplicated(), :]label_df = pd.DataFrame(data = {‘label’: label})label_df = label_df.loc[~label_df.index.duplicated(), :]transportation_df = pd.concat([features_df, label_df], axis= 1).dropna()transportation_df.head()
Next step would be to split the data into training and testing data:
transportation_df = spark.createDataFrame(transportation_df)trainSet, testSet = transportation_df.randomSplit([0.8, 0.2])
Now it’s time to create the Classifier model. The model has many hyperparameters that we need to know and tune in order to maximize model performance:
Tol: This parameter refers to the convergence tolerance or the value at which the weights begin to converge towards the predicted answer. The default value for this parameter is .000001. smaller values for the parameter may lead to overfitting while larger values may lead to more accurate results
Seed: The random seed for the randomly generated values of the model
Blocksize: This parameter refers to the number of records that will be included per iteration of the model. The default value for this parameter is 128. Higher block size will lead to overfitting while a lower one will provide more accurate results at the cost of runtime
Stepsize: This parameter refers to the model’s learning rate. the default value for this parameter is .03. A smaller step size will lead to a more accurate model at the cost of a larger run time and a larger one will lead to overfitting
Layers: Perhaps the most important hyperparameter of all, this parameter refers to the number of layers and nodes per layer that will be present in this model. The first layer must always be the number of features present in the data and the last layer must always be the number of outputted labels available. There must be at least one hidden layer in this array of layers.
With these requirements in mind let us look at the code for the MLPC model:
layers = [44,50,50,3]mlpc = MultilayerPerceptronClassifier(layers = layers, solver=’gd’, tol=.0000001, stepSize=.00001, blockSize= 30).setLabelCol(“label”).setFeaturesCol(“features”).setSeed(20).setMaxIter(500)model = mlpc.fit(trainSet)
Here we have made a Neural Net with two hidden layers each with 50 nodes. The model is trained using Gradient Descent and has a very high convergence tolerance and step size. Let us see the results:
result = model.transform(testSet)result.show(10)
from pyspark.ml.evaluation import MulticlassClassificationEvaluatorevaluator = MulticlassClassificationEvaluator(labelCol = ‘label’, predictionCol = ‘prediction’, metricName = ‘accuracy’)mlpacc = evaluator.evaluate(result)mlpacc
The overall accuracy is... unsatisfying, but this is the first of many experiments that we must run with this data in order to find the most effective number of layers and nodes to provide better results. Below is a table of tried layers and accuracies. If you would like to try this code, don’t hesitate to check out the notebook and run it yourself.
Immediate takeaways from this experiment:
More hidden layers do not necessarily mean better accuracyDeeper hidden layers do not necessarily mean better accuracy
More hidden layers do not necessarily mean better accuracy
Deeper hidden layers do not necessarily mean better accuracy
We can see from the table above that the model performs best when it remains simple rather than complex. Even so, we see that the accuracy for this particular dataset is barely exceeding that of the Gaussian Naive Bayes model from the first part of the series. Why could this be the case? Well, the simple answer is the imbalance in the data. When extracting this data from SafeGraph, three NAICS codes were utilized to retrieve data, and the data that was outputted was severely imbalanced, with the number of airport records being almost 4 times that of the number of bus stop records. This imbalance has shown itself to be an issue throughout this series and detrimentally affected the results of many models we trained on the data. To rectify the imbalance, random sampling of the different classes in the data was applied but doing so led to the data for the Airport and Train stations being underrepresented and thus still hurting the accuracy of the model.
Regardless of the results, This article provides an introduction to the notions of Spark Deep Learning through the MLPC model. The article shows the ease that Spark brings to creating Deep Learning models and furthermore shows the benefits and shortcomings of using such a complex model on a real-world dataset such as SafeGraph Patterns data.
In the next and final article of this series, we will look into the use of ensemble classification on this Patterns data and see in the use of multiple classifiers layered on top of one another helps to provide the best results.
Questions?
I invite you to ask them in the #safegraphdata channel of the SafeGraph Community, a free Slack community for data enthusiasts. Receive support, share your work, or connect with others in the GIS community. Through the SafeGraph Community, academics have free access to data on over 7 million businesses in the USA, UK, and Canada.