Exploring Human Memory Through Storytelling
Tags: Predictive Modeling, Towards Data Science, Analytics, Unstructured Data, Python

Learning & Modeling

The first question that was explored was whether the similarity score of a retold/imagined story could be predicted with its corresponding recalled story. Using similarity score as a proxy for memory accuracy, this process revealed vital features that are most predictive of similarity score. These features are important in determining which factors in the dataset affect memory the most.

The models considered include multiple linear regression, a lasso-regularized linear model, and a decision tree regressor. The linear regression was considered as a starting point to the analysis since it would not take a lot of time to implement and could help guide the analysis. The lasso and decision tree regressor were considered because of their built-in ability to perform feature selection, which further developed insights on what impacts a similarity score the most. The mean-squared errors were very small, so the linearity of the model was further analyzed, and it was discovered that the data was not very linear. This meant that predictions would not be very accurate.

Decision Tree Regressor’s Output

The second question that was explored was whether it is possible to mimic what the research paper included with the data set was trying to do and predict if a story is imagined or recalled based on this database. Most of the features provided weren’t helpful, so some new features were developed for this analysis. A total of seven features were used: the sentiment of the story with respect to the summary, the sentiment of the story with respect to the main event, total word count of stories, total word count of “cleaned” stories, the cosine similarity of the story with respect to the summary, the cosine similarity of the story with respect to the main event, and the average concreteness of the words.

The data frame including the new features

Sentiment analysis was based on the Summary and Main Event with respect to the story because the recalled story would have a larger absolute sentiment value than the imagined one. Concreteness evaluates the degree to which the concept denoted by a word refers to a recognizable entity, and it was introduced by Dr. Brysbaert and his team in 2014. The team provided a database with thirty-seven thousand English words and three thousand two-word expressions of concreteness with mean and standard deviation scores. For reference, there are 280 words or expressions ranking with the highest mean concreteness score of five in the database (e.g. sled and peacock), and the words with the lowest scores are “eh” and “essentialness” with a mean score of 1.04.

Results

When looking at the linear regressions and decision tree regressor, we found certain variables appeared throughout these models. They include the log amount of time (in days) since the event was told to the person who would go on to either retell the story or imagine a story. Draining, an indicator variable that asked how tired a person was while performing the task, also appeared. These results made intuitive sense but they were not the most insightful.

To predict the type of story, several binary classifier models were run because finding the best accuracy score would be ideal. The accuracy score for the logistic regression is the highest of all the binomial classifiers with a 62% accuracy score. Among all the features, the decision tree classifier indicated that the sentiment analysis for the summary and the total word count are the two most important features. By referencing the decision boundary graph below, it is evident that the total word count actually plays a large part within the classification.

The Decision Boundary for the Logistic Regression Model

The decision tree is also another simple way of showing the importance of the features. It is difficult to compare many features; however, if the similarity is large and the total word count is large, then there is a high possibility that it is an imagined story.

The Decision Tree Classifier Model

The logistic regression provides the best results, which is even better than the multi-layer perceptron classifier. It is predicted that the neural net would give better results.