Question: What is used as a feature set?
Evidence: In the first experiment we perform an attribution of individual scenes of H8 using the Support Vector Machine as a classifier and the frequencies of 500 most frequent rhythmic types and the frequencies of 500 most frequent words as a feature set. As training samples, individual scenes of plays written by Shakespeare, Fletcher, and Massinger are used that come roughly from the period when H8 was supposedly written, namely:
Shakespeare: The Tragedy of Coriolanus (5 scenes), The Tragedy of Cymbeline (27 scenes), The Winter’s Tale (12 scenes), The Tempest (9 scenes)
Fletcher: Valentinian (21 scenes), Monsieur Thomas (28 scenes), The Woman’s Prize (23 scenes), Bonduca (18 scenes)
Massinger: The Duke of Milan (10 scenes), The Unnatural Combat (11 scenes), The Renegado (25 scenes)
Altogether there are thus 53 training samples for Shakespeare, 90 training samples for Fletcher and 46 training samples for Massinger. In order to estimate the accuracy of the model, cross-validation is performed in the following way:
To avoid the risk of overfitting which may be caused by testing the model on the scenes from the same play as it was trained on, we do not perform a standard k-fold cross validation. Instead, we classify scenes of each play by a model trained on the rest, i.e. 5 scenes of Shakespeare’s Coriolanus are classified by a model trained on the scenes from the remaining 3 plays by Shakespeare, 4 plays by Fletcher and 5 plays by Massinger, 27 scenes of Cymbeline are classified in the same way and so on.
Since the training data are imbalanced (which may bias the results), we level the number of training samples per author by random selection.
To obtain more representative results, the entire process is repeated 30 times (with a new random selection in each iteration) thus resulting in 30 classifications of each scene.
For the sake of comparison of the attribution power of both feature subsets, cross-validations are performed not only of the combined models (500 words $\cup $ 500 rhythmic types), but also of the words-based models (500 words) and versification-based models (500 rhythmic types) alone.
As shown in Table TABREF14, the versification-based models yield a very high accuracy with the recognition of Shakespeare and Fletcher (0.97 to 1 with the exception of Valentinian), yet slightly lower accuracy with the recognition of Massinger (0.81 to 0.88). The accuracy of words-based models remains very high across all three authors (0.95 to 1); in three cases it is nevertheless outperformed by the combined model. We thus may conclude that combined models provide a reliable discriminator between Shakespeare’s, Fletcher’s and Massinger’s styles.
Table TABREF19 gives the results of the classifiers when applied to the individual scenes of H8 on the basis of which we may conclude:
It is very unlikely that Massinger took part in the text of H8. Out of 17 scenes only 2 are attributed to Massinger by any of the models (2.1, 4.2), and in both cases by a mere minority of votes.
The probability that the text of H8 is a result of collaboration between Shakespeare and Fletcher is very high: with 7 scenes all the 30 models agree upon Shakespeare’s authorship, with 5 scenes all the 30 models agree upon Fletcher’s authorship.
Our results correspond to the Spedding’s attribution to a high extent. With the exception of two scenes, the majority of models always predict the same author to which it is attributed by Spedding. The two exceptions are the second scene of act 3, where Spedding supposed mixed authorship, and the first scene of act 4, which was originally attributed to Fletcher.
In the first experiment we perform an attribution of individual scenes of H8 using the Support Vector Machine as a classifier and the frequencies of 500 most frequent rhythmic types and the frequencies of 500 most frequent words as a feature set.
Answer: The frequencies of 500 most frequent rhythmic types and the frequencies of 500 most frequent words.