Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper introduces a task of behavioral stylometry in the domain of chess plays. It uses a few-shot learning framework and uses the optimization objective inspired by the speaker identification domain to solve this problem. It demonstrated clear gains over previous baselines and has performed a satisfactory analysis that can be used by future researchers to extend this work.
main_review: **Originality**: this work opens up a new and meaningful task domain for few-shot learning and representation learning. Though the framework and optimization objective are heavily inspired by other domain, the application is very novel.

**Quality**: the quality of this paper is very high. The authors have conducted satisfactory experiments to evaluate the model's performance. The original experiment shows the promise of using few-shot learning as a framework to solve this task. The ablation studies (by whether to include first k moves) and evaluation of top players point the research community to future research directions.

**Clarity**: the structure and writing are very clear.

**Significance**: it is very important to introduce new problems to the machine learning community. Online chess communities have been growing for quite a few years, and this domain has a strong connection to human behavior modeling, teaching, and human-AI collaboration. It is left to the ML community to decide what are interesting problems that ML should solve in the domain of chess play. The authors of this paper built upon prior work and defined player identification as one of the important tasks to solve -- embeddings that capture the behavioral style of play are effective embeddings for chess moves. This has a huge implication for unsupervised learning and embedding evaluations in the future. It might not be THE task or THE evaluation metric that everyone needs to care about, but this is a very solid first step.

I don't have any significant issues with this paper, mainly because this paper introduces a new problem in a new domain. This is not to say that the paper is perfect. For example:
1. The paper proposes to build "embeddings" of player style, but the visualization in Figure 3 (b) (c), I can't easily recognize clusters, making me wonder if the embeddings do contain valuable information beyond player identification.
2. It seems that the data is public but the player IDs can only be obtained through private communication. I'm concerned that this level of privacy will impact future researchers using this as a valid task to test new algorithms. I hope the authors of this paper will spend more time thinking about how to build a research community around this domain and how to enable rapid model development in this domain in the future. 

limitations_and_societal_impact: The authors have addressed other limitations well.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper the authors propose a transformer-based approach for capturing and identifying individual styles in chess. This goal, which the authors name Behavior Stylometry, is framed as an essential step towards algorithmic personalization and person-agent interaction. The suggested framework borrows ideas from advances in speech processing and only utilizes a small sample in order to learn a representation of players' actions. Once trained, the model can accurately identify the users in a larger pool of unseen samples. Lastly, the authors suggest that their approach reveals structural properties for characterizing playing style in chess.
main_review: Overall, this paper seems methodologically sound to me, and the results seem compelling enough. I do have certain qualms, however.

* The paper seems to be very much tailored for chess applications. While that makes the paper coherent and easy to follow, it also diminishes its impact. It is hard to estimate how much of the proposed approach would generalize to other domains, for instance, or how easy it would be to adapt the proposed framework for other tasks. 

* The design of the loss function is meant to promote an embedding which keeps *individuals* as far apart in the embedding space as possible, but this approach seems to discount the fact that different people may have very similar playing styles, thus leading to artifacts in more general applications (I have more on that note, and how it relates to privacy concerns, see my last comment).

* From a training perspective, it is unclear how much intrinsic endogeneity may exist in the training data. That is to say, while you are separating games, entire subsequences within the game may be close to identical. To mitigate or at least study this aspect, it would make sense to cluster state-action subsequences within game trajectories and group/cluster them. How well would the model fare when seeing familiar people take slightly novel sequences?

* It would  have been interesting to see how well a naïve approach, such as comparing conditional action distributions, would have fared in this task.

* Lastly I'd like to point out that any approach for identifying users by a small sample of their actions may pose dire privacy concerns and should not be explored lightly. While I can see its utility in very restricted game-playing scenarios (for instance, opponent modeling), I believe it is a lot more valuable in typical contexts such as game-playing, recommendation etc. to characterize the *type* (indeed, the *style*) of a person or agent, rather than the unique identity of that person. Going for the latter goal seems to assume unnecessary privacy risk without adding much real-world utility to me.

limitations_and_societal_impact: I believe the greatest limitation of this work is its specificity to the chess domain, making it harder to estimate how well the approach would generalize in other contexts, and therefore its broader impact.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper describes a transformer-based approach to determining chess player identity from moves played.
main_review: **Review Criteria**

Originality: McIlroy-Young et al. (2020b) have previously studied behavioral stylometry in chess. This paper provides a more accurate and scalable solution to an existing task domain using transformer-based game embeddings. The architecture is adapted from work on speaker adaptation but is novel in its application to this domain. 

Quality: The results appear technically sound and the proposed method achieves high accuracy. The authors run appropriate ablations, such as not including opening moves, evaluating out-of-domain generalization, etc. [These ablations are especially important because the accuracy values are *shockingly* good; as an expert chess player (~ELO 2100), I am quite surprised that it is theoretically possible to identify chess players with such high accuracy, and I was quite skeptical of the results at first.]

Clarity: The task and model architecture are clearly described. Some details of the few-shot classification setup are difficult to understand. 

Significance: Behavioral stylometry in chess is not a particularly significant task, but the methods described could easily apply to other domains involving decision-making style, and chess serves as an appropriate and convincing testbed. In particular, the comparison between the embedding method and personalized "move-matching" models is interesting and likely to generalize to other domains. However, as noted below, stylometry is wrought with potential ethical issues that should be discussed further.

**Notes on Specific Sections**

Abstract: First paragraph of the abstract can be removed, since it reads more like an introduction and is in fact repeated in the introduction. I would recommend starting the abstract by describing what you did, rather than motivating the problem domain.

Introduction: It might be worth defining "style" and what it means in the context of chess. For example, one common distinction is between aggressive/tactical vs. positional play, with certain players (Shirov, Tal) being known for aggressive play, and certain openings (e.g., gambits) being more or less tactical in nature. Presumably, this method also picks up on elements of "style" which do not readily correspond to human-interpretable concepts.

Related Work/Methodology: Overall, this is all very clear and well-written. Possibly out-of-scope, but I wonder if you could achieve better results by initializing with board embeddings from the policy network of AlphaZero or some other neural chess engine. [But the results are already quite good, so this isn't top priority.]

Data: I'm not sure why the paper groups by the number of games played when the evaluation uses only 200 games for each player. However, sampling from ELO 1000-2000 is quite a large range, and I'm curious if these results would still hold for a narrower range. It seems possible to me that a model could be evaluating player strength rather than "style." Similarly, the top 1500 players from Chess.com/Lichess will also exhibit a lot variance in strength.

Evaluation: 
 - Task formulation: I found this section difficult to read, possibly because I had previously assumed D = R.
 - Can you confirm that k=15 means 15 moves for both players? Normally in chess a "move" consists of plays from both white and black, but I just wanted to confirm that's the case in your paper. There is technically opening theory that extends beyond the 15th move, but especially in Lichess 1000-2000 games, I wouldn't expect it to be very common. However, if k=15 actually meant 15 turns for both players combined (i.e., 7.5 moves), then this would definitely still be within the realm of opening theory, even for many amateur players.
 - Results for k=0/k=15 and varying sizes of support/query sets are useful, thanks!
 - Attention results seem straightforward and confirm my intuitions that openings are the most useful part of the game for stylometry, except for maybe the first move (where there are just less choices to be made). I'm less clear what takeaways I should have from the t-SNE plots, especially 3b and 3c. Does this just mean that choice of first move is an important factor in style but responses or not? Or how should I interpret 3b-3c?

Conclusion: Additional discussion of privacy concerns would be appreciated. See below.
limitations_and_societal_impact: The paper would benefit from further motivation of the problem domain and further discussion of ethical issues concerning behavioral stylometry. Is chess simply a testbed for general-purpose behavioral stylometry methods, or are there real-world applications of behavioral stylometry in chess? I see two main ethical issues with general-purpose behavioral stylometry, which merit further discussion:
 - Privacy Concerns -- this is briefly mentioned in the conclusion but deserves further discussion and citation of related ethics work. For example, the related work stylometry section mentions face verification, de-anonymization of programmers, and "identification of age and gender from blog posts" but does not mention the ethical issues raised by these application domains. Further, detecting user identity from online behavior carries many ethical risks and the tradeoffs are not well-described in this paper. For example, what are the "benefits of methodologies capable of stylometry" in this context? What are the risks? [Although I think this is mainly an issue in other stylometry domains, I don't particularly want my (anonymous) Lichess account to be connected to my (public) Chessbase data, either.]
 - Performance Ceilings -- high-accuracy stylometry is impossible in many domains, such as author identification. This can have ethical implications for e.g., forensic domains, where it is critical that policymakers and other non-technical individuals understand the limitations of these methods. Even in a domain like chess, methods for cheating detection (which became especially popular during virtual tournaments due to COVID-19) are limited in their accuracy, and accurately conveying that *it is probably impossible to identify players with perfect accuracy* is important for helping non-ML practitioners understand and make use of these results.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper introduces a transformer-based neural network architecture that can be used to learn and identify unique playing styles of individual chess players. The neural network learns an embedding over players' actions and by using the distance between the learned embedding and a sequence of chess moves, the network can identify the player taking those moves. While this problem has been studied before, the main contribution of the paper is the ability of the architecture to learn the playing styles of unseen players using a small number of example moves.
main_review: The main idea and motivation behind the paper is well defined and I found the problem of few-shot learning of player styles interesting. However, the comparison with a baseline used in the experiments from a previous paper is flawed -- see below.

Originality: While none of the techniques used in this paper are novel, the authors do employ previous techniques to a problem in which they are yet to be used. The use of learned embeddings to characterize player styles was clever.

Clarity: The paper leaves much to be desired in terms of clarity. I think the paper could have been written better. One subsection in which this is apparent is the description of the loss function. The explanation reads as if it structured in reverse --- that is, it would have made more sense if the similarity matrix was described before the embeddings. In fact, the description of the embeddings uses notations not defined until the similarity matrix was explained. I ended up having to refer the to the cited literature in order to understand this part. Furthermore, I think the paper focusses too much on the description of the architecture specifications rather than on the reason this architecture was chosen. Given that the authors claim that the strength of their proposed model lies in the use of transformers, it would have been prudent for the authors to describe why this is so --- why are transformers better? What characteristics of transformers make them more suitable to the task of behavioral stylometry?

Quality: The experimental evaluation has a potentially major issue in regards to the comparisons performed against previous methods. Authors, please clarify this in your response, if I misunderstood the comparison. As far as I can tell, the paper misrepresents its comparative performance. The paper presents behavioral stylometry on settings of k = 0 and k = 15. While the authors state that they compare against previous work at a setting of k = 15, this is not what they do. The results from previous work on an identical problem were performed at settings of k = 0, 10 and 30. The performance of the previous work reported in the paper were for k = 30. The prediction problem is much more difficult with higher k values (as the authors themselves make clear.) This invalidates any comparison with the previous algorithm. 

Significance: The paper does not present anything wholly unique, but the results reported are promising (if one discounts the potential major flaw described in the paragraph above). Another limiting factor of the paper's significance is that it uses a single problem domain in its evaluation.

COMMENT AFTER RESPONSE

Thanks for your response. In particular for clarifying the issue with the k-value. 
limitations_and_societal_impact: The authors do not sufficiently address the societal impacts of their work. There is one line in the conclusion that addresses this but it is not enough. Especially, given the security and privacy concerns that could arise from misuse of the research. While the work is limited to a relatively harmless domain, there is potential of its use in more sensitive domains. Given that identifying people based incomplete information can be abused in a number of situations, the authors should have spent more time elaborating on the positive/negative implications of their work.
review_title: null

Meta-review: UPDATE:  The revision has been reviewed and this paper has been accepted.  We recommend that the authors integrate the final sentence of the introduction into the end of Paragraph 2, where they are talking about applications of their ideas.  Some applications raise ethical considerations and it's important to admit that alongside the other applications rather than it appearing as an afterthought at the end.

----

This paper is very difficult to make a decision.  The reviewers all agree the paper is excellent technically (even the low reviewer score is not basing their score on the technical contribution).  However, there is also general agreement that the paper does not sufficiently address what amounts to serious ethical considerations of this work, specifically the opportunity for it or derivatives to be abused.  The authors acknowledge that a more substantial discussion is needed in the paper, so I trust they understand this significance of this concern.

However, I (and other reviewers) also feel the authors' response lacks a recognition of appropriate mitigations of possible abuse.  They primarily appeal to the public nature of the chess data as a consideration in why this line of research is ethically appropriate.  This is not sufficient.  Abuses, for example where a bad actor uses even public data to identify/track an otherwise anonymous person in order to discriminate or restrict rights, are not limited to use only private data.  In fact, it seems to me public data abuses are more likely or more damaging than private data abuses (as the latter limits the abuse to owners of private data).

I do believe advances of stylometry in identifying chess players is a pretty benign application, however there's little discussion about how general this work might be in its ability to do stylometry in other domains.  The authors mention an additional social benefit of understanding what identifiability exists in our datasets, but this argument needs more attention and peer review than can be completed through the author response/discussion.

As a consequence this paper is being **conditionally accepted**.  The final determination will only be made upon review of the revision.   The authors should take advantage of the extra page to address concerns as reflected in their own response(s), although without the emphasis that use of public data lessens the ethical concerns.  We hope that the authors' added discussion could be used to send a strong signal commensurate with the gravity of the situation, so that appropriate community norms can be developed.
Decision: Accept (Poster)