## 4 Results
### Predicting Institution Outcome Based on Patent Owner's Preliminary Response Brief  
In post-grant review litigation before PTAB, the goal of the patent owner's preliminary response brief is to avoid an institution of trial. At the initial stage, factual inferences are considered in the petitioner's favor, and the petitioner need only demonstrate a reasonable likelihood of success. Therefore, it is the patent owner's goal to show that there is a gap in the petitioner's logic, or that institution should be denied at the discretion of the judges, for example because the petition is redundant with arguments that were already made during prosecution before the patent was issued, duplicative of other petitions, or redundant to other parallel proceedings [24, 26]. To be sure, a patent owner may choose to waive their response, in which case, usually, although not always, trial is instituted; however, we do not consider those cases [22]. This paper focuses on developing a predictive and interpretable model that can predict a preliminary response brief's probability of winning based on the text and which, based on interpreting successful predictions, can provide insight as to what arguments and phrases may be connected to success. As explained in Section 4.1.1, the training data set must be constrained to a time when there is stability in the law and PTAB jurisprudence. For example, the training data set should exclude proceedings decided before April 2018, when a mixed outcome was possible. As another example, in 2019 PTAB also established legal precedents based on the _NHK_ and _Fnittiv_ cases that gave patent owners a clear roadmap to winning a denial of institution based on redundancy with parallel proceedings [26]. Accordingly, the training data set includes patent owners' preliminary response briefs in cases filed between 1 July 2018 and 30 November 2020. This provides a maximal number of training documents (2631 cases in total) to be sufficiently generalizable, while ensuring that it includes briefs which use contemporary legal arguments. Figure 8 shows the results of predicting the case outcome based on the text of the preliminary response brief, using both the XGBoost/TF-IDF and CNN-Attention (Figure 2) methods. Two test sets given limited data in a stable time period (one that may have temporal correlation with the training since it starts the day after the training period and one with one month separation): (1) All cases filed between 1 December 2020 and 31 March 2021. (2) All cases filed between 1 January 2021 and 31 March 2021, to ensure complete temporal separation from the training data set. After 1 April 2021, prediction accuracy drops, thus \begin{table}
\begin{tabular}{c c c c} \hline \hline
**CNN-Attention** & **LIME (CNN)** & **SHAP (XGBoost)** & **TF-IDF** \\ \hline notice & record & notice & program \\ hereby & stage & stage & digitally \\ commence & proceeding & record & television \\ entry & patentability & entry & interactive \\ pursuant & citing & pursuant & programs \\ stage & interactive & institution & guide \\ final & teaches & ordered & directory \\ made & hereby & explain & associated \\ shall & television & hereby & claims \\ given & pursuant & final & presents \\ impact & recites & disagree & maintaining \\ different & notice & arguments & data \\ ground & using & citing & recordings \\ exists & demonstrates & therefore & constitutional \\ record & dispute & date & stored \\ institution & discussed & combination & previously \\ material & arguments & petition & teaches \\ patentability & entry & authorized & erred \\ disclosure & associated & must & user \\ identified & request & recites & storing \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Top 20 words by mean attention (obtained from trained CNN classifier), LIME (for the CNN classifier), SHAP value (after XGBoost classification), and TF-IDF score (i.e., no classification) for the written decision in IPR2020-00806. limited to this 4 month period for testing.