# Patent Keyword Analysis Using Time Series and Copula Models
## 4 Experimental Results  
To verify the performance of our proposed method, we used the patent documents of Apple. Apple is a leading company in smart devices, and smart devices are dependent on diverse technologies. Apple's patent data was used in many research studies for patent data analysis [6; 17; 26]. We have searched all patent documents applied and registered by Apple until 2012 from the patent  
Figure 2: Patent keyword analysis process using the time series and copula models. databases [27; 28]. The number of valid patents finally selected from the collected patent data was 8119. In this paper, we extracted technological keywords from the Apple's valid patent documents using the text mining techniques [29]. The keyword data is a matrix consisting of a row of individual patent documents and a column of keywords. Each element of this matrix represents the occurred frequency value of the keyword in a patent document. Among them, we selected top five keywords with the highest frequency as follows; Device (6919), Data (6676), System (5936), User (4767), and Media (4157). The numbers in parentheses in the above keywords indicate the frequency of each keyword. These keywords were used to find the directional dependence by using Gaussian copula regression with beta logit [16]. First, we visually assessed the trend these five keywords showed over time. Figure 3 shows the time series plots of 'Device', 'Data', 'System', 'User', and 'Media'. The time series plots of main Apple keywords have increasing and nonstationary patterns. To make sure that Apple time series data are nonstationary, we performed the autocorrelation function (ACF) plots and the partial autocorrelation function (PACF) plots of Apple time series keywords. Figures 4 and 5 are the ACF plots and PACF plots which exist nonstationarily over several time lags. From the results of Figures 4 and 5, we can say that the Apple time series keywords are not stationary. It is appropriate that we apply the time series model considering heteroscedasticity to the five highest frequency keywords. The INGARCH(1,1), considering that the conditional distribution of count time data, Yt, follows a Poisson distribution, was applied to the five most highest frequency Apple keywords. Table 1 is a summary of parameter estimates for INGARCH(1,1). Figure 3: Count time series plots of the top five keywords. Figure 4: Autocorrelation function (ACF) plots of the top five keywords. Figure 5: Partial autocorrelation function (PACF) plots of the top five keywords. To estimate the parameters, we used the R data language and the 'tscount' package of R [30; 31]. Table 1 shows that the conditional means of Apple five keywords at time \(t\) depends on the value at time \(t\)-1 more than its own value at time \(t\)-1. By using these INGARCH(1,1) models summarizing Table 1, we generated standardized residuals of the five main Apple keywords for the Gaussian copula models for marginal regression (GCMR) analysis with a correlation structure in order to avoid the serial dependence in the component time series [32]. With these standardized residuals of the five main Apple keywords from INGARCH(1,1), we transformed these standardized residuals into uniform random variables. And then we considered the GCMR(Gaussian copula marginal regression) models with four correlation error structures, ARMA(p,q) where p = 1 and q = 2. For example, the model of 'Device' was represented as follows:  
\[X_{t}\big{|}F_{t-1}:Poisson(\lambda_{t}),\;\lambda_{t}=25.7+0.951X_{t-1}+(3.98E -10)\lambda_{t-1}. \tag{11}\]  
Table 2 shows an Akaike information criterion (AIC) comparison for the GCMR models with a correlation structure.