# Modelling of Web Domain Visits by Radial Basis Function Neural Networks and Support Vector Machine Regression

Vladimír Olej, Jana Filipová

► To cite this version:

Vladimír Olej, Jana Filipová. Modelling of Web Domain Visits by Radial Basis Function Neural Networks and Support Vector Machine Regression. 12th Engineering Applications of Neural Networks (EANN 2011) and 7th Artificial Intelligence Applications and Innovations (AIAI), Sep 2011, Corfu, Greece. pp.229-239, 10.1007/978-3-642-23960-1_28 . hal-01571473

HAL Id: hal-01571473

https://inria.hal.science/hal-01571473

Submitted on 2 Aug 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Modelling of Web Domain Visits by Radial Basis Function Neural Networks and Support Vector Machine Regression

Vladimír Olej, Jana Filipová

Institute of System Engineering and Informatics
Faculty of Economics and Administration, University of Pardubice, Studentská 84
532 10 Pardubice, Czech Republic,
{vladimir.olej@upce.cz, jana.filipova@upce.cz}

**Abstract.** The paper presents basic notions of web mining, radial basis function (RBF) neural networks and ε-insensitive support vector machine regression (ε-SVR) for the prediction of a time series for the website of the University of Pardubice. The model includes pre-processing time series, design RBF neural networks and ε-SVR structures, comparison of the results and time series prediction. The predictions concerning short, intermediate and long time series for various ratios of training and testing data. Prediction of web data can be benefit for a web server traffic as a complicated complex system.

**Keywords:** Web mining, radial basis function neural networks, ε-insensitive support vector machine regression, time series, prediction.

## 1 Introduction

Data modelling (prediction, classification, optimization) obtained by web mining [1] from the log files and data on a virtual server, as a complicated complex system that affects these data, there are a few problems worked out. The web server represents a complicated complex system; virtual system usually works with several virtual machines that operate over multiple databases. The quality of the activities of the system also affects the data obtained using web mining. Given virtual system is characterized by its operational parameters, which are changing over time, so it is a dynamic system. The data show a nonlinear characteristics, are heterogeneous, inconsistent, missing and uncertain. Currently there are a number of methods for modelling of the data obtained by web mining. These methods can generally be divided into methods of modelling with unsupervised learning [2,3] and supervised learning [4]. The present work builds on the modelling of web domains visit with uncertainty [5,6].

In the paper is presented a problem formulation with the aim of describing the time series web upce.cz (web presentation visits), including possibilities of pre-processing which is realized by means of simple mathematic-statistic methods. Next, there are introduced basic notions of RBFs [7] neural networks and ε-SVRs [4,8] for time
series prediction. Further, the paper includes a comparison of the prediction results of
the model designed. The model represents a time series prediction for web upce.cz
with the pre-processing done by simple mathematical statistical methods. The
prediction is carried out through RBF neural networks and ε-SVRs for different
durations of time series and various ratios O<sub>train</sub>:O<sub>test</sub> training O<sub>train</sub> and testing O<sub>test</sub>
dates (O= O<sub>train</sub> ∪ O<sub>test</sub>). There is expected, that for shorter time series prediction it
would be better to use RBF neural networks and for longer ε-SVRs.

## 2 Problem Formulation

The data for prediction of the time series web upce.cz over a given time period was
obtained from Google Analytics. This web mining tool, which makes use of Java-
Script code implemented for web presentation, offers a wide spectrum of operation
characteristics (web metrics). Metrics provided by Google Analytics can be divided
into following sections: visits, sources of access, contents and conversion. In the
section ‘visits’ it can be monitored for example the number of visitors, the number of
visits and number of pages viewed as well as the ratio of new and returning visitors.
Indicator geolocation, i.e. from which country are visitors most often, is needed to be
known because of language mutations, for example. In order to predict the visit rate to
the University of Pardubice, Czech Republic website (web upce.cz) it is important
monitoring the indicator of the number of visits within a given time period. One
‘visit’ here is defined as an unrepeated combination of IP address and cookies. A sub-
metrics is absolutely unique visit defined by an unrepeatable IP address and cookies
within given time period. The basic information obtained from Google Analytics
about web upce.cz during May 2009 considered of the following: The total visit rate
during given monthly cycles. A clear trend is obvious there, with Monday having the
highest visit rate, which in turn decreases as the week progresses; Saturday has the
lowest visit rate; The average number of pages visited is more than three; A visitor
stays on certain page five and half a minutes on average; The bounce rate is
approximately 60%; Visitors generally come directly to the website, which is
positive; The favourite pages is the main page, followed by the pages of the Faculty
of Economic and Administration and the Faculty of Philosophy.

The measurement of the visit rate of the University of Pardubice web page, (web
upce.cz) took place time periods of regular, controlled intervals. The result represents
a time series. The pre-processing of data was realized by means of simple
mathematic-statistic methods: a simple moving average (SMA), a central moving
average (CMA), a moving median (MM) along with simple exponential smoothing
(SES) and double exponential smoothing (DES) at time t. Pre-processing was used
with aim of smoothing the outliers, while maintaining the physical interpretation of
data.

The general formulation of the model of prediction of the visit rate for upce.cz can
be stated in thusly: y'=f(x'<sub>1</sub>, x'<sub>2</sub>, ..., x'<sub>m</sub>), m=5, where y' is the number of daily web
visits in time t+1, y is the number of daily web visits in time t, x'<sub>1</sub> is SMA, x'<sub>2</sub> is
CMA, x'<sub>3</sub> is MM, x'<sub>4</sub> is SES, and x'<sub>5</sub> is DES at time t. An example of the pre-
processing of the time series of web upce.cz is represented in Fig. 1.
<figure><img src="image_2.png" /><figcaption>Fig. 1. The pre-processing of web upce.cz visits by SMA</figcaption></figure>

On the basis of the different durational time series of web upce.cz, concrete prediction models for the visit rate to upce.cz web can be defined thusly

$$ 
\begin{aligned}
y' &= f(TS_{SMA}^{S}, TS_{CMA}^{S}, TS_{MM}^{S}, TS_{SES}^{S}, TS_{DES}^{S}), \\
y' &= f(TS_{SMA}^{I}, TS_{CMA}^{I}, TS_{MM}^{I}, TS_{SES}^{I}, TS_{DES}^{I}), \\
y' &= f(TS_{SMA}^{L}, TS_{CMA}^{L}, TS_{MM}^{L}, TS_{SES}^{L}, TS_{DES}^{L}),
\end{aligned} 
\quad (1) $$

where TS - time series, S - short TS (264 days), I - intermediate TS (480 days), L - long TS (752 days) at time t.

## 3 Basic Notions of RBF Neural Networks and ε-SVR

The term RBF neural network [7] refers to any kind of feed-forward neural networks that uses RBF as an activation function. RBF neural networks are based on supervised learning. The output $f(x, H, w)$ RBF of a neural network can be defined this way

$$ f(x, H, w) = \sum_{i=1}^{q} w_i \times h_i(x), \quad (2) $$

where $H=\{h_1(x), h_2(x), \dots, h_i(x), \dots, h_q(x)\}$ is a set of activation functions RBF of neurons (of RBF functions) in the hidden layer and $w_i$ are synapse weights. Each of the $m$ components of vector $x=(x_1, x_2, \dots, x_k, \dots, x_m)$ is an input value for the $q$ activation functions $h_i(x)$ of RBF neurons. The output $f(x, H, w)$ of RBF neural network represents a linear combination of outputs from $q$ RBF neurons and corresponding synapse weights $w$.

The activation function $h_i(x)$ of an RBF neural network in the hidden layer belongs to a special class of mathematical functions whose main characteristic is a monotonous rising or falling at an increasing distance from center $c_i$ of the activation function $h_i(x)$ of an RBF. Neurons in the hidden layer can use one of several activation functions $h_i(x)$ of an RBF neural network, for example a Gaussian activation function (a one-dimensional activation function of RBF), a rotary Gaussian activation function (a two-dimensional RBF activation function), multisquare and inverse multisquare activation functions or Cauchy's functions. Results may be presented in this manner
$$h(\mathbf{x}, C, R) = \sum_{i=1}^{q} \exp\left(-\frac{\|\mathbf{x} - \mathbf{c}_i\|^2}{r_i}\right), \quad (3)$$

where $\mathbf{x}=(x_1, x_2, \dots, x_k, \dots, x_m)$ represents the input vector, $C=\{c_1, c_2, \dots, c_i, \dots, c_q\}$ are centres of activation functions $h_i(x)$ an RBF neural network and $R=\{r_1, r_2, \dots, r_i, \dots, r_q\}$ are the radiuses of activation functions $h_i(x)$.

The neurons in the output layer represent only a weighted sum of all inputs coming from the hidden layer. An activation function of neurons in the output layer can be linear, with the unit of the output eventually being convert by jump instruction to binary form. The RBF neural network learning process requires a number of centres $c_i$ of activation function $h_i(x)$ of the RBF neural networks to be set as well as for the most suitable positions for RBF centres $c_i$ to be found. Other parameters are radiuses of centres $c_i$, rate of activation functions $h_i(x)$ of RBFs and synapse weights $W(q,n)$. These are setup between the hidden and output layers. The design of an appropriate number of RBF neurons in the hidden layer is presented in [4]. Possibilities of centres recognition $c_i$ are mentioned in [7] as a random choice. The position of the neurons is chosen randomly from a set of training data. This approach presumes that randomly picked centres $c_i$ will sufficiently represent data entering the RBF neural network. This method is suitable only for small sets of input data. Use on larger sets, often results in a quick and needless increase in the number of RBF neurons in the hidden layer, and therefore an unjustified complexity of the neural network. The second approach to locating centres $c_i$ of activation functions $h_i(x)$ of RBF neurons can be realized by a K-means algorithm.

In nonlinear regression $\varepsilon$-SVR [4,8,9,10,11] minimizes the loss function $L(d,y)$ with insensitive $\varepsilon$ [4,11]. Loss function $L(d,y)=|d-y|$, where $d$ is the desired response and $y$ is the output estimate. The construction of the $\varepsilon$-SVR for approximating the desired response $d$ can be used for the extension of loss function $L(d,y)$ as follows

$$L_{\varepsilon}(d, y) = \begin{cases} d - y - \varepsilon & \text{for } |d - y| \ge \varepsilon, \\ 0 & \text{else} \end{cases}, \quad (4)$$

where $\varepsilon$ is a parameter. Loss function $L_{\varepsilon}(d,y)$ is called a loss function with insensitive $\varepsilon$.

Let the nonlinear regression model in which the dependence of the scalar $d$ vector $\mathbf{x}$ expressed by $d=f(\mathbf{x}) + n$. Additive noise $n$ is statistically independent of the input vector $\mathbf{x}$. The function $f(\cdot)$, and noise statistics are unknown. Next, let the sample training data $(\mathbf{x}_i, d_i)$, $i=1,2, \dots, N$, where $\mathbf{x}_i$ and $d_i$ is the corresponding value of the output model $d$. The problem is to obtain an estimate of $d$, depending on $\mathbf{x}$. For further progress is expected to estimate $d$, called $y$, which is widespread in the set of nonlinear basis functions $\varphi(\mathbf{x}), j=0,1, \dots, m_l$ this way

$$y = \sum_{j=0}^{m_l} w_j \varphi_j(\mathbf{x}) = w^T \varphi(\mathbf{x}), \quad (5)$$

where $\varphi(\mathbf{x})=[\varphi_0(\mathbf{x}), \varphi_1(\mathbf{x}), \dots, \varphi_{m_l}(\mathbf{x})]^T$ and $w=[w_0, w_1, \dots, w_{m_l}]$. It is assumed that $\varphi_0(\mathbf{x})=1$ in order to the weight $w_0$ represents bias $b$. The solution to the problem is to minimize the empirical risk
$$R_{\text{emp}} = \frac{1}{N} \sum_{i=1}^{N} L_{\varepsilon}(d_i, y_i), \qquad (6)$$

under conditions of inequality $||w||^2 \le c_0$, where $c_0$ is a constant. The restricted optimization problem can be rephrased using two complementary sets of non-negative variables. Additional variables $\xi$ and $\xi'$ describe loss function $L_{\varepsilon}(d_i, y_i)$ with insensitivity $\varepsilon$. The restricted optimization problem can be written as an equivalent to minimizing the cost functional

$$\phi(w, \xi, \xi') = C \left( \sum_{i=1}^{N} (\xi_i + \xi'_i) \right) + \frac{1}{2} w^T w, \qquad (7)$$

under the constraints of two complementary sets of non-negative variables $\xi$ and $\xi'$. The constant $C$ is a user-specified parameter. Optimization problem (7) can be easily solved in the dual form. The basic idea behind the formulation of the dual-shaped structure is the Lagrangian function [9], the objective function and restrictions. Can then be defined Lagrange multipliers with their functions and parameters which ensure optimality of these multipliers. Optimization the Lagrangian function only describes the original regression problem. To formulate the corresponding dual problem a convex function can be obtained (for shorthand)

$$Q(\alpha_i, \alpha'_i) = \sum_{i=1}^{N} d_i(\alpha_i - \alpha'_i) - \varepsilon \sum_{i=1}^{N} (\alpha_i + \alpha'_i) - \frac{1}{2} \sum_{i=1}^{N} \sum_{j=1}^{N} (\alpha_i - \alpha'_i)(\alpha_j - \alpha'_j) K(x_i, x_j), \qquad (8)$$

where $K(x_i, x_j)$ is kernel function defined in accordance with Mercer's theorem [4,8]. Solving optimization problem is obtained by maximizing $Q(\alpha, \alpha')$ with respect to Lagrange multipliers $\alpha$ and $\alpha'$ and provided a new set of constraints, which hereby incorporated constant $C$ contained in the function definition $\phi(w, \xi, \xi')$. Data points covered by the $\alpha \neq \alpha'$, define support vectors [4,8,9].

# 4 Modelling and Analysis of the Results

The designed model in Fig. 2 demonstrates prediction modelling of the time series web upce.cz. Data pre-processing is carried out by means of data standardization. Thereby, the dependency on units is eliminated. Then the data are pre-processed through simple mathematical statistical methods (SMA, CMA, MM, SES, and DES). Data pre-processing makes the suitable physical interpretation of results possible. The pre-processing is run for time series of different duration namely ($TS^S_{SMA}$, $TS^S_{CMA}$, $TS^S_{MM}$, $TS^S_{SES}$, $TS^S_{DES}$), ($TS^I_{SMA}$, $TS^I_{CMA}$, $TS^I_{MM}$, $TS^I_{SES}$, $TS^I_{DES}$), ($TS^L_{SMA}$, $TS^L_{CMA}$, $TS^L_{MM}$, $TS^L_{SES}$, $TS^L_{DES}$). The prediction is made for the aforementioned pre-processed time series S, I, and L with the help of RBF neural networks, $\varepsilon$-SVR with a polynomial kernel function and $\varepsilon$-SVR with a RBF kernel function for various sets of training $O_{\text{train}}$ and testing $O_{\text{test}}$ data.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Prediction modelling of time series for web.upce.cz</figcaption></figure>

In Fig. 3 are shown the dependencies of Root Mean Squared Error (RMSE) on the number of neurons *q* in the hidden layer. The dependencies RMSE on the parameter *μ* are represented in Fig. 4. In Fig. 5 the dependencies RMSE are on the parameter *ν* for training *O*<sub>train</sub> and as well as the testing *O*<sub>test</sub> set, where:
* - *O*<sub>train_S</sub>,
* ♦ - *O*<sub>train_I</sub>,
* ■ - *O*<sub>train_L</sub>,
* × - *O*<sub>test_S</sub>,
* ▲ - *O*<sub>test_I</sub>,
* * - *O*<sub>test_L</sub>.

The parameter *μ* allows for an overrun of the local extreme in the learning process and the following progress of learning. The parameter *ν* represents the selection of centers RBFs as well as guarantees the correct allocation of neurons in the hidden layer for the given data entering the RBF neural network.

<figure><img src="image_4.png" /><figcaption>Fig. 3. RMSE dependencies on the parameter *q*</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4. RMSE dependencies on the parameter μ Fig. 5. RMSE dependencies on the parameter ν</figcaption></figure>

Conclusions presented in [4,7,10] are verified by the analysis of results (with 10-fold cross validation). RMSE<sub>train</sub> is lowered until it reaches to the value 0.3, when *q* neurons in the hidden layer increases in size 125 and with the greater ratio O<sub>train</sub>:O<sub>test</sub>.
RMSE<sub>test</sub> decreases only when the number *q* RBF neurons increases at a significantly slower ratio than the ratio O<sub>train</sub>:O<sub>test</sub>. Minimum RMSE<sub>test</sub> moves right with the increasing ratio O<sub>train</sub>:O<sub>test</sub>. Next, determination of the optimal number of *q* neurons in the hidden layer is necessary. Respectively, a lower number *q* neurons represents the increasing RMSE<sub>test</sub>. In Table 1, Table 2, and Table 3 are shown optimized results of the analysis of the experiments for different parameters of the RBF neural networks (with RBF activation function) with different durations of time series, various ratios of O<sub>train</sub>:O<sub>test</sub> training O<sub>train</sub> and testing O<sub>test</sub> of data sets and same amount of learning at *p*=600 cycles. Table 2 builds on the best set of *q* in Table 1 and Table 3 builds on the best set of *q* (Table 1) and *μ* (Table 2).

<table><caption>**Table 1.** Optimized results of RMSE analysis using the amount of neurons *q* in the hidden layer (*μ* and *ν* are constant for given TS)</caption><thead><tr><th>TS</th><th>O<sub>train</sub>:O<sub>test</sub></th><th>Number q of neurons</th><th>μ</th><th>ν</th><th>RMSE<sub>train</sub></th><th>RMSE<sub>test</sub></th></tr></thead><tbody><tr><td>S</td><td>50:50</td><td>80</td><td>0.9</td><td>1</td><td>0.383</td><td>0.408</td></tr><tr><td>S</td><td>66:34</td><td>80</td><td>0.9</td><td>1</td><td>0.331</td><td>0.463</td></tr><tr><td>S</td><td>80:20</td><td>80</td><td>0.9</td><td>1</td><td>0.343</td><td>0.365</td></tr><tr><td>I</td><td>50:50</td><td>125</td><td>0.9</td><td>1</td><td>0.372</td><td>0.316</td></tr><tr><td>I</td><td>66:34</td><td>125</td><td>0.9</td><td>1</td><td>0.295</td><td>0.440</td></tr><tr><td>I</td><td>80:20</td><td>125</td><td>0.9</td><td>1</td><td>0.352</td><td>0.326</td></tr><tr><td>L</td><td>50:50</td><td>25</td><td>0.9</td><td>1</td><td>0.409</td><td>0.402</td></tr><tr><td>L</td><td>66:34</td><td>25</td><td>0.9</td><td>1</td><td>0.404</td><td>0.415</td></tr><tr><td>L</td><td>80:20</td><td>25</td><td>0.9</td><td>1</td><td>0.409</td><td>0.390</td></tr></tbody></table>

<table><caption>**Table 2.** Optimized results of RMSE analysis using parameter *μ* (*q* and *ν* are constant for given TS)</caption><thead><tr><th>TS</th><th>O<sub>train</sub>:O<sub>test</sub></th><th>Number q of neurons</th><th>μ</th><th>ν</th><th>RMSE<sub>train</sub></th><th>RMSE<sub>test</sub></th></tr></thead><tbody><tr><td>S</td><td>50:50</td><td>80</td><td>0.5</td><td>1</td><td>0.473</td><td>0.385</td></tr><tr><td>S</td><td>66:34</td><td>80</td><td>0.5</td><td>1</td><td>0.341</td><td>0.445</td></tr><tr><td>S</td><td>80:20</td><td>80</td><td>0.5</td><td>1</td><td>0.311</td><td>0.408</td></tr><tr><td>I</td><td>50:50</td><td>125</td><td>0.3</td><td>1</td><td>0.313</td><td>0.344</td></tr><tr><td>I</td><td>66:34</td><td>125</td><td>0.3</td><td>1</td><td>0.302</td><td>0.428</td></tr><tr><td>I</td><td>80:20</td><td>125</td><td>0.3</td><td>1</td><td>0.343</td><td>0.365</td></tr><tr><td>L</td><td>50:50</td><td>25</td><td>0.3</td><td>1</td><td>0.400</td><td>0.387</td></tr><tr><td>L</td><td>66:34</td><td>25</td><td>0.3</td><td>1</td><td>0.398</td><td>0.409</td></tr><tr><td>L</td><td>80:20</td><td>25</td><td>0.3</td><td>1</td><td>0.406</td><td>0.391</td></tr></tbody></table>

<table><caption>**Table 3.** Optimized results of RMSE analysis using parameter *ν* (*q* and *μ* are constant for given TS)</caption><thead><tr><th>TS</th><th>O<sub>train</sub>:O<sub>test</sub></th><th>Number q of neurons</th><th>μ</th><th>ν</th><th>RMSE<sub>train</sub></th><th>RMSE<sub>test</sub></th></tr></thead><tbody><tr><td>S</td><td>50:50</td><td>80</td><td>0.5</td><td>1</td><td>0.385</td><td>0.473</td></tr><tr><td>S</td><td>66:34</td><td>80</td><td>0.5</td><td>1</td><td>0.341</td><td>0.445</td></tr><tr><td>S</td><td>80:20</td><td>80</td><td>0.5</td><td>1</td><td>0.311</td><td>0.408</td></tr><tr><td>I</td><td>50:50</td><td>125</td><td>0.3</td><td>3</td><td>0.357</td><td>0.375</td></tr><tr><td>I</td><td>66:34</td><td>125</td><td>0.3</td><td>3</td><td>0.327</td><td>0.405</td></tr><tr><td>I</td><td>80:20</td><td>125</td><td>0.3</td><td>3</td><td>0.376</td><td>0.441</td></tr><tr><td>L</td><td>50:50</td><td>25</td><td>0.3</td><td>1</td><td>0.400</td><td>0.387</td></tr><tr><td>L</td><td>66:34</td><td>25</td><td>0.3</td><td>1</td><td>0.398</td><td>0.409</td></tr><tr><td>L</td><td>80:20</td><td>25</td><td>0.3</td><td>1</td><td>0.406</td><td>0.391</td></tr></tbody></table>
The dependencies RMSE on the parameter *C* are represented in Fig. 6, the dependencies RMSE on the parameter *β* in Fig. 7, the dependencies RMSE on the parameter *γ* in Fig. 8 for training *O*<sub>train</sub> as well as testing *O*<sub>test</sub> set, where: ● - *O*<sub>train_S</sub>, ♦ - *O*<sub>train_I</sub>, ■ - *O*<sub>train_L</sub>, × - *O*<sub>test_S</sub>, ▲ - *O*<sub>test_I</sub>, * - *O*<sub>test_L</sub>. The parameters *C*, *ε* are functions of kernel functions *k(x, x<sub>i</sub>)* [4,11] variations. In the learning process *ε*-SVR are set using 10-fold cross validation. Parameter *C* controls the trade off between errors of the *ε*-SVR of training data and margin maximization; *ε* [4] selects support vectors in the regression structures, and *β* represents the rate of polynomial kernel function *k(x, x<sub>i</sub>)*. The coefficient *γ* characterizes polynomial and RBF kernel function.

<figure><img src="image_6.png" /><figcaption>Fig. 6. RMSE dependencies on the parameter *C*</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 7. RMSE dependencies on the parameter *β* Fig. 8. RMSE dependencies on the parameter *γ*</figcaption></figure>

The confirmation of conclusions presented in [4,11] is verified by the analysis of
results. RMSE<sub>test</sub> for the ε-SVR with RBF kernel function lowers toward zero value
with decreasing *C* (in the case the user experimentation) and higher ratio of O<sub>train</sub>:O<sub>test</sub>.
In the use of 10-fold cross validation, RMSE<sub>test</sub> moves towards zero with an increase
of parameter γ for O<sub>train_S</sub>. In the ε-SVR with polynomial kernel function RMSE<sub>test</sub>
significantly decreases when the parameter β decreases (Fig. 7). Minimum RMSE<sub>test</sub>
moves rights with an increase of the parameter γ (the minimum is between 0.2 to 0.3),
whereas an indirect correlation between the ratio O<sub>train</sub>:O<sub>test</sub> and RMSE<sub>test</sub> obtains.

In Table 4 and Table 5 are shown the optimized results of the analysis of the
experiments for different parameters of the ε-SVR (with RBF and a polynomial
kernel function) with different durations of time series, various ratios O<sub>train</sub>:O<sub>test</sub>
training O<sub>train</sub> and testing O<sub>test</sub> of data sets and same amount of learning at p=600
cycles. The tables are not represented by the partial results for the various parameters,
but only the resulting set of parameters for the TS and the ratio O<sub>train</sub>:O<sub>test</sub>.
<table><caption>**Table 4.** Optimized results of RMSE analysis using parameter *C*</caption><thead><tr><th>TS</th><th>O<sub>train</sub>:O<sub>test</sub></th><th>C</th><th>ε</th><th>γ</th><th>RMSE<sub>train</sub></th><th>RMSE<sub>test</sub></th></tr></thead><tbody><tr><td>S</td><td>50:50</td><td>8</td><td>0.2</td><td>0.4</td><td>0.343</td><td>0.414</td></tr><tr><td>S</td><td>66:34</td><td>10</td><td>0.1</td><td>0.4</td><td>0.365</td><td>0.314</td></tr><tr><td>S</td><td>80:20</td><td>10</td><td>0.1</td><td>0.4</td><td>0.355</td><td>0.306</td></tr><tr><td>I</td><td>50:50</td><td>10</td><td>0.1</td><td>0.4</td><td>0.331</td><td>0.369</td></tr><tr><td>I</td><td>66:34</td><td>10</td><td>0.1</td><td>0.4</td><td>0.342</td><td>0.361</td></tr><tr><td>I</td><td>80:20</td><td>9</td><td>0.1</td><td>0.4</td><td>0.350</td><td>0.336</td></tr><tr><td>L</td><td>50:50</td><td>6</td><td>0.1</td><td>0.2</td><td>0.382</td><td>0.409</td></tr><tr><td>L</td><td>66:34</td><td>4</td><td>0.1</td><td>0.2</td><td>0.383</td><td>0.416</td></tr><tr><td>L</td><td>80:20</td><td>4</td><td>0.1</td><td>0.2</td><td>0.384</td><td>0.445</td></tr></tbody></table>

<table><caption>**Table 5.** Optimized results of RMSE analysis using parameters γ and β</caption><thead><tr><th>TS</th><th>O<sub>train</sub>:O<sub>test</sub></th><th>C</th><th>ε</th><th>γ</th><th>β</th><th>RMSE<sub>train</sub></th><th>RMSE<sub>test</sub></th></tr></thead><tbody><tr><td>S</td><td>50:50</td><td>10</td><td>0.1</td><td>0.2</td><td>1</td><td>0.411</td><td>0.319</td></tr><tr><td>S</td><td>66:34</td><td>10</td><td>0.2</td><td>0.2</td><td>2</td><td>0.474</td><td>0.379</td></tr><tr><td>S</td><td>80:20</td><td>10</td><td>0.1</td><td>0.2</td><td>3</td><td>0.587</td><td>0.509</td></tr><tr><td>I</td><td>50:50</td><td>6</td><td>0.1</td><td>0.3</td><td>1</td><td>0.385</td><td>0.416</td></tr><tr><td>I</td><td>66:34</td><td>10</td><td>0.1</td><td>0.3</td><td>2</td><td>0.436</td><td>0.486</td></tr><tr><td>I</td><td>80:20</td><td>6</td><td>0.1</td><td>0.3</td><td>3</td><td>0.526</td><td>0.567</td></tr><tr><td>L</td><td>50:50</td><td>10</td><td>0.1</td><td>0.2</td><td>2</td><td>0.455</td><td>0.468</td></tr><tr><td>L</td><td>66:34</td><td>6</td><td>0.1</td><td>0.2</td><td>1</td><td>0.385</td><td>0.416</td></tr><tr><td>L</td><td>80:20</td><td>8</td><td>0.1</td><td>0.2</td><td>1</td><td>0.385</td><td>0.443</td></tr></tbody></table>

The original value of TS<sup>S</sup> (O<sub>test</sub>) *y* compared with predicted values of *y*′ (in which time is *t*+Δ*t*, Δ*t*=1 day) using RBF neural network is displayed in Fig. 9. In Fig. 10 TS<sup>S</sup> (O<sub>test</sub>) *y* is then compared to predicted values *y*′ using the ε-SVR (with RBF kernel function).

<figure><img src="image_8.png" /><figcaption>**Fig. 9.** TS<sup>S</sup> with values predicated by RBF</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>**Fig. 10.** TS<sup>S</sup> with values predicated by SVR</figcaption></figure>

In Table 6, there is a comparison of the RMSE<sub>train</sub> and RMSE<sub>test</sub> on the training and testing set to other designed and analyzed structures. For example, it was used a fuzzy inference system (FIS) Takagi-Sugeno [5,6], intuitionistic fuzzy inference system (IFIS) Takagi-Sugeno [5,6], feed-forward neural networks (FFNNs), RBF neural networks and ε-SVR<sup>1</sup> (ε-SVR<sup>2</sup>) with RBF (polynomial kernel function) with pre-processing input data by simple mathematical statistical methods.

<table><caption>**Table 6.** Comparison of the RMSE<sub>train</sub> and RMSE<sub>test</sub> on the training and testing data to other designed and analyzed structures of fuzzy inference systems and neural networks</caption><thead><tr><th></th><th>FIS</th><th>IFIS</th><th>FFNN</th><th>RBF</th><th>ε-SVR<sup>1</sup></th><th>ε-SVR<sup>2</sup></th></tr></thead><tbody><tr><td>RMSE<sub>train</sub></td><td>0.221</td><td>0.224</td><td>0.593</td><td>0.311</td><td>0.331</td><td>0.385</td></tr><tr><td>RMSE<sub>test</sub></td><td>0.237</td><td>0.239</td><td>0.687</td><td>0.408</td><td>0.369</td><td>0.416</td></tr></tbody></table>
# 5 Conclusion

The proposed model consists of data pre-processing and actual prediction using RBF neural networks as well as ε-SVR with polynomial and RBF kernel functions. Furthermore, the modelling was done for time series of different lengths and different parameters of neural networks. The analysis results for various ratios of $O_{train}:O_{test}$ show trends for RMSE$_{train}$ and RMSE$_{test}$. From the analysis of all obtained results of modelling time series by RBF neural network (ε-SVR) shows that RMSE$_{test}$ takes minimum values [4,10,11] for $TS^1$ ($TS^5$). Further direction of research in the area of modelling web domain visits (excluding the use of uncertainty [5,6] and modelling using RBF neural networks and machine learning using ε-SVR) is focused on different structures of neural networks. The crux of modelling are different lengths of time series, various ratios of $O_{train}:O_{test}$ and different techniques of their partitioning.

Prediction using web mining gives better characterization of webspace. On the basis of that system engineers can better characterize the load of complex virtual system and its dynamics. The RBF neural networks (ε-SVR) design was carried out in SPSS Clementine (STATISTICA).

# Acknowledgments.

This work was supported by the scientific research project of Ministry of Environment, the Czech Republic under Grant No: SP/4i2/60/07.

# References

[1] Cooley, R., Mobasher, B., Srivistava, J.: Web Mining: Information and Pattern Discovery on the World Wide Web. In: 9th IEEE International Conference on Tools with Artificial Intelligence, ICTAI '97, Newport Beach, CA (1997)

[2] Krishnapuram, R., Joshi, A., Yi, L.: A Fuzzy Relative of the K-medoids Algorithm with Application to Document and Snippet Clustering. In: IEEE International Conference on Fuzzy Systems, pp.1281-1286. Korea (1999)

[3] Pedrycz, W.: Conditional Fuzzy C-means. Pattern Recognition Letters, 17, 625-632 (1996)

[4] Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice-Hall Inc., New Jersey (1999)

[5] Olej, V., Hájek, P., Filipová, J.: Modelling of Web Domain Visits by IF-Inference System. WSEAS Transactions on Computers, Issue 10, 9, 1170-1180 (2010)

[6] Olej, V., Filipová, J., Hájek, P.: Time Series Prediction of Web Domain Visits by IF-Inference System. In: Proc. of the 14th WSEAS International Conference on Systems, Latest Trends on Computers, N. Mastorakis at al. (eds.), Vol.1, Greece, pp.156-161 (2010)

[7] Broomhead, D.S., Lowe, D.: Multivariate Functional Interpolation and Adaptive Networks. Complex Systems, 2, 321-355 (1988)

[8] Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)

[9] Smola, A., Scholkopf, J.: A Tutorial on Support Vector Regression. Statistics and Computing, 14, 199-222 (2004)

[10] Niyogi, P., Girosi, F.: On the Relationship between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Functions. Massachusetts Institute of Technology Artificial Intelligence Laboratory, Massachusetts (1994)

[11] Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer-Verlag New York (1995)