# Optimal Joint Call Admission Control with Vertical Handoff on Heterogeneous Networks

Diego Pacheco-Paramo, Vicent Pla, Vicente Casares-Giner, Jorge Martinez-Bauset

► To cite this version:

Diego Pacheco-Paramo, Vicent Pla, Vicente Casares-Giner, Jorge Martinez-Bauset. Optimal Joint Call Admission Control with Vertical Handoff on Heterogeneous Networks. 10th IFIP Networking Conference (NETWORKING), May 2011, Valencia, Spain. pp.121-134, 10.1007/978-3-642-20798-3_10. hal-01597976

HAL Id: hal-01597976

https://inria.hal.science/hal-01597976

Submitted on 29 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Optimal Joint Call Admission Control with Vertical Handoff on Heterogeneous Networks

Diego Pacheco-Paramo, Vicent Pla, Vicente Casares-Giner, and Jorge Martinez-Bauset

Dept. Comunicaciones, Universidad Politecnica de Valencia, UPV
Camino de Vera s/n, 46022, Valencia, Spain
{diepacpa@posgrado.upv.es, vpla@dcom.upv.es, vcasares@dcom.upv.es}
jmartinez@dcom.upv.es

**Abstract.** A joint management of radio resources in heterogeneous networks is considered to raise efficiency. In this paper we focus on joint schemes for admission control and access technology selection using vertical handoff. We study a system where both streaming and elastic traffic are supported, and two radio access technologies covering the same areas are available: TDMA and WCDMA. Optimal solutions for different optimization criteria which can be expressed in terms of blocking probabilities and throughput are found and properly described, and a new heuristic policy is proposed. Finally, in order to provide some understanding of the cost of a single vertical handoff, a new optimization criteria is defined where its cost is found in relation to that of voice and data blocking.

**Keywords:** Heterogeneous networks; Markov decision process; Joint call admission control.

## 1 Introduction

Coexistence of several access technologies demands the study of joint radio resource management (JRRM), given the objective of providing users with a permanent connection with the best possible performance [1]. Since coordinated control of several radio access technologies improves the use of limited resources [2], a challenge that arises is to find the most appropriate scheme for the main functions of JRRM, such as joint call admission control (JCAC) and vertical handoff (VH). Joint call admission control (JCAC) proposals such as those in [3] and [4], show the improvement in network's stability based on a load balancing scheme for a heterogeneous network, but they do not make use of the advantages of vertical handoff. The complexity that vertical handoff adds to the system is commented in [5] and several vertical handoff characteristics and algorithms are reviewed in [6]. Most of vertical handoff schemes like [7,8] base decisions on user preferences. We believe that the study of vertical handoff decision from the operator's point of view is a constructive and important issue.

In this article, we are interested in finding an optimal scheme for a heterogeneous network with two technologies, and where both streaming and elastic
traffic are supported. In [9] a Markov model is used for an heterogeneous net-
work (WLAN and CDMA), and a linear programming method is used to solve
the optimization problem. That paper explicitly acknowledges the inherent com-
plexity of the problem, which could make it computationally intractable for large
systems. This is a main concern in this work. Therefore, we have simplified some
characteristics of the system in such a way that results can be extrapolated to
more complex systems. Several solution methods for similar problems have been
used, such as genetic algorithms [10] or fuzzy-based solutions [11], [12]. In this
work, policy iteration is used since it allows to discern the main characteristics
of the optimal policies in a per iteration basis as several parameters vary.

The last part of the article is focused on the cost of a single vertical handoff, and how it relates to other costs such as those of blocking voice or data calls. Other solutions define a monetary cost of vertical handoff, and users decide according to this and other parameters [13]. In contrast, our interest is to explore the impact of this cost from the operator's point of view in relation to other performance parameters such as the blocking probabilities. The paper is structured as follows. In Section 2 we describe the Markov model of the system and the types of vertical handoff used. The solution method is described in Section 3. In Section 4 the optimal policies are analyzed and some heuristics are proposed and compared with optimal solutions. A study of the cost of performing a vertical handoff in relation to those of blocking voice and data calls is done in Section 5. Finally, Section 6 concludes the work.

## 2 System description and Markov model

The system of our interest is composed by two radio access technologies, TDMA and WCDMA. Additionally, both technologies provide voice and data services in the same area as it was proposed in [14]. As a call arrives to the system, a decision has to be made about if it is served in one or another technology.

Both service users (voice and data) are defined by a Poisson arrival process, with rate $\lambda_v(\lambda_d)$ for voice (data). The service time for voice is exponentially distributed with mean $1/\mu_v$. On the other hand, because data is best modelled as elastic traffic, its sojourn times will depend on the available resources. Therefore, the mean service time for data is exponentially distributed with mean $1/F(\sigma)$, where $\sigma$ is the average size of the information sent and $F$ refers to its division by the transmission rate.

## 2.1 State space

The state vector of the continuous time Markov chain (CTMC), is $\boldsymbol{s} = (s_1, s_2, s_3, s_4)$ where $s_1$ represents the number of ongoing voice sessions on TDMA, $s_2$ the data sessions on TDMA, $s_3$ the voice sessions on WCDMA and $s_4$ the data sessions on WCDMA. We define $\boldsymbol{C}$ as the fixed number of channels per slot in TDMA. A voice session will always use a whole channel, so there can only be $\boldsymbol{C}$ simultaneous voice sessions on this technology. On the other hand, data sessions can share
a channel when TDMA is at full capacity, in such a way that $n_c$ data sessions can be served per channel. This means that we can have a maximum of $C \cdot n_c$ simultaneous data sessions in TDMA. According to this, the first condition that a state must fulfill to be feasible is given by

$$s_1 \cdot n_c + s_2 \leq n_c \cdot C. \quad (1)$$

The capacity on WCDMA is defined by

$$s_3 \left( \frac{W/BR_{w,v}}{(E_b/N_0)_v} + 1 \right)^{-1} + s_4 \left( \frac{W/BR_{w,d}}{(E_b/N_0)_d} + 1 \right)^{-1} \leq \eta_{ul}, \quad (2)$$

where $W$ is the chip rate, $BR_{w,x}$ is the bit rate used for transmitting service $x$ in WCDMA, $(E_b/N_0)_x$ is the bit energy to noise density required for service $x$, and $\eta_{ul}$ is the uplink cell load factor. This is the same expression used in [15]. Considering that each technology has independent resources, the feasible combination of data and voice users can be calculated for each technology separately. We define $S$ as the set of feasible states, that is all the state vectors $s$ that fulfill the conditions defined in (1) and (2).

## 2.2 System Metrics

The parameters that define performance are the voice blocking probability, the data blocking probability, and the total throughput. The total blocking probability refers to the probability that the system is in a state where all calls are blocked, independent of the service it requires. In the same way the voice (data) blocking probability refers to the probability of being in those states where voice (data) calls are blocked. To calculate the throughput we have to consider that the bit rate is independent for each service and technology and that data sessions in TDMA can share a channel, which is reflected in the $min(C - s_1, s_2)$ factor of the following equation:

$$Th = \sum_{s \in S} P(s) (s_1 BR_{t,v} + s_3 BR_{w,v} + \min(C - s_1, s_2) BR_{t,d} + s_4 BR_{w,d}), \quad (3)$$

where $BR_{x,y}$ is the bit rate used for transmitting service $y$ (voice or data) in technology $x$ (TDMA or WCDMA).

## 2.3 Vertical Handoff

Inferred from the optimal policies obtained in [16], four vertical handoff types are defined:
<table><thead><tr><th>Type</th><th>Triggering Event</th><th>Conditions</th><th>Number/ Class of calls</th><th>From ⇒ To</th><th>VH-A</th><th>VH-B</th></tr></thead><tbody><tr><td>I</td><td>voice <b>arrival</b></td><td>Full occupation on WCDMA</td><td><b>N</b>* <b>data</b> calls</td><td>WCDMA⇒TDMA</td><td>✓</td><td>✓</td></tr><tr><td>II</td><td>data <b>arrival</b></td><td>Channel sharing on TDMA</td><td>One <b>voice</b> call</td><td>TDMA⇒WCDMA</td><td>✓</td><td>✓</td></tr><tr><td>III</td><td>voice or data <b>departure</b> from WCDMA</td><td>Channel sharing on TDMA</td><td>One <b>voice</b> call</td><td>TDMA⇒WCDMA</td><td>—</td><td>✓</td></tr><tr><td>IV</td><td>voice or data <b>departure</b> from TDMA</td><td>VH does not produce channel sharing on TDMA</td><td>One <b>data</b> call</td><td>WCDMA⇒TDMA</td><td>—</td><td>✓</td></tr></tbody></table>

## 3 Optimization problem

For each state $s \in S$, a decision must be made about if an arriving call should be admitted according to the service required. In a Markov decision process, a policy $\pi$ defines which actions $a=(a_s^v, a_s^d)$ should be taken at each state $s$ for voice $a_s^v$ and data $a_s^d$ arrivals. It should be clear that decision epochs occur only at arrivals. The main objective is to find among the possible policies the one that optimizes a chosen function.

The set of actions $A$ defines the possible values for $a_s^v$ and $a_s^d$, and it is defined in Table 1. Table 2 shows the transition rates $r_{su}$ from state $s$ to state $u$ ($s, u \in S$).

<table><caption>**Table 1.** set of actions A for MDPs with vertical handoff</caption><thead><tr><th>value</th><th>action</th></tr></thead><tbody><tr><td>0</td><td>Block call</td></tr><tr><td>1</td><td>Send call to TDMA</td></tr><tr><td>2</td><td>Send call to WCDMA</td></tr><tr><td>3</td><td>Vertical Handoff type I</td></tr><tr><td>4</td><td>Vertical Handoff type II</td></tr></tbody></table>

<footnote>* N is the necessary number of moved data calls such that one voice call can access WCDMA.

The system that uses vertical handoff types I and II is called VH-A, the one that uses all types is called VH-B, and the one that does not use vertical handoff is called NVH.</footnote>
<table><caption>**Table 2.** Transition rates from state **s** to **u**</caption><thead><tr><th>a<sub>s</sub><sup>v</sup></th><th>u</th><th>r<sub>su</sub></th></tr></thead><tbody><tr><td>1</td><td>s + e<sub>1</sub></td><td>&lambda;<sub>v</sub></td></tr><tr><td>2</td><td>s + e<sub>3</sub></td><td>&lambda;<sub>v</sub></td></tr><tr><td>3</td><td>s + N &middot; e<sub>2</sub> + e<sub>3</sub> - N &middot; e<sub>4</sub></td><td>&lambda;<sub>v</sub></td></tr></tbody></table><table><thead><tr><th>a<sub>s</sub><sup>d</sup></th><th>u</th><th>r<sub>su</sub></th></tr></thead><tbody><tr><td>1</td><td>s + e<sub>2</sub></td><td>&lambda;<sub>d</sub></td></tr><tr><td>2</td><td>s + e<sub>4</sub></td><td>&lambda;<sub>d</sub></td></tr><tr><td>4</td><td>s - e<sub>1</sub> + e<sub>2</sub> + e<sub>3</sub></td><td>&lambda;<sub>d</sub></td></tr></tbody></table>

## 3.1 Cost function

Since our interest relies on data and voice blocking probabilities, as well as the total throughput, we have defined two different objective functions. The first one, is the weighted sum of the voice and data blocking probabilities,

$$F_{BP} = BP_{voice} \cdot \alpha + BP_{data} \cdot (1 - \alpha). \quad (4)$$

The parameter $\alpha$, $0 \le \alpha \le 1$, is the one responsible for giving more or less weight to each blocking probability. Thus, $\alpha$ relates the way in which the blocking probabilities will be minimized. The cost function associated to the objective function for each feasible state **s** is

$$cost(s) = 1 - (\alpha \cdot F_v(a_s) + (1 - \alpha) \cdot F_d(a_s)), \quad (5)$$

where $F_v(a_s)=1$ if $a_s$ is 1 or 2, and 0 otherwise, being $x$ the service. The second objective function is the aggregated throughput, so in that case we try to maximize the value defined by (3). The reward for each state **s** is

$$cost(s) = s_1 BR_{t,v} + s_3 BR_{w,v} + \min(C - s_1, s_2)BR_{t,d} + s_4 BR_{w,d}. \quad (6)$$

## 3.2 Solution method

The method used to find the optimal policy $\pi_{opt}$ is policy iteration [17]. This method can search among the finite group of possible policies for the MDP and find the optimal in a finite number of steps. The relative values $V$ allow to relate the cost obtained in the actual state with costs expected from future actions, and are found using the next equation:

$$c^{\pi} - c_{\pi} \cdot e + V_{\pi} R_{\pi}^{T} = 0. \quad (7)$$

The $c^{\pi}$ in the previous expression is the vector of costs associated to being in each state, $R_{\pi}$ is the transition matrix, and $c_{\pi}$ is the value of the objective function for policy $\pi$. Once $V$ and $c_{\pi}$ are found using (7), it is possible to find
the action on each state that will minimize the objective function using the next expression:

$$ \min_a \{c_s^a - c_\pi + \sum_{s \neq u} r_{s,u}^a (v_\pi^u - v_\pi^s)\}, \qquad (8) $$

where $v_\pi^u$ and $v_\pi^s$ are the relative values for states $u$ and $s$ respectively when the policy $\pi$ is used, $c_s^a$ is the cost associated to state $s$ when action $a$ is taken, $r_{s,u}^a$ is the transition rate from state $s$ to state $u$, when the action for state $s$ is $a$. The set of actions will define a new policy and the process is repeated until the optimal policy is found.

## 4 Optimal Policy Analysis

To obtain and analyze the optimal solutions, the scenario that will be used unless otherwise stated is defined in Table 3. The values are chosen in order to keep the problem computationally tractable while at the same time keeping capacity proportionality among the technologies used. In this section we study MDP VH-A and MDP VH-B which are based on the systems defined in section 2.3. Since MDP VH-B performs vertical handoff types III and IV each time a departure occurs, they share the same action set $A$ of Table 1. The state space $S$ for both MDPs is defined by (1) and (2).

<table><caption>**Table 3.** Initial Scenario for Policy Iteration.</caption><thead><tr><th>WCDMA</th><th>TDMA</th></tr></thead><tbody><tr><td>W=3.84 Mcps<br>(E<sub>b</sub>/N<sub>0</sub>)<sub>v</sub>=14 dB<br>(E<sub>b</sub>/N<sub>0</sub>)<sub>d</sub>=14 dB</td><td>C = 4<br>n<sub>c</sub>= 2<br>BR<sub>t,v</sub>=12.2 kbps<br>BR<sub>t,d</sub>=44.8 kbps</td></tr><tr><td>BR<sub>w,v</sub>=12.2 kbps<br>BR<sub>w,d</sub>=44.8 kbps<br>η<sub>ul</sub>= 1</td><td></td></tr><tr><td>Clients</td><td></td></tr><tr><td>λ<sub>v</sub>= 0.025<br>λ<sub>d</sub>= 0.134<br>μ<sub>v</sub>= 0.0083<br>σ= 1 Mb</td><td></td></tr></tbody></table>

When using both optimization functions, the blocking function defined in (4) and the throughput function defined in (3), while varying $\lambda_v$ and $\lambda_d$ the main characteristics of the optimal policies for both MDPs can be summarized as follows:
<table><thead><tr><th>service</th><th>action</th></tr></thead><tbody><tr><td>voice</td><td>Send to WCDMA or use VH type I.</td></tr><tr><td>data</td><td>Send to TDMA while no sharing is needed.<br>If so, use VH type II or send to WCDMA.</td></tr></tbody></table>

However, there are differences for each case that should be analysed.

## 4.1 Blocking Function Optimization

When the blocking function is optimized and $λ_v$ varies from 0.005 to 0.095, the number of states that are left unused grows with $λ_v$ for both MDPs. For MDP VH-B it grows from 32.1% to 50.4% and for MDP VH-A grows from 10.4% to 41.7%. Therefore, MDP VH-B organizes calls in a more effective way given the impact of vertical handoff for departures. Also, there is some blocking of voice calls even when there is space left for the highest values of $λ_v$, when its influence is bigger on the optimization function. When $λ_v=0.095$, 1.2% of usable states block voice calls with MDP VH-B, while 1% do it with MDP VH-A.

When $λ_d$ grows from 0.05 to 0.225, the percentage of unused states for MDP VH-A remains constant in 41.6%, while it decreases from 49.4% to 39.6% with MDP VH-B. This decrease occurs because of vertical handoff type III, which is used more often as $λ_d$ grows. Also for high values of $λ_d$, there is some blocking of data calls even when there is space available. For $λ_d=0.225$, 2.56% of usable states block data calls for MDP VH-A and 2.98% of states do this for MDP VH-B.

## 4.2 Throughput Optimization

When the throughput is optimized and $λ_v$ grows from 0.005 to 0.095, the percentage of unused states also grows from 16% to 41.7% for MDP VH-A, but decreases from 61.2% to 49.5% for MDP VH-B. Also, there is some voice calls blocking while there is available space left when $λ_v$ is low. For $λ_v=0.005$, 0.71% of the usable states block voice calls for MDP VH-A while 0.25% of states do it for MDP VH-B. Therefore MDP VH-B manages resources in a more organized way, since the number of states left unused is always higher.

When $λ_d$ grows from 0.05 to 0.225, the percentage of unused states also grows from 41.6% to 43% for MDP VH-A and from 45.4% to 47.2% for MDP VH-B. Also, since voice calls contribute less to the total throughput, some states block voice calls when there is space left. This occurs for high values of $λ_d$. When $λ_d=0.225$, 0.7% of usable states block voice calls for MDP VH-A, while 0.56% of states do it for MDP VH-B.

## 4.3 Result Analysis for Vertical Handoff MDPs

Based on the results obtained in sections 4.1 and 4.2 we propose two new heuristic policies that make use of their main characteristics. Heuristic VH-A makes use
of vertical handoff types I and II, as it was seen for the optimal policies found using MDP VH-A. On the other hand, Heuristic VH-B also includes vertical handoff types III and IV, as it was done by the optimal policies obtained using MDP VH-B.

<table><thead><tr><th>Heuristic</th><th>Event</th><th>Action</th></tr></thead><tbody><tr><td>Heuristic</td><td>Voice arrival</td><td><ul><li>Send to WCDMA.</li><li>If it is not possible, use VH type I.</li><li>If it is not possible, send to TDMA.</li></ul></td></tr><tr><td>VH-A</td><td>Data arrival</td><td><ul><li>Send to TDMA if no channel sharing is needed.</li><li>If there is channel sharing, use VH type II.</li><li>If it is not possible, send to WCDMA.</li><li>If it is not possible, send to TDMA.</li></ul></td></tr><tr><td>Heuristic</td><td>Voice arrival</td><td><ul><li>Send to WCDMA.</li><li>If it is not possible, use VH type I.</li><li>If it is not possible, send to TDMA.</li></ul></td></tr><tr><td rowspan="3">VH-B</td><td>Data arrival</td><td><ul><li>Send to TDMA if no channel sharing is needed.</li><li>If there is channel sharing, use VH type II.</li><li>If it is not possible, send to WCDMA.</li><li>If it is not possible, send to TDMA.</li></ul></td></tr><tr><td>Voice departure</td><td><ul><li>Use VH type III if departs from WCDMA</li><li>Use VH type IV if departs from TDMA</li></ul></td></tr><tr><td>Data departure</td><td><ul><li>Use VH type III if departs from WCDMA</li><li>Use VH type IV if departs from TDMA</li></ul></td></tr></tbody></table>

In this section we compare the results obtained by the two new heuristic policies with those of the MDPs that use vertical handoff and the one that does not (MDP NVH). In Fig. 1(a) it is shown the blocking function for the MDPs and the two heuristic policies when $\lambda_v$ varies from 0.005 to 0.095. It is evident that the use of vertical handoff enhances the system behavior, since both heuristic have a lower optimal value for all $\lambda_v$. Also, if we restrict $BP_v$ and $PB_d$ below 2%, $\lambda_v$ can grow up to 0.07 for Heuristic VH-A, while for MDP NVH the maximum $\lambda_v$ was of 0.062, a difference of about 1 Erl. However, it is difficult to notice the effect of vertical handoff for service completion in Fig. 1(a), since the improvement is not as pronounced as one could expect.

In Fig. 1(b) is shown the blocking function as $\lambda_d$ varies from 0.05 to 0.225. Again, the performance of the heuristic policies is better than that of MDP NVH. When $\lambda_d=0.225$, the value of the blocking function for the heuristics VH-A and VH-B is very similar, and 85 % of the value obtained by MDP NVH. As $\lambda_d$ grows, the difference between the solution found by the MDPs with vertical handoff differ more from the heuristics and from the optimal solution found using MDP NVH. This is due to the rise of $PB_v$ on the heuristic policies, which is managed by the optimal policies by creating some data blocking states.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Blocking Function Optimization.</figcaption></figure>

When optimizing throughput, there is only a significant improvement of the solutions with vertical handoff over MDP NVH for the highest values of $\lambda_v$ as can be seen on Fig. 2(a). When $\lambda_v=0.095$, the throughput of the policies that use vertical handoff are very similar among them and above the value of MDP NVH for about 1.7%. This improvement represents about 44.3 kbps, almost 1 data channel or 4 voice channels. However, for this value of $\lambda_v$, $PB_v$ and $PB_d$ are way above acceptable levels, with 7.5% and 3.8% respectively for the solution of MDP VH-B, the one with the best results. Also, vertical handoff improves $PB_v$ and $PB_d$. Restricting these values to 2%, the maximum acceptable $\lambda_v$ for MDP VH-A is around 0.068, while for MDP NVH is of 0.064. This clearly indicates that the rearrangement done by vertical handoff improves the throughput by raising the maximum load capacity of the system.

In Fig. 2(b) appears the throughput for the same policies as $\lambda_d$ varies from 0.05 to 0.225. In this case, the improvement in throughput of the new policies over MDP NVH is very low. Also, the values for $PB_d$ are very similar, showing how difficult it is for the system to deal with high rates of $\lambda_d$. However, there is a significant difference in the values obtained for $PB_v$. When $PB_v$ reaches 2% for the solution obtained with MDP NVH, the value of Heuristic VH-A is of 0.5%, so there is a considerable reduction. This result also shows that the improvement on the throughput as $\lambda_d$ grows is given by voice calls, which contribute less to the total throughput given their data rate.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Throughput Optimization.</figcaption></figure>

# 5 Cost of the Vertical Handoff

In this section, a new objective function based on the blocking rates and the vertical handoff rate is defined as:

$$F_{VH} = \theta \cdot \zeta_{VB} + (1 - \theta) \cdot \zeta_{DB} + C_{VH} \cdot \zeta_{VH}, \quad (9)$$

where $\theta$, $0 \le \theta \le 1$, is the factor that defines the cost for blocking a single voice or data call, and $\zeta_{VB}$ and $\zeta_{DB}$ are the mean voice and data blocking rates. In the same way, $C_{VH}$ is the cost of performing a single vertical handoff, and $\zeta_{VH}$ is the mean rate of vertical handoffs performed. Hence, by assigning values to $\theta$ and $C_{VH}$, a new optimal policy that minimizes $F_{VH}$ can be found using a Markov decision process.

## 5.1 Markov decision process for VH cost

In this section we define three new MDPs based on (9). The first one does not use vertical handoff (MDP BR), the second one uses vertical handoff types I and II (MDP C-1), and the last one uses all four types of vertical handoff (MDP C-2). These MDPs are different from those exposed before because of their objective functions. The state space for all of them $S$ is defined by (1) and (2) and the set of actions $A$ is that of Table 1 for MDP C-1 and MDP C-2, while the set of actions for MDP BR does not include actions 3 and 4 from $A$ of Table 1. The cost function associated to the objective function for each feasible state $s$ for MDP BR is

$$cost(s) = \lambda_v \cdot G(a_s^v) \cdot \theta + \lambda_d \cdot G(a_s^d) \cdot (1 - \theta), \quad (10)$$
for MDP C1 is

$$ cost(s) = \lambda_v \cdot G(a_s^v) \cdot \theta + \lambda_d \cdot G(a_s^d) \cdot (1 - \theta) + C_{VH} \cdot (\lambda_v \cdot R(a_s^v) + \lambda_d \cdot R(a_s^d)), \quad (11) $$

and for MDP C2 is

$$ cost(s) = \lambda_v \cdot G(a_s^v) \cdot \theta + \lambda_d \cdot G(a_s^d) \cdot (1 - \theta) + C_{VH} \cdot (\lambda_v \cdot R(a_s^v) + \lambda_d \cdot R(a_s^d)) \\ + C_{VH} \cdot (T(s_{TDMA}^v) + T(s_{TDMA}^d) + T(s_{WCDMA}^v) + T(s_{WCDMA}^d)), \quad (12) $$

where the coefficients are explained in Table 4.

<table><caption>**Table 4.** Coefficients for optimization function</caption><thead><tr><th>SYMBOL</th><th>DEFINITION</th><th>VALUE</th></tr></thead><tbody><tr><td>G(a<sub>s</sub><sup>x</sup>)</td><td>Indicates if <i>s</i> is a blocking state for service <i>x</i></td><td><ul><li>If action a<sub>s</sub><sup>x</sup> is blocking, G(a<sub>s</sub><sup>x</sup>)=1.</li><li>Otherwise G(a<sub>s</sub><sup>x</sup>)=0.</li></ul></td></tr><tr><td>R(a<sub>s</sub><sup>x</sup>)</td><td>Indicates the number of calls that suffer vertical handoff when a call of service <i>x</i> arrives while the system is on state <i>s</i>.</td><td><ul><li>If a<sub>s</sub><sup>x</sup>=3, and all the conditions for vertical handoff type I are fulfilled, R(a<sub>s</sub><sup>v</sup>) = N.</li><li>If a<sub>s</sub><sup>x</sup>=4, and all the conditions for vertical handoff type II are fulfilled, R(a<sub>s</sub><sup>d</sup>) = 1.</li><li>Otherwise, R(a<sub>s</sub><sup>x</sup>) = 0.</li></ul></td></tr><tr><td>T(s<sub>TDMA</sub><sup>v</sup>)</td><td>Indicates the rate of calls that suffer vertical handoff when a voice call is served on TDMA while the system is on state <i>s</i>.</td><td><ul><li>If the conditions for vertical handoff type IV are fulfilled once a voice call is served on TDMA, T(s<sub>TDMA</sub><sup>v</sup>) = s<sub>1</sub> &middot; &mu;<sub>v</sub>.</li><li>Otherwise T(s<sub>TDMA</sub><sup>v</sup>) = 0.</li></ul></td></tr><tr><td>T(s<sub>TDMA</sub><sup>d</sup>)</td><td>Indicates the rate of calls that suffer vertical handoff when a data call is served on TDMA while the system is on state <i>s</i>.</td><td><ul><li>If the conditions for vertical handoff type IV are fulfilled once a data call is served on TDMA, T(s<sub>TDMA</sub><sup>d</sup>) = min(C-s<sub>1</sub>,s<sub>2</sub>) &middot; BR<sub>t,d</sub>/&sigma;.</li><li>Otherwise T(s<sub>TDMA</sub><sup>d</sup>) = 0.</li></ul></td></tr><tr><td>T(s<sub>WCDMA</sub><sup>v</sup>)</td><td>Indicates the rate of calls that suffer vertical handoff when a voice call is served on WCDMA while the system is on state <i>s</i>.</td><td><ul><li>If the conditions for vertical handoff type III are fulfilled once a voice call is served on WCDMA, T(s<sub>WCDMA</sub><sup>v</sup>) = s<sub>3</sub> &middot; &mu;<sub>v</sub>.</li><li>Otherwise T(s<sub>WCDMA</sub><sup>v</sup>) = 0.</li></ul></td></tr><tr><td>T(s<sub>WCDMA</sub><sup>d</sup>)</td><td>Indicates the rate of calls that suffer vertical handoff when a data call is served on WCDMA while the system is on state <i>s</i>.</td><td><ul><li>If the conditions for vertical handoff type III are fulfilled once a data call is served on WCDMA, T(s<sub>WCDMA</sub><sup>d</sup>) = s<sub>4</sub> &middot; BR<sub>w,d</sub>/&sigma;.</li><li>Otherwise T(s<sub>WCDMA</sub><sup>d</sup>) = 0.</li></ul></td></tr></tbody></table>
## 5.2 Results Analysis

In this section, policy iteration is used to solve the MDPs. The reference scenario is defined in Table 3 with $\theta= 0.5$, which means that the cost of blocking voice and data calls is the same.

In Fig. 3 we can see the optimal values for each of the MDPs introduced in the last section for different values of $C_{VH}$. MDP C2 has the lowest optimal value when $C_{VH}=0$ and is followed closely by MDP C1. However, as $C_{VH}$ grows, the value of MDP-R2 rapidly grows surpassing MDP C1 and MDP BR because of the large amount of vertical handoff of types III and IV performed. In fact, for values of $C_{VH}$ as low as 0.01, the optimal value of MDP C2 is higher than that of MDP BR. This means that when $C_{VH}$ is higher than this, it is not justified to use the policy this MDP uses. Keeping in mind that the cost of blocking voice and data calls is of 0.5, we could say that it is necessary that the cost of blocking voice and data calls to be about 50 times higher than that of vertical handoff for this policy to be useful.

On the other hand, results found using MDP C1 vary slowly and are limited by those of MDP BR. This occurs because if it is the vertical handoff which causes the objective function to grow, MDP C1 can always choose not to use it, and then its optimal policy will be identical to that of MDP BR. The interesting point here is to find that value of $C_{VH}$ that makes both MDPs to reach the same optimal policy and therefore the same results. For Fig. 3, when $C_{VH}=0.9$, both policies are the same. This means that when the cost of vertical handoff is about 1.8 times higher than that of blocking voice or data calls, vertical handoff is not useful anymore.

It is interesting to notice that when $C_{VH}=0$, the optimal policies of MDP C1 and MDP C2 are very similar to those of MDP VH-A and MDP VH-B, even though the objective functions are different. Therefore, the values of parameters such as throughput and blocking probabilities ($BP_v$ and $BP_d$) are similar as well. As $C_{VH}$ grows, the performance of these parameters degrade, and this happens at a faster rate for MDP C2 than for MDP C1. Hence, while the ratio of voice/data blocking cost to $C_{VH}$ is high, one could expect that the heuristic policies VH-A and VH-B will fairly represent the optimal policies. This last remark is true even for a higher rank of ratios in the case of MDP C1 and the heuristic VH-A, for the reasons explained earlier.

## 6 Conclusions

We have studied the optimal joint call admission control policy with vertical handoff in a system with heterogeneous technologies (WCDMA and TDMA) and services (voice and data). Two optimization functions were used, a blocking function and a throughput function. Four vertical handoff types are defined according to the service and the event that triggers them. Some simplifications such as technologies' capacity or coverage areas were restricted in order to obtain computational tractability. However, the solutions found apply for bigger systems and are useful as a starting point for more complex problems where areas do
<figure><img src="image_4.png" /><figcaption>Fig. 3. Optimization values for various $C_{VH}$.</figcaption></figure>

not always overlap or more networks are available. The analysis of the optimal policies allowed us to define new heuristic solutions that are simple enough as it is required, while at the same time improve the performance of other schemes that do not use vertical handoff. At the same time, it was shown how heuristic solutions perform very close to the optimal solutions for a wide range of arrival rates and both optimization criteria.

Finally, a study that combines the cost of blocking voice and data calls to that of performing vertical handoff was done. It should be noted that this is a very important topic when resource availability and utilization efficiency are main concerns, that is, the optimization and decisions are made from the operator's point of view, whose needs are important to understand. We concluded that only certain types of vertical handoff are useful, and only while the ratio blocking cost/handoff cost is between a specific range.

## Acknowledgements

This work was supported by the Spanish Government, the European Commission and Universidad Politécnica de Valencia through projects TIN2010-21378-C02-02, TIN2008-06739-C04-02/TSI and PAID-06-09.

## References

1. Wu L., Sandrasegaran K.: A survey on common radio resource management. The 2nd conference on Wireless Broadband and U-Wideband Communications, AusWireless. (2007)
2. Tolli A., Hakalin P., Holma H.: Performance Evaluation of Common Radio Resource Management. IEEE International conference on Communications (ICC). Vol.5, 3429-3433 (2002)

3. Pillekeit A., Derakhshan F., Jugl E., Mitschele-Thiel A.: Force-based Load Balancing in Co-located UMTS/GSM Networks. IEEE 60th VTC 2004-Fall. 4402-4406. (2004)

4. Suleiman K., Chan A., Dlodlo M.: Load Balancing in the call admission control of heterogeneous wireless networks. International Wireless Communications and Mobile Computing Conference 245-250. (2006)

5. Falowo O.E., Chan H.A.: Joint Call Admission Control Algorithms: Requirements, Approaches, and Design Considerations. Vol 31, 1200-1217 (2007)

6. Márquez-Barja J., Calafate C.T., Cano J-C., Manzoni P.: An Overview of Vertical Handover Techniques: Algorithms, Protocols and Tools. Computer Communications. (2010)

7. Stevens-Navarro E., Wong V.: Comparison between Vertical Handoff Decision Algorithms for Heterogeneous Wireless Networks. Vehicular Technology Conference. VTC 2006-Spring. IEEE 63rd. 947-951.(2006)

8. Goyal P., Saxena S.K.: A Dynamic Decision Model for Vertical Handoffs across Heterogeneous Wireless Networks. World Academy of Science, Engineering and Technology, Vol.31 (July 2008)

9. Yu F., Krishnamurthy V.: Optimal Joint Session Admission Control in Integrated WLAN and CDMA Cellular Networks with Vertical Handoff. IEEE transactions on Mobile Computing. Vol 6. 126-139. (2007)

10. Karabudak D., Hung C., Bing B. A call admission control scheme using genetic algorithms. 1151-1158. (2004)

11. Agusti R., Sallent O., Perez-Romero J., Giupponi L.: A fuzzy-neural based approach for joint radio resource management in a beyond 3G framework. 1st International Conference on Quality of Service in heterogeneous in Wire/Wireless networks. 216-224. (2004)

12. Falowo O.E., Chan H.A.: Fuzzy Logic Based Call Admission Control for Next Generation Wireless Networks. 3rd International Symposium on Wireless Communication Systems. 574-578 (2008)

13. Hasswa A., Nasser N., Hassanein H.: Generic Vertical Handoff Decision Function for Heterogeneous Wireless Networks. 2nd IFIP International Conference on Wireless and Optical Communications Networks.239-243(2005)

14. Gelabert X., Perez-Romero J., Sallent O., Agusti R.: A Markovian Approach to Radio Access Technology Selection in Heterogeneous Multiaccess/Multiservice Wireless Networks. IEEE transactions on mobile computing. Vol 7. Number 10. (2008)

15. Gilhousen K., Jacobs I., Padovani R., Viterbi A., Weaver L., Wheatley C.: On the Capacity of a Cellular CDMA System. IEEE transactions on vehicular technology. Vol 40. Number 2. (1991)

16. Pacheco-Paramo D., Pla V., Casares-Giner V., Martinez-Bauset J.: Optimal Radio Access Technology Selection on Heterogeneous Networks. http://personales.upv.es/diepacpa/HetNet.pdf

17. Bertsekas, D.P.: Dynamic Programming and Optimal Control. Athena Scientific. (2001)