# Design and Verification of Secure Mutual Authentication Protocols for Mobile Multihop Relay WiMAX Networks against Rogue Base/Relay Stations

Authors: Jie Huang; Chin-Tser Huang
Journal: Journal of Electrical and Computer Engineering
 (2016)

---

## Abstract

Mobile multihop relay (MMR) WiMAX networks have attracted lots of interest in the wireless communication industry recently because of its scalable coverage, improved data rates, and relatively low cost. However, security of MMR WiMAX networks is the main challenge to be addressed. In this paper, we first identify several possible attacks on MMR WiMAX networks in which a rogue base station (BS) or relay station (RS) can get authenticated and gain control over the connections and show that the current standard does not address this problem well. We then propose a set of new authentication protocols for protecting MMR WiMAX networks from rogue BS attack, rogue RS attack, and suppress-replay attack. Our protocols can provide centralized authentication by using a trusted authentication server to support mutual authentication between RS and BS, between RS and RS, and between mobile station (MS) and RS. Moreover, our protocols can also provide distributed authentication with a license issued by the trusted server. We use a formal tool called Scyther to analyze and verify the security properties of our protocols. The results show that our protocols can counter rogue BS and RS attacks and suppress-replay attack and are not susceptible to any known attacks.


---

## Body

## 1. Introduction

In the era of the Internet of Things (IoT) when every device is connected to the Internet, one of the most important technical enablers is the wireless technologies and infrastructure that make connecting different things (devices) possible. Among them, WiMAX (Worldwide Interoperability for Microwave Access) plays a very important role as it delivers lower-cost, longer-range, and high-bandwidth mobile broadband access for mobile clients and devices to connect to the Internet from anywhere at any time.WiMAX is a broadband wireless access technology designed for the advancement of IEEE 802.16 standard [1]. It is considered to be a replacement for WiFi-based mobile broadband connection, because of its better coverage and faster speed. WiMAX is ideal for high data-rate IP applications such as video conferencing, VoIP, online gaming, and HD video streaming. Another wireless broadband technology, LTE (Long Term Evolution) [1], can also provide high-speed data transmission for mobile phones and data terminals and has been adopted by many cellular service providers. In recent years, some people think that LTE has won over WiMAX in the standard war. However, according to the report from Intel Capital [2], WiMAX does not fade away and is still considered a very good option by many service providers, including Egyptian telecom startup, Orascom Telecom, and Netherlands startup Enertel Holding [2].In the early days when WiMAX was designed, WiMAX faced the paradox that increasing data rate will reduce reliability, and increasing minimum reliability service will reduce the coverage area [3]. One possible solution is to deploy more base stations (BSs) closely, but the high cost of deploying BSs will give away the original economic competitiveness of WiMAX. Therefore researchers switched to a more viable approach, which is to insert relatively cheaper fixed relay stations into the cell. This kind of networks is called multihop relay networks. In June 2009, the IEEE 802.16 Relay Task Group (TG) proposed the IEEE 802.16j-2009 amendment, whose main purpose is to expand the previous single-hop 802.16 standard to include multihop capabilities [4], enable the operations of multihop communications based on relay stations (RSs), specify the mobile multihop relay (MMR) deployment, and define two new types of elements, the multihop relay base station (MR-BS) and the relay station (RS). In 2012, 802.16 Working Group announced the latest version of the standard, IEEE 802.16-2012 [5], which incorporated 802.16j along with two other amendments, 802.16h and 802.16m.Security has been an open challenge in WiMAX since its commencement because of the open-air nature of wireless communications. To overcome the potential attacks to WiMAX, IEEE 802.16 standard specifies a security sublayer in the MAC (Media Access Control) layer. IEEE 802.16-2009 offers an improved authentication and authorization mechanism compared to its previous versions such as IEEE 802.16-2004. IEEE 802.16-2009 provides better encryption methods, more secure key management protocol, and an EAP-based authentication strategy. However, in an MMR WiMAX network, more security issues are exposed since messages have to be transmitted through one or more relay stations, which makes it more difficult to ensure the authenticity of messages and devices involved in the transmission. Therefore the latest standard IEEE 802.16-2012 defines an air interface between an MR-BS and a RS with the following additional security functionalities [5]:(i)
Trust within a certain cell, that is, MR-BS and a group of RSs in the MR cell maintain a set of trusted relationships, called Security Zone, in order to satisfy requirements of multihop relay system operations.
(ii)
Centralized security control in MR-BS, that is, MR-BS is in charge of the generation of the security association materials between RS and MR-BS.
(iii)
Transparency for mobile station (MS) connected to the network through one or more RSs, that is, any intermediate RS does not try to decrypt the user data or authenticate the MAC management message it receives from the MS but simply relays it to the next node.
(iv)
One RS does not have Authorization Key (AK) security context of any other RS.
(v)
The intermediate RS authenticates management messages it receives from other RSs using relay-specific shard keys.
(vi)
Protection of nonauthenticated Pairwise Master Key (PKM) messages by MR-BS and the access RS, that is, any nonauthenticated PKM messages which are transmitted between MS and MR-BS through the access RS will be protected by the HMAC/CMAC based on the shared security associations established between MR-BS and the access RS.However, even with these additional functionalities, researchers in [6] found that the security mechanism in IEEE 802.16j (part of current IEEE 802.16-2012 standard) is still not adequate in that it has vulnerabilities in its weak protection of some PKM messages and security zone key update and is susceptible to DoS attacks on BS and rogue BS and rogue RS attacks.In our previous conference paper [7], we focused on addressing the rogue BS and rogue RS attacks in MMR WiMAX networks with centralized authentication schemes and designed a set of protocols to address the aforementioned attacks. However, a formal verification we conducted later using the Scyther tool [8] shows that our previous protocols allow for a new attack called suppress-replay attack, which exploits the asynchronization of time stamps in BS and RS. Moreover, our previous work did not provide a distributed authentication scheme, which is considered to be a more favorable way to conduct authentication nowadays because of the multitude of distributed mobile clients and devices.In order to address these new issues found in our previous paper, we propose in this paper a complete authentication solution to address the rogue BS and rogue RS attacks in MMR WiMAX networks, the suppress-replay attack found in our previous scheme, and a distributed authentication scheme between RSs. We present three different mutual authentication protocols which utilize a trusted authentication server to support three possible scenarios in which the RS or MS connects to an MMR WiMAX network. Our protocols are conformant to the security requirements of IEEE 802.16-2012 standard. We also verify the correctness of our protocols by utilizing the former verification tool for security protocols called Scyther. According to the verification result, our protocols are able to counter against all the attacks we discussed in our paper.The remainder of this paper is organized as follows. In Section2, we give an overview of related works. Section 3 describes the possible rogue BS and rogue RS attacks on access service in MMR networks. In Section 4, we present our new schemes for securing the original authentication protocols for MMR networks against rogue BS and rogue RS. In Section 5, we give a formal analysis and verification of our protocols using the Scyther tool. Finally, we conclude our paper and discuss the future work in Section 6.


## 2. Related Work

A number of papers have been published regarding the security issues of WiMAX networks since IEEE 802.16 standard was developed. Xu et al. give a detailed analysis on privacy and key management protocols of the standard in [9, 10]. Several other papers addressed the security issues of one-way authentication and rogue base station attack such as [9, 11, 12]. However, these publications considered only the single-hop WiMAX when the MMR WiMAX network had not come to existence.The authentication issue has been studied in several other types of multihop networks, such as wireless mesh networks [13], cellular networks [14], and sensor networks [15]. However, the MMR WiMAX network is still very recent and has its own unique characteristics that need to be investigated separately.Distributed and centralized authentication approaches are two major options when it comes to authentication protocol design. In [16], Yang et al. discussed security issues in WiMAX MMR networks and the pros and cons of the two major types of authentication protocol design. In [17], Jin et al. propose an improved mutual authentication scheme in multihop WiMAX networks, in which they improve the X.509 certificate by using ECC algorithm instead of RSA, and modify the flow of mutual authentication to improve the security in multihop WiMAX networks. In [18, 19], Khan et al. proposed a modified PKM protocol using distributed authentication and localized key management scheme. In [3], Tie and Yi proposed a multihop ticket based handover authentication which adopted the idea from Kerberos and used a ticket to allow MS, RS, and BS to mutually authenticate each other. However, the authors in the aforementioned papers did not take rogue access node attack into consideration.In order to solve the problems like security zone key update, DDoS attack, and rogue RS attack, in [20], the authors propose a design of hybrid authentication and key distribution scheme to support the IEEE 802.16j (part of current IEEE 802.16-2012 standard) MMR requirements. Although the authors claim that this hybrid design is robust enough to prevent rogue node attack, they only consider the case when a rogue RS tries to join the network at initial phase, and they do not take rogue BS attack into account. The latter case will cause more severe damage to the network since a rogue BS can take control of the whole area within its communication range if it successfully joins the network as a legitimate BS. In another paper [21], the authors present a distributed scheme using decode and forward relays with localized authentication, which helps to authenticate MS and RS at initial network entry. However, this scheme still cannot solve the problem of rogue BS attack. In [22], the authors proposed a self-testing approach to defend against rogue BS attack of intelligent terminal. However, their work did not focus on MMR networks and thus cannot address the other security issues in MMR networks we discussed in Section 1. In [23], the authors discussed detection of rogue BS attack in WiMAX networks; however, their discussion did not address the security issues existing specifically in MMR networks.


## 3. Rogue BS and Rogue RS Attacks in MMR WiMAX Networks

Rogue stations have been one of the most common threats in wireless networks [23–25]. In order to design a secure wireless authentication protocol in MMR networks, rogue stations threat must be considered and addressed.Denial of Service (DoS) jamming is a type of the attacks that can involve a rogue station. In MMR, a rogue station can be a rogue BS or a rogue RS. In a DoS jamming attack, by jamming a legitimate BS, connectivity between client and a legitimate BS can be interrupted, which makes it possible for a rogue BS to stand in and impersonate the legitimate BS with fake credentials, trying to convince a joining client or RS to connect with it, so as to cause DoS or even redirect the traffic and hijack the communications.Another form of attack that might involve a rogue station, especially a rogue RS, is man-in-the-middle (MITM) attack [25]. When a client MS or RS initiates a connection, the rogue RS will intercept the connection, and then complete the connection to the intended legitimate RS and proxy all communications to the intended legitimate RS. The rogue RS is now in a position to inject data, modify messages and communications, or eavesdrop on a session that would normally be difficult to decode, such as encrypted sessions.One form of MITM attack involves asynchronization. If the clocks of the client and the legitimate station are not synchronized, it is possible for the rogue station to launch a suppress-replay attack [26]. In a suppress-replay attack, the rogue station can intercept messages that carry a timestamp corresponding to a future time due to an unsynchronized clock and extract from them the component containing the future timestamp. Then, the rogue station can combine the extracted timestamp component with valid components from other messages to create a fake message and replay it later when the timestamp in the fake message becomes current with respect to the clock of the legitimate recipient station.An example to demonstrate the suppress-replay attack is given as follows. Suppose we have a client MS whose clock is 2 minutes ahead of the one in the legitimate station BS. At client side time 1:10 pm and time 1:11 pm (which corresponds to BS time 1:08 pm and time 1:09 pm), client MS sends two messages, message 110 and message 111, to the legitimate BS trying to initiate a connection; each message contains its current MS timestamp and component with necessary authentication credentials to prove MS’s identity; we call this componentAuthentication Component (AC) here. When an attacker intercepts these two messages, the attacker can extract the AC from message 110 and the 1:11 pm timestamp from message 111 and then combine these two parts together to create a new message 111′. The attacker will then send out this newly created fake message 111′ when the time on the legitimate station BS becomes 1:11 pm. IEEE 802.16-2012 uses PKMv2 to counter possible rogue BS attack by using mutual authentication. However, there is an implicit assumption in PKMv2 that BS is always trustworthy; thus PKMv2 does not provide any protection measure to detect and counter the attack from a compromised BS.Moreover, the distributed security mode in MMR WiMAX networks also makes rogue RS attack more possible. This is because the authentication procedure between RS nodes is not performed by a centralized server but is based on the trust between nodes. If one node is compromised, its trust with other nodes is also compromised.Another issue with PKMv2 is that the preassumed trust of BS cannot prevent suppress-replay attack mentioned above. To address all of the aforementioned attacks in MMR, careful protocol design is required in order to achieve secure authentication in MMR networks.


## 4. Proposed Secure Authentication Protocol

We have introduced the related work on security issues in MMR WiMAX networks and have shown that current standards are not sufficient for addressing the rogue BS and rogue RS attacks. In this section, we present a set of new secure protocols to provide robust authentication in MMR WiMAX networks. Specifically, our protocols can defend against rogue BS and RS attack by using a trusted authentication server to provide dual authentication and security zone key. Our protocols support three scenarios of network access: RS connects to an MMR network through BS; RS connects to an MMR network through other RS; and MS connects to an MMR network through RS.


### 4.1. Assumptions

The proposed protocols are based on the following assumptions:(a)
Before the initial access authentication process, each RS, BS, and mobile user (MS) is preregistered with the Authentication Server (AS) by providing their MAC addresses and other necessary credentials. Each RS, BS, and MS shares its own public key(
K
R
S
,
K
B
S
,
K
M
S
) with AS, and each RS, BS, and MS also gets AS’s public key from AS. These keys were obtained during the preregistration phase.
(b)
AS is trusted by all the nodes in the MMR WiMAX network. It is believed by the nodes in this network that AS maintains a correct database of all legitimate registered nodes’ MAC addresses, each node’s corresponding public key, and other credentials. It is easier to ensure the physical security of AS because AS can always be indoor.


### 4.2. Notations

Before we describe the details of our protocols, we specify the simplified notation for each element used in the protocol:M
A
C
X:X’S MAC address (X can be either RS, BS, or MS),
S
e
q
_
X: a sequence number generated by stationX,
N
n
c
_
X: a nonce generated by station X,
K
X:X’s secret key shared with AS,
K
X
_
P
U
B:X’s public key stored inX’s certificate,
K
X
_
P
R
V:X’s private key corresponding to the public key stored inX’s certificate,
PMK: Pairwise Master Key,
SZK: a group key used in the security zone among BS and many RSs,
AK: Authorization Key,
M
D
[
M
]: message digest of messageM,
C
E
R
T
X:X’s digital certificate withX’s public key included,
E
K
X
[
M
]:M encrypted withX’s shared secret key,
E
K
S
Z
K
[
M
]:M encrypted with security zone key,
E
K
X
_
P
U
B
[
M
]:M encrypted withX’s public key stored inX’s certificate,
E
K
X
_
P
R
V
[
M
]:M encrypted withX’s private key corresponding to the public key stored inX’s certificate,
L
i
c
e
n
s
e
_
X: a signature issued to legitimate BS/RS/MS; the signature is generated by AS using AS’s private key,
E
X
P
R
X: expiration time for L
i
c
e
n
s
e
_
X.Here the format of L
i
c
e
n
s
e
_
X is {
E
K
A
S
_
P
R
V
[
M
A
C
X
∥
C
E
R
T
X
∥
E
X
P
R
X
]
}.


### 4.3. Scenario 1: RS Connects to the Network through BS

The first scenario in which a RS needs to connect to an MMR network via BS is shown in Figure1.Figure 1
RS connects to MMR networks via BS.A RS broadcasts the AUTH-REQ message when it wants to connect to an MMR WiMAX network. Normally the BS which is the closest to this RS will handle this message and send it to AS. AS will then perform the authentication and send back an AUTH-REPLY message. The detailed message format is specified as follows:(1)
R
S
→
B
S:
⁡
M
A
C
R
S
∥
S
e
q
_
R
S
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
R
S
∥
K
R
S
]
∥
C
E
R
T
R
S
(2)
B
S
→
A
S:
⁡
M
A
C
R
S
∥
S
e
q
_
R
S
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
R
S
∥
K
R
S
]
∥
C
E
R
T
R
S
∥
M
A
C
B
S
∥
S
e
q
_
B
S
∥
M
D
[
M
A
C
B
S
∥
S
e
q
_
B
S ∥ M
A
C
R
S
∥
K
B
S
]
∥
C
E
R
T
B
S
(3)
A
S
→
B
S:
⁡
E
K
B
S
{
M
A
C
A
S
∥
M
A
C
R
S
∥
S
e
q
_
B
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
B
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
B
S
]
}
∥
E
K
R
S
{
M
A
C
A
S ∥ M
A
C
B
S
∥
S
e
q
_
R
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
R
S
]
}
(4)
B
S
→
R
S:
⁡
E
K
R
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
R
S
]
}
∥
M
D
[
P
M
K ∥ A
K
∥
S
Z
K
]As we mentioned before, each RS is preregistered with AS. Therefore AS can match the MAC address in the message with the corresponding shared secret key in the database. When AS receives message(
2
), AS can use MAC address, sequence number, and the shared secret key to calculate the corresponding message digest in order to verify the authenticity of the sender (both BS and RS), since only AS and the corresponding legitimate node know the shared secret key. After successful authentication of BS and RS, AS will use the shared keys of BS and RS to encrypt the authentication reply message and send it back to BS and the requesting RS. If both BS and RS are legitimate, in message (
3
) AS will generate a PMK and a SZK and include them in the reply message. After BS receives and decrypts the AUTH-REPLY message from AS, it can be sure about the authenticity of the requesting RS. Thus BS can decide whether to grant RS the access to the network or not. BS also gets PMK from AS and uses it to generate AK. PMK is encrypted with BS’s shared key with AS. AS also sends the message digest of PMK in this message to BS, which is meant to allow BS to verify the integrity of this PMK it receives. In message (
3
), BS receives its L
i
c
e
n
s
e
_
B
S from AS, which works as a unique digital signature to prove its identity to other nodes before the license expires. This signature is generated by encrypting the concatenation of BS’s MAC address, BS’s certificate (e.g., X.509 certificate), and the expiration time of this license, using AS’s private key. As what has been described in our assumptions, every legitimate node in this network has the AS’s public key, so when BS gets its license, it can use AS’s public key to verify this license’s authenticity. Similar license also appears in message (4) in which it provides legitimate RS with a license to prove its identity to other nodes in the future. The usage of signature here in messages (
3
) and (
4
) can protect the network from malicious node with a fake license, since it should be computationally infeasible for a party who does not possess the private key to generate a valid signature.After successful authentication with BS/RS, AS assigns a security zone key, denoted as SZK, for nodes in the security zone to secure future communications between them. At AS, for each AUTH-REQ from RS, AS will check with BS’s MAC address in its received message to decide which security zone this requesting RS belongs to. After authentication is successfully completed, a zone key SZK which corresponds to the right BS will be assigned to the RS.In message(
4
), along with PMK, BS also sends the message digest of AK which it generated from PMK; the purpose is to let RS verify the integrity of the PMK it receives. If RS can use the PMK it received to generate the same AK which is consistent with the message digest M
D
[
A
K
] included in message (
4
), it can believe that the message is not compromised and the AK is good to use.We analyze why our protocol can protect the MMR WiMAX network against rogue BS or rogue RS attacks. In the case of rogue BS attack, since BS does not have RS’s secret key shared with AS, it cannot decrypt or modify the message destined to RS. Hence if RS gets the message from AS which notifies RS of this illegitimate BS, RS will not try to get access through this rogue BS again. Therefore the rogue BS will not have the opportunity to take control of RS in the network.In the case of rogue RS attack, when an access RS becomes a rogue one, it would not receive the AUTH-REPLY message with a license included. In the case of message hijacking or man-in-the-middle attack, even if an attacker can get the message which was supposed to be transmitted to the legitimate BS or RS, the attacker still cannot decrypt the message to get the license since he does not have the secret keyK
B
S or K
R
S. Therefore the attacker cannot obtain a valid license without getting authenticated by AS, which it cannot pass.


### 4.4. Scenario 2: RS Connects to the Network through Other RS

The second scenario in which a RS requests to connect to an MMR network through an edge RS is shown in Figure2. In this scenario, the requesting RS wants to set up connection with the BS via one or more intermediate RSs. Our protocol for this scenario contains three authentication phases: (i) initial verification of edge RS; (ii) dual authentication of BS and the requesting RS by AS; and (iii) distributed authentication when requesting RS holds a valid license.Figure 2
RS connects to MMR networks via an edge RS.In authentication phase (i), when the requestingR
S
A tries to get access to the network through another R
S
B (edge RS), R
S
A needs to verify the authenticity of this intermediate R
S
B first. The message format of phase (i) is specified as follows:(1)
RequestingR
S
A → Edge R
S
B:
⁡
M
A
C
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
]
(2)
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g  R
S
A:
⁡
M
A
C
R
S
(
B
)
∥
C
E
R
T
R
S
(
B
)
∥
L
i
c
e
n
s
e
_
R
S
(
B
)
∥
M
D
[
M
A
C
R
S
(
B
)
∥
S
e
q
_
R
S
(
A
)
]
∥
E
K
R
S
(
B
)
_
P
R
V
[
E
K
R
S
(
A
)
_
P
U
B
[
S
e
q
_
R
S
(
A
) ∥ N
n
c
_
R
S
(
B
)
]
]
(3)
RequestingR
S
A
→
E
d
g
e
R
S
B:
⁡
E
K
R
S
(
A
)
_
P
R
V
[
E
K
R
S
(
B
)
_
P
U
B
[
N
n
c
_
R
S
(
B
)
∥
N
n
c
_
R
S
(
A
)
]
]
(4)
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g
R
S
A:
⁡
E
K
R
S
(
B
)
_
P
R
V
[
E
K
R
S
(
A
)
_
P
U
B
[
N
n
c
_
R
S
(
A
)
]
]In messages(
1
) and (
2
), the requesting relay station R
S
A and the edge relay station R
S
B exchange their certificate, such that they know each other’s public key. In message (
2
), based on our assumption that each registered RS has the AS’s public key, the requesting R
S
A can use it to verify whether the edge R
S
B is a legitimate relay station by checking that the license is currently valid. However, if the authentication phase (i) stops at message (
2
), then it will be susceptible to areplay attack described as follows: a malicious relay station R
S
E listening in the middle makes a copy of message (
1
) sent by the requesting R
S
A, makes a separate run of this protocol by forwarding the copy of message (
1
) to a legitimate edge relay station R
S
B (i.e., pretending R
S
E itself is the requesting R
S
A), and replays the message (
2
) received from the R
S
B to R
S
A. When R
S
A receives the message, it will be convinced that the malicious R
S
E is a legitimate edge relay station. To address this problem of replay attack, we add messages (
3
) and (
4
), which is in challenge-response style. In message (
3
), R
S
A concatenates N
n
c
_
R
S
(
B
) (derived from decrypting message (
2
)) with a nonce of its choice N
n
c
_
R
S
(
A
), encrypts it using the public key of R
S
B and the private key of R
S
A, and sends it to R
S
B. R
S
B decrypts the received message (
3
) and extracts N
n
c
_
R
S
(
A
), encrypts it using the public key of R
S
A and the private key of R
S
B, and sends it to R
S
A in message (
4
). When R
S
A verifies that the N
n
c
_
R
S
(
A
) received in message (
4
) is the same as the nonce it chooses in message (
3
), R
S
A can believe that R
S
B is really the authentic edge relay station, rather than a malicious relay station of a replay attack.After edgeR
S
B’s authenticity has been verified in phase (i), the requesting R
S
A starts phase (ii), in which both BS and R
S
A need to be authenticated by AS. The message format of phase (ii) is specified as follows:〈1〉
RequestingR
S
A
→
E
d
g
e
R
S
B:
⁡
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
K
R
S
(
A
)
]
〈2〉
EdgeR
S
B
→
(
possibly  other  RS  in  between
)
→
B
S:
⁡
E
K
S
Z
K
{
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
K
R
S
(
A
)
]
}
〈3〉
B
S
→
A
S:
⁡
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
K
R
S
(
A
)
]
∥
M
A
C
B
S
∥
S
e
q
_
B
S ∥ M
D
[
M
A
C
B
S
∥
M
A
C
R
S
(
A
)
∥
S
e
q
_
B
S
∥
K
B
S
]
∥
C
E
R
T
B
S
〈4〉
A
S
→
B
S:
⁡
E
K
B
S
{
M
A
C
A
S
∥
M
A
C
R
S
(
A
)
∥
S
e
q
_
B
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
B
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
B
S
]
} ∥ E
K
R
S
(
A
)
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
(
A
)
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
R
S
(
A
)
]
}
〈5〉
B
S
→
E
d
g
e
R
S
B:
⁡
E
K
S
Z
K
{
E
K
R
S
(
A
)
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
(
A
)
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K ∥ K
R
S
(
A
)
]
}
∥
M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]
}
〈6〉
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g
R
S
A:
⁡
E
K
R
S
(
A
)
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
(
A
)
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K ∥ K
R
S
(
A
)
]
}
∥
M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]If the authentication betweenR
S
A and AS is successful and R
S
A is regarded to be legitimate, then R
S
A will be assigned with the security zone key SZK which R
S
A and R
S
B can use to secure future communications between them.Comparing the message format in phase (ii) with the message format in the first scenario in which RS connects to the network directly through BS, we can see that the major contents in the messages are very similar except that the security zone key SZK is being used here to encrypt all the messages that are transmitted in this security zone (messages2 and 5). This design is in accordance with the aforementioned security requirement of IEEE 802.16-2012 standard, which requires that trust is maintained within the security zone in order to support multihop relay system operations. At edge R
S
B, R
S
B decrypts the message it received with SZK and then forwards the decrypted messages to the requesting R
S
A. In this case the message is still secured because it is still encrypted by AS with R
S
A’s public key.If the requestingR
S
A has been authenticated by AS within a valid period of time before it tries to connect to the current network, R
S
A can skip phases (i) and (ii) and directly enter an optional phase (iii) in which a more efficient distributed authentication will be performed. The message format is described as follows:(1)
RequestingR
S
A
→
A
l
l neighborhood nodes:
⁡
M
A
C
R
S
A
∥
S
e
q
_
R
S
A
∥
C
E
R
T
R
S
A
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
) ∥ C
E
R
T
R
S
(
A
)
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
]
(2)
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g
R
S
A:
⁡
M
A
C
R
S
(
B
)
∥
C
E
R
T
R
S
(
B
)
∥
L
i
c
e
n
s
e
_
R
S
(
B
)
∥
M
A
C
B
S
∥
E
K
R
S
A
_
P
U
B
[
S
Z
K
]
∥
M
D
[
M
A
C
R
S
(
B
)
∥
S
e
q
_
R
S
(
A
) ∥ C
E
R
T
R
S
(
B
)
∥
L
i
c
e
n
s
e
_
R
S
(
B
)
∥
M
A
C
B
S
∥
S
Z
K
]In phase (iii),R
S
A first broadcasts to all its neighborhood nodes message (
1
), which contains R
S
A’s MAC address, sequence number, digital certificate, its license, and the message digest. Once a legitimate edge R
S
B receives this message, it can use AS’s public key to verify the authenticity of R
S
A. If R
S
A is verified to be authentic, then edge R
S
B can send R
S
A message (
2
) which includes R
S
B’s license, certificate, security zone key, and the MAC address of BS which is in charge of this security zone. When R
S
A gets message (
2
) and has the SZK and BS’s MAC address, it can establish secure communications with BS to generate and exchange AK and TEK.


### 4.5. Scenario 3: MS Connects to the Network through RS

The third scenario in which a mobile user (MS) needs to connect to an MMR network through an edge RS is shown in Figure3. In this scenario we have two cases to consider: the first one is when MS connects to the network for the first time through one or more RS; the second one is when MS has been authenticated by AS within a valid period of time before it tries to connect to the current network. For both cases, the message formats are similar to the message formats from the second scenario when a RS tries to get access to the network via other RS.Figure 3
MS connects to MMR networks via an edge RS.For MS which joins the network for the first time through an edge RS, there are two phases whose message format is specified as follows:Phase (i) (initial verification of the edge RS)
(1)
M
S
→
E
d
g
e
R
S:
⁡
M
A
C
M
S ∥ C
E
R
T
M
S ∥ S
e
q
_
M
S ∥ MD[M
A
C
M
S ∥ S
e
q
_
M
S]
(2)
Edge RS → MS:
⁡
M
A
C
R
S
∥
C
E
R
T
R
S
∥
L
i
c
e
n
s
e
_
R
S
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
M
S
]
∥
E
K
R
S
_
P
R
V
[
E
K
M
S
_
P
U
B
[
S
e
q
_
M
S
∥
N
n
c
_
R
S
]
]
(3)
MS → Edge RS:
⁡
E
K
M
S
_
P
R
V
[
E
K
R
S
_
P
U
B
[
N
n
c
_
R
S
∥
N
n
c
_
M
S
]
]
(4)
Edge RS → MS:
⁡
E
K
R
S
_
P
R
V
[
E
K
M
S
_
P
U
B
[
N
n
c
_
M
S
]
]Note that the four messages in phase (i) are similar to the four messages in phase (i) of Scenario 2 presented in the previous Section4.4, because we need to prevent the possible replay attack launched by a malicious relay station. We will not repeat the explanation of the replay attack here because it has been explained in detail in the previous subsection.Phase (ii) (dual authentication of MS and BS)
〈1〉
M
S
→
E
d
g
e
R
S:
⁡
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
K
M
S
]According to IEEE 802.16-2012 standard, here relay stations do not try to decrypt the user date or authenticate the MAC management message they receive from mobile stations but simply relay it. And RS does not have any key information associate with the MS:〈2〉
E
d
g
e
R
S
→
(
p
o
s
s
i
b
l
y
o
t
h
e
r
R
S
i
n
b
e
t
w
e
e
n
)
→
B
S:
⁡
E
K
S
Z
K
{
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
K
M
S
]
}
〈3〉
B
S
→
A
S:
⁡
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
K
M
S
]
∥
M
A
C
B
S
∥
S
e
q
_
B
S
∥
M
D
[
M
A
C
B
S ∥ M
A
C
M
S
∥
S
e
q
_
B
S
∥
K
B
S
]
∥
C
E
R
T
B
S
〈4〉
A
S
→
B
S:
⁡
E
K
B
S
{
M
A
C
A
S
∥
M
A
C
M
S
∥
S
e
q
_
B
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
B
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
B
S
]
} ∥ E
K
M
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
M
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
M
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
M
S
]
}
〈5〉
B
S
→
(
p
o
s
s
i
b
l
y
o
t
h
e
r
R
S
i
n
b
e
t
w
e
e
n
)
→
E
d
g
e
R
S:
⁡
E
K
S
Z
K
{
E
K
M
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
M
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
M
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K ∥ K
M
S
]
}
∥
M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]
}
〈6〉
E
d
g
e
R
S
→
M
S:
⁡
E
K
M
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
M
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
M
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
M
S
]
} ∥ M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]In message6, after MS gets PMK and generates related AK and TEK, MS can verify its AK with the AK in the received message to check their consistency. TEK will be used to secure future communications between BS and MS.For MS which has been authenticated by AS within a valid period of time before it tries to connect to the current network, a distributed authentication will be performed. The message format is specified as follows:(1)
M
S
→
A
l
l
n
e
i
g
h
b
o
r
h
o
o
d
n
o
d
e
s:
⁡
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
L
i
c
e
n
s
e
_
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
L
i
c
e
n
s
e
_
M
S
]
(2)
EdgeR
S
→
M
S:
⁡
M
A
C
R
S
∥
C
E
R
T
R
S
∥
L
i
c
e
n
s
e
_
R
S
∥
M
A
C
B
S
∥
E
K
M
S
_
P
U
B
[
S
Z
K
]
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
M
S
∥
C
E
R
T
R
S ∥ L
i
c
e
n
s
e
_
R
S
∥
M
A
C
B
S
∥
S
Z
K
]MS broadcasts message(
1
) to all its neighborhood nodes. Message (
1
) contains MS’s MAC address, sequence number, digital certificate, its license, and the message digest. Once a legitimate edge RS receives this message, it can use the AS’s public key to verify the authenticity of MS. If MS is verified to be authentic, then edge RS can send MS message (
2
) which includes RS’s license, certificate, security zone key between RS and MS, and the MAC address of BS which is in charge of this security zone. When MS gets message (
2
) and has the SZK and BS’s MAC address, with SZK it can establish a secure communication with BS before it has TEK to encrypt its messages.


## 4.1. Assumptions

The proposed protocols are based on the following assumptions:(a)
Before the initial access authentication process, each RS, BS, and mobile user (MS) is preregistered with the Authentication Server (AS) by providing their MAC addresses and other necessary credentials. Each RS, BS, and MS shares its own public key(
K
R
S
,
K
B
S
,
K
M
S
) with AS, and each RS, BS, and MS also gets AS’s public key from AS. These keys were obtained during the preregistration phase.
(b)
AS is trusted by all the nodes in the MMR WiMAX network. It is believed by the nodes in this network that AS maintains a correct database of all legitimate registered nodes’ MAC addresses, each node’s corresponding public key, and other credentials. It is easier to ensure the physical security of AS because AS can always be indoor.


## 4.2. Notations

Before we describe the details of our protocols, we specify the simplified notation for each element used in the protocol:M
A
C
X:X’S MAC address (X can be either RS, BS, or MS),
S
e
q
_
X: a sequence number generated by stationX,
N
n
c
_
X: a nonce generated by station X,
K
X:X’s secret key shared with AS,
K
X
_
P
U
B:X’s public key stored inX’s certificate,
K
X
_
P
R
V:X’s private key corresponding to the public key stored inX’s certificate,
PMK: Pairwise Master Key,
SZK: a group key used in the security zone among BS and many RSs,
AK: Authorization Key,
M
D
[
M
]: message digest of messageM,
C
E
R
T
X:X’s digital certificate withX’s public key included,
E
K
X
[
M
]:M encrypted withX’s shared secret key,
E
K
S
Z
K
[
M
]:M encrypted with security zone key,
E
K
X
_
P
U
B
[
M
]:M encrypted withX’s public key stored inX’s certificate,
E
K
X
_
P
R
V
[
M
]:M encrypted withX’s private key corresponding to the public key stored inX’s certificate,
L
i
c
e
n
s
e
_
X: a signature issued to legitimate BS/RS/MS; the signature is generated by AS using AS’s private key,
E
X
P
R
X: expiration time for L
i
c
e
n
s
e
_
X.Here the format of L
i
c
e
n
s
e
_
X is {
E
K
A
S
_
P
R
V
[
M
A
C
X
∥
C
E
R
T
X
∥
E
X
P
R
X
]
}.


## 4.3. Scenario 1: RS Connects to the Network through BS

The first scenario in which a RS needs to connect to an MMR network via BS is shown in Figure1.Figure 1
RS connects to MMR networks via BS.A RS broadcasts the AUTH-REQ message when it wants to connect to an MMR WiMAX network. Normally the BS which is the closest to this RS will handle this message and send it to AS. AS will then perform the authentication and send back an AUTH-REPLY message. The detailed message format is specified as follows:(1)
R
S
→
B
S:
⁡
M
A
C
R
S
∥
S
e
q
_
R
S
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
R
S
∥
K
R
S
]
∥
C
E
R
T
R
S
(2)
B
S
→
A
S:
⁡
M
A
C
R
S
∥
S
e
q
_
R
S
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
R
S
∥
K
R
S
]
∥
C
E
R
T
R
S
∥
M
A
C
B
S
∥
S
e
q
_
B
S
∥
M
D
[
M
A
C
B
S
∥
S
e
q
_
B
S ∥ M
A
C
R
S
∥
K
B
S
]
∥
C
E
R
T
B
S
(3)
A
S
→
B
S:
⁡
E
K
B
S
{
M
A
C
A
S
∥
M
A
C
R
S
∥
S
e
q
_
B
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
B
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
B
S
]
}
∥
E
K
R
S
{
M
A
C
A
S ∥ M
A
C
B
S
∥
S
e
q
_
R
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
R
S
]
}
(4)
B
S
→
R
S:
⁡
E
K
R
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
R
S
]
}
∥
M
D
[
P
M
K ∥ A
K
∥
S
Z
K
]As we mentioned before, each RS is preregistered with AS. Therefore AS can match the MAC address in the message with the corresponding shared secret key in the database. When AS receives message(
2
), AS can use MAC address, sequence number, and the shared secret key to calculate the corresponding message digest in order to verify the authenticity of the sender (both BS and RS), since only AS and the corresponding legitimate node know the shared secret key. After successful authentication of BS and RS, AS will use the shared keys of BS and RS to encrypt the authentication reply message and send it back to BS and the requesting RS. If both BS and RS are legitimate, in message (
3
) AS will generate a PMK and a SZK and include them in the reply message. After BS receives and decrypts the AUTH-REPLY message from AS, it can be sure about the authenticity of the requesting RS. Thus BS can decide whether to grant RS the access to the network or not. BS also gets PMK from AS and uses it to generate AK. PMK is encrypted with BS’s shared key with AS. AS also sends the message digest of PMK in this message to BS, which is meant to allow BS to verify the integrity of this PMK it receives. In message (
3
), BS receives its L
i
c
e
n
s
e
_
B
S from AS, which works as a unique digital signature to prove its identity to other nodes before the license expires. This signature is generated by encrypting the concatenation of BS’s MAC address, BS’s certificate (e.g., X.509 certificate), and the expiration time of this license, using AS’s private key. As what has been described in our assumptions, every legitimate node in this network has the AS’s public key, so when BS gets its license, it can use AS’s public key to verify this license’s authenticity. Similar license also appears in message (4) in which it provides legitimate RS with a license to prove its identity to other nodes in the future. The usage of signature here in messages (
3
) and (
4
) can protect the network from malicious node with a fake license, since it should be computationally infeasible for a party who does not possess the private key to generate a valid signature.After successful authentication with BS/RS, AS assigns a security zone key, denoted as SZK, for nodes in the security zone to secure future communications between them. At AS, for each AUTH-REQ from RS, AS will check with BS’s MAC address in its received message to decide which security zone this requesting RS belongs to. After authentication is successfully completed, a zone key SZK which corresponds to the right BS will be assigned to the RS.In message(
4
), along with PMK, BS also sends the message digest of AK which it generated from PMK; the purpose is to let RS verify the integrity of the PMK it receives. If RS can use the PMK it received to generate the same AK which is consistent with the message digest M
D
[
A
K
] included in message (
4
), it can believe that the message is not compromised and the AK is good to use.We analyze why our protocol can protect the MMR WiMAX network against rogue BS or rogue RS attacks. In the case of rogue BS attack, since BS does not have RS’s secret key shared with AS, it cannot decrypt or modify the message destined to RS. Hence if RS gets the message from AS which notifies RS of this illegitimate BS, RS will not try to get access through this rogue BS again. Therefore the rogue BS will not have the opportunity to take control of RS in the network.In the case of rogue RS attack, when an access RS becomes a rogue one, it would not receive the AUTH-REPLY message with a license included. In the case of message hijacking or man-in-the-middle attack, even if an attacker can get the message which was supposed to be transmitted to the legitimate BS or RS, the attacker still cannot decrypt the message to get the license since he does not have the secret keyK
B
S or K
R
S. Therefore the attacker cannot obtain a valid license without getting authenticated by AS, which it cannot pass.


## 4.4. Scenario 2: RS Connects to the Network through Other RS

The second scenario in which a RS requests to connect to an MMR network through an edge RS is shown in Figure2. In this scenario, the requesting RS wants to set up connection with the BS via one or more intermediate RSs. Our protocol for this scenario contains three authentication phases: (i) initial verification of edge RS; (ii) dual authentication of BS and the requesting RS by AS; and (iii) distributed authentication when requesting RS holds a valid license.Figure 2
RS connects to MMR networks via an edge RS.In authentication phase (i), when the requestingR
S
A tries to get access to the network through another R
S
B (edge RS), R
S
A needs to verify the authenticity of this intermediate R
S
B first. The message format of phase (i) is specified as follows:(1)
RequestingR
S
A → Edge R
S
B:
⁡
M
A
C
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
]
(2)
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g  R
S
A:
⁡
M
A
C
R
S
(
B
)
∥
C
E
R
T
R
S
(
B
)
∥
L
i
c
e
n
s
e
_
R
S
(
B
)
∥
M
D
[
M
A
C
R
S
(
B
)
∥
S
e
q
_
R
S
(
A
)
]
∥
E
K
R
S
(
B
)
_
P
R
V
[
E
K
R
S
(
A
)
_
P
U
B
[
S
e
q
_
R
S
(
A
) ∥ N
n
c
_
R
S
(
B
)
]
]
(3)
RequestingR
S
A
→
E
d
g
e
R
S
B:
⁡
E
K
R
S
(
A
)
_
P
R
V
[
E
K
R
S
(
B
)
_
P
U
B
[
N
n
c
_
R
S
(
B
)
∥
N
n
c
_
R
S
(
A
)
]
]
(4)
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g
R
S
A:
⁡
E
K
R
S
(
B
)
_
P
R
V
[
E
K
R
S
(
A
)
_
P
U
B
[
N
n
c
_
R
S
(
A
)
]
]In messages(
1
) and (
2
), the requesting relay station R
S
A and the edge relay station R
S
B exchange their certificate, such that they know each other’s public key. In message (
2
), based on our assumption that each registered RS has the AS’s public key, the requesting R
S
A can use it to verify whether the edge R
S
B is a legitimate relay station by checking that the license is currently valid. However, if the authentication phase (i) stops at message (
2
), then it will be susceptible to areplay attack described as follows: a malicious relay station R
S
E listening in the middle makes a copy of message (
1
) sent by the requesting R
S
A, makes a separate run of this protocol by forwarding the copy of message (
1
) to a legitimate edge relay station R
S
B (i.e., pretending R
S
E itself is the requesting R
S
A), and replays the message (
2
) received from the R
S
B to R
S
A. When R
S
A receives the message, it will be convinced that the malicious R
S
E is a legitimate edge relay station. To address this problem of replay attack, we add messages (
3
) and (
4
), which is in challenge-response style. In message (
3
), R
S
A concatenates N
n
c
_
R
S
(
B
) (derived from decrypting message (
2
)) with a nonce of its choice N
n
c
_
R
S
(
A
), encrypts it using the public key of R
S
B and the private key of R
S
A, and sends it to R
S
B. R
S
B decrypts the received message (
3
) and extracts N
n
c
_
R
S
(
A
), encrypts it using the public key of R
S
A and the private key of R
S
B, and sends it to R
S
A in message (
4
). When R
S
A verifies that the N
n
c
_
R
S
(
A
) received in message (
4
) is the same as the nonce it chooses in message (
3
), R
S
A can believe that R
S
B is really the authentic edge relay station, rather than a malicious relay station of a replay attack.After edgeR
S
B’s authenticity has been verified in phase (i), the requesting R
S
A starts phase (ii), in which both BS and R
S
A need to be authenticated by AS. The message format of phase (ii) is specified as follows:〈1〉
RequestingR
S
A
→
E
d
g
e
R
S
B:
⁡
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
K
R
S
(
A
)
]
〈2〉
EdgeR
S
B
→
(
possibly  other  RS  in  between
)
→
B
S:
⁡
E
K
S
Z
K
{
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
K
R
S
(
A
)
]
}
〈3〉
B
S
→
A
S:
⁡
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
C
E
R
T
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
)
∥
K
R
S
(
A
)
]
∥
M
A
C
B
S
∥
S
e
q
_
B
S ∥ M
D
[
M
A
C
B
S
∥
M
A
C
R
S
(
A
)
∥
S
e
q
_
B
S
∥
K
B
S
]
∥
C
E
R
T
B
S
〈4〉
A
S
→
B
S:
⁡
E
K
B
S
{
M
A
C
A
S
∥
M
A
C
R
S
(
A
)
∥
S
e
q
_
B
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
B
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
B
S
]
} ∥ E
K
R
S
(
A
)
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
(
A
)
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
R
S
(
A
)
]
}
〈5〉
B
S
→
E
d
g
e
R
S
B:
⁡
E
K
S
Z
K
{
E
K
R
S
(
A
)
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
(
A
)
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K ∥ K
R
S
(
A
)
]
}
∥
M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]
}
〈6〉
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g
R
S
A:
⁡
E
K
R
S
(
A
)
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
R
S
(
A
)
∥
P
M
K
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K ∥ K
R
S
(
A
)
]
}
∥
M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]If the authentication betweenR
S
A and AS is successful and R
S
A is regarded to be legitimate, then R
S
A will be assigned with the security zone key SZK which R
S
A and R
S
B can use to secure future communications between them.Comparing the message format in phase (ii) with the message format in the first scenario in which RS connects to the network directly through BS, we can see that the major contents in the messages are very similar except that the security zone key SZK is being used here to encrypt all the messages that are transmitted in this security zone (messages2 and 5). This design is in accordance with the aforementioned security requirement of IEEE 802.16-2012 standard, which requires that trust is maintained within the security zone in order to support multihop relay system operations. At edge R
S
B, R
S
B decrypts the message it received with SZK and then forwards the decrypted messages to the requesting R
S
A. In this case the message is still secured because it is still encrypted by AS with R
S
A’s public key.If the requestingR
S
A has been authenticated by AS within a valid period of time before it tries to connect to the current network, R
S
A can skip phases (i) and (ii) and directly enter an optional phase (iii) in which a more efficient distributed authentication will be performed. The message format is described as follows:(1)
RequestingR
S
A
→
A
l
l neighborhood nodes:
⁡
M
A
C
R
S
A
∥
S
e
q
_
R
S
A
∥
C
E
R
T
R
S
A
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
∥
M
D
[
M
A
C
R
S
(
A
)
∥
S
e
q
_
R
S
(
A
) ∥ C
E
R
T
R
S
(
A
)
∥
L
i
c
e
n
s
e
_
R
S
(
A
)
]
(2)
EdgeR
S
B
→
R
e
q
u
e
s
t
i
n
g
R
S
A:
⁡
M
A
C
R
S
(
B
)
∥
C
E
R
T
R
S
(
B
)
∥
L
i
c
e
n
s
e
_
R
S
(
B
)
∥
M
A
C
B
S
∥
E
K
R
S
A
_
P
U
B
[
S
Z
K
]
∥
M
D
[
M
A
C
R
S
(
B
)
∥
S
e
q
_
R
S
(
A
) ∥ C
E
R
T
R
S
(
B
)
∥
L
i
c
e
n
s
e
_
R
S
(
B
)
∥
M
A
C
B
S
∥
S
Z
K
]In phase (iii),R
S
A first broadcasts to all its neighborhood nodes message (
1
), which contains R
S
A’s MAC address, sequence number, digital certificate, its license, and the message digest. Once a legitimate edge R
S
B receives this message, it can use AS’s public key to verify the authenticity of R
S
A. If R
S
A is verified to be authentic, then edge R
S
B can send R
S
A message (
2
) which includes R
S
B’s license, certificate, security zone key, and the MAC address of BS which is in charge of this security zone. When R
S
A gets message (
2
) and has the SZK and BS’s MAC address, it can establish secure communications with BS to generate and exchange AK and TEK.


## 4.5. Scenario 3: MS Connects to the Network through RS

The third scenario in which a mobile user (MS) needs to connect to an MMR network through an edge RS is shown in Figure3. In this scenario we have two cases to consider: the first one is when MS connects to the network for the first time through one or more RS; the second one is when MS has been authenticated by AS within a valid period of time before it tries to connect to the current network. For both cases, the message formats are similar to the message formats from the second scenario when a RS tries to get access to the network via other RS.Figure 3
MS connects to MMR networks via an edge RS.For MS which joins the network for the first time through an edge RS, there are two phases whose message format is specified as follows:Phase (i) (initial verification of the edge RS)
(1)
M
S
→
E
d
g
e
R
S:
⁡
M
A
C
M
S ∥ C
E
R
T
M
S ∥ S
e
q
_
M
S ∥ MD[M
A
C
M
S ∥ S
e
q
_
M
S]
(2)
Edge RS → MS:
⁡
M
A
C
R
S
∥
C
E
R
T
R
S
∥
L
i
c
e
n
s
e
_
R
S
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
M
S
]
∥
E
K
R
S
_
P
R
V
[
E
K
M
S
_
P
U
B
[
S
e
q
_
M
S
∥
N
n
c
_
R
S
]
]
(3)
MS → Edge RS:
⁡
E
K
M
S
_
P
R
V
[
E
K
R
S
_
P
U
B
[
N
n
c
_
R
S
∥
N
n
c
_
M
S
]
]
(4)
Edge RS → MS:
⁡
E
K
R
S
_
P
R
V
[
E
K
M
S
_
P
U
B
[
N
n
c
_
M
S
]
]Note that the four messages in phase (i) are similar to the four messages in phase (i) of Scenario 2 presented in the previous Section4.4, because we need to prevent the possible replay attack launched by a malicious relay station. We will not repeat the explanation of the replay attack here because it has been explained in detail in the previous subsection.Phase (ii) (dual authentication of MS and BS)
〈1〉
M
S
→
E
d
g
e
R
S:
⁡
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
K
M
S
]According to IEEE 802.16-2012 standard, here relay stations do not try to decrypt the user date or authenticate the MAC management message they receive from mobile stations but simply relay it. And RS does not have any key information associate with the MS:〈2〉
E
d
g
e
R
S
→
(
p
o
s
s
i
b
l
y
o
t
h
e
r
R
S
i
n
b
e
t
w
e
e
n
)
→
B
S:
⁡
E
K
S
Z
K
{
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
K
M
S
]
}
〈3〉
B
S
→
A
S:
⁡
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
K
M
S
]
∥
M
A
C
B
S
∥
S
e
q
_
B
S
∥
M
D
[
M
A
C
B
S ∥ M
A
C
M
S
∥
S
e
q
_
B
S
∥
K
B
S
]
∥
C
E
R
T
B
S
〈4〉
A
S
→
B
S:
⁡
E
K
B
S
{
M
A
C
A
S
∥
M
A
C
M
S
∥
S
e
q
_
B
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
B
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
B
S
]
} ∥ E
K
M
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
M
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
M
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
M
S
]
}
〈5〉
B
S
→
(
p
o
s
s
i
b
l
y
o
t
h
e
r
R
S
i
n
b
e
t
w
e
e
n
)
→
E
d
g
e
R
S:
⁡
E
K
S
Z
K
{
E
K
M
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
M
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
M
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K ∥ K
M
S
]
}
∥
M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]
}
〈6〉
E
d
g
e
R
S
→
M
S:
⁡
E
K
M
S
{
M
A
C
A
S
∥
M
A
C
B
S
∥
S
e
q
_
M
S
∥
P
M
K
∥
L
i
c
e
n
s
e
_
M
S
∥
S
Z
K
∥
M
D
[
P
M
K
∥
S
Z
K
∥
K
M
S
]
} ∥ M
D
[
P
M
K
∥
A
K
∥
S
Z
K
]In message6, after MS gets PMK and generates related AK and TEK, MS can verify its AK with the AK in the received message to check their consistency. TEK will be used to secure future communications between BS and MS.For MS which has been authenticated by AS within a valid period of time before it tries to connect to the current network, a distributed authentication will be performed. The message format is specified as follows:(1)
M
S
→
A
l
l
n
e
i
g
h
b
o
r
h
o
o
d
n
o
d
e
s:
⁡
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
L
i
c
e
n
s
e
_
M
S
∥
M
D
[
M
A
C
M
S
∥
S
e
q
_
M
S
∥
C
E
R
T
M
S
∥
L
i
c
e
n
s
e
_
M
S
]
(2)
EdgeR
S
→
M
S:
⁡
M
A
C
R
S
∥
C
E
R
T
R
S
∥
L
i
c
e
n
s
e
_
R
S
∥
M
A
C
B
S
∥
E
K
M
S
_
P
U
B
[
S
Z
K
]
∥
M
D
[
M
A
C
R
S
∥
S
e
q
_
M
S
∥
C
E
R
T
R
S ∥ L
i
c
e
n
s
e
_
R
S
∥
M
A
C
B
S
∥
S
Z
K
]MS broadcasts message(
1
) to all its neighborhood nodes. Message (
1
) contains MS’s MAC address, sequence number, digital certificate, its license, and the message digest. Once a legitimate edge RS receives this message, it can use the AS’s public key to verify the authenticity of MS. If MS is verified to be authentic, then edge RS can send MS message (
2
) which includes RS’s license, certificate, security zone key between RS and MS, and the MAC address of BS which is in charge of this security zone. When MS gets message (
2
) and has the SZK and BS’s MAC address, with SZK it can establish a secure communication with BS before it has TEK to encrypt its messages.


## 5. Formal Analysis of Proposed Protocols

Next, we present the formal analysis and verification of the proposed protocols using a tool called Scyther [8]. Scyther is an automated tool for verification, falsification, and analysis of security protocols [8]. Its effectiveness and correctness have been proved and its operational semantics can be found in [27] for interested readers. Scyther can verify protocols with unbounded number of sessions and roles, if computational resources allow [8]. It is also currently the only existing tool capable of verifying synchronization [27]. Synchronization is an important property in mutual authentication protocols. It indicates that all the messages are transmitted exactly in the order as described by the protocol, which can be used to detect suppress-replay attacks [8]. Hence we include it as one of the properties we need to verify in our proposed protocols, and we choose Scyther since it is the only tool which can do such verification [8]. To have the most accurate verification results, we choose the latest stable version Scyther v1.1.3 released in 2014.


### 5.1. Model Description

In our model, we describe the behavior of the protocol entities in terms of their roles, that is, an initiator or a responder (receiver) or both. For example, for protocol 1, we have three agents, RS, BS, and AS. All of them play two roles: message initiator and message responder. In Scyther, arun is a unique execution of a role that is performed by an agent; that is, each agent executes its runs to implement the protocol and preserve the secrecy of the credentials (e.g., keys, licenses, and sequence number) it claims to achieve. Each agent has several variable definitions for the credentials used in the messages this agent will send or receive when playing different roles. Each agent also has a sequence of events that entails what messages this agent will send or receive, along with its claims. In Scyther, a claim is defined asclaim (A, c, P), which means that agentA expects goalc to hold with parameterP. If an attack exists, then the claim the agent wants to achieve will not hold. Such a claim is called a falsified claim [8].A claim has the locality property, so once agents get the messages, they will be able to view the state of the system from a local perspective. Therefore the protocol needs to make sure that the agent is able to have the knowledge of some properties of global state of the system from the local perspective; for example, the agent is able to know that something is beyond the intruder’s knowledge, or that a specific agent is active. However, for the same protocol, the claim on the same secret credential based on different agents might not always hold; that is, it is possible that for agentA as the message initiator and agentB as the message responderclaim (A, c, P) holds whileclaim (B, c, P) does not hold [8].In order to verify a security protocol using Scyther, we need to specify an attacker and a set of agents executing several runs. Scyther will trace all possible attacks that the attacker might launch and determine whether a security claim the agent holds is true or attacks exist. In the verification of our proposed protocols, we focus on two security properties:secrecy of keys andauthentication. To verify these two properties, we use two types of claims in our model: secrecy claim and authentication claim [8].(i) Secrecy Claim. A secrecy claim is defined asclaim (A, Secret, P). It is a statement that the credentials P included in this claim byA will not be obtained or spoofed by the attacker. Secrecy means that the information in question is not to the knowledge of an attacker, even if it is transmitted over an untrusted network.When agents are communicating data (public or secret credentials) with untrusted agents, the transmitted data is also open to the attacker. Although transmitted data is public in the air now, this does not imply that our protocol is broken. Rather, what we need is a secrecy claim saying that if an agent only speaks with trusted agents, then the data being transmitted or shared is kept in secret [8].(ii) Authentication Claim. In Scyther, authentication focuses on the verification that when a role in a protocol is executed, we can guarantee that in the current network there exists at least one entity that is communicating with this role. However, only knowing that some entity is communicating with the role is not enough to guarantee the correctness and robustness of an authentication protocol; we want to use Scyther to verify and show a stronger guarantee that when the protocol is being executed, the intended entity is aware of the communication, and the messages exchanged between the entity and the role follow the protocol description.An authentication claim is defined asclaim (A, Nisynch). It means agentA sends and receives messages in the order which is exactly the same as what has been described in the protocol [8].


### 5.2. Security Properties to Be Verified

We aim to verify that our proposed protocols have the following three security properties:information confidentiality,no theft of service possible [28], andmessage sequence synchronization. Successful verification of these three properties can prove the robustness of our proposed protocols against the possible rogue RS and rogue BS attack described in Section 3. In our verification, we include an additional restriction that only claims concerning sessions between trusted agents are evaluated.To illustrate our claims in the verification, we define a termKeyFields. Each data message exchanged between agents is composed of a set of key elements, for instance, pmk, ak, and szk.KeyFields is denoted as the set of elements. The three properties are explained in detail as follows.


#### 5.2.1. Information Confidentiality

This property is satisfied if the access nodes, that is, RS and MS, can make sure that all exchanged keys are kept in secret. This property requires that each individual elementα in K
e
y
F
i
e
l
d
s should be kept in secret.The general formalization of the information confidentiality property is given below:⁡
∀
α
∈
K
e
y
F
i
e
l
d
s
:
c
l
a
i
m
(
MS
/
RS
,
S
e
c
r
e
t
,
α
)
holds.This formalization can be expanded into the following six claims:Claim1:claim
(
MS
,
S
e
c
r
e
t
,
pmk
) holds.
Claim2:claim
(
MS
,
S
e
c
r
e
t
,
ak
) holds.
Claim3:claim
(
MS
,
S
e
c
r
e
t
,
szk
) holds.
Claim4:claim
(
RS
,
S
e
c
r
e
t
,
pmk
) holds.
Claim5:claim
(
RS
,
S
e
c
r
e
t
,
ak
) holds.
Claim6:claim
(
RS
,
S
e
c
r
e
t
,
szk
) holds.


#### 5.2.2. No Theft of Service Possible

This property is satisfied if (i) AS can be ensured that neither an unauthorized BS nor an unauthorized RS can be able to impersonate a legitimate one and get access to the network, and (ii) BS has the guarantee that an unauthenticated RS cannot gain access to the services provided, nor could it impersonate a legitimate user. A service should always be bound to an authenticated user. This property is similar to the information confidentiality property but involves different agents of the protocol. Its formal definition is given as follows:⁡
∀
α
∈
K
e
y
F
i
e
l
d
s
:
c
l
a
i
m
(
AS
/
BS
,
S
e
c
r
e
t
,
α
)
holds.This formalization is expanded into claims7–13:Claim7:claim
(
A
S
,
S
e
c
r
e
t
,
p
m
k
) holds.
Claim8:claim
(
A
S
,
S
e
c
r
e
t
,
s
z
k
) holds.
Claim9:claim
(
A
S
,
S
e
c
r
e
t
,
k
(
R
,
A
)
) holds.
Claim10:claim
(
A
S
,
S
e
c
r
e
t
,
k
(
B
,
A
)
) holds.
Claim11:claim
(
B
S
,
S
e
c
r
e
t
,
pmk
) holds.
Claim12:claim
(
B
S
,
S
e
c
r
e
t
,
ak
) holds.
Claim13:claim
(
B
S
,
S
e
c
r
e
t
,
szk
) holds.


#### 5.2.3. Message Sequence Synchronization

This property is satisfied if all of RS, BS, and AS can be ensured that the corresponding send and receive messages are executed in the order exactly the same as what has been specified in the protocol. Its formal definition is given as follows:Claim14:claim
(
R
S
,
N
i
s
y
n
c
h
) holds.
Claim15:claim
(
B
S
,
N
i
s
y
n
c
h
) holds.
Claim16:claim
(
A
S
,
N
i
s
y
n
c
h
) holds.
Claim17:claim
(
M
S
,
N
i
s
y
n
c
h
) holds.


### 5.3. Formal Verifications

Table1 shows the message elements that are used in our formal model.Table 1
Key element
Description
PMK
Pairwise Master Key
AK
Authorization Key
SZK
A group key used in the security zone among BS and many RSs
k(R, A)
K
R
S, RS’s secret key shared with AS
k(B, A)
K
B
S, BS’s secret key shared with ASRecall that, according to our protocol description in Section4, several keys will be transmitted. Hence we need to verify our protocols to see whether they can satisfy our secrecy claim and authentication claim.Below are the specific verification results from Scyther. There are three parts in each of the following figures of verification results. The first part is “Claim” which contains several columns indicating which elementα in KeyFields should be kept in secret in a specific node. For example, “R1, Secret pmk” represents key pmk in node R1 should be kept secret. The second part is “Status” which indicates the status of verification with a variety of possible attacks. “Ok” means the specific claim holds (i.e., passes the verification). “Failed” means possible attack(s) exists in this claim, indicating that the design of the authorization protocol might have some potential flaws. The third part is “Comments”; this part provides more specific information regarding the status. The expected results of our protocols are that all claims should hold; that is, we expect to see all status as “Ok” and no attacks are found.


#### 5.3.1. Protocol 1: RS Connects to the Network through BS

From the verification results in Figure4, we can see that all of our properties from Claim 4 to Claim 13 in Section 5.2 regarding key elements hold and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore, all the key information can be transmitted and exchanged safely.Figure 4
Verification result for Protocol 1.


#### 5.3.2. Protocol 2: RS Connects to the Network through Other RS(s)

(i) Verification Results for Phase 1. From the verification results in Figure 5, we can see that Claim 14 holds and no possible attacks are detected. This proves that our goal of message sequence synchronization is satisfied, and the requesting RS can successfully authenticate a legitimate edge RS.Figure 5
Verification result for Protocol 2-Phase 1, initial authentication.(ii) Verification Results for Phase 2. The verification results for phase 2 are shown in Figure 6, from which we can see that all of our claims from Claim 4 to Claim 13 regarding key element are held and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore all the key information can be transmitted and exchanged safely.Figure 6
Verification result for Protocol 2-Phase 2, mutual authentication.(iii) Verification Results for Phase 3. The verification results for phase 3 are shown in Figure 7, from which we can see that Claim 6 regarding the confidentiality of SZK holds and no possible attacks are detected. This proves that our goals of information confidentiality and no theft of service possible are satisfied.Figure 7
Verification result for Protocol 2-Phase 3, distributed authentication.


#### 5.3.3. Protocol 3: MS Connects to the Network through RS(s)

(i) Verification Results for Phase 1. The verification results for phase 1 are shown in Figure 8, from which we can see that Claims 17 and 14 hold and no possible attacks are detected. This proves that our goal of message sequence synchronization is satisfied, and the MS can successfully authenticate a legitimate edge RS.Figure 8
Verification result for Protocol 3-Phase 1, initial authentication.(ii) Verification Results for Phase 2. From Figure 9, we can see that all of our claims from Claim 1 to Claim 13 regarding key element are held and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore all the key information can be transmitted and exchanged safely.Figure 9
Verification result for Protocol 3-Phase 2, initial authentication.(iii) Verification Results for Phase 3. The verification results for phase 3 are shown in Figure 10, from which we can see that Claims 6 and 3 regarding the confidentiality of SZK hold and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied.Figure 10
Verification result for Protocol 3-Phase 3, distributed authentication.As seen in the above formal analysis, our claims for the secrecy and uniqueness of the exchanged key elements, the no theft of service possible, and message sequence synchronization are satisfied in all of the three protocols.


## 5.1. Model Description

In our model, we describe the behavior of the protocol entities in terms of their roles, that is, an initiator or a responder (receiver) or both. For example, for protocol 1, we have three agents, RS, BS, and AS. All of them play two roles: message initiator and message responder. In Scyther, arun is a unique execution of a role that is performed by an agent; that is, each agent executes its runs to implement the protocol and preserve the secrecy of the credentials (e.g., keys, licenses, and sequence number) it claims to achieve. Each agent has several variable definitions for the credentials used in the messages this agent will send or receive when playing different roles. Each agent also has a sequence of events that entails what messages this agent will send or receive, along with its claims. In Scyther, a claim is defined asclaim (A, c, P), which means that agentA expects goalc to hold with parameterP. If an attack exists, then the claim the agent wants to achieve will not hold. Such a claim is called a falsified claim [8].A claim has the locality property, so once agents get the messages, they will be able to view the state of the system from a local perspective. Therefore the protocol needs to make sure that the agent is able to have the knowledge of some properties of global state of the system from the local perspective; for example, the agent is able to know that something is beyond the intruder’s knowledge, or that a specific agent is active. However, for the same protocol, the claim on the same secret credential based on different agents might not always hold; that is, it is possible that for agentA as the message initiator and agentB as the message responderclaim (A, c, P) holds whileclaim (B, c, P) does not hold [8].In order to verify a security protocol using Scyther, we need to specify an attacker and a set of agents executing several runs. Scyther will trace all possible attacks that the attacker might launch and determine whether a security claim the agent holds is true or attacks exist. In the verification of our proposed protocols, we focus on two security properties:secrecy of keys andauthentication. To verify these two properties, we use two types of claims in our model: secrecy claim and authentication claim [8].(i) Secrecy Claim. A secrecy claim is defined asclaim (A, Secret, P). It is a statement that the credentials P included in this claim byA will not be obtained or spoofed by the attacker. Secrecy means that the information in question is not to the knowledge of an attacker, even if it is transmitted over an untrusted network.When agents are communicating data (public or secret credentials) with untrusted agents, the transmitted data is also open to the attacker. Although transmitted data is public in the air now, this does not imply that our protocol is broken. Rather, what we need is a secrecy claim saying that if an agent only speaks with trusted agents, then the data being transmitted or shared is kept in secret [8].(ii) Authentication Claim. In Scyther, authentication focuses on the verification that when a role in a protocol is executed, we can guarantee that in the current network there exists at least one entity that is communicating with this role. However, only knowing that some entity is communicating with the role is not enough to guarantee the correctness and robustness of an authentication protocol; we want to use Scyther to verify and show a stronger guarantee that when the protocol is being executed, the intended entity is aware of the communication, and the messages exchanged between the entity and the role follow the protocol description.An authentication claim is defined asclaim (A, Nisynch). It means agentA sends and receives messages in the order which is exactly the same as what has been described in the protocol [8].


## 5.2. Security Properties to Be Verified

We aim to verify that our proposed protocols have the following three security properties:information confidentiality,no theft of service possible [28], andmessage sequence synchronization. Successful verification of these three properties can prove the robustness of our proposed protocols against the possible rogue RS and rogue BS attack described in Section 3. In our verification, we include an additional restriction that only claims concerning sessions between trusted agents are evaluated.To illustrate our claims in the verification, we define a termKeyFields. Each data message exchanged between agents is composed of a set of key elements, for instance, pmk, ak, and szk.KeyFields is denoted as the set of elements. The three properties are explained in detail as follows.


### 5.2.1. Information Confidentiality

This property is satisfied if the access nodes, that is, RS and MS, can make sure that all exchanged keys are kept in secret. This property requires that each individual elementα in K
e
y
F
i
e
l
d
s should be kept in secret.The general formalization of the information confidentiality property is given below:⁡
∀
α
∈
K
e
y
F
i
e
l
d
s
:
c
l
a
i
m
(
MS
/
RS
,
S
e
c
r
e
t
,
α
)
holds.This formalization can be expanded into the following six claims:Claim1:claim
(
MS
,
S
e
c
r
e
t
,
pmk
) holds.
Claim2:claim
(
MS
,
S
e
c
r
e
t
,
ak
) holds.
Claim3:claim
(
MS
,
S
e
c
r
e
t
,
szk
) holds.
Claim4:claim
(
RS
,
S
e
c
r
e
t
,
pmk
) holds.
Claim5:claim
(
RS
,
S
e
c
r
e
t
,
ak
) holds.
Claim6:claim
(
RS
,
S
e
c
r
e
t
,
szk
) holds.


### 5.2.2. No Theft of Service Possible

This property is satisfied if (i) AS can be ensured that neither an unauthorized BS nor an unauthorized RS can be able to impersonate a legitimate one and get access to the network, and (ii) BS has the guarantee that an unauthenticated RS cannot gain access to the services provided, nor could it impersonate a legitimate user. A service should always be bound to an authenticated user. This property is similar to the information confidentiality property but involves different agents of the protocol. Its formal definition is given as follows:⁡
∀
α
∈
K
e
y
F
i
e
l
d
s
:
c
l
a
i
m
(
AS
/
BS
,
S
e
c
r
e
t
,
α
)
holds.This formalization is expanded into claims7–13:Claim7:claim
(
A
S
,
S
e
c
r
e
t
,
p
m
k
) holds.
Claim8:claim
(
A
S
,
S
e
c
r
e
t
,
s
z
k
) holds.
Claim9:claim
(
A
S
,
S
e
c
r
e
t
,
k
(
R
,
A
)
) holds.
Claim10:claim
(
A
S
,
S
e
c
r
e
t
,
k
(
B
,
A
)
) holds.
Claim11:claim
(
B
S
,
S
e
c
r
e
t
,
pmk
) holds.
Claim12:claim
(
B
S
,
S
e
c
r
e
t
,
ak
) holds.
Claim13:claim
(
B
S
,
S
e
c
r
e
t
,
szk
) holds.


### 5.2.3. Message Sequence Synchronization

This property is satisfied if all of RS, BS, and AS can be ensured that the corresponding send and receive messages are executed in the order exactly the same as what has been specified in the protocol. Its formal definition is given as follows:Claim14:claim
(
R
S
,
N
i
s
y
n
c
h
) holds.
Claim15:claim
(
B
S
,
N
i
s
y
n
c
h
) holds.
Claim16:claim
(
A
S
,
N
i
s
y
n
c
h
) holds.
Claim17:claim
(
M
S
,
N
i
s
y
n
c
h
) holds.


## 5.2.1. Information Confidentiality

This property is satisfied if the access nodes, that is, RS and MS, can make sure that all exchanged keys are kept in secret. This property requires that each individual elementα in K
e
y
F
i
e
l
d
s should be kept in secret.The general formalization of the information confidentiality property is given below:⁡
∀
α
∈
K
e
y
F
i
e
l
d
s
:
c
l
a
i
m
(
MS
/
RS
,
S
e
c
r
e
t
,
α
)
holds.This formalization can be expanded into the following six claims:Claim1:claim
(
MS
,
S
e
c
r
e
t
,
pmk
) holds.
Claim2:claim
(
MS
,
S
e
c
r
e
t
,
ak
) holds.
Claim3:claim
(
MS
,
S
e
c
r
e
t
,
szk
) holds.
Claim4:claim
(
RS
,
S
e
c
r
e
t
,
pmk
) holds.
Claim5:claim
(
RS
,
S
e
c
r
e
t
,
ak
) holds.
Claim6:claim
(
RS
,
S
e
c
r
e
t
,
szk
) holds.


## 5.2.2. No Theft of Service Possible

This property is satisfied if (i) AS can be ensured that neither an unauthorized BS nor an unauthorized RS can be able to impersonate a legitimate one and get access to the network, and (ii) BS has the guarantee that an unauthenticated RS cannot gain access to the services provided, nor could it impersonate a legitimate user. A service should always be bound to an authenticated user. This property is similar to the information confidentiality property but involves different agents of the protocol. Its formal definition is given as follows:⁡
∀
α
∈
K
e
y
F
i
e
l
d
s
:
c
l
a
i
m
(
AS
/
BS
,
S
e
c
r
e
t
,
α
)
holds.This formalization is expanded into claims7–13:Claim7:claim
(
A
S
,
S
e
c
r
e
t
,
p
m
k
) holds.
Claim8:claim
(
A
S
,
S
e
c
r
e
t
,
s
z
k
) holds.
Claim9:claim
(
A
S
,
S
e
c
r
e
t
,
k
(
R
,
A
)
) holds.
Claim10:claim
(
A
S
,
S
e
c
r
e
t
,
k
(
B
,
A
)
) holds.
Claim11:claim
(
B
S
,
S
e
c
r
e
t
,
pmk
) holds.
Claim12:claim
(
B
S
,
S
e
c
r
e
t
,
ak
) holds.
Claim13:claim
(
B
S
,
S
e
c
r
e
t
,
szk
) holds.


## 5.2.3. Message Sequence Synchronization

This property is satisfied if all of RS, BS, and AS can be ensured that the corresponding send and receive messages are executed in the order exactly the same as what has been specified in the protocol. Its formal definition is given as follows:Claim14:claim
(
R
S
,
N
i
s
y
n
c
h
) holds.
Claim15:claim
(
B
S
,
N
i
s
y
n
c
h
) holds.
Claim16:claim
(
A
S
,
N
i
s
y
n
c
h
) holds.
Claim17:claim
(
M
S
,
N
i
s
y
n
c
h
) holds.


## 5.3. Formal Verifications

Table1 shows the message elements that are used in our formal model.Table 1
Key element
Description
PMK
Pairwise Master Key
AK
Authorization Key
SZK
A group key used in the security zone among BS and many RSs
k(R, A)
K
R
S, RS’s secret key shared with AS
k(B, A)
K
B
S, BS’s secret key shared with ASRecall that, according to our protocol description in Section4, several keys will be transmitted. Hence we need to verify our protocols to see whether they can satisfy our secrecy claim and authentication claim.Below are the specific verification results from Scyther. There are three parts in each of the following figures of verification results. The first part is “Claim” which contains several columns indicating which elementα in KeyFields should be kept in secret in a specific node. For example, “R1, Secret pmk” represents key pmk in node R1 should be kept secret. The second part is “Status” which indicates the status of verification with a variety of possible attacks. “Ok” means the specific claim holds (i.e., passes the verification). “Failed” means possible attack(s) exists in this claim, indicating that the design of the authorization protocol might have some potential flaws. The third part is “Comments”; this part provides more specific information regarding the status. The expected results of our protocols are that all claims should hold; that is, we expect to see all status as “Ok” and no attacks are found.


### 5.3.1. Protocol 1: RS Connects to the Network through BS

From the verification results in Figure4, we can see that all of our properties from Claim 4 to Claim 13 in Section 5.2 regarding key elements hold and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore, all the key information can be transmitted and exchanged safely.Figure 4
Verification result for Protocol 1.


### 5.3.2. Protocol 2: RS Connects to the Network through Other RS(s)

(i) Verification Results for Phase 1. From the verification results in Figure 5, we can see that Claim 14 holds and no possible attacks are detected. This proves that our goal of message sequence synchronization is satisfied, and the requesting RS can successfully authenticate a legitimate edge RS.Figure 5
Verification result for Protocol 2-Phase 1, initial authentication.(ii) Verification Results for Phase 2. The verification results for phase 2 are shown in Figure 6, from which we can see that all of our claims from Claim 4 to Claim 13 regarding key element are held and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore all the key information can be transmitted and exchanged safely.Figure 6
Verification result for Protocol 2-Phase 2, mutual authentication.(iii) Verification Results for Phase 3. The verification results for phase 3 are shown in Figure 7, from which we can see that Claim 6 regarding the confidentiality of SZK holds and no possible attacks are detected. This proves that our goals of information confidentiality and no theft of service possible are satisfied.Figure 7
Verification result for Protocol 2-Phase 3, distributed authentication.


### 5.3.3. Protocol 3: MS Connects to the Network through RS(s)

(i) Verification Results for Phase 1. The verification results for phase 1 are shown in Figure 8, from which we can see that Claims 17 and 14 hold and no possible attacks are detected. This proves that our goal of message sequence synchronization is satisfied, and the MS can successfully authenticate a legitimate edge RS.Figure 8
Verification result for Protocol 3-Phase 1, initial authentication.(ii) Verification Results for Phase 2. From Figure 9, we can see that all of our claims from Claim 1 to Claim 13 regarding key element are held and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore all the key information can be transmitted and exchanged safely.Figure 9
Verification result for Protocol 3-Phase 2, initial authentication.(iii) Verification Results for Phase 3. The verification results for phase 3 are shown in Figure 10, from which we can see that Claims 6 and 3 regarding the confidentiality of SZK hold and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied.Figure 10
Verification result for Protocol 3-Phase 3, distributed authentication.As seen in the above formal analysis, our claims for the secrecy and uniqueness of the exchanged key elements, the no theft of service possible, and message sequence synchronization are satisfied in all of the three protocols.


## 5.3.1. Protocol 1: RS Connects to the Network through BS

From the verification results in Figure4, we can see that all of our properties from Claim 4 to Claim 13 in Section 5.2 regarding key elements hold and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore, all the key information can be transmitted and exchanged safely.Figure 4
Verification result for Protocol 1.


## 5.3.2. Protocol 2: RS Connects to the Network through Other RS(s)

(i) Verification Results for Phase 1. From the verification results in Figure 5, we can see that Claim 14 holds and no possible attacks are detected. This proves that our goal of message sequence synchronization is satisfied, and the requesting RS can successfully authenticate a legitimate edge RS.Figure 5
Verification result for Protocol 2-Phase 1, initial authentication.(ii) Verification Results for Phase 2. The verification results for phase 2 are shown in Figure 6, from which we can see that all of our claims from Claim 4 to Claim 13 regarding key element are held and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore all the key information can be transmitted and exchanged safely.Figure 6
Verification result for Protocol 2-Phase 2, mutual authentication.(iii) Verification Results for Phase 3. The verification results for phase 3 are shown in Figure 7, from which we can see that Claim 6 regarding the confidentiality of SZK holds and no possible attacks are detected. This proves that our goals of information confidentiality and no theft of service possible are satisfied.Figure 7
Verification result for Protocol 2-Phase 3, distributed authentication.


## 5.3.3. Protocol 3: MS Connects to the Network through RS(s)

(i) Verification Results for Phase 1. The verification results for phase 1 are shown in Figure 8, from which we can see that Claims 17 and 14 hold and no possible attacks are detected. This proves that our goal of message sequence synchronization is satisfied, and the MS can successfully authenticate a legitimate edge RS.Figure 8
Verification result for Protocol 3-Phase 1, initial authentication.(ii) Verification Results for Phase 2. From Figure 9, we can see that all of our claims from Claim 1 to Claim 13 regarding key element are held and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied. Therefore all the key information can be transmitted and exchanged safely.Figure 9
Verification result for Protocol 3-Phase 2, initial authentication.(iii) Verification Results for Phase 3. The verification results for phase 3 are shown in Figure 10, from which we can see that Claims 6 and 3 regarding the confidentiality of SZK hold and no possible attacks are detected. This proves that our goals ofinformation confidentiality andno theft of service possible are satisfied.Figure 10
Verification result for Protocol 3-Phase 3, distributed authentication.As seen in the above formal analysis, our claims for the secrecy and uniqueness of the exchanged key elements, the no theft of service possible, and message sequence synchronization are satisfied in all of the three protocols.


## 6. Concluding Remarks

In this paper, we present a set of new secure authentication protocols to address the attack of rogue BS/RS attack in the MMR WiMAX networks. First, relay station and mobile user authentication provide access control. Second, in order to protect the MMR network from rogue BS attack, an authentication server is used to conduct mutual authentication for both BS and RS/MS in the network. In this way, a rogue station can be detected at early stage and its harm to this network can be minimized. Third, a security zone key is generated and securely delivered to an authenticated BS by AS. BS can then distribute this SZK to legitimate RS within the network; in this way the communications within the network can be secured using SZK. Fourth, in order to reduce the overhead introduced by mutual authentications between nodes, a license and a distributed authentication are used as a pass ticket for nodes which have been authenticated within a valid period of time. Such a node with a valid license can get access to the network without having to go through the complete mutual authentication in AS; and a node with a valid license is trustworthy to give away SZK or provide authentication message forwarding. This method is useful for reducing the overhead especially when fast handoff happens frequently. To verify the security properties of our protocols, we apply the Scyther tool to conduct a formal verification. The verification does not find any attack with our protocols, which proves that our protocols satisfy three desirable security goals, namely,information confidentiality,no theft of service possible, andmessage sequence synchronization.In the future work, we will perform a numerical analysis on the authentication performance of our protocols in terms of key processing time, response time, and total overhead of provisioning the dual authentication, the security zone key, and the license.



---

*Source: 9859830-2016-09-08.xml*