Open5GS-ERANSIM on AWS
Tags: aws, 5g, opensource, 5gran
Published: 2023-11-06 13:20:19

Delving into new technologies brings a variety of challenges to professionals in every field. For developers, the quest for a dependable and powerful infrastructure to deploy their code is paramount. Solution Architects, on the other hand, seek in-depth insights into testing environments to deliver superior solutions and identify possible defects or areas for enhancement proactively. Ultimately, this leads us to a universal query: Where can we find the liberty to fully exercise our technical capabilities?

Fortunately, the open-source community offers a remedy to these concerns, with a single, unique challenge: the execution and ongoing management of open-source software. Emphasizing collaboration is vital, and it is with this spirit that I intend to arm both 5G developers and Solution Architects through this post with a 5G RAN-Core Open-Source platform, which is ideal for comprehensive End-to-End simulations and diligent monitoring of Key Performance Indicators (KPIs).

##Proposed Topology:

AWS Topology includes Components such as:

- VPC
- Internet Gateway
- NAT gateway
- EC2 (T2.Medium) -> Selection based on Memory utilization. The cheapest option would be T4G.nano
For a cost estimation, please refer to the following link: 
[AWS_calculator](https://calculator.aws/)

![AWS Topology](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i4w4bxt3x1cw7amiwloh.jpg)

##IP Design & Requirements:

###5G-RAN&CORE

- The first image shows the VM requirements for Open5GS and ERANSIM open-source projects:

![5G-RAN&Core](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yt2m806wctie7xac7nlw.jpg)

- The following tables describe the 5G-RAN and 5G Core components main configurations (some of them are default values when deploying software packages)'

####gNB Configuration
![gNB_configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iubbgvrchdaawuv55bsh.jpg)

####AMF Configuration
![AMF_configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/33qkf0wdocd7v2ppndzz.jpg)

####SMF Configuration
![SMF_configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s4u6epo21odmcm4pbr73.jpg)

####UPF Configuration
![UPF_Configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sk5ccvwdfrwt02ed88ts.jpg)

###5G-UE


![5G-UE Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k4kawtbghaccrvzb847j.jpg)
- The table below describes UE setup:


![UE_Setup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/67tv0zj8rekyozxsbqza.jpg)

###Other Software Packages

One of the common issues you can face while deploying open-source projects is software version compatibility. The Below table details all tested software versions used for this 5GRAN-Core architecture.


![Software Packages](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dl2rxux9hkkiff3cofxq.jpg)
##Configuration:

For the initial software installation, I used the following references, which already explain the step-by-step installation:

1. Open5GS & UERANSIM 
[5G-Core Setup](https://medium.com/rahasak/5g-core-network-setup-with-open5gs-and-ueransim-cd0e77025fd7)
2. UERANSIM
[UERANSIM Installation](https://github.com/aligungr/UERANSIM/wiki/Installation)

In this blog though, I want to discuss the issues I found while executing the above guidelines.

##Troubleshooting:

**1. During Open5GS - UE Device Provisioning:**

This is a key step to perform E2E test cases, as we need to provision 5G UEs into UDR to complete initial registration.

During Step-4 (Register UE Device)

```
# install nodejs
sudo apt update
sudo apt install curl
curl -fsSL https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt install nodejs

# clone webui
git clone https://github.com/open5gs/open5gs.git

# run webui with npm
cd webui
npm run dev --host 0.0.0.0
```

The following error is shown:

```
node:internal/modules/cjs/loader:1080
throw err;
^

Error: Cannot find module 'co'
Require stack:

/home/freicy/open5gs/webui/server/index.js
at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
at Module._load (node:internal/modules/cjs/loader:922:27)
at Module.require (node:internal/modules/cjs/loader:1143:19)
at require (node:internal/modules/cjs/helpers:121:18)
at Object. (/home/freicy/open5gs/webui/server/index.js:6:12)
at Module._compile (node:internal/modules/cjs/loader:1256:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
at Module.load (node:internal/modules/cjs/loader:1119:32)
at Module._load (node:internal/modules/cjs/loader:960:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [ '/home/freicy/open5gs/webui/server/index.js' ]
}
```
In order to solve this error, NodeJS must be installed in a docker container as follows:
Reference: [NodeJS Installation](https://github.com/open5gs/open5gs/issues/2564)

```
$ cd docker
$ docker compose run dev
root@ip-10-0-14-98:~# sudo apt update
root@ip-10-0-14-98:~#sudo apt install -y ca-certificates curl gnupg
root@ip-10-0-14-98:~# sudo mkdir -p /etc/apt/keyrings
root@ip-10-0-14-98:~# curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
root@ip-10-0-14-98:~# NODE_MAJOR=20
root@ip-10-0-14-98:~# echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
root@ip-10-0-14-98:~# sudo apt update
root@ip-10-0-14-98:~# sudo apt install nodejs -y
root@ip-10-0-14-98:~# sudo npm run dev
```
Once Web interface is available, you should be able to login using default admin credentials: 
username - admin
password - 1423


![UE_GUI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qw70igwjjla7dg3k5myf.jpg)

After provisioning 5G-UE, the information will be shown as below:


![UE_Information](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/70o975dckzslho09abrd.jpg)

**2. During UERANSIM installation:**

After executing the following steps:

```
sudo snap install cmake --classic
cd ~/UERANSIM
make
```
The following error showed up:

```
/home/ubuntu/UERANSIM/src/ue.cpp: In function ‘nr::ue::UeConfig* ReadConfigYaml()’:
/home/ubuntu/UERANSIM/src/ue.cpp:164:17: error: ‘struct nr::ue::UeConfig’ has no member named ‘tunPrefix’
  164 |         result->tunPrefix = yaml::GetString(config, "tunPrefix", 1, 12);
      |                 ^~~~~~~~~
/home/ubuntu/UERANSIM/src/ue.cpp: In function ‘nr::ue::UeConfig* GetConfigByUe(int)’:
/home/ubuntu/UERANSIM/src/ue.cpp:362:8: error: ‘struct nr::ue::UeConfig’ has no member named ‘tunPrefix’
  362 |     c->tunPrefix = g_refConfig->tunPrefix;
      |        ^~~~~~~~~
/home/ubuntu/UERANSIM/src/ue.cpp:362:33: error: ‘struct nr::ue::UeConfig’ has no member named ‘tunPrefix’
  362 |     c->tunPrefix = g_refConfig->tunPrefix;
      |                                 ^~~~~~~~~
gmake[3]: *** [CMakeFiles/nr-ue.dir/build.make:76: CMakeFiles/nr-ue.dir/src/ue.cpp.o] Error 1
gmake[3]: Leaving directory '/home/ubuntu/UERANSIM/cmake-build-release'
gmake[2]: *** [CMakeFiles/Makefile2:270: CMakeFiles/nr-ue.dir/all] Error 2
gmake[2]: Leaving directory '/home/ubuntu/UERANSIM/cmake-build-release'
gmake[1]: *** [Makefile:91: all] Error 2
gmake[1]: Leaving directory '/home/ubuntu/UERANSIM/cmake-build-release'
make: *** [makefile:12: build] Error 2
```
**Workaround:**

I made the following changes, commenting lines #163-164 and 362:

```
vi /home/ubuntu/UERANSIM/src/ue.cpp
...
356     c->imeiSv = g_refConfig->imeiSv;
357     c->supi = g_refConfig->supi;
358     c->protectionScheme = g_refConfig->protectionScheme;
359     c->homeNetworkPublicKey = g_refConfig->homeNetworkPublicKey.copy();
360     c->homeNetworkPublicKeyId = g_refConfig->homeNetworkPublicKeyId;
361     c->routingIndicator = g_refConfig->routingIndicator;
362     //c->tunPrefix = g_refConfig->tunPrefix;
363     c->hplmn = g_refConfig->hplmn;
364     c->configuredNssai = g_refConfig->configuredNssai;
365     c->defaultConfiguredNssai = g_refConfig->defaultConfiguredNssai;
366     c->supportedAlgs = g_refConfig->supportedAlgs;
367     c->gnbSearchList = g_refConfig->gnbSearchList;
...
```
```
148     // If we have multiple UEs in the same process, then log names should be separated.
149     result->prefixLogger = g_options.count > 1;
150 
151     if (yaml::HasField(config, "supi"))
152         result->supi = Supi::Parse(yaml::GetString(config, "supi"));
153     if (yaml::HasField(config, "protectionScheme"))
154         result->protectionScheme = yaml::GetInt32(config, "protectionScheme", 0, 255);
155     if (yaml::HasField(config, "homeNetworkPublicKeyId"))
156         result->homeNetworkPublicKeyId = yaml::GetInt32(config, "homeNetworkPublicKeyId", 0, 255);
157     if (yaml::HasField(config, "homeNetworkPublicKey"))
158         result->homeNetworkPublicKey = OctetString::FromHex(yaml::GetString(config, "homeNetworkPublicKey", 64, 64));
159     if (yaml::HasField(config, "imei"))
160         result->imei = yaml::GetString(config, "imei", 15, 15);
161     if (yaml::HasField(config, "imeiSv"))
162         result->imeiSv = yaml::GetString(config, "imeiSv", 16, 16);
163     /*if (yaml::HasField(config, "tunPrefix"))
164         result->tunPrefix = yaml::GetString(config, "tunPrefix", 1, 12);*/
165 
166     yaml::AssertHasField(config, "integrity");
167     yaml::AssertHasField(config, "ciphering");
```
**Explanation:**
Commented lines refer to _TunPrefix_ attribute for 5G-UE, which is not a mandatory attribute for PDU Session Establishment.


##E2E Verification:

### NG Setup Procedure:
For verification, I will use the following commands from UERANSIM Server:

```
ubuntu@ip-10-0-31-9:~/UERANSIM/build$ pwd
/home/ubuntu/UERANSIM/build
ubuntu@ip-10-0-31-9:~/UERANSIM/build$ ./nr-gnb -c ../config/open5gs-gnb.yaml
```

```
gGNB:
UERANSIM v3.2.6
[2023-11-06 12:16:16.825] [sctp] [info] Trying to establish SCTP connection... (10.0.14.98:38412)
[2023-11-06 12:16:16.829] [sctp] [info] SCTP connection established (10.0.14.98:38412)
[2023-11-06 12:16:16.829] [sctp] [debug] SCTP association setup ascId[4]
[2023-11-06 12:16:16.830] [ngap] [debug] Sending NG Setup Request
[2023-11-06 12:16:16.831] [ngap] [debug] NG Setup Response received
[2023-11-06 12:16:16.831] [ngap] [info] NG Setup procedure is successful
```

```
AMF:

11/06 12:30:35.332: [amf] INFO: gNB-N2 accepted[10.0.31.9]:60504 in ng-path module (../src/amf/ngap-sctp.c:113)
11/06 12:30:35.332: [amf] INFO: gNB-N2 accepted[10.0.31.9] in master_sm module (../src/amf/amf-sm.c:741)
11/06 12:30:35.332: [amf] INFO: [Added] Number of gNBs is now 1 (../src/amf/context.c:1185)
11/06 12:30:35.332: [amf] INFO: gNB-N2[10.0.31.9] max_num_of_ostreams : 10 (../src/amf/amf-sm.c:780)

```

### UE Registration & PDU Session Establishment:

**gNB**
```
ubuntu@ip-10-0-31-9:~/UERANSIM/build$ ./nr-ue -c ../config/open5gs-ue.yaml
UERANSIM v3.2.6
[2023-11-06 12:31:38.643] [rrc] [debug] UE[1] new signal detected
[2023-11-06 12:31:38.644] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
[2023-11-06 12:31:38.645] [rrc] [debug] New signal detected for cell[1], total [1] cells in coverage
[2023-11-06 12:31:38.646] [nas] [info] Selected plmn[901/70]
[2023-11-06 12:31:38.646] [rrc] [info] Selected cell plmn[901/70] tac[1] category[SUITABLE]
[2023-11-06 12:31:38.646] [nas] [info] UE switches to state [MM-DEREGISTERED/PS]
[2023-11-06 12:31:38.646] [nas] [info] UE switches to state [MM-DEREGISTERED/NORMAL-SERVICE]
[2023-11-06 12:31:38.646] [nas] [debug] Initial registration required due to [MM-DEREG-NORMAL-SERVICE]
[2023-11-06 12:31:38.650] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
[2023-11-06 12:31:38.650] [nas] [debug] Sending Initial Registration
[2023-11-06 12:31:38.652] [nas] [info] UE switches to state [MM-REGISTER-INITIATED]
[2023-11-06 12:31:38.652] [rrc] [debug] Sending RRC Setup Request
[2023-11-06 12:31:38.652] [rrc] [info] RRC Setup for UE[1]
[2023-11-06 12:31:38.653] [rrc] [info] RRC connection established
[2023-11-06 12:31:38.653] [rrc] [info] UE switches to state [RRC-CONNECTED]
[2023-11-06 12:31:38.653] [nas] [info] UE switches to state [CM-CONNECTED]
[2023-11-06 12:31:38.653] [ngap] [debug] Initial NAS message received from UE[1]
[2023-11-06 12:31:38.666] [nas] [debug] Authentication Request received
[2023-11-06 12:31:38.673] [nas] [debug] Security Mode Command received
[2023-11-06 12:31:38.673] [nas] [debug] Selected integrity[2] ciphering[0]
[2023-11-06 12:31:38.694] [ngap] [debug] Initial Context Setup Request received
[2023-11-06 12:31:38.694] [nas] [debug] Registration accept received
[2023-11-06 12:31:38.695] [nas] [info] UE switches to state [MM-REGISTERED/NORMAL-SERVICE]
[2023-11-06 12:31:38.695] [nas] [debug] Sending Registration Complete
[2023-11-06 12:31:38.695] [nas] [info] Initial Registration is successful
[2023-11-06 12:31:38.695] [nas] [debug] Sending PDU Session Establishment Request
[2023-11-06 12:31:38.695] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
[2023-11-06 12:31:38.899] [nas] [debug] Configuration Update Command received
[2023-11-06 12:31:38.917] [ngap] [info] PDU session resource(s) setup for UE[1] count[1]
[2023-11-06 12:31:38.918] [nas] [debug] PDU Session Establishment Accept received
[2023-11-06 12:31:38.918] [nas] [info] PDU Session establishment is successful PSI[1]
```

**AMF**
```
ubuntu@ip-10-0-14-98:~$ sudo tail -f /var/log/open5gs/amf.log
11/06 12:31:38.654: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:401)
11/06 12:31:38.654: [amf] INFO: [Added] Number of gNB-UEs is now 1 (../src/amf/context.c:2523)
11/06 12:31:38.654: [amf] INFO:     RAN_UE_NGAP_ID[1] AMF_UE_NGAP_ID[1] TAC[1] CellID[0x10] (../src/amf/ngap-handler.c:562)
11/06 12:31:38.655: [amf] INFO: [suci-0-901-70-0000-0-0-0000000001] Unknown UE by SUCI (../src/amf/context.c:1789)
11/06 12:31:38.655: [amf] INFO: [Added] Number of AMF-UEs is now 1 (../src/amf/context.c:1570)
11/06 12:31:38.655: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:1061)
11/06 12:31:38.655: [gmm] INFO: [suci-0-901-70-0000-0-0-0000000001]    SUCI (../src/amf/gmm-handler.c:157)
11/06 12:31:38.898: [gmm] INFO: [imsi-901700000000001] Registration complete (../src/amf/gmm-sm.c:1993)
11/06 12:31:38.898: [amf] INFO: [imsi-901700000000001] Configuration update command (../src/amf/nas-path.c:612)
11/06 12:31:38.898: [gmm] INFO:     UTC [2023-11-06T12:31:38] Timezone[0]/DST[0] (../src/amf/gmm-build.c:558)
11/06 12:31:38.898: [gmm] INFO:     LOCAL [2023-11-06T12:31:38] Timezone[0]/DST[0] (../src/amf/gmm-build.c:563)
11/06 12:31:38.898: [amf] INFO: [Added] Number of AMF-Sessions is now 1 (../src/amf/context.c:2544)
11/06 12:31:38.898: [gmm] INFO: UE SUPI[imsi-901700000000001] DNN[internet] S_NSSAI[SST:1 SD:0xffffff] (../src/amf/gmm-handler.c:1247)
11/06 12:31:38.922: [amf] INFO: [imsi-901700000000001:1:11][0:0:NULL] /nsmf-pdusession/v1/sm-contexts/{smContextRef}/modify (../src/amf/nsmf-handler.c:837)
```
**UPF**

```
ubuntu@ip-10-0-14-98:~$ sudo tail -f /var/log/open5gs/upf.log
11/06 11:29:26.714: [pfcp] INFO: ogs_pfcp_connect() [127.0.0.4]:8805 (../lib/pfcp/path.c:61)
11/06 11:29:26.714: [upf] INFO: PFCP associated [127.0.0.4]:8805 (../src/upf/pfcp-sm.c:184)
11/06 12:31:38.911: [upf] INFO: [Added] Number of UPF-Sessions is now 1 (../src/upf/context.c:206)
11/06 12:31:38.911: [gtp] INFO: gtp_connect() [127.0.0.4]:2152 (../lib/gtp/path.c:60)
11/06 12:31:38.911: [upf] INFO: UE F-SEID[UP:0x876 CP:0x9c2] APN[internet] PDN-Type[1] IPv4[10.45.0.2] IPv6[] (../src/upf/context.c:483)
11/06 12:31:38.911: [upf] INFO: UE F-SEID[UP:0x876 CP:0x9c2] APN[internet] PDN-Type[1] IPv4[10.45.0.2] IPv6[] (../src/upf/context.c:483)
11/06 12:31:38.919: [gtp] INFO: gtp_connect() [10.0.31.9]:2152 (../lib/gtp/path.c:60)
```
##KPI Monitoring (Ongoing):

Let's talk about the multiple KPIs we can refer for this 5G RAN-Core solution.

**NFVI:**
- CPU utilization (%)
- Network in (bytes)
- Network out (bytes)
- Disk reads (bytes)
- Disk read operations (operations)
- Disk writes (bytes)
- Memory Utilization

Thanks to AWS existing Monitoring tools, such as CloudWatch, we can achieve NFVI monitoring at a granular level:


![5G_RAN_Monitoring](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x1kwoz2irue0zq6uyq9o.jpg)

**5G:**

- Initial Registration Failure ratio (AMF)
- Service Request Failure ratio (AMF-SMF)
- Number of Subscribers (SMF)
- PCFP Association Setup/Session Establishment Failure Ratio (SMF-UPF)
- Packet Loss (UPF)
- Data Volume & Guaranteed Data rate (UPF)

**RAN**
- Loading..

Thank you for reading all the way through this post. In upcoming ones, I will discuss the challenges and benefits of deploying this same architecture in AWS Contenarized solution (AWS EKS)

Happy Learning!