HOWTO Improve your Developer Homelab/network with XCP-ng, pFSense, Ubiquity Unifi
Tags: homelab, linux, xcpng, unifi
Published: 2019-06-18 13:49:23

# Introduction

In the background you'll see a simple PC build, in this case

- Asus Z270 Prime-A
- 16GB Crucial RAM
- Intel 6700K
- [StarTech.com Dual Port PCI Express (PCIe x4) Gigabit Ethernet Server Adapter - 2 Port Network Card - Intel i350 NIC - GbE Network Card](https://amzn.to/2ISOpz4)
- [EVGA Supernova 750 P2, 80+ Platinum 750W, Fully Modular PSU](https://amzn.to/31H3Zqo)

In the Asus UEFI make sure to enable Intel Virtualisation option.

# System Config

- Installed XCP-ng v7.6 onto single SATA SSD.
- XCP-ng is tied to a _FreeNAS backend_.
- Mainboard Intel NIC used for XCP-ng management access - connected to internal EdgeRouter-x (ER-x) at gateway 10.0.0.1/24

# Appliance ports (yes, it's a PC though in this case...)

It's important to note that these are configured accordingly in the XCP-ng control software

- NIC0: Mainboard Intel NIC
- NIC1: I350 port 1 // pfSense WAN (xn0)
- NIC2: I350 port 2 // pfSense LAN (xn1)

# Ubiquity Unifi Switch 8 Managed PoE+ Gigabit Switch

- I have Unifi controller software running in an Ubuntu VM, on my primary XCP-ng server.  This handles all provisioning and control over Unifi hardware.
- The switch used here is the [Unifi Switch 8 Managed PoE+ Gigabit Switch with SFP](https://www.ui.com/unifi-switching/unifi-switch-8-150w/) shown below on the left in silver/grey.

- Blue cable --> Connects to the 10.0.0.0/24 network from the ER-x router
- Yellow cable --> pfSense LAN
- pfSense WAN port is also tied to the 10.0.0.0/24 network from the ER-x router
- Black cable --> Connect to my MacBook Pro.

# Ubiquity Unifi Switch 8 Port Configuration

- Port 1: Set to default 'All' Network, which by default is VLAN1 (trunking port).
- Port 2: Set to a custom Profile tied to Native LAN **VLAN2** // This is just a LAN profile with VLAN ID 2 and DHCP set to 'off'.  Profile is called **Profile-VLAN2**
- Port 3: Black cable to the MBP - in the image below is set to Network **LAN**

Notice how the laptop has a 10.0.0.0/24 IP.

![](https://thepracticaldev.s3.amazonaws.com/i/hkqpl66qniacxpidqmif.jpg)

Now, if we change the switch port Profile to **Profile-VLAN2** -- the magic happens here -- the laptop now gets a 192.168.100.0/24 IP -- YAY!!  This is crucial, as this is a virtual port (xn1) within pfSense.

![](https://thepracticaldev.s3.amazonaws.com/i/awh06fn0ji0du5k7fsjo.jpg)

# Great, so why would you do this?

By simply changing the port profile, any downstream traffic can now either be routed (1) straight from the main router (ER-x) or (2) through the ER-x ---> into the pFSense "Appliance" --> out the Black wire.

**This setup allows you to *switch* the physical network tied to the Black cable** 😀

I've also got IDS (Intrusion detection system) running in pFSense in the form of Suricata, which at the moment monitors the WAN interface.

I can also configure pFSense as an OpenVPN client, and all traffic through it will then be routed through a VPN -- my ultimate goal.

## What's the advantage of XCP-ng (XenServer)

XCP-ng (an opensource fork of Citrix XenServer) allows us to run pFSense in a Virtual Machine (VM) which allows us to setup periodic backups of said VM via cron-jobs from within XOA (Xen Orchestra).

Since these are coupled to a FreeNAS backend, all backups are stored securely in ZFS.

## Downsides to this approach

Right now, I don't have any need for VLANs through pFSense, but had I wanted to tag a VLAN entering the pFSense WAN port -- this is not possible as pfSense reports there are now VLAN capable interfaces attached.

Xen/XCP-ng has support for SR-IOV but I haven't gone down that rabbit hole.

## Generating VLANs

I wanted to be able to tag traffic coming out of xn1 (pfSense-LAN port) with a custom VLAN tag, but couldn't find away to do this; (1) it seems pFSense doesn't have the ability to do this (??) but only tag/decode incoming traffic via the WAN interface (2) it wouldn't be possible since XCP-ng insulates it from VLAN capable hardware (which the I350 NICs are capable off, 802.1q).

> I'd appreciate any correction to my statements above, as I'm sure there'll be someone better informed regarding tagging ports from within pFsense -- if it's possible, do let me know!

If you got this far, share your thoughts -- I'm keen to hear from you!!

Tweet to me @bsodmike :)