akkadotnet/akka.net — pull request #134 by Aaronontheweb (2014-05-25 21:21:57+00:00)
Title: [Need Review] Virtualized IP addresses for remote actor systems

## The Problem

As far as I could tell when I was porting Akka's remoting system to C#, TypeSafe instituted a strong coupling between a network transport's physical IP address and the virtual IP address that is used to address that actor system, for reasons that amount to "Akka remoting is designed for use within a tightly controlled set of nodes" - you can see Roland Kuhn expand upon that here: http://stackoverflow.com/questions/22576020/akka-remote-over-the-internet

Meaning, that if I want to address a node via TCP using the following valid Akka.NET address:

````
akka.tcp://system1@10.14.86.185:8081/user/ChatServer
````

`System1` would have to bind exclusively to port 10.14.86.185 in order to receive those messages. So for instance, if I launched `System1` with the following configuration:

````
#this is the new upcoming remoting support, which enables multiple transports
helios.tcp {
transport-class = ""Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote""
applied-adapters = []
transport-protocol = tcp
port = 8081
hostname = 0.0.0.0 #listens on ALL ips for this machine
}
````

So `System` binds on port 0.0.0.0 - listening on all network interfaces. Any messages that an external actor system wants to send to `System`` that are addressed to 10.14.86.185 will be dropped by `System` with an "invalid address" error message.

So here's where this becomes a problem: suppose you're operating on EC2 and need to have some actors communicate via an Elastic IP address, which uses NAT under the hood if you're running in a VPC (which 100% of new EC2 deployments are by default now.)

Effectively, `System1` cannot bind to any possible IP address that can be reached by any external system outside the VPC - since you can't bind a socket to an IP address that isn't assigned to a network interface on the machine. In vanilla Akka, you can use some of the new Akka.IO stuff to communicate directly with the node using HTTP or TCP; this is brand new code that was released to market as I was porting the Akka Remoting layer a couple of months back.

Given that we don't have a dedicated company working on Akka.NET and thus won't have access to Akka.IO for a sufficiently long period of time, and because my company is trying to deploy a mature product on top of Akka.NET this week and I don't want to have to put my system administrator on suicide watch by forcing him to work around this (frankly, unnecessary and draconian IMO) rule, I'm going to propose a break away from Akka's stance on the subject.

## The Solution

It took all of 10 lines of code to add support for this, but I've made it possible to decouple a system's physical listening address and it's addressable actor system address via configuration.

In this update I introduced a new optional configuration element called `public-hostname` to deal with this issue and updated the ChatServer example to use it here: https://github.com/Aaronontheweb/akka.net/blob/f9a11b0190ec036b2595671bcbd66bef846f9fd6/Examples/Chat/ChatServer/Program.cs

The TCP configuration section from before would now look like this:

````
#this is the new upcoming remoting support, which enables multiple transports
helios.tcp {
transport-class = ""Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote""
applied-adapters = []
transport-protocol = tcp
port = 8081
hostname = 0.0.0.0 #listens on ALL ips for this machine
public-hostname = 10.14.86.185 #only accept messages addressed to this external IP
}
````

In this scenario, the low-level socket would bind on port 0.0.0.0:8081 - it can accept connections from the NAT and any other form of traffic that can reach port 8081 on any valid network interface on that machine.

However, the actor system's publicly known address is 10.14.86.185, the external IP that the socket can't be bound to directly by `System1` but can be reached by external actor systems that need to remote into `System1`. `System1` can accept traffic from any IP, but it will only accept messages that are specifically addressed to 10.14.86.185 - otherwise they get dropped, same as before. This gives Akka.NET users the ability to use Akka.Remote in commonplace production environments that use NATs.

By default, if the `public-hostname` field is not specified, then Akka defaults to its normal behavior - tight coupling between physical IP address and system address.

I'm going to be working on Akka.Cluster and other portions of our remoting layer in the future, and I don't think this change will have a negative impact on our ability to implement that. Having dealt with Cassandra clustering professionally for a couple of years and all sorts of fun issues with their external / internal IP addressing schemes, I don't think this poses an issue as long as the (1) `public-hostname` is considered to be the canonical IP for the system, (2) `physical-hostname` is a static address that always refers to the same node (i.e. it's not the IP for a load-balancer), and (3) that's what gets broadcast to other peers in the cluster.

Would appreciate some input on this from other community members - I felt that sticking with the Akka orthodoxy on the subject created more problems than it solved in this case.