Package: vpcrouter
# VPC-router## Introduction### SummaryThe vpc-router implements automatic route failover and backup routes for AmazonVPC environments.vpc-router lets users avoid route table limitations and build large Kubernetesclusters with the performance and visibility of native VPC networking.It can also be used independently of Kubernetes whenever you need to manageroutes, backup routes and route failover in VPC environments.It provides a plugin architecture for the easy integration with other cloudorchestration systems.### Detailsvpc-router is a utility for setting and deleting routes in Amazon EC2 VPC routetables and specifically for automatically managing route failover.Each route is specified by a destination CIDR as well as a list of IP addressesof EC2 instances, which are eligible to receive packets for the CIDR. Aninstance from the list is chosen and a route is set. vpc-router continuouslymonitors instance health and performs an immediate route failover to anotherinstance in the set in case of a detected instance failure.Routes can be configured in different ways, but most commonly, vpc-router takesroute configs from storage (a config file, in the future also a KV store) orvia HTTP requests. It will make sure that routes in the VPC route table areupdated as needed with every detected change to the route config.By default, it applies all route updates to all the route tables it can findwithin a specified VPC.### Project originThis program was developed for the [Romana project](http://romana.io) toovercome the limit imposed on VPC route table entries (50 by default), whichconstricts the size of clusters. Avoiding this limit typically required runningan overlay network, which does not offer the performance and visibility ofnative VPC networking.Some users prefer to run CNI network providers that support more advancenetwork policy APIs. However, most CNI pod networks require an overlay whenclusters are split across Availability Zones (AZs), preventing HA clusters fromdelivering native VPC network performance. Romana, using vpc-router can buildCNI pod networks across zones without an overlay.While vpc-router was specifically designed for use with Romana and to takeadvantage of its topology aware IPAM in these Kubernetes deployment scenarios,it does not depend on either project and can also be used stand-alone.Plugins for integration with different environments are provided. For example,a [plugin to integrate with Romana](https://github.com/romana/vpcrouter-romana-plugin).Health-checks are also implemented via plugins. This means that vpc-router mayeither directly contact EC2 instances to check their health, or it may insteadconnect to AWS status and alert information, or use the node status provided byorchestration systems, such as Kubernetes.## Installation and runningYou can either run vpc-router out of the source directory, or perform a fullinstall, it can run outside of the VPC or within, on a cluster node or not.### Installation via pipThe vpc-router is in the Python Package Index (PyPi). Therefore, the simplestway to install it is just:pip install vpcrouter### Run vpc-router out of the source directoryIf you wish to work with the vpc-router sources, or [contribute](#contributing)to the project, you might want to run vpc-router directly from the sources.After downloading the code, create a virtual environment, activate it andinstall the required libraries. You can then use the `vpcrouter-runner.py`helper script to run vpc-router without a full install:$ git clone git@github.com:paninetworks/vpc-router.git$ virtualenv vpcrouter$ source vpcrouter/bin/activate$ cd vpc-router$ pip install -r requirements/deploy.txt$ ./vpcrouter-runner.py .....### Deploying in production environmentPlease see our documentation on how to[deploy vpc-router in production](DEPLOY.md), which covers issues such as:* Performing a proper installation* IaM permissions for running on EC2 instances* Running vpc-router in a container## Contributing### Feedback, bug reports, issue trackerWe welcome any contributions, bug reports or feedback. Please use our[issue tracker](https://github.com/romana/vpc-router/issues) to file bugs or requestadditional features.### Developing vpc-routerIn order to develop or extend vpc-router, please read the [developerdocumentation](DEVELOPERS.md) for information that might be useful to get youstarted.## Built-in HTTP server to see internal state and configvpc-router comes with a built-in HTTP server. By default it listens on`localhost:33289`. Send a GET request (with a web-browser, curl or wget, or anyclient you wish) to `http://localhost:33290/` to receive a JSON formattedoutput with the current internal configuration of vpc-router.The listen address and port can be modified with the `-a` (address) and `-p`(port) command line options.## Configuration### The route specvpc-router requires a route spec configuration in JSON format. It lookssomething like this:{"10.55.16.0/24" : [ "10.33.20.142" ],"10.66.17.0/24" : [ "10.33.20.93", "10.33.30.22" ]}Each entry in the dictionary is keyed on the route's CIDR and then lists anumber of eligible hosts, which can act as the target/router for this route.vpc-router randomly chooses an instance from a route's set of hosts.If a route to a specified CIDR does not exist in the VPC's route tables, or ifit does not specify a target that's contained in the specified host list,vpc-router creates or updates the route.The health of those hosts is continuously monitored. If a host acting as routerfails, vpc-router immediately switches the route to a different host from theset, if an alternative is available and healthy.Note that vpc-router takes control of the routing tables and removes anyentries of this type (interfaces on instances as target) if they are not partof the route spec.## Modes of operationThe modes for the detection of configuration updates are implemented viaplugins. It is therefore easy to extend vpc-router to integrate withvarious orchestration systems. [How to write plugins](PLUGINS.md) is documentedseparately.A plugin may either accept a route spec in the format described above, or itmay be used to translate other information into the properly formattedroute spec.### Internal mode pluginsOut of the box, plugins for three different configuration update modes areincluded in the vpc-router source:* configfile: Continuosly monitor a route spec configuration file for anychanges. The config file should contain the route spec in exactly the formatdescribed above.* http: Receive updated route specs via HTTP POSTs. The POSTed data should bethe route-spec in exactly the format described above.* fixedconf: With this a static config can be provided on the command line. Itis mostly used as a simple example for plugin developers. It does workand might be useful in some cases, but is not commonly going to be used inproduction. It translates the command line parameters into a route spec ofthe required format.### External mode pluginsIt is also possible to write external plugins, which live in their ownrepository. There is currently one example of this:* romana: The[Romana plugin](https://github.com/romana/vpcrouter-romana-plugin) providesseamless integration with the [Romana project](http://romana.io/), for thecreation of Kubernetes and OpenStack clusters without overlays, justnatively routed network traffic. The vpc-router with the Romana plugin thenallows those clusters to span multiple VPC Availability Zones, all whileusing native VPC networking and without being hindered by VPC's 50 routelimit. The Romana plugin watches the network topology knowledge from Romanaand translates it into the required route spec for vpc-router.### Mode 'configfile'The following command starts vpc-router as a service daemon in 'configfile'mode:$ vpcrouter -m configfile -f route-spec.conf -r us-east-1 -v vpc-350d6a51The used options are:* `-m configfile` tells vpc-router to take config changes from a specifiedroute spec file.* `-f` specifies the name of the route spec config file.* `-r` specifies the AWS region to which vpc-router should connect. Note: Thiscan be omitted if vpc-router is run on an instance in the region.* `-v` specifies the VPC for which vpc-router should perform route updates.Note: This can be omitted if vpc-router is run on an instance within the VPC.In 'configfile' mode the `-f` option must be used to specify the route specconfig file. It must exist when the server is started. The server thencontinuously monitors this file for changes.You can see an example route spec file in `examples/route_spec_1.conf`.### Mode 'http'The following command starts vpc-router as a service daemon in the 'http'mode. It utilizes the built-in HTTP server to listen for new route specs:$ vpcrouter -m http -r us-east-1 -v vpc-350d6a51The used options are:* `-m http` tells vpc-router to start listening on a certain address and portfor HTTP POST requests containing new route specs.* `-r` specifies the AWS region to which vpc-router should connect. Note: Thiscan be omitted if vpc-router is run on an instance in the region.* `-v` specifies the VPC for which vpc-router should perform route updates.Note: This can be omitted if vpc-router is run on an instance within the VPC.A new route spec can be POSTed to the `/route_spec` URL. The current route speccan be retrieved with a GET to that URL.For example:$ curl -X "POST" -H "Content-type:application/json" "http://localhost:33289/route_spec" -d '{"10.55.0.0/16" : [ "10.33.20.142" ], "10.66.17.0/24" : [ "10.33.20.93", "10.33.30.22" ]}'### Mode 'romana'For integration with the [Romana project](http://romana.io/), please see the[vpc-router Romana plugin](https://github.com/romana/vpcrouter-romana-plugin).## Continuous monitoringContinuos monitoring is performed for all hosts listed in the route spec. If aninstance does not appear healthy anymore and it is a current target for a routethen the route will be automatically updated to point to an alternate target,if a healthy one is available.The health-check itself is implemented via plugins, which gives vpc-router theflexibility to use a wide variety of information to determine whether an EC2routing instance is healthy. By default, it uses the 'icmpecho' plugin, whichutilizes an ICMPecho ('ping') request to actively check the responsiveness ofinstances. A 'tcp' plugin, which attempts TCP connection attempts on aspecified port, is also provided.Use the `--health` option to select the health monitor plugin, for example:$ vpcrouter --health tcp --tcp_check_port 22 --tcp_check_interval 5 ...or:$ vpcrouter --health icmpecho --icmp_check_interval 5 ...### The 'multi' plugin: Combining more than one health monitor pluginA `multi` plugin is also provided with vpc-router. This plugin allows thecombining of multiple, simpler health-monitoring plugins into complex ormulti-layered instance health monitoring. Use the `--health` option to selectthe `multi` plugin. Then use the `--multi_plugins` option to specify the listof simple health plugins that should be used simultaneously (a list of columnseparated health-monitor plugin names). Any additional command line optionsadded by those plugins can then also be specified.As an example, let's configure vpc-router to use the `icmpecho` plugin (with amonitoring interval of 2 seconds) as well as the `tcp` plugin, which shouldmonitor port 80.$ vpcrouter --health multi --multi_plugins icmpecho:tcp \--icmp_check_interval 2 --tcp_check_port 80 ...An instance is considered 'failed' if ANY of the specified sub-plugins reportsthe instance as failed.### Considering an instance as 'healthy' againIf a health monitoring plugin reports an instance as 'failed', it will beconsidered 'failed' for some amount of time (the exact time depends on theplugin, but usually it's 10 times the selected monitoring interval, when usingthe 'multi' plugin, it will be 20 times the largest interval of the specifiedsub-plugins).The health monitoring occasionally attempts to re-check failed instances to seeif they have recovered. If not, it will report them as failed again.If there has not been a 'failed' report about an instance within that timewindow, the instance will automatically be considered as 'healthy' again. Thisdoes not mean that routes are failing back to that instance, it just means thatthis instance becomes eligible to be a target for routes again.## TODOA 'todo' list is maintained in the[issue tracker](https://github.com/romana/vpc-router/issues) of the project.