Package: netunnel
NETunnelNETunnel is a tool to create network tunnels over HTTP/S written in Python 3.It can be used both in a client-server and in a server-server model.Getting StartedInstallingpipinstallnetunnelWe officially support Python 3.6+.UsageThe following example creates an HTTP tunnel from the client to the server's port 22 (SSHD service).Running the server: (In production, use --config-path to preserve changes)$python-mnetunnel.server
Theserverisrunninginstatelessmode.Use--config-pathtogenerateaconfigfile
netunnel_server-INFO-Generatingdefaultsecret-key
netunnel_server-INFO-Startingserveron127.0.0.1:4040Running the client:$python-mnetunnel.client--remote-port22Openingtunneltotheserver...
Tunnelentrancesocket:127.0.0.1:54781
Tunnelexitsocket:127.0.0.1:22The server's SSHD service is now accessible from the client:$ssh-p54781localhostPlease take a look at theexamplesdirectory for additional usages.How it worksThe client connects to the server and creates a websocket connection we call a "Channel". The channel
is used by server to send commands to the client and it performs heartbeat pings so both sides will know
if there are connection issues.The client makes a POST request to create a tunnel.Either the client or the server (depends on the tunnel's direction) listens on a socket locally
for incoming connections.For every new connection, it generates a websocket connection to the remote, and stream the data
from the connection over to the websocket and vice versa.Whenever a websocket is created, the remote connects to a socket locally and stream data from the
websocket to the socket and vice versa.Connection1 ->                      Websocket1 ->               ->
Connection2 ->  Tunnel Entrance ->  Websocket2 ->  Tunnel Exit  -> Service
Connection3 ->                      Websocket3 ->               ->Under the hoodThere are 2 core objects which performs the tunnelling:InputTunnel- The tunnel's entrance which listens on a socket.OutputTunnel- The tunnel's exit which creates connections to a socket.When a normal tunnel is created, the client creates an InputTunnel and the server creates
an OutputTunnel, while reverse tunnels are essentially the server is creating InputTunnel while
the client is creating an OutputTunnel.InputTunnel is initialized with a feeder of websockets that the client/server provides, so
that the implementation can be generic. In reverse tunnels, The server uses the channel to request
the client for a new websocket when it needs to feed a new websocket.Server ConfigurationsThe server's configuration file is optional, but recommended in production environments.When running the NETunnel server, you can provide a path to a configuration file using-cor--config-pathflags,
and the server will generate a default configuration file at that location.
If there is an existing configuration in that path, the server will load it, and merge it with its default
configurations, and for any change that was made dynamically to the server using the API, it will commit it to
the configuration file.The configuration file is in JSON format and support the following keys:allowed_tunnel_destinations- A key-value mapping of IPs and ports(as strings separated by comma) allowed to be
used as a tunnel's exit sockets. The special symbol*supported to allow all ports for a certain IP.
Defaults to{"127.0.0.1": "*"}secret_key- A passphrase used as an encryption key for sensitive settings to avoid storing them in the disk as plain text.
The key is generated automatically, but we recommend using the-s/--secret-keywhen running the server which will avoid
storing the key in the configuration file. Setting the environment variableNETUNNEL_SECRET_KEYwill behave just the
same as the flag, and won't be stored in the configuration. If you wish to decrypt, encrypt, or generate a key manually, seepython -m netunnel.common.security.peers- A list of remote NETunnel servers that can be used to set static tunnels (SeePeersin the Additional Features).
For an example of how to set a peer, look atexamples/server-server.allow_unverified_ssl_peers- When set totrue, remote peers certificates won't be verified. Defaults tofalse.revision- Currently unused. This will be used for configuration migrations purposes. You should not modify
this field manually in any use case.http_proxy- Settings for an optional global HTTP proxy to use for any requests the server may need to make, for
example to remote peers. The setting include a key-value mapping of the following:proxy_url- The URL to the remote proxy serverusername- An encrypted (using thesecret_key) username stringpassword- An encrypted (using thesecret_key) password stringA useful feature of NETunnel configuration is that it can parse environment variables on load to modify the default
values of any key. The configuration will search for variables starting with the prefixNETUNNEL_, following by the
uppercase of any existing key. The value is expected to be in JSON format.For example, in POSIX environments, running:exportNETUNNEL_ALLOWED_TUNNEL_DESTINATIONS='{"127.0.0.1": "22"}'exportNETUNNEL_ALLOW_UNVERIFIED_SSL_PEERS='true'python-mnetunnel.serverWill change the defaultallowed_tunnel_destinationsto{"127.0.0.1": "22"}and the defaultallow_unverified_ssl_peerstotrue.An example for a configuration file:examples/netunnel.example.confAdditional FeaturesPeers - A NETunnel server can register remote NETunnel servers called peers. The peers are stored in the
configuration and can be used to create static tunnels.Static Tunnels - NETunnel supports permanent tunnels between servers. This is a useful feature
for when we want a long term tunnels between machines. It can be used by making the netunnel server to run as a service
and create a configuration file with peers and static tunnels which initialized on startup. Both peers and static tunnels
can also be created dynamically via the server's API.
An example for a server-server model can be found here:examples/server-serverHTTP proxy support - You can use--proxy-url--proxy-username--proxy-passwordto configure a proxy
for the client. When used in a server-server model, there can be a global proxy used by the server to connect with.
The credentials of the global proxy in that case are stored encrypted in the server's configuration using a secret_key.Authentication plugins - By default, no authentication is made between NETunnel instances.
This can be configured by inherit the auth classes onnetunnel/common/auth.pyand pass them
to the client and server using--auth-plugin. A plugin example:examples/secret-auth-pluginChangelog1.0.5 (2021-03-30)FeaturesAdd a dynamic factory reset REST API callMiscSet marshmallow requirement to less than 4.0.01.0.4 (2021-01-25)MiscInclude LICENSE, CHANGES.md and README.md in the PyPi package.1.0.3 (2021-01-25)MiscExtend supported version ofimportlib-metadatato the lowest.