Setting up TWS & IBC on EC2 instance
Tags: aws, interactivebrokers, ec2, tws
Published: 2022-10-11 07:15:09

Probably if you are a developer who was always interested in the stock market or any other type of investing, you have at least once thought about creating some sort of system that will help you get the market data and then use it for analysis

Especially these days when the stock market is crashing, the Japanese yen and British pound are hitting record lows against the dollar, etc.

![I am fine meme](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8wh5766xwjiti3fs2802.jpg)

Probably this is you when you open your portfolio

---

And because you definitely do not want to be in such a situation, you made a research on what kind of options you have to get the market data and found something called IB Trader Workstation (or TWS)

You start digging more and turns out that this TWS is actually exactly what you need - it gives you access not only to historical data but real-time too. Moreover, you get access to analysis tools, interactive, customizable charts, you can execute orders, etc.

Sounds quite interesting, right?
So, you go and install it on your local machine, play with it for a week as you absolutely loved it

---

Then you finally want to start coding and you stumble into TWS API which is a simple interface to TWS through which you can do a lot of stuff - from automating your trading strategies & requesting market data to monitoring your account balance and portfolio in real-time.

And the main requirement for this API to work is a running instance of Trader Workstation app (or IB Gateway, but we are not here to talk about it)

As TWS been already installed on your machine, you start writing code using TWS API - either its native version or any third-party libraries such as ib_insync.

You test your code and it works perfectly fine!

But then you start thinking that running TWS on your local machine is not a long-term solution and you decide to switch everything to the cloud

---

You think about which cloud platform you should use and decide to go with AWS, cause Jeff Bezos is the best

![Jeff Bezos stonks meme](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z0gwwzhvbph6y4bbhycf.jpg)

The first thing you have to do is launch the EC2 instance which we will use to run our Trader Workstation app

We are going to use Ubuntu as our AMI and for the instance type we will choose t2.medium (cause if you choose a less powerful instance type, TWS will work a little bit slow)

I suppose now you have a running EC2 Ubuntu instance with the instance type mentioned above and now let's start working on the installation of TWS on this EC2 machine

---

First, let's connect to your EC2 instance.
One of the easiest ways is through using SSH (you also need to have access to the file with the key pair that you used when creating the EC2 instance)

Choose your EC2 instance and click on the "Connect" button

![Connect button](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1mf64bppq0vlerdcqqsj.png)

From here, choose "SSH client" and just copy the example code

![SSH code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mmehghp9gv0ak8r7n9te.png)

Then, use the terminal of your choice, and run the copied command (make sure that you run this command from the same folder where your key file is located)

You successfully connected to your EC2 and now let's run the following general commands

```
sudo apt update
sudo apt -y upgrade
```

After that, we have to install 2 additional things: XVFB and VNC server

**XVFB** - is an X server that can run on machines with no display hardware and no physical input devices. It emulates a dumb framebuffer using virtual memory. 
In simple words, we will use it to add GUI to our EC2 instance.

**VNC** - is basically a screen-sharing system that is used to remotely control another computer.
It works on a client/server model which means that we install the VNC server on the remote computer - the one we want to control (in our case the EC2 instance) and on our local machine we install the VNC Client that we use to control another computer

To install XVFB run this command:

```
sudo apt install -y xvfb
```

As our VNC server, we will use **x11vnc** which allows us to view remotely and interact with real X displays (i.e. a display corresponding to a physical monitor, keyboard, and mouse) with any VNC viewer.

```
sudo apt install -y x11vnc
```

Then, run this command to start virtual frame buffer

```
/usr/bin/Xvfb :0 -ac -screen 0 1024x768x24 &
```
When you run this command, you should get the ID of the Xvfb background process

![Xvfb process ID](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zci435rxhp28r3ynj3nl.png)

To start the VNC server, we will run this command where we specify 2 passwords - full access (keyboard + mouse) and second is view only.

```
/usr/bin/x11vnc -ncache 10 -ncache_cr -viewpasswd view_only_password -passwd full_access_password -display :0 -forever -shared -bg -noipv6
```

Also, we define what server display it should connect to (in our case, it is :0).
_**noipv6**_ option states that it should not try to use IPv6 for any listening or connecting sockets.
The _**shared**_ option allows more than one viewer to connect at the same time and the _**forever**_ option tells our VNC server to keep listening for more connections rather than exiting as soon as the first client disconnects

![VNC Server port](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jlaoivok5rys7qs807nj.png)

Now VNC server is listening on port 5900 and we can connect to it using any VNC client you want 

I am going to use [TightVNC](https://www.tightvnc.com/download.php) but there are many other options such as TigerVNC, UltraVNC, etc. 

Before connecting to our server, we have to make a few adjustments to the security group assigned to our EC2 instance.
Basically, we need to allow inbound traffic to port 5900 and we can easily do this by going to the AWS console, choosing our instance's security group and editing its inbound rules

![Chooseing EC2 security group](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e906nechlx06mplpenxo.png)

Here we add a new rule of type Custom TCP and define our IP as a source

![Adding new inbound rule](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qvnq6ly8f3p6d9brdwj6.png)

After that, we connect to the VNC Server using TightVNC Viewer.
In the hostname field, we enter the public DNS of our EC2 instance

![EC2 public DNS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ll4sjpwwwtdwud8zxuq7.png)

And we use the full access password that you entered when starting the VNC Server

![VNC authentication](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wpgdrj68pr5gq1byyjly.png)

After connecting to our VNC server, let's make our life a little bit easier and install metacity - a lightweight window manager.

```
sudo apt install -y metacity
```

And run this command:

```
DISPLAY=:0 metacity &
```

---

Now we are ready to install the Trader Workstation app.
First, let's install offline TWS for Linux on our local machine

[Offline TWS for Linux](https://www.interactivebrokers.com/en/index.php?f=16044&ns=T) 
 
After installation is complete, we will copy this file to our EC2 instance.

In order to do that, we will use the **_scp_** CLI tool that allows you to securely copy files and directories between two locations

```
scp -i /path_to_key_file tws-latest-standalone-linux-x64.sh ubuntu@public-ip-address-of-ec2-instance:
```
Do not forget to add ":" at the end of a public IP address

This command should copy the installation file from our local machine to the EC2 instance (to the home directory)

We can check this by running the _**ls**_ command:

![Copied file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zao1xsckgph7pflrgo2v.png)

Now let's make this file executable 

```
chmod a+x ~/tws-latest-standalone-linux-x64.sh
```

And as we have Xvfb installed, we can run the TWS installer on display ":0"

```
DISPLAY=:0 ~/tws-latest-standalone-linux-x64.sh
```

In TightVNC Viewer, you can see that the installation window is opened

![TWS installer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gndi4o5uqsut0q7441i6.png)

Follow the steps to complete the TWS installation.
When you finish installation, you can run the _**ls**_ command

![New Jts folder](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wde26rc5icux0pu3546i.png)

As you can see, the Jts folder was created which contains all TWS-required files

Now you just have to use TWS credentials to log in and configure TWS as you want

![TWS login page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hywctqebnwx4q9ec24fg.png)

And now you have the Trader Workstation app hosted on an EC2 instance!

---

So, let's say you've been using it for a few days and noticed that sometimes your TWS loses connection and asks for manual authentication
And it is definitely not the way you wanted it to work

Wouldn't it be perfect if there were a tool that can run TWS in 'hands-free' mode (without any manual intervention), right?

I agree that it would be nice to have such a tool and fortunately for us, there is one called [IBC](https://github.com/IbcAlpha/IBC/blob/master/userguide.md)

You can follow the link above and read more about what IBC can do, but for our purpose, we will use it to start the TWS app and login automatically in case our TWS loses connection

First, we need to install it by running this command:

```
wget https://github.com/IbcAlpha/IBC/releases/download/3.14.0/IBCLinux-3.14.0.zip
```

Then, let's install unzip command (if you don't have it installed already)

```
sudo apt install -y unzip
```

Before extracting the contents of the downloaded ZIP file, I want you to create a new folder - _**/opt/ibc/**_

We do this because several IBC script files assume the default paths used for particular files (config file, TWS & IBC program files) and by using these default paths we will minimize customizing the configuration file and the shell scripts.

```
mkdir -p opt/ibc/
cd opt/ibc
```

Now let's unzip a file

```
unzip ../../IBCLinux-3.14.0.zip
```

Run these 2 commands to make sure all .sh files are executable:

```
chmod u+x *.sh
chmod u+x scripts/*.sh
```

Let's go back to the home directory and create another new folder - **_ibc_** where we will store the configuration file

```
cd ../..
mkdir ibc
```

Copy configuration file from /opt/ibc to the newly created folder

```
cp opt/ibc/config.ini ~/ibc/
```

Now we have to make a few changes to that config file (use the editor of your choice):

```
vi ibc/config.ini
```

Here we have to change the username & password fields

![Modify config file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nszwpdte2cjs4979mi4q.png)

Save the changes and let's go back to the /opt/ibc/ folder
Here open the twsstart.sh file using an editor of your choice

```
cd opt/ibc/
vi twsstart.sh
```

For the trading mode, specify the one you need - either live or paper (I will use live)

And also, we need to change the TWS version.
We can  find the TWS major version number by running the TWS app, clicking the _**Help**_ button on the top navigation menu, then _**About Trader Workstation**_ option

![TWS major version](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/svy7frijyuoo8ia0xr3m.png)

In the displayed information you'll see a TWS version

Going back to twsstart.sh file, modify the TWS_MAJOR_VRSN property
And for the IBC_PATH parameter add ~ before /opt/ibc

![Modify script file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jkxv25nu80u2luy509od.png)

Save the changes and let's try to start IBC!
Before doing this, make sure that the previously started TWS is no longer running

```
DISPLAY=:0 ~/opt/ibc/twsstart.sh
```

After running this command, you should see TWS been started in VNC Viewer

![IBC started](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0ihbe77g9ruile0v73ue.png)

![TWS started](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/07do29pm0j5qqcwtfnwf.png)

And if you want to make IBC run every hour to check whether TWS lost connection and requires re-login, we can use crontab and schedule a cronjob that will run this command (DISPLAY=:0 ~/opt/ibc/twsstart.sh) every hour

Run this command:

```
crontab -e
```

And add the following line to the end of the file (you can adjust the time according to your needs)

```
0 * * * * DISPLAY=:0 /home/ubuntu/opt/ibc/twsstart.sh
```

---

And that's it, guys.

I hope that you have learned something new today!
I would appreciate it if you could like this post or leave a comment below!

Also, I want to mention that I used this [amazing article](https://dimon.ca/how-to-setup-ibc-and-tws-on-headless-ubuntu-in-10-minutes/#h.udghm8581dwb) as a reference and my post is just a shorter and easier version of that article

Also, feel free to follow me on [GitHub](https://github.com/Kai4ik) and [Medium](https://medium.com/@kai4ik)!

Adios, mi amigos)