Project: Rudder (Provisioning / Security & Compliance)
File: webapp.md

This is a straightforward guide aims to setup all tools and settings on a Linux OS, to be able to modify Rudder source code and to test it locally. Please follow this guide step by step. If you want to learn more about the tools feel free to read the document link to it ! Note that the NCF Api and the Technique Editor are not part of the webapp and will require further steps to setup. (Link and explanations provided below in the relative section) Vagrant Virtual Box and Oracle VM VirutalBox Extension pack IntelliJ OpenJDK 8 at least git netstat - - - - - - - - - - - - - If you are starting with a clean machine, you might want to take it all, should be needed at some point `apt update && apt install -y git openssh-server python3 python3-pip python python-pip openjdk-11-jdk net-tools ldap-utils nodejs npm maven` `pip install docopt requests pexpect urllib3` (or `pip3`) ```bash add-apt-repository "deb http://download.virtualbox.org/virtualbox/debian bionic contrib" wget -qO- https://www.virtualbox.org/download/oraclevbox2016.asc | sudo apt-key add - apt update vboxlatest=$(wget -qO- https://download.virtualbox.org/virtualbox/LATEST.TXT) vboxversion=$(echo $vboxlatest | cut -c1-3) apt install -y virtualbox-${vboxversion} wget https://download.virtualbox.org/virtualbox/${vboxlatest}/OracleVMVirtualBoxExtensionPack-${vboxlatest}.vbox-extpack echo y | sudo vboxmanage extpack install --replace OracleVMVirtualBoxExtensionPack-${vboxlatest}.vbox-extpack rm -rf OracleVMVirtualBoxExtensionPack-${vboxlatest}.vbox-extpack mkdir -p /etc/vbox echo '* 10.0.0.0/8 192.168.0.0/16' >> /etc/vbox/networks.conf ``` ```bash vagrantversion=$(wget -qO- https://raw.githubusercontent.com/hashicorp/vagrant/stable-website/version.txt) wget https://releases.hashicorp.com/vagrant/${vagrantversion}/vagrant${vagrantversion}x86_64.deb apt install -y ./vagrant${vagrantversion}x86_64.deb rm -rf vagrant${vagrantversion}x86_64.deb ``` ```bash wget https://download.jetbrains.com/idea/ideaIC-2019.3.3.tar.gz sudo tar -xzf ideaIC-2019.3.3.tar.gz -C /opt rm -rf ideaIC-2019.3.3.tar.gz ``` For later use: `/opt/idea-IC-193.6494.35/bin/idea.sh` to start intellij ```bash wget "http://apache.mirrors.ovh.net/ftp.apache.org/dist/directory/studio/2.0.0.v20180908-M14/ApacheDirectoryStudio-2.0.0.v20180908-M14-linux.gtk.x86_64.tar.gz" sudo tar -xzf ApacheDirectoryStudio-2.0.0.v20180908-M14-linux.gtk.x86_64.tar.gz -C /opt rm -rf ApacheDirectoryStudio-2.0.0.v20180908-M14-linux.gtk.x86_64.tar.gz ``` For later use: `/opt/ApacheDirectoryStudio/ApacheDirectoryStudio` to start Apache Directory Studio ```bash wget -qO elm.gz https://github.com/elm/compiler/releases/download/0.19.0/binary-for-linux-64-bit.gz gzip -d elm.gz chmod +x elm mv elm /usr/local/bin/ ``` ```bash wget -qO elm-0.19.1.gz https://github.com/elm/compiler/releases/download/0.19.1/binary-for-linux-64-bit.gz gzip -d elm-0.19.1.gz chmod +x elm-0.19.1 mv elm-0.19.1 /usr/local/bin/ ``` Please follow these to install Rudder Test framework Follow these instructions to then create a symlink in `rudder-tests` : ``` ln -s <rudder-api-client directory's path> ``` `cd` into the directory of the previously installed RTF. We'll name it: `rudder-test` here. `./platform/` contains examples of platform configurations. Please read https://github.com/Normation/rudder-tests#adding-a-platform-or-an-os for further information. Create a `<devenvname>.json` file in `./platform/` and put your platform's configuration in it. Here is the most minimalistic example of a functional configuration: ```json { "default":{ "run-with": "vagrant", "rudder-version": "6.0", "system": "debian9", "inventory-os": "debian" }, "server": { "rudder-setup": "dev-server" } } ``` Depending on your needs, a relay and some agents can be added: ```json ... "relay": { "rudder-setup": "relay" }, "agent1": { "rudder-setup": "agent", "system" : "debian8", "server": "relay" }, "agent2": { "rudder-setup": "agent", "system" : "debian7" }, ... ``` Note: for testing purpose a local development environment is needed: this implies that the `server` field must at least contain `"rudder-setup":

But if you only want to test Rudder without making any changes to the source code, replace `"rudder-setup": "dev-server"` by `"rudder-setup": "server"` Note: you will not be able to access Rudder at this stage, you will need to setup and run Rudder through IntelliJ to be able to access it locally. When your file is ready, you will have to create and prepare the box by running : ``` ./rtf platform setup <env's name> ``` Important note: always do either: `./rtf platform setup <env's name>` from the `rudder-tests` folder, or `vagrant up <VMid>`. The `<VMid>` can be found by doing `vagrant global-status` as the user that created the VM in the first place Now you can test that the environment is working by typing: ``` sudo netstat -laputn | grep 15432 ``` The output should be: ``` tcp 0 0 0.0.0.0:15432 0.0.0.0:* LISTEN ``` You can find the port number back in the Vagrantfile. For an eventual later use: this will not be necessary in this guide, but if you want to connect to the box, use (in `rudder-test` directory) : ``` vagrant ssh <env's name>_server ``` Meaning following steps are to be executed on your machine, not on the created VM unless specified otherwise, the place repos are cloned is not important. Maybe keep them together in `~/rudder/<repos>`. Create necessary groups ``` groupadd rudder groupadd rudder-policy-reder usermod -a -G rudder <username> usermod -a -G rudder-policy-reader <username> ``` Please note that this change will need to start a new shell or session to be taken into account Create some file for the webapp: ``` mkdir -p /var/rudder/inventories/incoming /var/rudder/share /var/rudder/inventories/accepted-nodes-updates /var/rudder/inventories/received /var/rudder/inventories/failed /var/log/rudder/core /var/log/rudder/compliance/ /var/rudder/run/ touch /var/log/rudder/core/rudder-webapp.log /var/log/rudder/compliance/non-compliant-reports.log /var/rudder/run/api-token ``` Add permissions ``` sudo chown -R <username> /var/rudder/ sudo chown -R <username> /var/log/rudder sudo chgrp -R rudder /var/rudder/configuration-repository sudo chmod -R 770 /var/rudder/share ``` Clone rudder-techniques repo by : ``` git clone https://github.com/Normation/rudder-techniques.git ``` Move technique's directory to `/var/rudder/configuration-repository/` ``` cp -r rudder-techniques/techniques /var/rudder/configuration-repository/ ``` These steps get you the base techniques. To sync it: git add and commit techniques directory in `/var/rudder/configuration-repository/` ``` git add techniques/ git commit -m "what ever you want" ``` Meaning following steps are to be executed on the newly created VM machine. But first, you need to establish a connection (1.) Connect to your vagrant box env, in `rudder-tests` directory : ``` vagrant ssh <env's name>_server ``` In vagrant box, find in

the two lines starting with `rootdn` and `rootpw` and keep them for the next section rootdn should look like that : `rootdn "cn=Manager,cn=rudder-configuration"` Run Apache Directory Studio (installed in Part 0) in LDAP -> New Connection define a connection's name Hostname : localhost Port : 1389 Click on Check Network Parameter, a windows should appear and tell you that the connection was established successfully if so click on next Complete with following parameters : Bind DN or user : `cn=Manager,cn=rudder-configuration` (`rootdn` parameter from previous step) Bind password : the `rootpw` value click on Check the Authentication, it should be a success, click on apply and close if so Your LDAP connection is now working! In IntelliJ install plugins : Scala, Elm, Jetty Runner and File watcher (recommended) Import your own forked rudder repository (for example): ``` git clone git@github.com:<gituser>/rudder.git ``` In IntelliJ : Import Project or File -> New -> Module from existing sources and choose `/rudder/webapp/sources/rudder` or you can choose only the modules that you need (compliance will be faster) Then select "Import Module from external model" and go for Maven In Run -> Edit Configuration add a new configuration and choose Jetty Runner Jetty Runner Folder : _downloaded jar of jetty-runner with version BELOW 11 from https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-runner/ (jetty 11 is not supported yet because of the lack of support for the servlet version used by rudder)_ Module : <all modules> Path : `/rudder` WebApp Folder : ``` <rudder's path directory>/webapp/sources/rudder/rudder-web/src/main/webapp ``` Classes Folder : ``` <rudder's path directory>/webapp/sources/rudder/rudder-web/target/classes ``` Runs on Port : 8080 VM Args : ``` -DrjrDisableannotation=true -Drun.mode=production -XX:MaxMetaspaceSize=360m -Xms256m -Xmx2048m -XX:-UseLoopPredicate -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:+UseStringDeduplication -Drudder.configFile=/home/<user>/rudder-tests/dev/configuration.properties ``` Beware, replace `<user>` If you are using JRebel add these following arguments : ``` -noverify -Drebel.liftplugin=true -Drebel.springplugin=false -noverify -agentpath:<jrebel's path directory>/lib/libjrebel64.so ``` In IntelliJ : File -> Project Structure -> Project settings Make sure that the Project java SDK is at least at version 8, same for Project language level In `Modules` make sure that every your modules have a Language level at 8 at least For every modules mark src/main/Scala as Sources Setup display format file manager Use the template (copy it from your local clone of the `rudder` repo) and create `~/.m2/settings.xml` and replace ` <localRepository>[PATH TO .m2 DIRECTORY]</localRepository>` line by the path to this directory.\ Do so manually or replace `<user>` by the actual user of the machine and use the following command: ```bash cp rudder/contributing/settings.xml /home/<user>/.m2/settings.xml sed -i "s/\[PATH TO \.m2 DIRECTORY\]/\/home\/<user>\//g" /home/<user>/.m2/settings.xml ``` If you are in the Rudder's Organization put your `Username` and `Password` in the relative fields (\[CAPS\]) of the very same `settings.xml` file. To import dependencies that Rudder use you will have to run : ``` mvn clean install ``` Inside `rudder/webapp/sources/`. Note: (rudder-code, rudder-web, rudder-rest, rudder-templates, rudder-templates-cli) Note: It can take several minutes to download all necessary dependencies. The purpose of the File Watcher is to automatically apply actions on defined file types when they are modified. In Rudder we use it to move generate frontend file (css, html, elm) to the right directory when they are

In Files -> Settings -> tools -> File Watchers Add 3 new Watcher : for CSS, HTML and Elm file with these configurations (change `file type` according to the watcher) : The `Program` field value is the same for these 3 watchers Otherwise at every modification you will need to run the `src/main/build.sh` script in the module. Also, the requirement of the `src/main/build.sh` is to have installed node dependencies. So, inside `rudder/webapp/sources/rudder/rudder-web/src/main/`, run : ```bash npm install ``` Clone ncf repository ``` git clone https://github.com/Normation/ncf.git sudo ln -s <path/to/ncf/repo> /usr/share/ncf mkdir -p /var/rudder/configuration-repository/ncf ``` Setup apache configuration: Install apache (httpd or apache2) Run the following commands (done on Fedora with httpd, path/service are different on deb systems) ``` cp <path/to/rudder/repo>/contributing/rudder.conf /etc/httpd/conf.d/ sed -i "s#<pathToncfRepo>#<path/to/ncf/repo>#" /etc/httpd/conf.d/rudder.conf service httpd restart ``` Congratulation the process is over ! You can now compile and run Rudder in IntelliJ ! You can access the application by running it from IntelliJ. The url is: Warning : make sure your development's environment is running before running Rudder. `./rtf platform setup <env's name>` in rudder-test directory. Otherwise you will get errors in IntelliJ's console. Let's code ! :rocket: All the previous steps are summed up in the following script: .\ Do not run this script as sudo! Though some commands require sudo privileges, you may be prompted to type sudo password Important: as a requirement, if not done yet you need to fork the following repo: `https://github.com/Normation/rudder`. The first param is your local user The second is your gitusername (used to clone your `Normation/rudder` fork) This script should be copied anywhere on your machine and simply executed. ```bash ./setupdevenv.sh <user> <gituser> ``` Important note: This script does not setup Intellij () and Apache Directory Studio (). These still have to be set manually. To start intellij, run: `/opt/idea-IC-193.6494.35/bin/idea.sh` To start Apache Directory Studio: `/opt/ApacheDirectoryStudio/ApacheDirectoryStudio` Important note: this script should only be ran once. Every other startup, only do the following command: `./rtf platform setup debian9_dev` All rudder repos will be cloned (including rtf and ncf) in `/home/<user>/rudder/` Disclaimer: this script might not work on your machine. If it does not, it still is a good guideline when trying to setup a dev environment since it traces every single step of this readme + rtf setup readme If you want to learn how to use Rudder and its web interface, consult the documentation here : https://docs.rudder.io/reference/5.0/usage/web_interface.html :shipit: If you want to submit your code, please feel to contribute by following the , we would be happy to review your code and and see new contributors join the boat! :heart: If you detect any bugs in the application please feel free to report it by signing up here if you don't have already an account: https://issues.rudder.io/ :bug: If you want to discuss about Rudder and get some helps, you can join our Gitter : https://gitter.im/normation/rudder :speech_balloon: