Package: dane-server
DANE-serverDANE-server is the back-end component ofDANEand takes care of task routing as well as the (meta)data storage. A task submitted to
DANE-server is registered in a database, and then its.run()function is called. Running a task involves assigning it to a worker via a message queue.A specific task is run by publishing the task to aRabbitMQ Topic Exchange,
on this exchange the task is routed based on its Task Key. The task key corresponds to thebinding_keyof a worker,
and each worker with this binding_key listens to a shared queue. Once a worker is available it will take the next task from the queue and process it.DANE-server depends on theDANEpackage for the logic of how to iterate over tasks, and how to interpret a task
in general.Local InstallationDANE-server has been tested with Python 3 and is installable through pip:pip install dane-serverBesides the python base, the DANE-server also relies on anElasticsearchserver (version 7.9) for storage,
andRabbitMQ(tested with version 3.7) for messaging.After installing all dependencies it is necessary to configure the DANE server, how to do this is described here:https://dane.readthedocs.io/en/latest/intro.html#configurationThe base config for DANE-server consists of the following parameters, which you might want to overwrite:LOGGING: 
    DIR: "./dane-server-logs/"
    LEVEL: "DEBUG"
DANE_SERVER:
    TEMP_FOLDER: "/home/DANE/DANE-data/TEMP/"
    OUT_FOLDER: "/home/DANE/DANE-data/OUT/"UsageNOTE: DANE-server is still in development, as such authorisation (amongst other featueres) has not yet been added. Use at your own peril.Run the server component (which listens to the RabbitMQ) as follows:dane-serverBesides the server component we also need the API, which we can start with:dane-apiIf no errors occur then this should start a webserver (at port 5500) which will handle API requests,
while in the background the server will handle interaction with the DB and RabbitMQ.APIThe DANE api is documented with a swagger UI, available at:http://localhost:5500/DANE/ExamplesExamples of how to work with DANE can be found at:https://dane.readthedocs.io/en/latest/examples.htmlDockerTo run DANE-server, using Docker make sure to install a Docker Engine, e.g. Docker Desktop for OSX.Build the Docker imagesAs the DANE-server has two separate processes. Two images need to be created:One for running the Task SchedulerOne for running the APIRun the following from the main directory of this repo:docker build -t dane-server -f Dockerfile.ts .
docker build -t dane-server-api -f Dockerfile.api .Note: currently the build relies on thees-index-cfgbranch of DANE (seerequirements.txt)After the images have been successfully built, it is possible to run DANE-server via Kubernetes as wellKubernetesThese instructions are optimized forminikube, which is for local development only. For deployment to a proper k8s cluster, you're on your own for now...Note that the provided Kubernetes config only provisions your k8s cluster with:Endpoint to external Elasticsearch (make sure you got one running)RabbitMQDANE server (task scheduler)DANE server APIIn order to get a bunch of workers setup, you can check the k8s config files inDANE-asr-workerrepository (later on more examples should follow).Create a configmap for config.ymlFirst make sure to create the config.yml from the config-k8s.yml:cp config-k8s.yml config.ymlNow before applying the Kubernetes filedane-server-k8s.yamlto your cluster, first create a ConfigMap for config.ymlkubectl create configmap dane-server-cfg --from-file config.ymlNow the ConfigMap is there, make sure to check that dane-server-k8s.yml points to a existing Elasticsearch host. After that you can go ahead and run:kubectl apply -f dane-server-k8s.yamlConfigure your local DNS to access the API (and RabbitMQ dashboard)Check the ip assigned to thedane-server-ingress(anddane-rabbitmq-ingress) by running:kubectl get ingressgrab the IP from theADDRESScolumn and put this in your/etc/hostsfile:{IP}    api.dane.nl rabbitmq.dane.nlNote: you can assign different domain names by editing the Ingresses indane-server-k8s.yaml