Repository: eclipse-iofog/iofog-python-sdk (language: python)

Issue:
Python SDK Updates for 1.3.0



Discussion / hints:

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
new file mode 100644
index 0000000..226a62f
--- /dev/null
+++ b/CHANGELOG.md
@@ -0,0 +1,16 @@
+# Changelog
+
+## [v3.0.0-alpha1] - 23 March 2021
+
+* Remove deploy module
+* Add rest.controller module
+* Rename client module to microservices
+* Add microservices.log module
+
+## [v1.3.0]
+
+* Ability to deploy ioFog microservices, and connections through SDK are now available.
+    * This can be imported through iofog.deploy options.
+    * Previous standard iofog can be imported through iofog.microservices
+* Added standards for rest calls made to ioFog under deploy.create_rest_call
+* Fixed issue with python3 object handling
diff --git a/Docker/Dockerfile.recieve b/Docker/Dockerfile.recieve
deleted file mode 100644
index 59febc5..0000000
--- a/Docker/Dockerfile.recieve
+++ /dev/null
@@ -1,10 +0,0 @@
-ARG TAG_NAME=latest
-FROM python:${TAG_NAME}
-
-COPY test/recieve.py recieve.py
-
-RUN pip install ws4py
-
-RUN python -m pip install iofog-python-sdk
-
-ENTRYPOINT ["python", "recieve.py"]
\ No newline at end of file
diff --git a/Docker/Dockerfile.send b/Docker/Dockerfile.send
deleted file mode 100644
index 41381ea..0000000
--- a/Docker/Dockerfile.send
+++ /dev/null
@@ -1,10 +0,0 @@
-ARG TAG_NAME=latest
-FROM python:${TAG_NAME}
-
-COPY test/send.py send.py
-
-RUN pip install ws4py
-
-RUN python -m pip install iofog-python-sdk
-
-ENTRYPOINT ["python", "send.py"]
\ No newline at end of file
diff --git a/README.md b/README.md
index f703406..6167caf 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,17 @@
-# iofog-python-sdk
+# ioFog Python SDK
+
+This SDK provides the following modules:
+* microservices: Clients for Microservices to talk to ioFog (e.g. ioMessage and Web Socket clients)
+* rest.controller: Client for Controller REST API
+
+## Installation
+
+Install python package:
+```bash
+sudo python3 -m pip install iofog
+```
+
+## Microservices
 
 This module lets you easily build an ioElement. It gives you all the functionality to interact with ioFog via Local API. It contains all necessary methods for IoMessage transformation as well.
 
@@ -6,42 +19,38 @@ This module lets you easily build an ioElement. It gives you all the functionali
  - fetch next unread messages from ioFog (get_next_messages)
  - fetch messages for time period and list of accessible publishers (get_next_messages_from_publishers_within_timeframe)
  - get config options (get_config)
+ - get Agent's Edge Resources (get_edge_resources)
  - create IoMessage, encode(decode) to(from) raw bytes, marshall(unmarshall) into(from) JSON object (IoMessage class methods)
  - connect to ioFog Control Channel via WebSocket (establish_control_ws_connection)
  - connect to ioFog Message Channel via WebSocket (establish_message_ws_connection) and publish new message via this channel (post_message_via_socket)
 
-## Code snippets: 
-
-Install python package:
-```bash
-sudo python2 -m pip install iofog-python-sdk
-```
+### Usage
 
 Import iofog client and additional classes to your project:
 ```python
-from iofog_python_sdk.client import IoFogClient
-from iofog_python_sdk.exception import IoFogException
-from iofog_python_sdk.iomessage import IoMessage
-from iofog_python_sdk.listener import *
+from iofog.microservices.client import Client
+from iofog.microservices.exception import IoFogException
+from iofog.microservices.iomessage import IoMessage
+from iofog.microservices.listener import *
 ```
 
 Create IoFog client with default settings:
 ```python
 try:
-    client = IoFogClient()
+    client = Client()
 except IoFogException as e:
  # client creation failed, e contains description
 ```
 
-Or specify host, port, ssl and container id explicitly:
+Or specify host, port, and container id explicitly:
 ```python
 try:
-    client = IoFogClient(id='container_id', host='iofog_host', port=6666)
+    client = Client(id='container_id', host='iofog_host', port=6666)
 except IoFogException as e:
  # client creation failed, e contains description
 ```
 
-#### REST calls
+##### REST calls
 
 Get list of next unread IoMessages:
 ```python
@@ -93,14 +102,23 @@ except IoFogException, ex:
  # some error occurred, ex contains description
 ```
 
+Get Agent's Edge Resources:
+```python
+try:
+    edge_resources = client.get_edge_resources()
+except IoFogException, ex:
+ # some error occurred, ex contains description
+```
 
-#### WebSocket calls
+##### WebSocket calls
 
 To use websocket connections you should implement listeners as follows:
 ```python
 class MyControlListener(IoFogControlWsListener):
     def on_control_signal(self):
         # do smth on control signal
+    def on_edge_resources_signal(self):
+        # Agent's Edge Resource list has been updated
 
 
 class MyMessageListener(IoFogMessageWsListener):
@@ -126,7 +144,8 @@ client.post_message_via_socket(io_msg_instance)
 ```
 
 
-#### Message utils
+##### Message utils
+
 Construct IoMessage from JSON(both json string and python dictionary are acceptable):
 ```python
 msg = IoMessage.from_json(json_msg)
@@ -146,3 +165,57 @@ Pack IoMessage into bytearray:
 ```python
 msg_bytes = io_msg_instance.to_bytearray()
 ```
+
+#### Logging
+
+```python
+from iofog.microservices.log import Logger
+
+log = Logger("logger1")
+log.debug("hello")
+log.info("world")
+log.warning("good")
+log.error("bye")
+```
+
+## Controller REST API Client
+
+This module provides a client for talking to [ioFog Controller's REST API](https://iofog.org/docs/2/reference-controller/rest-api.html).
+
+### Usage
+
+```python
+# Import Client class
+from iofog.rest.controller.client import Client
+
+# Instantiate Client
+client = Client(
+    host="123.123.123.123",
+    port=51121,
+    email="hello@world.io"
+    password="2uhi40ghas9")
+
+# Get Controller status and API version
+status = client.get_status()
+print(status['versions']['controller'])
+
+# Create an Agent
+name = "agent1"
+host = "123.123.123.124"
+resp = client.create_agent(name, host)
+
+# Upgrade an Agent
+resp = client.upgrade_agent(name)
+
+# Create an Application
+yaml_file = "/tmp/app.yaml"
+resp = client.create_app_from_yaml(yaml_file)
+
+# Get an Application
+app_name = "app1"
+resp = client.get_app(app_name)
+
+# Delete an Application
+resp = client.delete_app(app_name)
+```
+ 
\ No newline at end of file
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 0b5042d..d736bb1 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -1,13 +1,17 @@
 trigger:
+  tags:
+    include:
+      - v*
   branches:
     include:
-      - master
       - develop
+      - release*
   paths:
     exclude:
       - README.md
-
-pr: none
+      - CHANGELOG.md
+      - CONTRIBUTING
+      - NOTICE
 
 variables:
   group: 'pipelines'
@@ -17,25 +21,45 @@ jobs:
   pool:
     vmImage: 'ubuntu-16.04'
   steps:
-    - task: UsePythonVersion-3.x
+    - task: UsePythonVersion@0
       inputs:
-        versionSpec: '3.5'
+        versionSpec: '3.7'
         addToPath: true
         architecture: 'x64'
+
     - script: |
-        bash ./build.sh 'python3'
-      displayName: 'Building-3.x'
+        set -e
+        test/bootstrap.bash
+      displayName: 'Bootstrap tests'
+
     - script: |
-        bash ./test.sh
-      displayName: 'Testing-3.x'
-    - task: UsePythonVersion-2.7
-      inputs:
-        versionSpec: '2.7'
-        addToPath: true
-        architecture: 'x64'
+        set -e
+        test/setup.bash
+      displayName: 'Setup test environment'
+
     - script: |
-        bash ./build.sh
-      displayName: 'Building-2.7'
+        set -e
+        test/run.bash
+      displayName: 'Run tests'
+
     - script: |
-        bash ./test.sh
-      displayName: 'Testing-2.7'
\ No newline at end of file
+        test/teardown.bash
+      displayName: 'Teardown test environment'
+
+    - task: CopyFiles@2
+      inputs:
+        SourceFolder: $(System.DefaultWorkingDirectory)
+        TargetFolder: $(Build.ArtifactStagingDirectory)
+        Contents: |
+          setup.py
+          requirements.txt
+          README.md
+          iofog/**/*
+        OverWrite: true
+      displayName: 'Copy artifacts'
+
+    - task: PublishBuildArtifacts@1
+      inputs:
+        PathtoPublish: '$(Build.ArtifactStagingDirectory)'
+        ArtifactName: 'sdk'
+      displayName: 'Publish artefacts'
diff --git a/build.sh b/build.sh
index d995059..20d1e2f 100755
--- a/build.sh
+++ b/build.sh
@@ -10,7 +10,3 @@ fi
 # Build Python Images
 docker build --build-arg TAG_NAME=${PYTHON_TAG} -t iofog/test-python-sdk-send:${PYTHON_VERSION} -f ./Docker/Dockerfile.send .
 docker build --build-arg TAG_NAME=${PYTHON_TAG} -t iofog/test-python-sdk-recieve:${PYTHON_VERSION} -f ./Docker/Dockerfile.recieve .
-
-# Push Python Images
-docker push iofog/test-python-sdk-send:${PYTHON_VERSION}
-docker push iofog/test-python-sdk-recieve:${PYTHON_VERSION}
\ No newline at end of file
diff --git a/iofog_python_sdk/__init__.py b/iofog/__init__.py
similarity index 100%
rename from iofog_python_sdk/__init__.py
rename to iofog/__init__.py
diff --git a/iofog/microservices/__init__.py b/iofog/microservices/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/iofog_python_sdk/client.py b/iofog/microservices/client.py
similarity index 88%
rename from iofog_python_sdk/client.py
rename to iofog/microservices/client.py
index 019b650..a6cb911 100644
--- a/iofog_python_sdk/client.py
+++ b/iofog/microservices/client.py
@@ -8,30 +8,21 @@
 #  SPDX-License-Identifier: EPL-2.0
 #********************************************************************************
 
-import argparse
 import logging
 import os
 
 import subprocess
 
-from iofog_python_sdk.httpclient import IoFogHttpClient
-from iofog_python_sdk.definitions import *
-from iofog_python_sdk.wsclient import IoFogControlWsClient, IoFogMessageWsClient
-from iofog_python_sdk.listener import *
-from iofog_python_sdk.exception import *
+from iofog.microservices.httpclient import IoFogHttpClient
+from iofog.microservices.definitions import *
+from iofog.microservices.wsclient import IoFogControlWsClient, IoFogMessageWsClient
+from iofog.microservices.listener import *
+from iofog.microservices.exception import *
 
-parser = argparse.ArgumentParser()
-parser.add_argument("-l", "--log", dest="logLevel", choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
-                    help="Set the logging level", default='INFO')
-args = parser.parse_args()
-
-
-class IoFogClient:
+class Client:
     def __init__(self, id=None, ssl=None, host=None, port=None):
         self.logger = logging.getLogger(IOFOG_LOGGER)
-        self.logger.setLevel(args.logLevel)
         ch = logging.StreamHandler()
-        ch.setLevel(args.logLevel)
         formatter = logging.Formatter(
             '%(levelname)5s [%(asctime)-15s] %(module)10s - <Thread: %(threadName)15s> - %(message)s')
         ch.setFormatter(formatter)
@@ -93,6 +84,12 @@ def get_config(self):
         except Exception as e:
             raise IoFogException(e)
 
+    def get_edge_resources(self):
+        try:
+            return self.http_client.get_edge_resources()
+        except Exception as e:
+            raise IoFogException(e)
+
     def get_next_messages(self):
         try:
             return self.http_client.get_next_messages()
diff --git a/iofog_python_sdk/definitions.py b/iofog/microservices/definitions.py
similarity index 94%
rename from iofog_python_sdk/definitions.py
rename to iofog/microservices/definitions.py
index b8ccc2e..e4c2314 100644
--- a/iofog_python_sdk/definitions.py
+++ b/iofog/microservices/definitions.py
@@ -18,6 +18,7 @@
 IOFOG_LOGGER = 'iofog_logger'
 
 URL_GET_CONFIG = '/v2/config/get'
+URL_GET_EDGE_RESOURCES = '/v2/edgeResources'
 URL_GET_NEXT_MESSAGES = '/v2/messages/next'
 URL_GET_PUBLISHERS_MESSAGES = '/v2/messages/query'
 URL_POST_MESSAGE = '/v2/messages/new'
@@ -34,6 +35,7 @@
 CODE_CONTROL_SIGNAL = 0xC
 CODE_MSG = 0xD
 CODE_RECEIPT = 0xE
+CODE_EDGE_RESOURCE_SIGNAL = 0xF
 
 WS_ATTEMPT_LIMIT = 5
 IO_MESSAGE_VERSION = 4
@@ -67,6 +69,7 @@
 TIME_FRAME_END = 'timeframeend'
 PUBLISHERS = 'publishers'
 CONFIG = 'config'
+EDGE_RESOURCES = 'edgeResources'
 MESSAGES = 'messages'
 STATUS = 'status'
 COUNT = 'count'
diff --git a/iofog_python_sdk/exception.py b/iofog/microservices/exception.py
similarity index 100%
rename from iofog_python_sdk/exception.py
rename to iofog/microservices/exception.py
diff --git a/iofog_python_sdk/httpclient.py b/iofog/microservices/httpclient.py
similarity index 82%
rename from iofog_python_sdk/httpclient.py
rename to iofog/microservices/httpclient.py
index 68549e4..fc4882a 100644
--- a/iofog_python_sdk/httpclient.py
+++ b/iofog/microservices/httpclient.py
@@ -15,10 +15,10 @@
 except ImportError:
     from urllib2 import HTTPError #for python 2
 
-from iofog_python_sdk.definitions import *
-from iofog_python_sdk.util import make_post_request
-from iofog_python_sdk.iomessage import IoMessage
-from iofog_python_sdk.exception import IoFogHttpException
+from iofog.microservices.definitions import *
+from iofog.microservices.util import make_post_request
+from iofog.microservices.iomessage import IoMessage
+from iofog.microservices.exception import IoFogHttpException
 
 
 class IoFogHttpClient:
@@ -29,6 +29,7 @@ def __init__(self, container_id, ssl, host, port):
 
         self.url_base_rest = "{}://{}:{}".format(protocol_rest, host, port)
         self.url_get_config = self.url_base_rest + URL_GET_CONFIG
+        self.url_get_edge_resources = self.url_base_rest + URL_GET_EDGE_RESOURCES
         self.url_get_next_messages = self.url_base_rest + URL_GET_NEXT_MESSAGES
         self.url_get_publishers_messages = self.url_base_rest + URL_GET_PUBLISHERS_MESSAGES
         self.url_post_message = self.url_base_rest + URL_POST_MESSAGE
@@ -42,6 +43,13 @@ def get_config(self):
         except HTTPError as e:
             raise IoFogHttpException(e.code, e.read())
         return json.loads(config_resp[CONFIG])
+    
+    def get_edge_resources(self):
+        try:
+            edge_resources_response = make_get_request(self.url_get_edge_resources, APPLICATION_JSON, self.request_body_id)
+        except HTTPError as e:
+            raise IoFogHttpException(e.code, e.read())
+        return json.loads(edge_resources_response[EDGE_RESOURCES])
 
     def get_next_messages(self):
         try:
diff --git a/iofog_python_sdk/iomessage.py b/iofog/microservices/iomessage.py
similarity index 99%
rename from iofog_python_sdk/iomessage.py
rename to iofog/microservices/iomessage.py
index 31a6769..ae5d126 100644
--- a/iofog_python_sdk/iomessage.py
+++ b/iofog/microservices/iomessage.py
@@ -13,10 +13,10 @@
 from struct import pack, unpack
 import sys
 
-from iofog_python_sdk.util import *
+from iofog.microservices.util import *
 import base64
 
-from iofog_python_sdk.definitions import *
+from iofog.microservices.definitions import *
 
 
 # todo check fields == None?
diff --git a/iofog_python_sdk/listener.py b/iofog/microservices/listener.py
similarity index 93%
rename from iofog_python_sdk/listener.py
rename to iofog/microservices/listener.py
index b317b3b..352fc50 100644
--- a/iofog_python_sdk/listener.py
+++ b/iofog/microservices/listener.py
@@ -15,6 +15,9 @@ def __init__(self):
     def on_control_signal(self):
         pass
 
+    def on_edge_resources_signal(self):
+        pass
+
 
 class IoFogMessageWsListener:
     def __init__(self):
diff --git a/iofog/microservices/log.py b/iofog/microservices/log.py
new file mode 100644
index 0000000..2c5ba35
--- /dev/null
+++ b/iofog/microservices/log.py
@@ -0,0 +1,76 @@
+#********************************************************************************
+#  Copyright (c) 2018 Edgeworx, Inc.
+#
+#  This program and the accompanying materials are made available under the
+#  terms of the Eclipse Public License v. 2.0 which is available at
+#  http://www.eclipse.org/legal/epl-2.0
+#
+#  SPDX-License-Identifier: EPL-2.0
+#********************************************************************************
+
+from logging.handlers import RotatingFileHandler
+import json_logging, logging
+import datetime
+import json
+import os
+import socket
+
+DEVICE_KEY="deviceId"
+MSG_KEY="msg"
+hostname = socket.gethostname()
+
+class CustomJSONLog(logging.Formatter):
+    """
+    Customized logger
+    """
+
+    def format(self, record):
+        content = json.loads(record.getMessage())
+        json_log_object = {"timestamp": datetime.datetime.utcnow().isoformat(),
+                           "hostname": hostname,
+                           "level": record.levelname.lower(),
+                           "message": content[MSG_KEY]}
+        if DEVICE_KEY in content:
+            json_log_object[DEVICE_KEY] = content[DEVICE_KEY]
+        return json.dumps(json_log_object)
+
+json_logging.init_non_web(custom_formatter=CustomJSONLog, enable_json=True)
+
+def encode_content(msg, id):
+    if id is None:
+        return json.dumps({MSG_KEY: msg})
+    return json.dumps({MSG_KEY: msg, DEVICE_KEY: id})
+
+class Logger:
+
+    def __init__(self, name, device_id=None, log_dir=None):
+        self.logger = logging.getLogger(name)
+        self.device_id = device_id
+        # Create log file
+        if log_dir is None:
+            log_dir = "/var/log/iofog-microservices"
+        else:
+            log_dir = log_dir.rstrip('/')
+        self.file = "{}/{}.log".format(log_dir, name)
+        if not os.path.exists(self.file):
+            with open(self.file, 'w+') as f:
+                pass
+        # Register log file
+        self.logger.addHandler(RotatingFileHandler(filename=self.file, maxBytes=10*1024*1024, backupCount=5))
+
+    def info(self, msg):
+        self.logger.setLevel(logging.INFO)
+        self.logger.info(encode_content(msg, self.device_id))
+
+    def debug(self, msg):
+        self.logger.setLevel(logging.DEBUG)
+        self.logger.debug(encode_content(msg, self.device_id))
+
+    def warning(self, msg):
+        self.logger.setLevel(logging.WARN)
+        self.logger.warning(encode_content(msg, self.device_id))
+
+    def error(self, msg):
+        self.logger.setLevel(logging.ERROR)
+        self.logger.error(encode_content(msg, self.device_id))
+
diff --git a/iofog_python_sdk/util.py b/iofog/microservices/util.py
similarity index 85%
rename from iofog_python_sdk/util.py
rename to iofog/microservices/util.py
index 5cb31ea..dd4a646 100644
--- a/iofog_python_sdk/util.py
+++ b/iofog/microservices/util.py
@@ -17,7 +17,7 @@
     import urllib2 as urllib_request # for python 2
 
 from struct import pack
-from iofog_python_sdk.definitions import CODE_MSG
+from iofog.microservices.definitions import CODE_MSG
 
 
 def num_to_bytearray(num):
@@ -48,6 +48,11 @@ def make_post_request(url, body_type, body):
     response = urllib_request.urlopen(req)
     return json.loads(response.read())
 
+def make_get_request(url, body_type, body):
+    req = urllib_request.Request(url, body, {'Content-Type': body_type}, method="GET")
+    response = urllib_request.urlopen(req)
+    return json.loads(response.read())
+
 
 def prepare_iomessage_for_sending_via_socket(io_msg):
     msg_bytes = io_msg.to_bytearray()
diff --git a/iofog_python_sdk/wsclient.py b/iofog/microservices/wsclient.py
similarity index 93%
rename from iofog_python_sdk/wsclient.py
rename to iofog/microservices/wsclient.py
index 869b4a3..6227b83 100644
--- a/iofog_python_sdk/wsclient.py
+++ b/iofog/microservices/wsclient.py
@@ -13,9 +13,9 @@
 
 import time
 
-import iofog_python_sdk.util as util
-from iofog_python_sdk.iomessage import IoMessage
-from iofog_python_sdk.definitions import *
+import iofog.microservices.util as util
+from iofog.microservices.iomessage import IoMessage
+from iofog.microservices.definitions import *
 from ws4py.client.threadedclient import WebSocketClient
 from ws4py.framing import OPCODE_PONG
 
@@ -125,6 +125,10 @@ def received_message(self, message):
             self.logger.debug('Received control')
             self.listener.on_control_signal()
             self.send(bytearray([CODE_ACK]), binary=True)
+        elif opt_code == CODE_EDGE_RESOURCE_SIGNAL:
+            self.logger.debug('Received Edge Resource Signal')
+            self.listener.on_edge_resources_signal()
+            self.send(bytearray([CODE_ACK]), binary=True)
 
 
 class IoFogMessageWsClient(IoFogWsClient):
diff --git a/iofog/rest/__init__.py b/iofog/rest/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/iofog/rest/controller/__init__.py b/iofog/rest/controller/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/iofog/rest/controller/client.py b/iofog/rest/controller/client.py
new file mode 100644
index 0000000..13d3ca2
--- /dev/null
+++ b/iofog/rest/controller/client.py
@@ -0,0 +1,177 @@
+from iofog.rest.controller.request import *
+import yaml
+
+class Client:
+    """
+    Iofog Controller REST API client.
+    """
+
+
+    def __init__(self, host, port, email, password, name="", surname=""):
+        self.base_path = "http://" + host + ":" + str(port) + "/api/v3"
+        try:
+            self._login(email, password)
+        except Exception:
+            if name == "" or surname == "":
+                raise Exception("Must provide name and surname if User does not already exist")
+            self.create_user(email, password, name, surname)
+            self._login(email, password)
+        self.error_yaml_spec = "YAML file does not follow required specification. See https://iofog.org/docs/2/reference-iofogctl/reference-application.html"
+
+    def create_user(self, email, password, name, surname):
+        url = "{}/user/signup".format(self.base_path)
+        body = {
+            "email": email,
+            "password": password,
+            "lastName": name,
+            "firstName": surname
+        }
+        return request("POST", url, "", body)
+
+    def _login(self, email, password):
+        url = "{}/user/login".format(self.base_path)
+        body = {
+            "email": email,
+            "password": password
+        }
+        self.token = request("POST", url, "", body)["accessToken"]
+
+    def get_status(self):
+        url = "{}/status".format(self.base_path)
+        return request("GET", url)
+
+    def create_agent(self, name, host, arch=""):
+        fog_type_id = 0
+        if arch != "":
+            if arch not in [ "x86", "arm" ]:
+                raise Exception("Agent architecture {} not supported".format(arch))
+            fog_type_id = 1 if arch == "x86" else 2
+        url = "{}/iofog".format(self.base_path)
+        body = {
+            "name": name,
+            "fogType": fog_type_id,
+            "host": host
+        }
+        return request("POST", url, self.token, body)["uuid"]
+
+    def _get_provision_key(self, agent_id):
+        url = "{}/iofog/{}/provisioning-key".format(self.base_path, agent_id)
+        return request("GET", url, self.token)["key"]
+    
+    def _delete_agent(self, agent_id):
+        url = "{}/iofog/{}".format(self.base_path, agent_id)
+        request("DELETE", url, self.token)
+    
+    def get_agent_uuid(self, name):
+        url = "{}/iofog-list".format(self.base_path)
+        resp = request("GET", url, self.token)
+        for fog in resp["fogs"]:
+            if fog["name"] == name:
+                return fog["uuid"]
+
+    def delete_agent(self, name):
+        uuid = self.get_agent_uuid(name)
+        if uuid is not None:
+            self._delete_agent(uuid)
+    
+    def get_provision_key(self, agent_name):
+        uuid = self.get_agent_uuid(agent_name)
+        if uuid is None:
+            raise Exception("Could not get Agent UUID")
+        return self._get_provision_key(uuid)
+
+    def upgrade_agent(self, agent_name):
+        uuid = self.get_agent_uuid(agent_name)
+        url = "{}/iofog/{}/version/upgrade".format(self.base_path, uuid)
+        return request("POST", url, self.token)
+
+    def patch_agent(self, agent_name, config):
+        uuid = self.get_agent_uuid(agent_name)
+        url = "{}/iofog/{}".format(self.base_path, uuid)
+        return request("PATCH", url, self.token, config)
+
+    def create_app(self, name, msvcs, routes):
+        if routes is None:
+            routes = []
+        url = "{}/application/".format(self.base_path)
+        body = {
+            "name": name,
+            "routes": routes,
+            "microservices": msvcs,
+            "isActivated": True,
+            "isSystem": False
+        }
+        return request("POST", url, self.token, body)
+    
+    def create_app_from_yaml(self, file):
+        with open(file) as file:
+            doc = yaml.full_load(file)
+        if "metadata" not in doc or "spec" not in doc or "microservices" not in doc["spec"]:
+            raise Exception(self.error_yaml_spec)
+        name = doc["metadata"]["name"]
+        msvcs = doc["spec"]["microservices"]
+        routes = None
+        if "routes" in doc["spec"]:
+            routes = doc["spec"]["routes"]
+        json_msvcs = self._jsonify_yaml_msvcs(msvcs)
+        json_routes = self._jsonify_yaml_routes(routes)
+        return self.create_app(name, json_msvcs, json_routes)
+
+    def delete_app(self, name):
+        url = "{}/application/{}".format(self.base_path, name)
+        return request("DELETE", url, self.token)
+
+    def get_app(self, name):
+        url = "{}/application/{}".format(self.base_path, name)
+        return request("GET", url, self.token)
+
+    def _jsonify_yaml_routes(self, routes):
+        json_routes = []
+        if routes is None:
+            return json_routes
+        for route in routes:
+            json_route = route
+            json_route["name"] = "{}-{}".format(route["from"], route["to"])
+        return json_routes
+    
+    def _jsonify_yaml_msvcs(self, msvcs):
+        json_msvcs = []
+        for msvc in msvcs:
+            json_msvc = dict()
+            # images
+            images = []
+            for arch in [ "x86", "arm" ]:
+                if arch in msvc["images"]:
+                    fog_type_id = 1 if arch == "x86" else 2
+                    imageDict = {
+                        "fogTypeId": fog_type_id,
+                        "containerImage": msvc["images"][arch]
+                    }
+                    images.append(imageDict)
+            json_msvc["images"] = images
+            # container
+            for pasta in [ "env", "rootHostAccess", "ports", "volumes", "commands" ]:
+                if pasta in msvc["container"]:
+                    json_msvc[pasta] = msvc["container"][pasta]
+            if "volumes" in json_msvc:
+                json_msvc["volumeMappings"] = json_msvc["volumes"]
+                json_msvc.pop("volumes")
+            # msvc
+            for pasta in [ "name" ]:
+                if pasta in msvc:
+                    json_msvc[pasta] = msvc[pasta]
+            json_msvc["registryId"] = 1
+            if "config" in msvc:
+                json_msvc["config"] = json.dumps(msvc["config"])
+            if "env" in json_msvc:
+                for env in json_msvc["env"]:
+                    env["value"] = str(env["value"])
+            if "ports" in json_msvc:
+                for port in json_msvc["ports"]:
+                    if "public" in port:
+                        port["publicPort"] = port["public"]
+                        port.pop("public")
+            json_msvc["iofogUuid"] = self.get_agent_uuid(msvc["agent"]["name"])
+            json_msvcs.append(json_msvc)
+    
+        return json_msvcs
diff --git a/iofog/rest/controller/request.py b/iofog/rest/controller/request.py
new file mode 100644
index 0000000..bdc6d8f
--- /dev/null
+++ b/iofog/rest/controller/request.py
@@ -0,0 +1,32 @@
+import requests
+import json
+
+
+def request(method, address, auth_token="", body={}):
+    switch = {
+        "POST": requests.post,
+        "GET": requests.get,
+        "DELETE": requests.delete,
+        "PATCH": requests.patch,
+    }
+    headers = {}
+    data = {}
+    if body:
+        headers["Content-Type"] = 'application/json'
+        data = json.dumps(body, indent=4)
+
+    if auth_token:
+        headers["Authorization"] = auth_token
+    else:
+        headers["cache-control"] = "no-cache"
+
+    response = switch[method](address, data=data, headers=headers, timeout=30)
+    try:
+        response.raise_for_status()
+    except requests.HTTPError as e:
+        print(e.response.content)
+        raise e
+    responseDict = {}
+    if response.content:
+        responseDict = json.loads(response.content)
+    return responseDict
\ No newline at end of file
diff --git a/iofog_python_sdk/agent_service.py b/iofog_python_sdk/agent_service.py
deleted file mode 100644
index 9207e49..0000000
--- a/iofog_python_sdk/agent_service.py
+++ /dev/null
@@ -1,33 +0,0 @@
-#********************************************************************************
-#  Copyright (c) 2018 Edgeworx, Inc.
-#
-#  This program and the accompanying materials are made available under the
-#  terms of the Eclipse Public License v. 2.0 which is available at
-#  http://www.eclipse.org/legal/epl-2.0
-#
-#  SPDX-License-Identifier: EPL-2.0
-#********************************************************************************
-
-from iofog_python_sdk.create_rest_call import rest_call
-
-class agent_service:
-    def get_agent_per_microservice(controller_address, auth_token, microservices):
-        data = {}
-        agent_uuids = {}
-        post_address = "{}/iofog-list".format(controller_address)
-        json_response = rest_call(data, post_address, auth_token, method="GET")
-        for microserviceKey in microservices:
-            microservice = microservices[microserviceKey]
-            agent_uuids[microserviceKey] = next(x for x in json_response["fogs"] if x["name"] == microservice["agent-name"])
-        return agent_uuids
-
-
-    def get_agent_info(agent):
-        config = {}
-        config.update(agent)
-        return config
-
-
-    def update_agent(controller_address, uuid, fog_info, auth_token):
-        url = "{}/iofog/{}".format(controller_address, uuid)
-        return rest_call(fog_info, url, auth_token, method="PATCH").response
diff --git a/iofog_python_sdk/catalog_service.py b/iofog_python_sdk/catalog_service.py
deleted file mode 100755
index 293d180..0000000
--- a/iofog_python_sdk/catalog_service.py
+++ /dev/null
@@ -1,102 +0,0 @@
-from iofog_python_sdk.create_rest_call import rest_call
-from iofog_python_sdk.pretty_print import print_info
-from iofog_python_sdk.microservice_service import microservices as msv
-
-
-class catalog_service:
-    def create_catalog_curl_data(self, catalog_item):
-        data = {}
-        data["name"] = catalog_item["name"]
-        data["images"] = []
-        data["images"].append({'containerImage': '{}'.format(catalog_item["images"]["arm"]), 'fogTypeId': 2})
-        data["images"].append({'containerImage': '{}'.format(catalog_item["images"]["x86"]), 'fogTypeId': 1})
-        data["registryId"] = 1
-
-        return data
-
-    def update_catalog(self, controller_address, auth_token, catalog_item):
-        # Delete all microservices that uses this specific catalog item
-        print_info("====> Deleting all microservices currently using this catalog item")
-        microself = msv()
-        msv.delete_all_by_catalog_id(microself, controller_address, auth_token, catalog_item["id"])
-        # Update catalog item
-        data = catalog_service.create_catalog_curl_data(self, catalog_item)
-        if data == {}:
-            # If data has failed to be created, exit here, and echo which service failed
-            return "{} failed to create curl data".format(catalog_item)
-        post_address = "{}/catalog/microservices/{}".format(controller_address, catalog_item["id"])
-        json_response = rest_call(data, post_address, auth_token, method="PATCH").response
-
-    def add_to_catalog(self, controller_address, auth_token, catalog_item):
-        data = catalog_service.create_catalog_curl_data(self, catalog_item)
-        if data == {}:
-            # If data has failed to be created, exit here, and echo which service failed
-            return "{} failed to create curl data".format(catalog_item)
-        post_address = "{}/catalog/microservices".format(controller_address)
-        return rest_call(data, post_address, auth_token).response["id"]
-
-
-    def delete_by_id(self, controller_address, catalog_id, auth_token):
-        post_address = "{}/catalog/microservices/{}".format(controller_address, catalog_id)
-        return rest_call({}, post_address, auth_token, method="DELETE").response
-
-
-    def delete_items(self, controller_address, catalog_items, auth_token):
-        for catalog_id in catalog_items:
-            catalog_service.delete_by_id(self, controller_address, catalog_id, auth_token)
-
-    def get_catalog(self, controller_address, auth_token):
-        post_address = "{}/catalog/microservices".format(controller_address)
-        return rest_call({}, post_address, auth_token, method="GET").response["catalogItems"]
-
-    def get_catalog_item_by_name(self, controller_address, name, auth_token):
-        catalog_items = catalog_service.get_catalog(self, controller_address, auth_token)
-        return next((x for x in catalog_items if x["name"] == name), None)
-
-    def is_same(self, yaml_item, existing_item):
-        if len(yaml_item["images"]) != len(existing_item["images"]):
-            return False
-        for image_type in yaml_item["images"]:
-            image = yaml_item["images"][image_type]
-            same = next((x for x in existing_item["images"] if (image_type == "x86" and x["fogTypeId"] == 1 and x["containerImage"] == image) or (image_type == "arm" and x["fogTypeId"] == 2 and x["containerImage"] == image)), False)
-            if same == False:
-                return False
-        return True
-
-    def setup(self, controller_address, auth_token, microservices):
-        # For each microservice check if catalog item exists
-        print_info("====> Reading the catalog")
-        updated = False
-        existing_catalog_items = catalog_service.get_catalog(self, controller_address, auth_token)
-        catalog_ids = {}
-        for microserviceKey in microservices:
-            microservice = microservices[microserviceKey]
-            catalog_item = {
-                "images": microservice["images"],
-                "name": microservice["microservice"]["name"] + "_catalog"
-            }
-            catalog_id = ""
-            existing_catalog_item = next((x for x in existing_catalog_items if x["name"] == catalog_item["name"]), None)
-            # If it does not exists yet, create
-            if  existing_catalog_item == None:
-                print_info("====> Adding to the catalog")
-                updated = True
-                catalog_id = catalog_service.add_to_catalog(self, controller_address, auth_token, catalog_item)
-            # Otherwise, patch to update images
-            else:
-                catalog_item["id"] = existing_catalog_item["id"]
-                # Check if images have changed
-                if catalog_service.is_same(self, catalog_item, existing_catalog_item) == False:
-                    print_info("====> Updating a catalog item (Delete / Recreate)")
-                    updated = True
-                    # update_catalog(controller_address, auth_token, catalog_item)
-                    catalog_service.delete_by_id(self, controller_address, existing_catalog_item["id"], auth_token)
-                    catalog_id = catalog_service.add_to_catalog(self, controller_address, auth_token, catalog_item)
-                else:
-                    catalog_id = catalog_item["id"]
-            catalog_ids[microserviceKey] = catalog_id
-        if updated == False:
-            print_info("====> Catalog is up-to-date")
-        else:
-            print_info("====> Catalog updated")
-        return catalog_ids
diff --git a/iofog_python_sdk/create_rest_call.py b/iofog_python_sdk/create_rest_call.py
deleted file mode 100644
index e1b672a..0000000
--- a/iofog_python_sdk/create_rest_call.py
+++ /dev/null
@@ -1,47 +0,0 @@
-import requests
-import json
-
-from requests.exceptions import HTTPError
-from iofog_python_sdk.pretty_print import *
-
-class rest_call:
-    #
-    # create a rest call, with two optional variables, auth_token for before you can retrieve it, and
-    # get if this needs to be a GET call instead of POST
-    #
-    def __init__(self, data, address, auth_token="none", method="POST"):
-        switch = {
-            "POST": requests.post,
-            "GET": requests.get,
-            "DELETE": requests.delete,
-            "PATCH": requests.patch,
-        }
-        headers = {}
-        headers["Content-Type"] = 'application/json'
-
-        # Dump the data to Json so the curl call can take it in.
-        data = json.dumps(data, indent=4)
-
-        if auth_token == "none":
-            headers["cache-control"] = "no-cache"
-        else:
-            headers["Authorization"] = auth_token
-
-        print("==== " + method + " CALL ====")
-        print("Sending data: " + data)
-        print("To addr: " + address)
-
-        try:
-            r = switch[method](address, data=data, headers=headers, timeout=30)
-            r.raise_for_status()
-        except HTTPError as http_err:
-            print_error("HTTP error occurred: " + str(http_err))
-        except Exception as err:
-            print_error("Other error occurred: " + str(err))
-        else:
-            jsonResponse = ""
-            if r.text:
-                jsonResponse = json.loads(r.text)
-            print("Response: " + str(jsonResponse))
-            print('=====')
-            self.response = jsonResponse
diff --git a/iofog_python_sdk/flow_service.py b/iofog_python_sdk/flow_service.py
deleted file mode 100644
index 2834ffd..0000000
--- a/iofog_python_sdk/flow_service.py
+++ /dev/null
@@ -1,69 +0,0 @@
-from iofog_python_sdk.create_rest_call import rest_call
-import time
-
-
-class Flow:
-
-    def __init__(self):
-        self.id = ""
-
-    def create_flow(self, controller_address, auth_token, flow):
-        data = {}
-
-        data["name"] = flow
-        post_address = "{}/flow".format(controller_address)
-        jsonResponse = rest_call(data, post_address, auth_token).response
-        self.id = jsonResponse["id"]
-
-    def create_flow_if_not_exist(self, controller_address, auth_token, flow):
-        try:
-          id = Flow.get_id(self, controller_address, flow, auth_token)
-          return id
-        except StopIteration:
-          return Flow.create_flow(self, controller_address, auth_token, flow)
-
-    def restart_flow(self, controller_address, flow_id, flow_name, auth_token):
-      flow = Flow.get_flow_by_id(self, controller_address, flow_id, auth_token)
-      if flow.get("isActivated", False) == True:
-        Flow.stop_flow(self, controller_address, flow_id, flow_name, auth_token)
-        time.sleep(3)
-      Flow.start_flow(self, controller_address, flow_id, flow_name, auth_token)
-
-    def start_flow(self, controller_address, flow_id, flow_name, auth_token):
-        data = {}
-        data["name"] = flow_name
-        data["isActivated"] = True
-        post_address = "{}/flow/{}".format(controller_address, flow_id)
-        jsonRespone = rest_call(data, post_address, auth_token, method="PATCH").response
-
-    def stop_flow(self, controller_address, flow_id, flow_name, auth_token):
-        data = {}
-        data["name"] = flow_name
-        data["isActivated"] = False
-        post_address = "{}/flow/{}".format(controller_address, flow_id)
-        jsonRespone = rest_call(data, post_address, auth_token, method="PATCH").response
-
-    def delete_flow(self, controller_address, flow_id, auth_token):
-        data = {}
-        post_address = "{}/flow/{}".format(controller_address, flow_id)
-        jsonRespone = rest_call(data, post_address, auth_token, method="DELETE").response
-
-    def get_flow_by_name(self, controller_address, flow_name, auth_token):
-        flows = Flow.get_all(self, controller_address, auth_token)
-        return next(x for x in flows if x["name"] == flow_name)
-
-    def get_flow_by_id(self, controller_address, flow_id, auth_token):
-        data = {}
-        post_address = "{}/flow/{}".format(controller_address, flow_id)
-        json_response = rest_call(data, post_address, auth_token, method="GET").response
-        return json_response
-
-    def get_id(self, controller_address, flow_name, auth_token):
-        self.id = Flow.get_flow_by_name(self, controller_address, flow_name, auth_token)["id"]
-
-    def get_all(self, controller_address, auth_token):
-        data = {}
-        flow_ids = {}
-        post_address = "{}/flow".format(controller_address)
-        json_response = rest_call(data, post_address, auth_token, method="GET").response
-        return json_response["flows"]
diff --git a/iofog_python_sdk/microservice_service.py b/iofog_python_sdk/microservice_service.py
deleted file mode 100644
index 4131eec..0000000
--- a/iofog_python_sdk/microservice_service.py
+++ /dev/null
@@ -1,220 +0,0 @@
-import json
-
-from iofog_python_sdk.create_rest_call import rest_call
-from iofog_python_sdk.pretty_print import print_info, print_error
-from iofog_python_sdk.flow_service import Flow
-
-class microservices:
-    def __init__(self):
-        self.data = {}
-
-    def create_microservice_curl_data(self, microservice, flow_id, fog_uuid, catalog_id):
-        self.data["name"] = microservice["name"]
-
-        if len(microservice.get("volumes", [])) > 0:
-            self.data["volumeMappings"] = microservice["volumes"]
-
-        if len(microservice.get("env", [])) > 0:
-            self.data["env"] = microservice["env"]
-
-        if len(microservice.get("ports", [])) > 0:
-            self.data["ports"] = microservice["ports"]
-
-        if "config" in microservice:
-            self.data["config"] = json.dumps(microservice.get("config", {}))
-
-        self.data["rootHostAccess"] = microservice.get("root-host", False)
-        self.data["flowId"] = flow_id
-        self.data["iofogUuid"] = fog_uuid
-        self.data["catalogItemId"] = catalog_id
-
-        return self.data
-
-    def update_microservice_curl_data(self, microservice, fog_uuid, catalog_id):
-        self.data["name"] = microservice["name"]
-
-        if len(microservice.get("volumes", [])) > 0:
-            self.data["volumeMappings"] = microservice["volumes"]
-
-        if len(microservice.get("env", [])) > 0:
-            self.data["env"] = microservice["env"]
-
-        # if len(microservice.get("ports", [])) > 0:
-        #     data["ports"] = microservice["ports"]
-
-        if "config" in microservice:
-            self.data["config"] = json.dumps(microservice.get("config", {}))
-
-        self.data["rootHostAccess"] = microservice.get("root-host", False)
-        self.data["iofogUuid"] = fog_uuid
-        # data["catalogItemId"] = catalog_id
-
-        return self.data
-
-    def create_route(controller_address, auth_token, route):
-        post_address = "{}/microservices/{}/routes/{}".format(controller_address, route["from"], route["to"])
-        json_response = rest_call({}, post_address, auth_token).response
-
-    def delete_route(controller_address, auth_token, route):
-        post_address = "{}/microservices/{}/routes/{}".format(controller_address, route["from"], route["to"])
-        json_response = rest_call({}, post_address, auth_token, method="DELETE").response
-
-    def update_routing(controller_address, microservices_per_name, auth_token, routes):
-        print_info("====> Update routing")
-        updated = False
-        for route in routes:
-            # Check if route exists
-            msvc = microservices_per_name.get(route["from"], None)
-            dest_msvc = microservices_per_name.get(route["to"], None)
-            if msvc == None:
-                print_error("No source microservice for route: {} - Microservices: {}".format(route, microservices_per_name))
-                continue
-            if dest_msvc == None:
-                print_error("No destination microservice for route: {} - Microservices: {}".format(route, microservices_per_name))
-                continue
-            route_exists = msvc != None and next((x for x in msvc.get("routes", []) if x == dest_msvc["uuid"]), False)
-            # Create missing route
-            if route_exists == False:
-                print_info("====> Create new route")
-                updated = True
-                microservices.create_route(controller_address, auth_token,
-                    {
-                        "from": microservices_per_name[route["from"]]["uuid"],
-                        "to": microservices_per_name[route["to"]]["uuid"]
-                    })
-        # Delete unecessary routes
-        for microservice_name in microservices_per_name:
-            microservice = microservices_per_name[microservice_name]
-            microservice_routes = microservice.get("routes", [])
-            for route in microservice_routes:
-                route_needed = next((x for x in routes if x["from"] == microservice["name"] and microservices_per_name.get(x["to"], {"uuid": None})["uuid"] == route), False)
-                if route_needed == False:
-                    print_info("====> Delete outdated route")
-                    updated = True
-                    microservices.delete_route(controller_address, auth_token,
-                    {
-                        "from": microservice["uuid"],
-                        "to": route
-                    })
-        if updated == False:
-            print_info("====> Routing is up-to-date.")
-        else:
-            print_info("====> Routing updated.")
-
-    def create_microservice(self, controller_address, microservice, fog_uuid, catalog_id, flow_id, auth_token):
-        data = microservices.create_microservice_curl_data(microservice, flow_id, fog_uuid, catalog_id)
-        post_address = "{}/microservices".format(controller_address)
-        json_response = rest_call(data, post_address, auth_token).response
-        return json_response
-
-    def get_microservice_port_mapping(self, controller_address, microservice_uuid, auth_token):
-        data = microservices().data
-        post_address = "{}/microservices/{}/port-mapping".format(controller_address, microservice_uuid)
-        json_response = rest_call(data, post_address, auth_token, method="GET").response
-        return json_response["ports"]
-
-    def delete_microservice_port_mapping(self, controller_address, microservice_uuid, mapping, auth_token):
-        post_address = "{}/microservices/{}/port-mapping/{}".format(controller_address, microservice_uuid, mapping["internal"])
-        json_response = rest_call(self.data, post_address, auth_token, method="DELETE").response
-        return json_response
-
-    def create_microservice_port_mapping(self, controller_address, microservice_uuid, mapping, auth_token):
-        data = microservices().data
-        data = mapping
-        post_address = "{}/microservices/{}/port-mapping".format(controller_address, microservice_uuid)
-        json_response = rest_call(data, post_address, auth_token).response
-        return json_response
-
-    def update_ports(self, controller_address, microservice, auth_token):
-        print_info("====> Getting current port mapping")
-        updated = False
-        ports = microservice.get("ports", [])
-        existing_port_mappings = microservices.get_microservice_port_mapping(controller_address, microservice["uuid"], auth_token)
-        # Remove false port mapping
-        for existing_mapping in existing_port_mappings:
-            valid = next((x for x in ports if x["internal"] == existing_mapping["internal"] and x["external"] == existing_mapping["external"]), False)
-            if valid == False:
-                print_info("====> Remove outdated port mapping")
-                updated = True
-                microservices.delete_microservice_port_mapping(controller_address, microservice["uuid"], existing_mapping, auth_token)
-        # Create missing port mapping
-        for new_mapping in ports:
-            exists = next((x for x in existing_port_mappings if x["internal"] == new_mapping["internal"] and x["external"] == new_mapping["external"]), False)
-            if exists == False:
-                print_info("====> Create new port mapping")
-                updated = True
-                microservices.create_microservice_port_mapping(controller_address, microservice["uuid"], new_mapping, auth_token)
-        if updated == False:
-            print_info("====> Port mapping is up-to-date")
-        else:
-            print_info("====> Port mapping updated")
-
-
-    def update_microservice(self, controller_address, microservice, fog_uuid, catalog_id, auth_token):
-        self.data = microservices.update_microservice_curl_data(self, microservice, fog_uuid, catalog_id)
-        post_address = "{}/microservices/{}".format(controller_address, microservice["uuid"])
-        json_response = rest_call(self.data, post_address, auth_token, method="PATCH").response
-        # Update port mapping
-        microservices.update_ports(controller_address, microservice, auth_token)
-        return json_response
-
-    def get_microservices_by_flow_id(self, controller_address, flow_id, auth_token):
-        data = microservices()
-        post_address = "{}/microservices?flowId={}".format(controller_address, flow_id)
-        json_response = rest_call(data, post_address, auth_token, method="GET").response
-        return json_response["microservices"]
-
-    def get_all_microservices(self, controller_address, auth_token):
-        # Get all flows
-        flow_self = Flow()
-        flows = Flow.get_all(flow_self, controller_address, auth_token)
-        all_msvcs = []
-        for flow in flows:
-            all_msvcs.extend(microservices.get_microservices_by_flow_id(self, controller_address, flow["id"], auth_token))
-        return all_msvcs
-
-    def get_microservice_by_name(self, controller_address, microservice_name, flow_id, auth_token):
-        msvcs = microservices.get_microservices_by_flow_id(self, controller_address, flow_id, auth_token)
-        return next(x for x in msvcs if x["name"] == microservice_name)
-
-    def setup(self, controller_address, flow_id, fog_per_microservice, catalog_ids, auth_token, microservices, routes):
-        route = ""
-        microservices_per_name = {}
-        # Get exisiting microservices
-        for microserviceKey in microservices:
-            microservice = microservices[microserviceKey]
-            name = microservice["microservice"]["name"]
-            try:
-                msvc = microservices.get_microservice_by_name(controller_address, name, flow_id, auth_token)
-                microservices_per_name[name] = msvc
-            except StopIteration:
-                microservices_per_name[name] = None
-                continue
-        # Create missing microservices
-        for microserviceKey in microservices:
-            microservice = microservices[microserviceKey]["microservice"]
-            name = microservice["name"]
-            fog_uuid = fog_per_microservice[microserviceKey]["uuid"]
-            catalog_id = catalog_ids[microserviceKey]
-            msvc = None
-            if microservices_per_name[name] == None:
-                msvc = microservices.create_microservice(controller_address, microservice, fog_uuid, catalog_id, flow_id, auth_token)
-            else:
-                microservice["uuid"] = microservices_per_name[name]["uuid"]
-                microservices.update_microservice(controller_address, microservice, fog_uuid, catalog_id, auth_token)
-                msvc = {**microservice, **microservices_per_name[name]}
-            microservices_per_name[microservice["name"]] = msvc
-        # Update routing
-        microservices.update_routing(controller_address, microservices_per_name, auth_token, routes)
-
-    def delete_microservice(self, controller_address, microservice, auth_token):
-        post_address = "{}/microservices/{}".format(controller_address, microservice["uuid"])
-        json_response = rest_call({}, post_address, auth_token, method="DELETE").response
-        return json_response
-
-    def delete_all_by_catalog_id(self, controller_address, auth_token, catalog_id):
-        msvcs = microservices.get_all_microservices(self, controller_address, auth_token)
-        print("All microservices: {}".format(msvcs))
-        for msvc in msvcs:
-            if msvc["catalogItemId"] == catalog_id:
-                microservices.delete_microservice(self, controller_address, msvc, auth_token)
\ No newline at end of file
diff --git a/iofog_python_sdk/pretty_print.py b/iofog_python_sdk/pretty_print.py
deleted file mode 100644
index 40b183b..0000000
--- a/iofog_python_sdk/pretty_print.py
+++ /dev/null
@@ -1,54 +0,0 @@
-from colorama import init
-
-#
-# Pretty Print is a rather simple helper utility that exposes some helper methods to make printing logging
-# information more colorful and easier to understand.
-#
-# Initialize colorama
-init(autoreset=True)
-
-# These are the colors that we'll use in our output
-NO_FORMAT = '\033[0m'
-C_SKYBLUE1 = '\033[38;5;117m'
-C_DEEPSKYBLUE4 = '\033[48;5;25m'
-RED = '\033[38;5;1m'
-GREEN = '\033[38;5;28m'
-
-
-#
-# Display a nice title line for any output. You can optionally populate it with a string
-#
-# Usage: pretty_title("Bootstrapping ioFog")
-#
-def pretty_title(title):
-    print_info("## " + title + " ####################################################")
-
-
-#
-# Display a nice header for any command line script. You can optionally populate it with a string
-#
-# Usage: pretty_header("Bootstrapping ioFog")
-#
-def pretty_header(title):
-    print_info("## " + title + " ####################################################")
-    print_info("## Copyright (C) 2019, Edgeworx, Inc.\n")
-
-
-# Basic subtle output
-def print_info(message):
-    print(C_SKYBLUE1 + message + NO_FORMAT)
-
-
-# Highlighted output with a background
-def print_notify(message):
-    print(C_DEEPSKYBLUE4 + message + NO_FORMAT)
-
-
-# Hurrah!
-def print_success(message):
-    print(GREEN + message + NO_FORMAT)
-
-
-# Houston, we have a problem!
-def print_error(message):
-    print(RED + str(message) + NO_FORMAT)
diff --git a/requirements.txt b/requirements.txt
index e69de29..da63530 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -0,0 +1,4 @@
+PyYAML==5.3.1
+requests==2.24.0
+ws4py==0.5.1
+json-logging==1.2.11
\ No newline at end of file
diff --git a/setup.py b/setup.py
index f96e2cd..af5858d 100644
--- a/setup.py
+++ b/setup.py
@@ -8,23 +8,28 @@
 #  SPDX-License-Identifier: EPL-2.0
 #********************************************************************************
 
-from distutils.core import setup
+import setuptools
 
-setup(
-    name='iofog-python-sdk',
-    version='1.0.0',
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+setuptools.setup(
+    name='iofog',
+    version='3.0.0-alpha1',
     project_urls={
-        'Documentation': 'https://github.com/ioFog/iofog-python-sdk/blob/master/README.md',
-        'Source': 'https://github.com/ioFog/iofog-python-sdk.git',
-        'Tracker': 'https://github.com/ioFog/iofog-python-sdk/issues',
+        'Documentation': 'https://github.com/eclipse-iofog/iofog-python-sdk/blob/master/README.md',
+        'Source': 'https://github.com/eclipse-iofog/iofog-python-sdk.git',
+        'Tracker': 'https://github.com/eclipse-iofog/iofog-python-sdk/issues',
         'Eclipse ioFog': 'http://iofog.org'
     },
-    packages=['iofog_python_sdk'],
-    url='https://github.com/ioFog/iofog-python-sdk',
+    packages=setuptools.find_packages(),
+    url='https://github.com/eclipse-iofog/iofog-python-sdk',
     license='EPL-2.0',
     author='Eclipse ioFog',
     author_email='edgemaster@iofog.org',
-    description='Native python SDK for Eclipse ioFog development.',
-    requires=['ws4py'],
-    keywords='iofog IoT Eclipse fog computing edgeworx',
+    description='Python SDK for Eclipse ioFog development.',
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    requires=['ws4py', 'json', 'requests'],
+    keywords='ioFog IoT Eclipse fog computing edgeworx',
 )


Test patch:
diff --git a/test.sh b/test.sh
deleted file mode 100644
index 95e8296..0000000
--- a/test.sh
+++ /dev/null
@@ -1,31 +0,0 @@
-#!/usr/bin/env bash
-
-PYTHON_VERSIONS=( "python3" "python2" )
-FAILURES=( 0 0 )
-
-idx=0
-for version in ${PYTHON_VERSIONS[@]}; do
-
-    docker pull iofog/test-python-sdk-send::${version}
-    docker pull iofog/test-python-sdk-recieve:${version}
-
-    docker run -d iofog/test-python-sdk-send:${version}
-    docker run -d iofog/test-python-sdk-recieve:${version}
-
-    RECIEVE_CONTAINER_ID=$( docker ps | grep "python recieve.py" | awk '{print $1}' )
-    SEND_CONTAINER_ID=$( docker ps | grep "python send.py" | awk '{print $1}' )
-
-    if [[ "${RECIEVE_CONTAINER_ID}" -eq "" || "${SEND_CONTAINER_ID}" -eq "" ]]; then
-        echo "${version} has failed to send/recieve data"
-        ${FAILURES[idx]}= 1
-    fi
-    idx+=1
-done
-
-for failure in ${FAILURES[@]}; do
-    if [[ ${failure} == 1 ]]; then
-        exit 1
-    fi
-done
-
-exit 0
diff --git a/test/__init__.py b/test/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/test/bootstrap.bash b/test/bootstrap.bash
new file mode 100755
index 0000000..ce92607
--- /dev/null
+++ b/test/bootstrap.bash
@@ -0,0 +1,12 @@
+#!/bin/bash
+
+set -e
+
+# iofogctl
+curl -s https://packagecloud.io/install/repositories/iofog/iofogctl/script.deb.sh | sudo bash
+sudo apt install -qy iofogctl
+iofogctl version
+
+# pytest etc
+pip install -r requirements.txt
+pip install -r test/requirements.txt
diff --git a/test/conf/app.yaml b/test/conf/app.yaml
new file mode 100644
index 0000000..038e8ec
--- /dev/null
+++ b/test/conf/app.yaml
@@ -0,0 +1,45 @@
+---
+apiVersion: iofog.org/v2
+kind: Application
+metadata:
+  name: func-app
+spec:
+  microservices:
+  - name: func-app-server
+    agent:
+      name: agent-1
+      config:
+        bluetoothEnabled: true # this will install the iofog/restblue microservice
+        abstractedHardwareEnabled: false
+    images:
+      arm: edgeworx/healthcare-heart-rate:arm-v1
+      x86: edgeworx/healthcare-heart-rate:x86-v1
+      registry: remote # public docker
+    container:
+      rootHostAccess: false
+      ports: []
+    config:
+      test_mode: true
+      data_label: 'Anonymous_Person'
+  # Simple JSON viewer for the heart rate output
+  - name: func-app-ui
+    agent:
+      name: agent-1
+    images:
+      arm: edgeworx/healthcare-heart-rate-ui:arm
+      x86: edgeworx/healthcare-heart-rate-ui:x86
+      registry: remote
+    container:
+      rootHostAccess: false
+      ports:
+        # The ui will be listening on port 80 (internal).
+        - external: 5000
+          internal: 80
+          public: 5000
+      volumes:
+      - hostDestination: /tmp/iofog
+        containerDestination: /data
+        accessMode: rw
+      env:
+        - key: BASE_URL
+          value: http://localhost:8080/data
\ No newline at end of file
diff --git a/test/conf/ecn.yaml b/test/conf/ecn.yaml
new file mode 100644
index 0000000..42f5659
--- /dev/null
+++ b/test/conf/ecn.yaml
@@ -0,0 +1,22 @@
+---
+apiVersion: iofog.org/v2
+kind: LocalControlPlane
+metadata:
+  name: local-1
+spec:
+  iofogUser:
+    name: Foo
+    surname: Bar
+    email: user@domain.com
+    password: g9hr823rhuoi
+  controller:
+    container:
+      image: gcr.io/focal-freedom-236620/controller:develop
+---
+apiVersion: iofog.org/v2
+kind: LocalAgent
+metadata:
+  name: local-1
+spec:
+  container:
+    image: gcr.io/focal-freedom-236620/agent:develop
diff --git a/test/conf/vars.bash b/test/conf/vars.bash
new file mode 100644
index 0000000..a13a523
--- /dev/null
+++ b/test/conf/vars.bash
@@ -0,0 +1,1 @@
+NS="pytest"
diff --git a/test/conftest.py b/test/conftest.py
new file mode 100644
index 0000000..c370f57
--- /dev/null
+++ b/test/conftest.py
@@ -0,0 +1,1 @@
+pytest_plugins = ("dependency",)
diff --git a/test/log.py b/test/log.py
new file mode 100644
index 0000000..18521d1
--- /dev/null
+++ b/test/log.py
@@ -0,0 +1,43 @@
+import pytest
+import random
+import string
+from iofog.microservices.log import Logger
+
+@pytest.fixture(autouse=True)
+def _fixture():
+    return {}
+
+@pytest.mark.dependency()
+def test_first_logger(_fixture):
+    id = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(10))
+    name = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(10))
+    logger = Logger(name, device_id=id, log_dir="/tmp")
+    logger.info("hello")
+    logger.debug("world")
+    logger.warning("good")
+    logger.error("bye")
+    with open("/tmp/{}.log".format(name)) as log_file:
+        for line in log_file:
+            assert id in line
+            assert "deviceId" in line
+            assert "message" in line
+            assert "level" in line
+            assert "hostname" in line
+            assert "timestamp" in line
+
+
+@pytest.mark.dependency()
+def test_second_logger(_fixture):
+    name = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(10))
+    logger = Logger(name, log_dir="/tmp/")
+    logger.info("hello")
+    logger.debug("world")
+    logger.warning("good")
+    logger.error("bye")
+    with open("/tmp/{}.log".format(name)) as log_file:
+        for line in log_file:
+            assert "deviceId" not in line
+            assert "message" in line
+            assert "level" in line
+            assert "hostname" in line
+            assert "timestamp" in line
\ No newline at end of file
diff --git a/test/microservices.py b/test/microservices.py
new file mode 100644
index 0000000..7094580
--- /dev/null
+++ b/test/microservices.py
@@ -0,0 +1,38 @@
+import pytest
+from iofog.microservices.client import Client
+from iofog.microservices.iomessage import IoMessage
+
+class TestMicroservicesState:
+    def __init__(self):
+        self.msg = IoMessage()
+        self.msg.infotype = "infotype"
+        self.msg.infoformat = "infoformat"
+        self.msg.tag = "tag"
+        self.msg.groupid = "groupid"
+        self.msg.authid = "authid"
+        self.msg.authgroup = "authgroup"
+        self.msg.hash = "hash"
+        self.msg.previoushash = "previoushash"
+        self.msg.nonce = "nonce"
+        self.msg.contentdata = "python3"
+        self.msg.contextdata = ""
+
+state = TestMicroservicesState()
+
+@pytest.fixture(autouse=True)
+def _fixture():
+    return state
+
+@pytest.mark.dependency()
+def test_send(_fixture):
+    client = Client("sender")
+    receipt = client.post_message(_fixture.msg)
+    assert isinstance(receipt, dict)
+
+@pytest.mark.dependency(depends=["test_send"])
+def test_recieve(_fixture):
+    client = Client("reciever")
+    msgs = client.get_next_messages()
+    for msg in msgs:
+        assert isinstance(msg, bytes)
+        assert msg == _fixture.msg.contentdata
diff --git a/test/recieve.py b/test/recieve.py
deleted file mode 100644
index 6e3b512..0000000
--- a/test/recieve.py
+++ /dev/null
@@ -1,23 +0,0 @@
-from iofog_python_sdk.client import IoFogClient
-from iofog_python_sdk.exception import IoFogException
-from iofog_python_sdk.iomessage import IoMessage
-from iofog_python_sdk.listener import *
-
-
-def main():
-    try:
-        client = IoFogClient()
-    except IoFogException as e:
-        print("Failed to start client")
-
-    while True:
-        try:
-            messages = client.get_next_messages()
-            for message in messages:
-                if message.contentdata == "python2" or message.contentdata == "python3":
-                    print("It worked")
-                else:
-                    exit(1)
-        except IoFogException as e:
-            print("Could not get next message")
-            exit(e)
diff --git a/test/requirements.txt b/test/requirements.txt
new file mode 100644
index 0000000..0040581
--- /dev/null
+++ b/test/requirements.txt
@@ -0,0 +1,5 @@
+PyYAML==5.3.1
+requests==2.24.0
+pytest==5.4.3
+pytest-dependency==0.5.1
+ws4py==0.5.1
diff --git a/test/rest.py b/test/rest.py
new file mode 100644
index 0000000..11cbe7a
--- /dev/null
+++ b/test/rest.py
@@ -0,0 +1,59 @@
+import pytest
+from iofog.rest.controller.client import Client
+
+class TestRestState:
+    def __init__(self):
+        self.email = "serge@edgeworx.io"
+        self.password = "wfhoi982bv1sfdjoi"
+        self.name = "Serge"
+        self.surname = "Radinovich"
+        self.controller_address = "localhost"
+        self.controller_port = 51121
+
+state = TestRestState()
+
+@pytest.fixture(autouse=True)
+def _fixture():
+    return state
+
+@pytest.mark.dependency()
+def test_create_user(_fixture):
+    _fixture.client = Client(_fixture.controller_address,
+                             _fixture.controller_port,
+                             _fixture.email,
+                             _fixture.password,
+                             _fixture.name,
+                             _fixture.surname)
+
+@pytest.mark.dependency(depends=["test_create_user"])
+def test_get_status(_fixture):
+    status = _fixture.client.get_status()
+    assert isinstance(status, dict)
+    assert status["status"] == "online"
+    assert isinstance(status["timestamp"], int)
+    assert isinstance(status["uptimeSec"], float)
+    assert isinstance(status["versions"], dict)
+    assert isinstance(status["versions"]["controller"], str)
+    assert isinstance(status["versions"]["ecnViewer"], str)
+
+@pytest.mark.dependency(depends=["test_create_user"])
+def test_create_agent(_fixture):
+    name = "agent-1"
+    uuid = _fixture.client.create_agent(name, "localhost", "x86")
+    assert isinstance(uuid, str)
+    assert(_fixture.client.get_agent_uuid(name) == uuid)
+
+@pytest.mark.dependency(depends=["test_create_agent"])
+def test_create_app(_fixture):
+    resp = _fixture.client.create_app_from_yaml("test/conf/app.yaml")
+    assert isinstance(resp, dict)
+    print(resp)
+
+@pytest.mark.dependency(depends=["test_create_app"])
+def test_delete_app(_fixture):
+    _fixture.client.delete_app("func-app")
+
+@pytest.mark.dependency(depends=["test_delete_app"])
+def test_delete_agent(_fixture):
+    name = "agent-1"
+    _fixture.client.delete_agent(name)
diff --git a/test/run.bash b/test/run.bash
new file mode 100755
index 0000000..bc94245
--- /dev/null
+++ b/test/run.bash
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+set -e
+
+# Get test names from args, run all if empty
+TESTS="$1"
+if [ -z "$TESTS" ]; then
+    TESTS=("rest" "log")
+fi
+
+# Run specified tests
+for TEST in ${TESTS[@]}; do
+    pytest -v "test/${TEST}.py"
+done
diff --git a/test/send.py b/test/send.py
deleted file mode 100644
index 4ce9fb2..0000000
--- a/test/send.py
+++ /dev/null
@@ -1,68 +0,0 @@
-from iofog_python_sdk.client import IoFogClient
-from iofog_python_sdk.exception import IoFogException
-from iofog_python_sdk.iomessage import IoMessage
-from iofog_python_sdk.listener import *
-
-import sys
-
-def build_message():
-    msg = IoMessage()
-    msg.infotype = "infotype"
-    msg.infoformat = "infoformat"
-    msg.contextdata = ""
-    msg.tag = "tag"
-    msg.groupid = "groupid"
-    msg.authid = "authid"
-    msg.authgroup = "authgroup"
-    msg.hash = "hash"
-    msg.previoushash = "previoushash"
-    msg.nonce = "nonce"
-
-    return msg
-
-def python3():
-    try:
-        client = IoFogClient()
-    except IoFogException as e:
-        print("Client Failed on Python3")
-        exit(e)
-
-    msg = build_message()
-    msg.contentdata = "python3"
-
-    try:
-        receipt = client.post_message(msg)
-    except IoFogException as e:
-        print("Message Failed on Python3")
-        exit(e)
-
-    print("Python3 working with SDK")
-
-
-def python2():
-    try:
-        client = IoFogClient()
-    except IoFogException as e:
-        print("Client Failed on Python2")
-        exit(e)
-
-    msg = build_message()
-    msg.contentdata = "python2"
-
-    try:
-        receipt = client.post_message(msg)
-    except IoFogException as e:
-        print("Message Failed on Python2")
-        exit(e)
-
-    print("Python2 working with SDK")
-
-def main():
-    if sys.version_info[0] > 3:
-        print("test python3")
-        while True:
-            python3()
-    else:
-        print("test python2")
-        while True:
-            python2()
\ No newline at end of file
diff --git a/test/setup.bash b/test/setup.bash
new file mode 100755
index 0000000..039c312
--- /dev/null
+++ b/test/setup.bash
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+set -e
+
+# Get variables
+. test/conf/vars.bash
+
+# Create namespace
+iofogctl create namespace "$NS" -v
+iofogctl configure current-namespace "$NS" -v
+
+# Deploy local ECN
+iofogctl deploy -f test/conf/ecn.yaml -v
diff --git a/test/teardown.bash b/test/teardown.bash
new file mode 100755
index 0000000..0073a05
--- /dev/null
+++ b/test/teardown.bash
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Get variables
+. test/conf/vars.bash
+
+# Create namespace
+iofogctl delete namespace "$NS" --force -v