Repository: xlab-si/xopera-opera

Issue:
Policies in service templates should be serialized as a list
## Description
This issue describe the inconsistency spotted in opera's TOSCA YAML v1.3 parser. [Section 3.9 of TOSCA Simple Profile in YAML v1.3](https://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.3/os/TOSCA-Simple-Profile-YAML-v1.3-os.html#_Toc26969450) states that `policies` keyname within the `topology_template` should be represented as an optional list of TOSCA policy definitions that apply to the topology template.

So, the syntax for TOSCA policies here should look like this:

```yaml
topology_template:
    node_templates:
    # left off for brevity

    policies:
      - policy1:
          type: tosca.policies.placement
          # left off for brevity
      - policy2:
          type: tosca.policies.placement
          # left off for brevity
      - policy3:
          type: tosca.policies.placement
          # left off for brevity
```

When we try to use that syntax and deploy it with opera, we get an error (`Expected map.`), saying that opera expected a YAML dictionary/map for the `policies` keyname. And by looking into opera's parser we soon realize that policies are not serialized as a list, but rather as a map which is in contradiction with the TOSCA standard and it needs to be fixed. 

## Steps
To fix the issue in the TOSCA parser, we should modify the content in `src/opera/parser/tosca/v_1_3/topology_template.py` and also fix any tests or examples that use the wrong synax. 

## Current behaviour
Right now opera treats policies in `topology_template` as YAML maps.

## Expected results
Opera's TOSCA parser should serialize policies as YAML lists.


Discussion:

Patch:
diff --git a/docs/index.rst b/docs/index.rst
index db233d6..f7328c8 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -12,6 +12,8 @@ The following documentation explains the usage of xOpera TOSCA orchestrator. Wan
     installation
     examples
     documentation
+    saas
+
 ..
    Indices and tables
    ==================
diff --git a/docs/saas.rst b/docs/saas.rst
new file mode 100644
index 0000000..1080af4
--- /dev/null
+++ b/docs/saas.rst
@@ -0,0 +1,74 @@
+.. _SaaS:
+
+***********
+xOpera SaaS
+***********
+
+The Software as a Service edition of xOpera is available at https://xopera-radon.xlab.si/ui/_.
+
+It is a multi-user multi-platform multi-deployment multifunctional service offering all capabilities of the
+console-based ``opera``, providing all of its functionalities.
+
+Please read the warnings below, as you accept some inherent risks when using xOpera-SaaS
+
+Using the browser version is straightforward.
+
+Using the xOpera SaaS API through ``curl``::
+
+    csar_base64="$(base64 --wrap 0 test.csar)"
+    api="https://xopera-radon.xlab.si/api"
+    secret_base64="$(echo 'hello!' | base64 -)"
+
+    your_username=YOUR_USERNAME
+    your_password=YOUR_PASSWORD
+
+    # login process (would be automated by browser)
+    alias cookiecurl="curl -sSL --cookie-jar cookiejar.txt --cookie cookiejar.txt"
+    response="$(cookiecurl $api/credential)"
+    redirect_url="$(echo $response | xmllint --html --xpath "string(//form[@id='kc-form-login']/@action)" - 2>/dev/null)"
+    cookiecurl "$redirect_url" -d "username=$your_username" -d "password=$your_password" -d credentialId=""
+
+    # normal usage
+    cookiecurl "$api/credential"
+    cookiecurl "$api/credential" -XPOST -d "{\"name\": \"credential1\", \"path\": \"/tmp/credential.txt\", \"contents\": \"$secret_base64\"}"
+    cookiecurl "$api/credential"
+    cookiecurl "$api/credential/1"
+    cookiecurl "$api/workspace"
+    cookiecurl "$api/workspace" -XPOST -d '{"name": "workspace1"}'
+    cookiecurl "$api/workspace/1/credential/1" -XPUT
+    cookiecurl "$api/workspace/1/credential"
+    cookiecurl "$api/credential/1"
+    cookiecurl "$api/workspace/1"
+    cookiecurl "$api/workspace/1/project" -XPOST -d "{\"name\": \"myproject\", \"csar\": \"$csar_base64\"}"
+    cookiecurl "$api/workspace/1/project"
+    cookiecurl "$api/workspace/1"
+    cookiecurl "$api/workspace/1/project/1/creationStatus"
+    cookiecurl "$api/workspace/1/project/1/debugPackage"
+
+    # interaction with the project (identical to xopera-api), instructions copied from there
+    project_url="$api/workspace/1/project/1"
+    cookiecurl "$project_url/status"
+    cookiecurl "$project_url/validate" -XPOST -H "Content-Type: application/json" -d @inputs-request.json
+    cookiecurl "$project_url/deploy" -XPOST -H "Content-Type: application/json" -d @inputs-request.json
+    cookiecurl "$project_url/status" | jq
+    cookiecurl "$project_url/outputs"
+    cookiecurl "$project_url/undeploy" -XPOST
+
+For further interaction with each project, see
+`the xopera-api specification <https://github.com/xlab-si/xopera-api/blob/master/openapi-spec.yml>`_
+
+
+====================================================
+Warnings about your credentials and general security
+====================================================
+
+Your credentials - not for xOpera-SaaS, but those you add for services you access in CSARs etc - are stored in
+plaintext on the server xOpera-SaaS is running on.
+All assigned workspaces have access to them, as they have control of the filesystem, therefore all users with access
+to the workspace also have access to them.
+You need to use caution with the credentials you submit.
+
+If you request xOpera-SaaS server administrators to help you or access your project, they will also be in a position
+to access the credentials.
+Whenever possible, use temporary credentials with limited access to the smallest required set of capabilities
+to improve you security.
diff --git a/examples/attribute_mapping/service.yaml b/examples/attribute_mapping/service.yaml
index cfd8b1d..61b7370 100644
--- a/examples/attribute_mapping/service.yaml
+++ b/examples/attribute_mapping/service.yaml
@@ -20,7 +20,7 @@ node_types:
         operations:
           create:
             inputs:
-              id: { default: { get_property: [ SELF, enrolment_number ] } }
+              id: { default: { get_property: [ SELF, enrolment_number ] }, type: integer }
             outputs:
               student_id: [ SELF, student_id ]
             implementation: playbooks/create-student.yaml
@@ -51,8 +51,10 @@ relationship_types:
             inputs:
               student_id:
                 default: { get_attribute: [ TARGET, student_id ] }
+                type: string
               student_ids:
                 default: { get_attribute: [ SOURCE, student_ids ] }
+                type: list
             outputs:
               new_list: [ SOURCE, student_ids ]
             implementation: playbooks/teacher-teaches-student--preconfigure-source.yaml
diff --git a/examples/nginx_openstack/library/nginx/server/types.yaml b/examples/nginx_openstack/library/nginx/server/types.yaml
index 845e467..45a73e6 100644
--- a/examples/nginx_openstack/library/nginx/server/types.yaml
+++ b/examples/nginx_openstack/library/nginx/server/types.yaml
@@ -22,7 +22,6 @@ node_types:
             implementation:
               primary: playbooks/uninstall.yaml
 
-
 relationship_types:
   my.relationships.NginxSiteHosting:
     derived_from: tosca.relationships.HostedOn
@@ -33,5 +32,6 @@ relationship_types:
             inputs:
               marker:
                 default: { get_attribute: [ TARGET, host, id ] }
+                type: string
             implementation:
               primary: playbooks/reload.yaml
diff --git a/examples/nginx_openstack/library/openstack/vm/types.yaml b/examples/nginx_openstack/library/openstack/vm/types.yaml
index df58465..31411e7 100644
--- a/examples/nginx_openstack/library/openstack/vm/types.yaml
+++ b/examples/nginx_openstack/library/openstack/vm/types.yaml
@@ -34,15 +34,16 @@ node_types:
         operations:
           create:
             inputs:
-              vm_name:  { default: { get_property: [ SELF, name     ] } }
-              image:    { default: { get_property: [ SELF, image    ] } }
-              flavor:   { default: { get_property: [ SELF, flavor   ] } }
-              network:  { default: { get_property: [ SELF, network  ] } }
-              key_name: { default: { get_property: [ SELF, key_name ] } }
+              vm_name:  { default: { get_property: [ SELF, name     ] }, type: string }
+              image:    { default: { get_property: [ SELF, image    ] }, type: string }
+              flavor:   { default: { get_property: [ SELF, flavor   ] }, type: string }
+              network:  { default: { get_property: [ SELF, network  ] }, type: string }
+              key_name: { default: { get_property: [ SELF, key_name ] }, type: string }
               security_groups:
-                default: { get_property: [ SELF, security_groups  ] }
+                default: { get_property: [ SELF, security_groups ] }
+                type: string
             implementation: playbooks/create.yaml
           delete:
             inputs:
-              id: { default: { get_attribute: [ SELF, id ] } }
+              id: { default: { get_attribute: [ SELF, id ] }, type: string }
             implementation: playbooks/delete.yaml
diff --git a/examples/policy_triggers/service.yaml b/examples/policy_triggers/service.yaml
index a8addc9..a131c56 100644
--- a/examples/policy_triggers/service.yaml
+++ b/examples/policy_triggers/service.yaml
@@ -34,7 +34,7 @@ interface_types:
     operations:
       scale_down:
         inputs:
-          adjustment: { default: { get_property: [ SELF, name ] } }
+          adjustment: { default: 1, type: integer }
         description: Operation for scaling down.
         implementation: playbooks/scale_down.yaml
 
@@ -43,7 +43,7 @@ interface_types:
     operations:
       scale_up:
         inputs:
-          adjustment: { default: { get_property: [ SELF, name ] } }
+          adjustment: { default: 1, type: integer }
         description: Operation for scaling up.
         implementation: playbooks/scale_up.yaml
 
@@ -83,8 +83,8 @@ policy_types:
         condition:
           - not:
             - and:
-              - available_instances: [ { greater: 42 } ]
-              - available_space: [ { greater: 1000 } ]
+              - available_instances: [ { greater_than: 42 } ]
+              - available_space: [ { greater_than: 1000 } ]
         action:
           - call_operation:
               operation: radon.interfaces.scaling.ScaleDown.scale_down
@@ -116,8 +116,8 @@ policy_types:
         condition:
           - not:
             - and:
-              - available_instances: [ { greater: 42 } ]
-              - available_space: [ { greater: 1000 } ]
+              - available_instances: [ { greater_than: 42 } ]
+              - available_space: [ { greater_than: 1000 } ]
         action:
           - call_operation:
               operation: radon.interfaces.scaling.ScaleUp.scale_up
@@ -160,52 +160,46 @@ topology_template:
         key_name: my_key
       requirements:
         - host: workstation
-      capabilities:
-        host_capability:
-         properties:
-           num_cpus: 1
-           disk_size: 10 GB
-           mem_size: 4096 MB
 
   policies:
-    scale_down:
-      type: radon.policies.scaling.ScaleDown
-      properties:
-        cpu_upper_bound: 90
-        adjustment: 1
+    - scale_down:
+        type: radon.policies.scaling.ScaleDown
+        properties:
+          cpu_upper_bound: 90
+          adjustment: 1
 
-    scale_up:
-      type: radon.policies.scaling.ScaleUp
-      properties:
-        cpu_upper_bound: 90
-        adjustment: 1
+    - scale_up:
+        type: radon.policies.scaling.ScaleUp
+        properties:
+          cpu_upper_bound: 90
+          adjustment: 1
 
-    autoscale:
-      type: radon.policies.scaling.AutoScale
-      properties:
-        min_size: 3
-        max_size: 7
-      targets: [ openstack_vm ]
-      triggers:
-        radon.triggers.scaling:
-          description: A trigger for autoscaling
-          event: auto_scale_trigger
-          schedule:
-            start_time: 2020-04-08T21:59:43.10-06:00
-            end_time: 2022-04-08T21:59:43.10-06:00
-          target_filter:
-            node: openstack_vm
-            requirement: workstation
-            capability: host_capability
-          condition:
-            constraint:
-              - not:
-                - and:
-                  - available_instances: [ { greater: 42 } ]
-                  - available_space: [ { greater: 1000 } ]
-            period: 60 sec
-            evaluations: 2
-            method: average
-          action:
-            - call_operation: radon.interfaces.scaling.AutoScale.retrieve_info
-            - call_operation: radon.interfaces.scaling.AutoScale.autoscale
+    - autoscale:
+        type: radon.policies.scaling.AutoScale
+        properties:
+          min_size: 3
+          max_size: 7
+        targets: [ openstack_vm ]
+        triggers:
+          radon.triggers.scaling:
+            description: A trigger for autoscaling
+            event: auto_scale_trigger
+            schedule:
+              start_time: 2020-04-08T21:59:43.10-06:00
+              end_time: 2022-04-08T21:59:43.10-06:00
+            target_filter:
+              node: openstack_vm
+              requirement: workstation
+              capability: host_capability
+            condition:
+              constraint:
+                - not:
+                  - and:
+                    - available_instances: [ { greater_than: 42 } ]
+                    - available_space: [ { greater_than: 1000 } ]
+              period: 60 sec
+              evaluations: 2
+              method: average
+            action:
+              - call_operation: radon.interfaces.scaling.AutoScale.retrieve_info
+              - call_operation: radon.interfaces.scaling.AutoScale.autoscale
diff --git a/src/opera/parser/tosca/v_1_3/topology_template.py b/src/opera/parser/tosca/v_1_3/topology_template.py
index ce17793..a973590 100644
--- a/src/opera/parser/tosca/v_1_3/topology_template.py
+++ b/src/opera/parser/tosca/v_1_3/topology_template.py
@@ -3,6 +3,7 @@ from opera.template.topology import Topology
 
 from ..entity import Entity
 from ..map import Map
+from ..list import List
 from ..string import String
 
 from .group_definition import GroupDefinition
@@ -19,7 +20,7 @@ class TopologyTemplate(Entity):
         node_templates=Map(NodeTemplate),
         relationship_templates=Map(RelationshipTemplate),
         groups=Map(GroupDefinition),
-        policies=Map(PolicyDefinition),
+        policies=List(Map(PolicyDefinition)),
         outputs=Map(ParameterDefinition),
         # TODO(@tadeboro): substitution_mappings and workflows
     )


Test patch:
diff --git a/tests/integration/misc-tosca-types/service-template.yaml b/tests/integration/misc-tosca-types/service-template.yaml
index 610c071..27818b0 100644
--- a/tests/integration/misc-tosca-types/service-template.yaml
+++ b/tests/integration/misc-tosca-types/service-template.yaml
@@ -105,11 +105,11 @@ topology_template:
       members: [ my-workstation1, my-workstation2 ]
 
   policies:
-    test:
-      type: daily_test_policies.test
-      properties:
-        test_id: *test
-      targets: [ hello, setter, workstation_group ]
+    - test:
+        type: daily_test_policies.test
+        properties:
+          test_id: *test
+        targets: [ hello, setter, workstation_group ]
 
   outputs:
     output_prop:
diff --git a/tests/unit/opera/parser/tosca/v_1_3/test_topology_template.py b/tests/unit/opera/parser/tosca/v_1_3/test_topology_template.py
index 6ffaa6c..9423610 100644
--- a/tests/unit/opera/parser/tosca/v_1_3/test_topology_template.py
+++ b/tests/unit/opera/parser/tosca/v_1_3/test_topology_template.py
@@ -20,8 +20,8 @@ class TestParse:
               my_group:
                 type: group.type
             policies:
-              my_policy:
-                type: policy.type
+              - my_policy:
+                  type: policy.type
             outputs:
               my_output:
                 type: string