pyagena 1.0.4
Python utility library for agena.ai to create Bayesian network models from scratch or import existing models and export to agena.ai cloud or local API for calculations.

# Table of Contents

* [Description](#1-description)
* [Prerequisites and Installation](#2-prerequisites-and-installation)
* [Structure of pyagena Classes](#3-structure-of-pyagena-classes)
* [Class and Instance Methods](#4-class-and-instance-methods)
* [Importing a Model from .cmpx](#5-importing-a-model-from-cmpx)
* [Creating and Modifying a Model in python](#6-creating-and-modifying-a-model-in-python)
* [Agena.ai Cloud with pyagena](#7-agenaai-cloud-with-pyagena)
* [Local agena.ai API with pyagena](#8-local-agenaai-api-with-pyagena)
* [Importing and Exporting Datasets for Batch Calculation](#9-importing-and-exporting-datasets-for-batch-calculation)
* [pyagena Use Case Examples](#10-pyagena-use-case-examples)

# 1. Description

pyagena is a python environment for creating, modifying, and parsing Bayesian network models, and sending the models to agena.ai Cloud or to local agena.ai developer API to execute calculation requests. The environment allows users to read and modify Bayesian networks from .cmpx model files, create new Bayesian networks in python and export to .cmpx and .json files locally, as well as authenticate with agena.ai Cloud or use local agena.ai developer API for model calculations and sensitivity analyses. In the rest of this document, the python environment for agena.ai is referred to as pyagena. pyagena is developed for Python 3.8+.

# 2. Prerequisites and Installation

You can install pyagena using [pip](https://pypi.org/project/pip/):

```bash
pip install pyagena
```

pyagena requires the following python packages to be installed: `requests`, `pandas`, `networkx`, and `matplotlib`

In case these were not installed automatically, you can install them with the commands:

```bash
pip install requests
pip install pandas
pip install matplotlib
pip install networkx
```

Once installed, you can import the package in the python script (or a jupyter notebook file) to use. For example:

```python
import pyagena as ag
```

or

```python
from pyagena import *
```

# 3. Structure of pyagena Classes

The Bayesian networks (BNs) in pyagena are represented with several objects: `Model`, `Network`, `Node`, and `Dataset`. These python objects generally follow their equivalents defined in agena.ai models.

## 3.1 `Model` objects

These represent the overall BN. A single .cmpx file corresponds to a singe `Model`. A BN model can have multiple networks with their own nodes, links between these networks, and datasets. 

## 3.1.1 `id`

Id of the Model, optional. 

## 3.1.2 `networks`

A list of all the `Network` objects in the model. This field will be populated with the `create_network()` method. See below for more information about `Network`s.

## 3.1.3 `datasets`

A list of all the `Dataset` objects in the model. When a new `Model` is created, it will have the default first dataset "Case 1", and it is possible to add new datasets to the model afterwards with the `create_dataset()` method. See below for more information about `Dataset`s.

## 3.1.4 `network_links`

If the `Model` has multiple networks, it is possible to have links between these networks, following the agena.ai model network_links format.

To see how to create these links, see `add_network_link()` function later in this document.

## 3.1.5 `settings`

`Model` settings for calculations. It includes the following fields (the values in parantheses are the defaults if settings are not specified for a model):

* discreteTails (False)
* sampleSizeRanked (5)
* convergence (0.01)
* iterations (50)
* tolerance (1)

Model settings can be provided when creating a new model, if not provided the model will come with the default settings. Default settings can be changed later on (with the method `change_settings()`), or model settings can be reset back to default values (with the method `default_settings()`). See the correct input parameter format for these functions in the following section. Individual fields in model setting can be adjusted with `change_settings()` too. Individual setting value can be accessed with `syntax model.settings['convergence']`.

## 3.2 `Network` objects 

These represent each network in a BN model. Networks consist of nodes and in a BN model there might be more than one network. These networks can also be linked to each other with the use of input and output nodes. For such links, see `Model.network_links` field.

The fields that define a `Network` object are as follows:

### 3.2.1 `id`

Id of the `Network`. Mandatory field to create a new network.

### 3.2.2 `name`

Name of the network, optional. If not specified, `id` of the network is passed onto `name` field as well.

### 3.2.3 `description`

Description, optional. If not specified, the string "New Network" is assigned to `description` field by default.

### 3.2.4 `nodes`

A list of `Node` objects which are in the network. See below for more information about `Node`s.

Note that `Network` objects do have a `create_edge()` method but do not have a `links` field unlike the agena.ai models. As explained in `Node.parents` section below, this information is stored in `Node` objects in the python environment. When importing a .cmpx model, the information in `links` field is used to populate `Node.parents` fields for each node. Similarly, when exporting to a .cmpx/.json file, the parent-child information in `Node.parents` field is used to create the `links` field of the `Network` field of the .cmpx/.json.

## 3.3 `Node` objects

These represent the nodes in a BN. The fields that define a `Node` object are as follows:

### 3.3.1 `id`

Mandatory field to create a new `Node` object. This is the unique identifier of agena.ai model nodes.

### 3.3.2 `name`

Name of the node, optional. If not defined, `id` of the node will be passed onto the `name` field too.

### 3.3.3 `description`

Description of the node, optional. If not defined, "New Node" will be assigned to the `description` field.

### 3.3.4 `type`

Node type, it can be:

* Boolean
* Labelled
* Ranked
* DiscreteReal
* ContinuousInterval
* IntegerInterval

If it's not specified when creating a new node, the new node is "Boolean" by default if it's not a simulation node; and it is "ContinuousInterval" by default if it's a simulation node.

### 3.3.5 `parents`

Other `Node` objects can be pointed as parents of a `Node` object. To add parents to a node, see the node method `add_parent()` or network method `create_edge()`.

Something to keep in mind: the parent-child relationship information is stored at `Node` level in the python environment thanks to this field, as opposed to the separate `links` field of a .cmpx/.json file for the agena.ai models. When importing or exporting .cmpx files you do not need to think about this difference, as the file import and export functions handle the correct formats. This difference allows adding and removing `Node` objects as parents.

### 3.3.6 `simulated`

A boolean field to indicate whether the node is a simulation node or not.

### 3.3.7 `distr_type`

The table type of the node, it can be:

* Manual
* Expression
* Partitioned

To set the table (distribution) type of a node, see the node method `set_distr_type()` or the network method `set_node_distr_type()`.

### 3.3.8 `states`

States of the node (if not simulated). If states are not specified, depending on the `type`, sensible default states are assigned. Default states for different node types are:

* "Boolean" or "Labelled" node: "False", "True"
* "Ranked" node: "Low", "Medium", "High"
* "DiscreteReal" node: "0", "1"
* "IntegerInterval" node (if not simulated): "(-Infinity, -1]", "[0, 4]", "[5, Infinity)"
* "ContinuousInterval" node (if not simulated): "(-Infinity, -1)", "[-1, 1)", "[1, Infinity)"

And for a node with the table type (`distr_type`) "Expression", the default expression is: "Normal(0,1000000)".

To set new states to an existing nodes, see the node method `set_states()` or the network method `set_node_states()`.

### 3.3.9 `probabilities`

If the table type (`distr_type`) of the node is "Manual", the node will have state probabilities, values in its NPT. This field is a list of lists containing these values. The length of the list depends on the node states and the number of its parents. To see how to set probability values for a node, see the node method `set_probabilities()` or network method `set_node_probabilities()`. 

### 3.3.10 `expressions`

If the table type (`distr_type`) of the node is "Expression" or "Partitioned", the node will have expression(s) instead of the manually defined NPT values.

* If the node's table type is "Expression", the `expressions` field will be a list with a single expression.
* If the node's table type is "Partitioned", the `expressions` field will be a list of as many expressions as the number of parent node states on which the expression is partitioned.

To see how to set the expressions for a node, see the node method `set_expressions()` or the network method `set_node_expressions()`.

Possible expressions for the node types are listed below:

Boolean and Labelled:
* Comparative

IntegerInterval
* Binomial
* Exponential 
* Geometric 
* Hypergeometric 
* Negative Binomial 
* Normal 
* Poisson 
* TNormal 
* Triangular 
* Uniform 

ContinuousInterval
* Arithmetic
* Beta
* BetaPert
* ChiSquared
* Exponential
* ExtremeValue
* Gamma
* Logistic
* LogNormal
* Normal
* Student
* TNormal
* Triangle
* Uniform
* Weibull

Ranked
* TNormal

For further information on expressions, please refer to [agena.ai modeller manual, Sections 22 (Statistical distributions) and 23 (Expressions)](https://resources.agena.ai/materials/agena.ai%20modeller%20user%20manual.pdf).

### 3.3.11 `partitions`

If the table type (`distr_type`) of the node is "Partitioned", in addition to the expressions, the node will have the `partitions` field. This field is a list of strings, which are `id`s of the parent nodes on which the node expression is partitioned.

### 3.3.12 `variables`

The node variables are called constants on agena.ai Modeller. This field, if specified, sets the constant value for the node observations. For the correct syntax of defining a variable, see `set_variable()` function.

## 3.4 `Dataset` objects 

These represent the set of observations in a BN model (cases). A `Model` can have multiple `Dataset` objects in its `datasets` field.  When a new `Model` is created, it always comes with a default `Dataset` object with the `id` "Case 1" and with blank observations. It is possible to add more datasets (cases) with their `id`s. Each `Dataset` object under a `Model` can be called a new "case".

### 3.4.1 `id`

Id of the dataset (case).

### 3.4.2 `observations`

Under each dataset (case), observations for all the observed nodes in all the networks of the model (in terms of their states or values) are listed. If it's hard evidence, observation for a node will have a single value with the weight of 1. If a node in the model has a value in its `variable` field, this value will be passed onto the dataset (case) with the weight of 1.

### 3.4.3 `results`

This field is defined only for when a .cmpx model with calculations is imported or after the model is calculated with the agena.ai Cloud or local API. When creating a new BN in the python environment, this field is not filled in. The `results` field stores the posterior probability and inference results upon model calculation on agena.ai Cloud or local agena.ai developer API.

# 4. Class and Instance Methods

The `Model`, `Network`, `Node`, and `Dataset` objects have their own respective methods to help their definition and manipulate their fields. The python instance methods are used with the `.` sign following an instance of the class. For example,

```python
example_node.add_parent(example_parent_node)
```

or

```python
example_network.remove_node(example_node)
```

or

```python
example_model.create_dataset(example_case)
```

Before going into the details of methods, a note on displaying success messages and logging info as you go on with the pyagena usage: If you'd like to see the success messages and logging info, you can set the following once at the beginning of the work:

```python
set_verbose(True)
```

This is for success messages and logging info. The warning and error messages are displayed by default.

To also display timestamps of the messages:

```python
include_timestamps(True)
```

## 4.1 `Model` methods

A `Model` object consists of networks, network links, datasets, and settings. Once a new `Model` is created, it is possible to create `Network`s and `Dataset`s in it. By default, a new `Model` object comes with a single empty dataset (case) called "Case 1". Following methods can be used to modify `Model` objects: 

### 4.1.1 `create_network(id, name=None, description=None)`

A method to create a new `Network` object in a `Model` and add it to the `networks` field of the `Model` object. The input `id` is mandatory. The new `Network` object is added to the `Model` if the `Model` does not already have a `Network` with the given id.

### 4.1.2 `remove_network(id)`

A method to remove an existing `Network` object from the model using its `id`. Note that removing a `Network` from a `Model` doesn't automatically remove its possible network links to other `Network`s in the model. `network-links` field of a `Model` should be adjusted accordingly if needed.

### 4.1.3 `get_network(network_id)`

A method to access a `Network` object in a `Model`. The `Network` can be assigned to a variable and used.

### 4.1.4 `add_network_link(source_network_id, source_node_id, target_network_id, target_node_id, link_type, pass_state = optional)`

This is the method to add links to a model between its networks. These links start from a "source node" in a network and go to a "target node" in another network. To create the link, the source and target nodes in the networks need to be specified together with the network they belong to (by the `Node` and `Network` `id`s). The input parameters are as follows:

* `source_network_id` = `Network.id` of the network the source node belongs to
* `source_node_id` = `Node.id` of the source node
* `target_network_id` = `Network.id` of the network the target node belongs to
* `target_node_id` = `Node.id` of the target node
* `link_type` = a string of the link type name. If not specified, it is `Marginals`. It can be one of the following:
    * Marginals
    * Mean
    * Median
    * Variance
    * StandardDeviation
    * LowerPercentile
    * UpperPercentile
    * State
* `pass_state` = one of the `Node.states` of the source node. It has to be specified only if the `link_type` of the link is `"State"`, otherwise is left blank.

Note that links between networks are allowed only when the source and target nodes fit certain criteria. Network links are allowed if:

* Both nodes are the same type and either of them is simulated
* Both nodes are the same type and neither is simulated and both have the same number of states
* Source node is not numeric interval or discrete real and target node is simulated

### 4.1.5 `remove_network_link(source_network_id, source_node_id,target_network_id, target_node_id)`

A method to remove network links, given the `id`s of the source and target nodes (and the networks they belong to).

### 4.1.6 `remove_all_network_links()`

A method to remove all existing network links in a model.

### 4.1.7 `create_dataset(dataset_id)`

It is possible to add multiple cases to a model. These cases are new `Dataset` objects added to the `datasets` field of a model. Initially these cases have no observations and are only defined by their `id`s. The cases are populated with the `enter_observation()` function. The `create_dataset()` function takes the `id` of the new dataset to be added as input. 

### 4.1.8 `remove_dataset(dataset_id)`

A method to remove an existing scenario from the model. Input parameter `dataset_id` is the string which is the `id` of a dataset (case).

### 4.1.9 `get_dataset(dataset_id)`

A method to access a `Dataset` object in a `Model`. The `Dataset` can be assigned to a variable and used.

### 4.1.10 `enter_observation(network_id, node_id, value, dataset_id = optional, variable_input = False)`

A method to enter observation to a dataset in the model. Calls the dataset method `enter_observation` in the background.

To enter the observation to a specific dataset (case), the dataset id must be given as the input parameter `dataset`. If `dataset` is left blank, the entered observation will by default go to the first dataset (case) of the model (called "Case 1" by default). This means that if there is no extra datasets created for a model (which by default comes with "Case 1"), any observation entered will be set for this dataset (mimicking the behaviour of entering observation to agena.ai Modeller).

The observation is defined with the mandatory input parameters:
* `network_id` = `Network.id` of the network the observed node belongs to
* `node_id` = `Node.id` of the observed node
* `value` = this parameter can be:
    * the value or state of the observation for the observed node (if it is hard evidence)
    * the array of multiple values and their weights (if it is soft evidence)
    * the value of the observation for a variable of the observed node
* `dataset_id` = optional, `Dataset.id` of the dataset which will have the observation
* `variable_input` = a boolean parameter, set to `True` if the entered observation is a variable (constant) id for the node instead of an observed value.

A node can have only one observation. This observation can be hard evidence (single state or a single value) or soft evidence (multiple states or multiple values with their weights). If the node has variable(s), the variable value(s) can also be observed. Each variable of the node can have an observation. If the node with variables has observations for its variables and an observation for itself, the variable observations will be ignored and only the node observation will be used in calculations.

### 4.1.11 `remove_observation(network_id, node_id, dataset_id = optional)`

A method to remove a specific observation from the model. Calls the dataset method `remove_observation` in the background. It requires the id of the node which has the observation to be removed and the id of the network the node belongs to.

### 4.1.12 `clear_dataset_observations(dataset_id)`

A method to clear all observations in a specific dataset (case) in the model. Calls the dataset method `clear_all_observations()` in the background.

### 4.1.13 `clear_all_observations()`

A method to clear all observations defined in a model. This function removes all observations from all datasets (cases).

### 4.1.14 `change_settings(setting arguments)`

A method to change model settings. The input parameters can be some or all of the `settings` fields. For example:

```python
example_model.change_settings(convergence=0.001, iterations=75)
```

### 4.1.15 `default_settings()`

A method to reset model settings back to default values. The default values for model settings are:

* discreteTails = False
* sampleSizeRanked = 5
* convergence = 0.01
* iterations = 50
* tolerance = 1

### 4.1.16 `save_to_file(filename)`

A method to export the `Model` to a .cmpx or a .json file. This method passes on all the information about the model, its datasets, its networks, their nodes, and model settings to a file in the correct format readable by agena.ai.

Input parameter `filename` must have a file extension of '.cmpx' or '.json'.

### 4.1.17 `import_data(filename)`

A method to import dataset observations to the model using either a .csv or a .json input file. See examples below for the correct format of the input .csv and .json for data import.

### 4.1.18 `export_data(filename, dataset_ids = optional, include_inputs = False, include_outputs = True)`

A method to export dataset results or observations from the model to an either .csv or .json output file. See examples below for the format of the output .csv and .json files.

### 4.1.19 `from_cmpx(filepath = "/path/to/model/file.cmpx")`

This is the class method to create a `Model` object from a .cmpx file. The method parses the .cmpx file and creates the python objects based on the model in the file. To see its use, see examples below.

### 4.1.20 `create_csv_template()`

This method creates an empty CSV file for the model with the correct format so that it can be filled in and used for `import_data(filename)`. Note that this template includes every single node in the model, not all of which might be observable - you can delete the columns of the nodes which will not be observed. See examples below for its format and use.

### 4.1.21 `create_sensitivity_config(...)`

A method to create a sensitivity configuration object if a sensitivity analysis request will be sent to agena.ai Cloud servers or the local API. Its parameters are:

* `targetNode` = target node ID for the analysis
* `sensitivityNodes` = a list of sensitivity node IDs
* (optional) `network` = ID of the network to perform analysis on. If missing, the first network in the model is used
* (optional) `dataSet` = ID of the dataSet (scenario) to use for analysis
* (optional) `report_settings` = a dictionary for settings for the sensitivity analysis report. The elements of the dictionary are:
    * `"summaryStats"` (a list with some/all of the following fields)
        * mean
        * median
        * variance
        * standardDeviation
        * upperPercentile
        * lowerPercentile
    * `"sumsLowerPercentileValue"` (set the reported lower percentile value.
Default is 25)
    * `"sumsUpperPercentileValue"` (set the reported upper percentile value.
Default is 75)
    * `"sensLowerPercentileValue"` (lower percentile value to limit sensitivity node data by. Default is 0)
    * `"sensUpperPercentileValue"` (upper percentile value to limit sensitivity node data by. Default is 100)

For the use of the function, see Sections [8](#8-agenaai-cloud-with-pyagena) and [9](#9-local-agenaai-api-with-pyagena).

## 4.2 `Network` methods

A `Network` object is created in a `Model` object. Once a network of a model is created, the following methods can be used to modify and manipulate the object and its nodes.

### 4.2.1 `create_node(id, name=optional, description=optional, type=optional, simulated=optional, states=optional)`

A method to create a new `Node` object in a `Network` and add the new node to the `nodes` field of the `Network` object. The `id` of the new node is mandatory whereas `name`, `description`, `type`, `simulated`, and `states` are optional arguments. A new `Node` object is created with the arguments and added to the network if the network does not already have a node with the given id.

### 4.2.2 `remove_node(node_id)`

A method to remove an existing `Node` object from the network. Note that removing a Node from a network doesn't automatically remove it from its previous parent-child relationships in the network. You need to adjust such relationships separately on `Node` level.

### 4.2.3 `get_node(node_id)`

A method to access a `Node` object in a `Network`. The `Node` can be assigned to a variable and used.

### 4.2.4 `create_edge(child_id, parent_id)`

A method to add a parent `Node` to another `Node` in a `Network`. It takes node ids as parameters and runs the `add_parent()` method of a node.

### 4.2.5 `remove_edge(child_id, parent_id)`

A method to remove a parent `Node` from another `Node` in a `Network`. It takes node ids as parameters and runs the `remove_parent()` method of a node.

### 4.2.6 `set_node_probabilities(node_id, new_probs, by_row=False)`

A method to modify `Node`s in a `Network`. On the defined `Node` with the `node_id` parameter, it calls the node method `set_probabilities()`.

### 4.2.7 `set_node_states`

A method to modify `Node`s in a `Network`. On the defined `Node` with the `node_id` parameter, it calls the node method `set_states()`.

### 4.2.8 `set_node_expressions`

A method to modify `Node`s in a `Network`. On the defined `Node` with the `node_id` parameter, it calls the node method `set_expressions()`.

### 4.2.9 `set_node_variable`

A method to modify `Node`s in a `Network`. On the defined `Node` with the `node_id` parameter, it calls the node method `set_variable()`.

### 4.2.10 `set_node_distr_type`

A method to modify `Node`s in a `Network`. On the defined `Node` with the `node_id` parameter, it calls the node method `set_distr_type()`.

### 4.2.11 `plot()`

A method to plot the graphical structure of a BN network.

## 4.3 `Node` methods

`Node` objects are created in a `Network` object (see above). Once the nodes of a network are created, it is possible to modify them.

Some `Node` fields can be modified with a direct access to the field. For example, to update the name or a description information of a `Node`, simply use:

```python
example_node.name = "new node name"
```

or

```python
example_node.description = "new node description"
```

Because changing the name or description of a `Node` does not cause any compatibility issues. However, some fields such as table type or parents will have implications for other fields. Changing the node parents will change the size of its NPT, changing the node's table type from "Manual" to "Expression" will mean the state probabilities are now defined in a different way. Therefore, to modify such fields of a `Node`, use the corresponding method described below. These methods will ensure all the sensible adjustments are made when a field of a `Node` has been changed. To modify the nodes, you can either use the node methods described in this section, or the network methods described in the next section which allow you to modify nodes in the network. Once a network and its nodes are created, it is recommended to use the network methods to modify its nodes.

If you are directly working with `Node` objects, these are the methods `Node` objects can call for various purposes with their input parameters shown in parantheses:

### 4.3.1 `set_states(states)`

The method to update the states of the `Node` object. The node states can be defined upon creation if the node is a discrete node. If states are not specified during creation, sensible defaults will be assigned based on the node type. With `set_states()` it is possible to update node states later on. If the number of new states given with this method is the same as the previous number of node states, state names will be updated. If `set_states()` changes the number of node states, node probability table size will be adjusted accordingly and probability values will reset to uniform.

### 4.3.2 `add_parent(new_parent)`

The method to add a new parent to a node. Equivalent of adding an arc between two nodes on agena.ai Modeller. The input parameter `new_parent` is another `Node` object. If `new_parent` is already a parent for the node, the function does not update the `parents` field of the node.

When a new parent is added to a node, its NPT values and expressions are reset/resized accordingly. 

### 4.3.3 `remove_parent(old_parent)` 

The method to remove one of the existing parents of a node. Equivalent of removing the arc between two nodes on agena.ai Modeller. The input parameter `old_parent` is a `Node` object which has already been added to the `parents` field of the node.

When an existing parent is removed from a node, its NPT values and expressions are reset/resized accordingly.

### 4.3.4 `set_distr_type(new_distr_type)`

A method to set the table type (`distr_type`) of a node. If a `Node` is `simulated`, its table type can be "Expression" or "Partitioned" - the latter is only if the node has discrete parent nodes. If a `Node` is `not simulated`, its table type can be "Manual", "Expression", or "Partitioned (if the node has discrete parent nodes)".

Changing the node's distribution type (table type) adjusts its `states`/`probabilities`/`expressions`` parameters accordingly.

### 4.3.5 `set_probabilities(new_probs, by_row = False)`

The method to set the probability values if the table type (`distr_type`) of a `Node` is "Manual". `new_probs` is a list of lists containing numerical values, and the length of the input list depends on the number of the states of the node and of its parents.

You can format the input list in two different orders. If the parameter `by_row` is set to `True`, the method will read the input list to fill in the NPT row by row; if set to `False` (it is `False` by default), the method will read the input list to fill in the NPT column by columnn. This behaviour is illustrated with use case examples later in this document.

### 4.3.6 `set_expressions(new_expr, partition_parents = optional)`
The method to set the probability values if the table type (`distr_type`) of a `Node` is "Expression" or "Partitioned". If the table type is "Expression", `new_expr` is a list of size one and `partition_parents` is left untouched. If the table type is "Partitioned", `new_expr` is a list of expressions for each parent state, and `partition_parents` is a list of strings for each partitioned parent node's `id`. See the following sections for examples.

### 4.3.7 `set_variable(variable_name, variable_value)`

A method to set variables (constants) for a node. Takes the `variable_name` and `variable_value` inputs which define a new variable (constant) for the node.

### 4.3.8 `remove_variable(variable_name)`

A method to remove one of the existing variables (constants) from a node, using the `variable_name`.

## 4.4 `Dataset` Methods

The `Dataset` objects in the models hold the observations and calculation results. The datasets have the following methods.

### 4.4.1 `enter_observation(network_id, node_id, value, variable_name=optional)`

A method to enter an observation to a dataset. See Section [4.1.10](#4110-enter_observationnetwork_id-node_id-value-dataset_id--optional-variable_input--false) for details. Section 4.1.10 is the model method which calls this dataset method after using its parameter `dataset_id` to determine to which dataset the observation is entered. It is possible to use the dataset method directly to enter observations to the corresponding dataset.

### 4.4.2 `remove_observation(network_id, node_id)`

A method to remove an observation from a dataset. See Section [4.1.11](#4111-remove_observationnetwork_id-node_id-dataset_id--optional) for details. Section 4.1.11 is the model method which calls this dataset method after using its parameter `dataset_id` to determine from which dataset is the observation is removed.

### 4.4.3 `clear_all_observations()`

A method to remove all observations from a dataset. If the model method `clear_dataset_observations(dataset_id)` is used, it calls this dataset method. Not to be confused with the model method `clear_all_observations()` which removes all observations from all datasets in the model.

### 4.4.4 `get_result(network_id, node_id)`

If the dataset contains calculation results, you can access the results of a specific node in the model with this method.

## 4.5 agena.ai Cloud Related Functions

pyagena allows users to send their models to agena.ai Cloud servers for calculation. The functions around the server capabilities (including authentication) are described in [Section 8](#8-agenaai-cloud-with-pyagena).

## 4.6 agena.ai Local API Related Functions

pyagena allows users to connect to the local agena.ai developer API for calculation. The functions about the local developer API communication are descibed in [Section 9](#9-local-agenaai-api-with-pyagena).

Local API related functions come with the parameter `verbose` (set to False by default). If you'd like to see detailed console output, you can call the functions explained in [Section 9](#9-local-agenaai-api-with-pyagena) with `verbose = True`. Note that, for this to work, you also need to have enabled global `set_verbose(True)` setting.

# 5. Importing a Model from .cmpx

To import an existing agena.ai model (from a .cmpx file), create a new `Model` object using the `from_cmpx()` option:

```python
new_model = Model.from_cmpx("/path/to/model/file.cmpx")
```

This creates a python `Model` object with all the information taken from the .cmpx file. All fields and sub-fields of the `Model` object are accessible now. For example, you can see the networks in this model with:

```python
new_model.networks
```

Each network in the model is a `Network` object, therefore you can access its fields with the same logic, for example to see the id of the first network and all the nodes in the first network in the BN, use respectively:

```python
new_model.networks[0].id
```

```python
new_model.networks[0].nodes
```

Alternatively you can assign the networks to variables with the get method for further use:

```python
net = new_model.get_network(network_id) #assign desired network to a variable
net.nodes #display fields of the selected network
```

Similarly, each node in a network itself is a `Node` object. You can display all the fields of a node. Example uses for the second node in the first network of a model:

```python
new_model.networks[0].nodes[0].id
```

```python
new_model.networks[0].nodes[1].states
```

Like the networks, you can assign nodes to variables with the get method for further use:

```python
node_one = net.get_node(node_id) #assign desired node to a variable
node_one.states #display fields of the selected node

```
Once the python model is created from the imported .cmpx file, the `Model` object as well as all of its `Network`, `Dataset`, and `Node` objects can be manipulated using python methods.

# 6. Creating and Modifying a Model in python

It is possible to create an agena.ai model entirely in python, without a .cmpx file to begin with. Once all the networks and nodes of a model are created and defined in python, you can export the model to a .cmpx or .json file to be used with agena.ai calculations and inference, locally or on agena.ai Cloud. In this section, creating a model is shown step by step.

## 6.1 Creating a Model

You can create an empty model and modify its networks and nodes as described below. To create a new model object:

```python
example_model = Model()
```

which will initiate a new model object with no networks and one default empty dataset (Case 1).

## 6.2 Creating Networks

You can create network(s) in the model with the `create_network()` method of a model. For example:

```python
example_net = example_model.create_network(id="example_net")
```

which will create a new network in the model with no nodes. Note that if you assign the creation method to a variable, the network variable is usable directly in addition to the model now including the new network.

## 6.3 Creating Nodes

You can create nodes in a network with the `create_node()` method of a network. 

It is recommended to assign the network in a model to a new variable for the ease of future operations. If not assigned upon creation, you can use `get_network()`, for example:

```python
example_net = example_model.get_network("example_net")
```

Now we can use the variable `net` to create and modify nodes in it:

```python
node_one = net.create_node(id="node_one")
```

Similar no network creation, you can assign the newly created node to its own variable at the time of creation.

If the optional fields are not specified, the nodes will be created with the defaults. The default values for the fields, if they are not specified, are:

* name = node.id
* description = "New Node"
* simulated = False
* type = 
    * if simulated: "ContinuousInterval"
    * if not simulated: "Boolean"
* states =
    * if Boolean or Labelled: ["False", "True"]
    * if Ranked: ["Low", "Medium", "High"]
    * if DiscreteReal: ["0.0", "1.0"]

Once a new node is created, depending on the type and number of states, other fields are given sensible default values too. These fields are distr_type (table type), probabilities or expressions. To specify values in these fields, you need to use the relevant set functions (described above and explained later). The default values for these fields are:

* distr_type = 
    * if simulated: "Expression"
    * if not simulated: "Manual"
* probabilities = 
    * if distr_type is Manual: discrete uniform distribution, each state has the probability of (1/number of states)
* expressions = 
    * if distr_type is Expression: "Normal(0,1000000)"

Look at the following new node creation examples:

```python
net.create_node(id = "node_one")
```

```python
net.create_node(id = "node_two", name = "Second Node")
```

```python
net.create_node(id = "node_three", type = "Ranked")
```

```python
net.create_node(id = "node_four", type = "Ranked", states = ["Very low", "Low", "Medium", "High", "Very high"])
```

Looking up some example values in the fields that define these nodes:

* node_one.id = "node_one"
* node_one.name = "node_one"
* node_one.description = "New Node"
* node_one.type = "Boolean"
* node_one.states = ["False", "True"]
* node_two.id = "node_two"
* node_two.name = "Second Node"
* node_three.type = "Ranked"
* node_three.states =  ["Low", "Medium", "High"]
* node_four.states =  ["Very low", "Low", "Medium", "High", "Very high"]
* node_one.distr_type = "Manual"
* node_one.probabilities = [[0.5, 0.5]]
* node_three.probabilities = [[0.3333, 0.3333, 0.3333]]
* node_four.probabilities = [[0.2, 0.2, 0.2, 0.2, 0.2]]

Note that probabilities will be a list (of size one) of lists even when there is no parent node.

## 6.4 Modifying Nodes

To modify nodes they can either be obtained with the network method `get_node()` and assigned to a new variable, or preferably the network methods which allow the node modification can be used. For the former, as an example:

```python
node_one = net.get_node("node_one")
```

and now you can use the variable `node_one` to modify the node in the network.

To update node information, some fields can be simply overwritten with direct access to the field if it does not affect other fields. These fields are node `name` and `description`. For example:

```python
node_one.description = "first node we have created"
```

Other fields can be specified with the relevant set functions. For these, the node methods can directly be used, or preferably the network methods which allow the node modification can be used. When working on a network, it is recommended to use the network methods to modify its nodes, but below both alternatives are shown with examples.

To update the node states, you can use `set_states()` on a node:

```python
node_one.set_states(["Negative","Positive"])
```

or `set_node_states()` on a network:

```python
net.set_node_states("node_one", states = ["Negative", "Positive"])
```

Note that the input parameter for `set_states()` or `set_node_states()` is a list of node names. If this method changes the number of node states, the NPT will be adjusted accordingly and state probabilities will reset to uniform.

To set probability values for a node with a manual table (distr_type), you can use `set_probabilities()` on a node:

```python
node_one.set_probabilities([[0.2,0.8]])
```

or `set_node_probabilities()` on a network:

```python
net.set_node_probabilities("node_one", probabilities = [[0.2,0.8]])
```

Note that the `set_probabilities()` or `set_node_probabilities()` take **a list of lists** as input, even when the node has no parents and its NPT has only one row of probabilities. If the node has parents, the NPT will have multiple rows which should be in the input list.

Assume that `node_one` and `node_two` are the parents of `node_three` (how to add parent nodes is illustrated later in this section). Now assume that you want `node_three` to have the following NPT:

<table>
<tbody>
  <tr>
    <td><strong>node_one</strong></td>
    <td colspan="2"><strong>Negative</strong></td>
    <td colspan="2"><strong>Positive</strong></td>
  </tr>
  <tr>
    <td><strong>node_two</strong></td>
    <td><