Repository: HLR/DomiKnowS (language: python)

Issue:
Update README.md



Discussion / hints:

Fix patch:
diff --git a/.gitignore b/.gitignore
index af92530cd..35d239730 100644
--- a/.gitignore
+++ b/.gitignore
@@ -246,3 +246,10 @@ data/MNIST/raw/train-labels-idx1-ubyte.gz
 ### example folder
 /examples/
 /examples
+.vscode/settings.json
+domiknows/program/model/gbi copy.py
+
+# venv
+venv
+.externalToolBuilders/org.eclipse.jdt.core.javabuilder.launch
+.settings/org.eclipse.ltk.core.refactoring.prefs
diff --git a/.vscode/launch.json b/.vscode/launch.json
index dee191fe3..a1d1f68a7 100644
--- a/.vscode/launch.json
+++ b/.vscode/launch.json
@@ -19,7 +19,8 @@
             "request": "launch",
             "program": "${file}",
             "cwd": "${fileDirname}",
-            "console": "integratedTerminal"
+            "console": "integratedTerminal",
+            "justMyCode": false
         }
     ]
 }
\ No newline at end of file
diff --git a/.vscode/settings.json b/.vscode/settings.json
index 495aff81c..50c7e1cce 100644
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -21,6 +21,7 @@
         "logsumexp",
         "lossprogram",
         "mloss",
+        "nand",
         "nconstr",
         "Optim",
         "primaldual",
diff --git a/Docs b/Docs
deleted file mode 160000
index b4f631aab..000000000
--- a/Docs
+++ /dev/null
@@ -1,1 +0,0 @@
-Subproject commit b4f631aab1e7398b8ccf0ca347209a9dbbabe7c2
diff --git a/README.md b/README.md
index b2cff78b8..466d1d2ea 100644
--- a/README.md
+++ b/README.md
@@ -5,19 +5,29 @@ DomiKnowS is a Python library that facilitates the integration of domain knowled
 While several approaches for integrating symbolic and sub-symbolic models have been introduced, no generic library facilitates programming for such integration with various underlying algorithms. DomiKnowS aims to simplify the programming for knowledge integration in training and inference phases while separating the knowledge representation from learning algorithms.
 
 
-- [Getting Started](https://github.com/HLR/DomiKnowS/blob/c457c31bc0196c18748813f4ec444c3fea0c24a8/Getting%20Started.md): Provides detailed instructions on how to get started with DomiKnowS, including installation, setting up the environment, and basic usage.
-- [Documentation](https://github.com/HLR/DomiKnowS/tree/c457c31bc0196c18748813f4ec444c3fea0c24a8): Provides comprehensive documentation on the DomiKnowS, including classes, methods, and their usage.
-  - Contribute to DomiKnowS: Report Issues [see here](https://github.com/HLR/DomiKnowS/blob/Doc/Issue%20Report.md), share your suggestions [see here](https://github.com/HLR/DomiKnowS/blob/Doc/Suggestions.md) and create a pull request [see here](https://github.com/HLR/DomiKnowS/blob/Doc/Pull%20Request.md).
-- [Walkthrough Example](https://github.com/HLR/DomiKnowS/tree/c457c31bc0196c18748813f4ec444c3fea0c24a8/Walkthrough%20Examples): Contains examples that demonstrate the usage of DomiKnowS for various tasks, such as image classification, sequence modeling, and reinforcement learning. ( For more examples see [Examples Branch](https://github.com/HLR/DomiKnowS/tree/Tasks) )
-- [FAQ](https://github.com/HLR/DomiKnowS/blob/Doc/FAQ.md): Read our FAQ file if you have any questions.
-- [License](https://github.com/HLR/DomiKnowS/blob/c457c31bc0196c18748813f4ec444c3fea0c24a8/LICENSE.md): Contains information about the license of DomiKnowS and its terms of use.
+
+## Contents
+
+- [Getting Started](https://github.com/HLR/DomiKnowS/blob/Doc/Getting%20Started.md): Provides detailed instructions on how to get started with DomiKnowS, including installation, setting up the environment, and basic usage.
+- [Example Tasks](https://github.com/HLR/DomiKnowS/blob/Doc/Getting%20Started.md): Contains examples that demonstrate the usage of DomiKnowS for various tasks, such as image classification, sequence modeling, and reinforcement learning. ( For more example see [Examples Branch](https://github.com/HLR/DomiKnowS/tree/Tasks) )
+- [Documentation](https://github.com/HLR/DomiKnowS/tree/Doc): Provides comprehensive documentation on the DomiKnowS, including classes, methods, and their usage.
+- [Contributing](https://github.com/HLR/DomiKnowS/blob/Doc/Suggestions.md): Explains how you can contribute to the development of DomiKnowS, including reporting issues, suggesting enhancements, and submitting pull requests.
+- [License](https://github.com/HLR/DomiKnowS/blob/Doc/LICENSE.md): Contains information about the license of DomiKnowS and its terms of use.
 - [DomiKnowS Website](https://hlr.github.io/domiknows-nlp/): Contains documentation, example links, and an introductory video to DomiKnowS
 
+
+## Quick Start
+
+1. Install DomiKnowS using `pip install DomiKnowS`.
+2. Install Gurobi following the instructions [here](https://github.com/HLR/DomiKnowS/blob/develop/GurobiREADME.md).
+3. Refer to the [Getting Started](https://github.com/HLR/DomiKnowS/blob/Doc/Getting%20Started.md) documentation for detailed instructions on how to define graph declarations, model declarations, initialize programs, and compose and execute programs using DomiKnowS.
+
 ## Publications
 
 - [DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning](https://arxiv.org/abs/2108.12370)
 - [GLUECons: A Generic Benchmark for Learning Under Constraints](https://arxiv.org/abs/2302.10914)
 
+
 ## Acknowledgements
 
 DomiKnowS is developed and maintained by [HLR](https://hlr.github.io/). We would like to acknowledge the contributions of the open-source community and express our gratitude to the developers of Gurobi for their excellent optimization solver.
diff --git a/Tasks b/Tasks
deleted file mode 160000
index 44b3d0b24..000000000
--- a/Tasks
+++ /dev/null
@@ -1,1 +0,0 @@
-Subproject commit 44b3d0b24f0f7f940e95fbee0ddd318bf706a976
diff --git a/domiknows/__init__.py b/domiknows/__init__.py
index fc7d7175a..2c8f8690e 100644
--- a/domiknows/__init__.py
+++ b/domiknows/__init__.py
@@ -1,1 +1,1 @@
-from .utils import setProductionLogMode, getRegrTimer_logger, getProductionModeStatus, getReuseModel, setDnSkeletonMode, getDnSkeletonMode
\ No newline at end of file
+from .utils import setProductionLogMode, getRegrTimer_logger, getProductionModeStatus, getReuseModel, setDnSkeletonMode, getDnSkeletonMode, getDnSkeletonModeFull
\ No newline at end of file
diff --git a/domiknows/base.py b/domiknows/base.py
index bdbee1fb2..250f8faca 100644
--- a/domiknows/base.py
+++ b/domiknows/base.py
@@ -269,6 +269,9 @@ def attach(self, sub, name=None):
         if isinstance(sub, NamedTreeNode):
             if sub.sup is not None:
                 sub.sup.detach(sub)
+            # if sub.name in set(self.concepts.keys()):
+            #     raise ValueError('Concept name {} already exists in {}.'.format(sub.name, self.name))
+            # else:
             sub._sup = self # FIXME: using private method
         if isinstance(sub, Named):
             if name is None:
diff --git a/domiknows/data/allennlp/__init__.py b/domiknows/data/allennlp/__init__.py
deleted file mode 100644
index e69de29bb..000000000
diff --git a/domiknows/data/allennlp/reader.py b/domiknows/data/allennlp/reader.py
deleted file mode 100644
index 7fe5c7072..000000000
--- a/domiknows/data/allennlp/reader.py
+++ /dev/null
@@ -1,176 +0,0 @@
-from typing import Iterator
-from collections import OrderedDict, defaultdict
-import uuid
-from six import with_metaclass
-from allennlp.data.dataset_readers import DatasetReader
-from allennlp.data.fields import MetadataField
-from allennlp.data import Instance
-import cls
-from ...utils import optional_arg_decorator, optional_arg_decorator_for
-
-
-PLURAL_SUFFIX = '()'
-
-
-class SensableReaderMeta(cls.ClsMeta):
-    def __init__(cls, name, bases, namespace):
-        cls.tokens_dict = OrderedDict()
-        cls.textfield_dict = OrderedDict()
-        cls.field_dict = OrderedDict()
-        super(SensableReaderMeta, cls).__init__(name, bases, namespace)
-
-
-class SensableReader(with_metaclass(SensableReaderMeta, DatasetReader)):
-    def __init__(self, lazy=False) -> None:
-        super(SensableReader, self).__init__(lazy=lazy)
-        self.key_sensors = defaultdict(list)
-        self.key_tokens = {}
-        self.token_indexers = defaultdict(dict)
-
-    def _to_instance(self, raw_sample, metas=None) -> Instance:
-        cls = type(self)
-        fields = {}
-        if metas is not None:
-
-            for key, value in metas.items():
-                fields[key] = MetadataField(value)
-
-        # prepare tokens
-        for key, update_tokens in cls.tokens_dict.items():
-            update_tokens(self, fields, raw_sample)
-
-        # prepare `TextField`s
-        for key, update_textfield in cls.textfield_dict.items():
-            update_textfield(self, fields, raw_sample)
-
-        # prepare other field
-        for key, update_field in cls.field_dict.items():
-            update_field(self, fields, raw_sample)
-
-        return Instance(fields)
-
-    def _read(self, file_path, metas=None) -> Iterator[Instance]:
-        for raw_sample in self.raw_read(file_path):
-
-            yield self._to_instance(raw_sample, metas)
-
-    def read(self, file_path, metas=None) -> Iterator[Instance]:
-        from copy import copy
-        sub = copy(self)
-        sub._read = lambda file_path: self._read(file_path, metas=metas)
-        return DatasetReader.read(sub, file_path)
-
-    @cls
-    def field(cls, key):
-        if key.endswith(PLURAL_SUFFIX):
-            raise ValueError(
-                'Data field key cannot end with "{}"'.format(PLURAL_SUFFIX))
-
-        def decorator(func):
-            def update_field(self_, fields, raw_sample):
-                field = func(self_, fields, raw_sample)
-                if field is not None:
-                    for sensor in self_.key_sensors[key]:
-                        fields[sensor.fullname] = field
-                return fields
-            cls.field_dict[key] = update_field
-            return update_field
-        return decorator
-
-    @cls
-    def tokens(cls, key):
-        if key.endswith(PLURAL_SUFFIX):
-            raise ValueError(
-                'Data field key cannot end with "{}"'.format(PLURAL_SUFFIX))
-
-        def decorator(func):
-            def update_tokens(self_, fields, raw_sample):
-                tokens = func(self_, fields, raw_sample)
-                if tokens is not None:
-                    self_.key_tokens[key] = tokens
-                    # an additional metafield for the token list
-                    for sensor in self_.key_sensors[key]:
-                        fields[sensor.fullname] = MetadataField(tokens)
-                return tokens
-            cls.tokens_dict[key] = update_tokens
-            return update_tokens
-        return decorator
-
-    @cls
-    def textfield(cls, key):
-        if key.endswith(PLURAL_SUFFIX):
-            raise ValueError(
-                'Data field name cannot end with "{}"'.format(PLURAL_SUFFIX))
-
-        def decorator(func):
-            def update_textfield(self_, fields, raw_sample):
-                for sensor in self_.key_sensors[key]: # sensors associate with 'pos_tag'
-                    # it would be possible different sensor refers to different tokens
-                    tokens = self_.key_tokens[sensor.tokens_key]
-                    field = func(self_, fields, tokens)
-                    if field is not None:
-                        fields[sensor.fullname] = field
-
-                return fields
-            cls.textfield_dict[key] = update_textfield
-
-            return update_textfield
-        return decorator
-
-    @cls(True)
-    def fields(cls):
-        def decorator(func):
-            def update_field(self_, fields, raw_sample):
-                for key, field in func(self_, fields, raw_sample):
-                    if field is not None:
-                        for sensor in self_.key_sensors[key]:
-                            fields[sensor.fullname] = field
-                return fields
-            cls.field_dict[func.__name__ + PLURAL_SUFFIX] = update_field
-            return update_field
-        return decorator
-
-    def claim(self, key, sensor):
-        self.key_sensors[key].append(sensor)
-
-    def get_fieldname(self, key):
-        return self.key_sensors[key][0].fullname # using any[0]?
-
-
-@optional_arg_decorator_for(lambda cls: issubclass(cls, SensableReader))
-def keep_keys(cls, *keys, exclude=[]):
-    tokens_dict = OrderedDict()
-    textfield_dict = OrderedDict()
-    field_dict = OrderedDict()
-
-    # reversed order ensure MRO priority by override
-    for base in reversed(cls.__bases__):
-        if issubclass(base, SensableReader):
-            if len(keys) == 0:
-                tokens_dict.update({k:v for k, v in base.tokens_dict.items() if k not in exclude})
-                textfield_dict.update({k:v for k, v in base.textfield_dict.items() if k not in exclude})
-                field_dict.update({k:v for k, v in base.field_dict.items() if k not in exclude})
-            else:
-                for key in keys:
-                    if key in exclude:
-                        continue
-                    if key in base.tokens_dict:
-                        tokens_dict[key] = base.tokens_dict[key]
-                    if key in base.textfield_dict:
-                        textfield_dict[key] = base.textfield_dict[key]
-                    if key in base.field_dict:
-                        field_dict[key] = base.field_dict[key]
-    for key in keys:
-        if (key not in tokens_dict) and (key not in textfield_dict) and (key not in field_dict) and (key not in exclude):
-            raise ValueError('Cannot find key {} from any of the bases of {}'.format(key, cls.__name__))
-
-    # update with current lastly
-    tokens_dict.update(cls.tokens_dict)
-    textfield_dict.update(cls.textfield_dict)
-    field_dict.update(cls.field_dict)
-
-    cls.tokens_dict = tokens_dict
-    cls.textfield_dict = textfield_dict
-    cls.field_dict = field_dict
-
-    return cls
diff --git a/domiknows/data/reader.py b/domiknows/data/reader.py
index adb2ee4f5..97b094663 100644
--- a/domiknows/data/reader.py
+++ b/domiknows/data/reader.py
@@ -3,8 +3,9 @@
 
 
 class RegrReader:
-    def __init__(self, file, type="json"):
+    def __init__(self, file, type="json", prefix=""):
         self.file = file
+        self.prefix = prefix
         if type == "json":
             with open(file, 'r') as myfile:
                 data = myfile.read()
diff --git a/domiknows/graph/__init__.py b/domiknows/graph/__init__.py
index cee509af3..49fa0aba1 100644
--- a/domiknows/graph/__init__.py
+++ b/domiknows/graph/__init__.py
@@ -9,4 +9,5 @@
 from .candidates import CandidateSelection, combinationC
 from .property import Property
 from .trial import Trial
-from .dataNode import DataNode, DataNodeBuilder
\ No newline at end of file
+from .dataNode import DataNode, DataNodeBuilder
+from .dataNodeDummy import createDummyDataNode, satisfactionReportOfConstraints
\ No newline at end of file
diff --git a/domiknows/graph/allennlp/__init__.py b/domiknows/graph/allennlp/__init__.py
deleted file mode 100644
index cf50be8ce..000000000
--- a/domiknows/graph/allennlp/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-from .base import AllenNlpGraph
\ No newline at end of file
diff --git a/domiknows/graph/allennlp/base.py b/domiknows/graph/allennlp/base.py
deleted file mode 100644
index 6130c7697..000000000
--- a/domiknows/graph/allennlp/base.py
+++ /dev/null
@@ -1,439 +0,0 @@
-import os
-import logging
-import pickle
-from glob import glob
-from collections import defaultdict
-from itertools import chain
-from typing import Dict, List, Tuple, Union
-import torch
-import numpy as np
-import pandas as pd
-from allennlp.data.vocabulary import Vocabulary
-from allennlp.data.iterators import BucketIterator
-from allennlp.common.params import Params
-from allennlp.training import Trainer
-from allennlp.training.optimizers import Optimizer
-from allennlp.training.learning_rate_schedulers import LearningRateScheduler
-from allennlp.nn.util import device_mapping
-from ..trial import Trial
-from ...utils import WrapperMetaClass
-from ...solver import ilpOntSolverFactory
-from ...solver.ilpOntSolverFactory import ilpOntSolverFactory
-from ...solver.ilpOntSolver import ilpOntSolver
-from ...solver.allennlpInferenceSolver import AllennlpInferenceSolver
-from ...solver.allennlplogInferenceSolver import AllennlplogInferenceSolver
-from ...sensor.allennlp.base import ReaderSensor
-from ...sensor.allennlp.learner import SentenceEmbedderLearner
-from ...sensor.allennlp.sensor import SentenceEmbedderSensor
-from .. import Graph, Property
-from ..dataNode import DataNode
-from .model import GraphModel
-from .utils import evaluate
-
-
-DEBUG_TRAINING = 'REGR_DEBUG' in os.environ and os.environ['REGR_DEBUG']
-
-
-class AllenNlpGraph(Graph, metaclass=WrapperMetaClass):
-    __metaclass__ = WrapperMetaClass
-
-    def __init__(
-        self,
-        *args,
-        log_solver=False,
-        **kwargs
-    ):
-        vocab = None # Vocabulary()
-        self.model = GraphModel(self, vocab, *args, **kwargs)
-        if log_solver:
-            self.solver = ilpOntSolverFactory.getOntSolverInstance(self, AllennlplogInferenceSolver)
-        else:
-            self.solver = ilpOntSolverFactory.getOntSolverInstance(self, AllennlpInferenceSolver)
-        self.solver_log_to(None)
-        # do not invoke super().__init__() here
-
-    def get_multiassign(self):
-        def func(node):
-            if isinstance(node, Property) and len(node) > 1:
-                return node
-            return None
-        return list(self.traversal_apply(func))
-
-    @property
-    def poi(self):
-        return self.get_multiassign()
-
-    def get_sensors(self, *tests):
-        def func(node):
-            if isinstance(node, Property):
-                yield from node.find(*tests)
-        return list(self.traversal_apply(func))
-
-    def solver_log_to(self, log_path:str=None):
-        #solver_logger = logging.getLogger(ilpOntSolver.__module__)
-        solver_logger = self.solver.myLogger
-        solver_logger.propagate = False
-        if DEBUG_TRAINING:
-            solver_logger.setLevel(logging.DEBUG)
-            param = {'precision': 4,
-                     'threshold': np.inf,
-                     'edgeitems': 3,
-                     'linewidth': np.inf
-                    }
-            torch.set_printoptions(**param)
-            np.set_printoptions(**param)
-            pd.options.display.max_rows = None
-            pd.options.display.max_columns = None
-        else:
-            solver_logger.setLevel(logging.INFO)
-        solver_logger.handlers = []
-        if log_path is not None:
-            if DEBUG_TRAINING:
-                handler = logging.FileHandler(log_path)
-            else:
-                logFilesize = 5 * 1024 * 1024 * 1024 # 5G
-                logBackupCount = 4
-                logFileMode = 'a'
-                handler = logging.handlers.RotatingFileHandler(log_path,
-                                                               mode=logFileMode,
-                                                               maxBytes=logFilesize,
-                                                               backupCount=logBackupCount,
-                                                               encoding=None,
-                                                               delay=0)
-            solver_logger.addHandler(handler)
-
-    def save(self, path, **objects):
-        if not os.path.exists(path):
-            os.makedirs(path)
-        with open(os.path.join(path, 'model.th'), 'wb') as fout:
-            torch.save(self.model.state_dict(), fout)
-        self.model.vocab.save_to_files(os.path.join(path, 'vocab'))
-        for k, v in objects.items():
-            with open(os.path.join(path, k + '.pkl'), 'wb') as fout:
-                pickle.dump(v, fout)
-
-    def load(self, path, model:Union[str, int]='default', vocab=None):
-        model_file = None
-        if isinstance(model, int):
-            model_file = 'model_state_epoch_{}'.format(model)
-        elif isinstance(model, str):
-            if model.endswith('.th'):
-                model_file = model
-            elif model == 'best':
-                model_file = 'best.th'
-            elif model == 'last':
-                model_file = sorted(glob(os.path.join(path, "model_state_epoch_*.th")), key=lambda p: os.path.getmtime(p)).pop()
-            elif model == 'default':
-                model_file = 'model.th'
-        if model_file is None:
-            raise ValueError(('`model` in `load()` must be one of the following: '
-                              'an integer for epoch number, '
-                              'a string name ends with ".th", '
-                              'the string "best" for best model, '
-                              'the string "last" for last saved model, '
-                              'or the string "default" for just "model.th".'))
-        vocab = vocab or os.path.join(path, 'vocab')
-
-        if torch.cuda.is_available() and not DEBUG_TRAINING:
-            device = 0
-            self.model.cuda()
-        else:
-            device = -1
-
-        #print('Loading vocab from {}'.format(vocab))
-        self.model.vocab = Vocabulary.from_files(vocab)
-        self.model.extend_embedder_vocab()
-        #print('Loading model from {}'.format(model_file))
-        with open(model_file, 'rb') as fin:
-            self.model.load_state_dict(torch.load(fin, map_location=device_mapping(device)))
-
-    @property
-    def reader(self):
-        readers = set(self.get_sensors(ReaderSensor))
-        assert len(readers) == 1 # consider only 1 reader now
-        return readers.pop()
-
-    def train(self, data_config, train_config):
-        reader = self.reader
-        train_dataset = reader.read(os.path.join(data_config.relative_path, data_config.train_path), metas={'dataset_type':'train'})
-        valid_dataset = reader.read(os.path.join(data_config.relative_path, data_config.valid_path), metas={'dataset_type':'valid'})
-        self.update_vocab_from_instances(train_dataset + valid_dataset, train_config.pretrained_files)
-
-        # prepare optimizer
-        #print({n: p.size() for n, p in self.model.named_parameters()})
-        optimizer = Optimizer.from_params(self.model.named_parameters(), Params(train_config.optimizer))
-
-        # prepare scheduler
-        if 'scheduler' in train_config:
-            scheduler = LearningRateScheduler.from_params(optimizer, Params(train_config.scheduler))
-        else:
-            scheduler = None
-
-        # prepare iterator
-        sorting_keys = [(sensor.fullname, 'num_tokens') for sensor in self.get_sensors(SentenceEmbedderSensor)]
-        iterator = BucketIterator(sorting_keys=sorting_keys,
-                                  track_epoch=True,
-                                  **train_config.iterator)
-        iterator.index_with(self.model.vocab)
-
-        # prepare model
-        training_state = self.model.training
-        self.model.train()
-
-        trainer = self.get_trainer(train_dataset, valid_dataset,
-                                   optimizer=optimizer,
-                                   learning_rate_scheduler=scheduler,
-                                   iterator=iterator,
-                                   **train_config.trainer)
-
-        if train_config.trainer.serialization_dir is not None:
-            self.solver_log_to(os.path.join(train_config.trainer.serialization_dir, 'solver.log'))
-
-        metrics = trainer.train()
-
-        # restore model
-        self.model.train(training_state)
-
-        return metrics
-
-    def get_trainer(
-        self,
-        train_dataset,
-        valid_dataset,
-        summary_interval=100,
-        histogram_interval=100,
-        should_log_parameter_statistics=True,
-        should_log_learning_rate=True,
-        **kwargs
-    ) -> Trainer:
-        # prepare GPU
-        if torch.cuda.is_available() and not DEBUG_TRAINING:
-            device = 0
-            self.model.cuda()
-        else:
-            device = -1
-
-        trainer = Trainer(model=self.model,
-                          train_dataset=train_dataset,
-                          validation_dataset=valid_dataset,
-                          shuffle=not DEBUG_TRAINING,
-                          cuda_device=device,
-                          summary_interval=summary_interval,
-                          histogram_interval=histogram_interval,
-                          should_log_parameter_statistics=should_log_parameter_statistics,
-                          should_log_learning_rate=should_log_learning_rate,
-                          **kwargs)
-
-        return trainer
-
-    def update_vocab_from_instances(self, instances, pretrained_files=None):
-        #import pdb; pdb.set_trace()
-        vocab = Vocabulary.from_instances(instances, pretrained_files=pretrained_files)
-        self.model.vocab = vocab
-        self.model.extend_embedder_vocab()
-
-    def test(self, data_path, log_path=None, batch=1, log_violation=False):
-        # prepare GPU
-        if torch.cuda.is_available() and not DEBUG_TRAINING:
-            device = 0
-            self.model.cuda()
-        else:
-            device = -1
-
-        reader = self.reader
-        dataset = reader.read(data_path, metas={'dataset_type':'test'})
-
-        # prepare iterator
-        sentence_sensors = self.get_sensors(SentenceEmbedderLearner)
-        sorting_keys = [(sensor.fullname, 'num_tokens') for sensor in sentence_sensors]
-        iterator = BucketIterator(batch_size=batch,
-                                  sorting_keys=sorting_keys,
-                                  track_epoch=True)
-        iterator.index_with(self.model.vocab)
-
-        # prepare model
-        training_state = self.model.training
-        self.model.eval()
-
-        self.solver_log_to(log_path)
-        metrics = evaluate(model=self.model,
-                           instances=dataset,
-                           data_iterator=iterator,
-                           cuda_device=device,
-                           batch_weight_key=None,
-                           log_violation=log_violation)
-
-        # restore model
-        self.model.train(training_state)
-
-        return metrics
-
-    def populate(self, data_item, root, *concepts, query=None, label_fn=None, data_dict=None):
-        trial = Trial()
-        if data_dict is None:
-            data_dict = defaultdict(list)
-
-        def peel(node_value, repeat_unpeelable=False):
-            # get a iterator for each instance in the fist level
-            # {a:[1,2,3], b:[[4.1,4.2],[5.1,5.2],[6.1,6.2]], c:{d:[7,8,9]}}
-            # ->
-            # [{a:1, b:[4.1,4.2], c{d:7}}, {a:2, b:[5.1,5.2], c{d:8}}, {a:3, b:[6.1,6.2], c{d:9}}]
-            if isinstance(node_value, (torch.Tensor, np.ndarray, List, Tuple)):
-                return iter(node_value)
-            elif isinstance(node_value, (Dict,)):
-                pvs = []
-                keys = []
-                for k, v in node_value.items():
-                    pv = peel(v, repeat_unpeelable)
-                    if pv:
-                        pvs.append(pv)
-                        keys.append(k)
-                return (dict(zip(keys, zpv)) for zpv in zip(*pvs))
-            # TODO: fields like "loss" fron allennlp might not be possible to be divided
-            # according to dim(/batch). Return None or return the identical one for all?
-            elif repeat_unpeelable:
-                return repeat(node_value)
-            return None
-
-        def contain(node, k):
-            return k.startswith(node.fullname)
-
-        def contained_by(node):
-            return lambda item: contain(node, item[0])
-
-        root_data = []
-        sub_data_item = {}
-        for k, v in filter(contained_by(self), data_item.items()):
-            node = self[k[len(self.fullname)+1:]]
-            if not isinstance(node, Property): # skip sensors and learners
-                continue
-            if contain(root, k):
-                if not root_data:
-                    for index, _ in enumerate(peel(v)):
-                        dataNode = DataNode(instanceID=len(root_data),
-                                            instanceValue=None,
-                                            ontologyNode=root)
-                        dataNode.index = index
-                        root_data.append(dataNode)
-                #import pdb; pdb.set_trace()
-                for dataNode, pv in zip(root_data, peel(v)):
-                    dataNode.__dict__[node.prop_name] = pv
-            else:
-                sub_data_item[k] = v
-        if label_fn:
-            for dataNode in root_data:
-                prob = label_fn(dataNode)
-                if prob is not None:
-                    trial[root, dataNode] = prob
-        data_dict[root].extend(root_data)
-        for rel in root.contains():
-            pass
-            #for dataNode in data_dict[concept]:
-            #    dataNode
-#             for rel in concept.is_a():
-#                 pass
-#             for rel in concept.has_a():
-#                 pass
-        return trial, data_dict
-
-#     @staticmethod
-#     def peel(data_item, index):
-#         # get a new data_item by selecting index for first dim
-#         def peel_one(node_value):
-#             # closure variable - index
-#             if isinstance(node_value, (torch.Tensor, List, Tuple)):
-#                 return node_value[index]
-#             elif isinstance(node_value, (Dict,)):
-#                 return {k, peel_one(v, index): for k, v: node_value.items()}
-#             # TODO: fields like "loss" fron allennlp might not be possible to be divided
-#             # according to dim(/batch). Return None or return the identical one for all?
-#             return None
-#         peeled = {}
-#         for k, v in data_item:
-#             v = peel_one(v)
-#             if v is not None:
-#                 peeled[k] = v
-#         return peeled
-
-#     @staticmethod
-#     def peeled(data_item):
-#         keys, values = zip(*data_item.items())
-#         zip(values)
-#         # get a new data_item by selecting index for first dim
-#         def peel_one(node_value):
-#             # closure variable - index
-#             if isinstance(node_value, (torch.Tensor, List, Tuple)):
-#                 return node_value[index]
-#             elif isinstance(node_value, (Dict,)):
-#                 return {k, peel_one(v, index): for k, v: node_value.items()}
-#             # TODO: fields like "loss" fron allennlp might not be possible to be divided
-#             # according to dim(/batch). Return None or return the identical one for all?
-#             return None
-#         peeled = {}
-#         for k, v in data_item:
-#             v = peel_one(v)
-#             if v is not None:
-#                 peeled[k] = v
-#         return peeled
-
-#     def populate(self, data_item, *concepts, batch=None, query=None, key='label'):
-#         from ..concept import Concept
-#         from ..property import Property
-#         from ..dataNode import DataNode
-
-#         trial = Trial()
-#         data_dict = defaultdict()
-
-#         def contained(item):
-#             k = item[0]
-#             return k.startswith(self.fullname)
-
-#         def extract_batch_value(node_value):
-#             if batch is None:
-#                 return node_value
-#             # closure variables - batch
-#             if isinstance(node_value, (torch.Tensor, List, Tuple)):
-#                 return node_value[batch]
-#             elif isinstance(node_value, (Dict,)):
-#                 return {k, extract_batch(v): for k, v: node_value.items()}
-#             return None # fields like "loss" fron allennlp might not be divided according to batch, or return the identical one for all?
-
-#         for node_key, node_value in filter(contained, data_item.items()):
-#             node = self[node_key]
-#             if isinstance(node, Property):
-#                 concept_node = node.sup
-#                 if concept_node not in data_dict: # TODO: limit to *concepts?
-#                     # create the nodes
-#                     # get rid of batch
-#                     node_value = extract_batch_value(node_value)
-#                     if node_value is None:
-#                         # skip if get none for batch
-#                         continue
-#                     data_dict[concept_node] = []
-#                     # TODO: estimate the number of argument
-#                     # FIXME: suppose 1 here, that means a single loop
-#                     for _ in node_value:
-#                         dataNode = DataNode(instanceID=len(data_dict[concept_node]),
-#                                             instanceValue=None,
-#                                             ontologyNode=concept_node)
-#                         data_dict[concept_node].append(dataNode)
-#                 # add a property
-#                 # FIXME: same problem as above `for _ in node_value`
-#                 for dataNode, value in zip(data_dict[concept_node], node_value):
-#                     dataNode.__dict__[node.name] = value
-#                     if node.name == key:
-#                         # TODO: how to inteprete the value here then?
-#                         # logistic output have two dim where [1] is the logit for being True
-#                         trial[concept_node, dataNode] = value[-1]
-
-#         # wire up
-#         for concept in concepts:
-#             for rel in concept.contains():
-#                 for dataNode in data_dict[concept]:
-#                     dataNode
-#             for rel in concept.is_a():
-#                 pass
-#             for rel in concept.has_a():
-#                 pass
-
-#         return trial, data_dict
diff --git a/domiknows/graph/allennlp/metrics.py b/domiknows/graph/allennlp/metrics.py
deleted file mode 100644
index ed538a908..000000000
--- a/domiknows/graph/allennlp/metrics.py
+++ /dev/null
@@ -1,283 +0,0 @@
-from typing import Optional
-
-from overrides import overrides
-import torch
-from sklearn import metrics
-
-from allennlp.common.checks import ConfigurationError
-from allennlp.training.metrics import Metric, F1Measure
-
-
-@Metric.register("auc_fixed")
-class Auc(Metric):
-    """
-    The AUC Metric measures the area under the receiver-operating characteristic
-    (ROC) curve for binary classification problems.
-    """
-
-    def __init__(self, positive_label=1):
-        super(Auc, self).__init__()
-        self._positive_label = positive_label
-        self._all_predictions = torch.FloatTensor()
-        self._all_gold_labels = torch.LongTensor()
-
-    def __call__(self,
-                 predictions: torch.Tensor,
-                 gold_labels: torch.Tensor,
-                 mask: Optional[torch.Tensor] = None):
-        """
-        Parameters
-        ----------
-        predictions : ``torch.Tensor``, required.
-            A one-dimensional tensor of prediction scores of shape (batch_size).
-        gold_labels : ``torch.Tensor``, required.
-            A one-dimensional label tensor of shape (batch_size), with {1, 0}
-            entries for positive and negative class. If it's not binary,
-            `positive_label` should be passed in the initialization.
-        mask: ``torch.Tensor``, optional (default = None).
-            A one-dimensional label tensor of shape (batch_size).
-        """
-
-        predictions, gold_labels = self.unwrap_to_tensors(
-            predictions, gold_labels)
-
-        # Sanity checks.
-        if gold_labels.dim() != 1:
-            raise ConfigurationError("gold_labels must be one-dimensional, "
-                                     "but found tensor of shape: {}".format(gold_labels.size()))
-        if predictions.dim() != 1:
-            raise ConfigurationError("predictions must be one-dimensional, "
-                                     "but found tensor of shape: {}".format(predictions.size()))
-
-        unique_gold_labels = torch.unique(gold_labels)
-        if unique_gold_labels.numel() > 2:
-            raise ConfigurationError("AUC can be used for binary tasks only. gold_labels has {} unique labels, "
-                                     "expected at maximum 2.".format(unique_gold_labels.numel()))
-
-        gold_labels_is_binary = list(torch.sort(
-            unique_gold_labels)[0].numpy()) <= [0, 1]
-        if not gold_labels_is_binary and self._positive_label not in unique_gold_labels:
-            raise ConfigurationError("gold_labels should be binary with 0 and 1 or initialized positive_label "
-                                     "{} should be present in gold_labels".format(self._positive_label))
-
-        if mask is None:
-            batch_size = gold_labels.shape[0]
-            mask = torch.ones(batch_size)
-        mask = mask.byte()
-
-        self._all_predictions = torch.cat([self._all_predictions,
-                                           torch.masked_select(predictions, mask).float()], dim=0)
-        self._all_gold_labels = torch.cat([self._all_gold_labels,
-                                           torch.masked_select(gold_labels, mask).long()], dim=0)
-
-    def get_metric(self, reset: bool = False):
-        if self._all_gold_labels.shape[0] == 0:
-            return 0.5
-        if not any(self._all_gold_labels == self._positive_label):
-            return float('nan')
-        false_positive_rates, true_positive_rates, _ = metrics.roc_curve(self._all_gold_labels.numpy(),
-                                                                         self._all_predictions.numpy(),
-                                                                         pos_label=self._positive_label)
-        auc = metrics.auc(false_positive_rates, true_positive_rates)
-        if reset:
-            self.reset()
-        return auc
-
-    @overrides
-    def reset(self):
-        self._all_predictions = torch.FloatTensor()
-        self._all_gold_labels = torch.LongTensor()
-
-
-@Metric.register("ap")
-class AP(Metric):
-    """
-    The Average Presision measurement for binary classification problems.
-    """
-
-    def __init__(self):
-        super(AP, self).__init__()
-        self._all_predictions = torch.FloatTensor()
-        self._all_gold_labels = torch.LongTensor()
-
-    def __call__(self,
-                 predictions: torch.Tensor,
-                 gold_labels: torch.Tensor,
-                 mask: Optional[torch.Tensor] = None):
-        """
-        Parameters
-        ----------
-        predictions : ``torch.Tensor``, required.
-            A one-dimensional tensor of prediction scores of shape (batch_size).
-        gold_labels : ``torch.Tensor``, required.
-            A one-dimensional label tensor of shape (batch_size), with {1, 0}
-            entries for positive and negative class. If it's not binary,
-            `positive_label` should be passed in the initialization.
-        mask: ``torch.Tensor``, optional (default = None).
-            A one-dimensional label tensor of shape (batch_size).
-        """
-
-        predictions, gold_labels = self.unwrap_to_tensors(
-            predictions, gold_labels)
-
-        # Sanity checks.
-        if gold_labels.dim() != 1:
-            raise ConfigurationError("gold_labels must be one-dimensional, "
-                                     "but found tensor of shape: {}".format(gold_labels.size()))
-        if predictions.dim() != 1:
-            raise ConfigurationError("predictions must be one-dimensional, "
-                                     "but found tensor of shape: {}".format(predictions.size()))
-
-        unique_gold_labels = torch.unique(gold_labels)
-        if unique_gold_labels.numel() > 2:
-            raise ConfigurationError("AUC can be used for binary tasks only. gold_labels has {} unique labels, "
-                                     "expected at maximum 2.".format(unique_gold_labels.numel()))
-
-        gold_labels_is_binary = list(torch.sort(
-            unique_gold_labels)[0].numpy()) <= [0, 1]
-        if not gold_labels_is_binary and self._positive_label not in unique_gold_labels:
-            raise ConfigurationError("gold_labels should be binary with 0 and 1 or initialized positive_label "
-                                     "{} should be present in gold_labels".format(self._positive_label))
-
-        if mask is None:
-            batch_size = gold_labels.shape[0]
-            mask = torch.ones(batch_size)
-        mask = mask.byte()
-
-        self._all_predictions = torch.cat([self._all_predictions,
-                                           torch.masked_select(predictions, mask).float()], dim=0)
-        self._all_gold_labels = torch.cat([self._all_gold_labels,
-                                           torch.masked_select(gold_labels, mask).long()], dim=0)
-
-    def get_metric(self, reset: bool = False):
-        if self._all_gold_labels.shape[0] == 0:
-            return 0.5
-        if sum(self._all_gold_labels) == 0:
-            return float('nan')
-
-        ap = metrics.average_precision_score(self._all_gold_labels.numpy(),
-                                             self._all_predictions.numpy())
-        if reset:
-            self.reset()
-        return ap
-
-    @overrides
-    def reset(self):
-        self._all_predictions = torch.FloatTensor()
-        self._all_gold_labels = torch.LongTensor()
-
-
-@Metric.register("pr_auc")
-class PRAuc(Metric):
-    """
-    The Average Presision measurement for binary classification problems.
-    """
-
-    def __init__(self):
-        super(PRAuc, self).__init__()
-        self._all_predictions = torch.FloatTensor()
-        self._all_gold_labels = torch.LongTensor()
-
-    def __call__(self,
-                 predictions: torch.Tensor,
-                 gold_labels: torch.Tensor,
-                 mask: Optional[torch.Tensor] = None):
-        """
-        Parameters
-        ----------
-        predictions : ``torch.Tensor``, required.
-            A one-dimensional tensor of prediction scores of shape (batch_size).
-        gold_labels : ``torch.Tensor``, required.
-            A one-dimensional label tensor of shape (batch_size), with {1, 0}
-            entries for positive and negative class. If it's not binary,
-            `positive_label` should be passed in the initialization.
-        mask: ``torch.Tensor``, optional (default = None).
-            A one-dimensional label tensor of shape (batch_size).
-        """
-
-        predictions, gold_labels = self.unwrap_to_tensors(
-            predictions, gold_labels)
-
-        # Sanity checks.
-        if gold_labels.dim() != 1:
-            raise ConfigurationError("gold_labels must be one-dimensional, "
-                                     "but found tensor of shape: {}".format(gold_labels.size()))
-        if predictions.dim() != 1:
-            raise ConfigurationError("predictions must be one-dimensional, "
-                                     "but found tensor of shape: {}".format(predictions.size()))
-
-        unique_gold_labels = torch.unique(gold_labels)
-        if unique_gold_labels.numel() > 2:
-            raise ConfigurationError("AUC can be used for binary tasks only. gold_labels has {} unique labels, "
-                                     "expected at maximum 2.".format(unique_gold_labels.numel()))
-
-        gold_labels_is_binary = list(torch.sort(
-            unique_gold_labels)[0].numpy()) <= [0, 1]
-        if not gold_labels_is_binary and self._positive_label not in unique_gold_labels:
-            raise ConfigurationError("gold_labels should be binary with 0 and 1 or initialized positive_label "
-                                     "{} should be present in gold_labels".format(self._positive_label))
-
-        if mask is None:
-            batch_size = gold_labels.shape[0]
-            mask = torch.ones(batch_size)
-        mask = mask.byte()
-
-        self._all_predictions = torch.cat([self._all_predictions,
-                                           torch.masked_select(predictions, mask).float()], dim=0)
-        self._all_gold_labels = torch.cat([self._all_gold_labels,
-                                           torch.masked_select(gold_labels, mask).long()], dim=0)
-
-    def get_metric(self, reset: bool = False):
-        if sum(self._all_gold_labels) == 0:
-            return float('nan')
-        p, r, t = metrics.precision_recall_curve(self._all_gold_labels.numpy(),
-                                                 self._all_predictions.numpy())
-        auc = metrics.auc(r, p)
-
-        if reset:
-            self.reset()
-        return auc
-
-    @overrides
-    def reset(self):
-        self._all_predictions = torch.FloatTensor()
-        self._all_gold_labels = torch.LongTensor()
-from typing import Tuple
-
-
-@Metric.register("precision")
-class Precision(F1Measure):
-    """
-    Extract Precision from F1Measure.
-    """
-
-    def get_metric(self, reset: bool = False) -> float:
-        metric = super().get_metric(reset=reset)
-        return metric[0]
-
-@Metric.register("epoch")
-class Epoch(Metric):
-    def __init__(self):
-        super(Epoch, self).__init__()
-        self.epoch_num = None
-
-    def __call__(self, data):
-        self.epoch_num = data['epoch_num'][0]  # (batch,) there are copies
-
-    def get_metric(self, reset: bool = False):
-        return self.epoch_num
-
-@Metric.register("dataset_type")
-class DatasetType(Metric):
-    def __init__(self):
-        super(DatasetType, self).__init__()
-        self.dataset_type = None
-        self.dataset_types = {}
-
-    def __call__(self, data):
-        self.dataset_type = data['dataset_type'][0]  # (batch,) there are copies
-        if self.dataset_type not in self.dataset_types:
-            self.dataset_types[self.dataset_type] = len(self.dataset_types)
-
-    def get_metric(self, reset: bool = False):
-        return self.dataset_types[self.dataset_type]
diff --git a/domiknows/graph/allennlp/model.py b/domiknows/graph/allennlp/model.py
deleted file mode 100644
index 095e707e1..000000000
--- a/domiknows/graph/allennlp/model.py
+++ /dev/null
@@ -1,330 +0,0 @@
-import os
-from typing import Dict, List, Tuple, Iterable
-from collections import OrderedDict
-from torch import Tensor, cuda
-from torch.nn import Module
-import torch
-from allennlp.models import Model
-from allennlp.data import Vocabulary
-from allennlp.nn.util import get_text_field_mask
-
-from allennlp.training.metrics import CategoricalAccuracy, F1Measure
-
-from .metrics import Epoch, DatasetType
-
-from .. import Graph
-from ..trial import Trial
-from ..property import Property
-from ...utils import prod, get_prop_result
-from ...sensor.allennlp import AllenNlpLearner
-from ...sensor.allennlp.base import ModuleSensor
-from ...sensor.allennlp.sensor import CandidateSensor
-from .utils import sequence_cross_entropy_with_logits, structured_perceptron_exact_with_logits
-
-
-DEBUG_TRAINING = 'REGR_DEBUG' in os.environ and os.environ['REGR_DEBUG']
-
-
-DataInstance = Dict[str, Tensor]
-
-
-def update_metrics(
-    graph: Graph,
-    data: DataInstance,
-    metrics: List[Tuple[str, Property, callable]]
-) -> DataInstance:
-    label_masks = {}
-    for sensor in graph.get_sensors(CandidateSensor):
-        sensor(data)
-        mask = data[sensor.fullname].clone().detach()
-        label_masks[mask.shape] = mask
-    for metric_name, class_index, prop, metric in metrics:
-        label, pred, mask = get_prop_result(prop, data)
-        label_mask = label_masks.get(mask.shape)
-        if label_mask is not None:
-            mask = mask * label_mask.float()
-        metric(pred, label, mask)
-    return data
-
-
-class GraphModel(Model):
-    def __init__(
-        self,
-        graph: Graph,
-        vocab: Vocabulary,
-        balance_factor: float = 0.5,
-        label_smoothing: float = 0.1,
-        focal_gamma: float = 2.,
-        inference_interval: int = 10,
-        inference_training_set: bool = False,
-        inference_testing_set: bool = True,
-        inference_loss: bool = False,
-        soft_penalty: float =  1.
-    ) -> None:
-        super().__init__(vocab)
-        self.inference_interval = inference_interval
-        self.inference_training_set = inference_training_set
-        self.inference_testing_set = inference_testing_set
-        self.inference_loss = inference_loss
-        self.meta = []
-        self.metrics = []
-        self.metrics_inferenced = []
-
-        self.balance_factor = balance_factor
-        self.label_smoothing = label_smoothing
-        self.focal_gamma = focal_gamma
-        self.soft_penalty = soft_penalty
-
-        self.graph = graph
-
-        self.meta.append(('epoch', Epoch()))
-        self.meta.append(('type', DatasetType()))
-
-        whole_metrics = {
-            'Accuracy': CategoricalAccuracy,
-        }
-        class_metrics = {
-            ('P', 'R', 'F1'): F1Measure,
-        }
-
-        for prop in self.graph.poi:
-            for metric_name, MetricClass in whole_metrics.items():
-                self.metrics.append((metric_name, None, prop, MetricClass()))
-                self.metrics_inferenced.append((metric_name, None, prop, MetricClass()))
-            for metric_name, MetricClass in class_metrics.items():
-                for sensor in prop.find(AllenNlpLearner):
-                    class_num = prod(sensor.output_dim)
-                    if class_num == 2:
-                        self.metrics.append((metric_name, None, prop, MetricClass(1)))
-                        self.metrics_inferenced.append((metric_name, None, prop, MetricClass(1)))
-                    else:
-                        for class_index in range(class_num):
-                            self.metrics.append((metric_name, class_index, prop, MetricClass(class_index)))
-                            self.metrics_inferenced.append((metric_name, class_index, prop, MetricClass(class_index)))
-
-        #i = 0  # TODO: this looks too bad
-        for prop in self.graph.poi:
-            for sensor in prop.find(ModuleSensor, lambda s: s.module is not None):
-                self.add_module(sensor.name, sensor.module)
-                #i += 1
-
-    def _need_inference(
-        self,
-        data: DataInstance
-    ) -> bool:
-        return self._report_inference(data) or self.inference_loss
-
-    def _report_inference(
-        self,
-        data: DataInstance
-    ) -> bool:
-        #import pdb; pdb.set_trace()
-        if DEBUG_TRAINING:
-            return True
-        dataset_type_key = 'dataset_type' # specify in domiknows.graph.allennlp.base.AllenNlpGraph
-        if (not self.inference_training_set and
-            dataset_type_key in data and
-            all(dataset_type == 'train' for dataset_type in data[dataset_type_key])):
-            return False
-        if (self.inference_testing_set and
-            dataset_type_key in data and
-            all(dataset_type == 'test' for dataset_type in data[dataset_type_key])):
-            return True
-        epoch_key = 'epoch_num'  # TODO: this key... is from Allennlp doc
-        if epoch_key not in data:
-            return True  # no epoch record, then always inference
-        epoch = min(data[epoch_key])
-        need = ((epoch + 1) % self.inference_interval) == 0  # inference every 10 epoch
-        return need
-
-    def _update_metrics(
-        self,
-        trivial_trial, trial
-    ) -> DataInstance:
-        for metric_name, metric in self.meta:
-            metric(trivial_trial)
-        update_metrics(self.graph, trivial_trial, self.metrics)
-        if self._need_inference(trivial_trial) and self._report_inference(trivial_trial):
-            update_metrics(self.graph, trial, self.metrics_inferenced)
-        return trial
-
-    def get_metrics(self, reset: bool=False) -> Dict[str, float]:
-        metrics = OrderedDict()
-
-        def add(metric_name, class_index, suffix, prop, metric):
-            try:
-                out = metric.get_metric(reset)
-            except RuntimeError:
-                # in case inferenced ones are not called
-                # RuntimeError("You never call this metric before.")
-                # then pass
-                return
-
-            if isinstance(out, Iterable):
-                for i, (metric_name_item, out_item) in enumerate(sorted(zip(metric_name, out))):
-                    key = prop.sup.name
-                    if class_index is not None:
-                        class_name = self.vocab.get_token_from_index(class_index, namespace='labels')
-                        key += '({})'.format(class_name)
-                    if suffix is not None:
-                        key += '{}'.format(suffix)
-                    key += '-{}'.format(metric_name_item)
-                    if i == 0:
-                        key = '\n' + key
-                    metrics[key] = out_item
-            else:
-                key = prop.sup.name
-                if class_index is not None:
-                    class_name = self.vocab.get_token_from_index(class_index, namespace='labels')
-                    key += '({})'.format(class_name)
-                if suffix is not None:
-                    key += '{}'.format(suffix)
-                key += '-{}'.format(metric_name)
-                key = '\n' + key
-                metrics[key] = out
-
-        for metric_name, class_index, prop, metric in self.metrics:
-            add(metric_name, class_index, None, prop, metric)
-        for metric_name, class_index, prop, metric in self.metrics_inferenced:
-            add(metric_name, class_index, '_i', prop, metric)
-
-        max_len = max(len(metric_name.lstrip()) for metric_name in metrics)
-        pretty_metrics = OrderedDict()
-        for metric_name, metric in sorted(metrics.items(), key=lambda x: x[0].strip()):
-            lspace_len = len(metric_name) - len(metric_name.lstrip())
-            lspace = metric_name[:lspace_len]
-            name = metric_name[lspace_len:]
-            pretty_name = '{}{:>{max_len}}'.format(lspace, name, max_len=max_len)
-            pretty_metrics[pretty_name] = metric
-
-        for i, (metric_name, metric) in enumerate(self.meta):
-            pretty_metrics['{}[ {} ]'.format('\n' if not i else '', metric_name)] = metric.get_metric(reset)
-        return pretty_metrics
-
-    def _update_loss(self, trivial_trial, trial):
-        trial['loss'] = self.loss_func(trivial_trial, trial)
-        return trial
-
-    i = 0
-    def forward(
-        self,
-        **data: DataInstance
-    ) -> DataInstance:
-        Trial.clear() # reset at every epoch
-
-#         if (self.i % 100 == 0):
-#             import gc
-#             gc.collect()
-#             if cuda.is_available():
-#                 cuda.empty_cache()
-#             self.i = 1
-#         else:
-#             self.i += 1
-
-        # make sure every node needed are calculated
-        for prop in self.graph.poi:
-            for name, sensor in prop.items():
-                sensor(data)
-        trivial_trial = Trial(data)
-
-        # inference
-        with trivial_trial:
-            if self._need_inference(trivial_trial):
-                trial = Trial()
-                self._inference(trial)
-            else:
-                trial = trivial_trial
-        self._update_loss(trivial_trial, trial)
-        self._update_metrics(trivial_trial, trial)
-
-        return dict(trial)
-
-    def _inference(
-        self,
-        data: DataInstance
-    ) -> DataInstance:
-        #import pdb; pdb.set_trace()
-        # process candidates
-        label_not_masks = {}
-        for sensor in self.graph.get_sensors(CandidateSensor):
-            sensor(data)
-            mask = data[sensor.fullname].clone().detach()
-            label_not_masks[mask.shape] = (1-mask).type(torch.uint8)
-        for prop in self.graph.poi:
-            # label (b, l)
-            # pred  (b, l, c)
-            # mask  (b, l)
-            label, pred, mask = get_prop_result(prop, data)
-            if mask.shape in label_not_masks:
-                # label_mask  (b, l)
-                label_not_mask = label_not_masks[mask.shape]
-                shape = pred.shape
-                # (b*l, c)
-                pred = pred.clone().detach().view(-1, shape[-1])
-                # (b*l, )
-                label_not_mask = label_not_mask.view(-1)
-                pred[label_not_mask, :] = torch.tensor(float("nan"))
-                # (b, l, c)
-                pred = pred.view(shape)
-                label_not_mask = label_not_mask.view(mask.shape)
-                data[prop.fullname] = pred
-                for learner in prop.find(AllenNlpLearner):
-                    data[learner.fullname] = pred
-        #data = inference(self.graph, self.graph.solver, data, self.vocab)
-        data = self.graph.solver.inferSelection(self.graph, data)
-
-        return data
-
-    def loss_func(
-        self,
-        data, data_i
-    ) -> DataInstance:
-        #import pdb; pdb.set_trace()
-        loss = 0
-        label_masks = {}
-        for sensor in self.graph.get_sensors(CandidateSensor):
-            sensor(data)
-            mask = data[sensor.fullname].clone().detach()
-            label_masks[mask.shape] = mask
-        for prop in self.graph.poi:
-            # label (b, l)
-            # pred  (b, l, c)
-            # mask  (b, l)
-            label, pred, mask = get_prop_result(prop, data)
-            if self.inference_loss:
-                # (b, l, c)
-                _, pred_i, _ = get_prop_result(prop, data_i)
-                # (b, l,)
-                pred_i = pred_i.argmax(dim=-1)
-            label = label.clone().detach()
-            mask = mask.clone().float()
-            label_mask = label_masks.get(mask.shape)
-            if label_mask is not None:
-                mask = mask * label_mask.float()
-            #import pdb; pdb.set_trace()
-
-            # class balance weighted
-            num_token = float(prod(pred.shape[:-1]))
-            num_classes = pred.shape[-1]
-            balance_bias = num_token / self.balance_factor
-
-            alpha = [((num_token + balance_bias) / ((label == class_index).sum().float() + balance_bias))
-                     for class_index in range(num_classes)]
-
-            if self.inference_loss:
-                loss += structured_perceptron_exact_with_logits(
-                    pred, label, mask, pred_i,
-                    label_smoothing=self.label_smoothing,
-                    gamma=self.focal_gamma,
-                    alpha=alpha,
-                    soft_penalty=self.soft_penalty
-                )
-            else:
-                loss += sequence_cross_entropy_with_logits(
-                    pred, label, mask,
-                    label_smoothing=self.label_smoothing,
-                    gamma=self.focal_gamma,
-                    alpha=alpha
-                )
-
-        return loss
diff --git a/domiknows/graph/allennlp/utils.py b/domiknows/graph/allennlp/utils.py
deleted file mode 100644
index 2be9cbf5d..000000000
--- a/domiknows/graph/allennlp/utils.py
+++ /dev/null
@@ -1,400 +0,0 @@
-from typing import Union, List
-import numpy
-import torch
-
-
-def sequence_cross_entropy_with_logits(logits: torch.FloatTensor,
-                                       targets: torch.LongTensor,
-                                       weights: torch.FloatTensor,
-                                       average: str = "batch",
-                                       label_smoothing: float = None,
-                                       gamma: float = None,
-                                       eps: float = 1e-8,
-                                       alpha: Union[float, List[float], torch.FloatTensor] = None
-                                      ) -> torch.FloatTensor:
-    """
-    Computes the cross entropy loss of a sequence, weighted with respect to
-    some user provided weights. Note that the weighting here is not the same as
-    in the :func:`torch.nn.CrossEntropyLoss()` criterion, which is weighting
-    classes; here we are weighting the loss contribution from particular elements
-    in the sequence. This allows loss computations for models which use padding.
-
-    Parameters
-    ----------
-    logits : ``torch.FloatTensor``, required.
-        A ``torch.FloatTensor`` of size (batch_size, sequence_length, num_classes)
-        which contains the unnormalized probability for each class.
-    targets : ``torch.LongTensor``, required.
-        A ``torch.LongTensor`` of size (batch, sequence_length) which contains the
-        index of the true class for each corresponding step.
-    weights : ``torch.FloatTensor``, required.
-        A ``torch.FloatTensor`` of size (batch, sequence_length)
-    average: str, optional (default = "batch")
-        If "batch", average the loss across the batches. If "token", average
-        the loss across each item in the input. If ``None``, return a vector
-        of losses per batch element.
-    label_smoothing : ``float``, optional (default = None)
-        Whether or not to apply label smoothing to the cross-entropy loss.
-        For example, with a label smoothing value of 0.2, a 4 class classification
-        target would look like ``[0.05, 0.05, 0.85, 0.05]`` if the 3rd class was
-        the correct label.
-    gamma : ``float``, optional (default = None)
-        Focal loss[*] focusing parameter ``gamma`` to reduces the relative loss for
-        well-classified examples and put more focus on hard. The greater value
-        ``gamma`` is, the more focus on hard examples.
-    alpha : ``float`` or ``List[float]``, optional (default = None)
-        Focal loss[*] weighting factor ``alpha`` to balance between classes. Can be
-        used independently with ``gamma``. If a single ``float`` is provided, it
-        is assumed binary case using ``alpha`` and ``1 - alpha`` for positive and
-        negative respectively. If a list of ``float`` is provided, with the same
-        length as the number of classes, the weights will match the classes.
-        [*] T. Lin, P. Goyal, R. Girshick, K. He and P. Dollár, "Focal Loss for
-        Dense Object Detection," 2017 IEEE International Conference on Computer
-        Vision (ICCV), Venice, 2017, pp. 2999-3007.
-
-    Returns
-    -------
-    A torch.FloatTensor representing the cross entropy loss.
-    If ``average=="batch"`` or ``average=="token"``, the returned loss is a scalar.
-    If ``average is None``, the returned loss is a vector of shape (batch_size,).
-
-    """
-    if average not in {None, "token", "batch"}:
-        raise ValueError("Got average f{average}, expected one of "
-                         "None, 'token', or 'batch'")
-
-    # make sure weights are float
-    weights = weights.float()
-    # sum all dim except batch
-    non_batch_dims = tuple(range(1, len(weights.shape)))
-    # shape : (batch_size,)
-    weights_batch_sum = weights.sum(dim=non_batch_dims)
-    # shape : (batch * sequence_length, num_classes)
-    logits_flat = logits.view(-1, logits.size(-1))
-    # shape : (batch * sequence_length, num_classes)
-    log_probs_flat = torch.nn.functional.log_softmax(logits_flat, dim=-1)
-    # shape : (batch * max_len, 1)
-    targets_flat = targets.view(-1, 1).long()
-    # focal loss coefficient
-    if gamma:
-        # shape : (batch * sequence_length, num_classes)
-        probs_flat = log_probs_flat.exp()
-        eps = torch.tensor(eps, device=probs_flat.device)
-        probs_flat = probs_flat.min(1 - eps)
-        probs_flat = probs_flat.max(eps)
-        # shape : (batch * sequence_length,)
-        probs_flat = torch.gather(probs_flat, dim=1, index=targets_flat)
-        # shape : (batch * sequence_length,)
-        focal_factor = (1. - probs_flat) ** gamma
-        # shape : (batch, sequence_length)
-        focal_factor = focal_factor.view(*targets.size())
-        weights = weights * focal_factor
-
-    if alpha is not None:
-        # shape : () / (num_classes,)
-        if isinstance(alpha, (float, int)):
-            # pylint: disable=not-callable
-            # shape : (2,)
-            alpha_factor = torch.tensor([1. - float(alpha), float(alpha)],
-                                        dtype=weights.dtype, device=weights.device)
-            # pylint: enable=not-callable
-        elif isinstance(alpha, (list, numpy.ndarray, torch.Tensor)):
-            # pylint: disable=not-callable
-            # shape : (c,)
-            alpha_factor = torch.tensor(alpha, dtype=weights.dtype, device=weights.device)
-            # pylint: enable=not-callable
-            if not alpha_factor.size():
-                # shape : (1,)
-                alpha_factor = alpha_factor.view(1)
-                # shape : (2,)
-                alpha_factor = torch.cat([1 - alpha_factor, alpha_factor])
-        else:
-            raise TypeError(('alpha must be float, list of float, or torch.FloatTensor, '
-                             '{} provided.').format(type(alpha)))
-        # shape : (batch, max_len)
-        alpha_factor = torch.gather(alpha_factor, dim=0, index=targets_flat.view(-1)).view(*targets.size())
-        weights = weights * alpha_factor
-
-    if label_smoothing is not None and label_smoothing > 0.0:
-        num_classes = logits.size(-1)
-        smoothing_value = label_smoothing / num_classes
-        # Fill all the correct indices with 1 - smoothing value.
-        one_hot_targets = torch.zeros_like(log_probs_flat).scatter_(-1, targets_flat, 1.0 - label_smoothing)
-        smoothed_targets = one_hot_targets + smoothing_value
-        negative_log_likelihood_flat = - log_probs_flat * smoothed_targets
-        negative_log_likelihood_flat = negative_log_likelihood_flat.sum(-1, keepdim=True)
-    else:
-        # Contribution to the negative log likelihood only comes from the exact indices
-        # of the targets, as the target distributions are one-hot. Here we use torch.gather
-        # to extract the indices of the num_classes dimension which contribute to the loss.
-        # shape : (batch * sequence_length, 1)
-        negative_log_likelihood_flat = - torch.gather(log_probs_flat, dim=1, index=targets_flat)
-    # shape : (batch, sequence_length)
-    negative_log_likelihood = negative_log_likelihood_flat.view(*targets.size())
-    # shape : (batch, sequence_length)
-    negative_log_likelihood = negative_log_likelihood * weights
-
-    if average == "batch":
-        # shape : (batch_size,)
-        per_batch_loss = negative_log_likelihood.sum(non_batch_dims) / (weights_batch_sum + 1e-13)
-        num_non_empty_sequences = ((weights_batch_sum > 0).float().sum() + 1e-13)
-        return per_batch_loss.sum() / num_non_empty_sequences
-    elif average == "token":
-        return negative_log_likelihood.sum() / (weights_batch_sum.sum() + 1e-13)
-    else:
-        # shape : (batch_size,)
-        per_batch_loss = negative_log_likelihood.sum(non_batch_dims) / (weights_batch_sum + 1e-13)
-        return per_batch_loss
-
-
-def structured_perceptron_exact_with_logits(logits: torch.FloatTensor,
-                                            targets: torch.LongTensor,
-                                            weights: torch.FloatTensor,
-                                            inferences: torch.FloatTensor = None,
-                                            average: str = "batch",
-                                            label_smoothing: float = None,
-                                            gamma: float = None,
-                                            eps: float = 1e-8,
-                                            alpha: Union[float, List[float], torch.FloatTensor] = None,
-                                            soft_penalty = 1
-                                           ) -> torch.FloatTensor:
-    if average not in {None, "token", "batch"}:
-        raise ValueError("Got average f{average}, expected one of "
-                         "None, 'token', or 'batch'")
-
-    # make sure weights are float
-    weights = weights.float()
-    # sum all dim except batch
-    non_batch_dims = tuple(range(1, len(weights.shape)))
-    # shape : (batch_size,)
-    weights_batch_sum = weights.sum(dim=non_batch_dims)
-    # shape : (batch * sequence_length, num_classes)
-    logits_flat = logits.view(-1, logits.size(-1))
-    # shape : (batch * sequence_length, num_classes)
-    log_probs_flat = torch.nn.functional.log_softmax(logits_flat, dim=-1)
-    # shape : (batch * max_len, 1)
-    targets_flat = targets.view(-1, 1).long()
-
-    # inference coefficient
-    if inferences is not None:
-        if len(inferences.shape) == len(logits.shape): # inferences shape: (batch, sequence_length, ..., num_classes)
-            # shape: (batch, sequence_length, ...)
-            inferences_weights = inferences.select(-1, 1)
-            #inferences_weights = torch.gather(inferences, dim=-1, index=targets)
-        else: # inferences shape: (batch, sequence_length, ...)
-            # shape: (batch, sequence_length, ...)
-            inferences_weights = inferences
-
-        #inferences_weights = (inferences_weights.long() != targets.long()).float()
-        #soft_penalty = 0.6  # 0 - structured loss only; 1 - conventional loss
-        #import pdb; pdb.set_trace()
-        inferences_true = (inferences_weights.long() == targets.long()).float()
-        inferences_weights = 1 - inferences_true * (1 - soft_penalty)
-
-        weights = weights * inferences_weights
-
-    # focal loss coefficient
-    if gamma:
-        # shape : (batch * sequence_length, num_classes)
-        probs_flat = log_probs_flat.exp()
-        eps = torch.tensor(eps, device=probs_flat.device)
-        probs_flat = probs_flat.min(1 - eps)
-        probs_flat = probs_flat.max(eps)
-        # shape : (batch * sequence_length,)
-        probs_flat = torch.gather(probs_flat, dim=1, index=targets_flat)
-        # shape : (batch * sequence_length,)
-        focal_factor = (1. - probs_flat) ** gamma
-        # shape : (batch, sequence_length)
-        focal_factor = focal_factor.view(*targets.size())
-        weights = weights * focal_factor
-
-    if alpha is not None:
-        # shape : () / (num_classes,)
-        if isinstance(alpha, (float, int)):
-            # pylint: disable=not-callable
-            # shape : (2,)
-            alpha_factor = torch.tensor([1. - float(alpha), float(alpha)],
-                                        dtype=weights.dtype, device=weights.device)
-            # pylint: enable=not-callable
-        elif isinstance(alpha, (list, numpy.ndarray, torch.Tensor)):
-            # pylint: disable=not-callable
-            # shape : (c,)
-            alpha_factor = torch.tensor(alpha, dtype=weights.dtype, device=weights.device)
-            # pylint: enable=not-callable
-            if not alpha_factor.size():
-                # shape : (1,)
-                alpha_factor = alpha_factor.view(1)
-                # shape : (2,)
-                alpha_factor = torch.cat([1 - alpha_factor, alpha_factor])
-        else:
-            raise TypeError(('alpha must be float, list of float, or torch.FloatTensor, '
-                             '{} provided.').format(type(alpha)))
-        # shape : (batch, max_len)
-        alpha_factor = torch.gather(alpha_factor, dim=0, index=targets_flat.view(-1)).view(*targets.size())
-        weights = weights * alpha_factor
-
-    if label_smoothing is not None and label_smoothing > 0.0:
-        num_classes = logits.size(-1)
-        smoothing_value = label_smoothing / num_classes
-        # Fill all the correct indices with 1 - smoothing value.
-        one_hot_targets = torch.zeros_like(log_probs_flat).scatter_(-1, targets_flat, 1.0 - label_smoothing)
-        smoothed_targets = one_hot_targets + smoothing_value
-        negative_log_likelihood_flat = - log_probs_flat * smoothed_targets
-        negative_log_likelihood_flat = negative_log_likelihood_flat.sum(-1, keepdim=True)
-    else:
-        # Contribution to the negative log likelihood only comes from the exact indices
-        # of the targets, as the target distributions are one-hot. Here we use torch.gather
-        # to extract the indices of the num_classes dimension which contribute to the loss.
-        # shape : (batch * sequence_length, 1)
-        negative_log_likelihood_flat = - torch.gather(log_probs_flat, dim=1, index=targets_flat)
-    # shape : (batch, sequence_length)
-    negative_log_likelihood = negative_log_likelihood_flat.view(*targets.size())
-    # shape : (batch, sequence_length)
-    negative_log_likelihood = negative_log_likelihood * weights
-
-    if average == "batch":
-        # shape : (batch_size,)
-        per_batch_loss = negative_log_likelihood.sum(non_batch_dims) / (weights_batch_sum + 1e-13)
-        num_non_empty_sequences = ((weights_batch_sum > 0).float().sum() + 1e-13)
-        return per_batch_loss.sum() / num_non_empty_sequences
-    elif average == "token":
-        return negative_log_likelihood.sum() / (weights_batch_sum.sum() + 1e-13)
-    else:
-        # shape : (batch_size,)
-        per_batch_loss = negative_log_likelihood.sum(non_batch_dims) / (weights_batch_sum + 1e-13)
-        return per_batch_loss
-
-
-import logging
-from typing import Any, Iterable, Dict
-from allennlp.common.tqdm import Tqdm
-from allennlp.common.checks import check_for_gpu
-from allennlp.models.model import Model
-from allennlp.data import Instance
-from allennlp.data.iterators import DataIterator
-from allennlp.nn import util as nn_util
-from ...utils import get_prop_result
-from ...sensor.allennlp.sensor import SentenceSensor
-import torch
-
-
-logger = logging.getLogger(__name__)
-
-# We want to warn people that tqdm ignores metrics that start with underscores
-# exactly once. This variable keeps track of whether we have.
-class HasBeenWarned:
-    tqdm_ignores_underscores = False
-
-
-def show_violation(graph, data):
-    viol_logger = logging.getLogger(__name__ + '.violation')
-    sentence_sensor = next(graph.get_sensors(SentenceSensor))
-    sentence = data[sentence_sensor.fullname]
-
-    for prop in graph.poi:
-        label, pred, mask = get_prop_result(prop, data)
-        pred_label = torch.argmax(pred, dim=-1) # hard label for hard violation
-        for rel in prop.sup.is_a():
-            if prop.prop_name not in rel.dst:
-                continue
-            dst_label, dst_pred, dst_mask = get_prop_result(rel.dst[prop.prop_name], data)
-            dst_pred_label = torch.argmax(dst_pred, dim=-1) # hard label for hard violation
-            # pred_label: b, l*
-            violations = torch.nonzero((pred_label > dst_pred_label).float() * mask)
-            for violation in violations:
-                b = violation[0]
-                l = violation[1:]
-                viol_logger.info('\nExample: %s', sentence[b])
-                viol_logger.info('%s %s', l, [sentence[b][ll] for ll in l])
-                viol_logger.info('%s > %s', rel.src, rel.dst)
-
-        for i, rel in enumerate(prop.sup.has_a()):
-            if prop.prop_name not in rel.dst:
-                continue
-            dst_label, dst_pred, dst_mask = get_prop_result(rel.dst[prop.prop_name], data)
-            dst_pred_label = torch.argmax(dst_pred, dim=-1) # hard label for hard violation
-            # pred_label: b, li-1*, li, li+1*
-            # dst_pred: b, li
-            # b, 1*, 1i, 1*
-            for ll in range(len(pred_label.shape)-1):
-                if i != ll:
-                    dst_pred_label = torch.unsqueeze(dst_pred_label, dim=ll+1)
-            violations = torch.nonzero((pred_label > dst_pred_label).float() * mask)
-            for violation in violations:
-                #import pdb; pdb.set_trace()
-                b = violation[0]
-                l = violation[1:]
-                viol_logger.info('\nExample:%s', sentence[b])
-                viol_logger.info('%s(%s)==[%s,%s]==%s', rel.src, ','.join(['{}:{}'.format(ll,sentence[b][ll]) for ll in l]), pred[(b,*l,0)], pred[(b,*l,1)], pred_label[(b,*l)])
-                dst_lo = [l[i],]
-                dst_l = [0,]* len(l)
-                dst_l[i] = l[i]
-                viol_logger.info('%s(%s)==[%s,%s]==%s', rel.dst, sentence[b][l[i]], dst_pred[(b,*dst_lo,0)], dst_pred[(b,*dst_lo,1)], dst_pred_label[(b,*dst_l)])
-                viol_logger.debug(dst_pred[b])
-
-
-def evaluate(model: Model,
-             instances: Iterable[Instance],
-             data_iterator: DataIterator,
-             cuda_device: int,
-             batch_weight_key: str,
-             log_violation: bool=False) -> Dict[str, Any]:
-    check_for_gpu(cuda_device)
-    with torch.no_grad():
-        model.eval()
-
-        iterator = data_iterator(instances,
-                                 num_epochs=1,
-                                 shuffle=False)
-        logger.info("Iterating over dataset")
-        generator_tqdm = Tqdm.tqdm(iterator, total=data_iterator.get_num_batches(instances))
-
-        # Number of batches in instances.
-        batch_count = 0
-        # Number of batches where the model produces a loss.
-        loss_count = 0
-        # Cumulative weighted loss
-        total_loss = 0.0
-        # Cumulative weight across all batches.
-        total_weight = 0.0
-
-        for batch in generator_tqdm:
-            batch_count += 1
-            batch = nn_util.move_to_device(batch, cuda_device)
-            output_dict = model(**batch)
-            loss = output_dict.get("loss")
-            if log_violation:
-                show_violation(model.graph, output_dict)
-
-            metrics = model.get_metrics()
-
-            if loss is not None:
-                loss_count += 1
-                if batch_weight_key:
-                    weight = output_dict[batch_weight_key].item()
-                else:
-                    weight = 1.0
-
-                total_weight += weight
-                total_loss += loss.item() * weight
-                # Report the average loss so far.
-                metrics["loss"] = total_loss / total_weight
-
-            if (not HasBeenWarned.tqdm_ignores_underscores and
-                        any(metric_name.startswith("_") for metric_name in metrics)):
-                logger.warning("Metrics with names beginning with \"_\" will "
-                               "not be logged to the tqdm progress bar.")
-                HasBeenWarned.tqdm_ignores_underscores = True
-            description = ', '.join(["%s: %.2f" % (name, value) for name, value
-                                     in metrics.items() if not name.startswith("_")]) + " ||"
-            generator_tqdm.set_description(description, refresh=False)
-
-        final_metrics = model.get_metrics(reset=True)
-        if loss_count > 0:
-            # Sanity check
-            if loss_count != batch_count:
-                raise RuntimeError("The model you are trying to evaluate only sometimes " +
-                                   "produced a loss!")
-            final_metrics["loss"] = total_loss / total_weight
-
-        return final_metrics
diff --git a/domiknows/graph/candidates.py b/domiknows/graph/candidates.py
index 60740fd2c..f45d0fbc0 100644
--- a/domiknows/graph/candidates.py
+++ b/domiknows/graph/candidates.py
@@ -60,7 +60,19 @@ def intersection_of_lists(lists):
     ordered_common_elements = [elem for elem in lists[0] if elem in common_elements]
     return ordered_common_elements
 
-def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
+def findDatanodesForRootConcept(dn, rootConcept):
+    if dn.myBuilder != None and "DataNodesConcepts" in dn.myBuilder:
+        if rootConcept.name in dn.myBuilder["DataNodesConcepts"]:
+            return dn.myBuilder["DataNodesConcepts"][rootConcept.name]
+
+    dns = dn.findDatanodes(select = rootConcept)
+    
+    if dn.myBuilder != None:
+        dn.myBuilder["DataNodesConcepts"][rootConcept.name] = dns
+        
+    return dns
+    
+def getCandidates(dn, e, variable, lcVariablesDns, lc, logger, integrate = False):
     conceptName = e[0].name
                         
     # -- Collect dataNode for the logical constraint (path)
@@ -71,10 +83,14 @@ def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
         if variable.name == None:
             logger.error('The element %s of logical constraint %s has no name for variable'%(conceptName, lc.lcName))
             return None
-                                    
-        rootConcept = dn.findRootConceptOrRelation(conceptName)
-        _dns = dn.findDatanodes(select = rootConcept)
-        dnsList = [[dn] for dn in _dns]
+        
+        # Check if we already found this variable
+        if variable.name in lcVariablesDns:
+            dnsList = lcVariablesDns[variable.name]  
+        else:
+            rootConcept = dn.findRootConceptOrRelation(conceptName)
+            rootDns = findDatanodesForRootConcept(dn, rootConcept)
+            dnsList = [[rDn] for rDn in rootDns]
     else: # Path specified
         from domiknows.graph.logicalConstrain import eqL
         if not isinstance(variable.v, eqL):
@@ -84,6 +100,13 @@ def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
             
         # -- Prepare paths
         path = variable.v
+        if not isinstance(path, tuple):
+            if isinstance(path, str):
+                path = (path,)
+        
+        if not isinstance(path, (tuple, eqL)):
+            raise TypeError("Path must be a tuple or eqL")
+        
         paths = []
         
         if isinstance(path, eqL):
@@ -116,8 +139,8 @@ def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
         
             if referredVariableName not in lcVariablesDns: # Not yet defined - it has to be the current lc element dataNodes list
                 rootConcept = dn.findRootConceptOrRelation(conceptName)
-                _dns = dn.findDatanodes(select = rootConcept)
-                referredDns = [[dn] for dn in _dns]
+                rootDns = findDatanodesForRootConcept(dn, rootConcept)
+                referredDns = [[rDn] for rDn in rootDns]
                 integrate = True
                 new_iterate = True
             else: # Already defined in the logical constraint from the v part 
@@ -125,7 +148,7 @@ def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
                 referredDns = lcVariablesDns[referredVariableName] # Get DataNodes for referred variables already defined in the logical constraint
                 
             # Get variables from dataNodes selected  based on referredVariableName
-            for listOfDataNodes in referredDns:
+            for indexDn, listOfDataNodes in enumerate(referredDns):
                 eDns = [] 
                 
                 for currentReferredDataNode in listOfDataNodes:
@@ -134,14 +157,14 @@ def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
                     
                     # -- Get DataNodes for the edge defined by the path part of the v
                     if isinstance(path, eqL):
-                        currentEDns = getEdgeDataNode(currentReferredDataNode, v)
+                        currentEDns = getEdgeDataNode(currentReferredDataNode, v, indexDn, lcVariablesDns)
                         currentEDnsNew = [currentEDn for currentEDn in currentEDns if currentEDn is not None]
                         if currentEDnsNew and len(currentEDnsNew):
                             eDns.extend(currentEDnsNew)
                         else:
                             pass
                     else:
-                        currentEDns = getEdgeDataNode(currentReferredDataNode, v[1:]) 
+                        currentEDns = getEdgeDataNode(currentReferredDataNode, v[1:], indexDn, lcVariablesDns) 
                         # currentEDnsNew = [currentEDn for currentEDn in currentEDns if currentEDn is not None]
                         if currentEDns is not None and len(currentEDns):
                             eDns.extend(currentEDns)
@@ -214,7 +237,7 @@ def getCandidates(dn, e, variable, lcVariablesDns, lc, logger):
 
 # Find DataNodes starting from the given DataNode following provided path
 #     path can contain eqL statement selecting DataNodes from the DataNodes collecting on the path
-def getEdgeDataNode(dn, path):
+def getEdgeDataNode(dn, path, currentIndexDN, lcVariablesDns):
     # Path is empty
     
     if isinstance(path, eqL):
@@ -234,38 +257,67 @@ def getEdgeDataNode(dn, path):
     # Path has at least 2 elements - will perform recursion
 
     if isinstance(path[0], eqL): # check if eqL
-        path0 = path[0].e[0][0]
+        if isinstance(path[0].e[0], tuple):
+            path0 = path[0].e[0][0]
+        else:
+            path0 = path[0].e[0]
     else:
         path0 = path[0]
 
     relDns = None         
     if dn.isRelation(path0):
-        relDns = dn.getDnsForRelation(path0)
+        relDns = dn.getDnsForRelation(path0)            
     elif isinstance(path0, str):
         relDns = dn.getDnsForRelation(path0)
     else: # if not relation then has to be attribute in eql
-        relDns = []
+        path0Dns = relDns
         attributeName = path[0].e[1]
-        attributeValue = dn.getAttribute(attributeName)
         
-        if torch.is_tensor(attributeValue) and attributeValue.ndimension() == 0:
-            attributeValue = attributeValue.item()
+        relDns = []
+        if attributeName == "instanceID":
             
-        requiredValue = path[0].e[2]
-         
-        if attributeValue in requiredValue:
-            # return [dn]
-            relDns.append(dn)
-        elif (True in  requiredValue ) and attributeValue == 1:
-            # return [dn]
-            relDns.append(dn)
-        elif (False in  requiredValue ) and attributeValue == 0:
-            attributeValue = False
+            for pDns in path0Dns:
+                attributeValue = pDns.getInstanceID()
+                referredCandidateID = path[0].e[2]
+                
+                if referredCandidateID in lcVariablesDns and len(lcVariablesDns[referredCandidateID]) - 1 >= currentIndexDN:
+                    currentReferredCandidates = lcVariablesDns[referredCandidateID][currentIndexDN]
+                    for currentReferredCandidate in currentReferredCandidates:
+                        currentReferredCandidateID = currentReferredCandidate.getInstanceID()
+                        
+                        if currentReferredCandidateID == attributeValue:
+                            if dn not in relDns:
+                                relDns.append(dn)
+                    
+                # Check if it is a valid relation link with not empty set of connected datanodes      
+                if relDns is None or len(relDns) == 0 or relDns[0] is None:
+                    return [None]
+                else:
+                    return relDns
         else:
-            # return [None]
-            # relDns.append(None)
-            pass
-      
+            relDns = []
+            attributeName = path[0].e[1]
+            
+            attributeValue = dn.getAttribute(attributeName)
+                
+            if torch.is_tensor(attributeValue) and attributeValue.ndimension() == 0:
+                attributeValue = attributeValue.item()
+                
+            requiredValue = path[0].e[2]
+                
+            if attributeValue in requiredValue:
+                # return [dn]
+                relDns.append(dn)
+            elif (True in  requiredValue ) and attributeValue == 1:
+                # return [dn]
+                relDns.append(dn)
+            elif (False in  requiredValue ) and attributeValue == 0:
+                attributeValue = False
+            else:
+                # return [None]
+                # relDns.append(None)
+                pass
+    
     # Check if it is a valid relation link  with not empty set of connected datanodes      
     if relDns is None or len(relDns) == 0 or relDns[0] is None:
         return [None]
@@ -280,15 +332,19 @@ def getEdgeDataNode(dn, path):
                 path0e1 = path[0].e[1].name
                 
             if path0e1 in cDn.attributes or ("rootDataNode" in cDn.attributes and (path0.name + "/" + path0e1) in cDn.attributes["rootDataNode"].attributes["propertySet"]):
-                if cDn.getAttribute(path0e1).item() in path[0].e[2]:
-                    _cDns.append(cDn)
+                if torch.is_tensor(cDn.getAttribute(path0e1)):
+                    if cDn.getAttribute(path0e1).item() in path[0].e[2]:
+                        _cDns.append(cDn)
+                else:
+                    if cDn.getAttribute(path0e1) in path[0].e[2]:
+                        _cDns.append(cDn) 
                 
         relDns = _cDns
     
     # recursion
     rDNS = []
     for cDn in relDns:
-        rDn = getEdgeDataNode(cDn, path[1:])
+        rDn = getEdgeDataNode(cDn, path[1:], currentIndexDN, lcVariablesDns)
         
         if rDn:
             rDNS.extend(rDn)
diff --git a/domiknows/graph/concept.py b/domiknows/graph/concept.py
index 485b2e480..41e6c3067 100644
--- a/domiknows/graph/concept.py
+++ b/domiknows/graph/concept.py
@@ -3,24 +3,48 @@
 from typing import Type
 from .base import Scoped, BaseGraphTree
 from ..utils import enum
-
+# from .relation import Contains, HasA, IsA
 
 @Scoped.class_scope
 @BaseGraphTree.localize_namespace
 class Concept(BaseGraphTree):
-    _rels = {}  # catogrory_name : creation callback
+    _rels = {}  # category_name : creation callback
+    newVariableIndex = 0
+
+    @classmethod
+    def get_new_variable_index(cls):
+        # This method returns the current value of newVariableIndex and increments it
+        index = cls.newVariableIndex
+        cls.newVariableIndex += 1
+        return index
 
     @classmethod
     def relation_type(cls, name=None):
         def update(Rel):
             if name is not None:
                 Rel.name = classmethod(lambda cls: name)
+                Rel.relation_cls_name = name
 
             def create(src, *args, auto_constraint=None, **kwargs):
                 # add one-by-one
+                if Rel.relation_cls_name == "contains":
+                    arg_names = [arg.name for arg in args]
+                    arg_names = ", ".join(arg_names)
+                    assert len(args) == 1, f"The Contains relationship defined from {src.name} concept to concepts {arg_names} is not valid. The contains relationship can only be between one source and one destination concepts."
+                elif Rel.relation_cls_name == "has_a":
+                    if args:
+                        arg_names = [arg.name for arg in args]
+                        arg_names = " ".join(arg_names)
+                    elif kwargs:
+                        arg_names = [arg.name for arg in kwargs.values()]
+                        arg_names = " ".join(arg_names)
+                    assert len(args) >= 2 or len(kwargs) >= 2, f"The HasA relationship defined from {src.name} concept to concepts {arg_names} is not valid. The HasA relationship must be between one source and at least two destination concepts."
                 rels = []
+                
                 for argument_name, dst in chain(enum(args, cls=Concept, offset=len(src._out)), enum(kwargs, cls=Concept)):
                     # will be added to _in and _out in Rel constructor
+                    if 'is_a' in dst._out:
+                        dst = dst._out['is_a'][0].dst
                     rel_inst = Rel(src, dst, argument_name=argument_name, auto_constraint=auto_constraint)
                     rels.append(rel_inst)
                 return rels
@@ -30,41 +54,82 @@ def create(src, *args, auto_constraint=None, **kwargs):
 
         return update
 
-    def __init__(self, name=None):
+    def __init__(self, name=None, batch=False):
         '''
         Declare an concept.
         '''
+        self.batch = batch
         BaseGraphTree.__init__(self, name)
+        self._in = OrderedDict()  # relation category_name : list of relation inst
+        self._out = OrderedDict()  # relation category_name : list of relation inst
+        
+        self.var_name = None
+        
+    def get_batch(self):
+        return self.batch
 
-        self._in = OrderedDict()  # relation catogrory_name : list of relation inst
-        self._out = OrderedDict()  # relation catogrory_name : list of relation inst
-
+    def get_var_name(self):
+        return self.var_name
+    
     def __str__(self):
         return self.name
     
     def __rept__(self):
         return type(self) + ":" + self.name
     
+    def assign_suggest_name(self, name=None):
+        cls = type(self)
+        if name is None:
+            name = cls.suggest_name()            
+        assert cls._names[name] == 0, f"The name {name} has been already used in this graph for a concept before, please use a unique name."
+        cls._names[name] += 1
+        self.name = name
+        cls._objs[name] = self
+        
+    def findRootGraph(self, superGraph):
+        if superGraph._sub  == None:
+            return superGraph
+        
+        return self.findRootGraph(superGraph._sub)
+    
     def processLCArgs(self, *args, conceptT=None, **kwargs):
         from domiknows.graph.logicalConstrain import eqL, V
+        
+        error = None
+        
         if len(args) > 1 and isinstance(args[1], eqL):
             nameX = args[0]
             path = (nameX, args[1])
                                     
-            return [conceptT, V(name=nameX, v=path)]
-        elif len(args) and isinstance(args[0], str):
+            return [conceptT, V(name=nameX, v=path), error]
+        
+        if len(args) > 1:
+            selfInfo = self._context[0].findConceptInfo(self)
+            
+            relation_attrs = selfInfo.get('relationAttrs', {})
+    
+            if len(relation_attrs) != len(args):
+                error = ("extraV", ) + tuple(args)
+            else:
+                newVariable = 'p' + str(600 + Concept.get_new_variable_index())
+                
+                info = {arg: V(name=None, v=(newVariable, value))  for arg, value in zip(args, relation_attrs.keys())}
+                args = (newVariable,)
+                error = ('VarMaps', info)
+
+        if len(args) and isinstance(args[0], str):
             name = args[0]
             
             if "path" in kwargs:
                 path = kwargs['path']
                 
-                return [conceptT, V(name=name, v=path)]
+                return [conceptT, V(name=name, v=path), error]
             else:
-                return [conceptT, V(name=name)]
+                return [conceptT, V(name=name), error]
         elif "path" in kwargs:
             path = kwargs['path']
                                     
-            return [conceptT, V(name=None, v=path)]
+            return [conceptT, V(name=None, v=path), error]
         else:
             return [conceptT]
 
@@ -166,10 +231,12 @@ def ecHandle(*args, **kwargs):
                     
                     return ecHandle
             else:
-                raise AttributeError(*e.args)
+                raise AttributeError(f"The concept {self.name} does not have an attribute with the value '{e.args}'. You should probably use a different base concept for this.")
         def handle(*args, **kwargs):
             if not args and not kwargs:
                 return self._out.setdefault(rel, [])
+            if 'name' in kwargs:
+                raise TypeError(f"You cannot name the concepts or relations in the graph, you have done this to connect {self.name} to {args[0].name}")
             return Rel(self, *args, **kwargs)
         return handle
 
diff --git a/domiknows/graph/dataNode.py b/domiknows/graph/dataNode.py
index ef207e6a2..04b450770 100644
--- a/domiknows/graph/dataNode.py
+++ b/domiknows/graph/dataNode.py
@@ -1,14 +1,18 @@
 import torch
 from collections import OrderedDict, namedtuple
-from time import process_time_ns
+from time import perf_counter, perf_counter_ns
 import re
+from itertools import count
 
-from .dataNodeConfig import dnConfig 
+from .dataNodeConfig import dnConfig
 
-from ordered_set import OrderedSet 
+from ordered_set import OrderedSet
 
 from domiknows import getRegrTimer_logger, getProductionModeStatus
 from domiknows.solver import ilpOntSolverFactory
+from domiknows.utils import getDnSkeletonMode
+from domiknows.graph.relation import Contains
+
 
 import logging
 from logging.handlers import RotatingFileHandler
@@ -39,7 +43,7 @@
         logBackupCount = dnConfig['log_backupCount']
     if 'log_fileMode' in dnConfig:
         logFileMode = dnConfig['log_fileMode']
-        
+
 # Create file handler and set level to info
 import pathlib
 pathlib.Path("logs").mkdir(parents=True, exist_ok=True)
@@ -58,93 +62,195 @@
 _DataNode__Logger.addHandler(ch)
 # Don't propagate
 _DataNode__Logger.propagate = False
-        
+
 # --- Create loggers
-_DataNodeBulder__Logger  = logging.getLogger("dataNodeBuilder")
-_DataNodeBulder__Logger.setLevel(logLevel)
+_DataNodeBuilder__Logger  = logging.getLogger("dataNodeBuilder")
+_DataNodeBuilder__Logger.setLevel(logLevel)
 # Add ch to logger
-_DataNodeBulder__Logger.addHandler(ch)
+_DataNodeBuilder__Logger.addHandler(ch)
 # Don't propagate
-_DataNodeBulder__Logger.propagate = False
+_DataNodeBuilder__Logger.propagate = False
 
-_DataNodeBulder__Logger.info('--- Starting new run ---')
+_DataNodeBuilder__Logger.info('--- Starting new run ---')
 
-# Class representing single data instance with relation links to other data nodes
 class DataNode:
-    def __init__(self, instanceID = None, instanceValue = None, ontologyNode = None, relationLinks = {}, attributes = {}):
-
+    """
+    Represents a single data instance in a graph with relation links to other data nodes.
+
+    Attributes:
+        - myBuilder (DatanodeBuilder): DatanodeBuilder used to construct this datanode.
+        - instanceID (various): The data instance ID (e.g., paragraph number, sentence number).
+        - instanceValue (various): Optional value of the instance (e.g., text, bitmap).
+        - ontologyNode (Node): Reference to the node in the ontology graph.
+        - graph (Graph): Graph to which the DataNode belongs.
+        - relationLinks (dict): Dictionary mapping relation name to RelationLinks.
+        - impactLinks (dict): Dictionary with dataNodes impacting this dataNode.
+        - attributes (dict): Dictionary with node's attributes.
+        - current_device (str): The current device being used ('cpu' or 'cuda').
+        - gurobiModel (NoneType): Placeholder for Gurobi model.
+        - myLoggerTime (Logger): Logger for time measurement.
+    """
+    _ids = count(1)
+
+    def __init__(self, myBuilder = None, instanceID = None, instanceValue = None, ontologyNode = None, graph = None, relationLinks = {}, attributes = {}):
+        """Initialize a DataNode instance.
+
+        Args:
+            myBuilder (DatanodeBuilder): DatanodeBuilder used to construct this datanode.
+            instanceID (various): The data instance ID.
+            instanceValue (various): Optional value of the instance.
+            ontologyNode (Node): Reference to the node in the ontology graph.
+            graph (Graph): Graph to which the DataNode belongs.
+            relationLinks (dict): Dictionary mapping relation name to RelationLinks.
+            attributes (dict): Dictionary with node's attributes.
+
+        Attributes:
+            myBuilder (DatanodeBuilder): DatanodeBuilder used to construct this datanode.
+            instanceID (various): The data instance ID.
+            instanceValue (various): Optional value of the instance.
+            ontologyNode (Node): Reference to the node in the ontology graph.
+            graph (Graph): Graph to which the DataNode belongs.
+            relationLinks (dict): Dictionary mapping relation name to RelationLinks.
+            impactLinks (dict): Dictionary with dataNodes impacting this dataNode.
+            attributes (dict): Dictionary with node's attributes.
+            current_device (str): The current device being used ('cpu' or 'cuda').
+            gurobiModel (NoneType): Placeholder for Gurobi model.
+            myLoggerTime (Logger): Logger for time measurement.
+        """
+        self.id = next(self._ids)
+        self.myBuilder = myBuilder                       # DatanodeBuilder used to construct this datanode
         self.instanceID = instanceID                     # The data instance id (e.g. paragraph number, sentence number, phrase  number, image number, etc.)
         self.instanceValue = instanceValue               # Optional value of the instance (e.g. paragraph text, sentence text, phrase text, image bitmap, etc.)
         self.ontologyNode = ontologyNode                 # Reference to the node in the ontology graph (e.g. Concept) which is the type of this instance (e.g. paragraph, sentence, phrase, etc.)
-        
-        self.graph =  self.ontologyNode.sup
+
+        if ontologyNode is not None:
+            self.graph = self.ontologyNode.sup
+            if graph is not None:
+                self.graph = graph
+
         if relationLinks:
-            self.relationLinks = relationLinks           # Dictionary mapping relation name to RealtionLinks
+            self.relationLinks = relationLinks           # Dictionary mapping relation name to RelationLinks
         else:
             self.relationLinks = {}
-            
+
         self.impactLinks = {}                            # Dictionary with dataNodes impacting this dataNode by having it as a subject of its relation
-        
+
         if attributes:
             self.attributes = attributes                 # Dictionary with node's attributes
         else:
             self.attributes = {}
-            
-        self.current_device = 'auto'
-                     
+
+        self.current_device = 'cpu'
+        if torch.cuda.is_available():
+            if torch.cuda.device_count() > 1:
+                self.current_device = 'cuda:1'
+            else:
+                self.current_device = 'cuda' if torch.cuda.is_available() else 'cpu'
+
+        self.gurobiModel = None
+
+        self.myLoggerTime = getRegrTimer_logger()
+
     class DataNodeError(Exception):
+        """Exception raised for DataNode-related errors."""
         pass
-    
+
     def __str__(self):
+        """Return the string representation of the DataNode object.
+
+        Returns:
+            str: String representation of the instance.
+        """
         if self.instanceValue:
             return self.instanceValue
         else:
             return '{} {}'.format(self.ontologyNode.name, self.instanceID)
-        
+
     def __repr__(self):
+        """Return the unambiguous string representation of the DataNode object.
+
+        Returns:
+            str: Unambiguous string representation of the instance.
+        """
         if self.instanceValue:
             return self.instanceValue
         else:
             return '{} {}'.format(self.ontologyNode.name, self.instanceID)
-        
-    def __reprDeep__(self,  strRep = ""):
-        
+
+    def __reprDeep__(self, strRep=""):
+        """Return the deep string representation of the DataNode object including its relations.
+
+        Args:
+            strRep (str): Accumulated string representation.
+
+        Returns:
+            str: Deep string representation of the instance.
+        """
         rel = [*self.getRelationLinks().keys()]
         if 'contains' in rel:
             rel.remove('contains')
-            
-        relSting = None
+
+        relString = None
         if len(rel) > 0:
-            relSting = ' (' + rel + ')'
-            
-        if relSting:
+            relString = ' (' + str(rel) + ')'
+
+        if relString:
             strRep += self.ontologyNode.name + str(rel)
         else:
-            strRep += self.ontologyNode.name 
-            
+            strRep += self.ontologyNode.name
+
         childrenDns = {}
         for cDn in self.getChildDataNodes():
             if cDn.getOntologyNode().name not in childrenDns:
                 childrenDns[cDn.getOntologyNode().name] = []
-
             childrenDns[cDn.getOntologyNode().name].append(cDn)
-                
+
         strRep += '\n'
         for childType in childrenDns:
             strRep += '\t' + childrenDns[childType][0].__repr__(strRep)
-        
+
         return strRep
-    
+
     def getInstanceID(self):
+        """Get the instance ID of the DataNode object.
+
+        Returns:
+            various: Instance ID of the DataNode object.
+        """
         return self.instanceID
-    
+
     def getInstanceValue(self):
+        """Get the instance value of the DataNode object.
+
+        Returns:
+            various: Instance value of the DataNode object.
+        """
         return self.instanceValue
-    
+
     def getOntologyNode(self):
+        """Get the ontology node related to the DataNode object.
+
+        Returns:
+            Node: Ontology node related to the DataNode object.
+        """
         return self.ontologyNode
 
     def visualize(self, filename: str, inference_mode="ILP", include_legend=False, open_image=False):
+        """Visualize the current DataNode instance and its attributes.
+
+        This method creates a graph visualization using the Graphviz library. The
+        visualization includes attributes and relationships.
+
+        Args:
+            filename (str): The name of the file where the Graphviz output will be stored.
+            inference_mode (str, optional): The mode used for inference ("ILP" by default).
+            include_legend (bool, optional): Whether or not to include a legend in the visualization.
+            open_image (bool, optional): Whether or not to automatically open the generated image.
+
+        Raises:
+            Exception: If the specified inference_mode is not found in the DataNode.
+
+        """
         if include_legend:
             # Build Legend subgraph
             legend = graphviz.Digraph(name='cluster_legend',comment='Legend')
@@ -198,7 +304,7 @@ def visualize(self, filename: str, inference_mode="ILP", include_legend=False, o
             else:
                 # Normal nodes
                 g.attr('node', shape='rectangle')
-                
+
                 # Format attribute
                 attr_str = str(attribute)
                 if isinstance(attribute, torch.Tensor):
@@ -213,34 +319,63 @@ def visualize(self, filename: str, inference_mode="ILP", include_legend=False, o
         main_graph.subgraph(g)
 
         main_graph.render(filename, format='png', view=open_image)
-    
+
     # --- Attributes methods
-    
+
     def getAttributes(self):
-        return self.attributes     
-    
+        """Get all attributes of the DataNode.
+
+        Returns:
+            dict: Dictionary containing all attributes of the DataNode.
+        """
+        return self.attributes
+
+    def hasAttribute(self, key):
+        """Check if the DataNode has a specific attribute.
+
+        Args:
+            key (str): The key of the attribute to check for.
+
+        Returns:
+            bool: True if the attribute exists, False otherwise.
+        """
+        # Your code for checking attribute existence
+        # ...
+        return False
+
     def getAttribute(self, *keys):
+        """Retrieve a specific attribute using a key or a sequence of keys.
+
+        The method accepts multiple keys in the form of positional arguments,
+        combining them to identify the attribute to retrieve.
+
+        Args:
+            *keys (str or tuple or Concept): The key(s) to identify the attribute.
+
+        Returns:
+            object: The value of the attribute if it exists, or None otherwise.
+        """
         key = ""
         keyBis  = ""
         index = None
-        
+
         conceptFound = False
         for _, kConcept in enumerate(keys):
             if key != "":
                 key = key + "/"
                 keyBis = keyBis + "/"
-                
+
             # Handle different way of representing concept in the key list
             if isinstance(kConcept, str): # Concept name
-                cocneptForK = None
+                conceptForK = None
                 if not conceptFound:
-                    cocneptForK = self.findConcept(kConcept) # Find concept
-                
-                if not conceptFound and cocneptForK is not None:  
+                    conceptForK = self.findConcept(kConcept) # Find concept
+
+                if not conceptFound and conceptForK is not None:
                     conceptFound = True
-                    if isinstance(cocneptForK, tuple):
-                        key = key + '<' + cocneptForK[0].name +'>'
-                        index = cocneptForK[2]
+                    if isinstance(conceptForK, tuple):
+                        key = key + '<' + conceptForK[0].name +'>'
+                        index = conceptForK[2]
                         keyBis = keyBis + kConcept
                     else:
                         key = key + '<' + kConcept +'>'
@@ -256,8 +391,8 @@ def getAttribute(self, *keys):
                 conceptFound = True
                 key = key + '<' + kConcept.name +'>'
                 keyBis = keyBis + kConcept.name
-            
-        # Use key and keyBis to get the dn attribute     
+
+        # Use key and keyBis to get the dn attribute
         if key in self.attributes:
             if index is None:
                 return self.attributes[key]
@@ -268,109 +403,216 @@ def getAttribute(self, *keys):
                 return self.attributes[keyBis]
             else:
                 return self.attributes[keyBis][index]
-        elif "rootDataNode" in self.attributes:
-            rootDataNode = self.attributes["rootDataNode"]
-            if "variableSet" in rootDataNode.attributes:
+        elif "rootDataNode" in self.attributes or "variableSet" in self.attributes:
+            if "rootDataNode" in self.attributes:
+                rootDataNode = self.attributes["rootDataNode"]
                 keyInVariableSet = self.ontologyNode.name + "/" + key
-                if keyInVariableSet in rootDataNode.attributes["variableSet"]:
-                    return rootDataNode.attributes["variableSet"][keyInVariableSet][self.instanceID]
-                elif keyInVariableSet in rootDataNode.attributes["propertySet"]:
-                    return rootDataNode.attributes["propertySet"][keyInVariableSet][self.instanceID]
-                
-        return None   
-           
+
+                if "variableSet" in rootDataNode.attributes:
+                    if keyInVariableSet in rootDataNode.attributes["variableSet"]:
+                        return rootDataNode.attributes["variableSet"][keyInVariableSet][self.instanceID]
+                    elif "propertySet" in rootDataNode.attributes and keyInVariableSet in rootDataNode.attributes["propertySet"]:
+                        return rootDataNode.attributes["propertySet"][keyInVariableSet][self.instanceID]
+            elif "variableSet" in self.attributes:
+                if key in self.attributes["variableSet"]:
+                    return self.attributes["variableSet"][key]
+                elif "propertySet" in self.attributes and key in self.attributes["propertySet"]:
+                    return self.attributes["propertySet"][key]
+
+        return None
+
     # --- Relation Link methods
-     
+
     def getRelationLinks(self, relationName = None, conceptName = None):
+        """Retrieve relation links for a given relation and concept name.
+
+        This method retrieves relation links based on the relation name and/or
+        the concept name. It supports the flexibility to look up based on either
+        just a relation name, just a concept name, or both. If neither is given,
+        it returns all relation links.
+
+        Args:
+            relationName (str or None): The name of the relation to filter by. If None, no filtering is done based on the relation name.
+            conceptName (str or None): The name of the concept to filter by. If None, no filtering is done based on the concept name.
+
+        Returns:
+            list: A list of DataNodes that match the given relation and concept names, or an empty list if no matches are found.
+        """
         if relationName is None:
             if conceptName is None:
                 return self.relationLinks
             else:
                 conceptCN = []
-                
+
                 for r in self.relationLinks:
                     for dn in self.relationLinks[r]:
                         if dn.ontologyNode.name == conceptName:
                             conceptCN.append(dn)
-            
+
                 return conceptCN
-        
+
         if not isinstance(relationName, str):
             relationName = relationName.name
-            
+
         if relationName in self.relationLinks:
             relDNs = self.relationLinks[relationName]
-            
+
             if conceptName is None:
                 return relDNs
             else:
                 conceptCN = []
-            
+
             if not isinstance(conceptName, str):
                 conceptName = conceptName.name
-            
+
             for dn in relDNs:
                 if dn.ontologyNode.name == conceptName:
                     conceptCN.append(dn)
-            
+
             return conceptCN
         else:
             return []
-        
+
     def addRelationLink(self, relationName, dn):
+        """Add a relation link between the current DataNode and another DataNode.
+
+        This method establishes a relation link from the current DataNode to another
+        DataNode ('dn') under a given relation name. It also updates the impactLinks
+        for the target DataNode.
+
+        Args:
+            relationName (str): The name of the relation to add.
+            dn (DataNode): The target DataNode to link to.
+
+        Returns:
+            None
+        """
         if relationName is None:
             return
-        
+
         if relationName not in self.relationLinks:
             self.relationLinks[relationName] = []
-            
+
         if dn in self.relationLinks[relationName]:
-            return 
-        
+            return
+
         self.relationLinks[relationName].append(dn)
-        
+
         # Impact
         if relationName not in dn.impactLinks:
             dn.impactLinks[relationName] = []
-            
+
         if self not in dn.impactLinks[relationName]:
             dn.impactLinks[relationName].append(self)
 
     def removeRelationLink(self, relationName, dn):
+        """Remove a relation link between the current DataNode and another DataNode.
+
+        This method removes a relation link from the current DataNode to another
+        DataNode ('dn') under a given relation name. It also updates the impactLinks
+        for the target DataNode.
+
+        Args:
+            relationName (str): The name of the relation to remove.
+            dn (DataNode): The target DataNode to unlink from.
+
+        Returns:
+            None
+        """
         if relationName is None:
             return
-        
+
         if relationName not in self.relationLinks:
             return
-        
+
         self.relationLinks[relationName].remove(dn)
-        
+
         # Impact
         if relationName in  dn.impactLinks:
             dn.impactLinks[relationName].remove(self)
-            
-    def getLinks(self):
+
+    def getLinks(self, relationName = None, conceptName = None):
+        """Get links associated with the DataNode based on the relation and concept names.
+
+        This method retrieves the DataNodes linked to the current DataNode through
+        either relation links or impact links. You can filter these links based on
+        the name of the relation or the name of the concept (ontology node).
+
+        Args:
+            relationName (str, optional): The name of the relation to filter by.
+                                          Defaults to None.
+            conceptName (str, optional): The name of the ontology node (concept) to filter by.
+                                         Defaults to None.
+
+        Returns:
+            dict or list: A dictionary containing the DataNodes linked through relation or
+                          impact links. If relationName or conceptName is provided,
+                          returns a list of DataNodes that match the criteria.
+        """
         keys = self.relationLinks.keys() | self.impactLinks.keys()
-        
+
         links = {}
         for k in keys:
             if k not in self.relationLinks:
                 links[k] = self.impactLinks[k]
                 continue
-            
+
             if k not in self.impactLinks:
                 links[k] = self.relationLinks[k]
                 continue
-            
+
             links[k] = self.impactLinks[k] + self.relationLinks[k]
-            
-        return links
+
+        if relationName is None:
+            if conceptName is None:
+                return links
+            else:
+                conceptCN = []
+
+                for r in links:
+                    for dn in links[r]:
+                        if dn.ontologyNode.name == conceptName:
+                            conceptCN.append(dn)
+
+                return conceptCN
+
+        if not isinstance(relationName, str):
+            relationName = relationName.name
+
+        if relationName in links:
+            relDNs = links[relationName]
+
+            if conceptName is None:
+                return relDNs
+            else:
+                conceptCN = []
+
+            if not isinstance(conceptName, str):
+                conceptName = conceptName.name
+
+            for dn in relDNs:
+                if dn.ontologyNode.name == conceptName:
+                    conceptCN.append(dn)
+
+            return conceptCN
+        else:
+            return []
 
     # --- Contains (children) relation methods
-    
+
     def getChildDataNodes(self, conceptName = None):
+        """Retrieve child DataNodes based on the concept name.
+
+        Args:
+            conceptName (str, optional): The name of the concept to filter the child DataNodes.
+                                         Defaults to None.
+
+        Returns:
+            list: A list of child DataNodes that match the given concept name. Returns None if
+                  there are no child DataNodes.
+        """
         containsDNs = self.getRelationLinks('contains')
-        
+
         if conceptName is None:
             return containsDNs
 
@@ -378,7 +620,7 @@ def getChildDataNodes(self, conceptName = None):
             return None
 
         conceptCN = []
-            
+
         for dn in containsDNs:
             if isinstance(conceptName, str):
                 if dn.ontologyNode.name == conceptName:
@@ -386,88 +628,145 @@ def getChildDataNodes(self, conceptName = None):
             else:
                 if dn.ontologyNode == conceptName:
                     conceptCN.append(dn)
-        
+
         return conceptCN
-            
+
     def addChildDataNode(self, dn):
+        """Add a child DataNode to the current DataNode.
+
+        Args:
+            dn (DataNode): The DataNode to be added as a child.
+        """
         relationName = 'contains'
-        
+
         if (relationName in self.relationLinks) and (dn in self.relationLinks[relationName]):
             return
-        
+
         self.addRelationLink(relationName, dn)
 
     def removeChildDataNode(self, dn):
+        """Remove a child DataNode from the current DataNode.
+
+        Args:
+            dn (DataNode): The DataNode to be removed.
+        """
         relationName = 'contains'
 
         self.removeRelationLink(relationName, dn)
-        
+
     def resetChildDataNode(self):
+        """Reset all child DataNodes from the current DataNode.
+        """
         relationName = 'contains'
 
         self.relationLinks[relationName] = []
-        
+
     # --- Equality methods
-    
-    def getEqualTo(self, equalName = "equalTo", conceptName = None):
+
+    def getEqualTo(self, equalName="equalTo", conceptName=None):
+        """Retrieve DataNodes that are equal to the current DataNode.
+
+        Args:
+            equalName (str, optional): The name of the relation for equality. Defaults to "equalTo".
+            conceptName (str, optional): The name of the concept to filter the DataNodes.
+                                         Defaults to None.
+
+        Returns:
+            list: A list of DataNodes that are considered equal to the current DataNode.
+        """
         if conceptName:
             dns = self.getRelationLinks(relationName=equalName)
-            
+
             filteredDns = []
             for dn in dns:
                 if dn.getOntologyNode().name == conceptName:
                     filteredDns.append(dn)
-                    
+
             return filteredDns
         else:
             return self.getRelationLinks(relationName=equalName)
-    
-    def addEqualTo(self, equalDn, equalName = "equalTo"):
+
+    def addEqualTo(self, equalDn, equalName="equalTo"):
+        """Add a DataNode that is considered equal to the current DataNode.
+
+        Args:
+            equalDn (DataNode): The DataNode to be added.
+            equalName (str, optional): The name of the relation for equality. Defaults to "equalTo".
+        """
         self.addRelationLink(equalName, equalDn)
-        
-    def removeEqualTo(self, equalDn, equalName = "equalTo"):
+
+    def removeEqualTo(self, equalDn, equalName="equalTo"):
+        """Remove a DataNode that is considered equal to the current DataNode.
+
+        Args:
+            equalDn (DataNode): The DataNode to be removed.
+            equalName (str, optional): The name of the relation for equality. Defaults to "equalTo".
+        """
         self.removeRelationLink(equalName, equalDn)
 
+
     # --- Query methods
-    
-    # Recursively search for concepts and relations in the data graph
+
     def findConceptsAndRelations(self, dn, conceptsAndRelations = None, visitedDns = None):
+        """Recursively search for concepts and relations in the data graph starting from a given dataNode (dn).
+
+        This method will traverse through linked dataNodes to find concepts and relations. If 'variableSet'
+        is present in the attributes, it will return those concepts directly.
+
+        Args:
+            dn (DataNode): The dataNode from which to start the search.
+            conceptsAndRelations (set, optional): A set to store found concepts and relations. Defaults to None.
+            visitedDns (set, optional): A set to keep track of visited dataNodes to prevent cycles. Defaults to None.
+
+        Returns:
+            set: A set containing the names of all found concepts and relations.
+
+        """
         if 'variableSet' in self.attributes:
             conceptsAndRelations = set()
             for key in self.attributes['variableSet']:
                 if "/label" in key:
                     continue
                 conceptsAndRelations.add(key[key.index('<')+1:key.index('>')])
-            
+
             return conceptsAndRelations
-        else: 
+        else:
             if conceptsAndRelations is None:
                 conceptsAndRelations = set()
             if visitedDns is None:
                 visitedDns = set()
-                
+
             # Find concepts in dataNode - concept are in attributes from learning sensors
             for att in dn.attributes:
-                if att[0] == '<' and att[-1] == '>':  
+                if att[0] == '<' and att[-1] == '>':
                     if att[1:-1] not in conceptsAndRelations:
                         conceptsAndRelations.add(att[1:-1])
                         _DataNode__Logger.info('Found concept %s in dataNode %s'%(att[1:-1],dn))
-                        
-            # Recursively find concepts and relations in linked dataNodes 
+
+            # Recursively find concepts and relations in linked dataNodes
             links = dn.getLinks()
             if links:
                 for link in links:
                     for lDn in links[link]:
                         if lDn in visitedDns:
                             continue
-                        
+
                         visitedDns.add(lDn)
                         self.findConceptsAndRelations(lDn, conceptsAndRelations = conceptsAndRelations, visitedDns = visitedDns)
-    
+
             return conceptsAndRelations
 
-    # Find concept and relation names of DataNodes - used in concept.py
     def findConceptsNamesInDatanodes(self, dns = None, conceptNames = None, relationNames = None):
+        """Finds all unique concept and relation names in a list of DataNodes.
+
+        Args:
+            dns (list, optional): A list of DataNodes to be searched. Defaults to None.
+            conceptNames (set, optional): A set to store the names of concepts found. Defaults to None.
+            relationNames (set, optional): A set to store the names of relations found. Defaults to None.
+
+        Returns:
+            tuple: A tuple containing two sets: (conceptNames, relationNames).
+        """
         if conceptNames is None:
             conceptNames=set()
         if relationNames is None:
@@ -475,75 +774,96 @@ def findConceptsNamesInDatanodes(self, dns = None, conceptNames = None, relation
 
         if dns is None:
             dns = [self]
-            
+
         for dn in dns:
             conceptNames.add(dn.getOntologyNode().name)
             for relName, _ in dn.getRelationLinks().items():
                 if relName != 'contains':
                     relationNames.add(relName)
-                
+
             self.findConceptsNamesInDatanodes(dns=dn.getChildDataNodes(), conceptNames = conceptNames, relationNames = relationNames)
-            
+
         return conceptNames, relationNames
-    
-    # Find the root parent of relation of the given relation
+
     def findRootConceptOrRelation(self, relationConcept, usedGraph = None):
+        """Finds the root concept or relation of a given relation or concept.
+
+        Args:
+            relationConcept (str or Object): The relation or concept to find the root for.
+            usedGraph (Object, optional): The ontology graph where the relation or concept exists. Defaults to None.
+
+        Returns:
+            Object or str: The root concept or relation.
+        """
         if usedGraph is None:
             usedGraph = self.ontologyNode.getOntologyGraph()
-        
+
         if isinstance(relationConcept, str):
             _relationConcepts = self.findConcept(relationConcept)
-            
+
             if _relationConcepts:
                 relationConcept = _relationConcepts[0]
             else:
-                return relationConcept 
+                return relationConcept
 
         # Does this concept or relation has parent (through _isA)
+
+        if isinstance(relationConcept, tuple):
+            relationConcept = relationConcept[0]
+
         try:
             isAs = relationConcept.is_a()
         except (AttributeError, TypeError):
             isAs = []
-        
+
         for _isA in isAs:
             _relationConcept = _isA.dst
-            
+
             return  self.findRootConceptOrRelation(_relationConcept, usedGraph)
-        
+
         # If the provided concept or relation is root (has not parents)
-        return relationConcept 
+        return relationConcept
 
     def __testDataNode(self, dn, test):
+        """Tests a DataNode based on various types of conditions.
+
+        Args:
+            dn (Object): The DataNode to be tested.
+            test (tuple/list/str/int): The conditions to test the DataNode.
+
+        Returns:
+            bool: True if the DataNode satisfies the conditions, False otherwise.
+        """
         if test is None:
             return False
-            
+
         if isinstance(test, tuple) or isinstance(test, list): # tuple with at least three elements (concept, key elements, value of attribute)
             _test = []
             for t in test:
                 if isinstance(t, tuple):
                     r = self.__testDataNode(dn, t)
-                    
+
                     if not r:
                         return False
                 else:
                     _test.append(t)
-             
+
             if len(_test) == 0:
                 return True
             else:
                 test = _test
-               
-            if len(test) >= 3:     
+
+            if len(test) >= 3:
                 if isinstance(test[0], str):
                     if dn.getOntologyNode().name != test[0]:
                         return False
                 else:
                     if dn.getOntologyNode().name != test[0].name:
                         return False
-                    
+
                 keys = test[1:-1]
                 v = dn.getAttribute(*keys)
-                
+
                 last = test[-1]
                 if v == last:
                     return True
@@ -551,40 +871,56 @@ def __testDataNode(self, dn, test):
                     return False
         else:
             test = [test]
-    
+
         for i, t in enumerate(test):
             if isinstance(t, int):
                 if dn.getInstanceID() == t:
                     return True
                 else:
                     return False
-                
+
             if t == "instanceID" and i < len(test) - 1:
                 if dn.getInstanceID() == test[i+1]:
                     return True
                 else:
                     return False
-                
+
             if not isinstance(t, str):
                 t = t.name
-            
+
             if t == dn.getOntologyNode().name:
                 return True
             else:
                 return False
-    
+
     def getDnsForRelation(self, rel):
+        """Get DataNodes associated with a given relation.
+
+        The method first finds the root concept or relation for the given 'rel'.
+        Depending on what it finds, it returns the corresponding DataNodes.
+
+        Args:
+            rel (str/Object): The relation or concept for which DataNodes are needed.
+
+        Returns:
+            list: A list of DataNodes corresponding to the relation, or [None] if not found.
+        """
+        
         relRoot = self.findRootConceptOrRelation(rel)
-            
+
         if relRoot is None:
             return [None]
-        
+
+        if isinstance(relRoot, Contains):
+
+            relRoot = "contains" #TODO fix the generic name for multiple constaints
+
         if not isinstance(relRoot, str):
-            relRoot = relRoot.name     
-        
+            relRoot = relRoot.name  
+
         if relRoot.endswith(".reversed"):
             relRoot = relRoot[:-len(".reversed")]
-            if relRoot in self.impactLinks: 
+            if relRoot in self.impactLinks:
                 return self.impactLinks[relRoot]
             else:
                 return [None]
@@ -592,341 +928,422 @@ def getDnsForRelation(self, rel):
             return self.relationLinks[relRoot]
         else:
             return [None]
-            
-    # Find dataNodes in data graph for the given concept 
+
     def findDatanodes(self, dns = None, select = None, indexes = None, visitedDns = None, depth = 0):
+        """Find and return DataNodes based on the given query conditions.
+
+        Args:
+            dns (list): List of DataNodes to start with.
+            select (object): Query condition for selecting DataNodes.
+            indexes (dict): Optional query filtering.
+            visitedDns (OrderedSet): Keeps track of already visited DataNodes.
+            depth (int): Depth of the recursive call.
+
+        Returns:
+            list: List of DataNodes that satisfy the query condition.
+        """
         # If no DataNodes provided use self
         if not depth and dns is None:
             dns = [self]
-            
+
         returnDns = []
-        
+
         # If empty list of provided DataNodes then return - it is a recursive call with empty list
         if dns is None or len(dns) == 0:
             return returnDns
-        
+
         # No select provided - query not defined - return
         if select is None:
             if depth == 0 and not returnDns:
                 _DataNode__Logger.warning('Not found any DataNode - no value for the select part of query provided')
-                
+
             return returnDns
-       
-        # Check each provided DataNode if it satisfy the select part of the query  
+
+        # Check each provided DataNode if it satisfy the select part of the query
         for dn in dns:
             # Test current DataNote against the query
             if self.__testDataNode(dn, select):
                 if dn not in returnDns:
-                    returnDns.append(dn) 
-                            
+                    returnDns.append(dn)
+
             if not visitedDns:
                 visitedDns = OrderedSet()
-                             
+
             visitedDns.add(dn)
-                    
+
         # Call recursively
         newDepth = depth + 1
         for dn in dns:
             # Visit  DataNodes in links
-            for r, rValue in dn.getLinks().items():            
-               
+            for r, rValue in dn.getLinks().items():
+
                 # Check if the nodes already visited
                 dnsToVisit = OrderedSet()
                 for rDn in rValue:
                     if rDn not in visitedDns:
                         dnsToVisit.add(rDn)
-                    
+
                 if not dnsToVisit:
                     continue
-                
+
                 # Visit DataNodes in the current relation
-                _returnDns = self.findDatanodes(dnsToVisit, select = select, indexes = indexes, visitedDns = visitedDns, depth = newDepth)
-        
-                if _returnDns is not None:
-                    for _dn in _returnDns:
-                        if _dn not in returnDns:
-                            returnDns.append(_dn)
+                currentRelationDns = self.findDatanodes(dnsToVisit, select = select, indexes = indexes, visitedDns = visitedDns, depth = newDepth)
+
+                if currentRelationDns is not None:
+                    for currentRDn in currentRelationDns:
+                        if currentRDn not in returnDns:
+                            returnDns.append(currentRDn)
 
         if depth: # Finish recursion
             return returnDns
-        
+
         # If index provided in query then filter the found results for the select part of query through the index part of query
         if (indexes != None):
-            _returnDns = [] # Will contain results from returnDns satisfying the index
-            
+            currentReturnDns = [] # Will contain results from returnDns satisfying the index
+
             for dn in returnDns:
-                fit = True       
+                fit = True
                 for indexName, indexValue in indexes.items():
-                    
+
                     relDns = dn.getDnsForRelation(indexName)
-                    
+
                     if relDns is None or len(relDns) == 0 or relDns[0] is None:
                         fit = False
                         break
-                    
+
                     found = False
-                    for _dn in relDns:
+                    for rDn in relDns:
                         if isinstance(indexValue, tuple):
                             _test = []
                             for t in indexValue:
                                 if isinstance(t, tuple):
-                                    r = self.__testDataNode(_dn, t)
-                                    
+                                    r = self.__testDataNode(rDn, t)
+
                                     if r:
                                         found = True
                                         break
                                 else:
                                     _test.append(t)
-                             
+
                             if len(_test) == 0:
                                 continue
                             else:
                                 indexValue = _test
-                                
-                        if self.__testDataNode(_dn, indexValue):
+
+                        if self.__testDataNode(rDn, indexValue):
                             found = True
                             break
-                        
+
                     if not found:
                         fit = False
                         break
-                        
+
                 if fit:
-                    if dn not in _returnDns:
-                        _returnDns.append(dn)
-                       
-            returnDns = _returnDns
-        
+                    if dn not in currentReturnDns:
+                        currentReturnDns.append(dn)
+
+            returnDns = currentReturnDns
+
         # If not fund any results
         if depth == 0 and not returnDns:
             _DataNode__Logger.debug('Not found any DataNode for - %s -'%(select))
-    
+
         # Sort results according to their ids
         if returnDns:
             returnDnsNotSorted = OrderedDict()
             for dn in returnDns:
                 returnDnsNotSorted[dn.getInstanceID()] = dn
-                    
+
             returnDnsSorted = OrderedDict(sorted(returnDnsNotSorted.items()))
-        
+
             returnDns = [*returnDnsSorted.values()]
-        
+
         return returnDns
-          
+
     # Get root of the dataNode
     def getRootDataNode(self):
+        """Get the root DataNode.
+
+        Returns:
+            object: The root DataNode.
+        """
         if "contains" in self.impactLinks:
             return self.impactLinks["contains"][0].getRootDataNode()
         else:
             return self
-    
+
     # Keeps hashMap of concept name queries in findConcept to results
     conceptsMap = {}
-    
-    # Find concept in the graph based on concept name
+
     def findConcept(self, conceptName, usedGraph = None):
+        """Find concept based on the name in the ontology graph.
+
+        Args:
+            conceptName (str or Concept): The name of the concept to find.
+            usedGraph (object): The ontology graph to search within.
+
+        Returns:
+            tuple or None: A tuple containing details about the found concept or None if not found.
+        """
         if '<' in conceptName:
             conceptName = conceptName[1:-1]
-            
+
         if not usedGraph:
             usedGraph = self.ontologyNode.getOntologyGraph()
-            
+
         if usedGraph not in self.conceptsMap:
             self.conceptsMap[usedGraph] = {}
-            
+
         usedGraphConceptsMap = self.conceptsMap[usedGraph]
-        
+
         if isinstance(conceptName, Concept):
             conceptName = conceptName.name()
-            
+
         if conceptName in usedGraphConceptsMap:
             return usedGraphConceptsMap[conceptName]
-        
+
         subGraph_keys = [key for key in usedGraph._objs]
         for subGraphKey in subGraph_keys:
             subGraph = usedGraph._objs[subGraphKey]
-            
+
             for conceptNameItem in subGraph.concepts:
                 if conceptName == conceptNameItem:
                     concept = subGraph.concepts[conceptNameItem]
-                    
+
                     usedGraphConceptsMap[conceptName] =  (concept, concept.name, None, 1)
                     return usedGraphConceptsMap[conceptName]
-                
+
                 elif isinstance(subGraph.concepts[conceptNameItem], EnumConcept):
                     vlen = len(subGraph.concepts[conceptNameItem].enum)
-                    
+
                     if conceptName in subGraph.concepts[conceptNameItem].enum:
                         concept = subGraph.concepts[conceptNameItem]
-                        
+
                         usedGraphConceptsMap[conceptName] = (concept, conceptName, subGraph.concepts[conceptNameItem].get_index(conceptName), vlen)
                         return usedGraphConceptsMap[conceptName]
 
         usedGraphConceptsMap[conceptName] = None
-        
+
         return usedGraphConceptsMap[conceptName]
 
-    # Check if concept is relation
     def isRelation(self, conceptRelation, usedGraph = None):
+        """Check if a concept is a relation.
+
+        Args:
+            conceptRelation (str or Concept): The concept or relation to check.
+            usedGraph (object, optional): The ontology graph to use. Defaults to the one associated with self.
+
+        Returns:
+            bool: True if the concept is a relation, otherwise False.
+        """
         if usedGraph is None:
             usedGraph = self.ontologyNode.getOntologyGraph()
-        
+
         if isinstance(conceptRelation, str):
             conceptRelation = self.findConcept(conceptRelation)
-            
+
             if conceptRelation == None:
                 return False
-            
+
             conceptRelation = conceptRelation[0]
-            
+
         from  domiknows.graph.relation import Relation
         if isinstance(conceptRelation, Relation):
             return True
-        
-        if len(conceptRelation.has_a()) > 0:  
+
+        if len(conceptRelation.has_a()) > 0:
             return True
-        
+
         for _isA in conceptRelation.is_a():
             _conceptRelation = _isA.dst
-            
+
             if self.__isRelation(_conceptRelation, usedGraph):
                 return True
-        
-        return False 
-    
+
+        return False
+
     def getRelationAttrNames(self, conceptRelation, usedGraph = None):
+        """Get attribute names for a given relation or concept that is a relation.
+
+        Args:
+            conceptRelation (Concept): The concept or relation to check for attributes.
+            usedGraph (object, optional): The ontology graph to use. Defaults to the ontology graph associated with self.
+
+        Returns:
+            OrderedDict or None: An ordered dictionary of attribute names and their corresponding concepts, or None if no attributes found.
+        """
         if usedGraph is None:
             usedGraph = self.ontologyNode.getOntologyGraph()
-            
-        if len(conceptRelation.has_a()) > 0:  
+
+        if len(conceptRelation.has_a()) > 0:
             relationAttrs = OrderedDict()
-            for _, rel in enumerate(conceptRelation.has_a()): 
-                dstName = rel.dst.name                
+            for _, rel in enumerate(conceptRelation.has_a()):
+                dstName = rel.dst.name
                 relationAttr = self.findConcept(dstName, usedGraph)[0]
-    
+
                 relationAttrs[rel.name] = relationAttr
-                
+
             return relationAttrs
-        
+
         for _isA in conceptRelation.is_a():
             _conceptRelation = _isA.dst
-            
+
             resultForCurrent = self.__getRelationAttrNames(_conceptRelation, usedGraph)
-            
+
             if bool(resultForCurrent):
                 return resultForCurrent
-        
-        return None 
+
+        return None
 
     # cache
     collectedConceptsAndRelations = None
-    
-    # Collect all the concepts and relation from the data graph and translate them to tuple form
+
     def collectConceptsAndRelations(self, conceptsAndRelations = None):
+        """Collect all the concepts and relations from the data graph and transform them into tuple form.
+
+        Args:
+            conceptsAndRelations (set, optional): A set to store the found concepts and relations. Defaults to None.
+
+        Returns:
+            list: A list of tuples, each representing a concept or relation with additional information.
+        """
         if conceptsAndRelations is None:
             conceptsAndRelations = set()
-            
+
         if self.collectedConceptsAndRelations:
             return self.collectedConceptsAndRelations
-        
+
         # Search the graph starting from self for concepts and relations
-        candR = self.findConceptsAndRelations(self) 
-        
+        candR = self.findConceptsAndRelations(self)
+        self.rootConcepts = []
+
         returnCandR = []
-        
+
         # Process founded concepts - translate them to tuple form with more information needed for logical constraints and metrics
         for c in candR:
             _concept = self.findConcept(c)[0]
-            
+
             if _concept is None:
                 continue
-            
+
             if isinstance(_concept, tuple):
                 _concept = _concept[0]
-            
+
             # Check if this is multiclass concept
             if isinstance(_concept, EnumConcept):
+                self.rootConcepts.append((_concept, len(_concept.enum)))
+
                 for i, a in enumerate(_concept.enum):
-                    
+
                     if conceptsAndRelations and a not in conceptsAndRelations:
                         # continue
                         pass
-                    
+
                     returnCandR.append((_concept, a, i, len(_concept.enum))) # Create tuple representation for multiclass concept
             else:
+                self.rootConcepts.append((_concept, 1))
+
                 if conceptsAndRelations and c not in conceptsAndRelations and _concept not in conceptsAndRelations:
                     continue
-                
+
                 returnCandR.append((_concept, _concept.name, None, 1)) # Create tuple representation for binary concept
-        
+
         self.collectedConceptsAndRelations = returnCandR
         return self.collectedConceptsAndRelations
-        
-    def __getILPSolver(self, conceptsRelations = None):
+
+    def getILPSolver(self, conceptsRelations = None):
+        """Get the ILP Solver instance based on the given concepts and relations.
+
+        Args:
+            conceptsRelations (list, optional): A list of concepts and relations to be considered. Defaults to None.
+
+        Returns:
+            tuple: An instance of ILP Solver and the list of processed concepts and relations.
+
+        Raises:
+            DataNodeError: If the ILP Solver is not initialized.
+        """
         if conceptsRelations is None:
             conceptsRelations = []
 
         _conceptsRelations = []
-        
+
         # Get ontology graphs and then ilpOntsolver
         myOntologyGraphs = {self.ontologyNode.getOntologyGraph()}
-        
+
         for currentConceptOrRelation in conceptsRelations:
             if isinstance(currentConceptOrRelation, str):
                 currentConceptOrRelation = self.findConcept(currentConceptOrRelation)
-            
+
             _conceptsRelations.append(currentConceptOrRelation)
-            
+
             if isinstance(currentConceptOrRelation, tuple):
                 currentOntologyGraph = currentConceptOrRelation[0].getOntologyGraph()
             else:
                 currentOntologyGraph = currentConceptOrRelation.getOntologyGraph()
-            
+
             if currentOntologyGraph is not None:
                 myOntologyGraphs.add(currentOntologyGraph)
-                
+
         myilpOntSolver = ilpOntSolverFactory.getOntSolverInstance(myOntologyGraphs)
-        
+
         if not myilpOntSolver:
             _DataNode__Logger.error("ILPSolver not initialized")
             raise DataNode.DataNodeError("ILPSolver not initialized")
-        
+
         return myilpOntSolver, _conceptsRelations
-    
+
     #----------------- Solver methods
 
-    # Collect inferred results of the given type (e.g. ILP, softmax, argmax, etc) from the given concept
     def collectInferredResults(self, concept, inferKey):
+        """Collect inferred results based on the given concept and inference key.
+
+        Args:
+            concept (Concept or tuple): The concept for which to collect inferred results.
+            inferKey (str): The type of inference, e.g., 'ILP', 'softmax', 'argmax'.
+
+        Returns:
+            torch.Tensor: Tensor containing collected attribute list.
+        """
         collectAttributeList = []
-        
+
         if not isinstance(concept, tuple):
             if not isinstance(concept, Concept):
                 concept = self.findConcept(concept)
                 if concept is None:
                     return torch.tensor(collectAttributeList)
-                
+
             if isinstance(concept, EnumConcept):
                 concept = (concept, concept.name, None, len(concept.enum))
             else:
                 concept = (concept, concept.name, None, 1)
-    
+
         rootConcept = self.findRootConceptOrRelation(concept[0])
-        
+
         if not rootConcept:
             return torch.tensor(collectAttributeList)
-        
+
         rootConceptDns = self.findDatanodes(select = rootConcept)
-        
+
         if not rootConceptDns:
             return torch.tensor(collectAttributeList)
 
+        if getDnSkeletonMode() and "variableSet" in self.attributes:
+            vKeyInVariableSet = rootConcept.name + "/<" + concept[0].name +">"
+
+            # inferKey
+            inferKeyInVariableSet = vKeyInVariableSet + "/" + inferKey
+
+            if self.hasAttribute(inferKeyInVariableSet):
+                return self.getAttribute(inferKeyInVariableSet)
+
         keys = [concept, inferKey]
-        
+
         for dn in rootConceptDns:
             rTensor = dn.getAttribute(*keys)
             if rTensor is None:
                 continue
-            
+
             if torch.is_tensor(rTensor):
                 if len(rTensor.shape) == 0 or len(rTensor.shape) == 1 and rTensor.shape[0] == 1:
                     collectAttributeList.append(rTensor.item())
@@ -942,205 +1359,625 @@ def collectInferredResults(self, concept, inferKey):
                 collectAttributeList.append(1)
             else:
                 collectAttributeList.append(0)
-                
+
         if collectAttributeList and torch.is_tensor(collectAttributeList[0]):
             return torch.stack(tuple(collectAttributeList), dim=0)
-        
-        return torch.as_tensor(collectAttributeList)        
-    
-    # Calculate argMax and softMax
+
+        return torch.as_tensor(collectAttributeList)
+
     def infer(self):
-        conceptsRelations = self.collectConceptsAndRelations() 
-        
+        """Calculate argMax and softMax for the ontology-based data structure."""
+        conceptsRelations = self.collectConceptsAndRelations()
+
         for c in conceptsRelations:
             cRoot = self.findRootConceptOrRelation(c[0])
-            dns = self.findDatanodes(select = cRoot)
-            
-            if not dns:
-                continue
-            
-            vs = []
-            
-            for dn in dns:
-                v = dn.getAttribute(c[0])
-                
-                if v is None:
-                    vs = []
-                    break
-                elif not torch.is_tensor(v):
-                    vs = []
-                    break
-                else:
+
+            # ----- skeleton - tensor
+            if getDnSkeletonMode() and "variableSet" in self.attributes:
+                vKeyInVariableSet = cRoot.name + "/<" + c[0].name +">"
+
+                # softmax
+                softmaxKeyInVariableSet = vKeyInVariableSet + "/softmax"
+
+                if not self.hasAttribute(softmaxKeyInVariableSet):
+                    vKeyInVariableSetValues = self.attributes["variableSet"][vKeyInVariableSet]
                     if c[2] is not None:
-                        vs.append(v[c[2]])
+                        v = vKeyInVariableSetValues[:, c[2]]
                     else:
-                        if len(v.size()) != 1 or v.size()[0] != 2:
-                            vs = []
-                            break
+                        v = vKeyInVariableSetValues[:, 1]
+
+                    # check if v is None or not a tensor
+                    if v is None or not torch.is_tensor(v):
+                        continue
+
+                    if not(isinstance(v, torch.FloatTensor) or isinstance(v, torch.cuda.FloatTensor)):
+                        v = v.float()
+
+                    vSoftmaxT = torch.nn.functional.softmax(v, dim=-1)
+                    self.attributes["variableSet"][softmaxKeyInVariableSet] = vSoftmaxT
+
+                # argmax
+                argmaxKeyInVariableSet = vKeyInVariableSet + "/argmax"
+                if not self.hasAttribute(argmaxKeyInVariableSet):
+                    vKeyInVariableSetValues = self.attributes["variableSet"][vKeyInVariableSet]
+                    if c[2] is not None:
+                        v = vKeyInVariableSetValues[:, c[2]]
+                    else:
+                        v = vKeyInVariableSetValues[:, 1]
+
+                    vArgmaxTInxexes = torch.argmax(v, dim=-1)
+                    vArgmax = torch.zeros_like(v).scatter_(-1, vArgmaxTInxexes.unsqueeze(-1), 1.)
+
+                    self.attributes["variableSet"][argmaxKeyInVariableSet] = vArgmax
+
+                # This is test
+                if False:
+                    dns = self.findDatanodes(select = cRoot)
+                    for dn in dns:
+                        keyArgmax = "<" + c[0].name + ">/argmax"
+                        keySoftMax = "<" + c[0].name + ">/softmax"
+
+                        index = c[2]
+                        if index is None:
+                            index = 1
+
+                        s = dn.getAttribute(keySoftMax)[c[2]]
+                        a = dn.getAttribute(keyArgmax)[c[2]]
+                        continue
+
+            else:
+                # ---- loop through dns
+                dns = self.findDatanodes(select = cRoot)
+
+                if not dns:
+                    continue
+
+                vs = []
+
+                for dn in dns:
+                    v = dn.getAttribute(c[0])
+
+                    if v is None:
+                        vs = []
+                        break
+                    elif not torch.is_tensor(v):
+                        vs = []
+                        break
+                    else:
+                        if c[2] is not None:
+                            vs.append(v[c[2]])
                         else:
-                            vs.append(v[1])
-            
-            if not vs:
-                continue
-            
-            t = torch.tensor(vs)
-            t[torch.isnan(t)] = 0 # NAN  -> 0
-            
-            vM = torch.argmax(t).item() # argmax
-            
-            # Elements for softmax
-            tExp = torch.exp(t)
-            tExpSum = torch.sum(tExp).item()
-            
-            keyArgmax = "<" + c[0].name + ">/argmax"
-            keySoftMax = "<" + c[0].name + ">/softmax"
-            
-            # Add argmax and softmax to DataNodes
-            for dn in dns:    
-                if keyArgmax not in dn.attributes:
-                    dn.attributes[keyArgmax] = torch.empty(c[3], dtype=torch.float)
-                    
-                if dn.getInstanceID() == vM:
-                    dn.attributes[keyArgmax][c[2]] = 1
-                else:
-                    dn.attributes[keyArgmax][c[2]] = 0
-                
-                if keySoftMax not in dn.attributes:
-                    dn.attributes[keySoftMax] = torch.empty(c[3], dtype=torch.float)
-                    
-                dnSoftmax = tExp[dn.getInstanceID()]/tExpSum
-                dn.attributes[keySoftMax][c[2]] = dnSoftmax.item()
-
-    # Calculate local for datanote argMax and softMax
-    def inferLocal(self):
-        conceptsRelations = self.collectConceptsAndRelations() 
-        
+                            if len(v.size()) != 1 or v.size()[0] != 2:
+                                vs = []
+                                break
+                            else:
+                                vs.append(v[1])
+
+                if not vs:
+                    continue
+
+                t = torch.tensor(vs)
+                t[torch.isnan(t)] = 0 # NAN  -> 0
+
+                vM = torch.argmax(t).item() # argmax
+
+                # Elements for softmax
+                tExp = torch.exp(t)
+                tExpSum = torch.sum(tExp).item()
+
+                keyArgmax = "<" + c[0].name + ">/argmax"
+                keySoftMax = "<" + c[0].name + ">/softmax"
+
+                # Add argmax and softmax to DataNodes
+                for dn in dns:
+                    if keyArgmax not in dn.attributes:
+                        dn.attributes[keyArgmax] = torch.empty(c[3], dtype=torch.float)
+
+                    if dn.getInstanceID() == vM:
+                        dn.attributes[keyArgmax][c[2]] = 1
+                    else:
+                        dn.attributes[keyArgmax][c[2]] = 0
+
+                    if keySoftMax not in dn.attributes:
+                        dn.attributes[keySoftMax] = torch.empty(c[3], dtype=torch.float)
+
+                    dnSoftmax = tExp[dn.getInstanceID()]/tExpSum
+                    dn.attributes[keySoftMax][c[2]] = dnSoftmax.item()
+
+    def inferLocal(self, keys=("softmax", "argmax"), Acc=None):
+        """
+        Infer local probabilities and information for given concepts and relations.
+
+        Args:
+            keys (tuple): Tuple containing the types of information to infer ('softmax', 'argmax', etc.).
+            Acc (dict, optional): A dictionary containing some form of accumulated data for normalization.
+
+        Attributes affected:
+            - This function manipulates the 'attributes' dictionary attribute of the class instance.
+
+        Notes:
+            - The method uses PyTorch for tensor operations.
+            - Logging is done to capture the time taken for inferring local probabilities.
+        """
+        startInferLocal = perf_counter()
+
+        # Go through keys and remove anything from each of them which is before slash, including slash
+        keys = [key[key.rfind('/')+1:] for key in keys]
+
+        conceptsRelations = self.collectConceptsAndRelations()
+
+        normalized_keys = set([
+                    "normalizedProb", "meanNormalizedProb",
+                    "normalizedProbAll", "meanNormalizedProbStd",
+                    "normalizedProbAcc", "entropyNormalizedProbAcc",
+                    "normalizedJustAcc",
+                    ])
+
+        if "softmax" in keys or normalized_keys.intersection(set(keys)):
+            needSoftmax = True
+        else:
+            needSoftmax = False
+
         for c in conceptsRelations:
             cRoot = self.findRootConceptOrRelation(c[0])
+            inferLocalKeys = list(keys) # used to check if all keys are calculated
+
+            # ----- skeleton - tensor
+            if getDnSkeletonMode() and "variableSet" in self.attributes:
+
+                vKeyInVariableSet = cRoot.name + "/<" + c[0].name +">"
+
+                if needSoftmax:
+                    localSoftmaxKeyInVariableSet = vKeyInVariableSet + "/local/softmax"
+
+                    if "softmax" in inferLocalKeys:
+                        inferLocalKeys.remove("softmax")
+
+                    if not self.hasAttribute(localSoftmaxKeyInVariableSet):
+                        v = self.attributes["variableSet"][vKeyInVariableSet]
+
+                        # check if v is None or not a tensor
+                        if v is None or not torch.is_tensor(v):
+                            continue
+
+                        if not(isinstance(v, torch.FloatTensor) or isinstance(v, torch.cuda.FloatTensor)):
+                            v = v.float()
+
+                        vSoftmaxT = torch.nn.functional.softmax(v, dim=-1)
+                        self.attributes["variableSet"][localSoftmaxKeyInVariableSet] = vSoftmaxT
+
+                if "argmax" in keys:
+                    localArgmaxKeyInVariableSet = vKeyInVariableSet + "/local/argmax"
+                    inferLocalKeys.remove("argmax")
+
+                    if not self.hasAttribute(localArgmaxKeyInVariableSet):
+                        v = self.attributes["variableSet"][vKeyInVariableSet]
+
+                        vArgmaxTInxexes = torch.argmax(v, dim=1)
+                        vArgmax = torch.zeros_like(v).scatter_(1, vArgmaxTInxexes.unsqueeze(1), 1.)
+
+                        self.attributes["variableSet"][localArgmaxKeyInVariableSet] = vArgmax
+
+            # check if we already processed all keys using skeleton
+            if not inferLocalKeys:
+                continue
+
+            # ---- loop through dns
             dns = self.findDatanodes(select = cRoot)
-            
             if not dns:
                 continue
-            
+
             vs = []
-            
+
             for dn in dns:
-                keySoftmax = "<" + c[0].name + ">/local/softmax"
-                if keySoftmax in dn.attributes:
-                    continue
-                
-                v = dn.getAttribute(c[0])
-                vSoftmaxT = torch.nn.functional.softmax(v, dim=-1)
-                
-                # Replace nan with 1/len
-                #for i, s in enumerate(vSoftmaxT):
-                #   if s != s:
-                #       vSoftmaxT[i] = 1/len(v)
-                
-                dn.attributes[keySoftmax] = vSoftmaxT
-                
-                keyArgmax  = "<" + c[0].name + ">/local/argmax"
-                vArgmax = torch.clone(v)
-                vArgmaxIndex = torch.argmax(v).item()
-                
-                for i, _ in enumerate(v):
-                    if i == vArgmaxIndex:
-                        vArgmax[i] = 1
-                    else:
-                        vArgmax[i] = 0
-                                
-                dn.attributes[keyArgmax] = vArgmax
-        
-    # Calculate ILP prediction for data graph with this instance as a root based on the provided list of concepts and relations
-    def inferILPResults(self, *_conceptsRelations, fun=None, epsilon = 0.00001, minimizeObjective = False, ignorePinLCs = False):
-        if len(_conceptsRelations) == 0:
-            _DataNode__Logger.info('Called with empty list of concepts and relations for inference')
+                if needSoftmax:
+                    keySoftmax = "<" + c[0].name + ">/local/softmax"
+
+                    if not dn.hasAttribute(keySoftmax):
+                        v = dn.getAttribute(c[0])
+
+                        # check if v is None or not a tensor
+                        if v is None or not torch.is_tensor(v):
+                            continue
+
+                        if not(isinstance(v, torch.FloatTensor) or isinstance(v, torch.cuda.FloatTensor)):
+                            v = v.float()
+
+                        vSoftmaxT = torch.nn.functional.softmax(v, dim=-1)
+                        dn.attributes[keySoftmax] = vSoftmaxT.squeeze(0)
+
+                if "normalizedProb" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/normalizedProb"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        # Clamps the softmax probabilities
+                        vector = torch.clamp(vSoftmaxT, min=1e-18, max=1 - 1e-18)
+
+                        # Calculates their entropy;
+                        entropy = torch.distributions.Categorical(torch.log(vector)).entropy() / vector.shape[0]
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = (1/entropy.item()) * (vector/torch.mean(vector))
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "normalizedProbAcc" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/normalizedProbAcc"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        # Clamps the softmax probabilities
+                        vector = torch.clamp(vSoftmaxT, min=1e-18, max=1 - 1e-18)
+
+                        ### Calculate the multiplier factor
+                        if Acc and c[0].name in Acc:
+                            multiplier = pow(Acc[c[0].name], 4)
+                        else:
+                            multiplier = 1
+
+                        # Calculates their entropy;
+                        entropy = torch.distributions.Categorical(torch.log(vector)).entropy() / vector.shape[0]
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = (1/entropy.item()) * (vector/torch.mean(vector))
+
+                        if multiplier != 1:
+                            vNormalizedProbT = vNormalizedProbT * multiplier
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "entropyNormalizedProbAcc" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/entropyNormalizedProbAcc"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        # Clamps the softmax probabilities
+                        vector = torch.clamp(vSoftmaxT, min=1e-18, max=1 - 1e-18)
+
+                        ### Calculate the multiplier factor
+                        if Acc and c[0].name in Acc:
+                            multiplier = pow(Acc[c[0].name], 4)
+                        else:
+                            multiplier = 1
+
+                        # Calculates their entropy;
+                        entropy = torch.distributions.Categorical(torch.log(vector)).entropy() / vector.shape[0]
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = (1/entropy.item()) * vector
+
+                        if multiplier != 1:
+                            vNormalizedProbT = vNormalizedProbT * multiplier
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "normalizedJustAcc" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/normalizedJustAcc"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        ### Calculate the multiplier factor
+                        if Acc and c[0].name in Acc:
+                            multiplier = pow(Acc[c[0].name], 8)
+                        else:
+                            multiplier = 1
+
+                        # Calculates their entropy;
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = vSoftmaxT
+
+                        if multiplier != 1:
+                            vNormalizedProbT = vNormalizedProbT * multiplier
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "meanNormalizedProb" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/meanNormalizedProb"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        vector = vSoftmaxT
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = vector/torch.mean(vector)
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "normalizedProbAll" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/normalizedProbAll"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        # Clamps the softmax probabilities
+                        vector = torch.clamp(vSoftmaxT, min=1e-18, max=1 - 1e-18)
+
+                        # Calculates their entropy;
+                        entropy = torch.distributions.Categorical(torch.log(vector)).entropy() / vector.shape[0]
+
+                        signs = vector - torch.mean(vector)
+                        signs[signs < 0] = -1
+                        signs[signs >= 0] = +1
+                        adjustment = signs * torch.pow(vector - torch.mean(vector), 4)
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = (1/entropy.item()) * (vector/torch.mean(vector)) + adjustment
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "meanNormalizedProbStd" in keys:
+                    keyNormalizedProb = "<" + c[0].name + ">/local/meanNormalizedProbStd"
+                    if not dn.hasAttribute(keyNormalizedProb): # Already calculated ?
+                        vSoftmaxT = dn.getAttribute(keySoftmax)
+
+                        vector = vSoftmaxT
+
+                        signs = vector - torch.mean(vector)
+                        signs[signs < 0] = -1
+                        signs[signs >= 0] = +1
+                        adjustment = signs * torch.pow(vector - torch.mean(vector), 2)
+
+                        # Multiplies the reverse of entropy to the vector divided by its mean value. P
+                        vNormalizedProbT = (adjustment/torch.pow(torch.mean(vector), 2))
+
+                        dn.attributes[keyNormalizedProb] = vNormalizedProbT
+
+                if "argmax" in keys:
+                    keyArgmax  = "<" + c[0].name + ">/local/argmax"
+                    if not dn.hasAttribute(keyArgmax):
+                        v = dn.getAttribute(c[0])
+                        vArgmax = torch.zeros(v.shape).squeeze(0)
+                        vArgmaxCalculated = torch.argmax(v, keepdim=True)
+                        vArgmaxIndex = torch.argmax(v).item()
+                        vArgmax[vArgmaxIndex] = 1
+
+                        dn.attributes[keyArgmax] = vArgmax
+
+        endInferLocal = perf_counter()
+        elapsedInferLocalInMs = (endInferLocal - startInferLocal) * 1000
+        self.myLoggerTime.info('Infer Local Probabilities - keys: %s, time: %dms', keys, elapsedInferLocalInMs)
+
+    def inferILPResults(self, *_conceptsRelations, key=("local", "softmax"), fun=None, epsilon=0.00001, minimizeObjective=False, ignorePinLCs=False, Acc=None):
+        """
+        Calculate ILP (Integer Linear Programming) prediction for a data graph using this instance as the root.
+        Based on the provided list of concepts and relations, it initiates ILP solving procedures.
+
+        Parameters:
+        - *_conceptsRelations: tuple
+            The concepts and relations used for inference.
+        - key: tuple, optional
+            The key to specify the inference method, default is ("local", "softmax").
+        - fun: function, optional
+            Additional function to be applied during ILP, default is None.
+        - epsilon: float, optional
+            The small value used for any needed approximations, default is 0.00001.
+        - minimizeObjective: bool, optional
+            Whether to minimize the objective function during ILP, default is False.
+        - ignorePinLCs: bool, optional
+            Whether to ignore pin constraints, default is False.
+        - Acc: object, optional
+            An accumulator for collecting results, default is None.
+
+        Raises:
+        - DataNodeError: When no concepts or relations are found for inference.
+
+        Returns:
+        - None: This function operates in-place and does not return a value.
+        """
+        if not _conceptsRelations:
+            _DataNode__Logger.info('Called with empty list of concepts and relations for inference.')
         else:
-            _DataNode__Logger.info('Called with - %s - list of concepts and relations for inference'%([x.name if isinstance(x, Concept) else x for x in _conceptsRelations]))
-            
-        # Check if concepts and/or relations have been provided for inference, if provide translate then to tuple concept info form
-        _conceptsRelations = self.collectConceptsAndRelations(_conceptsRelations) # Collect all concepts and relations from graph as default set
+            _DataNode__Logger.info(f'Called with the following list of concepts and relations for inference: {[x.name if isinstance(x, Concept) else x for x in _conceptsRelations]}')
 
-        if len(_conceptsRelations) == 0:
-            _DataNode__Logger.error('Not found any concepts or relations for inference in provided DataNode %s'%(self))
-            raise DataNode.DataNodeError('Not found any concepts or relations for inference in provided DataNode %s'%(self))
-        else:        
-            _DataNode__Logger.info('Found - %s - as a set of concepts and relations for inference'%([x[1] if isinstance(x, tuple) else x for x in _conceptsRelations]))
-                
-        myilpOntSolver, conceptsRelations = self.__getILPSolver(_conceptsRelations)
-        
-        # Call ilpOntsolver with the collected probabilities for chosen candidates
-        _DataNode__Logger.info("Calling ILP solver")
-        
-        self.inferLocal()
-        myilpOntSolver.calculateILPSelection(self, *conceptsRelations, fun=fun, epsilon = epsilon, minimizeObjective = minimizeObjective, ignorePinLCs = ignorePinLCs)    
-        
-    def inferGBIResults(self, *_conceptsRelations, model, builder):
+        # Check if a full data node is created; if not, create it as it's needed for ILP inference
+        if self.myBuilder:
+            self.myBuilder.createFullDataNode(self)
+
+        # Collect all relevant concepts and relations from the data graph
+        _conceptsRelations = self.collectConceptsAndRelations(_conceptsRelations)
+
+        if not _conceptsRelations:
+            _DataNode__Logger.error(f'No concepts or relations found for inference in the provided DataNode {self}.')
+            raise DataNode.DataNodeError(f'No concepts or relations found for inference in the provided DataNode {self}.')
+        else:
+            _DataNode__Logger.info(f'Found the following set of concepts and relations for inference: {[x[1] if isinstance(x, tuple) else x for x in _conceptsRelations]}')
+
+        myILPOntSolver, conceptsRelations = self.getILPSolver(_conceptsRelations)
+
+        _DataNode__Logger.info("Initiating ILP solver")
+
+        if "local" in key:
+            keys = (key[1],)
+            self.inferLocal(keys=keys, Acc=Acc)
+
+        startILPInfer = perf_counter()
+        if self.graph.batch and self.ontologyNode == self.graph.batch and 'contains' in self.relationLinks:
+            batchConcept = self.graph.batch
+            self.myLoggerTime.info(f'Batch processing ILP for {batchConcept}')
+
+            for batchIndex, dn in enumerate(self.relationLinks['contains']):
+                startILPBatchStepInfer = perf_counter()
+                myILPOntSolver.calculateILPSelection(dn, *conceptsRelations, key=key, fun=fun, epsilon=epsilon, minimizeObjective=minimizeObjective, ignorePinLCs=ignorePinLCs)
+                endILPBatchStepInfer = perf_counter()
+
+                elapsed = endILPBatchStepInfer - startILPBatchStepInfer
+                if elapsed > 1:
+                    self.myLoggerTime.info(f'Finished step {batchIndex} for batch ILP Inference - time: {elapsed:.2f}s')
+                else:
+                    self.myLoggerTime.info(f'Finished step {batchIndex} for batch ILP Inference - time: {elapsed*1000:.2f}ms')
+        else:
+            myILPOntSolver.calculateILPSelection(self, *conceptsRelations, key=key, fun=fun, epsilon=epsilon, minimizeObjective=minimizeObjective, ignorePinLCs=ignorePinLCs)
+
+        endILPInfer = perf_counter()
+
+        elapsed = endILPInfer - startILPInfer
+        if elapsed > 1:
+            self.myLoggerTime.info(f'Completed ILP Inference - total time: {elapsed:.2f}s')
+        else:
+            self.myLoggerTime.info(f'Completed ILP Inference - total time: {elapsed*1000:.2f}ms')
+
+        self.myLoggerTime.info('')
+
+    def inferGBIResults(self, *_conceptsRelations, model, kwargs):
+        """
+        Infer Grounded Belief Inference (GBI) results based on given concepts and relations.
+
+        Parameters:
+        - _conceptsRelations: tuple or list
+            Concepts and relations for which GBI is to be calculated. If empty, collects all from the graph.
+        - model: object
+            Solver model to be used in the GBI calculation.
+
+        Returns:
+        None. The function modifies the state of the `self.graph` object to store GBI results.
+
+        Logging:
+        - Logs whether the function was called with an empty or non-empty list of concepts and relations.
+        - Logs other debug and informational messages.
+
+        Side Effects:
+        - Modifies the state of the `self.graph` object to store GBI results.
+        """
         if len(_conceptsRelations) == 0:
             _DataNode__Logger.info('Called with empty list of concepts and relations for inference')
         else:
             _DataNode__Logger.info('Called with - %s - list of concepts and relations for inference'%([x.name if isinstance(x, Concept) else x for x in _conceptsRelations]))
-            
+
         # Check if concepts and/or relations have been provided for inference, if provide translate then to tuple concept info form
         _conceptsRelations = self.collectConceptsAndRelations(_conceptsRelations) # Collect all concepts and relations from graph as default set
 
         from domiknows.program.model.gbi import GBIModel
-        myGBIModel = GBIModel(self.graph, model)
-        myGBIModel.calculateGBISelection(builder, _conceptsRelations)
-        
-    # Calculate the percentage of results satisfying each logical constraint 
+        from inspect import signature
+        cmodelSignature = signature(GBIModel.__init__)
+
+        cmodelKwargs = {}
+        for param in cmodelSignature.parameters.values():
+            paramName = param.name
+            if paramName in kwargs:
+                cmodelKwargs[paramName] = kwargs[paramName]
+
+        myGBIModel = GBIModel(self.graph, solver_model=model, **cmodelKwargs)
+        myGBIModel.calculateGBISelection(self, _conceptsRelations)
+
     def verifyResultsLC(self, key = "/local/argmax"):
-                
-        myilpOntSolver, _ = self.__getILPSolver(conceptsRelations = self.collectConceptsAndRelations())
+        """
+        Verify the results of ILP (Integer Linear Programming) by checking the percentage of
+        results satisfying each logical constraint (LC).
+
+        Parameters:
+        - key: str, optional
+            Specifies the method used for verification. Supported keys are those containing "local" or "ILP".
+            Default is "/local/argmax".
+
+        Raises:
+        - DataNodeError: When an unsupported key is provided.
+
+        Returns:
+        - verifyResult: object
+            The result of the verification, typically a data structure containing percentages of
+            results that satisfy each logical constraint.
+        """
+        myilpOntSolver, _ = self.getILPSolver(conceptsRelations = self.collectConceptsAndRelations())
+
+        # Check if full data node is created and create it if not
+        if self.myBuilder != None:
+            self.myBuilder.createFullDataNode(self)
+
+        if "local" in key:
+            self.inferLocal(keys=[key])
+        elif "ILP" in key:
+            self.infer()
+        else:
+            _DataNode__Logger.error("Not supported key %s for verifyResultsLC"%(key))
 
-        self.inferLocal()
-        self.infer()
         verifyResult = myilpOntSolver.verifyResultsLC(self, key = key)
-        
+
         return verifyResult
-    
-    # T-norms: L - Lukasiewicz, G - Godel, P - Product
-    #tnorms = ['L', 'G', 'P']
-    tnormsDefault = 'P'
-    # sampleSize = -1 means Semantic Sample
-    def calculateLcLoss(self, tnorm=tnormsDefault, sample = False, sampleSize = 0, sampleGlobalLoss = False):
-        
-        myilpOntSolver, conceptsRelations = self.__getILPSolver(conceptsRelations = self.collectConceptsAndRelations())
+
+    def calculateLcLoss(self, tnorm='P',counting_tnorm=None, sample=False, sampleSize=0, sampleGlobalLoss=False):
+        """
+        Calculate the loss for logical constraints (LC) based on various t-norms.
+
+        Parameters:
+        - tnorm: str, optional
+            Specifies the t-norm used for calculations. Supported t-norms are 'L' (Lukasiewicz),
+            'G' (Godel), and 'P' (Product). Default is 'P'.
+        - sample: bool, optional
+            Specifies whether sampling is to be used. Default is False.
+        - sampleSize: int, optional
+            Specifies the sample size if sampling is enabled. A value of -1 indicates Semantic Sample.
+            Default is 0.
+        - sampleGlobalLoss: bool, optional
+            Specifies whether to calculate the global loss in case of sampling. Default is False.
+
+        Returns:
+        - lcResult: object
+            The calculated loss for logical constraints, typically a numeric value or data structure.
+
+        Raises:
+        - DataNodeError: When an unsupported tnorm is provided or other internal errors occur.
+        """
+        self.myBuilder.createFullDataNode(self)
+
+        myilpOntSolver, conceptsRelations = self.getILPSolver(conceptsRelations=self.collectConceptsAndRelations())
 
         self.inferLocal()
-        lcResult = myilpOntSolver.calculateLcLoss(self, tnorm = tnorm, sample = sample, 
-                                                  sampleSize = sampleSize, sampleGlobalLoss = sampleGlobalLoss, conceptsRelations = conceptsRelations)
-        
+        lcResult = myilpOntSolver.calculateLcLoss(self, tnorm=tnorm,counting_tnorm=counting_tnorm, sample=sample,
+                                                  sampleSize=sampleSize, sampleGlobalLoss=sampleGlobalLoss,
+                                                  conceptsRelations=conceptsRelations)
+
         return lcResult
 
+
     def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, average='binary'):
+        """
+        Calculate inference metrics for given concepts and relations.
+
+        Parameters:
+        - conceptsRelations: tuple or list
+            Concepts and relations for which metrics are to be calculated. If empty, it collects all.
+        - inferType: str, optional (default is 'ILP')
+            The inference type to use. Can be 'ILP' or other types supported.
+        - weight: torch.Tensor or None, optional
+            Weight tensor to be used in the calculation.
+        - average: str, optional (default is 'binary')
+            Type of average to be used in metrics calculation. Can be 'binary', 'micro', etc.
+
+        Returns:
+        - result: dict
+            Dictionary containing calculated metrics (TP, FP, TN, FN, P, R, F1) for each concept.
+
+        Logging:
+        - Various logs are printed for debugging and information.
+        """
         if not conceptsRelations:
             _DataNode__Logger.info("Calling %s metrics with empty conceptsRelations"%(inferType))
             conceptsRelations = self.collectConceptsAndRelations(conceptsRelations) # Collect all concepts and relations from graph as default set
             _DataNode__Logger.info("Found conceptsRelations in DataNode- %s"%(conceptsRelations))
         else:
             _DataNode__Logger.info("Calling %s metrics with conceptsRelations - %s"%(inferType, conceptsRelations))
-                
+
         weightOriginal = weight
         if weight is None:
             weight = torch.tensor(1)
         else:
             _DataNode__Logger.info("Using weight %s"%(weight))
-         
-        # Will store calculated metrics an related data   
-        result = {}   
-        tp, fp, tn, fn  = [], [], [], []  
-        isBinary = False  
+
+        # Will store calculated metrics an related data
+        result = {}
+        tp, fp, tn, fn  = [], [], [], []
+        isBinary = False
         isMulticlass = False
         isMulticlassLabel = False
-        
+
         # Calculate metrics for each provided concept
         for cr in conceptsRelations:
             # Check format of concepts and translate them to tuple in order to accommodate multiclass concepts
             if not isinstance(cr, tuple): # Not tuple concept form yet
                 if not isinstance(cr, Concept): # If string find the corresponding concept
                     cr = self.findConcept(cr)
-                    
+
                     if cr is None: # Sting mapping to concept is not found
                         _DataNode__Logger.error("% string is not a concept - not able to calculate metrics"%(cr))
                         continue
@@ -1151,7 +1988,7 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                 else:
                     _DataNode__Logger.error("% string is not a concept - not able to calculate metrics"%(cr))
                     continue
-            
+
             _DataNode__Logger.info("Calculating metrics for concept %s"%(cr[0]))
 
             # Collect date for metrics from DataNode
@@ -1170,28 +2007,28 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                 continue
             else:
                 _DataNode__Logger.info("Concept %s labels from DataNode %s"%(cr[1], labelsR))
-            
+
             if not torch.is_tensor(preds):
                 _DataNode__Logger.error("Concept %s labels is not a Tensor - not able to calculate metrics"%(cr[1]))
                 continue
-            
+
             if not torch.is_tensor(labelsR):
                 _DataNode__Logger.error("Concept %s predictions is not a Tensor - not able to calculate metrics"%(cr[1]))
                 continue
-            
+
             # Move to CPU
             if preds.is_cuda: preds = preds.cpu()
             if labelsR.is_cuda: labelsR = labelsR.cpu()
-            
+
             # Translate labels - if provided as True/False to long
             labels = torch.clone(labelsR)
             labels = labels.long()
             preds = preds.long()
-           
+
             # -- Multiclass processing
-            
+
             # Check if concept is a label from Multiclass
-            if cr[2] is not None: # Multiclass label given multiclass index (cr[2]) 
+            if cr[2] is not None: # Multiclass label given multiclass index (cr[2])
                 isMulticlassLabel = True
                 average = None
                 labelsList = [i for i in range(cr[3])]
@@ -1204,17 +2041,17 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                 if preds.shape[0] == len(labelsR):
                     predsOriginal = preds
                     preds = torch.nonzero(preds, as_tuple=True)[1]
-                    
+
                     if preds.shape[0] != len(labelsR):
                         _DataNode__Logger.warning("Concept %s predictions tensor has some predictions not calculated - %s"%(cr[1], predsOriginal))
-                    
+
                     _DataNode__Logger.info("Concept %s is Multiclass "%(cr[1]))
                     _DataNode__Logger.info("Using average %s for Multiclass metrics calculation"%(average))
 
                 else:
                     _DataNode__Logger.error("Incompatible lengths for %s between inferred results %s and labels %s"%(cr[1], len(preds), len(labelsR)))
                     continue
-                
+
                 _DataNode__Logger.info("Calculating metrics for all class Labels of  %s "%(cr[1]))
                 multiclassLabels = cr[0].enum
                 result = self.getInferMetrics(*multiclassLabels, inferType=inferType, weight = weightOriginal, average=average)
@@ -1223,34 +2060,34 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                 labelsList = None
 
             # ---
-            
+
             # Check if date prepared correctly
             if preds.dim() != 1:
                 _DataNode__Logger.error("Concept %s predictions is Tensor with dimension %s > 1- not able to calculate metrics"%(cr[1], preds.dim()))
                 continue
-            
+
             if labels.dim() != 1:
                 _DataNode__Logger.error("Concept %s labels is Tensor with dimension %s > 1- not able to calculate metrics"%(cr[1], labels.dim()))
                 continue
-            
+
             if  preds.size()[0] != labels.size()[0]:
                 _DataNode__Logger.error("Concept %s labels size %s is not equal to prediction size %s - not able to calculate metrics"%(cr[1], labels.size()[0], preds.size()[0]))
                 continue
-            
+
             # Prepare the metrics result storage
             result[cr[1]] = {'cr': cr, 'inferType' : inferType, 'TP': torch.tensor(0.), 'FP': torch.tensor(0.), 'TN': torch.tensor(0.), 'FN': torch.tensor(0.)}
-            
+
             # To numpy for sklearn
-            labels = labels.numpy() 
+            labels = labels.numpy()
             preds = preds.numpy()
-            
+
             import numpy as np
             if np.sum(labels) == 0:
                 _DataNode__Logger.warning("Concept %s - found all zero labels %s"%(cr[1], labels))
             else:
                 _DataNode__Logger.info("Concept %s - labels used for metrics calculation %s"%(cr[1], labels))
             result[cr[1]]['labels'] = labels
-            
+
             if np.sum(preds) == 0:
                 _DataNode__Logger.warning("Concept %s - found all zero predictions %s"%(cr[1], preds))
             else:
@@ -1260,33 +2097,33 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
             # Calculate confusion matrix
             try:
                 if isMulticlass:
-                    cm = metrics.confusion_matrix(labels, preds)
+                    cm = metrics.confusion_matrix(labels, preds, labels=labelsList)
                 elif isMulticlassLabel:
                     cm = metrics.multilabel_confusion_matrix(labels, preds, labels=labelsList)
                     cm = cm[cr[2]]
                 elif isBinary:
                     cm = metrics.confusion_matrix(labels, preds)
                     _tn, _fp, _fn, _tp = cm.ravel()
-        
-                    tp.append(_tp) 
-                    result[cr[1]]['TP'] = _tp # true positive 
-        
+
+                    tp.append(_tp)
+                    result[cr[1]]['TP'] = _tp # true positive
+
                     fp.append(_fp)
                     result[cr[1]]['FP'] = _fp # false positive
-        
+
                     tn.append(_tn)
                     result[cr[1]]['TN'] = _tn # true negative
-        
+
                     fn.append(_fn)
                     result[cr[1]]['FN'] = _fn # false positive
                 else:
                     pass
-                    
+
                 result[cr[1]]['confusion_matrix'] = cm
                 _DataNode__Logger.info("Concept %s confusion matrix %s"%(cr[1], result[cr[1]]['confusion_matrix']))
             except ValueError as ve: # Error when both labels and preds as zeros
                 _DataNode__Logger.warning("Concept %s - both labels and predictions are all zeros - not able to calculate confusion metrics"%(cr[1]))
-            
+
             # Calculate precision P - tp/(tp + fp)
             _p = metrics.precision_score(labels, preds, average=average, labels=labelsList, zero_division=0) # precision or positive predictive value (PPV)
             if isMulticlassLabel:
@@ -1306,7 +2143,7 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                 _DataNode__Logger.warning("Concept %s recall %s"%(cr[1], _r))
             else:
                 _DataNode__Logger.info("Concept %s recall %s"%(cr[1], _r))
-             
+
             # Calculate F1 score - (P X R)/(P + R)
             _f1 = metrics.f1_score(labels, preds, average=average, labels=labelsList, zero_division=0) # f1
             if isMulticlassLabel:
@@ -1319,31 +2156,31 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
 
         # --- Calculate Total metrics for binary concept
         if isBinary:
-            result['Total'] = {}  
+            result['Total'] = {}
             tpT = (torch.tensor(tp)).sum()
-            result['Total']['TP'] = tpT 
-            fpT = (torch.tensor(fp)).sum() 
+            result['Total']['TP'] = tpT
+            fpT = (torch.tensor(fp)).sum()
             result['Total']['FP'] = fpT
-            tnT = (torch.tensor(tn)).sum() 
+            tnT = (torch.tensor(tn)).sum()
             result['Total']['TN'] = tnT
-            fnT = (torch.tensor(fn)).sum() 
+            fnT = (torch.tensor(fn)).sum()
             result['Total']['FN'] = fnT
-            
+
             if tpT + fpT:
-                pT = tpT / (tpT + fpT)                
+                pT = tpT / (tpT + fpT)
                 result['Total']['P'] = pT
                 if pT == 0:
                     _DataNode__Logger.warning("Total precision is %s"%(pT))
                 else:
                     _DataNode__Logger.info("Total precision is %s"%(pT))
-                    
+
                 rT = tpT / (tpT + fnT)
                 result['Total']['R'] = rT
                 if rT == 0:
                     _DataNode__Logger.warning("Total recall is %s"%(rT))
                 else:
                     _DataNode__Logger.info("Total recall is %s"%(rT))
-                
+
                 if pT + rT:
                     f1T = 2 * pT * rT / (pT + rT)
                     result['Total']['F1'] = f1T
@@ -1351,7 +2188,7 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                         _DataNode__Logger.warning("Total F1 is %s"%(f1T))
                     else:
                         _DataNode__Logger.info("Total F1 is %s"%(f1T))
-                        
+
                 elif tpT + (fpT + fnT)/2:
                     f1T = tpT/(tpT + (fpT + fnT)/2)
                     result['Total']['F1'] = f1T
@@ -1360,91 +2197,176 @@ def getInferMetrics(self, *conceptsRelations, inferType='ILP', weight = None, av
                     else:
                         _DataNode__Logger.info("Total F1 is %s"%(f1T))
                 else:
-                    _DataNode__Logger.warning("No able to calculate F1 for Total") 
+                    _DataNode__Logger.warning("No able to calculate F1 for Total")
         else:
             result['Total'] = {"No Total metrics for multiclass concept"}
 
         return result
-    
+
 # Class constructing the data graph based on the sensors data during the model execution
 class DataNodeBuilder(dict):
-    
+    """
+    DataNodeBuilder class that extends Python's built-in dictionary.
+
+    Attributes:
+    - context (str): The context in which the DataNodeBuilder is being used, defaults to "build".
+    - myLoggerTime: Logger time instance for logging purposes.
+    - skeletonDataNode: Data structure for the basic DataNode skeleton.
+    - skeletonDataNodeFull: Data structure for the full DataNode skeleton.
+
+    Methods:
+    - __init__: Initializes the DataNodeBuilder instance.
+    - __getitem__: Overrides dict's __getitem__ to fetch item for a given key.
+    - __changToTuple: Converts list elements to tuple form for use as dictionary keys.
+    """
+
     context = "build"
-    def __init__(self, *args, **kwargs ):
-        dict.__init__(self, *args, **kwargs )
-        _DataNodeBulder__Logger.info("")
-        _DataNodeBulder__Logger.info("Called")
+
+    def __init__(self, *args, **kwargs):
+        """
+        Initialize the DataNodeBuilder instance.
+
+        Parameters:
+        - args: Positional arguments to pass to the dict constructor.
+        - kwargs: Keyword arguments to pass to the dict constructor.
+
+        Side Effects:
+        - Logs the instance creation.
+        - Initializes various instance variables.
+        """
+        dict.__init__(self, *args, **kwargs)
+        _DataNodeBuilder__Logger.info("")
+        _DataNodeBuilder__Logger.info("Called")
         self.myLoggerTime = getRegrTimer_logger()
-        
-        from domiknows.utils import getDnSkeletonMode
+
+        from domiknows.utils import getDnSkeletonMode, getDnSkeletonModeFull
         self.skeletonDataNode = getDnSkeletonMode()
-        dict.__setitem__(self, "DataNodesConcepts", set())
+        self.skeletonDataNodeFull = getDnSkeletonModeFull()
+
+        dict.__setitem__(self, "DataNodesConcepts", {})
+        dict.__setitem__(self, "KeysInOrder", [])
+
+        if args:
+            dict.__setitem__(self, "data_item", args[0])
 
     def __getitem__(self, key):
-        return dict.__getitem__(self, key)
+        """
+        Override dictionary's __getitem__ to fetch item for a given key.
+
+        Parameters:
+        - key: The key to look for in the dictionary.
 
+        Returns:
+        The value associated with the provided key.
+        """
+        return dict.__getitem__(self, key)
 
-    # Change elements of value to tuple if they are list - in order to use the value as dictionary key
     def __changToTuple(self, v):
+        """
+        Change elements of value to tuple if they are list, in order to use the value as dictionary keys.
+
+        Parameters:
+        - v: The value to be converted.
+
+        Returns:
+        The value converted to tuple form if it was a list; otherwise, the original value.
+        """
         if isinstance(v, list):
             _v = []
             for v1 in v:
                 _v.append(self.__changToTuple(v1))
-                
+
             return tuple(_v)
         else:
             return v
-        
-    # Add variable name to set
+
     def __addVariableNameToSet(self, vName):
+        """
+        Add a variable name to the internal 'variableSet'.
+
+        This method checks if 'variableSet' exists in the dictionary. If it does not exist,
+        it is created. The provided variable name is then added to this set.
+
+        Args:
+            vName (str): The variable name to add to the set.
+        """
         variableSetName = 'variableSet'
         if not dict.__contains__(self, variableSetName):
             dict.__setitem__(self, variableSetName, set())
-        
+
         variableSet = dict.__getitem__(self, variableSetName)
         variableSet.add(vName)
-        
-    # Add property name to set
+
     def __addPropertyNameToSet(self, pName):
+        """
+        Add a property name to the internal 'propertySet'.
+
+        This method checks if 'propertySet' exists in the dictionary. If it does not exist,
+        it is created. The provided property name is then added to this set.
+
+        Args:
+            pName (str): The property name to add to the set.
+        """
         propertySetName = 'propertySet'
         if not dict.__contains__(self, propertySetName):
             dict.__setitem__(self, propertySetName, set())
-        
-        variableSet = dict.__getitem__(self, propertySetName)
-        variableSet.add(pName)
-            
-    # Add or increase generic counter counting number of setitem calls
+
+        propertySet = dict.__getitem__(self, propertySetName)
+        propertySet.add(pName)
+
     def __addSetitemCounter(self):
-        globalCounterName = 'Counter' + '_setitem'
+        """
+        Add or increment a global counter for the number of '__setitem__' calls.
+
+        This method checks if a global counter (named 'Counter_setitem') exists in the dictionary.
+        If it does not exist, it is created and initialized with 1. If it exists, it is incremented by 1.
+        """
+        globalCounterName = 'Counter_setitem'
         if not dict.__contains__(self, globalCounterName):
             dict.__setitem__(self, globalCounterName, 1)
         else:
-            currentCounter =  dict.__getitem__(self, globalCounterName)
+            currentCounter = dict.__getitem__(self, globalCounterName)
             dict.__setitem__(self, globalCounterName, currentCounter + 1)
-            
-    # Add or increase sensor counter counting number of setitem calls with the given sensor key
+
     def __addSensorCounters(self, skey, value):
+        """
+        Add or increment a sensor-specific counter for the number of '__setitem__' calls
+        with the given sensor key and value.
+
+        This method constructs a unique counter name based on the sensor key ('skey').
+        If the counter doesn't exist, it's created and initialized with the given 'value'.
+        If the counter already exists, it's incremented.
+
+        A flag named 'recent' is also used to indicate whether the counter was recently incremented.
+
+        Args:
+            skey (list): The list of keys representing the sensor. Used to construct the unique counter name.
+            value (Any): The value to be counted. If it's a list, it will be converted to a tuple.
+
+        Returns:
+            bool: True if the counter for the given '_value' was recently incremented, False otherwise.
+        """
         _value = value
         if isinstance(value, list):
             _value = self.__changToTuple(_value)
-            
+
         counterNanme = 'Counter'
         for s in skey: # skey[2:]:
             counterNanme = counterNanme + '/' + s
-            
+
         if not dict.__contains__(self, counterNanme):
             try:
                 dict.__setitem__(self, counterNanme, {_value : {"counter": 1, "recent" : True}})
             except TypeError:
                 return False
-            
+
             return False
         else:
             currentCounter =  dict.__getitem__(self, counterNanme)
-            
+
             if _value in currentCounter:
-                currentCounter[_value]["counter"] = currentCounter[_value]["counter"] + 1 
-                
+                currentCounter[_value]["counter"] = currentCounter[_value]["counter"] + 1
+
                 if currentCounter[_value]["recent"]:
                     return True
                 else:
@@ -1452,425 +2374,577 @@ def __addSensorCounters(self, skey, value):
                     return False
             else:
                 currentCounter[_value]  = {"counter": 1, "recent" : True}
-                
+
                 return False
-            
-    # Find concept in the graph based on concept name
+
     def __findConcept(self, conceptName, usedGraph):
+        """
+        Search for a concept in the graph based on its name.
+
+        Args:
+            conceptName (str): The name of the concept to search for.
+            usedGraph (Graph object): The graph object where to look for the concept.
+
+        Returns:
+            Concept object: The concept object if found, otherwise None.
+        """
         subGraph_keys = [key for key in usedGraph._objs]
         for subGraphKey in subGraph_keys:
             subGraph = usedGraph._objs[subGraphKey]
-            
+
             for conceptNameItem in subGraph.concepts:
                 if conceptName == conceptNameItem:
                     concept = subGraph.concepts[conceptNameItem]
-                    
+
                     return concept
-        return None 
-        
-    # Collect concept information defined in the graph
+        return None
+
     def __findConceptInfo(self, usedGraph, concept):
-        conceptInfo = {}
-        
-        conceptInfo['concept'] = concept
-        
-        conceptInfo['relation'] = False
-        conceptInfo['relationAttrs'] = {}
-        for _, rel in enumerate(concept.has_a()): 
-            conceptInfo['relation'] = True
-            conceptName = rel.dst.name
-                            
-            conceptAttr = self.__findConcept(conceptName, usedGraph)
-
-            conceptInfo['relationAttrs'][rel.name] = conceptAttr
-            
-        conceptInfo['root'] = False
-        # Check if the concept is root concept 
-        if ('contains' not in concept._in):
-            conceptInfo['root'] = True
-          
-        conceptInfo['contains'] = []  
-        # Check if concept contains other concepts
-        if ('contains' in concept._out):
-            for _contain in concept._out['contains']:
-                conceptInfo['contains'].append(_contain.dst)
-                
-        conceptInfo['containedIn'] = []  
-        # Check if concept is contained in other concepts
-        if ('contains' in concept._in):
-            for _contain in concept._in['contains']:
-                conceptInfo['containedIn'].append(_contain.src)
-                           
+        """
+        Collects and returns information about a given concept as defined in the graph.
+
+        Args:
+            usedGraph (Graph object): The graph object where to look for the concept information.
+            concept (Concept object): The concept object for which information is to be collected.
+
+        Returns:
+            dict: A dictionary containing various pieces of information about the concept.
+                  - 'concept': The concept itself.
+                  - 'relation': A boolean indicating whether the concept has any relations.
+                  - 'relationAttrs': A dictionary mapping relation names to their corresponding concept objects.
+                  - 'root': A boolean indicating if the concept is a root concept.
+                  - 'contains': A list of concepts that this concept contains.
+                  - 'containedIn': A list of concepts in which this concept is contained.
+        """
+        conceptInfo = {
+            'concept': concept,
+            'relation': bool(concept.has_a()),
+            'relationAttrs': {rel.name: self.__findConcept(rel.dst.name, usedGraph) for _, rel in enumerate(concept.has_a())},
+            'root': not ('contains' in concept._in),
+            'contains': [contain.dst for contain in concept._out.get('contains', [])],
+            'containedIn': [contain.src for contain in concept._in.get('contains', [])]
+        }
+
         return conceptInfo
-            
+
     def __updateConceptInfo(self,  usedGraph, conceptInfo, sensor):
+        """
+        Updates concept information based on the given sensor.
+
+        Args:
+            usedGraph (Graph object): The graph object where to look for the concept.
+            conceptInfo (dict): The existing dictionary containing information about the concept.
+            sensor (Sensor object): The sensor object that is being processed.
+
+        Attributes Updated in conceptInfo dictionary:
+            - 'relationAttrData': A boolean indicating if the destination attribute is equal to the concept.
+            - 'label': A boolean indicating if the sensor has a label attribute and it is set.
+            - 'relationName': The name of the relation associated with the sensor.
+            - 'relationTypeName': The type name of the relation as a string.
+            - 'relationAttrsGraph': Copy of existing 'relationAttrs' if present.
+            - 'relationAttrs': A dictionary with updated source and destination attributes.
+            - 'relationMode': The mode of the relation.
+
+        Note:
+            - This method uses `EdgeSensor` from domiknows.sensor.pytorch.relation_sensors for certain operations.
+            - The method updates the 'conceptInfo' dictionary in-place.
+        """
         from domiknows.sensor.pytorch.relation_sensors import EdgeSensor
         conceptInfo["relationAttrData"] = False
         conceptInfo['label'] = False
-        if hasattr(sensor, 'label') and sensor.label: 
+        if hasattr(sensor, 'label') and sensor.label:
             conceptInfo['label'] = True
 
         if (isinstance(sensor, EdgeSensor)):
-            
+
             conceptInfo['relationName'] = sensor.relation.name
             conceptInfo['relationTypeName'] = str(type(sensor.relation))
-                    
+
             if 'relationAttrs' in conceptInfo:
                 conceptInfo['relationAttrsGraph'] = conceptInfo['relationAttrs']
-                
+
             conceptInfo['relationAttrs'] = {}
-          
+
             conceptInfo['relationMode'] = sensor.relation.mode
-            conceptInfo['relationAttrs']["src"] = self.__findConcept(sensor.src.name, usedGraph)  
-            conceptInfo['relationAttrs']["dst"] = self.__findConcept(sensor.dst.name, usedGraph)  
+            conceptInfo['relationAttrs']["src"] = self.__findConcept(sensor.src.name, usedGraph)
+            conceptInfo['relationAttrs']["dst"] = self.__findConcept(sensor.dst.name, usedGraph)
 
             if conceptInfo['relationAttrs']["dst"] == conceptInfo['concept']:
                 conceptInfo['relationAttrData'] = True
 
     def __isRootDn(self, testedDn, checkedDns, visitedDns):
+        """
+        Determine if a given DataNode (testedDn) is a root node in the graph based on its impactLinks.
+
+        Args:
+            testedDn (DataNode): The DataNode object that is being tested for its 'root' status.
+            checkedDns (set): A set of DataNodes that have already been examined or should be considered for this check.
+            visitedDns (set, optional): A set of DataNodes that have already been visited during recursion to avoid infinite loops.
+
+        Returns:
+            bool: Returns True if the testedDn is a root node, False otherwise.
+
+        Note:
+            - The method is recursive and visits each node only once to avoid infinite loops.
+            - 'impactLinks' is an attribute of DataNode that shows which DataNodes impact the current DataNode.
+        """
         if visitedDns == None:
             visitedDns = set()
-            
+
         visitedDns.add(testedDn)
-        
+
         if not testedDn.impactLinks and testedDn in checkedDns:
             return False
-        
-        isRoot = True    
+
+        isRoot = True
         for _, iDnList in testedDn.impactLinks.items(): # Check if its impacts are connected to Dn in the new Root list
             if iDnList:
                 for iDn in iDnList:
                     if iDn in visitedDns:
                         continue
-                    
+
                     if self.__isRootDn(iDn, checkedDns, visitedDns):
                         isRoot = False
                         break
-                    
+
             if not isRoot:
                 break
-            
+
         return isRoot
-    
+
     def __updateRootDataNodeList(self, *dns):
+        """
+        Update the list of root dataNodes in the dictionary based on newly added dataNodes and existing ones.
+
+        Args:
+            dns (tuple): A tuple containing the dataNodes to be added to the root list. It can contain nested lists.
+
+        Returns:
+            None: The function updates the list of root dataNodes in place and doesn't return any value.
+
+        Notes:
+            - The function first identifies existing root dataNodes and then updates this list based on the new ones.
+            - It uses the `impactLinks` attribute of dataNodes to determine whether a dataNode should be considered a root.
+            - If the 'dataNode' key doesn't exist in the dictionary, it will be added.
+        """
         if not dns:
             return
-    
+
         # Get existing roots dataNodes
         if dict.__contains__(self, 'dataNode'):
             dnsRoots = dict.__getitem__(self, 'dataNode')
-            _DataNodeBulder__Logger.debug('Existing elements in the root dataNodes list - %s'%(dnsRoots))
+            _DataNodeBuilder__Logger.debug('Existing elements in the root dataNodes list - %s'%(dnsRoots))
         else:
             dnsRoots = []
-         
-        newDnsRoots = []
-        
+
         # First flatten the list of new dataNodes
-        flattenDns = [] 
-        if isinstance(dns[0], list): 
-            for dList in dns:
-                flattenDns.extend(dList)
-                
-            if flattenDns and isinstance(flattenDns[0], list): 
-                _flattenDns = []
-                
-                for dList in flattenDns:
-                    _flattenDns.extend(dList)
-                    
-                flattenDns = _flattenDns  
-        else:
-            flattenDns = dns
-            
-        for fd in flattenDns:
-            if fd not in dnsRoots:
-                dnsRoots.append(fd)
-        
-        # Update list of existing root dataNotes     
-        for dnE in dnsRoots:            
-            if not dnE.impactLinks: #  Has not impact link
-                if dnE not in newDnsRoots: # Not yet in the new Root list
-                    newDnsRoots.append(dnE)  
+        def flatten(dns):
+            for dn in dns:
+                if isinstance(dn, list):
+                    yield from flatten(dn)
+                else:
+                    yield dn
+
+        # Flatten the list of new dataNodes
+        flattenDns = list(flatten(dns))
+
+        # Create a set of all unique dataNodes in dnsRoots and flattenDns
+        allDns = set(dnsRoots)
+        allDns.update(flattenDns)
+
+        # -- Update list of existing root dataNotes
+
+        # Will be used to store new root dataNodes
+        newDnsRoots = []
+
+        # Loop over all known unique dataNodes
+        #for dnE in allDns:
+        # Check if the dataNode is a root dataNode because it has no impact link
+        # if not dnE.impactLinks: # Has no impact link
+        #     if dnE not in newDnsRoots: # Not yet in the new Root list
+        #         # Add it to the new Root list
+        #         newDnsRoots.append(dnE)
+        # else:
+        #     # Check if the current dataNode is still a root dataNode
+        #     if self.__isRootDn(dnE, dnsRoots, visitedDns = None):
+        #         newDnsRoots.append(dnE)
+
+        # Count the number of incoming links for each dataNode
+        incomingLinks = {dn: 0 for dn in allDns}
+        dnTypes = {}
+        for dn in allDns:
+            if dn.ontologyNode in dnTypes:
+                dnTypes[dn.ontologyNode].append(dn)
             else:
-                if self.__isRootDn(dnE, dnsRoots, visitedDns = None):
-                    newDnsRoots.append(dnE)  
+                dnTypes[dn.ontologyNode] = [dn]
+
+            for il in dn.impactLinks:
+                if il in incomingLinks:
+                    incomingLinks[dn] += 1
+                else:
+                    incomingLinks[dn] = 1
+
+        # Find the root dataNodes which have no incoming links
+        newDnsRoots = [dn for dn in allDns if incomingLinks[dn] == 0 or not dn.impactLinks]
+        newDnsRoots = sorted(newDnsRoots, key=lambda dn: len(dnTypes[dn.ontologyNode]), reverse=False)
+
+        # if newDnsRoots is empty
+        if not newDnsRoots:
+            newDnsRoots = allDns
+            #newDnsRoots = sorted(newDnsRoots, key=lambda dn: incomingLinks[dn], reverse=True)
+            newDnsRoots = sorted(newDnsRoots, key=lambda dn: len(dnTypes[dn.ontologyNode]), reverse=False)
+
+        # Set the updated root list
+        if not getProductionModeStatus():
+            _DataNodeBuilder__Logger.info('Updated elements in the root dataNodes list - %s'%(newDnsRoots))
+        dict.__setitem__(self, 'dataNode', newDnsRoots) # Updated the dict
 
-        # Set the updated root list 
-        _DataNodeBulder__Logger.info('Updated elements in the root dataNodes list - %s'%(newDnsRoots))
-        dict.__setitem__(self, 'dataNode', newDnsRoots) # Updated the dict 
-    
         return
-    
-    # Build or update relation dataNode in the data graph for a given key
+
     def __buildRelationLink(self, vInfo, conceptInfo, keyDataName):
+        """
+        Build or update relation dataNode in the data graph for a given key.
+
+        Args:
+            vInfo (object): Holds information about the value (e.g., tensor details).
+            conceptInfo (dict): Information about the concept the dataNode represents.
+            keyDataName (str): The key name for the attribute in question.
+
+        Returns:
+            None: The method updates the data graph in-place.
+        """
         relationName = conceptInfo['concept'].name
-         
+
         # Check if data graph started
         existingRootDns = dict.__getitem__(self, 'dataNode') # DataNodes roots
-        
+
         if not existingRootDns:
-            _DataNodeBulder__Logger.error('No dataNode created yet - abandon processing relation link dataNode value for %s and attribute %s'%(relationName,keyDataName))
+            _DataNodeBuilder__Logger.error('No dataNode created yet - abandon processing relation link dataNode value for %s and attribute %s'%(relationName,keyDataName))
             return # No graph yet - information about relation should not be provided yet
-        
-        # Find if DatnNodes for this relation have been created
+
+        # Find if DataNodes for this relation have been created
         existingDnsForRelation = self.findDataNodesInBuilder(select = relationName)
-        
+
         existingDnsForRelationNotSorted = OrderedDict()
         for dn in existingDnsForRelation:
             existingDnsForRelationNotSorted[dn.getInstanceID()] = dn
-                
+
         existingDnsForRelationSorted = OrderedDict(sorted(existingDnsForRelationNotSorted.items()))
-            
+
         # This is an information about relation attributes
         if conceptInfo['relationAttrData']:
             index = keyDataName.index('.')
             attrName = keyDataName[0:index]
-            
+
             relationAttrsCacheName = conceptInfo['concept'].name + "RelationAttrsCache"
-            
+
             if not dict.__contains__(self, relationAttrsCacheName):
                 dict.__setitem__(self, relationAttrsCacheName, {})
-        
+
             relationAttrsCache =  dict.__getitem__(self, relationAttrsCacheName)
             relationAttrsCache[attrName] = vInfo.value
-                
-            _DataNodeBulder__Logger.info('Caching received data for %s related to relation %s dataNode, found %i existing dataNode of this type - provided value has length %i'
+
+            if not getProductionModeStatus():
+                _DataNodeBuilder__Logger.info('Caching received data for %s related to relation %s dataNode, found %i existing dataNode of this type - provided value has length %i'
                                          %(keyDataName,relationName,len(existingDnsForRelation),vInfo.len))
-            
+
             # Find if all the needed attribute were initialized
             allAttrInit = True
             for relationAttributeName, _ in conceptInfo['relationAttrsGraph'].items():
                 if relationAttributeName not in relationAttrsCache:
                     allAttrInit = False
                     break
-            
+
             if allAttrInit: #Create links for the relation DataNode
                 # Find DataNodes connected by this relation based on graph definition
                 existingDnsForAttr = OrderedDict() # DataNodes for Attributes of the relation
                 for relationAttributeName, relationAttributeConcept in conceptInfo['relationAttrsGraph'].items():
                     _existingDnsForAttr = self.findDataNodesInBuilder(select = relationAttributeConcept.name)
-                     
+
                     if _existingDnsForAttr:
                         existingDnsForAttr[relationAttributeName] = _existingDnsForAttr
-                        _DataNodeBulder__Logger.info('Found %i dataNodes of the attribute %s for concept %s'%(len(_existingDnsForAttr),relationAttributeName,relationAttributeConcept.name))
+                        if not getProductionModeStatus():
+                            _DataNodeBuilder__Logger.info('Found %i dataNodes of the attribute %s for concept %s'%(len(_existingDnsForAttr),relationAttributeName,relationAttributeConcept.name))
                     else:
                         existingDnsForAttr[relationAttributeName] = []
-                        _DataNodeBulder__Logger.warning('Not found dataNodes of the attribute %s for concept %s'%(relationAttributeName,relationAttributeConcept.name))
-                                    
+                        _DataNodeBuilder__Logger.warning('Not found dataNodes of the attribute %s for concept %s'%(relationAttributeName,relationAttributeConcept.name))
+
                 attributeNames = [*existingDnsForAttr]
-                
+
                 # Create links between this relation and instance dataNode based on the candidate information provided by sensor for each relation attribute
                 for relationDnIndex, relationDn in existingDnsForRelationSorted.items():
                     for attributeIndex, attribute in enumerate(attributeNames):
-                          
                         candidatesForRelation = relationAttrsCache[attribute][relationDnIndex]
-                        
+
                         for candidateIndex, candidate in enumerate(candidatesForRelation):
                             isInRelation = candidate.item()
                             if isInRelation == 0:
                                 continue
-                            
+
                             candidateDn = existingDnsForAttr[attribute][candidateIndex]
-                            
-                            if attributeIndex == 0:
-                                candidateDn.addRelationLink(attribute, relationDn)
-                            
-                            relationDn.addRelationLink(attribute, candidateDn)  
-                            relationDn.attributes[keyDataName] = vInfo.value[relationDnIndex] # Add / /Update value of the attribute
-                
-                _DataNodeBulder__Logger.info('Create links between the relation %s and instance dataNode of types'%(conceptInfo['concept'].name))
+
+                            #if attributeIndex == 0:
+                            #   candidateDn.addRelationLink(attribute, relationDn)
+
+                            relationDn.addRelationLink(attribute, candidateDn)
+                            if (not self.skeletonDataNode):
+                                relationDn.attributes[keyDataName] = vInfo.value[relationDnIndex] # Add / /Update value of the attribute
+
+                if not getProductionModeStatus():
+                    _DataNodeBuilder__Logger.info('Create links between the relation %s and instance dataNode of types'%(conceptInfo['concept'].name))
             else:
                 # Just add the sensor value to relation DataNodes
-                if keyDataName in self:
-                    _DataNodeBulder__Logger.info('Updating attribute %s in relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
-                else:
-                    _DataNodeBulder__Logger.info('Adding attribute %s to relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
-                    
-                for i, rDn in existingDnsForRelationSorted.items(): # Loop through all relation links dataNodes
-                    rDn.attributes[keyDataName] = vInfo.value[i] # Add / /Update value of the attribute
+                if not getProductionModeStatus():
+                    if keyDataName in self:
+                        _DataNodeBuilder__Logger.info('Updating attribute %s in relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
+                    else:
+                        _DataNodeBuilder__Logger.info('Adding attribute %s to relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
+
+                if (not self.skeletonDataNode):
+                    for i, rDn in existingDnsForRelationSorted.items(): # Loop through all relation links dataNodes
+                        rDn.attributes[keyDataName] = vInfo.value[i] # Add / /Update value of the attribute
 
             self.__updateRootDataNodeList(list(existingDnsForRelationSorted.values()))
-        else:    
+        else:
             # -- DataNode with this relation already created  - update it with new attribute value
-            if keyDataName in self:
-                _DataNodeBulder__Logger.info('Updating attribute %s in relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
-            else:
-                _DataNodeBulder__Logger.info('Adding attribute %s to relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
- 
+            if not getProductionModeStatus():
+                if keyDataName in self:
+                    _DataNodeBuilder__Logger.info('Updating attribute %s in relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
+                else:
+                    _DataNodeBuilder__Logger.info('Adding attribute %s to relation link dataNodes %s'%(keyDataName,conceptInfo['concept'].name))
+
             if len(existingDnsForRelation) != vInfo.len:
-                _DataNodeBulder__Logger.error('Number of relations is %i and is different then the length of the provided tensor %i'%(len(existingDnsForRelation),vInfo.len))
+                _DataNodeBuilder__Logger.error('Number of relations is %i and is different then the length of the provided tensor %i'%(len(existingDnsForRelation),vInfo.len))
                 raise ValueError('Number of relations is %i and is different then the length of the provided tensor %i'%(len(existingDnsForRelation),vInfo.len))
- 
-            if len(existingDnsForRelationSorted) == 1:
-                if vInfo.dim == 0:
-                    existingDnsForRelationSorted[0].attributes[keyDataName] = vInfo.value # Add / /Update value of the attribute
-            elif vInfo.dim > 0:
-                for i, rDn in existingDnsForRelationSorted.items(): # Loop through all relations links dataNodes
-                    rDn.attributes[keyDataName] = vInfo.value[i] # Add / /Update value of the attribute
-            else:
-                pass
 
-    def __createInitialdDataNode(self, vInfo, conceptInfo, keyDataName):
+            if (not self.skeletonDataNode):
+                if len(existingDnsForRelationSorted) == 1:
+                    if vInfo.dim == 0:
+                        existingDnsForRelationSorted[0].attributes[keyDataName] = vInfo.value # Add / /Update value of the attribute
+                elif vInfo.dim > 0:
+                    for i, rDn in existingDnsForRelationSorted.items(): # Loop through all relations links dataNodes
+                        rDn.attributes[keyDataName] = vInfo.value[i] # Add / /Update value of the attribute
+                else:
+                    pass
+
+    def __createInitialDataNode(self, vInfo, conceptInfo, keyDataName):
+        """
+        Create initial data nodes for the data graph.
+
+        Args:
+            vInfo (object): Contains information about the value, like its length.
+            conceptInfo (dict): Information about the concept associated with the data node.
+            keyDataName (str): The name of the key for which the data node is being created.
+
+        Returns:
+            list: A list of created DataNode objects.
+        """
         conceptName = conceptInfo['concept'].name
 
         dns = []
-                   
-        _DataNodeBulder__Logger.info('Creating initial dataNode - provided value has length %i'%(vInfo.len))
+
+        if not getProductionModeStatus():
+            _DataNodeBuilder__Logger.info('Creating initial dataNode - provided value has length %i'%(vInfo.len))
 
         if vInfo.len == 1: # Will use "READER" key as an id of the root dataNode
             instanceValue = ""
-            
+
             if "READER" in self:
                 instanceID = dict.__getitem__(self, "READER")
             else:
                 instanceID = 0
-                
-            _dn = DataNode(instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
-            
-            _dn.attributes[keyDataName] = vInfo.value
-            
-            _DataNodeBulder__Logger.info('Created single dataNode with id %s of type %s'%(instanceID,conceptName))
-            dns.append(_dn)
+
+            initialDn = DataNode(myBuilder = self, instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
+
+            if (not self.skeletonDataNode):
+                initialDn.attributes[keyDataName] = vInfo.value
+
+            if not getProductionModeStatus():
+                _DataNodeBuilder__Logger.info('Created single dataNode with id %s of type %s'%(instanceID,conceptName))
+            dns.append(initialDn)
         elif vInfo.len > 1:
             for vIndex, v in enumerate(vInfo.value):
                 instanceValue = ""
                 instanceID = vIndex
-                _dn = DataNode(instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
-                
-                _dn.attributes[keyDataName] = v
-                
-                dns.append(_dn)
-                        
-            _DataNodeBulder__Logger.info('Created %i dataNodes of type %s'%(len(dns),conceptName))
-                    
+                newInitialDn = DataNode(myBuilder = self, instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
+
+                if (not self.skeletonDataNode):
+                    newInitialDn.attributes[keyDataName] = v
+
+                dns.append(newInitialDn)
+
+            if not getProductionModeStatus():
+                _DataNodeBuilder__Logger.info('Created %i dataNodes of type %s'%(len(dns),conceptName))
+
         self.__updateRootDataNodeList(dns)
-        
+
         return dns
-    
+
     def __createSingleDataNode(self, vInfo, conceptInfo, keyDataName):
+        """
+        Create initial data nodes for the data graph.
+
+        Args:
+            vInfo (object): Contains information about the value, like its length.
+            conceptInfo (dict): Information about the concept associated with the data node.
+            keyDataName (str): The name of the key for which the data node is being created.
+
+        Returns:
+            list: A list of created DataNode objects.
+        """
         conceptName = conceptInfo['concept'].name
-        _DataNodeBulder__Logger.info('Received information about dataNodes of type %s - value dim is %i and length is %i'%(conceptName,vInfo.dim,vInfo.len))
+        if not getProductionModeStatus():
+            _DataNodeBuilder__Logger.info('Received information about dataNodes of type %s - value dim is %i and length is %i'%(conceptName,vInfo.dim,vInfo.len))
 
-        # -- Create a single the new dataNode 
+        # -- Create a single the new dataNode
         instanceValue = ""
         instanceID = 0
-        _dn = DataNode(instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
-        _dn.attributes[keyDataName] = vInfo.value
-                
-        _DataNodeBulder__Logger.info('Single new dataNode %s created'%(_dn))
-
-        self.__updateRootDataNodeList(_dn)
-                
-        return [_dn]
-        
+        newSingleDn = DataNode(myBuilder = self, instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
+        if (not self.skeletonDataNode):
+            newSingleDn.attributes[keyDataName] = vInfo.value
+
+        if not getProductionModeStatus():
+            _DataNodeBuilder__Logger.info('Single new dataNode %s created'%(newSingleDn))
+
+        self.__updateRootDataNodeList(newSingleDn)
+
+        return [newSingleDn]
+
     def __createMultiplyDataNode(self, vInfo, conceptInfo, keyDataName):
+        """
+        Create multiple data nodes based on various conditions.
+
+        Args:
+            vInfo (object): Information about the value, like its dimension and length.
+            conceptInfo (dict): Information about the concept associated with the data nodes.
+            keyDataName (str): The name of the key for which the data node is being created.
+
+        Returns:
+            list: A list of the created DataNode objects.
+        """
         conceptName = conceptInfo['concept'].name
-        
-        # Master List of lists of created dataNodes - each list in the master list represent set of new dataNodes connected to the same parent dataNode 
+
+        # Master List of lists of created dataNodes - each list in the master list represent set of new dataNodes connected to the same parent dataNode
         # (identified by the index in the master list)
-        dns = [] 
-                
-        _DataNodeBulder__Logger.info('Received information about dataNodes of type %s - value dim is %i and length is %i'%(conceptName,vInfo.dim,vInfo.len))
+        dns = []
+
+        if not getProductionModeStatus():
+            _DataNodeBuilder__Logger.info('Received information about dataNodes of type %s - value dim is %i and length is %i'%(conceptName,vInfo.dim,vInfo.len))
 
         # --- Create dataNodes
-        
+
         # Check the type of sensor data
-        if vInfo.dim == 0: 
-            _DataNodeBulder__Logger.warning('Provided value is empty %s - abandon the update'%(vInfo.value))
+        if vInfo.dim == 0:
+            _DataNodeBuilder__Logger.warning('Provided value is empty %s - abandon the update'%(vInfo.value))
             return
         elif vInfo.dim == 1: # List with indexes for new DataNodes and data for attribute
-            _DataNodeBulder__Logger.info('Adding %i new dataNodes of type %s'%(vInfo.len,conceptName))
+            if not getProductionModeStatus():
+                _DataNodeBuilder__Logger.info('Adding %i new dataNodes of type %s'%(vInfo.len,conceptName))
 
-            dns1 = []
             for vIndex, v in enumerate(vInfo.value):
                 instanceValue = ""
                 instanceID = vIndex
-                
+
                 # Create new DataNode
-                _dn = DataNode(instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
-                
+                newDn = DataNode(myBuilder = self, instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
+
                 # add attribute
-                _dn.attributes[keyDataName] = v
-                
-                dns1.append(_dn)
-                      
-            # Single list of new DateNodes              
-            dns.append(dns1)              
+                if (not self.skeletonDataNode):
+                    newDn.attributes[keyDataName] = v
+
+                dns.append(newDn)
         elif vInfo.dim == 2: # Two dimensional relation information
             if "relationMode" in conceptInfo:
                 relatedDnsType = conceptInfo["relationAttrs"]['src']
                 relatedDns = self.findDataNodesInBuilder(select = relatedDnsType)
-                
+
                 if len(vInfo.value) > 0:
                     try:
-                        requiredLenOFReltedDns = len(vInfo.value[0])
+                        requiredLenOFRelatedDns = len(vInfo.value[0])
                     except IndexError:
-                        requiredLenOFReltedDns = 0
+                        requiredLenOFRelatedDns = 0
                 else:
-                    requiredLenOFReltedDns = 0
-                    
-                if requiredLenOFReltedDns != len(relatedDns):
-                    _DataNodeBulder__Logger.warning('Value of %s expects %i related dataNode of type %s but the number of existing dataNodes is %i - abandon the update'
-                                                    %(conceptInfo['relationName'],requiredLenOFReltedDns,relatedDnsType,len(relatedDns)))
+                    requiredLenOFRelatedDns = 0
+
+                if requiredLenOFRelatedDns != len(relatedDns):
+                    _DataNodeBuilder__Logger.warning('Value of %s expects %i related dataNode of type %s but the number of existing dataNodes is %i - abandon the update'
+                                                    %(conceptInfo['relationName'],requiredLenOFRelatedDns,relatedDnsType,len(relatedDns)))
                     return
-           
-                _DataNodeBulder__Logger.info('Create %i new dataNodes of type %s'%(vInfo.len,conceptName))
-                
-                if not conceptInfo['relation']:
-                    _DataNodeBulder__Logger.info('It is a contain update of type - %s'%(conceptInfo["relationMode"]))
-                    if conceptInfo["relationMode"] == "forward":
-                        _DataNodeBulder__Logger.info('%s is contain in %s'%(conceptName, relatedDnsType))
-                    else:
-                        _DataNodeBulder__Logger.info('%s is contain in %s'%(relatedDnsType, conceptName))
+
+                if not getProductionModeStatus():
+                    _DataNodeBuilder__Logger.info('Create %i new dataNodes of type %s'%(vInfo.len,conceptName))
+
+                    if not conceptInfo['relation']:
+                        _DataNodeBuilder__Logger.info('It is a contain update of type - %s'%(conceptInfo["relationMode"]))
+                        if conceptInfo["relationMode"] == "forward":
+                            _DataNodeBuilder__Logger.info('%s is contain in %s'%(conceptName, relatedDnsType))
+                        else:
+                            _DataNodeBuilder__Logger.info('%s is contain in %s'%(relatedDnsType, conceptName))
 
                 for i in range(0,vInfo.len):
                     instanceValue = ""
                     instanceID = i
-                    _dn = DataNode(instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
-                        
-                    _dn.attributes[keyDataName] = vInfo.value[i]
-                    dns.append(_dn)
-                    
+                    newDn = DataNode(myBuilder = self, instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
+
+                    if (not self.skeletonDataNode):
+                        newDn.attributes[keyDataName] = vInfo.value[i]
+                    dns.append(newDn)
+
                     # If it is not a regular relation but (Create contain relation between the new DataNode and existing DataNodes
                     if not conceptInfo['relation']:
                         if conceptInfo["relationMode"] == "forward":
                             for index, isRelated in enumerate(vInfo.value[i]):
                                 if isRelated == 1:
-                                    relatedDns[index].addChildDataNode(_dn)                            
+                                    relatedDns[index].addChildDataNode(newDn)
                         elif conceptInfo["relationMode"] == "backward":
                             for index, isRelated in enumerate(vInfo.value[i]):
                                 if isRelated == 1:
-                                    _dn.addChildDataNode(relatedDns[index])  
+                                    newDn.addChildDataNode(relatedDns[index])
             else:
-                _DataNodeBulder__Logger.info('Create %i new dataNodes of type %s'%(vInfo.len,conceptName))
+                if not getProductionModeStatus():
+                    _DataNodeBuilder__Logger.info('Create %i new dataNodes of type %s'%(vInfo.len,conceptName))
                 for i in range(0,vInfo.len):
                     instanceValue = ""
                     instanceID = i
-                    _dn = DataNode(instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
-                        
-                    dns.append(_dn)
+                    newDn = DataNode(myBuilder = self, instanceID = instanceID, instanceValue = instanceValue, ontologyNode = conceptInfo['concept'])
+
+                    dns.append(newDn)
         else:
-            _DataNodeBulder__Logger.warning('It is an unsupported sensor input - %s'%(vInfo))
-                
-        self.__updateRootDataNodeList(dns)   
+            _DataNodeBuilder__Logger.warning('It is an unsupported sensor input - %s'%(vInfo))
+
+        self.__updateRootDataNodeList(dns)
         return dns
-            
+
     def __updateDataNodes(self, vInfo, conceptInfo, keyDataName):
+        """
+        Update existing data nodes based on various conditions.
+
+        Notes:
+        - This function is not called when `skeletonDataNode` is on.
+
+        Args:
+            vInfo (object): Information about the value, like its dimension and length.
+            conceptInfo (dict): Information about the concept associated with the data nodes.
+            keyDataName (str): The name of the key for which the data node is being updated.
+
+        """
         conceptName = conceptInfo['concept'].name
         existingDnsForConcept = self.findDataNodesInBuilder(select = conceptName) # Try to get DataNodes of the current concept
 
         if not existingDnsForConcept:
             existingDnsForConcept = self.findDataNodesInBuilder(select = conceptName)
-            
+
         if not existingDnsForConcept:
             return
-                                                                    
-        if keyDataName in existingDnsForConcept[0].attributes:
-            _DataNodeBulder__Logger.info('Updating attribute %s in existing dataNodes - found %i dataNodes of type %s'%(keyDataName, len(existingDnsForConcept),conceptName))
-        else:
-            _DataNodeBulder__Logger.info('Adding attribute %s in existing dataNodes - found %i dataNodes of type %s'%(keyDataName, len(existingDnsForConcept),conceptName))
-        
-        if len(existingDnsForConcept) > vInfo.len: # Not enough elements in the value 
-            _DataNodeBulder__Logger.warning('Provided value has length %i but found %i existing dataNode - abandon the update'%(vInfo.len,len(existingDnsForConcept)))
+
+        if not getProductionModeStatus():
+            if keyDataName in existingDnsForConcept[0].attributes:
+                _DataNodeBuilder__Logger.info('Updating attribute %s in existing dataNodes - found %i dataNodes of type %s'%(keyDataName, len(existingDnsForConcept),conceptName))
+            else:
+                _DataNodeBuilder__Logger.info('Adding attribute %s in existing dataNodes - found %i dataNodes of type %s'%(keyDataName, len(existingDnsForConcept),conceptName))
+
+        if len(existingDnsForConcept) > vInfo.len: # Not enough elements in the value
+            _DataNodeBuilder__Logger.warning('Provided value has length %i but found %i existing dataNode - abandon the update'%(vInfo.len,len(existingDnsForConcept)))
         elif len(existingDnsForConcept) == vInfo.len: # Number of  value elements matches the number of found dataNodes
             if len(existingDnsForConcept) == 0:
                 return
@@ -1887,16 +2961,16 @@ def __updateDataNodes(self, vInfo, conceptInfo, keyDataName):
                     if isinstance(existingDnsForConcept[vIndex], DataNode): # Check if DataNode
                         existingDnsForConcept[vIndex].attributes[keyDataName] = v
                     else:
-                        _DataNodeBulder__Logger.error('Element %i in the list is not a dataNode - skipping it'%(vIndex))
+                        _DataNodeBuilder__Logger.error('Element %i in the list is not a dataNode - skipping it'%(vIndex))
                         raise ValueError('Element %i in the list is not a dataNode - skipping it'%(vIndex))
-        
+
                 if keyDataName[0] == '<' and keyDataName[-1] == '>':
                     if "contains" in existingDnsForConcept[0].impactLinks:
                         dnParent = existingDnsForConcept[0].impactLinks["contains"][0]
                         dnParent.attributes[keyDataName] = vInfo.value
         elif len(existingDnsForConcept) < vInfo.len: # Too many elements in the value
-            _DataNodeBulder__Logger.warning('Provided value has length %i but found %i existing dataNode - abandon the update'%(vInfo.len,len(existingDnsForConcept)))
-            
+            _DataNodeBuilder__Logger.warning('Provided value has length %i but found %i existing dataNode - abandon the update'%(vInfo.len,len(existingDnsForConcept)))
+
         # Check if this is the contain relation update or attribute update
         if "relationMode" in conceptInfo and  not conceptInfo["relation"]:
             relatedDnsType = conceptInfo["relationAttrs"]['src']
@@ -1904,94 +2978,144 @@ def __updateDataNodes(self, vInfo, conceptInfo, keyDataName):
             relatedDns = self.findDataNodesInBuilder(select = relatedDnsType)
 
             if vInfo.dim:
-                requiredLenOFReltedDns = len(vInfo.value[0])
+                requiredLenOFRelatedDns = len(vInfo.value[0])
             else:
-                requiredLenOFReltedDns = len(vInfo.item())
-            
-            if requiredLenOFReltedDns != len(relatedDns):
-                _DataNodeBulder__Logger.error('Provided value expected %i related dataNode of type %s but the number of existing dataNodes is %i - abandon the update'
-                                              %(requiredLenOFReltedDns,relatedDnsType,len(relatedDns)))
+                requiredLenOFRelatedDns = len(vInfo.item())
+
+            if requiredLenOFRelatedDns != len(relatedDns):
+                _DataNodeBuilder__Logger.error('Provided value expected %i related dataNode of type %s but the number of existing dataNodes is %i - abandon the update'
+                                              %(requiredLenOFRelatedDns,relatedDnsType,len(relatedDns)))
                 raise ValueError('Provided value expected %i related dataNode of type %s but the number of existing dataNodes is %i - abandon the update'
-                                              %(requiredLenOFReltedDns,relatedDnsType,len(relatedDns)))
+                                              %(requiredLenOFRelatedDns,relatedDnsType,len(relatedDns)))
+
+
+            if not getProductionModeStatus():
+                _DataNodeBuilder__Logger.info('It is a contain update of type - %s'%(conceptInfo["relationMode"]))
+                if conceptInfo["relationMode"] == "forward":
+                    _DataNodeBuilder__Logger.info('%s is contain in %s'%(conceptName, relatedDnsType))
+                else:
+                    _DataNodeBuilder__Logger.info('%s is contain in %s'%(relatedDnsType, conceptName))
 
-                
-            _DataNodeBulder__Logger.info('It is a contain update of type - %s'%(conceptInfo["relationMode"]))
-            if conceptInfo["relationMode"] == "forward":
-                _DataNodeBulder__Logger.info('%s is contain in %s'%(conceptName, relatedDnsType))
-            else:
-                _DataNodeBulder__Logger.info('%s is contain in %s'%(relatedDnsType, conceptName))
-                
             for i in range(0,vInfo.len):
-                _dn = existingDnsForConcept[i]
-                    
+                exitingDn = existingDnsForConcept[i]
+
                 if conceptInfo["relationMode"] == "forward":
                     for index, isRelated in enumerate(vInfo.value[i]):
                         if isRelated == 1:
-                            relatedDns[index].addChildDataNode(_dn)                            
+                            relatedDns[index].addChildDataNode(exitingDn)
                 elif conceptInfo["relationMode"] == "backward":
                     for index, isRelated in enumerate(vInfo.value[i]):
                         if isRelated == 1:
-                            _dn.addChildDataNode(relatedDns[index])  
-                
-            self.__updateRootDataNodeList(existingDnsForConcept)   
-                      
-    # Build or update dataNode in the data graph for a given relationAttributeConcept
+                            exitingDn.addChildDataNode(relatedDns[index])
+
+            self.__updateRootDataNodeList(existingDnsForConcept)
+
     def __buildDataNode(self, vInfo, conceptInfo, keyDataName):
+        """
+        Build or update a data node in the data graph for a given relationAttributeConcept.
+
+        Notes:
+            - This function will either create initial data nodes, create single data nodes,
+              create multiple data nodes, or update existing data nodes based on various conditions.
+
+        Args:
+            vInfo (object): Information about the value, like its dimension and length.
+            conceptInfo (dict): Information about the concept associated with the data nodes.
+            keyDataName (str): The name of the key for which the data node is being updated or created.
+
+        Returns:
+            object: Newly created or updated data nodes.
+        """
         conceptName = conceptInfo['concept'].name
-       
+
         if not dict.__contains__(self, 'dataNode'):   # ------ No DataNode yet
-            return self.__createInitialdDataNode(vInfo, conceptInfo, keyDataName) # Done - End the method
+            return self.__createInitialDataNode(vInfo, conceptInfo, keyDataName) # Done - End the method
         else:
             # ---------- DataNodes already created
             existingDnsForConcept = self.findDataNodesInBuilder(select = conceptName) # Try to get DataNodes of the current concept
-            
-            if len(existingDnsForConcept) == 0:# Check if DataNode for this concept already created                    
+
+            if len(existingDnsForConcept) == 0:# Check if DataNode for this concept already created
                 # No DataNode of this concept created yet
-    
+
                 # If attribute value is a single element - will create a single new DataNode
-                if vInfo.len == 1 and vInfo.dim < 2: 
+                if vInfo.len == 1 and vInfo.dim < 2:
                     return self.__createSingleDataNode(vInfo, conceptInfo, keyDataName)
                 else: # -- Value is multiple elements
                     return self.__createMultiplyDataNode(vInfo, conceptInfo, keyDataName)
             else: # DataNode with this concept already created - update it
-                self.__updateDataNodes(vInfo, conceptInfo, keyDataName)
-                
+                if (not self.skeletonDataNode):
+                    self.__updateDataNodes(vInfo, conceptInfo, keyDataName)
+
     def __addEquality(self, vInfo, conceptInfo, equalityConceptName, keyDataName):
+        """
+        Add equality relations between existing data nodes of specified concepts based on the provided value information.
+
+        Args:
+            vInfo (object): Information about the value matrix that indicates equality, like its shape.
+            conceptInfo (dict): Information about the concept associated with one set of data nodes.
+            equalityConceptName (str): The name of the second concept associated with another set of data nodes.
+            keyDataName (str): The name of the key for which the data node is being checked for equality.
+
+        Notes:
+            - Logging statements are used to indicate the progress and success of the equality addition.
+            - Checks are made to ensure that data nodes exist for both specified concepts before proceeding.
+            - The function uses the instance IDs of the data nodes and the shape of `vInfo.value` to establish the equality.
+        """
         conceptName = conceptInfo['concept'].name
         existingDnsForConcept = self.findDataNodesInBuilder(select = conceptName)
         existingDnsForEqualityConcept = self.findDataNodesInBuilder(select = equalityConceptName)
-        
+
         if not existingDnsForConcept and not existingDnsForEqualityConcept:
-            _DataNodeBulder__Logger.warning('No datNodes created for concept %s and equality concept %s'%(conceptName,equalityConceptName))
+            _DataNodeBuilder__Logger.warning('No datNodes created for concept %s and equality concept %s'%(conceptName,equalityConceptName))
             return
-        
+
         if not existingDnsForConcept:
-            _DataNodeBulder__Logger.warning('No datNodes created for concept %s'%(conceptName))
+            _DataNodeBuilder__Logger.warning('No datNodes created for concept %s'%(conceptName))
             return
-        
+
         if not existingDnsForEqualityConcept:
-            _DataNodeBulder__Logger.warning('No datNodes created for equality concept %s'%(equalityConceptName))
+            _DataNodeBuilder__Logger.warning('No datNodes created for equality concept %s'%(equalityConceptName))
             return
-        
-        _DataNodeBulder__Logger.info('Added equality between dataNodes of types %s and %s'%(conceptName,equalityConceptName))
+
+        if not getProductionModeStatus():
+            _DataNodeBuilder__Logger.info('Added equality between dataNodes of types %s and %s'%(conceptName,equalityConceptName))
 
         for conceptDn in existingDnsForConcept:
             for equalDn in existingDnsForEqualityConcept:
-                
+
                 if conceptDn.getInstanceID() >= vInfo.value.shape[0]:
                     continue
-                
+
                 if equalDn.getInstanceID() >= vInfo.value.shape[1]:
                     continue
-                
+
                 if vInfo.value[conceptDn.getInstanceID(), equalDn.getInstanceID()]:
-                    _DataNodeBulder__Logger.info('DataNodes of %s is equal to %s'%(conceptDn,equalDn))
+                    if not getProductionModeStatus():
+                        _DataNodeBuilder__Logger.info('DataNodes of %s is equal to %s'%(conceptDn,equalDn))
                     conceptDn.addEqualTo(equalDn)
 
-    # Method processing value of the attribute - determining it it should be treated as a single element. 
-    #     It returns a tuple with elements specifying the length of the first dimension of the value, 
-    #     the number of dimensions of the value and the original value itself
     def __processAttributeValue(self, value, keyDataName):
+        """
+        Processes the attribute value to determine its structure and nature.
+
+        This method analyzes the attribute value and categorizes it based on its dimensionality,
+        whether it is a scalar or a list, and its length. It returns a named tuple with this
+        information, which can be used for further processing or logging.
+
+        Args:
+            value (Union[torch.Tensor, list, scalar]): The value of the attribute to be processed.
+            keyDataName (str): The name of the attribute for which the value is processed.
+
+        Returns:
+            namedtuple: A named tuple 'ValueInfo' with fields 'len', 'value', and 'dim' where
+                - 'len' is the length of the first dimension of the value.
+                - 'value' is the original or processed value.
+                - 'dim' is the number of dimensions of the value.
+
+        Notes:
+            - Tensor or list with a length of 1 is considered as scalar.
+            - It supports Tensor, list, and scalar data types.
+        """
         ValueInfo = namedtuple('ValueInfo', ["len", "value", 'dim'])
 
         if isinstance(value, torch.Tensor):
@@ -2002,13 +3126,13 @@ def __processAttributeValue(self, value, keyDataName):
                 lenV = 1
         else:
             lenV = len(value)
-            
+
         if not isinstance(value, (torch.Tensor, list)): # It is scalar value
-            return ValueInfo(len = 1, value = value, dim=0) 
-            
+            return ValueInfo(len = 1, value = value, dim=0)
+
         if isinstance(value, torch.Tensor) and dimV == 0: # It is a Tensor but also scalar value
             return ValueInfo(len = 1, value = value.item(), dim=0)
-        
+
         if (lenV == 1): # It is Tensor or list with length 1 - treat it as scalar
             if isinstance(value, list) and not isinstance(value[0], (torch.Tensor, list)) : # Unpack the value
                 return ValueInfo(len = 1, value = value[0], dim=0)
@@ -2016,12 +3140,12 @@ def __processAttributeValue(self, value, keyDataName):
                 return ValueInfo(len = 1, value = torch.squeeze(value, 0), dim=0)
 
         #  If it is Tensor or list with length 2 but it is for attribute providing probabilities - assume it is a scalar value
-        if isinstance(value, list) and lenV ==  2 and keyDataName[0] == '<': 
+        if isinstance(value, list) and lenV ==  2 and keyDataName[0] == '<':
             return ValueInfo(lenV = 1, value = value, dim=0)
         elif isinstance(value, torch.Tensor) and lenV ==  2 and dimV  == 0 and keyDataName[0] == '<':
             return ValueInfo(len = 1, value = value, dim=0)
 
-        if isinstance(value, list): 
+        if isinstance(value, list):
             if not isinstance(value[0], (torch.Tensor, list)) or (isinstance(value[0], torch.Tensor) and value[0].dim() == 0):
                 return ValueInfo(len = lenV, value = value, dim=1)
             elif not isinstance(value[0][0], (torch.Tensor, list)) or (isinstance(value[0][0], torch.Tensor) and value[0][0].dim() == 0):
@@ -2029,61 +3153,105 @@ def __processAttributeValue(self, value, keyDataName):
             elif not isinstance(value[0][0][0], (torch.Tensor, list)) or (isinstance(value[0][0][0], torch.Tensor) and value[0][0][0].dim() == 0):
                 return ValueInfo(len = lenV, value = value, dim=3)
             else:
-                _DataNodeBulder__Logger.warning('Dimension of nested list value for key %s is more then 3 returning dimension 4'%(keyDataName))
+                _DataNodeBuilder__Logger.warning('Dimension of nested list value for key %s is more then 3 returning dimension 4'%(keyDataName))
                 return ValueInfo(len = lenV, value = value, dim=4)
 
         elif isinstance(value, torch.Tensor):
             return ValueInfo(len = lenV, value = value, dim=dimV)
-    
+
     def collectTime(self, start):
+        """
+        Collects the time taken for the __setitem__ operation and stores it in internal lists.
+
+        This method calculates the time elapsed for a __setitem__ operation and appends that,
+        along with the start and end timestamps, to respective lists stored in the object.
+
+        Args:
+            start (int): The start time of the __setitem__ operation in nanoseconds.
+
+        Notes:
+            - The time taken for each __setitem__ operation is stored in a list named 'DataNodeTime'.
+            - The start time for each __setitem__ operation is stored in a list named 'DataNodeTime_start'.
+            - The end time for each __setitem__ operation is stored in a list named 'DataNodeTime_end'.
+        """
         # Collect time used for __setitem__
-        end = process_time_ns()
-        currenTime =  end - start
-        if not dict.__contains__(self, "DataNodeTime"):
-            dict.__setitem__(self, "DataNodeTime", [])
+        end = perf_counter_ns()
+        currentTime =  end - start
+
+        timeList = self.setdefault("DataNodeTime", [])
+        timeList.append(currentTime)
+        startTimeList = self.setdefault("DataNodeTime_start", [])
+        startTimeList.append(start)
+        endTimeList = self.setdefault("DataNodeTime_end", [])
+        endTimeList.append(end)
 
-        timeList = dict.__getitem__(self, "DataNodeTime")
-        timeList.append(currenTime)
-        
-    # Overloaded __setitem Dictionary method - tracking sensor data and building corresponding data graph
     def __setitem__(self, _key, value):
+        """
+        Overloaded __setitem__ method for the DataNodeBuilder class.
+        This method is responsible for adding or updating key-value pairs in the dictionary-like object.
+
+        Parameters:
+        -----------
+        _key : Sensor, Property, Concept, or str
+            The key to insert into the dictionary. It can be an instance of Sensor, Property, Concept classes, or a string.
+        value : any
+            The value to associate with the key. It can be of any data type.
+
+        Behavior:
+        ---------
+        - If `_key` is a Sensor and its `build` attribute is set to False, the value is directly inserted without further processing.
+        - If `_key` is a Property, the value is directly inserted without further processing.
+        - If `_key` is a Concept or a string containing a Concept, additional logic is invoked to update the associated graph and indices.
+        - If the system is not in production mode, additional logging and checks are performed.
+
+        Returns:
+        --------
+        None
+
+        Side Effects:
+        -------------
+        - Updates the underlying dictionary.
+        - May update an associated graph.
+        - Logs messages to _DataNodeBuilder__Logger.
+        - Updates the DataNode's timing metrics.
+        """
         from ..sensor import Sensor
 
-        start = process_time_ns()
+        start = perf_counter_ns()
         self.__addSetitemCounter()
-        
+
         if isinstance(_key, (Sensor, Property, Concept)):
             key = _key.fullname
             if  isinstance(_key, Sensor) and not _key.build:
                 if isinstance(value, torch.Tensor):
-                    _DataNodeBulder__Logger.debug('No processing (because build is set to False) - key - %s, key type - %s, value - %s, shape %s'%(key,type(_key),type(value),value.shape))
+                    _DataNodeBuilder__Logger.debug('No processing (because build is set to False) - key - %s, key type - %s, value - %s, shape %s'%(key,type(_key),type(value),value.shape))
                 elif isinstance(value, list):
-                    _DataNodeBulder__Logger.debug('No processing (because build is set to False) - key - %s, key type - %s, value - %s, length %s'%(key,type(_key),type(value),len(value)))
+                    _DataNodeBuilder__Logger.debug('No processing (because build is set to False) - key - %s, key type - %s, value - %s, length %s'%(key,type(_key),type(value),len(value)))
                 else:
-                    _DataNodeBulder__Logger.debug('No processing (because build is set to False) - key - %s, key type - %s, value - %s'%(key,type(_key),type(value)))
+                    _DataNodeBuilder__Logger.debug('No processing (because build is set to False) - key - %s, key type - %s, value - %s'%(key,type(_key),type(value)))
 
                 self.collectTime(start)
                 return dict.__setitem__(self, _key, value)
-            
+
             if  isinstance(_key, Property):
                 if isinstance(value, torch.Tensor):
-                    _DataNodeBulder__Logger.debug('No processing Property as key - key - %s, key type - %s, value - %s, shape %s'%(key,type(_key),type(value),value.shape))
+                    _DataNodeBuilder__Logger.debug('No processing Property as key - key - %s, key type - %s, value - %s, shape %s'%(key,type(_key),type(value),value.shape))
                 elif isinstance(value, list):
-                    _DataNodeBulder__Logger.debug('No processing Property as key - key - %s, key type - %s, value - %s, length %s'%(key,type(_key),type(value),len(value)))
+                    _DataNodeBuilder__Logger.debug('No processing Property as key - key - %s, key type - %s, value - %s, length %s'%(key,type(_key),type(value),len(value)))
                 else:
-                    _DataNodeBulder__Logger.debug('No processing Property as key - key - %s, key type - %s, value - %s'%(key,type(_key),type(value)))
+                    _DataNodeBuilder__Logger.debug('No processing Property as key - key - %s, key type - %s, value - %s'%(key,type(_key),type(value)))
 
                 self.collectTime(start)
                 return dict.__setitem__(self, _key, value)
         elif isinstance(_key, str):
             key = _key
         else:
-            _DataNodeBulder__Logger.error('key - %s, type %s is not supported'%(_key,type(_key)))
+            _DataNodeBuilder__Logger.error('key - %s, type %s is not supported'%(_key,type(_key)))
             self.collectTime(start)
             return
-        
+
         skey = key.split('/')
-        
+
         # Check if the key with this value has been set recently
         # If not create a new sensor for it
         # If yes stop __setitem__ and return - the same value for the key was added last time that key was set
@@ -2091,289 +3259,486 @@ def __setitem__(self, _key, value):
             self.myLoggerTime.info(f"DataNode Builder skipping repeated value for sensor  - {skey}")
             self.collectTime(start)
             return # Stop __setitem__ for repeated key value combination
-        
-        if isinstance(value, torch.Tensor):
-            _DataNodeBulder__Logger.info('key - %s, key type - %s, value - %s, shape %s'%(key,type(_key),type(value),value.shape))
-        elif isinstance(value, list):
-            _DataNodeBulder__Logger.info('key - %s, key type - %s, value - %s, length %s'%(key,type(_key),type(value),len(value)))
-        else:
-            _DataNodeBulder__Logger.info('key - %s, key type - %s, value - %s'%(key,type(_key),type(value)))
+
+        if not getProductionModeStatus():
+            if isinstance(value, torch.Tensor):
+                _DataNodeBuilder__Logger.info('key - %s, key type - %s, value - %s, shape %s'%(key,type(_key),type(value),value.shape))
+            elif isinstance(value, list):
+                _DataNodeBuilder__Logger.info('key - %s, key type - %s, value - %s, length %s'%(key,type(_key),type(value),len(value)))
+            else:
+                _DataNodeBuilder__Logger.info('key - %s, key type - %s, value - %s'%(key,type(_key),type(value)))
 
         if value is None:
-            _DataNodeBulder__Logger.error('The value for the key %s is None - abandon the update'%(key))
+            _DataNodeBuilder__Logger.error('The value for the key %s is None - abandon the update'%(key))
             self.collectTime(start)
             return dict.__setitem__(self, _key, value)
-                
-        if len(skey) < 2:            
-            _DataNodeBulder__Logger.error('The key %s has only two elements, needs at least three - abandon the update'%(key))
+
+        if len(skey) < 2:
+            _DataNodeBuilder__Logger.error('The key %s has only two elements, needs at least three - abandon the update'%(key))
             self.collectTime(start)
             return dict.__setitem__(self, _key, value)
-        
+
         usedGraph = dict.__getitem__(self, "graph")
 
         # Find if the key include concept from graph
-        
+
         graphPathIndex = usedGraph.cutGraphName(skey)
         keyWithoutGraphName = skey[graphPathIndex:]
-        graphPath =  ''.join(map(str, skey[:graphPathIndex])) 
-       
+        graphPath =  ''.join(map(str, skey[:graphPathIndex]))
+
         # Check if found concept in the key
         if not keyWithoutGraphName:
-            _DataNodeBulder__Logger.warning('key - %s has not concept part - returning'%(key))
+            _DataNodeBuilder__Logger.warning('key - %s has not concept part - returning'%(key))
             self.collectTime(start)
             return dict.__setitem__(self, _key, value)
-            
+
         # Find description of the concept in the graph
         if isinstance(_key, Sensor):
             try:
-                conceptName = _key.concept.name 
+                conceptName = _key.concept.name
             except TypeError as _:
                 conceptName = keyWithoutGraphName[0]
         else:
             conceptName = keyWithoutGraphName[0]
         concept = self.__findConcept(conceptName, usedGraph)
-                
+
         if not concept:
-            _DataNodeBulder__Logger.warning('conceptName - %s has not been found in the used graph %s - returning'%(conceptName,usedGraph.fullname))
+            _DataNodeBuilder__Logger.warning('conceptName - %s has not been found in the used graph %s - returning'%(conceptName,usedGraph.fullname))
             self.collectTime(start)
             return dict.__setitem__(self, _key, value)
-        
+
         conceptInfo = self.__findConceptInfo(usedGraph, concept)
-        
+
         if isinstance(_key, Sensor):
             self.__updateConceptInfo(usedGraph, conceptInfo, _key)
 
         DataNodesConcepts = dict.__getitem__(self, "DataNodesConcepts")
-        if (not self.skeletonDataNode) or ("relationName" in conceptInfo) or ("dataNode" not in self) or (conceptName not in DataNodesConcepts):
-            DataNodesConcepts.add(conceptName)
-            dict.__setitem__(self, "DataNodesConcepts", DataNodesConcepts)
-
-            # Create key for DataNode construction
-            keyDataName = "".join(map(lambda x: '/' + x, keyWithoutGraphName[1:-1]))
-            keyDataName = keyDataName[1:] # __cut first '/' from the string
-                            
-            if conceptInfo['label']:
-                keyDataName += '/label'
-                
-            vInfo = self.__processAttributeValue(value, keyDataName)
-            
-            # Decide if this is equality between concept data, dataNode creation or update for concept or relation link
-            if keyDataName.find("_Equality_") > 0:
-                equalityConceptName = keyDataName[keyDataName.find("_Equality_") + len("_Equality_"):]
-                self.__addEquality(vInfo, conceptInfo, equalityConceptName, keyDataName)
-            else:                       
-                _DataNodeBulder__Logger.debug('%s found in the graph; it is a concept'%(conceptName))
-                index = self.__buildDataNode(vInfo, conceptInfo, keyDataName)   # Build or update Data node
-                
-                if index:
-                    indexKey = graphPath  + '/' +conceptName + '/index'
-                    dict.__setitem__(self, indexKey, index)
-                    from collections.abc import Sequence
-                    if self.skeletonDataNode:
-                        if "allDns" not in self:
-                            dict.__setitem__(self, "allDns", set())
-                        allDns = dict.__getitem__(self, "allDns")
-                        try:
-                            if isinstance(index[0], Sequence):
-                                index = index[0]
-                            allDns.update(index)
-                        except TypeError as ty:
-                            pass
-                
-                if conceptInfo['relation']:
-                    _DataNodeBulder__Logger.debug('%s is a relation'%(conceptName))
-                    self.__buildRelationLink(vInfo, conceptInfo, keyDataName) # Build or update relation link
-        else:
-            keyInRootDataNode = skey[-3] + "/" + skey[-2]
-            if conceptInfo['label']:
-                keyInRootDataNode += "/label"
-            if "<" in keyInRootDataNode:
-                self.__addVariableNameToSet((_key, keyInRootDataNode))
-            else:       
-                self.__addPropertyNameToSet((_key, keyInRootDataNode))
-        
+        # Only build the datanode if it is not a full skeleton mode or if the first initial datanode has not been created yet
+        if not self.skeletonDataNodeFull or ("dataNode" not in self):
+            # Only build the datanode if it is not a skeleton mode or if the datanodes for the given concept have not been created yet
+            # or if the concept is a relation
+            if (not self.skeletonDataNode) or (conceptName not in DataNodesConcepts) or ("relationName" in conceptInfo):
+                # Create key for DataNode construction
+                keyDataName = "".join(map(lambda x: '/' + x, keyWithoutGraphName[1:-1]))
+                keyDataName = keyDataName[1:] # __cut first '/' from the string
+
+                if conceptInfo['label']:
+                    keyDataName += '/label'
+
+                vInfo = self.__processAttributeValue(value, keyDataName)
+
+                # Decide if this is equality between concept data, dataNode creation or update for concept or relation link
+                if keyDataName.find("_Equality_") > 0:
+                    equalityConceptName = keyDataName[keyDataName.find("_Equality_") + len("_Equality_"):]
+                    self.__addEquality(vInfo, conceptInfo, equalityConceptName, keyDataName)
+                else:
+                    _DataNodeBuilder__Logger.debug('%s found in the graph; it is a concept'%(conceptName))
+                    index = self.__buildDataNode(vInfo, conceptInfo, keyDataName)   # Build or update Data node
+
+                    if index:
+                        indexKey = graphPath  + '/' + conceptName + '/index'
+                        dict.__setitem__(self, indexKey, index)
+                        from collections.abc import Sequence
+                        if self.skeletonDataNode:
+                            allDns = self.setdefault("allDns", set())
+
+                            # Add the index to the "allDns" set
+                            try:
+                                if isinstance(index[0], Sequence):
+                                    index = index[0]
+                                allDns.update(index)
+                            except TypeError as ty:
+                                pass
+
+                        DataNodesConcepts[conceptName] = index
+                        #dict.__setitem__(self, "DataNodesConcepts", DataNodesConcepts)
+
+                    if conceptInfo['relation']:
+                        _DataNodeBuilder__Logger.debug('%s is a relation'%(conceptName))
+                        self.__buildRelationLink(vInfo, conceptInfo, keyDataName) # Build or update relation link
+
+        if self.skeletonDataNode:
+            if conceptName in skey:
+                # Find the index of "conceptName" in skey
+                index = skey.index(conceptName)
+
+                # Join "conceptName" with the next element in skey
+                keyInRootDataNode = "/".join(skey[index:index+2])
+
+                # Add "/label" to the key if the concept has a label marked
+                if conceptInfo['label']:
+                    keyInRootDataNode += "/label"
+
+                # Check if the key contains "<" and add it to the variable set if it does, otherwise add it to the property set
+                if "<" in keyInRootDataNode:
+                    self.__addVariableNameToSet((_key, keyInRootDataNode))
+                else:
+                    self.__addPropertyNameToSet((_key, keyInRootDataNode))
+            else:
+                # throw an exception
+                raise Exception("The key does not contain conceptName")
+
+        # Add key to the list of keys in order
+        if self.skeletonDataNodeFull:
+            KeysInOrder = dict.__getitem__(self, "KeysInOrder")
+            KeysInOrder.append(_key)
+
         # Add value to the underling dictionary
         r = dict.__setitem__(self, _key, value)
-        
+
         if not r:
             pass # Error when adding entry to dictionary ?
-        
+
         self.collectTime(start)
-        return r                
-                                             
+        return r
+
     def __delitem__(self, key):
+        """
+        Overloaded __delitem__ method for the DataNodeBuilder class.
+        This method is responsible for deleting a key-value pair from the dictionary-like object.
+
+        Parameters:
+        -----------
+        key : any hashable type
+            The key to be deleted from the dictionary.
+
+        Returns:
+        --------
+        None
+        """
         return dict.__delitem__(self, key)
 
     def __contains__(self, key):
+        """
+        Overloaded __contains__ method for the DataNodeBuilder class.
+        This method checks if the key is present in the dictionary-like object.
+
+        Parameters:
+        -----------
+        key : any hashable type
+            The key to be checked for existence in the dictionary.
+
+        Returns:
+        --------
+        bool
+            True if the key exists, False otherwise.
+        """
         return dict.__contains__(self, key)
-    
-    # Add or increase generic counter counting number of the getitem method calls
+
     def __addGetDataNodeCounter(self):
+        """
+        Method to increment a counter that keeps track of the number of times the __getitem__ method is called.
+
+        Parameters:
+        -----------
+        None
+
+        Returns:
+        --------
+        None
+
+        Side Effects:
+        -------------
+        - Updates the internal counter for __getitem__ calls.
+        """
         counterName = 'Counter' + 'GetDataNode'
         if not dict.__contains__(self, counterName):
             dict.__setitem__(self, counterName, 1)
         else:
             currentCounter =  dict.__getitem__(self, counterName)
             dict.__setitem__(self, counterName, currentCounter + 1)
-            
-    def findDataNodesInBuilder(self, select = None, indexes = None):
+
+    def findDataNodesInBuilder(self, select=None, indexes=None):
+        """
+        Method to find data nodes that meet certain criteria within the DataNodeBuilder object.
+
+        Parameters:
+        -----------
+        select : function or None, optional
+            A function to apply to each DataNode to determine if it should be selected. Defaults to None.
+        indexes : list or None, optional
+            A list of indexes to specifically look for. Defaults to None.
+
+        Returns:
+        --------
+        list
+            A list of DataNodes that meet the given criteria.
+        """
         existingRootDns = dict.__getitem__(self, 'dataNode') # DataNodes roots
-        
+
         if not existingRootDns:
             foundDns = []
         else:
-            foundDns = existingRootDns[0].findDatanodes(dns = existingRootDns, select = select, indexes = indexes) 
-        
+            foundDns = existingRootDns[0].findDatanodes(dns=existingRootDns, select=select, indexes=indexes)
+
         return foundDns
-    
-    def needsBatchRootDN(self):
-        if dict.__contains__(self, 'dataNode'):
-            _dataNode = dict.__getitem__(self, 'dataNode')
-            
-            if len(_dataNode) == 1:
-                return False
-            else:
-                typesInDNs = set()
-                for i, d in enumerate(_dataNode):
-                    if i == 0:
-                        continue
-                    
-                    typesInDNs.add(d.getOntologyNode().name)
-                
-                if len(typesInDNs) > 1:
-                    return False
-                
-            return True
-        else:
-            return False 
-    
-    def addBatchRootDN(self):
+
+    def createFullDataNode(self, rootDataNode):
+        """
+        Method to create a full data node based on the current skeleton of the DataNodeBuilder object.
+
+        Parameters:
+        -----------
+        rootDataNode : DataNode object
+            The root data node to which attributes will be added.
+
+        Returns:
+        --------
+        None
+
+        Side Effects:
+        -------------
+        - Modifies internal state to reflect that a full data node has been created.
+        - Logs time taken to create the full data node.
+
+        Notes:
+        ------
+        - This method operates under the assumption that the DataNodeBuilder is initially in skeleton mode.
+        """
+        if not self.skeletonDataNodeFull:
+            return # Do nothing if not in full skeleton mode
+
+        startCreateFullDataNode = perf_counter()
+        self.skeletonDataNodeFull = False # Set temporary flag to False to allow creation of full dataNode
+
+        keysInOrder = dict.__getitem__(self, "KeysInOrder")
+
+        for key in keysInOrder:
+            # Run the original values through __setitem__ to build the full dataNode
+            self.__setitem__(key, dict.__getitem__(self, key))
+
+        if self.skeletonDataNode:
+            # Get the "allDns" set from the data node, or create a new empty set if it doesn't exist
+            allDns = self.get("allDns", set())
+
+            # Iterate over the data nodes in "allDns" and add the "rootDataNode" attribute to them
+            for dn in allDns:
+                if dn == rootDataNode:
+                    continue
+                dn.attributes["rootDataNode"] = rootDataNode
+
+        self.skeletonDataNodeFull = True # Return flag to the original
+
+        endCreateFullDataNode = perf_counter()
+        elapsedCreateFullDataNode = (endCreateFullDataNode - startCreateFullDataNode) * 1000
+        self.myLoggerTime.info(f'Creating Full Datanode: {elapsedCreateFullDataNode}ms')
+
+    def createBatchRootDN(self):
+        """
+        Creates a batch root DataNode when certain conditions are met.
+
+        Conditions for creating a new batch root DataNode:
+        - If the DataNodeBuilder object already has a single root DataNode, no new root DataNode will be created.
+        - If the DataNodeBuilder object has DataNodes of different types, a batch root DataNode cannot be created.
+
+        Parameters:
+        -----------
+        None
+
+        Returns:
+        --------
+        None
+
+        Side Effects:
+        -------------
+        - Modifies the 'dataNode' attribute of the DataNodeBuilder object.
+        - Logs messages based on the production mode status and whether a new root DataNode is created or not.
+
+        Raises:
+        -------
+        - ValueError: When the DataNodeBuilder object has no DataNodes, or existing DataNodes have no connected graph.
+
+        Notes:
+        ------
+        - This method makes use of internal logging for debugging and timing.
+        """
         if dict.__contains__(self, 'dataNode'):
-            _dataNode = dict.__getitem__(self, 'dataNode')
+            existingDns = dict.__getitem__(self, 'dataNode')
+            if len(existingDns) == 1:
+                rootDn = existingDns[0]
+                if not getProductionModeStatus():
+                    _DataNodeBuilder__Logger.info(f'No new Batch Root DataNode created - DataNode Builder already has single Root DataNode with id {rootDn.instanceID} of type {rootDn.getOntologyNode().name}')
+                return
+
+            # Check if there are more than one type of DataNodes in the builder
+            typesInDNs = set()
+            for i, d in enumerate(existingDns):
+                typesInDNs.add(d.getOntologyNode().name)
+
+            # If there are more than one type of DataNodes in the builder, then it is not possible to create new Batch Root DataNode
+            if len(typesInDNs) > 1:
+                _DataNodeBuilder__Logger.warn('DataNode Builder has DataNodes of different types: %s, not possible to create batch Datanode' % (typesInDNs))
+                return
 
-            supGraph = None
-            if len(_dataNode) == 1:
-                rootDn = _dataNode[0]
-                _DataNodeBulder__Logger.warning('No new Batch Root DataNode created - DataNode Builder has single DataNode with id %s of type %s'
-                                                %(rootDn.instanceID,rootDn.getOntologyNode().name))
+            # Create the Batch Root DataNode
+            supGraph = existingDns[1].getOntologyNode().sup
+            if supGraph is None:
+                raise ValueError('Not able to create Batch Root DataNode - existing DataNodes in the Builder have concept type %s not connected to any graph: %s'%(typesInDNs))
+
+            batchRootDNValue = ""
+            batchRootDNID = 0
+
+            if 'batch' in supGraph.concepts:
+                batchRootDNOntologyNode = supGraph.concepts['batch' ]
             else:
-                typesInDNs = set()
-                for i, d in enumerate(_dataNode):
-                    if i == 0:
-                        continue
-                    
-                    typesInDNs.add(d.getOntologyNode().name)
-                    supGraph = d.getOntologyNode().sup
-                
-                if len(typesInDNs) > 1:
-                    raise ValueError('Not able to create Batch Root DataNode - DataNode Builder has DataNodes of different types: %s'%(typesInDNs))  
-                
-                if supGraph == None:
-                    raise ValueError('Not able to create Batch Root DataNode - existing DataNodes in the Builder have concept type %s not connected to any graph: %s'%(typesInDNs))  
-
-                batchRootDNValue = ""
-                batchRootDNID = 0
                 batchRootDNOntologyNode = Concept(name='batch')
-                supGraph.attach(batchRootDNOntologyNode)
-                
-                batchRootDN= DataNode(instanceID = batchRootDNID, instanceValue = batchRootDNValue, ontologyNode = batchRootDNOntologyNode)
-            
-                for i, d in enumerate(_dataNode):
-                    batchRootDN.addChildDataNode(d)  
-                  
-                dns = []
-                dns.append(batchRootDN)  
-                self.__updateRootDataNodeList(dns)
-
-                _DataNodeBulder__Logger.info('Created single Batch Root DataNode with id %s of type %s'%(batchRootDNID,batchRootDNOntologyNode))
+            supGraph.attach(batchRootDNOntologyNode)
+
+            batchRootDN = DataNode(myBuilder = self, instanceID = batchRootDNID, instanceValue = batchRootDNValue, ontologyNode = batchRootDNOntologyNode)
+
+            for i, d in enumerate(existingDns):
+                batchRootDN.addChildDataNode(d)
+
+            # The new Root DataNode it the batch Root DataNode
+            self.__updateRootDataNodeList([batchRootDN])
+
+            if not getProductionModeStatus():
+                _DataNodeBuilder__Logger.info('Created single Batch Root DataNode with id %s of type %s'%(batchRootDNID,batchRootDNOntologyNode))
+            self.myLoggerTime.info('Created single Batch Root DataNode with id %s of type %s'%(batchRootDNID,batchRootDNOntologyNode))
         else:
-            raise ValueError('DataNode Builder has no DataNode started yet')   
-        
-    # Method returning constructed DataNode - the fist in the list
+            raise ValueError('DataNode Builder has no DataNode started yet')
+
     def getDataNode(self, context="interference", device='auto'):
+        """
+        Retrieves and returns the first DataNode from the DataNodeBuilder object based on the given context and device.
+
+        Parameters:
+        -----------
+        context : str, optional
+            The context under which to get the DataNode, defaults to "interference".
+        device : str, optional
+            The torch device to set for the DataNode, defaults to 'auto'.
+
+        Returns:
+        --------
+        DataNode or None
+            Returns the first DataNode if it exists, otherwise returns None.
+
+        Side Effects:
+        -------------
+        - Updates the torch device for the returned DataNode based on the 'device' parameter.
+        - Logs various messages based on the context and production mode.
+
+        Raises:
+        -------
+        None
+
+        Notes:
+        ------
+        - This method makes use of internal logging for debugging and timing.
+
+        """
         self.__addGetDataNodeCounter()
-        
+
         if context=="interference":
             if self.skeletonDataNode:
                 self.myLoggerTime.info("DataNode Builder is using skeleton datanode mode")
             if 'Counter' + '_setitem' in self:
                 self.myLoggerTime.info("DataNode Builder the set method called - %i times"%(self['Counter' + '_setitem']))
             if 'DataNodeTime' in self:
+                # self['DataNodeTime'] is in nanoseconds, so divide by 1000000 to get milliseconds
                 elapsedInMsDataNodeBuilder = sum(self['DataNodeTime'])/1000000
-                self.myLoggerTime.info(f"DataNode Builder used - {elapsedInMsDataNodeBuilder:.8f}ms")
-        
+                self.myLoggerTime.info(f"DataNode Builder time usage - {elapsedInMsDataNodeBuilder:.5f}ms")
+
+                #self.myLoggerTime.info(f"DataNode Builder elapsed time in ns - {self['DataNodeTime']}")
+                #self.myLoggerTime.info(f"DataNode Builder start time in ns - {self['DataNodeTime_start']}")
+                #self.myLoggerTime.info(f"DataNode Builder end time in ns - {self['DataNodeTime_end']}")
+
+        # If DataNode it created then return it
         if dict.__contains__(self, 'dataNode'):
-            _dataNode = dict.__getitem__(self, 'dataNode')
-            
-            if len(_dataNode) > 0:  
-                returnDn = _dataNode[0]
-                
+            existingDns = dict.__getitem__(self, 'dataNode')
+
+            if len(existingDns) != 0:
+                returnDn = existingDns[0]
+
                 # Set the torch device
                 returnDn.current_device = device
-                if returnDn.current_device=='auto': # if not set use cpu or cuda if available
+                if returnDn.current_device == 'auto': # if not set use cpu or cuda if available
                     returnDn.current_device = 'cpu'
                     if torch.cuda.is_available():
                         returnDn.current_device = 'cuda'
-                    
-                if len(_dataNode) == 1:
-                    _DataNodeBulder__Logger.info('Returning dataNode with id %s of type %s'%(returnDn.instanceID,returnDn.getOntologyNode().name))
+
+                if len(existingDns) != 1:
+                    typesInDNs = {d.getOntologyNode().name for d in existingDns[1:]}
+                    _DataNodeBuilder__Logger.warning(f'Returning first dataNode with id {returnDn.instanceID} of type {returnDn.getOntologyNode().name} - there are total {len(existingDns)} dataNodes of types {typesInDNs}')
+                    self.myLoggerTime.info(f'Returning first dataNode with id {returnDn.instanceID} of type {returnDn.getOntologyNode().name} - there are total {len(existingDns)} dataNodes of types {typesInDNs}')
                 else:
-                    typesInDNs = set()
-                    for i, d in enumerate(_dataNode):
-                        if i == 0:
-                            continue
-                        
-                        typesInDNs.add(d.getOntologyNode().name)
-                        
-                    _DataNodeBulder__Logger.warning('Returning first dataNode with id %s of type %s - there are total %i dataNodes of types %s'
-                                                    %(returnDn.instanceID,returnDn.getOntologyNode(),len(_dataNode),typesInDNs))
+                    if not getProductionModeStatus():
+                        _DataNodeBuilder__Logger.info(f'Returning dataNode with id {returnDn.instanceID} of type {returnDn.getOntologyNode().name}')
+                    self.myLoggerTime.info(f'Returning dataNode with id {returnDn.instanceID} of type {returnDn.getOntologyNode().name}')
 
                 if self.skeletonDataNode:
-                    variableSetName = 'variableSet'
-                    if dict.__contains__(self, variableSetName):
-                        variableSet = dict.__getitem__(self, variableSetName)
-                        returnDn.attributes[variableSetName] = {}
-                        for _k, k in variableSet:
-                            returnDn.attributes[variableSetName][k] = self[_k]
-                            
-                    propertySetName = 'propertySet'
-                    if dict.__contains__(self, variableSetName):
-                        propertySet = dict.__getitem__(self, propertySetName)
-                        returnDn.attributes[propertySetName] = {}
-                        for _k, k in propertySet:
-                            returnDn.attributes[propertySetName][k] = self[_k]
-                            
-                    allDnsName = 'allDns' 
-                    if dict.__contains__(self, allDnsName):
-                        allDns = dict.__getitem__(self, allDnsName)
-                        for dn in allDns:
-                            if dn == returnDn:
-                                continue
-                            dn.attributes["rootDataNode"] = returnDn
-        
+                    # Get the "variableSet" dictionary from the data node, or create a new empty dictionary if it doesn't exist
+                    variableSet = self.get("variableSet", {})
+
+                    # Create a dictionary of the items in "variableSet" with the keys and values swapped
+                    variableSetDict = {k2: self[k1] for k1, k2 in dict(variableSet).items()}
+
+                    # Add the "variableSet" dictionary to the return data node attributes
+                    returnDn.attributes["variableSet"] = variableSetDict
+
+                    # Get the "propertySet" dictionary from the data node, or create a new empty dictionary if it doesn't exist
+                    propertySet = self.get("propertySet", {})
+
+                    # Create a dictionary of the items in "propertySet"
+                    propertySetDict = {k2: self[k1] for k1, k2 in dict(propertySet).items()}
+
+                    # Add the "propertySet" dictionary to the return data node attributes
+                    returnDn.attributes["propertySet"] = propertySetDict
+
+                    # Get the "allDns" set from the data node, or create a new empty set if it doesn't exist
+                    allDns = self.get("allDns", set())
+
+                    # Iterate over the data nodes in "allDns" and add the "rootDataNode" attribute to them
+                    for dn in allDns:
+                        if dn == returnDn:
+                            continue
+                        dn.attributes["rootDataNode"] = returnDn
+
                 return returnDn
-        
-        _DataNodeBulder__Logger.error('Returning None - there are no dataNode')
+
+        _DataNodeBuilder__Logger.error('Returning None - there are no dataNode')
         return None
-    
-    # Method returning all constructed DataNodes 
+
     def getBatchDataNodes(self):
+        """
+        Retrieves and returns all DataNodes stored in the DataNodeBuilder object.
+
+        Returns:
+        --------
+        list or None
+            Returns a list of all existing DataNodes if they exist; otherwise returns None.
+
+        Side Effects:
+        -------------
+        - Logs various messages about the internal state and time usage of the DataNodeBuilder object.
+
+        Raises:
+        -------
+        None
+
+        Notes:
+        ------
+        - This method makes use of internal logging for debugging and timing.
+        """
         self.__addGetDataNodeCounter()
-        
+
         if 'Counter' + '_setitem' in self:
             self.myLoggerTime.info("DataNode Builder the set method called - %i times"%(self['Counter' + '_setitem' ]))
         if 'DataNodeTime' in self:
+            # self['DataNodeTime'] is in nanoseconds, so divide by 1000000 to get milliseconds
             elapsedInMsDataNodeBuilder = sum(self['DataNodeTime'])/1000000
-            self.myLoggerTime.info(f"DataNode Builder used - {elapsedInMsDataNodeBuilder:.8f}ms")
-        
+            self.myLoggerTime.info(f"DataNode Builder time usage - {elapsedInMsDataNodeBuilder:.5f}ms")
+
         if dict.__contains__(self, 'dataNode'):
-            _dataNode = dict.__getitem__(self, 'dataNode')
-            
-            if len(_dataNode) > 0:  
-                
-                _DataNodeBulder__Logger.info('Returning %i dataNodes - %s'%(len(_dataNode),_dataNode))
+            existingDns = dict.__getitem__(self, 'dataNode')
 
-                return _dataNode
-        
-        _DataNodeBulder__Logger.error('Returning None - there are no dataNodes')
+            if len(existingDns) > 0:
+
+                if not getProductionModeStatus():
+                    _DataNodeBuilder__Logger.info('Returning %i dataNodes - %s'%(len(existingDns),existingDns))
+
+                return existingDns
+
+        _DataNodeBuilder__Logger.error('Returning None - there are no dataNodes')
         return None
diff --git a/domiknows/graph/dataNodeDummy.py b/domiknows/graph/dataNodeDummy.py
new file mode 100644
index 000000000..d1eeabcda
--- /dev/null
+++ b/domiknows/graph/dataNodeDummy.py
@@ -0,0 +1,355 @@
+from . import DataNode
+import torch
+from domiknows.graph import LogicalConstrain, fixedL, ifL, V
+from domiknows.graph import EnumConcept
+
+dataSizeInit = 5
+
+def findConcept(conceptName, usedGraph):
+    subGraph_keys = [key for key in usedGraph._objs]
+    for subGraphKey in subGraph_keys:
+        subGraph = usedGraph._objs[subGraphKey]
+       
+        for conceptNameItem in subGraph.concepts:
+            if conceptName == conceptNameItem:
+                concept = subGraph.concepts[conceptNameItem]
+               
+                return concept
+            
+    return None 
+
+def findConceptInfo(usedGraph, concept):
+    conceptInfo = {
+        'concept': concept,
+        'multiplicity': len(concept.enum) if isinstance(concept, EnumConcept) else 2,
+        'relation': bool(concept.has_a()),
+        'relationAttrs': {rel.name: findConcept(rel.dst.name, usedGraph) for _, rel in enumerate(concept.has_a())},
+        'contains': [contain.dst for contain in concept._out.get('contains', [])],
+        'containedIn': [contain.src for contain in concept._in.get('contains', [])],
+        'is_a': [contain.dst for contain in concept._out.get('is_a', [])]
+    }
+
+    if not conceptInfo['containedIn'] and not conceptInfo['is_a'] and not conceptInfo['relation']:
+        conceptInfo['root'] = True
+    else:
+        conceptInfo['root'] = False
+        
+    return conceptInfo
+
+def addDatanodes(concept, conceptInfos, datanodes, allDns, level=1):
+    currentConceptInfo = conceptInfos[concept.name]
+    instanceID = currentConceptInfo.get('count', 0)
+
+    for dn in datanodes:
+        dns = []
+        for _ in range(dataSizeInit * level):
+            newDN = DataNode(instanceID = instanceID, ontologyNode = currentConceptInfo['concept'])
+            dn.addChildDataNode(newDN)
+            dns.append(newDN)
+            instanceID += 1
+        
+        for contain in currentConceptInfo['contains']:
+            addDatanodes(contain, conceptInfos, dns, allDns, level = level+1)
+            
+        currentConceptInfo['count'] = currentConceptInfo.get('count', 0) + dataSizeInit * level
+        currentConceptInfo.setdefault('dns', {}).setdefault(dn.ontologyNode.name, []).extend(dns)
+        currentConceptInfo['dns'].setdefault('all', []).extend(dns)
+        
+        allDns.extend(dns)
+
+def createDummyDataNode(graph):
+    rootDataNode = None # this will be the created dummy Datanode graph root 
+    rootConcept = None # find the root concept
+    
+    conceptInfos = {} # info collected about all concepts
+    allDns = []       # all created datanodes
+    
+    # Collect concepts info from main graph
+    for currentConceptKey, currentConcept in graph.concepts.items():
+        conceptInfo = findConceptInfo(graph, currentConcept)
+        conceptInfos[currentConceptKey] = conceptInfo
+        if conceptInfo['root']:
+            rootConcept = currentConceptKey 
+        
+    # Collect concepts info from subgraph
+    for subGraphKey, subGraph in graph.subgraphs.items():
+        for currentConceptKey, currentConcept in subGraph.concepts.items():
+            conceptInfo = findConceptInfo(subGraph, currentConcept)
+            conceptInfos[currentConceptKey] = conceptInfo
+            if conceptInfo['root']:
+                rootConcept = currentConceptKey 
+                  
+    if rootConcept:
+        # Add root datanodes
+        rootConceptInfo = conceptInfos[rootConcept]
+        rootDataNode = DataNode(instanceID = 1, ontologyNode = rootConceptInfo['concept'])
+        rootDataNode.attributes["variableSet"] = {}
+        
+        # Add children datanodes
+        for contain in rootConceptInfo['contains']:
+            addDatanodes(contain, conceptInfos, [rootDataNode], allDns)
+            
+        # Add relations
+        for currentConceptKey in conceptInfos:
+            relationConceptInfo = conceptInfos[currentConceptKey]
+            relationDns = []
+            
+            if relationConceptInfo['relation'] and not relationConceptInfo['is_a']:
+                for d, attr in enumerate(relationConceptInfo['relationAttrs']):
+                    attrConceptInfo = conceptInfos[relationConceptInfo['relationAttrs'][attr].name]
+                    
+                    instanceID = relationConceptInfo.get('count', 0)
+                   
+                    for i in range(attrConceptInfo['count']):
+                        if d == 0:
+                            newDN = DataNode(instanceID = instanceID, ontologyNode = relationConceptInfo['concept'])
+                            relationDns.append(newDN)
+                            instanceID += 1
+                        else:
+                            if i < len(relationDns):
+                                newDN = relationDns[i]
+                            else:
+                                break
+                            
+                        newDN.addRelationLink(attr, attrConceptInfo["dns"]["all"][i])
+                        
+                    relationConceptInfo['count'] = relationConceptInfo.get('count', 0) + instanceID
+
+                allDns.extend(relationDns)       
+                relationConceptInfo.setdefault('dns', {})['all'] = relationDns
+                    
+        # Add probabilities
+        for currentConceptKey in conceptInfos:
+            conceptInfo = conceptInfos[currentConceptKey]
+            if conceptInfo['is_a']:
+                conceptRootConceptInfo = conceptInfos[conceptInfo['is_a'][0].name]
+                
+                if 'count' not in conceptRootConceptInfo:
+                    m = 1
+                else:
+                    m = conceptRootConceptInfo['count']
+                    
+                # Assuming m, multiplicity, and rootDataNode.current_device are already defined
+                random_tensor = torch.rand(m, conceptInfo['multiplicity'], device=rootDataNode.current_device)
+
+                # Normalize the tensor so that the sum of each row equals 1
+                final_tensor = random_tensor / random_tensor.sum(dim=1, keepdim=True)
+                
+                if 'count' not in conceptRootConceptInfo:
+                    rootDataNode.attributes['<' + conceptInfo['concept'].name + '>'] = final_tensor
+                else:
+                    rootDataNode.attributes["variableSet"][conceptRootConceptInfo['concept'].name + '/<' + conceptInfo['concept'].name + '>'] = final_tensor
+                continue
+        
+    # Iterate over the data nodes in "allDns" and add the "rootDataNode" attribute to them
+    for dn in allDns:
+        if dn == rootDataNode:
+            continue
+        dn.attributes["rootDataNode"] = rootDataNode
+        
+    if not rootDataNode.attributes.get("variableSet"):
+        # Remove "variableSet" from rootDataNode.attributes
+        rootDataNode.attributes.pop("variableSet", None)  # None is the default value if "variableSet" is not found
+          
+    return rootDataNode
+
+def construct_ls_path_string(value):
+    path_elements = []
+    if isinstance(value[1], tuple):
+        for subpath in value:
+            # Convert each subpath element to a string
+            subpath_elements = ["\'" + subpath[0] + "\'"] + [str(e) for e in subpath[1:]]
+            path_elements.append("(" + ", ".join(subpath_elements) + ")")
+    else:
+        # Convert each value element to a string
+        value_elements = ["\'" + value[0] + "\'"] + [str(e) for e in value[1:]]
+        path_elements.append(", ".join(value_elements))
+        
+    return "".join(path_elements)
+
+def lcConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, currentLc, lcResult, lcTestIndex, lcSatisfactionMsg, headLc):
+    nestedLc = []
+    for _, e in enumerate(currentLc.e):
+        if isinstance(e, LogicalConstrain):
+            currentNestedLc = next(lcIterator)
+            nestedLc.append(currentNestedLc)
+     
+    nestedLcIterator = reversed(nestedLc)  
+    
+    currentLcInput = lcSatisfactionTest['lcs'][currentLc]
+
+    if lcResult:
+        lcSatisfactionMsg += f'{currentLc}{headLc} is satisfied (True) because:\n'
+    else:
+        lcSatisfactionMsg += f'{currentLc}{headLc} is Not satisfied (False) because:\n'
+        
+    # Create an iterator for the operands of currentLc
+    operands_iterator = iter(currentLcInput)
+    index = 0
+    for operand in operands_iterator:
+        # Get the first key and its corresponding value
+        currentOperand = currentLcInput[operand][lcTestIndex][0].item()
+         
+        if isinstance(currentLc.e[index], V):
+            if len(currentLc.e) < index+1:
+                continue
+            index += 1
+            
+        if operand.startswith("_lc"):
+            lcSatisfactionMsg += f'\t{currentLc.e[index]}(**) -> {currentOperand}\n'
+       
+            # call nested lc satisfaction
+            operandResult = currentOperand
+            if isinstance(currentLc.e[index], ifL):
+                lcSatisfactionMsg = ifConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, next(nestedLcIterator),operandResult, lcTestIndex, lcSatisfactionMsg, "(**)")
+            else:
+                lcSatisfactionMsg = lcConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, next(nestedLcIterator), operandResult, lcTestIndex, lcSatisfactionMsg, "(**)")
+        elif operand.startswith("_"): # anonymous - no variable in the lc
+            lcSatisfactionMsg += f'\t{currentLc.e[index][1]} -> {currentOperand}\n'
+        else:
+            if len(currentLc.e) > index + 1 and isinstance(currentLc.e[index + 1], V) and currentLc.e[index + 1].v is not None:
+                currentV = currentLc.e[index + 1].v
+                path = construct_ls_path_string(currentV)
+                lcSatisfactionMsg += f'\t{currentLc.e[index][1]}({path}) -> {currentOperand}\n'
+            else:
+                lcSatisfactionMsg += f'\t{currentLc.e[index][1]}(\'{operand}\') -> {currentOperand}\n'
+                
+        index += 1
+            
+    return lcSatisfactionMsg
+
+def ifConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, currentLc, ifResult, lcTestIndex, lcSatisfactionMsg, headLc):
+    currentLcInput = lcSatisfactionTest['lcs'][currentLc]
+
+    nestedLc = []
+    for _, e in enumerate(currentLc.e):
+        if isinstance(e, LogicalConstrain):
+            currentNestedLc = next(lcIterator)
+            nestedLc.append(currentNestedLc)
+     
+    nestedLcIterator = reversed(nestedLc)    
+           
+    if ifResult:
+        lcSatisfactionMsg += f'{currentLc}{headLc} is satisfied (True) because:\n' 
+    else:
+        lcSatisfactionMsg += f'{currentLc}{headLc} is Not satisfied (False) because:\n'
+        
+    # Create an iterator for the operands of currentLc
+    operands_iterator = iter(currentLcInput)
+    index = 0
+    promiseIndex = None
+    conclusionIndex = None
+    for operand in operands_iterator:
+        # Get the first key and its corresponding value
+        currentOperand = currentLcInput[operand][lcTestIndex][0].item()
+        
+        if isinstance(currentLc.e[index], V):
+            if len(currentLc.e) < index+1:
+                continue
+            index += 1
+            
+        if operand.startswith("_lc"):
+            lcSatisfactionMsg += f'\t{currentLc.e[index]}(**) -> {currentOperand}\n'
+        elif operand.startswith("_"): # anonymous - no variable in the lc
+            lcSatisfactionMsg += f'\t{currentLc.e[index][1]} -> {currentOperand}\n'
+        else:
+            if len(currentLc.e) > index+1 and isinstance(currentLc.e[index+1], V) and currentLc.e[index+1].v != None: 
+                currentV = currentLc.e[index + 1].v
+                path = construct_ls_path_string(currentV)
+                lcSatisfactionMsg += f'\t{currentLc.e[index][1]}({path}) -> {currentOperand}\n'
+            else:
+                lcSatisfactionMsg += f'\t{currentLc.e[index][1]}(\'{operand}\') -> {currentOperand}\n'
+                
+        if promiseIndex == None:
+            promiseIndex = index
+            promiseResult = currentOperand
+        else:
+            conclusionIndex = index
+            conclusionResult = currentOperand
+            
+        index += 1
+        
+    if ifResult:
+        lcSatisfactionMsg += f'\t{currentLc} premise is {promiseResult} and its conclusion is {conclusionResult}\n' 
+    else:
+        lcSatisfactionMsg += f'\tWhen in {currentLc} the premise is True, the conclusion should also be True\n'
+
+    if isinstance(currentLc.e[promiseIndex], LogicalConstrain):
+        if isinstance(currentLc.e[promiseIndex], ifL):
+            lcSatisfactionMsg = ifConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, next(nestedLcIterator), promiseResult, lcTestIndex, lcSatisfactionMsg, "(**)")
+        else:
+            lcSatisfactionMsg = lcConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, next(nestedLcIterator), promiseResult, lcTestIndex, lcSatisfactionMsg, "(**)")
+            
+    if isinstance(currentLc.e[conclusionIndex], LogicalConstrain):
+        if isinstance(currentLc.e[conclusionIndex], ifL):
+            lcSatisfactionMsg = ifConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, next(nestedLcIterator), conclusionResult, lcTestIndex, lcSatisfactionMsg, "(**)")
+        else:
+            lcSatisfactionMsg = lcConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, next(nestedLcIterator), conclusionResult, lcTestIndex, lcSatisfactionMsg, "(**)")
+    
+    return lcSatisfactionMsg
+
+def satisfactionReportOfConstraints(dn):
+    m = None     
+    sampleSize = 1
+    p = sampleSize
+   
+    key = "/local/softmax"
+    dn.inferLocal()
+    
+    mySolver, _ = dn.getILPSolver(conceptsRelations = dn.collectConceptsAndRelations())
+    mySolver.current_device = dn.current_device
+    mySolver.myLcLossSampleBooleanMethods.sampleSize = sampleSize        
+    mySolver.myLcLossSampleBooleanMethods.current_device = dn.current_device
+
+    lcCounter = 0 # Count processed lcs
+    lcSatisfaction = {}
+    for graph in mySolver.myGraph: # Loop through graphs
+        for _, lc in graph.logicalConstrains.items(): # loop trough lcs in the graph
+
+            if not lc.headLC or not lc.active: # Process only active and head lcs
+                continue
+                
+            if type(lc) is fixedL: # Skip fixedL lc
+                continue
+                
+            lcCounter +=  1
+            
+            lcName = lc.lcName
+                
+            lcSatisfaction[lcName] = {}
+            lcSatisfactionTest = lcSatisfaction[lcName]
+            
+            lcResult, lcVariables, inputLc = \
+                mySolver.constructLogicalConstrains(lc, mySolver.myLcLossSampleBooleanMethods, m, dn, p, key = key, headLC = True, loss = True, sample = True)
+            lcSatisfactionTest['lcResult'] = lcResult
+            lcSatisfactionTest['lcVariables'] = lcVariables
+            lcSatisfactionTest['lcs'] = inputLc
+
+    for lcName in lcSatisfaction:
+        lcSatisfactionTest = lcSatisfaction[lcName]
+        lcSatisfactionMsgs = {}
+        lcSatisfactionMsgs["Satisfied"] = []
+        lcSatisfactionMsgs["NotSatisfied"] = []
+        lenOfLcTests = len(lcSatisfactionTest['lcResult'])
+        
+        for lcTestIndex in range(lenOfLcTests):
+            lcIterator = reversed(lcSatisfactionTest['lcs'])
+
+            currentLc = next(lcIterator)
+            if lcSatisfactionTest['lcResult'][lcTestIndex][0] == None:
+                continue
+            lcResult = not lcSatisfactionTest['lcResult'][lcTestIndex][0].item()
+            lcSatisfactionMsg = ''
+            if isinstance(currentLc, ifL):
+                lcSatisfactionMsg = ifConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, currentLc, lcResult, lcTestIndex, lcSatisfactionMsg, "")
+            else:
+                lcSatisfactionMsg = lcConstrainSatisfactionMsg(lcSatisfactionTest, lcIterator, currentLc, lcResult, lcTestIndex, lcSatisfactionMsg, "")
+            
+            if lcResult:
+                lcSatisfactionMsgs["Satisfied"].append(lcSatisfactionMsg)
+            else:
+                lcSatisfactionMsgs["NotSatisfied"].append(lcSatisfactionMsg)
+
+        lcSatisfactionTest['lcSatisfactionMsgs'] = lcSatisfactionMsgs
+        
+    return lcSatisfaction
\ No newline at end of file
diff --git a/domiknows/graph/graph.py b/domiknows/graph/graph.py
index 5dc837698..40c5c6e8b 100644
--- a/domiknows/graph/graph.py
+++ b/domiknows/graph/graph.py
@@ -1,5 +1,7 @@
 from collections import OrderedDict, namedtuple
 from itertools import chain
+import inspect
+from distutils.dep_util import newer
 
 if __package__ is None or __package__ == '':
     from base import BaseGraphTree
@@ -11,7 +13,42 @@
 
 @BaseGraphTree.localize_namespace
 class Graph(BaseGraphTree):
+    """
+    Represents a graph structure, extending from BaseGraphTree.
+    
+    Class Attributes:
+        varNameReversedMap (dict): Class-level variable to store a reversed mapping for variable names.
+        
+    Instance Attributes:
+        _concepts (OrderedDict): Stores the concepts in an ordered dictionary.
+        _logicalConstrains (OrderedDict): Stores the logical constraints in an ordered dictionary.
+        _relations (OrderedDict): Stores the relations in an ordered dictionary.
+        _batch (NoneType): Currently not set, reserved for batch operations.
+        cacheRootConcepts (dict): Cache for root concepts, initialized as an empty dictionary.
+        auto_constraint (Any): Specifies whether to automatically create constraints.
+        reuse_model (bool): Flag to indicate whether to reuse an existing model.
+        ontology (tuple or Graph.Ontology): The ontology associated with the graph.
+    """
+    varNameReversedMap = {}  # Class variable
+        
     def __init__(self, name=None, ontology=None, iri=None, local=None, auto_constraint=None, reuse_model=False):
+        '''Initializes an instance of the graph class.
+    
+        Args:
+        name (str, optional): The name of the object. Defaults to None.
+        ontology (Graph.Ontology or str, optional): The ontology associated with the object. If provided as a string, it will be paired with the 'local' parameter. Defaults to None.
+        iri (str, optional): The IRI (Internationalized Resource Identifier) of the ontology. Used when 'ontology' is None. Defaults to None.
+        local (str, optional): The local namespace. Paired with either 'iri' or 'ontology' if provided as a string. Defaults to None.
+        auto_constraint (Any, optional): Specifies whether to automatically create constraints. The type and use are determined by the class. Defaults to None.
+        reuse_model (bool, optional): Flag to indicate whether to reuse an existing model. Defaults to False.
+    
+        Attributes:
+        _concepts (OrderedDict): Stores the concepts in an ordered dictionary.
+        _logicalConstrains (OrderedDict): Stores the logical constraints in an ordered dictionary.
+        _relations (OrderedDict): Stores the relations in an ordered dictionary.
+        _batch (NoneType): Currently not set, reserved for batch operations.
+        cacheRootConcepts (dict): Cache for root concepts, initialized as an empty dictionary.
+        '''
         BaseGraphTree.__init__(self, name)
         if ontology is None:
             self.ontology = (iri, local)
@@ -24,36 +61,749 @@ def __init__(self, name=None, ontology=None, iri=None, local=None, auto_constrai
         self._concepts = OrderedDict()
         self._logicalConstrains = OrderedDict()
         self._relations = OrderedDict()
+        self._batch = None
+        self.cacheRootConcepts = {}
 
     def __iter__(self):
         yield from BaseGraphTree.__iter__(self)
         yield from self._concepts
         yield from self._relations
 
+      
+    def findRootConceptOrRelation(self, relationConcept):
+        # If the result is already in cache, return it
+        if relationConcept in self.cacheRootConcepts:
+            return self.cacheRootConcepts[relationConcept]
+
+        try:
+            isAs = relationConcept.is_a()
+        except (AttributeError, TypeError):
+            isAs = []
+        
+        for _isA in isAs:
+            parentRelationConcept = _isA.dst
+            
+            # Recursive call, but result is cached if previously computed
+            root = self.findRootConceptOrRelation(parentRelationConcept)
+            
+            # Store result in cache
+            self.cacheRootConcepts[relationConcept] = root
+            return root
+
+        # If the provided concept or relation is root (has no parents)
+        # Store result in cache
+        self.cacheRootConcepts[relationConcept] = relationConcept
+        
+        return relationConcept
+    
+    def findConcept(self, conceptName):
+        '''Finds the root concept or relation for a given concept or relation.
+    
+        This method performs a recursive search to identify the root concept or relation.
+        If a result has been previously computed, it retrieves the result from cache to avoid redundant computation.
+        
+        Args:
+        relationConcept (Any): The concept or relation for which the root is to be found. The type depends on the implementation.
+    
+        Returns:
+        Any: The root concept or relation.
+    
+        Raises:
+        AttributeError, TypeError: If the attribute 'is_a' is not available or if the type is incorrect.
+        '''
+        subGraph_keys = [key for key in self._objs]
+        for subGraphKey in subGraph_keys:
+            subGraph = self._objs[subGraphKey]
+           
+            for conceptNameItem in subGraph.concepts:
+                if conceptName == conceptNameItem:
+                    concept = subGraph.concepts[conceptNameItem]
+                   
+                    return concept
+            
+        return None 
+
+    def findConceptInfo(self, concept):
+        '''Finds and returns various information related to a given concept.
+    
+        This method compiles a dictionary containing different attributes and relations of the concept. 
+        It looks for the 'has_a', 'contains', and 'is_a' relationships and also identifies if the concept is a root.
+        
+        Args:
+        concept (Any): The concept for which information is to be found. The type depends on the implementation.
+        
+        Returns:
+        OrderedDict: A dictionary containing the following keys:
+            - 'concept': The original concept.
+            - 'relation': Boolean indicating if the concept has a 'has_a' relationship.
+            - 'has_a': List of 'has_a' relations.
+            - 'relationAttrs': An ordered dictionary of relation attributes.
+            - 'contains': List of concepts that the original concept contains.
+            - 'containedIn': List of concepts that contain the original concept.
+            - 'is_a': List of concepts that the original concept is a type of.
+            - 'root': Boolean indicating if the concept is a root concept.
+        '''
+        has_a = concept.has_a()
+        
+        if not has_a:
+            is_a = [contain.dst for contain in concept._out.get('is_a', [])]
+            
+            for i in is_a:
+                has_a = i.has_a()
+                
+                if has_a:
+                    break
+        
+        conceptInfo = OrderedDict([
+            ('concept', concept),
+            ('relation', bool(concept.has_a())),
+            ('has_a', concept.has_a()),
+            ('relationAttrs', OrderedDict((rel, self.findConcept(rel.dst.name)) for _, rel in enumerate(has_a))),
+            ('contains', [contain.dst for contain in concept._out.get('contains', [])]),
+            ('containedIn', [contain.src for contain in concept._in.get('contains', [])]),
+            ('is_a', [contain.dst for contain in concept._out.get('is_a', [])])
+        ])
+    
+        if not conceptInfo['containedIn'] and not conceptInfo['is_a'] and not conceptInfo['relation']:
+            conceptInfo['root'] = True
+        else:
+            conceptInfo['root'] = False
+            
+        return conceptInfo
+        
+    # find all variables defined in the logical constrain and report error if some of them are defined more than once
+    def find_lc_variable(self, lc, found_variables=None, headLc=None):
+        '''Finds all variables defined in a logical constraint and reports errors for duplicates.
+    
+        This method traverses through the elements of a logical constraint to find all the variables 
+        that have been defined. It checks for incorrect cardinality definitions, multiple definitions of the 
+        same variable, and variables that are not associated with any concept among other things.
+        
+        Args:
+        lc (LogicalConstrain): The logical constraint to be processed.
+        found_variables (dict, optional): Dictionary to store found variables. The key is the variable name, 
+                                          and the value is a tuple containing the logical constraint, variable name, 
+                                          and the concept associated with the variable.
+                                          Defaults to None.
+        headLc (str, optional): The name of the parent logical constraint. Defaults to None.
+        
+        Returns:
+        dict: A dictionary containing all found variables.
+        
+        Raises:
+        Exception: If there are issues with the variable definitions or cardinality.
+        '''
+        if lc.cardinalityException:
+            if lc.name != headLc:
+                exceptionStr1 = f"{lc.typeName} {headLc} has incorrect cardinality definition in nested {lc} logical operator- " 
+            else:
+                exceptionStr1 = f"{lc.typeName} {headLc} has incorrect cardinality definition - "
+            
+            exceptionStr2 = f"integer {lc.cardinalityException} has to be last element in the same Logical operator for counting or existing logical operators!"
+            raise Exception(f"{exceptionStr1} {exceptionStr2}")
+
+        if found_variables is None:
+            found_variables = {}
+
+        from domiknows.graph import V, LogicalConstrain, LcElement
+
+        e_before = None
+        for e in lc.e:
+            # checking if element is a variable
+            if isinstance(e, V) and e and e.name:
+                variable_name = e.name
+                if e_before:
+                    if variable_name in found_variables:
+                        exceptionStr1 = f"In logical constraint {headLc} {lc} variable {variable_name} associated with concept {e_before[1]} already defined "
+                        exceptionStr2 = f"in {found_variables[variable_name][0]} and associated with concept {found_variables[variable_name][2][1]}"
+                        raise Exception(exceptionStr1 + exceptionStr2)
+            
+                    variable_info = (lc, variable_name, e_before)
+                    found_variables[variable_name] = variable_info
+                else:
+                    exceptionStr = f"In logical constraint {headLc} {lc} variable {variable_name} is not associated with any concept"
+                    raise Exception(exceptionStr)
+                
+            # checking for extra variable:
+            elif e and isinstance(e, tuple) and e[0] == 'extraV':
+                predicate = lc.e[0][1]
+                exceptionStr1 = f"Logical constraint {headLc} {lc}: Each predicate can only have one new variable definition. For the predicate {predicate}, you have used both {e[1]} and {e[2]} as new variables."
+                exceptionStr2 = f"Either wrap both under on variable, if you intended to initialize {e[1]} based on another value, then the second argument should be a path=(...)."
+                raise Exception(f"{exceptionStr1} {exceptionStr2}")
+            # checking if element is a tuple 
+            elif isinstance(e, tuple) and e and isinstance(e[0], LcElement) and not isinstance(e[0], LogicalConstrain):
+                self.find_lc_variable(e[0], found_variables=found_variables, headLc=headLc)
+                current_lc_element = e[0]
+                current_lc_element_concepts = [c for c in current_lc_element.e if isinstance(c, tuple) and not isinstance(c, V)]
+
+                if len(current_lc_element_concepts) != len(e[1]):
+                    raise Exception(f"Logical constraint {headLc} {lc} has incorrect definition of combination {e} - number of variables does not match number of concepts in combination")
+
+                if len(e) >= 2 and isinstance(e[1], tuple):
+                    for v in e[1]:
+                        if not isinstance(v, str):
+                            raise Exception(f"Logical constraint {headLc} {lc} has incorrect definition of combination {e} - all variables should be strings")
+
+                    for index, v in enumerate(e[1]):
+                        variable_name = v
+                        variable_info = (lc, variable_name, current_lc_element_concepts[index])
+                        found_variables[variable_name] = variable_info
+
+            # Checking if element is a LogicalConstrain
+            elif isinstance(e, LogicalConstrain):
+                self.find_lc_variable(e, found_variables=found_variables, headLc=headLc)
+
+            e_before = e
+
+        return found_variables
+
+    def check_if_all_used_variables_are_defined(self, lc, found_variables, used_variables=None, headLc=None):
+        '''Checks if all variables used in a logical constraint are properly defined.
+    
+        This method traverses through the elements of a logical constraint to identify all the variables 
+        that are used but not defined. It also handles variable names in different types of paths.
+        
+        Args:
+        lc (LogicalConstrain): The logical constraint to be processed.
+        found_variables (dict): Dictionary containing all variables that have been defined.
+                                The key is the variable name and the value is a tuple containing information
+                                about the variable.
+        used_variables (dict, optional): Dictionary to store variables that are used. The key is the variable name,
+                                         and the value is a list of tuples, each containing the logical constraint,
+                                         variable name, the type of the element that uses it, and the path to the variable.
+                                         Defaults to None.
+        headLc (str, optional): The name of the parent logical constraint. Defaults to None.
+        
+        Returns:
+        dict: A dictionary containing all used variables.
+        
+        Raises:
+        Exception: If there are variables that are used but not defined, or if there are errors in the path definitions.
+        '''
+        from .logicalConstrain import eqL
+        
+        if used_variables is None:
+            used_variables = {}
+
+        from domiknows.graph import V, LogicalConstrain
+
+        def handle_variable_name(lc_variable_name, lcPath):
+            if lc_variable_name not in found_variables:
+                raise Exception(f"Variable {lc_variable_name} found in {headLc} {lc} is not defined. You should first use {lc_variable_name} without putting it in a path to define it.")
+
+            if lc_variable_name not in used_variables:
+                used_variables[lc_variable_name] = []
+
+            lcElementType = lc.e[i-1]
+            variable_info = (lc, lc_variable_name, lcElementType, lcPath)
+            used_variables[lc_variable_name].append(variable_info)
+
+        for i, e in enumerate(lc.e):
+            if isinstance(e, V) and e.v: # has path
+                if isinstance(e.v, eqL):
+                    continue
+                elif isinstance(e.v, str):
+                    handle_variable_name(e.v, e.v)
+                elif isinstance(e, tuple):
+                    if isinstance(e.v[0], str): # single path
+                        handle_variable_name(e.v[0], e.v)
+                    elif isinstance(e.v[0], tuple): # path union
+                        for t in e.v:
+                            if isinstance(t[0], str):
+                                handle_variable_name(t[0], t)
+                            else:
+                                raise Exception(f"Path {t} found in {headLc} {lc} is not correct")
+                    else:
+                        raise Exception(f"Path {e} found in {headLc} {lc} is not correct")
+                else:
+                    raise Exception(f"Path {e} found in {headLc} {lc} is not correct")
+            elif isinstance(e, LogicalConstrain):
+                self.check_if_all_used_variables_are_defined(e, found_variables, used_variables=used_variables, headLc=headLc)
+
+        return used_variables
+
+    def getPathStr(self, path):
+        '''Converts a path of concepts and relations to a string representation.
+    
+        This method iterates over a given path, which can include instances of the Relation and Concept classes,
+        and constructs a string representation of the path.
+        
+        Args:
+        path (list): A list of path elements which can be instances of Relation or Concept classes.
+                     The first element in the list is not processed, and the list should be non-empty.
+        
+        Returns:
+        str: A string representation of the path, excluding the first element.
+        '''
+        from .concept import Concept
+        from .relation import Relation
+        pathStr = ""
+        for pathElement in path[1:]:
+            if isinstance(pathElement, (Relation,)):
+                if pathElement.var_name:
+                    pathStr += pathElement.var_name + " "
+                else:
+                    pathStr += pathElement.name + " "
+            elif isinstance(pathElement, (Concept,)):
+                pathStr += pathElement.var_name + " "
+            else:
+                pathStr += str(pathElement)
+               
+        return pathStr.strip()
+                
+    def check_path(self, path, resultConcept, variableConceptParent, lc_name, foundVariables, variableName):
+        '''Checks the validity of a path in terms of relations and concepts.
+    
+        This function checks the validity of a given path, including ensuring that each relation
+        or concept in the path has the correct type. It raises exceptions with informative error messages 
+        if the path is not valid.
+    
+        Args:
+        path (list): The path to check, starting from the source concept.
+        resultConcept (tuple): The expected end concept of the path.
+        variableConceptParent (Concept): The parent concept for the source of the path.
+        lc_name (str): The name of the logical constraint where the path is defined.
+        foundVariables (dict): Dictionary of found variables in the scope.
+        variableName (str): The name of the variable being checked.
+    
+        Raises:
+        Exception: Various types of exceptions are raised for different kinds of path invalidity.
+        '''
+        from .relation import IsA, HasA, Relation
+        from .logicalConstrain import eqL
+        from .concept import Concept
+        
+        requiredLeftConcept = variableConceptParent.name # path element has to be relation with this type to the left
+        requiredEndOfPathConcept = resultConcept[1] # path has to result in this concept
+        requiredEndOfPathConceptRoot =  self.findRootConceptOrRelation(resultConcept[0]).name  
+        expectedRightConcept =  None
+        lastPathElement = False
+        pathStr = self.getPathStr(path)
+        pathVariable = path[0]
+        pathPart = path[0]
+            
+        if len(path) == 1:
+            if requiredLeftConcept == requiredEndOfPathConceptRoot:
+                return
+            else:
+                exceptionStr1 = f"The variable {pathVariable}, defined in the path for {lc_name} is not valid. The concept of {pathVariable} is a of type {requiredLeftConcept},"
+                exceptionStr2 = f"but the required concept by the logical constraint element is {requiredEndOfPathConceptRoot}."
+                exceptionStr3 = f"The variable used inside the path should match its type with {requiredEndOfPathConceptRoot}."
+                raise Exception(f"{exceptionStr1} {exceptionStr2} {exceptionStr3}")
+            
+        for pathIndex, pathElement in enumerate(path[1:], start=1):   
+            if isinstance(pathElement, (eqL,)):
+                continue
+            if isinstance(pathElement, (str,)): # It is a string check if we have corresponding relation in the graph
+                if pathElement in self.varNameReversedMap:
+                    pathElement = self.varNameReversedMap[pathElement]
+                else:
+                    exceptionStr1 = f"The Path '{pathStr}' from the variable {pathVariable}, defined in {lc_name} is not valid."
+                    exceptionStr2 = f"The required source type after {pathPart} is a {requiredLeftConcept},"
+                    exceptionStr3 = f"but the used variable {pathElement} is a string which is not a valid name of a graph relationship."
+                    raise Exception(f"{exceptionStr1} {exceptionStr2}")
+                
+            if pathIndex < len(path) - 1:
+                expectedRightConcept = pathElement.dst.name
+                expectedRightConceptRoot = expectedRightConcept
+            else:
+                expectedRightConcept = requiredEndOfPathConcept
+                lastPathElement = True
+                expectedRightConceptRoot = requiredEndOfPathConceptRoot                
+            
+            if isinstance(pathElement, (HasA, IsA, Relation)):
+                pathElementSrc = pathElement.src.name
+                pathElementDst = pathElement.dst.name
+                pathElementVarName = pathElement.var_name if pathElement.var_name else ""
+
+                # Check if there is a problem with reversed usage of the current path element - it has to be possible to reverse the order to fix it
+                if requiredLeftConcept == pathElementDst and expectedRightConceptRoot == pathElementSrc:                    
+                    exceptionStr1 = f"The Path '{pathStr}' from the variable {pathVariable}, defined in {lc_name} is not valid"
+                    exceptionStr2 = f"The relation {pathElementVarName} is from a {pathElementSrc} to a {pathElementDst}, but you have used it from a {pathElementDst} to a {pathElementSrc}."
+                    if not pathElement.is_reversed:
+                        exceptionStr3 = f"You can change '{pathElement.var_name}' to '{pathElement.var_name}.reversed' to go from {pathElementDst} to the {pathElementSrc}, which is what is required here."
+                    else:
+                        exceptionStr3 = f"You can change  '{pathElement.var_name}.reversed' to '{pathElement.var_name}' to go from {pathElementSrc} to the {pathElementDst}, which is what is required here."
+                        f"You can use without the .reversed property to change the direction."
+                    raise Exception(f"{exceptionStr1} {exceptionStr2} {exceptionStr3}")
+                # Check if the current path element is correctly connected to the left (source) - has matching type
+                elif requiredLeftConcept != pathElementSrc:
+                    exceptionStr1 = f"The Path '{pathStr}' from the variable {pathVariable}, defined in {lc_name} is not valid."
+                    exceptionStr2 = f"The required source type after {pathPart} is a {requiredLeftConcept},"
+                    exceptionStr3 = f"but the used variable {pathElementVarName} is a relationship defined between a {pathElementSrc} and a {pathElementDst}, which is not correctly used here."
+                    raise Exception(f"{exceptionStr1} {exceptionStr2} {exceptionStr3}")
+                # Check if the current path element is correctly connected to the right (destination) - has matching type
+                elif expectedRightConceptRoot != pathElementDst: 
+                    exceptionStr1 = f"The Path '{pathStr}' from the variable {pathVariable}, defined in {lc_name} is not valid."
+                    if lastPathElement: # if this is the last element it has to match the concept in which this path is embedded
+                        exceptionStr2 = f"The required destination type after {pathPart} is a {expectedRightConcept}."
+                    else: # if this it intermediary path element that if is expected that it will match next path element source type
+                        exceptionStr2 = f"The expected destination type after {pathPart} is a {expectedRightConcept}."
+                    exceptionStr3 = f"The used variable {pathElementVarName} is a relationship defined between a {pathElementSrc} and a {pathElementDst}, which is not correctly used here."
+                    raise Exception(f"{exceptionStr1} {exceptionStr2} {exceptionStr3}")
+                
+                # Move along the path with the requiredLeftConcept and pathVariable
+                requiredLeftConcept = pathElementDst
+                pathPart += " " + pathElementVarName
+            else:
+                if isinstance(pathElement, (Concept,)):
+                    exceptionStr1 = f"You have used the notion {expectedRightConcept}(path=('{pathVariable}', {pathStr})) which is incorrect."
+                    exceptionStr2 = f"{pathElement} is a concept and cannot be used as part of the path."
+                    exceptionStr3 = f"- If you meant that '{pathVariable}' should be of type {expectedRightConcept}: {expectedRightConcept}(path=('{pathVariable}'))"
+                    exceptionStr4 = f"- If you meant another entity 'y' should be of type {expectedRightConcept} which is somehow related to '{pathVariable}': {expectedRightConcept}(path=('x', edge1, edge2, ...))"
+                    exceptionStr5 = f"where edge1, edge2, ... are relations that connect '{pathVariable}' to 'y'."
+                    raise Exception(f"{exceptionStr1} {exceptionStr2} {exceptionStr3} {exceptionStr4} {exceptionStr5}")
+                else: # all other types not allowed in path
+                    pathElementType = type(pathElement)
+                    exceptionStr1 = f"The Path '{pathStr}' from the variable {pathVariable}, after {pathPart} is not valid."
+                    exceptionStr2 = f"The used variable {pathElement} is a {pathElementType}, path element can be only relation or eqL logical constraint used to filter candidates in the path."
+                    raise Exception(f"{exceptionStr1} {exceptionStr2}")
+
+
+    def are_keys_new(self, given_dict, dict_list):
+        """
+        Check if all keys in 'given_dict' are not present in any dictionary within 'dict_list'.
+
+        This method iterates over each key in 'given_dict' and checks if it exists in any of the dictionaries
+        contained within 'dict_list'. If a key from 'given_dict' is found in any dictionary in 'dict_list',
+        the method returns False, indicating that not all keys are new. Otherwise, it returns True,
+        indicating all keys in 'given_dict' are new (i.e., not present in any dictionary in 'dict_list').
+
+        Parameters:
+        given_dict (dict): A dictionary whose keys are to be checked.
+        dict_list (list of dict): A list of dictionaries against which the keys of 'given_dict' are to be checked.
+
+        Returns:
+        bool: True if all keys in 'given_dict' are new, False otherwise.
+        """
+        for key in given_dict:
+            for d in dict_list:
+                if key in d:
+                    return False
+        return True
+
+    def collectVarMaps(self, lc, varMapsList):
+        """
+        Collects variable mappings (VarMaps) from a logical constraint (lc) and updates the list of collected VarMaps.
+    
+        This method recursively traverses the elements of the logical constraint 'lc' to identify and process VarMaps.
+        It differentiates between the definition of new variables and the usage of existing ones. For new variables, 
+        it clones the current VarMap, adds the name of the logical constraint, and appends it to 'varMapsList'. For 
+        existing variables, it updates the path variable in the current VarMap to match the one used in their 
+        definition. The method modifies 'lc' by removing VarMaps that define new variables.
+    
+        Parameters:
+        lc (LogicalConstrain): The logical constraint from which VarMaps are to be collected.
+        varMapsList (list): A list that accumulates VarMaps. This list collects only the definitions of variables.
+    
+        Returns:
+        list: The updated list of variable mappings (VarMaps) after processing 'lc'.
+        
+        Note:
+        - The method assumes the existence of specific types and structures within 'lc', such as 'VarMaps' tuples.
+        - The method is recursive and alters the structure of 'lc' by removing defining VarMaps.
+        """
+        from domiknows.graph import LogicalConstrain
+        import copy
+        
+        newE = []
+        # collect VarMaps from lc
+        for e in lc.e:
+            if isinstance(e, LogicalConstrain):
+                self.collectVarMaps(e, varMapsList) # recursive
+                newE.append(e)
+            # check if VarMap
+            elif isinstance(e, tuple) and e[0] == 'VarMaps':
+                currentVarMap = e[1]
+                
+                # check if variables in the current VarMap are new, have not been found already
+                # If they are new it means it it their definition in the lc
+                if self.are_keys_new(currentVarMap, varMapsList):
+                    cloned_CurrentVarMaps = copy.deepcopy(currentVarMap)
+                    # add info about the lc to varMap  - this is the lc in which this variables are defined
+                    cloned_CurrentVarMaps["lcName"] = lc.name 
+                    
+                    # Add this VarMaps to the collected VarMapss in the varMapsList
+                    # This list collect only definition of variables 
+                    # The current varMap will be removed from the current lc
+                    varMapsList.append(cloned_CurrentVarMaps)
+                else:
+                    # variables in VarMaps are already found 
+                    # It means that this is the usage of these variables in the lc
+                    for variableName in currentVarMap:
+                        # Find previous definition in varMapsList of the variable in the current VarMap
+                        definedVaribleList = [d.get(variableName, None) for d in varMapsList]
+                        
+                        if definedVaribleList:
+                            definedVarible = definedVaribleList[0]
+                            variable = currentVarMap[variableName]
+                        
+                            # Update the path variable in the current varMap to the one used in variable definition
+                            if isinstance(variable, tuple) and len(variable) > 1 and isinstance(variable[1], tuple):
+                                new_inner_tuple = (definedVarible[1][0],) + variable[1][1:]
+                                variable = (variable[0], new_inner_tuple) + variable[2:]
+                                currentVarMap[variableName] = variable
+
+                    # keep this varMap in the lc - it will be used when processing the lc variable syntax
+                    newE.append(e)
+            else:
+                # it is not varMap - keep it in lc
+                newE.append(e)
+              
+        lc.e = newE # Update logical constraint element - defining VarMaps are removed
+        return varMapsList
+                   
+    def handleVarsPath(self, lc, varMaps):
+        """
+        Processes and updates the variable paths in a logical constraint (lc) based on the mappings provided in varMaps.
+    
+        This method iterates through the elements of 'lc' and performs various transformations based on the type
+        of each element and the presence of variable mappings in 'varMaps'. The method handles nested logical 
+        constraints recursively, updates variables already in V form, and modifies variable paths using mappings 
+        from 'varMaps'. Additionally, it removes all 'VarMaps' elements from 'lc'.
+    
+        Parameters:
+        lc (LogicalConstrain): The logical constraint to be processed.
+        varMaps (dict): A dictionary containing mappings of variable names to their respective V instances or paths.
+    
+        Note:
+        - The method assumes a specific structure of 'lc' and 'varMaps', with 'lc' containing elements like 
+          LogicalConstrain, V, Concept, and tuples with 'VarMaps'.
+        - It employs a flag 'needsVariableUpdate' to track if the next element requires variable path updates.
+        - The method is recursive for nested logical constraints and alters the structure of 'lc'.
+        """
+
+        from domiknows.graph import V, LogicalConstrain, Concept
+        
+        # process logical constraint variable syntax is used - translate it to the path V syntax of logical constraints 
+        newE = []
+        needsVariableUpdate = False # flag set by lc element concept if VarMap is present in the current lc
+        for i, e in enumerate(lc.e):
+            if isinstance(e, LogicalConstrain):
+                self.handleVarsPath(e, varMaps) # recursive
+            elif isinstance(e, V) and e.name in varMaps: # check if the current lc element is already in the V  form and its name is in varMaps
+                # this is the variable syntax for a single variable usage
+                # replace the V with the mapping for this found in varMap
+                e = varMaps[e.name]
+            elif needsVariableUpdate: # ths flag was set by the previous lc element
+                usedVarMap = lc.e[i+1][1] # get varMaps to use
+                usedVarMapIt= iter(usedVarMap.items())
+                firstUsedV = next(usedVarMapIt)[1][1]
+                secondUsedV = next(usedVarMapIt)[1][1]
+
+                # create new paths based on variables from varMaps
+                firstNewPath = firstUsedV + (firstUsedV[1].reversed,)
+                secondNewPath = secondUsedV + (secondUsedV[1].reversed,)
+                
+                path = (firstNewPath, secondNewPath)  
+                
+                updated_e = V(name="", v=path)
+                e = updated_e
+                needsVariableUpdate = False
+            elif isinstance(e, tuple) and isinstance(e[0], Concept) and len(lc.e) > i+2:
+                if isinstance(lc.e[i+2], tuple) and lc.e[i+2][0] == 'VarMaps':
+                    # set the flag if the next element needs to be updated because the variable syntax is used
+                    needsVariableUpdate = True
+            
+            if isinstance(e, tuple) and e[0] == 'VarMaps':
+                pass # removed all VarMaps from lc
+            else:
+                newE.append(e)
+                    
+        lc.e = newE #Update logical constraint element - all VarMaps are removed
+                
+    def __exit__(self, exc_type, exc_value, traceback):
+        '''Handles the exiting logic for the context manager.
+    
+        This method performs clean-up operations like mapping variable names and 
+        validating logical constraints. It is automatically called when exiting the 
+        'with' block of the context manager.
+    
+        Args:
+        exc_type (type): The type of the exception that caused the context manager to exit.
+        exc_value (Exception): The instance of the exception that caused the context manager to exit.
+        traceback (traceback): A traceback object encapsulating the call stack at the point 
+                               where the exception originally occurred.
+    
+        Side-effects:
+        Modifies internal state to include variable name mappings and validates logical constraints.
+        '''
+        super().__exit__(exc_type, exc_value, traceback)
+            
+        #image_name = "./image" + self.name
+        #self.visualize(image_name, open_image=True)
+        
+        # Get the current frame and then go one level back
+        frame = inspect.currentframe().f_back
+
+        from . import Concept
+        from .relation import IsA, HasA
+    
+        # --- Iterate through all local variables in that frame
+        for var_name, var_value in frame.f_locals.items():
+            # Check if any of them are instances of the Concept class or Relation subclass
+            if isinstance(var_value, (Concept, HasA, IsA)):
+                # If they are, and their var_name attribute is not already set,
+                # set it to the name of the variable they are stored in.
+                if var_value.var_name is None:
+                    var_value.var_name = var_name
+                    self.varNameReversedMap[var_value.name] = var_value
+                    if isinstance(var_value, (HasA,)):
+                        var_value.reversed.var_name = var_name + ".reversed"
+                        self.varNameReversedMap[var_value.reversed.name] = var_value.reversed
+
+        # --- Process logical constraints variable syntax
+        for lc_name, lc in self.logicalConstrains.items():
+            if not lc.active or not lc.headLC:
+                continue
+
+            # collect VarMaps - store info about lc variable syntax if used in this logical constraint
+            varMapsList = self.collectVarMaps(lc, [])
+            
+            # process variable syntax - translate it to the path syntax
+            if varMapsList:
+                self.handleVarsPath(lc, varMapsList[0])
+                    
+        # --- Check if the logical constrains are correct ---
+        
+        lc_info = {}
+        LcInfo = namedtuple('CurrentLcInfo', ['foundVariables', 'usedVariables', 'headLcName'])
+        
+        # --- Gather information about variables used and defined in the logical constrains and 
+        #     report errors if some of them are not defined and used or defined more than once
+        for lc_name, lc in self.logicalConstrains.items():
+            if not lc.active or not lc.headLC:
+                continue
+                
+            # find variable defined in the logical constrain - report error if some of them are defined more than once
+            found_variables = self.find_lc_variable(lc, headLc=lc.name)
+
+            # find all variables used in the logical constrain - report error if some of them are not defined
+            # gather paths defined in the logical constrain per variable
+            used_variables = self.check_if_all_used_variables_are_defined(lc, found_variables, headLc=lc.name)
+            
+            # save information about variables used and defined in the logical constrain
+            current_lc_info = LcInfo(found_variables, used_variables, lc.name)
+            lc_info[lc_name] = current_lc_info
+        
+        # --- Check if the paths defined in the logical constrains are correct
+        for lc_name, lc in self.logicalConstrains.items():
+            if not lc.active or not lc.headLC:
+                continue
+            
+            # current logical constrain info and variables found and used in the current logical constrain
+            current_lc_info = lc_info[lc_name]
+            usedVariables = current_lc_info.usedVariables
+            foundVariables = current_lc_info.foundVariables
+            headLcName = current_lc_info.headLcName
+            
+            # loop over all variables used in the logical constrain
+            for variableName, pathInfos in usedVariables.items():
+                # get information about the variable in the found variables record
+                variableConcept = foundVariables[variableName][2][0]
+                #variableConceptWhat = variableConcept.what()
+                
+                # get the root parent of the variable concept
+                variableConceptParent = self.findRootConceptOrRelation(variableConcept)
+                
+                # loop over all paths defined using the variable as starting point
+                for pathInfo in pathInfos:
+                    path = pathInfo[3]
+                    resultConcept = pathInfo[2]
+                    
+                    if isinstance(path[0], tuple): # this path is a combination of paths 
+                        for subpath in path: 
+                            if len(subpath) < 1:  
+                                continue  # skip this subpath it is empty
+                                
+                            self.check_path(subpath, resultConcept, variableConceptParent, headLcName, foundVariables, variableName)
+                            
+                    else: # this path is a single path
+                        if len(path) < 1:
+                            continue # skip this path it is empty
+                            
+                        self.check_path(path, resultConcept, variableConceptParent, headLcName, foundVariables, variableName)
+                       
     @property
     def ontology(self):
+        '''Gets the ontology associated with the object.
+    
+        Returns:
+        Graph.Ontology: The ontology associated with the object.
+        '''
         return self._ontology
 
-    @property
-    def auto_constraint(self):
-        if self._auto_constraint is None and self.sup:
-            return self.sup.auto_constraint
-        return self._auto_constraint or False  # if None, return False instead
-
-    @auto_constraint.setter
-    def auto_constraint(self, value):
-        self._auto_constraint = value
-
     @ontology.setter
     def ontology(self, ontology):
+        '''Sets the ontology for the object.
+        
+        Args:
+        ontology (Graph.Ontology|str|tuple): The ontology to set. This can be either 
+                                             an instance of Graph.Ontology, a string, 
+                                             or a tuple containing two elements.
+    
+        Raises:
+        TypeError: If the provided ontology is not of the correct type.
+        '''
         if isinstance(ontology, Graph.Ontology):
             self._ontology = ontology
         elif isinstance(ontology, str):
             self._ontology = Graph.Ontology(ontology)
         elif isinstance(ontology, tuple) and len(ontology) == 2:
             self._ontology = Graph.Ontology(*ontology)
+        else:
+            raise TypeError("Invalid ontology type.")
 
+    @property
+    def batch(self):
+        '''Gets the current batch.
+        
+        Returns:
+        Any: The current batch.
+        '''
+        return self._batch
+    
+    @batch.setter
+    def batch(self, value):
+        '''Sets the batch.
+    
+        Args:
+        value (Any): The batch value to set.
+        '''
+        self._batch = value
+    
+    @property
+    def auto_constraint(self):
+        '''Determines whether to automatically enforce constraints.
+    
+        If the auto_constraint is not set, it will defer to the `sup` attribute.
+    
+        Returns:
+        bool: True if constraints should be automatically enforced, False otherwise.
+        '''
+        if self._auto_constraint is None and self.sup:
+            return self.sup.auto_constraint
+        return self._auto_constraint or False  # if None, return False instead
+    
+    @auto_constraint.setter
+    def auto_constraint(self, value):
+        '''Sets whether to automatically enforce constraints.
+        
+        Args:
+        value (bool): The value to set for automatic constraint enforcement.
+        '''
+        self._auto_constraint = value
+    
     def get_properties(self, *tests):
+        '''Finds and returns properties that meet the given conditions.
+    
+        Args:
+        tests (callable): Variable-length argument list of test conditions each property must pass.
+    
+        Returns:
+        list: A list of properties that meet all the given test conditions.
+
+        '''
         def func(node):
             if isinstance(node, Property):
                 for test in tests:
@@ -63,11 +813,27 @@ def func(node):
                     return node
             return None
         return list(self.traversal_apply(func))
-
+    
     def get_sensors(self, *tests):
+        '''Finds and returns sensors that meet the given conditions.
+    
+        Args:
+        tests (callable): Variable-length argument list of test conditions each sensor must pass.
+    
+        Returns:
+        list: A list of sensors that meet all the given test conditions.
+        '''
         return list(chain(*(prop.find(*tests) for prop in self.get_properties())))
 
     def get_apply(self, name):
+        '''Finds and returns the concept or relation with the given name.
+    
+        Args:
+        name (str): The name of the concept or relation to find.
+    
+        Returns:
+        Object: The concept or relation with the specified name, or the result of BaseGraphTree.get_apply if not found.
+        '''
         if name in self.concepts:
             return self.concepts[name]
         if name in self.relations:
@@ -75,6 +841,21 @@ def get_apply(self, name):
         return BaseGraphTree.get_apply(self, name)
 
     def set_apply(self, name, sub):
+        '''Sets the `Concept` or `Relation` for a given name in the object.
+
+        If the sub is a `Graph`, it delegates to the parent class `BaseGraphTree.set_apply`.
+        If it is a `Concept` or `Relation`, it adds it to the respective dictionary.
+    
+        Args:
+        name (str): The name to set for the `Concept` or `Relation`.
+        sub (Graph|Concept|Relation): The object to set for the name. This can be an instance of `Graph`, 
+                                      `Concept`, or `Relation`.
+    
+        TODO:
+        1. Handle cases where a concept has the same name as a subgraph.
+        2. Handle other types for the 'sub' parameter.
+    
+        '''
         if __package__ is None or __package__ == '':
             from concept import Concept
             from relation import Relation
@@ -86,14 +867,31 @@ def set_apply(self, name, sub):
             BaseGraphTree.set_apply(self, name, sub)
         elif isinstance(sub, Concept):
             self.concepts[name] = sub
+            if sub.get_batch():
+                self.batch = sub
         elif isinstance(sub, Relation):
             self.relations[name] = sub
         else:
             # TODO: what are other cases
             pass
 
-
     def visualize(self, filename, open_image=False):
+        '''Visualizes the graph using Graphviz.
+
+        Creates a directed graph and populates it with nodes for concepts and
+        edges for relations. It also recursively adds subgraphs. Finally, it
+        either saves the graph to a file or returns the graph object depending on
+        the `filename` parameter.
+    
+        Args:
+        filename (str|None): The name of the file where the graph will be saved.
+                             If None, the graph object is returned instead.
+        open_image (bool): Whether to open the image after rendering.
+                          Defaults to False.
+    
+        Returns:
+        graphviz.Digraph|None: The graph object if filename is None, otherwise None.
+        '''
         import graphviz
         concept_graph = graphviz.Digraph(name=f"{self.name}")
         concept_graph.attr(label=f"Graph: {self.name}") 
@@ -118,18 +916,41 @@ def visualize(self, filename, open_image=False):
 
     @property
     def subgraphs(self):
+        """Ordered dictionary containing the subgraphs.
+
+        Returns:
+            OrderedDict: An ordered dictionary of the subgraphs.
+        """
         return OrderedDict(self)
 
     @property
     def concepts(self):
+        """Getter for the concepts.
+
+        Returns:
+            dict: Dictionary of concepts.
+        """
         return self._concepts
 
     @property
     def logicalConstrains(self):
+        """Getter for the logical constraints.
+
+        Returns:
+            dict: Dictionary of logical constraints.
+        """
         return self._logicalConstrains
 
     @property
     def logicalConstrainsRecursive(self):
+        """Generator function that yields logical constraints recursively.
+
+        This method goes through all nodes and yields the logical constraints 
+        if the node is an instance of Graph.
+
+        Yields:
+            tuple: A tuple containing key-value pairs of logical constraints.
+        """
         def func(node):
             if isinstance(node, Graph):
                 yield from node.logicalConstrains.items()
@@ -137,17 +958,78 @@ def func(node):
 
     @property
     def relations(self):
+        """Getter for the relations.
+
+        Returns:
+            dict: Dictionary of relations.
+        """
         return self._relations
 
     def what(self):
+        """Method to get the summary of the graph tree.
+
+        This method provides a dictionary containing the base graph tree and its concepts.
+
+        Returns:
+            dict: Dictionary containing 'concepts' and the base graph tree.
+        """
         wht = BaseGraphTree.what(self)
         wht['concepts'] = dict(self.concepts)
         return wht
+    
+    def print_predicates(self,):
+        """Generate and return a list of predicates with their variables.
+
+        This method goes through the subgraphs and concepts to generate a list of predicates,
+        determining the variables they use based on their relations and superclass structure.
+
+        Returns:
+            list: A list of strings, each representing a predicate and its variables.
+        """
+        predicate_list = dict()
+        variable_list = ['x', 'y', 'z', 'a', 'b', 'r', 'c', 'l', 'i']
+        concepts = list(self.concepts.values())
+        for subgraph in self.subgraphs.values():
+            concepts.extend(list(subgraph.concepts.values()))
+        for concept in concepts:
+            predicate_name = concept.name
+            variables = variable_list[0]
+            check = False
+            if concept._out:
+                if 'has_a' in concept._out:
+                    num_relations = len(concept._out['has_a'])
+                    variables = ", ".join(variable_list[:num_relations])
+                    check = True
+            if not check:
+                new_node = concept
+                while 'is_a' in new_node._out:
+                    if new_node._out['is_a'][0].dst.name in predicate_list:
+                        variables = predicate_list[new_node._out['is_a'][0].dst.name][1]
+                        break
+                    else:
+                        if 'has_a' in new_node.sup._out:
+                            num_relations = len(new_node.sup._out['has_a'])
+                            variables = ", ".join(variable_list[:num_relations])
+                            break
+                    new_node = new_node._out['is_a'][0].dst
+            if hasattr(concept, 'enum'):
+                ### there should be another variable to indicate the type
+                variables = f"{variables}, t"
+                predicate_list[predicate_name] = [predicate_name, variables, concept.enum]
+            else:
+                predicate_list[predicate_name] = [predicate_name, variables]
+        final_list = []
+        for predicate in predicate_list.values():
+            if len(predicate) == 3:
+                final_list.append(f"{predicate[0]}({predicate[1]}) --> 't' stand for the possible types which should be selected from this list: {predicate[2]}")
+            else:
+                final_list.append(f"{predicate[0]}({predicate[1]})")
+        return final_list
+
+    Ontology = namedtuple('Ontology', ['iri', 'local'], defaults=[None])
+    """Namedtuple for Ontology data structure.
 
-    # NB: for namedtuple, defaults are right-most first
-    # `local` default to None,
-    # python 3.7+
-    # Ontology = namedtuple('Ontology', ('iri', 'local'), defaults=(None,))
-    # python 2.6+
-    Ontology = namedtuple('Ontology', ('iri', 'local'))
-    Ontology.__new__.__defaults__ = (None,)
+    Attributes:
+        iri (str): The IRI of the ontology. Default is None.
+        local (str): The local identification of the ontology. Default is None.
+    """
\ No newline at end of file
diff --git a/domiknows/graph/logicalConstrain.py b/domiknows/graph/logicalConstrain.py
index 5791d9a0c..0f5872707 100644
--- a/domiknows/graph/logicalConstrain.py
+++ b/domiknows/graph/logicalConstrain.py
@@ -11,27 +11,57 @@
 
 class LcElement:
     def __init__(self, *e,  name = None):
+        from .relation import Relation
+
+        self.typeName = "Logical Element"
+        if isinstance(self, LogicalConstrain):
+            self.typeName = "Logical Constraint"
+            
         if not e:
-            myLogger.error("Logical Element initialized is empty")
-            raise LcElement.LcElementError("Logical Element initialized is empty")
+            raise LcElement.LcElementError(f"{self.typeName} is empty")
         
         updatedE = []
-        for _, eItem in enumerate(e):
-            if isinstance(eItem, (LcElement, Concept)):
-                updatedE.append(eItem)
-            elif callable(eItem):
-                newEItem = eItem.__call__()
-                updatedE.extend(newEItem)
-            elif isinstance(eItem, list):
-                updatedE.extend(eItem)
-            else:
-                updatedE.append(eItem)
+        lenE = len(e)
+        self.cardinalityException = ""
+        
+        if isinstance(self, eqL):
+            expected_types = {
+                0: (Concept, Relation, tuple),
+                1: str,
+                2: set
+            }
+            
+            for i, eItem in enumerate(e):
+                expected_type = expected_types.get(i)
+                if expected_type and isinstance(eItem, expected_type):
+                    updatedE.append(eItem)
+                else:
+                    error_message = f"{self.typeName} is incorrect - {eItem} is not a valid {i} element of the {self.typeName}"
+                    raise LcElement.LcElementError(error_message)
+        else:
+            for i, eItem in enumerate(e):
+                if isinstance(eItem, (LcElement, Concept, Relation, tuple)): # lc element without path
+                    updatedE.append(eItem)
+                elif callable(eItem): # multiclass label
+                    newEItem = eItem.__call__() # generated label
+                    updatedE.extend(newEItem)
+                elif isinstance(eItem, list): # lc element with path
+                    updatedE.extend(eItem)
+                elif isinstance(eItem, int):
+                    if i<lenE-1:
+                        self.cardinalityException = eItem
+    
+                    updatedE.append(eItem)
+                else:
+                    raise LcElement.LcElementError(f"{self.typeName} is incorrect - {eItem} is not a valid element of the {self.typeName}")
                 
         self.e = updatedE
         
         updatedE = []
         for _, eItem in enumerate(self.e):
-            if isinstance(eItem, Concept):
+            if eItem == None:
+                continue
+            if isinstance(eItem, Concept): # binary concept mapping to tuple representation
                 updatedE.append((eItem, eItem.name, None, 1))
             else:
                 updatedE.append(eItem)
@@ -44,37 +74,38 @@ def __init__(self, *e,  name = None):
         conceptOrLc = None
         
         for _, eItem in enumerate(self.e):
-            if isinstance(eItem, LcElement):
+            if isinstance(eItem, (LcElement, Relation)):
                 conceptOrLc = eItem
                 break
-            elif isinstance(eItem, tuple):
+            elif isinstance(eItem, tuple): # Concept
                 if isinstance(eItem[0], Concept):
                     conceptOrLc = eItem[0]
                     break
     
         if conceptOrLc is None:
             myLogger.error("Logical Element is incorrect")
-            raise LcElement.LcElementError("Logical Element is incorrect")
+            raise LcElement.LcElementError(f"{self.typeName} {self.name} is incorrect - no concept or logical constrain found")
             
         if isinstance(conceptOrLc, Concept):
             if self.__getContext(conceptOrLc):
                 self.graph = self.__getContext(conceptOrLc)[-1]
+        elif isinstance(conceptOrLc, (Relation,)):
+            self.graph = conceptOrLc.graph
         elif isinstance(conceptOrLc, str):
             self.graph = None
         else:
             self.graph = conceptOrLc.graph
                 
         if self.graph == None:
-            myLogger.error("Logical Element initialized is not associated with graph")
-            raise LcElement.LcElementError("Logical Element initialized is not associated with graph")
-                     
+            raise LcElement.LcElementError(f"{self.typeName} is incorrect - no graph found for the {self.typeName}")
+        
         # Create Logical Element id based on number of existing Logical Element in the graph
         self.lcName = "LC%i"%(len(self.graph.logicalConstrains))
         
         if name is not None:
             self.name = name
         else:
-            self.name = self.lcName
+            self.name = self.lcName            
      
     class LcElementError(Exception):
         pass
@@ -103,7 +134,7 @@ def __init__(self, *e, p=100, active = True, sampleEntries  = False, name = None
         assert self.graph is not None
         self.graph.logicalConstrains[self.lcName] = self
         
-         # Go though constraint, find nested constrains and change their property headLC to indicate that their are nested and should not be process individually
+        # Go though constraint, find nested constrains and change their property headLC to indicate that their are nested and should not be process individually
         for e_item in self.e:
             if isinstance(e_item, LogicalConstrain):
                 e_item.headLC = False
@@ -162,8 +193,8 @@ def strEs(self):
                             else:
                                 new_v.append(v.name)
 
-                    #new_v = [v if isinstance(v, (str, tuple, LogicalConstrain)) else v.name for v in eItem[1]]
-                    new_V.append("path={}".format(tuple(new_v)))
+                        #new_v = [v if isinstance(v, (str, tuple, LogicalConstrain)) else v.name for v in eItem[1]]
+                        new_V.append("path={}".format(tuple(new_v)))
                 strsE.append("{}".format(tuple(new_V)))
             elif isinstance(eItem, Concept):
                 strsE.append(eItem.name)
@@ -331,143 +362,76 @@ def createILPConstrains(self, lcName, lcFun, model, v, headConstrain):
         # Return results from created ILP constraints - 
         # None if headConstrain is True or no ILP constraint created, ILP variable representing the value of ILP constraint, loss calculated
         return rVars
-    
-    
-     # Collects combination setups of ILP variables for logical methods calls for the created Logical Constraint - recursive method
-    def _collectCombinationILPVariableSetups(self, lcVariableName, lcVariableNames, v, lcVars = []): 
-        
-        # Get set of ILP variables lists for the current variable name
-        cLcVariables = v[lcVariableName]
-        
-        # List of lists containing sets of ILP variables for particular position 
-        newLcVars = []
-        
-        # --- Update the lcVars setup with ILP variables from this iteration
-        
-        if not lcVars: # If ILP variables setup is not initialized yet - this is the first iteration of the _collectILPVariableSetups method
-            if cLcVariables is None:
-                newV = [[None]]
-                newLcVars.append(newV)
-            else:
-                for cV in cLcVariables:
-                    newV = []
-                    for cvElement in cV:
-                        if cvElement is None:
-                            pass
-                        newElement = [cvElement]
-                        newV.append(newElement)
-                    newLcVars.append(newV)
-        else: # Many ILP variables in the current set
-            for indexLcV, lcV in enumerate(lcVars):
-                newV = []
-                if cLcVariables is None:
-                    for _, lcVelement in enumerate(lcV):
-                        newLcVelement = lcVelement.copy()
-                        newLcVelement.append(None)
-                        
-                        newV.append(newLcVelement)
-                else:
-                    for cV in cLcVariables:
-                        for indexElement, lcVelement in enumerate(lcV):
-                        
-                            newLcVelement = lcVelement.copy()
-                            newLcVelement.append(cV[indexElement])
-                            
-                            newV.append(newLcVelement)
-                            
-                    newLcVars.append(newV)                
-                            
-        if lcVariableNames:
-            # Recursive call - lcVars contains currently collected ILP variables setups
-            return self._collectCombinationILPVariableSetups(lcVariableNames[0], lcVariableNames[1:], v, lcVars=newLcVars)
-        else:
-            # Return collected setups
-            return newLcVars
 
-    def createILPConstrainsCombination(self, lcName, lcFun, model, v, headConstrain):
-        if len(v) < 2:
-            myLogger.error("%s Logical Constraint created with %i sets of variables which is less then two"%(lcName, len(v)))
-            return None
-        
-        # Input variable names
+    def createILPCount(self, model, myIlpBooleanProcessor, v, headConstrain, cOperation, cLimit, integrate,
+                       logicMethodName="COUNT"):
         try:
             lcVariableNames = [e for e in iter(v)]
         except StopIteration:
             pass
-                
-        lcVariableName0 = lcVariableNames[0]
 
-        rVars = [] # Output variables
-            
-        # Collect variables setups for ILP constraints
-        sVar = self._collectCombinationILPVariableSetups(lcVariableName0, lcVariableNames[1:], v)
-        
-        # Apply collected setups and create ILP constraint
-        for z in sVar:
-            tVars = [] # Collect ILP constraints results
-            for t in z:
-                tVars.append(lcFun(model, *t, onlyConstrains = headConstrain))
-                
-            rVars.append(tVars)
-        
-        # Return results from created ILP constraints - 
-        # None if headConstrain is True or no ILP constraint created, ILP variable representing the value of ILP constraint, loss calculated
-        return rVars
-        
+        if cLimit == None:
+            cLimit = 1
 
-    def createILPCount(self, model, myIlpBooleanProcessor, v, headConstrain, cOperation, cLimit, integrate, logicMethodName = "COUNT"):         
-        try:
-            lcVariableNames = [e for e in iter(v)]
-        except StopIteration:
-            pass
-        
-        if cLimit== None:
-            cLimit
-        lcVariableName0 = lcVariableNames[0] # First variable
-        lcVariableSet0 =  v[lcVariableName0]
+        lcVariableName0 = lcVariableNames[0]  # First variable
+        lcVariableSet0 = v[lcVariableName0]
 
-        zVars = [] # Output ILP variables
-        
-        for i, _ in enumerate(lcVariableSet0):
-            varsSetup = []
+        zVars = []  # Output ILP variables
+        # for i, _ in enumerate(lcVariableSet0):
+        #     varsSetup = []
+        #
+        #     var = []
+        #     for currentV in iter(v):
+        #         var.extend(v[currentV][i])
+        #
+        #     if len(var) == 0:
+        #         if not (headConstrain or integrate):
+        #             zVars.append([None])
+        #
+        #         continue
+        #
+        #     if headConstrain or integrate:
+        #         varsSetup.extend(var)
+        #     else:
+        #         varsSetup.append(var)
+        varsSetup = []
+
+        var = [currentV[0] for currentV in iter(lcVariableSet0)]
+
+        if headConstrain or integrate:
+            varsSetup.extend(var)
+        else:
+            varsSetup.append(var)
+
+        # -- Use ILP variable setup to create constrains
+        if headConstrain or integrate:
+            zVars.append([myIlpBooleanProcessor.countVar(model, *varsSetup, onlyConstrains=headConstrain,
+                                                         limitOp=cOperation, limit=cLimit,
+                                                         logicMethodName=logicMethodName)])
+        else:
+            for current_var in varsSetup:
+                zVars.append([myIlpBooleanProcessor.countVar(model, *current_var, onlyConstrains=headConstrain,
+                                                             limitOp=cOperation, limit=cLimit,
+                                                             logicMethodName=logicMethodName)])
 
-            var = []
-            for currentV in iter(v):
-                var.extend(v[currentV][i])
-                
-            if len(var) == 0:
-                if not (headConstrain or integrate):
-                    zVars.append([None])
-                    
-                continue
-            
-            if headConstrain or integrate:
-                varsSetup.extend(var)
-            else:
-                varsSetup.append(var)
-             
-            # -- Use ILP variable setup to create constrains   
-            if headConstrain or integrate:
-                zVars.append([myIlpBooleanProcessor.countVar(model, *varsSetup, onlyConstrains = headConstrain, limitOp = cOperation, limit=cLimit, 
-                                                             logicMethodName = logicMethodName)])
-            else:
-                for current_var in varsSetup:
-                    zVars.append([myIlpBooleanProcessor.countVar(model, *current_var, onlyConstrains = headConstrain, limitOp = cOperation, limit=cLimit, 
-                                                             logicMethodName = logicMethodName)])
-           
         if model is not None:
             model.update()
-            
+
         return zVars
     
-    def createILPAccumulatedCount(self, model, myIlpBooleanProcessor, v, headConstrain, cOperation, cLimit, integrate, logicMethodName = "COUNT"):         
+    def createILPAccumulatedCount(self, model, myIlpBooleanProcessor, v, headConstrain, cOperation, cLimit, integrate, logicMethodName = "COUNT"):  
+        
+        # Ignore value of integrate
+        integrate = True
+               
         try:
             lcVariableNames = [e for e in iter(v)]
         except StopIteration:
             pass
         
-        if cLimit== None:
-            cLimit
+        if cLimit == None:
+            cLimit = 1
+            
         lcVariableName0 = lcVariableNames[0] # First variable
         lcVariableSet0 =  v[lcVariableName0]
 
@@ -494,8 +458,10 @@ def createILPAccumulatedCount(self, model, myIlpBooleanProcessor, v, headConstra
              
         # -- Use ILP variable setup to create constrains   
         if headConstrain or integrate:
-            zVars.append([myIlpBooleanProcessor.countVar(model, *varsSetup, onlyConstrains = headConstrain, limitOp = cOperation, limit=cLimit, 
-                                                         logicMethodName = logicMethodName)])
+            r = myIlpBooleanProcessor.countVar(model, *varsSetup, onlyConstrains = headConstrain, limitOp = cOperation, limit=cLimit, 
+                                                     logicMethodName = logicMethodName)
+            for _ in lcVariableSet0:
+                zVars.append([r])
         else:
             for current_var in varsSetup:
                 zVars.append([myIlpBooleanProcessor.countVar(model, *current_var, onlyConstrains = headConstrain, limitOp = cOperation, limit=cLimit, 
@@ -583,6 +549,9 @@ def __call__(self, model, myIlpBooleanProcessor, v, headConstrain = False, integ
      
 class eqL(LogicalConstrain):
     def __init__(self, *e, active = True, sampleEntries = False, name = None):
+        #if e is len 2 and element index 1 is of type String
+        if len(e) == 2 and isinstance(e[1], str):
+            e = (e[0],  "instanceID", e[1])  
         LogicalConstrain.__init__(self, *e, p=100)
         self.headLC = False
     
diff --git a/domiknows/graph/relation.py b/domiknows/graph/relation.py
index e1f0202ab..638bf2cb1 100644
--- a/domiknows/graph/relation.py
+++ b/domiknows/graph/relation.py
@@ -62,6 +62,7 @@ def __init__(self, src, dst, argument_name, reverse_of=None, auto_constraint=Non
             self.is_reversed = True
         self.reversed = reverse_of
         self.auto_constraint = auto_constraint
+        self.var_name = None
 
     @property
     def mode(self):
@@ -103,6 +104,10 @@ def dst(self):
     @dst.setter
     def dst(self, dst):
         self._dst = dst
+        
+    @property
+    def graph(self):
+        return self._sup
 
     def what(self):
         return {'src': self.src, 'dst': self.dst}
diff --git a/domiknows/program/batchprogram.py b/domiknows/program/batchprogram.py
index 6fbea58d6..92286d894 100644
--- a/domiknows/program/batchprogram.py
+++ b/domiknows/program/batchprogram.py
@@ -13,7 +13,7 @@ def __init__(self, *args, batch_size=1, **kwargs):
         self.batch_size = batch_size
 
     def train_epoch(self, dataset):
-        # do not use super() because it call zero_grad for every step definitly
+        # do not use super() because it call zero_grad for every step definitely
         self.model.mode(Mode.TRAIN)
         self.model.reset()
         self.opt.zero_grad()
diff --git a/domiknows/program/lossprogram.py b/domiknows/program/lossprogram.py
index 7269da1c5..5a40b4767 100644
--- a/domiknows/program/lossprogram.py
+++ b/domiknows/program/lossprogram.py
@@ -264,15 +264,24 @@ def test_epoch(self, dataset, **kwargs):
 #         self.cmodel.mode(Mode.TEST)
         yield from super().test_epoch(dataset)
         
-    def populate_epoch(self, dataset):
+    def populate_epoch(self, dataset, grad = False):
         self.model.mode(Mode.POPULATE)
 #         self.cmodel.mode(Mode.POPULATE)
         self.model.reset()
-        with torch.no_grad():
+        if not grad:
+            with torch.no_grad():
+                for i, data_item in enumerate(dataset):
+                    _, _, *output = self.model(data_item)
+                    yield detuple(*output[:1])
+        else:
             for i, data_item in enumerate(dataset):
+                for dataKey in data_item:
+                    if data_item[dataKey].dtype in [torch.float32, torch.float64, torch.complex64, torch.complex128]:
+                        data_item[dataKey].requires_grad= True
+                    
                 _, _, *output = self.model(data_item)
                 yield detuple(*output[:1])
-        
+                
 class PrimalDualProgram(LossProgram):
     logger = logging.getLogger(__name__)
 
@@ -371,10 +380,6 @@ class GBIProgram(LossProgram):
     logger = logging.getLogger(__name__)
 
     def __init__(self, graph, Model, poi, beta=1, **kwargs):
-        mySolverModel= SolverModel(graph,
-                           poi=poi,
-                           inferTypes=['local/argmax', 'local/softmax'],
-                           metric={})
-        super().__init__(graph, Model, CModel=GBIModel, beta=beta, solver_model = mySolverModel, poi=poi, **kwargs)
+        super().__init__(graph, Model, CModel=GBIModel, beta=beta, poi=poi, **kwargs)
         from domiknows.utils import setDnSkeletonMode
         setDnSkeletonMode(True)
\ No newline at end of file
diff --git a/domiknows/program/metric.py b/domiknows/program/metric.py
index cdda5d5f4..3636d3f79 100644
--- a/domiknows/program/metric.py
+++ b/domiknows/program/metric.py
@@ -1,81 +1,102 @@
 from collections import defaultdict
 from typing import Any
-
 import numpy as np
 import torch
-from torch.nn import functional as F
-
+from sklearn.metrics import confusion_matrix, classification_report
 from ..base import AutoNamed
 from ..utils import wrap_batch
 
 
 class CMWithLogitsMetric(torch.nn.Module):
-    def forward(self, input, target, data_item, prop, weight=None):
+    """
+    A utility class for computing confusion matrix metrics from logits.
+
+    Inherits from:
+        torch.nn.Module
+    """
+
+    def forward(self, input, target, _, prop, weight=None):
+        """
+        Computes True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN) values
+        from given logits and target.
+
+        Args:
+            input (torch.Tensor): The logits tensor.
+            target (torch.Tensor): The ground truth labels tensor.
+            _ : Placeholder, not used.
+            prop: Placeholder, not used.
+            weight (torch.Tensor, optional): Weights to apply to the input. Defaults to tensor of value 1.
+
+        Returns:
+            dict: A dictionary containing TP, FP, TN, and FN values.
+        """
         if weight is None:
             weight = torch.tensor(1, device=input.device)
         else:
             weight = weight.to(input.device)
+            
         preds = input.argmax(dim=-1).clone().detach().to(dtype=weight.dtype)
         labels = target.clone().detach().to(dtype=weight.dtype, device=input.device)
         tp = (preds * labels * weight).sum()
         fp = (preds * (1 - labels) * weight).sum()
         tn = ((1 - preds) * (1 - labels) * weight).sum()
         fn = ((1 - preds) * labels * weight).sum()
+        
         return {'TP': tp, 'FP': fp, 'TN': tn, 'FN': fn}
 
 
-class BinaryCMWithLogitsMetric(CMWithLogitsMetric):
-    def forward(self, input, target, data_item, prop, weight=None):
-        target = target.argmax(dim=-1)
-        return super().forward(input, target, data_item, prop, weight)
-
-
-class MultiClassCMWithLogitsMetric(CMWithLogitsMetric):
-    def __init__(self, num_classes, weight=None):
-        super().__init__()
-        self.num_classes = num_classes
-        self.weight = weight
-
-    def forward(self, input, target, data_item, prop, weight=None):
-        from torch.nn import functional
-        target = functional.one_hot(target, num_classes=self.num_classes)
-        input = functional.one_hot(input.argmax(dim=-1), num_classes=self.num_classes)
-        input = torch.stack((-input, input), dim=-1)
-        if weight is None:
-            weight = self.weight
-        return super().forward(input, target, data_item, prop, weight)
-
-
-def calc_TP_FP_TN_FN_for_single_class(val):
-    y = val["labels"]
-    p = val["preds"]
-    TP,FP,TN,FN=0,0,0,0
-    for i,j in zip(y,p):
-        if i==j and i==1:
-            TP+=1
-        elif i==j and i==0:
-            TN+=1
-        elif not i==j and i==1:
-            FN+=1
-        elif not i == j and i == 0:
-            FP += 1
-    return {"TP": TP, 'FP': FP, 'TN': TN, 'FN': FN}
+class DatanodeCMMetric(torch.nn.Module):
+    """
+    A utility class for computing confusion matrix metrics using datanode inference results.
 
+    Inherits from:
+        torch.nn.Module
 
-class DatanodeCMMetric(torch.nn.Module):
+    Attributes:
+        inferType (str): The type of inference used to derive metrics.
+    """
     def __init__(self, inferType='ILP'):
+        """
+        Initializes the DatanodeCMMetric with a specified inference type.
+
+        Args:
+            inferType (str, optional): The type of inference. Defaults to 'ILP'.
+        """
         super().__init__()
         self.inferType = inferType
 
     def forward(self, input, target, data_item, prop, weight=None):
-        if (data_item.needsBatchRootDN()):
-            data_item.addBatchRootDN()
-        datanode = data_item.getDataNode(context=self.inferType)
+        """
+        Computes the confusion matrix metrics using data from the provided datanode.
+
+        Args:
+            input (torch.Tensor): The input tensor.
+            target (torch.Tensor): The ground truth labels.
+            data_item: The datanode containing the inference metrics.
+            prop: The property associated with the inference.
+            weight (torch.Tensor, optional): An optional weight tensor. Defaults to None.
+
+        Returns:
+            dict/None: A dictionary containing the TP, FP, TN, FN values, or 
+                       information on class names, labels, and predictions; 
+                       returns None if the property name is not found in the results.
+        """
+        datanode = data_item
         result = datanode.getInferMetrics(prop.name, inferType=self.inferType)
         if len(result.keys())==2:
             if str(prop.name) in result:
                 val =  result[str(prop.name)]
-                return calc_TP_FP_TN_FN_for_single_class(val)
+                conf_mat = confusion_matrix(val["labels"], val["preds"])
+                if conf_mat.size == 1:
+                    if val["labels"][0] == 1:
+                        TP = conf_mat[0, 0]
+                        FP = FN = TN = 0
+                    else:
+                        TN = conf_mat[0, 0]
+                        TP = FP = FN = 0
+                else:
+                    TN, FP, FN, TP = conf_mat.ravel()
+                return {"TP": TP, 'FP': FP, 'TN': TN, 'FN': FN}
             else:
                 return None
         else:
@@ -87,30 +108,80 @@ def forward(self, input, target, data_item, prop, weight=None):
 
 
 class MetricTracker(torch.nn.Module):
+    """
+    A utility class for tracking metrics for all datanodes.
+
+    Attributes:
+        metric (callable): The metric function to track.
+        list (list): A list of metric values for all the datanodes.
+        dict (defaultdict): A dictionary of metric values grouped by keys.
+    """
     def __init__(self, metric):
+        """
+        Initializes the MetricTracker with a specified metric function.
+
+        Args:
+            metric (callable): The metric function to be tracked.
+        """
         super().__init__()
         self.metric = metric
         self.list = []
         self.dict = defaultdict(list)
 
     def reset(self):
+        """
+        Resets the internal storage (both list and dict) to their initial empty state.
+        """
         self.list.clear()
         self.dict.clear()
 
     def __call__(self, *args, **kwargs) -> Any:
+        """
+        Computes the metric using the provided arguments and stores the result in the internal list.
+
+        Returns:
+            Any: The computed metric value.
+        """
         value = self.metric(*args, **kwargs)
         self.list.append(value)
         return value
 
     def __call_dict__(self, keys, *args, **kwargs) -> Any:
+        """
+        Computes the metric using the provided arguments and stores the result in the internal dictionary with the specified key.
+
+        Args:
+            keys: The key under which the metric value should be stored.
+
+        Returns:
+            Any: The computed metric value.
+        """
         value = self.metric(*args, **kwargs)
         self.dict[keys].append(value)
         return value
 
     def __getitem__(self, keys):
+        """
+        Returns a lambda function that computes and stores the metric in the internal dictionary for a specified key.
+
+        Args:
+            keys: The key under which the metric value should be stored.
+
+        Returns:
+            callable: A lambda function to compute and store the metric value.
+        """
         return lambda *args, **kwargs: self.__call_dict__(keys, *args, **kwargs)
 
     def kprint(self, k):
+        """
+        Custom key printing function based on the type and properties of the key.
+
+        Args:
+            k: The key to be printed.
+
+        Returns:
+            str: A string representation of the key.
+        """
         if (
             isinstance(k, tuple) and
             len(k) == 2 and
@@ -121,14 +192,21 @@ def kprint(self, k):
             return k
 
     def value(self, reset=False):
+        """
+        Retrieves the value(s) of the computed metric(s).
+
+        Args:
+            reset (bool, optional): If True, resets the internal storage after retrieving the value. Defaults to False.
+
+        Returns:
+            Any: The metric value(s), either as a single value, list, or dictionary.
+        """
         if self.list and self.dict:
             raise RuntimeError('{} cannot be used as list-like and dict-like the same time.'.format(type(self)))
         if self.list:
             value = wrap_batch(self.list)
             value = super().__call__(value)
         elif self.dict:
-            #value = wrap_batch(self.dict)
-            #value = super().__call__(value)
             func = super().__call__
             value = {self.kprint(k): func(v) for k, v in self.dict.items()}
         else:
@@ -138,6 +216,12 @@ def value(self, reset=False):
         return value
 
     def __str__(self):
+        """
+        Provides a string representation of the computed metric value(s).
+
+        Returns:
+            str: A string representation of the metric value(s).
+        """
         value = self.value()
         
         if isinstance(value, dict):
@@ -163,10 +247,53 @@ def __str__(self):
         return str(value)
 
 class MacroAverageTracker(MetricTracker):
+    """
+    A utility class that extends the MetricTracker to compute macro-average of metrics for datanodes.
+
+    Inherits from:
+        MetricTracker
+    """
+    def __init__(self, metric):
+        """
+        Initializes the MacroAverageTracker with a specified metric function.
+
+        Args:
+            metric (callable): The metric function to be tracked.
+        """
+        super().__init__(metric)
+        
     def forward(self, values):
+        """
+        Computes the macro-average for the given values.
+
+        Args:
+            values (Any): The input values (can be single value, list, tensor, or dictionary of values).
+
+        Returns:
+            Any: The macro-averaged value. The structure (tensor, list, or dictionary) of the output
+                 mirrors the structure of the input.
+        """
         def func(value):
+            """
+            Computes the mean of the provided tensor after detaching it.
+
+            Args:
+                value (torch.Tensor): A tensor value.
+
+            Returns:
+                torch.Tensor: The mean of the tensor.
+            """
             return value.clone().detach().mean()
         def apply(value):
+            """
+            Recursively applies the mean computation based on the type of the input value.
+
+            Args:
+                value (Any): The input value to be averaged.
+
+            Returns:
+                Any: The averaged value.
+            """
             if isinstance(value, dict):
                 return {k: apply(v) for k, v in value.items()}
             elif isinstance(value, torch.Tensor):
@@ -177,24 +304,43 @@ def apply(value):
         return retval
 
 
-class ValueTracker(MetricTracker):
-    def forward(self, values):
-        return values
-
-def frp_from_matrix(i,matrix):
-    matrix=np.array(matrix)
-    TP=matrix[i][i]
-    TN=matrix.sum()-matrix[i].sum()-matrix[:,i].sum()+matrix[i][i]
-    FN=matrix[i].sum()-matrix[i][i]
-    FP=matrix[:,i].sum()-matrix[i][i]
-    return TP,TN,FP,FN
-
 class PRF1Tracker(MetricTracker):
+    """
+    A tracker to calculate and monitor precision, recall, F1 score, and accuracy metrics.
+ 
+    Inherits from the MetricTracker class.
+    
+    Methods:
+    - forward: Processes input values to compute various metrics like precision, recall, F1 score, and accuracy.
+    """
     def __init__(self, metric=CMWithLogitsMetric(),confusion_matrix=False):
         super().__init__(metric)
         self.confusion_matrix=confusion_matrix
-
+        """
+        Initialize the PRF1Tracker instance.
+        
+        Parameters:
+        - metric (Metric, optional): An instance of the metric to be tracked. Defaults to CMWithLogitsMetric().
+        - confusion_matrix (bool, optional): Whether to create confusion matrix values or not. Defaults to False.
+        """
     def forward(self, values):
+        """
+        Processes the input values and computes precision, recall, F1 score, and accuracy metrics.
+
+        Parameters:
+        - values: Input data containing raw class names and predictions.
+        
+        Returns:
+        - dict: A dictionary containing calculated metrics.
+
+        If the input contains class names it means it is for a multiclass concept:
+            Returns a classification report with metrics for each class and overall metrics 
+            like 'weighted avg', 'macro avg', and 'accuracy' after negative classes are removed.
+
+        Else:
+
+            Returns metrics: 'P' (Precision), 'R' (Recall), 'F1' (F1 Score), and 'accuracy' for the bincaryclass concept.
+        """
         if values[0] and not "class_names" in values[0]:
 
             CM = wrap_batch(values)
@@ -219,50 +365,34 @@ def forward(self, values):
             else:
                 tn = CM['TN'].sum().float()
 
+            if not torch.is_tensor(tp):
+                tp = torch.tensor(tp)
+            if not torch.is_tensor(fp):
+                fp = torch.tensor(fp)
+            if not torch.is_tensor(fn):
+                fn = torch.tensor(fn)
+            if not torch.is_tensor(tn):
+                tn = torch.tensor(tn)
+                
             if tp:
                 p = tp / (tp + fp)
                 r = tp / (tp + fn)
                 f1 = 2 * p * r / (p + r)
             else:
-                p = torch.zeros_like(torch.tensor(tp))
-                r = torch.zeros_like(torch.tensor(tp))
-                f1 = torch.zeros_like(torch.tensor(tp))
+                p = torch.zeros_like(tp)
+                r = torch.zeros_like(tp)
+                f1 = torch.zeros_like(tp)
             if (tp + fp + fn + tn):
                 accuracy=(tp + tn) / (tp + fp + fn + tn)
             return {'P': p, 'R': r, 'F1': f1,"accuracy":accuracy}
         elif values[0]:
-            output={}
             names=values[0]["class_names"][:]
             n=len(names)
-
-            matrix=[[0 for i in range(n)] for j in range(n)]
-            for batch in values:
-                for label,pred in zip(batch["labels"],batch["preds"]):
-                    matrix[label][pred]+=1
-            if self.confusion_matrix:
-                output[str(names)]=matrix
-            for name in names:
-                TP,TN,FP,FN=frp_from_matrix(names.index(name),matrix)
-                if (TP+FP):
-                    output[name+" Precision"]=TP/(TP+FP)
-                else:
-                    output[name + " Precision"] = 0
-                if (TP+FN):
-                    output[name + " Recall"] =TP/(TP+FN)
-                else:
-                    output[name + " Recall"]=0
-                if (output[name+" Precision"]+output[name + " Recall"]):
-                    output[name + " F1"] =2*(output[name+" Precision"]*output[name + " Recall"])/(output[name+" Precision"]+output[name + " Recall"])
-                else:
-                    output[name + " F1"]=0
-                if (TP+TN+FP+FN):
-                    output[name + " Accuracy"] =(TP+TN)/(TP+TN+FP+FN)
-                else:
-                    output[name + " Accuracy"]=0
-            output["Total Accuracy of All Classes"]=sum([matrix[i][i] for i in range(n)])/sum([sum(matrix[i]) for i in range(n)])
-            return output
-
-
-class BinaryPRF1Tracker(PRF1Tracker):
-    def __init__(self, metric=BinaryCMWithLogitsMetric()):
-        super().__init__(metric)
+            labels = np.concatenate([batch["labels"] for batch in values])
+            preds = np.concatenate([batch["preds"] for batch in values])
+            labels = labels[preds >= 0]
+            preds = preds[preds >= 0]
+
+            report = classification_report(labels, preds, labels=np.arange(n), output_dict=True,zero_division=0)
+            report = {**{names[i]: report[str(i)] for i in range(n)}, **{'weighted avg': report['weighted avg'], 'macro avg': report['macro avg'], 'accuracy': report['accuracy']}}
+            return report
diff --git a/domiknows/program/model/gbi.py b/domiknows/program/model/gbi.py
index 25e0ea3ef..bc652db34 100644
--- a/domiknows/program/model/gbi.py
+++ b/domiknows/program/model/gbi.py
@@ -1,67 +1,78 @@
 from collections import OrderedDict
 import warnings
+import copy
 
 import torch
-import copy
-from torch.optim import SGD
+from torch import nn
+from torch.optim import Adam, SGD
+import torch.nn.functional as F
 
 from domiknows.program.metric import MacroAverageTracker, MetricTracker
+from domiknows.program.loss import NBCrossEntropyLoss
 from domiknows.program.model.base import Mode
 
-from .pytorch import SolverModel
-from ...graph.concept import EnumConcept
-from domiknows.utils import detuple
-import torch.nn.functional as F
-import argparse
-import numpy
+from domiknows.utils import getDnSkeletonMode
 
 # Gradient-based Inference
 class GBIModel(torch.nn.Module):
-    def __init__(self, graph, solver_model, gbi_iters = 100, device='auto'):
+    def __init__(
+            self,
+            graph,
+            solver_model=None,
+            gbi_iters = 50,
+            lr=1e-1,
+            reg_weight=1,
+            reset_params=True,
+            device='auto'
+        ):
+        """
+        This class specifies a model that uses gradient-based inference (GBI) for each inference step.
+        
+        :param graph: The `graph` parameter is an object that represents the logical constraints of a
+        graph. It contains information about the nodes, edges, and constraints of the graph
+        :param solver_model: Underlying model to perform GBI on. (Optional)
+        :param gbi_iters: The maximum number of gradient update steps to perform. GBI will exit early if all constraints are specified. Defaults to 50. (Optional)
+        :param lr: The step size of each update step. Defaults to 1e-1. (Optional)
+        :param reg_weight: The weight of the regularization loss. Increasing this value will result in parameter updates that are closer to the original, unoptimized parameters. Defaults to 1. (Optional)
+        :param reset_params: If set to `True`, the parameters of the model will be reset to the original (non-optimized) parameters after GBI is complete. If set to `False`, the parameters will *only* be reset if the loss becomes `NaN` or the constraints aren't satisfied after `gbi_iters` updates. Set this to `True` if GBI is to only be used during inference. Defaults to `True`. (Optional)
+        :param device: The device to use for GBI updates. Defaults to 'auto'. (Optional)
+        """
+
         super().__init__()
         
-        self.server_model= solver_model
+        if solver_model is None:
+            self.solver_model = self
+        else:
+            self.server_model = solver_model
+            
         self.gbi_iters = gbi_iters
+        self.lr = lr
+        self.reg_weight = reg_weight
+        self.reset_params = reset_params
+
         self.device = device
         
         self.constr = OrderedDict(graph.logicalConstrainsRecursive)
         nconstr = len(self.constr)
         if nconstr == 0:
             warnings.warn('No logical constraint detected in the graph. '
-                          'PrimalDualModel will not generate any constraint loss.')
-            
-        self.lmbd = torch.nn.Parameter(torch.zeros(nconstr).float())
-        self.lmbd_index = {}
-
-        self.iter_step = 0
-        self.warmpup = 80
-        
-        for i, (key, lc) in enumerate(self.constr.items()):
-            self.lmbd_index[key] = i
+                          'GBIModel will not generate any constraint loss.')
             
-        self.reset_parameters()
-        self.loss = MacroAverageTracker(lambda x:x)
+        self.loss = MacroAverageTracker(NBCrossEntropyLoss())
         
-    def reset_parameters(self):
-        torch.nn.init.constant_(self.lmbd, 0.0)
-
     def reset(self):
-        if hasattr(self, 'loss') and self.loss and isinstance(self.loss, MetricTracker):
-            self.loss.reset()
-
-    def get_lmbd(self, key):
-        if self.lmbd[self.lmbd_index[key]] < 0:
-            with torch.no_grad():
-                self.lmbd[self.lmbd_index[key]] = 0
-        return self.lmbd[self.lmbd_index[key]]
+        if hasattr(self, 'loss') and self.solver_model.loss and isinstance(self.solver_model.loss, MetricTracker):
+            self.solver_model.loss.reset()
 
  # --- GBI methods
     def get_constraints_satisfaction(self, node):
         """
-        Get constraint satisfaction from datanode
-        Returns number of satisfied constraints and total number of constraints
+        Get constraint satisfaction from datanode. Returns number of satisfied constraints and total number of constraints.
+
+        :params: node: The DataNode to get constraint satisfaction from.
         """
-        verifyResult = node.verifyResultsLC()
+
+        verifyResult = node.verifyResultsLC(key = "/local/argmax")
 
         assert verifyResult
 
@@ -74,148 +85,242 @@ def get_constraints_satisfaction(self, node):
 
         return num_satisfied, num_constraints
     
-    def get_lambda(self, model, lr=1e-3):
-        model_l = copy.deepcopy(model)
+    def reg_loss(self, model_updated, model, exclude_names=set()):
+        """
+        Calculates regularization loss for GBI. The loss is defined as the L2 distance between the original and updated model parameters.
+
+        :param model_updated: The model being optimized.
+        :param model: The original, unoptimized model. The parameters for this model should be frozen.
+        :param exclude_names: A set of parameter names to exclude from the regularization loss calculation. Defaults to an empty set. (Optional)
+        """
 
-        c_opt = SGD(model_l.parameters(), lr=lr)
+        orig_params = {}
+        lambda_params = {}
 
-        return model_l, c_opt
+        for key in model.keys():
+            key_prefix = '/'.join(key.split('/')[:3])
 
-    def reg_loss(self, model_lambda, model, exclude_names=set()):
-        orig_params = []
-        lambda_params = []
+            # print(key, key_prefix)
 
-        for (name, w_orig), (_, w_curr) in zip(model.named_parameters(), model_lambda.named_parameters()):
-            if name not in exclude_names:
-                orig_params.append(w_orig.flatten())
-                lambda_params.append(w_curr.flatten())
+            w_orig = model[key]
+            w_curr = model_updated[key]
+
+            if key_prefix not in orig_params:
+                orig_params[key_prefix] = []
+                lambda_params[key_prefix] = []
+
+            if key not in exclude_names:
+                orig_params[key_prefix].append(w_orig.flatten())
+                lambda_params[key_prefix].append(w_curr.flatten())
             else:
-                print('skipping %s' % name)
+                print('skipping %s' % key)
+
+        result_norm = 0.0
+
+        for key in orig_params.keys():
+            orig_params_module = torch.cat(orig_params[key], dim=0)
+            lambda_params_module = torch.cat(lambda_params[key], dim=0)
 
-        orig_params = torch.cat(orig_params, dim=0)
-        lambda_params = torch.cat(lambda_params, dim=0)
+            result_norm += torch.linalg.norm(orig_params_module - lambda_params_module, dim=0, ord=2)
 
-        return torch.linalg.norm(orig_params - lambda_params, dim=0, ord=2)
+        return result_norm
 
-    def populate_forward(self, model, data_item):
+    def set_pretrained(self, model, orig_params):
         """
-        Forward pass through torch model.
-        Returns DataNode and DataNodeBuilder.
+        Resets the parameters of a PyTorch model to the original, unoptimized parameters.
+
+        :param model: The PyTorch model to reset the parameters of.
+        :param orig_params: The original, unoptimized parameters of the model.
         """
-        _, _, *output = model(data_item)
-        node = detuple(*output[:1])
-        return node, output[1]
-    # ----
-    
-    def forward(self, builder, build=None):
-        if builder.needsBatchRootDN():
-            builder.addBatchRootDN()
-        datanode = builder.getDataNode(device=self.device)
+        model.load_state_dict(orig_params)
+
+    def get_argmax_from_node(self, node):
+        probs_named = {}
+        for var_name, var_val in node.getAttribute('variableSet').items():
+            if var_name.endswith('>'):
+                probs_named[var_name] = F.log_softmax(var_val, dim=-1).flatten()
+
+        argmax_vals = {name: torch.argmax(param).item() for name, param in probs_named.items()}
+        return argmax_vals
 
-        datanode.infer()
+    def forward(self, datanode, build=None, verbose=False):
+        """
+        Performs a forward pass on the model and updates the parameters using gradient-based inference (GBI).
+
+        :param datanode: The DataNode to perform GBI on.
+        :param build: Defaults to `None`. (Optional)
+        :param verbose: Print intermediate values during inference. Defaults to `False`. (Optional)
+        """
 
         # Get constraint satisfaction for the current DataNode
         num_satisfied, num_constraints = self.get_constraints_satisfaction(datanode)
+        model_has_GBI_inference = False
 
         # --- Test if to start GBI for this data_item
         if num_satisfied == num_constraints:
-            return 0.0, datanode, builder
-        
+            return 0.0, datanode, datanode.myBuilder
         # ------- Continue with GBI
-                
-        # -- Make copy of original model
-        model_l, c_opt = self.get_lambda(self.server_model, lr=1e-1)
+
+        # to be optimized by gbi
+        optimized_parameters = {name: param for name, param in self.server_model.named_parameters()}
+
+        # for regularization calculation
+        original_parameters = {name: param.clone().detach() for name, param in self.server_model.named_parameters()}
+
+        # for reloading parameters after gbi
+        reload_parameters = self.server_model.state_dict()
+        for name, param in reload_parameters.items():
+            reload_parameters[name] = param.clone().detach()
         
-        # -- model_l is the model that gets optimized by GBI
-        model_l.mode(Mode.TRAIN)
-        model_l.train()
-        model_l.reset()        
+        # Print original and cloned parameters to verify they are the same but different in memory location
+        for name, param in self.server_model.named_parameters():
+            are_equal = torch.equal(param, original_parameters[name])
+            same_memory = param.storage().data_ptr() == original_parameters[name].untyped_storage().data_ptr()
+            if same_memory or not are_equal:
+                print(f"{name} -> Equal: {are_equal}, Same Memory: {same_memory}")
+    
+        # Data to be used for inference
+        x = datanode.myBuilder["data_item"]
+                        
+        # -- self.server_mode is the model that gets optimized by GBI
+        self.server_model.reset()        
 
+        modelLParams = self.server_model.parameters()
+        c_opt = SGD(modelLParams, lr=self.lr)
+        
         # Remove "GBI" from the list of inference types if model has it
-        if hasattr(model_l, 'inferTypes'):
-            if 'GBI' in model_l.inferTypes:
-                model_l.inferTypes.remove('GBI')
+        if hasattr(self.server_model, 'inferTypes'):
+            if 'GBI' in self.server_model.inferTypes:
+                self.server_model.inferTypes.remove('GBI')
+                model_has_GBI_inference = True
         
-        no_of_not_satisfied = 0
+        node_l = None
         for c_iter in range(self.gbi_iters):
-            c_opt.zero_grad()
-
-            # -- forward pass through model_l
-            node_l, builder_l = self.populate_forward(model_l, builder) # data_item
+            # perform inference using weights from self.server_mode
+            loss, metric, node_l, _ = self.server_model(x)
 
             num_satisfied_l, num_constraints_l = self.get_constraints_satisfaction(node_l)
 
-            if num_satisfied_l == num_constraints_l:
-                is_satisfied = 1
-                no_of_not_satisfied = 0
-            else:
-                is_satisfied = 0
-                no_of_not_satisfied += 1
-
             # -- collect probs from datanode (in skeleton mode) 
-            probs = {}
+            probs = []
             for var_name, var_val in node_l.getAttribute('variableSet').items():
-                if not var_name.endswith('/label') and var_val.requires_grad:
-                    probs[var_name] = torch.sum(F.log_softmax(var_val, dim=-1))
+                if var_name.endswith('>'):# and var_val.requires_grad:
+                    probs.append(F.log_softmax(var_val, dim=-1).flatten())
 
-            # get total log prob
-            log_probs = 0.0
-            for c_prob in probs.values():
-                eps=1e-7
-                t = F.relu(c_prob)
-                tLog = torch.log(t+eps)
-                log_probs += torch.sum(tLog)
+            log_probs_cat = torch.cat(probs, dim=0)
+            log_probs = log_probs_cat.mean()
+
+            if verbose:
+                print('probs mean:')
+                print(log_probs)
 
             #  -- Constraint loss: NLL * binary satisfaction + regularization loss
             # reg loss is calculated based on L2 distance of weights between optimized model and original weights
-            c_loss = -1 * log_probs * is_satisfied + self.reg_loss(model_l, self.server_model)
+            reg_loss =  self.reg_loss(optimized_parameters, original_parameters)
+            c_loss = log_probs * ((num_constraints_l - num_satisfied_l) / num_constraints_l) + self.reg_weight * reg_loss
 
-            print("iter=%d, c_loss=%d, satisfied=%d"%(c_iter, c_loss.item(), num_satisfied_l))
+            if c_loss != c_loss:
+                # if parameter reset is normally disabled, reset parameters here
+                if not self.reset_params:
+                    self.set_pretrained(self.server_model, reload_parameters)
 
+                break
+            
+            if verbose:
+                print("iter={}, c_loss={:.4f}, c_loss.grad_fn={}, num_constraints_l={}, satisfied={}".format(c_iter, c_loss.item(), c_loss.grad_fn.__class__.__name__, num_constraints_l, num_satisfied_l))
+                print("reg_loss={:.4f}, reg_loss.grad_fn={}, log_probs={:.4f}, log_probs.grad_fn={}\n".format(reg_loss.item(), reg_loss.grad_fn.__class__.__name__, log_probs.item(), log_probs.grad_fn.__class__.__name__))
+                
             # --- Check if constraints are satisfied
             if num_satisfied_l == num_constraints_l:
                 # --- End early if constraints are satisfied
-                self.server_model 
-                return c_loss, datanode, builder
-            elif no_of_not_satisfied > 3: # three consecutive iterations where constraints are not satisfied
-                return c_loss, datanode, builder # ? float("nan")
+                if model_has_GBI_inference:
+                    self.server_model.inferTypes.append('GBI')
+
+                # reset model parameters
+                self.server_model.zero_grad()
+                if self.reset_params:
+                    self.set_pretrained(self.server_model, reload_parameters)
+                
+                print(f'Finishing GBI - Constraints are satisfied after {c_iter} iteration')
+                return c_loss, node_l, node_l.myBuilder
                         
-            # --- Backward pass on model_l
+            # --- Backward pass on self.server_model
             if c_loss.requires_grad:
+                # Compute gradients
                 c_loss.backward()
-            #  -- Update model_l
-            c_opt.step()
+                
+                if verbose:
+                    # Print the params of the model parameters which have grad
+                    print("Params before model step which have grad")
+                    for name, param in self.server_model.named_parameters():
+                        if param.grad is not None and torch.sum(torch.abs(param.grad)) > 0:
+                            print(name, 'param sum ', torch.sum(torch.abs(param)).item())
+                                            
+                # Update self.server_model params based on gradients
+                c_opt.step()
+                
+                if verbose:
+                    # Print the params of the model parameters which have grad
+                    print("Params after model step which have grad")
+                    for name, param in self.server_model.named_parameters():
+                        if param.grad is not None and torch.sum(torch.abs(param.grad)) > 0:
+                            print(name, 'param sum ', torch.sum(torch.abs(param)).item())
+            
+        if model_has_GBI_inference:
+            self.server_model.inferTypes.append('GBI')
         
-        return c_loss, datanode, builder # ? float("nan")
+        # reset model parameters
+        self.server_model.zero_grad()
+
+        # reset parameters regardless of self.reset_params flag because constraints aren't satisfied
+        self.set_pretrained(self.server_model, reload_parameters)
+
+        print(f'Finishing GBI - Constraints not are satisfied after {self.gbi_iters} iteration')
+        return c_loss, node_l, node_l.myBuilder
  
-    def calculateGBISelection(self, builder, conceptsRelations):
-        self.forward(builder)
-        
-        if builder.needsBatchRootDN():
-            builder.addBatchRootDN()
-        datanode = builder.getDataNode(device=self.device)
+    def calculateGBISelection(self, datanode, conceptsRelations):
+        c_loss, updatedDatanode, updatedBuilder = self.forward(datanode)
         
-        for c in conceptsRelations:
-            cRoot = datanode.findRootConceptOrRelation(c[0])
-            dns = datanode.findDatanodes(select = cRoot)
+        for currentConcept, conceptLen in updatedDatanode.rootConcepts:
+            cRoot = updatedDatanode.findRootConceptOrRelation(currentConcept)
+            dns = updatedDatanode.findDatanodes(select = cRoot)
+            originalDns = datanode.findDatanodes(select = cRoot)
+            
+            currentConceptRelationName = currentConcept.name
+            keyGBI  = "<" + currentConceptRelationName + ">/GBI"
             
-            keyArgmax  = "<" + c[0].name + ">/local/argmax"
-            keyGBI  = "<" + c[0].name + ">/GBI"
-
             if not dns:
                 continue
                         
-            for dn in dns: 
-                v = dn.getAttribute(c[0])
-                 
-                vGBI = torch.clone(v)
-                vArgmaxIndex = torch.argmax(v).item()
+            gbiForConcept  = None
+            if getDnSkeletonMode() and "variableSet" in updatedDatanode.attributes:
+                gbiForConcept = torch.zeros([len(dns), conceptLen], dtype=torch.float, device=updatedDatanode.current_device)
+                   
+            for i, (dn, originalDn) in enumerate(zip(dns, originalDns)): 
+                v = dn.getAttribute(currentConcept) # Get learned probabilities
+                #print(f'Net(depth={i}inGBI); pred: {torch.argmax(v, dim=-1)}')
+                if v is None:
+                    continue
                 
-                for i, _ in enumerate(v):
-                    if i == vArgmaxIndex:
-                        vGBI[i] = 1
-                    else:
-                        vGBI[i] = 0
+                # Calculate GBI results
+                vGBI = torch.zeros(v.size(), dtype=torch.float, device=updatedDatanode.current_device)
+                vArgmaxIndex = torch.argmax(v).item()
+                #print(f'vArgmaxIndex: {vArgmaxIndex}')
+                vGBI[vArgmaxIndex] = 1
                                 
-                dn.attributes[keyGBI] = vGBI
+                # Add GBI inference result to the original datanode 
+                if gbiForConcept is not None:
+                    if conceptLen == 1: # binary concept
+                        gbiForConcept[i] = vGBI[1]
+                    else:
+                        gbiForConcept[i] = vGBI
+                else:
+                    originalDn.attributes[keyGBI] = vGBI
+            
+            if gbiForConcept is not None:
+                rootConceptName = cRoot.name
+                keyGBIInVariableSet = rootConceptName + "/" + keyGBI
+                datanode.attributes["variableSet"][keyGBIInVariableSet]=gbiForConcept
+                updatedDatanode.attributes["variableSet"][keyGBIInVariableSet]=gbiForConcept
 
+        return
\ No newline at end of file
diff --git a/domiknows/program/model/ilpu.py b/domiknows/program/model/ilpu.py
index bc7728731..1041a9878 100644
--- a/domiknows/program/model/ilpu.py
+++ b/domiknows/program/model/ilpu.py
@@ -15,8 +15,7 @@ def poi_loss(self, data_item, prop, sensors):
             return None
 
         builder = data_item
-        if (builder.needsBatchRootDN()):
-            builder.addBatchRootDN()
+        builder.createBatchRootDN()
         datanode = builder.getDataNode(device=self.device)
         concept = prop.sup
         values = []
diff --git a/domiknows/program/model/iml.py b/domiknows/program/model/iml.py
index e72897f34..91fa2357d 100644
--- a/domiknows/program/model/iml.py
+++ b/domiknows/program/model/iml.py
@@ -15,8 +15,7 @@ def poi_loss(self, data_item, prop, sensors):
             return None
 
         builder = data_item
-        if (builder.needsBatchRootDN()):
-            builder.addBatchRootDN()
+        builder.createBatchRootDN()
         datanode = builder.getDataNode(device=self.device)
         concept = prop.sup
         values = []
diff --git a/domiknows/program/model/lossModel.py b/domiknows/program/model/lossModel.py
index 6ae93d794..5499fa8f2 100644
--- a/domiknows/program/model/lossModel.py
+++ b/domiknows/program/model/lossModel.py
@@ -12,13 +12,38 @@ class LossModel(torch.nn.Module):
     logger = logging.getLogger(__name__)
 
     def __init__(self, graph, 
-                 tnorm=DataNode.tnormsDefault, 
+                 tnorm='P',
+                 counting_tnorm=None,
                  sample = False, sampleSize = 0, sampleGlobalLoss = False, device='auto'):
+        """
+        This function initializes a LossModel object with the given parameters and sets up the
+        necessary variables and constraints.
+        
+        :param graph: The `graph` parameter is an object that represents the logical constraints of a
+        graph. It contains information about the nodes, edges, and constraints of the graph
+        :param tnorm: The `tnorm` parameter specifies the type of t-norm to be used in the model.
+        T-norms are a family of binary operations that are used to model logical conjunction (AND) in
+        fuzzy logic. The default value is 'P', which stands for the product t-norm, defaults to P
+        (optional)
+        :param sample: The `sample` parameter is a boolean flag that determines whether to use sampling
+        during training. If set to `True`, the model will use sampling to estimate the loss function. If
+        set to `False`, the model will not use sampling and will use the exact loss function, defaults
+        to False (optional)
+        :param sampleSize: The `sampleSize` parameter determines the size of the sample used for
+        training. It specifies the number of samples that will be randomly selected from the dataset for
+        each training iteration, defaults to 0 (optional)
+        :param sampleGlobalLoss: The parameter `sampleGlobalLoss` is a boolean flag that determines
+        whether to sample the global loss during training. If `sampleGlobalLoss` is set to `True`, the
+        global loss will be sampled. Otherwise, it will not be sampled, defaults to False (optional)
+        :param device: The `device` parameter specifies the device (CPU or GPU) on which the model will
+        be trained and evaluated. It can take the following values:, defaults to auto (optional)
+        """
         super().__init__()
         self.graph = graph
         self.build = True
         
         self.tnorm = tnorm
+        self.counting_tnorm = counting_tnorm
         self.device = device
         
         self.sample = sample
@@ -59,6 +84,13 @@ def reset(self):
             self.loss.reset()
 
     def get_lmbd(self, key):
+        """
+        The function `get_lmbd` returns a clamped value from a dictionary based on a given key.
+        
+        :param key: The key parameter is used to access a specific value in the lmbd dictionary
+        :return: the value of `self.lmbd[self.lmbd_index[key]]` after clamping it to a maximum value of
+        `self.lmbd_p[self.lmbd_index[key]]`.
+        """
         return self.lmbd[self.lmbd_index[key]].clamp(max=self.lmbd_p[self.lmbd_index[key]])
 
     def forward(self, builder, build=None):
@@ -68,12 +100,11 @@ def forward(self, builder, build=None):
         if not build and not isinstance(builder, DataNodeBuilder):
             raise ValueError('PrimalDualModel must be invoked with `build` on or with provided DataNode Builder.')
         
-        if (builder.needsBatchRootDN()):
-            builder.addBatchRootDN()
+        builder.createBatchRootDN()
         datanode = builder.getDataNode(device=self.device)
         
         # Call the loss calculation returns a dictionary, keys are matching the constraints
-        constr_loss = datanode.calculateLcLoss(tnorm=self.tnorm, sample=self.sample, sampleSize = self.sampleSize)
+        constr_loss = datanode.calculateLcLoss(tnorm=self.tnorm,counting_tnorm=self.counting_tnorm, sample=self.sample, sampleSize = self.sampleSize)
 
         lmbd_loss = []
         if self.sampleGlobalLoss and constr_loss['globalLoss']:
@@ -97,20 +128,34 @@ def forward(self, builder, build=None):
         # (*out, datanode, builder)
         return lmbd_loss, datanode, builder
     
+# The `PrimalDualModel` class is a subclass of `LossModel` that implements a primal-dual optimization
+# algorithm.
 class PrimalDualModel(LossModel):
     logger = logging.getLogger(__name__)
 
-    def __init__(self, graph, tnorm=DataNode.tnormsDefault, device='auto'):
-        super().__init__(graph, tnorm=tnorm, device=device)
+    def __init__(self, graph, tnorm='P',counting_tnorm=None, device='auto'):
+        """
+        The above function is the constructor for a class that initializes an object with a graph,
+        tnorm, and device parameters.
+        
+        :param graph: The `graph` parameter is the input graph that the coding assistant is being
+        initialized with. It represents the structure of the graph and can be used to perform various
+        operations on the graph, such as adding or removing nodes and edges, calculating node and edge
+        properties, and traversing the graph
+        :param tnorm: The tnorm parameter is used to specify the type of t-norm to be used in the graph.
+        A t-norm is a binary operation that generalizes the concept of conjunction (logical AND) to
+        fuzzy logic. The 'P' value for tnorm indicates that the product t-norm should, defaults to P
+        (optional)
+        :param device: The `device` parameter specifies the device on which the computations will be
+        performed. It can take the following values:, defaults to auto (optional)
+        """
+        super().__init__(graph, tnorm=tnorm, counting_tnorm = counting_tnorm, device=device)
         
 class SampleLossModel(torch.nn.Module):
     logger = logging.getLogger(__name__)
 
-    # def __init__(self, graph, sample = False, sampleSize = 0, sampleGlobalLoss = False):
-    #     super().__init__(graph, sample=sample, sampleSize=sampleSize, sampleGlobalLoss=sampleGlobalLoss)
-
     def __init__(self, graph, 
-                 tnorm=DataNode.tnormsDefault, 
+                 tnorm='P', 
                  sample = False, sampleSize = 0, sampleGlobalLoss = False, device='auto'):
         super().__init__()
         self.graph = graph
@@ -149,12 +194,31 @@ def reset(self):
             self.loss.reset()
 
     def get_lmbd(self, key):
+        """
+        The function `get_lmbd` returns the value of `self.lmbd` at the index specified by
+        `self.lmbd_index[key]`, ensuring that the value is non-negative.
+        
+        :param key: The `key` parameter is used to access a specific element in the `self.lmbd` list. It
+        is used as an index to retrieve the corresponding value from the list
+        :return: the value of `self.lmbd[self.lmbd_index[key]]`.
+        """
         if self.lmbd[self.lmbd_index[key]] < 0:
             with torch.no_grad():
                 self.lmbd[self.lmbd_index[key]] = 0
         return self.lmbd[self.lmbd_index[key]]
 
     def forward(self, builder, build=None):
+        """
+        The `forward` function calculates the loss for a PrimalDualModel using a DataNodeBuilder and
+        returns the loss value, the DataNode, and the builder.
+        
+        :param builder: The `builder` parameter is an instance of the `DataNodeBuilder` class. It is
+        used to create a batch root data node and retrieve a data node
+        :param build: The `build` parameter is an optional argument that specifies whether the
+        `DataNodeBuilder` should be invoked or not. If `build` is `None`, then the value of `self.build`
+        is used. If `build` is `True`, then the `createBatchRootDN()` method
+        :return: three values: lmbd_loss, datanode, and builder.
+        """
         if build is None:
             build = self.build
         self.iter_step += 1
@@ -162,9 +226,8 @@ def forward(self, builder, build=None):
         if not build and not isinstance(builder, DataNodeBuilder):
             raise ValueError('PrimalDualModel must be invoked with `build` on or with provided DataNode Builder.')
         
-        if (builder.needsBatchRootDN()):
-            builder.addBatchRootDN()
-        
+        builder.createBatchRootDN()
+
 #       self.loss.reset()
 
         datanode = builder.getDataNode(device=self.device)
diff --git a/domiknows/program/model/pytorch.py b/domiknows/program/model/pytorch.py
index fc8da1573..696fee0f0 100644
--- a/domiknows/program/model/pytorch.py
+++ b/domiknows/program/model/pytorch.py
@@ -12,20 +12,44 @@
 from .base import Mode
 from ..tracker import MacroAverageTracker
 from ..metric import MetricTracker
+from domiknows.utils import setDnSkeletonMode, getDnSkeletonMode
 
 
 class TorchModel(torch.nn.Module):
-    def __init__(self, graph, device='auto'):
+
+    def __init__(self, graph, device='auto', ignore_modules=False):
+        """
+        The function initializes an object with a graph, device, and build flag.
+        
+        :param graph: The `graph` parameter is an object that represents the computational graph. It
+        contains the nodes and edges that define the operations and data flow in the graph
+        :param device: The `device` parameter specifies the device on which the graph will be executed.
+        It can be set to `'auto'` to automatically select the device based on the availability of GPUs.
+        Alternatively, it can be set to a specific device, such as `'cpu'` or `'cuda:0', defaults to auto
+        (optional)
+        :param ignore_modules: The `ignore_modules` parameter is a boolean flag that determines whether
+        or not to ignore any modules in the graph. If set to `True`, any modules present in the graph
+        will be ignored during training. If set to `False`, all modules in the graph will be considered
+        during training, defaults to False (optional)
+        """
         super().__init__()
         self.graph = graph
         self.mode(Mode.TRAIN)
         self.build = True
         self.device = device
 
-        for learner in self.graph.get_sensors(TorchLearner):
-            self.add_module(learner.fullname, learner.model)
+        if not ignore_modules: ### added for the inference only models which do not update the initial parameters
+            for learner in self.graph.get_sensors(TorchLearner):
+                self.add_module(learner.fullname, learner.model)
 
     def mode(self, mode=None):
+        """
+        The mode function sets the mode of the object and performs certain actions based on the mode.
+        
+        :param mode: The `mode` parameter is used to specify the mode of operation for the code. It can
+        take one of the following values:
+        :return: The method is returning the value of the `_mode` attribute.
+        """
         if mode is None:
             return self._mode
         if mode in (Mode.TEST, Mode.POPULATE):
@@ -73,6 +97,22 @@ def data_hash(self, data_item):
             raise ValueError(f'To enable cache for {self}, data item must either contain a identifier key "id" or picklable (might be slow): \n{key_message}\n{dump_message}')
 
     def forward(self, data_item, build=None):
+        """
+        The `forward` function takes a data item, performs various operations on it based on the graph
+        structure, and returns the output.
+        
+        :param data_item: The `data_item` parameter is the input data that will be passed through the
+        forward pass of the neural network. It can be either a dictionary or an iterable object. If it
+        is an iterable, it will be converted into a dictionary where the keys are the indices of the
+        elements in the iterable
+        :param build: The `build` parameter is a boolean flag that indicates whether the method should
+        build a data node or not. If `build` is `True`, the method will build a data node and return the
+        loss, metric, data node, and builder. If `build` is `False`, the method
+        :return: If the `build` parameter is `True`, the function returns a tuple containing `(loss,
+        metric, datanode, builder)`. 
+        If the `build` parameter is `False`, the function returns a tuple containing the values returned
+        by the `populate` method.
+        """
         if build is None:
             build = self.build
         data_hash = None
@@ -88,14 +128,25 @@ def forward(self, data_item, build=None):
             sensor.fill_hash(data_hash)
         for sensor in self.graph.get_sensors(ReaderSensor):
             sensor.fill_data(data_item)
+            
         if build:
             data_item.update({"graph": self.graph, 'READER': 0})
+            # build datanode with tensor dictionary for attributes when not ILP or GBI inference needed
+            if getDnSkeletonMode():
+                if not hasattr(self, 'inferTypes') or ('ILP' not in self.inferTypes and 'GBI' not in self.inferTypes):
+                    setDnSkeletonMode(True, full=True)
             builder = DataNodeBuilder(data_item)
-            *out, = self.populate(builder)
-            if (builder.needsBatchRootDN()):
-                builder.addBatchRootDN()
-            datanode = builder.getDataNode(context="build", device=self.device)
-            return (*out, datanode, builder)
+            out = self.populate(builder)
+            
+            if len(out) == 2:
+                builder.createBatchRootDN()
+                datanode = builder.getDataNode(context="build", device=self.device)
+                loss = out[0]
+                metric = out[1]
+            else:
+                datanode, loss, metric = out
+                
+            return (loss, metric, datanode, builder)
         else:
             *out, = self.populate(data_item)
             return (*out,)
@@ -109,8 +160,31 @@ def model_helper(Model, *args, **kwargs):
 
 
 class PoiModel(TorchModel):
-    def __init__(self, graph, poi=None, loss=None, metric=None, device='auto'):
-        super().__init__(graph, device)
+    def __init__(self, graph, poi=None, loss=None, metric=None, device='auto', ignore_modules=False):
+        """
+        The function initializes an object with various parameters for graph processing.
+        
+        :param graph: The graph parameter is used to specify the graph structure. It represents the
+        connections between nodes in the graph
+        :param poi: The "poi" parameter stands for "point of interest". It is used to specify a specific
+        node or set of nodes in the graph that you are interested in. This can be useful when you want to
+        perform operations or calculations only on a subset of nodes in the graph. If no point of interest
+        :param loss: The `loss` parameter is used to specify the loss function that will be used during
+        training. It is typically a function that measures the difference between the predicted output and
+        the true output. The choice of loss function depends on the specific task and the type of data being
+        used. Some common loss functions include
+        :param metric: The `metric` parameter is used to specify the evaluation metric to be used during
+        training and evaluation of the model. It is typically a function that takes the predicted values and
+        the ground truth values as inputs and returns a scalar value representing the performance of the
+        model. Examples of common evaluation metrics include accuracy,
+        :param device: The `device` parameter specifies the device on which the computation will be
+        performed. It can take the following values:, defaults to auto (optional)
+        :param ignore_modules: The `ignore_modules` parameter is a boolean flag that determines whether to
+        ignore any modules in the graph. If set to `True`, any modules present in the graph will be ignored
+        during the initialization process. If set to `False` (default), all modules in the graph will be
+        considered during initialization, defaults to False (optional)
+        """
+        super().__init__(graph, device, ignore_modules=ignore_modules)
         if poi is None:
             self.poi = self.default_poi()
         else:
@@ -133,6 +207,12 @@ def __init__(self, graph, poi=None, loss=None, metric=None, device='auto'):
             self.metric = {'': metric}
 
     def default_poi(self):
+        """
+        The function `default_poi` returns a list of properties from a graph that have more than one
+        instance of the `TorchSensor` class.
+        :return: a list of properties that have more than one instance of the `TorchSensor` class in the
+        graph.
+        """
         poi = []
         for prop in self.graph.get_properties():
             if len(list(prop.find(TorchSensor))) > 1:
@@ -140,6 +220,14 @@ def default_poi(self):
         return poi
 
     def find_sensors(self, prop):
+        """
+        The function `find_sensors` finds pairs of sensors in a given property that have one sensor
+        labeled as the target and the other sensor as the output.
+        
+        :param prop: The parameter "prop" is expected to be an object that has a method called "find"
+        which takes a class name as an argument and returns a list of objects of that class. In this
+        case, it is being used to find objects of the class "TorchSensor"
+        """
         for sensor1, sensor2 in combinations(prop.find(TorchSensor), r=2):
             if sensor1.label:
                 target_sensor = sensor1
@@ -164,19 +252,52 @@ def reset(self):
                     metric.reset()
 
     def poi_loss(self, data_item, _, sensors):
+        """
+        The function calculates the loss for a given data item using a set of sensors.
+        
+        :param data_item: The `data_item` parameter represents a single data item that is being
+        processed. It could be any type of data, depending on the context of your code
+        :param _: The underscore "_" is a convention in Python to indicate that a variable is not going
+        to be used in the code. It is often used as a placeholder for a variable that needs to be
+        present for the function signature but is not actually used within the function. In this case,
+        it seems that the variable
+        :param sensors: The `sensors` parameter is a list of sensor functions. These sensor functions
+        take a `data_item` as input and return some output
+        :return: the calculated local_loss.
+        """
         if not self.loss:
             return 0
+        
         outs = [sensor(data_item) for sensor in sensors]
+        
         if len(outs[0]) == 0:
             return None
-        local_loss = self.loss[(*sensors,)](*outs)
+        
+        selfLoss = self.loss[(*sensors,)]
+        local_loss = selfLoss(*outs)
         
         if local_loss != local_loss:
             raise Exception("Calculated local_loss is nan") 
         
         return local_loss
 
-    def poi_metric(self, data_item, prop, sensors):
+    def poi_metric(self, data_item, prop, sensors, datanode=None):
+        """
+        The `poi_metric` function calculates a local metric based on the given data item, property,
+        sensors, and optional datanode.
+        
+        :param data_item: The `data_item` parameter is a single data item that is being processed. It
+        could be any type of data, depending on the context of your code
+        :param prop: The "prop" parameter is a property or attribute of the data item that is being
+        evaluated
+        :param sensors: The `sensors` parameter is a list of sensor functions. These sensor functions
+        take a `data_item` as input and return a value. The `sensors` list contains multiple sensor
+        functions that will be called to collect data for the metric calculation
+        :param datanode: The `datanode` parameter is an optional argument that represents the data node
+        for which the metric is being calculated. It is used as an input to the metric function to
+        provide additional context or information for the calculation
+        :return: the local_metric value.
+        """
         if not self.metric:
             return None
         outs = [sensor(data_item) for sensor in sensors]
@@ -184,13 +305,28 @@ def poi_metric(self, data_item, prop, sensors):
             return None
         local_metric = {}
         for key, metric in self.metric.items():
-            local_metric[key] = metric[(*sensors,)](*outs, data_item=data_item, prop=prop)
+            local_metric[key] = metric[(*sensors,)](*outs, data_item=datanode, prop=prop)
         if len(local_metric) == 1:
             local_metric = list(local_metric.values())[0]
             
         return local_metric
 
-    def populate(self, builder, run=True):
+    def populate(self, builder, datanode = None, run=True):
+        """
+        The `populate` function evaluates sensors, calculates loss and metrics, and returns the total
+        loss and metric values.
+        
+        :param builder: The `builder` parameter is an object that is used to construct and populate data
+        nodes in a data structure. It is likely an instance of a class that provides methods for
+        creating and manipulating data nodes
+        :param datanode: The `datanode` parameter is an optional argument that represents a data node in
+        the builder. It is used to store the metric values calculated during the population process. If
+        `datanode` is not provided, a new batch root data node is created and assigned to `datanode`
+        :param run: The `run` parameter is a boolean flag that determines whether the sensors should be
+        evaluated or not. If `run` is `True`, the sensors will be evaluated by calling their `__call__`
+        method. If `run` is `False`, the sensors will not be evaluated, defaults to True (optional)
+        :return: two values: `loss` and `metric`.
+        """
         loss = 0
         metric = {}
         
@@ -208,15 +344,54 @@ def populate(self, builder, run=True):
                         if local_loss is not None:
                             loss += local_loss
                     if self.metric:
-                        local_metric = self.poi_metric(builder, prop, sensors)
+                        if datanode is None:
+                            builder.createBatchRootDN()
+                            datanode = builder.getDataNode()
+            
+                        local_metric = self.poi_metric(builder, prop, sensors, datanode=datanode)
                         if local_metric is not None:
                             metric[(*sensors,)] = local_metric
         
         return loss, metric
 
 class SolverModel(PoiModel):
-    def __init__(self, graph, poi=None, loss=None, metric=None, inferTypes=None, inference_with = None, device='auto'):
-        super().__init__(graph, poi=poi, loss=loss, metric=metric, device=device)
+    def __init__(self, graph, poi=None, loss=None, metric=None, inferTypes=None, inference_with = None, probKey = ("local" , "softmax"), device='auto', probAcc=None, ignore_modules=False, kwargs = None):
+        """
+        The function initializes an object with various parameters for graph-based inference.
+        
+        :param graph: The graph parameter is used to specify the graph structure that the model will be
+        trained on. It can be a graph object or a graph file
+        :param poi: The "poi" parameter stands for "point of interest". It is used to specify a specific
+        node or set of nodes in the graph that you want to focus on or perform operations on. If you don't
+        specify any value for "poi", it will default to None, meaning that the entire graph
+        :param loss: The `loss` parameter is used to specify the loss function to be used during training.
+        It is typically a function that measures the difference between the predicted output and the true
+        output. The choice of loss function depends on the specific task and the type of data being used.
+        Some common loss functions include mean
+        :param metric: The `metric` parameter is used to specify the evaluation metric to be used during
+        training and inference. It is typically a function that takes the predicted values and the ground
+        truth values as inputs and returns a scalar value representing the performance of the model. Common
+        examples of evaluation metrics include accuracy, precision, recall
+        :param inferTypes: The `inferTypes` parameter is used to specify the types of nodes in the graph
+        that should be inferred. It is a list of strings, where each string represents a type of node. Only
+        nodes with these types will be inferred during the graph inference process
+        :param inference_with: The "inference_with" parameter is used to specify the type of inference to be
+        performed. It determines how the model will make predictions or inferences based on the input data.
+        The possible values for this parameter can vary depending on the specific implementation or
+        framework being used. Some common options include "classification
+        :param probKey: The `probKey` parameter is a tuple that specifies the type of probability
+        distribution to use for inference. The first element of the tuple specifies the type of distribution
+        to use for local inference, and the second element specifies the type of distribution to use for
+        softmax inference
+        :param device: The `device` parameter specifies the device on which the computations will be
+        performed. It can take the following values:, defaults to auto (optional)
+        :param probAcc: The `probAcc` parameter is the values of accuracies used inside the new ILP variation:
+        :param ignore_modules: The `ignore_modules` parameter is a boolean flag that determines whether to
+        ignore certain modules during the inference process. If set to `True`, the inference process will
+        not consider these modules. If set to `False`, all modules will be considered during inference,
+        defaults to False (optional)
+        """
+        super().__init__(graph, poi=poi, loss=loss, metric=metric, device=device, ignore_modules=ignore_modules)
         
         if inferTypes == None:
             self.inferTypes = ['ILP']
@@ -227,8 +402,23 @@ def __init__(self, graph, poi=None, loss=None, metric=None, inferTypes=None, inf
             self.inference_with = []
         else:
             self.inference_with = inference_with
+            
+        self.probKey = probKey
+        self.probAcc = probAcc
+        self.kwargs = kwargs
 
     def inference(self, builder):
+        """
+        The `inference` function takes a builder object, iterates over a list of properties, and performs
+        inference using different types of models based on the `inferTypes` list.
+        
+        :param builder: The `builder` parameter is an object that is used to construct a computation
+        graph. It is typically used to define the inputs, operations, and outputs of a neural network
+        model
+        :return: the `datanode` object.
+        """
+        # import time
+        # start = time.time()
         for i, prop in enumerate(self.poi):
             for sensor in prop.find(TorchSensor):
                 sensor(builder)
@@ -239,27 +429,53 @@ def inference(self, builder):
 #         print("Done with the computation")
 
         # Check if this is batch
-        if (builder.needsBatchRootDN()):
-            builder.addBatchRootDN()
+        # end = time.time()
+        # print("Time taken for computation: ", end-start)
+        # start = time.time()
+        builder.createBatchRootDN()
         datanode = builder.getDataNode(device=self.device)
+        # end = time.time()
+        # print("Time taken for creating datanode in its builder: ", end-start)
         # trigger inference
 #         fun=lambda val: torch.tensor(val, dtype=float).softmax(dim=-1).detach().cpu().numpy().tolist()
+        # start = time.time()
         if datanode:
             for infertype in self.inferTypes:
+                # sub_start = time.time()
                 {
-                    'ILP': lambda :datanode.inferILPResults(*self.inference_with, fun=None, epsilon=None),
                     'local/argmax': lambda :datanode.inferLocal(),
                     'local/softmax': lambda :datanode.inferLocal(),
                     'argmax': lambda :datanode.infer(),
                     'softmax': lambda :datanode.infer(),
-                    'GBI': lambda :datanode.inferGBIResults(*self.inference_with, model=self, builder=builder),
+                    'ILP': lambda :datanode.inferILPResults(*self.inference_with, key=self.probKey, fun=None, epsilon=None, Acc=self.probAcc),
+                    'GBI': lambda :datanode.inferGBIResults(*self.inference_with, model=self, kwargs = self.kwargs),
                 }[infertype]()
+                # sub_end = time.time()
+                # print("Time taken for inference of type ", infertype, " : ", sub_end-sub_start)
     #         print("Done with the inference")
-        return builder
+        # end = time.time()
+        # print("Time taken for inference after datanode creation: ", end-start)
+        return datanode
 
     def populate(self, builder, run=True):
-        data_item = self.inference(builder)
-        return super().populate(builder, run=False)
+        """
+        The `populate` function takes a `builder` object, performs inference on it, and then calls the
+        `populate` method of the superclass with the resulting `datanode`, returning the `datanode`,
+        `lose`, and `metric` values.
+        
+        :param builder: The "builder" parameter is an object that is used to build or construct the data
+        node. It is likely an instance of a class that has methods for creating and manipulating data
+        nodes
+        :param run: The "run" parameter is a boolean flag that determines whether to run the population
+        process immediately after populating the data node. If set to True, the population process will
+        be executed; if set to False, the population process will be skipped, defaults to True
+        (optional)
+        :return: three values: datanode, lose, and metric.
+        """
+        datanode = self.inference(builder)
+        lose, metric = super().populate(builder, datanode = datanode, run=False)
+        
+        return datanode, lose, metric
     
 
 class PoiModelToWorkWithLearnerWithLoss(TorchModel):
@@ -320,8 +536,7 @@ def loss(self):
     def metric(self):
         # return self.metrics_tracker
         pass
-    
-    
+  
 class PoiModelDictLoss(PoiModel):
     def __init__(self, graph, poi=None, loss=None, metric=None, dictloss=None, device='auto'):
         self.loss_tracker = MacroAverageTracker()
@@ -367,7 +582,7 @@ def poi_loss(self, data_item, prop, sensors):
             elif not str(prop.name) in self.dictloss:
                 self.losses[pred, target] = self.dictloss["default"](data_item, prop, pred(data_item), target(data_item))
                 if torch.isnan(self.losses[pred, target]).any() :
-                    print("here it is")
+                    print("The loss has resulted in a Nan value")
                 return self.losses[pred, target]
             else:
                 self.losses[pred, target] = self.dictloss[str(prop.name)](data_item, prop, pred(data_item), target(data_item))
@@ -384,9 +599,7 @@ def populate(self, builder, run=True):
             return loss, metric
         else:
             return super().populate(builder, run=True)
-        
-    
-    
+   
 # class PoiModelDictLoss(TorchModel):
 #     def __init__(self, graph, poi=None, loss=None, metric=None):
 #         super().__init__(graph)
@@ -485,8 +698,7 @@ def inference(self, builder):
 #         print("Done with the computation")
 
         # Check if this is batch
-        if (builder.needsBatchRootDN()):
-            builder.addBatchRootDN()
+        builder.createBatchRootDN()
         datanode = builder.getDataNode(device=self.device)
         # trigger inference
 #         fun=lambda val: torch.tensor(val, dtype=float).softmax(dim=-1).detach().cpu().numpy().tolist()
diff --git a/domiknows/program/program.py b/domiknows/program/program.py
index 6493e4f3b..684859ed7 100644
--- a/domiknows/program/program.py
+++ b/domiknows/program/program.py
@@ -157,21 +157,40 @@ def __call__(self, stepName, metricName, metricResult):
 
 class LearningBasedProgram():
     def __init__(self, graph, Model, logger=None, db=False, **kwargs):
+        """
+        This function initializes an object with a graph, a model, a logger, and other optional
+        parameters.
+        
+        :param graph: The `graph` parameter is an object that represents a graph or network structure.
+        It is likely used in the `Model` class to define the architecture of the neural network
+        :param Model: The `Model` parameter is the class that represents the machine learning model you
+        want to use. It should have an `__init__` method that takes the `graph` and any additional
+        keyword arguments (`**kwargs`) as parameters. The `Model` class is used to create an instance of
+        the
+        :param logger: The `logger` parameter is an optional logger object that can be used for logging
+        messages and debugging information. If no logger is provided, a default logger will be used
+        :param db: The `db` parameter is a boolean flag that indicates whether or not to perform
+        database updates. If `db` is `True`, database updates will be performed. If `db` is `False`, no
+        database updates will be performed, defaults to False (optional)
+        """
         self.graph = graph
 
         self.logger = logger or logging.getLogger(__name__)
         self.dbUpdate = None if db else dbUpdate(graph)
 
         from inspect import signature
-        modelSignature = signature(Model.__init__)
+        self.modelSignature = signature(Model.__init__)
         
-        modelKwargs = {}
-        for param in modelSignature.parameters.values():
+        self.kwargs = kwargs
+        self.modelKwargs = {}
+        for param in self.modelSignature.parameters.values():
             paramName = param.name
             if paramName in kwargs:
-                modelKwargs[paramName] = kwargs[paramName]
+                self.modelKwargs[paramName] = kwargs[paramName]
+            if paramName == "kwargs":
+                self.modelKwargs[paramName] = kwargs
                 
-        self.model = Model(graph, **modelKwargs)
+        self.model = Model(graph, **self.modelKwargs)
         self.opt = None
         self.epoch = None
         self.stop = None
@@ -195,6 +214,17 @@ def to(self, device='auto'):
             self.model.device = self.device
 
     def calculateMetricDelta(self, metric1, metric2):
+        """
+        The function calculates the difference between two metrics and returns the delta.
+        
+        :param metric1: The first metric, represented as a dictionary. Each key in the dictionary
+        represents a category, and the corresponding value is another dictionary where the keys
+        represent subcategories and the values represent the metric values
+        :param metric2: The `metric2` parameter is a dictionary representing a metric. It has a nested
+        structure where the keys represent categories and the values represent subcategories and their
+        corresponding values
+        :return: a dictionary called metricDelta.
+        """
         metricDelta = {}
         for k, v in metric1.value().items():
             metricDelta[k] = {}
@@ -207,6 +237,18 @@ def calculateMetricDelta(self, metric1, metric2):
         return metricDelta
 
     def call_epoch(self, name, dataset, epoch_fn, **kwargs):
+        """
+        The function `call_epoch` logs information about the loss and metrics of a model during an epoch
+        and updates a database if specified.
+        
+        :param name: The name of the epoch or task being performed. It is used for logging purposes
+        :param dataset: The `dataset` parameter is the input dataset that will be used for training or
+        evaluation. It is typically a collection of data samples that the model will process
+        :param epoch_fn: The `epoch_fn` parameter is a function that represents a single epoch of
+        training or evaluation. It takes the `dataset` as input and performs the necessary operations
+        for that epoch, such as forward and backward passes, updating model parameters, and calculating
+        metrics
+        """
         if dataset is not None:
             self.logger.info(f'{name}:')
             desc = name if self.epoch is None else f'Epoch {self.epoch} {name}'
@@ -269,6 +311,31 @@ def train(
         test_every_epoch=False,
         Optim=None,
         **kwargs):
+        """
+        The `train` function is used to train a model on a given training set, with optional validation
+        and testing sets, for a specified number of epochs.
+        
+        :param training_set: The training set is the dataset used to train the model. It typically
+        consists of input data and corresponding target labels
+        :param valid_set: The valid_set parameter is used to specify the validation dataset. It is
+        typically a separate portion of the training dataset that is used to evaluate the model's
+        performance during training and tune hyperparameters
+        :param test_set: The `test_set` parameter is used to specify the dataset that will be used for
+        testing the model's performance after each epoch of training. It is typically a separate dataset
+        from the training and validation sets, and is used to evaluate the model's generalization
+        ability on unseen data
+        :param device: The device on which the model will be trained (e.g., 'cpu' or 'cuda')
+        :param train_epoch_num: The number of epochs to train the model for. An epoch is a complete pass
+        through the entire training dataset, defaults to 1 (optional)
+        :param test_every_epoch: The parameter "test_every_epoch" is a boolean flag that determines
+        whether to perform testing after every epoch during training. If set to True, testing will be
+        performed after each epoch. If set to False, testing will only be performed once at the end of
+        training, defaults to False (optional)
+        :param Optim: The `Optim` parameter is used to specify the optimizer to be used for training the
+        model. It should be a class that implements the optimization algorithm, such as
+        `torch.optim.SGD` or `torch.optim.Adam`. The optimizer is responsible for updating the model's
+        parameters based on the computed gradients
+        """
         if device is not None:
             self.to(device)
         if Optim is not None and list(self.model.parameters()):
@@ -292,23 +359,52 @@ def train(
         self.stop = None
 
     def train_epoch(self, dataset, **kwargs):
+        """
+        The function `train_epoch` trains a model on a dataset for one epoch, updating the model's
+        parameters based on the calculated loss and performing gradient descent if an optimizer is
+        provided.
+        
+        :param dataset: The `dataset` parameter is the training dataset that contains the input data and
+        corresponding labels. It is used to iterate over the data items during training
+        """
         self.model.mode(Mode.TRAIN)
         self.model.reset()
         for data_item in dataset:
+            
             if self.opt is not None:
                 self.opt.zero_grad()
+                
             loss, metric, *output = self.model(data_item)
             if self.opt and loss:
                 loss.backward()
                 self.opt.step()
+                
             yield (loss, metric, *output[:1])
 
     def test(self, dataset, device=None, **kwargs):
+        """
+        The function `test` is used to test a model on a given dataset, with an optional device argument
+        for specifying the device to run the test on.
+        
+        :param dataset: The dataset parameter is the dataset object that contains the testing data. It
+        is used to evaluate the performance of the model on the testing data
+        :param device: The "device" parameter is used to specify the device on which the model should be
+        tested. It can be set to "None" if you want to test the model on the CPU, or it can be set to a
+        specific device such as "cuda" if you want to test the model on
+        """
         if device is not None:
             self.to(device)
         self.call_epoch('Testing', dataset, self.test_epoch, **kwargs)
 
     def test_epoch(self, dataset, **kwargs):
+        """
+        The function `test_epoch` is used to evaluate a model on a dataset during the testing phase,
+        yielding the loss, metric, and output for each data item.
+        
+        :param dataset: The `dataset` parameter is the input dataset that you want to test your model
+        on. It could be a list, generator, or any other iterable object that provides the data items to
+        be tested. Each data item should be in a format that can be processed by your model
+        """
         self.model.mode(Mode.TEST)
         self.model.reset()
         with torch.no_grad():
@@ -320,25 +416,85 @@ def populate(self, dataset, device=None):
         if device is not None:
             self.to(device)
         yield from self.populate_epoch(dataset)
+        
+    def populate_one(self, data_item, grad = False, device=None):
+        if device is not None:
+            self.to(device)
+        return next(self.populate_epoch([data_item], grad = grad))
 
-    def populate_epoch(self, dataset):
+    def populate_epoch(self, dataset, grad = False):
+        """
+        The `populate_epoch` function is used to iterate over a dataset and yield the output of a model
+        for each data item, either with or without gradient calculations.
+        
+        :param dataset: The `dataset` parameter is the input data that you want to use to populate the
+        model. It could be a list, array, or any other iterable object that contains the data items
+        :param grad: The `grad` parameter is a boolean flag that determines whether or not to compute
+        gradients during the epoch. If `grad` is set to `False`, the epoch will be executed in
+        evaluation mode without computing gradients. If `grad` is set to `True`, the epoch will be
+        executed in training, defaults to False (optional)
+        """
         self.model.mode(Mode.POPULATE)
         self.model.reset()
-        with torch.no_grad():
+        
+        try:
+            lenI = len(dataset)
+            print(f"\nNumber of iterations in epoch: {lenI}")
+        except:
+            pass
+
+        if not grad:
+            with torch.no_grad():
+                for i, data_item in tqdm(enumerate(dataset)):
+                    # import time
+                    # start = time.time()
+                    loss, metric, datanode, builder = self.model(data_item)
+                    # end = time.time()
+                    # print("Time taken for one data item in populate epoch: ", end - start)
+                    yield detuple(datanode)
+        else:
             for i, data_item in tqdm(enumerate(dataset)):
+                # import time
+                # start = time.time()
+                data_item["modelKwargs"] = self.modelKwargs
                 _, _, *output = self.model(data_item)
+                # end = time.time()
+                # print("Time taken for one data item in populate epoch: ", end - start)
                 yield detuple(*output[:1])
 
-    def populate_one(self, data_item):
-        return next(self.populate_epoch([data_item]))
-
     def save(self, path, **kwargs):
+        """
+        The function saves the state dictionary of a model to a specified path using the torch.save()
+        function.
+        
+        :param path: The path where the model's state dictionary will be saved
+        """
         torch.save(self.model.state_dict(), path, **kwargs)
 
     def load(self, path, **kwargs):
+        """
+        The function loads a saved model state dictionary from a specified path.
+        
+        :param path: The path parameter is the file path to the saved model state dictionary
+        """
         self.model.load_state_dict(torch.load(path, **kwargs))
 
     def verifyResultsLC(self,data,constraint_names=None,device=None):
+        """
+        The function `verifyResultsLC` calculates and prints the accuracy of constraint verification
+        results for a given dataset.
+        
+        :param data: The `data` parameter is the input data that will be used to populate the datanode.
+        It is passed to the `populate` method of the current object (`self`) along with an optional
+        `device` parameter
+        :param constraint_names: The `constraint_names` parameter is a list of constraint names that you
+        want to verify the results for. If this parameter is not provided or is set to `None`, then the
+        function will verify the results for all constraints available in the `verifyResult` dictionary
+        :param device: The `device` parameter is used to specify the device on which the calculations
+        should be performed. It is an optional parameter and if not provided, the default device will be
+        used
+        :return: None.
+        """
         import numpy as np
         datanode_ac,datanode_t=[],[]
         all_ac, all_t = [], []
@@ -346,7 +502,7 @@ def verifyResultsLC(self,data,constraint_names=None,device=None):
         names=[]
         FIRST=True
         for datanode in self.populate(data, device=device):
-            datanode.inferILPResults()
+            # datanode.inferILPResults()
             verifyResult = datanode.verifyResultsLC()
             if FIRST:
                 if constraint_names is None:
diff --git a/domiknows/program/util_model/transformer_layer.py b/domiknows/program/util_model/transformer_layer.py
new file mode 100644
index 000000000..811059966
--- /dev/null
+++ b/domiknows/program/util_model/transformer_layer.py
@@ -0,0 +1,448 @@
+from transformers.pytorch_utils import apply_chunking_to_forward, find_pruneable_heads_and_indices, prune_linear_layer
+from transformers.activations import ACT2FN
+from transformers.configuration_utils import PretrainedConfig
+import math
+import torch
+import torch.nn as nn
+from typing import List, Optional, Tuple, Union
+
+
+class ILPTransformerConfig(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`BertModel`] or a
+    [`TFBertModel`]. It is used to instantiate a BERT model according to the specified arguments,
+    defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration
+    to that of the BERT [bert-base-uncased](https://huggingface.co/bert-base-uncased) architecture.
+
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model
+    outputs. Read the documentation from [`PretrainedConfig`] for more information.
+
+
+    Args:
+        vocab_size (`int`, *optional*, defaults to 30522):
+            Vocabulary size of the BERT model. Defines the number of different tokens that can be represented by the
+            `inputs_ids` passed when calling [`BertModel`] or
+            [`TFBertModel`].
+        hidden_size (`int`, *optional*, defaults to 768):
+            Dimensionality of the encoder layers and the pooler layer.
+        num_hidden_layers (`int`, *optional*, defaults to 12):
+            Number of hidden layers in the Transformer encoder.
+        num_attention_heads (`int`, *optional*, defaults to 12):
+            Number of attention heads for each attention layer in the Transformer encoder.
+        intermediate_size (`int`, *optional*, defaults to 3072):
+            Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer encoder.
+        hidden_act (`str` or `Callable`, *optional*, defaults to `"gelu"`):
+            The non-linear activation function (function or string) in the encoder and pooler. If string,
+            `"gelu"`, `"relu"`, `"silu"` and `"gelu_new"` are supported.
+        hidden_dropout_prob (`float`, *optional*, defaults to 0.1):
+            The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
+        attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1):
+            The dropout ratio for the attention probabilities.
+        max_position_embeddings (`int`, *optional*, defaults to 512):
+            The maximum sequence length that this model might ever be used with. Typically set this to something large
+            just in case (e.g., 512 or 1024 or 2048).
+        type_vocab_size (`int`, *optional*, defaults to 2):
+            The vocabulary size of the `token_type_ids` passed when calling [`BertModel`] or
+            [`TFBertModel`].
+        initializer_range (`float`, *optional*, defaults to 0.02):
+            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
+        layer_norm_eps (`float`, *optional*, defaults to 1e-12):
+            The epsilon used by the layer normalization layers.
+        position_embedding_type (`str`, *optional*, defaults to `"absolute"`):
+            Type of position embedding. Choose one of `"absolute"`, `"relative_key"`,
+            `"relative_key_query"`. For positional embeddings use `"absolute"`. For more information on
+            `"relative_key"`, please refer to [Self-Attention with Relative Position Representations (Shaw et al.)](https://arxiv.org/abs/1803.02155). For more information on `"relative_key_query"`, please refer to
+            *Method 4* in [Improve Transformer Models with Better Relative Position Embeddings (Huang et al.)](https://arxiv.org/abs/2009.13658).
+        use_cache (`bool`, *optional*, defaults to `True`):
+            Whether or not the model should return the last key/values attentions (not used by all models). Only
+            relevant if `config.is_decoder=True`.
+        classifier_dropout (`float`, *optional*):
+            The dropout ratio for the classification head.
+
+    Examples:
+
+    ```python
+    >>> from transformers import BertModel, BertConfig
+
+    >>> # Initializing a BERT bert-base-uncased style configuration
+    >>> configuration = BertConfig()
+
+    >>> # Initializing a model from the bert-base-uncased style configuration
+    >>> model = BertModel(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+    model_type = "bert"
+
+    def __init__(
+        self,
+        vocab_size=30522,
+        hidden_size=384,
+        num_hidden_layers=1,
+        num_attention_heads=6,
+        intermediate_size=1024,
+        hidden_act="gelu",
+        hidden_dropout_prob=0.1,
+        attention_probs_dropout_prob=0.1,
+        max_position_embeddings=2048,
+        type_vocab_size=2,
+        initializer_range=0.02,
+        layer_norm_eps=1e-12,
+        pad_token_id=0,
+        position_embedding_type="absolute",
+        use_cache=True,
+        classifier_dropout=None,
+        **kwargs
+    ):
+        super().__init__(pad_token_id=pad_token_id, **kwargs)
+
+        self.vocab_size = vocab_size
+        self.hidden_size = hidden_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.hidden_act = hidden_act
+        self.intermediate_size = intermediate_size
+        self.hidden_dropout_prob = hidden_dropout_prob
+        self.attention_probs_dropout_prob = attention_probs_dropout_prob
+        self.max_position_embeddings = max_position_embeddings
+        self.type_vocab_size = type_vocab_size
+        self.initializer_range = initializer_range
+        self.layer_norm_eps = layer_norm_eps
+        self.position_embedding_type = position_embedding_type
+        self.use_cache = use_cache
+        self.classifier_dropout = classifier_dropout
+
+
+class ILPTransformerEmbeddings(nn.Module):
+    """Construct the embeddings from word, position and token_type embeddings."""
+
+    def __init__(self, config):
+        super().__init__()
+        self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id)
+        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
+        self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size)
+
+        # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load
+        # any TensorFlow checkpoint file
+        self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        # position_ids (1, len position emb) is contiguous in memory and exported when serialized
+        self.position_embedding_type = getattr(config, "position_embedding_type", "absolute")
+        self.register_buffer(
+            "position_ids", torch.arange(config.max_position_embeddings).expand((1, -1)), persistent=False
+        )
+        self.register_buffer(
+            "token_type_ids", torch.zeros(self.position_ids.size(), dtype=torch.long), persistent=False
+        )
+
+    def forward(
+        self,
+        input_ids: Optional[torch.LongTensor] = None,
+        token_type_ids: Optional[torch.LongTensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        past_key_values_length: int = 0,
+    ) -> torch.Tensor:
+        if input_ids is not None:
+            input_shape = input_ids.size()
+        else:
+            input_shape = inputs_embeds.size()[:-1]
+
+        seq_length = input_shape[1]
+
+        if position_ids is None:
+            position_ids = self.position_ids[:, past_key_values_length : seq_length + past_key_values_length]
+
+        # Setting the token_type_ids to the registered buffer in constructor where it is all zeros, which usually occurs
+        # when its auto-generated, registered buffer helps users when tracing the model without passing token_type_ids, solves
+        # issue #5664
+        if token_type_ids is None:
+            if hasattr(self, "token_type_ids"):
+                buffered_token_type_ids = self.token_type_ids[:, :seq_length]
+                buffered_token_type_ids_expanded = buffered_token_type_ids.expand(input_shape[0], seq_length)
+                token_type_ids = buffered_token_type_ids_expanded
+            else:
+                token_type_ids = torch.zeros(input_shape, dtype=torch.long, device=self.position_ids.device)
+
+        if inputs_embeds is None:
+            inputs_embeds = self.word_embeddings(input_ids)
+        token_type_embeddings = self.token_type_embeddings(token_type_ids)
+
+        embeddings = inputs_embeds + token_type_embeddings
+        if self.position_embedding_type == "absolute":
+            position_embeddings = self.position_embeddings(position_ids)
+            embeddings += position_embeddings
+        embeddings = self.LayerNorm(embeddings)
+        embeddings = self.dropout(embeddings)
+        return embeddings
+
+
+class ILPTransformerSelfAttention(nn.Module):
+    def __init__(self, config, position_embedding_type=None):
+        super().__init__()
+        if config.hidden_size % config.num_attention_heads != 0 and not hasattr(config, "embedding_size"):
+            raise ValueError(
+                f"The hidden size ({config.hidden_size}) is not a multiple of the number of attention "
+                f"heads ({config.num_attention_heads})"
+            )
+
+        self.num_attention_heads = config.num_attention_heads
+        self.attention_head_size = int(config.hidden_size / config.num_attention_heads)
+        self.all_head_size = self.num_attention_heads * self.attention_head_size
+
+        self.query = nn.Linear(config.hidden_size, self.all_head_size)
+        self.key = nn.Linear(config.hidden_size, self.all_head_size)
+        self.value = nn.Linear(config.hidden_size, self.all_head_size)
+
+        self.dropout = nn.Dropout(config.attention_probs_dropout_prob)
+        self.position_embedding_type = position_embedding_type or getattr(
+            config, "position_embedding_type", "absolute"
+        )
+        if self.position_embedding_type == "relative_key" or self.position_embedding_type == "relative_key_query":
+            self.max_position_embeddings = config.max_position_embeddings
+            self.distance_embedding = nn.Embedding(2 * config.max_position_embeddings - 1, self.attention_head_size)
+
+        self.is_decoder = config.is_decoder
+
+    def transpose_for_scores(self, x: torch.Tensor) -> torch.Tensor:
+        new_x_shape = x.size()[:-1] + (self.num_attention_heads, self.attention_head_size)
+        x = x.view(new_x_shape)
+        return x.permute(0, 2, 1, 3)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        attention_mask: Optional[torch.FloatTensor] = None,
+        head_mask: Optional[torch.FloatTensor] = None,
+        encoder_hidden_states: Optional[torch.FloatTensor] = None,
+        encoder_attention_mask: Optional[torch.FloatTensor] = None,
+        past_key_value: Optional[Tuple[Tuple[torch.FloatTensor]]] = None,
+        output_attentions: Optional[bool] = False,
+    ) -> Tuple[torch.Tensor]:
+        mixed_query_layer = self.query(hidden_states)
+
+        # If this is instantiated as a cross-attention module, the keys
+        # and values come from an encoder; the attention mask needs to be
+        # such that the encoder's padding tokens are not attended to.
+        is_cross_attention = encoder_hidden_states is not None
+
+        if is_cross_attention and past_key_value is not None:
+            # reuse k,v, cross_attentions
+            key_layer = past_key_value[0]
+            value_layer = past_key_value[1]
+            attention_mask = encoder_attention_mask
+        elif is_cross_attention:
+            key_layer = self.transpose_for_scores(self.key(encoder_hidden_states))
+            value_layer = self.transpose_for_scores(self.value(encoder_hidden_states))
+            attention_mask = encoder_attention_mask
+        elif past_key_value is not None:
+            key_layer = self.transpose_for_scores(self.key(hidden_states))
+            value_layer = self.transpose_for_scores(self.value(hidden_states))
+            key_layer = torch.cat([past_key_value[0], key_layer], dim=2)
+            value_layer = torch.cat([past_key_value[1], value_layer], dim=2)
+        else:
+            key_layer = self.transpose_for_scores(self.key(hidden_states))
+            value_layer = self.transpose_for_scores(self.value(hidden_states))
+
+        query_layer = self.transpose_for_scores(mixed_query_layer)
+
+        use_cache = past_key_value is not None
+        if self.is_decoder:
+            # if cross_attention save Tuple(torch.Tensor, torch.Tensor) of all cross attention key/value_states.
+            # Further calls to cross_attention layer can then reuse all cross-attention
+            # key/value_states (first "if" case)
+            # if uni-directional self-attention (decoder) save Tuple(torch.Tensor, torch.Tensor) of
+            # all previous decoder key/value_states. Further calls to uni-directional self-attention
+            # can concat previous decoder key/value_states to current projected key/value_states (third "elif" case)
+            # if encoder bi-directional self-attention `past_key_value` is always `None`
+            past_key_value = (key_layer, value_layer)
+
+        # Take the dot product between "query" and "key" to get the raw attention scores.
+        attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
+
+        if self.position_embedding_type == "relative_key" or self.position_embedding_type == "relative_key_query":
+            query_length, key_length = query_layer.shape[2], key_layer.shape[2]
+            if use_cache:
+                position_ids_l = torch.tensor(key_length - 1, dtype=torch.long, device=hidden_states.device).view(
+                    -1, 1
+                )
+            else:
+                position_ids_l = torch.arange(query_length, dtype=torch.long, device=hidden_states.device).view(-1, 1)
+            position_ids_r = torch.arange(key_length, dtype=torch.long, device=hidden_states.device).view(1, -1)
+            distance = position_ids_l - position_ids_r
+
+            positional_embedding = self.distance_embedding(distance + self.max_position_embeddings - 1)
+            positional_embedding = positional_embedding.to(dtype=query_layer.dtype)  # fp16 compatibility
+
+            if self.position_embedding_type == "relative_key":
+                relative_position_scores = torch.einsum("bhld,lrd->bhlr", query_layer, positional_embedding)
+                attention_scores = attention_scores + relative_position_scores
+            elif self.position_embedding_type == "relative_key_query":
+                relative_position_scores_query = torch.einsum("bhld,lrd->bhlr", query_layer, positional_embedding)
+                relative_position_scores_key = torch.einsum("bhrd,lrd->bhlr", key_layer, positional_embedding)
+                attention_scores = attention_scores + relative_position_scores_query + relative_position_scores_key
+
+        attention_scores = attention_scores / math.sqrt(self.attention_head_size)
+        if attention_mask is not None:
+            # Apply the attention mask is (precomputed for all layers in ILPTransformerModel forward() function)
+            attention_scores = attention_scores + attention_mask
+
+        # Normalize the attention scores to probabilities.
+        attention_probs = nn.functional.softmax(attention_scores, dim=-1)
+
+        # This is actually dropping out entire tokens to attend to, which might
+        # seem a bit unusual, but is taken from the original Transformer paper.
+        attention_probs = self.dropout(attention_probs)
+
+        # Mask heads if we want to
+        if head_mask is not None:
+            attention_probs = attention_probs * head_mask
+
+        context_layer = torch.matmul(attention_probs, value_layer)
+
+        context_layer = context_layer.permute(0, 2, 1, 3).contiguous()
+        new_context_layer_shape = context_layer.size()[:-2] + (self.all_head_size,)
+        context_layer = context_layer.view(new_context_layer_shape)
+
+        outputs = (context_layer, attention_probs) if output_attentions else (context_layer,)
+
+        if self.is_decoder:
+            outputs = outputs + (past_key_value,)
+        return outputs
+
+
+class ILPTransformerSelfOutput(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+
+    def forward(self, hidden_states: torch.Tensor, input_tensor: torch.Tensor) -> torch.Tensor:
+        hidden_states = self.dense(hidden_states)
+        hidden_states = self.dropout(hidden_states)
+        hidden_states = self.LayerNorm(hidden_states + input_tensor)
+        return hidden_states
+
+
+class ILPTransformerAttention(nn.Module):
+    def __init__(self, config, position_embedding_type=None):
+        super().__init__()
+        self.self = ILPTransformerSelfAttention(config, position_embedding_type=position_embedding_type)
+        self.output = ILPTransformerSelfOutput(config)
+        self.pruned_heads = set()
+
+    def prune_heads(self, heads):
+        if len(heads) == 0:
+            return
+        heads, index = find_pruneable_heads_and_indices(
+            heads, self.self.num_attention_heads, self.self.attention_head_size, self.pruned_heads
+        )
+
+        # Prune linear layers
+        self.self.query = prune_linear_layer(self.self.query, index)
+        self.self.key = prune_linear_layer(self.self.key, index)
+        self.self.value = prune_linear_layer(self.self.value, index)
+        self.output.dense = prune_linear_layer(self.output.dense, index, dim=1)
+
+        # Update hyper params and store pruned heads
+        self.self.num_attention_heads = self.self.num_attention_heads - len(heads)
+        self.self.all_head_size = self.self.attention_head_size * self.self.num_attention_heads
+        self.pruned_heads = self.pruned_heads.union(heads)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        attention_mask: Optional[torch.FloatTensor] = None,
+        head_mask: Optional[torch.FloatTensor] = None,
+        encoder_hidden_states: Optional[torch.FloatTensor] = None,
+        encoder_attention_mask: Optional[torch.FloatTensor] = None,
+        past_key_value: Optional[Tuple[Tuple[torch.FloatTensor]]] = None,
+        output_attentions: Optional[bool] = False,
+    ) -> Tuple[torch.Tensor]:
+        self_outputs = self.self(
+            hidden_states,
+            attention_mask,
+            head_mask,
+            encoder_hidden_states,
+            encoder_attention_mask,
+            past_key_value,
+            output_attentions,
+        )
+        attention_output = self.output(self_outputs[0], hidden_states)
+        outputs = (attention_output,) + self_outputs[1:]  # add attentions if we output them
+        return outputs
+
+
+class ILPTransformerIntermediate(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.dense = nn.Linear(config.hidden_size, config.intermediate_size)
+        if isinstance(config.hidden_act, str):
+            self.intermediate_act_fn = ACT2FN[config.hidden_act]
+        else:
+            self.intermediate_act_fn = config.hidden_act
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        hidden_states = self.dense(hidden_states)
+        hidden_states = self.intermediate_act_fn(hidden_states)
+        return hidden_states
+
+
+class ILPTransformerOutput(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.dense = nn.Linear(config.intermediate_size, 2)
+        # self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+
+    def forward(self, hidden_states: torch.Tensor, input_tensor: torch.Tensor) -> torch.Tensor:
+        hidden_states = self.dropout(hidden_states)
+        hidden_states = self.dense(hidden_states)
+        # hidden_states = self.LayerNorm(hidden_states + input_tensor)
+        return hidden_states
+
+
+class ILPTransformerLayer(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.chunk_size_feed_forward = config.chunk_size_feed_forward
+        self.seq_len_dim = 1
+        self.attention = ILPTransformerAttention(config)
+        self.is_decoder = False
+        self.add_cross_attention = config.add_cross_attention
+        self.intermediate = ILPTransformerIntermediate(config)
+        self.output = ILPTransformerOutput(config)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        attention_mask: Optional[torch.FloatTensor] = None,
+        head_mask: Optional[torch.FloatTensor] = None,
+        past_key_value: Optional[Tuple[Tuple[torch.FloatTensor]]] = None,
+        output_attentions: Optional[bool] = False,
+    ) -> Tuple[torch.Tensor]:
+        # decoder uni-directional self-attention cached key/values tuple is at positions 1,2
+        self_attn_past_key_value = past_key_value[:2] if past_key_value is not None else None
+        self_attention_outputs = self.attention(
+            hidden_states,
+            attention_mask,
+            head_mask,
+            output_attentions=output_attentions,
+            past_key_value=self_attn_past_key_value,
+        )
+        attention_output = self_attention_outputs[0]
+
+        outputs = self_attention_outputs[1:]  # add self attentions if we output attention weights
+
+        layer_output = apply_chunking_to_forward(
+            self.feed_forward_chunk, self.chunk_size_feed_forward, self.seq_len_dim, attention_output
+        )
+        outputs = (layer_output,) + outputs
+        return outputs
+
+    def feed_forward_chunk(self, attention_output):
+        intermediate_output = self.intermediate(attention_output)
+        layer_output = self.output(intermediate_output, attention_output)
+        return layer_output
diff --git a/domiknows/sensor/allennlp/__init__.py b/domiknows/sensor/allennlp/__init__.py
deleted file mode 100644
index c9d342890..000000000
--- a/domiknows/sensor/allennlp/__init__.py
+++ /dev/null
@@ -1,1 +0,0 @@
-from .base import AllenNlpSensor, AllenNlpLearner
\ No newline at end of file
diff --git a/domiknows/sensor/allennlp/base.py b/domiknows/sensor/allennlp/base.py
deleted file mode 100644
index 1b18ed92f..000000000
--- a/domiknows/sensor/allennlp/base.py
+++ /dev/null
@@ -1,169 +0,0 @@
-from typing import List, Dict, Tuple, NoReturn, Any
-import torch
-from ...graph import Property
-from .. import Sensor, Learner
-from .module import WrapperModule
-
-
-class AllenNlpSensor(Sensor):
-    def __init__(
-        self,
-        *pres: List[Property],
-        output_only: bool=False
-    ) -> NoReturn:
-        Sensor.__init__(self)
-        self.pres = pres
-        self.output_only = output_only
-        self.pre_dims = []
-        for pre in pres:
-            for sensor in pre.find(AllenNlpSensor):
-                dim = sensor.output_dim
-                self.pre_dims.append(dim)
-                break
-            else:
-                raise RuntimeError('Could not determin input dim for {} from pre-requirement {}'.format(self.fullname, pre.fullname))
-
-    def update_context(
-        self,
-        data_item: Dict[str, Any],
-        force=False
-    ) -> Dict[str, Any]:
-        # update prerequired first
-        for pre in self.pres:
-            if not isinstance(pre, dict):
-                raise RuntimeError('{}'.format(self.fullname))
-            for name, sensor in pre.items():
-                # choose one result to update finally
-                # TODO: consider priority or confidence or merge somehow
-                if isinstance(sensor, AllenNlpSensor) and not sensor.output_only:
-                    sensor(data_item)
-                    break
-            else:  # no break
-                raise RuntimeError('Not able to find a sensor for {} as prereqiured by {}'.format(
-                    pre.fullname, self.fullname))
-
-        # then call forward
-        Sensor.update_context(self, data_item, force)
-
-    @property
-    def output_dim(self):
-        return None
-
-class ReaderSensor(AllenNlpSensor):
-    def __init__(
-        self,
-        reader,
-        key: str,
-        output_only: bool=False
-    ) -> NoReturn:
-        AllenNlpSensor.__init__(self, output_only=output_only) # *pres=[]
-        self.key = key
-        self.reader = reader
-        reader.claim(key, self)
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        return data_item[self.fullname]
-
-
-
-class ModuleSensor(AllenNlpSensor):
-    def __init__(
-        self,
-        *pres: List[Property],
-        output_only: bool=False
-    ) -> NoReturn:
-        AllenNlpSensor.__init__(self, *pres, output_only=output_only)
-        self._module = None
-
-    def create_module(self):
-        raise NotImplementedError('Implemented of create_module is required in subclass of ModuleSensor.')
-
-    @property
-    def module(self):
-        if self._module is None:
-            module = self.create_module()
-            self._module = WrapperModule(module, self.pres)
-        return self._module
-
-    @property
-    def output_dim(self):
-        return (self.module.get_output_dim(),)
-
-
-class AllenNlpLearner(ModuleSensor, Learner):
-    def parameters(self):
-        return self.module.parameters()
-
-
-class PreArgsModuleSensor(ModuleSensor):
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        try:
-            return self.module(*(data_item[pre.fullname] for pre in self.pres))
-        except:
-            print('Error during forward with sensor {}'.format(self.fullname))
-            print('module:', self.module)
-            raise
-        return None
-
-
-class PreArgsModuleLearner(PreArgsModuleSensor, AllenNlpLearner):
-    pass
-
-
-class SinglePreSensor(AllenNlpSensor):
-    def __init__(
-        self,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        AllenNlpSensor.__init__(self, pre, output_only=output_only)
-
-    @property
-    def pre(self):
-        return self.pres[0]
-
-    @property
-    def pre_dim(self):
-        return self.pre_dims[0]
-
-
-class SinglePreLearner(SinglePreSensor, PreArgsModuleLearner):
-    def __init__(
-        self,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        PreArgsModuleLearner.__init__(self, pre, output_only=output_only)
-
-
-class MaskedSensor(AllenNlpSensor):
-    def get_mask(self, data_item: Dict[str, Any]):
-        raise NotImplementedError('Implement get_mask in subclass of MaskedSensor.')
-
-
-class SinglePreMaskedSensor(SinglePreSensor, MaskedSensor):
-    def get_mask(self, data_item: Dict[str, Any]):
-        for sensor in self.pre.find(MaskedSensor):
-            break
-        else:
-            print(self.pre)
-            raise RuntimeError('{} require at least one pre-required sensor to be MaskedSensor.'.format(self.fullname))
-        return sensor.get_mask(data_item)
-
-
-class SinglePreMaskedLearner(SinglePreMaskedSensor, SinglePreLearner):
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        return self.module(data_item[self.pre.fullname], self.get_mask(data_item))
-
-
-class SinglePreArgMaskedPairSensor(PreArgsModuleSensor, SinglePreMaskedSensor):
-    pass
diff --git a/domiknows/sensor/allennlp/learner.py b/domiknows/sensor/allennlp/learner.py
deleted file mode 100644
index f16683f79..000000000
--- a/domiknows/sensor/allennlp/learner.py
+++ /dev/null
@@ -1,217 +0,0 @@
-from typing import List, Dict, Any, Union, NoReturn
-from collections.abc import Iterable
-import numpy as np
-import torch
-from allennlp.modules.token_embedders import Embedding
-from allennlp.modules.text_field_embedders import BasicTextFieldEmbedder
-from torch.nn import Module, Dropout, Sequential, Linear, ZeroPad2d, Conv2d, LogSoftmax, ReLU
-from ...utils import prod
-from ...graph import Property
-from .base import AllenNlpLearner, SinglePreLearner, SinglePreMaskedLearner, MaskedSensor
-from .sensor import SentenceSensor, SinglePreMaskedSensor, SentenceEmbedderSensor
-from .module import DropoutRNN, Permute, Uncap
-
-
-class SentenceEmbedderLearner(SentenceEmbedderSensor, AllenNlpLearner):
-    def create_module(self):
-        self.embedding = Embedding(
-            num_embeddings=0, # later load or extend
-            embedding_dim=self.embedding_dim,
-            pretrained_file=self.pretrained_file,
-            vocab_namespace=self.key,
-            trainable=True,
-        )
-        return BasicTextFieldEmbedder({self.key: self.embedding})
-
-
-class TripletEmbedderLearner(SentenceEmbedderLearner):
-    def create_module(self):
-        embedder = super().create_module()
-        uncapper = Uncap(3, embedder)
-        return uncapper
-
-    @property
-    def output_dim(self):
-        return (self.embedding_dim,)
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        return self.module(data_item[self.fullname], self.get_mask(data_item))
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        en_mask = super().get_mask(data_item)
-        #import pdb; pdb.set_trace()
-        batch, en_len = en_mask.shape
-        un_len = en_len ** (1. / 3)
-        un_len = int(np.round(un_len))
-        # (b, ul, ul, ul)
-        un_mask = torch.zeros_like(en_mask).reshape((batch, un_len, un_len, un_len))
-        # (b,)
-        ul = en_mask.sum(dim=1).float() ** (1. / 3)
-        ul = ul.round().int()
-        for mm, ull in zip(un_mask, ul):
-            mm[:ull,:ull,:ull] = 1
-        return un_mask
-
-class RNNLearner(SinglePreMaskedLearner):
-    def create_module(self):
-        return DropoutRNN(prod(self.pre_dim), self.layers, self.dropout, self.bidirectional)
-
-    def __init__(
-        self,
-        pre: Property,
-        output_only: bool=False,
-        layers: int=1,
-        dropout: float=0.5,
-        bidirectional: bool=True
-    ) -> NoReturn:
-        self.layers = layers
-        self.dropout = dropout
-        self.bidirectional = bidirectional
-        SinglePreMaskedLearner.__init__(self, pre, output_only=output_only)
-
-
-class MLPLearner(SinglePreLearner, SinglePreMaskedSensor):
-    def create_module(self):
-        dims = list(self.dims) # convert or copy
-        if isinstance(self.activation, list):
-            activations = self.activation
-        else:
-            activations = [self.activation,] * len(dims)
-        if isinstance(self.dropout, list):
-            dropouts = self.dropout
-        else:
-            dropouts = [self.dropout,] * len(dims)
-
-        dims.insert(0, prod(self.pre_dim))
-        for i in range(len(dims) - 1):
-            if dims[i + 1] is None:
-                dims[i + 1] = dims[i]
-        layers = []
-        for dim_in, dim_out, activation, dropout in zip(dims[:-1], dims[1:], activations, dropouts):
-            layers.append(Linear(in_features=dim_in, out_features=dim_out))
-            if activation is not None:
-                layers.append(activation)
-            layers.append(Dropout(dropout))
-
-        module = Sequential(*layers)
-        return module
-
-    @property
-    def output_dim(self):
-        dims = list(self.dims) # convert or copy
-        dims.insert(0, prod(self.pre_dim))
-        for i in range(len(dims) - 1):
-            if dims[i + 1] is None:
-                dims[i + 1] = dims[i]
-        return (dims[-1],)
-
-    def __init__(
-        self,
-        dims: List[int],
-        pre: Property,
-        activation: Union[Module, List[Module]]=ReLU(),
-        dropout: Union[float, List[float]]=0.5,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.dims = dims
-        self.activation = activation
-        self.dropout = dropout
-        SinglePreLearner.__init__(self, pre, output_only=output_only)
-
-
-class ConvLearner(MLPLearner):
-    def create_module(self):
-        dims = list(self.dims) # convert or copy
-        if isinstance(self.kernel_size, Iterable):
-            kernel_sizes = self.kernel_size
-        else:
-            kernel_sizes = [self.kernel_size,] * len(dims)
-        if isinstance(self.activation, Iterable):
-            activations = self.activation
-        else:
-            activations = [self.activation,] * len(dims)
-        if isinstance(self.dropout, Iterable):
-            dropouts = self.dropout
-        else:
-            dropouts = [self.dropout,] * len(dims)
-
-        dims.insert(0, prod(self.pre_dim))
-        for i in range(len(dims) - 1):
-            if dims[i + 1] is None:
-                dims[i + 1] = dims[i]
-        layers = []
-        for dim_in, dim_out, kernel_size, activation, dropout in zip(dims[:-1], dims[1:], kernel_sizes, activations, dropouts):
-            layers.append(Permute(0, 3, 1, 2)) # (b,l,l,c) -> (b,c,l,l)
-            padding_t = padding_l = int(np.floor((float(kernel_size) - 1) / 2))
-            padding_b = padding_r = int(np.ceil((float(kernel_size) - 1) / 2))
-            layers.append(ZeroPad2d(padding=(padding_l, padding_r, padding_t, padding_b)))
-            layers.append(Conv2d(in_channels=dim_in, out_channels=dim_out, kernel_size=kernel_size))
-            layers.append(Permute(0, 2, 3, 1)) # (b,c,l,l) -> (b,l,l,c)
-            if activation is not None:
-                layers.append(activation)
-            layers.append(Dropout(dropout))
-
-        module = Sequential(*layers)
-        return module
-
-    def __init__(
-        self,
-        dims: List[int],
-        kernel_size: Union[int, List[int]],
-        pre: Property,
-        activation: Union[Module, List[Module]]=ReLU(),
-        dropout: Union[float, List[float]]=0.5,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.dims = dims
-        self.kernel_size = kernel_size
-        self.activation = activation
-        self.dropout = dropout
-        SinglePreLearner.__init__(self, pre, output_only=output_only)
-
-
-class LogisticRegressionLearner(SinglePreLearner, SinglePreMaskedSensor):
-    def create_module(self):
-        fc = Linear(in_features=prod(self.pre_dim), out_features=2)
-        sm = LogSoftmax(dim=-1)
-        module = Sequential(fc,
-                            #sm,
-                           )
-        return module
-
-    @property
-    def output_dim(self):
-        return (2,)
-
-    def __init__(
-        self,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        SinglePreLearner.__init__(self, pre, output_only=output_only)
-
-
-class SoftmaxLogitLearner(SinglePreLearner, SinglePreMaskedSensor):
-    def create_module(self):
-        fc = Linear(in_features=prod(self.pre_dim), out_features=self.classes)
-        sm = LogSoftmax(dim=-1)
-        module = Sequential(fc,
-                            #sm,
-                           )
-        return module
-
-    @property
-    def update_output_dim(self):
-        return (self.classes,)
-
-    def __init__(
-        self,
-        classes: int,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.classes = classes
-        SinglePreLearner.__init__(self, pre, output_only=output_only)
diff --git a/domiknows/sensor/allennlp/module.py b/domiknows/sensor/allennlp/module.py
deleted file mode 100644
index 0f21ff478..000000000
--- a/domiknows/sensor/allennlp/module.py
+++ /dev/null
@@ -1,468 +0,0 @@
-import numpy as np
-import torch
-from torch.nn import Module, GRU, Dropout
-from allennlp.modules.seq2seq_encoders import PytorchSeq2SeqWrapper
-from ...utils import find_base
-import itertools
-
-
-class WrapperModule(Module):
-    # use a wrapper to keep pre-requireds and avoid side-effect of sequencial or other modules
-    def __init__(self, module, pres):
-        Module.__init__(self)
-        self.main_module = module
-        for pre in pres:
-            from .base import ModuleSensor
-            for sensor in pre.find(ModuleSensor, lambda s: not s.output_only):
-                self.add_module(sensor.fullname, sensor.module)
-
-    def forward(self, *args, **kwargs):
-        return self.main_module(*args, **kwargs)
-
-    def get_output_dim(self):
-        return self.main_module.get_output_dim()
-
-
-class BaseModule(Module):
-    def __init__(self, output_dim):
-        Module.__init__(self)
-        self.output_dim = output_dim
-        self.default_device = None
-
-    def get_output_dim(self):
-        return self.output_dim
-
-
-class DropoutRNN(BaseModule):
-    def __init__(self, embedding_dim, layers=1, dropout=0.5, bidirectional=True):
-        output_dim = embedding_dim
-        if bidirectional:
-            output_dim *= 2
-        BaseModule.__init__(self, output_dim)
-
-        self.embedding_dim = embedding_dim
-        self.bidirectional = bidirectional
-
-        self.rnn = PytorchSeq2SeqWrapper(GRU(embedding_dim,
-                                             embedding_dim,
-                                             num_layers=layers,
-                                             batch_first=True,
-                                             dropout=dropout,
-                                             bidirectional=bidirectional))
-        self.dropout = Dropout(dropout)
-
-    def forward(self, x, mask):
-        return self.dropout(self.rnn(x, mask))
-
-
-class Concat(Module):
-    def forward(self, *x):
-        # TODO: flatten
-        return torch.cat(x, dim=-1)
-
-
-class CartesianProduct(Module):
-    def forward(self, x, y):  # (b,l1,f1) x (b,l2,f2) -> (b, l1, l2, f1+f2)
-        # TODO: flatten
-        xs = x.size()
-        ys = y.size()
-        assert xs[0] == ys[0]
-        # torch cat is not broadcasting, do repeat manually
-        xx = x.view(xs[0], xs[1], 1, xs[2]).repeat(1, 1, ys[1], 1)
-        yy = y.view(ys[0], 1, ys[1], ys[2]).repeat(1, xs[1], 1, 1)
-        return torch.cat([xx, yy], dim=3)
-
-
-class SelfCartesianProduct(CartesianProduct):
-    def forward(self, x):
-        return CartesianProduct.forward(self, x, x)
-
-
-class CartesianProduct3(Module):
-    # (b,l1,f1) x (b,l2,f2) -> (b, l1, l2, l3, f1+f2+f3)
-    def forward(self, x, y, z):
-        # TODO: flatten
-        xs = x.size()
-        ys = y.size()
-        zs = z.size()
-        assert xs[0] == ys[0] and ys[0] == zs[0]
-        # torch cat is not broadcasting, do repeat manually
-        xx = x.view(xs[0], xs[1], 1, 1, xs[2]).repeat(1, 1, ys[1], zs[1], 1)
-        yy = y.view(ys[0], 1, ys[1], 1, ys[2]).repeat(1, xs[1], 1, zs[1], 1)
-        zz = z.view(ys[0], 1, 1, ys[1], zs[2]).repeat(1, xs[1], ys[1], 1, 1)
-        return torch.cat([xx, yy, zz], dim=-1)
-
-
-class SelfCartesianProduct3(CartesianProduct3):
-    def forward(self, x):
-        return CartesianProduct3.forward(self, x, x, x)
-
-
-class JointOldCandidate(Module):
-    def forward(self, *tensors):
-        #import pdb; pdb.set_trace()
-        batch = tensors[0].shape[0]
-        num = len(tensors)
-        # b, l, c
-        logits_sum = tensors[0].clone()
-        # n, b, l, c
-        addends = [logits.clone() for logits in tensors[1:]]
-        for i, logits in enumerate(addends):
-            for tensor in addends[i:]:
-                # b, ..., 1, l, c
-                tensor.unsqueeze_(-3)
-            # b, l, ..., l, c = b, ..., l, 1, c + b, ..., 1, l, c 
-            logits_sum = logits_sum.unsqueeze_(-2) + logits # this are the logits
-        # b, l, ..., l ~ {0, 1}
-        candidate = logits_sum.argmax(-1) # find the largest index along logits dim
-        # remove self pair
-        # b, l, ..., l ~ {0}
-        candidate_2 = torch.zeros_like(candidate)[0]
-        for indices in itertools.product(*(range(d) for d in candidate.shape[1:])):
-            if len(indices) == len(set(indices)):
-                # if no same index
-                candidate_2[indices] = 1
-        # b, l, ..., l ~ {0, 1}
-        candidate_2 = candidate_2.expand(batch, *((-1,) * num))
-        return candidate * candidate_2
-
-
-class JointCandidate(Module):
-    def forward(self, *tensors):
-        #import pdb; pdb.set_trace()
-        batch = tensors[0].shape[0]
-        num = len(tensors)
-        # b, l ~ {0, 1}
-        candidate_sum = tensors[0].clone().argmax(-1)
-        # n-1, b, l ~ {0, 1}
-        addends = [logits.clone().argmax(-1) for logits in tensors[1:]]
-        for i, candidate_tensor in enumerate(addends):
-            for tensor in addends[i:]:
-                # b, ..., 1, l ~ {0, 1}
-                tensor.unsqueeze_(-2)
-            # b, l, ..., l = b, ..., l, 1 + b, ..., 1, l
-            candidate_sum = candidate_sum.unsqueeze_(-1) * candidate_tensor
-        # b, l, ..., l ~ {0, 1}
-        candidate = candidate_sum
-        # remove self pair
-        # b, l, ..., l ~ {0}
-        candidate_2 = torch.zeros_like(candidate)[0]
-        for indices in itertools.product(*(range(d) for d in candidate.shape[1:])):
-            if len(indices) == len(set(indices)):
-                # if no same index
-                candidate_2[indices] = 1
-        # b, l, ..., l ~ {0, 1}
-        candidate_2 = candidate_2.expand(batch, *((-1,) * num))
-        return candidate * candidate_2
-
-
-class NGram(Module):
-    def __init__(self, ngram):
-        Module.__init__(self)
-        self.ngram = ngram
-
-    def forward(self, x):
-        #import pdb; pdb.set_trace()
-        shifted = []
-        size = x.size() # (b, l, c)
-        for i in torch.arange(self.ngram):
-            shifted_x = torch.zeros((size[0], size[1]+self.ngram, size[2]), device=x.device)
-            shifted_x[:, i:i-self.ngram, :] = x
-            shifted.append(shifted_x)
-        new_x = torch.cat(shifted, dim=-1)
-        offset = int((self.ngram-1) / 2)
-        return new_x[:, offset:offset-self.ngram, :]
-
-
-class PairTokenDistance(BaseModule):
-    def __init__(self, emb_num, window):
-        BaseModule.__init__(self, emb_num)
-        self.emb_num = emb_num # must define emb_num (to have lb and ub) before window
-        self.window = window
-
-    @property
-    def window(self):
-        return self._window
-
-    @window.setter
-    def window(self, window):
-        self._window = window
-        ul = np.floor(window / 2)
-        self._base = find_base(ul, self.ub - 1)
-
-    @property
-    def base(self):
-        return self._base
-
-    @property
-    def emb_num(self):
-        return self._emb_num
-
-    @emb_num.setter
-    def emb_num(self, emb_num):
-        self._emb_num = emb_num
-        self._lb = -np.floor((emb_num - 1) / 2)
-        self._ub = np.ceil((emb_num - 1) / 2)
-
-    @property
-    def lb(self):
-        return self._lb
-
-    @property
-    def ub(self):
-        return self._ub
-
-    def forward(self, x):
-        batch = len(x)
-        length = max(len(xx) for xx in x)
-        #(l*2)
-        dist = torch.arange(-length + 1, length, device=self.default_device)
-        rows = []
-        for i in range(length):
-            rows.append(dist[i:i + length].view(1, -1))
-        #(l, l)
-        dist = torch.cat(tuple(reversed(rows)))
-        #print(dist)
-        sign = dist.sign()
-        dist = dist.abs()
-        dist = dist.float()
-        dist = (dist.log() / np.log(self.base) + 1).floor()
-        #print(dist)
-        dist[dist < 0] = 0
-        dist = dist * sign.to(dtype=dist.dtype, device=dist.device)
-        dist[dist < self.lb] = self.lb
-        dist[dist > self.ub] = self.ub
-        dist = dist - self.lb
-        dist = dist.long()
-        #print(dist)
-        #(n, n)
-        eye = torch.eye(self.emb_num, device=dist.device)
-        #import pdb; pdb.set_trace()
-        #(1, l, l, n)
-        dist = eye.index_select(0, dist.view(-1)).view(1, length, length, -1)
-        #(b, l, l, n)
-        dist = dist.repeat(batch, 1, 1, 1)
-        return dist
-
-
-class PairTokenDependencyRelation(BaseModule):
-    def __init__(self):
-        BaseModule.__init__(self, 6)
-
-    def forward(self, x):
-        #import pdb; pdb.set_trace()
-        batch = len(x)
-        length = max(len(xx) for xx in x)
-        #(b,l,l,f)
-        dep = torch.zeros((batch, length, length, self.get_output_dim()), device=self.default_device)
-        for i, span in enumerate(x):
-            for j, token in enumerate(span):
-                # children: immediate syntactic children
-                for r_token in token.children:
-                    dep[i, j, r_token.i, 0] = 1
-                # lefts: leftward immediate children
-                for r_token in token.lefts:
-                    dep[i, j, r_token.i, 1] = 1
-                # rights: rightward immediate children
-                for r_token in token.rights:
-                    dep[i, j, r_token.i, 2] = 1
-                # conjuncts: coordinated tokens
-                for r_token in token.conjuncts:
-                    dep[i, j, r_token.i, 3] = 1
-                # head: syntactic parent, or "governor"
-                dep[i, j, token.head.i, 4] = 1
-                # ancestors: rightmost token of this toke's syntactic descendants
-                for r_token in token.ancestors:
-                    dep[i, j, r_token.i, 5] = 1
-
-        return dep
-
-
-class TripPhraseDistRelation(Module):
-    def forward(self, x):
-        #import pdb; pdb.set_trace()
-        batch, length, feat = x.shape
-        device = x.device
-        dist = torch.zeros(batch, length, length, length, feat * 2, device=device)
-
-        permutations = itertools.permutations(range(0, length), 3)
-        # permutations ensure no repeat index at the same time
-        for lm, tr, sp in permutations:
-            # (batch, lm:sp, feat)
-            lm2sp = x[:, min(lm, sp):max(lm, sp), :]
-            # (batch, tr:sp, feat)
-            tr2sp = x[:, min(tr, sp):max(tr, sp), :]
-            # (batch, feat)
-            dist[:, lm, tr, sp, :] = torch.cat((lm2sp.mean(dim=1), tr2sp.mean(dim=1)), dim=1)
-
-        return dist
-
-
-class LowestCommonAncestor(Module):
-    def forward(self, token_lists, features):
-        #import pdb; pdb.set_trace()
-        batch = features.shape[0]
-        length = features.shape[1]
-        feat = features.shape[2]
-
-        docs = []
-        for token_list in token_lists:
-            doc = None
-            for token in token_list:
-                if doc is None:
-                    doc = token.doc
-                else:
-                    assert doc == token.doc
-            docs.append(doc)
-
-        # (b,lx,lx)
-        lcas = torch.zeros((batch, length, length), device=features.device, dtype=torch.long)
-        for doc, lca in zip(docs, lcas):
-            # (l,l) ~ [-1, l-1]
-            lca_np = doc.get_lca_matrix()
-            # (lx,lx) ~ [0, l]
-            lca[:lca_np.shape[0], :lca_np.shape[0]] = torch.as_tensor(lca_np) + 1
-        # (b, lx+1, f)
-        features = torch.cat((torch.zeros((batch, 1, feat), device=features.device), features), dim=1)
-        # (b, lx*lx)
-        lcas = lcas.view(batch, length * length)
-        lcas_emb = []
-        for feature, lca in zip(features, lcas):
-            # [(1, lx*lx, f)...]
-            lcas_emb.append(feature.index_select(0, lca).view(1, length * length, feat))
-        # (b, lx, lx, f)
-        lcas_emb = torch.cat(lcas_emb, dim=0).view(batch, length, length, feat)
-
-        return lcas_emb
-
-
-class PairTokenDependencyDistance(BaseModule):
-    def __init__(self, emb_num, window):
-        BaseModule.__init__(self, emb_num * 2)
-
-        self.emb_num = emb_num
-        self.window = window
-        #self.emb_dim = 64
-        #self.embedding = torch.nn.Embedding(self.emb_num, self.emb_dim)
-
-    @property
-    def window(self):
-        return self._window
-
-    @window.setter
-    def window(self, window):
-        self._window = window
-        # leave one for zero-distance, making one-distance another unique (b**0===1 <- b!=0)
-        self.base = find_base(window, self.emb_num - 1)
-
-    def forward(self, token_lists):
-        #import pdb; pdb.set_trace()
-        batch = len(token_lists)
-        length = max(len(token_list) for token_list in token_lists)
-
-        docs = []
-        for token_list in token_lists:
-            doc = None
-            for token in token_list:
-                if doc is None:
-                    doc = token.doc
-                else:
-                    assert doc == token.doc
-            docs.append(doc)
-
-        # (b,lx,lx)
-        lcas = torch.zeros((batch, length, length), device=self.default_device, dtype=torch.long)
-
-        for doc, lca in zip(docs, lcas):
-            # (l,l) ~ [-1, l-1]
-            lca_np = doc.get_lca_matrix()
-            # (lx,lx) ~ [-1, l-1]
-            lca[:lca_np.shape[0], :lca_np.shape[0]] = torch.as_tensor(lca_np)
-        lcas = lcas.float()
-        # (b,lx,lx) ~ [0, l-1] U {inf}
-        lcas[lcas==-1] = torch.as_tensor(np.inf)
-        # (lx,)
-        index = torch.arange(length, device=lcas.device, dtype=torch.float)
-        # (b,lx,lx) = (b,lx,lx) - (lx,1) ~ [0, l-1] U {inf}
-        right_dist = (lcas - index).abs()
-
-        # (b,lx,lx) ~ [0, emb_num-2] U {-inf, inf} + 1 ~ [1, emb_num-1] U {-inf, inf}
-        right_dist = (right_dist.log() / np.log(self.base) + 1).floor()
-        # (b,lx,lx) ~ [0, emb_num-1]
-        right_dist[right_dist < 0] = 0
-        right_dist[right_dist > self.emb_num - 1] = self.emb_num - 1
-        right_dist = right_dist.long()
-
-        #left_dist = right_dist.transpose(1, 2)
-
-        # (lx, lx)
-        eye = torch.eye(self.emb_num, device=lcas.device)
-        # (b,lx,lx, emb_num)
-        #left_oh = eye.index_select(0, left_dist.view(-1)).view(batch, length, length, -1)
-        right_oh = eye.index_select(0, right_dist.view(-1)).view(batch, length, length, -1)
-        left_oh = right_oh.transpose(1, 2)
-
-        # (b,lx,lx,emb_dim)
-        #left_emb = self.embedding(left_dist)
-        # (b,lx,lx,emb_dim)
-        #right_emb = self.embedding(right_dist)
-
-        # (b,lx,lx, emb_num*2)
-        dist = torch.cat((left_oh, right_oh), dim=-1)
-        return dist
-
-
-class Permute(Module):
-    def __init__(self, *dims):
-        Module.__init__(self)
-        self.dims = dims
-
-    def forward(self, x):
-        return x.permute(self.dims)
-
-
-class Uncap(Module):
-    def __init__(self, dim, pre_module=None):
-        Module.__init__(self)
-        self.dim = dim
-        self.pre_module = pre_module
-
-    def forward(self, x, mask):
-        if self.pre_module:
-            x = self.pre_module(x)
-
-        #import pdb; pdb.set_trace()
-        batch, en_len, feat = x.shape
-        device = x.device
-        # note that mask is generated to match this module already
-
-        # estimate ul
-        un_len = en_len ** (1. / self.dim)
-        # should always be a natural number if dim is matched as input config
-        # could be some slight different due to numerical stability
-        un_len = int(np.round(un_len))
-
-        dist = torch.zeros_like(x).reshape((batch, *((un_len,) * self.dim), feat))
-        # process each differnt length base on mask
-        for dd, xx, mm in zip(dist, x, mask):
-            # dd - (ul..., f)
-            # xx - (el, f)
-            # mm - (ul...,)
-
-            # ()
-            ull = mm.sum().float() ** (1. / 3)
-            ull = ull.round().int()
-
-            # convert ijk index to 1-d array index
-            def encap_index(*indices):
-                return sum(i * un_len ** j for j, i in enumerate(reversed(indices)))
-
-            # headfirst solution
-            # iter through all and move only the last dim together
-            for indices in itertools.product(range(ull), repeat=self.dim-1):
-                dd[indices] = xx[encap_index(*indices)]
-
-            # more efficient solution:
-            # try torch.unfold or torch.as_strided
-
-        return dist
diff --git a/domiknows/sensor/allennlp/sensor.py b/domiknows/sensor/allennlp/sensor.py
deleted file mode 100644
index 346f35100..000000000
--- a/domiknows/sensor/allennlp/sensor.py
+++ /dev/null
@@ -1,376 +0,0 @@
-from typing import List, Dict, Any, NoReturn
-from collections import OrderedDict
-import torch
-from torch.nn import Module
-from allennlp.modules.token_embedders import Embedding, PretrainedBertEmbedder
-from allennlp.modules.text_field_embedders import BasicTextFieldEmbedder
-from allennlp.nn.util import get_text_field_mask
-from ...graph import Property
-from ...utils import prod, guess_device
-from .base import ReaderSensor, ModuleSensor, SinglePreMaskedSensor, MaskedSensor, PreArgsModuleSensor, SinglePreArgMaskedPairSensor
-from .module import Concat, SelfCartesianProduct, SelfCartesianProduct3, NGram, PairTokenDistance, PairTokenDependencyRelation, PairTokenDependencyDistance, LowestCommonAncestor, TripPhraseDistRelation, JointCandidate
-
-
-class SentenceSensor(ReaderSensor):
-    def __init__(
-        self,
-        reader,
-        key: str,
-        output_only: bool=False
-    ) -> NoReturn:
-        ReaderSensor.__init__(self, reader, key, output_only=output_only) # *pres=[]
-        self.embedders = OrderedDict() # list of SentenceEmbedderLearner
-
-    def add_embedder(self, key, embedder):
-        self.reader.claim(key, embedder)
-        self.embedders[key] = embedder
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        # This sensor it self can do nothing
-        # mayby with self.embedders something more can happen?
-        return None
-
-    @property
-    def output_dim(self):
-        return ()
-
-
-class LabelSensor(ReaderSensor):
-    def __init__(
-        self,
-        reader,
-        key: str,
-        output_only: bool=True
-    ) -> NoReturn:
-        ReaderSensor.__init__(self, reader, key, output_only=output_only)
-
-    @property
-    def output_dim(self):
-        return ()
-
-    
-class LabelMaskSensor(LabelSensor):
-    pass
-
-
-class ConcatSensor(PreArgsModuleSensor, MaskedSensor):
-    def create_module(self):
-        return Concat()
-
-    @property
-    def output_dim(self):
-        output_dim = 0
-        for pre_dim in self.pre_dims:
-            if len(pre_dim) == 0:
-                output_dim += 1
-            else:
-                output_dim += prod(pre_dim) # assume flatten
-        return (output_dim,)
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        for pre in self.pres:
-            for sensor in pre.find(MaskedSensor):
-                return sensor.get_mask(data_item)
-            else:
-                # not found
-                continue
-            # found
-            break
-        else:
-            raise RuntimeError('{} require at least one pre-required sensor to be MaskedSensor.'.format(self.fullname))
-
-        return None # not going to here
-
-
-class CartesianProductSensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return SelfCartesianProduct()
-
-    @property
-    def output_dim(self):
-        if len(self.pre_dim) == 0:
-            output_dim = 2
-        else:
-            output_dim = prod(self.pre_dim) * 2 # assume flatten
-        return (output_dim,)
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        for sensor in self.pre.find(MaskedSensor):
-            break
-        else:
-            print(self.pre)
-            raise RuntimeError('{} require at least one pre-required sensor to be MaskedSensor.'.format(self.fullname))
-
-        mask = sensor.get_mask(data_item).float()
-        ms = mask.size()
-        mask = mask.view(ms[0], ms[1], 1).matmul(
-            mask.view(ms[0], 1, ms[1]))  # (b,l,l)
-        return mask
-
-
-class CandidateSensor(MaskedSensor):
-    pass
-
-class CandidateReaderSensor(ReaderSensor, CandidateSensor):
-    pass
-
-class JointCandidateSensor(PreArgsModuleSensor, CandidateSensor):
-    def create_module(self):
-        return JointCandidate()
-
-    @property
-    def output_dim(self):
-        return ()
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        masks = []
-        for pre in self.pres:
-            for sensor in pre.find(MaskedSensor):
-                break
-            else:
-                print(self.pre)
-                raise RuntimeError('{} require at least one pre-required sensor to be MaskedSensor.'.format(self.fullname))
-            # (b,l)
-            mask = sensor.get_mask(data_item).float()
-            masks.append(mask)
-
-        masks_num = len(masks)
-        mask = masks[0]
-        for i in range(1, masks_num):
-            for j in range(i, masks_num):
-                masks[j].unsqueeze_(-2)
-            mask = mask.unsqueeze_(-1).matmul(masks[i])
-        return mask
-
-
-class CartesianProduct3Sensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return SelfCartesianProduct3()
-
-    @property
-    def output_dim(self):
-        if len(self.pre_dim) == 0:
-            output_dim = 3
-        else:
-            output_dim = prod(self.pre_dim) * 3 # assume flatten
-        return (output_dim,)
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        for sensor in self.pre.find(MaskedSensor):
-            break
-        else:
-            print(self.pre)
-            raise RuntimeError('{} require at least one pre-required sensor to be MaskedSensor.'.format(self.fullname))
-
-        mask = sensor.get_mask(data_item).float()
-        ms = mask.size()
-        #(b,l,l)
-        mask1 = mask.view(ms[0], ms[1], 1).matmul(mask.view(ms[0], 1, ms[1]))
-        mask2 = mask1.view(ms[0], ms[1], ms[1], 1).matmul(mask.view(ms[0], 1, 1, ms[1]))
-        
-        return mask2
-
-class SentenceEmbedderSensor(SinglePreMaskedSensor, ModuleSensor):
-    def create_module(self):
-        self.embedding = Embedding(
-            num_embeddings=0, # later load or extend
-            embedding_dim=self.embedding_dim,
-            pretrained_file=self.pretrained_file,
-            vocab_namespace=self.key,
-            trainable=False,
-        )
-        return BasicTextFieldEmbedder({self.key: self.embedding})
-
-    def __init__(
-        self,
-        key: str,
-        embedding_dim: int,
-        pre,
-        pretrained_file: str=None,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.key = key
-        self.embedding_dim = embedding_dim
-        self.pretrained_file = pretrained_file
-        ModuleSensor.__init__(self, pre, output_only=output_only)
-
-        for pre_sensor in pre.find(SentenceSensor):
-            pre_sensor.add_embedder(key, self)
-            self.tokens_key = pre_sensor.key # used by reader.update_textfield()
-            break
-        else:
-            raise TypeError()
-
-    def update_context(
-        self,
-        data_item: Dict[str, Any],
-        force=False
-    ) -> Dict[str, Any]:
-        if self.fullname in data_item and isinstance(data_item[self.fullname], dict):
-            data_item[self.fullname + '_index'] = data_item[self.fullname] # reserve
-            force = True
-        return SinglePreMaskedSensor.update_context(self, data_item, force)
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        return self.module(data_item[self.fullname])
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        # TODO: make sure update_context has been called
-        return get_text_field_mask(data_item[self.fullname + '_index'])
-
-
-class SentenceBertEmbedderSensor(SentenceEmbedderSensor):
-    def create_module(self):
-        self.embedding = PretrainedBertEmbedder(
-            pretrained_model = self.pretrained_model,
-        )
-        return BasicTextFieldEmbedder({self.key: self.embedding},
-                                      embedder_to_indexer_map={self.key: [
-                                          self.key,
-                                          "{}-offsets".format(self.key),
-                                          "{}-type-ids".format(self.key)]},
-                                      allow_unmatched_keys=True)
-
-    def __init__(
-        self,
-        key: str,
-        pretrained_model: str,
-        pre,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.key = key
-        self.pretrained_model = pretrained_model
-        ModuleSensor.__init__(self, pre, output_only=output_only)
-
-        for pre_sensor in pre.find(SentenceSensor):
-            pre_sensor.add_embedder(key, self)
-            self.tokens_key = pre_sensor.key # used by reader.update_textfield()
-            break
-        else:
-            raise TypeError()
-
-
-class NGramSensor(PreArgsModuleSensor, SinglePreMaskedSensor):
-    def create_module(self):
-        return NGram(self.ngram)
-
-    @property
-    def output_dim(self):
-        #import pdb; pdb.set_trace()
-        return tuple(dim * self.ngram for dim in self.pre_dim)
-
-    def __init__(
-        self,
-        ngram: int,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.ngram = ngram
-        PreArgsModuleSensor.__init__(self, pre, output_only=output_only)
-
-
-class TokenDistantSensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return PairTokenDistance(self.emb_num, self.window)
-
-    def __init__(
-        self,
-        emb_num: int,
-        window: int,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.emb_num = emb_num
-        self.window = window
-        SinglePreArgMaskedPairSensor.__init__(self, pre, output_only=output_only)
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        device, _ = guess_device(data_item).most_common(1)[0]
-        self.module.main_module.default_device = device
-        return super().forward(data_item)
-
-
-class TripPhraseDistSensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return TripPhraseDistRelation()
-
-    @property
-    def output_dim(self):
-        return (self.pre_dim[0] * 2,)
-
-    def get_mask(self, data_item: Dict[str, Any]):
-        for sensor in self.pre.find(MaskedSensor):
-            break
-        else:
-            print(self.pre)
-            raise RuntimeError('{} require at least one pre-required sensor to be MaskedSensor.'.format(self.fullname))
-
-        mask = sensor.get_mask(data_item).float()
-        ms = mask.size()
-        #(b,l,l)
-        mask1 = mask.view(ms[0], ms[1], 1).matmul(mask.view(ms[0], 1, ms[1]))
-        mask2 = mask1.view(ms[0], ms[1], ms[1], 1).matmul(mask.view(ms[0], 1, 1, ms[1]))
-
-        return mask2
-
-
-class TokenDepSensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return PairTokenDependencyRelation()
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        #import pdb; pdb.set_trace()
-        device, _ = guess_device(data_item).most_common(1)[0]
-        self.module.main_module.default_device = device
-        return super().forward(data_item)
-
-
-class TokenLcaSensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return LowestCommonAncestor()
-
-    @property
-    def output_dim(self):
-        return self.pre_dims[1]
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        return PreArgsModuleSensor.forward(self, data_item)
-
-
-class TokenDepDistSensor(SinglePreArgMaskedPairSensor):
-    def create_module(self):
-        return PairTokenDependencyDistance(self.emb_num, self.window)
-
-    def __init__(
-        self,
-        emb_num: int,
-        window: int,
-        pre: Property,
-        output_only: bool=False
-    ) -> NoReturn:
-        self.emb_num = emb_num
-        self.window  = window
-        SinglePreArgMaskedPairSensor.__init__(self, pre, output_only=output_only)
-
-    def forward(
-        self,
-        data_item: Dict[str, Any]
-    ) -> Any:
-        device, _ = guess_device(data_item).most_common(1)[0]
-        self.module.main_module.default_device = device
-        return super().forward(data_item)
diff --git a/domiknows/sensor/pytorch/sensors.py b/domiknows/sensor/pytorch/sensors.py
index 827daa534..393301670 100644
--- a/domiknows/sensor/pytorch/sensors.py
+++ b/domiknows/sensor/pytorch/sensors.py
@@ -7,7 +7,23 @@
 
 
 class TorchSensor(Sensor):
+    """
+    A second main sensor base class that builds on the bare sensor class and updates and propagates context based on the given data item.
+    This class must be inherited and reimplemeted and is not usable.
+    
+    Inherits from:
+    - Sensor: The base class for sensors.
+    """
     def __init__(self, *pres, edges=None, label=False, device='auto'):
+        """
+        Initializes the TorchSensor with the provided parameters.
+
+        Args:
+        - *pres: Variable-length argument list of predecessors.
+        - edges (optional): Edges associated with this sensor.
+        - label (bool, optional): Flag to indicate if this sensor is a label. Defaults to False.
+        - device (str, optional): The device to run torch operations on. It can be 'auto', 'cuda', or 'cpu'. Defaults to 'auto'.
+        """
         super().__init__()
         if not edges:
             edges = []
@@ -27,11 +43,25 @@ def __init__(self, *pres, edges=None, label=False, device='auto'):
 
     def __call__(
         self,
-        data_item: Dict[str, Any]
+        data_item: Dict[str, Any],
+        force=False
     ) -> Any:
+        """
+        Allows instances of this class to be called as functions. Updates the context and returns data for this sensor.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to process.
+        - force (bool, optional): Flag to force recalculation even if result is cached. Default is False.
+
+        Returns:
+        - Any: The data corresponding to this sensor.
+
+        Raises:
+        - Raises any exceptions that might occur during the update_context call.
+        """
         self.context_helper = data_item
         try:
-            self.update_context(data_item)
+            self.update_context(data_item, force=force)
         except Exception as ex:
             print('Error {} during updating data item {} with sensor {}'.format(ex, data_item, self.fullname))
             raise
@@ -41,6 +71,13 @@ def update_context(
         self,
         data_item: Dict[str, Any],
         force=False):
+        """
+        Updates the context of the given data item for this torch sensor. The fucntion that is callaed when __call__ is used.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to update.
+        - force (bool, optional): Flag to force recalculation even if result is cached. Default is False.
+        """
         if not force and self in data_item:
             # data_item cached results by sensor name. override if forced recalc is needed
             val = data_item[self]
@@ -60,6 +97,15 @@ def update_context(
 
     @staticmethod
     def non_label_sensor(sensor):
+        """
+        Checks if the provided sensor is not a label sensor.
+
+        Args:
+        - sensor: The sensor to check.
+
+        Returns:
+        - bool: True if the sensor is not a label sensor, otherwise False.
+        """
         if not isinstance(sensor, Sensor):
             return False
         elif isinstance(sensor, TorchSensor):
@@ -72,6 +118,13 @@ def update_pre_context(
         data_item: Dict[str, Any],
         concept=None
     ) -> Any:
+        """
+        Updates the context for the predecessors of this sensor.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to update context for.
+        - concept (optional): The concept associated with this sensor. Defaults to the sensor's own concept.
+        """
         concept = concept or self.concept
         for edge in self.edges:
             for sensor in edge.find(self.non_label_sensor):
@@ -81,6 +134,20 @@ def update_pre_context(
                 sensor(data_item=data_item)
 
     def fetch_value(self, pre, selector=None, concept=None):
+        """
+        Fetches the value for a predecessor using an optional selector.
+
+        Args:
+        - pre: The predecessor to fetch the value for.
+        - selector (optional): An optional selector to find a specific value.
+        - concept (optional): The concept associated with this sensor. Defaults to the sensor's own concept.
+
+        Returns:
+        - The fetched value for the given predecessor.
+
+        Raises:
+        - Raises KeyError if the provided selector key doesn't exist.
+        """
         concept = concept or self.concept
         if selector:
             try:
@@ -91,28 +158,64 @@ def fetch_value(self, pre, selector=None, concept=None):
             return self.context_helper[concept[pre]]
 
     def define_inputs(self):
+        """
+        Defines the inputs for this sensor based on its predecessors.
+        """
         self.inputs = []
         for pre in self.pres:
             self.inputs.append(self.fetch_value(pre))
 
     def forward(self,) -> Any:
+        """
+        Computes the forward pass for this torch sensor.
+
+        Raises:
+        - NotImplementedError: Indicates that subclasses should provide their implementation.
+        """
         raise NotImplementedError
 
     @property
     def prop(self):
+        """
+        Returns the superior of this sensor. This property is used to get the property associated with the sensor.
+
+        Raises:
+        - ValueError: If the sensor doesn't have a superior.
+        """
         if self.sup is None:
             raise ValueError('{} must be used with with property assignment.'.format(type(self)))
         return self.sup
 
     @property
     def concept(self):
+        """
+        Returns the concept associated with this sensor.
+
+        Raises:
+        - ValueError: If the sensor doesn't have a concept associated with it.
+        """
         if self.prop.sup is None:
             raise ValueError('{} must be used with with concept[property] assignment.'.format(type(self)))
         return self.prop.sup
 
 
 class FunctionalSensor(TorchSensor):
+    """
+    A functional sensor extending the TorchSensor with functionality for forward pass operations making it directly usable.
+
+    Inherits from:
+    - TorchSensor: A base class for torch-based sensors in the graph.
+    """
     def __init__(self, *pres, forward=None, build=True, **kwargs):
+        """
+        Initializes the FunctionalSensor with the provided parameters.
+
+        Args:
+        - *pres: Variable-length argument list of predecessors.
+        - forward (callable, optional): The forward function to use for this sensor. Defaults to None.
+        - build (bool, optional): Flag indicating whether to build the sensor immediately. Defaults to True.
+        - **kwargs: Additional keyword arguments.
+        """
         super().__init__(*pres, **kwargs)
         self.forward_ = forward
         self.build = build
@@ -122,6 +225,13 @@ def update_pre_context(
         data_item: Dict[str, Any],
         concept=None
     ):
+        """
+        Updates the context for the predecessors of this sensor. Extends the behavior to handle more types.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to update context for.
+        - concept (optional): The concept associated with this sensor. Defaults to the sensor's own concept.
+        """
         from ...graph.relation import Transformed, Relation
         concept = concept or self.concept
         for edge in self.edges:
@@ -149,6 +259,14 @@ def update_context(
         data_item: Dict[str, Any],
         force=False,
         override=True):
+        """
+        Updates the context of the given data item for this functional sensor.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to update.
+        - force (bool, optional): Flag to force recalculation even if result is cached. Default is False.
+        - override (bool, optional): Flag to decide if overriding the parent node is allowed. Default is True.
+        """
         if not force and self in data_item:
             # data_item cached results by sensor name. override if forced recalc is needed
             val = data_item[self]
@@ -162,6 +280,17 @@ def update_context(
             data_item[self.prop] = val  # override state under property name
 
     def fetch_value(self, pre, selector=None, concept=None):
+        """
+        Fetches the value for a predecessor using an optional selector. Extends the behavior to handle more types.
+
+        Args:
+        - pre: The predecessor to fetch the value for.
+        - selector (optional): An optional selector to find a specific value.
+        - concept (optional): The concept associated with this sensor. Defaults to the sensor's own concept.
+
+        Returns:
+        - The fetched value for the given predecessor.
+        """
         from ...graph.relation import Transformed, Relation
         concept = concept or self.concept
         if isinstance(pre, (str, Relation)):
@@ -173,12 +302,29 @@ def fetch_value(self, pre, selector=None, concept=None):
         return pre
 
     def forward_wrap(self):
+        """
+        Wraps the forward method ensuring the results are on the appropriate device.
+
+        Returns:
+        - The result of the forward method, moved to the appropriate device if necessary.
+        """
         value = self.forward(*self.inputs)
         if isinstance(value, torch.Tensor) and value.device is not self.device:
             value = value.to(device=self.device)
         return value
 
     def forward(self, *inputs, **kwinputs):
+        """
+        Computes the forward pass for this functional sensor, making use of a provided forward function if available.
+
+        Args:
+        - *inputs: Variable-length argument list of inputs for the forward function.
+        - **kwinputs: Additional keyword inputs for the forward function.
+
+        Returns:
+        - The result of the forward computation.
+        - Calls the superclass forward method if no forward function was provided during initialization.
+        """
         if self.forward_ is not None:
             return self.forward_(*inputs, **kwinputs)
         return super().forward()
@@ -219,16 +365,43 @@ def forward(self, *args, **kwargs) -> Any:
 
 
 class JointSensor(FunctionalSensor):
+    """
+    Represents a joint sensor that generates multiple properties.
+
+    Inherits from:
+    - FunctionalSensor: A sensor with defined forward functionality.
+    """
     def __init__(self, *args, bundle_call=False, **kwargs):
+        """
+        Initializes the JointSensor with the provided parameters.
+
+        Args:
+        - *args: Variable-length argument list.
+        - bundle_call (bool, optional): Indicates if component sensors should be called when this sensor is called.
+                                        Defaults to False.
+        - **kwargs: Additional keyword arguments.
+        """
         super().__init__(*args, **kwargs)
         self._components = None
         self.bundle_call = bundle_call
 
     @property
     def components(self):
+        """
+        Returns the list of component sensors associated with this joint sensor.
+
+        Returns:
+        - List of component sensors.
+        """
         return self._components
 
     def attached(self, sup):
+        """
+        Configures the joint sensor when attached to a parent node.
+
+        Args:
+        - sup: The parent node to which this sensor is attached.
+        """
         from ...graph.relation import Relation
         from .relation_sensors import EdgeSensor
         super().attached(sup)
@@ -245,6 +418,12 @@ def attached(self, sup):
                 self.components.append(sensor)
 
     def __iter__(self):
+        """
+        Returns an iterator over the component sensors. Builds the component sensors if they haven't been built yet.
+
+        Yields:
+        - Each individual component sensor.
+        """
         self.build = False
         if self.components is None:
             self._components = []
@@ -257,6 +436,16 @@ def __iter__(self):
             yield from self.components
 
     def __call__(self, *args, **kwargs):
+        """
+        Calls the joint sensor, potentially also invoking its component sensors if `bundle_call` is set to True.
+
+        Args:
+        - *args: Variable-length argument list.
+        - **kwargs: Additional keyword arguments.
+
+        Returns:
+        - The value from the main sensor's call.
+        """
         value = super().__call__(*args, **kwargs)
         if self.bundle_call and self.components is not None:
             for sensor in self.components:
@@ -268,6 +457,14 @@ def update_context(
         data_item: Dict[str, Any],
         force=False,
         override=True):
+        """
+        Updates the context of the given data item for this joint sensor.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to update.
+        - force (bool, optional): Flag to force recalculation even if result is cached. Default is False.
+        - override (bool, optional): Flag to decide if overriding of the parent data is allowed. Default is True.
+        """
         super().update_context(data_item, force=force, override=override and self.components is None)
 
 
diff --git a/domiknows/sensor/sensor.py b/domiknows/sensor/sensor.py
index d28e3bf38..9e53c954a 100644
--- a/domiknows/sensor/sensor.py
+++ b/domiknows/sensor/sensor.py
@@ -4,11 +4,31 @@
 
 
 class Sensor(BaseGraphTreeNode):
+    """
+    Represents the bare parent sensor that can update and propagate context of a datanode based on the given data and create new properties.
+
+    Inherits from:
+    - BaseGraphTreeNode: A parent node class that provides basic graph functionalities.
+    """
     def __call__(
         self,
         data_item: Dict[str, Any],
         force=False
     ) -> Any:
+        """
+        Allows instances of this class to be called as functions. Updates context and returns the data 
+        corresponding to this sensor from the given data in the context of the datanode.
+
+        Args:
+        - data_item (Dict[str, Any]): The data of the given datanode.
+        - force (bool, optional): Flag to force recalculation even if result is cached. Default is False.
+
+        Returns:
+        - Any: The updated data corresponding to this sensor.
+
+        Raises:
+        - Raises any exceptions that might occur during the update_context call.
+        """
         try:
             self.update_context(data_item, force)
         except:
@@ -21,6 +41,14 @@ def update_context(
         data_item: Dict[str, Any],
         force=False
     ):
+        """
+        Updates the context of the given data item based on this sensor. If forced, or if the result isn't cached,
+        it computes the forward pass and caches the result.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to update.
+        - force (bool, optional): Flag to force recalculation even if result is cached. Default is False.
+        """
         if not force and (self in data_item):
             # data_item cached results by sensor name. override if forced recalc is needed
             val = data_item[self]
@@ -30,6 +58,15 @@ def update_context(
         self.propagate_context(data_item, self, force)
 
     def propagate_context(self, data_item, node, force=False):
+        """
+        Propagates the context from this sensor to the given node's superior, if needed. It ensures the data is consistent 
+        and updated throughout the graph.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to propagate context through.
+        - node (BaseGraphTreeNode): The node to propagate context to.
+        - force (bool, optional): Flag to force propagation even if the result is cached. Default is False.
+        """
         if node.sup is not None and (node.sup not in data_item or force):
             data_item[node.sup] = data_item[self]
             self.propagate_context(data_item, node.sup, force)
@@ -38,4 +75,13 @@ def forward(
         self,
         data_item: Dict[str, Any]
     ) -> Any:
+        """
+        Computes the forward pass for the given data item. This method should be implemented by subclasses. This function defines how to calcualte the new properties based on the current data.
+
+        Args:
+        - data_item (Dict[str, Any]): The data dictionary to compute the forward pass for.
+
+        Raises:
+        - NotImplementedError: Indicates that subclasses should provide their implementation.
+        """
         raise NotImplementedError
diff --git a/domiknows/sensor/torch/__init__.py b/domiknows/sensor/torch/__init__.py
deleted file mode 100644
index e69de29bb..000000000
diff --git a/domiknows/sensor/torch/learner.py b/domiknows/sensor/torch/learner.py
deleted file mode 100644
index e691446a8..000000000
--- a/domiknows/sensor/torch/learner.py
+++ /dev/null
@@ -1,94 +0,0 @@
-import torch
-
-from .. import Learner
-from .sensor import TorchSensor, FunctionalSensor
-
-
-class TorchLearner(TorchSensor, Learner):
-    pass
-
-class FunctionalLearner(FunctionalSensor, TorchLearner):
-    pass
-
-
-class ModuleLearner(FunctionalLearner):
-    @staticmethod
-    def Module(**kwargs):
-        pass
-
-    def __init__(self, pre, target=False, module=None, **kwargs):
-        super().__init__(pre, target=target)
-        self.module = module or self.Module(**kwargs)
-
-    def forward_func(self, input):
-        return self.module(input)
-
-    def parameters(self):
-        return self.module.parameters()
-
-
-class EmbedderLearner(ModuleLearner):
-    Module = torch.nn.Embedding
-
-    def mask(self, data_item):
-        input = next(self.get_args(data_item, sensor_filter=lambda s: not s.target))
-        if self.module.padding_idx is not None:
-            mask = input.clone().detach() != self.module.padding_idx
-        else:
-            mask = torch.ones_like(input)
-        return mask
-
-class NorminalEmbedderLearner(EmbedderLearner):
-    def __init__(self, pre, vocab_key=None, num_embeddings=None, target=False, module=None, **kwargs):
-        self.vocab_key = vocab_key
-        self.modele_args = kwargs
-        if num_embeddings is None:
-            super().__init__(pre, num_embeddings=1, **kwargs, target=target)
-        else:
-            super().__init__(pre, num_embeddings=num_embeddings, **kwargs, target=target)
-
-    def update_module(self, num_embeddings):
-        self.module = self.Module(num_embeddings=num_embeddings, **self.modele_args)
-
-    def mask(self, data_item):
-        input = next(self.get_args(data_item, sensor_filter=lambda s: not s.target))
-        if self.module.padding_idx is not None:
-            mask = input.clone().detach() != self.module.padding_idx
-        else:
-            mask = torch.ones_like(input)
-        return mask
-
-    def forward_func(self, input):
-        indexes, tokens = input
-        if isinstance(indexes, list):
-            max_len = max(map(len, indexes))
-            def pad(output):
-                return torch.cat((
-                    torch.tensor(output, dtype=torch.long), 
-                    torch.zeros(max_len - len(output), dtype=torch.long)
-                    ))
-            indexes = torch.stack(tuple(map(pad, indexes)))
-        device = next(self.parameters()).device
-        indexes = indexes.to(device=device)
-        return super().forward_func(indexes)
-
-class RNNLearner(ModuleLearner):
-    Module=torch.nn.LSTM
-
-    def forward_func(self, input):
-        output, _ = super().forward_func(input)
-        return output
-
-
-class MLPLearner(ModuleLearner):
-    Module = torch.nn.Linear
-
-
-class LRLearner(ModuleLearner):
-    @staticmethod
-    def Module(**kwargs):
-        kwargs['out_features'] = 1
-        return torch.nn.Linear(**kwargs)
-
-    def forward_func(self, input):
-        return super().forward_func(input).squeeze(-1)
diff --git a/domiknows/sensor/torch/sensor.py b/domiknows/sensor/torch/sensor.py
deleted file mode 100644
index e99b4ca8b..000000000
--- a/domiknows/sensor/torch/sensor.py
+++ /dev/null
@@ -1,201 +0,0 @@
-import abc
-from typing import Any, Dict
-from itertools import chain
-from domiknows.sensor.sensor import Sensor
-from domiknows.graph.property import Property
-from domiknows.utils import consume
-
-import torch
-
-
-class SkipSensor(Exception):
-    pass
-
-
-class TorchSensor(Sensor):
-    def __init__(self, target=False):
-        super().__init__()
-        self.target = target
-
-    def propagate_context(self, data_item, node, force=False):
-        if not self.target:  # avoid target to be mixed in forward calculation
-            super().propagate_context(data_item, node, force)
-
-    def mask(self, data_item: Dict[str, Any]) -> Any:
-        # allow safely skip mask
-        raise SkipSensor
-
-
-class DataSensor(TorchSensor):
-    def __init__(self, key, target=False):
-        super().__init__(target=target)
-        self.key = key
-
-    def forward(self, data_item):
-        return data_item[self.key]
-
-
-class LabelSensor(DataSensor):
-    def __init__(self, key):
-        super().__init__(key, target=True)
-
-
-class FunctionalSensor(TorchSensor):
-    def __init__(self, *pres, target=False):
-        super().__init__(target)
-        self.pres = pres
-
-    def get_args(self, data_item, skip_none_prop=False, sensor_fn=None, sensor_filter=None):
-        for pre in self.pres:
-            if isinstance(pre, Property):
-                for _, sensor in pre.items():
-                    if sensor_filter and not sensor_filter(sensor):
-                        continue
-                    try:
-                        if sensor_fn:
-                            yield sensor_fn(sensor, data_item)
-                        else:
-                            yield sensor(data_item)
-                        break
-                    except SkipSensor:
-                        pass
-                else:  # no break
-                    raise RuntimeError('Not able to find a sensor for {} as prereqiured by {}'.format(
-                        pre.fullname, self.fullname))
-            else:
-                if not skip_none_prop:
-                    yield pre
-
-    @abc.abstractmethod
-    def forward_func(self, *args, **kwargs):
-        raise NotImplementedError
-
-    def forward(
-        self,
-        data_item: Dict[str, Any],
-    ) -> Dict[str, Any]:
-        args = self.get_args(data_item, sensor_filter=lambda s: not s.target)
-        return self.forward_func(*args)
-
-    def mask(self, data_item):
-        masks = list(self.get_args(data_item, sensor_fn=lambda s, c: s.mask(c)))
-        masks_num = len(masks)
-        mask = masks[0]
-        mask = mask.float()
-        for i in range(1, masks_num):
-            for j in range(i, masks_num):
-                masks[j].unsqueeze_(-2)
-            masks[i] = masks[i].float()
-            mask = mask.unsqueeze_(-1).matmul(masks[i])
-        return mask
-
-
-class CartesianSensor(FunctionalSensor):
-    def forward_func(self, *inputs):
-        # torch cat is not broadcasting, do repeat manually
-        input_iter = iter(inputs)
-        output = next(input_iter)
-        for input in input_iter:
-            dob, *dol, dof = output.shape
-            dib, *dil, dif = input.shape
-            assert dob == dib
-            output = output.view(dob, *dol, *(1,)*len(dil), dof).repeat(1, *(1,)*len(dol), *dil, 1)
-            input = input.view(dib, *(1,)*len(dol), *dil, dif).repeat(1, *dol, *(1,)*len(dil), 1)
-            output = torch.cat((output, input), dim=-1)
-        return output
-
-
-from collections import OrderedDict, Counter
-
-class NorminalSensor(FunctionalSensor):
-    UNK = '__UNK__'
-    SPECIAL_TOKENS = [UNK]
-
-    def __init__(self, *pres, vocab_counter=None, least_count=0, special_tokens=[], target=False):
-        super().__init__(*pres, target=target)
-        self.least_count = least_count
-        self.special_tokens = self.SPECIAL_TOKENS.copy()
-        self.special_tokens.extend(special_tokens)
-        self.fixed = False
-        self._vocab = None
-        self.set_vocab(vocab_counter)
-
-    def set_vocab(self, vocab_counter=None):
-        self.vocab_counter = vocab_counter or Counter()
-        self.update_vocab(fixed=True)
-
-    def update_counter(self, outputs):
-        comsum(self.vocab_counter.update, chain(*outputs))
-
-    def update_vocab(self, fixed=None):
-        if self.fixed:
-            raise RuntimeError('Vocab of {} has been fixed!'.format(self))
-        vocab_list = self.special_tokens.copy()
-        vocab_count = list(filter(lambda kv: kv[1] > self.least_count, self.vocab_counter))
-        if vocab_count:
-            vocab_to_add, _ = zip(*vocab_count)
-        else:
-            vocab_to_add = []
-        vocab_list.extend(vocab_to_add)
-        self._vocab = OrderedDict((token, idx) for idx, token in enumerate(vocab_list))
-        if fixed is not None:
-            self.fixed = fixed
-
-    def encode(self, output):
-        return self._vocab.get(output, self._vocab[self.UNK])
-
-    def forward(self, data_item):
-        raw_outputs = super().forward(data_item)
-        if not self.fixed:
-            self.update_counter(raw_outputs)
-        encoded_outputs = [list(map(self.encode, raw_output)) for raw_output in raw_outputs]
-        max_len = max(map(len, encoded_outputs))
-        def pad(output):
-            return torch.cat((
-                torch.tensor(output, dtype=torch.long), 
-                torch.zeros(max_len - len(output), dtype=torch.long)
-                ))
-        encoded_tensor = torch.stack(tuple(map(pad, encoded_outputs)))
-        return encoded_tensor, raw_outputs
-
-class SpacyTokenizorSensor(NorminalSensor):
-    from spacy.lang.en import English
-    nlp = English()
-
-    def forward_func(self, sentences):
-        tokens = self.nlp.tokenizer.pipe(sentences)
-        return list(tokens)
-
-
-TRANSFORMER_MODEL = 'bert-base-uncased'
-
-class BertTokenizorSensor(FunctionalSensor):
-    from transformers import BertTokenizer
-    tokenizer = BertTokenizer.from_pretrained(TRANSFORMER_MODEL)
-
-    def forward_func(self, sentences):
-        tokens = self.tokenizer.batch_encode_plus(
-            sentences,
-            return_tensors='pt',
-            return_attention_mask=True,
-            #return_offsets_mapping=True,
-        )
-        tokens['tokens'] = self.tokenizer.convert_ids_to_tokens(tokens['input_ids'], skip_special_tokens=True)
-        return tokens
-
-class BertEmbeddingSensor(FunctionalSensor):
-    from transformers import BertModel
-    model = BertModel.from_pretrained(TRANSFORMER_MODEL)
-
-    # {
-    #     tokens: list[List[str]], sting based tokens
-    #     input_ids: list[List[int]],
-    #     -token_type_ids: list[List[int]] if return_token_type_ids is True (default)
-    #     attention_mask: list[List[int]] if return_attention_mask is True (default)
-    #     -overflowing_tokens: list[List[int]] if a ``max_length`` is specified and return_overflowing_tokens is True
-    #     -num_truncated_tokens: List[int] if a ``max_length`` is specified and return_overflowing_tokens is True
-    #     -special_tokens_mask: list[List[int]] if ``add_special_tokens`` if set to ``True`` and return_special_tokens_mask is True
-    # }
-    def forward_func(self, tokens):
-        outputs = self.model(tokens['input_ids'])
-        return outputs
diff --git a/domiknows/solver/.gitignore b/domiknows/solver/.gitignore
new file mode 100644
index 000000000..6d9594aa4
--- /dev/null
+++ b/domiknows/solver/.gitignore
@@ -0,0 +1,1 @@
+/logs/
diff --git a/domiknows/solver/allennlpInferenceSolver.py b/domiknows/solver/allennlpInferenceSolver.py
deleted file mode 100644
index 3e8cdccb3..000000000
--- a/domiknows/solver/allennlpInferenceSolver.py
+++ /dev/null
@@ -1,153 +0,0 @@
-import abc
-from typing import Dict
-from collections import defaultdict
-import numpy as np
-import torch
-from ..graph import Graph
-from ..sensor.allennlp.sensor import SentenceEmbedderSensor, SentenceSensor
-from ..sensor.allennlp.base import AllenNlpLearner
-from ..utils import get_prop_result
-from .ilpOntSolver import ilpOntSolver
-
-
-DataInstance = Dict[str, torch.Tensor]
-
-
-class AllennlpInferenceSolver(ilpOntSolver):
-    __metaclass__ = abc.ABCMeta
-
-    def inferSelection(self, graph: Graph, data: DataInstance) -> DataInstance:
-        # build concept (property) group by rank (# of has-a)
-        prop_dict = defaultdict(list)
-
-        def concept_rank(concept):
-            # get inheritance rank
-            in_rank = 0
-            for is_a in concept.is_a():
-                in_rank += concept_rank(is_a.dst)
-            # get self rank
-            rank = len(concept.has_a())
-            # TODO: it would be better to have new syntax to support override
-            # determine override condition
-            if in_rank > rank:
-                rank = in_rank
-            return rank or 1
-
-        for prop in graph.poi:
-            concept = prop.sup
-            prop_dict[concept_rank(concept)].append(prop)
-
-        if prop_dict:
-            max_rank = max(prop_dict.keys())
-        else:
-            max_rank = 0
-
-        # find base, assume only one base for now
-        # FIXME: that [0] means we are only considering problems with only one sentence
-        sentence_sensor = graph.get_sensors(SentenceSensor)[0]
-        # Note: SentenceEmbedderSensor is not reliable. For example BERT indexer will introduce redundant wordpiece tokens
-
-        sentence = data[sentence_sensor.fullname]
-        batch_size = len(sentence)
-        mask_len = [len(s) for s in sentence]  # (b, )
-        length = max(mask_len)
-        device = None
-
-        values = [defaultdict(dict) for _ in range(batch_size)]
-        for rank, props in prop_dict.items():
-            for prop in props:
-                #name = '{}_{}'.format(prop.sup.name, prop.name)
-                #name = prop.name  # current implementation has concept name in prop name
-                name = prop.sup.name # we need concept name to match that in OWL
-                # TODO: what if a concept has several properties to be predict?
-                # pred_logit - (b, l...*r, c) - for dim-c, [0] is neg, [1] is pos
-                # mask - (b, l...*r)
-                label, pred_logit, mask = get_prop_result(prop, data)
-                if not device:
-                    device = pred_logit.device
-                # pred - (b, l...*r, c)
-                pred = torch.nn.functional.softmax(pred_logit, dim=-1)
-                pos_index = torch.tensor(1, device=device).long()
-                # (b, l...*r)
-                batched_value = pred.index_select(-1, pos_index).squeeze(dim=-1)
-                # copy and detach, time consuming I/O
-                batched_value = batched_value.clone().cpu().detach().numpy()
-
-                for batch_index, batch_index_d in zip(range(batch_size), torch.arange(batch_size, dtype=torch.long, device=device)):
-                    # (l...*r)
-                    value = batched_value[batch_index_d]
-                    # apply mask
-                    # (l'...*r)
-                    value = value[(slice(0, mask_len[batch_index]),) * rank]
-                    values[batch_index][rank][name] = value
-        #import pdb; pdb.set_trace()
-
-        results = []
-        # inference per instance
-        for batch_index in range(batch_size):
-            # prepare tokens
-            tokens = ['{}_{}'.format(i, token)
-                      for i, token in enumerate(sentence[batch_index])]
-            # prepare tables
-            table_list = []
-            for rank in range(1, max_rank + 1):
-                if rank in values[batch_index]:
-                    table_list.append(values[batch_index][rank])
-                else:
-                    table_list.append(None)
-            #import pdb; pdb.set_trace()
-            # Do inference
-            try:
-                # following statement should be equivalent to
-                # - EMR:
-                # result_list = self.calculateILPSelection(tokens, concept_dict, relation_dict)
-                # - SPRL:
-                # result_list = self.calculateILPSelection(tokens, concept_dict, None, triplet_dict)
-                result_table_list = self.calculateILPSelection(tokens, *table_list)
-
-                #import pdb; pdb.set_trace()
-                if all([result_table is None for result_table in result_table_list]):
-                    raise RuntimeError('No result from solver. Check any log from the solver.')
-            except:
-                # whatever, raise it
-                raise
-
-            # collect result in batch
-            result = defaultdict(dict)
-            for rank, props in prop_dict.items():
-                for prop in props:
-                    name = prop.sup.name
-                    # (l'...*r)
-                    # return structure is a pd?
-                    # result_table_list started from 0
-                    result[rank][name] = result_table_list[rank - 1][name]
-            results.append(result)
-        #import pdb; pdb.set_trace()
-
-        # put results back
-        for rank, props in prop_dict.items():
-            for prop in props:
-                name = prop.sup.name
-                instance_value_list = []
-                for batch_index in range(batch_size):
-                    # (l'...*r)
-                    instance_value = results[batch_index][rank][name]
-                    # (l...*r)
-                    instance_value_pad = np.empty([length, ] * rank)
-                    instance_value_pad[(slice(0, mask_len[batch_index]),) * rank] = instance_value
-                    # (l...*r)
-                    instance_value_d = torch.tensor(instance_value_pad, device=device)
-                    instance_value_list.append(instance_value_d)
-                # (b, l...*r)
-                batched_value = torch.stack(instance_value_list, dim=0)
-                # (b, l...*r, 2)
-                batched_value = torch.stack([1 - batched_value, batched_value], dim=-1)
-                # undo softmax
-                logits_value = torch.log(batched_value / (1 - batched_value))  # Go to +- inf
-                # Put it back finally
-                #import pdb; pdb.set_trace()
-                data[prop.fullname] = logits_value
-                for learner in prop.find(AllenNlpLearner):
-                    data[learner.fullname] = logits_value
-
-        return data
diff --git a/domiknows/solver/allennlplogInferenceSolver.py b/domiknows/solver/allennlplogInferenceSolver.py
deleted file mode 100644
index 7d5b829e3..000000000
--- a/domiknows/solver/allennlplogInferenceSolver.py
+++ /dev/null
@@ -1,151 +0,0 @@
-import abc
-from typing import Dict
-from collections import defaultdict
-import numpy as np
-import torch
-from ..graph import Graph
-from ..sensor.allennlp.sensor import SentenceEmbedderSensor, SentenceSensor
-from ..sensor.allennlp.base import AllenNlpLearner
-from ..utils import get_prop_result
-from .ilpOntSolver import ilpOntSolver
-
-
-DataInstance = Dict[str, torch.Tensor]
-
-
-class AllennlplogInferenceSolver(ilpOntSolver):
-    __metaclass__ = abc.ABCMeta
-
-    def inferSelection(self, graph: Graph, data: DataInstance) -> DataInstance:
-        # build concept (property) group by rank (# of has-a)
-        prop_dict = defaultdict(list)
-
-        def concept_rank(concept):
-            # get inheritance rank
-            in_rank = 0
-            for is_a in concept.is_a():
-                in_rank += concept_rank(is_a.dst)
-            # get self rank
-            rank = len(concept.has_a())
-            # TODO: it would be better to have new syntax to support override
-            # determine override condition
-            if in_rank > rank:
-                rank = in_rank
-            return rank or 1
-
-        for prop in graph.poi:
-            concept = prop.sup
-            prop_dict[concept_rank(concept)].append(prop)
-
-        if prop_dict:
-            max_rank = max(prop_dict.keys())
-        else:
-            max_rank = 0
-
-        # find base, assume only one base for now
-        # FIXME: that [0] means we are only considering problems with only one sentence
-        sentence_sensor = graph.get_sensors(SentenceSensor)[0]
-        # Note: SentenceEmbedderSensor is not reliable. For example BERT indexer will introduce redundant wordpiece tokens
-
-        sentence = data[sentence_sensor.fullname]
-        batch_size = len(sentence)
-        mask_len = [len(s) for s in sentence]  # (b, )
-        length = max(mask_len)
-        device = None
-
-        values = [defaultdict(dict) for _ in range(batch_size)]
-        for rank, props in prop_dict.items():
-            for prop in props:
-                #name = '{}_{}'.format(prop.sup.name, prop.name)
-                #name = prop.name  # current implementation has concept name in prop name
-                name = prop.sup.name # we need concept name to match that in OWL
-                # TODO: what if a concept has several properties to be predict?
-                # pred_logit - (b, l...*r, c) - for dim-c, [0] is neg, [1] is pos
-                # mask - (b, l...*r)
-                label, pred_logit, mask = get_prop_result(prop, data)
-                if not device:
-                    device = pred_logit.device
-                # pred - (b, l...*r, c)
-                pred = torch.nn.functional.log_softmax(pred_logit, dim=-1)
-                # copy and detach, time consuming I/O
-                # batched_value - (b, l...*r, c)
-                batched_value = pred.clone().cpu().detach().numpy()
-
-                for batch_index in range(batch_size):
-                    # (l...*r, c)
-                    value = batched_value[batch_index]
-                    # apply mask
-                    # (l'...*r, c)
-                    value = value[(slice(0, mask_len[batch_index]),) * rank]
-                    values[batch_index][rank][name] = value
-        #import pdb; pdb.set_trace()
-
-        results = []
-        # inference per instance
-        for batch_index in range(batch_size):
-            # prepare tokens
-            tokens = ['{}_{}'.format(i, token)
-                      for i, token in enumerate(sentence[batch_index])]
-            # prepare tables
-            table_list = []
-            for rank in range(1, max_rank + 1):
-                if rank in values[batch_index]:
-                    table_list.append(values[batch_index][rank])
-                else:
-                    table_list.append(None)
-            #import pdb; pdb.set_trace()
-            # Do inference
-            try:
-                # following statement should be equivalent to
-                # - EMR:
-                # result_list = self.calculateILPSelection(tokens, concept_dict, relation_dict)
-                # - SPRL:
-                # result_list = self.calculateILPSelection(tokens, concept_dict, None, triplet_dict)
-                result_table_list = self.calculateILPSelection(tokens, *table_list)
-
-                #import pdb; pdb.set_trace()
-                if all([result_table is None for result_table in result_table_list]):
-                    raise RuntimeError('No result from solver. Check any log from the solver.')
-            except:
-                # whatever, raise it
-                raise
-
-            # collect result in batch
-            result = defaultdict(dict)
-            for rank, props in prop_dict.items():
-                for prop in props:
-                    name = prop.sup.name
-                    # (l'...*r)
-                    # return structure is a pd?
-                    # result_table_list started from 0
-                    result[rank][name] = result_table_list[rank - 1][name]
-            results.append(result)
-        #import pdb; pdb.set_trace()
-
-        # put results back
-        for rank, props in prop_dict.items():
-            for prop in props:
-                name = prop.sup.name
-                instance_value_list = []
-                for batch_index in range(batch_size):
-                    # (l'...*r)
-                    instance_value = results[batch_index][rank][name]
-                    # (l...*r)
-                    instance_value_pad = np.empty([length, ] * rank)
-                    instance_value_pad[(slice(0, mask_len[batch_index]),) * rank] = instance_value
-                    # (l...*r)
-                    instance_value_d = torch.tensor(instance_value_pad, device=device)
-                    instance_value_list.append(instance_value_d)
-                # (b, l...*r)
-                batched_value = torch.stack(instance_value_list, dim=0)
-                # (b, l...*r, 2)
-                batched_value = torch.stack([1 - batched_value, batched_value], dim=-1)
-                # undo softmax
-                logits_value = torch.log(batched_value / (1 - batched_value))  # Go to +- inf
-                # Put it back finally
-                #import pdb; pdb.set_trace()
-                data[prop.fullname] = logits_value
-                for learner in prop.find(AllenNlpLearner):
-                    data[learner.fullname] = logits_value
-
-        return data
diff --git a/domiknows/solver/gekkoILPBooleanMethods.py b/domiknows/solver/gekkoILPBooleanMethods.py
deleted file mode 100644
index 517de03ee..000000000
--- a/domiknows/solver/gekkoILPBooleanMethods.py
+++ /dev/null
@@ -1,356 +0,0 @@
-if __package__ is None or __package__ == '':
-    from domiknows.solver.ilpConfig import ilpConfig 
-else:
-    from .ilpConfig import ilpConfig 
-    
-if ilpConfig['ilpSolver'] == "Gurobi":
-    from gurobipy import *
-elif ilpConfig['ilpSolver'] == "GEKKO":
-    from gekko import GEKKO
-        
-class gekkoILPBooleanProcessor:
-    
-    # variable controlling what ILP solver is used
-    ilpSolver = "Gurobi" # "Gurobi", "GEKKO", "None"
-    
-    def __init__(self, _ildConfig = ilpConfig) -> None:
-        super().__init__()
-        
-        if _ildConfig is not None:
-            self.ilpSolver = _ildConfig['ilpSolver']
-
-    # Negation
-    def notVar(m, _var):
-        if ilpSolver=="Gurobi":
-             _notVar=m.addVar(vtype=GRB.BINARY, name="not_%s" % (_var))
-             
-             m.addConstr(1 - _var, GRB.EQUAL, _notVar)
-        elif ilpSolver == "GEKKO":
-            _notVar=m.Var(0, lb=0, ub=1, integer=True, name="not_%s" % (_var))
-            
-            m.Equation(1 - _var - _notVar == 0)
-    
-        return _notVar
-    
-    # Conjunction 2 variable
-    def and2Var(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _andVar=m.addVar(vtype=GRB.BINARY, name="and_%s_%s" % (_var1, _var2))
-            
-            m.addConstr(_andVar - _var1, GRB.LESS_EQUAL, 0)
-            m.addConstr(_andVar - _var2, GRB.LESS_EQUAL, 0)
-            
-            m.addConstr(_var1 + _var2 - _andVar - 1, GRB.LESS_EQUAL, 0)
-        elif ilpSolver == "GEKKO":
-            _andVar=m.Var(0, lb=0, ub=1, integer=True, name="and_%s_%s" % (_var1, _var2))
-    
-            m.Equation(_andVar - _var1 <= 0)
-            m.Equation(_andVar - _var2 <= 0)
-            
-            m.Equation(_var1 + _var2 - _andVar <= 1)
-            
-        return _andVar
-    
-    # Conjunction
-    def andVar(m, *_var):
-        if len(_var) < 1:
-            return None
-        
-        if len(_var) == 1:
-            for currentVar in _var:
-                return currentVar
-        
-        _andVarName = "and"
-        for currentVar in _var:
-            _andVarName += "_%s" % (currentVar)
-            
-        if ilpSolver=="Gurobi":
-            _andVar = m.addVar(vtype=GRB.BINARY, name=_andVarName)
-        elif ilpSolver == "GEKKO":
-            _andVar = m.Var(0, lb=0, ub=1, integer=True, name=_andVarName)
-    
-        for currentVar in _var:
-            if ilpSolver=="Gurobi":
-                m.addConstr(_andVar - currentVar, GRB.LESS_EQUAL, 0)
-            elif ilpSolver == "GEKKO":
-                m.Equation(_andVar - currentVar <= 0)
-    
-        if ilpSolver=="Gurobi":
-            _varSumLinExpr = LinExpr()
-            for currentVar in _var:
-                _varSumLinExpr.addTerms(1.0, currentVar)
-        
-            m.addConstr(_varSumLinExpr, GRB.LESS_EQUAL, _andVar + len(_var) - 1)
-        elif ilpSolver == "GEKKO":
-            m.Equation(m.sum(list(_var)) - _andVar - len(_var) <= -1)
-    
-        return _andVar
-    
-    # Disjunction 2 variables
-    def or2Var(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _orVar=m.addVar(vtype=GRB.BINARY, name="or_%s_%s" % (_var1, _var2))
-            
-            m.addConstr(_orVar, GRB.LESS_EQUAL, _var1)
-            m.addConstr(_orVar, GRB.LESS_EQUAL, _var2)
-            
-            m.addConstr(_var1 + _var2, GRB.GREATER_EQUAL, _orVar)
-        elif ilpSolver == "GEKKO":
-            _orVar=m.Var(0, lb=0, ub=1, integer=True, name="or_%s_%s" % (_var1, _var2))
-     
-            m.Equation(_orVar - _var1 <= 0)
-            m.Equation(_orVar - _var2 <= 0)
-            
-            m.Equation(_var1 + _var2 - _orVar >= 0)
-    
-        return _orVar
-    
-    # Disjunction
-    def orVar(m, *_var):
-        if len(_var) < 1:
-            return None
-        
-        if len(_var) == 1:
-            for currentVar in _var:
-                return currentVar
-        
-        _orVarName = "or"
-        for currentVar in _var:
-            _orVarName += "_%s" % (currentVar)
-        
-        if ilpSolver=="Gurobi":
-            _orVar = m.addVar(vtype=GRB.BINARY, name=_orVarName)
-        elif ilpSolver == "GEKKO":
-            _orVar = m.Var(0, lb=0, ub=1, integer=True, name=_orVarName)
-    
-        for currentVar in _var:
-            if ilpSolver=="Gurobi":
-                m.addConstr(_orVar, GRB.LESS_EQUAL, currentVar)
-            elif ilpSolver == "GEKKO":
-                m.Equation(_orVar - currentVar <= 0)
-    
-        if ilpSolver=="Gurobi":
-            _varSumLinExpr = LinExpr()
-            for currentVar in _var:
-                _varSumLinExpr.addTerms(1.0, currentVar)
-        
-            m.addConstr(_varSumLinExpr, GRB.GREATER_EQUAL, _orVar)
-        elif ilpSolver == "GEKKO":
-             m.Equation(m.sum(list(_var)) - _orVar >= 0)
-             
-        return _orVar
-    
-    # Nand (Alternative denial) 2 variables
-    def nand2Var(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _nandVar = m.addVar(vtype=GRB.BINARY, name="nand_%s_%s" % (_var1, _var2))
-            
-            m.addConstr(1 - _nandVar, GRB.LESS_EQUAL, _var1)
-            m.addConstr(1 - _nandVar, GRB.LESS_EQUAL, _var2)
-        
-            m.addConstr(_var1 + _var2, GRB.LESS_EQUAL, 2 - _nandVar)
-        elif ilpSolver == "GEKKO":
-            _nandVar = m.Var(0, lb=0, ub=1, integer=True, name="nand_%s_%s" % (_var1, _var2))
-    
-            m.Equation(1 - _nandVar - _var1 <= 0)
-            m.Equation(1 - _nandVar - _var2 <= 0)
-        
-            m.Equation(_var1 + _var2 + _nandVar <= 2)
-        
-        return _nandVar
-    
-    # Nand (Alternative denial)
-    def nandVar(m, *_var):
-        if len(_var) < 1:
-            return None
-        
-        if len(_var) == 1:
-            for currentVar in _var:
-                return currentVar
-        
-        _nandVarName = "nand"
-        for currentVar in _var:
-            _nandVarName += "_%s" % (currentVar)
-            
-        if ilpSolver=="Gurobi":
-            _nandVar = m.addVar(vtype=GRB.BINARY, name=_nandVarName)
-        elif ilpSolver == "GEKKO":
-            _nandVar = m.Var(0, lb=0, ub=1, integer=True, name=_nandVarName)
-    
-        for currentVar in _var:
-            if ilpSolver=="Gurobi":
-                m.addConstr(1 - _nandVar, GRB.LESS_EQUAL, currentVar)
-            elif ilpSolver == "GEKKO":
-                m.Equation(currentVar + _nandVar >= 1)
-        
-        if ilpSolver=="Gurobi":
-            _varSumLinExpr = LinExpr()
-            for currentVar in _var:
-                _varSumLinExpr.addTerms(1.0, currentVar)
-    
-            m.addConstr(_varSumLinExpr, GRB.LESS_EQUAL, len(_var) - _nandVar)
-        elif ilpSolver == "GEKKO":
-            m.Equation(m.sum(list(_var)) + _nandVar <= len(_var))
-    
-        return _nandVar
-    
-    # Nor (Joint Denial) i2 variables
-    def nor2Var(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _norVar = m.addVar(vtype=GRB.BINARY, name="nor_%s_%s"%(_var1, _var2))
-            
-            m.addConstr(_var1, GRB.LESS_EQUAL, 1 - _norVar)
-            m.addConstr(_var2, GRB.LESS_EQUAL, 1 - _norVar)
-            
-            m.addConstr(_var1 + _var2, GRB.GREATER_EQUAL, 1 - _norVar)
-        elif ilpSolver == "GEKKO":
-            _norVar = m.Var(0, lb=0, ub=1, integer=True, name="nor_%s_%s"%(_var1, _var2))
-            
-            m.Equation(_norVar + _var1 <= 1)
-            m.Equation(_norVar + _var2 <= 1)
-            
-            m.Equation(_var1 + _var2 + _norVar >= 1)
-            
-        return _norVar
-    
-    # Nor (Joint Denial)
-    def norVar(m, *_var):
-        if len(_var) < 1:
-            return None
-        
-        if len(_var) == 1:
-            for currentVar in _var:
-                return currentVar
-        
-        _norVarName = "nor"
-        for currentVar in _var:
-            _norVarName += "_%s"%(currentVar)
-           
-        if ilpSolver=="Gurobi":
-            _norVar = m.addVar(vtype=GRB.BINARY, name=_norVarName)
-        elif ilpSolver == "GEKKO":
-            _norVar = m.Var(0, lb=0, ub=1, integer=True, name=_norVarName)
-    
-        for currentVar in _var:
-            if ilpSolver=="Gurobi":
-                m.addConstr(currentVar, GRB.LESS_EQUAL, 1 - _norVar)
-            elif ilpSolver == "GEKKO":
-                m.Equation(currentVar + _norVar  <= 1)
-        
-        if ilpSolver=="Gurobi":
-            _varSumLinExpr = LinExpr()
-            for currentVar in _var:
-                _varSumLinExpr.addTerms(1.0, currentVar)
-            
-            m.addConstr(_varSumLinExpr, GRB.GREATER_EQUAL, 1 - _norVar)
-        elif ilpSolver == "GEKKO":
-            m.Equation(m.sum(list(_var)) + _nandVar >= 1)
-    
-        return _norVar
-    
-    # Exclusive Disjunction
-    def xorVar(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _xorVar = m.addVar(vtype=GRB.BINARY, name="xor_%s_%s"%(_var1, _var2))
-            
-            m.addConstr(_var1 + _var2 + _xorVar, GRB.LESS_EQUAL, 2)
-            m.addConstr(-_var1 - _var2 + _xorVar, GRB.LESS_EQUAL, 0)
-            m.addConstr(_var1 - _var2 + _xorVar, GRB.GREATER_EQUAL, 0)
-            m.addConstr(-_var1 + _var2 + _xorVar, GRB.GREATER_EQUAL, 0)
-        elif ilpSolver == "GEKKO":
-            _xorVar = m.Var(0, lb=0, ub=1, integer=True, name="xor_%s_%s"%(_var1, _var2))
-            
-            m.Equation(_var1 + _var2 + _xorVar <= 2)
-            m.Equation(-_var1 - _var2 + _xorVar <= 0)
-            m.Equation(_var1 - _var2 + _xorVar >= 0)
-            m.Equation(-_var1 + _var2 + _xorVar >= 0)
-            
-        return _xorVar
-    
-    # Implication
-    def ifVar(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _ifVar = m.addVar(vtype=GRB.BINARY, name="if_%s_%s"%(_var1, _var2))
-            
-            m.addConstr(1 - _var1 , GRB.LESS_EQUAL, _ifVar)
-            m.addConstr(_var2 , GRB.LESS_EQUAL, _ifVar)
-            m.addConstr(1 - _var1 + _var2, GRB.GREATER_EQUAL, _ifVar)
-        elif ilpSolver == "GEKKO":
-            _ifVar = m.Var(0, lb=0, ub=1, integer=True, name="if_%s_%s"%(_var1, _var2))
-            
-            m.Equation(_var1 + _ifVar >= 1)
-            m.Equation(_var2 - _ifVar <= 0)
-            m.Equation(_ifVar + _var1 - _var2 <= 1)
-            
-        return _ifVar
-           
-    # Equivalence 
-    def epqVar(m, _var1, _var2):
-        if ilpSolver=="Gurobi":
-            _epqVar = m.addVar(vtype=GRB.BINARY, name="epq_%s_%s"%(_var1, _var2))
-            
-            m.addConstr(_var1 + _var2 - _epqVar, GRB.LESS_EQUAL, 1)
-            m.addConstr(_var1 + _var2 + _epqVar , GRB.GREATER_EQUAL, 1)
-            m.addConstr(-_var1 + _var2 + _epqVar, GRB.LESS_EQUAL, 1)
-            m.addConstr(_var1 - _var2 + _epqVar, GRB.LESS_EQUAL, 1)
-        elif ilpSolver == "GEKKO":
-            _epqVar = m.Var(0, lb=0, ub=1, integer=True, name="epq_%s_%s"%(_var1, _var2))
-            
-            m.Equation(_var1 + _var2 - _epqVar <= 1)
-            m.Equation(_var1 + _var2 + _epqVar >= 1)
-            m.Equation(-_var1 + _var2 + _epqVar <= 1)
-            m.Equation(_var1 - _var2 + _epqVar <= 1)
-     
-        return _epqVar
-    
-    def main() :
-        # Create a new Gurobi model
-        m = Model("andVarTest")
-        m.Params.InfUnbdInfo = 1
-        m.params.outputflag = 1
-        
-        _var1 = m.addVar(vtype=GRB.BINARY, name="_var1")
-        _var2 = m.addVar(vtype=GRB.BINARY, name="_var2")
-    
-        m.addConstr(_var1, GRB.EQUAL, 1)
-        m.addConstr(_var2, GRB.EQUAL, 1)
-    
-        _andVar = andVar(m, _var1, _var2)
-        m.update()
-    
-        Q = LinExpr()
-        Q += _andVar
-        m.setObjective(Q, GRB.MAXIMIZE)
-    
-        m.update()
-        
-        print(m.getObjective()) 
-        for const in m.getConstrs():
-            print(m.getConstrByName(const.ConstrName))
-            
-        m.optimize()
-           
-        print(m.printStats())
-    
-        if m.status == GRB.Status.OPTIMAL:
-            print('Optimal solution was found')
-        elif m.status == GRB.Status.INFEASIBLE:
-            print('Model was proven to be infeasible.')
-            exit()
-        elif m.status == GRB.Status.INF_OR_UNBD:
-            print('Model was proven to be infeasible or unbound.')
-            exit()
-        elif m.status == GRB.Status.UNBOUNDED:
-            print('Model was proven to be unbound.')
-            exit()
-        else:
-            print('Optimal solution not was found - error code %i'%(m.status))
-            exit()
-    
-        m.update()
-        m.printAttr('x') 
-        
-        #print(_andVar.getAttr(GRB.Attr.X))
-    
-    if __name__ == '__main__' :
-        main()
\ No newline at end of file
diff --git a/domiknows/solver/gekkoILPOntSolver.py b/domiknows/solver/gekkoILPOntSolver.py
deleted file mode 100644
index 7376e6cc6..000000000
--- a/domiknows/solver/gekkoILPOntSolver.py
+++ /dev/null
@@ -1,804 +0,0 @@
-# numpy
-import numpy as np
-
-# ontology
-from owlready2 import *
-
-from gekko import GEKKO
-
-if __package__ is None or __package__ == '': 
-    from domiknows.solver.ilpConfig import ilpConfig
-    from domiknows.solver.ilpOntSolver import ilpOntSolver
- 
-    from domiknows.solver.gekkoILPBooleanMethods import gekkoILPBooleanProcessor
-else:
-    from .ilpConfig import ilpConfig 
-    from .ilpOntSolver import ilpOntSolver
-
-    from .gekkoILPBooleanMethods import gekkoILPBooleanProcessor
-
-import logging
-import datetime
-
-class gekkoILPOntSolver(ilpOntSolver):
-    ilpSolver = 'GEKKO'
-
-    def __init__(self, graph, ontologiesTuple, _ilpConfig=ilpConfig) -> None:
-        super().__init__(graph, ontologiesTuple, _ilpConfig=ilpConfig)
-        self.myIlpBooleanProcessor = gekkoILPBooleanProcessor()
-    
-    def addTokenConstrains(self, m, tokens, conceptNames, x, graphResultsForPhraseToken):
-        if graphResultsForPhraseToken is None:
-            return None
-        
-        # Create variables for token - concept and negative variables
-        for tokenIndex, token in enumerate(tokens):            
-            for conceptName in conceptNames: 
-                x[token, conceptName]=m.Var(0, lb=0, ub=1, integer=True, name="x_%s_%s"%(token, conceptName))
-                #it = np.nditer(graphResultsForPhraseToken[conceptName], flags=['c_index', 'multi_index'])
-                
-                currentProbability = graphResultsForPhraseToken[conceptName][tokenIndex]
-                self.myLogger.info("Probability for concept %s and token %s is %f"%(conceptName,token,currentProbability))
-
-                if currentProbability < 1.0: #ilpOntSolver.__negVarTrashhold:
-                    x[token, conceptName]=m.Var(0, lb=0, ub=1, integer=True, name="x_%s_%s"%(token, conceptName+'-neg'))
-
-        # Add constraints forcing decision between variable and negative variables 
-        for conceptName in conceptNames:
-            for token in tokens:
-                if (token, conceptName+'-neg') in x:
-                    m.Equation(x[token, conceptName] + x[token, conceptName+'-neg'] <= 1)
-         
-        self.myLogger.info("Created %i ILP variables for tokens"%(len(x)))
-        
-        # -- Add constraints based on concept disjoint statements in ontology - not(and(var1, var2)) = nand(var1, var2)
-        foundDisjoint = dict() # too eliminate duplicates
-        for conceptName in conceptNames:
-            
-            currentConcept = self.myOnto.search_one(iri = "*%s"%(conceptName))
-            
-            if currentConcept is None :
-                continue
-            
-            self.myLogger.debug("Concept \"%s\" from data set mapped to \"%s\" concept in ontology"%(currentConcept.name, conceptName))
-                
-            for d in currentConcept.disjoints():
-                disjointConcept = d.entities[1]._name
-                    
-                if currentConcept._name == disjointConcept:
-                    disjointConcept = d.entities[0]._name
-                        
-                    if currentConcept._name == disjointConcept:
-                        continue
-                        
-                if disjointConcept not in conceptNames:
-                     continue
-                        
-                if conceptName in foundDisjoint:
-                    if disjointConcept in foundDisjoint[conceptName]:
-                        continue
-                
-                if disjointConcept in foundDisjoint:
-                    if conceptName in foundDisjoint[disjointConcept]:
-                        continue
-                            
-                for token in tokens:
-                    #m.Equation(self.myIlpBooleanProcessor.nandVar(m, x[token, conceptName], x[token, disjointConcept]) >= 1)
-                    # short version ensuring that logical expression is SATISFY - no generating variable holding the result of evaluating the expression
-                    m.Equation(x[token, conceptName] + x[token, disjointConcept] <= 1)          
-                               
-                if not (conceptName in foundDisjoint):
-                    foundDisjoint[conceptName] = {disjointConcept}
-                else:
-                    foundDisjoint[conceptName].add(disjointConcept)
-                           
-            if conceptName in foundDisjoint:
-                self.myLogger.info("Created - disjoint - constrains between concept \"%s\" and concepts %s"%(conceptName,foundDisjoint[conceptName]))
-
-        # -- Add constraints based on concept equivalent statements in ontology - and(var1, av2)
-        foundEquivalent = dict() # too eliminate duplicates
-        for conceptName in conceptNames:
-            
-            currentConcept = self.myOnto.search_one(iri = "*%s"%(conceptName))
-                
-            if currentConcept is None :
-                continue
-                
-            for equivalentConcept in currentConcept.equivalent_to:
-                if equivalentConcept.name not in conceptNames:
-                     continue
-                        
-                if conceptName in foundEquivalent:
-                    if equivalentConcept.name in foundEquivalent[conceptName]:
-                        continue
-                
-                if equivalentConcept.name in foundEquivalent:
-                    if conceptName in foundEquivalent[equivalentConcept.name]:
-                        continue
-                            
-                for token in tokens:
-                    m.Equation(self.myIlpBooleanProcessor.andVar(m, x[token, conceptName], x[token, equivalentConcept]) >= 1)
-
-                if not (conceptName in foundEquivalent):
-                    foundEquivalent[conceptName] = {equivalentConcept.name}
-                else:
-                    foundEquivalent[conceptName].add(equivalentConcept.name)
-           
-            if conceptName in foundEquivalent:
-                self.myLogger.info("Created - equivalent - constrains between concept \"%s\" and concepts %s"%(conceptName,foundEquivalent[conceptName]))
-    
-        # -- Add constraints based on concept subClassOf statements in ontology - var1 -> var2
-        for conceptName in conceptNames :
-            
-            currentConcept = self.myOnto.search_one(iri = "*%s"%(conceptName))
-                
-            if currentConcept is None :
-                continue
-                
-            for ancestorConcept in currentConcept.ancestors(include_self = False) :
-                if ancestorConcept.name not in conceptNames :
-                     continue
-                            
-                for token in tokens:
-                    m.Equation(self.myIlpBooleanProcessor.ifVar(m, x[token, conceptName], x[token, ancestorConcept]) >= 1)
-
-                self.myLogger.info("Created - subClassOf - constrains between concept \"%s\" and concept \"%s\""%(conceptName,ancestorConcept.name))
-
-        # -- Add constraints based on concept intersection statements in ontology - and(var1, var2, var3, ..)
-        for conceptName in conceptNames :
-            
-            currentConcept = self.myOnto.search_one(iri = "*%s"%(conceptName))
-                
-            if currentConcept is None :
-                continue
-                
-            for conceptConstruct in currentConcept.constructs(Prop = None) :
-                if type(conceptConstruct) is And :
-                    
-                    for token in tokens:
-                        _varAnd = m.addVar(name="andVar_%s"%(constrainName))
-        
-                        andList = []
-                    
-                        for currentClass in conceptConstruct.Classes :
-                            andList.append(x[token, currentClass.name])
-    
-                        andList.append(x[token, conceptName])
-                        
-                        m.Equation(self.myIlpBooleanProcessor.andVar(m, andList) >= 1)
-        
-        # -- Add constraints based on concept union statements in ontology -  or(var1, var2, var3, ..)
-        for conceptName in conceptNames :
-            
-            currentConcept = self.myOnto.search_one(iri = "*%s"%(conceptName))
-                
-            if currentConcept is None :
-                continue
-                
-            for conceptConstruct in currentConcept.constructs(Prop = None) :
-                if type(conceptConstruct) is Or :
-                    
-                    for token in tokens:
-                        _varOr = m.addVar(name="orVar_%s"%(constrainName))
-        
-                        orList = []
-                    
-                        for currentClass in conceptConstruct.Classes :
-                            orList.append(x[token, currentClass.name])
-    
-                        orList.append(x[token, conceptName])
-                        
-                        m.Equation(self.myIlpBooleanProcessor.orVar(m, orList) >= 1)
-
-        # -- Add constraints based on concept objectComplementOf statements in ontology - xor(var1, var2)
-        for conceptName in conceptNames :
-            
-            currentConcept = self.myOnto.search_one(iri = "*%s"%(conceptName))
-                
-            if currentConcept is None :
-                continue
-                
-            for conceptConstruct in currentConcept.constructs(Prop = None) :
-                if type(conceptConstruct) is Not :
-                    
-                    complementClass = conceptConstruct.Class
-
-                    for token in tokens:       
-                        m.Equation(self.myIlpBooleanProcessor.xorVar(m, x[token, conceptName], x[token, complementClass.name]) >= 1)
-
-                    self.myLogger.info("Created - objectComplementOf - constrains between concept \"%s\" and concept \"%s\""%(conceptName,complementClass.name))
-                        
-        # ---- No supported yet
-    
-        # -- Add constraints based on concept disjonitUnion statements in ontology -  Not supported by owlready2 yet
-        
-        # -- Add constraints based on concept oneOf statements in ontology - ?
-        
-        # Add objectives
-        X_Q = None
-        for tokenIndex, token in enumerate(tokens):
-            for conceptName in conceptNames:
-                if X_Q is None:
-                    X_Q = graphResultsForPhraseToken[conceptName][tokenIndex]*x[token, conceptName]
-                else:
-                    X_Q += graphResultsForPhraseToken[conceptName][tokenIndex]*x[token, conceptName]
-
-                if (token, conceptName+'-neg') in x: 
-                    if X_Q is None:
-                        X_Q = (1-graphResultsForPhraseToken[conceptName][tokenIndex])*x[token, conceptName+'-neg']
-                    else:
-                        X_Q += (1-graphResultsForPhraseToken[conceptName][tokenIndex])*x[token, conceptName+'-neg']
-
-        return X_Q
-     
-    def addRelationsConstrains(self, m, tokens, conceptNames, x, y, graphResultsForPhraseRelation):
-        if graphResultsForPhraseRelation is None:
-            return None
-        
-        relationNames = graphResultsForPhraseRelation.keys()
-            
-        # Create variables for relation - token - token and negative variables
-        for relationName in relationNames:            
-            for token1Index, token1 in enumerate(tokens): 
-                for token2Index, token2 in enumerate(tokens):
-                    if token1 == token2:
-                        continue
-    
-                    y[relationName, token1, token2]=m.Var(0, lb=0, ub=1, integer=True, name="y_%s_%s_%s"%(relationName, token1, token2))
-
-                    if graphResultsForPhraseRelation[relationName][token2Index][token1Index] < 1.0: #ilpOntSolver.__negVarTrashhold:
-                        y[relationName+'-neg', token1, token2]=m.Var(0, lb=0, ub=1, integer=True, name="y_%s-neg_%s_%s"%(relationName, token1, token2))
-
-        # Add constraints forcing decision between variable and negative variables 
-        for relationName in relationNames:
-            for token in tokens: 
-                for token1 in tokens:
-                    if token == token1:
-                        continue
-                    
-                    if (relationName+'-neg', token, token1) in y: 
-                        m.Equation(y[relationName, token, token1] + y[relationName+'-neg', token, token1] <= 1)
-                                
-        self.myLogger.info("Created %i ilp variables for relations"%(len(y)))
-
-        # -- Add constraints based on property domain and range statements in ontology - P(x,y) -> D(x), R(y)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue
-    
-            self.myLogger.debug("Relation \"%s\" from data set mapped to \"%s\" concept in ontology"%(currentRelation.name, relationName))
-    
-            currentRelationDomain = currentRelation.get_domain() # domains_indirect()
-            currentRelationRange = currentRelation.get_range()
-                    
-            for domain in currentRelationDomain:
-                if domain._name not in conceptNames:
-                    continue
-                        
-                for range in currentRelationRange:
-                    if range.name not in conceptNames:
-                        continue
-                            
-                    for token in tokens:
-                        for token1 in tokens:
-                            if token == token1:
-                                continue
-                             
-                            #m.Equation(self.myIlpBooleanProcessor.ifVar(m, y[currentRelation._name, token, token1], x[token, domain._name]) >= 1)
-                            #m.Equation(self.myIlpBooleanProcessor.ifVar(m, y[currentRelation._name, token, token1], x[token, range._name]) >= 1)
-                                
-                            # short version ensuring that logical expression is SATISFY - no generating variable holding the result of evaluating the expression
-                            m.Equation(x[token, domain._name] - y[currentRelation._name, token, token1] >= 0)
-                            m.Equation(x[token1, range._name] - y[currentRelation._name, token, token1] >= 0)
-                                
-                    self.myLogger.info("Created - domain-range - constrains for relation \"%s\" for domain \"%s\" and range \"%s\""%(relationName,domain._name,range._name))
-
-          # -- Add constraints based on property subProperty statements in ontology R subproperty of S - R(x, y) -> S(x, y)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            for superProperty in currentRelation.is_a:
-                if superProperty.name not in graphResultsForPhraseRelation:
-                    continue
-                
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        #m.Equation(self.myIlpBooleanProcessor.ifVar(m, y[relationName, token, token1], y[superProperty.name, token, token1]) >= 1)
-                        m.Equation(y[superProperty.name, token, token1] - y[relationName, token, token1] >= 0)
-            
-        # -- Add constraints based on property equivalentProperty statements in ontology -  and(R, S)
-        foundEquivalent = dict() # too eliminate duplicates
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue
-                
-            for equivalentProperty in currentRelation.equivalent_to:
-                if equivalentProperty.name not in graphResultsForPhraseRelation:
-                     continue
-                        
-                if relationName in foundEquivalent:
-                    if equivalentProperty.name in foundEquivalent[relationName]:
-                        continue
-                
-                if equivalentProperty.name in foundEquivalent:
-                    if relationName in foundEquivalent[equivalentProperty.name]:
-                        continue
-                            
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        m.Equation(self.myIlpBooleanProcessor.andVar(m, y[relationName, token, token1], y[equivalentProperty.name, token, token1]) >= 1)
-
-                    if not (relationName in foundEquivalent):
-                        foundEquivalent[relationName] = {equivalentProperty.name}
-                    else:
-                        foundEquivalent[relationName].add(equivalentProperty.name)
-        
-        # -- Add constraints based on property inverseProperty statements in ontology - S(x,y) -> R(y,x)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            currentRelationInverse = currentRelation.get_inverse_property()
-            
-            if not currentRelationInverse:
-                continue
-            
-            if currentRelationInverse.name not in graphResultsForPhraseRelation:
-                continue
-                 
-            if currentRelationInverse is not None:
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        m.Equation(self.myIlpBooleanProcessor.ifVar(m, y[equivalentProperty.name, token, token1], y[relationName, token, token1]) >= 1)
-  
-        # -- Add constraints based on property functionalProperty statements in ontology - at most one P(x,y) for x
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            functionalLinExpr = None
-
-            if FunctionalProperty in currentRelation.is_a:
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        functionalLinExpr += y[relationName, token, token1]
-                
-                constrainName = 'c_%s_%s_%s_FunctionalProperty'%(token, token1, relationName)
-                m.addConstr(newLinExpr, GRB.LESS_EQUAL, 1, name=constrainName)
-        
-        # -- Add constraints based on property inverseFunctionaProperty statements in ontology - at most one P(x,y) for y
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            if InverseFunctionalProperty in currentRelation.is_a:
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        functionalLinExpr += y[relationName, token1, token]
-    
-                constrainName = 'c_%s_%s_%s_InverseFunctionalProperty'%(token, token1, relationName)
-                m.addConstr(newLinExpr, GRB.LESS_EQUAL, 1, name=constrainName)
-        
-        # -- Add constraints based on property reflexiveProperty statements in ontology - P(x,x)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            if ReflexiveProperty in currentRelation.is_a:
-                for token in tokens: 
-                    constrainName = 'c_%s_%s_ReflexiveProperty'%(token, relationName)
-                    m.addConstr(y[relationName, token, token], GRB.EQUAL, 1, name=constrainName)  
-                        
-        # -- Add constraints based on property irreflexiveProperty statements in ontology - not P(x,x)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            if IrreflexiveProperty in currentRelation.is_a:
-                for token in tokens: 
-                    constrainName = 'c_%s_%s_ReflexiveProperty'%(token, relationName)
-                    m.addConstr(y[relationName, token, token], GRB.EQUAL, 0, name=constrainName)  
-                    
-        # -- Add constraints based on property symetricProperty statements in ontology - R(x, y) -> R(y,x)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            if SymmetricProperty in currentRelation.is_a:
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        constrainName = 'c_%s_%s_%s_SymmetricProperty'%(token, token1, relationName)
-                        m.addGenConstrIndicator(y[relationName, token, token1], True, y[relationName, token1, token], GRB.EQUAL, 1)
-        
-        # -- Add constraints based on property asymetricProperty statements in ontology - not R(x, y) -> R(y,x)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            if AsymmetricProperty in currentRelation.is_a:
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        constrainName = 'c_%s_%s_%s_AsymmetricProperty'%(token, token1, relationName)
-                        m.addGenConstrIndicator(y[relationName, token, token1], True, y[relationName, token1, token], GRB.EQUAL, 0)  
-                        
-        # -- Add constraints based on property transitiveProperty statements in ontology - P(x,y) and P(y,z) - > P(x,z)
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-            
-            if TransitiveProperty in currentRelation.is_a:
-                for token in tokens: 
-                    for token1 in tokens:
-                        if token == token1:
-                            continue
-                        
-                        constrainName = 'c_%s_%s_%s_TransitiveProperty'%(token, token1, relationName)
-                        #m.addGenConstrIndicator(y[relationName, token, token1], True, y[relationName, token1, token], GRB.EQUAL, 1)  
-                               
-        # -- Add constraints based on property allValueFrom statements in ontology
-    
-        # -- Add constraints based on property hasValueFrom statements in ontology
-    
-        # -- Add constraints based on property objectHasSelf statements in ontology
-        
-        # -- Add constraints based on property disjointProperty statements in ontology
-    
-        # -- Add constraints based on property key statements in ontology
-    
-    
-        # -- Add constraints based on property exactCardinality statements in ontology
-    
-        # -- Add constraints based on property minCardinality statements in ontology
-        
-        # -- Add constraints based on property maxCardinality statements in ontology
-        for relationName in graphResultsForPhraseRelation:
-            currentRelation = self.myOnto.search_one(iri = "*%s"%(relationName))
-                
-            if currentRelation is None:
-                continue   
-    
-        # ---- Related to DataType properties - not sure yet if we need to support them
-        
-            # -- Add constraints based on property dataSomeValuesFrom statements in ontology
-        
-            # -- Add constraints based on property dataHasValue statements in ontology
-    
-            # -- Add constraints based on property dataAllValuesFrom statements in ontology
-    
-        # Add objectives
-        Y_Q  = None
-        for relationName in relationNames:
-            for token1Index, token1 in enumerate(tokens):
-                for token2Index, token2 in enumerate(tokens):
-                    if token1 == token2 :
-                        continue
-    
-                    if Y_Q is None:
-                        Y_Q = graphResultsForPhraseRelation[relationName][token2Index][token1Index]*y[relationName, token1, token2]
-                    else:
-                        Y_Q += graphResultsForPhraseRelation[relationName][token2Index][token1Index]*y[relationName, token1, token2]
-
-                    if (relationName+'-neg', token, token1) in y: 
-                        if Y_Q is None:
-                            Y_Q = (1-graphResultsForPhraseRelation[relationName][token2Index][token1Index])*y[relationName+'-neg', token1, token2]
-                        else:
-                            Y_Q += (1-graphResultsForPhraseRelation[relationName][token2Index][token1Index])*y[relationName+'-neg', token1, token2]
-        
-        return Y_Q
-    
-    def addTripleRelationsConstrains(self, m, tokens, conceptNames, x, y, z, graphResultsForPhraseTripleRelation):
-        if graphResultsForPhraseTripleRelation is None:
-            return None
-        
-        tripleRelationNames = graphResultsForPhraseTripleRelation.keys()
-            
-        # Create variables for relation - token - token -token and negative variables
-        for tripleRelationName in tripleRelationNames:            
-            for tokenIndex, token in enumerate(tokens): 
-                for token1Index, token1 in enumerate(tokens):
-                    if token1 == token:
-                        continue
-                        
-                    for token2Index, token2 in enumerate(tokens):
-                        if token2 == token:
-                            continue
-                        
-                        if token2 == token1:
-                            continue
-                    
-                        z[tripleRelationName, token, token1, token2]=m.Var(0, lb=0, ub=1, integer=True, name="y_%s_%s_%s_%s"%(tripleRelationName, token, token1, token2))
-
-    
-                        if graphResultsForPhraseTripleRelation[tripleRelationName][token2Index][token1Index][tokenIndex] < ilpOntSolver.__negVarTrashhold:
-                            z[tripleRelationName+'-neg', token, token1, token2]=m.Var(0, lb=0, ub=1, integer=True, name="y_%s-neg_%s_%s_%s"%(tripleRelationName, token, token1, token2))
-
-                        
-        # Add constraints forcing decision between variable and negative variables 
-        for tripleRelationName in tripleRelationNames:
-            for token in tokens: 
-                for token1 in tokens:
-                    if token1 == token:
-                        continue
-                        
-                    for token2 in tokens:
-                        if token2 == token:
-                            continue
-                        
-                        if token2 == token1:
-                            continue
-                
-                        if (tripleRelationName+'-neg', token, token1, token2) in z: 
-                            m.Equation(z[tripleRelationName, token, token1, token2] + z[tripleRelationName+'-neg', token, token1, token2] <= 1)
-    
-        self.myLogger.info("Created %i ilp variables for triple relations"%(len(z)))
-
-        # -- Add constraints 
-        for tripleRelationName in graphResultsForPhraseTripleRelation:
-            currentTripleRelation = self.myOnto.search_one(iri = "*%s"%(tripleRelationName))
-                
-            if currentTripleRelation is None:
-                continue
-    
-            self.myLogger.debug("Triple Relation \"%s\" from data set mapped to \"%s\" concept in ontology"%(currentTripleRelation.name, currentTripleRelation))
-            
-            ancestorConcept = None
-            for _ancestorConcept in currentTripleRelation.ancestors(include_self = False):
-                if _ancestorConcept.name == "Thing":
-                     continue
-                 
-                ancestorConcept = _ancestorConcept
-                break
-            
-            if ancestorConcept is None:
-                break
-            
-            tripleProperties = {}
-            triplePropertiesRanges = {}    
-
-            for property in self.myOnto.object_properties():
-                _domain = property.domain
-                
-                if _domain is None:
-                    break
-                
-                domain = _domain[0]._name
-                if domain is ancestorConcept.name:                    
-                    for superProperty in property.is_a:
-                        if superProperty is None:
-                            continue
-                        
-                        if superProperty.name == 'first':
-                            tripleProperties['1'] = property
-                            _range = property.range
-                
-                            if _range is None:
-                                break
-                
-                            range = _range[0]._name
-                            triplePropertiesRanges['1'] = range
-                        elif superProperty.name == 'second':
-                            tripleProperties['2'] = property
-                            _range = property.range
-                
-                            if _range is None:
-                                break
-                
-                            range = _range[0]._name
-                            triplePropertiesRanges['2'] = range
-                        elif superProperty.name == 'third':
-                            tripleProperties['3'] = property
-                            _range = property.range
-                
-                            if _range is None:
-                                break
-                
-                            range = _range[0]._name
-                            triplePropertiesRanges['3'] = range
-                                        
-            for toke1n in tokens:
-                for token2 in tokens:
-                    if token2 == token1:
-                        continue
-                        
-                    for token3 in tokens:
-                        if token3 == token1:
-                            continue
-                        
-                        if token3 == token2:
-                            continue
-                     
-                        # TODO
-                                    
-                        self.myLogger.info("Created - triple - constrains for relation \"%s\" for tokens \"%s\", \"%s\", \"%s\""%(tripleRelationName,token1,token2,token3))
-    
-        # Add objectives
-        Z_Q  = None
-        for tripleRelationName in tripleRelationNames:
-            for token1Index, token1 in enumerate(tokens): 
-                for token2Index, token2 in enumerate(tokens):
-                    if token2 == token1:
-                        continue
-                        
-                    for token3Index, token3 in enumerate(tokens):
-                        if token3 == token1:
-                            continue
-                        
-                        if token3 == token2:
-                            continue
-
-                        if Z_Q is None:
-                            Z_Q = graphResultsForPhraseTripleRelation[tripleRelationName][token3Index][token2Index][token1Index]*z[tripleRelationName, token1, token2, token3]
-                        else:
-                            Z_Q += graphResultsForPhraseTripleRelation[tripleRelationName][token3Index][token2Index][token1Index]*z[tripleRelationName, token1, token2, token3]
-    
-                        if (tripleRelationName+'-neg', token1, token2, token3) in z: 
-                            if Z_Q is None:
-                                Z_Q = (1-graphResultsForPhraseTripleRelation[tripleRelationName][token3Index][token2Index][token1Index])*z[tripleRelationName+'-neg', token1, token2, token3]
-                            else:
-                                Z_Q += (1-graphResultsForPhraseTripleRelation[tripleRelationName][token3Index][token2Index][token1Index])*z[tripleRelationName+'-neg', token1, token2, token3]
-            
-        return Z_Q
-        
-    def calculateILPSelection(self, phrase, fun=None, epsilon = 0.00001, graphResultsForPhraseToken=None, graphResultsForPhraseRelation=None, graphResultsForPhraseTripleRelation=None):
-        start = datetime.datetime.now()
-        self.myLogger.info('Start for phrase %s'%(phrase))
-
-        if graphResultsForPhraseRelation is not None:
-            for relation in graphResultsForPhraseRelation:
-                self.myLogger.info('graphResultsForPhraseRelation for relation \"%s\" \n%s'%(relation, graphResultsForPhraseRelation[relation]))
-        
-        if graphResultsForPhraseTripleRelation is not None:
-            for tripleRelation in graphResultsForPhraseTripleRelation:
-                self.myLogger.info('graphResultsForPhraseTripleRelation for relation \"%s\" \n%s'%(tripleRelation, graphResultsForPhraseTripleRelation[tripleRelation]))
-
-        conceptNames = graphResultsForPhraseToken.keys()
-        tokens = [x for x, _ in phrase]
-                    
-        try:
-            m = GEKKO()
-            solverDisp=False # remote solver used
-            m.options.LINEAR = 1
-            m.options.IMODE = 3 #steady state optimization
-            #m.options.CSV_READ = 0
-            #m.options.CSV_WRITE = 0 # no result files are written
-            m.options.WEB  = 0
-            gekkoTresholdForTruth = 0.5
-            
-            # Variables for concept - token
-            x={}
-    
-            # Variables for relation - token, token
-            y={}
-            
-            # Variables for relation - token, token, token
-            z={}
-                
-            # -- Set objective
-            Q = None
-            
-            X_Q = self.addTokenConstrains(m, tokens, conceptNames, x, graphResultsForPhraseToken)
-            if X_Q is not None:
-                if Q is None:
-                    Q = X_Q
-                else:
-                    Q += X_Q
-            
-            Y_Q = self.addRelationsConstrains(m, tokens, conceptNames, x, y, graphResultsForPhraseRelation)
-            if Y_Q is not None:
-                Q += Y_Q
-                
-            Z_Q = self.addTripleRelationsConstrains(m, tokens, conceptNames, x, y, z, graphResultsForPhraseTripleRelation)
-            if Z_Q is not None:
-                Q += Z_Q
-            
-            m.Obj((-1) * Q)
-
-            # Token is associated with a single concept
-            #for token in tokens:
-            #   constrainName = 'c_%s'%(token)
-            #    m.addConstr(quicksum(x[token, conceptName] for conceptName in conceptNames), GRB.LESS_EQUAL, 1, name=constrainName)
-            
-            startOptimize = datetime.datetime.now()
-
-            m.solve(disp=solverDisp)
-
-            endOptimize = datetime.datetime.now()
-            elapsedOptimize = endOptimize - startOptimize
-
-            self.myLogger.info('Optimal solution was found - elapsed time: %ims'%(elapsedOptimize.microseconds/1000))
-
-            # Collect results for tokens
-            tokenResult = None
-            if graphResultsForPhraseToken is not None:
-                tokenResult = dict()
-                
-                for conceptName in conceptNames:
-                    tokenResult[conceptName] = np.zeros(len(tokens))
-                            
-                    for tokenIndex, token in enumerate(tokens):             
-                        if x[token, conceptName].value[0] > gekkoTresholdForTruth:
-                            tokenResult[conceptName][tokenIndex] = 1
-                            self.myLogger.info('Solution \"%s\" is \"%s\"'%(token,conceptName))
-
-            # Collect results for relations
-            relationResult = None
-            if graphResultsForPhraseRelation is not None: 
-                relationResult = dict()
-                relationNames = graphResultsForPhraseRelation.keys()
-                
-                for relationName in relationNames:                        
-                    for token1Index, token1 in enumerate(tokens):
-                        for token2Index, token2 in enumerate(tokens):
-                            if token2 == token1:
-                                continue
-                            
-                            if y[relationName, token1, token2].value[0] > gekkoTresholdForTruth:
-                                relationResult[relationName][token2Index][token1Index] = 1
-                                
-                                self.myLogger.info('Solution \"%s\" is in relation \"%s\" with \"%s\"'%(token2,relationName,token1))
-                        
-            # Collect results for triple relations
-            tripleRelationResult = None
-            if graphResultsForPhraseTripleRelation is not None:
-                tripleRelationResult = {}
-                tripleRelationNames = graphResultsForPhraseTripleRelation.keys()
-                
-                for tripleRelationName in tripleRelationNames:
-                    tripleRelationResult[tripleRelationName] = np.zeros((len(tokens), len(tokens), len(tokens)))
-                
-                pass # TODO
-        
-        except:
-            self.myLogger.error('Error')
-            raise
-           
-        end = datetime.datetime.now()
-        elapsed = end - start
-        self.myLogger.info('End - elapsed time: %ims'%(elapsed.microseconds/1000))
-        
-        # return results of ILP optimization
-        return tokenResult, relationResult, tripleRelationResult
\ No newline at end of file
diff --git a/domiknows/solver/gurobiILPBooleanMethods.py b/domiknows/solver/gurobiILPBooleanMethods.py
index 39fb8fd37..d89458cfa 100644
--- a/domiknows/solver/gurobiILPBooleanMethods.py
+++ b/domiknows/solver/gurobiILPBooleanMethods.py
@@ -669,7 +669,7 @@ def countVar(self, m, *var, onlyConstrains = False, limitOp = 'None', limit = 1,
                     if m: m.update()
                     
                     m.addConstr(varsInfo['varSumLinExpr'] - BigM *varCOUNT >= updatedLimit - BigM, name='Count %s:'%(logicMethodName))
-                    m.addConstr(varsInfo['varSumLinExpr'] - BigM *varCOUNT <= updatedLimit, name='Count %s:'%(logicMethodName))   
+                    m.addConstr(varsInfo['varSumLinExpr'] - BigM *varCOUNT <= updatedLimit - (1 - varCOUNT), name='Count %s:'%(logicMethodName))  
                                  
             # This check is common for '<=' and '=='
             elif updatedLimit < 0: # The constraint not is satisfied - the limit is negative or zero so ilp sum cannot be less than it - ilp sum is zero or more
diff --git a/domiknows/solver/gurobiILPOntSolver.py b/domiknows/solver/gurobiILPOntSolver.py
index 9bc69b0de..68d6fc37a 100644
--- a/domiknows/solver/gurobiILPOntSolver.py
+++ b/domiknows/solver/gurobiILPOntSolver.py
@@ -1,4 +1,5 @@
-from time import process_time, process_time_ns
+import math
+from time import perf_counter, perf_counter_ns
 from collections import OrderedDict
 import logging
 
@@ -20,9 +21,11 @@
 from domiknows.graph import LcElement, LogicalConstrain, V, fixedL, ifL, forAllL
 from domiknows.graph import CandidateSelection
 from domiknows.utils import getReuseModel
+from domiknows.utils import getDnSkeletonMode
 
 from domiknows.graph.candidates import getCandidates
 
+
 class gurobiILPOntSolver(ilpOntSolver):
     ilpSolver = 'Gurobi'
 
@@ -51,234 +54,208 @@ def get_logical_constraints(self, ):
     def reset_logical_constraints(self, ):
         self.logical_constraints = self.myGraph[0].logicalConstrains ### Can myGraph really be multiple graphs?
         
-    def valueToBeSkipped(self, x):
-        return ( 
-                x != x or  # nan 
-                abs(x) == float('inf')  # inf 
-                ) 
+    def getConcept(self, concept):
+        return concept[0]
+    
+    def getConceptName(self, concept):
+        return concept[0].name
+    
+    def conceptIsBinary(self, concept):
+        return concept[2] is None
     
+    def conceptIsMultiClass(self, concept):
+        return concept[2] is not None
+    
+    # Check if value is NaN or if and has to be skipped
+    def valueToBeSkipped(self, x):
+        return math.isnan(x) or math.isinf(x)
+        
     # Get Ground Truth for provided concept
     def __getLabel(self, dn, conceptRelation, fun=None, epsilon = None):
         value = dn.getAttribute(conceptRelation, 'label')
-        
         return value
         
-    # Get and calculate probability for provided concept
-    def __getProbability(self, dn, conceptRelation, fun=None, epsilon = 0.00001):
-        if not dn:
-            valueI = None
-        else:
-            valueI = dn.getAttribute(conceptRelation, "local" , "softmax")
-                    
-        if valueI is None: # No probability value - return negative probability 
-            return [float("nan"), float("nan")]
+    # Get and tailor probability for provided concept and instance (represented as dn)
+    def getProbability(self, dn, conceptRelation, key = ("local" , "softmax"), fun=None, epsilon = 0.00001):
+        valueI = dn.getAttribute(conceptRelation, *key) if dn else None
+        labelIndex = conceptRelation[2]
         
-        if conceptRelation[2] is not None:
-            value = torch.empty(2, dtype=torch.float)
-            
-            value[0] = 1 - valueI[conceptRelation[2]]
-            value[1] = valueI[conceptRelation[2]]
+        if valueI is None:
+            value = None
         else:
-            value = valueI
-
-        # Process probability through function and apply epsilon
-        if epsilon is not None:
-            if value[0] > 1-epsilon:
-                value[0] = 1-epsilon
-            elif value[1] > 1-epsilon:
-                value[1] = 1-epsilon
-                
-            if value[0] < epsilon:
-                value[0] = epsilon
-            elif value[1] < epsilon:
-                value[1] = epsilon
-               
-            # Apply fun on probabilities 
+            labelIndex = conceptRelation[2]
+            # If value is for multi-class concept, then we need to convert it to binary
+            value = torch.zeros(2, dtype=torch.float) if self.conceptIsMultiClass(conceptRelation) else valueI.squeeze(0)
+            value[0] = 1 - valueI[labelIndex] if self.conceptIsMultiClass(conceptRelation) else value[0]
+            value[1] = valueI[labelIndex] if self.conceptIsMultiClass(conceptRelation) else value[1]
+            
+        # If softmax process probability through function and apply epsilon
+        if "softmax" in key and value is not None and not math.isnan(value[0]) and epsilon is not None:
+            value[0] = max(epsilon, min(1-epsilon, value[0]))
+            value[1] = max(epsilon, min(1-epsilon, value[1]))
+                    
+            # Apply fun on probabilities if defined
             if fun is not None:
                 value = fun(value)
-            
+        
         return value # Return probability
     
+    def createILPVariable(self, m, dn, currentConceptRelation, notV = False):
+        if notV:
+            xkey = '<' + self.getConceptName(currentConceptRelation) + '>/ILP/notx'
+        else:
+            xkey = '<' + self.getConceptName(currentConceptRelation) + '>/ILP/x'  
+
+        # Create a placeholder in Datanode for variable if it does not exist
+        currentConceptLabelLength =  currentConceptRelation[3]
+        dn.attributes.setdefault(xkey, [None] * currentConceptLabelLength)
+        
+        # Check Datanode if ILP variable is already created
+        if self.conceptIsMultiClass(currentConceptRelation):
+            xNew = dn.attributes[xkey][currentConceptRelation[2]]
+        else:
+            xNew =  dn.attributes[xkey][0]
+            
+        # Return ILP variable if it already exists
+        if xNew is not None:
+            return xNew
+            
+        # --- Else create ILP variable
+        currentLabel = currentConceptRelation[1]
+
+        if notV:
+            xVarName = "%s_%s_is_not_%s"%(dn.getOntologyNode(), dn.getInstanceID(), currentLabel)
+        else:
+            xVarName = "%s_%s_is_%s"%(dn.getOntologyNode(), dn.getInstanceID(), currentLabel)
+            
+        xNew = m.addVar(vtype=GRB.BINARY,name=xVarName) 
+
+        # Add ILP Variable to Datanode
+        if self.conceptIsMultiClass(currentConceptRelation):
+            dn.attributes[xkey][currentConceptRelation[2]] = xNew
+        else:
+            dn.attributes[xkey][0] = xNew
+        
+        return xNew
+        
+    def getDatanodesForConcept(self, rootDn, currentName, conceptToDNSCash=None):
+        if conceptToDNSCash is None or currentName is None:
+            conceptToDNSCash = {}
+           
+            if currentName is None:
+                return  # Just reset cash
+            
+        if currentName in conceptToDNSCash:
+            dns = conceptToDNSCash[currentName]
+        else:
+            rootConcept = rootDn.findRootConceptOrRelation(currentName)
+            dns = rootDn.findDatanodes(select=rootConcept)
+            conceptToDNSCash[currentName] = dns
+            
+        return dns
+    
+    # Count number of variables for logical constraints
     def countLCVariables(self, rootDn, *conceptsRelations):
-        # Collect LC variables 
         ilpVarCount = {}
-        conceptToDNSCash = {}
         
         for currentConceptRelation in conceptsRelations: 
-            currentName = currentConceptRelation[0].name
-            
-            if currentName in conceptToDNSCash:
-                dns = conceptToDNSCash[currentName]
-            else:
-                rootConcept = rootDn.findRootConceptOrRelation(currentName)
-                dns = rootDn.findDatanodes(select = rootConcept)
-                conceptToDNSCash[currentName] = dns
+            currentName = self.getConceptName(currentConceptRelation)
+            currentLabel = currentConceptRelation[1]
+
+            dns = self.getDatanodesForConcept(rootDn, currentName)
                 
-            currentConceptName = currentName + "_" + currentConceptRelation[1]
+            currentConceptName = currentName + "_" + currentLabel
             ilpVarCount[currentConceptName] = len(dns)
             
-            if currentConceptRelation[2] is None:
+            if self.conceptIsMultiClass(currentConceptName):
                 ilpVarCount[currentConceptName] += len(dns)
        
         return ilpVarCount
-    
-    def createObjective(self, xDict, rootDn, *conceptsRelations, dnFun = None, fun=None, epsilon = 0.00001):
+  
+    # Create ILP variables for logical constraints and objective
+    def createILPVariables(self, m, x, rootDn, *conceptsRelations, key = ("local" , "softmax"), fun=None, epsilon = 0.00001):
         Q = None
-        conceptToDNSCash = {}        
+        # Reset the cash for concept to datanodes
+        self.getDatanodesForConcept(rootDn, None)
         
-        x = list(xDict.values())
-        index = 0
+        # Create ILP variables 
         for currentConceptRelation in conceptsRelations: 
-            currentName = currentConceptRelation[0]
-            
-            if currentName in conceptToDNSCash:
-                dns = conceptToDNSCash[currentName]
+            currentLabel = currentConceptRelation[1]
+            if self.conceptIsMultiClass(currentConceptRelation):
+                currentLabelIndex = currentConceptRelation[2]
             else:
-                rootConcept = rootDn.findRootConceptOrRelation(currentName)
-                dns = rootDn.findDatanodes(select = rootConcept)
-                conceptToDNSCash[currentName] = dns
+                currentLabelIndex = 0
+            
+            if currentLabel is None:
+                self.myLogger.debug("Concept %s does not have label"%currentConceptRelation)
        
+            # Get datanodes for concept
+            dns = self.getDatanodesForConcept(rootDn, self.getConceptName(currentConceptRelation))
             for dn in dns:
-                if x[index] == None:
-                    continue
+                xNew = x.get((self.getConcept(currentConceptRelation), currentLabel, dn.getInstanceID(), currentLabelIndex), None)
+                if self.conceptIsBinary(currentConceptRelation):
+                    xNotNew = x.get((self.getConcept(currentConceptRelation), 'Not_' + currentLabel, dn.getInstanceID(), currentLabelIndex), None)
+
+                if xNew is None:
+                    # Create ILP variable
+                    xNew = self.createILPVariable(m, dn, currentConceptRelation)
+                    if self.conceptIsBinary(currentConceptRelation):
+                        xNotNew  = self.createILPVariable(m, dn, currentConceptRelation, notV=True)
+
+                # ---- Prepare ILP variables for constraints and objective for the current run
                 
-                currentProbability = dnFun(dn, currentConceptRelation, fun=fun, epsilon=epsilon)
+                # Create placeholder for variable in the current x set for run
+                x[self.getConcept(currentConceptRelation), currentLabel, dn.getInstanceID(), currentLabelIndex] = None
+                if self.conceptIsBinary(currentConceptRelation):
+                    x[self.getConcept(currentConceptRelation), 'Not_'+  currentLabel, dn.getInstanceID(), currentLabelIndex]  = None
+                    
+                # Get probability for variable in the current run
+                currentProbability = self.getProbability(dn, currentConceptRelation, key=key, fun=fun, epsilon=epsilon)
                 
+                # Skip variable for the current run if probability is None
                 if currentProbability == None or (torch.is_tensor(currentProbability) and currentProbability.dim() == 0) or len(currentProbability) < 2:
-                    self.myLogger.warning("Probability not provided for variable concept %s in dataNode %s - skipping it"%(currentName,dn.getInstanceID()))
-                    x[index] = None
-                
-                # Check if probability is NaN or if and has to be skipped
+                    self.myLogger.warning("Probability not provided for variable concept %s in dataNode %s - skipping it"%(self.getConceptName(currentConceptRelation),dn.getInstanceID()))
+                # Skip variable for the current run if probability is NaN or Inf
                 elif self.valueToBeSkipped(currentProbability[1]):
-                    self.myLogger.info("Probability is %f for concept %s and dataNode %s - skipping it"%(currentProbability[1],currentConceptRelation[1],dn.getInstanceID()))
-                    x[index] = None
-                else:
+                    self.myLogger.info("Probability is %f for concept %s and dataNode %s - skipping it"%(currentProbability[1],  currentLabel, dn.getInstanceID()))
+                else:    
+                    # Add variable to the x set for current run
+                    x[self.getConcept(currentConceptRelation),  currentLabel, dn.getInstanceID(), currentLabelIndex] = xNew
+                    if self.conceptIsBinary(currentConceptRelation):
+                        x[self.getConcept(currentConceptRelation), 'Not_' + currentLabel, dn.getInstanceID(), currentLabelIndex] = xNotNew
+
+                    # Add variable to objective
                     if Q is None:
-                        Q = currentProbability[1] * x[index]
-                    else:
-                        Q += currentProbability[1] * x[index] 
-                    index += 1   
-                       
-                # Create negative variable for binary concept
-                if currentConceptRelation[2] is None: # ilpOntSolver.__negVarTrashhold:
-                    if x[index-1] == None:
-                        x[index] = None
-                        continue
-                    
-                    # Check if probability is NaN or if and has to be created based on positive value
-                    if self.valueToBeSkipped(currentProbability[0]):
-                        currentProbability[0] = 1 - currentProbability[1]
-                        self.myLogger.info("No ILP negative variable for concept %s and dataNode %s - created based on positive value %f"
-                                           %(dn.getInstanceID(), currentName, currentProbability[1]))
-                        
-                    Q += currentProbability[0] * x[index] 
-                    index += 1  
-           
-        return Q
-    
-    def createILPVariables(self, m, x, rootDn, *conceptsRelations, dnFun = None, fun=None, epsilon = 0.00001):
-        Q = None
-        
-        # Create ILP variables 
-        for currentConceptRelation in conceptsRelations: 
-            rootConcept = rootDn.findRootConceptOrRelation(currentConceptRelation[0])
-            dns = rootDn.findDatanodes(select = rootConcept)
-            xkey = '<' + currentConceptRelation[0].name + '>/ILP/x'  
-       
-            for dn in dns:
-                # Init x to None
-                if currentConceptRelation[2] is not None:
-                    x[currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]] = None
-                else:
-                    x[currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), 0] = None
-                    
-                if currentConceptRelation[2] is None:
-                    if currentConceptRelation[2] is not None:
-                        x[currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]] = None
+                        Q = currentProbability[1] * xNew
                     else:
-                        x[currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), 0] = None
-                        
-                currentProbability = dnFun(dn, currentConceptRelation, fun=fun, epsilon=epsilon)
+                        Q += currentProbability[1] * xNew       
+                            
+                    if self.conceptIsBinary(currentConceptRelation): 
+                        Q += currentProbability[0] * xNotNew    
                 
-                if currentProbability == None or (torch.is_tensor(currentProbability) and currentProbability.dim() == 0) or len(currentProbability) < 2:
-                    self.myLogger.warning("Probability not provided for variable concept %s in dataNode %s - skipping it"%(currentConceptRelation[0].name,dn.getInstanceID()))
-                    continue
+            if self.conceptIsMultiClass(currentConceptRelation):
+                self.myLogger.debug("No creating ILP negative variables for multiclass concept %s"%(currentLabel))
                 
-                # Check if probability is NaN or if and has to be skipped
-                if self.valueToBeSkipped(currentProbability[1]):
-                    self.myLogger.info("Probability is %f for concept %s and dataNode %s - skipping it"%(currentProbability[1],currentConceptRelation[1],dn.getInstanceID()))
-                    continue
-    
-                xNew = None
-                if currentConceptRelation[2] is not None:
-                    if (currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]) in x:
-                        xNew = x[currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]]
-                else:
-                    if (currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), 0) in x:
-                        xNew = x[currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), 0]
-                
-                if xkey not in dn.attributes:
-                    dn.attributes[xkey] = [None] * currentConceptRelation[3]
-                        
-                if xNew is None:
-                    # Create variable
-                    xVarName = "%s_%s_is_%s"%(dn.getOntologyNode(), dn.getInstanceID(), currentConceptRelation[1])
-                    xNew = m.addVar(vtype=GRB.BINARY,name=xVarName) 
-                    
-                    if currentConceptRelation[2] is not None:
-                        x[currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]] = xNew
-                    else:
-                        x[currentConceptRelation[0], currentConceptRelation[1], dn.getInstanceID(), 0] = xNew
+            if getDnSkeletonMode() and "variableSet" in rootDn.attributes:
+                rootConcept = rootDn.findRootConceptOrRelation(currentConceptRelation)
+                rootConceptName = rootConcept.name
 
-                if currentConceptRelation[2] is not None:
-                    dn.attributes[xkey][currentConceptRelation[2]] = xNew
-                else:
-                    dn.attributes[xkey][0] = xNew
-                
-                if Q is None:
-                    Q = currentProbability[1] * xNew
-                else:
-                    Q += currentProbability[1] * xNew       
-    
-                # Check if probability is NaN or if and has to be created based on positive value
-                if self.valueToBeSkipped(currentProbability[0]):
-                    currentProbability[0] = 1 - currentProbability[1]
-                    self.myLogger.info("No ILP negative variable for concept %s and dataNode %s - created based on positive value %f"
-                                       %(dn.getInstanceID(), currentConceptRelation[0].name, currentProbability[1]))
-    
-                # Create negative variable for binary concept
-                if currentConceptRelation[2] is None: # ilpOntSolver.__negVarTrashhold:
-                    xNotNew  = None
+                xkey = '<' + self.getConceptName(currentConceptRelation) + '>/ILP/x'
+                xkeyInVariableSet = rootConceptName + "/" + xkey
+                if not rootDn.hasAttribute(xkey):
+                    rootDn.attributes["variableSet"][xkeyInVariableSet] = []
+                xForConcept = [value for key, value in x.items() if key[0] == self.getConcept(currentConceptRelation) and key[1] == currentLabel]
+                rootDn.attributes["variableSet"][xkeyInVariableSet].append(xForConcept)
                     
-                    if currentConceptRelation[2] is not None:
-                        if (currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]) in x:
-                            xNotNew= x[currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]]
-                    else:
-                        if (currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), 0) in x:
-                            xNotNew = x[currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), 0]
+                if self.conceptIsBinary(currentConceptRelation):
+                    notxkey = '<' + self.getConceptName(currentConceptRelation) + '>/ILP/notx'
+                    notxkeyInVariableSet = rootConceptName + "/" + notxkey
+                    if not rootDn.hasAttribute(notxkey):
+                        rootDn.attributes["variableSet"][notxkeyInVariableSet] = []
                     
-                    notxkey = '<' + currentConceptRelation[0].name + '>/ILP/notx'
-                
-                    if notxkey not in dn.attributes:
-                        dn.attributes[notxkey] = [None] * currentConceptRelation[3]
-                        
-                    if xNotNew is None:
-                        xNotVarName = "%s_%s_is_not_%s"%(dn.getOntologyNode(), dn.getInstanceID(), currentConceptRelation[1])
-                        xNotNew = m.addVar(vtype=GRB.BINARY,name=xNotVarName)
-                        if currentConceptRelation[2] is not None:
-                            x[currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), currentConceptRelation[2]] = xNotNew
-                        else:
-                            x[currentConceptRelation[0], 'Not_'+currentConceptRelation[1], dn.getInstanceID(), 0] = xNotNew
-                        
-                    if currentConceptRelation[2] is not None:
-                        dn.attributes[notxkey][currentConceptRelation[2]] = xNotNew
-                    else:
-                        dn.attributes[notxkey][0] = xNotNew
-                                        
-                    Q += currentProbability[0] * xNotNew    
-                
-            if currentConceptRelation[2] is not None:
-                self.myLogger.debug("No creating ILP negative variables for multiclass concept %s"%( currentConceptRelation[1]))
+                    xnotForConcept = [value for key, value in x.items() if key[0] == self.getConcept(currentConceptRelation) and key[1] == 'Not_' + currentLabel]
+                    rootDn.attributes["variableSet"][notxkeyInVariableSet].append(xnotForConcept)
+
 
         m.update()
 
@@ -704,7 +681,7 @@ def addLogicalConstrains(self, m, dn, lcs, p, key = None):
             key = "/ILP/xP" # to get ILP variable from datanodes
         
         for lc in lcs:   
-            startLC = process_time_ns() # timer()
+            startLC = perf_counter_ns()
             m.update()
             startNumConstrs = m.NumConstrs
             
@@ -714,12 +691,12 @@ def addLogicalConstrains(self, m, dn, lcs, p, key = None):
                 self.myLogger.debug('Skipping not active Logical Constraint %s(%s) - %s'%(lc.lcName, lc, [str(e) for e in lc.e]))
                 continue
 
-            result = self.__constructLogicalConstrains(lc, self.myIlpBooleanProcessor, m, dn, p, key = key, headLC = True)
+            result = self.constructLogicalConstrains(lc, self.myIlpBooleanProcessor, m, dn, p, key = key, headLC = True)
             
             m.update()
             endNumConstrs = m.NumConstrs
             newNumConstrs = endNumConstrs - startNumConstrs
-            endLC = process_time_ns() # timer()
+            endLC = perf_counter_ns()
             elapsedInNsLC = endLC - startLC
             elapsedInMsLC = elapsedInNsLC/1000000
             
@@ -885,7 +862,7 @@ def getMLResult(self, dn, xPkey, e, p, loss = False, sample = False):
                 xP = torch.zeros(usedSampleSize, device = self.current_device, requires_grad=True)
         else:
             xV = dn.getAttribute(xPkey)
-            xEp = dn.getAttribute(xPkey).expand(usedSampleSize, len(xV))
+            xEp = dn.getAttribute(xPkey).expand(usedSampleSize, len(xV.squeeze(0)))
             xP = xEp[:,e[1]]
           
         if sampleSize > -1: 
@@ -960,7 +937,7 @@ def __addLossTovDns(self, loss, vDns):
         else:
             return vDns
     
-    def __constructLogicalConstrains(self, lc, booleanProcessor, m, dn, p, key = None, lcVariablesDns = None, headLC = False, loss = False, sample = False, vNo = None, verify=False):
+    def constructLogicalConstrains(self, lc, booleanProcessor, m, dn, p, key = None, lcVariablesDns = None, headLC = False, loss = False, sample = False, vNo = None, verify=False):
         if key == None:
             key = ""
             
@@ -1025,7 +1002,7 @@ def __constructLogicalConstrains(self, lc, booleanProcessor, m, dn, p, key = Non
 
                 elif isinstance(e, (Concept, tuple)): # -- Concept 
                     # -- Get dataNodes candidates 
-                    dnsList = getCandidates(dn, e, variable, lcVariablesDns, lc, self.myLogger)
+                    dnsList = getCandidates(dn, e, variable, lcVariablesDns, lc, self.myLogger, integrate = integrate)
                     lcVariablesDns[variableName] = dnsList
                                 
                     if isinstance(lc, CandidateSelection):
@@ -1129,7 +1106,7 @@ def __constructLogicalConstrains(self, lc, booleanProcessor, m, dn, p, key = Non
                 if isinstance(e, LcElement):
 
                     if isinstance(e, CandidateSelection): # -- nested LogicalConstrain - process recursively 
-                        lcVariablesDnsNew = self.__constructLogicalConstrains(e, booleanProcessor, m, dn, p, key = key, 
+                        lcVariablesDnsNew = self.constructLogicalConstrains(e, booleanProcessor, m, dn, p, key = key, 
                                                                      lcVariablesDns = lcVariablesDns, headLC = False, loss = loss, sample = sample, vNo=vNo, verify=verify)
                          
                         lcVariablesDns = lcVariablesDnsNew
@@ -1149,13 +1126,13 @@ def __constructLogicalConstrains(self, lc, booleanProcessor, m, dn, p, key = Non
                         self.myLogger.info('Processing Nested %s(%s) - %s'%(e.lcName, e, e.strEs()))
 
                         if sample:
-                            vDns, sampleInfoLC, lcVariablesLC = self.__constructLogicalConstrains(e, booleanProcessor, m, dn, p, key = key, 
+                            vDns, sampleInfoLC, lcVariablesLC = self.constructLogicalConstrains(e, booleanProcessor, m, dn, p, key = key, 
                                                                                    lcVariablesDns = lcVariablesDns, headLC = False, loss = loss, sample = sample, vNo=vNo, verify=verify)
                             sampleInfo = {**sampleInfo, **sampleInfoLC} # sampleInfo|sampleInfoLC in python 9
                             
                             lcVariablesSet = {**lcVariablesSet, **lcVariablesLC}
                         else:
-                            vDns = self.__constructLogicalConstrains(e, booleanProcessor, m, dn, p, key = key, 
+                            vDns = self.constructLogicalConstrains(e, booleanProcessor, m, dn, p, key = key, 
                                                                      lcVariablesDns = lcVariablesDns, headLC = False, loss = loss, sample = sample, vNo=vNo, verify=verify)
                             
                             vDns = self.__addLossTovDns(loss, vDns)
@@ -1214,7 +1191,7 @@ def __constructLogicalConstrains(self, lc, booleanProcessor, m, dn, p, key = Non
     # ---------------
                 
     # -- Main method of the solver - creating ILP constraints plus objective, invoking the ILP solver and returning the result of the ILP solver classification  
-    def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00001, minimizeObjective = False, ignorePinLCs = False):
+    def calculateILPSelection(self, dn, *conceptsRelations, key = ("local" , "softmax"), fun=None, epsilon = 0.00001, minimizeObjective = False, ignorePinLCs = False):
         if self.ilpSolver == None:
             self.myLogger.warning('ILP solver not provided - returning')
             self.myLoggerTime.warning('ILP solver not provided - returning')
@@ -1223,22 +1200,22 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
         
         self.current_device = dn.current_device
         
-        self.myLogger.info('Calculating ILP Inferencing ')
-        self.myLoggerTime.info('Calculating ILP Inferencing ')
-
-        start = process_time() # timer()
+        self.myLogger.info('Calculating ILP Inference ')
+        self.myLoggerTime.info('Calculating ILP Inference ')
+        start = perf_counter()
 
         gurobiEnv = Env("", empty=True)
         gurobiEnv.setParam('OutputFlag', 0)
         gurobiEnv.start()  
         
+        # Check if existing ILP model can be reuse without recreating ILP constraints
         try:
             # Find count of instance in each concept 
             ilpVarCount = self.countLCVariables(dn, *conceptsRelations)
 
             reusingModel = False
             if self.reuse_model:
-                # Find it there is saved model with this count of instances per concept
+                # Find it there is saved ILP model with this count of instances per concept
                 for modelDec in self.model:
                     if ilpVarCount == modelDec[0]:
                         m = modelDec[1]
@@ -1248,16 +1225,23 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
                     
             if not reusingModel:
                 # If not reusing the model or if the right model was yet saved - create new Gurabi model
-                m = Model("decideOnClassificationResult" + str(start), gurobiEnv)
+                if dn.gurobiModel == None:
+                    m = Model("decideOnClassificationResult" + str(start), gurobiEnv)
+                    dn.gurobiModel = m
+                else:
+                    m = dn.gurobiModel
+                    m.reset()
+                    mConstr = m.getConstrs()
+                    m.remove(mConstr)
+                    m.update()
+                    
                 m.params.outputflag = 0
                 x = OrderedDict()
                 
-                # Create ILP Variables for concepts and objective
-                Q = self.createILPVariables(m, x, dn, *conceptsRelations, dnFun = self.__getProbability, fun=fun, epsilon = epsilon)
-            else:
-                Q = self.createObjective(x, dn, *conceptsRelations, dnFun = self.__getProbability, fun=fun, epsilon = epsilon)
+            # Handle ILP Variables for concepts and objective
+            Q = self.createILPVariables(m, x, dn, *conceptsRelations, key=key, fun=fun, epsilon = epsilon)
             
-            endVariableInit = process_time() # timer()
+            endVariableInit = perf_counter()
             elapsedVariablesInMs = (endVariableInit - start) *1000
             self.myLoggerTime.info('ILP Variables Init - time: %ims'%(elapsedVariablesInMs))
 
@@ -1270,7 +1254,7 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
             else:
                 self.myLoggerTime.info('Reusing ILP Model - LCs already present in the model')
 
-            endGraphAndOntologyConstraints = process_time() # timer()
+            endGraphAndOntologyConstraints = perf_counter()
             elapsedGandOConstraintsInMs = (endGraphAndOntologyConstraints - endVariableInit) * 1000
             self.myLoggerTime.info('ILP Graph and Ontology Constraints - time: %ims'%(elapsedGandOConstraintsInMs))
             
@@ -1309,9 +1293,9 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
             lcP = OrderedDict(sorted(_lcP.items(), key=lambda t: t[0], reverse = True))
             for p in lcP:
                 self.myLogger.info('Found %i logical constraints with p %i - %s\n'%(len(lcP[p]),p,lcP[p]))
-                self.myLoggerTime.info('Starting ILP inferencing - Found %i logical constraints'%(len(lcP[p])))
+                self.myLoggerTime.info('Starting ILP interference - Found %i logical constraints'%(len(lcP[p])))
             
-            # Search through set of logical constraints for subset satisfying and the mmax/min calculated objective value
+            # Search through set of logical constraints for subset satisfying and the max/min calculated objective value
             lcRun = {} # Keeps information about subsequent model runs
             ps = [] # List with processed p 
             
@@ -1324,7 +1308,7 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
                     xP = {}
                     lckey = "/ILP/xP"
                    
-                    pStart= process_time() # timer()
+                    pStart= perf_counter()
                     
                     for _x in x:
                         # Map variables to the new copy model
@@ -1353,7 +1337,7 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
                                 
                             dns[0].attributes[xPkey][p][_x[3]] = mP.getVarByName(x[_x].VarName)
                             
-                            pEnd= process_time() # timer()
+                            pEnd = perf_counter()
                             self.myLoggerTime.info('ILP Model init for p %i - time: %ims'%(p, (pEnd - pStart)*1000))
                 else:
                     mP = m
@@ -1371,7 +1355,7 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
     
                 # ----------- Add LC constraints to the ILP model
                 
-                endLogicalConstraintsPrep = process_time() # timer()
+                endLogicalConstraintsPrep = perf_counter()
                 elapsedLogicalConstraintsPrepInMs = (endLogicalConstraintsPrep - endGraphAndOntologyConstraints)*1000
                 self.myLoggerTime.info('ILP Logical Constraints Preprocessing - time: %ims'%(elapsedLogicalConstraintsPrepInMs))
                 
@@ -1383,24 +1367,27 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
                         self.model.append((ilpVarCount, mP, xP))
                         import sys
                         memoryUsage = sys.getsizeof(mP)
-                        self.myLoggerTime.info('ILP Logical Constraints Preprocessing - memory use by saved Gurobi models: %f'%(memoryUsage))
+                        # Convert bytes to kilobytes
+                        memoryUsage_kB = memoryUsage / 1024
+                        self.myLoggerTime.info(f'ILP Logical Constraints Preprocessing - memory use by saved Gurobi models: {memoryUsage_kB:.2f} kB')
+
 
                 self.myLogger.info('Optimizing model for LCs with probabilities %s with %i ILP variables and %i ILP constraints'%(p,mP.NumVars,mP.NumConstrs))
                 self.myLoggerTime.info('Optimizing model for LCs with probabilities %s with %i ILP variables and %i ILP constraints'%(p,mP.NumVars,mP.NumConstrs))
 
-                endLogicalConstraints = process_time() # timer()
+                endLogicalConstraints = perf_counter()
                 elapsedLogicalConstraintsInMs = (endLogicalConstraints - endLogicalConstraintsPrep) *1000
                 self.myLoggerTime.info('ILP Logical Constraints - time: %ims'%(elapsedLogicalConstraintsInMs))
             
                 #mP.update()
                 #mP.display() 
-                startOptimize = process_time() # timer()
+                startOptimize = perf_counter()
 
                 # ----------- Run ILP model - Find solution 
                 mP.optimize()
                 mP.update()
                                 
-                endOptimize = process_time() # timer()
+                endOptimize = perf_counter()
                 elapsedOptimizeInMs = (endOptimize - startOptimize) * 1000
     
                 # ----------- Check model run result
@@ -1483,16 +1470,33 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
                     c_root_dns = dn.findDatanodes(select = c_root)
                     
                     ILPkey = '<' + c[0].name + '>/ILP'
+                    
                     xkey = ILPkey + '/x'
                     xPkey = ILPkey + '/xP'
                     xNotPkey = ILPkey + '/notxP'
+                    
+                    ilpTensor = None
+                    if getDnSkeletonMode() and "variableSet" in dn.attributes:
+                        ilpKeyInVariableSet = c_root.name + "/<" + c[0].name +">" + "/ILP"
+                        
+                        if ilpKeyInVariableSet in dn.attributes["variableSet"]:
+                            ilpTensor = dn.attributes["variableSet"][ilpKeyInVariableSet]
+                        else:
+                            ilpTensor = torch.zeros([len(c_root_dns), c[3]], dtype=torch.float, device=self.current_device)
                    
-                    for cDn in c_root_dns:
+                    for i, cDn in enumerate(c_root_dns):
                         dnAtt = cDn.getAttributes()
                         
                         if xkey not in dnAtt and xPkey not in dnAtt:
                             if xVars[xVarsIndex] == None or not reusingModel:
-                                dnAtt[ILPkey] = torch.tensor([float("nan")], device=self.current_device)
+                                
+                                if ilpTensor is not None:
+                                   ilpTensor[i][index] = float("nan")
+                                else:
+                                    if ILPkey not in dnAtt:
+                                        dnAtt[ILPkey] = torch.empty(c[3], dtype=torch.float)
+                                    
+                                    dnAtt[ILPkey][index] = float("nan")
                                 
                                 # Update index for x variables 
                                 if c[2] is None:
@@ -1534,22 +1538,30 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
                         elif solution == 1: 
                             solution = 1
 
-                        if ILPkey not in dnAtt:
+                        if ilpTensor is None and ILPkey not in dnAtt:
                             dnAtt[ILPkey] = torch.empty(c[3], dtype=torch.float)
                         
                         if xkey not in dnAtt:
                             dnAtt[xkey] = torch.empty(c[3], dtype=torch.float)
                        
-                        dnAtt[ILPkey][index] = solution
-                        if pUsed: 
+                        if ilpTensor is not None:
+                           ilpTensor[i][index] = solution
+                        else:
+                            dnAtt[ILPkey][index] = solution
+                            
+                        if pUsed: # Set main ILP variable x based on max P ILP variable x
                             dnAtt[xkey][index] = dnAtt[xPkey][maxP][index]
-                            if xNotPkey in dnAtt:
+                            if xNotPkey in dnAtt: # do this for not x as well
                                 dnAtt[xNotPkey][index] = dnAtt[xNotPkey][maxP][index]
 
-                        ILPV = dnAtt[ILPkey][index]
-                        if ILPV == 1:
-                            self.myLogger.info('\"%s\" is \"%s\"'%(cDn,c[1]))
+                        
+                        if solution == 1:
+                            self.myLogger.info('\"%s\" is \"%s\"'%(cDn, c[1]))
                             #self.myLoggerTime.info('\"%s\" is \"%s\"'%(cDn,c[1]))
+                    
+                    if ilpTensor is not None:
+                        dn.attributes["variableSet"][ilpKeyInVariableSet] = ilpTensor
+
             else:
                 pass
                                        
@@ -1559,25 +1571,23 @@ def calculateILPSelection(self, dn, *conceptsRelations, fun=None, epsilon = 0.00
             
             raise
            
-        endResultPrep = process_time() # timer()
+        endResultPrep = perf_counter()
         elapsedResultPrepInMs = (endResultPrep - endOptimize) *1000
         self.myLoggerTime.info('ILP Preparing Return Results - time: %ims'%(elapsedResultPrepInMs))
             
         self.myLogger.info('')
 
-        end = process_time() # timer()
+        end = perf_counter()
         elapsedInS = end - start
-        
         if elapsedInS > 1:
-            self.myLogger.info('End ILP Inferencing - total time: %fs'%(elapsedInS))
-            self.myLoggerTime.info('End ILP Inferencing - total time: %fs'%(elapsedInS))
+            self.myLogger.info('End ILP Inference - internal total time: %fs'%(elapsedInS))
+            self.myLoggerTime.info('End ILP Inference - internal total time: %fs'%(elapsedInS))
         else:
             elapsedInMs = elapsedInS *1000
-            self.myLogger.info('End ILP Inferencing - total time: %ims'%(elapsedInMs))
-            self.myLoggerTime.info('End ILP Inferencing - total time: %ims'%(elapsedInMs))
+            self.myLogger.info('End ILP Inference - internal total time: %ims'%(elapsedInMs))
+            self.myLoggerTime.info('End ILP Inference - internal total time: %ims'%(elapsedInMs))
             
         self.myLogger.info('')
-        self.myLoggerTime.info('')
         
         # ----------- Return
         return
@@ -1675,7 +1685,7 @@ def generateSemanticSample(self, rootDn, conceptsRelations):
         productSize = 1
         productArgs = []
         for currentConceptRelation in conceptsRelations:
-            currentConceptName = currentConceptRelation[0].name
+            currentConceptName = self.getConceptName(currentConceptRelation)
             
             if currentConceptName not in masterConcepts:
                 masterConcepts[currentConceptName] = OrderedDict()
@@ -1689,7 +1699,7 @@ def generateSemanticSample(self, rootDn, conceptsRelations):
                 masterConcepts[currentConceptName]["dns"] = dns
                 
                 conceptRange = 2
-                if currentConceptRelation[2] is not None:
+                if self.conceptIsMultiClass(currentConceptRelation):
                     conceptRange = currentConceptRelation[3]
                 masterConcepts[currentConceptName]["range"] = [x for x in range(conceptRange)]
                 
@@ -1729,10 +1739,10 @@ def generateSemanticSample(self, rootDn, conceptsRelations):
                         
                     if isFiexd != None:
                         if isFiexd == 1:
-                            dn.getAttributes()[mConceptInfo['xkey']][sampleSize][i] = torch.ones(productSize, device = self.current_device)
+                            dn.getAttributes()[mConceptInfo['xkey']][sampleSize][i] = torch.ones((productSize,), device = self.current_device)
                             continue
                         
-                    dn.getAttributes()[mConceptInfo['xkey']][sampleSize][i] = torch.zeros(productSize, device = self.current_device)
+                    dn.getAttributes()[mConceptInfo['xkey']][sampleSize][i] = torch.zeros((productSize,), device = self.current_device)
              
         from itertools import product
         for j, p in enumerate(product(*productArgs, repeat=1)):
@@ -1752,8 +1762,8 @@ def generateSemanticSample(self, rootDn, conceptsRelations):
         return productSize
         
     # -- Calculated loss values for logical constraints
-    def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleGlobalLoss = False, conceptsRelations = None):
-        start = process_time() # timer()
+    def calculateLcLoss(self, dn, tnorm='L',counting_tnorm=None, sample = False, sampleSize = 0, sampleGlobalLoss = False, conceptsRelations = None):
+        start = perf_counter()
 
         m = None 
         p = 0
@@ -1775,6 +1785,8 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
         else:
             myBooleanMethods = self.myLcLossBooleanMethods
             self.myLcLossBooleanMethods.setTNorm(tnorm)
+            if counting_tnorm:
+                self.myLcLossBooleanMethods.setCountingTNorm(counting_tnorm)
             
             self.myLogger.info('Calculating loss ')
             self.myLoggerTime.info('Calculating loss ')
@@ -1787,7 +1799,7 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
         lcLosses = {}
         for graph in self.myGraph: # Loop through graphs
             for _, lc in graph.logicalConstrains.items(): # loop trough lcs in the graph
-                startLC = process_time_ns() # timer()
+                startLC = perf_counter_ns()
 
                 if not lc.headLC or not lc.active: # Process only active and head lcs
                     continue
@@ -1808,7 +1820,7 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
                 if sample:
                     # lossList will contain boolean results for lc evaluation for the given sample element
                     # sampleInfo - will contain list of variable exiting in the given lc with their sample and probabilities
-                    lossList, sampleInfo, inputLc = self.__constructLogicalConstrains(lc, myBooleanMethods, m, dn, p, key = key, headLC = True, loss = True, sample = sample)
+                    lossList, sampleInfo, inputLc = self.constructLogicalConstrains(lc, myBooleanMethods, m, dn, p, key = key, headLC = True, loss = True, sample = sample)
                     current_lcLosses['lossList'] = lossList
                     current_lcLosses['sampleInfo'] = sampleInfo
                     current_lcLosses['input'] = inputLc
@@ -1824,17 +1836,17 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
                         current_lcLosses['lossRate'].append(liList)
                 else:
                     # lossList will contain float result for lc loss calculation
-                    lossList = self.__constructLogicalConstrains(lc, myBooleanMethods, m, dn, p, key = key, headLC = True, loss = True, sample = sample)
+                    lossList = self.constructLogicalConstrains(lc, myBooleanMethods, m, dn, p, key = key, headLC = True, loss = True, sample = sample)
                     current_lcLosses['lossList'] = lossList
                              
-                endLC = process_time_ns() # timer()
+                endLC = perf_counter_ns()
                 elapsedInNsLC = endLC - startLC
                 elapsedInMsLC = elapsedInNsLC/1000000
                 current_lcLosses['elapsedInMsLC'] = elapsedInMsLC
                             
         if not sample: # Loss value
             for currentLcName in lcLosses:
-                startLC = process_time_ns() # timer()
+                startLC = perf_counter_ns()
 
                 current_lcLosses = lcLosses[currentLcName]
                 lossList = current_lcLosses['lossList']
@@ -1869,19 +1881,23 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
                 else:
                     current_lcLosses['loss'] = None
 
-                endLC = process_time_ns() # timer()
+                endLC = perf_counter_ns()
                 elapsedInNsLC = endLC - startLC
                 elapsedInMsLC = elapsedInNsLC/1000000
                 current_lcLosses['elapsedInMsLC'] += elapsedInMsLC
 
-                self.myLoggerTime.info('Processing time for %s with %i entries is: %ims'%(currentLcName, len(lossList),  current_lcLosses['elapsedInMsLC']))
+                if lossList is not None:
+                    self.myLoggerTime.info('Processing time for %s with %i entries is: %ims'%(currentLcName, len(lossList),  current_lcLosses['elapsedInMsLC']))
+                else:
+                    self.myLoggerTime.info('Processing time for %s with %i entries is: %ims'%(currentLcName, 0,  current_lcLosses['elapsedInMsLC']))
+
                 [h.flush() for h in self.myLoggerTime.handlers]
             
         else: # -----------Sample
             globalSuccesses = torch.ones(sampleSize, device = self.current_device)
             
             for currentLcName in lcLosses:
-                startLC = process_time_ns() # timer()
+                startLC = perf_counter_ns()
 
                 current_lcLosses = lcLosses[currentLcName]
                 lossList = current_lcLosses['lossList']
@@ -1943,7 +1959,7 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
                 current_lcLosses['lcVariables'] = lcVariables
                 current_lcLosses['countSuccesses'] = countSuccesses
 
-                endLC = process_time_ns() # timer()
+                endLC = perf_counter_ns()
                 elapsedInNsLC = endLC - startLC
                 elapsedInMsLC = elapsedInNsLC/1000000
                 current_lcLosses['elapsedInMsLC'] += elapsedInMsLC
@@ -1956,7 +1972,7 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
                 if currentLcName in ["globalSuccessCounter", "globalSuccesses"]:
                     continue
                 
-                startLC = process_time_ns() # timer()
+                startLC = perf_counter_ns()
 
                 current_lcLosses = lcLosses[currentLcName]
                 lossList = current_lcLosses['lossList']
@@ -2011,7 +2027,7 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
                     current_lcLosses['lcVariables'].append(lcVariables)
             
                 # Calculate processing time
-                endLC = process_time_ns() # timer()
+                endLC = perf_counter_ns()
                 elapsedInNsLC = endLC - startLC
                 elapsedInMsLC = elapsedInNsLC/1000000
                 current_lcLosses['elapsedInMsLC'] += elapsedInMsLC
@@ -2031,20 +2047,18 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
         self.myLogger.info('Processed %i logical constraints'%(lcCounter))
         self.myLoggerTime.info('Processed %i logical constraints'%(lcCounter))
             
-        end = process_time() # timer()
+        end = perf_counter()
         elapsedInS = end - start
         
         if elapsedInS > 1:
-            self.myLogger.info('End of Loss Calculation - total time: %fs'%(elapsedInS))
-            self.myLoggerTime.info('End of Loss Calculation - total time: %fs'%(elapsedInS))
+            self.myLogger.info('End of Loss Calculation - total internl time: %fs'%(elapsedInS))
+            self.myLoggerTime.info('End of Loss Calculation - total internl time: %fs'%(elapsedInS))
         else:
             elapsedInMs = (end - start) *1000
-            self.myLogger.info('End of Loss Calculation - total time: %ims'%(elapsedInMs))
-            self.myLoggerTime.info('End of Loss Calculation - total time: %ims'%(elapsedInMs))
+            self.myLogger.info('End of Loss Calculation - total internl time: %ims'%(elapsedInMs))
+            self.myLoggerTime.info('End of Loss Calculation - total internl time: %ims'%(elapsedInMs))
             
-        
         self.myLogger.info('')
-        self.myLoggerTime.info('')
 
         [h.flush() for h in self.myLoggerTime.handlers]
             
@@ -2052,7 +2066,7 @@ def calculateLcLoss(self, dn, tnorm='L', sample = False, sampleSize = 0, sampleG
     
     # -- Calculated loss values for logical constraints
     def verifyResultsLC(self, dn, key = "/argmax"):
-        start = process_time() # timer()
+        start = perf_counter()
 
         m = None 
         p = 0
@@ -2067,7 +2081,7 @@ def verifyResultsLC(self, dn, key = "/argmax"):
         lcVerifyResult = OrderedDict()
         for graph in self.myGraph: # Loop through graphs
             for _, lc in graph.logicalConstrains.items(): # loop trough lcs in the graph
-                startLC = process_time_ns() # timer()
+                startLC = perf_counter_ns()
 
                 if not lc.headLC or not lc.active: # Process only active and head lcs
                     continue
@@ -2085,7 +2099,7 @@ def verifyResultsLC(self, dn, key = "/argmax"):
                 current_verifyResult = lcVerifyResult[lcName]
                 
                 # verifyList will contain result for the lc verification
-                verifyList, lcVariables = self.__constructLogicalConstrains(lc, myBooleanMethods, m, dn, p, key = key, headLC = True, verify=True)
+                verifyList, lcVariables = self.constructLogicalConstrains(lc, myBooleanMethods, m, dn, p, key = key, headLC = True, verify=True)
                 current_verifyResult['verifyList'] = verifyList
                 
                 verifyListLen = 0
@@ -2097,7 +2111,7 @@ def verifyResultsLC(self, dn, key = "/argmax"):
                 if verifyListLen:
                     current_verifyResult['satisfied'] = (verifyListSatisfied / verifyListLen) * 100
                 else:
-                    current_verifyResult['satisfied'] = float("nan")
+                    current_verifyResult['satisfied'] = 0 # float("nan")
                     
                 # If  this if logical constraints
                 if type(lc) is ifL or type(lc) is forAllL: # if LC
@@ -2135,9 +2149,9 @@ def verifyResultsLC(self, dn, key = "/argmax"):
                     if ifVerifyListLen:
                         current_verifyResult['ifSatisfied'] = (ifVerifyListSatisfied / ifVerifyListLen) *100
                     else:
-                        current_verifyResult['ifSatisfied'] = float("nan")
+                        current_verifyResult['ifSatisfied'] = 0 #float("nan")
 
-                endLC = process_time_ns() # timer()
+                endLC = perf_counter_ns()
                 elapsedInNsLC = endLC - startLC
                 elapsedInMsLC = elapsedInNsLC/1000000
                 current_verifyResult['elapsedInMsLC'] = elapsedInMsLC
@@ -2145,7 +2159,7 @@ def verifyResultsLC(self, dn, key = "/argmax"):
         self.myLogger.info('Processed %i logical constraints'%(lcCounter))
         self.myLoggerTime.info('Processed %i logical constraints'%(lcCounter))
             
-        end = process_time() # timer()
+        end = perf_counter()
         elapsedInS = end - start
         
         if elapsedInS > 1:
diff --git a/domiknows/solver/ilpOntSolverFactory.py b/domiknows/solver/ilpOntSolverFactory.py
index 21c3ba51c..c16c25eac 100644
--- a/domiknows/solver/ilpOntSolverFactory.py
+++ b/domiknows/solver/ilpOntSolverFactory.py
@@ -59,12 +59,6 @@ def getOntSolverInstance(cls, graph, *SupplementalClasses, _ilpConfig=ilpConfig,
                 else:
                     from .gurobiILPOntSolver1 import gurobiILPOntSolver
                 SolverClass = cls.getClass(gurobiILPOntSolver, *SupplementalClasses)
-            elif _ilpConfig['ilpSolver'] == "GEKKO":
-                if __package__ is None or __package__ == '':
-                    from domiknows.solver.gekkoILPOntSolver import gekkoILPOntSolver
-                else:
-                    from .gekkoILPOntSolver import gekkoILPOntSolver
-                SolverClass = cls.getClass(gekkoILPOntSolver, *SupplementalClasses)
             elif _ilpConfig['ilpSolver'] == "mini_prob_debug":
                 if __package__ is None or __package__ == '':
                     from domiknows.solver.mini_solver_debug import MiniProbSolverDebug
diff --git a/domiknows/solver/lcLossBooleanMethods.py b/domiknows/solver/lcLossBooleanMethods.py
index 20f5c6d74..df1c56cc3 100644
--- a/domiknows/solver/lcLossBooleanMethods.py
+++ b/domiknows/solver/lcLossBooleanMethods.py
@@ -10,7 +10,8 @@ class lcLossBooleanMethods(ilpBooleanProcessor):
     
     def __init__(self, _ildConfig = ilpConfig) -> None:
         super().__init__()
-        self.tnorm = DataNode.tnormsDefault
+        self.tnorm = 'P'
+        self.counting_tnorm = None
         self.grad = True
         
         self.myLogger = logging.getLogger(ilpConfig['log_name'])
@@ -27,6 +28,22 @@ def setTNorm(self, tnorm='L'):
             raise Exception('Unknown type of t-norms formulation - %s'%(tnorm))
 
         self.tnorm = tnorm
+
+    def setCountingTNorm(self, tnorm='L'):
+        if tnorm =='L':
+            if self.ifLog: self.myLogger.info("Using Lukasiewicz t-norms Formulation")
+        elif tnorm =='G':
+            if self.ifLog: self.myLogger.info("Using Godel t-norms Formulation")
+        elif tnorm =='P':
+            if self.ifLog: self.myLogger.info("Using Product t-norms Formulation")
+        elif tnorm =='SP':
+            if self.ifLog: self.myLogger.info("Using Simplified Product t-norms Formulation")
+        #elif tnorm =='LSE':
+        #    if self.ifLog: self.myLogger.info("Using Log Sum Exp Formulation")
+        else:
+            raise Exception('Unknown type of t-norms formulation - %s'%(tnorm))
+
+        self.counting_tnorm = tnorm
         
     def _isTensor(self, v):
         if v is None:
@@ -366,10 +383,54 @@ def epqVar(self, _, var1, var2, onlyConstrains = False):
             return epqLoss
         else:
             return epqSuccess
-     
+
+    def calc_probabilities(self, t, s):
+
+        n = len(t)
+        dp = torch.zeros(s + 1, device=self.current_device, dtype=torch.float64)
+        dp[0] = 1.0
+        dp.requires_grad_()
+        for i in range(n):
+            dp_new = dp.clone()
+            dp_new[1:min(s, i + 1) + 1] = dp[1:min(s, i + 1) + 1] * (1 - t[i]) + dp[:min(s, i + 1)] * t[i]
+            dp_new[0] = dp[0] * (1 - t[i])
+            dp = dp_new
+        return dp
+
     def countVar(self, _, *var, onlyConstrains = False, limitOp = '==', limit = 1, logicMethodName = "COUNT"):
         logicMethodName = "COUNT"
-        
+
+        method=self.counting_tnorm if self.counting_tnorm else self.tnorm
+        #if method=="LSE": # log sum exp
+        #    exists_at_least_one = lambda t, beta=100.0: torch.clamp(-torch.log((1 / beta) * torch.log(torch.sum(torch.exp(beta * t)))), min=0,max=1)
+        #    exists_at_least_s = lambda t, s, beta=10.0: torch.clamp(torch.relu(s - torch.sum(torch.sigmoid(beta * (t - 0.5)))),max=1)
+        #    exists_at_most_s = lambda t, s, beta=10.0: torch.clamp(torch.relu(torch.sum(torch.sigmoid(beta * (t - 0.5))) - s),max=1)
+        #    exists_exactly_s = lambda t, s, beta=10.0: torch.clamp(torch.abs(s - torch.sum(torch.sigmoid(beta * (t - 0.5)))),max=1)
+        if method=="G": # Godel logic
+            exists_at_least_one = lambda t: 1 - torch.max(t)
+            exists_at_least_s = lambda t, s: 1- torch.min(torch.sort(t, descending=True)[0][:s])
+            exists_at_most_s = lambda t, s: 1 - torch.min(torch.sort(1 - t, descending=True)[0][:len(t)-s])
+            exists_exactly_s = lambda t, s: 1 - torch.min(torch.min(torch.sort(t, descending=True)[0][:s]) , torch.min(torch.sort(1 - t, descending=True)[0][:len(t)-s]))
+        elif method == "L": # Łukasiewicz logic
+            exists_at_least_one = lambda t: 1 - torch.min(torch.sum(t), torch.ones(1, device=self.current_device, requires_grad=True, dtype=torch.float64))
+            exists_at_least_s = lambda t, s: 1 - torch.max(torch.sum(torch.sort(t, descending=True)[0][:s])-(s-1), torch.zeros(1, device=self.current_device, requires_grad=True, dtype=torch.float64))
+            exists_at_most_s = lambda t, s: 1 - torch.max(torch.sum(torch.sort(1 - t, descending=True)[0][:len(t)-s])-(len(t)-s-1), torch.zeros(1, device=self.current_device, requires_grad=True, dtype=torch.float64))
+            exists_exactly_s = lambda t, s: 1 - torch.max(torch.sum(torch.sort(t, descending=True)[0][:s])-(s-1)+torch.sum(torch.sort(1 - t, descending=True)[0][:len(t)-s])-(len(t)-s-1), torch.zeros(1, device=self.current_device, requires_grad=True, dtype=torch.float64))
+
+        elif method == "P":  #  Product logic
+
+            exists_at_least_one = lambda t: torch.prod(1 - t)
+            exists_at_least_s = lambda t, s: 1 - torch.sum(self.calc_probabilities(t, len(t))[s:])
+            exists_at_most_s = lambda t, s: 1 - torch.sum(self.calc_probabilities(t, s))
+            exists_exactly_s = lambda t, s: 1 - self.calc_probabilities(t, s)[s]
+
+        else: # "SP" # Simplified product logic
+            exists_at_least_one = lambda t: torch.prod(1 - t)
+            exists_at_least_s = lambda t, s: 1 - torch.prod(torch.sort(t, descending=True)[0][:s])
+            exists_at_most_s = lambda t, s: 1 - torch.prod(torch.sort(1 - t, descending=True)[0][:len(t) - s])
+            exists_exactly_s = lambda t, s: 1 - self.calc_probabilities(t, s)[s]
+
+
         if self.ifLog: self.myLogger.debug("%s called with: %s"%(logicMethodName,var))
 
         var = self._fixVar(var)
@@ -382,8 +443,8 @@ def countVar(self, _, *var, onlyConstrains = False, limitOp = '==', limit = 1, l
         tOne = torch.ones(1, device=self.current_device, requires_grad=True, dtype=torch.float64)
         
         if  limitOp == '==': # == limit
-            countLoss = torch.minimum(torch.maximum(torch.abs(torch.sub(limit, varSum)), tZero), tOne) # min(max(abs(varSum - limit), 0), 1)
-
+            #countLoss = torch.minimum(torch.maximum(torch.abs(torch.sub(limit, varSum)), tZero), tOne) # min(max(abs(varSum - limit), 0), 1)
+            countLoss=exists_exactly_s(varSum, limit)
             if onlyConstrains:
                 return countLoss
             else:
@@ -391,11 +452,24 @@ def countVar(self, _, *var, onlyConstrains = False, limitOp = '==', limit = 1, l
                 return countSuccess
         else:
             if limitOp == '>=': # > limit
-                countSuccess = torch.minimum(torch.maximum(torch.sub(varSum, limit), tZero), tOne) # min(max(varSum - limit, 0), 1)
-                
+                #Existsl
+                if onlyConstrains:
+                    if limit ==1:return exists_at_least_one(varSum)
+                    else: return exists_at_least_s(varSum, limit)
+                else:
+                    if limit == 1:
+                        countSuccess = 1 - exists_at_least_one(varSum)
+                    else:
+                        countSuccess = 1 - exists_at_least_s(varSum, limit)
+
             elif limitOp == '<=': # < limit
-                countSuccess = torch.minimum(torch.maximum(torch.sub(limit, varSum), tZero), tOne) # min(max(limit - varSum, 0), 1)
-                    
+                #atmostL
+
+                if onlyConstrains:
+                    return exists_at_most_s(varSum, limit)
+                else:
+                    countSuccess = 1 - exists_at_most_s(varSum, limit)
+
             if onlyConstrains:
                 countLoss = torch.sub(tOne, countSuccess)
                 return countLoss
diff --git a/domiknows/solver/lcLossSampleBooleanMethods.py b/domiknows/solver/lcLossSampleBooleanMethods.py
index 2a99a1bef..3d1019db2 100644
--- a/domiknows/solver/lcLossSampleBooleanMethods.py
+++ b/domiknows/solver/lcLossSampleBooleanMethods.py
@@ -174,9 +174,9 @@ def epqVar(self, _, var1, var2, onlyConstrains = False):
             
             return epqLoss
         else:       
-            return epqSuccess     
-    
-    def countVar(self, _, *var, onlyConstrains = False, limitOp = '==', limit = 1, logicMethodName = "COUNT"):
+            return epqSuccess
+
+    def countVar(self, _, *var, onlyConstrains=False, limitOp='==', limit=1, logicMethodName="COUNT"):
         # -- Consider None
         fixedVar = []
         for v in var:
@@ -190,35 +190,35 @@ def countVar(self, _, *var, onlyConstrains = False, limitOp = '==', limit = 1, l
                 elif limitOp == '==':
                     fixedVar.append(torch.zeros([self.sampleSize], device=self.current_device))
         # --
-         
-        limitTensor = torch.full([self.sampleSize], limit, device = self.current_device)
-       
-        # Calculate sum 
+        # V = 100,
+        limitTensor = torch.full([self.sampleSize], limit, device=self.current_device)
+
+        # Calculate sum
         varSum = torch.zeros([self.sampleSize], device=self.current_device)
         if fixedVar:
-            varSum = fixedVar[0].int()
-            
+            varSum = fixedVar[0].clone()
+
         for i in range(1, len(fixedVar)):
-            varSum.add_(fixedVar[i].int())
+            varSum.add_(fixedVar[i])
 
         # Check condition
         if limitOp == '>=':
-            #if varSum >= limit:
+            # if varSum >= limit:
             countSuccess = torch.ge(varSum, limitTensor)
         elif limitOp == '<=':
-            #if varSum <= limit:
+            # if varSum <= limit:
             countSuccess = torch.le(varSum, limitTensor)
         elif limitOp == '==':
-            #if varSum == limit:
+            # if varSum == limit:
             countSuccess = torch.eq(varSum, limitTensor)
-                
+
         if onlyConstrains:
             countLoss = torch.logical_not(countSuccess)
-            
+
             return countLoss
-        else:       
-            return countSuccess    
-        
+        else:
+            return countSuccess
+
     def fixedVar(self, _, var, onlyConstrains = False):
         if self.ifNone([var]):
             return None
diff --git a/domiknows/utils.py b/domiknows/utils.py
index 0b0c3fa67..bf0da1c17 100644
--- a/domiknows/utils.py
+++ b/domiknows/utils.py
@@ -130,13 +130,25 @@ def getReuseModel():
     return reuseModel
 
 dnSkeletonMode = False
-def setDnSkeletonMode(dnSkeleton):
+dnSkeletonModeFull = False
+def setDnSkeletonMode(dnSkeleton, full=False):
+    if not isinstance(dnSkeleton, bool):
+        dnSkeleton = False
+        return
+    
     global dnSkeletonMode
     dnSkeletonMode = dnSkeleton
+    if full:
+        global dnSkeletonModeFull
+        dnSkeletonModeFull = True
     
 def getDnSkeletonMode():
     global dnSkeletonMode
     return dnSkeletonMode
+
+def getDnSkeletonModeFull():
+    global dnSkeletonModeFull
+    return dnSkeletonModeFull
     
 def printablesize(ni):
     if hasattr(ni, 'shape'):
@@ -168,16 +180,16 @@ def enum(inst, cls=None, offset=0):
         enum = inst.items()
     elif isinstance(inst, dict):
         enum = inst.items()
-        if inst:  # if dict not empty
-            # NB: stacklevel
-            #     1 - utils.enum()
-            #     2 - concept.Concept.relation_type.<local>.update.<local>.create()
-            #     3 - concept.Concept.__getattr__.<local>.handle()
-            #     4 - __main__()
-            warnings.warn('Please use OrderedDict rather than dict to prevent unpredictable order of arguments.' +
-                          'For this instance, {} is used.'
-                          .format(OrderedDict((k, v.name) for k, v in inst.items())),
-                          stacklevel=4)
+        # if inst:  # if dict not empty
+        #     # NB: stacklevel
+        #     #     1 - utils.enum()
+        #     #     2 - concept.Concept.relation_type.<local>.update.<local>.create()
+        #     #     3 - concept.Concept.__getattr__.<local>.handle()
+        #     #     4 - __main__()
+        #     warnings.warn('Please use OrderedDict rather than dict to prevent unpredictable order of arguments.' +
+        #                   'For this instance, {} is used.'
+        #                   .format(OrderedDict((k, v.name) for k, v in inst.items())),
+        #                   stacklevel=4)
     elif isinstance(inst, Iterable):
         enum = enumerate(inst, offset)
     else:
diff --git a/requirements.txt b/requirements.txt
index 573fb6617..d11a6fa39 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
 acls==1.0.2
-Owlready2==0.30
+Owlready2
 gurobipy
 pandas>=1.1.5
 torch>=1.8.1
@@ -7,6 +7,6 @@ transformers==4.6
 ordered-set
 setuptools
 graphviz
-pymongo[tls]
 dnspython
 scikit-learn
+spacy
diff --git a/setup.py b/setup.py
index 8430d47a3..1fbb4e994 100644
--- a/setup.py
+++ b/setup.py
@@ -22,9 +22,9 @@
     long_description_content_type='text/markdown',
     
     url='https://github.com/HLR/DomiKnowS',
-    author='Andrzej Uszok',
-    author_email='auszok@ihmc.org',
-
+    author='Andrzej Uszok, Parisa Kordjamshidi',
+    author_email='auszok@ihmc.org, kordjams@msu.edu',
+    
     packages=find_packages(include=['domiknows', 'domiknows.*', 'README.md']),
     
     install_requires=[


Test patch:
diff --git a/test_regr/dummy_datanode/graph.py b/test_regr/dummy_datanode/graph.py
new file mode 100644
index 000000000..00e960291
--- /dev/null
+++ b/test_regr/dummy_datanode/graph.py
@@ -0,0 +1,71 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL
+
+Graph.clear()
+Concept.clear()
+Relation.clear()
+
+with Graph('global') as graph:
+    with Graph('linguistic') as ling_graph:
+        char = Concept(name='char')
+        word = Concept(name='word')
+        phrase = Concept(name='phrase')
+        sentence = Concept(name='sentence')
+        (rel_sentence_contains_word,) = sentence.contains(word)
+        (rel_sentence_contains_phrase,) = sentence.contains(phrase)
+        (rel_phrase_contains_word,) = phrase.contains(word)
+        (rel_word_contains_char,) = word.contains(char)
+
+        pair = Concept(name='pair')
+        (rel_pair_word1, rel_pair_word2, ) = pair.has_a(arg1=word, arg2=word)
+
+    with Graph('application', auto_constraint=True) as app_graph:
+        people = word(name='people')
+        organization = word(name='organization')
+        location = word(name='location')
+        other = word(name='other')
+        o = word(name='O')
+
+        #disjoint(people, organization, location, other, o)
+
+        # LC0
+        nandL(people, organization, active = True)
+        
+        work_for = pair(name='work_for')
+        located_in = pair(name='located_in')
+        live_in = pair(name='live_in')
+        orgbase_on = pair(name='orgbase_on')
+        kill = pair(name='kill')
+
+        '''
+        work_for.has_a(people, organization)
+        located_in.has_a(location, location)
+        live_in.has_a(people, location)
+        orgbase_on.has_a(organization, location)
+        kill.has_a(people, people)
+        '''
+        
+        # LC2
+        ifL(existsL(work_for('x')), andL(people(path=('x', rel_pair_word1)), organization(path=('x', rel_pair_word2))), active = True)
+         
+        # Exist person which lives and works in the same location
+        existsL(
+            andL(
+                people('p'), # person p
+                live_in('l', path=('p', rel_pair_word1.reversed)), 
+                work_for('w', path=('p', rel_pair_word1.reversed)),
+                orgbase_on('o', path=(('w', rel_pair_word2, rel_pair_word1.reversed),('l', rel_pair_word2, rel_pair_word2.reversed)))
+                )
+        )
+        '''
+        existsL(
+            andL(
+                people('p'), # person p
+                location('p_live_in_location', path=('p', rel_pair_word1.reversed, rel_pair_word2)),
+                location('p_work_in_location', path=('p_live_in_location')) # the same location
+                )
+        )
+        '''          
+        # LC3
+        ifL(word('x'), atMostL(people, organization, location, other, o), active = True)
+        
\ No newline at end of file
diff --git a/test_regr/dummy_datanode/graph_multi.py b/test_regr/dummy_datanode/graph_multi.py
new file mode 100644
index 000000000..f81f258c3
--- /dev/null
+++ b/test_regr/dummy_datanode/graph_multi.py
@@ -0,0 +1,39 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL
+from domiknows.graph import EnumConcept
+from itertools import combinations
+
+Graph.clear()
+Concept.clear()
+Relation.clear()
+
+with Graph('global') as graph_multi:
+    with Graph('linguistic') as ling_graph:
+        ling_graph.ontology = ('http://ontology.ihmc.us/ML/PhraseGraph.owl', './')
+       
+        word = Concept(name='word')
+        phrase = Concept(name='phrase')
+        sentence = Concept(name='sentence')
+        (rel_sentence_contains_word,) = sentence.contains(word)
+        (rel_sentence_contains_phrase,) = sentence.contains(phrase)
+        (rel_phrase_contains_word,) = phrase.contains(word)
+
+        pair = Concept(name='pair')
+        (rel_pair_phrase1, rel_pair_phrase2, ) = pair.has_a(arg1=phrase, arg2=phrase)
+
+    with Graph('application') as app_graph:
+        entity = phrase(name='entity')
+        entity_label = entity(name="entity_label", ConceptClass=EnumConcept, values=["people", "organization", 'location', 'other', 'O'])
+        pair_label = pair(name="pair_label", ConceptClass=EnumConcept, values=["work_for", "located_in", 'live_in', 'orgbase_on', 'kill'])
+#        
+        for l1, l2 in combinations(entity_label.attributes, 2):
+            nandL(l1, l2)
+
+        for l1, l2 in combinations(pair_label.attributes, 2):
+            nandL(l1, l2)
+        
+        ifL(pair_label.work_for('x', 'y'), andL(entity_label.people('x'), entity_label.organization('y')))
+        ifL(pair_label.located_in('x', 'y'), andL(entity_label.location('x'), entity_label.location('y')))
+        ifL(pair_label.live_in('x', 'y'), andL(entity_label.people('x'), entity_label.location('y')))
+        ifL(pair_label.orgbase_on('x', 'y'), andL(entity_label.organization('x'), entity_label.location('y')))
+        ifL(pair_label.kill('x', 'y'), andL(entity_label.people('x'), entity_label.people('y')))
diff --git a/test_regr/dummy_datanode/test_attribute.py b/test_regr/dummy_datanode/test_attribute.py
new file mode 100644
index 000000000..f6b59a0d3
--- /dev/null
+++ b/test_regr/dummy_datanode/test_attribute.py
@@ -0,0 +1,31 @@
+from domiknows.graph.dataNodeDummy import createDummyDataNode, satisfactionReportOfConstraints
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import nandL, orL, andL, existsL, notL, atLeastL, atMostL, ifL
+from domiknows.graph.relation import disjoint
+
+Graph.clear()
+Concept.clear()
+Relation.clear()
+
+with Graph('test_graph') as graph:
+    image = Concept(name='image')
+
+    a = image(name='a')
+    b = image(name='b')
+    c = image(name='c')
+    d = image(name='d')
+    e = image(name='e')
+
+    ifL(a,notL(b))
+    ifL(a,notL(c))
+    nandL(d,e)
+
+ac_, t_ = 0, 0
+
+datanode=createDummyDataNode(graph)
+print(datanode.getAttribute(a))
+datanode.inferILPResults()
+verifyResult = datanode.verifyResultsLC()
+verifyResultILP = datanode.verifyResultsLC()
+
+report = satisfactionReportOfConstraints(datanode)
diff --git a/test_regr/dummy_datanode/test_dummy_datanode.py b/test_regr/dummy_datanode/test_dummy_datanode.py
new file mode 100644
index 000000000..fd90c4761
--- /dev/null
+++ b/test_regr/dummy_datanode/test_dummy_datanode.py
@@ -0,0 +1,24 @@
+import pytest
+from domiknows.graph import createDummyDataNode, satisfactionReportOfConstraints
+from graph import graph
+
+def test_dummy_data_node_inference():
+    testDummyDn = createDummyDataNode(graph)
+    
+    # run satisfactionReportOfConstraints
+    try:
+        satisfactionReportOfConstraints(testDummyDn)
+    except Exception:
+        pytest.fail("satisfaction report raised an exception")
+        
+    # Checking if inferILPResults doesn't raise any exception
+    try:
+        testDummyDn.inferILPResults()
+    except Exception:
+        pytest.fail("inferILPResults raised an exception")
+
+    # Checking if infer doesn't raise any exception
+    try:
+        testDummyDn.infer()
+    except Exception:
+        pytest.fail("infer raised an exception")
\ No newline at end of file
diff --git a/test_regr/dummy_datanode/test_dummy_datanode_multipy.py b/test_regr/dummy_datanode/test_dummy_datanode_multipy.py
new file mode 100644
index 000000000..1d58dd15e
--- /dev/null
+++ b/test_regr/dummy_datanode/test_dummy_datanode_multipy.py
@@ -0,0 +1,24 @@
+import pytest
+from domiknows.graph import createDummyDataNode, satisfactionReportOfConstraints
+from graph_multi import graph_multi
+
+def test_dummy_data_node_inference():
+    testDummyDn = createDummyDataNode(graph_multi)
+    
+    # run satisfactionReportOfConstraints
+    try:
+        satisfactionReportOfConstraints(testDummyDn)
+    except Exception:
+        pytest.fail("satisfaction report raised an exception")
+        
+    # Checking if inferILPResults doesn't raise any exception
+    try:
+        testDummyDn.inferILPResults()
+    except Exception:
+        pytest.fail("inferILPResults raised an exception")
+
+    # Checking if infer doesn't raise any exception
+    try:
+        testDummyDn.infer()
+    except Exception:
+        pytest.fail("infer raised an exception")
\ No newline at end of file
diff --git a/test_regr/examples/PMDExistL/graph.py b/test_regr/examples/PMDExistL/graph.py
new file mode 100644
index 000000000..a456f6905
--- /dev/null
+++ b/test_regr/examples/PMDExistL/graph.py
@@ -0,0 +1,31 @@
+def get_graph(args):
+    from domiknows.graph import Graph, Concept, Relation
+    from domiknows.graph.logicalConstrain import orL, existsL, ifL, notL, andL, atMostL, atLeastL, exactL
+    from domiknows.graph import EnumConcept
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+
+    with Graph('global_PMD') as graph:
+        a = Concept(name='a')
+        b = Concept(name='b')
+        a_contain_b,  = a.contains(b)
+
+        b_answer = b(name="answer_b", ConceptClass=EnumConcept, values=["zero", "one"])
+
+        expected_zero = b_answer.__getattr__("zero")
+        expected_one = b_answer.__getattr__("one")
+
+        expected_value = expected_zero if args.expected_value == 0 else expected_one
+
+        if args.atLeastL and args.atMostL:
+            atLeastL(expected_value, args.expected_atLeastL)
+            atMostL(expected_value, args.expected_atMostL)
+        elif args.atMostL:
+            atMostL(expected_value, args.expected_atMostL)
+        elif args.atLeastL:
+            atLeastL(expected_value, args.expected_atLeastL)
+        else:
+            exactL(expected_value, args.expected_atLeastL)
+
+    return graph, a, b, a_contain_b, b_answer
diff --git a/test_regr/examples/PMDExistL/main.py b/test_regr/examples/PMDExistL/main.py
new file mode 100644
index 000000000..4b7393b42
--- /dev/null
+++ b/test_regr/examples/PMDExistL/main.py
@@ -0,0 +1,80 @@
+import sys
+import argparse
+from typing import Any
+import numpy as np
+import torch
+from domiknows.sensor.pytorch.sensors import ReaderSensor
+from domiknows.sensor.pytorch.relation_sensors import EdgeSensor, FunctionalSensor
+from domiknows.sensor.pytorch.learners import ModuleLearner
+from domiknows.sensor import Sensor
+from domiknows.program.metric import MacroAverageTracker
+from domiknows.program.loss import NBCrossEntropyLoss
+from domiknows.program.lossprogram import PrimalDualProgram
+from domiknows.program.model.pytorch import SolverModel
+
+from utils import TestTrainLearner,return_contain,create_dataset,evaluate_model,train_model
+sys.path.append('../../../../domiknows/')
+from graph import get_graph
+
+Sensor.clear()
+
+def parse_arguments() -> argparse.Namespace:
+
+    parser = argparse.ArgumentParser(description="Machine Learning Experiment")
+    parser.add_argument("--counting_tnorm", choices=["G", "P", "L", "SP"], default="G", help="The tnorm method to use for the counting constraints")
+    parser.add_argument("--atLeastL", default=False, type=bool, help="Use at least L constraint")
+    parser.add_argument("--atMostL", default=False, type=bool, help="Use at most L constraint")
+    parser.add_argument("--epoch", default=500, type=int, help="Number of training epochs")
+    parser.add_argument("--expected_atLeastL", default=3, type=int, help="Expected value for at least L")
+    parser.add_argument("--expected_atMostL", default=3, type=int, help="Expected value for at most L")
+    parser.add_argument("--expected_value", default=0, type=int, help="Expected value")
+    parser.add_argument("--N", default=10, type=int, help="N parameter")
+    parser.add_argument("--M", default=8, type=int, help="M parameter")
+    return parser.parse_args()
+
+
+
+def setup_graph(args: argparse.Namespace, a: Any, b: Any, a_contain_b: Any, b_answer: Any) -> None:
+    a["index"] = ReaderSensor(keyword="a")
+    b["index"] = ReaderSensor(keyword="b")
+    b["temp_answer"] = ReaderSensor(keyword="label")
+    b[a_contain_b] = EdgeSensor(b["index"], a["index"], relation=a_contain_b, forward=return_contain)
+    b[b_answer] = ModuleLearner(a_contain_b, "index", module=TestTrainLearner(args.N), device="cpu")
+    b[b_answer] = FunctionalSensor(a_contain_b, "temp_answer", forward=lambda _, label: label, label=True)
+
+def main(args: argparse.Namespace):
+    np.random.seed(0)
+    torch.manual_seed(0)
+
+    graph, a, b, a_contain_b, b_answer = get_graph(args)
+    dataset = create_dataset(args.N, args.M)
+    setup_graph(args, a, b, a_contain_b, b_answer)
+    program = PrimalDualProgram(
+        graph, SolverModel, poi=[a, b, b_answer],
+        inferTypes=['local/argmax'],
+        loss=MacroAverageTracker(NBCrossEntropyLoss()),
+        beta=10, device='cpu', tnorm="L", counting_tnorm=args.counting_tnorm
+    )
+    expected_value = args.expected_value
+
+    before_count = evaluate_model(program, dataset, b_answer).get(expected_value, 0)
+    train_model(program, dataset, args.epoch)
+
+    pass_test_case = True
+    actual_count = evaluate_model(program, dataset, b_answer).get(expected_value, 0)
+
+
+    if args.atLeastL:
+        pass_test_case &= (actual_count >= args.expected_atLeastL)
+    if args.atMostL:
+        pass_test_case &= (actual_count <= args.expected_atMostL)
+    if not args.atLeastL and not args.atMostL:
+        pass_test_case &= (actual_count == args.expected_atLeastL)
+
+    print(f"Test case {'PASSED' if pass_test_case else 'FAILED'}")
+    print(f"expected_value, before_count, actual_count,pass_test_case): {expected_value, before_count, actual_count,pass_test_case}")
+    return pass_test_case, before_count, actual_count
+
+if __name__ == "__main__":
+    args = parse_arguments()
+    main(args)
diff --git a/test_regr/examples/PMDExistL/testcase.py b/test_regr/examples/PMDExistL/testcase.py
new file mode 100644
index 000000000..161c530ae
--- /dev/null
+++ b/test_regr/examples/PMDExistL/testcase.py
@@ -0,0 +1,78 @@
+import subprocess
+import sys
+import os
+import itertools
+import json
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from collections import defaultdict
+
+def run_test(params):
+    # Convert params to command-line arguments
+    args = []
+    for key, value in params.items():
+        if not str(value)=="False":
+            args.extend([f'--{key}', str(value)])
+
+    # Get the path to the Python interpreter in the current virtual environment
+    python_executable = sys.executable
+
+    # Construct the command to run the main script
+    cmd = [python_executable, 'main.py'] + args
+
+    # Run the command in a subprocess
+    try:
+        # Use UTF-8 encoding and replace any characters that can't be decoded
+        result = subprocess.run(cmd, capture_output=True, text=True, encoding='utf-8', errors='replace', check=True)
+        return params, True, result.stdout
+    except subprocess.CalledProcessError as e:
+        return params, False, e.stderr
+
+def run_tests():
+    """Run tests with different combinations of input arguments."""
+    # Define the parameter combinations to test
+    param_combinations = {
+        'counting_tnorm': ["G" , "P","SP","L" ],
+        'atLeastL': [True, False],
+        'atMostL': [True, False],
+        'epoch': [1000],
+        'expected_atLeastL': [2],
+        'expected_atMostL': [5],
+        'expected_value': [0,1],
+        'N': [10],
+        'M': [8]
+    }
+
+    # Generate all combinations of parameters
+    keys, values = zip(*param_combinations.items())
+    combinations = [dict(zip(keys, v)) for v in itertools.product(*values)]
+
+    # Run tests for each combination using ProcessPoolExecutor with max_workers=4
+    results = defaultdict(list)
+    total_combinations = len(combinations)
+
+    with ProcessPoolExecutor(max_workers=4) as executor:
+        for i, (params, test_passed, output) in enumerate(executor.map(run_test, combinations), 1):
+            counting_tnorm = params['counting_tnorm']
+            results[counting_tnorm].append((params, 'PASSED' in output, output))
+            print(f"\nCompleted test {i}/{total_combinations}:")
+            print(f"Parameters: {params}")
+            print(output.split("\n")[-2])
+            print(f"Passed: {'PASSED' in output}")
+
+    # Print summary of results per counting_tnorm
+    print("\n--- Test Summary ---")
+    for counting_tnorm, tnorm_results in results.items():
+        passed_tests = sum(1 for _, passed, _ in tnorm_results if passed)
+        print(f"\nResults for counting_tnorm = {counting_tnorm}:")
+        print(f"Passed {passed_tests} out of {len(tnorm_results)} tests.")
+
+        # Print details of failed tests for this counting_tnorm
+        if passed_tests < len(tnorm_results):
+            print(f"Failed tests for counting_tnorm = {counting_tnorm}:")
+            for params, passed, output in tnorm_results:
+                if not passed:
+                    print(f"Parameters: {params}")
+                    print(f"Error output: {output}")
+
+if __name__ == "__main__":
+    run_tests()
\ No newline at end of file
diff --git a/test_regr/examples/PMDExistL/utils.py b/test_regr/examples/PMDExistL/utils.py
new file mode 100644
index 000000000..94e73d393
--- /dev/null
+++ b/test_regr/examples/PMDExistL/utils.py
@@ -0,0 +1,73 @@
+from typing import List, Dict, Any
+import numpy as np
+import torch
+from torch import nn
+from tqdm import tqdm
+from domiknows.sensor.pytorch.learners import TorchLearner
+from domiknows.program.model.base import Mode
+from domiknows.program.lossprogram import PrimalDualProgram
+
+class DummyLearner(TorchLearner):
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return torch.stack((torch.ones(len(x)) * 4, torch.ones(len(x)) * 6), dim=-1)
+
+class TestTrainLearner(nn.Module):
+    def __init__(self, input_size: int):
+        super().__init__()
+        self.layers = nn.Sequential(
+            nn.Linear(input_size, input_size),
+            nn.Linear(input_size, 2)
+        )
+
+    def forward(self, _, x: torch.Tensor) -> torch.Tensor:
+        return self.layers(x)
+
+def return_contain(b: torch.Tensor, _: Any) -> torch.Tensor:
+    return torch.ones(len(b)).unsqueeze(-1)
+
+def create_dataset(N: int, M: int) -> List[Dict[str, Any]]:
+    return [{
+        "a": [0],
+        "b": [((np.random.rand(N) - np.random.rand(N))).tolist() for _ in range(M)],
+        "label": [0] * M
+    }]
+
+def train_model(program: PrimalDualProgram, dataset: List[Dict[str, Any]], num_epochs: int) -> None:
+    program.model.train()
+    program.model.reset()
+    program.cmodel.train()
+    program.cmodel.reset()
+    program.model.mode(Mode.TRAIN)
+
+    opt = torch.optim.Adam(program.model.parameters(), lr=1e-2)
+    copt = torch.optim.Adam(program.cmodel.parameters(), lr=1e-3)
+
+    for _ in tqdm(range(num_epochs), desc="Training with PMD"):
+        for data in dataset:
+            opt.zero_grad()
+            copt.zero_grad()
+            mloss, _, *output = program.model(data)
+            closs, *_ = program.cmodel(output[1])
+
+            loss = mloss * 0 + (closs if torch.is_tensor(closs) else 0)
+            if loss.item() < 0:
+                print("Negative loss", loss.item())
+                break
+            if loss:
+                loss.backward()
+                opt.step()
+                copt.step()
+
+def evaluate_model(program: PrimalDualProgram, dataset: List[Dict[str, Any]], b_answer: Any) -> Dict[int, int]:
+    program.model.eval()
+    program.model.reset()
+    program.cmodel.eval()
+    program.cmodel.reset()
+    program.model.mode(Mode.TEST)
+
+    final_result_count = {}
+    for datanode in program.populate(dataset=dataset):
+        for child in datanode.getChildDataNodes():
+            pred = child.getAttribute(b_answer, 'local/argmax').argmax().item()
+            final_result_count[pred] = final_result_count.get(pred, 0) + 1
+    return final_result_count
\ No newline at end of file
diff --git a/test_regr/examples/conll04/graph.py b/test_regr/examples/conll04/graph.py
index 5aca9559a..0f0dad031 100644
--- a/test_regr/examples/conll04/graph.py
+++ b/test_regr/examples/conll04/graph.py
@@ -36,16 +36,18 @@
         live_in = pair(name='live_in')
         orgbase_on = pair(name='orgbase_on')
         kill = pair(name='kill')
-
+        
+        ''' We should not define it
         work_for.has_a(people, organization)
         located_in.has_a(location, location)
         live_in.has_a(people, location)
         orgbase_on.has_a(organization, location)
         kill.has_a(people, people)
-
+        '''
         # LC2
-        ifL(existsL(work_for('x')), andL(people(path=('x', rel_pair_word1.name)), organization(path=('x', rel_pair_word2.name))), active = True)
-        
+        #ifL(existsL(work_for('x')), andL(people(path=('x', rel_pair_word1.name)), organization(path=('x', rel_pair_word2.name))), active = True)
+        ifL(existsL(work_for('x', 'y')), andL(people('x'), organization('y')), active = True)
+
         # LC3
         ifL(word, atMostL(people, organization, location, other, o), active = True)
         
\ No newline at end of file
diff --git a/test_regr/examples/conll04/test_main.py b/test_regr/examples/conll04/test_main.py
index 0b95d3499..0c91be56b 100644
--- a/test_regr/examples/conll04/test_main.py
+++ b/test_regr/examples/conll04/test_main.py
@@ -129,7 +129,7 @@ def model_declaration(config, case):
     graph.detach()
 
     from domiknows.utils import setDnSkeletonMode
-    setDnSkeletonMode(True)
+    setDnSkeletonMode(False)
     
     lcConcepts = {}
     for _, g in graph.subgraphs.items():
@@ -326,7 +326,7 @@ def test_main_conll04(case):
                        
             #assert len(child_node.getChildDataNodes()) == len(case.char.raw[child_node.instanceID])
             num_pairs = case.pair.pa1_backward[:,child_node.instanceID].sum()
-            assert len(child_node.getRelationLinks(relationName = "arg1")) == num_pairs # has relation named "pair"with each word (including itself)
+            assert len(child_node.getLinks(relationName = "arg1")) == num_pairs # has relation named "pair"with each word (including itself)
 
             #assert (child_node.getAttribute('emb') == case.word.emb[child_node.instanceID]).all()
             assert (child_node.getAttribute('<people>') == case.word.people[child_node.instanceID]).all()
@@ -406,8 +406,8 @@ def test_main_conll04(case):
         assert len(JohnDN.getChildDataNodes(conceptName=char)) == 5
         assert len(JohnDN.getChildDataNodes(conceptName=phrase)) == 0
         
-        assert len(JohnDN.getRelationLinks()) == 2
-        assert len(JohnDN.getRelationLinks(relationName="arg1")) == 4
+        assert len(JohnDN.getRelationLinks()) == 1
+        assert len(JohnDN.getLinks(relationName="arg1")) == 4
 
         # Get value of attribute o/ILP for word 2
         #assert tokenResult['O'][2] == 1
@@ -418,33 +418,17 @@ def test_main_conll04(case):
         assert sum([dn.getAttribute(o, 'ILP').item() for dn in datanode.findDatanodes(select = word)]) == 2
         
         # ------------ Relations Results
-        
-        # Get value of attribute work_for/ILP for pair between 0 and 3
-        #assert pairResult['work_for'][0][3] == nan
-        assert datanode.findDatanodes(select = pair, indexes = {"arg1" : 0, "arg2": 3})[0].getAttribute(work_for, 'ILP').item() != \
-               datanode.findDatanodes(select = pair, indexes = {"arg1" : 0, "arg2": 3})[0].getAttribute(work_for, 'ILP').item()
-        
-        #assert datanode.findDatanodes(select = pair, indexes = {"arg1" : (word, 'raw', 'John'), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP') == 1
-
-        assert datanode.findDatanodes(select = pair, indexes = {"arg1" : ((word,), (word, 'raw', 'John')), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP').item() \
-               != \
-               datanode.findDatanodes(select = pair, indexes = {"arg1" : ((word,), (word, 'raw', 'John')), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP').item()
-        assert datanode.findDatanodes(select = pair, indexes = {"arg1" : (word, (word, 'raw', 'John')), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP').item() !=\
-               datanode.findDatanodes(select = pair, indexes = {"arg1" : (word, (word, 'raw', 'John')), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP').item()
-         
-        assert datanode.findDatanodes(select = pair, indexes = {"arg1" : (0, (word, 'raw', 'John')), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP') != \
-               datanode.findDatanodes(select = pair, indexes = {"arg1" : (0, (word, 'raw', 'John')), "arg2": (word, 'raw', "IBM")})[0].getAttribute(work_for, 'ILP')
-        
+                
         # Sum all value of attribute work_for/ILP  for the pair relation from 0
         #assert sum(pairResult['work_for'][1]) == 1
         assert sum([dn.getAttribute(work_for, 'ILP').item() if dn.getAttribute(work_for, 'ILP').item() == dn.getAttribute(work_for, 'ILP').item() else 0
                     for dn in datanode.findDatanodes(select = pair, indexes = {"arg1" : 1}) 
-                    ]) == 0
+                    ]) == 2
         
         # Sum all value of attribute work_for/ILP  for the pair relation from 1
         #assert sum(pairResult['work_for'][0]) == 0
         assert sum([dn.getAttribute(work_for, 'ILP').item() if not math.isnan(dn.getAttribute(work_for, 'ILP').item()) 
-                    else 0 for dn in datanode.findDatanodes(select = pair, indexes = {"arg1" : 0})]) == 0
+                    else 0 for dn in datanode.findDatanodes(select = pair, indexes = {"arg1" : 0})]) == 1
         
         # Sum all value of attribute work_for/ILP  for the pair relation from 2
         #assert sum(pairResult['work_for'][2]) == 0
@@ -452,7 +436,7 @@ def test_main_conll04(case):
     
         # Sum all value of attribute work_for/ILP  for the pair relation from 3
         #assert sum(pairResult['work_for'][3]) == 0
-        assert sum([dn.getAttribute(work_for, 'ILP').item() for dn in datanode.findDatanodes(select = pair, indexes = {"arg1" : 3})]) == 0
+        assert sum([dn.getAttribute(work_for, 'ILP').item() for dn in datanode.findDatanodes(select = pair, indexes = {"arg1" : 3})]) == 1
         
         # ------------ Calculate logical constraints losses 
         for tnorm in ['L', 'G', "P"]:
@@ -479,6 +463,9 @@ def test_main_conll04(case):
                         assert round(lcResult[ifLLCid]['lossTensor'][i].item(), 4) == round(case.lc2LossTensor[tnorm][i].item(), 4)
         
         #------- Calculate sample logical constraints losses 
+       
+        #sampleResult = datanode.calculateLcLoss(sample = True, sampleSize = -1)
+        sampleResult = datanode.calculateLcLoss(sample = True, sampleSize = 1)
         sampleResult = datanode.calculateLcLoss(sample = True, sampleSize = 1000)
 
                         
diff --git a/test_regr/examples/graph_coloring/graph.py b/test_regr/examples/graph_coloring/graph.py
index 098a18f9a..55cbd9dc9 100644
--- a/test_regr/examples/graph_coloring/graph.py
+++ b/test_regr/examples/graph_coloring/graph.py
@@ -1,21 +1,50 @@
-from domiknows.graph import Graph, Concept, Relation
-from domiknows.graph.logicalConstrain import orL, existsL
 
 
-Graph.clear()
-Concept.clear()
-Relation.clear()
+def get_graph(constraint,atmost,atleast,test_number):
+    from domiknows.graph import Graph, Concept, Relation
+    from domiknows.graph.logicalConstrain import orL, existsL, ifL, notL, andL, atMostAL, atLeastAL
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
 
-with Graph('global') as graph:
-        world = Concept(name='world')
-        city = Concept(name='city')
+    with Graph('global'+str(test_number)) as graph:
+        world = Concept(name='world_'+str(test_number))
+        city = Concept(name='city_'+str(test_number))
         (world_contains_city,) = world.contains(city)
-         
-        neighbor = Concept(name='neighbor')
-        (city1, city2) = neighbor.has_a(arg1=city, arg2=city)
         
-        firestationCity = city(name='firestationCity')
+        neighbor = Concept(name='neighbor_'+str(test_number))
+        (city1, city2) = neighbor.has_a(arg1=city, arg2=city)
         
-        # Constraints - For each city x either it is a firestationCity or exists a city y which is in neighbor relation to city x and y is a firestationCity        
-        #orL(firestationCity, V(name='x'), existsL(firestationCity, V(name='y', v=('x', neighbor.name, city2.name)), 'y'), V(name='z'))
-        orL(firestationCity('x'), existsL(firestationCity('y', path=('x', city1.name, city2.name))))
\ No newline at end of file
+        firestationCity = city(name='firestationCity_'+str(test_number))
+
+
+        atMostAL(firestationCity,atmost)
+        atLeastAL(firestationCity,atleast)
+
+        if constraint=="existL":
+            orL(firestationCity('x'), existsL(
+                ifL(neighbor("z",path=('x', city1.reversed))
+                    ,firestationCity("p",path=('z', city2))
+                )
+            ))
+        elif constraint=="orLnotLexistL":
+            orL(firestationCity('x'), notL(existsL(
+                ifL(neighbor("z",path=('x', city1.reversed))
+                    , firestationCity("p",path=('z', city2))
+                )
+            )))
+        elif constraint=="ifLnotLexistL":
+            ifL(firestationCity('x'), notL(existsL(
+                ifL(neighbor("z",path=('x', city1.reversed))
+                    , firestationCity("p",path=('z', city2))
+                )
+            )))
+        elif constraint=="orL":
+            ifL(neighbor('z'), orL(
+                firestationCity("x",path=('z', city1)),
+                firestationCity("p",path=('z', city2))
+            ))
+        else:
+            pass
+            #print("no constraint.")
+    return graph, world, city, world_contains_city, neighbor, city1, city2, firestationCity
\ No newline at end of file
diff --git a/test_regr/examples/graph_coloring/graph2.py b/test_regr/examples/graph_coloring/graph2.py
index 930e49a05..b4864dc52 100644
--- a/test_regr/examples/graph_coloring/graph2.py
+++ b/test_regr/examples/graph_coloring/graph2.py
@@ -16,8 +16,9 @@
 
         firestationCity = city(name='firestationCity')
         
-        # Constraints - For each city x either it is a firestationCity or exists a city y which is in cityLink relation with neighbor attribute equal True to city x and y is a firestationCity
-        orL(firestationCity('x'), existsL(firestationCity(path=('x', eqL(cityLink, city1.name, {True}),  city2))))
+        # Constraints - For each city x either it is a firestationCity or 
+        # exists a city y which is in cityLink relation with neighbor attribute equal True to city x and y is a firestationCity
+        orL(firestationCity('x'), existsL(firestationCity(path=('x', city1.reversed, eqL(cityLink, "neighbor", {True}),  city2))))
         
         # No less then 1 firestationCity in the world
         atLeastAL(firestationCity, p=90)
@@ -26,7 +27,7 @@
         atMostAL(firestationCity, 3, p=80)
         
         # Each city has no more then 4 neighbors which are not firestationCity
-        ifL(city('x'), atMostL(notL(firestationCity(path=('x', eqL(cityLink, city1.name, {True}), city2))), 3), p=90)
+        ifL(city('x'), atMostL(notL(firestationCity(path=('x', city1.reversed, eqL(cityLink, "neighbor", {True}),  city2))), 3), p=90)
 
         # Exactly 2 firestationCity in the world 
         exactAL(firestationCity, 2, p=55)
\ No newline at end of file
diff --git a/test_regr/examples/graph_coloring/main.py b/test_regr/examples/graph_coloring/main.py
index d190603ca..e9452bde8 100644
--- a/test_regr/examples/graph_coloring/main.py
+++ b/test_regr/examples/graph_coloring/main.py
@@ -1,29 +1,53 @@
-import sys
-import pytest
-
-sys.path.append('.')
-sys.path.append('../..')
-
 
-def model_declaration():
-    import torch
-
-    from domiknows.sensor.pytorch.sensors import ReaderSensor
-    from domiknows.sensor.pytorch.relation_sensors import EdgeSensor, CompositionCandidateReaderSensor
-    from domiknows.program import LearningBasedProgram
-    from domiknows.program.model.pytorch import model_helper, PoiModel
+import sys
+sys.path.append('../../../../domiknows/')
+
+import torch, argparse, time
+from domiknows.sensor.pytorch.sensors import ReaderSensor
+from domiknows.sensor.pytorch.relation_sensors import EdgeSensor, CompositionCandidateReaderSensor
+from domiknows.program import SolverPOIProgram
+
+from domiknows.sensor.pytorch.learners import TorchLearner
+from graph import get_graph
+
+for test_number, test_cases in enumerate([
+    [],
+    ["--firestations", "4","--atleastaL", "3"],
+    ["--firestations", "4","5","6","7","--atmostaL", "2"],
+    ["--firestations", "1","2","6","--constraint","ifLnotLexistL"],
+    ["--firestations", "1", "--constraint", "orLnotLexistL"]
+]):
+    from domiknows.sensor import Sensor
+    Sensor.clear()
+
+    parser = argparse.ArgumentParser(description='Check a graphical structure and existL constraint in domiknows')
+    parser.add_argument('--firestations', dest='firestations', default=[], help='list of cities to be defauled as firestation from 1 to 9', type=int, nargs='+')
+    parser.add_argument('--constraint', dest='constraint',default="None", choices=["None","orL", "existL","ifLnotLexistL","orLnotLexistL"], help="Choose a constraint")
+    parser.add_argument('--atmostaL', dest='atmostaL',default=10,type=int)
+    parser.add_argument('--atleastaL', dest='atleastaL',default=0,type=int)
+
+    args = parser.parse_args(test_cases)
+    
+    graph, world, city, world_contains_city, neighbor, city1, city2, firestationCity = get_graph(args.constraint,args.atmostaL,args.atleastaL,test_number)
+    
 
-    from graph import graph, world, city, world_contains_city, neighbor, city1, city2, firestationCity
-    from sensors import DummyCityLearner
+    class DummyCityLearner(TorchLearner):
+        def __init__(self, *pre, fire_stations=[i+1 for i in range(9)]):
+            TorchLearner.__init__(self,*pre)
+            self.fire_stations=fire_stations
 
-    graph.detach()
+        def forward(self, x):
+            result = torch.zeros(len(x), 2)
+            result[:, 1] = -1000
+            for i in self.fire_stations:
+                result[i-1,1]=1000
+            return result
 
-    # --- City
+    dataset = [{'world': [0], 'city': [1, 2, 3, 4, 5, 6, 7, 8, 9], 'links': {1: [2, 3, 4, 5], 2: [1, 6], 3: [1], 4: [1], 5: [1], 6: [2, 7, 8, 9], 7: [6], 8: [6], 9: [6]}}]
     world['index'] = ReaderSensor(keyword='world')
     city['index'] = ReaderSensor(keyword='city')
     city[world_contains_city] = EdgeSensor(city['index'], world['index'], relation=world_contains_city, forward=lambda x, _: torch.ones_like(x).unsqueeze(-1))
 
-
     def readNeighbors(index, data, arg1, arg2):
         city1, city2 = arg1, arg2
         if city1.getAttribute('index') in data[int(city2.getAttribute('index'))]:
@@ -33,55 +57,24 @@ def readNeighbors(index, data, arg1, arg2):
 
     neighbor[city1.reversed, city2.reversed] = CompositionCandidateReaderSensor(city['index'], keyword='links', relations=(city1.reversed, city2.reversed), forward=readNeighbors)
 
-    # --- Learners
-    city[firestationCity] = DummyCityLearner('index')
-    
-    program = LearningBasedProgram(graph, model_helper(PoiModel, poi=[world, city, city[firestationCity], neighbor]))
-    return program
-
-
-@pytest.mark.gurobi
-def test_graph_coloring_main():
-    from reader import CityReader
-    from graph import city, neighbor, firestationCity
-
-    lbp = model_declaration()
-
-    dataset = CityReader().run()  # Adding the info on the reader
-
-    for datanode in lbp.populate(dataset=dataset):
-        assert datanode != None
-        assert len(datanode.getChildDataNodes()) == 9
-
-        for child_node in datanode.getChildDataNodes():
-            assert child_node.ontologyNode == city
-            assert child_node.getAttribute('<' + firestationCity.name + '>')[0] == 0
-            assert child_node.getAttribute('<' + firestationCity.name + '>')[1] == 1
-
-        # call solver
-        conceptsRelations = (firestationCity)  
-        datanode.inferILPResults(*conceptsRelations, fun=None, minimizeObjective=True) 
-
-        result = []
-        for child_node in datanode.getChildDataNodes():
-            s = child_node.getAttribute('index')
-            f = child_node.getAttribute(firestationCity, 'ILP').item()
-            if f > 0:
-                r = (s, True)
-            else:
-                r = (s, False)
-            result.append(r)
-
-        for child_index, child_node in enumerate(datanode.getChildDataNodes()):
-            if child_index + 1 == 1:
-                assert child_node.getAttribute(firestationCity, 'ILP').item() == 1
-            elif child_index + 1 == 6:
-                assert child_node.getAttribute(firestationCity, 'ILP').item() == 1
-            else:
-                assert child_node.getAttribute(firestationCity, 'ILP').item() == 0
-                
-    verifyResult = datanode.verifyResultsLC()
-
 
-if __name__ == '__main__':
-    pytest.main([__file__])
+    city[firestationCity] = DummyCityLearner('index',fire_stations=list(args.firestations))
+    program = SolverPOIProgram(graph, poi=[world, city, city[firestationCity], neighbor])
+
+    for datanode in program.populate(dataset=dataset):    
+        datanode.inferILPResults() 
+        fire_stations=[int(child_node.getAttribute('<' + firestationCity.name + '>',"ILP").item()) for child_node in datanode.getChildDataNodes()]
+        if test_number==0:
+            assert sum(fire_stations)==0
+        if test_number==1:
+            assert sum(fire_stations)==3
+            assert fire_stations[4-1]==1
+        if test_number==2:
+            assert sum(fire_stations)==2
+            assert fire_stations[4-1]+fire_stations[5-1]+fire_stations[6-1]+fire_stations[7-1]==2
+        if test_number==3:
+            assert sum(fire_stations)==2
+            assert fire_stations[1-1] and fire_stations[6-1]
+        if test_number==4:
+            assert sum(fire_stations)==1
+            fire_stations[1-1]==0
diff --git a/test_regr/examples/orbs/graph.py b/test_regr/examples/orbs/graph.py
new file mode 100644
index 000000000..dab19dc4f
--- /dev/null
+++ b/test_regr/examples/orbs/graph.py
@@ -0,0 +1,45 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import orL, existsL, ifL, notL, andL, atMostL, atLeastL
+
+Graph.clear()
+Concept.clear()
+Relation.clear()
+
+def get_graph(constraint,atmost,atleast):
+
+    with Graph('global') as graph:
+
+        csp = Concept(name='csp')
+        csp_range = Concept(name='csp_range')
+        enforce_csp_range = csp_range(name='enforce_csp_range')
+
+        orbs = Concept(name='orbs')
+
+        (csp_contains_csp_range,) = csp.contains(csp_range)
+        (csp_range_contains_orbs,) = csp_range.contains(orbs)
+
+        colored_orbs=orbs(name='colored_orbs')
+        if constraint=="simple_constraint":
+            atMostL(colored_orbs,atmost)
+            atLeastL(colored_orbs,atleast)
+        elif constraint=="foreach_bag_atLeastAL":
+            ifL(csp_range("x"), 
+                atLeastL(colored_orbs("y",path=("x",csp_range_contains_orbs)),atleast)
+            )
+        elif constraint=="foreach_bag_atMostAL":
+            ifL(csp_range("x"), 
+                atMostL(colored_orbs("y",path=("x",csp_range_contains_orbs)),atmost)
+            )
+        elif constraint=="foreach_bag_existsL":
+            ifL(csp_range("x"), 
+                existsL(colored_orbs("y",path=("x",csp_range_contains_orbs)))
+            )
+        elif constraint=="foreach_bag_existsL_notL": # try with --colored
+            ifL(csp_range("x"), 
+                existsL(notL(colored_orbs("y",path=("x",csp_range_contains_orbs))))
+            )
+        else:
+            print("no contraint")
+    
+    return graph, csp,csp_range,orbs,csp_contains_csp_range,csp_range_contains_orbs,colored_orbs,enforce_csp_range
+
diff --git a/test_regr/examples/orbs/main.py b/test_regr/examples/orbs/main.py
new file mode 100644
index 000000000..a4953a972
--- /dev/null
+++ b/test_regr/examples/orbs/main.py
@@ -0,0 +1,75 @@
+import sys, torch, argparse, time
+sys.path.append('../../domiknows/')
+
+
+from domiknows.sensor.pytorch.sensors import ReaderSensor
+from domiknows.sensor.pytorch.relation_sensors import EdgeSensor, CompositionCandidateReaderSensor
+from domiknows.program import SolverPOIProgram
+
+from domiknows.sensor.pytorch.learners import TorchLearner
+from graph import get_graph
+parser = argparse.ArgumentParser(description='Check a csp structure and atmostal/atleastal constraint in domiknows')
+parser.add_argument('--colored', dest='colored', default=False,action='store_true',help="color every orb")
+parser.add_argument('--constraint', dest='constraint',default="foreach_bag_atLeastAL", choices=["None","simple_constraint","foreach_bag_existsL","foreach_bag_existsL_notL","foreach_bag_atLeastAL","foreach_bag_atMostAL"], help="Choose a constraint")
+parser.add_argument('--atmostaL', dest='atmostaL',default=25,type=int)
+parser.add_argument('--atleastaL', dest='atleastaL',default=2,type=int)
+
+args = parser.parse_args()
+
+graph, csp,csp_range,orbs,csp_contains_csp_range,csp_range_contains_orbs,colored_orbs,enforce_csp_range = get_graph(args.constraint,args.atmostaL,args.atleastaL)
+graph.detach()
+
+class DummyCityLearner(TorchLearner):
+    def __init__(self, *pre, colored=args.colored):
+        TorchLearner.__init__(self,*pre)
+        self.colored=colored
+
+    def forward(self, x):
+        result = torch.zeros(len(x), 2)
+        result[:, 1] = 1000*self.colored-1000*(1-self.colored)
+        return result
+    
+class DummyCityLearner2(TorchLearner):
+    def forward(self, x):
+        result = torch.zeros(len(x), 2)
+        result[:, 1] = 1000
+        return result
+
+dataset = [{'csp': [0], 'csp_range': [0, 1, 2, 3, 4], 'orbs': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]},]
+
+csp['index'] = ReaderSensor(keyword='csp')
+csp_range['index'] = ReaderSensor(keyword='csp_range')
+orbs['index'] = ReaderSensor(keyword='orbs')
+
+def connect(x,y):
+    connection=torch.zeros((len(x),len(y)))
+    for i in x.tolist():
+        for j in x.tolist():
+            connection[i-1,j+5*(i-1)]=1
+    return connection.T
+
+csp_range[csp_contains_csp_range] = EdgeSensor(csp['index'], csp_range['index'], relation=csp_contains_csp_range, forward=lambda _, x: torch.ones_like(x).unsqueeze(-1))
+orbs[csp_range_contains_orbs] = EdgeSensor(csp_range['index'], orbs['index'], relation=csp_range_contains_orbs, forward=connect)
+csp_range[enforce_csp_range] = DummyCityLearner2('index')
+orbs[colored_orbs] = DummyCityLearner('index')
+
+program = SolverPOIProgram(graph, poi=[orbs[colored_orbs],csp_range[enforce_csp_range],csp_range[csp_contains_csp_range],orbs[csp_range_contains_orbs]])
+
+orbs_before_inference,orbs_after_inference=[],[]
+
+for datanode in program.populate(dataset=dataset):
+    print("before inference")
+    print("orb color:",end="")
+    for csp_range_datanode in datanode.getChildDataNodes():
+        print(*[int(orb_node.getAttribute('<colored_orbs>',"local/softmax")[1].item()) for orb_node in csp_range_datanode.getChildDataNodes()],end=" | ")
+        
+    assert sum([int(orb_node.getAttribute('<colored_orbs>',"local/softmax")[1].item()) for orb_node in csp_range_datanode.getChildDataNodes()])==0
+    datanode.inferILPResults() 
+        
+    print("\n\nafter inference")
+    print("orb color:",end="")
+    for csp_range_datanode in datanode.getChildDataNodes():
+        print(*[int(orb_node.getAttribute('<colored_orbs>',"ILP").item()) for orb_node in csp_range_datanode.getChildDataNodes()],end=" | ")
+
+    assert sum([int(orb_node.getAttribute('<colored_orbs>',"ILP").item()) for orb_node in csp_range_datanode.getChildDataNodes()])==2
+print()
\ No newline at end of file
diff --git a/test_regr/graph_errors/README.md b/test_regr/graph_errors/README.md
new file mode 100644
index 000000000..b6c6cfc4c
--- /dev/null
+++ b/test_regr/graph_errors/README.md
@@ -0,0 +1,3 @@
+## Graph Errors examples
+
+This folder contains the graphs with deliberate errors to test logical constrain parsing and error reporting.
\ No newline at end of file
diff --git a/test_regr/graph_errors/graph_cardinality.py b/test_regr/graph_errors/graph_cardinality.py
new file mode 100644
index 000000000..b204b6e9e
--- /dev/null
+++ b/test_regr/graph_errors/graph_cardinality.py
@@ -0,0 +1,58 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+
+    with Graph('global') as graph:
+        with Graph('linguistic') as ling_graph:
+            text_document = Concept(name='text_document')
+        
+        with Graph('application', auto_constraint=False) as app_graph:
+            entity = Concept(name='entity')
+            person = entity(name='person', auto_constraint=False)
+            organization = entity(name='organization', auto_constraint=False)
+            location = entity(name='location', auto_constraint=False)
+            date = entity(name='date')
+            other_entity_types = entity(name='other_entity_types')
+        
+            relation = Concept(name='relation')
+            employment = relation(name='employment')
+            located = relation(name='located')
+            part_whole = relation(name='part-whole')
+            personal_social = relation(name='personal-social')
+            other_relation = relation(name='other')
+        
+            pair = Concept(name='pair')
+            (rel_pair_entity1, rel_pair_entity2) = pair.has_a(arg1=entity, arg2=entity)
+        
+            if fix_constraint:
+                ifL(
+                    entity('x'),
+                    atMostL(
+                        person(path=('x')),
+                        organization(path=('x')),
+                        location(path=('x')),
+                        date(path=('x')),
+                        other_entity_types(path=('x')),
+                        1
+                    ),
+                    name="constraint_only_one_entity_fixed"
+                )
+            else:
+                ifL(
+                    entity('x'),
+                    atMostL(
+                        1,
+                        person(path=('x')),
+                        organization(path=('x')),
+                        location(path=('x')),
+                        date(path=('x')),
+                        other_entity_types(path=('x'))
+                    ),
+                    name="constraint_only_one_entity"
+                )
+
+    return graph
diff --git a/test_regr/graph_errors/graph_concept_in_path.py b/test_regr/graph_errors/graph_concept_in_path.py
new file mode 100644
index 000000000..d98094b86
--- /dev/null
+++ b/test_regr/graph_errors/graph_concept_in_path.py
@@ -0,0 +1,58 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+
+    with Graph('global') as graph:
+        with Graph('linguistic') as ling_graph:
+            text_document = Concept(name='text_document')
+        
+        with Graph('application', auto_constraint=False) as app_graph:
+            entity = Concept(name='entity')
+            person = entity(name='person', auto_constraint=False)
+            organization = entity(name='organization', auto_constraint=False)
+            location = entity(name='location', auto_constraint=False)
+            date = entity(name='date')
+            other_entity_types = entity(name='other_entity_types')
+        
+            relation = Concept(name='relation')
+            employment = relation(name='employment')
+            located = relation(name='located')
+            part_whole = relation(name='part-whole')
+            personal_social = relation(name='personal-social')
+            other_relation = relation(name='other')
+        
+            pair = Concept(name='pair')
+            (rel_pair_entity1, rel_pair_entity2) = pair.has_a(arg1=entity, arg2=entity)
+        
+            if fix_constraint:
+                ifL(
+                    entity('x'),
+                    atMostL(
+                        person(path=('x')),
+                        organization(path=('x')),
+                        location(path=('x')),
+                        date(path=('x')),
+                        other_entity_types(path=('x')),
+                        1
+                    ),
+                    name="constraint_only_one_entity_fixed"
+                )
+            else:
+                ifL(
+                    entity('x'),
+                    atMostL(
+                        person(path=('x', entity)),
+                        organization(path=('x', entity)),
+                        location(path=('x', entity)),
+                        date(path=('x', entity)),
+                        other_entity_types(path=('x', entity)),
+                        1
+                    ),
+                    name="constraint_only_one_entity"
+                )
+
+    return graph
diff --git a/test_regr/graph_errors/graph_nli_1.py b/test_regr/graph_errors/graph_nli_1.py
new file mode 100644
index 000000000..e176ce770
--- /dev/null
+++ b/test_regr/graph_errors/graph_nli_1.py
@@ -0,0 +1,50 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL, notL
+from domiknows.graph.concept import EnumConcept
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+        
+    with Graph('global') as graph:
+        with Graph('linguistic') as lgraph:
+            pair = Concept(name='pair')
+            premise = Concept(name='premise')
+            hypothesis = Concept(name='hypothesis')
+            (rel_pair_premise, rel_pair_hypothesis) = pair.has_a(arg1=premise, arg2=hypothesis)
+
+        with Graph('nli_graph') as ngraph:
+            nli_class = pair(name='nli_class', ConceptClass=EnumConcept,
+                            values=['entailment', 'contradiction', 'neutral'])
+
+            if fix_constraint:
+                ifL(
+                    andL(
+                        pair('x'),
+                        pair('y'),
+                        pair(path=('x', rel_pair_hypothesis, rel_pair_hypothesis.reversed)),
+                        pair(path=('y')),
+                    ),
+                    ifL(
+                        nli_class.entailment(path=('x')),
+                        notL(nli_class.contradiction(path=('y')))
+                    ),
+                    name="pair_symmetry_constraint"
+                ) 
+            else:
+                ifL(
+                    andL(
+                        pair('x'),
+                        pair('y'),
+                        pair(path=('x', rel_pair_premise, rel_pair_hypothesis.reversed)),
+                        pair(path=('y')),
+                    ),
+                    ifL(
+                        nli_class.entailment(path=('x')),
+                        notL(nli_class.contradiction(path=('y')))
+                    ),
+                    name="pair_symmetry_constraint"
+                ) 
+
+    return graph           
\ No newline at end of file
diff --git a/test_regr/graph_errors/graph_nli_2.py b/test_regr/graph_errors/graph_nli_2.py
new file mode 100644
index 000000000..80f91901e
--- /dev/null
+++ b/test_regr/graph_errors/graph_nli_2.py
@@ -0,0 +1,50 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL, notL
+from domiknows.graph.concept import EnumConcept
+
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+    with Graph('global') as graph:
+        with Graph('linguistic') as lgraph:
+            pair = Concept(name='pair')
+            premise = Concept(name='premise')
+            hypothesis = Concept(name='hypothesis')
+            (rel_pair_premise, rel_pair_hypothesis) = pair.has_a(arg1=premise, arg2=hypothesis)
+
+        with Graph('nli_graph') as ngraph:
+            nli_class = pair(name='nli_class', ConceptClass=EnumConcept,
+                            values=['entailment', 'contradiction', 'neutral'])
+
+            if fix_constraint:
+                ifL(
+                    andL(
+                        pair('x'),
+                        pair('y'),
+                        pair(path=('x')),
+                        pair(path=('y')),
+                    ),
+                    ifL(
+                        nli_class.entailment(path=('x')),
+                        notL(nli_class.contradiction(path=('y')))
+                    ),
+                    name="pair_symmetry_constraint"
+                )
+            else:
+                ifL(
+                    andL(
+                        pair('x'),
+                        pair('y'),
+                        pair(path=('x', rel_pair_premise)),
+                        pair(path=('y', rel_pair_premise.reversed)),
+                    ),
+                    ifL(
+                        nli_class.entailment(path=('x', rel_pair_hypothesis)),
+                        notL(nli_class.contradiction(path=('y', rel_pair_hypothesis.reversed)))
+                    ),
+                    name="pair_symmetry_constraint"
+                )
+
+    return graph
\ No newline at end of file
diff --git a/test_regr/graph_errors/graph_nli_3.py b/test_regr/graph_errors/graph_nli_3.py
new file mode 100644
index 000000000..98b0866c3
--- /dev/null
+++ b/test_regr/graph_errors/graph_nli_3.py
@@ -0,0 +1,51 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL, notL
+from domiknows.graph.concept import EnumConcept
+
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+    
+    with Graph('global') as graph:
+        with Graph('linguistic') as lgraph:
+            pair = Concept(name='pair')
+            premise = Concept(name='premise')
+            hypothesis = Concept(name='hypothesis')
+            (rel_pair_premise, rel_pair_hypothesis) = pair.has_a(arg1=premise, arg2=hypothesis)
+
+        with Graph('nli_graph') as ngraph:
+            nli_class = pair(name='nli_class', ConceptClass=EnumConcept,
+                            values=['entailment', 'contradiction', 'neutral'])
+
+            if fix_constraint:
+                ifL(
+                    andL(
+                        pair('x'),
+                        pair('y'),
+                        pair(path=('x')),
+                        pair(path=('y')),
+                    ),
+                    ifL(
+                        nli_class.entailment(path=('x')),
+                        notL(nli_class.contradiction(path=('y')))
+                    ),
+                    name="pair_symmetry_constraint"
+                ) 
+            else:
+                ifL(
+                    andL(
+                        pair('x'),
+                        pair('y'),
+                        pair(path=('x', rel_pair_premise)),
+                        pair(path=('y', rel_pair_premise)),
+                    ),
+                    ifL(
+                        nli_class.entailment(path=('x')),
+                        notL(nli_class.contradiction(path=('y')))
+                    ),
+                    name="pair_symmetry_constraint"
+                ) 
+
+    return graph
\ No newline at end of file
diff --git a/test_regr/graph_errors/graph_path.py b/test_regr/graph_errors/graph_path.py
new file mode 100644
index 000000000..043d7d8b7
--- /dev/null
+++ b/test_regr/graph_errors/graph_path.py
@@ -0,0 +1,56 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+    
+    GRAPH_CONSTRAIN = True  ### This can be either True or False 
+    with Graph('global') as graph:
+        ### The definition of a linguistic graph is based on the base concepts
+        with Graph('linguistic') as ling_graph:
+            text_document = Concept(name='text_document')
+            
+        ### Defining the graph information of the application based on the task concepts
+        with Graph('application', auto_constraint=False) as app_graph:
+            named_entity = Concept(name='named_entity')
+            person = named_entity(name='person', auto_constraint=False)
+            organization = named_entity(name='organization', auto_constraint=False)
+            location = named_entity(name='location', auto_constraint=None)
+            date = named_entity(name='date')
+            miscellaneous = named_entity(name='miscellaneous')
+            
+            relation = Concept(name='relation')
+            employment = relation(name='employment')
+            located = relation(name='located')
+            part_whole = relation(name='part-whole')
+            personal_social = relation(name='personal-social')
+            other_relation = relation(name='other')
+            
+            ### The 'pair' relationship would connect two named entities to each other
+            pair = Concept(name='pair')
+            ### has_a relationship is used to indicate a many-to-many relationship between two named entities stored in variables of type pair
+            (rel_pair_entity1, rel_pair_entity2) = pair.has_a(arg1=named_entity, arg2=named_entity)
+            
+            ### Defining the constraints
+            if fix_constraint:
+                ifL(
+                    pair('x'), 
+                    andL(
+                        person(path=('x', rel_pair_entity1)),
+                        organization(path=('x', rel_pair_entity2))
+                    ),
+                    name="LC_employment"
+                )
+            else:
+                ifL(
+                    employment('x'), 
+                    andL(
+                        person(path=('x', rel_pair_entity1)),
+                        organization(path=('x', rel_pair_entity2))
+                    ),
+                    name="LC_employment"
+                )
+            
+    return graph
diff --git a/test_regr/graph_errors/graph_path_reverse.py b/test_regr/graph_errors/graph_path_reverse.py
new file mode 100644
index 000000000..442800bba
--- /dev/null
+++ b/test_regr/graph_errors/graph_path_reverse.py
@@ -0,0 +1,56 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL, orL
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+    
+    GRAPH_CONSTRAIN = True  ### This can be either True or False 
+    with Graph('global') as graph:
+        ### The definition of a linguistic graph is based on the base concepts
+        with Graph('linguistic') as ling_graph:
+            text_document = Concept(name='text_document')
+            
+        ### Defining the graph information of the application based on the task concepts
+        with Graph('application', auto_constraint=False) as app_graph:
+            named_entity = Concept(name='named_entity')
+            person = named_entity(name='person', auto_constraint=False)
+            organization = named_entity(name='organization', auto_constraint=False)
+            location = named_entity(name='location', auto_constraint=None)
+            date = named_entity(name='date')
+            miscellaneous = named_entity(name='miscellaneous')
+            
+            ### The 'pair' relationship would connect two named entities to each other
+            pair = Concept(name='pair')
+            ### has_a relationship is used to indicate a many-to-many relationship between two named entities stored in variables of type pair
+            (rel_pair_entity1, rel_pair_entity2) = pair.has_a(arg1=named_entity, arg2=named_entity)
+            
+            employment = pair(name='employment')
+            located = pair(name='located')
+            part_whole = pair(name='part-whole')
+            personal_social = pair(name='personal-social')
+            other_relation = pair(name='other')
+            
+            
+            ### Defining the constraints
+            if fix_constraint:
+                ifL(
+                    person('x'), 
+                    orL(
+                        employment(path=('x', rel_pair_entity1.reversed)),
+                        personal_social(path=('x', rel_pair_entity1.reversed))
+                    ),
+                    name="LC_person_attendance"
+                )
+            else:
+                ifL(
+                    person('x'), 
+                    orL(
+                        employment(path=('x', rel_pair_entity1)),
+                        personal_social(path=('x', rel_pair_entity1))
+                    ),
+                    name="LC_person_attendance"
+                )
+            
+    return graph
diff --git a/test_regr/graph_errors/graph_relations.py b/test_regr/graph_errors/graph_relations.py
new file mode 100644
index 000000000..f941a5c58
--- /dev/null
+++ b/test_regr/graph_errors/graph_relations.py
@@ -0,0 +1,34 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+
+    with Graph('CIFAR10Graph') as graph:
+        image_batch = Concept(name='image_batch')
+        image = Concept(name='image')
+        rgb = Concept(name='RGB')
+        class_label = Concept(name='class_label')
+
+        (rel_image_batch_contains_image, rel_image_batch_contains_rgb) = image_batch.contains(image, rgb)
+        (rel_image_has_class_label,) = image.has_a(arg1=class_label)
+
+        airplane = class_label(name='airplane')
+        automobile = class_label(name='automobile')
+        bird = class_label(name='bird')
+        cat = class_label(name='cat')
+        deer = class_label(name='deer')
+        dog = class_label(name='dog')
+        frog = class_label(name='frog')
+        horse = class_label(name='horse')
+        ship = class_label(name='ship')
+        truck = class_label(name='truck')
+
+        animal = class_label(name='animal')
+        vehicle = class_label(name='vehicle')
+        fine_grained_animal = class_label(name='fine_grained_animal')
+        fine_grained_vehicle = class_label(name='fine_grained_vehicle') 
+
+    return graph
diff --git a/test_regr/graph_errors/graph_variable_resue.py b/test_regr/graph_errors/graph_variable_resue.py
new file mode 100644
index 000000000..b9c4a3215
--- /dev/null
+++ b/test_regr/graph_errors/graph_variable_resue.py
@@ -0,0 +1,48 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, orL, nandL, atMostL, existsL
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+
+    with Graph('global') as graph:
+        with Graph('linguistic') as ling_graph:
+            text_document = Concept(name='text_document')
+        
+        with Graph('application', auto_constraint=False) as app_graph:
+            entity = Concept(name='entity')
+            person = entity(name='person', auto_constraint=False)
+            organization = entity(name='organization', auto_constraint=False)
+            location = entity(name='location', auto_constraint=False)
+            date = entity(name='date')
+            other_entity_types = entity(name='other_entity_types')
+        
+            relation = Concept(name='relation')
+            employment = relation(name='employment')
+            located = relation(name='located')
+            part_whole = relation(name='part-whole')
+            personal_social = relation(name='personal-social')
+            other_relation = relation(name='other')
+        
+            pair = Concept(name='pair')
+            (rel_pair_entity1, rel_pair_entity2) = pair.has_a(arg1=entity, arg2=entity)
+        
+            if fix_constraint:
+                ifL(
+                    orL(
+                        person('x'),
+                        organization(path=('x')),
+                    ),
+                    name="constraint_only_one_entity_fixed"
+                ),  
+            else:
+                ifL(
+                    orL(
+                        person('x'),
+                        organization('x'),
+                    ),
+                    name="constraint_only_one_entity_fixed"
+                ),
+
+    return graph
diff --git a/test_regr/graph_errors/graph_variable_type.py b/test_regr/graph_errors/graph_variable_type.py
new file mode 100644
index 000000000..f7ed8fa9b
--- /dev/null
+++ b/test_regr/graph_errors/graph_variable_type.py
@@ -0,0 +1,38 @@
+from domiknows.graph import Graph, Concept, Relation
+from domiknows.graph.logicalConstrain import ifL, andL, nandL, atMostL, existsL, notL
+from domiknows.graph.concept import EnumConcept
+
+
+def setup_graph(fix_constraint=False):
+    Graph.clear()
+    Concept.clear()
+    Relation.clear()
+    
+    with Graph('Accident_and_Weather_Conditions') as graph:
+
+        # Basic Concepts
+        accident_details = Concept(name='accident_details')
+        weather_details = Concept(name='weather_details')
+
+        # Task Concepts
+        accident_cause = accident_details(name='accident_cause', ConceptClass=EnumConcept, 
+                                        values=['human error', 'mechanical failure', 'road conditions', 'weather conditions', 'other'])
+        weather_condition = weather_details(name='weather_condition', ConceptClass=EnumConcept, 
+                                        values=['clear', 'cloudy', 'rainy', 'snowy', 'foggy', 'stormy', 'other'])
+        
+        # Constraints
+        # If the accident cause is weather condition, then the weather quality is not clear.
+        
+        if fix_constraint:
+            ifL(
+                accident_cause.__getattr__('weather conditions')('x'),
+                notL(weather_condition.__getattr__('clear')('y'))
+            )
+        else:
+            ifL(
+                accident_cause.__getattr__('weather conditions')('x'),
+                notL(weather_condition.__getattr__('clear')(path=('x'))),
+                name = "testLC"
+            )
+
+    return graph
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_cardinality.py b/test_regr/graph_errors/test_graph_cardinality.py
new file mode 100644
index 000000000..df6886f04
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_cardinality.py
@@ -0,0 +1,29 @@
+import pytest
+from graph_cardinality import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+        sanitized_pattern = re.sub(r'[^\x20-\x7E]', '', 
+                                   "Logical Constraint constraint_only_one_entity has incorrect cardinality definition in nested atMostL logical operator - "
+                                   "integer 1 has to be last element in the same Logical operator for counting or existing logical operators!").replace(" ", "")
+        
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+
+        ### fixed with ChatGPT
+        ##### Logical Constraint constraint_only_one_entity has incorrect cardinality definition in nested atMostL logical operator - 
+        # integer 1 has to be last element in the same Logical operator for counting or existing logical operators!
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_concept_in_path.py b/test_regr/graph_errors/test_graph_concept_in_path.py
new file mode 100644
index 000000000..679d83d46
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_concept_in_path.py
@@ -0,0 +1,33 @@
+import pytest
+from graph_concept_in_path import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+        sanitized_pattern = re.sub(r'[^\x20-\x7E]', '', 
+                                   "You have used the notion person(path=('x', entity)) which is incorrect."
+                                   "entity is a concept and cannot be used as part of the path."
+                                   "- If you meant that 'x' should be of type person: person(path=('x'))"
+                                   "- If you meant another entity 'y' should be of type person which is somehow related to 'x': person(path=('x', edge1, edge2, ...))"
+                                   "where edge1, edge2, ... are relations that connect 'x' to 'y'.").replace(" ", "")
+        
+        ### you have used the notion person(path=('x', entity)) which is incorrect.
+        ### entity is a concept and cannot be used as part of the path. 
+        ### - If you meant that 'x' should be of type person: person(path=('x'))
+        ### - if you meant another entity 'y' should be of type person which is somehow related to 'x': person(path=('x', edge1, edge2, ...)) 
+        ###   where edge1, edge2, ... are relations that connect 'x' to 'y'
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_nli_1.py b/test_regr/graph_errors/test_graph_nli_1.py
new file mode 100644
index 000000000..e6b82584b
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_nli_1.py
@@ -0,0 +1,32 @@
+import pytest
+from graph_nli_1 import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+
+        sanitized_pattern = re.sub(
+            r'[^\x20-\x7E]', 
+            '', 
+            ("The Path 'rel_pair_premise rel_pair_hypothesis.reversed' from the variable x, defined in pair_symmetry_constraint is not valid."
+             "The required source type after x rel_pair_premise is a premise,"
+             "but the used variable rel_pair_hypothesis.reversed is a relationship defined between a hypothesis and a pair, which is not correctly used here.")
+        ).replace(" ", "")
+        
+        print(sanitized_pattern)
+
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_nli_2.py b/test_regr/graph_errors/test_graph_nli_2.py
new file mode 100644
index 000000000..fee53bf01
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_nli_2.py
@@ -0,0 +1,27 @@
+import pytest
+from graph_nli_2 import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+        sanitized_pattern = re.sub(r'[^\x20-\x7E]', 
+                                   '', 
+                                   "The Path 'rel_pair_premise' from the variable x, defined in pair_symmetry_constraint is not valid."
+                                   "The required destination type after x is a pair."
+                                   "The used variable rel_pair_premise is a relationship defined between a pair and a premise, which is not correctly used here.").replace(" ", "")
+        
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_nli_3.py b/test_regr/graph_errors/test_graph_nli_3.py
new file mode 100644
index 000000000..5c0117878
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_nli_3.py
@@ -0,0 +1,30 @@
+import pytest
+from graph_nli_3 import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+
+        sanitized_pattern = re.sub(
+            r'[^\x20-\x7E]', 
+            '', 
+            ("The Path 'rel_pair_premise' from the variable x, defined in pair_symmetry_constraint is not valid."
+             "The required destination type after x is a pair."
+             "The used variable rel_pair_premise is a relationship defined between a pair and a premise, which is not correctly used here.")
+        ).replace(" ", "")
+        
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_path.py b/test_regr/graph_errors/test_graph_path.py
new file mode 100644
index 000000000..f7fabdac5
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_path.py
@@ -0,0 +1,27 @@
+import pytest
+from graph_path import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+        sanitized_pattern = re.sub(r'[^\x20-\x7E]', 
+                                   '', 
+                                   "The Path 'rel_pair_entity1' from the variable x, defined in LC_employment is not valid."
+                                   "The required source type after x is a relation," 
+                                   "but the used variable rel_pair_entity1 is a relationship defined between a pair and a named_entity, which is not correctly used here.").replace(" ", "")
+        
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_path_reverse.py b/test_regr/graph_errors/test_graph_path_reverse.py
new file mode 100644
index 000000000..95bddcf98
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_path_reverse.py
@@ -0,0 +1,31 @@
+import pytest
+from graph_path_reverse import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+        sanitized_pattern = re.sub(r'[^\x20-\x7E]', 
+                                   '', 
+                                   "The Path 'rel_pair_entity1' from the variable x, defined in LC_person_attendance is not valid"
+                                   "The relation rel_pair_entity1 is from a pair to a named_entity, but you have used it from a named_entity to a pair." 
+                                   "You can change 'rel_pair_entity1' to 'rel_pair_entity1.reversed' to go from named_entity to the pair, which is what is required here.").replace(" ", "")
+        
+        ### The Path 'rel_pair_entity1' from the variable x, defined in LC_person_attendance is not valid.
+        ### The relation rel_pair_entity1 is from a pair to a named_entity, but you have used it from a named_entity to a pair. 
+        ### You can change `rel_pair_entity1` to `rel_pair_entity1.reversed` to go from named_entity to the pair, which is what is required here.
+        
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_variable_reuse.py b/test_regr/graph_errors/test_graph_variable_reuse.py
new file mode 100644
index 000000000..c224b0d6f
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_variable_reuse.py
@@ -0,0 +1,30 @@
+import pytest
+from graph_variable_resue import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+        sanitized_pattern = re.sub(r'[^\x20-\x7E]', '', 
+                                   "In logical constraint constraint_only_one_entity_fixed orL variable x associated with concept organization already defined "
+                                   "in orL and associated with concept person").replace(" ", "")
+        
+        ### you have used the notion person(path=('x', entity)) which is incorrect.
+        ### entity is a concept and cannot be used as part of the path. 
+        ### - If you meant that 'x' should be of type person: person(path=('x'))
+        ### - if you meant another entity 'y' should be of type person which is somehow related to 'x': person(path=('x', edge1, edge2, ...)) 
+        ###   where edge1, edge2, ... are relations that connect 'x' to 'y'
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file
diff --git a/test_regr/graph_errors/test_graph_variable_type.py b/test_regr/graph_errors/test_graph_variable_type.py
new file mode 100644
index 000000000..fc4acad87
--- /dev/null
+++ b/test_regr/graph_errors/test_graph_variable_type.py
@@ -0,0 +1,30 @@
+import pytest
+from graph_variable_type import setup_graph
+import re
+
+def test_setup_graph_exception():
+    try:
+        setup_graph()
+    except Exception as e:
+        sanitized_error_message = re.sub(r'[^\x20-\x7E]', '', str(e)).replace(" ", "")
+
+        sanitized_pattern = re.sub(
+            r'[^\x20-\x7E]', 
+            '', 
+            ("The variable x, defined in the path for testLC is not valid. The concept of x is a of type accident_details," 
+             "but the required concept by the logical constraint element is weather_details."
+             "The variable used inside the path should match its type with weather_details.")
+        ).replace(" ", "")
+        
+        print(repr(sanitized_error_message))
+        print(repr(sanitized_pattern))
+        
+        assert sanitized_error_message == sanitized_pattern, f"Exception message did not match: got {sanitized_error_message}"
+    else:
+        pytest.fail("Expected an exception but none was raised.")
+        
+def test_setup_graph_no_exception():
+    try:
+        setup_graph(fix_constraint=True)
+    except Exception as e:
+        pytest.fail(f"Unexpected Exception raised: {e}")
\ No newline at end of file