GitHub repository: IntegratedAlarmSystem-Group/ias

Issues:
issue_202: state: open
title: Kafka and zookeeper to store data in /var/lib
body: Actually the IAS configuration for Kafka and zookeeper writes data in `/opt/kafkadata` and `/opt/zookeeperdata` respectively.

This ticket is to:
- [ ] store kafa data in `/var/lib/kafka`
- [ ] store zookeeper data in `/var/lib/zookeeper`
- [ ] `startIasServices` and `stopIasServices` shall be updated for the new folders

The new folders, `/var/lib/kafka` and `/var/lib/zookeeper`, must be created as root and proper privileges and ownership assigned before running the services.
The procedure must be reported in the installation notes.

It must be taken into account that the configuration provided by IAS are for testing purposes only. The configuration in production should be tuned for your system.


user: acaproni
issue_201: state: open
title: The Multiplicity TF to take into account the validity of its inputs
body: The Multiplicity TF (MTF) sets the alarm in output when the number of set alarms in input is equal or greater than the threshold.
At the present, if one of the inputs is INVALID, the output of the MTF is also invalid.

The validity of the output should depend on the number of inputs that are valid as follows, assuming a threshold t:
- if the number of valid inputs that are SET is >=t then the output must be SET and valid
- if the number of invalid inputs (i) plus the number of valid SET inputs (s) is such that i+s<t then the output must be UNSET and valid
- for all other cases the output must be INVALID (because the invalid inputs can/cannot trigger the alarm in output)

The purpose of this ticket is to fix the cases when the invalid inputs do not affect the output i.e. when a) the threshold is passed independently of invalid inputs or b) cannot be passed even if the invalid inputs would be set.
user: acaproni
issue_200: state: open
title: Create an example/tutorial
body: IAS should provide a complete example for newcomers.
An example would help understanding how to generate and display alarms starting from scratch.

The example shall show how 
- [ ] to create and configure the CDB to generate one alarm running a single Converter, a single ComputingElement, a single Dasu and a single Supervisor (i.e. the simplest scenario)
- [ ] check the correctness of the CDB
- [ ] run the IAS Converter
- [ ] run the IAS Supervisor that generates the alarm of the example
- [ ] get (or display) the alarms
user: acaproni
issue_199: state: open
title: Allow auto-ACK of alarms for specific alarms
body: For certain types of alarms, the operators might not be requested to ACK them: the IAS should allow to automatically ACK alarms that are properly flagged.
By default an alarm that is not flagged requires an ACK from the operator to move from the cleared to the nominal state.

This ticket requires
- a modification of the CDB to be able to mark an Alarm IASIO as automatically acknowledgeable
- a change in the state machine to pass directly from the Cleared to the normal state
user: acaproni
issue_198: state: open
title: Test the IAS against more instances of Kafka
body: Up to now the IAS have been used with only one kafka server but it should be tested against more servers i.e. more servers in brokers property.

For this test, we need to run kafka in a at least a couple of hosts. IAS tools shall be able to connect and get and put messages in the kafka topics. IAS tools could be deployed in a host where kafka is not installed.

This test should 
- check if it is possible to save the urls of more kafka server in the configuration and if they are properly returned by the CDB library
- check that the IAS tools connect to all the servers
- check that messages are red and written from the topics (that might be located in a host other than that where the tools run)
- check what happens when one kafka server is shut down (i.e. check if replication works).
user: acaproni
issue_194: state: open
title: The Alarm should not expose ack() to the TF
body: In the implementation of the acknowledgement (#167) , the Alarm class exposes the ack() method that would allow a misbehaving TF to ack an alarm programmatically: the ACK shall only be performed in response to an operator action i.e. a command issued by the alarm GUI for example.
user: acaproni
issue_193: state: open
title: Extend the value of the `InOut` for UNKNOWN, ERROR, BROKEN use cases
body: This ticket replaces #170 and #169 because they must be fixed in the same context.

In the actual implementation (v12), the value of a InOut[T] is a `Option[_>: T]`. None (that corresponds to a `null` value in `IASValue`) is used only when the value has not yet been set (for example the ASCE has never executed the TF). `Some(value)` represents the case when the value is correctly produced. The error in the TF is handled by the ASCE that is marked as faulty and its TF never executed again so an error is never sent to the BSDB. 
There are 2 disadvantages in handling the error in this way:
1. the latest good value produced by the TF slowly becomes UNRELIABLE hiding that fact the there has been an error in the TF and debugging is complicate because the fact that a value is UNRELIABLE does not necessarily mean that there has been an error in the TF
3. the fact that a TF failed once does not mean that it fails all the time (for example a division by 0 can happen only for certain specific values)

This ticket is to change that value in the `InOut` to catch the following cases:
- UNKNOWN: the value is not known because the TF has never been executed (corresponds to the `None` of the actual implementation)
- ERROR: the TF returned an error for example a division by 0 or an exception in TF (in the actual implementation the TF throws an exception that is caught by the ASCE: the ASCE is marked as malfunctioning and the TF is never executed again but the error is not propagated in the BSDB)
- BROKEN: the TF was too slow and the ASCE has been marked as broken (this would help debugging but c
- Value[T]: the TF produced a valid output (corresponds to the `Some` of the actual implementation.

Note that:
- [ ] the ASCEs (and the plugins) are supposed to send the value even if the value is UNKNOWN or there has been an error
- [ ] in case of error, the ASCE must not be marked as broken but it must try to run the TF again and produce a good value or another error (to catch the case of a TF that fails only for a subset of inputs)
- [ ] when the TF is too slow and the ASCE is marked as broken, it shall produce a `BROKEN` value
- [ ] the validity shall not be touched in case of UNKNOWN, ERROR, BROKEN if they are produced inside the valid time frame: the validity should not be modified in the context of this ticket
- [ ] the state of UNKNOWN, BROKEN or ERROR shall be passed to the `IASValue` to be propagated to the BSDB and from there to DASUs and ASCEs
- [ ] the LTDB shall be updated to cope with the new definition of the value

The result of an ASCE being UNKNOWN|ERROR|BROKEN|Value[T] can be shown in the alarm panel to let the operators (and software developers) understand what is happening in the alarm system.
user: acaproni
labels: E-ELT request: Changes triggered by the ELT protype
issue_191: state: open
title: Converters to have an entry in the CDB
body: Converters should have an entry in the CDB as other tools have.
The entry, based on the ID of the converter, should, at least, allow to
* configure the log level
* configure the size of the in-memory cache (@see #168)
* allow to define the Kafka group and ID (this would allow to partition the load)

The CdbChecker should be updated to cope with this change in the CDB.

This change is backward incompatible meaning with that that Converters without a deinition in the CDB will not start.
user: acaproni
issue_187: state: open
title: CdbChecker to check template constraints on ASCEs and DASUs
body: The CdbCheck shall check the setting of templates in the ASCEs and DASUs as descibed in the [comment of the issue #80](https://github.com/IntegratedAlarmSystem-Group/ias/issues/80#issuecomment-380602168)
user: acaproni
issue_185: state: open
title: CdbChecker to check correctness of plugin configurations
user: acaproni
issue_173: state: open
title: Check group.id of kafka consumers
body: This ticket is to check that the group.id property of clients are properly set taking into account that 
* IAS tools that consume all the events of a topic (like the Supervisor) must have a unique `group.id`
* IAS tools that collaborate consuming events (like the converters) must share the same `group.id`

From a short check the Supervisors already define a unique `group.id` (#113) but the HB consumer of the IasMonitor does not.

To be in the safe side and make the used group explicit, the group identifier shall be added to the constructor or the start methods of all the kafka consumer.
user: acaproni
issue_151: state: open
title: The core to provide a python API to read JSON CDB
body: Actually the display reads JSON CDB with its own python classes because the core does not provide them.
However in this way the defaults values are not available in the display that has to duplicate them.
This potentially create a misalignment difficult to identify.

A good starting point could be of moving python sources from the display to the core.
user: acaproni
issue_148: state: open
title: IAS monitor should send alarms directly to the web server
body: Actual implementation of the IAS monitoring tool (@see #126) publish alarms in the BSDB like DASU and plugins do. For this to work the Monitor seems to simulate the sending of the alarms by a plugin i.e. the full running ID of such IASIOs contain a PLUGIN and CONVERTER ID that do not exist).
To avoid such workaround, the ID of the IASIOs must be reviewed as anticipated in #145 

Alarms could be send with websockets for example or other means. Adding anew notification method should be straightforward because the implementation is decoupled by a interface (a trait as it is scala code)
user: acaproni
issue_147: state: open
title: IasMonitor to check Kafka connectors
body: The first version of IAS monitoring has been released with v7. It monitor all the IAS tools that send heartbeats.
It should also monitor kafka connectors that do not publish HBs but whose status can be queried with the kafka REST interface.
This document seems very useful: https://docs.confluent.io/current/connect/managing/monitoring.html
user: acaproni
issue_146: state: open
title: Allow the plugins to set additional properties to monitor points
body: It would be useful if the plugin allows to set additional properties to the monitor points. 
The converter mast pass such properties the the converted IASIOs that already support that feature.
TFs can already get the properties of the inputs

Properties must be i the same format used along the core i.e. a map of key, value strings.

Python plugin must support this feature too. 

The Additional properties field in the JSON strings sent to the plugin topic must be optional to avoid backward incompatibilities.

This feature allows the plugin to send more information than the value of a monitor point.
user: acaproni
issue_143: state: open
title: The IASIO configuration of the CDB to provide a TAG to show instead of the ID in the panels
body: The ID of an IASIO is currently displayed in the panels to identify alarms. However the identifier is selected depending on computational needs and as such shows internal information and does not bring useful information, is difficult to read and could also be misleading.
The decouple the internals of the IAS from the information displayed by the panels, the configuration of the IASIO in the CDB should, optionally, provide a TAG to display in the GUIs instead of the identifier. If the TAG is not provided the identifier can be used to reduce the editing of the CDB even if discouraged.

For the tamplate, there must be a way to map the tempalted identifier into a templated TAG that remains generic, clear and human readable when translated in concrete identifier.
At the present a templated identifers looks like `WindSpeed[!#4!]` that is hard to read. Specialized GUIs already musk that IDs to increase readability but it shows up in the tables for example and it can be hard to correlate the alarm displayed in one way in a panel with those in the tables.
If templated TAG is `PS of DA` it can be mapped to `PS of DA45` for the concrete instance 25 that is much more readable than `Araay_PS_DA[!#45!]` identifier used internally


user: acaproni
issue_134: state: open
title: PATH environment variables has  a relative path
body: In line 51 of the  $IAS_SOURCE/Tools/config/ias-bash-profile.sh

PATH="../../bin:$IAS_ROOT/bin:$JAVA_HOME/bin:$JRE_HOME/bin:$SCALA_HOME/bin:$PATH"

This causes that several binaries fails to execute due to non existent directories. "../../bin" does not always exists.
user: ahoffstadt
issue_125: state: open
title: Extends the definition of templates to allow list of values
body: Actually the definition of a template is composed of the identifier plus a range  (`[min,max]`) of possible instances.

It would be useful to allow also a list of values like
* 1,5,7,12
* Name1, Name2, Name 3

This ticket has been generated after a meeting with ALMA computing.
user: acaproni
issue_123: state: open
title: Improve template support from plugins
body: Actually it is possible to replicate a plugin by means of templates (#81). For example a plugin that get the status of the antenna `i` can be generated by the plugin `i`. In this case we will need to run one plugin for each replicated device.  Te use case of this replication is the plugin that gets the same monitor points by equal devices.
In ALMA we can think of a plugin that gets the status of one antenna.

This ticket is to support a different use case that is when one plugin gets the status of more devices but need to produce an output that can be used by replicated DASU.
In ALMA this is the case of the plugin that gets the status bits from the utility modules of *all* the antennas. This plugin is not replicated but we want the output to be replicated so that it can be the input of a replicated DASUs.

Actually a plugin can simply produce output by following the replication pattern describe in #80 but as it could possibly change, we should provide the plugin with a proper API.

user: acaproni
issue_120: state: open
title: Need to persist information when restarting ASCEs
body: Actually the ASCEs are stateless: they produce the output depending on the inputs whose updated value is eventually provided by the plugins or ASCEs/DASUs that constantly send the last computed value to the BSDB. If a DASU is restarted, it will produce the output after receiving the values of all of its inputs.

While this mechanism works most of the times, there are situations that need something more.
One example is provided by the wind speed: when it passes 30m/s a visual inspection is required. The ASCE can easily remember the point in time when the speed passed the threshold and compare with the time of the last inspection. If the wind speed drops below the threshold AND the visual inspection has been done then the alarm will be cleared.

But what if the DASU is restarted when the speed drops below 30 and the visual inspection has not yet been done? The newly restarted DASU will not set the alarm because the speed is below 30 even if the visual inspection has not yet been done. For this kind of situations we would like to persist the information so that the newly restarted DASU can set the alarm properly. In this case the DASU/ASCE must save the time when the wind speed passed the threshold of 30m/s and the time of the last visual inspection.

Note that the saving of the state is needed only to support the restarting of ASCEs/DASUs/Supervisors whose values of the inputs depends on something that happened in the past and may not be true at the present. I.e. persisting of information is not necessary if Supervisors are not restarted.


user: acaproni
issue_117: state: open
title: Add a global refresh rate in the plugin config file
body: To simplify the JSON configuration file of plugins, we could add a global refresh rate used by all the monitor point of the plugin. At the same time, such global refresh rate can be overridden by setting the refresh rate of individual monitor points.
user: acaproni
issue_105: state: open
title: DASUs and ASCEs should not process IASValue but InOut
body: `IASValue` is the view of an alarm or monitor point in the BSDB but Supervisors, DASUs and ASCEs all work wiith `InOut`.

The DASU shall convert `IASValues` in input to 'InOut and avoid propagating only `InOut` objectss between the ASCEs . This would avoid continuously converting `IASValue` to `InOut` and vice-versa while running the TFs of the ASCEs.
user: acaproni
issue_100: state: open
title: Logging starting IAS tools need improvements
body: `IASRun` starts IAS tools like supervisor or converters. It writes a log file into `$IAS_ROOT/log` whose name is like `iasRun2018-05-07_17:32:49.565.log`.

Browsing the logs files, I found that they only contain a single like: 
```
2018-05-07 14:33:09.360  | INFO | [iasRun 136] [MainThread] | Start IASRun
```
from which we cannot understand what tool did `IASRun` start.

This is due to the fact that the scripts to start tools like `iasSupervisor` runs a bash script (that does not produce a log) and only at the end call `IASRun`that does produce a log.

To fix this ticket we should replace the bash scripts like `iasSupervisor.sh` with python scripts. In this way there will be a log file with the name of the python script that is starting.

`IASRun.py`should be enhanced so that it log (at INFO) level some information about the call it is starting and which parameters, java properties and so on will be passed top the tool


user: acaproni
issue_84: state: open
title: Generation of time-dependent alarms
body: At the present the ASCEs produce the output when requested by the Supervisors/DASUs where they run. It happens when
* a new set of inputs has been provided
* a time interval elapsed

If none of the 2 conditions is true the output remains valid and its value actual: the execution of the TF is not triggered by the Supervisor.

While looking at the alarms generated by the weather conditions, we have seen that an alarm must be generated if the wind speed is greater then a threshold for a given amount of seconds, 12 seconds in this specific case.
With the actual implementation, the TF is able to detect that the delay elapsed only when one of the 2 conditions become true that can be too late.

The case of the wind speed is a valid use case. An alarm must be produced if the wind speed is greater then the threshold for more then 12 seconds.
The automatic refresh is set to 5 seconds by default so, if the wind speed does not change the shortest time to produce the alarm will be 15 seconds (3 times the automatic refresh).

The ASCE is not autonomous and cannot wake up and offer a new value after 12 seconds. The Supervisor and the DASU have no knowledge on what TF is executed by the ASCEs so they cannot tune the refresh at the time frame requested by the TF.
In this specific case, tuning the automatic refresh rate to 4 seconds instead of the default of 5 seconds would be enough. This can easily be done by setting a java property in the supervisor.

At a first glance, implementing this ticket is not trivial and I am not sure that generating an alarm of this type 3 seconds later then the required 12 seconds would really make a difference especially considering that the automatic refresh rate of the Supervisor can be tuned by setting a java property.
On the other hand, if the automatic refresh rate, set in the IAS, is changed to 10 seconds instead of the default 5 seconds then the alarm will be set about 20 seconds after the required time and that could be unacceptable.

For the time being, I keep this ticket open as a remainder for this issue. In future there could be more time-dependent use case to cope with.
user: acaproni
issue_78: state: open
title: Discuss and if it is the case serialize kafka records with apache avro
body: Changing the format of the JSON records published in the kafka topic often introduces compatibility issues in the web server sender.
One possible solution to mitigate could be the replacement of the JSON serialization of java pojos with [apache avro](http://avro.apache.org/).
We could actually keep both serialization in place and choose the one to use at run time. Plain JSON is easier to read so it is very useful for debugging.
Another big advantage of avro is the binding with different programming languages like python that can be useful developing guis with pyQt or for the web server.

I would like to discuss and if we all agree start using avro in the next release. I got inspired by reading the specification of confluent platform and in particular by [this](https://www.confluent.io/blog/avro-kafka-data/) post.
user: acaproni
issue_75: state: open
title: Let the DASU run the TFs of the ASCEs concurrently
body: The DASUs run the TFs of the ASCEs sequentially but they can be run concurrently at least for the ASCEs belonging to the same level. The implementation should not be very complicate by delegating the running of the TFs of the ASCEs to a `Feature` thread. 
The task to be run concurrently should not be limited to the TFs but also to the preparation of the data structures and the conversion of the result to be returned to the DASU.
user: acaproni
issue_66: state: open
title: IAS to provide a maven repository of jars
body: The core of the IAS should provide a maven repository of jars that would be useful for building third party tool like the plugins that are build without the need of building the entire core.

Having a maven repository of jar can also be a easy way to publish releases. We could let jenkins to populate the maven repository 

Eventually, we could replace ant with maven for building the core and populate the repository.


user: acaproni
issue_50: state: open
title: Review Supervisor, DASU and ASCE constructors
body: DASU and Supervisor constructors ignore the DAO and read it again with the passed CDB reader.

We should review and improve DASU and Supervisor constructor and uniform the ASCE constructor
user: acaproni
issue_17: state: open
title: IASIO type is defined twice
body: The type of a IASIO is defined twice: in the CDB and in the BasicTypes module.
One type is the carbon copy of the other but for different purposes: the type defined in Cdb, `IasTypeDao` is to store and get the type from the database; the type defined in the BasicTypes module, `IASTypes` is used at run-time to dynamically recognize the type of a monitor point alarm or values.

I am just writing helper methods to convert one type into another in the `IASTypes` enumerated.

To avoid converting from one type to another at run-time it would be a good idea to use only the type defined in the Cdb (possibly with a better name).
Using only one data type also ensure consistency when changing or extending the supported data type.

I also suggest to reduce the number of supported datatypes by either
* replace all floating point values with DOUBLE
* replace all integer flavors (SHORT, BYTE, etc.) with LONG
user: acaproni
issue_16: state: open
title: Plugin should flag the data produced by a remote system when their values are invalid
body: Raw data produced by a monitored software system should be filtered out if their values are not consistent.
For example if a temperature has a value of NaN or too high to be valid.

Open question: actually the validity of a monitor point or alarm produced by a remote system depends only on the time it has been produced. If it is too old the something must not be working as expected and we flag it as invalid. We should probably define another type of validity when the monitored system return values that have no meaning.
user: acaproni
issue_15: state: open
title: All messages should be signed
body: Messages sent by Plugins should be secured by using signatures and private-public pairs of keys.
user: sfehlandt
issue_8: state: open
title: Let the IAS components report metrics at runtime
body: Instead of implementing its own metric system, the IAS should adopt an existing metric reporting tool to provide statistics to monitor, at least, the most critical part of the code.
There are several tools for that. One that seems very popular is provided by 
[dropwizard](https://github.com/dropwizard/metrics).

user: acaproni
issue_2: state: open
title: Change CDB configuration to support mutliple scenarios
body: The ID of the IAS is an integer automatically generated by the DB. Other DB entities do not refer to the IAS (i.e. no foreign key) because there is only one instance of the IAS running at a given time.

By changing the ID of the IAS to a user provided string and adding foreign keys to the other tables, it would be possible to have more then one configured IAS in the database. At run time, it would be possible to select which IAS configuration to run providing the IAS identifier as  a java property or an environment variable. The simplest use case is to have one IAS for testing and one for production but this feature could be useful in operation to have for example different deployments.
user: acaproni
issue_1: state: open
title: Add delete capability to the CDB
body: The actual implementation of the CDB allows read and write/update items but does not allow to delete them.
Deletion should be added to the interfaces and implemented in RDB and file implementers.
user: acaproni