Repository: golemfactory/yagna (language: python)

Issue:
Certificate handling verification/tests
- [x] Text formatting change
- [x] Adding `ya-service-api` to `ya-provider` to print tables in cmdline output
- [x] Certificates related functional tests

Things to verify/test:
 - Tests/verification of handling certificates storage
   - startup with no certificates - error when manifest verification not disabled?
   - startup with same certs in storage but a different file name - warn, make sure there are no duplicates in openssl store (would it be possible, would openssl fail?)
   - startup with expired certificates - log it, but do we want to delete them on startup?
   - startup with certs missing root CA - log it, or fail?
   - do we want to keep param allowing to configure cert_dir?
   - fail when cert_dir does not exist, or create empty?
   - fail when no write permissions to cert dir (in case someone would want to configure `/etc/ssl/certs/` as a cert dir) ?
   - monitoring of cert storage:
     - deleted certificate disappears from internal store
     - added certificate appears in internal store
   
 - Tests/verification of handling certificates incoming in demand
   - rejection on untrusted/expired certificate
   - support of incoming certificate chains
     - can we handle them? (currently arriving PEM cert is parsed as a single cert, but it coule be parsed as a stack of certs, and then this stack probably could be passed as a chain to [X509StoreContext](https://docs.rs/openssl/latest/openssl/x509/struct.X509StoreContext.html#method.init) to verify it
     - do we want to support it?
     - if yes make sure it is not possible to pass root CA cert in such chain (malicious action).
     - if certificate chains are supported we should control issuing new certificates by third party using some x509v3 extension properties.
   - unsupported signature type - failure to verify can happen because Provider does not trust certificate Requestor uses, but also because Requestor had used wrong signature type (are there differences between openssl lib distributions / environments / versions?). From Requestor's point of view both cases will look the same. Maybe list of supported signatures should be hardcoded into API and yapapi?
   
- Tests/verification of certificates added/removed/listed through cmdline
  - Fail to add expired/invalid cert or just send warning message?
  - Do we want to list expired/invalid certificates (with e.g. warning messages)?
  - Do we want to support adding certificate chain (merged PEM certs)?
  - Do we want to support adding certificate by passing remote URL (probably not, if yes for now only by downloading them once)
  - Listed certificates keep their order (do not change position after consecutive list request, and, or deleting/adding)

General considerations:
  - Need to decide if we want to let third party sign other certificates (things like `pathlen`, keyUsage property, and policy to control who they can issue it to (organisation name etc.) - [x509v3 config](https://www.openssl.org/docs/manmaster/man5/x509v3_config.html))
  - There is a risk of accidental creation and leak of signed Manifest+signature+cert allowing request to anywhere using any command (especially when CRL/OCSP is not supported)
  - Certificates and signatures are used to verify manifest, but would we not want to also verify the requestor ? - Provider gets `requestorId` in demand and we could verify if it matches lets say `Common Name` from incoming certificate. It would instantly break our examples with shared signature and certificate, but maybe this verification could be conditional. It would be controlled by a dedicated Manifest field.



Discussion / hints:
@nieznanysprawiciel last 2 bullet points are regarding my worries about abusing Outbound Network feature (with a possible solution).
EDIT
Questions in this list do not really require answer, because these things will be / are decided during implementation. Just listed them to keep in mind covering the finally implemented behaviour in test.
>  Things to verify/test:
>  - Tests/verification of handling certificates storage
>    - startup with no certificates - error when manifest verification not disabled?

No because we allow to delete all certificates from cmdline

>    - startup with same certs in storage but a different file name - warn, make sure there are no duplicates in openssl store (would it be possible, would openssl fail?)

Duplicates are ignored because `openss` store does not care about them too. Only case when duplication is taken into account is removing certificates.

>    - startup with expired certificates - log it, but do we want to delete them on startup?

Ignored - expiration date is listed from cmdline on `keystore list` command

>    - startup with certs missing root CA - log it, or fail?

Ignored.

>    - do we want to keep param allowing to configure cert_dir?

Yes

>    - fail when cert_dir does not exist, or create empty?

Create new empty dir.

>    - fail when no write permissions to cert dir (in case someone would want to configure `/etc/ssl/certs/` as a cert dir) ?

No fail on startup, but obviously attempt to `add` or `remove` will fail.

>    - monitoring of cert storage:
>      - deleted certificate disappears from internal store

Yes

>      - added certificate appears in internal store

Yes

>    
>  - Tests/verification of handling certificates incoming in demand
>    - rejection on untrusted/expired certificate
>    - support of incoming certificate chains
>      - can we handle them? (currently arriving PEM cert is parsed as a single cert, but it coule be parsed as a stack of certs, and then this stack probably could be passed as a chain to [X509StoreContext](https://docs.rs/openssl/latest/openssl/x509/struct.X509StoreContext.html#method.init) to verify it
>      - do we want to support it?
>      - if yes make sure it is not possible to pass root CA cert in such chain (malicious action).
>      - if certificate chains are supported we should control issuing new certificates by third party using some x509v3 extension properties.
>    - unsupported signature type - failure to verify can happen because Provider does not trust certificate Requestor uses, but also because Requestor had used wrong signature type (are there differences between openssl lib distributions / environments / versions?). From Requestor's point of view both cases will look the same. Maybe list of supported signatures should be hardcoded into API and yapapi?


Fix patch:
diff --git a/Cargo.lock b/Cargo.lock
index 5a86ef0f1c..54f5a16192 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -1470,6 +1470,18 @@ dependencies = [
  "num_cpus",
 ]
 
+[[package]]
+name = "dashmap"
+version = "5.3.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3495912c9c1ccf2e18976439f4443f3fee0fd61f424ff99fde6a66b15ecb448f"
+dependencies = [
+ "cfg-if 1.0.0",
+ "hashbrown 0.12.3",
+ "lock_api 0.4.7",
+ "parking_lot_core 0.9.3",
+]
+
 [[package]]
 name = "data-encoding"
 version = "2.3.2"
@@ -2413,7 +2425,7 @@ version = "0.3.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "06c5d2f987ee8f6dff3fa1a352058dc59b990e447e4c7846aa7d804971314f7b"
 dependencies = [
- "dashmap",
+ "dashmap 4.0.2",
  "futures 0.3.21",
  "futures-timer",
  "no-std-compat",
@@ -2495,9 +2507,9 @@ dependencies = [
 
 [[package]]
 name = "hashbrown"
-version = "0.11.2"
+version = "0.12.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ab5ef0d4909ef3724cc8cce6ccc8572c5c817592e9285f5464f8e86f8bd3726e"
+checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888"
 
 [[package]]
 name = "hdrhistogram"
@@ -2838,12 +2850,12 @@ dependencies = [
 
 [[package]]
 name = "indexmap"
-version = "1.8.2"
+version = "1.9.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e6012d540c5baa3589337a98ce73408de9b5a25ec9fc2c6fd6be8f0d39e0ca5a"
+checksum = "10a35a97730320ffe8e2d410b5d3b69279b98d2c14bdb8b70ea89ecf7888d41e"
 dependencies = [
  "autocfg 1.1.0",
- "hashbrown 0.11.2",
+ "hashbrown 0.12.3",
 ]
 
 [[package]]
@@ -3405,7 +3417,7 @@ dependencies = [
  "hdrhistogram",
  "metrics-core",
  "metrics-util",
- "serde_yaml",
+ "serde_yaml 0.8.24",
 ]
 
 [[package]]
@@ -5427,9 +5439,9 @@ dependencies = [
 
 [[package]]
 name = "serde"
-version = "1.0.137"
+version = "1.0.143"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "61ea8d54c77f8315140a05f4c7237403bf38b72704d031543aa1d16abbf517d1"
+checksum = "53e8e5d5b70924f74ff5c6d64d9a5acd91422117c60f48c4e07855238a254553"
 dependencies = [
  "serde_derive",
 ]
@@ -5445,9 +5457,9 @@ dependencies = [
 
 [[package]]
 name = "serde_derive"
-version = "1.0.137"
+version = "1.0.143"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1f26faba0c3959972377d3b2d306ee9f71faee9714294e41bb777f83f88578be"
+checksum = "d3d8e8de557aee63c26b85b947f5e59b690d0454c753f3adeb5cd7835ab88391"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -5522,6 +5534,19 @@ dependencies = [
  "yaml-rust",
 ]
 
+[[package]]
+name = "serde_yaml"
+version = "0.9.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "79b7c9017c64a49806c6e8df8ef99b92446d09c92457f85f91835b01a8064ae0"
+dependencies = [
+ "indexmap",
+ "itoa 1.0.2",
+ "ryu",
+ "serde",
+ "unsafe-libyaml",
+]
+
 [[package]]
 name = "serial_test"
 version = "0.5.1"
@@ -5529,7 +5554,21 @@ source = "git+https://github.com/tworec/serial_test.git?branch=actix_rt_test#6a9
 dependencies = [
  "lazy_static",
  "parking_lot 0.11.2",
- "serial_test_derive",
+ "serial_test_derive 0.5.1",
+]
+
+[[package]]
+name = "serial_test"
+version = "0.9.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "92761393ee4dc3ff8f4af487bd58f4307c9329bbedea02cac0089ad9c411e153"
+dependencies = [
+ "dashmap 5.3.4",
+ "futures 0.3.21",
+ "lazy_static",
+ "log",
+ "parking_lot 0.12.1",
+ "serial_test_derive 0.9.0",
 ]
 
 [[package]]
@@ -5542,6 +5581,18 @@ dependencies = [
  "syn",
 ]
 
+[[package]]
+name = "serial_test_derive"
+version = "0.9.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4b6f5d1c3087fb119617cff2966fe3808a80e5eb59a8c1601d5994d66f4346a5"
+dependencies = [
+ "proc-macro-error",
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
 [[package]]
 name = "sha-1"
 version = "0.9.8"
@@ -5953,6 +6004,17 @@ version = "1.0.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
 
+[[package]]
+name = "tar"
+version = "0.4.38"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4b55807c0344e1e6c04d7c965f5289c39a8d94ae23ed5c0b57aabac549f871c6"
+dependencies = [
+ "filetime",
+ "libc",
+ "xattr",
+]
+
 [[package]]
 name = "tempdir"
 version = "0.3.7"
@@ -6642,6 +6704,12 @@ version = "0.1.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "39ec24b3121d976906ece63c9daad25b85969647682eee313cb5779fdd69e14e"
 
+[[package]]
+name = "unsafe-libyaml"
+version = "0.2.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "931179334a56395bcf64ba5e0ff56781381c1a5832178280c7d7f91d1679aeb0"
+
 [[package]]
 name = "url"
 version = "2.2.2"
@@ -7199,7 +7267,7 @@ checksum = "022ffb9c20a34581d5acb059af96323ebde68bf7e476dd8fe433365e3ff631d5"
 dependencies = [
  "serde",
  "serde_json",
- "serde_yaml",
+ "serde_yaml 0.8.24",
  "thiserror",
  "ya-client-model 0.3.2",
 ]
@@ -7212,7 +7280,7 @@ checksum = "2a59f53bc89329d33bc1d143027535ae36fd0a981e6c5bfb2363516ff32ddf8e"
 dependencies = [
  "serde",
  "serde_json",
- "serde_yaml",
+ "serde_yaml 0.8.24",
  "thiserror",
  "ya-client-model 0.4.0",
 ]
@@ -7225,7 +7293,7 @@ dependencies = [
  "regex",
  "serde",
  "serde_json",
- "serde_yaml",
+ "serde_yaml 0.8.24",
  "shlex 1.1.0",
  "tempdir",
  "thiserror",
@@ -7429,7 +7497,7 @@ dependencies = [
  "secp256k1 0.19.0",
  "serde",
  "serde_json",
- "serde_yaml",
+ "serde_yaml 0.8.24",
  "sha3 0.8.2",
  "shell-words",
  "signal-hook",
@@ -7520,13 +7588,14 @@ dependencies = [
  "semver 1.0.10",
  "serde",
  "serde_json",
- "serde_yaml",
- "sha2 0.9.9",
+ "serde_yaml 0.9.4",
+ "serial_test 0.9.0",
  "shlex 1.1.0",
  "snailquote",
  "structopt",
  "strum 0.24.0",
- "tempdir",
+ "tar",
+ "test-case",
  "thiserror",
  "url",
  "ya-agreement-utils 0.4.0",
@@ -7564,7 +7633,7 @@ dependencies = [
  "regex",
  "serde",
  "serde_json",
- "serial_test",
+ "serial_test 0.5.1",
  "sha3 0.8.2",
  "structopt",
  "strum 0.24.0",
@@ -7825,6 +7894,7 @@ dependencies = [
  "ya-manifest-utils",
  "ya-std-utils",
  "ya-utils-actix",
+ "ya-utils-cli",
  "ya-utils-path",
  "ya-utils-process",
  "yansi",
@@ -8010,12 +8080,9 @@ version = "0.1.1"
 dependencies = [
  "anyhow",
  "lazy_static",
- "log",
- "prettytable-rs",
  "serde",
- "serde_json",
- "serde_yaml",
  "url",
+ "ya-utils-cli",
 ]
 
 [[package]]
@@ -8178,6 +8245,18 @@ dependencies = [
  "tokio 1.19.2",
 ]
 
+[[package]]
+name = "ya-utils-cli"
+version = "0.1.0"
+dependencies = [
+ "anyhow",
+ "lazy_static",
+ "prettytable-rs",
+ "serde",
+ "serde_json",
+ "serde_yaml 0.9.4",
+]
+
 [[package]]
 name = "ya-utils-futures"
 version = "0.1.0"
diff --git a/Cargo.toml b/Cargo.toml
index 34d31c2d02..e546238c53 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -201,6 +201,7 @@ ya-file-logging = { path = "utils/file-logging" }
 ya-manifest-utils = { path = "utils/manifest-utils" }
 ya-transfer = { path = "utils/transfer" }
 ya-utils-actix = { path = "utils/actix_utils"}
+ya-utils-cli = { path = "utils/cli"}
 ya-utils-futures = { path = "utils/futures" }
 ya-utils-networking = { path = "utils/networking" }
 ya-utils-path = { path = "utils/path" }
diff --git a/agent/provider/Cargo.toml b/agent/provider/Cargo.toml
index 813102aaa6..37efb678fa 100644
--- a/agent/provider/Cargo.toml
+++ b/agent/provider/Cargo.toml
@@ -21,6 +21,7 @@ ya-compile-time-utils = "0.2"
 ya-core-model = { version = "^0.7", features = ['activity', 'payment'] }
 ya-file-logging = "0.1"
 ya-utils-actix = "0.1"
+ya-utils-cli = "0.1"
 ya-utils-path = "0.1"
 ya-utils-process = { version = "0.1", features = ['lock'] }
 ya-std-utils = "0.1"
diff --git a/agent/provider/src/cli/keystore.rs b/agent/provider/src/cli/keystore.rs
index 5516d65a53..e6f7911d90 100644
--- a/agent/provider/src/cli/keystore.rs
+++ b/agent/provider/src/cli/keystore.rs
@@ -3,7 +3,9 @@ use std::path::PathBuf;
 
 use structopt::StructOpt;
 
-use ya_manifest_utils::{util, Keystore, KeystoreLoadResult};
+use ya_manifest_utils::util::{self, CertBasicData, CertBasicDataVisitor};
+use ya_manifest_utils::KeystoreLoadResult;
+use ya_utils_cli::{CommandOutput, ResponseTable};
 
 use crate::startup_config::ProviderConfig;
 
@@ -44,9 +46,9 @@ impl KeystoreConfig {
 
 fn list(config: ProviderConfig) -> anyhow::Result<()> {
     let cert_dir = cert_dir_path(&config)?;
-    let keystore = Keystore::load(&cert_dir)?;
-    util::print_cert_list_header();
-    keystore.visit_certs(util::print_cert_list_row)?;
+    let table = CertTable::new();
+    let table = util::visit_certificates(&cert_dir, table)?;
+    table.print(&config)?;
     Ok(())
 }
 
@@ -55,16 +57,19 @@ fn add(config: ProviderConfig, add: Add) -> anyhow::Result<()> {
     let keystore_manager = util::KeystoreManager::try_new(&cert_dir)?;
     match keystore_manager.load_certs(&add.certs)? {
         KeystoreLoadResult::Loaded { loaded, skipped } => {
-            println!("Added certificates:");
-            util::print_cert_list(&loaded)?;
-            if !skipped.is_empty() {
+            println_conditional(&config, "Added certificates:");
+            let certs_data = util::to_cert_data(&loaded)?;
+            print_cert_list(&config, certs_data)?;
+            if !skipped.is_empty() && !config.json {
                 println!("Certificates already loaded to keystore:");
-                util::print_cert_list(&skipped)?;
+                let certs_data = util::to_cert_data(&skipped)?;
+                print_cert_list(&config, certs_data)?;
             }
         }
         KeystoreLoadResult::NothingNewToLoad { skipped } => {
-            println!("No new certificate to add. Skipped:");
-            util::print_cert_list(&skipped)?;
+            println_conditional(&config, "No new certificate to add. Skipped:");
+            let certs_data = util::to_cert_data(&skipped)?;
+            print_cert_list(&config, certs_data)?;
         }
     }
     Ok(())
@@ -76,11 +81,12 @@ fn remove(config: ProviderConfig, remove: Remove) -> anyhow::Result<()> {
     let ids: HashSet<String> = remove.ids.into_iter().collect();
     match keystore_manager.remove_certs(&ids)? {
         util::KeystoreRemoveResult::NothingToRemove => {
-            println!("No matching certificates to remove.");
+            println_conditional(&config, "No matching certificates to remove.");
         }
         util::KeystoreRemoveResult::Removed { removed } => {
             println!("Removed certificates:");
-            util::print_cert_list(&removed)?;
+            let certs_data = util::to_cert_data(&removed)?;
+            print_cert_list(&config, certs_data)?;
         }
     };
     Ok(())
@@ -89,3 +95,55 @@ fn remove(config: ProviderConfig, remove: Remove) -> anyhow::Result<()> {
 fn cert_dir_path(config: &ProviderConfig) -> anyhow::Result<PathBuf> {
     Ok(config.cert_dir.get_or_create()?)
 }
+
+fn print_cert_list(
+    config: &ProviderConfig,
+    certs_data: Vec<util::CertBasicData>,
+) -> anyhow::Result<()> {
+    let mut table = CertTable::new();
+    for data in certs_data {
+        table.add(data);
+    }
+    table.print(&config)?;
+    Ok(())
+}
+
+struct CertTable {
+    table: ResponseTable,
+}
+
+impl CertTable {
+    pub fn new() -> Self {
+        let columns = vec![
+            "ID".to_string(),
+            "Not After".to_string(),
+            "Subject".to_string(),
+        ];
+        let values = vec![];
+        let table = ResponseTable { columns, values };
+        Self { table }
+    }
+
+    pub fn print(self, config: &ProviderConfig) -> anyhow::Result<()> {
+        let output = CommandOutput::from(self.table);
+        output.print(config.json)?;
+        Ok(())
+    }
+
+    pub fn add(&mut self, data: CertBasicData) {
+        self.accept(data)
+    }
+}
+
+impl CertBasicDataVisitor for CertTable {
+    fn accept(&mut self, data: CertBasicData) {
+        let row = serde_json::json! {[ data.id, data.not_after, data.subject ]};
+        self.table.values.push(row);
+    }
+}
+
+fn println_conditional(config: &ProviderConfig, txt: &str) {
+    if !config.json {
+        println!("{txt}");
+    }
+}
diff --git a/core/identity/examples/appkey_service.rs b/core/identity/examples/appkey_service.rs
index 3c1d04e412..93ebaa132d 100644
--- a/core/identity/examples/appkey_service.rs
+++ b/core/identity/examples/appkey_service.rs
@@ -36,11 +36,11 @@ async fn main() -> anyhow::Result<()> {
         }
         Args::ClientAK(cmd) => {
             let ctx = CliCtx::default();
-            ctx.output(cmd.run_command(&ctx).await?);
+            ctx.output(cmd.run_command(&ctx).await?)?;
         }
         Args::ClientID(cmd) => {
             let ctx = CliCtx::default();
-            ctx.output(cmd.run_command(&ctx).await?);
+            ctx.output(cmd.run_command(&ctx).await?)?;
         }
     }
     Ok(())
diff --git a/core/serv-api/Cargo.toml b/core/serv-api/Cargo.toml
index 65c4a83b91..39e2de008c 100644
--- a/core/serv-api/Cargo.toml
+++ b/core/serv-api/Cargo.toml
@@ -8,9 +8,6 @@ edition = "2018"
 [dependencies]
 anyhow = "1.0"
 lazy_static = "1.4"
-log = "0.4"
-prettytable-rs = "0.8"
 serde = { version = "1.0", features = ["derive"] }
-serde_json = "1.0"
-serde_yaml = "0.8"
+ya-utils-cli = "0.1"
 url = "2.1.1"
diff --git a/core/serv-api/src/lib.rs b/core/serv-api/src/lib.rs
index be641ac4d7..d368f15373 100644
--- a/core/serv-api/src/lib.rs
+++ b/core/serv-api/src/lib.rs
@@ -1,9 +1,7 @@
-use anyhow::Result;
-use prettytable::{color, format, format::TableFormat, Attr, Cell, Row, Table};
-use serde::Serialize;
-use std::collections::HashMap;
 use std::path::PathBuf;
 
+pub use ya_utils_cli::{CommandOutput, ResponseTable};
+
 #[derive(Clone, Debug, Default)]
 pub struct MetricsCtx {
     pub push_enabled: bool,
@@ -22,263 +20,7 @@ pub struct CliCtx {
 }
 
 impl CliCtx {
-    pub fn output(&self, output: CommandOutput) {
+    pub fn output(&self, output: CommandOutput) -> Result<(), anyhow::Error> {
         output.print(self.json_output)
     }
 }
-
-pub enum CommandOutput {
-    NoOutput,
-    Object(serde_json::Value),
-    Table {
-        columns: Vec<String>,
-        values: Vec<serde_json::Value>,
-        summary: Vec<serde_json::Value>,
-        header: Option<String>,
-    },
-    FormattedObject(Box<dyn FormattedObject>),
-}
-
-impl CommandOutput {
-    pub fn object<T: Serialize>(value: T) -> Result<Self> {
-        Ok(CommandOutput::Object(serde_json::to_value(value)?))
-    }
-
-    pub fn print(&self, json_output: bool) {
-        match self {
-            CommandOutput::NoOutput => {
-                if json_output {
-                    println!("null");
-                }
-            }
-            CommandOutput::Table {
-                columns,
-                values,
-                summary,
-                header,
-            } => {
-                if json_output {
-                    let columns_size_eq_values_size = values.iter().all(|row| match row {
-                        serde_json::Value::Array(values) => values.len() == columns.len(),
-                        _ => false,
-                    });
-                    if columns_size_eq_values_size {
-                        let kvs: Vec<HashMap<&String, &serde_json::Value>> = values
-                            .iter()
-                            .map(|row| match row {
-                                serde_json::Value::Array(row_values)
-                                    if columns.len() == row_values.len() =>
-                                {
-                                    columns
-                                        .into_iter()
-                                        .enumerate()
-                                        .map(|(idx, key)| (key, &row_values[idx]))
-                                        .collect()
-                                }
-                                _ => unreachable!(),
-                            })
-                            .collect();
-                        println!(
-                            "{}",
-                            serde_json::to_string_pretty(&serde_json::json!(kvs)).unwrap()
-                        )
-                    } else {
-                        println!(
-                            "{}",
-                            serde_json::to_string_pretty(&serde_json::json!({
-                                "headers": columns,
-                                "values": values
-                            }))
-                            .unwrap()
-                        )
-                    }
-                } else {
-                    if let Some(txt) = header {
-                        println!("{}", txt);
-                    }
-                    print_table(columns, values, summary);
-                }
-            }
-            CommandOutput::Object(v) => {
-                if json_output {
-                    println!("{}", serde_json::to_string_pretty(&v).unwrap())
-                } else {
-                    match v {
-                        serde_json::Value::String(s) => {
-                            println!("{}", s);
-                        }
-                        v => println!("{}", serde_yaml::to_string(&v).unwrap()),
-                    }
-                }
-            }
-            CommandOutput::FormattedObject(formatted_object) => {
-                if json_output {
-                    println!(
-                        "{}",
-                        serde_json::to_string_pretty(&formatted_object.to_json().unwrap()).unwrap()
-                    )
-                } else {
-                    formatted_object.print().unwrap()
-                }
-            }
-        }
-    }
-}
-
-fn print_table(
-    columns: &Vec<String>,
-    values: &Vec<serde_json::Value>,
-    summary: &Vec<serde_json::Value>,
-) {
-    let mut table = Table::new();
-    //table.set_format(*format::consts::FORMAT_NO_LINESEP_WITH_TITLE);
-    table.set_format(*FORMAT_BASIC);
-
-    table.set_titles(Row::new(
-        columns
-            .iter()
-            .map(|c| {
-                Cell::new(c)
-                    .with_style(Attr::Bold)
-                    .with_style(Attr::ForegroundColor(color::GREEN))
-            })
-            .collect(),
-    ));
-    if values.is_empty() {
-        let _ = table.add_row(columns.iter().map(|_| Cell::new("")).collect());
-    }
-    for row in values {
-        if let Some(row_items) = row.as_array() {
-            use serde_json::Value;
-
-            let row_strings = row_items
-                .iter()
-                .map(|v| match v {
-                    Value::String(s) => s.to_string(),
-                    Value::Null => "".into(),
-                    v => v.to_string(),
-                })
-                .collect();
-            table.add_row(row_strings);
-        }
-    }
-    if !summary.is_empty() {
-        table.add_row(Row::empty());
-        table.add_empty_row();
-        let l = summary.len();
-        for (idx, row) in summary.into_iter().enumerate() {
-            if let Some(row_items) = row.as_array() {
-                use serde_json::Value;
-
-                let row_strings = Row::new(
-                    row_items
-                        .iter()
-                        .map(|v| {
-                            let c = Cell::new(&match v {
-                                Value::String(s) => s.to_string(),
-                                Value::Null => "".into(),
-                                v => v.to_string(),
-                            });
-
-                            if idx == l - 1 {
-                                c.with_style(Attr::Bold)
-                            } else {
-                                c
-                            }
-                        })
-                        .collect(),
-                );
-                table.add_row(row_strings);
-            }
-        }
-    }
-    let _ = table.printstd();
-}
-
-impl From<()> for CommandOutput {
-    fn from(_: ()) -> Self {
-        CommandOutput::NoOutput
-    }
-}
-
-impl From<ResponseTable> for CommandOutput {
-    fn from(table: ResponseTable) -> Self {
-        CommandOutput::Table {
-            columns: table.columns,
-            values: table.values,
-            summary: Vec::new(),
-            header: None,
-        }
-    }
-}
-
-pub trait FormattedObject {
-    fn to_json(&self) -> Result<serde_json::Value>;
-
-    fn print(&self) -> Result<()>;
-}
-
-pub struct ResponseTable {
-    pub columns: Vec<String>,
-    pub values: Vec<serde_json::Value>,
-}
-
-impl ResponseTable {
-    pub fn sort_by(mut self, arg_key: &Option<impl AsRef<str>>) -> Self {
-        let key = match arg_key {
-            None => return self,
-            Some(k) => k.as_ref(),
-        };
-        let idx =
-            match self
-                .columns
-                .iter()
-                .enumerate()
-                .find_map(|(idx, v)| if v == key { Some(idx) } else { None })
-            {
-                None => return self,
-                Some(idx) => idx,
-            };
-        self.values
-            .sort_by_key(|v| Some(v.as_array()?.get(idx)?.to_string()));
-        self
-    }
-
-    pub fn with_summary(self, summary: Vec<serde_json::Value>) -> CommandOutput {
-        CommandOutput::Table {
-            columns: self.columns,
-            values: self.values,
-            summary,
-            header: None,
-        }
-    }
-
-    pub fn with_header(self, header: String) -> CommandOutput {
-        CommandOutput::Table {
-            columns: self.columns,
-            values: self.values,
-            summary: Vec::new(),
-            header: Some(header),
-        }
-    }
-}
-
-lazy_static::lazy_static! {
-    pub static ref FORMAT_BASIC: TableFormat = format::FormatBuilder::new()
-        .column_separator('│')
-        .borders('│')
-        .separators(
-            &[format::LinePosition::Top],
-            format::LineSeparator::new('─', '┬', '┌', '┐')
-        )
-        .separators(
-            &[format::LinePosition::Title],
-            format::LineSeparator::new('─', '┼', '├', '┤')
-        )
-        .separators(
-            &[format::LinePosition::Bottom],
-            format::LineSeparator::new('─', '┴', '└', '┘')
-        )
-        .padding(2, 2)
-        .build();
-}
diff --git a/core/serv/src/main.rs b/core/serv/src/main.rs
index 97475d1862..9a36a4f3a8 100644
--- a/core/serv/src/main.rs
+++ b/core/serv/src/main.rs
@@ -112,7 +112,7 @@ impl CliArgs {
     pub async fn run_command(self) -> Result<()> {
         let ctx: CliCtx = (&self).try_into()?;
 
-        ctx.output(self.command.run_command(&ctx).await?);
+        ctx.output(self.command.run_command(&ctx).await?)?;
         Ok(())
     }
 }
diff --git a/utils/cli/Cargo.toml b/utils/cli/Cargo.toml
new file mode 100644
index 0000000000..bfcd7d2a3b
--- /dev/null
+++ b/utils/cli/Cargo.toml
@@ -0,0 +1,20 @@
+[package]
+name = "ya-utils-cli"
+version = "0.1.0"
+authors = ["Golem Factory <contact@golem.network>"]
+edition = "2018"
+homepage = "https://github.com/golemfactory/yagna"
+repository = "https://github.com/golemfactory/yagna"
+license = "LGPL-3.0"
+description = "Yagna CLI utils"
+keywords = ["golem", "yagna"]
+
+[dependencies]
+anyhow = "1.0"
+prettytable-rs = "0.8"
+serde = "1.0"
+serde_json = "1.0"
+serde_yaml = "0.9"
+lazy_static = "1.4"
+
+[dev-dependencies]
diff --git a/utils/cli/src/cmd.rs b/utils/cli/src/cmd.rs
new file mode 100644
index 0000000000..52785638db
--- /dev/null
+++ b/utils/cli/src/cmd.rs
@@ -0,0 +1,74 @@
+use anyhow::Result;
+use serde::Serialize;
+
+pub enum CommandOutput {
+    NoOutput,
+    Object(serde_json::Value),
+    Table {
+        columns: Vec<String>,
+        values: Vec<serde_json::Value>,
+        summary: Vec<serde_json::Value>,
+        header: Option<String>,
+    },
+}
+
+impl CommandOutput {
+    pub fn object<T: Serialize>(value: T) -> Result<Self> {
+        Ok(CommandOutput::Object(serde_json::to_value(value)?))
+    }
+
+    pub fn print(&self, json_output: bool) -> Result<()> {
+        if json_output {
+            self.print_json()?;
+        } else {
+            self.print_plain()?;
+        }
+        Ok(())
+    }
+
+    fn print_json(&self) -> anyhow::Result<()> {
+        match self {
+            CommandOutput::NoOutput => println!("null"),
+            CommandOutput::Table {
+                columns,
+                values,
+                summary: _,
+                header: _,
+            } => crate::table::print_json_table(columns, values)?,
+            CommandOutput::Object(value) => println!("{}", serde_json::to_string_pretty(&value)?),
+        }
+        Ok(())
+    }
+
+    fn print_plain(&self) -> anyhow::Result<()> {
+        match self {
+            CommandOutput::NoOutput => {},
+            CommandOutput::Table {
+                columns,
+                values,
+                summary,
+                header,
+            } => {
+                if let Some(txt) = header {
+                    println!("{}", txt);
+                }
+                crate::table::print_table(columns, values, summary)
+            },
+            CommandOutput::Object(value) => {
+                match value {
+                    serde_json::Value::String(s) => {
+                        println!("{}", s);
+                    }
+                    value => println!("{}", serde_yaml::to_string(&value)?),
+                }
+            }
+        }
+        Ok(())
+    }
+}
+
+impl From<()> for CommandOutput {
+    fn from(_: ()) -> Self {
+        CommandOutput::NoOutput
+    }
+}
diff --git a/utils/cli/src/lib.rs b/utils/cli/src/lib.rs
new file mode 100644
index 0000000000..b0dece1497
--- /dev/null
+++ b/utils/cli/src/lib.rs
@@ -0,0 +1,6 @@
+
+mod cmd;
+mod table;
+
+pub use cmd::CommandOutput;
+pub use table::ResponseTable;
diff --git a/utils/cli/src/table.rs b/utils/cli/src/table.rs
new file mode 100644
index 0000000000..66ddd1ff69
--- /dev/null
+++ b/utils/cli/src/table.rs
@@ -0,0 +1,188 @@
+use prettytable::{color, format, format::TableFormat, Attr, Cell, Row, Table};
+use std::collections::HashMap;
+use crate::cmd::CommandOutput;
+
+pub fn print_table(
+    columns: &Vec<String>,
+    values: &Vec<serde_json::Value>,
+    summary: &Vec<serde_json::Value>,
+) {
+    let mut table = Table::new();
+    table.set_format(*FORMAT_BASIC);
+
+    table.set_titles(Row::new(
+        columns
+            .iter()
+            .map(|c| {
+                Cell::new(c)
+                    .with_style(Attr::Bold)
+                    .with_style(Attr::ForegroundColor(color::GREEN))
+            })
+            .collect(),
+    ));
+    if values.is_empty() {
+        let _ = table.add_row(columns.iter().map(|_| Cell::new("")).collect());
+    }
+    for row in values {
+        if let Some(row_items) = row.as_array() {
+            use serde_json::Value;
+
+            let row_strings = row_items
+                .iter()
+                .map(|v| match v {
+                    Value::String(s) => s.to_string(),
+                    Value::Null => "".into(),
+                    v => v.to_string(),
+                })
+                .collect();
+            table.add_row(row_strings);
+        }
+    }
+    if !summary.is_empty() {
+        table.add_row(Row::empty());
+        table.add_empty_row();
+        let l = summary.len();
+        for (idx, row) in summary.into_iter().enumerate() {
+            if let Some(row_items) = row.as_array() {
+                use serde_json::Value;
+
+                let row_strings = Row::new(
+                    row_items
+                        .iter()
+                        .map(|v| {
+                            let c = Cell::new(&match v {
+                                Value::String(s) => s.to_string(),
+                                Value::Null => "".into(),
+                                v => v.to_string(),
+                            });
+
+                            if idx == l - 1 {
+                                c.with_style(Attr::Bold)
+                            } else {
+                                c
+                            }
+                        })
+                        .collect(),
+                );
+                table.add_row(row_strings);
+            }
+        }
+    }
+    let _ = table.printstd();
+}
+
+pub fn print_json_table(
+    columns: &Vec<String>,
+    values: &Vec<serde_json::Value>,
+) -> Result<(), anyhow::Error> {
+    let columns_size_eq_values_size = values.iter().all(|row| match row {
+        serde_json::Value::Array(values) => values.len() == columns.len(),
+        _ => false,
+    });
+    if columns_size_eq_values_size {
+        let kvs: Vec<HashMap<&String, &serde_json::Value>> = values
+            .iter()
+            .map(|row| match row {
+                serde_json::Value::Array(row_values)
+                    if columns.len() == row_values.len() =>
+                {
+                    columns
+                        .into_iter()
+                        .enumerate()
+                        .map(|(idx, key)| (key, &row_values[idx]))
+                        .collect()
+                }
+                _ => unreachable!(),
+            })
+            .collect();
+        println!(
+            "{}",
+            serde_json::to_string_pretty(&serde_json::json!(kvs))?
+        )
+    } else {
+        println!(
+            "{}",
+            serde_json::to_string_pretty(&serde_json::json!({
+                "headers": columns,
+                "values": values
+            }))?
+        )
+    }
+    Ok(())
+}
+
+pub struct ResponseTable {
+    pub columns: Vec<String>,
+    pub values: Vec<serde_json::Value>,
+}
+
+impl ResponseTable {
+    pub fn sort_by(mut self, arg_key: &Option<impl AsRef<str>>) -> Self {
+        let key = match arg_key {
+            None => return self,
+            Some(k) => k.as_ref(),
+        };
+        let idx =
+            match self
+                .columns
+                .iter()
+                .enumerate()
+                .find_map(|(idx, v)| if v == key { Some(idx) } else { None })
+            {
+                None => return self,
+                Some(idx) => idx,
+            };
+        self.values
+            .sort_by_key(|v| Some(v.as_array()?.get(idx)?.to_string()));
+        self
+    }
+
+    pub fn with_summary(self, summary: Vec<serde_json::Value>) -> CommandOutput {
+        CommandOutput::Table {
+            columns: self.columns,
+            values: self.values,
+            summary,
+            header: None,
+        }
+    }
+
+    pub fn with_header(self, header: String) -> CommandOutput {
+        CommandOutput::Table {
+            columns: self.columns,
+            values: self.values,
+            summary: Vec::new(),
+            header: Some(header),
+        }
+    }
+}
+
+impl From<ResponseTable> for CommandOutput {
+    fn from(table: ResponseTable) -> Self {
+        CommandOutput::Table {
+            columns: table.columns,
+            values: table.values,
+            summary: Vec::new(),
+            header: None,
+        }
+    }
+}
+
+lazy_static::lazy_static! {
+    pub static ref FORMAT_BASIC: TableFormat = format::FormatBuilder::new()
+        .column_separator('│')
+        .borders('│')
+        .separators(
+            &[format::LinePosition::Top],
+            format::LineSeparator::new('─', '┬', '┌', '┐')
+        )
+        .separators(
+            &[format::LinePosition::Title],
+            format::LineSeparator::new('─', '┼', '├', '┤')
+        )
+        .separators(
+            &[format::LinePosition::Bottom],
+            format::LineSeparator::new('─', '┴', '└', '┘')
+        )
+        .padding(2, 2)
+        .build();
+}
diff --git a/utils/manifest-utils/Cargo.toml b/utils/manifest-utils/Cargo.toml
index 9cb235b49a..c593381b4e 100644
--- a/utils/manifest-utils/Cargo.toml
+++ b/utils/manifest-utils/Cargo.toml
@@ -14,7 +14,7 @@ ya-agreement-utils = "0.4"
 
 serde = "1.0"
 serde_json = "1.0"
-serde_yaml = "0.8"
+serde_yaml = "0.9"
 thiserror = "1.0"
 
 anyhow = { version = "1.0" }
@@ -22,7 +22,6 @@ base64 = { version = "0.13" }
 chrono = { version = "0.4", features = ["serde"] }
 hex = { version = "0.4" }
 semver = { version = "1.0", features = ["serde"] }
-sha2 = { version = "0.9" }
 snailquote = { version = "0.3" }
 structopt = { version =  "0.3" }
 strum = { version = "0.24", features = ["derive"] }
@@ -31,6 +30,8 @@ openssl = { version = "0.10", features = ["vendored"] }
 md-5 = "0.10"
 
 [dev-dependencies]
-anyhow = "1.0.31"
-tempdir = "0.3.7"
-shlex = "1.1.0"
+anyhow = "1.0"
+serial_test = "0.9"
+shlex = "1.1"
+tar = "0.4"
+test-case = "2.2"
diff --git a/utils/manifest-utils/src/lib.rs b/utils/manifest-utils/src/lib.rs
index 3c69ba6afd..4eeec405ac 100644
--- a/utils/manifest-utils/src/lib.rs
+++ b/utils/manifest-utils/src/lib.rs
@@ -4,4 +4,6 @@ pub mod util;
 
 pub use manifest::*;
 pub use policy::{Keystore, Policy, PolicyConfig};
-pub use util::*;
+pub use util::{
+    decode_data, DecodingError, KeystoreLoadResult, KeystoreManager, KeystoreRemoveResult,
+};
diff --git a/utils/manifest-utils/src/policy.rs b/utils/manifest-utils/src/policy.rs
index 519f5ade87..717b2a84a5 100644
--- a/utils/manifest-utils/src/policy.rs
+++ b/utils/manifest-utils/src/policy.rs
@@ -9,10 +9,12 @@ use openssl::hash::MessageDigest;
 use openssl::pkey::{PKey, Public};
 use openssl::sign::Verifier;
 use openssl::x509::store::{X509Store, X509StoreBuilder};
-use openssl::x509::{X509ObjectRef, X509Ref, X509StoreContext, X509};
+use openssl::x509::{X509ObjectRef, X509StoreContext, X509};
 use structopt::StructOpt;
 use strum::{Display, EnumIter, EnumString, EnumVariantNames, IntoEnumIterator, VariantNames};
 
+use crate::util::{CertBasicDataVisitor, X509Visitor};
+
 /// Policy configuration
 #[derive(StructOpt, Clone, Debug, Default)]
 pub struct PolicyConfig {
@@ -147,7 +149,7 @@ impl Keystore {
             sig_alg.as_ref()
         ))?;
         let mut verifier = Verifier::new(msg_digest, pkey.as_ref())?;
-        if false == verifier.verify_oneshot(&sig, data.as_ref().as_bytes())? {
+        if !(verifier.verify_oneshot(&sig, data.as_ref().as_bytes())?) {
             return Err(anyhow::anyhow!("Invalid signature"));
         }
         Ok(())
@@ -165,19 +167,19 @@ impl Keystore {
         Ok(ids)
     }
 
-    pub fn visit_certs(
+    pub(crate) fn visit_certs<T: CertBasicDataVisitor>(
         &self,
-        visit_fn: impl Fn(&X509Ref) -> anyhow::Result<()>,
+        visitor: &mut X509Visitor<T>,
     ) -> anyhow::Result<()> {
         let inner = self.inner.read().unwrap();
         for cert in inner.objects().iter().flat_map(X509ObjectRef::x509) {
-            visit_fn(cert)?;
+            visitor.accept(cert)?;
         }
         Ok(())
     }
 
     fn load_file(store: &mut X509StoreBuilder, cert: &PathBuf) -> anyhow::Result<()> {
-        for cert in crate::parse_cert_file(cert)? {
+        for cert in crate::util::parse_cert_file(cert)? {
             store.add_cert(cert)?
         }
         Ok(())
@@ -195,7 +197,7 @@ impl Keystore {
             .map_err(|err| anyhow::anyhow!("Err: {}", err.to_string()))?;
         let cert_chain = openssl::stack::Stack::new()?;
         let mut ctx = X509StoreContext::new()?;
-        if false == ctx.init(&store, &cert, &cert_chain, |ctx| ctx.verify_cert())? {
+        if !(ctx.init(&store, &cert, &cert_chain, |ctx| ctx.verify_cert())?) {
             return Err(anyhow::anyhow!("Invalid certificate"));
         }
         Ok(cert.public_key()?)
diff --git a/utils/manifest-utils/src/util.rs b/utils/manifest-utils/src/util.rs
index 404cf78432..102bace60b 100644
--- a/utils/manifest-utils/src/util.rs
+++ b/utils/manifest-utils/src/util.rs
@@ -1,4 +1,5 @@
-use std::collections::{HashMap, HashSet};
+use std::collections::{BTreeMap, HashMap, HashSet};
+use std::convert::TryFrom;
 use std::ffi::OsStr;
 use std::fs::{self, DirEntry};
 use std::{fs::File, path::PathBuf};
@@ -60,46 +61,24 @@ fn os_str_to_string(os_str: &OsStr) -> String {
     os_str.to_string_lossy().to_string()
 }
 
-pub fn print_cert_list(certs: &Vec<X509>) -> anyhow::Result<()> {
-    print_cert_list_header();
+pub fn to_cert_data(certs: &Vec<X509>) -> anyhow::Result<Vec<CertBasicData>> {
+    let mut certs_data = Vec::new();
     for cert in certs {
-        print_cert_list_row(cert)?;
+        let data = CertBasicData::try_from(cert.as_ref())?;
+        certs_data.push(data);
     }
-    Ok(())
-}
-
-pub fn print_cert_list_header() {
-    println!("{0: <32} {1: <24} {2}", "ID", "Not After", "Subject");
-    println!(
-        "{0: <32} {1: <24} {2}",
-        "-----------", "-------", "---------"
-    );
-}
-
-pub fn print_cert_list_row(cert: &X509Ref) -> anyhow::Result<()> {
-    let id = cert_to_id(&cert)?;
-    let not_after = cert.not_after().to_string();
-    let mut subject = String::new();
-    add_cert_subject_entries(&mut subject, cert, Nid::COMMONNAME, "CN");
-    add_cert_subject_entries(&mut subject, cert, Nid::PKCS9_EMAILADDRESS, "E");
-    add_cert_subject_entries(&mut subject, cert, Nid::ORGANIZATIONNAME, "O");
-    add_cert_subject_entries(&mut subject, cert, Nid::ORGANIZATIONALUNITNAME, "OU");
-    add_cert_subject_entries(&mut subject, cert, Nid::COUNTRYNAME, "C");
-    add_cert_subject_entries(&mut subject, cert, Nid::STATEORPROVINCENAME, "ST");
-
-    println!("{0: <32} {1: <24} {2}", id, not_after, subject);
-    Ok(())
+    Ok(certs_data)
 }
 
 /// Adds entries with given `nid` to given `subject` String.
 fn add_cert_subject_entries(
-    subject: &mut String,
+    subject: &mut BTreeMap<String, String>,
     cert: &X509Ref,
     nid: Nid,
     entry_short_name: &str,
 ) {
     if let Some(entries) = cert_subject_entries(cert, nid) {
-        subject.push_str(&format!("{}: {} ", entry_short_name, entries));
+        subject.insert(entry_short_name.to_string(), entries);
     }
 }
 
@@ -125,7 +104,61 @@ fn cert_subject_entries(cert: &X509Ref, nid: Nid) -> Option<String> {
 pub fn cert_to_id(cert: &X509Ref) -> anyhow::Result<String> {
     let txt = cert.to_text()?;
     let digest = Md5::digest(&txt);
-    Ok(format!("{:x}", digest))
+    Ok(format!("{digest:x}"))
+}
+
+pub fn visit_certificates<T: CertBasicDataVisitor>(
+    cert_dir: &PathBuf,
+    visitor: T,
+) -> anyhow::Result<T> {
+    let keystore = Keystore::load(cert_dir)?;
+    let mut visitor = X509Visitor { visitor };
+    keystore.visit_certs(&mut visitor)?;
+    Ok(visitor.visitor)
+}
+
+pub struct CertBasicData {
+    pub id: String,
+    pub not_after: String,
+    pub subject: BTreeMap<String, String>,
+}
+
+impl TryFrom<&X509Ref> for CertBasicData {
+    type Error = anyhow::Error;
+
+    fn try_from(cert: &X509Ref) -> Result<Self, Self::Error> {
+        let id = cert_to_id(cert)?;
+        let not_after = cert.not_after().to_string();
+        let mut subject = BTreeMap::new();
+        add_cert_subject_entries(&mut subject, cert, Nid::COMMONNAME, "CN");
+        add_cert_subject_entries(&mut subject, cert, Nid::PKCS9_EMAILADDRESS, "E");
+        add_cert_subject_entries(&mut subject, cert, Nid::ORGANIZATIONNAME, "O");
+        add_cert_subject_entries(&mut subject, cert, Nid::ORGANIZATIONALUNITNAME, "OU");
+        add_cert_subject_entries(&mut subject, cert, Nid::COUNTRYNAME, "C");
+        add_cert_subject_entries(&mut subject, cert, Nid::STATEORPROVINCENAME, "ST");
+
+        Ok(CertBasicData {
+            id,
+            not_after,
+            subject,
+        })
+    }
+}
+
+pub trait CertBasicDataVisitor {
+    fn accept(&mut self, cert_data: CertBasicData);
+}
+
+pub(crate) struct X509Visitor<T: CertBasicDataVisitor> {
+    visitor: T,
+}
+
+impl<T: CertBasicDataVisitor> X509Visitor<T> {
+    pub(crate) fn accept(&mut self, cert: &X509Ref) -> anyhow::Result<()> {
+        let cert_data = CertBasicData::try_from(cert)?;
+        self.visitor.accept(cert_data);
+        Ok(())
+    }
 }
 
 pub struct KeystoreManager {
@@ -149,7 +182,7 @@ impl KeystoreManager {
 
         for cert_path in cert_paths {
             let mut new_certs = Vec::new();
-            let file_certs = parse_cert_file(&cert_path)?;
+            let file_certs = parse_cert_file(cert_path)?;
             if file_certs.is_empty() {
                 continue;
             }
@@ -175,7 +208,7 @@ impl KeystoreManager {
         if loaded.is_empty() {
             return Ok(KeystoreLoadResult::NothingNewToLoad { skipped });
         }
-        return Ok(KeystoreLoadResult::Loaded { loaded, skipped });
+        Ok(KeystoreLoadResult::Loaded { loaded, skipped })
     }
 
     pub fn remove_certs(self, ids: &HashSet<String>) -> anyhow::Result<KeystoreRemoveResult> {
@@ -210,11 +243,11 @@ impl KeystoreManager {
             if split_and_skip {
                 let file_stem = get_file_stem(&cert_file).expect("Cannot get file name stem");
                 let dot_extension = get_file_extension(&cert_file)
-                    .map_or_else(|| String::from(""), |ex| format!(".{}", ex));
+                    .map_or_else(|| String::from(""), |ex| format!(".{ex}"));
                 for (id, cert) in ids_cert {
                     let cert = cert.to_pem()?;
                     let mut file_path = self.cert_dir.clone();
-                    let filename = format!("{}.{}{}", file_stem, id, dot_extension);
+                    let filename = format!("{file_stem}.{id}{dot_extension}");
                     file_path.push(filename);
                     fs::write(file_path, cert)?;
                 }
@@ -229,17 +262,16 @@ impl KeystoreManager {
     /// Loads keychain file to `cert_dir`
     fn load_as_keychain_file(&self, cert_path: &PathBuf) -> anyhow::Result<()> {
         let file_name = get_file_name(cert_path)
-            .ok_or_else(|| format!("Cannot get filename of {:?}", cert_path))
-            .map_err(|err| anyhow::anyhow!(err))?;
+            .ok_or_else(|| anyhow::anyhow!(format!("Cannot get filename of {cert_path:?}")))?;
         let mut new_cert_path = self.cert_dir.clone();
         new_cert_path.push(file_name);
         if new_cert_path.exists() {
             let file_stem = get_file_stem(&new_cert_path).expect("Has to have stem");
             let dot_extension = get_file_extension(&new_cert_path)
-                .map(|ex| format!(".{}", ex))
+                .map(|ex| format!(".{ex}"))
                 .unwrap_or(String::from(""));
             for i in 0..u32::MAX {
-                let numbered_filename = format!("{0}.{1}{2}", file_stem, i, dot_extension);
+                let numbered_filename = format!("{file_stem}.{i}{dot_extension}");
                 new_cert_path = self.cert_dir.clone();
                 new_cert_path.push(numbered_filename);
                 if !new_cert_path.exists() {
@@ -260,16 +292,15 @@ impl KeystoreManager {
         cert_path: &PathBuf,
         certs: Vec<X509>,
     ) -> anyhow::Result<()> {
-        let file_stem = get_file_stem(&cert_path)
-            .ok_or_else(|| "Cannot get file name stem")
-            .map_err(|err| anyhow::anyhow!(err))?;
-        let dot_extension = get_file_extension(&cert_path)
-            .map(|ex| format!(".{}", ex))
+        let file_stem = get_file_stem(cert_path)
+            .ok_or_else(|| anyhow::anyhow!("Cannot get file name stem."))?;
+        let dot_extension = get_file_extension(cert_path)
+            .map(|ex| format!(".{ex}"))
             .unwrap_or(String::from(""));
         for cert in certs.into_iter() {
             let id = cert_to_id(&cert)?;
             let mut new_cert_path = self.cert_dir.clone();
-            new_cert_path.push(format!("{}.{}{}", file_stem, id, dot_extension));
+            new_cert_path.push(format!("{file_stem}.{id}{dot_extension}"));
             let cert = cert.to_pem()?;
             fs::write(new_cert_path, cert)?;
         }
@@ -292,19 +323,24 @@ pub enum KeystoreRemoveResult {
     Removed { removed: Vec<X509> },
 }
 
-#[test]
-fn base64_wrapped_lines_test() {
-    let wrapped_base64 = "
-    VGhlIHF1aWNrIGJyb3du
-    IGZveCBqdW1wcyBvdmVy
-    IHRoZSBsYXp5IGRvZw==";
-    let phrase = decode_data(wrapped_base64).expect("failed to decode base64 wrapped content");
-    let phrase = String::from_utf8_lossy(&phrase).to_string();
-    let expected = "The quick brown fox jumps over the lazy dog";
-    assert_eq!(
-        &phrase, expected,
-        "Manifest related base64 payload may be encoded by the user, 
-        and many tools wrap base64 output by default, 
-        so we should try to filter out whitespace"
-    )
+#[cfg(test)]
+pub mod tests {
+    use super::*;
+
+    #[test]
+    pub fn base64_wrapped_lines_test() {
+        let wrapped_base64 = "
+        VGhlIHF1aWNrIGJyb3du
+        IGZveCBqdW1wcyBvdmVy
+        IHRoZSBsYXp5IGRvZw==";
+        let phrase = decode_data(wrapped_base64).expect("failed to decode base64 wrapped content");
+        let phrase = String::from_utf8_lossy(&phrase).to_string();
+        let expected = "The quick brown fox jumps over the lazy dog";
+        assert_eq!(
+            &phrase, expected,
+            "Manifest related base64 payload may be encoded by the user, 
+            and many tools wrap base64 output by default, 
+            so we should try to filter out whitespace"
+        )
+    }
 }


Test patch:
diff --git a/utils/manifest-utils/resources/test/certificates.tar b/utils/manifest-utils/resources/test/certificates.tar
new file mode 100644
index 0000000000..19cb33efc3
Binary files /dev/null and b/utils/manifest-utils/resources/test/certificates.tar differ
diff --git a/utils/manifest-utils/resources/test/data.json.base64 b/utils/manifest-utils/resources/test/data.json.base64
new file mode 100644
index 0000000000..3285956213
--- /dev/null
+++ b/utils/manifest-utils/resources/test/data.json.base64
@@ -0,0 +1,1 @@
+ewogICAgImZvbyI6ICJiYXIiCn0K
\ No newline at end of file
diff --git a/utils/manifest-utils/resources/test/data.json.base64.foo_req_sign.sha256.base64 b/utils/manifest-utils/resources/test/data.json.base64.foo_req_sign.sha256.base64
new file mode 100644
index 0000000000..9da1198bdd
--- /dev/null
+++ b/utils/manifest-utils/resources/test/data.json.base64.foo_req_sign.sha256.base64
@@ -0,0 +1,1 @@
+bcvrLgBybQr8g9SvEzvXnNNo9mnTgOc1VwTffEfeMVPDjeMx8L4WkoCf7QIP1n8E7YwcMGCGYHgCMPl4br3/PyN8H12FFEgf5fkC8+GyrY1dzTFVuxFb+ONshnv/k9zAeLWuYh6+jQnQjdvhVszTZQEF7WFaa/A1RHWDeOp/CiSptTkqFXe0TDnluDTatdMwap8XAWEbU0ejk6nSJfQWnWUetlGlTBwGZEk8TVdDkgHr0g4MC4vw6sIUaLBHOK5kJSCbIP55bYWDz8+8h3NONqFNDAprk6gJu6/XG33JHePnNsbn5jzn49LbObk3h3lX5wv4orGFDiDzK0SPvuDLBw==
\ No newline at end of file
diff --git a/utils/manifest-utils/resources/test/data.json.base64.unknown_req_sign.sha256.base64 b/utils/manifest-utils/resources/test/data.json.base64.unknown_req_sign.sha256.base64
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/utils/manifest-utils/tests/certificate_store.rs b/utils/manifest-utils/tests/certificate_store.rs
new file mode 100644
index 0000000000..9e4ca7cbbe
--- /dev/null
+++ b/utils/manifest-utils/tests/certificate_store.rs
@@ -0,0 +1,121 @@
+pub mod common;
+
+#[macro_use]
+extern crate serial_test;
+
+use std::fs;
+
+use common::*;
+
+use test_case::test_case;
+
+use ya_manifest_utils::util::visit_certificates;
+
+/// Test utilities
+#[test_case(
+    &[],
+    &[],
+    &[],
+    &[];
+    "Does not fail when listing empty store"
+)]
+#[test_case(
+    &["foo_ca.cert.pem"],
+    &[],
+    &["c128af8c6d0ba34d940582c01443911d"],
+    &["foo_ca.cert.pem"]; 
+    "Can load one certificate"
+)]
+#[test_case(
+    &["foo_ca.cert.pem", "foo_inter.cert.pem"],
+    &["c128af8c6d0ba34d940582c01443911d", "4e0df976b534cb73794a7613b31af51c"],
+    &[],
+    &[];
+    "Can remove all certificates"
+)]
+#[test_case(
+    &["foo_ca-chain.cert.pem"],
+    &[],
+    &["4e0df976b534cb73794a7613b31af51c", "c128af8c6d0ba34d940582c01443911d"],
+    &["foo_ca-chain.cert.pem"]; 
+    "Load keychain loads two certificates and stores them in received form (single keychain file)"
+)]
+#[test_case(
+    &["foo_ca-chain.cert.pem"],
+    &["c128af8c6d0ba34d940582c01443911d"],
+    &["4e0df976b534cb73794a7613b31af51c"],
+    &["foo_ca-chain.cert.4e0df976b534cb73794a7613b31af51c.pem"]; 
+    "Load keychain and remove root CA results with intermediate cert and cert file with id in the name"
+)]
+#[test_case(
+    &["foo_ca-chain.cert.pem"], 
+    &["4e0df976b534cb73794a7613b31af51c"], 
+    &["c128af8c6d0ba34d940582c01443911d"], 
+    &["foo_ca-chain.cert.c128af8c6d0ba34d940582c01443911d.pem"]; 
+    "Load keychain and remove intermediate cert results with root CA and cert file with id in the name"
+)]
+#[test_case(
+    &["foo_ca.cert.pem", "foo_ca.cert.pem", "foo_ca.cert.pem"], 
+    &[],
+    &["c128af8c6d0ba34d940582c01443911d"],
+    &["foo_ca.cert.pem"]; 
+    "Adding duplicates results in a single certificate in the store"
+)]
+#[serial]
+fn certificate_store_test(
+    certs_to_add: &[&str],
+    ids_to_remove: &[&str],
+    expected_ids: &[&str],
+    expected_files: &[&str],
+) {
+    // Having
+    let (resource_cert_dir, test_cert_dir) = init_cert_dirs();
+    load_certificates(&resource_cert_dir, &test_cert_dir, certs_to_add);
+    remove_certificates(&test_cert_dir, ids_to_remove);
+    let mut visitor = TestCertDataVisitor::new(expected_ids);
+    // When
+    visitor = visit_certificates(&test_cert_dir, visitor).expect("Can visit loaded certificates");
+    // Then
+    visitor.test();
+    let certs = loaded_cert_files();
+    assert_eq!(certs, to_set(expected_files));
+}
+
+/// Name collision should be resolved
+#[test]
+#[serial]
+fn certificate_name_collision_test() {
+    // Having
+    let (resource_cert_dir, test_cert_dir) = init_cert_dirs();
+    let colliding_name = "foo_inter.cert.pem";
+
+    let mut colliding_file_1 = resource_cert_dir.clone();
+    colliding_file_1.push(colliding_name);
+
+    let mut colliding_file_2 = resource_cert_dir.clone();
+    colliding_file_2.push("copy");
+    fs::create_dir_all(&colliding_file_2).expect("Can create dir");
+    colliding_file_2.push(colliding_name);
+    let mut other = resource_cert_dir.clone();
+    other.push("foo_req.cert.pem");
+    fs::copy(other, colliding_file_2).expect("Can copy file");
+
+    load_certificates(
+        &resource_cert_dir,
+        &test_cert_dir,
+        &[colliding_name, &format!("copy/{colliding_name}")],
+    );
+    let mut visitor = TestCertDataVisitor::new(&[
+        "4e0df976b534cb73794a7613b31af51c",
+        "0e136cb330db24e5630e64ad46489557",
+    ]);
+    // When
+    visitor = visit_certificates(&test_cert_dir, visitor).expect("Can visit loaded certificates");
+    // Then
+    visitor.test();
+    let certs = loaded_cert_files();
+    assert_eq!(
+        certs,
+        to_set(&["foo_inter.cert.pem", "foo_inter.cert.0.pem"])
+    );
+}
diff --git a/utils/manifest-utils/tests/certificate_validation.rs b/utils/manifest-utils/tests/certificate_validation.rs
new file mode 100644
index 0000000000..e285f59de9
--- /dev/null
+++ b/utils/manifest-utils/tests/certificate_validation.rs
@@ -0,0 +1,84 @@
+pub mod common;
+
+#[macro_use]
+extern crate serial_test;
+
+use std::{fs, path::PathBuf};
+
+use common::*;
+use ya_manifest_utils::Keystore;
+
+#[test]
+#[serial]
+fn valid_certificate_test() {
+    // Having
+    let (resource_cert_dir, test_cert_dir) = init_cert_dirs();
+    load_certificates(
+        &resource_cert_dir,
+        &test_cert_dir,
+        &["foo_ca-chain.cert.pem"],
+    );
+
+    let request = prepare_request(resource_cert_dir);
+
+    // Then
+    let keystore = Keystore::load(&test_cert_dir).expect("Can load certificates");
+    keystore
+        .verify_signature(request.cert, request.sig, request.sig_alg, request.data)
+        .expect("Signature and cert can be validated")
+}
+
+#[test]
+#[serial]
+fn invalid_certificate_test() {
+    // Having
+    let (resource_cert_dir, test_cert_dir) = init_cert_dirs();
+    load_certificates(&resource_cert_dir, &test_cert_dir, &[]);
+
+    let request = prepare_request(resource_cert_dir);
+
+    // Then
+    let keystore = Keystore::load(&test_cert_dir).expect("Can load certificates");
+    let result =
+        keystore.verify_signature(request.cert, request.sig, request.sig_alg, request.data);
+    assert!(
+        result.is_err(),
+        "Keystore has no intermediate cert - verification should fail"
+    );
+    let err = result.err().expect("Error result");
+    let msg = format!("{err:?}");
+    assert_eq!(msg, "Invalid certificate");
+}
+
+struct SignedRequest {
+    cert: String,
+    sig: String,
+    sig_alg: String,
+    data: String,
+}
+
+fn prepare_request(resource_cert_dir: PathBuf) -> SignedRequest {
+    let resource_dir = common::test_resources_dir_path();
+
+    let mut cert = resource_cert_dir.clone();
+    cert.push("foo_req.cert.pem");
+    let mut cert = fs::read_to_string(cert).expect("Can read certificate file");
+    cert = base64::encode(cert);
+
+    let mut data = resource_dir.clone();
+    data.push("data.json.base64");
+    let data = fs::read_to_string(data).expect("Can read resource file");
+
+    let mut sig = resource_dir.clone();
+    sig.push("data.json.base64.foo_req_sign.sha256.base64");
+    let sig = fs::read_to_string(sig).expect("Can read resource file");
+
+    let sig_alg = "sha256".to_string();
+
+    SignedRequest {
+        cert,
+        sig,
+        sig_alg,
+        data,
+    }
+}
diff --git a/utils/manifest-utils/tests/common.rs b/utils/manifest-utils/tests/common.rs
new file mode 100644
index 0000000000..2036c8c9ca
--- /dev/null
+++ b/utils/manifest-utils/tests/common.rs
@@ -0,0 +1,128 @@
+use std::fs::{self, File};
+use std::sync::Once;
+use std::{collections::HashSet, path::PathBuf};
+
+use tar::Archive;
+
+use ya_manifest_utils::{
+    util::{self, CertBasicData, CertBasicDataVisitor},
+    KeystoreLoadResult, KeystoreRemoveResult,
+};
+
+static INIT: Once = Once::new();
+
+pub fn load_certificates(
+    resource_cert_dir: &PathBuf,
+    test_cert_dir: &PathBuf,
+    certfile_names: &[&str],
+) -> KeystoreLoadResult {
+    let cert_paths: Vec<PathBuf> = certfile_names
+        .iter()
+        .map(|certfile_name| {
+            let mut cert_path = resource_cert_dir.clone();
+            cert_path.push(certfile_name);
+            cert_path
+        })
+        .collect();
+    let keystore_manager =
+        util::KeystoreManager::try_new(test_cert_dir).expect("Can create keystore manager");
+    keystore_manager
+        .load_certs(&cert_paths)
+        .expect("Can load certificates")
+}
+
+pub fn remove_certificates(test_cert_dir: &PathBuf, cert_ids: &[&str]) -> KeystoreRemoveResult {
+    let keystore_manager =
+        util::KeystoreManager::try_new(test_cert_dir).expect("Can create keystore manager");
+    keystore_manager
+        .remove_certs(&to_set(cert_ids))
+        .expect("Can load certificates")
+}
+
+#[derive(Default)]
+pub struct TestCertDataVisitor {
+    expected: HashSet<String>,
+    actual: HashSet<String>,
+}
+
+impl TestCertDataVisitor {
+    #[allow(dead_code)]
+    pub fn new(expected: &[&str]) -> Self {
+        let mut visitor = Self::default();
+        visitor.expected = expected.iter().map(|s| s.to_string()).collect();
+        visitor
+    }
+
+    #[allow(dead_code)]
+    pub fn test(&self) {
+        assert_eq!(self.expected, self.actual)
+    }
+}
+
+impl CertBasicDataVisitor for TestCertDataVisitor {
+    fn accept(&mut self, cert_data: CertBasicData) {
+        self.actual.insert(cert_data.id);
+    }
+}
+
+pub fn init_cert_dirs() -> (PathBuf, PathBuf) {
+    let resource_cert_dir = resource_cert_dir_path();
+    INIT.call_once(|| {
+        if resource_cert_dir.exists() {
+            fs::remove_dir_all(&resource_cert_dir).expect("Can delete test cert resources dir");
+        }
+        fs::create_dir_all(&resource_cert_dir).expect("Can create temp dir");
+        unpack_cert_resources(&resource_cert_dir);
+    });
+    let store_cert_dir = store_cert_dir_path();
+    if store_cert_dir.exists() {
+        // we want to clean store cert dir before every test
+        fs::remove_dir_all(&store_cert_dir).expect("Can delete test temp dir");
+    }
+    fs::create_dir_all(&store_cert_dir).expect("Can create temp dir");
+    (resource_cert_dir, store_cert_dir)
+}
+
+pub fn loaded_cert_files() -> HashSet<String> {
+    let store_cert_dir = store_cert_dir_path();
+    let cert_dir = fs::read_dir(store_cert_dir).expect("Can read cert dir");
+    cert_dir
+        .into_iter()
+        .map(|file| file.expect("Can list cert files"))
+        .map(|x| x.file_name().to_string_lossy().to_string())
+        .collect()
+}
+
+pub fn to_set<T: AsRef<str>>(v: &[T]) -> HashSet<String> {
+    v.iter()
+        .map(|s| s.as_ref().to_string())
+        .collect::<HashSet<String>>()
+}
+
+fn unpack_cert_resources(cert_resources_dir: &PathBuf) {
+    let mut cert_archive = test_resources_dir_path();
+    cert_archive.push("certificates.tar");
+    let cert_archive = File::open(cert_archive).expect("Can open cert archive file");
+    let mut cert_archive = Archive::new(cert_archive);
+    cert_archive
+        .unpack(cert_resources_dir)
+        .expect("Can unack cert archive");
+}
+
+pub fn test_resources_dir_path() -> PathBuf {
+    let mut test_resources = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    test_resources.push("resources/test");
+    test_resources
+}
+
+fn resource_cert_dir_path() -> PathBuf {
+    let mut cert_dir = PathBuf::from(env!("CARGO_TARGET_TMPDIR"));
+    cert_dir.push("cert_resources");
+    cert_dir
+}
+
+fn store_cert_dir_path() -> PathBuf {
+    let mut cert_dir = PathBuf::from(env!("CARGO_TARGET_TMPDIR"));
+    cert_dir.push("cert_dir");
+    cert_dir
+}