Repository: phantomcyber/splunk-soar-sdk
Pull request #53: feat: add three delicious flavors of asset state
Author: login: sodle-splunk
Created: 2025-05-14T19:09:52Z | Merged: 2025-05-14T22:24:54Z



Diff:
diff --git a/docs/actions/action_anatomy.md b/docs/actions/action_anatomy.md
index 5b049901..b82e6508 100644
--- a/docs/actions/action_anatomy.md
+++ b/docs/actions/action_anatomy.md
@@ -77,7 +77,32 @@ IDE.
 
 You can learn more on actions testing in [the testing docs pages](/docs/testing/index.md).
 
-## 6. Action Description
+## 6. Asset State
+
+The Splunk SOAR platform can preserve some state data in between action runs. This data is stored on the filesystem of the SOAR instance or Automation Broker running the action, and is tied to each individual asset.
+
+State is divided into three dictionaries, which can be accessed from the `SOARClient` object:
+
+- `client.ingestion_state`: stores checkpoints and other data used during polling operations
+- `client.auth_state`: stores session tokens and other authentication data
+- `client.asset_cache`: stores other arbitrary data that can be reused between action runs
+
+These properties are automatically loaded from the filesystem before the action handler function is invoked, and any updates are saved to the filesystem after the handler exits.
+
+State is not encrypted by default, so apps should take care to encrypt any secret data before storing it.
+
+```python
+@app.action()
+def my_stateful_action(params: Params, asset: Asset, client: SOARClient) -> MyActionOutput:
+    if not (session_token := client.auth_state.get("session_token")):
+        session_token = my_api.get_session_token(asset.client_id, asset.client_secret)
+        client.auth_state["session_token"] = session_token
+
+    result = my_api.do_something(session_token)
+    return MyActionOutput(**result)
+```
+
+## 7. Action Description
 
 The docstring is another important part of creating your action. The SDK is using it to generate the action description
 for the app manifest and the documentation in the platform. This method not only documents well your code, but also
@@ -86,7 +111,7 @@ allows to build a documentation for the actions use in the SOAR platform.
 You can pass the `description` param in the `action` decorator to differentiate your code docstring from the actual
 description, but in most cases you will want both to be the same.
 
-## 7. Action Result
+## 8. Action Result
 
 **NOTE:** this functionality is still subject to change - the returned type in the future will be Output, so this doc
 will require updates.
diff --git a/src/soar_sdk/abstract.py b/src/soar_sdk/abstract.py
index 0552c124..732f1132 100644
--- a/src/soar_sdk/abstract.py
+++ b/src/soar_sdk/abstract.py
@@ -14,6 +14,10 @@ class SOARClient(ABC):
     This interface is still a subject to change, so consider it to be WIP.
     """
 
+    ingestion_state: dict
+    auth_state: dict
+    asset_cache: dict
+
     @abstractmethod
     def get_soar_base_url(self) -> str:
         pass
diff --git a/src/soar_sdk/connector.py b/src/soar_sdk/connector.py
index 2cc36954..981238b3 100644
--- a/src/soar_sdk/connector.py
+++ b/src/soar_sdk/connector.py
@@ -12,6 +12,11 @@
     from .actions_provider import ActionsProvider
 
 
+_INGEST_STATE_KEY = "ingestion_state"
+_AUTH_STATE_KEY = "auth_state"
+_CACHE_STATE_KEY = "asset_cache"
+
+
 class AppConnector(BaseConnector, SOARClient):
     """
     The connector mainly generated by the App Wizard. It is extended to properly work
@@ -29,7 +34,9 @@ def __init__(self, actions_provider: "ActionsProvider") -> None:
 
         self.actions_provider = actions_provider
 
-        self._state: dict = {}
+        self.ingestion_state: dict = {}
+        self.auth_state: dict = {}
+        self.asset_cache: dict = {}
 
     @classmethod
     def get_soar_base_url(cls) -> str:
@@ -70,24 +77,20 @@ def handle_action(self, param: dict[str, Any]) -> None:
     def initialize(self) -> bool:
         # Load the state in initialize, use it to store data
         # that needs to be accessed across actions
-        self._state = self.load_state() or {}
-
-        # TODO: get the asset config
-        # config = self.get_config()  # noqa: ERA001
-        """
-        # Access values in asset config by the name
-
-        # Required values can be accessed directly
-        required_config_name = config['required_config_name']
-
-        # Optional values should use the .get() function
-        optional_config_name = config.get('optional_config_name')
-        """
+        state = self.load_state() or {}
+        self.ingestion_state = state.get(_INGEST_STATE_KEY, {})
+        self.auth_state = state.get(_AUTH_STATE_KEY, {})
+        self.asset_cache = state.get(_CACHE_STATE_KEY, {})
 
         return True
 
     def finalize(self) -> bool:
-        self.save_state(self._state)
+        state = {
+            _INGEST_STATE_KEY: self.ingestion_state,
+            _AUTH_STATE_KEY: self.auth_state,
+            _CACHE_STATE_KEY: self.asset_cache,
+        }
+        self.save_state(state)
         return True
 
     def add_result(self, action_result: PhantomActionResult) -> PhantomActionResult:
diff --git a/tests/test_app_action.py b/tests/test_app_action.py
index 2964f199..0f81c091 100644
--- a/tests/test_app_action.py
+++ b/tests/test_app_action.py
@@ -121,6 +121,33 @@ def action_function(params: SampleParams, client: SOARClient) -> ActionOutput:
     assert client_mock.debug.call_count == 1
 
 
+def test_action_call_with_state(simple_app, sample_params):
+    initial_state = {"key": "initial"}
+    updated_state = {"key": "updated"}
+
+    @simple_app.action()
+    def action_function(params: SampleParams, soar: SOARClient) -> ActionOutput:
+        assert soar.ingestion_state == initial_state
+        assert soar.auth_state == initial_state
+        assert soar.asset_cache == initial_state
+
+        soar.ingestion_state = updated_state
+        soar.auth_state = updated_state
+        soar.asset_cache = updated_state
+
+    client_mock = mock.Mock()
+
+    client_mock.ingestion_state = initial_state
+    client_mock.auth_state = initial_state
+    client_mock.asset_cache = initial_state
+
+    action_function(sample_params, soar=client_mock)
+
+    assert client_mock.ingestion_state == updated_state
+    assert client_mock.auth_state == updated_state
+    assert client_mock.asset_cache == updated_state
+
+
 def test_app_action_simple_declaration(simple_app):
     @simple_app.action()
     def some_handler(params: Params) -> ActionOutput: ...
diff --git a/tests/test_connector.py b/tests/test_connector.py
index 852fdf2b..4102933e 100644
--- a/tests/test_connector.py
+++ b/tests/test_connector.py
@@ -2,7 +2,12 @@
 
 import pytest
 
-from soar_sdk.connector import AppConnector
+from soar_sdk.connector import (
+    AppConnector,
+    _INGEST_STATE_KEY,
+    _AUTH_STATE_KEY,
+    _CACHE_STATE_KEY,
+)
 from soar_sdk.input_spec import InputSpecification
 from tests.stubs import SampleActionParams
 
@@ -76,7 +81,13 @@ def test_app_connector_delegates_set_csrf_info(simple_connector: AppConnector):
 def test_app_connector_initialize_loads_state(simple_connector: AppConnector):
     """Test that initialize loads the state from load_state method."""
     # Mock the load_state method to return a specific state
-    test_state = {"key1": "value1", "key2": "value2"}
+    test_state_inner = {"test_key": "test_value"}
+    test_state = {
+        _INGEST_STATE_KEY: test_state_inner,
+        _AUTH_STATE_KEY: test_state_inner,
+        _CACHE_STATE_KEY: test_state_inner,
+    }
+
     simple_connector.load_state = mock.Mock(return_value=test_state)
 
     # Call initialize
@@ -89,7 +100,9 @@ def test_app_connector_initialize_loads_state(simple_connector: AppConnector):
     simple_connector.load_state.assert_called_once()
 
     # Verify the state was stored correctly
-    assert simple_connector._state == test_state
+    assert simple_connector.ingestion_state == test_state_inner
+    assert simple_connector.auth_state == test_state_inner
+    assert simple_connector.asset_cache == test_state_inner
 
 
 def test_app_connector_initialize_handles_empty_state(simple_connector: AppConnector):
@@ -107,14 +120,18 @@ def test_app_connector_initialize_handles_empty_state(simple_connector: AppConne
     simple_connector.load_state.assert_called_once()
 
     # Verify the state was initialized to an empty dict
-    assert simple_connector._state == {}
+    assert simple_connector.ingestion_state == {}
+    assert simple_connector.auth_state == {}
+    assert simple_connector.asset_cache == {}
 
 
 def test_app_connector_finalize_saves_state(simple_connector: AppConnector):
     """Test that finalize saves the current state using save_state."""
     # Set up a test state
     test_state = {"key1": "value1", "key2": "value2"}
-    simple_connector._state = test_state
+    simple_connector.ingestion_state = test_state
+    simple_connector.auth_state = test_state
+    simple_connector.asset_cache = test_state
 
     # Mock the save_state method
     simple_connector.save_state = mock.Mock()
@@ -126,4 +143,10 @@ def test_app_connector_finalize_saves_state(simple_connector: AppConnector):
     assert result is True
 
     # Verify save_state was called with the correct state
-    simple_connector.save_state.assert_called_once_with(test_state)
+    simple_connector.save_state.assert_called_once_with(
+        {
+            _INGEST_STATE_KEY: test_state,
+            _AUTH_STATE_KEY: test_state,
+            _CACHE_STATE_KEY: test_state,
+        }
+    )


Reviews:
user: login: phantom-jacob
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T21:25:27Z
user: login: sodle-splunk
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T21:31:58Z
user: login: phantom-jacob
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T21:32:47Z
user: login: sodle-splunk
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T21:43:13Z
user: login: sodle-splunk
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T21:43:36Z
user: login: sodle-splunk
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-14T21:54:40Z
user: login: phantom-jacob
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-14T22:24:26Z

Review comments:
diff_hunk: @@ -77,7 +77,32 @@ IDE.
 
 You can learn more on actions testing in [the testing docs pages](/docs/testing/index.md).
 
-## 6. Action Description
+## 6. Asset State
+
+The Splunk SOAR platform can preserve some state data in between action runs. This data is stored on the filesystem of the SOAR instance or Automation Broker running the action, and is tied to each individual asset.
+
+State is divided into three dictionaries, which can be accessed from the `SOARClient` object:
+
+- `client.ingestion_state`: stores checkpoints and other data used during polling operations
+- `client.auth_state`: stores session tokens and other authentication data
+- `client.asset_cache`: stores other arbitrary data that can be reused between action runs
+
+These properties are automatically loaded from the filesystem before the action handler function is invoked, and any updates are saved to the filesystem after the handler exits.
+
+State is not encrypted by default, so apps should take care to encrypt any secret data before storing it.
path: docs/actions/action_anatomy.md
user: login: phantom-jacob
body: I say the SDK should encrypt/decrypt the entire state transparently, until the platform does it for us eventually
created_at: 2025-05-14T21:14:13Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +77,32 @@ IDE.
 
 You can learn more on actions testing in [the testing docs pages](/docs/testing/index.md).
 
-## 6. Action Description
+## 6. Asset State
+
+The Splunk SOAR platform can preserve some state data in between action runs. This data is stored on the filesystem of the SOAR instance or Automation Broker running the action, and is tied to each individual asset.
+
+State is divided into three dictionaries, which can be accessed from the `SOARClient` object:
+
+- `client.ingestion_state`: stores checkpoints and other data used during polling operations
+- `client.auth_state`: stores session tokens and other authentication data
+- `client.asset_cache`: stores other arbitrary data that can be reused between action runs
+
+These properties are automatically loaded from the filesystem before the action handler function is invoked, and any updates are saved to the filesystem after the handler exits.
+
+State is not encrypted by default, so apps should take care to encrypt any secret data before storing it.
+
+```python
+@app.action()
+def my_stateful_action(params: Params, asset: Asset, client: SOARClient) -> MyActionOutput:
+    if not (session_token := client.auth_state.get("session_token")):
+        session_token = my_api.get_session_token(asset.client_id, asset.client_secret)
+        client.auth_state["session_token"] = session_token
path: docs/actions/action_anatomy.md
user: login: phantom-jacob
body: Discussion: would it be more intuitive for state to be attached to the asset? Technical implications aside, given that it is _asset state_, my first thought on reading this doc was that it would feel more natural being attached there, like so:

```python
@app.action()
def my_stateful_action(params: Params, asset: Asset, client: SOARClient) -> MyActionOutput:
    if not (session_token := asset.auth_state.get("session_token")):
        session_token = my_api.get_session_token(asset.client_id, asset.client_secret)
        asset.auth_state["session_token"] = session_token
```

Of course, that would preclude assets from defining `auth_state`, `ingestion_state`, and `cache` keys...
created_at: 2025-05-14T21:17:17Z
author_association: COLLABORATOR
diff_hunk: @@ -12,6 +12,11 @@
     from .actions_provider import ActionsProvider
 
 
+INGEST_STATE_KEY = "ingestion_state"
+AUTH_STATE_KEY = "auth_state"
+CACHE_STATE_KEY = "asset_cache"
path: src/soar_sdk/connector.py
user: login: phantom-jacob
body: What about:

```python

class AssetState(pydantic.BaseModel):
    ingestion_state: dict[str, Any] = pydantic.Field(default_factory=dict)
    auth_state: dict[str, Any] = pydantic.Field(default_factory=dict)
    asset_cache: dict[str, Any] = pydantic.Field(default_factory=dict)
```

And then we could serialize/deserialize it in load and save state, but give SDK devs type hints and autocomplete instead of constants
created_at: 2025-05-14T21:20:13Z
author_association: COLLABORATOR
diff_hunk: @@ -121,6 +121,36 @@ def action_function(params: SampleParams, client: SOARClient) -> ActionOutput:
     assert client_mock.debug.call_count == 1
 
 
+def test_action_call_with_state(simple_app, sample_params):
+    initial_state = {"key": "initial"}
+    updated_state = {"key": "updated"}
+
+    @simple_app.action()
+    def action_function(params: SampleParams, soar: SOARClient) -> ActionOutput:
+        assert soar.ingestion_state == initial_state
+        assert soar.auth_state == initial_state
+        assert soar.asset_cache == initial_state
+        soar.debug("TAG", "Progress was made")
path: tests/test_app_action.py
user: login: phantom-jacob
body: Shouldn't we be planning on removing these logging functions from `SOARClient`? Something something zen of python...
```
There should be one-- and preferably only one --obvious way to do it.
```
created_at: 2025-05-14T21:24:38Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +77,32 @@ IDE.
 
 You can learn more on actions testing in [the testing docs pages](/docs/testing/index.md).
 
-## 6. Action Description
+## 6. Asset State
+
+The Splunk SOAR platform can preserve some state data in between action runs. This data is stored on the filesystem of the SOAR instance or Automation Broker running the action, and is tied to each individual asset.
+
+State is divided into three dictionaries, which can be accessed from the `SOARClient` object:
+
+- `client.ingestion_state`: stores checkpoints and other data used during polling operations
+- `client.auth_state`: stores session tokens and other authentication data
+- `client.asset_cache`: stores other arbitrary data that can be reused between action runs
+
+These properties are automatically loaded from the filesystem before the action handler function is invoked, and any updates are saved to the filesystem after the handler exits.
+
+State is not encrypted by default, so apps should take care to encrypt any secret data before storing it.
path: docs/actions/action_anatomy.md
user: login: sodle-splunk
body: Would that require encryption_helper to be ported over first?
created_at: 2025-05-14T21:31:58Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +77,32 @@ IDE.
 
 You can learn more on actions testing in [the testing docs pages](/docs/testing/index.md).
 
-## 6. Action Description
+## 6. Asset State
+
+The Splunk SOAR platform can preserve some state data in between action runs. This data is stored on the filesystem of the SOAR instance or Automation Broker running the action, and is tied to each individual asset.
+
+State is divided into three dictionaries, which can be accessed from the `SOARClient` object:
+
+- `client.ingestion_state`: stores checkpoints and other data used during polling operations
+- `client.auth_state`: stores session tokens and other authentication data
+- `client.asset_cache`: stores other arbitrary data that can be reused between action runs
+
+These properties are automatically loaded from the filesystem before the action handler function is invoked, and any updates are saved to the filesystem after the handler exits.
+
+State is not encrypted by default, so apps should take care to encrypt any secret data before storing it.
path: docs/actions/action_anatomy.md
user: login: phantom-jacob
body: Yeah probably. So out of scope for this PR
created_at: 2025-05-14T21:32:47Z
author_association: COLLABORATOR
diff_hunk: @@ -12,6 +12,11 @@
     from .actions_provider import ActionsProvider
 
 
+INGEST_STATE_KEY = "ingestion_state"
+AUTH_STATE_KEY = "auth_state"
+CACHE_STATE_KEY = "asset_cache"
path: src/soar_sdk/connector.py
user: login: sodle-splunk
body: Devs already do not need to use these constants: the three state objects are available on the SOARClient as `soar.ingestion_state`, `soar.auth_state`, and `soar.asset_cache`. Autocomplete sees them and recognizes them as dicts. See `test_app_action` changes for example.

The constants are only here to prevent using the same string literal in two different places + tests, and I suppose we can/should make them `_PRIVATE`.
created_at: 2025-05-14T21:43:13Z
author_association: COLLABORATOR
diff_hunk: @@ -121,6 +121,36 @@ def action_function(params: SampleParams, client: SOARClient) -> ActionOutput:
     assert client_mock.debug.call_count == 1
 
 
+def test_action_call_with_state(simple_app, sample_params):
+    initial_state = {"key": "initial"}
+    updated_state = {"key": "updated"}
+
+    @simple_app.action()
+    def action_function(params: SampleParams, soar: SOARClient) -> ActionOutput:
+        assert soar.ingestion_state == initial_state
+        assert soar.auth_state == initial_state
+        assert soar.asset_cache == initial_state
+        soar.debug("TAG", "Progress was made")
path: tests/test_app_action.py
user: login: sodle-splunk
body: Yeah probably. I'll get rid of this debug line.
created_at: 2025-05-14T21:43:36Z
author_association: COLLABORATOR
diff_hunk: @@ -77,7 +77,32 @@ IDE.
 
 You can learn more on actions testing in [the testing docs pages](/docs/testing/index.md).
 
-## 6. Action Description
+## 6. Asset State
+
+The Splunk SOAR platform can preserve some state data in between action runs. This data is stored on the filesystem of the SOAR instance or Automation Broker running the action, and is tied to each individual asset.
+
+State is divided into three dictionaries, which can be accessed from the `SOARClient` object:
+
+- `client.ingestion_state`: stores checkpoints and other data used during polling operations
+- `client.auth_state`: stores session tokens and other authentication data
+- `client.asset_cache`: stores other arbitrary data that can be reused between action runs
+
+These properties are automatically loaded from the filesystem before the action handler function is invoked, and any updates are saved to the filesystem after the handler exits.
+
+State is not encrypted by default, so apps should take care to encrypt any secret data before storing it.
+
+```python
+@app.action()
+def my_stateful_action(params: Params, asset: Asset, client: SOARClient) -> MyActionOutput:
+    if not (session_token := client.auth_state.get("session_token")):
+        session_token = my_api.get_session_token(asset.client_id, asset.client_secret)
+        client.auth_state["session_token"] = session_token
path: docs/actions/action_anatomy.md
user: login: sodle-splunk
body: I see the argument for that.

However, I don't love that we'd be reserving yet more names in the Asset namespace. Side question: we should make sure that we raise an alarm if a dev tries to overshadow one of these names we're reserving.

It also adds some complexity to the loading/saving of state: The AppConnector is a subclass of both `BaseConnector` and `SOARClient`, so it's hooked into the initialize/finalize lifecycle, has access to the underlying save_state/load_state APIs, and has a relatively convenient place to put three new fields. Asset is much more indirect.
created_at: 2025-05-14T21:54:40Z
author_association: COLLABORATOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-14T19:10:27Z
body: <!-- semantic-release-breadcrumb -->
Merging this PR will release `1.0.0-beta.19` with the following release notes:

# [1.0.0-beta.19](https://github.com/phantomcyber/splunk-soar-sdk/compare/1.0.0-beta.18...1.0.0-beta.19) (2025-05-14)


### Bug Fixes

* make state keys private, and remove debug line from test ([3601de3](https://github.com/phantomcyber/splunk-soar-sdk/commit/3601de392bc62677f708a6aff914217dcf629588))


### Features

* add three delicious flavors of asset state ([c406a05](https://github.com/phantomcyber/splunk-soar-sdk/commit/c406a05df5effdeeda07b33a4ee6aaa8861d728e))
author_association: NONE
user: login: soar-sdk-semantic-release[bot]
created_at: 2025-05-14T22:27:25Z
body: :tada: This PR is included in version 1.0.0-beta.19 :tada:

The release is available on [GitHub release](https://github.com/phantomcyber/splunk-soar-sdk/releases/tag/1.0.0-beta.19)

Your **[semantic-release](https://github.com/semantic-release/semantic-release)** bot :package::rocket:
author_association: NONE