Repository: p2panda/p2panda
Pull request #710: Partial ordering algorithm for operations based on `previous` links 
Author: login: sandreae
Created: 2025-03-03T14:54:12Z | Merged: 2025-03-28T09:27:17Z

- generic `PartialOrder` struct for non-deterministically sorting items in a DAG by applying a partial order
- dependency store traits and memory implementation used by `PartialOrder`
- specialised wrapper around `PartialOrder` which sorts p2panda operations based on dependencies contained in their `previous` field

Closes: https://github.com/p2panda/p2panda/issues/644

## Next Steps
- fuzz testing
 
## 📋 Checklist

- [x] Add tests that cover your changes
- [x] Add this PR to the _Unreleased_ section in `CHANGELOG.md`
- [x] Link this PR to any issues it closes
- [x] New files contain a SPDX license header



Diff:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 124dba983..662ceedc0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,10 @@ Highlights are marked with a pancake 🥞
 
 ## [Unreleased]
 
+### Added
+
+- Partial ordering algorithm for operations [#710](https://github.com/p2panda/p2panda/pull/710)
+
 ### Fixed
 
 - Wait for relay connection initialisation and first direct address [#725](https://github.com/p2panda/p2panda/pull/725)
diff --git a/p2panda-stream/src/lib.rs b/p2panda-stream/src/lib.rs
index c2252d413..502ca6f81 100644
--- a/p2panda-stream/src/lib.rs
+++ b/p2panda-stream/src/lib.rs
@@ -13,8 +13,10 @@
 //! techniques for outdated operations etc.
 mod macros;
 pub mod operation;
+mod ordering;
 mod stream;
 #[cfg(test)]
 mod test_utils;
 
+pub use ordering::*;
 pub use stream::*;
diff --git a/p2panda-stream/src/ordering/mod.rs b/p2panda-stream/src/ordering/mod.rs
new file mode 100644
index 000000000..ba956c274
--- /dev/null
+++ b/p2panda-stream/src/ordering/mod.rs
@@ -0,0 +1,3 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod partial;
diff --git a/p2panda-stream/src/ordering/partial/mod.rs b/p2panda-stream/src/ordering/partial/mod.rs
new file mode 100644
index 000000000..f6e82e875
--- /dev/null
+++ b/p2panda-stream/src/ordering/partial/mod.rs
@@ -0,0 +1,383 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+use std::marker::PhantomData;
+
+use thiserror::Error;
+
+pub use crate::partial::store::{MemoryStore, PartialOrderStore};
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over a set of items which form a dependency graph.
+///
+/// A partial order sorts items based on their causal relationships. An item can be "before",
+/// "after" or "at the same time" as any other item.
+///
+/// This functionality is required when, for example, processing a set of messages where some
+/// messages _must_ be processed before others. A set such as this would naturally form a graph
+/// structure, each item would have a chain of dependencies. Another example would be a package
+/// dependency tree, where a certain package depends on one or many others. In order to understand
+/// which order we should install these packages, we need to partially order the set and process
+/// them from start to finish.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. The approach taken in this module establishes a partial order over all items in the
+/// set. The word "partial" indicates that some items may not be directly comparable. Items in
+/// different branches of the graph may not have a direct path between them, and so we don't know
+/// "which should come first". In fact, as there is no dependency relation between them, it makes
+/// no difference which comes first, and depending on the order items are processed the ordering
+/// process may arrive at different results (it is a non-deterministic algorithm).
+///
+/// Items in the process of being ordered are considered to be in one of two states. They are
+/// considered in a "ready" state when all their dependencies have themselves been processed, and
+/// in a "pending" state when their dependencies have not yet been processed.
+///
+/// If an item is in a "pending" state then it is held in a pending queue and if it's dependencies
+/// are later processed and "ready", then the so far "pending" item will be moved to the "ready"
+/// queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// ```text
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+/// ```
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// ```text
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+/// ```
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// ```text
+/// A <-- ?? <-- C
+///  \-- B1 <--/
+/// ```
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// ```text
+/// [A, B1]
+/// ```
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
+#[derive(Debug)]
+pub struct PartialOrder<K, S> {
+    /// Store for managing "ready" and "pending" items.
+    store: S,
+    _phantom: PhantomData<K>,
+}
+
+impl<K, S> PartialOrder<K, S>
+where
+    K: Clone + Copy + Display + StdHash + PartialEq + Eq,
+    S: PartialOrderStore<K>,
+{
+    pub fn new(store: S) -> Self {
+        Self {
+            store,
+            _phantom: PhantomData,
+        }
+    }
+
+    /// Pop the next item from the ready queue.
+    pub async fn next(&mut self) -> Result<Option<K>, PartialOrderError> {
+        self.store.take_next_ready().await
+    }
+
+    /// Process a new item which may be in a "ready" or "pending" state.
+    pub async fn process(&mut self, key: K, dependencies: &[K]) -> Result<(), PartialOrderError> {
+        if !self.store.ready(dependencies).await? {
+            self.store.mark_pending(key, dependencies.to_vec()).await?;
+            return Ok(());
+        }
+
+        self.store.mark_ready(key).await?;
+
+        // We added a new ready item to the store so now we want to process any pending items
+        // which depend on it as they may now have transitioned into a ready state.
+        self.process_pending(key).await?;
+
+        Ok(())
+    }
+
+    /// Recursively check if any pending items now have their dependencies met.
+    async fn process_pending(&mut self, key: K) -> Result<(), PartialOrderError> {
+        // Get all items which depend on the passed key.
+        let Some(dependents) = self.store.get_next_pending(key).await? else {
+            return Ok(());
+        };
+
+        // For each dependent check if it has all it's dependencies met, if not then we do nothing
+        // as it is still in a pending state.
+        for (next_key, next_deps) in dependents {
+            if !self.store.ready(&next_deps).await? {
+                continue;
+            }
+
+            self.store.mark_ready(next_key).await?;
+
+            // Recurse down the dependency graph by now checking any pending items which depend on
+            // the current item.
+            Box::pin(self.process_pending(next_key)).await?;
+        }
+
+        // Finally remove this item from the pending items queue.
+        self.store.remove_pending(key).await?;
+
+        Ok(())
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use std::collections::HashSet;
+
+    use super::{MemoryStore, PartialOrder};
+
+    #[tokio::test]
+    async fn partial_order() {
+        // Graph
+        //
+        // A <-- B <--------- D
+        //        \--- C <---/
+        //
+        let graph = [
+            ("A", vec![]),
+            ("B", vec!["A"]),
+            ("C", vec!["B"]),
+            ("D", vec!["B", "C"]),
+        ];
+
+        // A has no dependencies and so it's added straight to the processed set and ready queue.
+        let store = MemoryStore::default();
+        let mut checker = PartialOrder::new(store);
+        let item = graph[0].clone();
+        checker.process(item.0, &item.1).await.unwrap();
+        assert_eq!(checker.store.ready.len(), 1);
+        assert_eq!(checker.store.pending.len(), 0);
+        assert_eq!(checker.store.ready_queue.len(), 1);
+
+        // B has it's dependencies met and so it too is added to the processed set and ready
+        // queue.
+        let item = graph[1].clone();
+        checker.process(item.0, &item.1).await.unwrap();
+        assert_eq!(checker.store.ready.len(), 2);
+        assert_eq!(checker.store.pending.len(), 0);
+        assert_eq!(checker.store.ready_queue.len(), 2);
+
+        // D doesn't have both its dependencies met yet so it waits in the pending queue.
+        let item = graph[3].clone();
+        checker.process(item.0, &item.1).await.unwrap();
+        assert_eq!(checker.store.ready.len(), 2);
+        assert_eq!(checker.store.pending.len(), 1);
+        assert_eq!(checker.store.ready_queue.len(), 2);
+
+        // C satisfies D's dependencies and so both C & D are added to the processed set
+        // and ready queue.
+        let item = graph[2].clone();
+        checker.process(item.0, &item.1).await.unwrap();
+        assert_eq!(checker.store.ready.len(), 4);
+        assert_eq!(checker.store.pending.len(), 0);
+        assert_eq!(checker.store.ready_queue.len(), 4);
+
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("A"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("B"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("C"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("D"));
+        let item = checker.next().await.unwrap();
+        assert!(item.is_none());
+    }
+
+    #[tokio::test]
+    async fn partial_order_with_recursion() {
+        // Graph
+        //
+        // A <-- B <--------- D
+        //        \--- C <---/
+        //
+        let incomplete_graph = [
+            ("A", vec![]),
+            ("C", vec!["B"]),
+            ("D", vec!["C"]),
+            ("E", vec!["D"]),
+            ("F", vec!["E"]),
+            ("G", vec!["F"]),
+        ];
+
+        let store = MemoryStore::default();
+        let mut checker = PartialOrder::new(store);
+        for (key, dependencies) in incomplete_graph {
+            checker.process(key, &dependencies).await.unwrap();
+        }
+        assert_eq!(checker.store.ready.len(), 1);
+        assert_eq!(checker.store.pending.len(), 5);
+        assert_eq!(checker.store.ready_queue.len(), 1);
+
+        let missing_dependency = ("B", vec!["A"]);
+
+        checker
+            .process(missing_dependency.0, &missing_dependency.1)
+            .await
+            .unwrap();
+        assert_eq!(checker.store.ready.len(), 7);
+        assert_eq!(checker.store.pending.len(), 0);
+        assert_eq!(checker.store.ready_queue.len(), 7);
+
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("A"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("B"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("C"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("D"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("E"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("F"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("G"));
+        let item = checker.next().await.unwrap();
+        assert!(item.is_none());
+    }
+
+    #[tokio::test]
+    async fn complex_graph() {
+        // Graph
+        //
+        // A <-- B1 <-- C1 <--\
+        //   \-- ?? <-- C2 <-- D
+        //        \---- C3 <--/
+        //
+        let incomplete_graph = [
+            ("A", vec![]),
+            ("B1", vec!["A"]),
+            // This item is missing.
+            // ("B2", vec!["A"]),
+            ("C1", vec!["B1"]),
+            ("C2", vec!["B2"]),
+            ("C3", vec!["B2"]),
+            ("D", vec!["C1", "C2", "C3"]),
+        ];
+
+        let store = MemoryStore::default();
+        let mut checker = PartialOrder::new(store);
+        for (key, dependencies) in incomplete_graph {
+            checker.process(key, &dependencies).await.unwrap();
+        }
+
+        // A1, B1 and C1 have dependencies met and were already processed.
+        assert!(checker.store.ready.len() == 3);
+        assert_eq!(checker.store.pending.len(), 3);
+        assert_eq!(checker.store.ready_queue.len(), 3);
+
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("A"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("B1"));
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("C1"));
+        let item = checker.next().await.unwrap();
+        assert!(item.is_none());
+
+        // No more ready items.
+        assert_eq!(checker.store.ready_queue.len(), 0);
+
+        // Process the missing item.
+        let missing_dependency = ("B2", vec!["A"]);
+        checker
+            .process(missing_dependency.0, &missing_dependency.1)
+            .await
+            .unwrap();
+
+        // All items have now been processed and new ones are waiting in the ready queue.
+        assert_eq!(checker.store.ready.len(), 7);
+        assert_eq!(checker.store.pending.len(), 0);
+        assert_eq!(checker.store.ready_queue.len(), 4);
+
+        let mut concurrent_items = HashSet::from(["C2", "C3"]);
+
+        let item = checker.next().await.unwrap().unwrap();
+        assert_eq!(item, "B2");
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert_eq!(item, "D");
+        let item = checker.next().await.unwrap();
+        assert!(item.is_none());
+    }
+
+    #[tokio::test]
+    async fn very_out_of_order() {
+        // Graph
+        //
+        // A <-- B1 <-- C1 <--\
+        //   \-- B2 <-- C2 <-- D
+        //        \---- C3 <--/
+        //
+        let out_of_order_graph = [
+            ("D", vec!["C1", "C2", "C3"]),
+            ("C1", vec!["B1"]),
+            ("B1", vec!["A"]),
+            ("B2", vec!["A"]),
+            ("C3", vec!["B2"]),
+            ("C2", vec!["B2"]),
+            ("A", vec![]),
+        ];
+
+        let store = MemoryStore::default();
+        let mut checker = PartialOrder::new(store);
+        for (key, dependencies) in out_of_order_graph {
+            checker.process(key, &dependencies).await.unwrap();
+        }
+
+        assert!(checker.store.ready.len() == 7);
+        assert_eq!(checker.store.pending.len(), 0);
+        assert_eq!(checker.store.ready_queue.len(), 7);
+
+        let item = checker.next().await.unwrap();
+        assert_eq!(item, Some("A"));
+
+        let mut concurrent_items = HashSet::from(["B1", "B2", "C1", "C2", "C3"]);
+
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert!(concurrent_items.remove(item));
+        let item = checker.next().await.unwrap().unwrap();
+        assert_eq!(item, "D");
+        let item = checker.next().await.unwrap();
+        assert!(item.is_none());
+    }
+}
diff --git a/p2panda-stream/src/ordering/partial/operations.rs b/p2panda-stream/src/ordering/partial/operations.rs
new file mode 100644
index 000000000..7f4548df8
--- /dev/null
+++ b/p2panda-stream/src/ordering/partial/operations.rs
@@ -0,0 +1,296 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::marker::PhantomData;
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::partial::{
+    PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore,
+};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation dependency graph into a partial order. Here we have the addition of a `LogStore`
+/// and `OperationStore` implementation (traits from `p2panda-store`).
+#[derive(Debug)]
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    ///
+    /// It is assumed that any operations being processed by the PartialOrder struct are already
+    /// present in the store, as we may need to retrieve them later.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    _phantom: PhantomData<(L, E)>,
+}
+
+impl<L, E, OS, POS> PartialOrder<L, E, OS, POS>
+where
+    OS: OperationStore<L, E> + LogStore<L, E>,
+    POS: PartialOrderStore<Hash>,
+    E: Extensions,
+{
+    /// Construct a new dependency checker from an operation store and dependency store.
+    pub fn new(operation_store: OS, partial_order_store: POS) -> Self {
+        let inner_dependency_checker = InnerPartialOrder::new(partial_order_store);
+        PartialOrder {
+            operation_store,
+            inner: inner_dependency_checker,
+            _phantom: PhantomData,
+        }
+    }
+
+    /// Process a single operation.
+    pub async fn process(
+        &mut self,
+        operation: Operation<E>,
+    ) -> Result<(), OperationDependencyCheckerError> {
+        let hash = operation.hash;
+        let previous = operation.header.previous.clone();
+        self.inner.process(hash, &previous).await?;
+        Ok(())
+    }
+
+    /// Take the next ready operation from the queue.
+    pub async fn next(&mut self) -> Result<Option<Operation<E>>, OperationDependencyCheckerError> {
+        let Some(hash) = self.inner.next().await? else {
+            return Ok(None);
+        };
+
+        if let Some((header, body)) = self
+            .operation_store
+            .get_operation(hash)
+            .await
+            .map_err(|err| OperationDependencyCheckerError::StoreError(err.to_string()))?
+        {
+            let operation = Operation { hash, header, body };
+            Ok(Some(operation))
+        } else {
+            Err(OperationDependencyCheckerError::MissingOperation(hash))
+        }
+    }
+}
+
+#[derive(Debug, Error)]
+pub enum OperationDependencyCheckerError {
+    #[error(transparent)]
+    CheckerError(#[from] PartialOrderError),
+
+    #[error("store error: {0}")]
+    StoreError(String),
+
+    #[error("processed operation not found in store: {0}")]
+    MissingOperation(Hash),
+}
+
+#[cfg(test)]
+mod tests {
+    use std::collections::HashSet;
+
+    use p2panda_core::{Header, Operation, PrivateKey};
+    use p2panda_store::{MemoryStore, OperationStore};
+
+    use crate::ordering::partial::MemoryStore as PartialOrderMemoryStore;
+
+    use super::PartialOrder;
+
+    /// Create operations which form the following graph with their previous links:
+    ///
+    /// 0P0 <-- 0P1 <--\
+    ///     \-- OP2 <-- OP4
+    ///      \--OP3 <--/
+    ///
+    /// Each operation is inserted into the store in it's own log.
+    async fn setup(
+        private_key: &PrivateKey,
+        operation_store: &mut MemoryStore<u64>,
+    ) -> Vec<Operation> {
+        let mut header_0 = Header {
+            public_key: private_key.public_key(),
+            timestamp: 0,
+            ..Default::default()
+        };
+        header_0.sign(&private_key);
+        let operation_0 = Operation {
+            hash: header_0.hash(),
+            header: header_0.clone(),
+            body: None,
+        };
+
+        let mut header_1 = Header {
+            public_key: private_key.public_key(),
+            previous: vec![header_0.hash()],
+            timestamp: 1,
+            ..Default::default()
+        };
+        header_1.sign(&private_key);
+        let operation_1 = Operation {
+            hash: header_1.hash(),
+            header: header_1.clone(),
+            body: None,
+        };
+
+        let mut header_2 = Header {
+            public_key: private_key.public_key(),
+            previous: vec![header_0.hash()],
+            timestamp: 2,
+            ..Default::default()
+        };
+        header_2.sign(&private_key);
+        let operation_2 = Operation {
+            hash: header_2.hash(),
+            header: header_2.clone(),
+            body: None,
+        };
+
+        let mut header_3 = Header {
+            public_key: private_key.public_key(),
+            previous: vec![header_0.hash()],
+            timestamp: 3,
+            ..Default::default()
+        };
+        header_3.sign(&private_key);
+        let operation_3 = Operation {
+            hash: header_3.hash(),
+            header: header_3.clone(),
+            body: None,
+        };
+
+        let mut header_4 = Header {
+            public_key: private_key.public_key(),
+            previous: vec![header_1.hash(), header_2.hash(), header_3.hash()],
+            timestamp: 4,
+            ..Default::default()
+        };
+        header_4.sign(&private_key);
+        let operation_4 = Operation {
+            hash: header_4.hash(),
+            header: header_4.clone(),
+            body: None,
+        };
+
+        operation_store
+            .insert_operation(header_0.hash(), &header_0, None, &header_0.to_bytes(), &0)
+            .await
+            .unwrap();
+
+        operation_store
+            .insert_operation(header_1.hash(), &header_1, None, &header_1.to_bytes(), &1)
+            .await
+            .unwrap();
+
+        operation_store
+            .insert_operation(header_2.hash(), &header_2, None, &header_2.to_bytes(), &2)
+            .await
+            .unwrap();
+
+        operation_store
+            .insert_operation(header_3.hash(), &header_3, None, &header_3.to_bytes(), &3)
+            .await
+            .unwrap();
+
+        operation_store
+            .insert_operation(header_4.hash(), &header_4, None, &header_4.to_bytes(), &4)
+            .await
+            .unwrap();
+
+        vec![
+            operation_0,
+            operation_1,
+            operation_2,
+            operation_3,
+            operation_4,
+        ]
+    }
+
+    #[tokio::test]
+    async fn operations_with_previous() {
+        let private_key = PrivateKey::new();
+        let mut operation_store = MemoryStore::<u64>::default();
+        let partial_order_store = PartialOrderMemoryStore::default();
+        let mut dependency_checker =
+            PartialOrder::new(operation_store.clone(), partial_order_store);
+
+        // Setup test data in the store. They form a dependency graph with the following form:
+        //
+        // 0P0 <-- 0P1 <--\
+        //     \-- OP2 <-- OP4
+        //      \--OP3 <--/
+        //
+        let operations = setup(&private_key, &mut operation_store).await;
+
+        // Process each operation out-of-order.
+        let result = dependency_checker.process(operations[4].clone()).await;
+        assert!(result.is_ok());
+
+        let result = dependency_checker.process(operations[3].clone()).await;
+        assert!(result.is_ok());
+
+        let result = dependency_checker.process(operations[2].clone()).await;
+        assert!(result.is_ok());
+
+        let result = dependency_checker.process(operations[1].clone()).await;
+        assert!(result.is_ok());
+
+        let result = dependency_checker.process(operations[0].clone()).await;
+        assert!(result.is_ok());
+
+        // Calling next should give us the first operation which had all it's dependencies met, in
+        // this case it's the root of the graph OP1.
+        let next = dependency_checker.next().await.unwrap();
+        assert_eq!(next.unwrap(), operations[0]);
+
+        // Operations OP1, OP2 and OP3 all only depend on operation OP0 and they were created
+        // concurrently, we should see all of these operations next, but we don't know in what order.
+        let mut concurrent_operations =
+            HashSet::from([operations[1].hash, operations[2].hash, operations[3].hash]);
+
+        let next = dependency_checker.next().await.unwrap();
+        assert!(next.is_some());
+        let next = next.unwrap();
+        assert!(concurrent_operations.remove(&next.hash),);
+
+        let next = dependency_checker.next().await.unwrap();
+        assert!(next.is_some());
+        let next = next.unwrap();
+        assert!(concurrent_operations.remove(&next.hash),);
+
+        let next = dependency_checker.next().await.unwrap();
+        assert!(next.is_some());
+        let next = next.unwrap();
+        assert!(concurrent_operations.remove(&next.hash),);
+
+        // We know OP4 will be given last as it depended on OP1, OP2 and OP3.
+        let next = dependency_checker.next().await.unwrap();
+        assert_eq!(next.unwrap(), operations[4]);
+    }
+
+    #[tokio::test]
+    async fn missing_dependency() {
+        let private_key = PrivateKey::new();
+        let mut operation_store = MemoryStore::<u64>::default();
+        let partial_order_store = PartialOrderMemoryStore::default();
+        let mut dependency_checker =
+            PartialOrder::new(operation_store.clone(), partial_order_store);
+
+        let operations = setup(&private_key, &mut operation_store).await;
+
+        let result = dependency_checker.process(operations[1].clone()).await;
+        assert!(result.is_ok());
+        let result = dependency_checker.process(operations[2].clone()).await;
+        assert!(result.is_ok());
+        let result = dependency_checker.process(operations[3].clone()).await;
+        assert!(result.is_ok());
+        let result = dependency_checker.process(operations[4].clone()).await;
+        assert!(result.is_ok());
+
+        let next = dependency_checker.next().await.unwrap();
+        assert!(next.is_none());
+    }
+}
diff --git a/p2panda-stream/src/ordering/partial/store.rs b/p2panda-stream/src/ordering/partial/store.rs
new file mode 100644
index 000000000..ad9b9be40
--- /dev/null
+++ b/p2panda-stream/src/ordering/partial/store.rs
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet, VecDeque};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use crate::partial::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
+/// - maintain a list of items which don't have their dependencies met
+/// - return all pending items which depend on a given item
+#[allow(async_fn_in_trait)]
+pub trait PartialOrderStore<K>
+where
+    K: Clone + Copy + StdHash + PartialEq + Eq,
+{
+    /// Add an item to the store which has all it's dependencies met already. If this is the first
+    /// time the item has been added it should also be pushed to the end of a "ready" queue.
+    async fn mark_ready(&mut self, key: K) -> Result<bool, PartialOrderError>;
+
+    /// Add an item which does not have all it's dependencies met yet.
+    async fn mark_pending(
+        &mut self,
+        key: K,
+        dependencies: Vec<K>,
+    ) -> Result<bool, PartialOrderError>;
+
+    /// Get all pending items which directly depend on the given key.
+    async fn get_next_pending(
+        &self,
+        key: K,
+    ) -> Result<Option<HashSet<(K, Vec<K>)>>, PartialOrderError>;
+
+    /// Take the next ready item from the ready queue.
+    async fn take_next_ready(&mut self) -> Result<Option<K>, PartialOrderError>;
+
+    /// Remove all items from the pending queue which depend on the passed key.
+    async fn remove_pending(&mut self, key: K) -> Result<bool, PartialOrderError>;
+
+    /// Returns `true` of all the passed keys are present in the ready list.
+    async fn ready(&self, keys: &[K]) -> Result<bool, PartialOrderError>;
+}
+
+/// Memory implementation of the `PartialOrderStore` trait.
+#[derive(Clone)]
+pub struct MemoryStore<K> {
+    pub(crate) ready: HashSet<K>,
+    pub(crate) ready_queue: VecDeque<K>,
+    pub(crate) pending: HashMap<K, HashSet<(K, Vec<K>)>>,
+}
+
+impl<K> Default for MemoryStore<K> {
+    fn default() -> Self {
+        Self {
+            ready: HashSet::new(),
+            ready_queue: VecDeque::new(),
+            pending: HashMap::new(),
+        }
+    }
+}
+
+impl<K> PartialOrderStore<K> for MemoryStore<K>
+where
+    K: Clone + Copy + Debug + StdHash + PartialEq + Eq,
+{
+    async fn mark_ready(&mut self, key: K) -> Result<bool, PartialOrderError> {
+        let result = self.ready.insert(key);
+        if result {
+            self.ready_queue.push_back(key);
+        }
+        Ok(result)
+    }
+
+    async fn mark_pending(
+        &mut self,
+        key: K,
+        dependencies: Vec<K>,
+    ) -> Result<bool, PartialOrderError> {
+        let insert_occured = false;
+        for dep_key in &dependencies {
+            if self.ready.contains(dep_key) {
+                continue;
+            }
+
+            let dependents = self.pending.entry(*dep_key).or_default();
+            dependents.insert((key, dependencies.clone()));
+        }
+
+        Ok(insert_occured)
+    }
+
+    async fn get_next_pending(
+        &self,
+        key: K,
+    ) -> Result<Option<HashSet<(K, Vec<K>)>>, PartialOrderError> {
+        Ok(self.pending.get(&key).cloned())
+    }
+
+    async fn take_next_ready(&mut self) -> Result<Option<K>, PartialOrderError> {
+        Ok(self.ready_queue.pop_front())
+    }
+
+    async fn remove_pending(&mut self, key: K) -> Result<bool, PartialOrderError> {
+        Ok(self.pending.remove(&key).is_some())
+    }
+
+    async fn ready(&self, dependencies: &[K]) -> Result<bool, PartialOrderError> {
+        let deps_set = HashSet::from_iter(dependencies.iter().cloned());
+        let result = self.ready.is_superset(&deps_set);
+        Ok(result)
+    }
+}


Reviews:
user: login: adzialocha
body: Haven't checked the PR properly yet, but have some more general change requests:

* Think this can have more documentation. This will be a core functionality and we should be able to explain it's logic to people who don't know what this is
* The `stream` module is for `Stream` implementations (_not_ our namakemono specification "Stream" which probably will not be our future name for it anyhow), maybe you can think of another folder for that code?
* License headers :-)
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-03T15:01:55Z
user: login: adzialocha
body: Very nice and well readable code, especially the tests are great!

Next to the obvious CHANGELOG entry I think this could still need a bit of doc-string love and then I have some more detailled points, the crash resistant one being the most important maybe.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-06T16:29:03Z
user: login: adzialocha
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T18:15:36Z
user: login: sandreae
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T22:26:22Z
user: login: sandreae
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T22:46:22Z
user: login: sandreae
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T22:53:21Z
user: login: sandreae
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T22:53:50Z
user: login: sandreae
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T23:05:23Z
user: login: adzialocha
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T10:20:07Z
user: login: adzialocha
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T10:21:04Z
user: login: adzialocha
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T10:25:49Z
user: login: adzialocha
body: Thanks for tackling the transaction part!

I was wondering, maybe we can remove the caching part for now? Next to the API, I think it's opening up too many questions, we don't do it anywhere else and seems like an optimization we don't need yet. On top caching _is_ hard (what happens if an operation gets removed, the cache will not learn about that, is this causing any bugs etc.).

I'd be up for skipping that part for now and moving on with what we have (next to documentation improvements, that would be good to have before merging it). What do you think @sandreae?
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-14T18:13:40Z
user: login: sandreae
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T17:15:24Z
user: login: adzialocha
body: Great! The docs are really nice now! Thank you!

I've added a small intro what "partial ordering" means in the context of causality and fixed a minor broken sentence, I hope that was correct.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-28T09:24:00Z

Review comments:
diff_hunk: @@ -11,6 +11,8 @@
 //! techniques for outdated operations etc.
 mod macros;
 pub mod operation;
+#[allow(dead_code)]
+mod ordering;
path: p2panda-stream/src/lib.rs
user: login: adzialocha
body: I'd expose all of these methods, even if there's no `Stream` implementation yet developers should be able to access the "lower level" tools to come up with their own APIs. We already do this for ingest as well :+1:
created_at: 2025-03-06T15:35:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,7 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+mod partial;
+
+#[allow(unused_imports)]
path: p2panda-stream/src/ordering/mod.rs
user: login: adzialocha
body: See comment above, this can go then as well, we want to expose these things.
created_at: 2025-03-06T15:36:20Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: adzialocha
body: I like the examples, maybe we can put them in "comment" blocks aka:

```
A <-- ?? <-- C
  \-- B1 <--/
```
created_at: 2025-03-06T15:43:36Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
+#[derive(Debug)]
+pub struct PartialOrder<K, S> {
+    /// Store for managing "ready" and "pending" items.
+    store: S,
+
+    /// Sorted queue of "ready" items.
+    ready_queue: VecDeque<K>,
+}
+
+impl<K, S> PartialOrder<K, S>
+where
+    K: Clone + Copy + Display + StdHash + PartialEq + Eq,
+    S: PartialOrderStore<K>,
+{
+    pub fn new(store: S) -> Self {
+        Self {
+            store,
+            ready_queue: VecDeque::new(),
+        }
+    }
+
+    /// Pop the next item from the ready queue.
+    pub fn next(&mut self) -> Option<K> {
+        self.ready_queue.pop_front()
+    }
+
+    /// Process a new item which may be in a "ready" or "pending" state.
+    pub async fn process(&mut self, key: K, dependencies: Vec<K>) -> Result<(), PartialOrderError> {
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: adzialocha
body: `dependencies` can be a `&[K]`, with `K` being `Copy` it should be easy to handle.
created_at: 2025-03-06T16:03:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use super::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
+/// - maintain a list of items which don't have their dependencies met
+/// - return all pending items which depend on a given item
+pub trait PartialOrderStore<K>
+where
+    K: Clone + Copy + StdHash + PartialEq + Eq,
+{
+    /// Add an item to the store which has all it's dependencies met already.
+    async fn add_ready(&mut self, key: K) -> Result<bool, PartialOrderError>;
path: p2panda-stream/src/ordering/partial/store.rs
user: login: adzialocha
body: Just a thought: How about `mark_ready` and `mark_pending`?
created_at: 2025-03-06T16:05:06Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: adzialocha
body: The doc-string might need another round of edits maybe, I'm not sure if I understand it reading it for the first time, it might get harder for people who don't know the concept of "ordering", "partial ordering", "dependency checking" or "Directed acyclic graph".

Maybe you can try to frame it in a way where this introduces a user to _what_ they are doing here (from the perspective of someone who is rather new to the concept)?

> "Struct for establishing partial order over items in a Directed-Acyclic-Graph."

The DAG structure is correct but it seems to me that this is a side-effect of creating a structure which represents dependencies (a "dependency graph"). I think the latter is easier to understand because it explains better what the interesting information is we use in this algorithm. 

Also there is no explanation of why we want to get a partial order of this and what a partial order is. You say:

> "This approach establishes a partial order, meaning not all items in the graph are comparable, and is non-deterministic"

.. I'm not sure what "comparable" means in this context and _maybe_ I know what you mean with "non-deterministic" but compared to what and how?

> "The main requirement is that all dependencies of an item are sorted "before" the item itself, the exact order is not a concern."

Not sure if I understand that sentence 😅 
created_at: 2025-03-06T16:07:54Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
+#[derive(Debug)]
+pub struct PartialOrder<K, S> {
+    /// Store for managing "ready" and "pending" items.
+    store: S,
+
+    /// Sorted queue of "ready" items.
+    ready_queue: VecDeque<K>,
+}
+
+impl<K, S> PartialOrder<K, S>
+where
+    K: Clone + Copy + Display + StdHash + PartialEq + Eq,
+    S: PartialOrderStore<K>,
+{
+    pub fn new(store: S) -> Self {
+        Self {
+            store,
+            ready_queue: VecDeque::new(),
+        }
+    }
+
+    /// Pop the next item from the ready queue.
+    pub fn next(&mut self) -> Option<K> {
+        self.ready_queue.pop_front()
+    }
+
+    /// Process a new item which may be in a "ready" or "pending" state.
+    pub async fn process(&mut self, key: K, dependencies: Vec<K>) -> Result<(), PartialOrderError> {
+        if !self.store.ready(&dependencies).await? {
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: adzialocha
body: Looks very efficient ✨ 
created_at: 2025-03-06T16:09:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
+#[derive(Debug)]
+pub struct PartialOrder<K, S> {
+    /// Store for managing "ready" and "pending" items.
+    store: S,
+
+    /// Sorted queue of "ready" items.
+    ready_queue: VecDeque<K>,
+}
+
+impl<K, S> PartialOrder<K, S>
+where
+    K: Clone + Copy + Display + StdHash + PartialEq + Eq,
+    S: PartialOrderStore<K>,
+{
+    pub fn new(store: S) -> Self {
+        Self {
+            store,
+            ready_queue: VecDeque::new(),
+        }
+    }
+
+    /// Pop the next item from the ready queue.
+    pub fn next(&mut self) -> Option<K> {
+        self.ready_queue.pop_front()
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: adzialocha
body: Is it possible to mark an item as "ready" _after_ it got popped from the queue?

I'm thinking about crash resiliance here as we might get intro trouble when items got "freed" by being marked as ready, _then_ moved into the ready queue. IF the process crashes we would loose them and that information as the ready queue only lives in memory.

I can see that it would be easy to just mark them as ready as soon as they get returned, but maybe that messes with the logic in other parts?
created_at: 2025-03-06T16:14:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,376 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::{collections::HashMap, marker::PhantomData};
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::{PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation DAG into a partial order. Here we have the addition of a `LogStore` and
+/// `OperationStore` implementation (traits from `p2panda-store`) and an operation cache which
+/// helps us avoid unnecessary calls to the database.
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    /// In memory cache of operations we have processed, before retrieving "ready" operations from
+    /// the store they will be taken from here if present. The store can be cleared at any time
+    /// in order to reduce memory use.
+    operation_cache: HashMap<Hash, Operation<E>>,
path: p2panda-stream/src/ordering/partial/operations.rs
user: login: adzialocha
body: A ring buffer would be cool here, with a configurable size on start up. I think `VecDeque` can be used as a ring buffer here.
created_at: 2025-03-06T16:17:51Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,376 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::{collections::HashMap, marker::PhantomData};
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::{PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation DAG into a partial order. Here we have the addition of a `LogStore` and
+/// `OperationStore` implementation (traits from `p2panda-store`) and an operation cache which
+/// helps us avoid unnecessary calls to the database.
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    /// In memory cache of operations we have processed, before retrieving "ready" operations from
+    /// the store they will be taken from here if present. The store can be cleared at any time
+    /// in order to reduce memory use.
+    operation_cache: HashMap<Hash, Operation<E>>,
+    _phantom: PhantomData<(L, E)>,
+}
+
+impl<L, E, OS, POS> PartialOrder<L, E, OS, POS>
+where
+    OS: OperationStore<L, E> + LogStore<L, E>,
+    POS: PartialOrderStore<Hash>,
+    E: Extensions,
+{
+    /// Construct a new dependency checker from an operation store and dependency store.
+    pub fn new(operation_store: OS, partial_order_store: POS) -> Self {
+        let inner_dependency_checker = InnerPartialOrder::new(partial_order_store);
+        PartialOrder {
+            operation_store,
+            inner: inner_dependency_checker,
+            operation_cache: Default::default(),
+            _phantom: PhantomData,
+        }
+    }
+
+    /// Process a single operation.
+    pub async fn process(
+        &mut self,
+        operation: Operation<E>,
+    ) -> Result<(), OperationDependencyCheckerError> {
+        let hash = operation.hash;
+        let previous = operation.header.previous.clone();
+        self.operation_cache.insert(operation.hash, operation);
+        self.inner.process(hash, previous).await?;
+        Ok(())
+    }
+
+    /// Take the next ready operation from the queue.
+    pub async fn next(&mut self) -> Result<Option<Operation<E>>, OperationDependencyCheckerError> {
+        let Some(hash) = self.inner.next() else {
+            return Ok(None);
+        };
+
+        if let Some(operation) = self.operation_cache.remove(&hash) {
+            return Ok(Some(operation));
+        }
+
+        if let Some((header, body)) = self
+            .operation_store
+            .get_operation(hash)
+            .await
+            .map_err(|err| OperationDependencyCheckerError::StoreError(err.to_string()))?
+        {
+            let operation = Operation {
+                hash: header.hash(),
path: p2panda-stream/src/ordering/partial/operations.rs
user: login: adzialocha
body: Is it possible to avoid the `hash()` operation here? It would be more costly than just using the already computed hash.
created_at: 2025-03-06T16:21:12Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use super::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
path: p2panda-stream/src/ordering/partial/store.rs
user: login: adzialocha
body: Is it possible to only keep track of "pending" items and as soon as they are _not_ in the list anymore they are considered "ready"? That would allow to not grow the size of storage, though it would also not be much more than the operation itself already.
created_at: 2025-03-06T16:26:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,376 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::{collections::HashMap, marker::PhantomData};
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::{PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation DAG into a partial order. Here we have the addition of a `LogStore` and
+/// `OperationStore` implementation (traits from `p2panda-store`) and an operation cache which
+/// helps us avoid unnecessary calls to the database.
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    /// In memory cache of operations we have processed, before retrieving "ready" operations from
+    /// the store they will be taken from here if present. The store can be cleared at any time
+    /// in order to reduce memory use.
+    operation_cache: HashMap<Hash, Operation<E>>,
+    _phantom: PhantomData<(L, E)>,
+}
+
+impl<L, E, OS, POS> PartialOrder<L, E, OS, POS>
+where
+    OS: OperationStore<L, E> + LogStore<L, E>,
+    POS: PartialOrderStore<Hash>,
+    E: Extensions,
+{
+    /// Construct a new dependency checker from an operation store and dependency store.
+    pub fn new(operation_store: OS, partial_order_store: POS) -> Self {
+        let inner_dependency_checker = InnerPartialOrder::new(partial_order_store);
+        PartialOrder {
+            operation_store,
+            inner: inner_dependency_checker,
+            operation_cache: Default::default(),
+            _phantom: PhantomData,
+        }
+    }
+
+    /// Process a single operation.
+    pub async fn process(
+        &mut self,
+        operation: Operation<E>,
+    ) -> Result<(), OperationDependencyCheckerError> {
+        let hash = operation.hash;
+        let previous = operation.header.previous.clone();
+        self.operation_cache.insert(operation.hash, operation);
+        self.inner.process(hash, previous).await?;
+        Ok(())
+    }
+
+    /// Take the next ready operation from the queue.
+    pub async fn next(&mut self) -> Result<Option<Operation<E>>, OperationDependencyCheckerError> {
+        let Some(hash) = self.inner.next() else {
+            return Ok(None);
+        };
+
+        if let Some(operation) = self.operation_cache.remove(&hash) {
+            return Ok(Some(operation));
+        }
+
+        if let Some((header, body)) = self
+            .operation_store
path: p2panda-stream/src/ordering/partial/operations.rs
user: login: adzialocha
body: Interesting that this assumes that the operations are already stored, maybe this needs to be mentioned somewhere?

If we ingest operations _after_ the ordering we might need to change that in the future.
created_at: 2025-03-06T16:28:01Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
+#[derive(Debug)]
+pub struct PartialOrder<K, S> {
+    /// Store for managing "ready" and "pending" items.
+    store: S,
+
+    /// Sorted queue of "ready" items.
+    ready_queue: VecDeque<K>,
+}
+
+impl<K, S> PartialOrder<K, S>
+where
+    K: Clone + Copy + Display + StdHash + PartialEq + Eq,
+    S: PartialOrderStore<K>,
+{
+    pub fn new(store: S) -> Self {
+        Self {
+            store,
+            ready_queue: VecDeque::new(),
+        }
+    }
+
+    /// Pop the next item from the ready queue.
+    pub fn next(&mut self) -> Option<K> {
+        self.ready_queue.pop_front()
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: adzialocha
body: In a "robust" implementation which would use this I can see how one would have a single SQL transaction for whatever the application does next and marking this as "ready", to not loose any data in that transition.
created_at: 2025-03-06T18:15:36Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use super::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
+/// - maintain a list of items which don't have their dependencies met
+/// - return all pending items which depend on a given item
+pub trait PartialOrderStore<K>
+where
+    K: Clone + Copy + StdHash + PartialEq + Eq,
+{
+    /// Add an item to the store which has all it's dependencies met already.
+    async fn add_ready(&mut self, key: K) -> Result<bool, PartialOrderError>;
path: p2panda-stream/src/ordering/partial/store.rs
user: login: sandreae
body: Yeh, I like that!
created_at: 2025-03-10T22:26:22Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
+#[derive(Debug)]
+pub struct PartialOrder<K, S> {
+    /// Store for managing "ready" and "pending" items.
+    store: S,
+
+    /// Sorted queue of "ready" items.
+    ready_queue: VecDeque<K>,
+}
+
+impl<K, S> PartialOrder<K, S>
+where
+    K: Clone + Copy + Display + StdHash + PartialEq + Eq,
+    S: PartialOrderStore<K>,
+{
+    pub fn new(store: S) -> Self {
+        Self {
+            store,
+            ready_queue: VecDeque::new(),
+        }
+    }
+
+    /// Pop the next item from the ready queue.
+    pub fn next(&mut self) -> Option<K> {
+        self.ready_queue.pop_front()
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: sandreae
body: Well spotted, thanks. I've moved the `ready_queue` into the store, and added a new method `take_next_ready` to the API. I quite like this, don't think it complicates the store, and it allows for adding items to both the ready set and ready queue in one transaction with say a SQL implementation.
created_at: 2025-03-10T22:46:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use super::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
path: p2panda-stream/src/ordering/partial/store.rs
user: login: sandreae
body: I considered this approach, as we discussed it before, however I can't see a way to make it work. If the "pending" items set is empty and we receive a new item with dependencies `[x, y]` we wouldn't know if these were items which have been processed and ready (therefore removed from the pending set), or that we have never seen (and therefore never added to the pending set).
created_at: 2025-03-10T22:53:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use super::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
path: p2panda-stream/src/ordering/partial/store.rs
user: login: sandreae
body: Maybe I'm misunderstanding your suggestion though! 
created_at: 2025-03-10T22:53:50Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,376 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::{collections::HashMap, marker::PhantomData};
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::{PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation DAG into a partial order. Here we have the addition of a `LogStore` and
+/// `OperationStore` implementation (traits from `p2panda-store`) and an operation cache which
+/// helps us avoid unnecessary calls to the database.
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    /// In memory cache of operations we have processed, before retrieving "ready" operations from
+    /// the store they will be taken from here if present. The store can be cleared at any time
+    /// in order to reduce memory use.
+    operation_cache: HashMap<Hash, Operation<E>>,
path: p2panda-stream/src/ordering/partial/operations.rs
user: login: sandreae
body: It's quite optimal that `HashMap` is fast for look-up, we'd lose this property with a `VecDeque`. A capacity to the cache would be nice, it's of course possible to have your own logic to decide when to call `clear_cache()` though. 
created_at: 2025-03-10T23:05:23Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,376 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::{collections::HashMap, marker::PhantomData};
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::{PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation DAG into a partial order. Here we have the addition of a `LogStore` and
+/// `OperationStore` implementation (traits from `p2panda-store`) and an operation cache which
+/// helps us avoid unnecessary calls to the database.
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    /// In memory cache of operations we have processed, before retrieving "ready" operations from
+    /// the store they will be taken from here if present. The store can be cleared at any time
+    /// in order to reduce memory use.
+    operation_cache: HashMap<Hash, Operation<E>>,
path: p2panda-stream/src/ordering/partial/operations.rs
user: login: adzialocha
body: >  it's of course possible to have your own logic to decide when to call clear_cache() though.

I think my question is just why and how will I ever want to wonder about such internals and based on what factors, I would need to start to keep a counter around myself etc. and maintain state just to use our API. A ring buffer or something similar does the work for you with a simple config value of the maximum size and avoids a pitfall where people will _not_ call this manual method and cause a memory leak.
created_at: 2025-03-11T10:20:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,376 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::{collections::HashMap, marker::PhantomData};
+
+use p2panda_core::{Extensions, Hash, Operation};
+use p2panda_store::{LogStore, OperationStore};
+use thiserror::Error;
+
+use crate::ordering::{PartialOrder as InnerPartialOrder, PartialOrderError, PartialOrderStore};
+
+/// Struct for processing p2panda operations into a partial order based on dependencies expressed
+/// in their `previous` field.
+///
+/// This struct is a thin wrapper around ordering::PartialOrder struct which takes care of sorting
+/// the operation DAG into a partial order. Here we have the addition of a `LogStore` and
+/// `OperationStore` implementation (traits from `p2panda-store`) and an operation cache which
+/// helps us avoid unnecessary calls to the database.
+pub struct PartialOrder<L, E, OS, POS> {
+    /// A store containing p2panda operations.
+    operation_store: OS,
+
+    /// The inner PartialOrder struct which sorts the operation DAG into a partial order.
+    inner: InnerPartialOrder<Hash, POS>,
+
+    /// In memory cache of operations we have processed, before retrieving "ready" operations from
+    /// the store they will be taken from here if present. The store can be cleared at any time
+    /// in order to reduce memory use.
+    operation_cache: HashMap<Hash, Operation<E>>,
path: p2panda-stream/src/ordering/partial/operations.rs
user: login: adzialocha
body: I'm sure there is ways to build a cache which is both performant to look up and has an automatic (configurable) limit
created_at: 2025-03-11T10:21:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+use std::collections::{HashMap, HashSet};
+use std::fmt::Debug;
+use std::hash::Hash as StdHash;
+
+use super::PartialOrderError;
+
+/// Trait defining a store API for handling ready and pending dependencies.
+///
+/// An implementation of this store trait provides the following functionality:
+/// - maintain a list of all items which have all their dependencies met
path: p2panda-stream/src/ordering/partial/store.rs
user: login: adzialocha
body: Ah, yes! Let's not do it then!
created_at: 2025-03-11T10:25:49Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+
+pub mod operations;
+pub mod store;
+
+use std::collections::VecDeque;
+use std::fmt::{Debug, Display};
+use std::hash::Hash as StdHash;
+
+use thiserror::Error;
+
+use store::PartialOrderStore;
+
+/// Error types which may be returned from `PartialOrder` methods.
+#[derive(Debug, Error)]
+pub enum PartialOrderError {
+    #[error("store error: {0}")]
+    StoreError(String),
+}
+
+/// Struct for establishing partial order over items in a Directed-Acyclic-Graph.
+///
+/// There are various approaches which can be taken when wanting to linearize items in a graph
+/// structure. This approach establishes a partial order, meaning not all items in the graph are
+/// comparable, and is non-deterministic. The main requirement is that all dependencies of an item
+/// are sorted "before" the item itself, the exact order is not a concern.
+///
+/// Items are considered in a "ready" state when all dependencies are met, and in a "pending"
+/// state when their dependencies are not yet met. Calls to `next` will return "ready" items in
+/// the order that they became ready.
+///
+/// If an item is processed an in a "pending" state then it is held in a pending queue and if it's
+/// dependencies are later processed and "ready", then the so far "pending" item will be moved to
+/// the "ready" queue. This processing of pending items recursively checks all pending dependents.
+///
+/// Example graph:
+///
+/// A <-- B2 <-- C
+///   \-- B1 <--/
+///
+/// Both of the following are possible and valid orderings for the above graph:
+///
+/// [A, B1, B2, C]
+/// [A, B2, B1, C]
+///
+/// Items will not be placed into an partial order until all their dependencies are met, in the
+/// following example item C will not be visited as we have not processed all of it's
+/// dependencies.
+///
+/// Example graph:
+///
+/// A <-- ?? <-- C
+///   \-- B1 <--/
+///
+/// C is not processed yet as we are missing one of its dependencies:
+///
+/// [A, B1]
+///
+/// Note that no checks are made for cycles occurring in the graph, this should be validated on
+/// another layer.
path: p2panda-stream/src/ordering/partial/mod.rs
user: login: sandreae
body: Thanks for the feedback! I've made some (hopefully) improvements to the docs, let me know what you think.
created_at: 2025-03-27T17:15:24Z
author_association: MEMBER

PR comments:
user: login: sandreae
created_at: 2025-03-03T15:17:59Z
body: > The stream module is for Stream implementations (not our namakemono specification "Stream" which probably will not be our future name for it anyhow), maybe you can think of another folder for that code?

I couldn't think of a great place for this code to live... moving it to `p2panda-core` makes the most sense but this also didn't feel quite right. Do you have any ideas @adzialocha ?
author_association: MEMBER
user: login: adzialocha
created_at: 2025-03-03T15:27:34Z
body: > I couldn't think of a great place for this code to live... moving it to `p2panda-core` makes the most sense but this also didn't feel quite right. Do you have any ideas @adzialocha ?

Haven't checked closely now what this code does, but from first look it just seems to wrap what you have in `partial/mod.rs`? Maybe move the code just into the same file?
author_association: MEMBER