Repository: ad-freiburg/qlever
Pull request #1847: Fix handling of unconnected `OPTIONAL`s
Author: login: RobinTF
Created: 2025-03-03T16:30:39Z | Merged: 2025-03-11T16:57:50Z

`OPTIONAL` clauses that either appear at the beginning of the query or that are unconnected to the part of the query that stands before them behave either like an "ordinary" group graph pattern (if their result is non-empty) or like the neutral element of join (a single row with no bound variables) otherwise. So far, QLever only implemented the first case, leading to wrong results for unconnected `OPTIONAL`s that have 0 results. This is now fixed.
Fixes #1761

Linked issue #1761: Issue with multiple OPTIONALs and  COALESCE
1. If I have a query with multiple optionals where one of them produces no bindings, then the whole query produce no bindings.

```
PREFIX Platform: <http://www.researchspace.org/resource/system/>
PREFIX crm: <http://www.cidoc-crm.org/cidoc-crm/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

SELECT * WHERE {
  OPTIONAL {
    <http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor> crm:P67_refers_to ?refersToResourceConfig_1.
  }
  OPTIONAL {
    <http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor> rdf:type Platform:resource_configuration.
    BIND(<http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor> AS ?refersToResourceConfig_2)
  }
  BIND(COALESCE(?refersToResourceConfig_1, ?refersToResourceConfig_2) AS ?resourceConfiguration)
}
```

query plan:

![Image](https://github.com/user-attachments/assets/0ae662eb-4b0c-463f-8868-b31d1a91c7c5)

If I remove first optional then the query actually produce results.

Can be reproduced with a single quad:

```
<http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.researchspace.org/resource/system/resource_configuration> <http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor> .
```

2. I guess related issue with even simpler query, reproducible on any data. I have optional that produce no bindings and then want to use COALESCE to bind some variable to some default value.

```
PREFIX crm: <http://www.cidoc-crm.org/cidoc-crm/>

SELECT * WHERE {
  OPTIONAL {
    <http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor> crm:P67_refers_to ?refersToResourceConfig_1.
  }
  BIND(COALESCE(?refersToResourceConfig_1, <http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor>) AS ?resourceConfiguration)
}
```

If OPTIONAL is empty I would expect `?resourceConfiguration` to be bound to `<http://www.researchspace.org/resource/system/resource_configurations_container/data/Actor>`.

![Image](https://github.com/user-attachments/assets/f43aade9-bcc2-41b1-b56e-0736a370439e)

Diff:
diff --git a/src/engine/CMakeLists.txt b/src/engine/CMakeLists.txt
index d1eba3a437..be690afb1a 100644
--- a/src/engine/CMakeLists.txt
+++ b/src/engine/CMakeLists.txt
@@ -15,5 +15,6 @@ add_library(engine
         TextLimit.cpp LazyGroupBy.cpp GroupByHashMapOptimization.cpp SpatialJoin.cpp
         CountConnectedSubgraphs.cpp SpatialJoinAlgorithms.cpp PathSearch.cpp ExecuteUpdate.cpp
         Describe.cpp GraphStoreProtocol.cpp
-        QueryExecutionContext.cpp ExistsJoin.cpp SPARQLProtocol.cpp ParsedRequestBuilder.cpp)
+        QueryExecutionContext.cpp ExistsJoin.cpp SPARQLProtocol.cpp ParsedRequestBuilder.cpp
+        NeutralOptional.cpp)
 qlever_target_link_libraries(engine util index parser sparqlExpressions http SortPerformanceEstimator Boost::iostreams s2)
diff --git a/src/engine/NeutralOptional.cpp b/src/engine/NeutralOptional.cpp
new file mode 100644
index 0000000000..2a78ae9723
--- /dev/null
+++ b/src/engine/NeutralOptional.cpp
@@ -0,0 +1,144 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate() * 2, 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t col) {
+  return tree_->getMultiplicity(col);
+}
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+// State machine that forwards a lazy result and ensures that at least one row
+// is returned when no limit is present.
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+bool NeutralOptional::singleRowCroppedByLimit() const {
+  const auto& limit = getLimit();
+  return limit._offset > 0 || limit.limitOrDefault() == 0;
+}
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  IdTable singleRowTable{getResultWidth(), allocator()};
+  singleRowTable.push_back(std::vector(getResultWidth(), Id::makeUndefined()));
+  if (childResult->isFullyMaterialized()) {
+    if (childResult->idTable().empty()) {
+      return {singleRowCroppedByLimit() ? IdTable{getResultWidth(), allocator()}
+                                        : std::move(singleRowTable),
+              resultSortedOn(),
+              {}};
+    }
+    return {childResult->idTable().clone(), childResult->sortedBy(),
+            childResult->getSharedLocalVocab()};
+  }
+  if (singleRowCroppedByLimit()) {
+    return {std::move(childResult->idTables()), childResult->sortedBy()};
+  }
+  return {Result::LazyResult{WrapperWithEnsuredRow{
+              std::move(childResult->idTables()), std::move(singleRowTable)}},
+          childResult->sortedBy()};
+}
+
+// _____________________________________________________________________________
+VariableToColumnMap NeutralOptional::computeVariableToColumnMap() const {
+  auto variableColumns = tree_->getVariableColumns();
+  // Because the optional operation might not return any rows, and we add
+  // placeholder undefined rows in that case, we need to mark all columns as
+  // possibly undefined.
+  for (ColumnIndexAndTypeInfo& info : variableColumns | ql::views::values) {
+    info.mightContainUndef_ = ColumnIndexAndTypeInfo::PossiblyUndefined;
+  }
+  return variableColumns;
+}
diff --git a/src/engine/NeutralOptional.h b/src/engine/NeutralOptional.h
new file mode 100644
index 0000000000..165f5e9e38
--- /dev/null
+++ b/src/engine/NeutralOptional.h
@@ -0,0 +1,41 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#pragma once
+
+#include "engine/Operation.h"
+
+// Implement the case where an `OPTIONAL` clause is joined with the empty
+// pattern. Conceptually this is the same as an optional join with the neutral
+// element, but specialized and more efficient.
+class NeutralOptional : public Operation {
+  std::shared_ptr<QueryExecutionTree> tree_;
+
+ public:
+  NeutralOptional(QueryExecutionContext* qec,
+                  std::shared_ptr<QueryExecutionTree> tree);
+
+ private:
+  std::string getCacheKeyImpl() const override;
+  uint64_t getSizeEstimateBeforeLimit() override;
+  std::unique_ptr<Operation> cloneImpl() const override;
+  ProtoResult computeResult(bool requestLaziness) override;
+  VariableToColumnMap computeVariableToColumnMap() const override;
+
+  // Return true, if `_limit` is configured in a way that will prevent the
+  // neutral element from ever appearing in the result.
+  bool singleRowCroppedByLimit() const;
+
+ public:
+  std::vector<QueryExecutionTree*> getChildren() override;
+  std::string getDescriptor() const override;
+  size_t getResultWidth() const override;
+  size_t getCostEstimate() override;
+  float getMultiplicity(size_t col) override;
+  bool knownEmptyResult() override;
+  bool supportsLimit() const override;
+
+ protected:
+  std::vector<ColumnIndex> resultSortedOn() const override;
+};
diff --git a/src/engine/QueryPlanner.cpp b/src/engine/QueryPlanner.cpp
index 4b7d0529a6..a652d046f5 100644
--- a/src/engine/QueryPlanner.cpp
+++ b/src/engine/QueryPlanner.cpp
@@ -29,6 +29,7 @@
 #include "engine/Minus.h"
 #include "engine/MultiColumnJoin.h"
 #include "engine/NeutralElementOperation.h"
+#include "engine/NeutralOptional.h"
 #include "engine/OptionalJoin.h"
 #include "engine/OrderBy.h"
 #include "engine/PathSearch.h"
@@ -82,6 +83,15 @@ void mergeSubtreePlanIds(SubtreePlan& target, const SubtreePlan& a,
   target.idsOfIncludedTextLimits_ =
       a.idsOfIncludedTextLimits_ | b.idsOfIncludedTextLimits_;
 }
+
+// Helper function that assigns the node, filter and text limit ids from
+// `source` to `target`.
+void assignNodesFilterAndTextLimitIds(QueryPlanner::SubtreePlan& target,
+                                      const QueryPlanner::SubtreePlan& source) {
+  target._idsOfIncludedNodes = source._idsOfIncludedNodes;
+  target._idsOfIncludedFilters = source._idsOfIncludedFilters;
+  target.idsOfIncludedTextLimits_ = source.idsOfIncludedTextLimits_;
+}
 }  // namespace
 
 // _____________________________________________________________________________
@@ -394,9 +404,7 @@ vector<SubtreePlan> QueryPlanner::getGroupByRow(
     // Create a group by operation to determine on which columns the input
     // needs to be sorted
     SubtreePlan groupByPlan(_qec);
-    groupByPlan._idsOfIncludedNodes = parent._idsOfIncludedNodes;
-    groupByPlan._idsOfIncludedFilters = parent._idsOfIncludedFilters;
-    groupByPlan.idsOfIncludedTextLimits_ = parent.idsOfIncludedTextLimits_;
+    assignNodesFilterAndTextLimitIds(groupByPlan, parent);
     std::vector<Alias> aliases;
     if (pq.hasSelectClause()) {
       aliases = pq.selectClause().getAliases();
@@ -428,9 +436,7 @@ vector<SubtreePlan> QueryPlanner::getOrderByRow(
   for (const auto& parent : previous) {
     SubtreePlan plan(_qec);
     auto& tree = plan._qet;
-    plan._idsOfIncludedNodes = parent._idsOfIncludedNodes;
-    plan._idsOfIncludedFilters = parent._idsOfIncludedFilters;
-    plan.idsOfIncludedTextLimits_ = parent.idsOfIncludedTextLimits_;
+    assignNodesFilterAndTextLimitIds(plan, parent);
     vector<pair<ColumnIndex, bool>> sortIndices;
     // Collect the variables of the ORDER BY or INTERNAL SORT BY clause. Ignore
     // variables that are not visible in the query body (according to the
@@ -1894,7 +1900,7 @@ size_t QueryPlanner::findSmallestExecutionTree(
     return tie(a) < tie(b);
   };
   return ql::ranges::min_element(lastRow, compare) - lastRow.begin();
-};
+}
 
 // _____________________________________________________________________________
 std::vector<SubtreePlan> QueryPlanner::createJoinCandidates(
@@ -2452,7 +2458,38 @@ void QueryPlanner::checkCancellation(
   cancellationHandle_->throwIfCancelled(location);
 }
 
-// _______________________________________________________________
+// _____________________________________________________________________________
+bool QueryPlanner::GraphPatternPlanner::handleUnconnectedMinusOrOptional(
+    std::vector<SubtreePlan>& candidates, const auto& variables) {
+  using enum SubtreePlan::Type;
+  bool areVariablesUnconnected = ql::ranges::all_of(
+      variables,
+      [this](const Variable& var) { return !boundVariables_.contains(var); });
+  if (!areVariablesUnconnected) {
+    return false;
+  }
+  // A MINUS clause that doesn't share any variable with the preceding
+  // patterns behaves as if it isn't there.
+  auto type = candidates[0].type;
+  if (type == MINUS) {
+    return true;
+  }
+  // An OPTIONAL clause that doesn't share any variable with the preceding
+  // patterns behaves as if it is joined with the neutral element.
+  if (type == OPTIONAL) {
+    auto& newPlans = candidatePlans_.emplace_back();
+    ql::ranges::for_each(
+        candidates, [this, &newPlans](const SubtreePlan& plan) {
+          auto joinedPlan = makeSubtreePlan<NeutralOptional>(qec_, plan._qet);
+          assignNodesFilterAndTextLimitIds(joinedPlan, plan);
+          newPlans.push_back(std::move(joinedPlan));
+        });
+    return true;
+  }
+  return false;
+}
+
+// _____________________________________________________________________________
 void QueryPlanner::GraphPatternPlanner::visitGroupOptionalOrMinus(
     std::vector<SubtreePlan>&& candidates) {
   // Empty group graph patterns should have been handled previously.
@@ -2462,24 +2499,8 @@ void QueryPlanner::GraphPatternPlanner::visitGroupOptionalOrMinus(
   // actually behave like ordinary (Group)GraphPatterns.
   auto variables = candidates[0]._qet->getVariableColumns() | ql::views::keys;
 
-  using enum SubtreePlan::Type;
-  if (auto type = candidates[0].type;
-      (type == OPTIONAL || type == MINUS) &&
-      ql::ranges::all_of(variables, [this](const Variable& var) {
-        return !boundVariables_.contains(var);
-      })) {
-    // A MINUS clause that doesn't share any variable with the preceding
-    // patterns behaves as if it isn't there.
-    if (type == MINUS) {
-      return;
-    }
-
-    // All variables in the OPTIONAL are unbound so far, so this OPTIONAL
-    // actually is not an OPTIONAL.
-    for (auto& vec : candidates) {
-      vec.type = SubtreePlan::BASIC;
-    }
-  }
+  bool specialCaseHandled =
+      handleUnconnectedMinusOrOptional(candidates, variables);
 
   // All variables seen so far are considered bound and cannot appear as the
   // RHS of a BIND operation. This is also true for variables from OPTIONALs
@@ -2487,6 +2508,10 @@ void QueryPlanner::GraphPatternPlanner::visitGroupOptionalOrMinus(
   ql::ranges::for_each(
       variables, [this](const Variable& var) { boundVariables_.insert(var); });
 
+  if (specialCaseHandled) {
+    return;
+  }
+
   // If our input is not OPTIONAL and not a MINUS, this means that we can still
   // arbitrarily optimize among our candidates and just append our new
   // candidates.
diff --git a/src/engine/QueryPlanner.h b/src/engine/QueryPlanner.h
index fbc7654e5d..1c72fee94b 100644
--- a/src/engine/QueryPlanner.h
+++ b/src/engine/QueryPlanner.h
@@ -576,6 +576,12 @@ class QueryPlanner {
     void visitSubquery(parsedQuery::Subquery& subquery);
     void visitDescribe(parsedQuery::Describe& describe);
 
+    // Helper function for `visitGroupOptionalOrMinus`. SPARQL queries like
+    // `SELECT * { OPTIONAL { ?a ?b ?c }}`, `SELECT * { MINUS { ?a ?b ?c }}` or
+    // `SELECT * { ?x ?y ?z . OPTIONAL { ?a ?b ?c }}` need special handling.
+    bool handleUnconnectedMinusOrOptional(std::vector<SubtreePlan>& candidates,
+                                          const auto& variables);
+
     // This function is called for groups, optional, or minus clauses.
     // The `candidates` are the result of planning the pattern inside the
     // braces. This leads to all of those clauses currently being an
diff --git a/test/QueryPlannerTest.cpp b/test/QueryPlannerTest.cpp
index b92e6466a7..aa487ccec0 100644
--- a/test/QueryPlannerTest.cpp
+++ b/test/QueryPlannerTest.cpp
@@ -3258,3 +3258,17 @@ TEST(QueryPlanner, postQueryValuesClause) {
       h::CartesianProductJoin(h::NeutralElement(),
                               h::ValuesClause("VALUES (?x\t?y) { (1 2) }")));
 }
+
+// _____________________________________________________________________________
+TEST(QueryPlanner, OptionalJoinWithEmptyPattern) {
+  h::expect("SELECT * { OPTIONAL { ?a ?b ?c } }",
+            h::NeutralOptional(h::IndexScanFromStrings("?a", "?b", "?c")));
+  h::expect("SELECT * { ?a ?b ?c . OPTIONAL { ?d ?e ?f } }",
+            h::CartesianProductJoin(
+                h::IndexScanFromStrings("?a", "?b", "?c"),
+                h::NeutralOptional(h::IndexScanFromStrings("?d", "?e", "?f"))));
+  h::expect("SELECT * { OPTIONAL { ?a ?b ?c } . OPTIONAL { ?d ?e ?f } }",
+            h::CartesianProductJoin(
+                h::NeutralOptional(h::IndexScanFromStrings("?a", "?b", "?c")),
+                h::NeutralOptional(h::IndexScanFromStrings("?d", "?e", "?f"))));
+}
diff --git a/test/QueryPlannerTestHelpers.h b/test/QueryPlannerTestHelpers.h
index 87842cc8ea..e334ba3d4e 100644
--- a/test/QueryPlannerTestHelpers.h
+++ b/test/QueryPlannerTestHelpers.h
@@ -23,6 +23,7 @@
 #include "engine/Minus.h"
 #include "engine/MultiColumnJoin.h"
 #include "engine/NeutralElementOperation.h"
+#include "engine/NeutralOptional.h"
 #include "engine/OptionalJoin.h"
 #include "engine/OrderBy.h"
 #include "engine/PathSearch.h"
@@ -262,6 +263,8 @@ inline auto Join = MatchTypeAndUnorderedChildren<::Join>;
 
 constexpr auto OptionalJoin = MatchTypeAndOrderedChildren<::OptionalJoin>;
 
+constexpr auto NeutralOptional = MatchTypeAndOrderedChildren<::NeutralOptional>;
+
 constexpr auto Minus = MatchTypeAndOrderedChildren<::Minus>;
 
 // Return a matcher that matches a query execution tree that consists of
diff --git a/test/engine/CMakeLists.txt b/test/engine/CMakeLists.txt
index 41b2b463ad..9f8e05732b 100644
--- a/test/engine/CMakeLists.txt
+++ b/test/engine/CMakeLists.txt
@@ -13,3 +13,4 @@ addLinkAndRunAsSingleTest(SpatialJoinAlgorithmsTest engine)
 addLinkAndDiscoverTestSerial(QueryExecutionTreeTest engine)
 addLinkAndDiscoverTestSerial(DescribeTest engine)
 addLinkAndDiscoverTestSerial(ExistsJoinTest engine)
+addLinkAndDiscoverTest(NeutralOptionalTest engine)
diff --git a/test/engine/NeutralOptionalTest.cpp b/test/engine/NeutralOptionalTest.cpp
new file mode 100644
index 0000000000..d2d1deb16a
--- /dev/null
+++ b/test/engine/NeutralOptionalTest.cpp
@@ -0,0 +1,382 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include <gmock/gmock.h>
+
+#include "../util/IdTableHelpers.h"
+#include "../util/IndexTestHelpers.h"
+#include "../util/OperationTestHelpers.h"
+#include "./ValuesForTesting.h"
+#include "engine/NeutralOptional.h"
+#include "engine/QueryExecutionTree.h"
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getChildren) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, child};
+  EXPECT_THAT(no.getChildren(), ::testing::ElementsAre(child.get()));
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getCacheKey) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, child};
+  EXPECT_THAT(no.getCacheKey(),
+              ::testing::AllOf(::testing::StartsWith("NeutralOptional#"),
+                               ::testing::EndsWith(child->getCacheKey())));
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getDescriptor) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, std::move(child)};
+  EXPECT_THAT(no.getDescriptor(), ::testing::StartsWith("Optional"));
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getResultWidth) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{5, qec->getAllocator()},
+      std::vector<std::optional<Variable>>(5, std::nullopt));
+  NeutralOptional no{qec, std::move(child)};
+  EXPECT_EQ(no.getResultWidth(), 5);
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getCostEstimate) {
+  auto* qec = ad_utility::testing::getQec();
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{0, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{});
+    NeutralOptional no{qec, std::move(child)};
+    EXPECT_EQ(no.getCostEstimate(), 1);
+  }
+  {
+    IdTable idTable{0, qec->getAllocator()};
+    idTable.resize(42);
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, std::move(idTable), std::vector<std::optional<Variable>>{});
+    NeutralOptional no{qec, std::move(child)};
+    EXPECT_EQ(no.getCostEstimate(), 42 * 2);
+  }
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getSizeEstimate) {
+  auto* qec = ad_utility::testing::getQec();
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{0, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{});
+    NeutralOptional no{qec, std::move(child)};
+    EXPECT_EQ(no.getSizeEstimate(), 1);
+  }
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{0, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{});
+    NeutralOptional no{qec, std::move(child)};
+    no.setLimit({0, 0});
+    EXPECT_EQ(no.getSizeEstimate(), 0);
+  }
+  {
+    IdTable idTable{0, qec->getAllocator()};
+    idTable.resize(42);
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, std::move(idTable), std::vector<std::optional<Variable>>{});
+    NeutralOptional no{qec, std::move(child)};
+    no.setLimit({40, 1});
+    EXPECT_EQ(no.getSizeEstimate(), 40);
+  }
+  {
+    IdTable idTable{0, qec->getAllocator()};
+    idTable.resize(42);
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, std::move(idTable), std::vector<std::optional<Variable>>{});
+    NeutralOptional no{qec, std::move(child)};
+    EXPECT_EQ(no.getSizeEstimate(), 42);
+  }
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getMultiplicity) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{3, qec->getAllocator()},
+      std::vector<std::optional<Variable>>(3, std::nullopt));
+  NeutralOptional no{qec, child};
+  EXPECT_EQ(no.getMultiplicity(0), child->getMultiplicity(0));
+  EXPECT_EQ(no.getMultiplicity(1), child->getMultiplicity(1));
+  EXPECT_EQ(no.getMultiplicity(2), child->getMultiplicity(2));
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, knownEmptyResult) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, std::move(child)};
+  EXPECT_FALSE(no.knownEmptyResult());
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, supportsLimit) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, std::move(child)};
+  EXPECT_TRUE(no.supportsLimit());
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, clone) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, std::move(child)};
+  auto clone = no.clone();
+  ASSERT_TRUE(clone);
+  EXPECT_THAT(no, IsDeepCopy(*clone));
+  EXPECT_EQ(clone->getDescriptor(), no.getDescriptor());
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getResultSortedOn) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{3, qec->getAllocator()},
+      std::vector<std::optional<Variable>>(3, std::nullopt), false,
+      std::vector<ColumnIndex>{1, 0, 2});
+  NeutralOptional no{qec, child};
+  EXPECT_EQ(no.getResultSortedOn(), child->resultSortedOn());
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getExternallyVisibleVariableColumns) {
+  using namespace ::testing;
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{2, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{Variable{"?a"}, Variable{"?b"}});
+  NeutralOptional no{qec, std::move(child)};
+  EXPECT_THAT(no.getExternallyVisibleVariableColumns(),
+              UnorderedElementsAre(
+                  Pair(Variable{"?a"},
+                       ColumnIndexAndTypeInfo{
+                           0, ColumnIndexAndTypeInfo::PossiblyUndefined}),
+                  Pair(Variable{"?b"},
+                       ColumnIndexAndTypeInfo{
+                           1, ColumnIndexAndTypeInfo::PossiblyUndefined})));
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, ensureEmptyResultWhenLimitCutsOffEverything) {
+  auto* qec = ad_utility::testing::getQec();
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{1, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{std::nullopt});
+    NeutralOptional no{qec, std::move(child)};
+    no.setLimit({std::nullopt, 1});
+
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(false);
+    EXPECT_TRUE(result.idTable().empty());
+    EXPECT_TRUE(result.localVocab().empty());
+  }
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{1, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{std::nullopt});
+    NeutralOptional no{qec, std::move(child)};
+    no.setLimit({std::nullopt, 1});
+
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(true);
+    auto& idTables = result.idTables();
+    EXPECT_EQ(idTables.begin(), idTables.end());
+  }
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{1, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{std::nullopt});
+    NeutralOptional no{qec, std::move(child)};
+    no.setLimit({0});
+
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(false);
+    EXPECT_TRUE(result.idTable().empty());
+    EXPECT_TRUE(result.localVocab().empty());
+  }
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, IdTable{1, qec->getAllocator()},
+        std::vector<std::optional<Variable>>{std::nullopt});
+    NeutralOptional no{qec, std::move(child)};
+    no.setLimit({0});
+
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(true);
+    auto& idTables = result.idTables();
+    EXPECT_EQ(idTables.begin(), idTables.end());
+  }
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, ensureSingleRowWhenChildIsEmpty) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{1, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{std::nullopt});
+  NeutralOptional no{qec, child};
+
+  {
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(false);
+    EXPECT_EQ(result.idTable(), makeIdTableFromVector({{Id::makeUndefined()}}));
+    EXPECT_TRUE(result.localVocab().empty());
+  }
+
+  {
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(true);
+    auto& idTables = result.idTables();
+
+    auto it = idTables.begin();
+    ASSERT_NE(it, idTables.end());
+    // First table is empty from `ValuesForTesting`.
+    EXPECT_TRUE(it->idTable_.empty());
+    EXPECT_TRUE(it->localVocab_.empty());
+
+    ++it;
+    ASSERT_NE(it, idTables.end());
+    EXPECT_EQ(it->idTable_, makeIdTableFromVector({{Id::makeUndefined()}}));
+    EXPECT_TRUE(it->localVocab_.empty());
+
+    EXPECT_EQ(++it, idTables.end());
+  }
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, ensureResultIsProperlyPropagated) {
+  auto* qec = ad_utility::testing::getQec();
+  LocalVocab localVocab{};
+  localVocab.getIndexAndAddIfNotContained(LocalVocabEntry{
+      ad_utility::triple_component::Literal::fromStringRepresentation(
+          "\"Test\"")});
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, makeIdTableFromVector({{1}, {2}, {3}}),
+        std::vector<std::optional<Variable>>{std::nullopt}, false,
+        std::vector<ColumnIndex>{}, localVocab.clone());
+    NeutralOptional no{qec, child};
+
+    {
+      qec->getQueryTreeCache().clearAll();
+      auto result = no.computeResultOnlyForTesting(false);
+      EXPECT_EQ(result.idTable(), makeIdTableFromVector({{1}, {2}, {3}}));
+      EXPECT_EQ(result.localVocab().getAllWordsForTesting(),
+                localVocab.getAllWordsForTesting());
+    }
+
+    {
+      qec->getQueryTreeCache().clearAll();
+      auto result = no.computeResultOnlyForTesting(true);
+      auto& idTables = result.idTables();
+
+      auto it = idTables.begin();
+      ASSERT_NE(it, idTables.end());
+      EXPECT_EQ(it->idTable_, makeIdTableFromVector({{1}, {2}, {3}}));
+      EXPECT_EQ(it->localVocab_.getAllWordsForTesting(),
+                localVocab.getAllWordsForTesting());
+
+      EXPECT_EQ(++it, idTables.end());
+    }
+  }
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, makeIdTableFromVector({{1}, {2}, {3}}),
+        std::vector<std::optional<Variable>>{std::nullopt}, false,
+        std::vector<ColumnIndex>{}, localVocab.clone());
+    NeutralOptional no{qec, child};
+    no.setLimit({std::nullopt, 1});
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(false);
+    EXPECT_EQ(result.idTable(), makeIdTableFromVector({{2}, {3}}));
+    EXPECT_EQ(result.localVocab().getAllWordsForTesting(),
+              localVocab.getAllWordsForTesting());
+  }
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, makeIdTableFromVector({{1}, {2}, {3}}),
+        std::vector<std::optional<Variable>>{std::nullopt}, false,
+        std::vector<ColumnIndex>{}, localVocab.clone());
+    NeutralOptional no{qec, child};
+    no.setLimit({std::nullopt, 1});
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(true);
+    auto& idTables = result.idTables();
+
+    auto it = idTables.begin();
+    ASSERT_NE(it, idTables.end());
+    EXPECT_EQ(it->idTable_, makeIdTableFromVector({{2}, {3}}));
+    EXPECT_EQ(it->localVocab_.getAllWordsForTesting(),
+              localVocab.getAllWordsForTesting());
+
+    EXPECT_EQ(++it, idTables.end());
+  }
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, makeIdTableFromVector({{1}, {2}, {3}}),
+        std::vector<std::optional<Variable>>{std::nullopt}, false,
+        std::vector<ColumnIndex>{}, localVocab.clone());
+    NeutralOptional no{qec, child};
+    no.setLimit({2});
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(false);
+    EXPECT_EQ(result.idTable(), makeIdTableFromVector({{1}, {2}}));
+    EXPECT_EQ(result.localVocab().getAllWordsForTesting(),
+              localVocab.getAllWordsForTesting());
+  }
+
+  {
+    auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+        qec, makeIdTableFromVector({{1}, {2}, {3}}),
+        std::vector<std::optional<Variable>>{std::nullopt}, false,
+        std::vector<ColumnIndex>{}, localVocab.clone());
+    NeutralOptional no{qec, child};
+    no.setLimit({2});
+    qec->getQueryTreeCache().clearAll();
+    auto result = no.computeResultOnlyForTesting(true);
+    auto& idTables = result.idTables();
+
+    auto it = idTables.begin();
+    ASSERT_NE(it, idTables.end());
+    EXPECT_EQ(it->idTable_, makeIdTableFromVector({{1}, {2}}));
+    EXPECT_EQ(it->localVocab_.getAllWordsForTesting(),
+              localVocab.getAllWordsForTesting());
+
+    EXPECT_EQ(++it, idTables.end());
+  }
+}
diff --git a/test/util/OperationTestHelpers.h b/test/util/OperationTestHelpers.h
index f0832ed6cc..7001043b6d 100644
--- a/test/util/OperationTestHelpers.h
+++ b/test/util/OperationTestHelpers.h
@@ -8,6 +8,7 @@
 #include <chrono>
 #include <vector>
 
+#include "./GTestHelpers.h"
 #include "engine/Operation.h"
 #include "engine/QueryExecutionTree.h"
 


Reviews:
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-03T16:31:07Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-03T16:31:26Z
user: login: joka921
body: Thank you very much, 
I have some questions and suggestions.
Please start commenting your code (docstrings + non-obvious code parts)
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-05T09:13:52Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T10:16:51Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T10:18:00Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T10:18:50Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T10:19:55Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T10:20:53Z
user: login: RobinTF
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T10:22:13Z
user: login: joka921
body: Thank you very much.
There is some room for improvements
(don't copy the materialized result, leverage the fact that the columns are often "either defined or only contain a single element".
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-11T08:18:03Z
user: login: joka921
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-11T16:54:07Z

Review comments:
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: Not sure if this is the right thing to do here.
created_at: 2025-03-03T16:31:07Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: Not sure if this is correct here.
created_at: 2025-03-03T16:31:25Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: I don't understand the comment about the cloning etc.
There also is nothing to clone here? 
And like this, the operation is free, as the cost estimate always has to include the costs for the children.
created_at: 2025-03-05T08:38:08Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: Are you sure that not just `tree_->getSizeEstimate()`is correct?
If the subtree already has a limit, then it is the limit of a subquery, which is correct to return here.
created_at: 2025-03-05T08:39:35Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: You can just propagate it to the subtree so `return tree_->getMultiplicty(i)`.
created_at: 2025-03-05T08:40:18Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: I am not sure about the current semantics of this function, and whether we have to do something special if we say `true` here, can you please check and elaborate.
created_at: 2025-03-05T08:42:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: Not sure if this shouldn't be propagated earlier etc, but I haven't touched this mechanism in a while.
created_at: 2025-03-05T08:43:27Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
+
+  IdTable singleRowTable{getResultWidth(), allocator()};
+  singleRowTable.push_back(std::vector(getResultWidth(), Id::makeUndefined()));
+  if (childResult->isFullyMaterialized()) {
+    if (childResult->idTable().empty()) {
+      return {singleRowCroppedByLimit ? IdTable{getResultWidth(), allocator()}
+                                      : std::move(singleRowTable),
+              resultSortedOn(),
+              {}};
+    }
+    return {childResult->idTable().clone(), childResult->sortedBy(),
+            childResult->getSharedLocalVocab()};
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: ```suggestion
    return childResult;
```
You haven't done anything to the result, so no need for a deep clone here.
created_at: 2025-03-05T08:45:51Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: I am beginning to understand the limit business, can you please confirm:
The critical case is, if the result is empty, so we would have to return a neutral element, but limit and offset forbid this.
created_at: 2025-03-05T08:49:55Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
+
+  IdTable singleRowTable{getResultWidth(), allocator()};
+  singleRowTable.push_back(std::vector(getResultWidth(), Id::makeUndefined()));
+  if (childResult->isFullyMaterialized()) {
+    if (childResult->idTable().empty()) {
+      return {singleRowCroppedByLimit ? IdTable{getResultWidth(), allocator()}
+                                      : std::move(singleRowTable),
+              resultSortedOn(),
+              {}};
+    }
+    return {childResult->idTable().clone(), childResult->sortedBy(),
+            childResult->getSharedLocalVocab()};
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: You can even do the same thing for `singleRowCropped && empty anyway`,
then the code gets even simpler.
created_at: 2025-03-05T08:51:26Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
+
+  IdTable singleRowTable{getResultWidth(), allocator()};
+  singleRowTable.push_back(std::vector(getResultWidth(), Id::makeUndefined()));
+  if (childResult->isFullyMaterialized()) {
+    if (childResult->idTable().empty()) {
+      return {singleRowCroppedByLimit ? IdTable{getResultWidth(), allocator()}
+                                      : std::move(singleRowTable),
+              resultSortedOn(),
+              {}};
+    }
+    return {childResult->idTable().clone(), childResult->sortedBy(),
+            childResult->getSharedLocalVocab()};
+  }
+  if (singleRowCroppedByLimit) {
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: write a comment for the `singelRowCropped... business`.
created_at: 2025-03-05T08:52:22Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
+
+  IdTable singleRowTable{getResultWidth(), allocator()};
+  singleRowTable.push_back(std::vector(getResultWidth(), Id::makeUndefined()));
+  if (childResult->isFullyMaterialized()) {
+    if (childResult->idTable().empty()) {
+      return {singleRowCroppedByLimit ? IdTable{getResultWidth(), allocator()}
+                                      : std::move(singleRowTable),
+              resultSortedOn(),
+              {}};
+    }
+    return {childResult->idTable().clone(), childResult->sortedBy(),
+            childResult->getSharedLocalVocab()};
+  }
+  if (singleRowCroppedByLimit) {
+    return {std::move(childResult->idTables()), childResult->sortedBy()};
+  }
+  return {Result::LazyResult{WrapperWithEnsuredRow{
+              std::move(childResult->idTables()), std::move(singleRowTable)}},
+          childResult->sortedBy()};
+}
+
+// _____________________________________________________________________________
+VariableToColumnMap NeutralOptional::computeVariableToColumnMap() const {
+  auto variableColumns = tree_->getVariableColumns();
+  for (ColumnIndexAndTypeInfo& info : variableColumns | ql::views::values) {
+    info.mightContainUndef_ = ColumnIndexAndTypeInfo::PossiblyUndefined;
+  }
+  return variableColumns;
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: Add a comment that this typically is somewhat a pessimization and we would actually need something like
`only undef if the whole thing is the neutral element`.
created_at: 2025-03-05T08:54:02Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: Comment!
created_at: 2025-03-05T08:54:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
path: src/engine/NeutralOptional.cpp
user: login: joka921
body: Ah, are the semantics:
`if true, then I promise to already have applied the limit as soon as computeResult returns`,
and that this can always be enforced by pushing the limit to the child, as then at least the generic mechanism in Operation.cpp kicks in?
created_at: 2025-03-05T09:04:45Z
author_association: MEMBER
diff_hunk: @@ -2292,7 +2293,38 @@ void QueryPlanner::checkCancellation(
   cancellationHandle_->throwIfCancelled(location);
 }
 
-// _______________________________________________________________
+// _____________________________________________________________________________
+bool QueryPlanner::GraphPatternPlanner::handleUnconnectedMinusOrOptional(
+    std::vector<SubtreePlan>& candidates, const auto& variables) {
+  using enum SubtreePlan::Type;
+  if (ql::ranges::all_of(variables, [this](const Variable& var) {
+        return !boundVariables_.contains(var);
+      })) {
path: src/engine/QueryPlanner.cpp
user: login: joka921
body: Check for the inverse condition, then return false immediately, less nesting.
created_at: 2025-03-05T09:06:32Z
author_association: MEMBER
diff_hunk: @@ -2292,7 +2293,38 @@ void QueryPlanner::checkCancellation(
   cancellationHandle_->throwIfCancelled(location);
 }
 
-// _______________________________________________________________
+// _____________________________________________________________________________
+bool QueryPlanner::GraphPatternPlanner::handleUnconnectedMinusOrOptional(
+    std::vector<SubtreePlan>& candidates, const auto& variables) {
+  using enum SubtreePlan::Type;
+  if (ql::ranges::all_of(variables, [this](const Variable& var) {
+        return !boundVariables_.contains(var);
+      })) {
+    // A MINUS clause that doesn't share any variable with the preceding
+    // patterns behaves as if it isn't there.
+    auto type = candidates[0].type;
+    if (type == MINUS) {
+      return true;
+    }
+    // An OPTIONAL clause that doesn't share any variable with the preceding
+    // patterns behaves as if it is joined with the neutral element.
+    if (type == OPTIONAL) {
+      auto& newPlans = candidatePlans_.emplace_back();
+      ql::ranges::for_each(
+          candidates, [this, &newPlans](const SubtreePlan& plan) {
+            auto joinedPlan = makeSubtreePlan<NeutralOptional>(qec_, plan._qet);
+            joinedPlan._idsOfIncludedFilters = plan._idsOfIncludedFilters;
+            joinedPlan._idsOfIncludedNodes = plan._idsOfIncludedNodes;
+            joinedPlan.idsOfIncludedTextLimits_ = plan.idsOfIncludedTextLimits_;
path: src/engine/QueryPlanner.cpp
user: login: joka921
body: Isn't there a helper function that already does this?
created_at: 2025-03-05T09:07:46Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,380 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include <gmock/gmock.h>
+
+#include "../util/IdTableHelpers.h"
+#include "../util/IndexTestHelpers.h"
+#include "../util/OperationTestHelpers.h"
+#include "./ValuesForTesting.h"
+#include "engine/NeutralOptional.h"
+#include "engine/QueryExecutionTree.h"
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getChildren) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, child};
+  EXPECT_THAT(no.getChildren(), ::testing::ElementsAre(child.get()));
+}
+
+// _____________________________________________________________________________
+TEST(NeutralOptional, getCacheKey) {
+  auto* qec = ad_utility::testing::getQec();
+  auto child = ad_utility::makeExecutionTree<ValuesForTesting>(
+      qec, IdTable{0, qec->getAllocator()},
+      std::vector<std::optional<Variable>>{});
+  NeutralOptional no{qec, std::move(child)};
+  EXPECT_THAT(no.getCacheKey(), ::testing::StartsWith("NeutralOptional#"));
path: test/engine/NeutralOptionalTest.cpp
user: login: joka921
body: Also test that it contains the child's cache key.
created_at: 2025-03-05T09:09:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: This is how it's done in cartesian product join and bind
created_at: 2025-03-05T10:16:51Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: Yes, this enables us to clone fewer elements
created_at: 2025-03-05T10:18:00Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: I'm not sure about the semantics of this function when supports limit is true. I have to check how other functions do it.
created_at: 2025-03-05T10:18:50Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
+
+  IdTable singleRowTable{getResultWidth(), allocator()};
+  singleRowTable.push_back(std::vector(getResultWidth(), Id::makeUndefined()));
+  if (childResult->isFullyMaterialized()) {
+    if (childResult->idTable().empty()) {
+      return {singleRowCroppedByLimit ? IdTable{getResultWidth(), allocator()}
+                                      : std::move(singleRowTable),
+              resultSortedOn(),
+              {}};
+    }
+    return {childResult->idTable().clone(), childResult->sortedBy(),
+            childResult->getSharedLocalVocab()};
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: `childResult` is of type `std::shared_ptr<const Result>`, the function signature mandates a plain `Result`, so we need to copy.
created_at: 2025-03-05T10:19:55Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,138 @@
+//   Copyright 2025, University of Freiburg,
+//   Chair of Algorithms and Data Structures.
+//   Author: Robin Textor-Falconi <textorr@informatik.uni-freiburg.de>
+
+#include "engine/NeutralOptional.h"
+
+// _____________________________________________________________________________
+NeutralOptional::NeutralOptional(QueryExecutionContext* qec,
+                                 std::shared_ptr<QueryExecutionTree> tree)
+    : Operation{qec}, tree_{std::move(tree)} {}
+
+// _____________________________________________________________________________
+std::vector<QueryExecutionTree*> NeutralOptional::getChildren() {
+  return {tree_.get()};
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getCacheKeyImpl() const {
+  return absl::StrCat("NeutralOptional#", tree_->getCacheKey());
+}
+
+// _____________________________________________________________________________
+std::string NeutralOptional::getDescriptor() const { return "Optional"; }
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getResultWidth() const {
+  return tree_->getResultWidth();
+}
+
+// _____________________________________________________________________________
+size_t NeutralOptional::getCostEstimate() {
+  // Cloning is expensive, so we estimate the cost as the cost of the child.
+  // Does not apply to the lazy case.
+  return std::max<size_t>(tree_->getCostEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+uint64_t NeutralOptional::getSizeEstimateBeforeLimit() {
+  auto originalLimit = tree_->getRootOperation()->getLimit();
+  tree_->setLimit({});
+  absl::Cleanup resetLimit{
+      [this, &originalLimit]() { tree_->setLimit(originalLimit); }};
+  return std::max<size_t>(tree_->getSizeEstimate(), 1);
+}
+
+// _____________________________________________________________________________
+float NeutralOptional::getMultiplicity(size_t) { return 1; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::knownEmptyResult() { return false; }
+
+// _____________________________________________________________________________
+bool NeutralOptional::supportsLimit() const { return true; }
+
+// _____________________________________________________________________________
+std::unique_ptr<Operation> NeutralOptional::cloneImpl() const {
+  return std::make_unique<NeutralOptional>(getExecutionContext(),
+                                           tree_->clone());
+}
+
+// _____________________________________________________________________________
+std::vector<ColumnIndex> NeutralOptional::resultSortedOn() const {
+  return tree_->resultSortedOn();
+}
+
+namespace {
+struct WrapperWithEnsuredRow
+    : ad_utility::InputRangeFromGet<Result::IdTableVocabPair> {
+  Result::LazyResult originalRange_;
+  IdTable rowFallback_;
+  std::optional<Result::LazyResult::iterator> iterator_ = std::nullopt;
+  bool hasRows_ = false;
+  bool done_ = false;
+
+  explicit WrapperWithEnsuredRow(Result::LazyResult originalRange,
+                                 IdTable rowFallback)
+      : originalRange_{std::move(originalRange)},
+        rowFallback_{std::move(rowFallback)} {}
+
+  std::optional<Result::IdTableVocabPair> get() override {
+    if (done_) {
+      return std::nullopt;
+    }
+    if (iterator_) {
+      ++iterator_.value();
+    } else {
+      iterator_ = originalRange_.begin();
+    }
+    if (*iterator_ == originalRange_.end()) {
+      done_ = true;
+      return hasRows_ ? std::nullopt
+                      : std::optional{Result::IdTableVocabPair{
+                            std::move(rowFallback_), {}}};
+    }
+    hasRows_ |= !iterator_.value()->idTable_.empty();
+    return std::move(*iterator_.value());
+  }
+};
+}  // namespace
+
+// _____________________________________________________________________________
+ProtoResult NeutralOptional::computeResult(bool requestLaziness) {
+  const auto& limit = getLimit();
+  tree_->setLimit(limit);
+
+  auto childResult = tree_->getResult(requestLaziness);
+
+  bool singleRowCroppedByLimit =
+      limit._offset > 0 || limit.limitOrDefault() == 0;
path: src/engine/NeutralOptional.cpp
user: login: RobinTF
body: Yes
created_at: 2025-03-05T10:20:53Z
author_association: COLLABORATOR
diff_hunk: @@ -2292,7 +2293,38 @@ void QueryPlanner::checkCancellation(
   cancellationHandle_->throwIfCancelled(location);
 }
 
-// _______________________________________________________________
+// _____________________________________________________________________________
+bool QueryPlanner::GraphPatternPlanner::handleUnconnectedMinusOrOptional(
+    std::vector<SubtreePlan>& candidates, const auto& variables) {
+  using enum SubtreePlan::Type;
+  if (ql::ranges::all_of(variables, [this](const Variable& var) {
+        return !boundVariables_.contains(var);
+      })) {
+    // A MINUS clause that doesn't share any variable with the preceding
+    // patterns behaves as if it isn't there.
+    auto type = candidates[0].type;
+    if (type == MINUS) {
+      return true;
+    }
+    // An OPTIONAL clause that doesn't share any variable with the preceding
+    // patterns behaves as if it is joined with the neutral element.
+    if (type == OPTIONAL) {
+      auto& newPlans = candidatePlans_.emplace_back();
+      ql::ranges::for_each(
+          candidates, [this, &newPlans](const SubtreePlan& plan) {
+            auto joinedPlan = makeSubtreePlan<NeutralOptional>(qec_, plan._qet);
+            joinedPlan._idsOfIncludedFilters = plan._idsOfIncludedFilters;
+            joinedPlan._idsOfIncludedNodes = plan._idsOfIncludedNodes;
+            joinedPlan.idsOfIncludedTextLimits_ = plan.idsOfIncludedTextLimits_;
path: src/engine/QueryPlanner.cpp
user: login: RobinTF
body: I don't think so, but it would be neat
created_at: 2025-03-05T10:22:13Z
author_association: COLLABORATOR

PR comments:
user: login: codecov[bot]
created_at: 2025-03-03T17:09:51Z
body: ## [Codecov](https://app.codecov.io/gh/ad-freiburg/qlever/pull/1847?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ad-freiburg) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 90.24%. Comparing base [(`0daadaa`)](https://app.codecov.io/gh/ad-freiburg/qlever/commit/0daadaa3169e3ba50a4bb84f06f745e3210519d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ad-freiburg) to head [(`1470743`)](https://app.codecov.io/gh/ad-freiburg/qlever/commit/1470743a671838c9073df4fd383b706f2719f142?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ad-freiburg).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1847      +/-   ##
==========================================
+ Coverage   90.20%   90.24%   +0.03%     
==========================================
  Files         400      401       +1     
  Lines       38514    38616     +102     
  Branches     4319     4330      +11     
==========================================
+ Hits        34743    34850     +107     
+ Misses       2477     2473       -4     
+ Partials     1294     1293       -1     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/ad-freiburg/qlever/pull/1847?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ad-freiburg).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ad-freiburg).

author_association: NONE
user: login: sonarqubecloud[bot]
created_at: 2025-03-05T16:22:59Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=ad-freiburg_qlever&pullRequest=1847) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [2 New issues](https://sonarcloud.io/project/issues?id=ad-freiburg_qlever&pullRequest=1847&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=ad-freiburg_qlever&pullRequest=1847&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=ad-freiburg_qlever&pullRequest=1847&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=ad-freiburg_qlever&pullRequest=1847&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=ad-freiburg_qlever&pullRequest=1847&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=ad-freiburg_qlever&pullRequest=1847)


author_association: NONE
user: login: sparql-conformance[bot]
created_at: 2025-03-11T15:42:27Z
body: 
### Conformance check passed ✅


 No test result changes.

Details: https://qlever.cs.uni-freiburg.de/sparql-conformance-ui?cur=de41fcbd11cc1862b19e7b1fc928ce10f239fdad&prev=79793de2bf426adb6fff95d5ada5ce9007d818ac
author_association: NONE