Repository: github/codeql
Pull request #19477: C++: Fix infinite range analysis loop on invalid SSA
Author: login: MathiasVP
Labels: name: C++
name: documentation
Created: 2025-05-12T19:02:18Z | Merged: 2025-05-13T15:59:11Z

As part of the IR SSA/alias analysis we compute the which memory allocations overlap. In https://github.com/github/codeql/pull/17062 we mitigated a problem where, in very large function bodies, the set of overlapping memory allocations was basically a cartesian product.

As a result of this, we don't always have proper SSA information on these large function bodies. It's well-known that even one missing SSA edge can cause the shared range analysis to loop infinitely.

This PR excludes the functions with incomplete SSA information from the new range analysis. Ideally, I would like us to move away from using the IR SSA/alias analysis in the new range analysis library, but that seemed like too big of a task to start right now.

On a customer codebase there were 3 functions in the entire database which contained incomplete SSA and this caused the range analysis to loop infinitely in any of these three functions.

I haven't managed to reproduce a test for this. However, I will make sure that we have testing for this in DCA once this PR is merged.


Diff:
diff --git a/cpp/ql/lib/change-notes/2025-05-13-range-analysis-infinite-loop.md b/cpp/ql/lib/change-notes/2025-05-13-range-analysis-infinite-loop.md
new file mode 100644
index 000000000000..7452e024d53f
--- /dev/null
+++ b/cpp/ql/lib/change-notes/2025-05-13-range-analysis-infinite-loop.md
@@ -0,0 +1,4 @@
+---
+category: fix
+---
+* Fixed an infinite loop in `semmle.code.cpp.rangeanalysis.new.RangeAnalysis` when computing ranges in very large and complex function bodies.
\ No newline at end of file
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll b/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll
index 39975d8883c4..e517a75edf97 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/DataFlowPrivate.qll
@@ -1567,7 +1567,7 @@ private int countNumberOfBranchesUsingParameter(SwitchInstruction switch, Parame
         |
           exists(Ssa::UseImpl use | use.hasIndexInBlock(useblock, _, sv))
           or
-          exists(Ssa::DefImpl def | def.hasIndexInBlock(useblock, _, sv))
+          exists(Ssa::DefImpl def | def.hasIndexInBlock(sv, useblock, _))
         )
       )
   )
@@ -1814,7 +1814,7 @@ module IteratorFlow {
      */
     private predicate isIteratorWrite(Instruction write, Operand address) {
       exists(Ssa::DefImpl writeDef, IRBlock bb, int i |
-        writeDef.hasIndexInBlock(bb, i, _) and
+        writeDef.hasIndexInBlock(_, bb, i) and
         bb.getInstruction(i) = write and
         address = writeDef.getAddressOperand()
       )
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/SsaInternals.qll b/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/SsaInternals.qll
index 51829f13df51..bea6b68d5119 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/SsaInternals.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/dataflow/internal/SsaInternals.qll
@@ -191,7 +191,7 @@ abstract class DefImpl extends TDefImpl {
    * Holds if this definition (or use) has index `index` in block `block`,
    * and is a definition (or use) of the variable `sv`
    */
-  final predicate hasIndexInBlock(IRBlock block, int index, SourceVariable sv) {
+  final predicate hasIndexInBlock(SourceVariable sv, IRBlock block, int index) {
     this.hasIndexInBlock(block, index) and
     sv = this.getSourceVariable()
   }
@@ -891,12 +891,12 @@ private module SsaInput implements SsaImplCommon::InputSig<Location> {
   predicate variableWrite(BasicBlock bb, int i, SourceVariable v, boolean certain) {
     DataFlowImplCommon::forceCachingInSameStage() and
     (
-      exists(DefImpl def | def.hasIndexInBlock(bb, i, v) |
+      exists(DefImpl def | def.hasIndexInBlock(v, bb, i) |
         if def.isCertain() then certain = true else certain = false
       )
       or
       exists(GlobalDefImpl global |
-        global.hasIndexInBlock(bb, i, v) and
+        global.hasIndexInBlock(v, bb, i) and
         certain = true
       )
     )
@@ -934,10 +934,11 @@ module SsaCached {
 }
 
 /** Gets the `DefImpl` corresponding to `def`. */
+pragma[nomagic]
 private DefImpl getDefImpl(SsaImpl::Definition def) {
   exists(SourceVariable sv, IRBlock bb, int i |
     def.definesAt(sv, bb, i) and
-    result.hasIndexInBlock(bb, i, sv)
+    result.hasIndexInBlock(sv, bb, i)
   )
 }
 
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/IRFunction.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/IRFunction.qll
index 354ba41e3d1b..7fa08f57a006 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/IRFunction.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/IRFunction.qll
@@ -58,4 +58,12 @@ class IRFunction extends IRFunctionBase {
    * Gets all blocks in this function.
    */
   final IRBlock getABlock() { result.getEnclosingIRFunction() = this }
+
+  /**
+   * Holds if this function may have incomplete def-use information.
+   *
+   * Def-use information may be omitted for a function when it is too expensive
+   * to compute.
+   */
+  final predicate hasIncompleteSsa() { Construction::hasIncompleteSsa(this) }
 }
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll
index acb17006fefb..522cd393081e 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/AliasedSSA.qll
@@ -235,7 +235,7 @@ private newtype TMemoryLocation =
  *
  * Some of these memory locations will be filtered out for performance reasons before being passed to SSA construction.
  */
-abstract private class MemoryLocation0 extends TMemoryLocation {
+abstract class MemoryLocation0 extends TMemoryLocation {
   final string toString() {
     if this.isMayAccess()
     then result = "?" + this.toStringInternal()
@@ -874,7 +874,7 @@ private int numberOfOverlappingUses(MemoryLocation0 def) {
  * Holds if `def` is a busy definition. That is, it has a large number of
  * overlapping uses.
  */
-private predicate isBusyDef(MemoryLocation0 def) { numberOfOverlappingUses(def) > 1024 }
+predicate isBusyDef(MemoryLocation0 def) { numberOfOverlappingUses(def) > 1024 }
 
 /** Holds if `use` is a use that overlaps with a busy definition. */
 private predicate useOverlapWithBusyDef(MemoryLocation0 use) {
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll
index e0a6594e7400..1a1eb6a1876f 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll
@@ -731,6 +731,20 @@ private module Cached {
     or
     instruction = getChi(result.(UninitializedGroupInstruction))
   }
+
+  /**
+   * Holds if the def-use information for `f` may have been omitted because it
+   * was too expensive to compute. This happens if one of the memory allocations
+   * in `f` is a busy definition (i.e., it has many different overlapping uses).
+   */
+  pragma[nomagic]
+  cached
+  predicate hasIncompleteSsa(IRFunction f) {
+    exists(Alias::MemoryLocation0 defLocation |
+      Alias::isBusyDef(pragma[only_bind_into](defLocation)) and
+      defLocation.getIRFunction() = f
+    )
+  }
 }
 
 private Instruction getNewInstruction(OldInstruction instr) { getOldInstruction(result) = instr }
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/IRFunction.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/IRFunction.qll
index 354ba41e3d1b..7fa08f57a006 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/IRFunction.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/IRFunction.qll
@@ -58,4 +58,12 @@ class IRFunction extends IRFunctionBase {
    * Gets all blocks in this function.
    */
   final IRBlock getABlock() { result.getEnclosingIRFunction() = this }
+
+  /**
+   * Holds if this function may have incomplete def-use information.
+   *
+   * Def-use information may be omitted for a function when it is too expensive
+   * to compute.
+   */
+  final predicate hasIncompleteSsa() { Construction::hasIncompleteSsa(this) }
 }
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/internal/IRConstruction.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/internal/IRConstruction.qll
index 5e2461ba8b7c..594e37b668d6 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/internal/IRConstruction.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/raw/internal/IRConstruction.qll
@@ -220,6 +220,8 @@ Instruction getMemoryOperandDefinition(
   none()
 }
 
+predicate hasIncompleteSsa(IRFunction f) { none() }
+
 /**
  * Holds if the operand totally overlaps with its definition and consumes the
  * bit range `[startBitOffset, endBitOffset)`.
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/IRFunction.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/IRFunction.qll
index 354ba41e3d1b..7fa08f57a006 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/IRFunction.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/IRFunction.qll
@@ -58,4 +58,12 @@ class IRFunction extends IRFunctionBase {
    * Gets all blocks in this function.
    */
   final IRBlock getABlock() { result.getEnclosingIRFunction() = this }
+
+  /**
+   * Holds if this function may have incomplete def-use information.
+   *
+   * Def-use information may be omitted for a function when it is too expensive
+   * to compute.
+   */
+  final predicate hasIncompleteSsa() { Construction::hasIncompleteSsa(this) }
 }
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SSAConstruction.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SSAConstruction.qll
index e0a6594e7400..1a1eb6a1876f 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SSAConstruction.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SSAConstruction.qll
@@ -731,6 +731,20 @@ private module Cached {
     or
     instruction = getChi(result.(UninitializedGroupInstruction))
   }
+
+  /**
+   * Holds if the def-use information for `f` may have been omitted because it
+   * was too expensive to compute. This happens if one of the memory allocations
+   * in `f` is a busy definition (i.e., it has many different overlapping uses).
+   */
+  pragma[nomagic]
+  cached
+  predicate hasIncompleteSsa(IRFunction f) {
+    exists(Alias::MemoryLocation0 defLocation |
+      Alias::isBusyDef(pragma[only_bind_into](defLocation)) and
+      defLocation.getIRFunction() = f
+    )
+  }
 }
 
 private Instruction getNewInstruction(OldInstruction instr) { getOldInstruction(result) = instr }
diff --git a/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll b/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll
index 8bee2bf86a77..39283d07b54c 100644
--- a/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll
+++ b/cpp/ql/lib/semmle/code/cpp/ir/implementation/unaliased_ssa/internal/SimpleSSA.qll
@@ -73,6 +73,8 @@ class MemoryLocation extends TMemoryLocation {
   final predicate canReuseSsa() { canReuseSsaForVariable(var) }
 }
 
+class MemoryLocation0 = MemoryLocation;
+
 predicate canReuseSsaForOldResult(Instruction instr) { none() }
 
 abstract class VariableGroup extends Unit {
@@ -141,3 +143,9 @@ int getStartBitOffset(MemoryLocation location) { none() }
 
 /** Gets the end bit offset of a `MemoryLocation`, if any. */
 int getEndBitOffset(MemoryLocation location) { none() }
+
+/**
+ * Holds if `def` is a busy definition. That is, it has a large number of
+ * overlapping uses.
+ */
+predicate isBusyDef(MemoryLocation def) { none() }
diff --git a/cpp/ql/lib/semmle/code/cpp/rangeanalysis/new/internal/semantic/SemanticExprSpecific.qll b/cpp/ql/lib/semmle/code/cpp/rangeanalysis/new/internal/semantic/SemanticExprSpecific.qll
index 1b36ae2efc5e..1b83ae959d2c 100644
--- a/cpp/ql/lib/semmle/code/cpp/rangeanalysis/new/internal/semantic/SemanticExprSpecific.qll
+++ b/cpp/ql/lib/semmle/code/cpp/rangeanalysis/new/internal/semantic/SemanticExprSpecific.qll
@@ -112,7 +112,14 @@ module SemanticExprConfig {
   }
 
   /** Holds if no range analysis should be performed on the phi edges in `f`. */
-  private predicate excludeFunction(Cpp::Function f) { count(f.getEntryPoint()) > 1 }
+  private predicate excludeFunction(Cpp::Function f) {
+    count(f.getEntryPoint()) > 1
+    or
+    exists(IR::IRFunction irFunction |
+      irFunction.getFunction() = f and
+      irFunction.hasIncompleteSsa()
+    )
+  }
 
   SemType getUnknownExprType(Expr expr) { result = getSemanticType(expr.getResultIRType()) }
 


Reviews:
user: login: redsun82
body: Thanks for this work!

as an external viewer without much QL knowledge, it does seem there is some existing duplication in this code, among different versions of the IR implementation. I wonder if one could factor out some of that duplication?

Also, do you think there's any way to test this, or the code triggering this is too complex to easily put in a test? Can we have this covered in DCA? I think I recall you found this could be related to an endless loop in the SQL repo?

I will leave more knowledgeable QL devs to approve this.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T06:33:18Z
user: login: MathiasVP
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:51:09Z
user: login: MathiasVP
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T10:22:54Z
user: login: jketema
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T15:58:47Z

Review comments:
diff_hunk: @@ -1319,6 +1319,13 @@ predicate canReuseSsaForMemoryResult(Instruction instruction) {
   // We don't support reusing SSA for any location that could create a `Chi` instruction.
 }
 
+predicate hasIncompleteSsa(IRFunction f) {
path: cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll
user: login: redsun82
body: I understand this may be more or less an implementation detail, but should we still put a doc comment here?
created_at: 2025-05-13T06:25:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1319,6 +1319,13 @@ predicate canReuseSsaForMemoryResult(Instruction instruction) {
   // We don't support reusing SSA for any location that could create a `Chi` instruction.
 }
 
+predicate hasIncompleteSsa(IRFunction f) {
path: cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll
user: login: MathiasVP
body: Happy to do that. Note that this is still a draft since I wanted to check the implications on DCA before I made this more pretty 🙂
created_at: 2025-05-13T09:51:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1319,6 +1319,13 @@ predicate canReuseSsaForMemoryResult(Instruction instruction) {
   // We don't support reusing SSA for any location that could create a `Chi` instruction.
 }
 
+predicate hasIncompleteSsa(IRFunction f) {
path: cpp/ql/lib/semmle/code/cpp/ir/implementation/aliased_ssa/internal/SSAConstruction.qll
user: login: MathiasVP
body: I've force-pushed some QLDoc in 510df38da2476708271777dc17ad5bd52068f14c
created_at: 2025-05-13T10:22:54Z
author_association: CONTRIBUTOR

PR comments:
user: login: MathiasVP
created_at: 2025-05-13T10:07:56Z
body: > as an external viewer without much QL knowledge, it does seem there is some existing duplication in this code, among different versions of the IR implementation. I wonder if one could factor out some of that duplication?

The term you're looking for here is "parameterized modules" 😂

The IR implementation precedes parameterized modules and no one has found time to rewrite the IR to use parameterized modules yet. So right now the IR is "pyrameterized" via [identical-files.json](https://github.com/github/codeql/blob/main/config/identical-files.json) (so the duplication is actually not that bad since there's a script and a CI job to ensure that they're always identical)

> Also, do you think there's any way to test this, or the code triggering this is too complex to easily put in a test? Can we have this covered in DCA? I think I recall you found this could be related to an endless loop in the SQL repo?

Yeah, I plan on bumping a SHA on one of the projects in DCA once this PR goes in to make sure we have a test for this in DCA. I can't do that before this PR goes in since we can't test the "base SHA" since that will hit an infinite loop on the current main.
author_association: CONTRIBUTOR
user: login: MathiasVP
created_at: 2025-05-13T10:24:04Z
body: DCA looks uneventful. No performance implications (as expected), and no result changes (since this affects such a small number of function bodies).
author_association: CONTRIBUTOR
user: login: jketema
created_at: 2025-05-13T10:36:57Z
body: DCA does seem to show some slowdown? Just over 10% for abseil on Linux, as the worst offender.
author_association: CONTRIBUTOR
user: login: MathiasVP
created_at: 2025-05-13T10:38:50Z
body: > DCA does seem to show some slowdown? Just over 10% for abseil on Linux, as the worst offender.

Oh! You're right! Why wasn't that in `Summary summaries`? 😭 Anyway, thanks for the heads up. I'll take a look right away.

I suppose the join order issue highlighted by DCA in `RangeAnalysisImpl::ModulusAnalysisInstantiated::ssaModulus` is relevant 😅
author_association: CONTRIBUTOR
user: login: jketema
created_at: 2025-05-13T10:40:11Z
body: > > DCA does seem to show some slowdown? Just over 10% for abseil on Linux, as the worst offender.
> 
> Oh! You're right! Why wasn't that in `Summary summaries`? 😭 Anyway, thanks for the heads up. I'll take a look right away.

Because it never is...
author_association: CONTRIBUTOR
user: login: redsun82
created_at: 2025-05-13T10:50:42Z
body: > Oh! You're right! Why wasn't that in `Summary summaries`? 😭 Anyway, thanks for the heads up. I'll take a look right away.

Incidentally, it's the same question I asked yesterday [here](https://github.com/github/semmle-code/pull/52898#pullrequestreview-2832051661). As far as I could tell, we swept that kind of landing page alerts under a pretty high threshold in order to avoid wobble. That does seem to make the landing page less useful than it should...

I wonder if the wobble we tried to silence was platform specific (i.e. windows maybe?)
author_association: CONTRIBUTOR
user: login: MathiasVP
created_at: 2025-05-13T13:26:06Z
body: > DCA does seem to show some slowdown? Just over 10% for abseil on Linux, as the worst offender.

Should hopefully be fixed by 0836f0b413126532825744034cf5e46a80604da6. There were two issues:
- A small join order issue
- I had placed the predicate outside the `cached` module which added an extra stage along with some re-evaluation.

Additionally, I found another small join order issue that I fixed in f255fc2fd593dcd08e21062829384688690b297c.

I'll rerun DCA and see if the performance has been fixed 🤞
author_association: CONTRIBUTOR