Bug type: miscompilation
Component: InstCombine
Bug location: llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp: InstCombinerImpl::foldLogicOfFCmps
Issue title: Wrong canonicalization of a range check idiom
Labels: miscompilation
llvm:instcombine
floating-point
Reported by: bongjunj

https://github.com/llvm/llvm-project/blob/11dad2fa5138a50d60a5a34a2c7e074b976820e2/llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp#L1511-L1527


Alive2 report: https://alive2.llvm.org/ce/z/qBHAmS

```llvm
----------------------------------------
define i1 @test_and_olt_fmf_propagation_union.2(float %x) {
#0:
  %cmp1 = fcmp nnan ninf olt float %x, 0.000000
  %cmp2 = fcmp nnan nsz ogt float %x, -0.000000
  %#1 = select i1 %cmp2, i1 %cmp1, i1 %cmp2
  %cond = and i1 %#1, %cmp2
  ret i1 %cond
}
=>
define i1 @test_and_olt_fmf_propagation_union.2(float %x) {
#0:
  %#1 = fabs nnan ninf nsz float %x
  %#2 = fcmp nnan ninf nsz olt float %#1, 0.000000
  ret i1 %#2
}
Transformation doesn't verify!

ERROR: Target is more poisonous than source

Example:
float %x = #xff800000 (-oo)

Source:
i1 %cmp1 = poison
i1 %cmp2 = #x0 (0)
i1 %#1 = #x0 (0)
i1 %cond = #x0 (0)

Target:
float %#1 = poison
i1 %#2 = poison
Source value: #x0 (0)
Target value: poison

Summary:
  0 correct transformations
  1 incorrect transformations
  0 failed-to-prove transformations
  0 Alive2 errors
```

Comments: author: XChy
body: Thanks for the report.

We preserve `ninf` flag from `cmp1` when folding logical and `select cmp2, cmp1, false` into `fcmp ninf (fabs(x))`. But if `cmp2` equals to `false`, the select won't be poison whatever `cmp1` is. Thus, such fold makes it more poisonous. See https://alive2.llvm.org/ce/z/hnqeLC
author: XChy
body: And it looks like you are mutating the unit tests to fuzz LLVM. Looks interesting to me. If OK, could you provide some information about it?
author: bongjunj
body: > And it looks like you are mutating the unit tests to fuzz LLVM. Looks interesting to me. If OK, could you provide some information about it?

@XChy Yes we are designing a fuzzer on top of LLVM to test the compiler. We can't disclose the full detail at this time. Thank you.

Fix patch:
commit a2ba438f3e5635e368333213914c7452a6a6a2da
Author: XChy <xxs_chy@outlook.com>
Date:   Mon Oct 21 22:30:31 2024 +0800

    [InstCombine] Preserve the flag from RHS only if the `and` is bitwise (#113164)
    
    Fixes #113123
    Alive proof: https://alive2.llvm.org/ce/z/hnqeLC

diff --git a/llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp b/llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp
index d72013ba223d..835eae9f5725 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp
@@ -1531,8 +1531,10 @@ Value *InstCombinerImpl::foldLogicOfFCmps(FCmpInst *LHS, FCmpInst *RHS,
     }
     if (IsLessThanOrLessEqual(IsAnd ? PredL : PredR)) {
       BuilderTy::FastMathFlagGuard Guard(Builder);
-      Builder.setFastMathFlags(LHS->getFastMathFlags() |
-                               RHS->getFastMathFlags());
+      FastMathFlags NewFlag = LHS->getFastMathFlags();
+      if (!IsLogicalSelect)
+        NewFlag |= RHS->getFastMathFlags();
+      Builder.setFastMathFlags(NewFlag);
 
       Value *FAbs = Builder.CreateUnaryIntrinsic(Intrinsic::fabs, LHS0);
       return Builder.CreateFCmp(PredL, FAbs,


Regression tests:
file: llvm/test/Transforms/InstCombine/fcmp-range-check-idiom.ll
commands: opt -S -passes=instcombine %s
tests: test_name: test_and_olt_fmf_propagation_union_logical_rhs_poison
test_body: define i1 @test_and_olt_fmf_propagation_union_logical_rhs_poison(float %x) {
  %cmp1 = fcmp ninf olt float %x, 0x3C00000000000000
  %cmp2 = fcmp ogt float %x, 0xBC00000000000000
  %cond = select i1 %cmp2, i1 %cmp1, i1 false
  ret i1 %cond
}

test_name: test_and_olt_fmf_propagation_union_logical_both_poison
test_body: define i1 @test_and_olt_fmf_propagation_union_logical_both_poison(float %x) {
  %cmp1 = fcmp ninf olt float %x, 0x3C00000000000000
  %cmp2 = fcmp ninf ogt float %x, 0xBC00000000000000
  %cond = select i1 %cmp2, i1 %cmp1, i1 false
  ret i1 %cond
}

test_name: test_and_olt_fmf_propagation_union
test_body: define i1 @test_and_olt_fmf_propagation_union(float %x) {
  %cmp1 = fcmp nnan ninf olt float %x, 0x3C00000000000000
  %cmp2 = fcmp nnan nsz ogt float %x, 0xBC00000000000000
  %cond = and i1 %cmp1, %cmp2
  ret i1 %cond
}

test_name: test_and_olt_fmf_propagation_union_logical_lhs_poison
test_body: define i1 @test_and_olt_fmf_propagation_union_logical_lhs_poison(float %x) {
  %cmp1 = fcmp olt float %x, 0x3C00000000000000
  %cmp2 = fcmp ninf ogt float %x, 0xBC00000000000000
  %cond = select i1 %cmp2, i1 %cmp1, i1 false
  ret i1 %cond
}