Bug type: crash
Component: SimplifyLibCalls
Bug location: llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp: LibCallSimplifier::optimizeCall
Issue title: [SimplifyLibCalls] Calling convention assertion fails
Labels: llvm:crash
llvm:instcombine
Reported by: nikic

```llvm
; RUN: opt -S -passes=instcombine
define void @main(ptr %ptr) {
  %opendir = call fastcc ptr @opendir(ptr %ptr)
  %memset = call ptr @__memset_chk(ptr %opendir, i32 0, i64 596, i64 -1)
  ret void
}

declare ptr @__memset_chk(ptr, i32, i64, i64)

declare fastcc ptr @opendir(ptr)
```

> opt: /home/npopov/repos/llvm-project/llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp:3488: Value *llvm::LibCallSimplifier::optimizeStringMemoryLibCall(CallInst *, IRBuilderBase &): Assertion `(ignoreCallingConv(Func) || TargetLibraryInfoImpl::isCallingConvCCompatible(CI)) && "Optimizing string/memory libcall would change the calling convention"' failed.

Encountered when building openjpeg2.


Fix patch:
commit a001e9718fd974859f2797a9f9ed7bf87d364e4f
Author: Nikita Popov <npopov@redhat.com>
Date:   Fri Jan 5 10:41:44 2024 +0100

    [SimplifyLibCalls] Don't try to manually reprocess calls
    
    The current code for reprocessing the result of fortified libcall
    simplifications is not correct, because we might simplify to an
    argument of the original call, and if that is again a libcall,
    mistakenly think that this is actually the simplification result.
    
    Instead of trying to fix this, simply remove the code entirely,
    because InstCombine nowadays correctly handles reprocessing of
    SimplifyLibCall results.
    
    Fixes https://github.com/llvm/llvm-project/issues/77064.

diff --git a/llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp b/llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp
index 760a626c8b6f..a7cd68e860e4 100644
--- a/llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp
+++ b/llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp
@@ -3735,26 +3735,8 @@ Value *LibCallSimplifier::optimizeCall(CallInst *CI, IRBuilderBase &Builder) {
 
   // Also try to simplify calls to fortified library functions.
   if (Value *SimplifiedFortifiedCI =
-          FortifiedSimplifier.optimizeCall(CI, Builder)) {
-    // Try to further simplify the result.
-    CallInst *SimplifiedCI = dyn_cast<CallInst>(SimplifiedFortifiedCI);
-    if (SimplifiedCI && SimplifiedCI->getCalledFunction()) {
-      // Ensure that SimplifiedCI's uses are complete, since some calls have
-      // their uses analyzed.
-      replaceAllUsesWith(CI, SimplifiedCI);
-
-      // Set insertion point to SimplifiedCI to guarantee we reach all uses
-      // we might replace later on.
-      IRBuilderBase::InsertPointGuard Guard(Builder);
-      Builder.SetInsertPoint(SimplifiedCI);
-      if (Value *V = optimizeStringMemoryLibCall(SimplifiedCI, Builder)) {
-        // If we were able to further simplify, remove the now redundant call.
-        substituteInParent(SimplifiedCI, V);
-        return V;
-      }
-    }
+          FortifiedSimplifier.optimizeCall(CI, Builder))
     return SimplifiedFortifiedCI;
-  }
 
   // Then check for known library functions.
   if (TLI->getLibFunc(*Callee, Func) && isLibFuncEmittable(M, TLI, Func)) {


Regression tests:
file: llvm/test/Transforms/InstCombine/pr77064.ll
commands: opt -S -passes=instcombine < %s
tests: test_name: main
test_body: define void @main(ptr %ptr) {
  %opendir = call fastcc ptr @opendir(ptr %ptr)
  %memset = call ptr @__memset_chk(ptr %opendir, i32 0, i64 596, i64 -1)
  ret void
}

declare ptr @__memset_chk(ptr, i32, i64, i64)

declare fastcc ptr @opendir(ptr)