[OnePlus 3 Guides, News, & Discussion] GuestK00458:
What seems to work for removing on existing roms (so not on build stage) is to remove everything except org.ifaa.android.manager related stuff.

What I have removed is IFAAServices, Soterservices and everything related except in /system/system/framework.

This is what I have left:

/system/system/framework/arm/boot-org.ifaa.android.manager.art
/system/system/framework/arm/boot-org.ifaa.android.manager.oat
/system/system/framework/arm/boot-org.ifaa.android.manager.vdex
/system/system/framework/arm64/boot-org.ifaa.android.manager.art
/system/system/framework/arm64/boot-org.ifaa.android.manager.oat
/system/system/framework/arm64/boot-org.ifaa.android.manager.vdex
/system/system/framework/boot-org.ifaa.android.manager.vdex
/system/system/framework/org.ifaa.android.manager.jar

No delays and no issues in daily usage.

Edit:

Script part I am using now on non-deblobbed rom to remove just the stuff which can be removed (not the ifaa.android.manager files):

#!/sbin/sh
#
# https://xdaforums.com/oneplus-3/how-to/guide-deblob-alipay-ifaa-tencent-soter-t4064893/

echo -n "Deblob AliPay, Soter & IFAA stuff..."

# First mount firmware_mnt
mount /dev/block/by-name/modem /system/system/vendor/firmware_mnt

# Remove all ifaa, soter and alipay blobs in Firmware
bloblist1="
/system/system/vendor/firmware_mnt/image/alipay.b00
/system/system/vendor/firmware_mnt/image/alipay.b01
/system/system/vendor/firmware_mnt/image/alipay.b02
/system/system/vendor/firmware_mnt/image/alipay.b03
/system/system/vendor/firmware_mnt/image/alipay.b04
/system/system/vendor/firmware_mnt/image/alipay.b05
/system/system/vendor/firmware_mnt/image/alipay.b06
/system/system/vendor/firmware_mnt/image/alipay.mdt
/system/system/vendor/firmware_mnt/image/soter64.b00
/system/system/vendor/firmware_mnt/image/soter64.b01
/system/system/vendor/firmware_mnt/image/soter64.b02
/system/system/vendor/firmware_mnt/image/soter64.b03
/system/system/vendor/firmware_mnt/image/soter64.b04
/system/system/vendor/firmware_mnt/image/soter64.b05
/system/system/vendor/firmware_mnt/image/soter64.b06
/system/system/vendor/firmware_mnt/image/soter64.mdt
"

for blobname in $bloblist1
do
if [ -d "$blobname" ]; then
chmod -R 777 "$blobname" 2>/dev/null
rm -rf "${blobname:?}"/* 2>/dev/null
else
chmod 777 "$blobname" 2>/dev/null
rm -f "$blobname" 2>/dev/null
fi
done

# Remove all ifaa, soter and alipay stuff in vendor and lib
bloblist2="
/system/system/vendor/bin/hw/vendor.oneplus.hardware.ifaa@2.0-service
/system/system/vendor/bin/hw/vendor.qti.hardware.soter@1.0-service
/system/system/vendor/etc/init/vendor.oneplus.hardware.ifaa@2.0-service.rc
/system/system/vendor/etc/init/vendor.qti.hardware.soter@1.0-service.rc
/system/system/vendor/lib64/hw/vendor.qti.hardware.soter@1.0-impl.so
/system/system/vendor/lib64/vendor.qti.hardware.soter@1.0.so
/system/system/lib64/vendor.oneplus.hardware.ifaa@2.0.so
"

for blobname in $bloblist2
do
if [ -d "$blobname" ]; then
chmod -R 777 "$blobname" 2>/dev/null
rm -rf "${blobname:?}"/* 2>/dev/null
else
chmod 777 "$blobname" 2>/dev/null
rm -f "$blobname" 2>/dev/null
fi
done

# /system/system/framework/arm/boot-org.ifaa.android.manager.art
# /system/system/framework/arm/boot-org.ifaa.android.manager.oat
# /system/system/framework/arm/boot-org.ifaa.android.manager.vdex
# /system/system/framework/arm64/boot-org.ifaa.android.manager.art
# /system/system/framework/arm64/boot-org.ifaa.android.manager.oat
# /system/system/framework/arm64/boot-org.ifaa.android.manager.vdex
# /system/system/framework/boot-org.ifaa.android.manager.vdex
# /system/system/framework/org.ifaa.android.manager.jar

# Remove org.ifaa.android.manager.jar from environ.rc
# sed -i 's/:\/system\/framework\/org.ifaa.android.manager.jar//' /system/init.environ.rc

umount /system/system/vendor/firmware_mnt

echo " done!"