You could use RPC_TRANSACTION_REJECTED code here, which I introduce in #3730 (maybe rename it to transaction or block rejected...).