Only try to fall back to the default pubkey if it exists