"I've got your number" is a phrase that usually means one of two things.  One being that I have, literally, your phone or address number.  Or, it usually implies a negative judgement about the action being perceived.