If you remove your number from the list, other users will still be able to report it as spam and share information about it. Because the Truecaller database is built on user-contributed information, such activities by other users might still affect your visibility.