Question: I have e-mails and transcripts in German from conversations with customers. They include personal identifiable information that I need to remove. So it is a data anonymisation task. The text would be "Hello Mr. Smith", "Dear Mr Smith", "Hello Lisa" etc. followed by the conversation. I need to keep the conversation for further analysis. Three solutions come to my mind: A) compiling a list of names: At this stage, I do not know all the names that will be mentioned. I have no access to the CRM database. So compiling a list and adding it to the stopwords corpus will be time-consuming and/or error prone. B) Part-of-Speech Tagging (PoS) / Named Entity Recognition (NER): This would also delete product names and places. I need to keep this information. So NER is unfortunately not an option. C) Regular expression (regex): Use RegEx to match the salutation, e.g. "Dear", and delete the subsequent word. This answer gave me a good starting point, but it assumes that I know the word that follows the name I need to delete which I don't. import re print re.sub(r'(?<=copy )(.*)(?=from)', '', "copy table values from 'a.dat';")  How could I modify the code to delete the word that follows the salutation? I read up on lookaround and played around a bit on regex101 but couldn't figure it out. Also, would I need to tokenize the string first? A solution with pandas str.replace is welcome too.

Answer: Version 2a You could approach this by setting up lists of salutations and of honorifics (Mr, Mrs, Dr etc), and if any word in your data matched one of these working through the first few words you could remove it. This approach makes it easy to quickly add new salutations and honorifics as you spot them in the transcripts. This first piece of code is an improvement on my earlier answer (which is listed at the end). @Simone added the requirement that the salutation could be a pair of words (Good morning rather than just Hello - for example). The new code copes with this, and also works for double names (such as John Doe as well as just John). It also works with a fairly wide range of less conventional ways of addressing someone (and is not fooled by capitalisation or the lack of it) which might appear in an email or a transcript, as hopefully the example text in the code shows. However, although it is more bullet-proof than the first version, it is not perfect - as you might expect from code that is trying to second-guess human responses. import re text = ["Dear mrs chan Blah blah blah", "Dear fred smith, How are you?", "greetings ms duncan-jones, what Gives?", " My Dear Lisa , this is cool.", "Hello Dr Foster It's good, to hear, from you.", "Meera Whatcha thinkin?", "Good morning Ms Daisy Martin, Hope you are keeping well.", "Hi there seema hows things?", "Jaz Yes ok", "OK raffi you may be right"] salutations_1 = ["dear", "dearest", "hello", "hi", "hiya", "greetings", "salutations", "ok", "good", "my"] salutations_2 = ["morning", "day", "afternoon", "evening", "there", "dear", "dearest"] honorifics = ["mr", "mrs", "dr", 'ms', "sir", "master"] for line in text: line = line.replace(',', ' #', 1) # Mark the position of the first comma line = line.strip() words = re.split(r"[\s]+", line) try: # First salutation word if words[0].lower() in salutations_1: words = words[1:] # Second salutation word if words[0].lower() in salutations_2: words = words[1:] # Honorifics if words[0].lower() in honorifics: words = words[1:] # Remove 1st name (eg John) words = words[1:] # Remove comma marker if present if words[0] == '#': words = words[1:] # Remove possible second name (eg Doe) and comma marker if words[1] == '#': words = words[2:] joined = ' '.join(words) finished = joined.replace(' #', ',') # Restore any unused comma print(finished) except: print("**Line too short**")  This gives: Blah blah blah How are you? what Gives? this is cool. It's good, to hear, from you. Whatcha thinkin? Hope you are keeping well. hows things? Yes ok you may be right  It has removed all the salutations, honorifics and names without losing any of the accompanying text. This is the earlier version of the code, left here for completeness. import re text = ["Dear mrs smith Blah", "Dear fred, how are you. It is a while, since etc", " Dear Lisa, this is great", "Hello Dr Foster, It's good to hear from you."] salutations = ["dear", "hello", "hi"] honorifics = ["mr", "mrs", "dr"] for line in text: line = line.strip() words = re.split(r"[\s]+", line) #print(words) for word in words: if word.lower() in salutations: words.remove(word) for word in words: if word.lower() in honorifics: words.remove(word) del words[0] # Remove the name print(' '.join(words))  From the data above, this prints out: Blah how are you. It is a while, since etc this is great It's good to hear from you.