Question: I have a dataset that requires splitting, exploding, and tidying using regular expressions (regex) in Python and Pandas. The dataset consists of logs from multiple users sent through an old machine to a server API. Each cell may contain multiple messages, and my goal is to transform the data into a structured and tidy format. Here's a sample of the dataset: data = { 'text_plain': [ "5:57:11 H2045: Estatus OK updated (19:48:34) Mark P.: No Defects found on parcel", "11:04:38 Jill : Travel on Time 2:11:30 YHXO: Wheater conds OK", "6:53:07 Jill : Stable Props 22:38:15 Carl V : OK Status 6:15:34 IUJO-65: Door Open", "18:44:38 Van UHJ: Spider on site Alert", "/10:37:43/ H2046 : Movie Purchase Rejected", "10:33:46 Mark P.: Alert by Super User overwrite 21:55:22 Jill: push sent 6:54:41 YHXO: pull received", "23:20:04 Jill : Windows Closed", "5:16:58 Carl V: Is someone on the Front door?", "(17:11:49) IUJO-66 : No Response on Deck (5:10:43) Van UHJ : Flights delay 8:34:08 H2047: Buy Concert Tickets 9:05:42 Mark P.: Gen. OK", "7:00:15 Jill : Status not ok updated 21:22:34 YHXO: Front desk clear" ], 'id': [1,2,3,4,5,6,7,8,9,10] }  As you can see the data needs to be split based on the psuedo timestamp pattern that we see on the column "text_Plain" (will always be in 24-hour format thankfully) followed by a username and a message. The timestamp can be enclosed in parentheses or sometimes in slashes or sometimes not even enclosed at all as you can see, BUT the recurrent pattern that will be of used always to split the data will be r'[(/]?(\d{1,2}:\d{2}:\d{2})[/)]?'  My desired output will be: id timestamp user msg 1 5:57:11 H2045 Estatus OK updated 1 19:48:34 Mark P. No Defects found on parcel 2 11:04:38 Jill Travel on Time 2 2:11:30 YHXO Weather conds OK 3 6:53:07 Jill Stable Props 3 22:38:15 Carl V OK Status 3 6:15:34 IUJO-65 Door Open 4 18:44:38 Van UHJ Spider on site Alert 5 10:37:43 H2046 Movie Purchase Rejected 6 10:33:46 Mark P. Alert by Super User overwrite 6 21:55:22 Jill Push sent 6 6:54:41 YHXO Pull received 7 23:20:04 Jill Windows Closed 8 5:16:58 Carl V Is someone on the Front door? 9 17:11:49 IUJO-66 No Response on Deck 9 5:10:43 Van UHJ Flights delay 9 8:34:08 H2047 Buy Concert Tickets 9 9:05:42 Mark P. Gen. OK 10 7:00:15 Jill Status not ok updated 10 21:22:34 YHXO Front desk clear  My thought of process tells me that I should first split the col "text_Plain" by my regex pattern (without REMOVING IT) so that I can get something like this: ['5:57:11 H2045: Estatus OK updated', '(19:48:34) Mark P.: No Defects found on parcel'] '[11:04:38 Jill : Travel on Time', '2:11:30 YHXO: Wheater conds OK', '6:53:07 Jill : Stable Props'] ['22:38:15 Carl V : OK Status',' 6:15:34 IUJO-65: Door Open', '18:44:38 Van UHJ: Spider on site Alert', '/10:37:43/ H2046 : Movie Purchase Rejected']  To then extract my timestamp, user, msg and get the id (afterwards exploding my plain_text col) But, here is the catch: I have to deal with 5-20 thousands data logs a day and I need to write code that can actually prove to be faster than or more efficient than the one I'm using because I'm new to progrmaming and I have read that regex can be very slow and not efficient and I am keen to learn a better or faster way. This is my workaround at the moment but is taking WAY TOO long for my PC to handle: pattern = '(?<=[(/])(\d{1,2}:\d{2}:\d{2})(?=[/)])' df['new_text'] = df['text_plain'].str.split(pattern) df['new_text'] = df['new_text'].apply(lambda x: [s for s in x if s.strip() != '']) df = df.explode('new_text')  but it is failing as it is not keeping the pattern after the splitting and missing data along the way, also it uses .apply and I have read that we shouldn't use it with regex as it will make the operation way slower if you can clarify that would be awesome

Answer: Try (regex demo): out = ( df["text_plain"] .str.extractall( r"(?P<timestamp>\d+:\d+:\d+)\S*\s+(?P<user>[^:]+?)\s*:\s*(?P<msg>.*?)(?=\s*\S*\d+:\d+:\d+|\Z)" ) .droplevel(level=1) ) print(pd.merge(df[["id"]], out, left_index=True, right_index=True))  Prints:  id timestamp user msg 0 1 5:57:11 H2045 Estatus OK updated 0 1 19:48:34 Mark P. No Defects found on parcel 1 2 11:04:38 Jill Travel on Time 1 2 2:11:30 YHXO Wheater conds OK 2 3 6:53:07 Jill Stable Props 2 3 22:38:15 Carl V OK Status 2 3 6:15:34 IUJO-65 Door Open 3 4 18:44:38 Van UHJ Spider on site Alert 4 5 10:37:43 H2046 Movie Purchase Rejected 5 6 10:33:46 Mark P. Alert by Super User overwrite 5 6 21:55:22 Jill push sent 5 6 6:54:41 YHXO pull received 6 7 23:20:04 Jill Windows Closed 7 8 5:16:58 Carl V Is someone on the Front door? 8 9 17:11:49 IUJO-66 No Response on Deck 8 9 5:10:43 Van UHJ Flights delay 8 9 8:34:08 H2047 Buy Concert Tickets 8 9 9:05:42 Mark P. Gen. OK 9 10 7:00:15 Jill Status not ok updated 9 10 21:22:34 YHXO Front desk clear