Package: pdsniff
captures and processes network packets asynchronously using the Scapy library, and converts the results to a pandas DataFramepip install pdsniffTested against Windows 10 ( Necessary:https://npcap.com/#download)  / Python 3.10 / AnacondaFunction start_sniffingThe start_sniffing function is designed to capture and process network packets asynchronously using the Scapy library.
It takes several parameters, including count, result_list, killkey, and dumpfolder, to control the packet capturing and processing behavior.Here's what the function does:It initializes the result_list as a deque with a maximum length of 1 if not provided by the caller.
If a dumpfolder is provided, it creates the folder if it doesn't exist and determines the starting
index for saving pickled files based on existing files in the folder.
It defines an internal function _get_frame that handles the actual packet capturing and processing.
The packet capturing is done using a separate thread (KThread) to avoid blocking the main program execution.
Packets are captured and processed in batches of size count.
The processed data is converted into a pandas DataFrame with proper data types based on the provided dtypes.
If a dumpfolder is provided, the processed data frames are periodically saved as pickled files.Args:count(int):Thenumberofpacketstocaptureandprocessineachbatch.result_list(None,list,deque,optional):Listordequetostoretheprocesseddataframes.IfNone,adequewithamaximumlengthof1willbeused.DefaultstoNone.killkey(str,optional):Thehotkeycombinationtostoppacketcapturingandprocessing.Defaultsto"ctrl+alt+x".dumpfolder(str,optional):Thefolderpathtosaveprocesseddataframesaspickledfiles.Ifprovided,capturedpacketswillbeperiodicallysavedaspicklefilesinthisfolder.DefaultstoNone.**kwargs:AdditionalkeywordargumentstobepassedtotheScapysnifffunction.Returns:KThread:Athreadobjectthatcapturesandprocessesthenetworkpackets.Note:Thisfunctionstartsanewthread(KThread)forpacketcapturingandprocessing.Thefunctionwillkeeprunningandcapturingpacketsuntilthespecified`killkey`hotkeyispressed.Capturedpacketswillbeprocessedinbatchesof`count`andstoredinthe`result_list`.If`dumpfolder`isprovided,theprocesseddataframeswillalsobesavedaspicklefiles.The`result_list`parametercanbeusedtoaccessthecapturedandprocesseddataframesfromthecallingcode.Advantages:Asynchronous packet capturing allows the main program to continue executing without waiting for packets to arrive, leading to better program responsiveness.Batch processing reduces processing overhead and allows handling large volumes of packets efficiently.Saving processed data frames as pickled files facilitates data persistence and easy data retrieval for analysis or further processing.Function load_dump_filesThe load_dump_files function is responsible for loading and concatenating processed data frames from pickled files in the specified folder.Here's what the function doesIt reads all pickled files (.pkl) in the provided folder that have numeric filenames.
It concatenates the data frames from these files into a single pandas DataFrame.
The resulting DataFrame contains the concatenated data from the pickled files.Loadandconcatenateprocesseddataframesfrompickledfilesinthespecifiedfolder.Args:folder(str):Thefolderpathwherethepickleddataframesarestored.Returns:pd.DataFrame:ADataFramecontainingtheconcatenateddatafromthepickledfiles.Note:Thisfunctionreadsallpickledfiles(*.pkl)inthespecifiedfolderthathavenumericfilenames.ItconcatenatesthedataframesfromthesefilesandreturnsasingleDataFrame.The`folder`parametershouldbethepathtothefoldercontainingthepickledfiles.frompdsniffimportstart_sniffing,load_dump_filescount=1000result_list=[]# results will be appendedkillkey="ctrl+alt+x"folder_path="c:\\internetdump"t2=start_sniffing(count=count,result_list=result_list,killkey=killkey,dumpfolder=folder_path)df=load_dump_files(folder_path)