Package: procmon-parser
Parsing Procmon files with PythonProcmon (https://docs.microsoft.com/en-us/sysinternals/downloads/procmon) is a very powerful monitoring tool for Windows,
capable of capturing file system, registry, process/thread and network activity.Procmon uses internal file formats for configuration (PMC) and logs (PML).
Prior toprocmon-parser,PMCfiles could only be parsed and generated by the Procmon GUI, andPMLfiles
could be read only using the Procmon GUI, or by converting them to CSV or XML using Procmon command line.The goals ofprocmon-parserare:Parsing & BuildingPMCfiles - making it possible to dynamically add/remove filter rules, which can significantly
reduce the size of the log file over time as Procmon captures millions of events.ParsingPMLfiles - making it possible to directly load the rawPMLfile into convenient python objects
instead of having to convert the file to CSV/XML formats prior to loading.PMC (Process Monitor Configuration) ParserUsageLoading configuration of a pre-exported Procmon configuration:>>>fromprocmon_parserimportload_configuration,dump_configuration,Rule>>>withopen("ProcmonConfiguration.pmc","rb")asf:...config=load_configuration(f)>>>config["DestructiveFilter"]0>>>config["FilterRules"][Rule(Column.PROCESS_NAME,RuleRelation.IS,"System",RuleAction.EXCLUDE),Rule(Column.PROCESS_NAME,RuleRelation.IS,"Procmon64.exe",RuleAction.EXCLUDE),Rule(Column.PROCESS_NAME,RuleRelation.IS,"Procmon.exe",RuleAction.EXCLUDE),Rule(Column.PROCESS_NAME,RuleRelation.IS,"Procexp64.exe",RuleAction.EXCLUDE),Rule(Column.PROCESS_NAME,RuleRelation.IS,"Procexp.exe",RuleAction.EXCLUDE),Rule(Column.PROCESS_NAME,RuleRelation.IS,"Autoruns.exe",RuleAction.EXCLUDE),Rule(Column.OPERATION,RuleRelation.BEGINS_WITH,"IRP_MJ_",RuleAction.EXCLUDE),Rule(Column.OPERATION,RuleRelation.BEGINS_WITH,"FASTIO_",RuleAction.EXCLUDE),Rule(Column.RESULT,RuleRelation.BEGINS_WITH,"FAST IO",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"pagefile.sys",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$Volume",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$UpCase",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$Secure",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$Root",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$MftMirr",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$Mft",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$LogFile",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.CONTAINS,"$Extend",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$Boot",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$Bitmap",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$BadClus",RuleAction.EXCLUDE),Rule(Column.PATH,RuleRelation.ENDS_WITH,"$AttrDef",RuleAction.EXCLUDE),Rule(Column.EVENT_CLASS,RuleRelation.IS,"Profiling",RuleAction.EXCLUDE)]Adding some new rules>>>new_rules=[Rule('PID','is','1336','include'),Rule('Process_Name','contains','python')]>>>config["FilterRules"]=new_rules+config["FilterRules"]Dropping filtered events>>>config["DestructiveFilter"]=1Dumping the new configuration to a file>>>withopen("ProcmonConfiguration1337.pmc","wb")asf:...dump_configuration(config,f)File FormatFor the raw binary format of PMC files you can refer to thedocs, or take a look at the source code inconfiguration_format.py.PML (Process Monitor Log) ParserUsageprocmon-parserexports aProcmonLogsReaderclass for reading logs directly from a PML file:>>>fromprocmon_parserimportProcmonLogsReader>>>f=open("LogFile.PML","rb")>>>pml_reader=ProcmonLogsReader(f)>>>len(pml_reader)# number of logs53214>>>first_event=next(pml_reader)# reading the next event in the log>>>print(first_event)ProcessName=dwm.exe,Pid=932,Operation=RegQueryValue,Path="HKCU\Software\Microsoft\Windows\DWM\ColorPrevalence",Time=7/12/20201:18:10.7752429AM>>>print(first_event.process)#  Accessing the process of the event"C:\Windows\system32\dwm.exe",932>>>formoduleinfirst_event.process.modules[:3]:...print(module)# printing information about some modules"C:\Windows\system32\dwm.exe",address=0x7ff6fa980000,size=0x18000"C:\Windows\system32\d3d10warp.dll",address=0x7fff96700000,size=0x76c000"C:\Windows\system32\wuceffects.dll",address=0x7fff9a920000,size=0x3f000>>>first_event.stacktrace# get a list of the stack frames addresses from the event[18446735291098361031,18446735291098336505,18446735291095097155,140736399934388,140736346856333,140736346854333,140698742953668,140736303659045,140736303655429,140736303639145,140736303628747,140736303625739,140736303693867,140736303347333,140736303383760,140736303385017,140736398440420,140736399723393]>>>File FormatFor the raw binary format of PML files you can refer to thedocs, or take a look at the source code instream_logs_format.py.Currently the parser is only tested with PML files saved byProcmon.exeof versions v3.4.0 or higher.TODOThe PML format is very complex so there are some features (unchecked in the list) that are not supported yet:Getting the IRP name of the operation.Category column and Detail column, which contains different information about each operation type, is supported only for some of the operations:Network operationsUDP/TCP UnknownUDP/TCP OtherUDP/TCP SendUDP/TCP ReceiveUDP/TCP AcceptUDP/TCP ConnectUDP/TCP DisconnectUDP/TCP ReconnectUDP/TCP RetransmitUDP/TCP TCPCopyProcess operationsProcess DefinedProcess CreateProcess ExitThread CreateThread ExitLoad ImageThread ProfileProcess StartProcess StatisticsSystem StatisticsRegistry operationsRegOpenKeyRegCreateKeyRegCloseKeyRegQueryKeyRegSetValueRegQueryValueRegEnumValueRegEnumKeyRegSetInfoKeyRegDeleteKeyRegDeleteValueRegFlushKeyRegLoadKeyRegUnloadKeyRegRenameKeyRegQueryMultipleValueKeyRegSetKeySecurityRegQueryKeySecurityFilesystem OperationsVolumeDismountVolumeMountCreateFileMappingCreateFileCreatePipeReadFileWriteFileQueryInformationFileSetInformationFileQueryEAFileSetEAFileFlushBuffersFileQueryVolumeInformationSetVolumeInformationDirectoryControlFileSystemControlDeviceIoControlInternalDeviceIoControlShutdownLockUnlockFileCloseFileCreateMailSlotQuerySecurityFileSetSecurityFilePowerSystemControlDeviceChangeQueryFileQuotaSetFileQuotaPlugAndPlayProfiling OperationsThread ProfilingProcess ProfilingDebug Output ProfilingThese are a lot of operation types so I didn't manage to get to all of them yet :(If there is an unsupported operation which you think its details are interesting, please let me know :)TestsTo test that the parsing is done correctly, There are two fairly large Procmon PML files and their respective CSV format
log files, taken from 64 bit and 32 bit machine. The test checks that each event in the PML parsed byprocmon-parserequals to the respective event in the CSV.Contributingprocmon-parseris developed on GitHub ateronnen/procmon-parser.
Feel free to report an issue or send a pull request, use theissue tracker.