Package: pcaper
The package helps to assemble and iterate HTTP requests.
Pcaper provides class to read traffic files in pcap or har formats,
executable converters -pcap2txtandhar2txt.PcapParserbased ondpkt.HarParseruses built-in json package.pcaperextends dpkt.http.Request class.
Following fields of HTTP request are available:timestamp- timestamp of the last packet of original HTTP requestsrc- source IP addressdst- destination IP addresssport- source TCP portdport- destination TCP portmethod- HTTP request methodversion- HTTP protocol versionuri- HTTP request URIheaders- ordered dictionary of HTTP headersorigin_headers- ordered dictionary HTTP headers with case sensetive namesbody- HTTP request bodyorigin- original HTTP requestInstallationpipinstallpcaperImportimportpcaperpcap_parser=pcaper.PcapParser()har_parser=pcaper.HarParser()ExamplesIterate HTTP requestsRead pcap file, assemble and iterate HTTP requestsfrompcaperimportPcapParserpcap_parser=PcapParser()params={'input':'file.pcap',}forrequestinpcap_parser.read_pcap(params):print(request.origin)frompcaperimportHarParserhar_parser=HarParser()params={'input':'file.har'}forrequestinhar_parser.read_har(params):print(request.origin)Extract separate HTTP request headersYou can extract header by namereader=pcaper.PcapParser()params={'input':'file.pcap'}forrequestinreader.read_pcap(params):print(request.headers['host'])print(request.headers['user-agent'])Filter TCP/IP packetsIt is possible to filter out excess packetsreader=pcaper.PcapParser()params={'input':'file.pcap','filter':'tcp.dst == 1.1.1.1'}forrequestinreader.read_pcap(params):print(request.origin)You can combine tcp and ip filters in dpkt stylereader=pcaper.PcapParser()params={'input':'file.pcap','filter':'(ip.src == 10.4.0.136 or ip.dst == 10.1.40.61) and tcp.dport == 8888'}forrequestinreader.read_pcap(params):print(request.origin)It is possible to use excluding filter in dpkt stylereader=pcaper.PcapParser()params={'input':'file.pcap','filter':'tcp.dport != 8888 and ip.dst != 10.1.40.61'}forrequestinreader.read_pcap(params):print(request.origin)NoteNewpcapng formatis not supported bydpktpackage,
but you can convert input file frompcapngtopcapformat
with standard utility, which is installed withwiresharkpackage.mergecapfile.pcapng-wout.pcap-FpcapScriptspcap2txtThepcap2txtscript is installed to Python directory
and can be executed directly in command lineIt simplify parsing of pcap files. Just extract HTTP requests
including its headers and body and print out complete data to console or file.Print HTTP requests from pcap file:pcap2txtfile.pcapFilter TCP/IP packets, extract HTTP requests and write to external file:pcap2txt-f"tcp.dport == 8080 and ip.dst != 10.10.10.10"-ofile.outfile.pcapFilter HTTP packetspcap2txt-F'"rambler.ru" in http.uri'file.pcapYou can use logical expressions in filterspcap2txt-F'"keep-alive" in http.headers["connection"] or "Keep-alive" in http.headers["connection"]'file.pcapStandard Python string functions over HTTP request headerspcap2txt-F'"keep-alive" in http.headers["connection"].lower()'file.pcapUse excluding filters alsopcap2ammo-F'"rambler.ru" not in http.uri'file.pcapPrint statistics about counted requests:pcap2txt-f"ip.src == 10.10.10.10"-Sfile.pcapStats:total:1complete:1incorrect:0incomplete:0har2txtThehar2txtscript is installed to Python directory
and can be executed directly in command lineIt simplify parsing of har files. Just extract HTTP requests
including its headers and body and print out complete data to console or file.Print HTTP requests from har file:har2txtfile.harFilter HTTP packetshar2txt-F'http.verision == "1.1"'file.harUse excluding filters alsohar2txt-F'"rambler.ru" not in http.uri'file.harFilter packets with destination IP.pcaperextracts data from har file, which contains destination IP
(dstfiled), but doesn’t contain source IP, source and destination ports.har2txt-F'http.dst == "1.1.1.1"'file.harPrint statistics about counted requests:har2txt-S-F'http.dst == "10.10.10.10'file.harStats:total:1complete:1incorrect:0incomplete:0