Package: jspcap
jspcapThejspcapproject is an open source Python program focus onPCAPparsing and analysis, which works as a stream PCAP file extractor. With support ofjsformat, it shall support multiple output report formats.Note that the whole project only supportsPython 3.6or later.AboutModule StructureFoundationInterfaceReassemblyIPSuiteProtocolsUtilitiesCoreKitDumpKitEngine ComparisonInstallationUsageDocumentationInterfacesMacrosFormatsLayersEnginesProtocolsCLI UsageSamplesUsage SamplesCLI SamplesTODOAboutjspcapis an independent open source library, using onlyjsformatas its formatted output dumper.There is a project calledjspcapyworks onjspcap, which is a command line tool for PCAP extraction.Unlike popular PCAP file extractors, such asScapy,dpkt,pyshark, and etc,jspcapusesstreamingstrategy to read input files. That is to read frame by frame, decrease occupation on memory, as well as enhance efficiency in some way.Module StructureInjspcap, all files can be described as following six parts.Foundation (jspcap.foundation) -- synthesise file I/O and protocol analysis, coordinate information exchange in all network layersInterface (jspcap.interface) -- user interface for thejspcaplibrary, which standardise and simplify the usage of this libraryReassembly (jspcap.reassembly) -- base on algorithms described inRFC 815, implement datagram reassembly of IP and TCP packetsIPSuite (jspcap.ipsuite) -- collection of constructors forInternet Protocol SuiteProtocols (jspcap.protocols) -- collection of all protocol family, with detail implementation and methodsUtilities (jspcap.utilities) -- collection of four utility functions and classesCoreKit (jspcap.corekit) -- core utilities forjspcapimplementationDumpKit (jspcap.dumpkit) -- dump utilities forjspcapimplementationEngine ComparisonBesides, due to complexity ofjspcap, its extraction procedure takes around0.01seconds per packet, which is not ideal enough. Thus,jspcapintroduced alternative extraction engines to accelerate this procedure. By now,jspcapsupportsScapy,DPKT, andPyShark. Plus,jspcapsupports two strategies of multiprocessing (server&pipeline). For more information, please refer to the document.EnginePerformance (seconds per packet)default0.014525251388549805server0.12124489148457845pipeline0.014450424114863079scapy0.002443440357844035dpkt0.0003609057267506917pyshark0.0792640733718872InstallationNote thatjspcaponly supports Python versionssince 3.6Simply run the following to install the latest from PyPI:pip install jspcapOr install from the git repository:$ git clone https://github.com/JarryShaw/jspcap.git
$ python setup.py installUsageDocumentationInterfacesNAMEDESCRIPTIONextractextract a PCAP fileanalyseanalyse application layer packetsreassemblereassemble fragmented datagramstracetrace TCP packet flowsMacrosFormatsNAMEDESCRIPTIONJSONJavaScript Object Notation (JSON) formatPLISTmacOS Property List (PLIST) formatTREETree-View text formatPCAPPCAP formatLayersNAMEDESCRIPTIONRAWno specific layerLINKdata-link layerINETinternet layerTRANStransport layerAPPapplication layerEnginesNAMEDESCRIPTIONjsPCAPthe default engineMPServerthe multiprocessing engine with server process strategyMPPipelinethe multiprocessing engine with pipeline strategyDPKTtheDPKTengineScapytheScapyenginePySharkthePySharkengineProtocolsNAMEDESCRIPTIONRawRaw Packet DataARPAddress Resolution ProtocolEthernetEthernet ProtocolL2TPLayer Two Tunnelling ProtocolOSPFOpen Shortest Path FirstRARPReverse Address Resolution ProtocolVLAN802.1Q Customer VLAN Tag TypeAHAuthentication HeaderHIPHost Identity ProtocolHOPOPTIPv6 Hop-by-Hop OptionsIPInternet ProtocolIPsecInternet Protocol SecurityIPv4Internet Protocol version 4IPv6Internet Protocol version 6IPv6_FragFragment Header for IPv6IPv6_OptsDestination Options for IPv6IPv6_RouteRouting Header for IPv6IPXInternetwork Packet ExchangeMHMobility HeaderTCPTransmission Control ProtocolUDPUser Datagram ProtocolHTTPHypertext Transfer ProtocolDocumentation can be found in submodules ofjspcap. Or, you may find usage sample in thetestfolder. For further information, please refer to the source code -- the docstrings should help you :)ps:helpfunction in Python should always help you out.CLI UsageThe following part was originally described injspcapy, which is now deprecated and merged into this repository.As it shows in the help manual, it is quite easy to use:$ jspcapy --help
usage: jspcapy [-h] [-V] [-o file-name] [-f format] [-j] [-p] [-t] [-a] [-v]
               [-F] [-E PKG] [-P PROTOCOL] [-L LAYER]
               input-file-name

PCAP file extractor and formatted exporter

positional arguments:
  input-file-name       The name of input pcap file. If ".pcap" omits, it will
                        be automatically appended.

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit
  -o file-name, --output file-name
                        The name of input pcap file. If format extension
                        omits, it will be automatically appended.
  -f format, --format format
                        Print a extraction report in the specified output
                        format. Available are all formats supported by
                        jsformat, e.g.: json, plist, and tree.
  -j, --json            Display extraction report as json. This will yield
                        "raw" output that may be used by external tools. This
                        option overrides all other options.
  -p, --plist           Display extraction report as macOS Property List
                        (plist). This will yield "raw" output that may be used
                        by external tools. This option overrides all other
                        options.
  -t, --tree            Display extraction report as tree view text. This will
                        yield "raw" output that may be used by external tools.
                        This option overrides all other options.
  -a, --auto-extension  If output file extension omits, append automatically.
  -v, --verbose         Show more information.
  -F, --files           Split each frame into different files.
  -E PKG, --engine PKG  Indicate extraction engine. Note that except default
                        or jspcap engine, all other engines need support of
                        corresponding packages.
  -P PROTOCOL, --protocol PROTOCOL
                        Indicate extraction stops after which protocol.
  -L LAYER, --layer LAYER
                        Indicate extract frames until which layer.Under most circumstances, you should indicate the name of input PCAP file (extension may omit) and at least, output format (json,plist, ortree). Once format unspecified, the name of output file must have proper extension (*.json,*.plist, or*.txt), otherwiseFormatErrorwill raise.As forverbosemode, detailed information will print while extraction (as following examples). Andauto-extensionflag works for the output file, to indicate whether extensions should be appended.SamplesUsage SamplesAs described intestfolder,jspcapis quite easy to use, with simply three verbs as its main interface. Several scenarios are shown as below.extract a PCAP file and dump the result to a specific file (with no reassembly)importjspcap# dump to a PLIST file with no frame storage (property frame disabled)plist=jspcap.extract(fin='in.pcap',fout='out.plist',format='plist',store=False)# dump to a JSON file with no extension auto-completejson=jspcap.extract(fin='in.cap',fout='out.json',format='json',extension=False)# dump to a folder with each tree-view text file per frametree=jspcap.extract(fin='in.pcap',fout='out',format='tree',files=True)extract a PCAP file and fetch IP packet (both IPv4 and IPv6) from a frame (with no output file)>>>importjspcap>>>extraction=jspcap.extract(fin='in.pcap',nofile=True)>>>frame0=extraction.frame[0]# check if IP in this frame, otherwise ProtocolNotFound will be raised>>>flag=jspcap.IPinframe0>>>tcp=frame0[jspcap.IP]ifflagelseNoneextract a PCAP file and reassemble TCP payload (with no output file nor frame storage)importjspcap# set strict to make sure full reassemblyextraction=jspcap.extract(fin='in.pcap',store=False,nofile=True,tcp=True,strict=True)# print extracted packet if HTTP in reassembled payloadsforpacketinextraction.reassembly.tcp:forreassemblyinpacket.packets:ifjspcap.HTTPinreassembly.protochain:print(reassembly.info)CLI SamplesThe CLI (command line interface) ofjspcaphas two different access.through console scripts -- use command namejspcapydirectly (as shown in samples)through Python module --python -m jspcap [...]works exactly the same as aboveHere are some usage samples:export to a macOS Property List (Xcodehas special support for this format)$ jspcapy in --format plist --verbose
🚨Loading file 'in.pcap'
 - Frame   1: Ethernet:IPv6:ICMPv6
 - Frame   2: Ethernet:IPv6:ICMPv6
 - Frame   3: Ethernet:IPv4:TCP
 - Frame   4: Ethernet:IPv4:TCP
 - Frame   5: Ethernet:IPv4:TCP
 - Frame   6: Ethernet:IPv4:UDP
🍺Report file stored in 'out.plist'export to a JSON file (with no format specified)$ jspcapy in --output out.json --verbose
🚨Loading file 'in.pcap'
 - Frame   1: Ethernet:IPv6:ICMPv6
 - Frame   2: Ethernet:IPv6:ICMPv6
 - Frame   3: Ethernet:IPv4:TCP
 - Frame   4: Ethernet:IPv4:TCP
 - Frame   5: Ethernet:IPv4:TCP
 - Frame   6: Ethernet:IPv4:UDP
🍺Report file stored in 'out.json'export to a text tree view file (without extension autocorrect)$ jspcapy in --output out --format tree --verbose
🚨Loading file 'in.pcap'
 - Frame   1: Ethernet:IPv6:ICMPv6
 - Frame   2: Ethernet:IPv6:ICMPv6
 - Frame   3: Ethernet:IPv4:TCP
 - Frame   4: Ethernet:IPv4:TCP
 - Frame   5: Ethernet:IPv4:TCP
 - Frame   6: Ethernet:IPv4:UDP
🍺Report file stored in 'out'TODOspecifyRawpacketinterface verbsreview docstringsmergejspcapywrite documentationimplement IP and MAC address containersimplement option list extractorsimplement more protocols