Package: protodeep
🏄‍♂️ DescriptionProtoDeep is an easy to use tool that allows you to decode and analyze protobuf data.It isheavily basedon the well-establishedBlackbox Protobufpackage, and inspired byProtobuf Inspectorfor the custom definitions feature.This project was originally intended to be integrated into theGHunt RDTK, but the script grew a lot and ended up becoming a tool that can help many people.Features :CLI usagePython library usageMake your own definitionsEasily match / filter dataExport and compile on the flyExample of using ProtoDeep on the Google's Play StoresearchListendpoint, with custom definitions :✔️ RequirementsPython >= 3.10⚙️ Installation$pip3installpipx
$pipxensurepath
$pipxinstallprotodeepIt will automatically use venvs to avoid dependency conflicts with other projects.💃 UsageHelp menuUsage: main.py [-h] -t TYPE [-d DEFINITIONS] [-na] [-s] [-b] [-bi NUMBER] [-he] [-np] [-m MASK] [-mk MASK] [-mv MASK]
               [-f MASK] [-fk MASK] [-fv MASK] [-epf [PROTOFILE_FILENAME]] [-epd [PROTODEEP_FILENAME]]
               [-c [PYTHON_FILENAME]] [-n SCHEMA_NAME]
               [proto_file]

Positional Arguments:
  proto_file

Options:
  -h, --help            show this help message and exit
  -t, --type TYPE       Either protobuf (raw protobuf content), or protodeep (a ProtoDeep file).
  -d, --definitions DEFINITIONS
                        The file containing the custom protobuf definitions.
  -na, --no-autodetect  Don't try to autodetect if it's a raw HTTP request.
  -s, --stdin           Parse from stdin.
  -b, --base64          If this is a base64 input, so it automatically decodes it.
  -bi, --bruteforce-index NUMBER
                        The index up to which to try bruteforce to find Protobuf content. Default : 20
  -he, --hide-empty     Hide the empty values.
  -np, --no-print       Don't print the decoded protobuf.
  -m, --match MASK      Match anything with the given string. You can use '?' and '*' to wildcard match.Ex : "*token*"
  -mk, --match-keychain MASK
                        Match keychains with the given string.
  -mv, --match-value MASK
                        Match values with the given string.
  -f, --filter MASK     Filter anything with the given string. You can use '?' and '*' to wildcard match.
  -fk, --filter-keychain MASK
                        Filter keychains with the given string.
  -fv, --filter-value MASK
                        Filter values with the given string.
  -epf, --export-protofile [PROTOFILE_FILENAME]
                        Export the proto file with the definitions.
  -epd, --export-protodeep [PROTODEEP_FILENAME]
                        Export a protodeep file, to reuse in ProtoDeep.
  -c, --compile [PYTHON_FILENAME]
                        Compile protobuf into a Python file.
  -n, --name SCHEMA_NAME
                        Name of the schema when exporting into a proto file.ConceptsHere are the main concepts to know when using ProtoDeep :Output exampleKeychains : Since the protobuf is made of nested keys and values, keychains are a way to precisely identify a value in the decoded protobuf. It's the key sequence used to access the value.Pretty Keychains : This is the same as keychains, except that the keys are replaced by the names defined in the custom definitions.Type : Type of the value. Supported types are listed in thisblackboxprotobuf's file.Value : The value found in the protobuf data.Iterator : ProtoDeep will try to autodetect repeated messages, and will print elements of these arrays with thei<position>key, so you can know the position of the element in the list.Custom definitions fileDefinitions : It is a JSON file, containg a dict with the keychains as keys, and names as values. You can specify the type of a value by adding:<type>next to it. By doing so, ProtoDeep will detect it, and relaunch the decoding of the protobuf data with this new type.Note that it will only work when using protobuf data, not a protodeep file, since data has already been decoded.Have fun 🥰💞🧑‍💻 DevelopersTo use ProtoDeep as a lib, you can't use pipx because it uses a venv.So you should install ProtoDeep with pip :$pip3installprotodeepAnd now, you should be able toimport protodeepin your projects like this :fromprotodeep.libimportguess_schemawithopen('protobuf_data.bin','rb')asf:raw=f.read()protodeep_schema=guess_schema(data=raw)protodeep_schema.pretty_print(hide_empty=True,filter_any=["*term_to_filter*"])protodeep_schema.export_protodeep("obj.pdeep")TestingThanks tolearn-more, tests are now available, to test the CLI and lib usage !You can launch the tests by doing :$pip3install-rrequirements-dev.txt
$pytestTests are run automatically throughGitHub Actions.📕 CheatsheetSome examples so you know how to use protodeep :Reading a protobuf file:$protodeepprotobuf_data.bin-tprotobufRead a protobuf file, provide a custom definitions file, hide the output, export to protofile & protodeep, and compile a Python file called "final.py" :$protodeepprotobuf_data.bin-tprotobuf-dsearch_ps_defs.json-np-epf-epd-cfinal.pyNames for the arguments--export-protofile/--export-protodeep/--compileare optional. If they aren't set, a default name will be used.Read protobuf from stdin, provide a custom definitions file, match the keychain "11,1,1,2", hide the empty values, and filter lines where the word "access" and "denied" are present, and lines where the word "tiktok" is present:$curl-s<protobuf_endpoint>|protodeep--stdin-tprotobuf-mk"11,1,1,2"-he-f"*access*denied*"-f"*tiktok*"Matching / filtering arguments can be used as many times as you like.ThanksTheNCC Groupfor the super usefulblackboxprotobufprojectmildsunriseforprotobuf-inspectorThe HideAndSec team 💜 (blog :https://hideandsec.sh)SponsorsThanks to these awesome people for supporting me !You like my work ?Sponsor meon GitHub ! 🤗