Package: metext
MetExt : Metadata ExtractorMetExt (Metadata Extractor) is a Python library and CLI utility that allows to process data in different formats and extract data patterns of interest.InstallationMetExt requires Python v3.5+.You can install MetExt from PyPi.$ python -m pip install metextOr you can clone the repository install the project locally.$ git clone https://github.com/espoem/MetExt.git
$ cd MetExt
$ python -m pip install -e .Dependencies can also be installed from therequirements.txtfile.$ python -m pip install -r requirements.txtCLIHelpmetext --helpUsagemetext [-h] [-i INPUT] [-f FILE] [-r] [-o OUTPUT]
        [-d [{quopri,yenc,base58ripple,base64url,pem,base58btc,base85,hexdump,base91,binhex,percent,ascii85,base64,mime,base32,hex,z85,raw,base32crockford,gzip,uu,base32hex} [{quopri,yenc,base58ripple,base64url,pem,base58btc,base85,hexdump,base91,binhex,percent,
  ascii85,base64,mime,base32,hex,z85,raw,base32crockford,gzip,uu,base32hex} ...]]]
        [-e [{bch,xrp,sha256,bip32-xkey,url,ipv4,pem,sha512,ltc,email,chainlink,md5,uuid,btc-wif,sha1,urn,issn,eth,isbn,json,base64,ada,usdt,base32,doi,magnet,sha224,ipv6,uri,data_uri,btc,mac,dot,sha384} [{bch,xrp,sha256,bip32-xkey,url,ipv4,pem,sha512,ltc,email,
  chainlink,md5,uuid,btc-wif,sha1,urn,issn,eth,isbn,json,base64,ada,usdt,base32,doi,magnet,sha224,ipv6,uri,data_uri,btc,mac,dot,sha384} ...]]]
        [-F {text,csv,json}] [--version]Input paths support wildcards. If no input is provided, standard input is processed, i.e. input can be piped into metext. MetExt process input data in best effort, i.e. it tries to decode the data if encoded. The range of decoding to consider can be set via the argument-d. The list of patterns to search for is set with the argument-e. If not set, all patterns are searched for.If no decoding is wanted to apply, set-d raw.APIList supported modulesfrommetextimportlist_decoders_names,list_extractors_names,list_printers_namesif__name__=="__main__":print(list_decoders_names())print(list_extractors_names())print(list_printers_names())Analyse data for patternsfrommetextimportanalyse,print_analysis_outputif__name__=="__main__":# file-like objectwithopen("filepath","rb")asfp:result=analyse(fp,"raw")# without decoding, find all supported patternsprint(result)print_analysis_output(result)# by default prints to STDOUT in JSON formatprint_analysis_output(result,"output_filepath","csv")# prints to "output_filepath" in CSV format# input is list of filepaths# consider either no, or base64 encoding# extract ipv4 patternsresult=analyse(["filepath"],["raw","base64"],"ipv4")# input is string# limit decoding to provided decoders with custom kwargs# limit patterns to extractors with custom kwargsinput_data="some string"result=analyse(input_data,[("base64",{"charset":"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz+/"},)],[("base64",{})],)# input is bytes# decode data in best effort# extract all patternsinput_data="some byte string"result=analyse(input_data)ModulesThe tool's functionality is divided into individual modules (plugins) of a particular type. A new module can be created by inheriting from one of plugins' bases inplugin_base.pyfile. A plugin must have defined a unique pair PLUGIN_TYPE, PLUGIN_NAME.Supported decodersDecoderDescriptionhexHexadecimal values, optionally with delimiters or prefixeshexdumpHexdump format produced by toolshexdumpandxddbase32base32encoding as defined in RFC 4648base32hexbase32hexencoding as defined in RFC 4648base32crockfordCrockford's Base 32 encodingbase58btcBase 58 encoding using Bitcoin alphabetbase58rippleBase 58 encoding using Ripple alphabetbase64base64encoding as defined in RFC 4648.base64urlbase64urlencoding as defined in RFC 4648base85base85encoding as defined in RFC 1924ascii85Ascii85 encodingz85Z85 encodingbase91Base 91 encodingbinhexBinHex 4.0 encodingpemDecodes Base 64 data in PEM formatgzipDecompress GZIP datamimeDecodes MIME message bodypercentURL-encoding as defined in RFC 3986quopriQuoted-printable encoding as defined in RFC 2045uuUUEncodingyencyEnc encodingSupported pattern extractorsExtractorDescriptionbase32base32-encoded database64base64-encoded databtcBitcoin P2PKH, P2SH and Bech32 mainnet addressesbtc-wifBitcoin private key in wallet import format (WIF)bip32-keyExtended private and public keys as defined in BIP32ethEthereum addressltcLitecoin legacy and Bech32 addressxrpXRP (Ripple) classic addressusdtTether (Omni, Ethereum) addressbchBitcoin Cash legacy and CashAddr addresschainlinkChainlink (Ethereum) addressadaCardano addressdotPolkadot addressemailE-mail addressuuidUniversally unique identifier (UUID, GUID)md5MD5 hex digestsha1SHA-1 hex digestsha224SHA-224 hex digestsha256SHA-256 hex digestsha384SHA-384 hex digestsha512SHA-512 hex digestipv4IPv4 addressipv6IPv6 addressisbnISBN identifierissnISSN identifierdoiDOI identifierjsonJSON non-empty object or arraymacMAC addresspemData in PEM formaturiGeneric URI with a registered schemeurlURL with http(s) or ftp schemeurnURNdata_uriData URImagnetBitTorrent magnet link