Package: epoch_analyzer
A python module to analyze numbers to determine on which epoch they might be based and in what unit they might be expressed.InstallationInstall using pip:pip install epoch_analyzerfor windows:py-mpip install epoch_analyzerAlternatively, you can download or clone this repo and callpip install-e..Usage in PythonImport the EpochTester from the package epoch_analyzerfrom epoch_analyzer import EpochTesterIf you desire you can pass a min_date and max_date which are used for reference.Given the resulting object you can call test or convert, for either testing numbers for a probable format or (batch) converting lists of numbers to dates.Usage EpochTester.testYou can pass one number, an iterable list of numbers (for example a counter) to this method.For each number one test conversion is performed, it then check which format matches most of the values, a counter object is returned. By calling most_common on that object, the items can be iterated in order starting at the most likely options.fromepoch_analyzerimportEpochTestertester=EpochTester()results=tester.test({12345435,2999663906})formatch,occurrenceinresults.most_common():print("\t%s:\t%d%%"%(match,occurrence))Usage EpochTester.convertIf you have an unknown type and want conversion to all plausible options you can use this method. If you already know the format its better to fetch a convertor and use that, see alse the option described below.fromepoch_analyzerimportEpochTestertester=EpochTester()results=tester.convert({12345435,2999663906})forinput,matchesinresults.items():print("%d: #%dmatches"%(input,len(matches)))forlabel,resultinmatches:print("\t%s:\t%s%%"%(label,result))Output:2999663906:#1 matches4-Bytesbit-basedtimestampsince1970:2014-11-0519:52:34%12345435:#0 matchesUsage for specific conversionsIf you know with which epoch you are working and you are converting single numbers than the conversion is quite simple. In that case you don’t have to use the convert method but you just request the specific convertor and use that, see the following example.fromepoch_analyzerimportEpochTesterconvertor=EpochTester().get_convertor('4-Bytes bit-based timestamp since 1970')print(convertor.convert_to_date(2999663906))# prints '2014-11-05 19:52:34'Usage from the command lineWhen the module is correctly installed, the commandepochshould be available from your path. There are a lot of options, which are listed when calling the command without any arguments.Some examples:Just convert a single value:epoch 1394543556Output:Forinput1394543556:NumberOfSecondsSinceUnixEpoch(2014-03-1113:12:36)Convert a list of timestamp from a file and output them to a file as unix timestamp.epoch-finput.txt-u>output-timestamps.txtMake a summary of the matches from a list of timestamps from a file.epoch-finput.txt--summaryOutput:Summaryfor5inputs:1.NumberOfSecondsSinceUnixEpoch:40%2.NumberOfMinutesSinceMacOSXEpoch:40%3.NumberOfMiliSecondsSinceUnixEpoch:20%Supply a minimum (which is in this case out-of-range for this unixtime):epoch--min2014-12-011394543556Output:Forinput1394543556:NomatchingpatternwasfoundA space separated hexadecimal input is supported:epoch "aa bb"--hexOutput (Note that big and little endian are both tested):Forinput48042:NomatchingpatternwasfoundForinput43707:NomatchingpatternwasfoundUsage from the command line: scanning binary filesWhen the module is correctly installed (note that termcolor is a prerequisite), the commandepoch_scanshould be available from your path. There are a lot of options, which are listed when calling the command without any arguments.The table width is required to make columns and spot patterns. If your data has a fixed table width its most of the time easy to detect by just resizing your hex editor until you see clear columns of repeating or similair data being displayed. Compare the following two examples:ª..!B....tI.v..:r.#..gd.....l...._..2..9D.a..T..d...ª..!B ........
.wª..!B....tI.w..9r.#..gd.....l...._..2..9D.a..T..d...ª..!B ......
...wª..!B....tI.w..9r.#..gd.....l...._..2..9D.a..T..d...ª..!B ....
.....wª..!B....tI.w..9q."..id.....l...._..2..9D.a..T..d...ª..!B ..
.......wª..!B....tI.w..9q."..id.....l...._..2..9D.a..T..d...ª..!B
.........wª..!B....tI.v..:r.#..gd.....l...._..2..9D.a..T..d...ª..!B .........w
ª..!B....tI.w..9r.#..gd.....l...._..2..9D.a..T..d...ª..!B .........w
ª..!B....tI.w..9r.#..gd.....l...._..2..9D.a..T..d...ª..!B .........w
ª..!B....tI.w..9q."..id.....l...._..2..9D.a..T..d...ª..!B .........w
ª..!B....tI.w..9q."..id.....l...._..2..9D.a..T..d...ª..!B .........wSome examples:Scan for epoch values in the file above:epoch_scan-t44 raw.logOutput:Samplepickedfromoffset:4440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(bigend.)22282752=>1970-05-1000:08:004-Bytesbit-basedtimestampsince1970[0.900000]440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(littleend.)1677722196=>None4-Bytesbit-basedtimestampsince1970[0.900000]440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(littleend.)-1441202176=>2012-08-1216:00:004-Bytesbit-basedtimestampsince1970[0.900000]440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(littleend.)1109458944=>1986-08-1616:00:004-Bytesbit-basedtimestampsince1970[0.900000]440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(bigend.)-1441791967=>2012-08-0800:00:334-Bytesbit-basedtimestampsince1970[0.900000]440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(littleend.)1109458960=>1986-08-1616:00:164-Bytesbit-basedtimestampsince1970[0.900000]440361020154020064000019AA000021422000050000000000000077AA100021421000010774490177000539(littleend.)117506064=>None4-Bytesbit-basedtimestampsince1970[0.900000]The outputs shows a random sample from the file. Each match is highlighted (blue for big endian, green for little endian). The bytes are shown in capital hex and displayed in groups of 4 bytes. To aid the interpretation of the results, both the numeric value and the converted value are shown for the matching format.Other options which might be helpfull are:using a fixed sample -sdefining a –min or –max if you expect a specific periodlimit the number of items to process -c