Package: betfair-data
Betfair DataBetfair Data is a very fast, Rust based, Betfair historical data parsing library for python. It supports both the officialBetfair's historic dataand self recorded stream files.Installationpip install betfair_dataNote: requires Python >= 3.7.Exampleimportbetfair_dataasbfdpaths=["data/2021_12_DecRacingAUPro.tar","data/2021_10_OctRacingAUPro.tar","data/2021_11_NovRacingAUPro.tar",]market_count=0update_count=0forfileinbfd.Files(paths):market_count+=1formarketinfile:update_count+=1print(f"Markets{market_count}Updates{update_count}",end='\r')print(f"Markets{market_count}Updates{update_count}")Loading FilesYou can read in files quickly in a worker thread using the providedFilesutility. It supports reading in bz2, gzip or uncompressed (.json) stream files or .tar or .zip archives containing such files.paths=["data/2021_10_OctRacingAUPro.tar","data/2021_11_NovRacingAUPro.zip","data/self_recorded_market.gz","data/uncompressed_market.json",]files=bfd.Files(paths)Or you can use the glob library to find and select all the paths automatically.importglobpaths=glob.glob("data/betfair_official/*.tar")+glob.glob("data/self_recorded/*.gz")files=bfd.Files(paths)You can also load the file through any other means and pass the raw bytes and name into the File object constructor.# generator to read in filesdefload_files(paths:str):forpathinglob.glob(paths,recursive=True):withopen(path,"rb")asfile:yieldbfd.File(path,file.read())forfileinload_files("markets/*.json"):formarketinfile:passBenchmarksRunning over 3 months of Australian racing data on a 2021 M1 Macbook Pro.betfair_data (mutable)betfair_data (immutable/bflw compat)betfairlightweight lightweight=Truebetfairlightweight lightweight=False5m 12sec6m 50sec1hour 1min 45sec3hours 46mins 39sec~70 markets/sec~53.5 markets/sec~6 markets/sec~1.62 markets/sec~534,200 updates/sec~406,500 updates/sec~45,500 updates/sec~12,250 updates/secTypesIDE's should automatically detect the types and provide checking and auto complete. See thepyi stub filefor a comprehensive view of the types and method available.BetfairlightweightWe also support a format that is a drop in replacement forbetfairlightweightobjects. We have rigorously tested it against betfairlightweight to ensure a complete match of its structure, any differences should be submitted as issues with the corresponding differences and the data used to create them.frombetfair_dataimportbflwfiles=bflw.Files(paths)frombetfair_dataimportbflwfile_bytes=...file=bflw.File("file_name",file_bytes)LoggingLogging can be enabled and warnings are emitted for IO and JSON errors.importlogginglogging.basicConfig(level=logging.WARN,format='%(levelname)s%(name)s%(message)s')Example logged errorsWARNING betfair_data file: data/2021_10_OctRacingAUPro.tar/PRO/2021/Oct/4/30970292/1.188542184.bz2 err: (JSON Parse Error) expected value at line 1480 column 1
WARNING betfair_data file: data/2021_10_OctRacingAUPro.tar/PRO/2021/Oct/8/30985584/1.188739324.bz2 err: (JSON Parse Error) expected `:` at line 1 column 909
WARNING betfair_data file: data/2021_10_OctRacingAUPro.tar/PRO/2021/Oct/8/30985584/1.188739325.bz2 err: (JSON Parse Error) expected `:` at line 1 column 904
WARNING betfair_data file: data/2021_10_OctRacingAUPro.tar/PRO/2021/Oct/15/31001342/1.189124831.bz2 err: (JSON Parse Error) expected value at line 1335 column 1