Package: alpha-factory
This programme is to automatically generate alpha factors and filter
relatively good factors with back-testing methods. Time consuming parts
are optimized withnumbapackage.Dependenciespython >= 3.5pandas >= 0.22.0numpy >= 1.14.0RNWS >= 0.2.1numba >= 0.38.0single_factor_model>=0.3.0IPython 5.1.0empyricalalphalensNote: It is best to use the latest version ofllvmlitein order to
makenumbawork properly. Otherwise it may couse a kernel-dies
situation.Exampleload packages and read in datafromalpha_factoryimportgenerator_class,get_memory_use_pct,cleanfromRNWSimportreadimportnumpyasnpimportpandasaspdstart=20180101end=20180331factor_path='.'frame_path='.'df=pd.read_csv(frame_path+'/frames.csv')## read in datare=read.read_df('./re',file_pattern='re',start=start,end=end)cap=read.read_df('./cap',file_pattern='cap',header=0,dat_col='cap',start=start,end=end)open_price,close,vwap,adj,high,low,volume,sus=read.read_df('./mkt_data',file_pattern='mkt',start=start,end=end,header=0,dat_col=['open','close','vwap','adjfactor','high','low','volume','sus'])ind1,ind2,ind3=read.read_df('./ind',file_pattern='ind',start=start,end=end,header=0,dat_col=['level1','level2','level3'])inx_weight=read.read_df('./ZZ800_weight','Stk_ZZ800',start=start,end=end,header=None,inx_col=1,dat_col=3)Note:framescontains columns as:df_name,equation,dependency,type, wheretypeincludesdf,cap,group. In this caseframes.csvhavedf_name:re,cap,open_price,close,vwap,high,low,volume,ind1,ind2,ind3.You can also read data by usingpd.read_csvdirectly depending on
how you store your data.start to generateparms={'re':close.mul(adj).pct_change(),'cap':cap,'open_price':open_price,'close':close,'vwap':vwap,'high':high,'low':low,'volume':volume,'ind1':ind1,'ind2':ind2,'ind3':ind3}withgenerator_class(df,factor_path,**parms)asgen:gen.generator(batch_size=3,name_start='a')gen.generator(batch_size=3,name_start='a')gen.output_df(path=frame_path+'/frames_new.csv')continue to generate with existing frames and factorswithgenerator_class(df,factor_path,**parms)asgen:gen.reload_df(path=frame_path+'/frames_new.csv')gen.reload_factors(align=True)clean()foriinrange(5):gen.generator(batch_size=2,name_start='a')print('step%dmemory usage:\t%.1f%%\n'%(i,get_memory_use_pct()))ifget_memory_use_pct()>80:breakgen.output_df(path=frame_path+'/frames_new2.csv')Note: It is very important toalignall factors and initial
dataframes before generating.you can also choose how to store your factors by settingstore_methodbacktesting with stratified sampling approach and ic-ir meansure after generationdata_box_param={'ind':ind1,'price':vwap*adjfactor,'sus':sus,'ind_weight':inx_weight,'path':'./databox'}back_test_param={'sharpe_ratio_thresh':3,'n':5,'out_path':'.','back_end':'loky','n_jobs':6,'detail_root_path':None,'double_side_cost':0.003,'rf':0.03}icir_param={'ir_thresh':0.4,'out_path':'.','back_end':'loky','n_jobs':6}withgenerator_class(df,factor_path,**parms)asgen:foriinrange(5):gen.generator(batch_size=2,name_start='a')gen.output_df(path=frame_path+'/frames_new.csv')gen.getOrCreate_databox(**data_box_param)gen.back_test(**back_test_param)gen.icir(**icir_param)clean()ifget_memory_use_pct()>90:print('Memory exceeded')breakTo temporarily save (and reload) factor data you can usecreate_tmp_memoryandreload_tmp_memorymethods. This is usually
used beforeback_testandicirto release more memory for
parallel running.generate script of factorsfromalpha_factoryimportwrite_fileimportpandasaspddf2=pd.read_csv(frame_path+'/frames_new.csv')write_file(df2,'script.py')locate a factorfromalpha_factory.utiliseimportget_factor_pathfactor_name='a0'path=get_factor_path(factor_path,factor_name)only whenstorage_method='byTime'use your own functionsTo use your own functions you need to append your code in classfunctionsfrombasic_functions.pyin the sourse file and also
append the corresponding names infunctions.csvfromdatafile
in the sourse file.After that you can setdebug=Trueingeneratorfunction to check
if there is any bug from all those functions. If indeed there is, a new
embeded ipython would be activated to help you find out what is going on
in the loop.