Package: xgboost2sql
XGBoost模型转sql语句工具包现在是大数据量的时代，我们开发的模型要应用在特别大的待预测集上，使用单机的python，需要预测2、3天，甚至更久，中途很有可能中断。因此需要通过分布式的方式来预测。这个工具包就是实现了将训练好的python模型，转换成sql语句。将生成的sql语句可以放到大数据环境中进行分布式执行预测，能比单机的python预测快好几个量级思想碰撞微信微信公众号干饭人魔都数据干饭人仓库地址：https://github.com/ZhengRyan/xgboost2sql微信公众号文章：https://mp.weixin.qq.com/s/z3IjzMFKP7iEoag5KP6nAApipy包：https://pypi.org/project/xgboost2sql/环境准备可以不用单独创建虚拟环境，因为对包的依赖没有版本要求xgboost2sql安装pip install（pip安装）pipinstallxgboost2sql# to installpipinstall-Uxgboost2sql# to upgradeSource code install（源码安装）pythonsetup.pyinstall运行样例###【注意：：：核验对比python模型预测出来的结果和sql语句预测出来的结果是否一致请查看教程代码】"https://github.com/ZhengRyan/xgboost2sql/examples/tutorial_code.ipynb"导入相关依赖importxgboostasxgbfromsklearn.datasetsimportmake_classificationfromsklearn.model_selectionimporttrain_test_splitfromxgboost2sqlimportXGBoost2Sql训练1个xgboost二分类模型X,y=make_classification(n_samples=10000,n_features=10,n_informative=3,n_redundant=2,n_repeated=0,n_classes=2,weights=[0.7,0.3],flip_y=0.1,random_state=1024)X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=1024)###训练模型model=xgb.XGBClassifier(n_estimators=3)model.fit(X_train,y_train)#xgb.to_graphviz(model)使用xgboost2sql工具包将模型转换成的sql语句xgb2sql=XGBoost2Sql()sql_str=xgb2sql.transform(model)将sql语句保存xgb2sql.save()将sql语句打印出来print(sql_str)selectkey,1/(1+exp(-(tree_1_score+tree_2_score+tree_3_score)+(-0.0)))asscorefrom(selectkey,--tree1casewhen(f9<-1.64164519orf9isnull)thencasewhen(f3<-4.19117069orf3isnull)thencasewhen(f2<-1.31743848orf2isnull)then-0.150000006else-0.544186056endelsecasewhen(f3<1.23432565orf3isnull)thencasewhen(f7<-2.55682254orf7isnull)then-0.200000018elsecasewhen(f5<0.154983491orf5isnull)then0.544721723elsecasewhen(f3<0.697217584orf3isnull)then-0.150000006else0.333333373endendendelsecasewhen(f5<-1.0218116orf5isnull)thencasewhen(f0<-0.60882163orf0isnull)thencasewhen(f2<0.26019755orf2isnull)then0.0666666701else-0.300000012endelse-0.520000041endelse0.333333373endendendelsecasewhen(f9<1.60392439orf9isnull)thencasewhen(f3<0.572542191orf3isnull)thencasewhen(f5<0.653370142orf5isnull)thencasewhen(f7<-0.765973091orf7isnull)thencasewhen(f3<-0.432390809orf3isnull)then0.204000011else-0.485454559endelsecasewhen(f3<-1.20459461orf3isnull)then-0.5104478else0.441509455endendelsecasewhen(f7<0.133017987orf7isnull)thencasewhen(f8<0.320554674orf8isnull)then-0.290322572else0.368339777endelsecasewhen(f8<-0.211985052orf8isnull)then0.504000008else-0.525648415endendendelsecasewhen(f7<2.22314501orf7isnull)thencasewhen(f8<-0.00532855326orf8isnull)thencasewhen(f8<-0.204920739orf8isnull)then-0.533991575else-0.200000018endelse0.428571463endelsecasewhen(f3<1.33772755orf3isnull)thencasewhen(f0<-0.975171864orf0isnull)then0.163636371else0.51818186endelse-0endendendelsecasewhen(f3<1.77943277orf3isnull)thencasewhen(f7<-0.469875157orf7isnull)thencasewhen(f3<-0.536645889orf3isnull)thencasewhen(f9<1.89841866orf9isnull)then-0else0.333333373endelsecasewhen(f4<-2.43660188orf4isnull)then0.150000006else0.551020443endendelsecasewhen(f1<-0.0788691565orf1isnull)then0.150000006else-0.375endendelsecasewhen(f4<-1.73232496orf4isnull)then-0.150000006elsecasewhen(f6<-1.6080606orf6isnull)then-0.150000006elsecasewhen(f7<-0.259483218orf7isnull)then-0.558620751else-0.300000012endendendendendendastree_1_score,--tree2casewhen(f9<-1.64164519orf9isnull)thencasewhen(f3<-4.19117069orf3isnull)thencasewhen(f0<0.942570388orf0isnull)then-0.432453066else-0.128291652endelsecasewhen(f3<1.23432565orf3isnull)thencasewhen(f7<-2.55682254orf7isnull)then-0.167702854elsecasewhen(f5<0.154983491orf5isnull)thencasewhen(f1<2.19985676orf1isnull)then0.41752997else0.115944751endelse0.115584135endendelsecasewhen(f5<-1.0218116orf5isnull)thencasewhen(f0<-0.60882163orf0isnull)then-0.119530827else-0.410788596endelse0.28256765endendendelsecasewhen(f9<1.60392439orf9isnull)thencasewhen(f3<0.460727394orf3isnull)thencasewhen(f5<0.653370142orf5isnull)thencasewhen(f7<-0.933565617orf7isnull)thencasewhen(f3<-0.572475374orf3isnull)then0.182491601else-0.377898693endelsecasewhen(f3<-1.20459461orf3isnull)then-0.392539263else0.352721155endendelsecasewhen(f7<0.207098693orf7isnull)thencasewhen(f8<0.498489976orf8isnull)then-0.193351224else0.29298231endelsecasewhen(f8<-0.117464997orf8isnull)then0.400667101else-0.402199954endendendelsecasewhen(f7<1.98268723orf7isnull)thencasewhen(f8<-0.00532855326orf8isnull)thencasewhen(f7<1.36281848orf7isnull)then-0.408002198else-0.236123681endelsecasewhen(f5<1.14038813orf5isnull)then0.404326111else-0.110877581endendelsecasewhen(f3<1.56952488orf3isnull)thencasewhen(f5<2.14646816orf5isnull)then0.409404457else0.0696995854endelse-0.32059738endendendelsecasewhen(f3<1.77943277orf3isnull)thencasewhen(f7<-0.469875157orf7isnull)thencasewhen(f3<-0.536645889orf3isnull)thencasewhen(f9<1.89841866orf9isnull)then-0else0.28256765endelse0.419863999endelsecasewhen(f3<0.444227457orf3isnull)then-0.34664312else0.0693304539endendelsecasewhen(f4<-1.10089087orf4isnull)thencasewhen(f3<2.3550868orf3isnull)then0.0147894565else-0.331404865endelse-0.421277165endendendendastree_2_score,--tree3casewhen(f9<-1.64164519orf9isnull)thencasewhen(f3<-4.19117069orf3isnull)thencasewhen(f4<-1.30126143orf4isnull)then-0.0772174299else-0.374165356endelsecasewhen(f3<1.23432565orf3isnull)thencasewhen(f7<-2.55682254orf7isnull)then-0.142005175elsecasewhen(f5<0.154983491orf5isnull)thencasewhen(f7<3.59379435orf7isnull)then0.352122813else0.132789165endelse0.0924336985endendelsecasewhen(f5<-1.0218116orf5isnull)thencasewhen(f0<-0.60882163orf0isnull)then-0.0954768136else-0.351594836endelse0.245992288endendendelsecasewhen(f9<1.60392439orf9isnull)thencasewhen(f3<0.347133756orf3isnull)thencasewhen(f5<0.661561131orf5isnull)thencasewhen(f7<-0.933565617orf7isnull)thencasewhen(f3<-0.472413659orf3isnull)then0.116336405else-0.313245147endelsecasewhen(f3<-1.5402329orf3isnull)then-0.352897167else0.311400592endendelsecasewhen(f7<0.275665522orf7isnull)thencasewhen(f8<0.403402805orf8isnull)then-0.292606086else0.220064178endelsecasewhen(f8<-0.0442957953orf8isnull)then0.350784421else-0.336107522endendendelsecasewhen(f7<1.77503061orf7isnull)thencasewhen(f8<0.196157426orf8isnull)thencasewhen(f7<1.36281848orf7isnull)then-0.3376683else-0.0711223111endelsecasewhen(f7<-0.661211252orf7isnull)then0.434363276else-0.219307661endendelsecasewhen(f3<1.37940335orf3isnull)thencasewhen(f6<1.34894884orf6isnull)then0.367155522else0.124757253endelse-0.293739736endendendelsecasewhen(f3<1.77943277orf3isnull)thencasewhen(f7<-0.469875157orf7isnull)thencasewhen(f3<-0.536645889orf3isnull)thencasewhen(f9<1.89841866orf9isnull)then-0else0.245992288endelsecasewhen(f0<1.60565615orf0isnull)then0.357973605else0.193993196endendelsecasewhen(f9<1.89456153orf9isnull)then-0.276471078else0.111896731endendelsecasewhen(f1<1.35706067orf1isnull)thencasewhen(f4<-1.10089087orf4isnull)thencasewhen(f3<2.3550868orf3isnull)then0.0119848112else-0.284813672endelse-0.376859784endelsecasewhen(f2<-0.25748384orf2isnull)then0.0723158419else-0.253415495endendendendendastree_3_scorefromdata_table)