Package: numerblox
NumerBloxNumerBlox offers components that help with developing strong Numerai models and inference pipelines. From downloading data to submitting predictions, NumerBlox has you covered.All components can be used standalone and all processors are fully compatible to use withinscikit-learnpipelines.Documentation:crowdcent.github.io/numerblox1. InstallationInstall numerblox from PyPi by running:pip install numerbloxAlternatively you can clone this repository and install it in
development mode by installing usingpoetry:gitclonehttps://github.com/crowdcent/numerblox.git
pipinstallpoetrycdnumerblox
poetryinstallInstallation without dev dependencies can be done by adding--only mainto thepoetry installline.Test your installation using one of the education notebooks inexamples. Good places to start arequickstart.ipynbandnumerframe_tutorial.ipynb. Run it in your
Notebook environment to quickly test if your installation has succeeded.
The documentation contains examples and explanations for each component of NumerBlox.2. Core functionalityNumerBlox has the following features for both Numerai Classic and Signals:Data Download:Automated retrieval of Numerai datasets.NumerFrame:A custom Pandas DataFrame for easier Numerai data manipulation.Preprocessors:Customizable techniques for data preprocessing.Target Engineering:Tools for creating new target variables.Postprocessors:Ensembling, neutralization, and penalization.MetaPipeline:An era-aware pipeline extension of scikit-learn's Pipeline. Specifically designed to integrate with era-specific Postprocessors such as neutralization and ensembling. Can be optionally bypassed for custom implementations.MetaEstimators:Era-aware estimators that extend scikit-learn's functionality. Includes features like CrossValEstimator which allow for era-specific, multiple-folds fitting seamlessly integrated into the pipeline.Evaluation:Comprehensive metrics aligned with Numerai's evaluation criteria.Submitters:Facilitates secure and easy submission of predictions.Example notebooks for each of these components can be found in theexamples. Also check outthe documentationfor more information.3. Quick StartBelow are two examples of how NumerBlox can be used to train and do inference on Numerai data. For a full overview of all components check out the documentation. More advanced examples to leverage NumerBlox to the fullest can be found in theEnd-To-End Example section.3.1 Simple exampleThe example below shows how NumerBlox simplifies training and inference on an XGBoost model.
NumerBlox is used here for easy downloading, data parsing, evaluation, inference and submission. You can experiment with this setup yourself in the example notebookquickstart.ipynb.importpandasaspdfromxgboostimportXGBRegressorfromnumerblox.miscimportKeyfromnumerblox.numerframeimportcreate_numerframefromnumerblox.downloadimportNumeraiClassicDownloaderfromnumerblox.prediction_loadersimportExamplePredictionsfromnumerblox.evaluationimportNumeraiClassicEvaluatorfromnumerblox.submissionimportNumeraiClassicSubmitter# Download datadownloader=NumeraiClassicDownloader("data")# Training and validation datadownloader.download_training_data("train_val",version="4.3")df=create_numerframe("data/train_val/train_int8.parquet")# TrainX,y=df.get_feature_target_pair(multi_target=False)xgb=XGBRegressor()xgb.fit(X.values,y.values)# Evaluateval_df=create_numerframe("data/train_val/validation_int8.parquet")val_df['prediction']=xgb.predict(val_df.get_feature_data)val_df['example_preds']=ExamplePredictions("v4.3/validation_example_preds.parquet").fit_transform(None)['prediction'].valuesevaluator=NumeraiClassicEvaluator()metrics=evaluator.full_evaluation(val_df,example_col="example_preds",pred_cols=["prediction"],target_col="target")# Inferencedownloader.download_live_data("current_round",version="4.3")live_df=create_numerframe(file_path="data/current_round/live_int8.parquet")live_X,live_y=live_df.get_feature_target_pair(multi_target=False)preds=xgb.predict(live_X)# SubmitNUMERAI_PUBLIC_ID="YOUR_PUBLIC_ID"NUMERAI_SECRET_KEY="YOUR_SECRET_KEY"key=Key(pub_id=NUMERAI_PUBLIC_ID,secret_key=NUMERAI_SECRET_KEY)submitter=NumeraiClassicSubmitter(directory_path="sub_current_round",key=key)# Your prediction file with 'id' as index and defined 'cols' below.pred_dataf=pd.DataFrame(preds,index=live_df.index,columns=["prediction"])# Only works with valid key credentials and model_namesubmitter.full_submission(dataf=pred_dataf,cols="prediction",file_name="submission.csv",model_name="MY_MODEL_NAME")3.2. Advanced NumerBlox modelingThis example showcases how you can really push NumerBlox to create powerful pipelines. This pipeline approaches the Numerai Classic data as a classification problem. It fits multiple cross validation folds, reduces the classification probabilties to single values and create a weighted ensemble of these where the most recent folds get a higher weight. Lastly, the predictions are neutralized. The model is evaluated in validation data, inference is done on live data and a submission is done.
Lastly, we remove the download and submission directories to clean up the environment. This is especially convenient if you are running daily inference on your own server or a cloud VM.fromxgboostimportXGBClassifierfromsklearn.treeimportDecisionTreeClassifierfromsklearn.model_selectionimportTimeSeriesSplitfromnumerblox.metaimportCrossValEstimator,make_meta_pipelinefromnumerblox.prediction_loadersimportExamplePredictionsfromnumerblox.ensembleimportNumeraiEnsemble,PredictionReducerfromnumerblox.neutralizersimportFeatureNeutralizer# Download datadownloader=NumeraiClassicDownloader("data")# Training and validation datadownloader.download_training_data("train_val",version="4.3")df=create_numerframe("data/train_val/train_int8.parquet")# Setup model pipelinemodel=XGBClassifier()crossval=CrossValEstimator(estimator=model,cv=TimeSeriesSplit(n_splits=5),predict_func='predict_proba')pred_rud=PredictionReducer(n_models=5,n_classes=5)ens=NumeraiEnsemble(donate_weighted=True)neut=FeatureNeutralizer(proportion=0.5)full_pipe=make_meta_pipeline(preproc_pipe,crossval,pred_rud,ens,neut)# TrainX,y=df.get_feature_target_pair(multi_target=False)y_int=(y*4).astype(int)eras=df.get_era_datafeatures=df.get_feature_datafull_pipe.fit(X,y_int,numeraiensemble__eras=eras)# Evaluateval_df=create_numerframe("data/train_val/validation_int8.parquet")val_X,_=val_df.get_feature_target_pair(multi_target=False)val_eras=val_df.get_era_dataval_features=val_df.get_feature_dataval_df['prediction']=full_pipe.predict(val_X,eras=val_eras,features=val_features)val_df['example_preds']=ExamplePredictions("v4.3/validation_example_preds.parquet").fit_transform(None)['prediction'].valuesevaluator=NumeraiClassicEvaluator()metrics=evaluator.full_evaluation(val_df,example_col="example_preds",pred_cols=["prediction"],target_col="target")# Inferencedownloader.download_live_data("current_round",version="4.3")live_df=create_numerframe(file_path="data/current_round/live_int8.parquet")live_X,live_y=live_df.get_feature_target_pair(multi_target=False)live_eras=live_df.get_era_datalive_features=live_df.get_feature_datapreds=full_pipe.predict(live_X,eras=live_eras,features=live_features)# SubmitNUMERAI_PUBLIC_ID="YOUR_PUBLIC_ID"NUMERAI_SECRET_KEY="YOUR_SECRET_KEY"key=Key(pub_id=NUMERAI_PUBLIC_ID,secret_key=NUMERAI_SECRET_KEY)submitter=NumeraiClassicSubmitter(directory_path="sub_current_round",key=key)# Your prediction file with 'id' as index and defined 'cols' below.pred_dataf=pd.DataFrame(preds,index=live_df.index,columns=["prediction"])# Only works with valid key credentials and model_namesubmitter.full_submission(dataf=pred_dataf,cols="prediction",file_name="submission.csv",model_name="MY_MODEL_NAME")# Clean up environmentdownloader.remove_base_directory()submitter.remove_base_directory()4. ContributingBe sure to read theHow To Contribute sectionsection in the documentation for detailed instructions on
contributing.If you have questions or want to discuss new ideas for NumerBlox,
please create a Github issue first.5. Crediting sourcesSome of the components in this library may be based on forum posts,
notebooks or ideas made public by the Numerai community. We have done
our best to ask all parties who posted a specific piece of code for
their permission and credit their work in the documentation. If your
code is used in this library without credits, please let us know, so we
can add a link to your article/code.If you are contributing to NumerBlox and are using ideas posted
earlier by someone else, make sure to credit them by posting a link to
their article/code in documentation.