Package: hyperframes
HyperFrameThe aim of this project is to provide a high-dimensional analogue to the two-dimensional pandas DataFrame.This allows its user to organise information where the interaction of several factors is of interest.The HyperFrame allows for the easy setting and saving of data for storage, and the fast, interactive creation of two-dimensional pandas DataFrames of any combination of two factors for data exploration.importnumpyasnpimportpandasaspdfromsklearn.neighborsimportKNeighborsClassifierfromsklearn.linear_modelimportLogisticRegressionfromsklearn.ensembleimportRandomForestClassifierfromhyperframeimportHyperFramefromsklearn.model_selectionimporttrain_test_splitfromdemo.helpersimportmetrics,X,yX_train,X_test,y_train,y_test=\train_test_split(X,y,test_size=0.33,random_state=42)clf=KNeighborsClassifier()clf.fit(X_train,y_train)KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
                     metric_params=None, n_jobs=None, n_neighbors=5, p=2,
                     weights='uniform')Initialisationdimension_labels=["train_test","species","metric"]index_labels={"train_test":["train","test"],"species":["setosa","versicolor","virginica"],"metric":["precision","recall","f1"]}scores=HyperFrame(dimension_labels,index_labels)Setting datayhat=clf.predict(X_train)#iset alternative 1scores.iset(metrics(y_train,yhat),"train","","")<hyperframe.HyperFrame at 0x7ff4d4241320>yhat=clf.predict(X_test)#iset alternative 2scores.iset(metrics(y_test,yhat),train_test="test")<hyperframe.HyperFrame at 0x7ff4d4241320>Getting data#iget alternative 1scores.iget("train","","",return_type="pandas").round(2)precisionrecallf1setosa0.891.000.94versicolor0.710.710.71virginica0.800.710.75#iget alternative 2scores.iget(species="versicolor",return_type="pandas").round(2)precisionrecallf1train0.710.710.71test0.700.470.56#iget alternative 3scores.iget0("species","train_test",return_type="pandas").round(2){'metric': 'precision'}setosaversicolorvirginicatrain0.890.710.80test0.950.700.65Initialising a second HyperFramescores_lr=HyperFrame(dimension_labels,index_labels)clf=LogisticRegression(penalty="none",max_iter=1000)clf.fit(X_train,y_train)yhat=clf.predict(X_train)scores_lr.iset(metrics(y_train,yhat),"train","","")yhat=clf.predict(X_test)scores_lr.iset(metrics(y_test,yhat),"test","","")<hyperframe.HyperFrame at 0x7ff4d4231588>Mergingprint("scores shape:{}".format(scores.shape))print("scores_lr shape:{}".format(scores_lr.shape))scores shape: (2, 3, 3)
scores_lr shape: (2, 3, 3)scores_models=scores.merge(scores_lr,"model",["knn","logistic regression"])scores_models.iget("test","","f1","",return_type="pandas").round(2)knnlogistic regressionsetosa0.970.95versicolor0.560.58virginica0.720.72scores_models.iget("","","f1","logistic regression",return_type="pandas").round(2)setosaversicolorvirginicatrain0.920.740.78test0.950.580.72Initialising a third HyperFramescores_rf=HyperFrame(dimension_labels,index_labels)clf=RandomForestClassifier()clf.fit(X_train,y_train)yhat=clf.predict(X_train)scores_rf.iset(metrics(y_train,yhat),"train","","")yhat=clf.predict(X_test)scores_rf.iset(metrics(y_test,yhat),"test","","")<hyperframe.HyperFrame at 0x7ff4d41cd978>scores_rf.iget("test","","",return_type="pandas").round(2)precisionrecallf1setosa0.950.950.95versicolor0.750.400.52virginica0.610.880.72Expanding A DataFrameprint("scores_models shape:{}".format(scores_models.shape))print("scores_rf shape:{}".format(scores_rf.shape))scores_models shape: (2, 3, 3, 2)
scores_rf shape: (2, 3, 3)scores_models=scores_models.expand(scores_rf,"model","random forest")scores_models.iget("test","","f1","",return_type="pandas").round(2)knnlogistic regressionrandom forestsetosa0.970.950.95versicolor0.560.580.52virginica0.720.720.72Simple Mathematical Operationsscores.max("train_test").iget("","",return_type="pandas")precisionrecallf1setosa0.9500001.0000000.974359versicolor0.7142860.7142860.714286virginica0.8000000.8125000.750000scores.min("train_test","metric").iget("",return_type="pandas")setosa        0.885714
versicolor    0.466667
virginica     0.650000
dtype: float64scores.mean("train_test","species","metric")0.7810886435641339scores.sum()14.059595584154408Writing to filescores_models.write_file("./demo/scores_models")Reading from filescores_models=scores_models.read_file("./demo/scores_models")