Package: en-ems
En-EMS |Entropy-based Ensemble Members Selectionen-emsis a Python library for the selection of a set of mutually exclusive, collectivelly exaustive (MECE) ensemble members.The library implements the approach presented byDarbandsari and Coulibaly (2020)as step that antecedes the further merging of a set of ensemble forecasts.Theen-emspackage is built over thepyitlibpackage, which implements fundamental information theory methods.InstallingThe library can be installed using the traditional pip:pipinstallen-emsAnd is listed on the Python Package Index (pypi) asen-ems.UsingSuppose you have a file namedexample.csvwith the following content:Date,       Memb_A, Memb_B, ...,  Memb_Z, Obsv
2020/05/15, 1.12,   1.05,   ...,  0.5,    1.01
2020/05/16, 1.15,   1.12,   ...,  0.9,    1.10
2020/05/17, 1.13,   1.32,   ...,  1.1,    1.29
...         ...     ...     ...,  ...,    ...
2020/11/30, 1.22,   0.95,   ...,  0.3,    0.87In which the columns starting with "Memb_" hold the realization of one ensemble member for the time interval and "Obsv" holds the observed values for the same time interval.If your our objective is to select a MECE set considering obaservations, it can be done using the standard parameters by:importpandasaspdimportenems# read filedata_ensemble=pd.read_csv("example.csv").to_dict('list')data_obsv=data_ensemble["Obsv"]deldata_ensemble["Obsv"],data_ensemble["Date"]# perform selectionselection_log=enems.select_ensemble_members(data_ensemble,data_obsv)The variableselection_logwill be a dictionary containing a log of thetotal correlation,joint antropyand (if an observation was given) thetransinformationof the given and selected datasets. It also contains, as expected, the ids of the selected ensemble members.Example 1: No observation data availableMock data for a dataset with 75 supposed ensemble members and without observation records can be obtained with the functionenems.load_data_75().Here is a full example on how we can access the mock data, select a MECE subset and visualize the results using the popularmatplotlibis given:importmatplotlib.pyplotaspltimportenemsif__name__=="__main__":# ## LOAD DATA ################################################################################################### #test_data_df=enems.load_data_75()test_data=test_data_df.to_dict("list")# ## SELECT MECE SUBSET ########################################################################################## #selection_log=enems.select_ensemble_members(test_data,None,n_bins=10,bin_by="equal_intervals",beta_threshold=0.95,n_processes=1,verbose=False)# ## PLOT FUNCTIONS ############################################################################################## #defplot_ensemble_members(all_series:dict,selected_series:set,plot_title:str,output_file_path:str)->None:_,axs=plt.subplots(1,1,figsize=(7,2.5))axs.set_xlabel("Time")axs.set_ylabel("Value")axs.set_title(plot_title)axs.set_xlim(0,143)axs.set_ylim(0,5)[axs.plot(all_series[series_id],color="#999999",zorder=3,alpha=0.33)forseries_idinselected_series]plt.tight_layout()plt.savefig(output_file_path)plt.close()returnNonedefplot_log(n_total_members:int,log:dict,output_file_path:str)->None:_,axss=plt.subplots(1,2,figsize=(7.0,2.5))x_values=[n_total_members-i-1foriinrange(len(log["history"]["total_correlation"]))]axss[0].set_xlabel("Time")axss[0].set_ylabel("Total correlation")axss[0].plot(x_values,log["history"]["total_correlation"],color="#7777FF",zorder=3)axss[0].set_ylim(70,140)axss[0].set_xlim(x_values[0],x_values[-1])axss[1].set_xlabel("Time")axss[1].set_ylabel("Joint entropy")axss[1].axhline(log["original_ensemble_joint_entropy"],color="#FF7777",zorder=3,label="Full set")axss[1].plot(x_values,log["history"]["joint_entropy"],color="#7777FF",zorder=3,label="Selected set")axss[1].set_ylim(6.3,6.9)axss[1].set_xlim(x_values[0],x_values[-1])axss[1].legend()plt.tight_layout()plt.savefig(output_file_path)plt.close()returnNone# ## FUNCTIONS CALL ############################################################################################## #plot_log(len(test_data.keys()),selection_log,"test/log.svg")plot_ensemble_members(test_data,set(test_data.keys()),"All members (%d)"%len(test_data.keys()),"test/ensemble_all.svg")plot_ensemble_members(test_data,selection_log["selected_members"],"Selected members (%d)"%len(selection_log["selected_members"]),"test/ensemble_selected.svg")Which would give us the following plot:log.svgensemble_all.svgensemble_selected.svgExample 2:Additional mock observation data compatible with the mock ensemble members is distributed with the package. It can be accessed using the funcionenems.load_data_obs().An example on how to use it to trigger the full version of the algorithm can is presented:importmatplotlib.pyplotaspltimportnumpyasnpimportenemsif__name__=="__main__":# ## LOAD DATA ################################################################################################### #test_data_obs=enems.load_data_obs().valuestest_data_df=enems.load_data_75()test_data=test_data_df.to_dict("list")# ## PLOT FUNCTIONS ############################################################################################## #defplot_ensemble_members([...]):[...]defplot_log([...]):[...]# ## FUNCTIONS CALL ############################################################################################## #cur_selection_log=enems.select_ensemble_members(test_data,test_data_obs,n_bins=10,bin_by="equal_intervals",beta_threshold=0.95,n_processes=1,verbose=False)plot_log(len(test_data.keys()),cur_selection_log,"test/log_obs.svg")plot_ensemble_members(test_data,test_data_obs,set(test_data.keys()),"All members (%d)"%len(test_data.keys()),"test/ensemble_all_obs.svg")plot_ensemble_members(test_data,test_data_obs,cur_selection_log["selected_members"],"Selected members (%d)"%len(cur_selection_log["selected_members"]),"test/ensemble_selected_obs.svg")deltest_data_obs,cur_selection_logWhich would give us the following plot:log_obs.svgensemble_all_obs.svgensemble_selected_obs.svgFurther documentationFurther information about the library can be found in thedocsfolder of the Git repository of this project.The users are can find the complete theoretical explanation and assessment of the method in the original work ofDarbandsari and Coulibaly (2020).