Package: majora
IntroductionThroughout my data science journey, I have learned that it is a good practice to understand the data first and try to gather as many insights from it. Exploratory Data Analysis (EDA) is all about making sense of data in hand, before getting dirty with machine learning and sophisticated algorithm.While there are plenty of Python libraries that can help create beautiful and complex visualizations, I often find myself starting with the most simplistic analyses: count plot, histogram, scatter plot, boxplot, etc. This initial EDA workflow is very similar for each new data set. But unfortunately, they are tedious. Converting to correct data types, selecting the right variable type for the right plot, iterate through all possible variable combinations, adjusting plot aesthetics and labels, etc. These are the tasks I would love to do... once. As someone that does not find great joy in completing repetitive tasks, I set out to build a tool that allows me to be as lazy as possible.DescriptionMajora is a python library that automates common tasks in your exploratory data analysis. This includes missing values visualization, missing values handling, variable types handling, predictive modeling, and a variety of univariate and bivariate graphs. The goal is to provide a fast and effective tool for discovering insights, so you can quickly move on to the machine learning model.FeaturesSmart data type conversionAutomatic graph discoverySimple missing values identificaiton and handlingCART model with cross-validation and tree visualizationTable of ContentsTable of contentsInstallationDataset OverviewMissing ValuesIdentify Missing ValuesHandle Missing ValuesVariable Types HandlingIdentify data typesType conversionsVisualizationUnivariate plotsHistogramCount PlotsWord CloudBivariate plotsCorrelation plotsPrincipal Component AnalysisBox PlotsRelative Frequency PlotsCorrespondence AnalysisTrend plotDecision Tree VisualizerInstallation:warning: Majora is only compatible withPython 3.:warning: Decision Tree visualizer requiresgraphviz.Install Via GitHub>pipinstallgit+https://github.com/GrandPurpleOcelot/Auto-EDAUsagefrommajoraimport*;Initiate a class instance with input dataframe:heart=pd.read_csv('datasets/heart.csv')heart['target']=np.where(heart['target']==1,'has disease','no disease')report=majora(heart,target_variable='target')The available parameters are:df: the input pandas dataframe.target_variable: the target variable that Auto_EDA will focus on.Dataset Overviewreport.get_samples()get_samples() returns a df concatenated from head + random samples + tail of the dataset.>>>report.get_overview()NumberofVariables:303NumberofObservations:14MemoryUsage:0.052106Mbget_samples() returns number of variables, observations, and memory usage.Missing ValuesIdentify Missing Valuesreport.get_missings(missing_tag='?')The available parameters are:missing_tag: Sometimes missing values are denoted with a number or string (eg. '?'), enter the missing tag to replace them with NAsHandling missing values>>>report.handle_missings(strategy='deletion',drop_threshold=0.5)Droppedcolumns:['NMHC(GT)']Numberofdroppedrows:2416-->25.8%ofrowsremovedThe available parameters are:strategy: select a strategy to handle missing data. Options: 'deletion', 'encode', 'mean_mode''deletion': drop variables with > 70% missing (or a different threshold using argument 'drop_threshold') and remove observations that contain at least 1 missing value.'encode'(Encoding imputation): for numerical variable, encoding missing entries as -999. For categorical variable, encoding missing entries as string "unknown"'mean_mode'(Mean/mode imputation): for numerial variable, impute the missing entries with the mean. For categorical variable, impute the missing entries with the modedrop_threshold: if 'deletion' strategy is selected, any column that have fraction of missing values exceed drop_threshold will be dropped. drop_threshold = 1 will keep all columns. Default drop_threshold = 0.7.Variable TypesIdentify Data Typesreport.check_data_type()Type conversion suggestions:String datetime -> datetimeSmall interger (for example: boolean)  -> categorical typeString float -> floatMaximum cardinality (number of unique == number of observations) -> removeHandle Suggested Type Conversions:>>>report.change_data_type()ColumnDatetimeconvertstodatetimeVisualizationUnivariate PlotsHistogramExploratory type:numericaldatareport.histogram()The available parameters are:kde: boolean (default = False).Count PlotsExploratory type:categoricaldatareport.count_plots()Word CloudExploratory type:textdataDevelopment in progress...Bivariate PlotsCorrelation PlotsExploratory type: for numerical and numerical datareport.correlation()Principal Component AnalysisExploratory type:dimensionality reductionreport.pca()Box PlotsExploratory type:numericalandcategoricaldatareport.boxplots()Relative Frequency PlotsExploratory type:categoricalandcategoricaldatareport.cat_plots()Correspondence AnalysisExploratory type:categoricalandcategoricaldata```python
report.correspondence_analysis()
```Trend PlotExploratory type:timeseriesdatareport.timeseries_plots(grouper='M')The available parameters are:grouper: aggregate the timeseries with a time interval (default = 'W' for 1 week) using mean. This argument is used to reduce the datetime points we have to plot.Statistical ModelingUser can specify a target variable for classification/regression task using Classification And Regression Tree (CART).Classification Report (train on 75% of data, test on 25%)report.tree_model(max_depth=4)ClassificationReporton25%ofTestingData:precisionrecallf1-scoresupporthasdisease0.850.850.8541nodisease0.830.830.8335accuracy0.8476macroavg0.840.840.8476weightedavg0.840.840.8476Bar chart of relative feature importanceDecision tree visualization with Dtreeviz