Package: nafig
nafigDo you want to visualize missing values in your data? There are plenty amazing methods (checkmissingnofor example) but they all look bulky when your data has too many columns.nafigwill help you to build a perfect NA figure!Installation$pipinstall-Unafigor install withPoetry$poetryaddnafigUsageHere are some examples of the usage both for simulated and real world data. Checkthis notebookto play with code yourself!First, let's import the core function and other useful things:>>>fromnafig.plotsimportna_text_barplot# The core function>>>fromnafig.utilsimportcreate_example_data# To simulate data>>>importpandasaspd# To works with tables>>>df,feature_types=create_example_data()dfis just a pandas dataframe with missing values.feature_typesis an array, containing data type description for each column. This is just an example, so labels don't correspond to actual data types.>>>feature_types[:10]array(['Categorical','Categorical','Binary','Continuous','Continuous','Continuous','Binary','Continuous','Continuous','Binary'],dtype='<U11')This toy dataframe contains 300 columns. Visualization of missing data with heatmap would unfortunately be too bulky. How to explore missing data distribution in this dataset? Try NA text barplot!>>>na_text_barplot(df,hue=feature_types,line_height=1.5)Columns of the dataset are binned by percentage of the missing data in them. Colouring by feature types helps to understand, which types of data are missing. On Y-axis you can see the number of features in each group.You can vary the number of bins using num_bins parameter:>>>na_text_barplot(df,hue=feature_types,line_height=1.5,num_bins=20)>>>na_text_barplot(df,hue=feature_types,line_height=2,num_bins=2,fig_width=8,font_size=3)Now let's see some real data examples!House prices missing data visualizationData source:https://www.kaggle.com/c/house-prices-advanced-regression-techniques/data?select=train.csv>>>DATA_PATH="data/house-prices/train.csv">>>house_prices_df=pd.read_csv(DATA_PATH,index_col=0)This is a reasonably good data with most of the values present. But thanks to this plot, we can see, which features are the bad guys!>>>na_text_barplot(house_prices_df,fig_width=17,num_bins=20,line_height=1.5)Note that if you don't pass thehueparameter, features will be colored by the data type of the column. If you don't want to colorize features at all, sethuetoFalse.By settingremove_empty_binstoTrue, you can remove the empty bins. It will require a reader to pay more attention to the X-axis but will save you some space.>>>na_text_barplot(house_prices_df,fig_width=10,num_bins=20,line_height=1.5,remove_empty_bins=True)Seatle AirBnB dataset missing values vizualizationData source:https://www.kaggle.com/datasets/airbnb/seattle>>>airbnb_df=pd.read_csv("data/airbnb/listings.csv")This dataset has a bit more missing data. On the plot we can see that all integer features are almost complete, and someobjectand floating number columns contain missing values>>>na_text_barplot(airbnb_df,fig_width=18,line_height=1.8,font_size=9,remove_empty_bins=True)Feel free to explore other parameters! There are more to help you create a perfect missing values visualizationDevelopers section🚀 FeaturesDevelopment featuresSupports forPython 3.9and higher.Poetryas the dependencies manager. See configuration inpyproject.tomlandsetup.cfg.Automatic codestyle withblack,isortandpyupgrade.Ready-to-usepre-commithooks with code-formatting.Type checks withmypy; docstring checks withdarglint; security checks withsafetyandbanditTesting withpytest.Ready-to-use.editorconfig,.dockerignore, and.gitignore. You don't have to worry about those things.Deployment featuresGitHubintegration: issue and pr templates.Github Actionswith predefinedbuild workflowas the default CI/CD.Everything is already set up for security checks, codestyle checks, code formatting, testing, linting, docker builds, etc withMakefile. More details inmakefile-usage.Dockerfilefor your package.Always up-to-date dependencies with@dependabot. You will onlyenable it.Automatic drafts of new releases withRelease Drafter. You may see the list of labels inrelease-drafter.yml. Works perfectly withSemantic Versionsspecification.Makefile usageMakefilecontains a lot of functions for faster development.1. Download and remove PoetryTo download and install Poetry run:makepoetry-downloadTo uninstallmakepoetry-remove2. Install all dependencies and pre-commit hooksInstall requirements:makeinstallPre-commit hooks coulb be installed aftergit initviamakepre-commit-install3. CodestyleAutomatic formatting usespyupgrade,isortandblack.makecodestyle# or use synonymmakeformattingCodestyle checks only, without rewriting files:makecheck-codestyleNote:check-codestyleusesisort,blackanddarglintlibraryUpdate all dev libraries to the latest version using one comandmakeupdate-dev-deps4. Code securitymakecheck-safetyThis command launchesPoetryintegrity checks as well as identifies security issues withSafetyandBandit.makecheck-safety5. Type checksRunmypystatic type checkermakemypy6. Tests with coverage badgesRunpytestmaketest7. All lintersOf course there is a command torulerun all linters in one:makelintthe same as:maketest&&makecheck-codestyle&&makemypy&&makecheck-safety8. Dockermakedocker-buildwhich is equivalent to:makedocker-buildVERSION=latestRemove docker image withmakedocker-removeMore informationabout docker.9. CleanupDelete pycache filesmakepycache-removeRemove package buildmakebuild-removeDelete .DS_STORE filesmakedsstore-removeRemove .mypycachemakemypycache-removeOr to remove all above run:makecleanup📈 ReleasesYou can see the list of available releases on theGitHub Releasespage.We followSemantic Versionsspecification.We useRelease Drafter. As pull requests are merged, a draft release is kept up-to-date listing the changes, ready to publish when you’re ready. With the categories option, you can categorize pull requests in release notes using labels.List of labels and corresponding titlesLabelTitle in Releasesenhancement,feature🚀 Featuresbug,refactoring,bugfix,fix🔧 Fixes & Refactoringbuild,ci,testing📦 Build System & CI/CDbreaking💥 Breaking Changesdocumentation📝 Documentationdependencies⬆️ Dependencies updatesYou can update it inrelease-drafter.yml.GitHub creates thebug,enhancement, anddocumentationlabels for you. Dependabot creates thedependencieslabel. Create the remaining labels on the Issues tab of your GitHub repository, when you need them.🛡 LicenseThis project is licensed under the terms of theMITlicense. SeeLICENSEfor more details.📃 Citation@misc{nafig,author={VladimirShitov},title={Package for plotting figures with NA data distribution},year={2023},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/VladimirShitov/nafig}}}CreditsThis project was generated withpython-package-template