Package: pandas-alive
Pandas_AliveAnimated plotting extension for Pandas with MatplotlibPandas_Aliveis intended to provide a plotting backend for animatedmatplotlibcharts forPandasDataFrames, similar to the alreadyexisting Visualization feature of Pandas.WithPandas_Alive, creating stunning, animated visualisations is as easy as calling:df.plot_animated()Table of ContentsInstallationUsageCurrently Supported Chart TypesHorizontal Bar Chart RacesVertical Bar Chart RacesLine ChartsBar ChartsScatter ChartsPie ChartsBubble ChartsBubble Chart Example 1Bubble Chart Example 2GeoSpatial ChartsGeoSpatial Point ChartsPolygon GeoSpatial ChartsMultiple ChartsUrban PopulationLife Expectancy in G7 CountriesNSW COVID VisualisationItaly COVID VisualisationSimple Pendulum MotionHTML 5 VideosProgress Bars!Future FeaturesTutorialsInspirationRequirementsDocumentationContributingDevelopmentChangelogInstallationInstall withpip install pandas_aliveorconda install pandas_alive -c conda-forgeUsageAs this package was inspired bybar_chart_race, the example data set is sourced from there.Must begin with a pandas DataFrame containing 'wide' data where:Every row represents a single period of timeEach column holds the value for a particular categoryThe index contains the time component (optional)The data below is an example of properly formatted data. It shows total deaths from COVID-19 for the highest 20 countries by date.To produce the above visualisation:CheckRequirementsfirst to ensure you have the tooling installed!Callplot_animated()on the DataFrameEither specify a file name to write to withdf.plot_animated(filename='example.mp4')or usedf.plot_animated().get_html5_videoto return a HTML5 videoDone!Noteon custom figures in notebooks:
When setting up custom figures for animations inMatplotlibmake sure to use theFigure()syntax and notfigure()instance type. The latter causes animations inMatplotlib, and in turn inpandas_alive, to take twice as long to be generated when changing from 'Figure' to 'figure' syntax.More on 'Figure' vs 'figure' can be found in thisSO entry, and thisother SO entry.importpandas_alivecovid_df=pandas_alive.load_dataset()covid_df.plot_animated(filename='examples/example-barh-chart.gif')Currently Supported Chart TypesHorizontal Bar Chart Racesimportpandasaspdimportpandas_aliveelec_df=pd.read_csv("data/Aus_Elec_Gen_1980_2018.csv",index_col=0,parse_dates=[0],thousands=',')elec_df.fillna(0).plot_animated('examples/example-electricity-generated-australia.gif',period_fmt="%Y",title='Australian Electricity Generation Sources 1980-2018')importpandas_alivecovid_df=pandas_alive.load_dataset()defcurrent_total(values):total=values.sum()s=f'Total :{int(total)}'return{'x':.85,'y':.2,'s':s,'ha':'right','size':11}covid_df.plot_animated(filename='examples/summary-func-example.gif',period_summary_func=current_total)importpandasaspdimportpandas_aliveelec_df=pd.read_csv("data/Aus_Elec_Gen_1980_2018.csv",index_col=0,parse_dates=[0],thousands=',')elec_df.fillna(0).plot_animated('examples/fixed-example.gif',period_fmt="%Y",title='Australian Electricity Generation Sources 1980-2018',fixed_max=True,fixed_order=True)importpandas_alivecovid_df=pandas_alive.load_dataset()covid_df.plot_animated(filename='examples/perpendicular-example.gif',perpendicular_bar_func='mean')Vertical Bar Chart Racesimportpandas_alivecovid_df=pandas_alive.load_dataset()covid_df.plot_animated(filename='examples/example-barv-chart.gif',orientation='v')Line ChartsWith as many lines as data columns in the DataFrame.importpandas_alivecovid_df=pandas_alive.load_dataset()covid_df.diff().fillna(0).plot_animated(filename='examples/example-line-chart.gif',kind='line',period_label={'x':0.25,'y':0.9})Bar ChartsSimilar to line charts with time as the x-axis.importpandas_alivecovid_df=pandas_alive.load_dataset()covid_df.sum(axis=1).fillna(0).plot_animated(filename='examples/example-bar-chart.gif',kind='bar',period_label={'x':0.1,'y':0.9},enable_progress_bar=True,steps_per_period=2,interpolate_period=True,period_length=200)Scatter Chartsimportpandasaspdimportpandas_alivemax_temp_df=pd.read_csv("data/Newcastle_Australia_Max_Temps.csv",parse_dates={"Timestamp":["Year","Month","Day"]},)min_temp_df=pd.read_csv("data/Newcastle_Australia_Min_Temps.csv",parse_dates={"Timestamp":["Year","Month","Day"]},)merged_temp_df=pd.merge_asof(max_temp_df,min_temp_df,on="Timestamp")merged_temp_df.index=pd.to_datetime(merged_temp_df["Timestamp"].dt.strftime('%Y/%m/%d'))keep_columns=["Minimum temperature (Degree C)","Maximum temperature (Degree C)"]merged_temp_df[keep_columns].resample("Y").mean().plot_animated(filename='examples/example-scatter-chart.gif',kind="scatter",title='Max & Min Temperature Newcastle, Australia')Pie Chartsimportpandas_alivecovid_df=pandas_alive.load_dataset()covid_df.plot_animated(filename='examples/example-pie-chart.gif',kind="pie",rotatelabels=True,period_label={'x':0,'y':0})Bubble ChartsBubble charts are generated from a multi-indexed dataframes. Where the index is the time period (optional) and the axes are defined withx_data_label&y_data_labelwhich should be passed a string in the level 0 column labels.See an example multi-indexed dataframe at:https://github.com/JackMcKew/pandas_alive/tree/master/data/multi.csvWhen you setcolor_data_label=to a df column name,pandas_alivewill automatically add acolorbar.importpandas_alivemulti_index_df=pd.read_csv("data/multi.csv",header=[0,1],index_col=0)multi_index_df.index=pd.to_datetime(multi_index_df.index,dayfirst=True)map_chart=multi_index_df.plot_animated(kind="bubble",filename="examples/example-bubble-chart.gif",x_data_label="Longitude",y_data_label="Latitude",size_data_label="Cases",color_data_label="Cases",vmax=5,steps_per_period=3,interpolate_period=True,period_length=500,dpi=100)Bubble Chart Example 1Bubble Chart Example 2Jupyter notebook:pendulum_sample.ipynbGeoSpatial ChartsGeoSpatial charts can now be animated easily usinggeopandas!If using Windows,anacondais the easiest way to install with all GDAL dependancies.Must begin with ageopandasGeoDataFrame containing 'wide' data where:Every row represents a single geometry (Point or Polygon).The index contains the geometry label (optional)Each column represents a single period in time.These can be easily composed by transposing data compatible with the rest of the charts usingdf = df.T.GeoSpatial Point Chartsimportgeopandasimportpandas_aliveimportcontextilygdf=geopandas.read_file('data/nsw-covid19-cases-by-postcode.gpkg')gdf.index=gdf.postcodegdf=gdf.drop('postcode',axis=1)map_chart=gdf.plot_animated(filename='examples/example-geo-point-chart.gif',basemap_format={'source':contextily.providers.Stamen.Terrain})Polygon GeoSpatial ChartsSupports GeoDataFrames containing Polygons!importgeopandasimportpandas_aliveimportcontextilygdf=geopandas.read_file('data/italy-covid-region.gpkg')gdf.index=gdf.regiongdf=gdf.drop('region',axis=1)map_chart=gdf.plot_animated(filename='examples/example-geo-polygon-chart.gif',basemap_format={'source':contextily.providers.Stamen.Terrain})Multiple Chartspandas_alivesupports multiple animated charts in a single visualisation.Create a list of all charts to include in animationUseanimate_multiple_plotswith afilenameand the list of charts (this will usematplotlib.subplots)Done!importpandas_alivecovid_df=pandas_alive.load_dataset()animated_line_chart=covid_df.diff().fillna(0).plot_animated(kind='line',period_label=False,add_legend=False)animated_bar_chart=covid_df.plot_animated(n_visible=10)pandas_alive.animate_multiple_plots('examples/example-bar-and-line-chart.gif',[animated_bar_chart,animated_line_chart],enable_progress_bar=True)Urban Populationimportpandas_aliveurban_df=pandas_alive.load_dataset("urban_pop")animated_line_chart=(urban_df.sum(axis=1).pct_change().fillna(method='bfill').mul(100).plot_animated(kind="line",title="Total % Change in Population",period_label=False,add_legend=False))animated_bar_chart=urban_df.plot_animated(n_visible=10,title='Top 10 Populous Countries',period_fmt="%Y")pandas_alive.animate_multiple_plots('examples/example-bar-and-line-urban-chart.gif',[animated_bar_chart,animated_line_chart],title='Urban Population 1977 - 2018',adjust_subplot_top=0.85,enable_progress_bar=True)Life Expectancy in G7 Countriesimportpandas_aliveimportpandasaspddata_raw=pd.read_csv("https://raw.githubusercontent.com/owid/owid-datasets/master/datasets/Long%20run%20life%20expectancy%20-%20Gapminder%2C%20UN/Long%20run%20life%20expectancy%20-%20Gapminder%2C%20UN.csv")list_G7=["Canada","France","Germany","Italy","Japan","United Kingdom","United States",]data_raw=data_raw.pivot(index="Year",columns="Entity",values="Life expectancy (Gapminder, UN)")data=pd.DataFrame()data["Year"]=data_raw.reset_index()["Year"]forcountryinlist_G7:data[country]=data_raw[country].valuesdata=data.fillna(method="pad")data=data.fillna(0)data=data.set_index("Year").loc[1900:].reset_index()data["Year"]=pd.to_datetime(data.reset_index()["Year"].astype(str))data=data.set_index("Year")animated_bar_chart=data.plot_animated(period_fmt="%Y",perpendicular_bar_func="mean",period_length=200,fixed_max=True)animated_line_chart=data.plot_animated(kind="line",period_fmt="%Y",period_length=200,fixed_max=True)pandas_alive.animate_multiple_plots("examples/life-expectancy.gif",plots=[animated_bar_chart,animated_line_chart],title="Life expectancy in G7 countries up to 2015",adjust_subplot_left=0.2,adjust_subplot_top=0.9,enable_progress_bar=True)NSW COVID Visualisationimportgeopandasimportpandasaspdimportpandas_aliveimportcontextilyimportmatplotlib.pyplotaspltimporturllib.request,jsonwithurllib.request.urlopen("https://data.nsw.gov.au/data/api/3/action/package_show?id=aefcde60-3b0c-4bc0-9af1-6fe652944ec2")asurl:data=json.loads(url.read().decode())# Extract url to csv componentcovid_nsw_data_url=data["result"]["resources"][0]["url"]# Read csv from data API urlnsw_covid=pd.read_csv(covid_nsw_data_url)postcode_dataset=pd.read_csv("data/postcode-data.csv")# Prepare data from NSW health datasetnsw_covid=nsw_covid.fillna(9999)nsw_covid["postcode"]=nsw_covid["postcode"].astype(int)grouped_df=nsw_covid.groupby(["notification_date","postcode"]).size()grouped_df=pd.DataFrame(grouped_df).unstack()grouped_df.columns=grouped_df.columns.droplevel().astype(str)grouped_df=grouped_df.fillna(0)grouped_df.index=pd.to_datetime(grouped_df.index)cases_df=grouped_df# Clean data in postcode dataset prior to matchinggrouped_df=grouped_df.Tpostcode_dataset=postcode_dataset[postcode_dataset['Longitude'].notna()]postcode_dataset=postcode_dataset[postcode_dataset['Longitude']!=0]postcode_dataset=postcode_dataset[postcode_dataset['Latitude'].notna()]postcode_dataset=postcode_dataset[postcode_dataset['Latitude']!=0]postcode_dataset['Postcode']=postcode_dataset['Postcode'].astype(str)# Build GeoDataFrame from Lat Long dataset and make map chartgrouped_df['Longitude']=grouped_df.index.map(postcode_dataset.set_index('Postcode')['Longitude'].to_dict())grouped_df['Latitude']=grouped_df.index.map(postcode_dataset.set_index('Postcode')['Latitude'].to_dict())gdf=geopandas.GeoDataFrame(grouped_df,geometry=geopandas.points_from_xy(grouped_df.Longitude,grouped_df.Latitude),crs="EPSG:4326")gdf=gdf.dropna()# Prepare GeoDataFrame for writing to geopackagegdf=gdf.drop(['Longitude','Latitude'],axis=1)gdf.columns=gdf.columns.astype(str)gdf['postcode']=gdf.indexgdf.to_file("data/nsw-covid19-cases-by-postcode.gpkg",layer='nsw-postcode-covid',driver="GPKG")# Prepare GeoDataFrame for plottinggdf.index=gdf.postcodegdf=gdf.drop('postcode',axis=1)gdf=gdf.to_crs("EPSG:3857")#Web Mercatormap_chart=gdf.plot_animated(basemap_format={'source':contextily.providers.Stamen.Terrain},cmap='cool')cases_df.to_csv('data/nsw-covid-cases-by-postcode.csv')fromdatetimeimportdatetimebar_chart=cases_df.sum(axis=1).plot_animated(kind='line',label_events={'Ruby Princess Disembark':datetime.strptime("19/03/2020","%d/%m/%Y"),'Lockdown':datetime.strptime("31/03/2020","%d/%m/%Y")},fill_under_line_color="blue",add_legend=False)map_chart.ax.set_title('Cases by Location')grouped_df=pd.read_csv('data/nsw-covid-cases-by-postcode.csv',index_col=0,parse_dates=[0])line_chart=(grouped_df.sum(axis=1).cumsum().fillna(0).plot_animated(kind="line",period_label=False,title="Cumulative Total Cases",add_legend=False))defcurrent_total(values):total=values.sum()s=f'Total :{int(total)}'return{'x':.85,'y':.2,'s':s,'ha':'right','size':11}race_chart=grouped_df.cumsum().plot_animated(n_visible=5,title="Cases by Postcode",period_label=False,period_summary_func=current_total)importtimetimestr=time.strftime("%d/%m/%Y")plots=[bar_chart,line_chart,map_chart,race_chart]frommatplotlibimportrcParamsrcParams.update({"figure.autolayout":False})# make sure figures are `Figure()` instancesfigs=plt.Figure()gs=figs.add_gridspec(2,3,hspace=0.5)f3_ax1=figs.add_subplot(gs[0,:])f3_ax1.set_title(bar_chart.title)bar_chart.ax=f3_ax1f3_ax2=figs.add_subplot(gs[1,0])f3_ax2.set_title(line_chart.title)line_chart.ax=f3_ax2f3_ax3=figs.add_subplot(gs[1,1])f3_ax3.set_title(map_chart.title)map_chart.ax=f3_ax3f3_ax4=figs.add_subplot(gs[1,2])f3_ax4.set_title(race_chart.title)race_chart.ax=f3_ax4timestr=cases_df.index.max().strftime("%d/%m/%Y")figs.suptitle(f"NSW COVID-19 Confirmed Cases up to{timestr}")pandas_alive.animate_multiple_plots('examples/nsw-covid.gif',plots,figs,enable_progress_bar=True)Italy COVID Visualisationimportgeopandasimportpandasaspdimportpandas_aliveimportcontextilyimportmatplotlib.pyplotaspltregion_gdf=geopandas.read_file('data\geo-data\italy-with-regions')region_gdf.NOME_REG=region_gdf.NOME_REG.str.lower().str.title()region_gdf=region_gdf.replace('Trentino-Alto Adige/Sudtirol','Trentino-Alto Adige')region_gdf=region_gdf.replace("Valle D'Aosta/VallÃ©e D'Aoste\r\nValle D'Aosta/VallÃ©e D'Aoste","Valle d'Aosta")italy_df=pd.read_csv('data\Regional Data - Sheet1.csv',index_col=0,header=1,parse_dates=[0])italy_df=italy_df[italy_df['Region']!='NA']cases_df=italy_df.iloc[:,:3]cases_df['Date']=cases_df.indexpivoted=cases_df.pivot(values='New positives',index='Date',columns='Region')pivoted.columns=pivoted.columns.astype(str)pivoted=pivoted.rename(columns={'nan':'Unknown Region'})cases_gdf=pivoted.Tcases_gdf['geometry']=cases_gdf.index.map(region_gdf.set_index('NOME_REG')['geometry'].to_dict())cases_gdf=cases_gdf[cases_gdf['geometry'].notna()]cases_gdf=geopandas.GeoDataFrame(cases_gdf,crs=region_gdf.crs,geometry=cases_gdf.geometry)gdf=cases_gdfmap_chart=gdf.plot_animated(basemap_format={'source':contextily.providers.Stamen.Terrain},cmap='viridis')cases_df=pivotedfromdatetimeimportdatetimebar_chart=cases_df.sum(axis=1).plot_animated(kind='line',label_events={'Schools Close':datetime.strptime("4/03/2020","%d/%m/%Y"),'Phase I Lockdown':datetime.strptime("11/03/2020","%d/%m/%Y"),'1M Global Cases':datetime.strptime("02/04/2020","%d/%m/%Y"),'100k Global Deaths':datetime.strptime("10/04/2020","%d/%m/%Y"),'Manufacturing Reopens':datetime.strptime("26/04/2020","%d/%m/%Y"),'Phase II Lockdown':datetime.strptime("4/05/2020","%d/%m/%Y"),},fill_under_line_color="blue",add_legend=False)map_chart.ax.set_title('Cases by Location')line_chart=(cases_df.sum(axis=1).cumsum().fillna(0).plot_animated(kind="line",period_label=False,title="Cumulative Total Cases",add_legend=False))defcurrent_total(values):total=values.sum()s=f'Total :{int(total)}'return{'x':.85,'y':.1,'s':s,'ha':'right','size':11}race_chart=cases_df.cumsum().plot_animated(n_visible=5,title="Cases by Region",period_label=False,period_summary_func=current_total)importtimetimestr=time.strftime("%d/%m/%Y")plots=[bar_chart,race_chart,map_chart,line_chart]# Otherwise titles overlap and adjust_subplot does nothingfrommatplotlibimportrcParamsfrommatplotlib.animationimportFuncAnimationrcParams.update({"figure.autolayout":False})# make sure figures are `Figure()` instancesfigs=plt.Figure()gs=figs.add_gridspec(2,3,hspace=0.5)f3_ax1=figs.add_subplot(gs[0,:])f3_ax1.set_title(bar_chart.title)bar_chart.ax=f3_ax1f3_ax2=figs.add_subplot(gs[1,0])f3_ax2.set_title(race_chart.title)race_chart.ax=f3_ax2f3_ax3=figs.add_subplot(gs[1,1])f3_ax3.set_title(map_chart.title)map_chart.ax=f3_ax3f3_ax4=figs.add_subplot(gs[1,2])f3_ax4.set_title(line_chart.title)line_chart.ax=f3_ax4axes=[f3_ax1,f3_ax2,f3_ax3,f3_ax4]timestr=cases_df.index.max().strftime("%d/%m/%Y")figs.suptitle(f"Italy COVID-19 Confirmed Cases up to{timestr}")pandas_alive.animate_multiple_plots('examples/italy-covid.gif',plots,figs,enable_progress_bar=True)Simple Pendulum MotionJupyter notebook:pendulum_sample.ipynbHTML 5 VideosPandas_Alivesupports rendering HTML5 videos through the use ofdf.plot_animated().get_html5_video()..get_html5_videosaves the animation as an h264 video, encoded in base64 directly into the HTML5 video tag. This respects the rc parameters for the writer as well as the bitrate. This also makes use of the interval to control the speed, and uses the repeat parameter to decide whether to loop.This is typically used in Jupyter notebooks.importpandas_alivefromIPython.displayimportHTMLcovid_df=pandas_alive.load_dataset()animated_html=covid_df.plot_animated().get_html5_video()HTML(animated_html)Progress Bars!Generating animations can take some time, so enable progress bars by installingtqdmwithpip install tqdmorconda install tqdmand using the keywordenable_progress_bar=Truetogether withfilename=movie file name.By default Pandas_Alive will create atqdmprogress bar when saving to a file, for the number of frames to animate, and update the progres bar after each frame.importpandas_alivecovid_df=pandas_alive.load_dataset()# add a filename=movie.mp4 or movie.gif to save to, in order to see the progress bar in actioncovid_df.plot_animated(enable_progress_bar=True)Example of TQDM in action:Future FeaturesA list of future features that may/may not be developed is:Add to line & scatter charts the ability to plot 'X' vs 'Y', as already implemented with bubble plots.Add option of a colorbar for bubble plots when included in multiple plots. Currently only available for single bubble chart animations.Geographic charts (currently using OSM export image, potentialgeopandas)Loading bar support (potentialtqdmoralive-progress)Potentially support writing to GIF in memory withhttps://github.com/maxhumber/gifSupport custom figures & axes for multiple plots (eg, gridspec)TutorialsFind tutorials on how to usePandas_Aliveover at:https://jackmckew.dev/creating-animated-plots-with-pandas_alive.htmlhttps://jackmckew.dev/geopandas-and-pandas-alive.htmlJupyter notebooks intest_notebooksfolder.InspirationThe inspiration for this project comes from:bar_chart_racebyTed PetrouPandas-BokehbyPatrik HlobilRequirementsIf you get an error such asTypeError: 'MovieWriterRegistry' object is not an iterator, this signals there isn't a writer library installed on your machine.This package utilises thematplotlib.animation function, thus requiring a writer library.Ensure to have one of the supported tooling software installed prior to use!ffmpegImageMagickSee more athttps://matplotlib.org/3.2.1/api/animation_api.html#writer-classesIf the output file name has an extension of.gif,pandas_alivewill write this withPILin memory.DocumentationDocumentation is provided athttps://jackmckew.github.io/pandas_alive/ContributingPull requests are welcome! Please help to cover more and more chart types!DevelopmentTo get started in development, clone a copy of this repository to your PC. This will now enable you to create a Jupyter notebook or a standalone.pyfile, and importpandas_aliveas a local module. Now you can create new chart types inpandas_alive/charts.pyorpandas_alive/geocharts.pyto build to your hearts content!For Python packages for a development environment checkrequirements.txtif usingPIP, orpy38-pandas_alive.ymlif usingconda.If you are usingcondaand are new to setting up environments for collaboration on projects, here are some notes from a previous contributor usingconda:Python set up with conda for project collaborationIf you wish to contribute new Jupyter notebooks with different application examples, please place them in this directory:./examples/test_notebooks/.Changelog