Package: tagmaps
Tag MapsSpatio-Temporal Tag and Photo Location Clustering for generating Tag MapsTag Mapsare similar to Tag Clouds, but Tag Maps use the spatial information that is attached to geotagged photographs, in addition to tag frequency, to visualize tags on a map.
This library uses the single-linkage tree that is available fromHDBSCANto cut trees at a specific, automatic or user-defined distance for all available tags in the given dataset.
Afterwards, alpha shapes are generated as a means to allow 'soft' placement of tags on a map, according to their area of use. Two shapefiles are generated that can be used to visualize results, for example, in ESRI ArcGIS or Mapnik.Based on the papers:Dunkel, A. (2015). Visualizing the perceived environment using crowdsourced photo geodata. Landscape and Urban Planning, 142.DOI/PDFDunkel, A. (2016). Assessing the perceived environment through crowdsourced spatial photo content for application to the fields of landscape and urban planning. Thesis, TU Dresden Landscape and Environmental Planning.DOI/PDFDunkel, A. (2020). Tag Maps in der Landschaftsplanung. In book: Handbuch Methoden Visueller Kommunikation in der Räumlichen Planung.DOIOverview of processing steps (Toronto High Park example):a)individual photo locations (raw data)b)photo locations combined to clustersc)tag location clustering (HDBSCAN) and alpha-shape generationd)soft placement of all relevant tag clusters using alpha shapesThe label placement based on descending importance is currently implemented inArcGISandMapnik. See the folderresourcesfor information regarding ArcGIS and aJupyter Notebook for Mapnik.The following animationillustrates the ArcMap label placement algorithm for the TU Dresden Campus.InstallationThe recommended way to install the package is withconda install tagmaps -c conda-forge.For a detailed guide to setup tagmaps package in Windows 10, see thedocumentation.DocumentationSee thetagmaps documentationfor additional information, guides and tutorials. There is also an externalAPI referenceavailable.QuickstartCloneresourcesfolder somewhere locallygit clone https://github.com/Sieboldianus/TagMaps.git && cd TagMaps && git filter-branch --subdirectory-filter resourcesPlace geotagged data in/01_Inputsub-folderinformation on how to structure data is available in thedocumentationRuntagmapswithin folderresources. Output files will be saved to/02_Output2 shapefiles in auto-selected UTM projection, one containing all tag cluster and one with the overall location clustersVisualize shapefiles, e.g. using ESRI ArcGISdownloadBasemapLayout_World.mxdfromresources folderand replace missing links with 2 resulting shape-files in/02_Outputadjust minimum and maximum font sizes, weighting formula or other metrics to your needs.Some background:Tag Maps package can be used with any tagged & spatially referenced data, but it has been specifically developed with Social Media data in mind (Flickr, Twitter etc.).
There are two ways to load input data:Unfiltered raw dataUsetagmaps.add_record(record)where record is of typePostStructure(seeshared_structure.py)How you clean up data totally depends on the type, have a look at LoadData class inload_data.pyfor Twitter and  Flickr cleanupFiltered datathe result from 1 is aUserPostLocation(UPL), which is a reference of type 'CleanedPost'. A UPL means that all posts of a single user at a single coordinate are merged,
e.g. a reduced list of terms, tags and emoji based on global occurrence (i.e. no duplicates).The filtered data that is used for tagmaps can be exported usingtagmaps.write_cleaned_data().
Since this will remove all terms/tags/emoji that do not appear in the top 1000 (e.g.) occurring global list of terms,
this will produce a highly pseudonymized set of information, with only collectively relevant terms remaining.
The default value (1000) can be adjusted using themax_itemsargument, e.g. the smaller max_items, the higher is the effect of anonymization/generalization.CodeThe code has been completely refactored in January 2019, but there are still some missing pieces.
Particularly the API (that is:import tagmaps) is still in an early stage. See method main() inmain.pyfor examples on how to use tag maps package.ResourcesCheck outthis album on Flickrwith some more Tag Maps examplesThere's also an semi-interactive interface to explore some Tag MapshereCheck out my blogherewith some background informationAJekyll-Reveal presentationon theory & backgroundContributorsSome future goals:include topic modelingimprove automatic detection of general vs specific tags for an area (e.g. chi square)improve unit testing (pytest) for tagmaps packagemove from tkinter interface to browser based solutionBuilt WithThis project includes and makes use of several other projects/libraries/frameworks:Alpha ShapesKevin Dwyer/ Sean Gillies
Generating Concave Hull for Point CloudsHDBSCANMcInnes, J. Healy, S. Astels - BSD licensed
A high performance implementation of HDBSCAN clustering.ShapelyManipulation and analysis of geometric objectsSciPy and Convex HullSimple shapes for point clusters are generated using SciPy's excellent Convex Hull functionsFionaOGR's neat and nimble API for Python programmers.MapnikMapnik combines pixel-perfect image output with lightning-fast cartographic algorithms, backing OpenStreetMapLicenseGNU GPLv3Changelog & DownloadThis is a high-level summary of version progress. SeeCHANGELOG.mdfor a full list of changes.2022-05-10:TagMaps v0.22.0the project has finally migrated to apyproject.toml-only based packaging system, as described in thedeclarative config (pyproject.toml)the code structure now follows thesrc-layout.fiona was pinned to1.8.22in conda until#213(Windows installations only) is solved2022-07-27:TagMaps v0.21.0Add export to Mapnik (9db0d0d), enable with--mapnik_exportSee ajupyter lab notebookthat illustrates visualization of tag map data inMapnik2021-02-22:TagMaps v0.20.10fix emoji grapheme detection issue withemoji>=1.01several fixes in cx-freeze build, re-compile with python 3.92020-01-24:TagMaps v0.20.4mainly improvements of type annotations and code legibilityinclude type hints inapi-docsmigration from namedtuples to (awesome) new dataclass from Python 3.7
(this is the minimum requirement from v0.20.2 onwards)fix projection resulting in flipped geometries for pyproj>2.0.0fix various other small bugs2019-05-08:TagMaps v0.17.6as of this version, tagmaps package is available onconda-forgefixed a bug with newer versions of pyproj (>2.0.0) that would result in very slow projection performance2019-03-08:TagMaps v0.17.4First version of public API, e.g. load tagmaps to other packages withimport tagmapsorfrom tagmaps import TagMapsRefactor of LoadData and PrepareData in separate classes, use of contextmanager/ pipeline generatorImproved generation of Alpha ShapesBasic system integration test pipelineJupyter Notebook compatibility2019-01-23:TagMaps v0.11.1complete refactor of code with improved encapsulation, code now largely follows PEP conventionsbugfix: emoji handling now accurately recognizes grapheme clusters consisting of multiple unicode codepoints.interface: add feature to filter based on toplists for tags, emoji and locationsadded sample CC-BY dataset2018-01-31:TagMaps v0.9.2because Tag Maps can be generated from local to regional to continental scale, finding an algorithm that fits all was not straight forward. The current implementation will produce shapes for all of these scales without any user input.this alpha shape implementation is motivated fromKevin Dwyer/ Sean Gilliesgreat base codeauto-projection from geographic to projected Coordinate System: select the most suitable UTM Zone for projecting data.2018-01-17:TagMaps v0.9.1first build with pythoninitial commit, still lots of unnecessary comments/code parts left in code2010-03-30:TagMaps v0.0.1first implementation of tagmaps concept in ArcGIS Model Builder