Point Clouds. Scatterplots and Tables as User Interfaces of Artificial 'Intelligence'
Creators: name: Hunger, Francis
Publisher: name: Zenodo
Year: 2023
Subjects: user interface, visualization, scatter plot, table, umap, t-sne, machine learning, artificial intelligence

description: Scatterplots and tabular structures are the main graphical user interfaces for the diagrammatic depiction of large image data collections, which have been processed with visual recognition tools. This study discusses various media visualisations as case examples: ARTigo (LMU Munich), imgs.ai (UC Santa Barbara/Bildarchiv Foto Marburg), iArt (Universities of Munich and Hannover), Vikus Viewer (FH Potsdam), and X Degrees of Separation (Google). It further investigates how these projects employ visualisation algorithms like PCA, t-SNE and UMAP in relation to artificial weighted networks such as VGG19 or CLIP and provides practical proposals how to deal with the identified problems.
description: {"references": ["Biederman, Irving. 1987. \"Recognition-by-Components: A Theory of Human Image Understanding.\" Psychological Review 94 (2): 115\u201347. doi:10/c9dqh2", "B\u00f6nisch, Dominik. 2021. \"The Curator's Machine. Clustering von Musealen Sammlungsdaten Durch Annotieren Verdeckter Beziehungsmuster Zwischen Kunstwerken.\" Training the Archive \u2013 Working Paper, Aachen/Dortmund, May. doi:10.5281/ZENODO.4604880", "Bowker, Geoffrey C. 2014. \"The Theory/Data Thing Commentary.\" International Journal of Communication 8: 1795\u201399", "Bowker, Geoffrey C., and Susan Leigh Star. 1999. Sorting Things Out \u2013 Classification and Its Consequences. Cambridge, MA: MIT Press", "Br\u00fcggemann, Viktoria, Mark-Jan Bludau, Christopher Pietsch, and Marian D\u00f6rk. 2022. \"Von Der Wolke Zum Pfad \u2013 Visuelle Und Assoziative Exploration Zweier Kultureller Sammlungen.\" In Zenodo. doi: 10.5281/ ZENODO.6327939", "Bry, Fran\u00e7ois, Clemens Schefels, and Corina Schemainda. 2018. \"Eine Qualitative Analyse Der ARTigo-Annotationen: Serious Games, Games With A Purpose, Suchmaschine F\u00fcr Kunstwerke, Kunstgeschichte, Human Computation, Semantische Tiefe.\" Edited by Piotr Kuroczy\u0144ski, Peter Bell, and Lisa Dieckmann. Computing Art Reader. University Library Heidelberg, 96\u2013114. doi: 10.11588/ARTHISTORICUM.413.C5771", "Chun, Wendy Hui Kyong. 2021. Discriminating Data \u2013 Correlation, Neighborhoods, and the New Politics of Recogni-tion. Cambridge, MA: MIT Press", "Correll, Michael. 2022. \"Bad Ideas In Visualization.\" Medium. January 6. https://mcorrell.medium.com/bad-ideas-in-visualization-77d378148d35", "D. Haggerty, Kevin, and Richard V. Ericson. 2000. \"The Surveillant Assemblage.\" British Journal of Sociology 51 (4): 605\u201322. doi: 10.1080/00071310020015280", "Deffebach, Nancy. 2015. Mar\u00eda Izquierdo and Frida Kahlo: Challenging Visions in Modern Mexican Art. Latin American and Caribbean Arts and Culture. Austin, TX: University of Texas Press", "Dobson, James E. 2019. Critical Digital Humanities \u2013 The Search for a Methodology. Topics in the Digital Humani-ties. Urbana, Illinois: University of Illinois Press", "Few, Stephen. 2005. \"Effectively Communicating Numbers \u2013 Selecting the Best Means and Manner of Display.\" Proclarity White Paper. https://www.perceptualedge.com/articles/Whitepapers/Communicating_Numbers.pdf", "Few, Stephen 2012. Show Me the Numbers. Berkeley, CA: Analytics Press", "Friendly, Michael, and Daniel Denis. 2005. \"The Early Origins and Development of the Scatterplot.\" Journal of the History of the Behavioral Sciences 41 (2): 103\u201330. doi: 10.1002/jhbs.20078", "Galloway, Alexander R. 2012. The Interface Effect. Cambridge: Polity Press", "Geirhos, Robert, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. 2019. \"ImageNet-Trained CNNs Are Biased towards Texture \u2013 Increasing Shape Bias Improves Accuracy and Ro-bustness.\" ArXiv:1811.12231, January. http://arxiv.org/abs/1811.12231", "Gitelman, Lisa. 2013. Raw Data Is an Oxymoron. Cambridge, MA: MIT Press", "Glinka, Katrin, Christopher Pietsch, and Marian D\u00f6rk. 2017. \"Von Sammlungsspezifischen Visualisierungen Zu Nachnutzbaren Werkzeugen.\" In Books of Abstracts, edited by Michael Stolz and Patrick Helling, 204\u20137. Bern. https://uclab.fh-potsdam.de/wp/wp-content/uploads/dhd2017-aus-programm.pdf", "Hunger, Francis. 2021a. \"'Why so Many Windows?' \u2013 Wie Die Bilddatensammlung ImageNet Die Automatisierte Bilderkennung Historischer Bilder Beeinflusst.\" Training the Archive \u2013 Working Paper 2, Aachen/Dortmund, June. doi: 10.5281/ZENODO.4742621", "Hunger, Francis 2021b. \"Curation and Its Statistical Automation by Means of Artificial 'Intelligence.'\" Training the Archive \u2013 Working Paper 3, Aachen/Dortmund, November. doi: 10/gn8dn7", "Hunger, Francis 2023. \"Die Form Der Datenbank \u2013 Genealogien, Operationalit\u00e4ten Und Praxeologien Relationaler Datenbanken in Ost Und West.\" Dissertation, Weimar: Bauhaus Universit\u00e4t Weimar [in preparation].", "Hunger, Francis 2023. \"'Why so Many Windows?' \u2013 How the ImageNet Image Database Influences Automated Image Recognition of Historical Images.\" International Journal for Digital Art History, [pre-print].", "K\u00f6nig, D\u00e9nes. 1936. Theorie Der Endlichen Und Unendlichen Graphen. Edited by Horst Sachs and H. Sachs. 1936 Reprint. Teubner-Archiv Zur Mathematik 6. Leipzig: Teubner", "Kr\u00e4mer, Sybille. 2010. \"Notationen Schemata Und Diagramme \u2013\u202f\u00bbR\u00e4umlichkeit\u00ab\u202fals Darstellungsprinzip.\" In Notati-onen Und Choreographisches Denken., edited by Gabriele Brandstetter, Franck Hofmann, and Kirsten Maar, 29\u201345. Freiburg/Berlin/Wien: Rombach Verlag", "Kr\u00e4mer, Sybille, and Horst Bredekamp. 2003. \"Kultur, Technik, Kulturtechnik \u2013 Wider Die Diskursivierung Der Kultur.\" In Bild, Schrift, Zahl., edited by Sybille Kr\u00e4mer and Horst Bredekamp, 11\u201322. Kulturtechnik. M\u00fcnchen: Fink", "Kubovy, Michael, and Martin van den Berg. 2008. \"The Whole Is Equal to the Sum of Its Parts \u2013 A Probabilistic Model of Grouping by Proximity and Similarity in Regular Patterns.\" Psychological Review 115 (1): 131\u201354. doi: 10/b2vw3s.", "Latour, Bruno. 1987. Science in Action \u2013 How to Follow Scientists and Engineers through Society. Cambridge, MA: Harvard University Press", "Manovich, Lev. 2020. Cultural Analytics. Cambridge, MA: The MIT Press", "McInnes, Leland, John Healy, and James Melville. 2020. \"UMAP \u2013 Uniform Manifold Approximation and Projection for Dimension Reduction.\" ArXiv:1802.03426 [Cs, Stat], September. http://arxiv.org/abs/1802.03426", "Radford, Alec, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, et. al. 2021. \"Learning Transferable Visual Models From Natural Language Supervision.\" arXiv. http://arxiv.org/ abs/2103.00020", "Russell, Henry Norris. 1914. \"Relations between the Spectra and Other Characteristics of Stars.\" In Popular Astron-omy, 22:275\u201394. https://ia802300.us.archive.org/16/items/sim_popular-astronomy_1914-05_22_5/sim_popular-astronomy_1914-05_22_5.pdf", "Sarikaya, Alper, and Michael Gleicher. 2018. \"Scatterplots \u2013 Tasks, Data, and Designs.\" IEEE Transactions on Visua-lization and Computer Graphics 24 (1): 402\u201312. doi: 10.1109/TVCG.2017.2744184", "Schneider, Stefanie, and Hubertus Kohle. 2021. \"Bilderlesemaschinen \u2013 Einsatzszenarien Maschineller Lernverfah-ren Im Musealen Raum.\" Museum and the Internet (MAI) 2021 Tagung, May 28. https://www.youtube.com/ watch?v=lsSpzjtS-rM", "Simonyan, Karen, and Andrew Zisserman. 2015. \"Very Deep Convolutional Networks for Large-Scale Image Recognition.\" ArXiv:1409.1556 [Cs], April. http://arxiv.org/abs/1409.1556", "Springstein, Matthias. 2021. \"AW: IArt Fragen.\" Email to the Author", "Springstein, Matthias, Stefanie Schneider, Javad Rahnama, Eyke H\u00fcllermeier, Hubertus Kohle, and Ralph Ewerth. 2021. \"IART \u2013 A Search Engine for Art-Historical Images to Support Research in the Humanities.\" ArXiv:2108.01542 [Cs], August. http://arxiv.org/abs/2108.01542", "Taylor, Frederick Winslow. 1903. Shop Management. London, New York, NY: Harper &amp; Brothers", "Toshev, Alexander, and Christian Szegedy. 2014. \"DeepPose \u2013 Human Pose Estimation via Deep Neural Networks.\" 2014 IEEE Conference on Computer Vision and Pattern Recognition, June, 1653\u201360. doi: 10/ggtmv3", "Tufte, Edward R. 1983. The Visual Display of Quantitative Information. Cheshire: Graphics Press", "Wilke, C. 2019. Fundamentals of Data Visualization \u2013 A Primer on Making Informative and Compelling Figures. Sebastopol, CA: O'Reilly Media", "Zhaoping, Li. 2019. \"A New Framework for Understanding Vision from the Perspective of the Primary Visual Cortex.\" Current Opinion in Neurobiology 58 (October): 1\u201310. doi: 10/ghndbt"]}