Regressive Augmentation: Investigating Ubicomp’s Romantic Promises
Authors: Kerasidou Xaroula (Charalampia) 
Published: October 2020

Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as taking a walk in the woods. Mark Weiser on ubiquitous computing (21st Century Computer 104) In 2007, a forum entitled HCI 2020: Human Values in a Digital Age sought to address the questions: What will our world be like in 2020? Digital technologies will continue to proliferate, enabling ever more ways of changing how we live. But will such developments improve the quality of life, empower us, and make us feel safer, happier and more connected? Or will living with technology make it more tiresome, frustrating, angst-ridden, and security-driven? What will it mean to be human when everything we do is supported or augmented by technology? (Harper et al. 10) The forum came as a response to, what many call, post-PC technological developments; developments that seek to engulf our lives in digital technologies which in their various forms are meant to support and augment our everyday lives. One of these developments has been the project of ubiquitous computing along with its kin project, tangible computing. Ubiquitous computing (ubicomp) made its appearance in the late 1980s in the labs of Xerox’s Palo Alto Research Center (PARC) as the “third wave” in computing, following those of the mainframe and personal computing (Weiser, Open House 2). Mark Weiser, who coined the term, along with his collaborators at Xerox PARC, envisioned a “new technological paradigm” which would leave behind the traditional one-to-one relationship between human and computer, and spread computation “ubiquitously, but invisibly, throughout the environment” (Weiser, Gold and Brown 693). Since then, the field has grown and now counts several peer-reviewed journals, conferences, and academic and industrial research centres around the world, which have set out to study the new “post-PC computing” under names such as Pervasive Computing, Ambient Intelligence, Tangible Computing, The Internet of Things, etc. Instead of providing a comprehensive account of all the different ubicomp incarnations, this paper seeks to focus on the early projects and writings of some of ubicomp’s most prominent figures and tease out, as a way of critique, the origins of some of its romantic promises. From the outset, ubiquitous computing was heavily informed by a human-centred approach that sought to shift the focus from the personal computer back to its users. On the grounds that the PC has dominated the technological landscape at the expense of its human counterparts, ubiquitous computing promised a different human-machine interaction, with “machines that fit the human environment instead of forcing humans to enter theirs” (104, my italics) placing the two in opposite and antagonistic terrains. The problem comes about in the form of interaction between people and machines … So when the two have to meet, which side should dominate? In the past, it has been the machine that dominates. In the future, it should be the human. (Norman 140) Within these early ubicomp discourses, the computer came to embody a technological menace, the machine that threatened the liberal humanist value of being free and in control. For example, in 1999 in a book that was characterized as “the bible of ‘post-PC’ thinking” by Business Week, Donald Norman exclaimed: we have let ourselves to be trapped. … I don’t want to be controlled by a technology. I just want to get on with my life, … So down with PC’s; down with computers. All they do is complicate our lives. (72) And we read on the website of MIT’s first ubicomp project Oxygen: For over forty years, computation has centered about machines, not people. We have catered to expensive computers, pampering them in air-conditioned rooms or carrying them around with us. Purporting to serve us, they have actually forced us to serve them. Ubiquitous computing then, in its early incarnations, was presented as the solution; the human-centred, somewhat natural approach, which would shift the emphasis away from the machine and bring control back to its legitimate owner, the liberal autonomous human subject, becoming the facilitator of our apparently threatened humanness. Its promise? An early promise of regressive augmentation, I would say, since it promised to augment our lives, not by changing them, but by returning us to a past, better world that the alienating PC has supposedly displaced, enabling us to “have more time to be more fully human” (Weiser and Brown). And it sought to achieve this through the key characteristic of invisibility, which was based on the paradox that while more and more computers will permeate our lives, they will effectively disappear. Ubicomp’s Early Romantic Promises The question of how we can make computers disappear has been addressed in computer research in various ways. One of the earliest and most prominent of these is the approach, which focuses on the physicality of the world seeking to build tangible interfaces. One of the main advocates of this approach is MIT’s Tangible Media Group, led by Professor Hiroshi Ishii. The group has been working on their vision, which they call “Tangible Bits,” for almost two decades now, and in 2009 they were awarded the “Lasting Impact Award” at the ACM Symposium on User Interface Software and Technology (UIST) for their metaDesk platform, presented in 1997 (fig.1), which explores the coupling of everyday physical objects with digital information (Ullmer and Ishii). Also, in 2004 in a special paper titled “Bottles: A Transparent Interface as a Tribute to Mark Weiser”, Ishii presented once again an early project he and his group developed in 1999, and for which they were personally commented by Weiser himself. According to Ishii, bottles (fig. 2)—a system which comprises three glass bottles “filled with music” each representing a different musical instrument, placed on a Plexiglas “stage” and controlled by their physical manipulation (moving, opening or closing them)—no less, “illustrates Mark Weiser’s vision of the transparent (or invisible) interface that weaves itself into the fabric of everyday life” (1299). Figure 1: metaDesk platform (MIT Tangible Media Group) Figure 2: musicBottles (MIT Tangible Media Group) Tangible computing was based on the premise that we inhabit two worlds: the physical world and cyberspace, or as Ishii and Ullmer put it, the world of atoms and the world of bits claiming that there is gap between these two worlds that left us “torn between these parallel but disjoint spaces” (1). This agreed with Weiser’s argument that cyberspace, and specifically the computer, has taken centre stage leaving the real world—the real people, the real interactions—in the background and neglected. Tangible computing then sought to address this problem by "bridging the gaps between both cyberspace and the physical environment" (1). As Ishii and Ullmer wrote in 1997: The aim of our research is to show concrete ways to move beyond the current dominant model of GUI [Graphic User Interface] bound to computers with a flat rectangular display, windows, a mouse, and a keyboard. To make computing truly ubiquitous and invisible, we seek to establish a new type of HCI that we call "Tangible User Interfaces" (TUIs). TUIs will augment the real physical world by coupling digital information to everyday physical objects and environments. (2) “Our intention is to take advantage of natural physical affordances to achieve a heightened legibility and seamlessness of interaction between people and information” (2). In his earlier work computer scientist Paul Dourish turned to phenomenology and the concept of embodiment in order to develop an understanding of interaction as embodied. This was prior to his recent work with cultural anthropologist Bell where they examined the motivating mythology of ubiquitous computing along with the messiness of its lived experience (Dourish and Bell). Dourish, in this earlier work observed that one of the common critical features early tangible and ubiquitous computing shared is that “they both attempt to exploit our natural familiarity with the everyday environment and our highly developed spatial and physical skills to specialize and control how computation can be used in concert with naturalistic activities” (Context-Aware Computing 232). They then sought to exploit this familiarity in order to build natural computational interfaces that fit seamlessly within our everyday, real world (Where the Action Is 17). This idea of an existing set of natural tactile skills appears to come hand-in-hand with a nostalgic, romantic view of an innocent, simple, and long gone world that the early projects of tangible and ubiquitous computing sought to revive; a world where the personal computer not only did not fit, an innocent world in fact displaced by the personal computer. In 1997, Ishii and Ullmer wrote about their decision to start their investigations about the “future of HCI” in the museum of the Collection of Historic Scientific Instruments at Harvard University in their efforts to get inspired by “the aesthetics and rich affordances of these historical scientific instruments” concerned that, “alas, much of this richness has been lost to the rapid flood of digital technologies” (1). Elsewhere Ishii explained that the origin of his idea to design a bottle interface began with the concept of a “weather forecast bottle;” an idea he intended to develop as a present for his mother. “Upon opening the weather bottle, she would be greeted by the sound of singing birds if the next day’s weather was forecasted to be clear” (1300). Here, we are introduced to a nice elderly lady who has opened thousands of bottles while cooking for her family in her kitchen. This senior lady; who is made to embody the symbolic alignment between woman, the domestic and nature (see Soper, Rose, Plumwood); “has never clicked a mouse, typed a URL, nor booted a computer in her life” (Ishii 1300). Instead, “my mother simply wanted to know the following day’s weather forecast. Why should this be so complicated?” (1300, my italics). Weiser also mobilised nostalgic sentiments in order to paint a picture of what it would be to live with ubiquitous computing. So, for example, when seeking a metaphor for ubiquitous computing, he proposed “childhood – playful, a building of foundations, constant learning, a bit mysterious and quickly forgotten by adults” (Not a Desktop 8). He viewed the ubicomp home as the ideal retreat to a state of childhood; playfully reaching out to the unknown, while being securely protected and safely “at home” (Open House). These early ideas of a direct experience of the world through our bodily senses along with the romantic view of a past, simple, and better world that the computer threatened and that future technological developments promised, could point towards what Leo Marx has described as America’s “pastoral ideal”, a force that, according to Marx, is ingrained in the American view of life. Balancing between primitivism and civilisation, nature and culture, the pastoral ideal “is an embodiment of what Lovejoy calls ‘semi-primitivism’; it is located in a middle ground somewhere ‘between’, yet in a transcendent relation to, the opposing forces of civilisation and nature” (Marx 23). It appears that the early advocates of tangible and ubiquitous computing sought to strike a similar balance to the American pastoral ideal; a precarious position that managed to reconcile the disfavour and fear of Europe’s “satanic mills” with an admiration for the technological power of the Industrial Revolution, the admiration for technological development with the bucolic ideal of an unspoiled and pure nature. But how was such a balance to be achieved? How could the ideal middle state be achieved balancing the opposing forces of technological development and the dream of the return to a serene pastoral existence? According to Leo Marx, for the European colonisers, the New World was to provide the answer to this exact question (101). The American landscape was to become the terrain where old and new, nature and technology harmonically meet to form a libertarian utopia. Technology was seen as “‘naturally arising’ from the landscape as another natural ‘means of happiness’ decreed by the Creator in his design of the continent. So, far from conceding that there might be anything alien or ‘artificial’ about mechanization, technology was seen as inherent in ‘nature’; both geographic and human” (160). Since then, according to Marx, the idea of the “return” to a new Golden Age has been engrained in the American culture and it appears that it informs ubiquitous computing’s own early visions. The idea of a “naturally arising” technology which would facilitate our return to the once lost garden of security and nostalgia appears to have become a common theme within ubiquitous computing discourses making appearances across time and borders. So, for example, while in 1991 Weiser envisioned that ubiquitous technologies will make “using a computer as refreshing as taking a walk in the woods” (21st Century Computer 11), twelve years later Marzano writing about Philip’s vision of Ambient Intelligence promised that “the living space of the future could look more like that of the past than that of today” (9). While the pastoral defined nature in terms of the geographical landscape, early ubiquitous computing appeared to define nature in terms of the objects, tools and technologies that surround us and our interactions with them. While pastoral America defined itself in contradistinction to the European industrial sites and the dirty, smoky and alienating cityscapes, within those early ubiquitous computing discourses the role of the alienating force was assigned to the personal computer. And whereas the personal computer with its “grey box” was early on rejected as the modern embodiment of the European satanic mills, computation was welcomed as a “naturally arising” technological solution which would infuse the objects which, “through the ages, … are most relevant to human life—chairs, tables and beds, for instance, … the objects we can’t do without” (Marzano 9). Or else, it would infuse the—newly constructed—natural landscape fulfilling the promise that when the “world of bits” and the “world of atoms” are finally bridged, the balance will be restored. But how did these two worlds come into existence? How did bits and atoms come to occupy different and separate ontological spheres? Far from being obvious or commonsensical, the idea of the separation between bits and atoms has a history that grounds it to specific times and places, and consequently makes those early ubiquitous and tangible computing discourses part of a bigger story that, as documented (Hayles) and argued (Agre), started some time ago. The view that we inhabit the two worlds of atoms and bits (Ishii and Ullmer) was endorsed by both early ubiquitous and tangible computing, it was based on the idea of the separation of computation from its material instantiation, presenting the former as a free floating entity able to infuse our world. As we saw earlier, tangible computing took the idea of this separation as an unquestionable fact, which then served as the basis for its research goals. As we read in the home page of the Tangible Media Group’s website: Where the sea of bits meets the land of atoms, we are now facing the challenge of reconciling our dual citizenship in the physical and digital worlds. "Tangible Bits" is our vision of Human Computer Interaction (HCI): we seek a seamless coupling of bits and atoms by giving physical form to digital information and computation (my italics). The idea that digital information does not have to have a physical form, but is given one in order to achieve a coupling of the two worlds, not only reinforces the view of digital information as an immaterial entity, but also places it in a privileged position against the material world. Under this light, those early ideas of augmentation or of “awakening” the physical world (Ishii and Ullmer 3) appear to be based on the idea of a passive material world that can be brought to life and become worthy and meaningful through computation, making ubiquitous computing part of a bigger and more familiar story. Restaging the dominant Cartesian dualism between the “ensouled” subject and the “soulless” material object, the latter is rendered passive, manipulable, and void of agency and, just like Ishii’s old bottles, it is performed as a mute, docile “empty vessel” ready to carry out any of its creator’s wishes; hold perfumes and beverages, play music, or tell the weather. At the same time, computation was presented as the force that could breathe life to a mundane and passive world; a free floating, somewhat natural, immaterial entity, like oxygen (hence the name of MIT’s first ubicomp project), like the air we breathe that could travel unobstructed through any medium, our everyday objects and our environment. But it is interesting to see that in those early ubicomp discourses computation’s power did not extend too far. While computation appeared to be foregrounded as a powerful, almost magic, entity able to give life and soul to a soulless material world, at the same time it was presented as controlled and muted. The computational power that would fill our lives, according to Weiser’s ubiquitous computing, would be invisible, it wouldn’t “intrude on our consciousness” (Weiser Not a Desktop 7), it would leave no traces and bring no radical changes. If anything, it would enable us to re-establish our humanness and return us to our past, natural state promising not to change us, or our lives, by introducing something new and unfamiliar, but to enable us to “remain serene and in control” (Weiser and Brown). In other words, ubiquitous computing, as this early story goes, would not be alienating, complex, obtrusive, or even noticeable, for that matter, and so, at the end of this paper, we come full circle to ubicomp’s early goals of invisibility with its underpinnings of the precarious pastoral ideal. This short paper focused on some of ubicomp’s early stories and projects and specifically on its promise to return us to a past and implicitly better world that the PC has arguably displaced. By reading these early promises of, what I call, regressive augmentation through Marx’s work on the “pastoral ideal,” this paper sought to tease out, in order to unsettle, the origins of some of ubicomp’s romantic promises. References Agre, P. E. Computation and Human Experience. New York: Cambridge University Press, 1997. Dourish, P. “Seeking a Foundation for Context-Aware Computing.” Human–Computer Interaction 16.2-4 (2001): 229-241. ———. Where the Action Is: The Foundations of Embodied Interaction. Cambridge: MIT Press, 2001. Dourish, P. and Genevieve Bell. Divining a Digital Future: Mess and Mythology in Ubiquitous Computing. Cambridge, Massachusetts: MIT Press, 2011.Grimes, A., and R. Harper. “Celebratory Technology: New Directions for Food Research in HCI.” In CHI’08, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 2008. 467-476. Harper, R., T. Rodden, Y. Rogers, and A. Sellen (eds.). Being Human: Human-Computer Interaction in the Year 2020. Microsoft Research, 2008. 1 Dec. 2013 ‹http://research.microsoft.com/en-us/um/Cambridge/projects/hci2020/downloads/BeingHuman_A3.pdf›. Hayles, K. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago: University of Chicago Press, 1999. Ishii, H. “Bottles: A Transparent Interface as a Tribute to Mark Weiser.” IEICE Transactions on Information and Systems 87.6 (2004): 1299-1311. Ishii, H., and B. Ullmer. “Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms.” In CHI ’97, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. New York: ACM, 1997. 234-241. Marx, L. The Machine in the Garden: Technology and the Pastoral Ideal in America. 35th ed. New York: Oxford University Press, 2000. Marzano, S. “Cultural Issues in Ambient Intelligence”. In E. Aarts and S. Marzano (eds.), The New Everyday: Views on Ambient Intelligence. Rotterdam: 010 Publishers, 2003. Norman, D. The Invisible Computer: Why Good Oroducts Can Fail, the Personal Computer Is So Complex, and Information Appliances Are the Solution. Cambridge, Mass.: MIT Press, 1999. Plumwood, V. Feminism and the Mastery of Nature. London, New York: Routledge, 1993. Rose, G. Feminism and Geography. Cambridge: Polity, 1993. Soper, K. “Naturalised Woman and Feminized Nature.” In L. Coupe (ed.), The Green Studies Reader: From Romanticism to Ecocriticism. London: Routledge, 2000. Ullmer, B., and H. Ishii. “The metaDESK: Models and Prototypes for Tangible User Interfaces.” In UIST '97, Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 1997. 223-232. Weiser, M. “The Computer for the 21st Century." Scientific American 265.3 (1991): 94-104. ———. “The Open House.” ITP Review 2.0, 1996. 1 Dec. 2013 ‹http://makingfurnitureinteractive.files.wordpress.com/2007/09/wholehouse.pdf›. ———. “The World Is Not a Desktop." Interactions 1.1 (1994): 7-8. Weiser, M., and J.S. Brown. “The Coming Age of Calm Technology.” 1996. 1 Dec. 2013 ‹http://www.johnseelybrown.com/calmtech.pdf›. Weiser, M., R. Gold, and J.S. Brown. “The Origins of Ubiquitous Computing at PARC in the Late 80s.” Pervasive Computing 38 (1999): 693-696.