Masaki Fujihata
Simultaneous Echoes
In collaboration with Frank Lyons
Technical assistance: Takeshi Kawashima

Simultaneous Echoes is a practice-based research
project. The project explores how fragmented
musical elements, which were recorded indifferent locations and at different times can bereconstructed in cyberspace. The cyberspace isused as a musical notation, a ‘3D locative musicscore’. The fragmented sound and images of thelocation are sampled with a set of audio recording
and video cameras and with the support of GPS
data sampled in specifically selected locations inand around Londonderry and in collaboration with
communities of interest there including choirs.
As a music score constitutes a kind of thinking
space for a music composer, this spatial, locative
music score is also a media thinking space for
conducting music in a new way. It reframes
in innovative ways the process of producing
whole data, recording, capturing, editing anddisseminating music and soundscapes.

Masaki Fujihata, Simultaneous Echoes, Screen shot,2009

Biographies
Masaki Fujihata is one of the pioneers ofJapanese new media art, beginning his career
working in video and digital imaging in the early
1980s. He was one of the first artists to use
stereolithography, a technique in which a laser
polymerises a liquid resin as it sweeps its surface.
He also created the world’s smallest sculptures byusing the manufacturing techniques for integrated
circuits. However, he is most recognised for his
sophisticated interactive network installations,and his primary concern has been to employ
multimedia technology in order to examine thepossibilities for communication within virtual
spaces. His interactive works include Removable
Reality (1992), which used an infrared cordlessphone, and Impressing Velocity (1994), in which
he used a laptop computer equipped with GPS
to digitally map Mount Fuji, making it available
for viewers to explore interactively. Fujihata
has exhibited extensively throughout Japanand internationally, including at the 1983, 1984,
1996, and 2000 Siggraph conferences (USA),Ars Electronica (Linz), DEAF (Rotterdam),“CyberForum” ( Lisbon), VEAF (Vancouver) andhis work is part of the permanent collection of theZKM (Karlsruhe). www.fujihata.jp
Frank Lyons holds a PhD in Music Composition
and is a Senior Lecturer in the School of Creative
Arts, University of Ulster, UK. He has developed
an international profile as a composer andresearcher with performances of his works inAustralia, the US, Europe, the UK and Ireland byartists such as Ensemble Noszferatu, Darragh
Morgan, Carlos Bonell and Alan Banks. CD
releases include Mnemonics for solo guitar playedby Alan Banks (Oct 2005) and Blitzed, Dazed by
the Haze and Rush, all for violin and electronics,
played by Darragh Morgan.

15TH INTERNATIONAL SYMPOSIUM ON ELECTRONIC ART

New Catalogue.indd 107

107

19/8/09 15:49:22

Mapping Scapes of Sound and Vision
Notes on the field work Simultaneous Echoes, an audiovisual installationby Masaki Fujihata (visual artist) in collaboration with Frank Lyons (music
composer) for ISEA 2009, Londonderry/Derry, Northern Ireland
Yvonne Spielmann

Field Works
For more than a decade, Japanese media artist
Masaki Fujihata has explored In a series of fieldworks the interrelationships between the visual
representation of natural landscapes as weperceive them in painting, cartography, video
and films, and the audiovisual expression of ourphysical activity to walk and move in these scapesand meet the people who live and work therein.
The process of a field work is driven by the conceptthat the artist in person and small teams equippedwith recording technologies of video, microphone,and GPS (Global Positioning System) enter theselected field with the target to measure, map andvisualise real spaces according to the parametersof human activity and behaviour. For example,
Fujihata and crew were climbing up and down
Mount Fuji with laptop, video and GPS equipment
by foot in 1992, and several measuring teams inparallel were surrounding and crossing Lake Shinji
in 2002 by boat and bicycle, thereby collecting
the visual data from camera and mobile phone
imagery.
In the following field projects outside Japan,
Fujihata increased public participation andcollected position data and video images of his
encounter with other foreigners who live and workin and around the Swiss cityscape as interpreters.
While they walk and drive by car together in thecity of Geneva and its vicinity the audiovisualinterviews with the interpreters in Landing
Home in Geneva (2005) focus on questions of
their cultural homes. Respectively, Fujihata’s
participating investigation wishes to communicate
views and attitudes of how to translate cultures
across languages and borders. And with regard
to the contemporary division of mobile phone andInternet access zones, effective technologicalbarriers are also addressed. With the preceding108

New Catalogue.indd 108

project Alsace (2002), the border topic is evenmore sharply pointed out in the chosen locationof the field research because this multi-culturaland multi-lingual territory in-between France andGermany had changed its borders several times.
In search for people of Alsace who can talk about
the experience of intercultural communication inquotidian life and are knowledgeable of the historyof alternating citizenships in this region, Fujihata’s
recording team pursues pressing questions ofcitizenship, homeland and cultural history froman outside position.
The advantage being that the intervening artistinterviewer from Japan is not directly involved
or considered to be taking sides in European
geopolitical matters, however, at this point all
the field works are connected in Fujihata’s
overriding aim to immerse himself physically into
the processes of the project. Which means theartist himself, not unlike a multiple personality,
is in the field, is conducting the collection of
the data, making the interviews, designing
the virtualisation of the merged data inputs
and overseeing the presentation formats of
installations and workshops (which will involve
further participants). Following the approach
to visualise one’s own experience in the field,
captured video images, location data (GPS)and direction data of an angular sensor (used torelocate the camera angle in the data space) areprocessed in the computer. Synchronisation of
position/location and orientation/direction data isnecessary to reconstruct the walking experiencein the natural landscape and attach video images
in the computer space accordingly.

ISEA2009

19/8/09 15:49:22

tion
usic

In Alsace and Landing Home in Geneva,
GPS lines, in particular, are needed to createtranslocational nets of activity in the computer,which make visible the merged subjective–
objective scape of the project. The viewer/userof the interactive installation can navigate along
these lines that were initially drawn in the real
space. S/he will activate a variety of visual fields,
which appear like a swarm of wandering frames,
each of which contains short, one to two minutelong moving sequences of different moments intime and space. In total the accessible recorded/edited video interviews build an impression of the
field that represents the notion of a field work. The
subject of ‘work’ also means that the reconstructionof natural landsapes in cyberspace results fromprocesses of reworking of the real: images thatappear in interconnected frame structures are
only supposedly free flowing and crossing in thevirtual space. Rather, access to the live recorded
mini-narratives and soundscapes that overlap likea fragmented picture is controlled and limited by
the artist, who in restricting interactivity wants us tobecome aware of the intervening processes. This
may instigate critical reflection on transgressing
borderlines and how we can make connections
on virtual maps quite differently from cartographic
maps that we usually use for orientation.

s evenocationculturalce andtimes.
k about
ation in
historyjihata’sons ofy fromg artistnvolved
ropeanoint alljihata’s
ally into
ans theonality,tion ofsigning
inputsats ofinvolveproache field,(GPS)
used toce) aretion ofdata iseriencemages

In these manoeuvrable video sequences, theartist will then be seen and heard in the same
virtual space together with his team members andthe public participants. They explore the territory
in different directions and with various speeds
and vehicles in order to remap physical spaces
in cultural and intercultural terms. The resultant
installation shows the virtual representation ofcompressed and composite imagery and soundsof a reality that had been modelled according to

the depiction of the real traces. These are theroutes that many different people took who werein the same area, but not always at the sametime. Herein, real and virtual are mixed not only
audio-visually but also with regard to multiple
authorships.
In this respect, an interesting notion of interactivity
occurs. Fujihata’s interest is to artistically express
one’s personal processes of interacting withthe real environment. Against any seemingly
objective and moreover culturally neutral
endeavour to generate a scientific cartographic
map of the world we live in, the artists’ tools are:incorporation of our traces into newly generated
audiovisual scapes that the computer displays in
3D. Naturally, confronting the ‘objective’ (standardmap) and the ‘subjective’ (personal map) resultsin conflicting presentations, when the scale andshape of the displayed measure and image data
in 3D computer graphics only vaguely remind of
the underlying real object. Instead they give an
accurate impression of the velocity and directionsof walking and the multiple perspectives ofviewing.
As the work’s title Impressing Velocity suggests,
the deviation in the computer generated
visualisation of Mount Fuji’s shape reflects theexhaustive ascent and descent of the researchrecording team in direct translation: by making adent in the middle of the visual appearance of themountain when the climbing speed was high andreversely expanding top and bottom when theteam was slow and resting.

15TH INTERNATIONAL SYMPOSIUM ON ELECTRONIC ART

New Catalogue.indd 109

109

19/8/09 15:49:22

[Fig: Impressing Velocity].

The scaling device that is used to translate thereal into the virtual mountain is the human activity,
which through the means of computer graphics
draws a more realistic image of the mountain after
the effort it takes to walk up and down. Naturally,scaling here departs from both, the artistic fiction
in natural landscape painting and also fromcartographic realism. While the first leads topainterly views of Mount Fuji with maladjusted
proportions of width and height, most prominently
in the Thirty-six Views of Mount Fuji by Japanesepainter Hokusai (1826-1833), the latter iscompletely ignorant to any subjective views andwalks, to any lively encounter with the material atall. But in the 3D graphic visualization, personal
input can be conflated and superimposed with thearea map so that Mount Fuji’s contours becomesthe ‘raw material’ of experience based data. Their
expression transforms the whole object into an
exploding array of arrow heads in all directions.
This project highlights Fujihata’s idea toincorporate and – with the simulation technologiesof the computer – almost physically imprint110

New Catalogue.indd 110

one’s personal-physical impression of a naturalenvironment onto the presentation of the object
itself. Landscapes and cityscapes that are
external to our experience are then constructed
as if internally accessible so that recorded dataof people’s behaviour in a particular environment
will be used to remodel mountain, lake andlandscapes and make these look as if someone
had purposefully deformed their shape. Herein,
the artist departs from two opposing principles in
representation that are dominant in our culture,that is, the aesthetic concept of drawing what
we see, and the scientific notion of standardised
measuring principles that are externally exercisedto all natural phenomena in the geosphere,
regardless of cultural and subjective specificities.
Fujihata sets other priorities and interrelates both,
personal experience (seeing, hearing, moving)and manpower (walking, driving and carrying
equipment) as selected input data for the creationof altered scale and form.
Likewise the banks and the waters of Lake Shinji
in the virtual visualisation of this field projectfrom 2002, appear like a number of endlesslyintertwined Möbius strips of images that are
floating in the virtual space. In the computer,video images taken from boats were modelled ashorizontal image strips. It causes disturbances
when viewing them simultaneously displayed withother images from mobile phones and also overlaid
with the GPS lines that refer to the other fieldteams (on bicycle). All this is compressed densely
in a completely arbitrary perceptual environment,that is, nevertheless, based on actual field dataof the group experiences. Position data (fromGPS transmitters and mobile phones) and theorientation data (coming from an angle measuring
sensor) of the different teams who were in thefield are actually combined to build meandering

ISEA2009

19/8/09 15:49:25

loops that tell the crossings and meetings of theexperiencing-recording teams. Then, in the virtual
remapping process, the surface of the lake serves
like a sketch map onto which the video images are‘drawn’ from incoherent, logically incompatible
views created by fixed camera positions at bothsides and at the front of the steering boat as itruns across the lake.

[Fig: Lake Shinji]

visual tools to investigate the Irish cultural rootsof instrumental sounds in juxtaposition to thelandscapes of Northern Ireland. In comparing
the various field works, one common drive thatis also grounding the Northern Ireland project
Simultaneous Echoes strikes out: to stay in theregion and observe as long as it takes and go
wheresoever necessary, regardless of distanceand climate, in order to receive the relevant datafrom the people and in the natural environment.
The artistic team is convinced and proves, byinteracting with the field and becoming participant
you will get to the core of examining one’s own
geographical and geopolitical position in theencounter with other cultural specificities.
Simultaneous Echoes

The participatory and interactive notion of fieldworks is culturally enriched, when the concept ofinterfaces with permeable information flow in both
directions (artist/interviewer and interviewed/
performer) is exported from Japan to projects in
Europe. In this new context, Fujihata’s researchand interview techniques both demand carefulconsideration of the fact that the artist in the
production process holds a multi-facetted position
oscillating in-between strangeness and closeness.
Evidently, the artist is a participating interviewer
when asking a number of foreign interpretersin the intercultural communication environmentof the multi-lingual city of Geneva, Switzerland,
which they consider as their homeland. By thesame token, the dual inside-outside position
manifests in the scrutiny of overlapping nationaland cultural identities in the French-Germanborder region of Alsace that has changed its
national status in twentieth century quite often.
And recently, Fujihata expands the participatoryinteraction and collaborates with Irish composerFrank Lyons to co-creatively develop audio and

The field work Simultaneous Echoes is situatedin Londonderry and the larger vicinity and has
a different starting point. When visiting theregion, Masaki Fujihata felt the need to get
acquainted with the field by investigating the Irish
audio cultures and balance visual and soundinformation of the field in the installation. As aresult, the project was developed co-creatively
with composer Frank Lyons, who collaborates
on the media cultural research and the interplayof acoustic and visual scores. Shared concepts
were developed through dialogue and exchange
on an interpersonal basis between the different
expertises, and resulted in combined aesthetic
approaches to record, rework, defamiliarise andrecreate/recompose the ‘raw materials’.

15TH INTERNATIONAL SYMPOSIUM ON ELECTRONIC ART

New Catalogue.indd 111

111

19/8/09 15:49:26

[Fig. Simultaneous Echoes]

In the work process of Simultaneous Echoes,
the interplay of three different types of sounds,
instrumental sounds of recorded Irish drum andpiper players, natural sounds of wind, waterand sheep and compositional scores produce
novel soundscapes that have the spatialisationof sounds as the main characteristics. In thisrespect the mixing, assembling and sampling ofdifferent sounds is a way to show a technique offragmentation and compression as it corresponds
to similar compositional activity regarding thedesign of the visual scapes. Here again, different
types of video images, one to two minute long minimovies of instrument players, natural landscapes,
and concise historical landmarks in the vicinity of
Londonderry/Derry and the counties of Northern
Ireland are overlaid, converged and presented ina virtual three-dimensional scape of panoramas
that are interconnected by lines drawn from theGPS data of the recording position.
In the beginning, video technology and GPS
data were recorded and connected so that
panorama and frame types of moving images
could be reconstructed with their locational datain the virtual space. This activity of making avirtual map allows the artists to attach sounds toimages and vice versa that document the inital
112

New Catalogue.indd 112

encounter with Irish pipe and drum players, andwith the winds, waters and other sounds of thenatural landscape. In the artistic reworking offield data, namely location, movement, sounds
and images that express actual experiences of
behaving physically in these scapes, the leadingidea is to use these inputs in the virtual space todraw traces of the music performances and thetravelling routes. The computer graphic design
will demonstrate the distances, connections
and crossings of the research journey, but mostimportantly in the virtual display of the data
space there are no borders. In this new way ofremapping geographical locations an open virtualscape is constructed that enables the artists tocontrol interactively the movement and timing oflocations which were recorded with the images
and are now used to compose the audio scores.
In correspondence to the raw materials of visualand locational data, the musical composition
evolves as a field process and not a coherent,
conventional composition. Many elements areoverlaid and interact as freely as one manoeuvresin the 3D space.
The interplay between objective position
information of GPS and video images, which
together form the field data, and their appearancesin the computer where they get manipulated,fragmented and recomposed is not interesting
because of aesthetically appealing deformationand deviation from physical reality. But differently
from l’art pour l’art approaches the merger createsa new model of interactivity that is driven by thephysical impression of the real, which becomesvisible by using the GPS input to draw threedimensional maps in the computer. These GPSlines remind of geographical accuracy and theauthenticity of the events presented in virtuality.
In the realisation of the field work in computer
graphics, the input from reality is visualised as a

ISEA2009

19/8/09 15:49:33

wire frame and can be regarded as the matrix forbuilding a structure of design and composition.
Once this structure is interconnecting audio andvisual scores, us, the audience can experience
traces of the initially real routing of sounds andimages as they now travel seemingly freely in the
cyberspace. The reverse side that simultaneously
echoes against this joyful mix of juggling excerpts
unfolds forced synchronisation of position andorientation data with the multi channel sound layers.
These systems are created in correspondence
to the tonality of the moving panoramic images
in the computer and will produce unheard andunseen soundscapes.
Outside – Inside Positions
The general approach of Masaki Fujihata’s fieldprojects is to collect exact data of the traces ofsubjective experiences in the real environment.
This information, position data, sounds andimages, are presented in the virtual space ofthe computer as combined artistic-scientificresults of actual human behaviour, mobility andcommunication in the field. It stands in stark
contrast to the supposedly objective measuring
principles, which are common for standard
cartographic representation of the world in maps.
Fujihata’s unconventionally conceived field data
are used to aesthetically remap and remodel our
perception of location and orientation, because
he uses recording and display tools to represent
the real spaces according to actual subjective
parameters and not to cartographic rules. The
most subjective views are gained from the public
participation of people who inhabit, visit or travel inand through a specific region at different times andmeet with Fujihata’s recording team at different
locations. All of these activities get registeredand the enterprise develops in many directions. It

entails communicative, interview and interculturalsituations with and between locals and foreigners.
It also extends in multiple media dimensions, hasmany visual and auditive layers of recording anddisplay, and in the presentational mode of aninteractive installation reverses subject-object
distinction between the artist and his work.
Fujihata’s field of activity deliberately positions
itself in-between inside and outside the ‘objects’of scrutiny. And in prolongation of observing
borders, differences and translations from multiple
viewpoints, the artist being the interviewer at thesame is audio-visually integrated in the recorded
scene. This intervention with his processes offield research is further highlighted when we,the viewers and users, see and hear the artist
immersed into a real scene at the time when herecords it and that through the visible timeline we
can now access from various arbitrary viewpoints
in the virtual. Visual fields are connected to thereal events and at the same time express aselecting creative activity whereby the artist has
programmed certain interactive choices that allow
navigating between visual fields containing small
movies. Hereby, having control of the field andbeing part of its unfolding vividness, subject andobject positions become evident as flexible andinterchangeable.
By starting off with dialogue, the technologicalchoices in these works serve the need to transform
landscapes and cityscapes after the impression
these locations put on us (the ‘visiting’ artist–
interviewer and the people who permanently
or contemporarily inhabit these areas). Thenthe work expresses this initial impression as if
it were possible to talk back, to communicate
with the environment in different languagesof views and sounds in the virtual scapes.
Intervention has paradoxical meanings: one aim

15TH INTERNATIONAL SYMPOSIUM ON ELECTRONIC ART

New Catalogue.indd 113

113

19/8/09 15:49:33

is to readjust and relocate the scale, velocity,
extension and tuning of experienced ‘rawmaterials’ that need to be synchronised with acomplex set of interconnected parameters. Raw
materials in this respect are physical sounds
and images, notations of location and orientation
data. In the 3D virtual environment the scientificposition and orientation data and the artistic dataof recorded sounds and images get interwoven:panoramas and mobile frames are interconnected
through the time line of the position/location dataobtained from the GPS, and the orientation data
adjust the viewing angle. As a result a mountaincan appear with a dent, and organic sounds
will be sampled and transformed into acoustic
scapes. The other aim also manifests with thiswired connection between locational sounds andimages insofar this structure unfolds translocal,transcultural and transmedial connections on thelevel of virtual reality.

occur. These, with regard to the divisional
traditions of performing also entail the prospectof variation and flexible usage in the future. The
musical scores that are rendered into acousmatic
scapes express an idea of open fields. Therein,
the sounds have strong connections to the actualsite and Irish culture, on the one hand, and arealso presented as media elements that can berecombined, blurred and blended to present
another, dialogical aspect of cultural roots.
Metaphorically speaking, the arrangement ofelements of physical and/or arbitrarily connectedsound and image scapes in the work installationtransgresses the historical frame of Irish music
performance. This field work echoes between
past and present and simultaneously presents
travelling concepts of the inside knowledgeof the composer Frank Lyons and the outside
investigation of the Japanese artist Masaki
Fujihata.

In Simultaneous Echoes, the encounter ofdifferent spheres, real and virtual, acoustic andvisual in the travelling frames of various sizes,
shapes and behaviour expresses rather violently:intermedial interconnections between mobile
vision and sound materials clearly demonstratethat the assembling and transforming process
is driven primarily by the video with the aim toprovoke new forms of acoustic spaces. This
sampling process as it relates to videos, whichcontain sounds of the location and can be altered,exchanged and further manipulated, leads to
a new technique. It shall be called acousmaticcomposition. Nevertheless, the field recordingsof the winds, the pipers and drummers are theraw materials that get mixed and combined in the
broader context of demonstrating cultural roots inthe virtual environments. One idea is to work withfamiliar sounds and defamiliarise the sources sothat new and alternate compositional possibilities

A double viewpoint of the research approach in
Simultaneous Echoes becomes evident. Oneis driven by the external view of the Japanese
media artist, and the other unfolds in co-creativity
with the local knowledge of the Irish composer.
Together and with dual languages, here vision,
there sound, they observe the political divisionsof the troublesome history as these have become
manifest in the different usage of pipes and drums.
For example, the Irish pipes (Uillean Pipes) need
to be distinguished from the highland bag pipes
which were associated with Protestant culturesand used by marching bands. Furthermore, themarching drums refer to the British tradition in
distinction to the Irish drums (Bodhran). By mixing
and sampling these sounds – performed by realplayers with environmental ‘noise’–, singing andsynthetic sounds, specific and limited connotationsof the individual elements are transgressed. It isthen possible to exlore more freely and within a

114

New Catalogue.indd 114

ISEA2009

19/8/09 15:49:33

larger time span of past and present the roots androutes of a cultural scape. Technological-aesthetic
tools of the computer are employed and enlarged
to interrogate from a combined intercultural andintermedial perspective how people relate to thefield they live in by performing specific practices
of sounds and music.

with computers, where the artistic intervention
into traditional and natural fields produces anunderstanding of ethnoscapes.

The starting point of this endeavour was that all field
inputs need to be processed to match and meetin the virtual space. Therefore, it is necessary toadjust all the metaphorical travelling of concepts
in time and space and actually relocate andreposition the field data with an exact address.
Connecting lines that bridge between previous
creative activity and present access to the movies
and their sounds hereby render the medium toself-reflexivity. In the multi-layered sound andvision composition, the computer system is forced
to exhaust the concept of space into open scapesthat exceed conventional parameters. Scapes arerather ambiguous fields and depart from spatially,
historically and culturally bounded landscapesand cityscapes insofar as scapes are affectedby participatory activity. And this encompasses
processes of interaction between the observer
and theIn this respect, the unique co-creativity between
visual artist Masaki Fujihata and composer Frank
Lyons emerges with the articulation of acousmatic
aesthetics and manifests in the open forms of theinteractive installation work. Therein, processesof deterritorialisation lead to novel mappingof locational sounds and images in a specific,
unconventional design of virtuality that includes
the physical activity of research with regard to
the cultural specificity of the presentation andits location. The collaborative mixing of real andvirtual results in spacious sounds and sounding
imagery. It unfolds another dimension of art

15TH INTERNATIONAL SYMPOSIUM ON ELECTRONIC ART

New Catalogue.indd 115

115

19/8/09 15:49:33