Audrey Tang: We’re recording our conversation.

Kel O’Neill: OK.

Audrey Tang: We’ll provide a transcription for you to read it before we publish it.

Kel O’Neill: Sure.

Interviewer: I was asking...

Audrey Tang: She was asking, "What kind of profile resists like working in VR?" I’m like, "No, [laughs] because it really doesn’t take any learning, especially the hand controllers and this generation motion controllers. We were born learning this, so it’s actually easier to teach than keyboard."

Kel O’Neill: It’s interesting because I was showing a 360 video yesterday HTC in a Vive and someone had left the controller out, so the ghost of the controller was living inside of 360 video like I kept turning over and I would see the controller in my field of view.

Audrey Tang: Yeah, yeah.

Kel O’Neill: Which made me think just about how the medium is asking to be more and more interactive and more and more immersive. Then in a way, when I started out doing 360 video, there’s a time when that’s just going to be gone when videogrammetry replaces it. Everything’s going that way.

Audrey Tang: Yeah.

Kel O’Neill: There’s a real investment and involvement with this virtual space, probably social virtual space.

Audrey Tang: Yeah, I mean last year was mostly about solo experiences, but I think as of last October or so, we have pretty mature social platforms. There’s not a lot of massive adoption but technologically they’re mature.

Kel O’Neill: I talked to this guy who was a VR journalist about his time in social VR when he was interviewing me and my wife. We asked him how long he had stayed inside the platform. I can’t remember which one it was. It was like, "I was in there for..."

Audrey Tang: What was the platform again?

Kel O’Neill: I can’t remember.

Audrey Tang: OK.

Kel O’Neill: But he was like, "I was in there for 21 hours."

Audrey Tang: The only open‑source ones High Fidelity at the moment, or Hi‑Fi for short.

Kel O’Neill: I didn’t know this one.

Audrey Tang: There’s a lot of commercial ones as well.

Kel O’Neill: Trying to explain to people like that’s the reason behind Facebook’s acquisition of Oculus...

Audrey Tang: Uh‑huh, sure.

Kel O’Neill: ...that’s been a bit of a stumbling block because I think at this point, we’re still on the rollercoaster.

Audrey Tang: Right, right, right. This is...

Kel O’Neill: That’s you?

Audrey Tang: Yeah, this is me, 3‑D scan and everything, lip‑sync, also. There’s whole social plan face thing. As we can see, this is done by the guy who did Second Life. You see plenty of places with zero people at the moment.

Kel O’Neill: That’s so interesting. One of my mentors is Nonny de la Peña, who’s a VR journalist, basically, one of the first adopters of the Vive and also one of the first people to bring, or I think the first person to bring virtual reality to film festivals. One of the projects she did called, "Hunger in Los Angeles," her initial work before she got into VR was on Second Life.

Kel O’Neill: It was academic analysis of Second Life, so it makes a lot of sense...

Audrey Tang: What in Los Angeles?

Kel O’Neill: It’s called "Hunger in Los Angeles."

Audrey Tang: "Hunger in Los Angeles."

Kel O’Neill: Yeah, so if you look up Nonny de la Peña...

Audrey Tang: Like this?

Kel O’Neill: No, "s."

Audrey Tang: OK, "Hungry in Los Angeles."

Kel O’Neill: Then Nonny de la Peña is...Yeah, there you go. I think she got rid of Immersive Journalism. She has a slightly more updated website. By "slightly more updated," I mean a Squarespace website. [laughs]

Kel O’Neill: This piece was in many ways the beginning of virtual reality as we know it, because she had to figure out how to show this at Sundance. Originally, she was using some like military‑grade virtual reality helmet. This is outside a...it’s modeled after a...How do I put this? Sorry. I just had a moment of realizing that I was being recorded.

Audrey Tang: We can edit the transcript afterward.

Kel O’Neill: This is modeled after an incident that one of her researchers went out and recorded at a soup kitchen in Los Angeles where someone went into diabetic shock while waiting for their food. Because hunger in Los Angeles is a real problem, she wanted to figure out a way to actually get people to understand it.

Kel O’Neill: She essentially created this walk around volumetric virtual reality system that could put people into this situation. Obviously it’s made in Unity. Those pixels are quite blocky at this point, but it was very effective at the time. People bursting into tears when they take off the headsets or what have you.

Kel O’Neill: Her intern was a guy named Palmer Luckey [laughs] who was just somebody who was really into playing with phones in his garage. He helped to build the first system that was not using military technology to make this piece available to people at Sundance.

Audrey Tang: Is that a glove, or was there a hand‑capture device?

Kel O’Neill: There was no hand‑capture device. The hand is in the pocket. What it’s actually doing...See the cross on the head is what’s...I’m sure...Right? Get it?

Audrey Tang: Right, right, right.

Kel O’Neill: The cross on the head is the sensor that’s placing you within the virtual context.

Audrey Tang: Ooh, that’s nice.

Kel O’Neill: Yeah, so she’s done a lot with this as well, this ghosting effect which is also in her piece, "Project Syria."

Audrey Tang: I see.

Kel O’Neill: When my partner Eline and I, we won an award called the Tim Hetherington Visionary Award that allowed us to basically choose a mentor for the first VR project that we ever made, we chose her because she held her breath for so long and stuck there while nobody was believing in the technology.

Kel O’Neill: Yeah, it’s been pretty amazing to see her life change in the past three years as this has become more and more of a mainstream technology.

Audrey Tang: This is great, this is great. This is like recreating an actual event of a few minutes long and you can just immersively participate into it.

Kel O’Neill: Exactly. She has a very specific methodology with a very specific journalistic ethic behind it. She’ll do recordings, field recordings, or use existing news footage to recreate events. The piece of hers that I think is the most powerful for me is the piece called "Kiya," that’s about a domestic violence situation. Actually, about a murder.

Audrey Tang: What’s the keyword?

Kel O’Neill: Actually, you know what? I would say, instead of going to Immersivejournalism.com, because she’s in the process of closing this one out...

Audrey Tang: I was just going on here.

Kel O’Neill: ...her company now is called Emblematic.

Audrey Tang: Emblematic.

Kel O’Neill: Which is very up your alley, I feel like after reading that piece. It’s Emblematic Media, I believe. I don’t even know what it is now.

Audrey Tang: That’s it.

Kel O’Neill: You should be able to find it. Emblematic Group, yeah. [pointing at a Formula 1 VR experience on the screen] That’s a commercial experience. "Use of Force," is a project that was at Tribeca. She produces a lot. "Kiya" is the one on the right.

Audrey Tang: "Two sisters’ efforts to rescue their third sister, Kiya..."

Kel O’Neill: I think that that may actually be available on Steam. I’m not sure.

Audrey Tang: OK. It’s loading. I’m not sure what it’s doing. Oops.

Kel O’Neill: Do you want to log into Squarespace?

Audrey Tang: I think there’s just one screenshot there.

Kel O’Neill: Yeah. I honestly think that Steam may be the best way to get it if they put it up there yet. I think that they must have because that was an Al Jazeera commission and I would assume that they want to get it out there in any way that they can. Yep. You can get it.

Audrey Tang: There we go. Yeah, yeah. It’s interesting. Thank you. That’s great. I wish we got a video.

Kel O’Neill: But even this. You look at this. It’s puppets. It’s motion capture and then there’s a thin wrap of pixels on top of it. All of this now would be done by volumetrically, like with a volumetric scanner. Sorry, not volumetric scanner but with videogrammetry. Same thing, basically.

Audrey Tang: Basically, you act one from this menu?

Kel O’Neill: That’s an interesting thing and this leads to a conversation that I’m really interested to have with you, which is about perspective in VR. Because it’s something that has become a pretty recent obsession of mine and I feel like you have interesting thoughts on it, knowing a little bit about what you said about your experiences.

Audrey Tang: This is my piece...

Kel O’Neill: Yeah, which I’ve read.

Audrey Tang: Yeah. The relevant part here is the fact that I work with some students to recreate their classroom and connect those classrooms together and had interview with some high school students and finally with some anchor. This is the High Fidelity space that I mentioned and everything. It’s mo‑cap, or whatever Vive can offer as a mo‑cap substitute.

Kel O’Neill: I like how they’re all just sitting on nothing. That’s wonderful.

Audrey Tang: Yeah, perspectives. What would you like to talk about?

Kel O’Neill: It’s just that I think there’s a capacity in the technology to address perspective and motion in different ways. You can address it from an objective perspective, so you can be...The what do you call it? The Swayze effect, you can be a ghost which is more common in 360‑video. There’s also this idea of inhabiting yourself in a virtual space. There’s the idea of becoming someone else when in virtual space.

Kel O’Neill: Then there’s something that I felt after reading your piece that if you don’t know about it, you should, because, I think, it would be very up your alley and that’s the ability to merge with someone else’s identity through VR. Do you know the piece, "The Machine to be Another"?

Audrey Tang: I think so. Yeah.

Kel O’Neill: "Gender Swap," have you ever heard of that? Have you done it?

Audrey Tang: Sure, yeah.

Kel O’Neill: Where did you do it?

Audrey Tang: I started in Second Life. It was very long ago, actually, 2006, ’07, something. At that time, I was working on a Second Life search engine. The search engine was supposed to send those invisible robots to everybody’s spaces in Second Life and look around just like real robot would do.

Audrey Tang: Visually, because Second Life the engine only feeds the textures if you can see it, If it’s in your line of sight. It learned to crawl the spaces, literally, and collect all the vision information so that we can build a search engine so you can ask it like, "Where are all the red clothes, or white necklaces?" in the whole multi ‑list, so you’re looking in and searching around.

Audrey Tang: But one part of that led me to basically record the movement of the one real avatar because we want to learn how a person actually cross the space and how to tell which one is the interesting alley here or something or which one is not interesting and so on to save some CPU time.

Audrey Tang: As part of that work, I was just recording people’s movement and the ways they look. Of course, it’s not VR, it’s not mo‑cap, but it’s a predecessor of that. As part of that, I could just replay the path and the head position. That’s pretty much it. There’s nothing else in Second Life anyway. [laughs]

Audrey Tang: But that also means that I get to immerse it through, at least today, my co‑developers’ viewpoints and lifestyles and everything because we did capture all the visual information that they got. It’s like the Wayback Machine, the Web archives, but for Second Life.

Kel O’Neill: But there’s also a tagging possibility, right, where you could actually...? Like could you track metadata on the individuals and also track...? You could be like, "All brown‑haired people are in these spaces and..."

Audrey Tang: Exactly.

Kel O’Neill: That’s so interesting. I get it.

Audrey Tang: Basically, recorder but the recorder with depth perception because that’s what Second Life gives us. You can recreate just the part of the experience that’s relevant. You can take all the other persons and their accessories out and just focus on the interaction between two people in a very large part so the other people are just transparent. That’s the technology but...

Kel O’Neill: But it’s not in this time, or whatever, right?

Audrey Tang: Exactly. You can say to it, "Before entering this discussion, attend this replay of the meeting from the other side’s perspective," but only for five minutes or something. It’s like an inform session before a deliberation but it’s actually working on a fact level. That’s the main idea.

Kel O’Neill: Machine to be Another, their piece, "Gender Swap," is a real‑time meet‑in‑a‑world version of that using virtual reality as the interface.

Audrey Tang: I saw the media.

Kel O’Neill: That piece, I feel like that’s one of the most powerful applications that I’ve ever seen of virtual reality because it opens up this idea of real empathy. Because I think that people always talk about empathy being [inaudible 18:42] machine and all that rhetoric.

Kel O’Neill: Which I don’t really believe with VR, because I think that there’s an aspect of alienation that’s inherent to the media. That’s not for augmented reality. Right?

Audrey Tang: Right. I understand the issue.

Kel O’Neill: How come people don’t acknowledge that? Marketing?

Audrey Tang: I don’t think they had sufficient firsthand experience in both media to tell one from another. Because as you said, it’s clunky. With the WiFi appendage, I think, we’re getting there. Just to let you guys know what we’re talking about.

Kel O’Neill: I wrote an essay on it as well that I’ll send to you. It’s about who the people are behind the project. One of the interesting things about it is that there a lot of them have these...They’re what you would call "third culture kids." They’re from multiple cultures and therefore have a unique understanding of culture and identity.

Kel O’Neill: I feel like since we’re all going in that direction anyway, that was one of the most profound experiences that I’ve ever had in the medium.

Audrey Tang: "Library of Ourselves."

Kel O’Neill: Did "Library of Ourselves" actually happen? I feel like "Library of Ourselves" is, basically, Snapchat spectacles in a way. See? Oh no, this is a different one.

Kel O’Neill: Oh no. I’ve never seen this one. That’s amazing.

Audrey Tang: Awesome. Awesome.

Kel O’Neill: That’s amazing.

Kel O’Neill: Fantastic. I don’t know what they’re using now, but when I had that experience in 2014, which was the gender version of the one you just saw, that was using like a Chinese Oculus knock‑off with a low‑quality streaming camera.

Audrey Tang: Yeah, I’m aware of, yes. Just a second. Which was this one, right?

Kel O’Neill: That was, yeah, "Gender Swap."

Audrey Tang: This was the one you did?

Kel O’Neill: Yeah. I also had [laughs] another experience of it at Tribeca in 2015, where I did it with my friend and designer Clint who’s my collaborator. It was really funny because we have very similar perspectives on the world, so it didn’t really feel like anything. It just felt like dancing with myself.

Audrey Tang: Sure. [laughs] They’re orchestrated to basically perform the same movements just for the sake of recording.

Kel O’Neill: Actually, what’s interesting about it is there is some level of orchestration, but there’s also a Ouija board effect where nobody’s pushing and nobody’s pulling, but obviously, that’s not what’s actually happening. You fall into a subtle...

Audrey Tang: So that you dance in resonance.

Kel O’Neill: Exactly. Yeah. A lot of what makes that happen is if they emphasize that you slow down, as you work through the experience, there’s a meditative aspect to it as well.

Kel O’Neill: In interviewing them, I brought up this idea of the third entity. That’s the term that I used, which was something that’s borrowed from a theater teacher who talked about, "When person A meets person B, they create person C."

Kel O’Neill: There’s this sharing of me and you, and then there’s you distinctly, me distinctly, and they...The funny thing was one of the people who was part of the group that does this, he was like, "’The third entity,’ that’s the term that we use in our workshop." It’s something that’s there in all of our brains that we can reach for the third entity like we all understand the concept.

Audrey Tang: I work in psychoanalysis, so it’s like...

Kel O’Neill: Got it.

Audrey Tang: ...part of the setting. This is, of course, a very powerful demo. But I do think the scalable way of doing this is some blend between synthetic and the real, like in‑the‑flesh. Like this, I think, is very convincing.

Kel O’Neill: I agree.

Audrey Tang: The hand‑touching part.

Kel O’Neill: Here I’m sending you the essay that I wrote about it. There you go. That also opens up this question of, "How are you going to get people to buy these devices when they’re most effective in a performance setting?" I think that there’s been an unhealthy rush to sell the devices to an unwilling public and it’s going to take some time.

Audrey Tang: The defense is, "Probably, people are going to find them anyway."

Kel O’Neill: That’s true. But how far off do you think we are from room‑scale experiences with mobile VR?

Audrey Tang: I think Daydream is already pretty much there.

Kel O’Neill: But there are limitations to Daydream in terms of Z access so it’s still, you’re talking about an X‑Y flat experience on some level.

Audrey Tang: Oh yeah, you have to be sitting there. But the most powerful experiences so far that I’ve seen involves very little mobility, as far as the Vive and Oculus experience goes.

Kel O’Neill: That’s interesting. Do you view Tilt Brush as a powerful experience or do you view it just as an amusement?

Audrey Tang: [laughs] A lot of my experience is just sitting in a rotating chair. I think the rotating chair is part of the mobile VR experience.

Kel O’Neill: Yeah, it is.

Audrey Tang: It doesn’t work if you take that out.

Kel O’Neill: That’s one thing that we played with a lot in our first 360‑piece because it’s a piece where a lot of it is split screen, so you have an opening of two worlds touching each other.

Kel O’Neill: There’s something about watching the way that people rotate around to be facing front and seeing Kenya and facing at the back and seeing San Diego, there’s something about that, once again, choreography that they fall into when they’re doing that that really it feels magical. It feels like breaking new ground in the medium.

Kel O’Neill: I think that that’s one of the things that’s really interesting is that it feels, sometimes, working on this specific technology, like, "I am the ant, and all of my friends are ants who work in the field and we’re all just working on something that we don’t fully understand every aspect of."

Audrey Tang: Yeah, part of the swarm.

Kel O’Neill: Part of the swarm.

Audrey Tang: Yeah.

Kel O’Neill: Which you would think would be a disconcerting experience.

Audrey Tang: Why? We’re social animals. We’re part of the swarm.

Kel O’Neill: But...

Audrey Tang: Better than part of the herd.

Kel O’Neill: But it’s also the identity thread of it, of being part of something larger than yourself at least from the perspective of an American where individualism is supposedly what it’s all about all the time, you feel a little bit like you’re not supposed to be a finger or a fingernail. You’re supposed to be the whole body.

Kel O’Neill: I guess, the endpoint of that is terrible people [laughs] who are...I’m trying very hard not to say Donald Trump’s name right now.

Audrey Tang: I think there are symptoms, there are causes.

Kel O’Neill: The psychoanalysis thing, because you’re very good at listening clearly. Do you actually practice psychoanalysis?

Audrey Tang: Yeah, sure.

Kel O’Neill: On both sides of the couch?

Audrey Tang: A little bit. I study group therapy. I don’t actually analyze people on the couch, but I’ve been doing personality analysis for six years or more. But anyway, it’s a very long time. But I do facilitate groups, group therapy and group dynamics and things like that.

Audrey Tang: It’s the thing because anyone who works in social working or care worker or psychotherapy knows that it’s not really the therapist doing the healing. We’re just channelers, and a lot of experience doing that humbles us a little bit.

Kel O’Neill: Interesting.

Audrey Tang: It’s not an individualist‑collectivist thing. It’s just knowing that personalities, identities, they’re artifacts of communication. If you take the social part out of it that there’s no identities. You can cling to memories and replay and try to confine yourself into identity but it doesn’t have to be like that. That’s the main perspective I’m coming from.

Kel O’Neill: Did you ever read that Neil Gaiman, "Miracleman" reboot?

Audrey Tang: No.

Kel O’Neill: There’s a race of aliens in it whose form of creativity is changing bodies so they can swap their consciousness into different bodies depending on how they want to express themselves artistically.

Audrey Tang: Awesome.

Kel O’Neill: Yeah, amazing. But once that’s available to us through this technology or some other technology, what does that mean to a group versus an individual? What does that mean to identity overall?

Audrey Tang: They’re just false...I do think it’s like ends of the spectrum that you can fluctuate over the course of the day like from being very meditative to very social. I think it’s healthy to explain the phenomenon like this. But it’s not any fixed point in time where you can say, "This is me and the others are not, the others are just performance." No, it’s all us.

Kel O’Neill: It’s interesting because if you talk about it that way, you have the question of, "What limits--in terms of what you can put out into the world, how you can improve the world-- are there actually, and what are just the limits that you’ve created for yourself?"

Kel O’Neill: There is a certain amount of achievement or, I would almost say like, "help" that you can offer to other people that you may not be aware you can offer to other people.

Audrey Tang: If you say, "Time box five minutes, 15 minutes, or something to help other people even though they’re complete strangers," then, you discover part of yourself that you didn’t already know.

Kel O’Neill: Yeah, but there’s also something like, and this gets back to the empathy mission idea, that how you define help and what actually is help is sometimes hard to really know. Because the idea that these technologies are going to actually help anyone and not just lock us in some strange techno dystopia, that’s a leap of faith in and of itself. Like, are these things going to improve society?

Audrey Tang: Because I’ve been doing VR classes and everything for 16 years, so for me it’s always about what we ask of the technology, not the other way around.

Kel O’Neill: But with AR, I feel that’s very right on, because that’s a technology that feels very human and I feel much more in the driver’s seat of an AR experience and I feel like just if you look at the enterprise applications and you look at the social applications, it can definitely benefit people and expose layers of reality that we’re unaware of.

Kel O’Neill: The idea of augmenting reality is a little bit off. I feel like it’s more about exposing stuff that already exists.

Audrey Tang: It’s surfacing.

Kel O’Neill: Surfacing.

Audrey Tang: Yeah, surfacing different levels of reality.

Kel O’Neill: Exactly. But VR, the isolation, the sealing yourself off from the world, what is the benefit of that? Do you think that there’s a benefit to that socially, culturally?

Audrey Tang: To me it’s also a spectrum. There’s earphones. There’s earphones that have noise cancellation which is the audio kind of a VR and there’s earphones that have pass‑through. I have a pair of earphones that basically have, say, very good noise cancellation but it has a microphone that takes everything from the outside and places it to your ear...

Audrey Tang: ...if you enable the pass‑through mode. That is actually a lot of where VR is going. I gave classes to students in Kaohsiung and Hangzhou and I teach them to use the Tron mode of Vive, which is this camera in the front of Vive. You can enable it and then you’re in a VR space but you see this blue outline of the world. It’s AR.

Audrey Tang: I told them to use SketchUp or whatever to draw their classrooms, which are very regular and to the point where if they put it and switch to Tron mode, you see it’s abled but you also see the surroundings.

Kel O’Neill: That’s interesting. That’s very interesting.

Audrey Tang: This is also AR or mixed reality but coming from that isolation part of it.

Kel O’Neill: On a technology level, clearly, that spectrum is being acknowledged by the major hardware manufacturers or software manufacturers. In the case of Microsoft, they’re probably not going to be making many HoloLens as compared to other device manufacturers who make the augmented reality glasses that run HoloGraphic.

Kel O’Neill: Like the head of Microsoft’s HoloLens program is a woman named, Lorraine Bardeen who talks about that spectrum, how there’s VR on one side and there’s augmented reality on the other side and a lot of the technology that’s been developed for HoloLens is now being fed over to the virtual reality side.

Kel O’Neill: Which is what Acer and all the other companies that are now manufacturing, or have contracts to manufacture the Microsoft VR devices are using.

Audrey Tang: What I’m saying is that complete isolation is other than looking at the Milky Way for meditative purposes...

Audrey Tang: ...that’s probably the very good use of it, but it’s very limited and I think everybody will switch to the social side.

Kel O’Neill: Slowly move down the spectrum.

Audrey Tang: Probably within a year or two.

Kel O’Neill: I agree with that. I would say, "Google agrees with that as well," because Tango and Daydream are on a collision course it seems with each other.

Audrey Tang: We have a product concept now. [laughs]

Kel O’Neill: Right, and you have the camera going through. You have the Daydream, soft utopian Google Daydream, headset and if you have pass‑through going through the camera with Tango, you basically have like another box...

Audrey Tang: They just rolled out a phone that has both so...

Kel O’Neill: Amazing.

Audrey Tang: ... just need a cell phone.

Kel O’Neill: Wait. What is the phone that they rolled out?

Audrey Tang: I think it’s an Asus phone. ZenFone is a start. ZenFone AR.

Kel O’Neill: I think my friend, Andy, told me that this was going to happen. When did this come out?

Audrey Tang: A couple of weeks ago.

Kel O’Neill: Because it used to only...Tango used to be only on like a giant Phablet.

Audrey Tang: Yeah, this is Tango.

Audrey Tang: This is, actually, pretty well‑built. It’s not heavy. It’s not a giant Phablet. This is the usual three cams. The idea is that you can use Tango to do real‑time modeling and then put on Daydream to preview. It has weird amount of RAM.

Kel O’Neill: They don’t trust it yet to have the systems fully integrated. They don’t have...

Audrey Tang: It’s all software at this point. The HoloLens is ready and the software part is, at the moment, two virtual machines, so you can’t be in the same mode as Daydream as in Tango but it’s a software problem. I’m sure it could be solved, if they came together and joined and promote this.

Kel O’Neill: Didn’t they move Daydream over the VR department?

Audrey Tang: To what?

Kel O’Neill: Did they move Daydream, I’m sorry, did they move Tango over to the VR department at a certain point? I feel like they did.

Audrey Tang: I think it’s in a supergroup but it’s not really in the same department. Anyway, this is just the first. I’m pretty sure that there will be plenty of builds like this because it’s proven to work in a phone.

Kel O’Neill: It’s funny. Look at all those things. [pointing at the Tango and Daydream logos] Do you think the graphic designers are just waiting to push those together?

Audrey Tang: Yeah, probably.

Kel O’Neill: That’s hilarious. Have you played with Tango?

Audrey Tang: In its very, very early days but not in expansion. I had those three different cameras [laughs] separately in the software stages.

Kel O’Neill: Hilarious. You’ve developed for Daydream as well?

Audrey Tang: I did the SDK but I don’t have the Daydream device yet.

Kel O’Neill: I’ve heard really good things from developers, like my friends who are developing for it. But the thing that’s funny about all this, I’m so not technological. I’ve disengaged from all social media which Jonna thinks is hilarious. I’m not a developer at all. It’s just about... there’s something about all this stuff that really calls to me, like, I’m a filmmaker and an artist.

Audrey Tang: All this is just so that people can make art.

Audrey Tang: This is literally what the cameras do.

Kel O’Neill: Absolutely. What are you going to do?

Audrey Tang: I’ve been spending a lot of time with those AR devices.

Kel O’Neill: This is the audio?

Audrey Tang: AR glasses.

Kel O’Neill: OK. Those are the new ones.

Audrey Tang: But this is really AR because after I’ve been wearing this for hours, I didn’t really feel that it’s there.

Kel O’Neill: That’s interesting. Do you know Roundware?

Audrey Tang: No.

Kel O’Neill: Roundware is interesting. You’d have to have the iPhone in your pocket, but check it out. It’s an audio AR platform that never...It’s called Roundware. It’s open‑source. It’s really brilliant.

Audrey Tang: Roundware?

Kel O’Neill: Roundware, yeah.

Audrey Tang: Roundware.

Kel O’Neill: Halsey Burgund is the creator. [Audrey pulls up a Roundware link from Google] When did this come out? Did they build this on Roundware?

Audrey Tang: I have no idea.

Kel O’Neill: Yeah. I saw Halsey’s name in that. Man, you’re quick with keyboards.

Audrey Tang: It says, "Costs, game." I see. It’s a video game that’s built on this platform.

Kel O’Neill: He’s an artist in residence at MIT Open Doc Lab. Do you know when this is published? Because I know he wanted to start working with the (MIT) Game Lab. Oh, no. This is old.

Audrey Tang: Yeah, this is old.

Kel O’Neill: That must’ve been something that he was just playing around with. Basically, you can embed audio using Google Maps or using the Maps API. You can see it behind there. You can tell stories through it. He’s a pretty interesting guy. He’s got a good spirit.

Audrey Tang: It’s informed by everything that happens in your vicinity and just maps them to the auditory spectrum.

Kel O’Neill: You could do something as basic as having multiple drone pieces, drone‑like audio pieces playing in different zones and then as you pass through the pieces, you transition. Or you could just tell a complex story using the real world as your canvas.

Kel O’Neill: The thing that he’s actually added into it that’s really fun is that there’s the ability for users to tag the spaces as they’re going through the pieces with their own comments. As a user, you can encounter the...

Audrey Tang: It’s a participatory form of sounds. In High Fidelity, because it’s got the open‑source platform that’s built, there’s a really good mixer and they wrote their own audio codec for that. It’s basically, it’s been used for things like this, but also to mix real‑time jamming events.

Audrey Tang: For example, you have soloist and then...You always start with the drummer, actually. There was this drummer and then the drummer pipelines to the soloist and to guitar and whatever. Everybody adds a layer to the jam and with very minimal latency and the people in the High Fidelity world just listen to the final mixed sound but as they walk towards the different players.

Kel O’Neill: Spatialized?

Audrey Tang: Right. Exactly. Although they’re physically in different places, they could still jam using this kind of time‑delayed technology.

Kel O’Neill: I teach...There’s a State Department program I’m involved with where filmmakers from the Middle East come to the US and learn about American production methods. I do the immersive and interactive mentoring for them.

Kel O’Neill: I’ve been trying to explain wwhat social VR is to them and the best way that I can explain it is that you’re always the person sitting in the turning chair even if you are, like in a solitary VR experience, even if it’s very convincing how it shifts your perspective. But with social VR, I can be in Moscow and you can be in Taipei, and you can be a lion and I can be a snake and we believe each other.

Audrey Tang: I think this has tremendous potential for post‑conflict work.

Kel O’Neill: As evidence.

Audrey Tang: As evidenced by this. Seriously, I do think the more the conflict that they are, the more traumatic it is, the more socially needs for the healing process to happen.

Kel O’Neill: I see that.

Audrey Tang: In the clinics realm, group therapy or psychoanalysis, we just fix one endpoint form this large traumatic conflict thing. Of course, you hope that this person can carry some of the help or some of the resilience back to their home network. But wouldn’t that be even more effective if the home network can use such a VR to work on their...?

Kel O’Neill: Let me tell you something else. Do you know who Skip Rizzo is?

Audrey Tang: Nope.

Kel O’Neill: Skip Rizzo is another person who works at USC with Nonny de la Peña in a separate area. Skip does PTSD therapy for veterans of Iraq and Afghanistan and now, the Vietnam War as well, through VR. He was one of the first people to adopt VR as a therapeutic tool. It’s interesting because his experiences that he creates...[Finding a link about Skip Rizzo for Audrey] Let me just get this...To my knowledge, he’s not working on anything that’s social.

Kel O’Neill: But I find what you’re saying very interesting. The stuff he’s doing is AI‑based or immersion therapy‑based. Let me find...I have a video that I’ll share with you.

Audrey Tang: That’s great. So far, the proven literature for immersive therapy is mostly around desensitization, which is, of course, an important part, but very small part of it.

Kel O’Neill: Desensitization literally means "numbing," so the question is, "Are you actually getting to the root of the trauma when you do that?"

Audrey Tang: Exactly. It’s just scratching the surface so to speak.

Kel O’Neill: But it can do so much more.

Audrey Tang: Sure. Desensitization has its place. An example that worked, like, before having stage fright, anticipate stage fright and use the VR to simulate a huge room. [laughs]

Kel O’Neill: I’ve heard about that being very effective, actually.

Audrey Tang: It’s a very quick fix. It’s basically, that you engage stage fright two hours or two days before the speech itself.

Kel O’Neill: Have you seen any of the footage of the initial immersion therapy VR programs? "Fear of Heights," and all that stuff?

Audrey Tang: Yeah.

Kel O’Neill: There’s a certain camp appeal to pretty much all of the ’90s VR stuff. Especially, older media reports on ’90s VR. It’s really fun to watch how it is.

Audrey Tang: What are your current projects?

Kel O’Neill: That’s so interesting. I am here, in a way, because I feel like I need to set the reset button, just press the reset button and look a little bit inside and figure out what comes next. Because I feel like I was barreling along for two or three years and I want to take some time to rethink al lot of stuff and rethink a lot of the work that lies ahead.

Kel O’Neill: There’s a slate of virtual reality projects that’s developing right now that my wife and I are working on that are just starting to take shape. In addition to all of the work and meetings and everything that are happening here on the surface, underneath the surface, all that work is starting to build up because I never have any time.

Kel O’Neill: This conversation, an hour packs in all of the sitting back and thinking I get to do in two weeks or a month at home, because I’m always just working, working, working. We’ll see. I can’t give you any definitive answers. But I can say that there’s three or four things that are on the horizon that are just starting to bloom.

Audrey Tang: That’s great. We’re in a similar place. [laughs]

Interviewer: But Kel is going to have video shot during this trip, right?

Kel O’Neill: I think so. Yeah. I think so. The best thing I could say about that video that we’re intending on shooting, "It’s very amorphous right now." But essentially, we want to find a place in Taiwan that no one will recognize outside of Taiwan, so there’s a feeling of complete displacement in the environment.

Kel O’Neill: We want to find a place that has certain cultural signifiers attached to it that send you off in false directions to give viewers a sense that the world is a much bigger place than you can ever possibly imagine.

Audrey Tang: Expand their minds.

Kel O’Neill: What?

Audrey Tang: Expand their minds.

Kel O’Neill: Yeah. Basically. But I don’t know what that means and I don’t know whether it’s going to be documentary or fiction aspects. Do you know, I’m pretty obsessed right now with one‑shot virtual reality experiences? 360‑degree experiences that don’t have traditional editing. I think that that comes from, it’s a reaction to the last couple of pieces that we worked on where I just don’t...I don’t know.

Kel O’Neill: Every time we make something I want to go in exactly the opposite direction. But the odd thing about that is that you start building up enough work, it all start to look alike, no matter how much you’re thinking you’re like pulling the wheel toward another direction, it’s still coming out of you. That speaks to the limits of identity that you were talking about before.

Kel O’Neill: This spectrum of identity does actually exist. That you are who you are no matter what you’re creating, it’s always going to look like you. It’s always going to come out as you.

Audrey Tang: There’s going to be continuity, of course, because we don’t get every day as a blank slate. But I do think that the forms or the experiments or the creations, they are probably just the projections of your dimension through other people’s dimensions. You see what I mean?

Kel O’Neill: Yeah, totally.

Audrey Tang: The thing is to make sense to say that while not entirely open to everybody else’s dimensions, it does make sense to grow your own dimension out of it and, of course, you’re going to be continuous on that dimension of a life’s trajectory.

Kel O’Neill: If you’re talking about projecting your own dimensionality, there’s certain specificity that is the only key to universality. If I were to go out tomorrow and, say, "I’m going to make a universal piece of work that everybody is going to understand and it’s going to sum up the world," overall it would be the most nonspecific piece of shit ever made.

Kel O’Neill: But if I say, "I’m going to burrow into my own perspective and create something that’s very true to me," that sometimes feel like the only pathway to...

Audrey Tang: Universality.

Kel O’Neill: ...universality. Yeah.

Audrey Tang: My favorite novel, "Finnegans Wake."

Kel O’Neill: Wow, look at you.

Kel O’Neill: You’re like the only person that got through it.

Audrey Tang: I translated part of it, but it’s very time‑consuming.

Audrey Tang: In any case, what I’m saying is at the same time the vocabulary of experience is private, like entirely specific to that particular person, but because of the execution of not limiting to other people’s grammar, it’s not limited to other people’s language, it’s at the same time entirely universal, so I don’t think it’s two things.

Audrey Tang: It’s just being authentic to your experiences, which of course involves interactions with people who are your contemporaries, who carries with them cultures, and everything like this.

Audrey Tang: It’s not possible to say, "I create something," without all the embeddings [laughs] in the local culture. But it’s important at the same time to say that, "But I’m not adhered to that culture," or, "I’m not charged with furthering the culture." I think it’s the same thing to say that we’re not limited by identity. Of course every day we’re limited by the limits imposed by the previous days.

Kel O’Neill: Yeah.

Interviewer: Though I couldn’t fully follow your conversation it has been so interesting. There are a few things coming into my mind. I have a question for Audrey. You just mentioned that you taught in Kaohsiung and Hangzhou.

Audrey Tang: Yeah, yeah.

Interviewer: What was that occasion? Are you still teaching now as a minister?

Audrey Tang: The class finished a while ago. That was before my visit to Paris, but it was after I had become a minister, so around September and October, last one.

Interviewer: Is it possible you teach again?

Audrey Tang: Sure. Why not? The thing is that when I said, "I teach in Kaohsiung and Hangzhou," I mean that I took a half‑day off in my dormitory...

Interviewer: Makes sense.

Audrey Tang: ...and sent an avatar of me, photorealistic, to Kaohsiung and Hangzhou [laughs] asking them to connect to this virtual classroom. Then I asked all my students to give a VR model of themselves and recreate their classrooms and we staged a classroom together. I’m synchronously there [laughs] but we’re all in this VR space.

Interviewer: Is it possible...? After our workshop next week, I’m thinking to extend similar meaningful programs for the young students here, so if there’s a possibility I might come and listen to it, to follow, to try to maximize.

Audrey Tang: I do tour around the world in robotic doubles...

Audrey Tang: ...giving lectures and everything. Double Robotics is really pretty good for that purpose. Have you ever tried it?

Kel O’Neill: No. It’s fascinating to me.

Interviewer: Because we are trying to set up a virtual college on digital arts in Chengchi University.

Kel O’Neill: Oh yeah, OK. I thought you meant...

Audrey Tang: Snowden wrote it. He’s probably the most famous user.

Kel O’Neill: There’s a really good "Community" episode about it.

Audrey Tang: They have this too.

Kel O’Neill: Amazing. That’s hilarious.

Audrey Tang: They designed it so that stitching algorithm will cancel this entire thing out, so you get a floating perspective.

Kel O’Neill: I’m talking about work that I made a year and a half ago, and it’s like, "In my day stitching was terrible." Now it’s like it doesn’t matter, everything’s...

Audrey Tang: It doesn’t matter anymore. It is a solved problem.

Kel O’Neill: Solved problem. Literally everything that was published before like April, 2016, has these irreversible stitching errors.

Audrey Tang: Exactly.

Kel O’Neill: Will we accept that in the future?

Audrey Tang: Why not?

Kel O’Neill: Yeah, I guess. I hope so.

Audrey Tang: I’m sure some AI will look at those old films and completely patch over them...

Audrey Tang: ... and fix all the artifacts.

Kel O’Neill: Or just be like, "This is a terrible film. Make the human slaves fix them."

Audrey Tang: Seriously, the generative model is...

Kel O’Neill: Yeah.

Audrey Tang: You know what the generative model is.

Audrey Tang: Basically what this does is that it looks at huge amount of bedroom images and [inaudible 56:11] are really just initial few rows and a mission autocompletes based on...

Kel O’Neill: That’s amazing.

Audrey Tang: It’s very sub‑descriptive. [laughs] All this is synthetic. It’s as of a few months ago, a solved problem. You can very easily teach it to complete and remove the stitching artifacts in a very convincing way. It also generates album covers out of nowhere, out of noise. This is entirely dreamed up but still pretty convincing. Yeah.

Audrey Tang: Isn’t it nice?

Kel O’Neill: On some days, yeah. If you’re wearing the right pair of glasses and you’re looking at it, it’s very nice. I don’t know. The collaboration with technology, you seem to have very healthy attitude toward it. Does it ever feel to you like things are moving too quickly?

Audrey Tang: No. Not quick enough.

Kel O’Neill: What’s your ideal thing that will happen next month with technology?

Audrey Tang: It’s probably already happened. It’s just not evenly distributed. All these things like the Google Translate team, last October, solved the Chinese‑English translation problem.

Kel O’Neill: I haven’t downloaded the app, the magic app, yet.

Audrey Tang: Then after they solved it, it only took three or four months for a comparable performance, open‑source technology, OpenNMT, to appear on GitHub courtesy of Harvard University.

Audrey Tang: When I said, "Not quick enough," I mean, we had to wait three or four months but it’s, actually, quick enough for most of business uses because people would, of course, take even more time to evaluate and fit into their daily flows. But knowing that Chinese‑English translation is a solved problem, actually changes pretty much everything.

Kel O’Neill: Not to turn this into...Not to take us out of very dark alley, but I remember being in Accra in Ghana and seeing all the smoke rising from one neighborhood and being told later by our camera assistant that that’s where all of the computer waste was sent and lit on fire.

Audrey Tang: It is definitely a problem.

Kel O’Neill: The problems though, can those be solved as quickly as...? Can a problem involving real environmental degradation be solved as quickly as a problem like communication? I guess, you could say that the problem like communication, once it’s solved, can lead to the solutions to the problems of the environment and the problems of politics, if we just understand each other well.

Audrey Tang: Yeah, because then, the entire systematic fault, the systematic shortcoming is before everybody’s eyes and people can’t really go around ignoring it. But just do go back to your question.

Audrey Tang: I’m happy to appear like this anywhere in the world.

Kel O’Neill: He’ll send a robot to you.

Interviewer: My second thought is that, I really enjoy your relaxing conversation. Is it possible for you to have a similar occasion to be on the Google things you did just a few weeks ago, and talking about the VR and try to open the eyes for those in audience who are not very familiar with...?

Audrey Tang: You’re talking about my speech in Mix Taiwan about artificial intelligence and Daydreams?

Interviewer: Yeah. That’s right.

Audrey Tang: Sure. I’m happy to share this because I did give a talk in Paris which is what you read.

Interviewer: You two together, sitting together, have a very relaxing conversation.

Audrey Tang: That’s what we’re doing now, right? [laughs]

Interviewer: Yeah, but try to rebuild it online.

Audrey Tang: Online?

Interviewer: Yeah.

Audrey Tang: If we start filming then.

Kel O’Neill: We’re running out of time. I need more coffee.

Audrey Tang: You need more coffee. I’m sure. Drop by anytime and we’ll get a video recording.

Interviewer: Is it possible...? [non‑English speech] .

Kel O’Neill: I think I’m good for now.

Audrey Tang: [non‑English speech]

Interviewer: [non‑English speech] .

Audrey Tang: [non‑English speech] .

Interviewer: [non‑English speech]

Audrey Tang: You’re welcome to drop by any time and just to say that maybe next time, we actually do film ourselves so that it becomes an educational click of some sort.

Kel O’Neill: Can you then set us up as Second Life avatars?

Audrey Tang: Sure.

Kel O’Neill: Then it can be broadcast out as our virtual selves talking to each other?

Audrey Tang: Yeah. That’s what I did, actually, for a lot of interviews.

Kel O’Neill: Nice.

Audrey Tang: I just send my audio stream and then my avatar autocompletes the movement and everything.

Kel O’Neill: How are you doing your photogrammetry scans of yourself? Just with one, two, three detach or are you doing...?

Audrey Tang: It is just with regular, a few cameras in a cylinder rim.

Kel O’Neill: Where did you do yours?

Audrey Tang: In Paris, somewhere else in Paris. But in Taiwan, there’s a studio, I think, which is a very similar arrangement and they do deliver pretty good scans. That’s where I got Chen Ya-Lin to scan her image in the interview I show briefly of us hugging each other. It’s pretty convincing.

Kel O’Neill: We could do that.

Audrey Tang: We can do that.

Interviewer: Oh, OK.

Audrey Tang: Yeah. Or if you have your Adobe Creative CC account, there’s Adobe Fuse where you can very easily model yourself using a Sims‑like...It will be more cartoonish but actually details will be even more detailed. There’s a tradeoff. [laughs]

Kel O’Neill: See, that’s what I’m waiting for in terms of VR capture is just somebody to crack videogrammetry in a way that it’s not this 30‑camera, green screen, whatever. I actually think what you...

Audrey Tang: Tango’s going to. Supposedly, Tango is going to solve that.

Kel O’Neill: You know all of the open‑source stuff that James George and Alexander Porter and all the Scatter people did?

Audrey Tang: Mm‑hmm.

Kel O’Neill: It seems to me like they’re so closely aligned that all of the stuff that they did with the Kinect scans and the 5D mount and all that, it’s basically the same, right?

Audrey Tang: Yeah. It is.

Kel O’Neill: But Tango, you’re still talking about single perspective with this one so if you have three Tango phones, you could do...?

Audrey Tang: Or just one Tango phone on a rail.

Kel O’Neill: But with movement of subject. If you’re talking about the movement of subject like you’re actually talking about live cap of the subject and then blowing that out into room‑scale VR experience.

Audrey Tang: Yes.

Kel O’Neill: How would you do that? You just...

Audrey Tang: You just switch two types of phone systems.

Kel O’Neill: That’s a funny thing about VR, as well, is that we always thought that the more cameras, the better and what we’re slowly discovering is that two cameras are better than anything.

Audrey Tang: That’s why Vive is designed this way, firmware update issues notwithstanding.

Audrey Tang: But that’s all it takes, really.

Kel O’Neill: That’s going to be a moment when we actually have the ability to put volumetric scans into space with relative ease and simple authoring tools. That’s when the masters of the medium are going to arise.

Audrey Tang: Right. I have this sticker here, but it could be a masquerade or something. Supposedly that I wear this. Supposedly, this is Tango phone. Suppose it has a projection of a synthetic rendering of the foveation of where my eyes are looking so that people see a pair of eyes. Not real eyes but where my eyes are looking at.

Audrey Tang: Suppose that we both wear this, and both of our friends has this Tango mode scan on. Then, actually, we are like sensors.

Kel O’Neill: That’s very interesting.

Audrey Tang: Then we don’t have to install anything, as long as the field of view, we stay in each other’s field of view, we’ve got all the information we need.

Kel O’Neill: I think if you throw area learning into that, specifically through the settings that HoloLens has been playing with, then you have the ability to create this forest style, like world on top of world, on top of world, on top of world that’s psychedelic, to say the least.

Audrey Tang: Yeah.

Kel O’Neill: Cool.

Interviewer: Kel has been talking with me about his idea to have VR festival here.

Audrey Tang: Sure.

Interviewer: I’m going to work that through the pathway of the university. I think that was a good idea. I’m trying to invite Kel and his wife back to set some fires everywhere.

Audrey Tang: That’s great.

Interviewer: I’m expecting to work with you.

Audrey Tang: Of course, anytime. Drop by. Or I’ll send my digital double.

Interviewer: OK.

Kel O’Neill: It’s really, really great to meet you.

Audrey Tang: Sure.