Rethinking Programming“Environment”
Technical and Social Environment Design toward Convivial Computing
Convivial Computing Salon, May 5, 4-5pm London (Zoom)

Jun Kato*1, Keisuke Shimakage*2*1 National Institute of Advanced Industrial Science and Technology (AIST)*2 OTON GLASS, Inc.

5/5/2020

Rethinking Programming "Environments"

1

Rethinking programming “environment”
BACKGROUND

Programming environment design fordevelopment of programs that run in the real world
TOWARD CONVIVIAL COMPUTING• Technical environment design for collaborations• Social environment design for more inclusion
CONCLUSION

5/5/2020

Rethinking Programming "Environments"2

A bit of self introduction...

Phybots

DejaVuSongle Sync

A toolkit (API and runtime debugger)
for making “robotic things”

IDE extensions for developing
Interactive camera-based programs

APIs for controlling various devices
in synchronization with music

ACM DIS 2012

ACM UIST 2012

ACM Multimedia 2018

5/5/2020

Rethinking Programming "Environments"3

Phybots (ACM DIS 2012)

5/5/2020

Jun Kato, Daisuke Sakamoto, Takeo Igarashi

Rethinking Programming "Environments"

4

DejaVu (ACM UIST 2012)

Jun Kato, Sean McDirmid, Xiang CaoCanvas
Timeline

5/5/2020

Rethinking Programming "Environments"

5

Songle Sync (ACM Multimedia 2018)

Jun Kato, Masa Ogata, Takahiro Inoue, Masataka Goto

https://api.songle.jp

6Integrated Development Environment(IDE)

Toolkit (API)
Tutorial

Domain-specific language
Visual programming

Programming language

Code editor

IDE extensions
Debugger
5/5/2020

Rethinking Programming "Environments"

7

A bit of self introduction...

Phybots

DejaVuSongle Sync

A toolkit (API and runtime debugger)
for making “robotic things”

IDE extensions for developing
interactive camera-based programs

APIs for controlling various devices
in synchronization with music

ACM DIS 2012

ACM UIST 2012

ACM Multimedia 2018

PX for programs running in the real-world

ACM DIS 2017

f3.js: A Parametric Design Tool for Physical Computing Devices
for Both Interaction Designers and End-users

https://f3js.org
Jun Kato, Masataka Goto

Personal fabrication made easy

3D printers
Photo taken by Atsushi Tadokoro (CC BY 2.0)
https://www.flickr.com/photos/tadokoro/5138646645

Laser cutters

Sensor andactuator modules

How about programming and device assembling?

Different tools and expertise needed forsoftware and hardware design

SoftwareHardware• Programmers need to imagine hardware while writing code
• “new Button()” does not infer any hardware layout
5/5/2020

Rethinking Programming "Environments"11

f3.jsModule repository for
hardware metrics

APIs for enclosure layout

Live programming editor
for microcontroller firmwareand enclosure layout

IDE for creating laser-cut interfaces andmicrocontroller programs from single code base

f3.js project page for programmers

Integrated Development Environment(IDE)

Toolkit (API)
Tutorial

Domain-specific language

Programing environments
Visual programmingare Programming
designedlanguage
for programmers

Code editor

IDE extensions
5/5/2020

Debugger
Rethinking Programming "Environments"

14

Rethinking programming “environment”
BACKGROUND

Programming experience design fordevelopment of programs that run in the real world
TOWARD CONVIVIAL COMPUTING• Technical environment design for collaborations• Social environment design for more inclusion
CONCLUSION

5/5/2020

Rethinking Programming "Environments"

15

Programming environments are (usually)
different from runtime environments• Programmers develop programs and publish them
• Users install “apps” and use them
• Once published, the programs cannot be edited
5/5/2020

Rethinking Programming "Environments"

16

Some environments allow “remix”
Scratch – Imagine, Program, Share. https://scratch.mit.edu/“See inside”
to read and edit source code

“Remixes”
to play with edited programs

5/5/2020

Rethinking Programming "Environments"

17

Some environments allow “remix”

Still, programing environments“See inside”
are designed for programmers

to read and edit source code

“Remixes”
to play with edited programs

5/5/2020

Rethinking Programming "Environments"

18

What if we design it for “both”• Instead of compiled programs, source code is shared• Furthermore, users benefit from a respective UI design• Users can safely edit the programs without breaking them
5/5/2020

Rethinking Programming "Environments"19

f3.js project page for end-users[LIVE 2016] Jun Kato, Masataka Goto

Live Tuning:
Users can get a customized variation of the devicef3.js variable declaration dialog for end-users[PX 2017] Jun Kato, Masataka Goto

User-Generated Variables:
Users can propose parameters for the device spec

Summary of our technical environment design

A multi-layered UI design approach• Base layer for programmers• Another layer for users
Shneiderman. Promoting universal usability with multi-layer interface design. 2002.

Meta-design framework• Programmers as meta-designers• Users as designers• Programming environments become socio-technical systems
Fischer et al. Revisiting and broadening the meta-design framework for end-user development. 2017.
5/5/2020

Rethinking Programming "Environments"

22

Rethinking programming “environment”
BACKGROUND

Programming experience design fordevelopment of programs that run in the real world
TOWARD CONVIVIAL COMPUTING• Technical environment design for collaborations• Social environment design for more inclusion
CONCLUSION

5/5/2020

Rethinking Programming "Environments"23

f3.jsModule repository for
hardware metrics

APIs for enclosure layout

Multi-layered user interfaces
for programmers and users

A programming environment is usually designed toconsist of (merely) computational artifact

A merely technical approach is “scalable”
but sometimes in short of “social inclusion”• Computational support X is available for many people but forsome people with characteristics Y• For instance, f3.js can be used by a variety of people but is
not designed for people with visual impairment
5/5/2020

Rethinking Programming "Environments"25

Do-It-Yourself Empowerment as Experienced by Novice Makers with Disabilities
Meissner et al., ACM DIS 20175/5/2020

Rethinking Programming "Environments"

26

A smart glasses built with a 3D printer and Raspberry Pi to help thefather of one of the authors (Keisuke) who acquired dyslexia
5/5/2020

Rethinking Programming "Environments"27

as a toolkit
5/5/2020

Rethinking Programming "Environments"

28

People with visual impairments teamed upwith evangelists to develop smart glasses

5/5/2020

Rethinking Programming "Environments"

29

OTON GLASS case studies

5/5/2020

Rethinking Programming "Environments"30

An evangelist as part ofa programming environment• Social inclusion is a hybrid of technical and social implementations• Programming environment design can be community design• Programming as communication between diverse kinds of people
5/5/2020

Rethinking Programming "Environments"

31

Summary of our social environment design

Programming by a community of people• A programing environment consists of not only computational
artifacts but also a community of people to collaborate• “People are message-passing objects” [Salon 2020 Day 1]

Programming as communication• Unlike tailor-made model, “the programmer” with visual
impairment produces ideas and decides what to build• The programming activity inherently involves communicationand enables empowerment
5/5/2020

Rethinking Programming "Environments"

32

Rethinking programming “environment”
BACKGROUND

Programming experience design fordevelopment of programs that run in the real world
TOWARD CONVIVIAL COMPUTING• Technical environment design for collaborations• Social environment design for more inclusion
CONCLUSION

5/5/2020

Rethinking Programming "Environments"

33Programming Experience (PX) researchfor convivial computing
Programming environments are usually designed exclusively
for programmers but should be more inclusive!
They can be shared amongprogrammers and users

They can be hybrid of people
and computational artifacts
5/5/2020

Rethinking Programming "Environments"

34