Thomas Witte, Matthias Tichy | 31.10.2019

Code Manipulation throughInteractive Markers in a Live Preview

Page 2

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Some context• Quadcopter Lab as ademonstrator/student projects•• Small Lua based Domain Specific
Language (DSL) to define
missions/waypoints•• Programming novices often hesitant toexperiment out of fear to break things• → Simulation, easier understanding

Page 3

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Defining Quadcopter Missions• Simplify the language, syntax• Problems developing a strategy,
estimating dimensions etc.
→ live coding / preview

Page 4

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Overview• 3 main concepts:
Live evaluation that previews runtime values and draws visualization

Source location tracking that allows to write changes back to the codeMixing visualizations in RViz (integrates into existing visualizations, decoupled)• Goal:Helping novices exploring, experimenting, understanding codeSimplify debugging, remove barrier between development and runtime

Page 5

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

PrototypeStart simulation/robot
Code editor

Connected RViz

Output console

Page 6

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Live Preview• The code is executed after
each change and markers
are placed

Execute & place Markers
ChangeCodePage 7

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Live Preview• The code is executed after
each change and markers
are placed• Allow live modifications ofboth views
Problem:imperative language,changing state

Execute & place Markers
ChangeCodeMove
MarkerTrack source locations

Page 8

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Interactive Markers• Existing library, implemented in RViz
• Visualization integrates with other
ROS nodes• Can be implemented in othervisualization tool (e.g. AR)• Editor and live preview decoupled

Images from wiki.ros.org, CC 3.0 Attribution

Page 9Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Source Location Tracking (SLT)• My program returns 4 but I want it
to be 6; what do I have to change?
• Values can have a source location• The source location for a literal isits location in the program code.
• The source location is propagatedalong with the value.

Page 10

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

SLT – Simple Example• Marker control is linked to thevalue that placed the marker• Assignments etc. just copy thevalue• The source is modified at thevalue‘s location and reexecuted→ the marker is now at thedesired position

Page 11

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

SLT – Expressions• Forcing a desired value for an expression yields multiple results• 5+4=7: 5 → 3 | 4 → 2 | (5 → 4.5, 4 → 2.5) | …
• Left bias implemented: first operand that has a source attached is changed

Page 12

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

SLT – Examples, Generalizing effects• Incomplete heuristic, but itworks in many cases• Nice generalizing effect:Moving a marker keeps theshape elliptical• Language and Domain
agnostic

Page 13

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

tf Integration• Interaction with physical objects
should be visible in the preview• Capturing a pose: placing anobject and using live evaluation• Experimentation to explore
corner cases

Page 14

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Live Evaluation, \ operator• Simple script:
fly to x-coordinate of
the yellow helmet• pose command uses tf→ preview changes when
helmet is moved• Show the live coordinatesin the code

Page 15

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Live Evaluation, \ operator• Simple script:
fly to x-coordinate of
the yellow helmet• pose command uses tf→ preview changes when
helmet is moved• Show the live coordinatesin the code

Page 16

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Start simulation• Directly start script on a connected system (simulation/robot)• Separate interpreter, implementation of functions• write structure → move markers → see preview → run simulation (→ start robot)

Page 17

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Overview• 3 main concepts:
Live evaluation that previews runtime values and draws visualization

Source location tracking that allows to write changes back to the codeMixing visualizations in RViz (integrates into existing visualizations, decoupled)• Goal:

Helping novices exploring, experimenting, understanding codeSimplify debugging, remove barrier between development and runtime

Page 18

Code Manipulation through Interactive Markers in a Live Preview | Thomas Witte, Matthias Tichy | 31.10.2019

Overview• Prototype implementation:
Not yet released

Code available at github.com/sp-uulm/interactive_script• Future Work:Evaluate prototype and DSLIntegrate block-based language frontend

Integrate Augmented Reality visualization