# Presentation: 199631

## Martha: a next generationtestable language

**Tim Menzies**
- West Virginia University_[tim@menzies.us](mailto:tim@menzies.us)_

**World’s worst title**

**Not a “language” but**
**support  tools for **
**“****what else” queries**

## Problem

|  |  | us |  |
| --- | --- | --- | --- |
|  |  | What we know | What we don’t know |
| t
h
e
m | What they know | The known knowns | The unknown knowns |
|  | What they don’t know |  | The unknown unknowns |

- Leveson:
- common cause of software failure
- components that are known to fine in isolation but failing when combined due to an unknown interaction.

-  Brian O’Conner:
- Columbia incident- pre-launch foam-strike studies
- Did  not study a critical  combination of factors

**The unknown knowns**
- Factors that have been modeled separately,
- but never studied in conjunction

## Hypothesis

**The “enough” hypothesis:**
- Early in lifecycle we know enough  to find enough problems.
**“****Enough” :**
- models exist or can be created for enough of the components
- operational constrains known
- system  goals understood.
- goals can be converted into oracles
- behavior can be sampled, fast enough
**The “funnel” hypothesis (a.k.a. “low hanging fruit”)**
- Within a system of decisions, there are a few “master” decision and many “slaves”
- It is practical to find those master decisions
**Faster, better arguments:**
- Sample, learn, restrain
- Address the master decisions first,
- Makes the slave decisions superfluous
**Procedural encodings + range functions + memoing**
- More usable modeling langauge for most folks

## Approach

**Not: “what is”**
- a = f(b,c,d,...)
**Not: “what if”:**
- For a few scenarios..
-  jiggle “b,c,d,...”
- see how “a” changes
**Rather: “what else”**
- Memoed functions (with rollback)
- Hedges (fuzzy and-ors-nots)
- Not data types, but range functions
- Massive what-ifs (generates lots of output)
- An oracle scores each run
- A  data miner: finds ranges of “b,c,d...” that select for poor scores
- Repeat till happy :-)

**functions**

*monitored *
*      **by*

**models**

*contain*

**range **
**definitions **

*are*

**simulations**

*exercise*

**oracle**

*scored-by*

**data**
**miner**

*used-by*

*used-by*

**treatments**

*restrains*

*learns*

**hedges**

*are*

*use*

**memos**

- Aha!

*use*

*make*

## Importance/ Benefits

**Q1: Cold causes o-ring incidents? ****A1: No, so launch**

**Q2: Heat blocks o-ring incidents?**
**A2: Yes, don’t launch**

- The trouble isn't what people don't know;
- it's what they do know that isn't so. -- Will Rogers
- My ideal computer never says “hello world”;rather, it says “hello, that’s strange”. -- Tim Menzies

## Relevance To NASA

**Increasing use by NASA:**
- Rapid development of mission concepts
- E.g.
- JPL’s TeamX
- GSFC’s IMDC

**How to audit TeamX or IMDC sessions?**
- How ensure that, in the heat of the moment, critical features of a mission are not over-looked?
- Or worse, accidently  over-written by other decisions?

## Accomplishments

**Accomplishments**

**Taught Martha to graduate students **
- “just a bunch of functions”
**Case studies**
- FLEET: a fleet of autonomous agents
- CHEM: chemical production facility
- COST: software cost model
**Tasks                                     **parameter,  fault                           _simulation_, _optimization_, _tuning_,        _localization_, _diagnosis_, _configuration_
- CHEM
- COST
- FLEET
**Lessons:**
- “treatments” support a wide range of tasks
- Surprisingly fast learning of controllers
- Automating a “principle-investigator-in-a-box”

- FLEET

- CHEM

- COST

## Next Steps

**Better incremental data mining**
- 100 to 1000 times faster
- Anytime algorithm
- Smaller memory footprint
**Better sampling policies**
- ??? Use internals of data miner to define what to try next
**Release of a tool kit**
**More case studies**
- SMART (small ARRT)
- TEAM-X
- ECS support
- with James Kiper, Miami Uni; and  Leila Meshkat, JPL
- Other offers?