LSLTest

We need to have a simple protocol for performing automated tests of lsl functions. We want to be able to rez objects which will reset their state, be able to register their existence to a coordinator, and start them all, individually or in groups.
The exact protocol below is not a specific requirement -- any protocol which provides an unambiguous way to easily communicate with the coordinator and the test units is sufficient. The messages need to be easy enough for a human to type to control the controller or a single test unit.
Units are always MKS -- meters, kilograms, seconds unless otherwise specified.
Strife did some milestone work which may be useful: https://wiki.secondlife.com/wiki/User:Strife_Onizuka/Projects/TestHost
= Coordinator =
The coordinator holds testing sessions by registering test units and commanding them to start, run as individuals or groups, and collect their results.
== Requirements ==
* There should be no need to link any new objects, including test units, to the coordinator.
* There is no need for secure communication between test objects and the coordinator.
* The coordinator does not need to be a single lsl script.
* The coordinator will announce the ability for nearby test units to register.
* Once a test session has begun, the coordinator will not accept any more registrations.
* The coordinator can start a group of tests and collect results.
* The coordinator can start a single test unit and collect results.
* The coordinator can start all known tests.
* The coordinator will have a variable test broadcast channel.
* The coordinator will accept commands from its controller and broadcast appropriate commands to all, groups, or individual test units.
* Collect reported results from registered test units. 
** All pass/fail messages by group and test unit.
** All late registrations.
* It should be possible for an agent or another script to control the coordinator.
* Every test session will record a start time, test location,
* The collection of all tests which failed the last test session is an ad-hoc test group known only to the controller. For example we run all tests in 'quiet mode' and then all failures are treated as a new group which allows 'verbose mode' testing individually or all at once.
* If results are filling memory, the coordinator should output information prematurely, but needs to keep statistical totals as well as the known failed group.
* Every time summary or transient results are output, the session id, and current time are included in the output.
* The coordinator must know by the end of a test session and be able to report on:
** Which test units passed
** Which test units failed
** Which test units failed to report before timeout.
* At the end of every test session it is possible to get a summary from the coordinator.
* The summary reports:
** Number of test units in the last session
** Number of passed test units.
** Number of failed test units.
** Number of test units which failed to report before timeout.
== Communication ==
All commands should be said in a chat channel. The channel has to be controllable before the tests start.
A partial protocol is specified, though not all commands and communication are covered. The communication between the coordinator, controller, and test units should follow a "noun, verb, parameters, ..." or "noun, property, value" where possible.
The coordinator can send results and summaries to different channels. For spoken results, it simply says the results as needed and when a summary is requested. For email, the same data is emailed to the destination address. For http endpoints, the same data is sent via POST in the content body.
== States ==
There are five primary states for the coordinator.
=== Initialization ===
On startup, the coordinator is pristine. It only needs to collect who will be processing the results and how.
* The coordinator will listen on it's control channel. 
* if the control channel command is issued, the controller will change
==== Commands ====
* "coordinator, control-channel, 1234"
* "coordinator, report-destination, say, 0"
* "coordinator, report-destination, email, phoenix@secondlife.com"
* "coordinator, report-destination, http, http://example.com/" 
* "coordinator, registration-timeout, 10"
* "coordinator, set-up"
=== Registration ===
* During this phase, the coordinator will announce registration is available.
** llSay(test_broadcast_channel, "testunits, register");
* Test units will register their name and group
** llSay(test_broadcast_channel, "testunit, group_math, integer_limits");
=== Set Up ===
This phase is to control which tests to run, what channel to use for communication, and how verbose the reporting should be.
==== Requirements ====
* Able to change broadcast channel for test unit and coordinator communication.
* Able to 
* Able to start tests.
* Record start time of tests
==== Commands ====
* "coordinator, broadcast-channel, 1234"
** change the broadcast channel internally
** tell all registered test units to use the new channel
* "coordinator, test-timeout, 100"
* "coordinator, start, quiet, all"
** broadcast start command to every test unit or group with instructions to perform minimal reporting (summary only)
* "coordinator, start, normal, test_group, math"
** broadcast start command to specified group with instructions to perform normal reporting (failures and summary)
* "coordinator, start, verbose, test_unit, multiplication"
** broadcast start command to specified test unit with instructions to perform verbose reporting
=== Collect results ===
* Keep track of total known registrants running in this session.
* Track number of test units that report all tests passed.
* Track number of test units that report failure.
* Track number of test units that failed to report within the timeout.
=== Tests Complete ===
==== Commands ====
* "coordinator, summary, say, 0"
* "coordinator, summary, email, phoenix@secondlife.com"
* "coordinator, summary, http, http://example.com/" 
* "coordinator, set-up"
** return to setup
* "coordinator, reset"
** tell every registered test unit to return to the default broadcast channel.
** reset internal state, and go to the initialization phase.
= Controller =
We will also need a simple default controller which understands the coordinator protocol and provides a touch and dialog menu controls so that an avatar can approach the controller and run every test unit or test group.
== Requirements ==
* The controller must be separate from the coordinator -- not the same script, nor required to be in the same linked set.
* The controller does not need to be a single lsl script.
* The controller must provide avatar controller interface.
** An avatar can control the tests through a series of dialog menus or prompted typed commands.
** The controller should activate when an avatar touches it.
** There should be a shiny red button inviting the touch. :)
* The controller knows enough of the coordinator protocol to initiate a set of tests.
* The controller must be able to initiate:
** A single test unit
** A test group
** All tests
* The controller can assume that only one coordinator is available to run tests
* The controller should allow specification of test results endpoint:
** llSay on channel 0
** email to a specified address
** http
= Test Units =
Each test unit will have a small amount of boiler plate code to talk to the controller and understand the communication enough to change to another channel, start tests, and report test results.
== Requirements ==
* All test units reset to a default test broadcast channel of 0.
* All test units will be in a test group and the unit knows the group name.
* All test units have a name.
* Register to a coordinator on announcement of registration.
* Change communication channel on request of the coordinator
* Run its tests on the request of the coordinator
* Report test results with multiple levels of output
** verbose: everything
** normal: failures and summary information
** quiet: summary information only