Template verifier.py

This documentation and policy applies to code since the 1.18 viewer release.
Note: The build check has been replaced with an md5sum check. For all practical purposes, the message template file is frozen. Should a situation arise where it needs to change, the validation script below should be run manually, and the updated md5sum digest placed into indra/cmake/TemplateCheck.cmake
= The script =
== template_verifier.py ==
scripts/template_verifier.py
The script has two modes:
* --mode="development"
* --mode="production"
When run, it fetches the authorative message template (http://url_to_be_determined) to check messages for compatibility (using the below rules).
=== At prebuild time ===
The script fetches master_message_template.msg and runs a check against the message template in your branch.
* If the check fails, the build fails. 
* If the check succeeds, the build succeeds.
* If there are compatible changes in the message template, it prints out a summary of the changes (this might be in a hidden console depending on your development environment).
= The Policy =
Only compatible changes can be made to the production message template. The following are compatible changes:
* Adding new blocks to the end of a message
* Marking as UDPDeprecated an undeprecated message. See example.
** "Marking a message UDPDeprecated" means we've made extensions to the message that are only in the TCP/LLSD transport -- it can't be sent by UDP subsequently (this is OK because it will be possible to send almost any message via either UDP or TCP/LLSD). This also means that the message template no longer accurately describes the semantics of the message.
* Marking as Deprecated an undeprecated message or a UDPDeprecated message. See example.
** Marking a message Deprecated means that we don't send it anymore. We've either removed the functionality or replaced it with functionality that doesn't use the message system.
* Adding a new message (why are you doing this, you fool?)
* Arbitrary comment changes. More documentation is better, unless it's wrong, in which case it should be removed.
= Checks to enforce The Policy =
In order to enforce the compatibility rules, we have a well-known message template against which yours will be checked at certain times in the development cycle. Each check compares the master against your current message template. Every time you build or deploy, the production message template will be downloaded, and your changes compared against it.
The message template will be checked at the following points in time:
* During a build (on all three platforms)
* Just prior to deploy (via the farm command) to:
** Development grids
** Production grid
== Your changes in relation to "Base" ==
When the check is run, it determines your message template's relationship to the production grid's template. This relationship will be one of:
; Same : You have the same message template (modulo comments).
; Older : The production template has compatible changes that are more advanced than your template (e.g. the production template has more blocks in some message, production has deprecated some messages, etc). This means that the production template advanced while you were hacking on your branch, or you branched off an old release.
; Newer : Your template has compatible changes that are more advanced (e.g. you added some blocks)
; Mixed : Both you and the production template have sections that are newer than each other, but in different sections of the template. This is akin to a merge conflict. (e.g. you've added a new message, and the production template deprecated a message).
; Incompatible : You've made some changes that are incompatible with the production template. Bad developer, no biscuit. (e.g. changing the name of a variable, changing the size of a variable, deleting a block, pretty much anything)
== Conditions for passing The Compatibility Test ==
Each point where the compatibility check is made has different conditions that must be satisfied to pass. Depending on what you're doing, the potential message template states listed above may be acceptable or unacceptable. 
* If you are deploying to production OR checking in to release: 
** ACCEPTABLE: Same, Newer
** UNACCEPTABLE: Incompatible, Older, Mixed 
* If you're building locally OR deploying to a dev grid:
** ACCEPTABLE: Same, Newer, Older, Mixed
** UNACCEPTABLE: Incompatible
== Examples of Compatible Changes ==
THIS IS THE ORIGINAL BLOCK TO WHICH OTHERS IN THIS SECTION ARE COMPARED.
CHANGES IN EACH SUBSEQUENT SECTION BOLDED AND DISPLAYED IN GREEN.
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 } // four bytes
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
=== Marking UDPDeprecated ===
 {
 TestMessage Low 23 Trusted Zerocoded UDPDeprecated
 {
 TestBlock1 Single
 { Test1 U32 } // four bytes
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
=== Marking Deprecated ===
 {
 TestMessage Low 23 Trusted Zerocoded Deprecated
 {
 TestBlock1 Single
 { Test1 U32 } // four bytes
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
=== Adding Block To End ===
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 } // four bytes
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 {
 NewBlock Single
 { TestNew U32 }
 }
 }
=== Adding/Modifying Comments ===
 // Test message, always ignored
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 } // one word
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
== Examples of Incompatible Changes ==
THIS IS THE ORIGINAL BLOCK TO WHICH OTHERS IN THIS SECTION ARE COMPARED.
CHANGES IN EACH SUBSEQUENT SECTION BOLDED AND DISPLAYED IN RED.
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
=== Variable Changed Type ===
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 F64 }
 { Test2 U32 }
 }
 }
=== Variable Order Changed ===
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test2 U32 }
 { Test1 U32 }
 }
 }
=== Block Order Changed ===
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 {
 TestBlock1 Single
 { Test1 U32 }
 }
 }
=== Block Name Changed, Variable Name Changed ===
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlockX Single
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Change1 U32 }
 { Test2 U32 }
 }
 }
=== Message Number Changed ===
 {
 TestMessage Low 100 Trusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
=== Message Properties Changed ===
 {
 TestMessage High 23 NotTrusted Zerocoded
 {
 TestBlock1 Single
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }
=== Block Properties Changed ===
 {
 TestMessage Low 23 Trusted Zerocoded
 {
 TestBlock1 Multiple 2
 { Test1 U32 }
 }
 {
 NeighborBlock Multiple 4
 { Test0 U32 }
 { Test1 U32 }
 { Test2 U32 }
 }
 }