Reviewed all the changes needed for the RL updates.
Added RL Update.txt with notes on what all the changes are.
Added RL Updates.xlsx which summarizes which revisions affect which files.
Updated the RL unit test files to set hrl-discount to false and look for the values
that result from this setting.
This is the parameter setting in the current CSoar, and the test works with it now.
However, setting this parameter in the test should be removed when
the default parameter setting is updated.
Moved ReinforcementLearningCommand from the commands package to the learning.rl package.
Modified ReinforcementLearningCommand to be set up using the standard provider pattern.
Removed it from StandardCommands.
Added code to do the plain rl command.
Made an RLPrintHelper class so the calls look just like the ones
used by epmem to the standard utils.PrintHelper, but the output
formatting looks exactly like the current CSoar.
If CSoar is ever updated to use the standard formatting that epmem and smem use,
it should be easy to throw away the RLPrintHelper and use the standard one.
Made a major refactoring by creating the ReinforcementLearningParams class
and rewriting it and the things that use it to conform closely to the way
params are handled in epmem.
Have several parameters fully implemented, more are needed.
Both the JUnit test and some hand-run tests on the debugger work fine now.
Implemented the rest of the non-Experimental parameters.
All these seem to work correctly and the unit test uses them and it works.
The printout copies that of CSoar exactly, including the Experimental properties.
The latter are not really implemented yet, so the printout for them is faked.
The temporal-discount parameter is implemented and works but it is not printed
since CSoar does not print it.
Updated the notes in RL Update.txt.
Created the chunk-stop parameter and the code to set and print it.
The actual effect in the rete is not done yet.
Got all the code changes for 12380 done.
Upgraded both RLTests.java and ReinforcementLearningTest.java to look at
the current value of hrl-discount and use different expected values
for each possible case.
Still need a unit test that tests that chunnk-stop actually works.
Oops, needed to check in this test resource in its latest form as well.
Added unit tests to test that the chunk-stop parameter really works.
Complete the work for r12380.
This change makes independent versions of testRLUnit for each
possible value of hrl-discount.
Fixed bug r12630.
RL.rl_tabulate_reward_value_for_goal now does not create a slot if none exists.
All the tests pass.
Can't find the files needed for r12630.
Going on to the next.
We are not going to work on r12639 for the forseeable future.
Made the changes needed for r13271.
However, there are some issues I need to check with Bob.
In ReinforcementLearning.java, 147-150, I made the code conform
to what CSoar says almost, but the new if statement in CSoar calls
rl_reset_data, which does not exist in JSoar.
Either this is not needed in JSoar, in which case the whole event listener
can be eliminated, or it is needed and we need to see how to implement it.
Also, the code removed in this change was explicitly tested for in a test called
testSetsExplorationModeToEpsilonGreedyWhenEnabled.
Since the code this test tests is now gone, the test fails.
I deleted this test.
All other test pass just fine.

THESE THINGS SHOULD BE REVISITED WHEN I CAN GET INPUT FROM BOB M.
Started on some initial changes for r13404, which is a big one.
Mainly just implemented the decay-mode parameter in the rl command,
not the realy internal logic yet.]
Committing this much so I can finish up the stuff for 13271 before
drowning in the depths of 13404.
Added the rl_reset_data function, thus completing the code changes for r13271.
This completes this revision with the exception of having a unit test
that tests the functionality of rl_reset_data.
Partial implementation of r13404.
There's a bunch more to do, but this much passes all unit tests.
WATCH OUT!!  In the middle of working on 13404, I realized that several other revisions came after it in Bob's list but before it chronologically.
For this reason I'm going to commit these partial changes to 13404, put that on the shelf,
and then come back to it when all the earlier ones are done.
When returning to 13404 it would be good to review this commit and see where things are.