mozilla/socorro — pull request #951 by twobraids (2012-11-15 22:46:54+00:00)
Title: No Merge - simplest ini proposal yet

We've been struggling with how to set up ini files for what I see is a conflicting set of goals:

1) Simply works out of the box for dev
2) uses ini files because they are clear and easier to understand than Python code
3) Separates division of maintainance labor between dev and ops
4) Keeps local config out of the repo

After messing around with this seemingly forever, I'm going to propose a radical simplification:

one ini per app and optionally one ini per external resource. This scheme is simple to understand and simple to implement. Here's how it addresses the issues above.

1) the defaults will be for a stand alone system without using HBase. When deployed for the first time, -dist files are copied to their .ini equivalents. If it is desired to have a production system using HBase, the person doing the installation will have to run each app in turn having it write out an HBase aware ini file. We'll have instructions on how to do this. I believe it it is inevitable that we should create samples of ini files for more complicated configurations.

2) it turns out that ini files aren't as easy as initially thought. We added the +include feature to get more flexibility. Only with that could we even concieve of the separation of maintenance responsibity. We're finding that +include isn't enough and now were talking about optional includes and changing ini to allow for repeating/overriding key/value pairs. If we're going as far as inventing a domain a specific language, we ought to just use Python instead.

3) we've been trying to separate roles in ini file maintanance by using include files. This results in a profusion of ini files because ini files are line order dependent and do not allow repeating keys or sections. This adds only adds complexity (eight ini files for the processor alone). In this proposal, I've marked values interesting to ops with two different comments: "OPS - set once" and "OPS - may change". Let the OPS folks see the whole file rather than a proliferation of disjoint parts.

4) it has been argued that separating roles into include flies would mean that potential volitile key/value pairs would not get over written when a new version is deployed. I would suggest that a better solution the installation system should never overwrite an existing ini file when copying -dist files to ini files. If they layout or values in an -dist file change between versions of socorro, then there must be manual tuning of existing ini files anyway. Spliting -dist files into ones that are updatable while others are not is again adding complexity to resolve an infrequent problem.

The install system should copy -dist files to the .../config directory iff ini files do not already exisit there. The person doing an upgrade would then use the release notes and a diff tool to copy changes into their exisiting ini files in the .../config directory or /etc/socorro directory.

We ought to try it this way first and if we find that it is untenable, then we should consider more complicated automated methods.