Repository: mesonbuild/meson

Issue:
Feature Request: Deal with upgrades better
Hi Jussi,

I just upgraded from .27-research to .28-research and got an error message - which I expected. Is there a flag or target that could be added to force this upgrade to happen? And by upgrade, I mean delete the old stuff and regenerate. It's awkward to tell people to `rm -rf build; mkdir build; <cc flags> meson build` when I upgrade meson on the systems and their build breaks. I'm trying to replace our crusty make based build and this, IMO, is a wart.  (meson is still awesome, though!)

```
[1/1] Regenerating build files
Traceback (most recent call last):
  File "/opt/meson/share/meson/meson.py", line 211, in run
    app.generate()
  File "/opt/meson/share/meson/meson.py", line 126, in generate
    env = environment.Environment(self.source_dir, self.build_dir, self.meson_script_file, self.options)
  File "/opt/meson/share/meson/environment.py", line 69, in __init__
    self.coredata = coredata.load(cdf)
  File "/opt/meson/share/meson/coredata.py", line 199, in load
    (obj.version, version))
RuntimeError: Build tree has been generated with Meson version 0.27.0-research, which is incompatible with current version 0.28.0-research.
```



Discussion:
We could do it automatically, I guess, but doing this loses all build tree state and files that are there (it is common for people to copy extra files to their build dir to use for manual tests etc). Blowing those away without the user's consent is not really nice. It is also a fair bit of work trying to transition all configuration settings from one version to another automatically.

Once Meson gets a bit more popular, people will mostly get it via distro packages. Those update only rarely so this should not be such a big of an issue. If you are running latest trunk, well, that's one of the downsides of being on the bleeding edge. :)

So yes, the current situation is not really optimal, but fixing it properly would take a lot of resources that are, as always, scarce.

Would you find this acceptable? Would something like this work?
- Capture the `os.environ` when the user first calls `meson` and store somewhere in the build dir
- Capture the command line args meson was run with and store somewhere in the build dir
- Add an argument like `--reinitializie-meson-and-lose-all-build-directory-data` or maybe something a bit shorter
- When the users types `meson --reinitializie-meson-and-lose-all-build-directory-data <build-dir>`, meson would
  - Overwrite the current environment with the stored env.
  - Overwrite the current command line args with the stored ones
  - Blow away everything inside of the `build-dir`
  - Finally, meson would run itself like normal

To me the entire point of a separate build dir is that its contents can be considered volatile/junk. In any case, wouldn't it be enough to just remove some meson + ninja dirs, or does the build dir have to be empty initially?

I think the way autotools works in maintainer-mode is quite nice and worth emulating (I realise it's not the exact same use case).

> To me the entire point of a separate build dir is that its contents can be considered volatile/junk. In any case, wouldn't it be enough to just remove some meson + ninja dirs, or does the build dir have to be empty initially?

If meson is happy with a non empty build dir then so am I! No need to blow away everything.

> I think the way autotools works in maintainer-mode is quite nice and worth emulating (I realise it's not the exact same use case).

I just looked up [maintainer-mode](https://blogs.gnome.org/desrt/2011/09/08/am_maintainer_mode-is-not-cool/) - it looks like it would do the opposite of what meson does currently - ie: not update the ninja file unless a flag gets passed in. Personally, I think the way meson behaves now is ideal. 

To see what the problem is let's assume that you first run Meson on your project. Then you work on it and add a new Meson option `foo`. Then you change it from the default value. Once this is done you update. If the system is not careful and loses the value of this setting (for example, by going back to the default), then this may lead to unexpected behaviour for the user.

Even if you stored all these changes there is _still_ an error case. Suppose you edit your Meson options, update Meson and only _then_ try to compile. You have done two unrelated changes and there is no reliable way to recover from that.

The current approach is not not the most user friendly but it is a) explicit and unambiguous b) simple and understandable and c) does not have unexpected failure modes.

Adding a target that does the equivalent of `rm interesting files here; meson.py <src dir> <build dir>` is doable but it would erase all build configuration state.

Hmm, I hadn't considered changing values via mesonconf... automatic regeneration is tricky.

Is a released version, the user doesn't see the stack dump, right? I only see it because I'm running HEAD?

I just got pinged again about this again - total confusion.

Can we print out a more reasonable (for meson noobs) error message instead of a stack dump and a message that doesn't tell you how to resolve the issue?

Yes, we should print a better error message. A stack trace just looks terrible and people's eyes glaze over and they miss the message at the bottom.


Resolving patch:
diff --git a/mesonbuild/coredata.py b/mesonbuild/coredata.py
--- a/mesonbuild/coredata.py
+++ b/mesonbuild/coredata.py
@@ -149,18 +149,22 @@ def set_builtin_option(self, optname, value):
             raise RuntimeError('Tried to set unknown builtin option %s.' % optname)
 
 def load(filename):
-    with open(filename, 'rb') as f:
-        obj = pickle.load(f)
+    load_fail_msg = 'Coredata file {!r} is corrupted. Try with a fresh build tree.'.format(filename)
+    try:
+        with open(filename, 'rb') as f:
+            obj = pickle.load(f)
+    except pickle.UnpicklingError:
+        raise MesonException(load_fail_msg)
     if not isinstance(obj, CoreData):
-        raise RuntimeError('Core data file is corrupted.')
+        raise MesonException(load_fail_msg)
     if obj.version != version:
-        raise RuntimeError('Build tree has been generated with Meson version %s, which is incompatible with current version %s.'%
+        raise MesonException('Build directory has been generated with Meson version %s, which is incompatible with current version %s.\nPlease delete this build directory AND create a new one.'%
                            (obj.version, version))
     return obj
 
 def save(obj, filename):
     if obj.version != version:
-        raise RuntimeError('Fatal version mismatch corruption.')
+        raise MesonException('Fatal version mismatch corruption.')
     with open(filename, 'wb') as f:
         pickle.dump(obj, f)
 


Test patch: