Repository: mesonbuild/meson

Issue:
New choice in array option is ignored
**Describe the bug**
When a new choice "newchoice" is added to an existing array option "foo", _meson configure_ will not list the new choice in the "Possible Values" column, and _meson -Dfoo=newchoice_ complains that "newchoice" is not valid.

`ERROR: Options "newchoice" are not in allowed choices: "oldchoice"`

**To Reproduce**
[choice.tar.gz](https://github.com/mesonbuild/meson/files/4839477/choice.tar.gz)

1. Run _meson build_
2. Edit meson_options.txt as follows:
```
-        choices : ['oldchoice'],
+        choices : ['oldchoice', 'newchoice'],
```
3. Run _meson configure build_. The "foo" option should report "newchoice" as a Possible Value.

**Expected behavior**
The new choice should be considered by meson as a possible value and accepted when modifying the option with -D.

**system parameters**
* Operating system: Gentoo Linux
* Python version: 3.7
* Meson version: 0.52.1 (tested the master branch as well)



Discussion:
You need to pass --reconfigure in that case.
I'm afraid running _meson --reconfigure_ in the build directory doesn't help (neither does _meson setup --reconfigure_).
The problem seems to be in CoreData.merge_user_options. The code handles the addition of options, as well as the modification of an option type, but not the modification of choices.
Just to help for clarification, (or testing?) This is experienced when we add (or remove) support for a new pipeline in libcamera.

Currently available pipeline handlers are defined (and user selectable) by an option in meson_options.txt:
  https://git.linuxtv.org/libcamera.git/tree/meson_options.txt

```
option('pipelines',
        type : 'array',
        choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc'],
        description : 'Select which pipeline handlers to include')
```

This option is later parsed under src/libcamera/pipeline/meson.build to determine which subdirs to include

https://git.linuxtv.org/libcamera.git/tree/src/libcamera/pipeline/meson.build:
```
foreach pipeline : get_option('pipelines')
    subdir(pipeline)
endforeach
```

And, I can confirm I also see the issue here that Laurent states above, that even running 'meson --reconfigure build' does not update the default options available from the choices array:
```
git diff
diff --git a/meson_options.txt b/meson_options.txt
index ea38f1553240..c51508e52f99 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -16,7 +16,7 @@ option('gstreamer',
 
 option('pipelines',
         type : 'array',
-        choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc', 'vivid'],
+        choices : ['ipu3', 'raspberrypi', 'rkisp1', 'simple', 'uvcvideo', 'vimc', 'vivid', 'newsoc'],
         description : 'Select which pipeline handlers to include')
 
 option('qcam',
```

And the option can not be used, (either implicitly, or explicitly)
```
meson --reconfigure build -Ddocumentation=false -Dpipelines=newsoc
The Meson build system
Version: 0.54.3
Source dir: /home/libcamera/libcamera
Build dir: /home/libcamera/libcamera/build
Build type: native build

meson.build:3:0: ERROR: Options "newsoc" are not in allowed choices: "ipu3, raspberrypi, rkisp1, simple, uvcvideo, vimc, vivid"

A full log can be found at /home/libcamera/libcamera/build/meson-logs/meson-log.txt



```

Resolving patch:
diff --git a/mesonbuild/coredata.py b/mesonbuild/coredata.py
--- a/mesonbuild/coredata.py
+++ b/mesonbuild/coredata.py
@@ -689,14 +689,24 @@ def get_external_args(self, for_machine: MachineChoice, lang):
     def get_external_link_args(self, for_machine: MachineChoice, lang):
         return self.compiler_options[for_machine][lang]['link_args'].value
 
-    def merge_user_options(self, options: T.Dict[str, T.Union[str, bool, int]]) -> None:
+    def merge_user_options(self, options: T.Dict[str, UserOption[T.Any]]) -> None:
         for (name, value) in options.items():
             if name not in self.user_options:
                 self.user_options[name] = value
-            else:
-                oldval = self.user_options[name]
-                if type(oldval) != type(value):
-                    self.user_options[name] = value
+                continue
+
+            oldval = self.user_options[name]
+            if type(oldval) != type(value):
+                self.user_options[name] = value
+            elif oldval.choices != value.choices:
+                # If the choices have changed, use the new value, but attempt
+                # to keep the old options. If they are not valid keep the new
+                # defaults but warn.
+                self.user_options[name] = value
+                try:
+                    value.set_value(oldval.value)
+                except MesonException as e:
+                    mlog.warning('Old value(s) of {} are no longer valid, resetting to default ({}).'.format(name, value.value))
 
     def is_cross_build(self, when_building_for: MachineChoice = MachineChoice.HOST) -> bool:
         if when_building_for == MachineChoice.BUILD:
diff --git a/run_unittests.py b/run_unittests.py
--- a/run_unittests.py
+++ b/run_unittests.py
@@ -3325,6 +3325,46 @@ def test_free_stringarray_setting(self):
         self.setconf("-Dfree_array_opt=['a,b', 'c,d']", will_build=False)
         self.opt_has('free_array_opt', ['a,b', 'c,d'])
 
+    def test_options_with_choices_changing(self) -> None:
+        """Detect when options like arrays or combos have their choices change."""
+        testdir = Path(os.path.join(self.unit_test_dir, '84 change option choices'))
+        options1 = str(testdir / 'meson_options.1.txt')
+        options2 = str(testdir / 'meson_options.2.txt')
+
+        # Test that old options are changed to the new defaults if they are not valid
+        real_options = str(testdir / 'meson_options.txt')
+        self.addCleanup(os.unlink, real_options)
+
+        shutil.copy(options1, real_options)
+        self.init(str(testdir))
+        shutil.copy(options2, real_options)
+
+        self.build()
+        opts = self.introspect('--buildoptions')
+        for item in opts:
+            if item['name'] == 'combo':
+                self.assertEqual(item['value'], 'b')
+                self.assertEqual(item['choices'], ['b', 'c', 'd'])
+            elif item['name'] == 'arr':
+                self.assertEqual(item['value'], ['b'])
+                self.assertEqual(item['choices'], ['b', 'c', 'd'])
+
+        self.wipe()
+
+        # When the old options are valid they should remain
+        shutil.copy(options1, real_options)
+        self.init(str(testdir), extra_args=['-Dcombo=c', '-Darray=b,c'])
+        shutil.copy(options2, real_options)
+        self.build()
+        opts = self.introspect('--buildoptions')
+        for item in opts:
+            if item['name'] == 'combo':
+                self.assertEqual(item['value'], 'c')
+                self.assertEqual(item['choices'], ['b', 'c', 'd'])
+            elif item['name'] == 'arr':
+                self.assertEqual(item['value'], ['b', 'c'])
+                self.assertEqual(item['choices'], ['b', 'c', 'd'])
+
     def test_subproject_promotion(self):
         testdir = os.path.join(self.unit_test_dir, '12 promote')
         workdir = os.path.join(self.builddir, 'work')


Test patch:
diff --git a/test cases/unit/84 change option choices/meson.build b/test cases/unit/84 change option choices/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/unit/84 change option choices/meson.build	
@@ -0,0 +1 @@
+project('change option choices')
diff --git a/test cases/unit/84 change option choices/meson_options.1.txt b/test cases/unit/84 change option choices/meson_options.1.txt
new file mode 100644
--- /dev/null
+++ b/test cases/unit/84 change option choices/meson_options.1.txt	
@@ -0,0 +1,13 @@
+option(
+    'combo',
+    type : 'combo',
+    choices : ['a', 'b', 'c'],
+    value : 'a',
+)
+
+option(
+    'array',
+    type : 'array',
+    choices : ['a', 'b', 'c'],
+    value : ['a'],
+)
diff --git a/test cases/unit/84 change option choices/meson_options.2.txt b/test cases/unit/84 change option choices/meson_options.2.txt
new file mode 100644
--- /dev/null
+++ b/test cases/unit/84 change option choices/meson_options.2.txt	
@@ -0,0 +1,13 @@
+option(
+    'combo',
+    type : 'combo',
+    choices : ['b', 'c', 'd'],
+    value : 'b',
+)
+
+option(
+    'array',
+    type : 'array',
+    choices : ['b', 'c', 'd'],
+    value : ['b'],
+)