Repository: qtile/qtile

Issue:
migrations need version annotations
We have cases where migrations are non-idempotent, so we should only run them when upgrading from a particular version to a different one.

My suggestion is to write a `qtile_version = "$whatever` into users' configs after they've done a migration, and then we can parse that and only run migrations newer than that.

We'll need to annotate the migrations we have somehow, but we can do that reasonably easily (a decorator, make them callable classes, whatever).

Thoughts?


Discussion:
https://github.com/qtile/qtile/pull/2346#issuecomment-812575166
I like the idea. 

The scenario mentioned in that linked thread should, hopefully, be pretty rare but it may be enough to warrant this approach.
Do you mean the latest tag, and then compare based on semantic versioning? If so, you'd still have the problem mentioned in #2346 for those that run a non-tagged version of Qtile.
I think if you're running a non-tagged version you're on your own :). We can just assume you're at the latest or something.
Yeah I do agree it's a edge case, but a simple one to handle if we use a timestamp or a unique id per migration. Is there any downside I don't see?
> Is there any downside I don't see?

I guess I don't understand your design. The idea would be to write a big dictionary of stuff about what migrations we've performed into people's configs?
I think it's exactly the same as yours except that instead of writing `qtile_version = 0.18.0` you'd either write a migration id, i.e. `last_migration = 42`, or some sort of timestamp of the last `qtile migrate` run, for example `last_migration = 20210404120510`, that you would then compare against the timestamp of each migration.
Or since we keep a list of the migrations and their order, we could just put the name of the migration itself in the configuration. I think each option has subtle differences in terms of what you can or cannot do later, but any would do really.
Referring back to the `padding`/`padding_x` issue: how do you catch the scenario where you've got a new user who's never used `qtile migrate` before and has a config that's already compliant with that change so doesn't need udating? They may want `padding` in some widget but, the first time they use `qtile migrate` for future changes, it's going to make the change and break their confg.

If this is this the only edge case we are concerned about then I wonder if we're thinking about this the wrong way i.e. is the real issue here how the `PaddingMixin` works? Would it be easier to rethink how it's used?
> Referring back to the `padding`/`padding_x` issue: how do you catch the scenario where you've got a new user who's never used `qtile migrate` before and has a config that's already compliant with that change so doesn't need udating? They may want `padding` in some widget but, the first time they use `qtile migrate` for future changes, it's going to make the change and break their confg.

We can add the `qtile_version` or `last_migration` to the default config file when copying it.

> If this is this the only edge case we are concerned about then I wonder if we're thinking about this the wrong way i.e. is the real issue here how the `PaddingMixin` works? Would it be easier to rethink how it's used?

Yeah probably, but it's going to happen again for sure so let's just support it anyway.
> I think it's exactly the same as yours except that instead of writing qtile_version = 0.18.0 you'd either write a migration id, i.e. last_migration = 42

I see, that does seem nice. Then when we delete migrations because they're old, we can just keep the version_offset or something, so this number will just monotonically increase forever.
So I was thinking about this a bit more, and I think there's a bit of a chicken and egg problem with writing things into the config. Suppose a qtile update changes something (moves an import, for example). We won't be able to import this old config via importlib or similar, because the newer libqtile won't have the import in the place the old config expects it.

We could probably "parse" the value of migrations_performed or whatever it's called with regexes or something, but that feels a little ugly.

We could provide a command line `--migrate-from=74` that would perform the 74th migration onwards, but then people have to know the number they were previously at, which is kind of a pain. We could also do `--migrate-from=0.17.0` which might be more comprehensible, but might still be annoying to figure out if you install the new version and forget what you were previously on.

We could put the value in some xattr, but git doesn't track xattrs so people will probably lose that eventually.

Thoughts?
It may be ugly but I had assumed that that parsing route would be the way we needed to do this. While the other ideas are neater they're much more reliant on the user and introduces more chance of it going wrong.
Funny. Yeah we could either parse, which I think is reasonable in that use case, or write in another file (`.py` or not), e.g. `echo {migration_number} > .config/qtile/.migrate`. 
I thought about a separate file, but people won't know to commit that to git :(
Why would they need to commit it?
What happens if they move to a machine, lose this file, and a non-idempotent migration runs again? Presumably that will break things...
I don't think it's our responsibility to maintain a backup system for user's config files. Maybe we can just have it at `~/.config/qtile/migrations`, not hidden, in case they might miss it, but other than that I think it's up to them.
On the other hand I can foresee us having to debug people who break this :p 
> On the other hand I can foresee us having to debug people who break this :p

My thoughts exactly. Though maybe it's a technical solution to a social problem...

Resolving patch:
diff --git a/libqtile/scripts/migrate.py b/libqtile/scripts/migrate.py
--- a/libqtile/scripts/migrate.py
+++ b/libqtile/scripts/migrate.py
@@ -162,7 +162,7 @@ def do_migrate(args):
 
     for m in MIGRATIONS:
         q = bowler.Query(config_dir)
-        m(q).execute(interactive=args.interactive, write=True)
+        m(q).execute(interactive=not args.yes, write=True)
 
     changed = False
     for py, backup in file_and_backup(config_dir):
@@ -193,8 +193,8 @@ def add_subcommand(subparsers, parents):
         help="Use the specified configuration file (migrates every .py file in this directory)",
     )
     parser.add_argument(
-        "--interactive",
+        "--yes",
         action="store_true",
-        help="Interactively apply diff (similar to git add -p)",
+        help="Automatically apply diffs with no confirmation",
     )
     parser.set_defaults(func=do_migrate)


Test patch:
diff --git a/test/test_migrate.py b/test/test_migrate.py
--- a/test/test_migrate.py
+++ b/test/test_migrate.py
@@ -13,7 +13,7 @@
 
 def run_qtile_migrate(config):
     cmd = os.path.join(os.path.dirname(__file__), '..', 'bin', 'qtile')
-    argv = [cmd, "migrate", "-c", config]
+    argv = [cmd, "migrate", "--yes", "-c", config]
     subprocess.check_call(argv)