jiebi/CodeConvo huggingface dataset:
On today's master () - and probably others - building the attribute of many package produces an output that differs from the output on another architecture. Nix will display a hash mismatch. I tried with x8664-linux and aarch64-linux. Execute this on aarch64 and x8664. The aarch64 build produces the below output while x8664 is happy with the hash in nixpkgs. Please run and paste the results. system: host os: multi-user?: sandbox: version: nixpkgs: and system: host os: multi-user?: sandbox: version: channels(root): channels(andi): nixpkgs:
x8664 build produces same output for me: system: host os: multi-user?: sandbox: version: channels(root): nixpkgs:
I noticed something similar on macOS in Could some recent change have caused this? Perhaps ?
I reverted in a branch on my fork. My x86_64-linux laptop doesn't complain about the hash sum - as before. The aarch64-linux community box also doesn't complain anymore. Can anyone else confirm this?
I just made a diff of : The left side is aarch64, the right side is x86_64. Both are done on the same git revision ().
What a weird behavior that is missing.
Probably related:
I think this may be related: I also see different hashes on NixOS 19.03 vs. stable. I use the branch, but when CI builds the derivation on 19.03, the hashes do not match. If I build on NixOS 19.03 with from master, the hashes do match. This is quite annoying, because it makes it impossible to have package sets that work on NixOS stable and unstable without using rustPlatform from a nixpkgs newer than 19.03. I'll try to revert and see if I get the same hashes on 19.03 and master.
That's indeed it. When I revert , I get the same as 19.03. Maybe this change should be reverted? Since NixOS is not a rolling distribution and people run both stable and unstable, it would really be nice if package sets and overlays work on both versions. This also makes backporting changes easier.
Diff between the vendored dependencies: Here the differences also seem to be missing dotfiles (e.g. and ). I think this is the upstream PR that changed this behavior: (Edit: oops, I just saw that already mentioned the associated commit.)
is a critical update and reverting it will break multiple Rust packages. Rather I think we should get that change pushed into the channel, and then update any Rust packages whose changes as a result.
The only bummer is that it also breaks people's own packages on the channel as well.
Yeah, it would be bad to break people's derivations as an update to a stable release. At any rate, the 19.09 release notes should probably have a blurb that points out that hashes will change due to changes, so that people who update from 19.03 to 19.09 are prepared. There are probably also derivations in that have incorrect hashes now, since the file that is generated by is now excluded vendored dependencies.
I’ve updated the title of the issue to better reflect what I think we now know. Additionally, I’ve opened to update the hashes for all packages that haven’t been updated in master yet. I would appreciate a review on it. Once that’s merged, I can do the same thing for the affected release branch.
Looks like I misunderstood and this hasn’t made its was into a release branch yet, so we’re almost done here. Somebody just needs to add the release note for 19.09.
with the changelog PR () being merged, is there anything else missing?
Don't think so. :) Somebody can shout if I'm wrong.
Describe the bug The new version of Agda is working fine on other systems, but not NixOS. I've reported a bit more about what is known at To Reproduce Steps to reproduce the behavior: Agda 2.6.1 any file (e.g, ) Expected behavior Agda checks the file quickly and exits. Additional context Issue in the Agda repository: agda/agda Metadata Maintainer information:
I believe that this is fixed by rework of our Agda setup ! (Which is included in the most recent nixos-unstable.) Can you confirm? :-) $ echo module Main where && time agda Checking Main (). real 0m0.132s user 0m0.104s sys 0m0.028s Looks reasonable!
I haven't had chance to test the new stuff, but this issue probably should be closed anyway, as it's being worked on upstream (). I'm doing fine now using the workaround of adding option .
Describe the bug I encountered this bug when trying to use that does FFI. The particular call in general was the equivalent dlopen on the libc library and resolving to the system one which would cause a SIGSEGV. Digging into the source some more, you discover that Java sets a default value for if it's not explicitly set. Adding system libraries (i.e. /usr/lib) breaks/invalidates the hermetic invariant needed by Nix. These library paths are also searched first before those set in the DTRUNPATH of the ELF header as documented. I suggest the OVERRIDELIBPATH is either set to empty string maybe or even the contents of the ELF RUNPATH header itself. To Reproduce I was using JRuby which makes it easy to reproduce by using a gem which ultimately calls the JNR FFI. (The following reproduction results in the SIGSEGGV) Expected behavior Expected behavior is for the FFI call to work! This can be resolved by setting to the correct lib. NOTE: I could not merely set LDLIBRARYPATH because that would apply to everything in my nix-shell. Furthermore, wrapping java is somewhat tedious, seemed pragmatic for now. Notify maintainers Metadata Maintainer information: All JDK derivations
If you are curious for the JNR call that loads the C library:
OVERRIDELIBPATH is not sufficient, because it still SYSEXTDIR/lib, which is somewhere in . We'll need a patch. This also means we might as well write C code to get the right rpath value, even when . Although less useful, I'd be ok with a patch that just makes it . At least that will give a somewhat useful error message instead of a crash. ~It seems like OVERRIDELIBPATH was designed for cases like ours. Setting it to a static rpath value is tricky because that way we're duplicating it in a way that can't be later. It a statically known value. So I'd go with the empty value.~
I'm hoping (haven't tested) that with an empty OVERRIDELIBPATH the DTRUNPATH at least will be searched for. Do you want to work on this change together ?
I don't have a lot of experience yet with the JDK derivation (8.nix etc..)
I don't have a lot of experience with that either, but I do implement RubberDuck and I'm happy to review Nix+Java ecosystem stuff.
Tagging who seems to do a lot of JVM Java work as well.
Reopening: more versions to be done
I marked this as stale due to inactivity. &rarr;
[x] OpenJDK 8: [x] OpenJDK 11: [ ] OpenJDK 17
why does openjdk 17 not have a mr?
Do you have a specific problem? With the example above I could not reproduce the jvm crash.
sorry, i thought i was hitting this bug, but turns out that the bug i hit is different
Is there a way to reproduce this issue for openjdk17? Otherwise I'd propose to close the issue.
Afaict, it has not been written yet. You may not have the "right" libraries in to make it crash. I don't think we need a reproducer. We know that the unpatched behavior is bad, and it is highly unlikely that upstream's behavior changes, because the rest of the world depends on it. did you have a specific reason not to patch OpenJDK 17? Could you patch it?
That patch does not seem to be too complicated. I could prepare it, but I'd feel better if I had some use case to test if the patch is working.
I mostly just forgot about it sadly. Some other stuff picked up and it kind of fell by the wayside sorry about that. On Thu, Jul 28, 2022, 2:19 AM Benjamin Asbach
Closing since it looks like the last bit of this was fixed by . Happy to reopen if I'm mistaken.
Data that was once config and recorded on doesn't get updated when the configuration changes. This is a violation of purity. And more than that, it's very surprising when trying to install, hesitating between pgsql and sqlite, and struggling before understanding that the configuration doesn't get used anyway :p cc
that's a bit tough. We have a unit, that's supposed to do the installation. We use the command to install nextcloud (passing the configured database and adminpass settings as command parameters). This CLI generates , and installs nextcloud. The script checks the presence of , and skips installation on following starts. If you now change configuration from the module, we won't run the install command again, as the config file already exists, and assumes nextcloud is already installed.
Hmm… Maybe it would be possible to generate the somewhere in the store, and symlink to it? This way the next time runs it can in addition check whether points to the correct store path. Now, I don't really know what exactly does, so maybe it's not actually possible to do this because eg. one would need to re-run on certain configuration changes? In which case it'd be hard :/ but at least we can maybe make fail or something like that, so that the issue is at least noisy?
I don't recommend to do this, I just tried that (after upgrading one of my personal systems) and this tries to recreate a database schema which can end in very bad results. Unfortunately I doubt that this will work properly, nextcloud wants to merge this during the runtime. E.g. it adds a after a successful or adds trusted domains when running the corresponding command. I'm using this as well now and hitting these issues, unfortunately I don't have useful ideas either ATM, but I'm happy to discuss better solutions :)
Nextcloud edits it's in a lot of different places, so having it generated through Nix currently isn't possible. digs into that already, I a comments about our use case here too.
to avoid other people running into the reconfiguration issue for now, what about documenting that, maybe in ?
That's certainly a good thing to do.
If nobody else wants to take over, I'd volunteer to do this during the next week as investing time on improving the nextcloud module is one item on my todo list :) I was aware of being stateful, but thanks a lot for linking the corresponding ticket! I'll have a look at this as soon as I have sufficient time to :)
In the light of this issue, what's the correct way to upgrade nextcloud to a new major version? Do I just pull a newer , upgrade my system and hope that the systemd job will do the right thing?
The service runs an that should apply all changes like database migrations needed for the new Nextcloud version (beware, downgrades aren't possible). As long as you let Nextcloud do its job, you shouldn't get that much issues. IMHO the actual problem here is the configuration of the software within Nix(OS).
just a simple question (could not come to a conclusion after the lengthy thread) but I recently updated my nextcloud via nixops deploy and the update disabled the calendar module. Did I mess up or is that expected ? any way to configure this from nix ?
Assuming that you installed the calendar module via the UI, it is installed in . The beginning of my config reads: So in principle nextcloud can edit the apps installed via the web interface (otherwise you couldn't install them via the interface in the first place).
I don't think there's anything actionable here - the nextcloud nixos module is limited by how Nextcloud (imperatively) manages its configuration. The current state is properly documented in , and we raised our concerns to upstream in - closing this issue, feel free to reopen if there's more to be done.
I think one actionable item would be to have the nextcloud service fail with an explicit error if the configuration being used is not compatible with the configuration defined in the .nix. At least the user would be warned in practice that configuration is not really applied, and not only by the manual that they are relatively unlikely to actually read. Reopening for now, feel free to reclose if you think that's not a reasonable “solution” :)
The problem is we don't know whether the configuration on disk containing database connection information () differs from the one that would be generated through an installation via - we don't render the configuration on our own, but the installer writes it out as a side-effect.
If I'm not missing something obvious, checking configuration changes would translate to parsing/evaluating on every nextcloud startup as an , comparing every "interesting" key with the one provided through nixos configuration, and eventually then bailing out if they differ. I'm not sure if that's a road we'd like to go and keep up to date, while the underlying bug is a missing distinction between configuration and state upstream…
Since upstream mixes state and config it's questionable whether their configuration format is somehow supposed to be backwards-compatible (as it's only supposed to be used internally by Nextcloud). Hence I'm afraid that this will make i.e. upgrades even harder in case of configuration changes, so :-1: from me. As the issues are documented in the manual now, folks are warned and I think that this is a better option in contrast to adding more complexity to . I suggest we either close this for the reasons already stated or wait for an appropriate upstream fix.
I really don't feel good having stuff in that's actually a disguised command line… :/ Something that would sound nicer to me would be an executable that the user would have to manually run, and a bare-bones module that does not handle setup or configuration and just makes sure that the systemd unit is setup. At least the error is obvious (enabling the module without having run the executable would fail), all module options actually mean something, and the impure operation is made explicit. Either that or, similar to , rename all the options so it's explicit that they're used only for initial configuration. Would one of these solutions make sense to you?
Me neither, but currently we don't have much choice… I really like the fact that we can set up the Nextcloud instance in an unattended fashion, without having to worry about manual setting things up. I really don't want to end with more imperative setup steps than necessary. In theory, we could move some of the options in to to make it more obvious what isn't applied continuously. But TBH, I'd rather like to see the state/config mixing fixed upstream than having to introduce more mental complexity here, especially if this gets fixed and we end up with both incarnations. Really, this issue only occurs while a user changes nextcloud database parameters (mostly while tweaking his initial setup), and not having read the module documentation, where the behaviour is documented.
What about adding the following (subtile) log statement, to point users to the docs when they peek into the logs while things are not working as expected?
I agree that the log statement is the minimum we can do, and would at least give a hint about the way in which things are broken. Now, I guess the reason we're kind of disagreeing on the path forward is that you're much more hopeful than me that nextcloud will ever fix its stuff -- I'm assuming it'll get fixed in years at best (and even then…) -- and if it's in years then I'd much rather live with for years then spend a year deprecating it by making it conflict with new options than try to live with from now on and have users fail to understand what is going on. Now urhm, I'm investigating the module a bit deeper, and… is there any reason we can't put the database configuration etc. in ? We could then refresh on each activation, and it should be enough? If we have a way to couple it with something that automatically runs iff the database is not ready yet then the setup would actually be… almost perfect?
After having a deeper look at the sources ( at github, (6 commits ahead of v16.0.0RC1)) I guess that this should be possible (I didn't test it yet though). Configs will be loaded from using PHP's function[1] while all files ending with will be taken into account. Hence works as all config arrays will be merged together. This config will be loaded at [2] and is available as which is then used in the DI container[3]. If I understand the code correctly, I assume that it should be possible to render all config options in as config files found by glob will override values from [4]. [4]
Nice catch! This might indeed be possible - we should make very sure that config overrides work the way we think they will - seems to write the merged configuration, and not just the overrides (and merging this with other config in the wrong order can lead to wonderfully weird to debug issues) ;-) While I'd still prefer nextcloud splitting config and state properly, and being able to cope with read-only (base) config - it might be a direction worth exploring. What do you think?
Well, for the record it was actually idea :sweat_smile: I guess so, all the stuff from is also in my at the moment. I'll probably have sufficient time next weekend to check if I can trigger such a case. I can't promise that, but unless anybody beats me, I can hopefully prepare a fix for this :)
Did you gain some new insights on this?
Unfortunately not... I was pretty busy back then. But as this only requires some fiddling in the module and no patching of the actual package, the change shouldn't be too painful, so if you'd like to take over... :sweat_smile:
there's with a fix now :)
Steps to reproduce the behavior: Add any other context about the problem here. <!-- Please @ people who are in the list of the offending package or module. If in doubt, check for whoever last touched something. --CC
I don't have time to look at this right now, but if anyone could run git bisect that would be helpful.
Likely caused by the libgpiod update in
If so, maybe we should bring back libgpiod 1.x for openocd?
How many users does it even have? Might be a good idea to do the same thing that is done for ffmpeg and llvm, i.e. separate libgpiod1 and libgpiod2 packages.
Describe the bug Putting a valid certificate name in will cause nixos-rebuild to fail. To Reproduce Steps to reproduce the behavior: this line in . This command will fail. Expected behavior finishes sucessfully. Additional context This error message is related to : I guess this error is caused by switching to python 3. Notify maintainers <!-- Please @ people who are in the list of the offending package or module. If in doubt, check for whoever last touched something. -- Metadata Please run and paste the result. Maintainer information:
isn't compatible with Python 3, it seems. We'll need to update or carry a patch.
MRTC:
Building with (on nixpkgs unstable) leads to I found identical issue from 2019 in RedHat. They claim this patch fixes the issue, however I haven't tested this. (Mainly because I don't know how) I'd be happy to try to fix this, looks like just adding that patch will do the trick, but I'm new to nixpkgs Notify maintainers Metadata Please run and paste the result. Maintainer information:
The patch works. I'll submit a PR.
Is this different from ? Is the patch superfluous?
Oh I didn't see that, if that fixes it then this should probably be closed. The redhat issue makes me think that it is an issue with allegro so perhaps it should be merged anyhow to ensure it doesn't crop up later. But I don't really mind
I'm not sure that it should be closed just yet. I'd need to try building at the latest master without it. I did also run into the same problem you had.
I'm pretty sure this is fixed now
If you serve files from , nginx will set last-modified header to unix timestamp 1. The result is that browser will cache indefinitely. Not sure what we should do about it really, following nginx configuration helps:
Why do you need to turn etag off?
Ah, I see. nginx's etags aren't hashes.
I wonder if we could find a way to pull the hash out of the nix store path and put it into an etag header.
Thanks for finding this. I've been having issues with broken caching on my nginx/nixos server. Even when the files on the server were updated (files in the nix store), the browser was told that no change had happened. If I'm understanding this correctly, since nginx's etags are based on filesystem modification time; you can't have proper cache invalidation (ie. it will always send 304 not modified). Or is there a workaround?
+1 this is a problem. And turning off etag seems like overkill. Is there some way to 1) set the date on a file so that Nginx can pick it up, or 2) do something else awesome?
Is the zero-timestamp rule set in stone in Nix, or is there some way to override it if you really want a file in /nix/store to have a particular timestamp?
I think we could get away with: It's not perfect, as it will change on each Nix evaluation instead of being content addressed. Another way would be to turn the store path passed to nginx directive to translate into an etag.
The etag function is at called from and . Making it dependent on the realpath of the file seems doable.
Explanation how to configure nginx to use the hash from the nix path:
In my testing, this sends the correct etag, but does not send the 304 not modified, when the etag matches. Also, the header gets stripped out when compression is enabled.
It's working perfectly for me. I've gzip compression enabled, the etag header is sent and the browser gets a 304 when querying the header.
Nice! An example with a directory would be more common:
I've to correct what I said. your statement seems to be correct. What happens in my case is that Cloudflare is actually handling the caching of the ETag correctly, not nginx.
So this approach is completely useless without external caching. I'll investigate further and see if I can come up with a solution.
Here is a patch: If real path (after dereferencing symlinks) of is in Nix store, it will use path's hash as ETag. For example: I'll take some time to double check everything and only then send a PR. However, feel free to test: nix { pkgs, ... }: { = { enable = true; package = with pkgs; nginx.overrideAttrs (super: { patches = (super.patches or []) ++ [(fetchpatch { url = ; sha256 = "16pa1vwcm7kibkjsxpk3szaa2vnxdinml6v0fp4038i2qaav113k"; })]; }); }; }
This is really great! or anyone, do you have a moment to write an explanation of this new feature and its practical ramifications for anyone configuring an nginx server? Since it isn't typical nginx behavior, I think the nixos manual should explain it somewhere so it doesn't catch people by surprise.
I think is a good explanation, just needs to be put into the manual.
Cool, submitted .
Package name: bottles Latest released version: 51.11 <!-- Search your package here: --- Current version on the unstable channel: 51.9 Current version on the stable/release channel: 51.9 <!-- Type the name of your package and try to find an open pull request for the package If you find an open pull request, you can review it! There's a high chance that you'll have the new version right away while helping the community! --- Old/incorrect PR Notify maintainers (for last modified) <!-- If the result shows no maintainers, tag the person that last updated the package. --Note for maintainers: Please tag this issue in your PR. Add a :+1: [reaction] to [issues you find important]. [reaction]: [issues you find important]:
python-kasa recently updated to 0.6.x from 0.5.x. The home-assistant project does not yet support 0.6.x, as a result kasa functionality is degraded. Using HomeAsisstant from latest nixpkgs, attempt to get emeter data from kasa devices. I reported this issue upstream and they advised on the version issue. There are more logs visible there. Add a :+1: [reaction] to [issues you find important]. [reaction]: [issues you find important]:
I guess that if we would run the unit tests then the issue with 0.6.0 would have been detected.