jiebi/CodeConvo huggingface dataset:
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: Am I wrong?
I just tried it with Julia 1.4 installed with Nix, and as I suspected fails to build due to
We need to have a interface - that would support building binary packages, and make the interpreter aware of them. I wish I was a bit more motivated to do this, but upstream disappoints me regarding the humble task of compiling Julia as is, with the right selection of a few system dependencies - no response at all at & . I'm also dis-motivated to explore this aspect, since their build system is of plain makefiles, and they keep patches of dependencies from so long ago, without documenting whether they are needed / used or not...
Nix behind It is a shame to see julia's support in NixOS languish. It seems pretty clear to me that upstream is only interested in supporting the configuration obtained by cloning their repository and typing on a stock Linux distribution. I understand their perspective in many ways, as having julia and all of its packages "just work" requires a lot of care, and building with unsupported configurations is very likely to result in brokenness somewhere in the package ecosystem if not in julia itself. Julia is a bit special in this regard because of its emphasis on performance and also its need to support interfaces to other languages/frameworks that often have weird requirements. I have found that typically, there is very good reason for these patches, even though at times they may not be documented except in git and github history. (Admittedly, I am less involved in upstream than I used to be.)
I think Julia team initially evaluated Nix as their binary distributor () but decided against it—there was a discussion somewhere but can’t find it. IIRC lack of Windows / desire to tie into native pkg management systems were factors. Julia’s does a pretty great job, and uses a solver to properly resolve dependency versions based on constraints. I’ve been incredibly frustrated by the withPackages pattern, at least in python. I constantly run into problems due to nixpkgs ignoring version constraints. There are some other nice advantages to nix of course: binary dependencies are shared across language ecosystems for example, and there are lots of supremely annoying packages (I’m looking at you CUDA...) that Nix makes delightfully easy to install, use, and maintain parallel versions.
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: But until Nixpkgs supports multiple versions across the board, and has a solver baked in, I’m honestly not sure Nix brings enough advantages to bail on Edit: I think the approach done by poetry2nix and Mach-nix are the way to go for Julia packages + nix. yarn2nix is another shining example. I would love to see a solution in this vein—didn’t mean to douse any enthusiasm for people interested in helping improve the Julia situation on NixOS :)
Maybe it is worth to reaching out to Julia Computing folks at to get their attention on ?
We'd probably need to put this information in our attribute sets, just like our Python packages know what deps they need. Hmm that's a bit surprising to hear as the Python ecosystem seems most well organized. Have you ever encountered a real bug with a mismatched version of a dependency with a specific version specified in a package's ? IIRC fails if a version is used and it's not exactly what's available. If you've ever felt there's a certain python package of a certain version missing, have you considered contributing it? Even if not, was it too hard to override it? What do you mean "across the board"? That might be much more harder then it sounds, due to the constraints of Nix it self when being evaluated everytime you run and friends, see It's not a question of whether Nix brings an advantage or not, it's a question of whether there are any Julia packages that are built at other distros imperatively with a , and this produces (or even worse, downloads) shared objects or executables into (not in ), linked to Julia or other Nix deps that do reside in . If the answer is yes, than every Nix user will experience breakage after Julia is updated in his - not necessarily if it's a version bump of Julia, it could be a mere rebuild of Julia by hydra. Yea Mach-nix is an awesome idea well implemented ( thanks for the inspiration :heart:). I've also reached a similar conclusion at . However, this is my opinion for Rust and Go, and I tend to think that for a small ecosystem like Julia, perhaps it'll be too much - it shouldn't be that hard to manage all of these pacakges like Python is managed in Nixpkgs.
I opened
Frequently.
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: Especially with Scipy, matplotlib, bokeh, sklearn, tensorflow, pytorch, etc... Often, nixpkgs used to have it until it was updated. Except in very rare circumstances, we do not keep more than one version of a python package around. I have no interest wasting time working around bugs that authors tried to spare me by specifying precise versions. It seems a sort of cognitive dissonance in our next community that we all care so deeply about reproducible builds, but for convenience often disregard the officially supported dependencies. As a result, there are often subtle bugs in software on NixOS that do not appear on other platforms. (Eg x11vnc crashes, shared memory problems, etc). Nix supports crazy high specificity in versioning, well we past semantic versioning down the the bit in many cases. And yet in nixpkgs, by convention, we remove the old version anytime we update. It’s understandable that Hydra cannot build everything, but it’s always bothered me that I have to pin nixpkgs if I want a build to stay functional over 1+ years. is mostly declarative. Of course, Julia is constantly re-compiling code, and until there is true support for precompilation, I don’t see how it’s possible to use nix store. This is due to multiple dispatch. Suppose I wrote a library to do an efficient k-nearest-neighbor query using some fancy free structure. Perhaps the core algorithms simply require a function for your types. When you write this new function, Julia has to compile all the code in the k-nearest neighbor library for this new type. Thus, it is not possible right now to compile a package on its own as the core pattern in Julia involves extending the function of other packages via multiple dispatch. You can only compile a package in context of the code that is running. Therefore, the Nix store approach is not appropriate. See
I see. You are talking about "authors" of Nixpkgs' expressions? I'm still a bit perplexed, as I earlier said, and correct me if I'm wrong: should fail if is used in , and the exact version is not available. Sometimes (), we do hardcode such version requirement changes in our expressions, but hopefully that's done only when it's justified.
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: I agree (maybe a bit ). Generally I think, every dep's version related issue should be handled by someone like you, who's familiar with python packages you use. I'd at least (open PRs) to fix the issues I encounter :). I see. I guess the situation is more complex than I thought, but I don't think I'd be able to form a strong opinion until I'll get a Julia interpreter from an expression I'm satisfied with. Thanks for the link :).
Unfortunately, this pattern is done constantly (including in my own contributions!). The problem is that, except an extremely rare cases like tensorflow, nixpkgs does not allow multiple versions of same python package to co-exist. Every time we update a package, we remove the previous version from nixpkgs. Confounding this issue, we often disable tests because sometimes the effort is just too high when developers bake in assumptions that aren’t compatible with the Nix ethos (for some reason this seems to happen more with tests then source code...). The most frequent cause of failure is the code needs to download something to run the test. (I’ve tried to lobby in the past that the test phase should be permitted to have Internet access as long as it doesn’t modify /nix/store, but I digress). In any case, the end result is that using a python package on Nix typically means you aren’t using the exact dependencies that upstream declared, and you’re more likely to have a package that despite having changes in dependencies has never had the test run. Thus, amazingly, I’ve had more luck with Julia packages working on NixOS than Python when all other binaries are present in environment.
I don't think there's a strong policy in our Python ecosystem against multiple versions of packages - if this is what upstream recommends. Anyway, the whole topic is out of scope for this issue. Feel free to open a new issue and cc me and who's working hard to ship a 20.09 release.
Yes, sorry getting a little off topic here. I mainly mention it to suggest that our efforts in Julia that start from scratch Can learn from these issues. I’m not too familiar with the new binary builder, but was quite nice in that it would first search for libraries before attempting to install.
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: See this issue for some context: Note that a core Julia dev gave a pretty great endorsement on Nix (albeit back in 2014): I think it might be most productive to do this development outside of nixpkgs, and do it in Julia to hook in with nixpkgs as needed. That way users could seamlessly use Pkg in Julia. I’d love to and have it just work instead of the current hack that I wrote that a lot of people seem to be using ()
Short summary of the Discourse thread: for LLVM and libuv it is essential to use the patches/forks. For other packages it is probably unnecessary to use the exact version specified by upstream.
Thanks for being there with me :). I think I'm gonna work on this once we are through with , to avoid merge conflicts.
TIL Guix manages to package Julia 1.4.1: What's interesting, is that they but they build it with Julia's LLVM patches fetched from the internet - they don't rely on Julia's build system to build it's own LLVM. For similar dependencies which the real system's version can't be used, they do the same. I think we should try to copy this approach.
FWIW I did that in Fedora originally, but I stopped. Since LLVM makes breaking releases very quickly, you'll probably want to move to a new version that Julia doesn't support quite soon.
Maybe I don't understand, but why should I move to an LLVM version that julia doesn't support? Also, aren't the and versions don't introduce breaking changes between minor releases? We could stick to since that's what Guix does and I think Julia uses this version as a base too.
I don't really know how NixOS works, hence the "FWIW". If you used versioned LLVM packages and keep older major versions then it's probably OK.
Just offering a user perspective: Julia provides a package manager that provides mostly reproducible (in a practical/scientific reproducibility sense rather than a bit-identical sense) environments easily from manifests. Increasingly, these environments are isolated from the host OS by binary packages for dependencies being served by Julia. I don't want Nix to manage dependencies that can handle for me: Julia already handles those excellently and conveniently. I just want Nix to handle the (usually few) system dependencies.
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: What I would like as a NixOS user is for Nix to provide me with a working Julia 1.5 derivation that provides: julia, a FHS wrapper or something providing compatible versions of the few things that Julia's binary packages need (just libc, libstdc++, libgfortran; I think. ). When I say "compatible", I mean that these libraries should work with the binaries and with the julia binary provided by Nix. Not all Julia packages are BinaryBuilder-ised yet, so I'd also want an easy way to specify that I want compatible versions of other packages installed (e.g. Qt, cairo, etc). For packages that need python, I don't know if I'd like Julia or Nix to handle installing and managing that. The simplest approach would be to tell Julia to do it (there is pretty good support in Julia for distributing isolated environments of python + packages + dependencies; better than in , imo). I don't know how doable any of that is in Nix, just describing what a good outcome would look like to me.
I think this is quite doable. I would propose that we add a new package, , that is a "batteries included" build that contains system libraries for all common Julia packages. Would this approach work for BinaryBuilder? I haven't used this yet. If not, we could fall back to a FHSuserenv, but I'm reluctant to do so if at all avoidable as there are outstanding nix bugs () that create seg faults in some heavy data processing workflows.
I think that is a good proposal and I think it will work with BinaryBuilder packages (e.g. SDL2_jll). In any case, some approach that works with BinaryBuilder packages should be found because those are a critical part of the Julia environment now :) Some important packages (e.g. GR) still want to build and install their own software dynamically, and that doesn't work well in Nix, but most of those can plausibly move to the BinaryBuilder system (PR for GR ), so I think letting those be broken would be tolerable (and a big improvement over the status quo of Julia on nix). Perhaps we can get something working quickly by just patchelf'ing the official Julia builds for now? (I don't understand how all of this stuff works!)
Code change:
makeFlags = let arch = head (splitString "-" stdenv.system); <del> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> march = { x86_64 = stdenv.hostPlatform.platform.gcc.arch or "x86-64"; i686 = "pentium4"; aarch64 = "armv8-a"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); # Julia requires Pentium 4 (SSE2) or better <del> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; }.${arch} </del> <ins> cpuTarget = { x86_64 = "x86-64"; i686 = "pentium4"; aarch64 = "generic"; }.${arch} </ins> or (throw "unsupported architecture: ${arch}"); <del> # Julia applies a lot of patches to its dependencies, so for now do not use the system LLVM # https://github.com/JuliaLang/julia/tree/master/deps/patches </del> in [ "ARCH=${arch}" "MARCH=${march}"

Related discussion:
text: Is there anything presently blocking just getting julia 1.5 into nixpkgs? I don't care about ecosystem working with nixpkgs, just a good ol' fashioned julia install.
yes, see
If you just want Julia and don't mind about binary packages breaking you can just download Julia from the official site and patchelf it. On Sun, 4 Oct 2020, 09:32 Tyler Benster, <:
On the there was also the idea floated to patch to patchelf dynamically loaded binary dependencies.
Guix has a Julia package and it works quite well. Perhaps some ideas can be borrowed from there: It'd be really useful to get Julia working on Nix. It's the only big outstanding thing that doesn't work and prevents me from migrating some servers in my organization.
I tried to copy their expressions in and it doesn't work yet, but there's also another seemingly working attempt by at .
So, do we want anything else now that 1.5.3 is packaged (:greenheart: ? I think we: 't need clean up the expressions a bit - not use the file if it's not shared. I did (2) along with the julia_10 fix in As for (1) I will need a bit of feedback.
I guess some people like me still use julia_13 via test-disabling overrides, so a bit of time after 1.5 reaches channels for smooth transition could make sense?
Maybe backport 1.5 to 20.09?
OK so I disabled the tests for julia13 in and made the default there as well. All merged pull requests referenced here should be backported.
Code change:
{ lib <del> , stdenv , fetchFromGitea </del> , acl , attr , autoreconfHook <ins> , bzip2 , fetchFromGitea </ins> , libburn <ins> , libcdio , libiconv </ins> , libisofs , pkg-config <ins> , readline , stdenv </ins> , zlib }:

Related discussion:
text: Can't build xorriso due to source archive hash mismatch. Steps to reproduce the behavior: the substituter -shell -p xorriso nixpkgs 23.11.5408.
Probably it comes from a moving target... Let's go!
How exactly it happened? Here is the url: Is web archive failing?? P.S.: since Xorriso is just another name for libisoburn, I will remove xorriso and link-alias it to libisoburn.
Code change:
sha256 = "sha256-b9cBn5Nv+IxB58gi9G8QtRvXLWZZePZYZIPedbMMNr8="; }; <del> buildInputs = [ iptables-legacy libuuid openssl ]; </del> <ins> buildInputs = [ iptables-legacy libuuid openssl ] ++ lib.optionals (firewall == "nftables") [ libmnl libnftnl ]; </ins> nativeBuildInputs= [ pkg-config makeWrapper ]; # ./configure is not a standard configure file, errors with: # Option not recognized : --prefix= dontAddPrefix = true; <ins> configureFlags = [ "--firewall=${firewall}" # allow using various config options "--ipv6" "--leasefile" "--regex" "--vendorcfg" # hardening "--portinuse" ]; </ins> installFlags = [ "PREFIX=$(out)" "INSTALLPREFIX=$(out)" ]; <del> postFixup = '' for script in $out/etc/miniupnpd/ip{,6}tables_{init,removeall}.sh do wrapProgram $script --set PATH '${scriptBinEnv}:$PATH' done ''; </del> <ins> postFixup = { # Ideally we'd prefer using system's config.firewall.package here for iptables, # however for some reason switching --prefix to --suffix breaks the script iptables = '' for script in $out/etc/miniupnpd/ip{,6}tables_{init,removeall}.sh do wrapProgram $script --prefix PATH : '${scriptBinEnv}:$PATH' done ''; nftables = '' for script in $out/etc/miniupnpd/nft_{delete_chain,flush,init,removeall}.sh do wrapProgram $script --suffix PATH : '${scriptBinEnv}:$PATH' done ''; }.${firewall}; </ins> passthru.tests = { bittorrent-integration = nixosTests.bittorrent;

Related discussion:
text: We already have in , but only with iptables support. It would be nice to have the nftables version available as well.
Being worked on in , I'm trying to test myself but couldn't make it to work yet.
I think the PR is ready now, testing welcome!
Maybe add an option for will be better?
Code change:
}; buildInputs = lib.optional pulseSupport libpulseaudio <del> ++ lib.optional jackSupport libjack2 ++ lib.optional alsaSupport alsa-lib; </del> <ins> ++ lib.optionals jackSupport [ libjack2 soxr ] ++ lib.optional alsaSupport alsa-lib ++ lib.optional pcapSupport libpcap; </ins> nativeBuildInputs = [ cmake pkg-config ]; cmakeFlags = [ "-DPULSEAUDIO_ENABLE=${if pulseSupport then "ON" else "OFF"}" "-DALSA_ENABLE=${if alsaSupport then "ON" else "OFF"}" "-DJACK_ENABLE=${if jackSupport then "ON" else "OFF"}" <ins> "-DPCAP_ENABLE=${if pcapSupport then "ON" else "OFF"}" </ins> ]; cmakeDir = "../Receivers/unix";

Related discussion:
text: Scream does not compile with jack support when it is explicitly enabled through an overlay. nixpkgs.overlays = [ (final: prev: { scream = { jackSupport = true; }; }) ]
Error message?
There is no error, it just skips compiling it. someone else ran it and i think they said that it skips compiling it with jack support because soxr is missing -- Checking for module 'jack' -- Found jack, version 1.9.19 -- Checking for module 'soxr' -- No package 'soxr' found
See . In the meantime, you can try: nixpkgs.overlays = [ (final: prev: { scream = ( { jackSupport = true; }).overrideAttrs (old: { buildInputs = old.buildInputs ++ [ ]; }); }) ]