Change:
Describe the bug , which is depended on by , has been failing to build on both Linux and macOS since / , perhaps since was updated in . of a contemporary build attempt indicate that the immediate problem is that the current version of tokenizers depends on a vendored copy of v0.10.0 of the parkinglot crate, via its cargoDeps dependency. Further research suggests that the -issue reported in the logs was fixed in Amanieu/parkinglot which it looks like is included in v0.10.1 and successors. As a result, I think that someone just needs to refresh tokenizers' (possibly by way of simply updating the package) or, failing that, to try to get it to build with an older version of . (That said, I have yet to test these changes myself, so I don't know if they are truly enough or if additional modifications are required.) To Reproduce Expected behavior The build should succeed. Notify maintainers (also /cc (since this looks like an interaction with and since there is a commit linked to above updating in the scmdiff from the last successful build to the first failing build, per hydra.) Metadata $ nix-shell -p nix-info --run "nix-info -m" system: host os: multi-user?: sandbox: version: channels(...): nixpkgs: Maintainer information:
Thanks for the bug report (still hoping for Hydra notifications some day)! The linked PR should fix this issue. I am also working on some other PRs to make transformers work on Linux/AArch64 and Darwin/x86_64.

Discussion:
}; in rustPlatform.buildRustPackage rec { pname = "tokenizers"; <del> version = "0.8.1.rc1"; </del> <ins> version = "0.8.1"; </ins> src = fetchFromGitHub { owner = "huggingface"; repo = pname; rev = "python-v${version}"; <del> sha256 = "1bzvfffnjjskx8zlq1qsqfd47570my2wnbq4ip8i1hkz10q900qv"; </del> <ins> sha256 = "0sxdwx05hr87j2z32rk4rgwn6a26w9r7m5fgj6ah1sgagiiyxbjw"; </ins> }; <del> cargoSha256 = "0s5z3g1njb7wlyb32ba6xas4zc62c3zhmp1mrvghmaxpvljp6k7b"; </del> <ins> # Update parking_lot to be compatible with recent Rust versions, that # replace asm! by llvm_asm!: # # https://github.com/Amanieu/parking_lot/pull/223 # # Remove once upstream updates this dependency. cargoPatches = [ ./update-parking-lot.diff ]; cargoSha256 = "0cdkxmj8z2wdspn6r62lqlpvd0sj1z0cmb1zpqaajxvr0b2kjlj8"; </ins> sourceRoot = "source/bindings/python";

<ins> diff --git a/bindings/python/Cargo.lock b/bindings/python/Cargo.lock index f50db71..ea71817 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -269,7 +269,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" [[package]] name = "lock_api" -version = "0.3.3" +version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "scopeguard 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)", @@ -337,16 +337,16 @@ dependencies = [ [[package]] name = "parking_lot" -version = "0.10.0" +version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ - "lock_api 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot_core 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", + "lock_api 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot_core 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "parking_lot_core" -version = "0.7.0" +version = "0.7.2" source = "registry+https://github.com/rust-lang/crates.io-index" dependencies = [ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", @@ -409,7 +409,7 @@ dependencies = [ "inventory 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", "libc 0.2.68 (registry+https://github.com/rust-lang/crates.io-index)", "num-traits 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)", - "parking_lot 0.10.0 (registry+https://github.com/rust-lang/crates.io-index)", + "parking_lot 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)", "paste 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", "pyo3cls 0.9.2 (registry+https://github.com/rust-lang/crates.io-index)", "regex 1.3.6 (registry+https://github.com/rust-lang/crates.io-index)", @@ -768,7 +768,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" "checksum itoa 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)" = "b8b7a7c0c47db5545ed3fef7468ee7bb5b74691498139e4b3f6a20685dc6dd8e" "checksum lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646" "checksum libc 0.2.68 (registry+https://github.com/rust-lang/crates.io-index)" = "dea0c0405123bba743ee3f91f49b1c7cfb684eef0da0a50110f758ccf24cdff0" -"checksum lock_api 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "79b2de95ecb4691949fea4716ca53cdbcfccb2c612e19644a8bad05edcf9f47b" +"checksum lock_api 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "c4da24a77a3d8a6d4862d95f72e6fdb9c09a643ecdb402d754004a557f2bec75" "checksum maybe-uninit 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "60302e4db3a61da70c0cb7991976248362f30319e88850c487b9b95bbf059e00" "checksum memchr 2.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3728d817d99e5ac407411fa471ff9800a778d88a24685968b36824eaf4bee400" "checksum memoffset 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)" = "b4fc2c02a7e374099d4ee95a193111f72d2110197fe200272371758f6c3643d8" @@ -777,8 +777,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" "checksum number_prefix 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "17b02fc0ff9a9e4b35b3342880f48e896ebf69f2967921fe8646bf5b7125956a" "checksum onig 6.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "bd91ccd8a02fce2f7e8a86655aec67bc6c171e6f8e704118a0e8c4b866a05a8a" "checksum onig_sys 69.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3814583fad89f3c60ae0701d80e87e1fd3028741723deda72d0d4a0ecf0cb0db" -"checksum parking_lot 0.10.0 (registry+https://github.com/rust-lang/crates.io-index)" = "92e98c49ab0b7ce5b222f2cc9193fc4efe11c6d0bd4f648e374684a6857b1cfc" -"checksum parking_lot_core 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7582838484df45743c8434fbff785e8edf260c28748353d44bc0da32e0ceabf1" +"checksum parking_lot 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d3a704eb390aafdc107b0e392f56a82b668e3a71366993b5340f5833fd62505e" +"checksum parking_lot_core 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d58c7c768d4ba344e3e8d72518ac13e259d7c7ade24167003b8488e10b6740a3" "checksum paste 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "ab4fb1930692d1b6a9cfabdde3d06ea0a7d186518e2f4d67660d8970e2fa647a" "checksum paste-impl 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "a62486e111e571b1e93b710b61e8f493c0013be39629b714cb166bdb06aa5a8a" "checksum pkg-config 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)" = "05da548ad6865900e60eaba7f589cc0783590a92e940c26953ff81ddbab2d677" </ins>