Change:
While the system we have right now is good for contributing, I think each new nix expression should have a test in written along with it in each new PR. We should do this retroactively where possible for all currently existing packages too. This way, we can ensure everything works with hydra, and be alerted quickly for broken, but successfully built expressions that may not be actively maintained and (user) tested anymore. Would it be possible to change the policy in to do this? The flow I see is this: new expression for program X locally to ensure it works test in nixos/tests for it so hydra can do this automatically PR after everything fits . Is this a good idea, something we should implement? I see more benefits that drawbacks.
This will also be really great for stable releases, since if implemented in full, we can ensure all packages are at least minimally tested.
Some packages offer installed tests
NixOS tests for simple things are currently very frustrating to debug, because the overhead of an empty NixOS test is larger than the runtime of a non-NixOS-test LibreOffice functional test. And naive approaches to non-NixOS-based tests got rejected because launching DBus and X requires a minor duplication of functionality (and so it was rejected). Installed tests shipped with packages come pre-debugged (hopefully), so it should be less of a problem there. Of course, some might depend on network access, but that should be easy to detect and block the coresponding tests. So if you want to see more NixOS tests, your best bet probably includes something about a NixOS profile with minimal boot time. Maybe even with some code to boot it in a Nix build sandbox (used as a container).
Thank you for your contributions. This has been automatically marked as stale because it has had no activity for 180 days. If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity. Here are suggestions that might help resolve this more quickly: for maintainers and people that previously touched the related code and @ mention them in a comment. on the . on the on .

Discussion:
```ShellSession $ git diff > nixpkgs/pkgs/the/package/0001-changes.patch ``` <ins> ## Package tests {#sec-package-tests} Tests are important to ensure quality and make reviews and automatic updates easy. Nix package tests are a lightweight alternative to [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests). They can be used to create simple integration tests for packages while the module tests are used to test services or programs with a graphical user interface on a NixOS VM. Unittests that are included in the source code of a package should be executed in the `checkPhase`. ### Writing package tests {#ssec-package-tests-writing} This is an example using the `phoronix-test-suite` package with the current best practices. Add the tests in `passthru.tests` to the package definition like this: ```nix { stdenv, lib, fetchurl, callPackage }: stdenv.mkDerivation { … passthru.tests = { simple-execution = callPackage ./tests.nix { }; }; meta = { … }; } ``` Create `tests.nix` in the package directory: ```nix { runCommand, phoronix-test-suite }: let inherit (phoronix-test-suite) pname version; in runCommand "${pname}-tests" { meta.timeout = 3; } '' # automatic initial setup to prevent interactive questions ${phoronix-test-suite}/bin/phoronix-test-suite enterprise-setup >/dev/null # get version of installed program and compare with package version if [[ `${phoronix-test-suite}/bin/phoronix-test-suite version` != *"${version}"*  ]]; then echo "Error: program version does not match package version" exit 1 fi # run dummy command ${phoronix-test-suite}/bin/phoronix-test-suite dummy_module.dummy-command >/dev/null # needed for Nix to register the command as successful touch $out '' ``` ### Running package tests {#ssec-package-tests-running} You can run these tests with: ```ShellSession $ cd path/to/nixpkgs $ nix-build -A phoronix-test-suite.tests ``` ### Examples of package tests {#ssec-package-tests-examples} Here are examples of package tests: - [Jasmin compile test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/jasmin/test-assemble-hello-world/default.nix) - [Lobster compile test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/lobster/test-can-run-hello-world.nix) - [Spacy annotation test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/spacy/annotation-test/default.nix) - [Libtorch test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/science/math/libtorch/test/default.nix) - [Multiple tests for nanopb](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/nanopb/default.nix) </ins>