opened
peterhoeg
username_0
###### Motivation for this change

Also includes a basic nixos module.

This is ridiculously painful due to how dependencies are handled (or I'm missing something completely).

The basic dependencies for the application itself are declared up front, but each sub-module has a ```REQUIREMENTS``` array where other python packages are listed. homeassistant will then try to load those dependencies if the module is activated and if failing to do so, will fall back to using pip to install them.

I honestly don't know what the best way to deal with this is. It's a rabbit hole of dependencies that reminds me of .rpm hell in the old RH days. Additionally, these sub modules will sometimes require specific versions which means multiple versions now need to be maintained and the maintenance cost just shoots up.

Any thoughts @fridh?

###### Things done

- [X] Tested using sandboxing
  ([nix.useSandbox](http://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS,
    or option `build-use-sandbox` in [`nix.conf`](http://nixos.org/nix/manual/#sec-conf-file)
    on non-NixOS)
- Built on platform(s)
   - [X] NixOS
   - [ ] OS X
   - [ ] Linux
- [ ] Tested compilation of all pkgs that depend on this change using `nix-shell -p nox --run "nox-review wip"`
- [X] Tested execution of all binary files (usually in `./result/bin/`)
- [X] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md).

---
homeassistant: [WIP] init at 0.30.2
issue
created
schneefux
username_1
[requirements-all.txt](https://github.com/home-assistant/home-assistant/blob/dev/requirements_all.txt) includes all the packages you would possibly need. The dynamic `pip` installation is merely for convenience so you don't have to install packages you don't use.
comment
created
FRidh
username_2
Lots of dependencies -> lots of work. That's just how it is right now. Outside of Nixpkgs you could use `pypi2nix`.
comment
created
peterhoeg
username_0
first of all, thank you @fridh for all the feedback, much appreciated!

This is a WIP as I wasn't sure if my approach was correct, so I will get all the items cleaned up you mentioned. All tests will be enabled of course, I just wanted to see if I could get this thing up and running first.

But it offhand seems like there should be a better way. Coming from ruby where the combination of ```bundler``` and ```bundix``` makes it *super* easy, packaging a complex python app seems extremely work intensive.
comment
created
FRidh
username_2
@username_0 you're welcome. 

There should indeed be a better way, but unfortunately upstream doesn't provide any. Python packaging is quite a mess. People can put whatever code they want in `setup.py`. Therefore, the only way to find out what relations packages have, is to actually build them. This is the approach `pypi2nix` takes. But with extension types you need to override the results, and fixing test suites is still something you need to do yourself (if you're interested in the tests, that is).

As soon as Hydra allows network access during evaluation time, I will further work on https://github.com/NixOS/nixpkgs/pull/16005, which at least reduces the amount of work the packaging requires.
comment
created
FRidh
username_2
(hmm, but that should actually work now since https://github.com/NixOS/nix/commit/06bbfb6004942bfcddd930e746ee7a2bfe5c3872)
comment
created
peterhoeg
username_0
So for the time being, what would you recommend for packaging home assistant?

The way I see it, we can do one of the following:

1) Package the bare minimum (ie the server without any modules and its requirements). Then let it download via PIP as it launches.

2) Item 1 + how ever many additional modules I can be bothered to manually write expressions for.

3) Use pypi2nix on the ```requirements_all.txt``` file and use it as an external source.

4) Wait for a more elegant solution.


2 seems a bit pointless as there will be modules that require PIP anyway and there is not much difference between having 70% and 97% of the packages dynamically installed.

3 requires additional work on the part of the person using home assistant which doesn't seem like a nice solution.

That really only leaves us with option 1.
comment
created
FRidh
username_2
This won't work because it cannot write in the store. You would have to specify another directory to install the modules, however, the Nix Python won't be able to find those modules.


I suggest for now that you create a new repo in which you define the module, and where you create the expressions with `pypi2nix`. If one wants to use `homeassistant`, they can clone the repo, and import it. There are other modules like that, e.g. [musnix](https://github.com/musnix/musnix). This saves you a lot of time, while keeping it fairly straightforward for others to use your module. Not everything needs to be in Nixpkgs.
comment
created
peterhoeg
username_0
@fridh assuming the home-assistant binary installs using ```--user```, shouldn't both python modules in the store and local user modules be available?

In any case, thanks to @garbas, ```pypi2nix``` is now able to generate the expressions for home-assistant, so I will be sticking that in a separate repository and move the nixos bits there too in the same way as musnix. 

Would you (the collective nix(os) maintainers be open to moving that repository under the main NixOS github organization if we handle changes similarly to nixpkgs (ie, only trusted project members can merge)? That should enable us to ship modules that cannot make it into the main repository but still be of the same high standard.
comment
created
FRidh
username_2
I think at some point it might make sense to put such repo's together under one umbrella.
comment
created
peterhoeg
username_0
For anyone else stumbling over this issue, @garbas has created a dedicated repos for python packages generated using pypi2nix which is where I'll be submitting the home-assistant stuff.  https://github.com/garbas/nixpkgs-python

@username_2, once again thank you for taking the time to explain things.
comment