Repository: AppDaemon/appdaemon (language: python)

Issue:
Suggestion: Improve CI pipeline
## Introduction
Thanks for this amazing project!
I have been using it for quite some time to easily integrate an external device (a TP-link router) in Home Assistant.
After seeing the reported issues in the Discord chat about the docker builds failing, I would like to chime in and give an helping hand, having had some previous experience with CI systems (Gitlab CI and Github Actions) and Docker containers.
After a quick glance at the project, I have some questions regarding its development cycle:

## Continuous integration platform
It appears that multiple CI platforms are used:
- [semaphore ci](https://github.com/AppDaemon/appdaemon/blob/8de3a24ce447f954c8c7fe58d80eb6257c318508/.semaphore/semaphore.yml)
- [azure pipelines](https://github.com/AppDaemon/appdaemon/blob/8de3a24ce447f954c8c7fe58d80eb6257c318508/azure-pipelines.yml)
-  __Github Actions__: there has been a previous attempt in #1022 to introduce it, but at the moment in the `dev` branch the configuration has been removed, as seen from this commit: 363f1651fe5bbcb43814b712fa52be211946326f

### Question
Is there a reason why different CI platforms are needed? Wouldn't it be better to use a single one to consolidate the efforts and provide a single interface to see the latest build status?
Since the project is already using Github as its hosting platform, I am inclined to proposed __Github Actions__ as the CI provider, since it intuitively integrates with the rest of the ecosystem (github issues, pull request, releases, docker registry, etc..).

## Proposed changes
- [x] Open a discussion to decide if multiple CI platforms are indeed needed and preferable. In case a single one is preferred:
  - [x] Consolidate the workflow on a single platform to do automated testing and deployment
- [x] Define the automated steps that should happen in the CI pipeline (e.g.: should we run tests for every git push? create docker images on git tags?)
- [ ] Integrate the changes proposed in #1646 in a CI configuration (automatically generate release notes and release artifacts as part of the Github project)

## Implementation plan
General pipeline configuration:
- [x] Build dev as commits are pushed
- [x] Build releases whenever a TAG is created

The following is an outline of the ideal pipeline, summarized from the comments below:
- [x] Build and deploy docker multi arch images
- [x] Documentation (built using readthedocs)
- [x] Update dependencies (currently `dependabot` is used)
- [ ] Automatic release notes


Discussion / hints:
Hi Carlo - and thanks for reaching out!

I think consolidating all the CI pipeline would be a great idea - we got where we are today because the various pieces were added piecemeal over the last few years. A well meaning person would contribute some config for their favorite CI pipeline to solve a specific problem, then usually move on leaving me to maintain something I had no understanding of ;)

Keeping it all in one place sounds like a great idea, but I have limited experience in the CI side of things, and as you have seen, when things break, it is a scramble to fix them.

So, with the following points in mind, if it is a good fit, I would be happy to build a single pipeline, and I am comfortable with the choice of Git if it makes the most sense. The ideal pipeline would:

- Handle Pre commit hooks (black & flake) (Azure used for this now)
- Build and deploy docker multi arch images (semaphore currently)
- Documentation (Readthe docs currently in use, I would probably expect to keep that as I am not aware of any alternative)
- Update dependencies (currently dependably, not sure if there is an alternative)
- Some automation of release notes etc. (we have none at the moment)

Even consolidating the semaphore and azure pipelines would be a win in my opinion. However, there are some important non-technical considerations too:

- AppDaemon has zero funding of any sort so would need to use whatever free tiers of tools are available
- Whatever the chosen pipeline is, it would be very helpful if the original designer was available to help out if things went wrong, ideally interactively over discord

In terms of CI process, I think what we have at the moment works well:

- Build dev as commits are pushed
- Build releases whenever a TAG is created

So, given the above do you think Git CI would be a fit, and would yo use willing to help?  If so, what would you suggest for next steps? I know little about this subject so would really want you to help build the pipeline from scratch to agreed specs, then ideally be around afterwards to help out if anything goes wrong, if not, I will have just exchanged one set of arcane incomprehensible systems for another :) One advantage of your proposal is, as you said that it is all under one roof so hopefully it will be simpler to understand and maintain which would be a win in its own right.

Let me know what you think - I do like this idea a lot!
I can understand your frustration, being an open source project maintainer and having a changing sets of hands with the coming and going of each volunteer can be problematic.
For this reason I think a shared design and (a bit of) documentation on the chosen CI solution is fundamental, since in that case anyone willing to work on the implemented solution can understand what has been built and give its contribution, especially when things go wrong.

I know this sounds a bit cliche, but a well-managed CI pipeline reduces the need for human intervention, since each step is documented as code in the pipeline definition, thereby reducing the need for manual steps in the development of an application. This could be helpful in the maintenance aspects of this project, for instance reducing the need for manually publishing release artifacts and documentation.
I think the Docker container errors encountered the other day could be avoided by simply building the Docker image automatically as part of the pipeline, thereby easily catching any change that may introduce errors in the build process!

## Ideal candidate
I think Github Actions might be ideal since less third-party solutions there are, less things can break at the wrong time, plus at the same time it is all consolidated under the same roof, as you said.
It ticks the following boxes:
- free for open-source projects like this
- tightly integrated with github issues and pull requests, so the results of the pipeline are easily integrated as part of the development process
- can be easily customized without the need of an external service (for instance now semaphore and azure pipelines are configured via their external dashboard, with their own user access and secret management controls).
 With Github Actions every options is here inside the Github repository.
 
## Next steps
I can help out in the implementation. I cannot guarantee my 100% availability (aren't we all volunteers after all? :wink: ), but I will do my best.
To start, I can work on automating the setup of the dependencies when checking out the project, as part of a basic CI step.
Starting from there the sky is the limit! :)

P.S.: Does this project have any unit testing? (`pytest` or the like of it?)

OK, sounds good - I am looking forward to seeing how things progress. And of course I don't expect 100% availability - I just hope that you might respond to an email or on discord within a day or two if I find an issue I can't figure out :) Although as you say, if its 1 consistent pipeline I stand a better chance of getting familiar with it and needing less help.

As far as unit testing goes - no we don't have any. It's another thing on my long list to become an expert at ... I did at one point make a start writing my own framework for this but I got bogged down. If you can recommend a framework, and at least show me how to get started I can add this into the mix as well, as it is a good idea, and something I have been meaning to get to at some point.
> As far as unit testing goes - no we don't have any. It's another thing on my long list to become an expert at ... I did at one point make a start writing my own framework for this but I got bogged down. If you can recommend a framework, and at least show me how to get started I can add this into the mix as well, as it is a good idea, and something I have been meaning to get to at some point.

I have no familiarity at all with the AppDaemon code, so proposing one framework vs another might be hazardous, since I don't know the complexity and intricacies of the code we are dealing with.
In some other projects I used with great success `pytest`. It seems to have lots of popularity in the Python world, with plugins and extensions behind it for various use cases.

Ok, presuming that pytest can easily be integrated with the new pipeline I’ll take a look – thanks.


Fix patch:
diff --git a/.flake8 b/.flake8
new file mode 100644
index 000000000..de7af22d4
--- /dev/null
+++ b/.flake8
@@ -0,0 +1,9 @@
+[flake8]
+# Increase from the default 79
+max-line-length = 500
+
+extend-ignore = E203, E266
+# E203 whitespace before ‘,’, ‘;’, or ‘:’
+# E266 too many leading ‘#’ for block comment
+
+max-complexity=18
\ No newline at end of file
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 30255d496..0eb775f4a 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -1,22 +1,15 @@
 repos:
-- repo: https://github.com/ambv/black
-  rev: 22.3.0
-  hooks:
-    - id: black
-      language_version: python3
-      args:
-        - '--line-length=120'
 - repo: https://github.com/pre-commit/pre-commit-hooks
   rev: v2.1.0
   hooks:
     - id: end-of-file-fixer
     - id: trailing-whitespace
+- repo: https://github.com/ambv/black
+  rev: 22.3.0
+  hooks:
+    - id: black
+      language_version: python3
 - repo: https://github.com/PyCQA/flake8
   rev: 3.7.8
   hooks:
     - id: flake8
-      args:
-        - '--max-line-length=500'
-        - '--ignore=E203,E266,E501,W503'
-        - '--max-complexity=18'
-        - '--select=B,C,E,F,W,T4,B9'
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 000000000..26a9add67
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,12 @@
+[project]
+name = "appdaemon"
+
+# Taken from: https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html
+[tool.pytest.ini_options]
+addopts = [
+    "--import-mode=importlib",
+]
+
+# black configuration
+[tool.black]
+line-length = 120


Test patch:
diff --git a/.github/workflows/python-tests.yml b/.github/workflows/python-tests.yml
new file mode 100644
index 000000000..478c79c7a
--- /dev/null
+++ b/.github/workflows/python-tests.yml
@@ -0,0 +1,40 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+
+name: Python lint and tests
+
+on:
+  push:
+  pull_request:
+    branches: ["dev"]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.7", "3.8", "3.9", "3.10"]
+
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v3
+      with:
+        python-version: ${{ matrix.python-version }}
+        cache: 'pip' # caching pip dependencies
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+        python -m pip install flake8 pytest
+    - name: Lint with flake8
+      run: |
+        # stop the build if there are Python syntax errors or undefined names
+        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+        # exit-zero treats all errors as warnings.
+        flake8 . --count --exit-zero --statistics
+    - name: Test with pytest
+      run: |
+        python -m pytest
diff --git a/tests/test_main.py b/tests/test_main.py
new file mode 100644
index 000000000..57bd16bde
--- /dev/null
+++ b/tests/test_main.py
@@ -0,0 +1,8 @@
+
+from appdaemon.__main__ import ADMain
+
+
+# TODO: fake test to assert that pytest works correctly
+def test_main():
+    main = ADMain()
+    assert main is not None
\ No newline at end of file