pynetcheck 0.2.2
Python Network device checker using Pytest and IP Fabric.
License: MIT
Requires Python: >=3.8,<4.0
Keywords: ipfabric,ip-fabric,community-fabric

#  PyNetCheck

## Python Network Checker

This project provides examples to extend the IP Fabric platform with custom device testing.

*This project is still in development and more test cases and examples will be added at a later time.*

## Requirements

* `Configuration saved` task must be enabled in [IP Fabric Discovery Settings](https://docs.ipfabric.io/latest/IP_Fabric_Settings/Discovery_and_Snapshots/Discovery_Settings/disabled_discovery_tasks/).
* `.env` file with IP Fabric URL and credentials.

Example `.env` file:

```bash
IPF_URL=https://demo.ipfabric.io

# Use IPF_TOKEN OR (IPF_USERNAME AND IPF_PASSWORD):
IPF_TOKEN=TOKEN
# IPF_USERNAME=user
# IPF_PASSWORD='p@ssw0rd'

# OPTIONAL:

# IPF_SNAPSHOT defaults to $last
IPF_SNAPSHOT=$last
# IPF_SNAPSHOT=7e2d4bef-3f90-4c9c-851d-fc2f0990db35

# IPF_VERIFY defaults to True and can also be set to a path
IPF_VERIFY=true
# IPF_VERIFY="path/to/client.pem"

# IPF_TIMEOUT defaults to HTTPX default of 5.0 see https://www.python-httpx.org/advanced/#timeout-configuration
# IPF_TIMEOUT only accepts int/float arguments
# To set to None use or advanced configuration use: `ipf = IPFClient(timeout=None)`
IPF_TIMEOUT=5
```

## Installation

The project is available on PyPi and can be installed via pip:

```bash
pip install pynetcheck
```
## Running

To run tests with builtin cases, use the following command:

```bash
(venv) >pynetcheck --tb=line                         
========================================================================================== test session starts ==========================================================================================
platform win32 -- Python 3.9.9, pytest-7.4.2, pluggy-1.3.0
rootdir: C:\Code\_EXAMPLES\config_vulnerability\pynetcheck
configfile: pytest.ini
plugins: anyio-4.0.0, depends-1.0.1, html-reporter-0.2.9
collected 13 items                                                                                                                                                                                       

pynetcheck\tests\cve_2023_20198\ios_xe_test.py ..sFFFF.s.s.s                                                                                                                                       [100%]

=============================================================================================== FAILURES ================================================================================================ 
C:\Code\_EXAMPLES\pynetcheck\tests\cve_2023_20198\ios_xe_test.py:34: AssertionError: Startup - HTTP secure-server Enabled
C:\Code\_EXAMPLES\pynetcheck\tests\cve_2023_20198\ios_xe_test.py:52: AssertionError: Startup - HTTP secure-server Vulnerable
C:\Code\_EXAMPLES\pynetcheck\tests\cve_2023_20198\ios_xe_test.py:30: AssertionError: Running - HTTP server Enabled
C:\Code\_EXAMPLES\pynetcheck\tests\cve_2023_20198\ios_xe_test.py:50: AssertionError: Running - HTTP server Vulnerable
======================================================================================== short test summary info ========================================================================================
FAILED pynetcheck\tests\cve_2023_20198\ios_xe_test.py::TestHTTPServerIPF::test_https_server_disabled[L77R11-LEAF5] - AssertionError: Startup - HTTP secure-server Enabled
FAILED pynetcheck\tests\cve_2023_20198\ios_xe_test.py::TestHTTPServerIPF::test_https_server_vulnerable[L77R11-LEAF5] - AssertionError: Startup - HTTP secure-server Vulnerable
FAILED pynetcheck\tests\cve_2023_20198\ios_xe_test.py::TestHTTPServerIPF::test_http_server_disabled[L67CSR16] - AssertionError: Running - HTTP server Enabled
FAILED pynetcheck\tests\cve_2023_20198\ios_xe_test.py::TestHTTPServerIPF::test_http_server_vulnerable[L67CSR16] - AssertionError: Running - HTTP server Vulnerable
================================================================================ 4 failed, 5 passed, 4 skipped in 1.94s ================================================================================= 
```

To run using a directory that stores a list of configuration files:

```bash
pynetcheck --config-dir /path/to/dir
```

To run against your own test cases:

```bash
>pynetcheck --tb=line --testpaths tests    
========================================================================================== test session starts ==========================================================================================
platform win32 -- Python 3.9.9, pytest-7.4.2, pluggy-1.3.0
rootdir: C:\Code\_EXAMPLES\config_vulnerability
configfile: pyproject.toml
plugins: anyio-4.0.0, depends-1.0.1, html-reporter-0.2.9
collected 357 items                                                                                                                                                                                      

tests\test_ssh.py FFFFFFFFFFFFFFFFFFF.FFF.FFF......FFF.FFFF.FFFFFFFFFFFFFFFFFFFFFFFF....F..........FF.F.....F.......FFFFF.F..FFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFF.FFFFFFFFFFFF.FF [ 49%]
FF.FFFFFFFFFFFFFFF.FFFFFFF.FFFFFFFFFFFFF.FFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFF..FFFF..FFFFFFFF.FFFFF.FFFF.FF.F...F.F.FFFFFFFFF.FFFFF..FF.....FF.FFFFFFFFFFF.FFF......F....FFFFFFFFFF              [100%]
```

## Results

### HTML

Results are stored in the [pytest_html_report.html](https://gitlab.com/ip-fabric/integrations/pynetcheck/-/raw/main/example/pytest_html_report.html) which can be viewed in any browser.  

![img.png](https://gitlab.com/ip-fabric/integrations/pynetcheck/-/raw/main/example/pytest_html.png)

### Exporting

The `pytest-html-reporter` also provides the ability to export via CSV or Excel formats, example: [pytest.csv](example/pytest.csv).

Table modified to show only the relevant information:

| Suite                               | Test Case                                  | Status | Time (s) | Error Message                                               |
|-------------------------------------|--------------------------------------------|--------|----------|-------------------------------------------------------------|
| tests/cve_2023_20198/ios_xe_test.py | test_saved_config_consistency              | PASS   | 0.21     |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_https_server_vulnerable[L77R12-LEAF6] | SKIP   | 0        |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_https_server_vulnerable[L77R11-LEAF5] | FAIL   | 0        | E   AssertionError: Startup - HTTP secure-server Vulnerable |
| tests/cve_2023_20198/ios_xe_test.py | test_https_server_vulnerable[L67CSR16]     | SKIP   | 0        |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_https_server_disabled[L77R12-LEAF6]   | PASS   | 0        |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_https_server_disabled[L77R11-LEAF5]   | FAIL   | 0        | E   AssertionError: Startup - HTTP secure-server Enabled    |
| tests/cve_2023_20198/ios_xe_test.py | test_https_server_disabled[L67CSR16]       | PASS   | 0        |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_http_server_vulnerable[L77R12-LEAF6]  | SKIP   | 0        |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_http_server_vulnerable[L77R11-LEAF5]  | SKIP   | 0        |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_http_server_vulnerable[L67CSR16]      | FAIL   | 0        | E   AssertionError: Running - HTTP server Vulnerable        |
| tests/cve_2023_20198/ios_xe_test.py | test_http_server_disabled[L77R12-LEAF6]    | PASS   | 0.13     |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_http_server_disabled[L77R11-LEAF5]    | PASS   | 0.15     |                                                             |
| tests/cve_2023_20198/ios_xe_test.py | test_http_server_disabled[L67CSR16]        | FAIL   | 0.15     | E   AssertionError: Running - HTTP server Enabled           |