Repository: DiamondLightSource/pythonSoftIOC (language: python)

Issue:
devIocStats: SYS_CPU_LOAD and IOC_CPU_LOAD values are incorrect
When enabling `softioc.devIocStats`, the values of <ioc>:SYS_CPU_LOAD and <ioc>:IOC_CPU_LOAD are incorrect.

To reproduce, compose `ioc.py` based on tutorial:
```
from softioc import softioc, builder
import cothread

builder.SetDeviceName("MY-DEVICE-PREFIX")
ai = builder.aIn('AI', initial_value=5)
ao = builder.aOut('AO', initial_value=12.45, on_update=lambda v: ai.set(v))
builder.LoadDatabase()
softioc.devIocStats('my_ioc')
softioc.iocInit()

def update():
    while True:
        ai.set(ai.get() + 1)
        cothread.Sleep(1)
cothread.Spawn(update)
softioc.interactive_ioc(globals())
```

Result when running `ioc.py`:
```
Starting iocInit
############################################################################
## EPICS 7.0.6.1
## Rev. 7.0.6.99.2.0
############################################################################
iocRun: All initialization complete
Python 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> dbpr('my_ioc:SYS_CPU_LOAD')
my_ioc:SYS_CPU_LOAD 0
AMSG:               ASG :               DESC: System CPU Load                   
DISA: 0             DISP: 0             DISV: 1             
NAME: my_ioc:SYS_CPU_LOAD               NAMSG:              RVAL: 0             
SEVR: MAJOR         STAT: HIHI          SVAL: 0             TPRO: 0             
VAL : 2442.3690941135
>>> dbpr('my_ioc:IOC_CPU_LOAD')
my_ioc:IOC_CPU_LOAD 0
AMSG:               ASG :               DESC: IOC CPU Load  DISA: 0             
DISP: 0             DISV: 1             NAME: my_ioc:IOC_CPU_LOAD               
NAMSG:              RVAL: 0             SEVR: NO_ALARM      STAT: NO_ALARM      
SVAL: 0             TPRO: 0             VAL : 20.0192856949861
>>> 
```

True cpu load is about 0.3 - 0.4%:
```
$ ps -p 4186753 -o %cpu,%mem 
%CPU %MEM
 0.3  0.1
$ head -n 1 /proc/stat
cpu  221105886 1802813 52192077 7864068116 1410091 0 1526417 0 0 0
```


Discussion / hints:
Thank you for pointing this out. It looks like there's more broken PVs than just those ones. For example:

```
>>> dbpr("my_ioc:CPU_CNT")
AMSG:               ASG :               DESC: Number of CPUs                    
DISA: 0             DISV: 1             NAME: my_ioc:CPU_CNT                    
NAMSG:              RVAL: 0             SEVR: NO_ALARM      STAT: NO_ALARM      
SVAL: 0             TPRO: 0             VAL : 1  
```

As you can see, it looks like devIocStats is not correctly picking up the number of CPUs on my system (I have 4). This number is used in a lot of calculations, so is probably a/the source of the issue.

Looking at the devIocStatsOSD.h file, it seems there is a [defaults version](https://github.com/epics-modules/iocStats/blob/master/devIocStats/os/default/devIocStatsOSD.h) but there is no override header file for [Linux](https://github.com/epics-modules/iocStats/tree/master/devIocStats/os/Linux). This means the number of CPUs defaults to 1, and is never adjusted. 

(We use the Linux distribution as that's how we set up the build in [setup.py](https://github.com/dls-controls/pythonSoftIOC/blob/289897eecae1a3abe00e91b8f79e443e6bc4fea7/setup.py#LL44C54-L44C80) - the `OS_CLASS` variable returns `Linux`)

I'm tempted to say this is an issue in devIocStats, although I'd probably have to do more investigation to prove that.
I think something in the EPICS build system will pick up the `posix` version too, which we are not currently pulling into the build. 

This file would appear to give the right CPU count:
https://github.com/epics-modules/iocStats/blob/fa12cb2172e3477debd1372f57e68d2ae21f7c1b/devIocStats/os/posix/devIocStatsOSD.h#L42-L43

I think the best thing to do would be to build devIocStats once standalone and look at the gcc command it generates for Linux, which files it picks up, and replicate that in setup.py
Here's the gcc line from devIocStats:
```
/usr/bin/gcc  -D_GNU_SOURCE -D_DEFAULT_SOURCE          -D_X86_64_ -DUNIX  -Dlinux      -O3 -g   -Wall -Werror-implicit-function-declaration      -mtune=generic     -m64  -fPIC -I. -I../O.Common -I. -I. -I../os/Linux -I../os/posix -I../os/default -I.. -I../../include/compiler/gcc -I../../include/os/Linux -I../../include         -I/repos/epics/support/sncseq/include -I/repos/epics/epics-base/include/compiler/gcc -I/repos/epics/epics-base/include/os/Linux -I/repos/epics/epics-base/include        -MM -MF osdCpuUtilization.d  ../os/Linux/osdCpuUtilization.c
```

Fix patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 7afd2070..ef898e67 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -14,6 +14,10 @@ Changed:
 
 - `Improve string representation of RecordWrapper instances <../../pull/130>`
 
+Fixed:
+
+- `Fix devIocStats by including posix headers <../../pull/134>`
+
 4.3.0_ - 2023-04-04
 -------------------
 
diff --git a/setup.py b/setup.py
index 34157bce..1b70ecd2 100644
--- a/setup.py
+++ b/setup.py
@@ -41,6 +41,7 @@
 ]
 
 devIocStats_src = os.path.join("softioc", "iocStats", "devIocStats")
+devIocStats_posix = os.path.join(devIocStats_src, "os", "posix")
 devIocStats_os = os.path.join(devIocStats_src, "os", get_config_var('OS_CLASS'))
 devIocStats_default = os.path.join(devIocStats_src, "os", "default")
 
@@ -52,14 +53,26 @@
     else:
         sources.append(os.path.join(devIocStats_default, f))
 
+include_dirs = [
+    epicscorelibs.path.include_path,
+    devIocStats_src,
+    devIocStats_os,
+    devIocStats_default
+]
+
+if get_config_var("POSIX"):
+    # If we're on a POSIX system, insert the POSIX folder into the list after
+    # the os-specific one so that os-specific header files are used first.
+    include_dirs.insert(
+        include_dirs.index(devIocStats_os) + 1,
+        devIocStats_posix
+    )
+
 # Extension with all our C code
 ext = Extension(
     name='softioc._extension',
     sources = sources,
-    include_dirs=[
-        epicscorelibs.path.include_path,
-        devIocStats_src, devIocStats_os, devIocStats_default
-    ],
+    include_dirs = include_dirs,
     dsos = [
         'epicscorelibs.lib.qsrv',
         'epicscorelibs.lib.pvAccessIOC',


Test patch:
diff --git a/tests/test_deviocstats.py b/tests/test_deviocstats.py
new file mode 100644
index 00000000..5047200e
--- /dev/null
+++ b/tests/test_deviocstats.py
@@ -0,0 +1,71 @@
+# File for tests related to devIocStats support module, which at time of writing
+# is built alongside PythonSoftIOC and optionally turned on at runtime
+
+import multiprocessing
+import pytest
+
+from conftest import (
+    create_random_prefix,
+    TIMEOUT,
+    select_and_recv,
+    get_multiprocessing_context
+)
+
+from softioc import asyncio_dispatcher, builder, softioc
+
+def deviocstats_test_func(
+        device_name,
+        child_conn):
+    """Start the IOC with the specified validate method"""
+
+    builder.SetDeviceName(device_name)
+
+    dispatcher = asyncio_dispatcher.AsyncioDispatcher()
+    builder.LoadDatabase()
+    softioc.devIocStats(device_name)
+    softioc.iocInit(dispatcher)
+
+    child_conn.send("R")
+
+    # Keep process alive while main thread runs CAGET
+    if child_conn.poll(TIMEOUT):
+        val = child_conn.recv()
+        assert val == "D", "Did not receive expected Done character"
+
+@pytest.mark.asyncio
+async def test_deviocstats():
+
+    ctx = get_multiprocessing_context()
+
+    parent_conn, child_conn = ctx.Pipe()
+
+    device_name = create_random_prefix()
+
+    process = ctx.Process(
+        target=deviocstats_test_func,
+        args=(device_name, child_conn),
+    )
+
+    process.start()
+
+    from aioca import caget, purge_channel_caches
+
+    try:
+        # Wait for message that IOC has started
+        select_and_recv(parent_conn, "R")
+
+        # Suppress potential spurious warnings
+        purge_channel_caches()
+
+        cpu_cnt = await caget(device_name + ":CPU_CNT")
+        assert cpu_cnt == multiprocessing.cpu_count()
+
+        ioc_cpu_load = await caget(device_name + ":IOC_CPU_LOAD")
+        assert ioc_cpu_load == pytest.approx(0, abs=1e-2)
+
+
+    finally:
+        # Suppress potential spurious warnings
+        purge_channel_caches()
+        parent_conn.send("D")  # "Done"
+        process.join(timeout=TIMEOUT)