Repository: zephyrproject-rtos/zephyr

Issue #48608: boards: mps2_an385: Unstable system timer
Created: 2022-08-03T08:47:45Z
Describe the bug
The system timer on the mps2_an385 platform, when run inside QEMU with the "icount" mode disabled, is unstable and may return an invalid uptime value (e.g. when calling k_uptime_get_32()), leading to frequent test failures and system crashes.
While the mps2_an385 platform normally runs with the "icount" mode enabled in order to ensure stable timing, it still runs the networking tests with the icount mode disabled:

  
    
      zephyr/boards/Kconfig
    
    
        Lines 58 to 61
      in
      c11b785
    
  
  
    

        
          
           config QEMU_ICOUNT 
        

        
          
           	bool "QEMU icount mode" 
        

        
          
           	depends on QEMU_TARGET 
        

        
          
           	default y if !NETWORKING && !BT 
        
    
  


This leads to the networking tests randomly failing on the mps2_an385 platform and forces the twister to retry running these tests until they pass.
While the above has always been an issue, we have recently introduced, in the commit 3d1bbe5, a test case that is more sensitive to the timing stability and this led to a significantly increased test failure rate, to the degree that simply increasing the retry count is no longer acceptable (it is already at 4).
The underlying cause of this timing instability seems to have never been investigated, so investigate and fix this issue accordingly.
To Reproduce
Run tests/kernel/timer/timer_monotonic on mps2_an385 with CONFIG_QEMU_ICOUNT=n:
cmake -GNinja -DBOARD=mps2_an385 -DCONFIG_QEMU_ICOUNT=n ..
ninja run

Logs and console output
The test fails most of the times; though, it may occasionally succeed.
Attempt 1
Running TESTSUITE timer_fn
===================================================================
START - test_timer
k_ticks_to_cyc_floor32(1) = 250000
sys_clock_hw_cycles_per_sec() = 25000000
START - test monotonic timer
diff = -16498779 (t_last = 49847299 : t_now = 33348520);i = 427277

    Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_monotonic/src/main.c:80: test_timer: (errors is true)
errors = 1

 FAIL - test_timer in 3.649 seconds
===================================================================
TESTSUITE timer_fn failed.
===================================================================
PROJECT EXECUTION FAILED

Attempt 2
Running TESTSUITE timer_fn
===================================================================
START - test_timer
k_ticks_to_cyc_floor32(1) = 250000
sys_clock_hw_cycles_per_sec() = 25000000
START - test monotonic timer
diff = -16498882 (t_last = 66367902 : t_now = 49869020);i = 656041

    Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_monotonic/src/main.c:80: test_timer: (errors is true)
errors = 1

 FAIL - test_timer in 2.954 seconds
===================================================================
TESTSUITE timer_fn failed.
===================================================================
PROJECT EXECUTION FAILED

Expected behavior
Test always passes.
Impact
Networking tests are rendered ineffective on mps2_an385 because of an unreasonably high failure rate.
Environment (please complete the following information):

OS: Ubuntu 20.04
Toolchain: Zephyr SDK 0.14.2
Commit SHA: bc0d0a3

Additional context
Related discussion on Discord: https://discord.com/channels/720317445772017664/733037890514321419/1004039756587479091
The networking test failure that motivated this:
https://github.com/zephyrproject-rtos/zephyr/runs/7644823137?check_suite_focus=true#step:11:1864
INFO    - 1157/1157 mps2_an385                tests/net/socket/tcp/net.socket.tcp.preempt        FAILED Failed (qemu 87.725s)
...
===================================================================
START - test_v4_msg_waitall

    Assertion failed at WEST_TOPDIR/zephyr/tests/net/socket/tcp/src/main.c:203: test_context_cleanup: (wait_for_n_tcp_contexts(0, TCP_TEARDOWN_TIMEOUT) not equal to 0)
Not all TCP contexts properly cleaned up
 FAIL - test_v4_msg_waitall in 0.391 seconds
...

Duplicate issue #57405: net.socket.tls.preempt fails on mps2_an385
Created: 2023-04-29T15:18:40Z
Describe the bug
The ztests for net.socket.tls.preempt are failing for mps2_an385 it looked like the tests may be timing out due to a failure to allocate a lock due to running out of memory. These tests have been disabled and need to be reenabled.
To Reproduce
Steps to reproduce the behavior:

On linux setup a network bridge and the net-tools using https://docs.zephyrproject.org/3.1.0/connectivity/networking/native_posix_setup.html
Run the test set mentioned above with twister or west
Currently one test set will pass and one test set will fail
See error in the json report.

Expected behavior
The test should pass.
Impact
If the out of memory error is in fact happening, the way it seems to happen for me due to something other than user error, it is perplexing as to why this would happen since these boards have 4M of memory.

Closing comment (2023-10-04T12:48:26Z): @jgl-meta tests for this platform were disabled here: #63220 due to #48608. Should we close this issue as a duplicate of #48608?

It indeed does look like it.
Closing as duplicate.