Repository: cython/cython

Issue:
Cython coverage tool blames that function definition lines are not covered
I'm trying to enable a coverage for cython modules for [multidict](https://github.com/aio-libs/multidict) library. `Cython.Coverage` plugin works but the [coverage output](https://codecov.io/gh/aio-libs/multidict/src/cf1458ba151cffd74972cf00471ed09b9d6ee0dc/multidict/_multidict.pyx) looks incorrect.

The tool blames that method definitions are not executed lines but it's definitely wrong because method bodies are executed and test covered.

What I do wrong?



Discussion:
Did you try latest master?

I've tried cython 0.24.1
On master I have another error:

```
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/home/andrew/.virtualenvs/multidict/lib/python3.5/site-packages/pytest_cov/plugin.py", line 212, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/home/andrew/.virtualenvs/multidict/lib/python3.5/site-packages/pytest_cov/engine.py", line 150, in finish
INTERNALERROR>     self.cov.combine()
INTERNALERROR>   File "/home/andrew/.virtualenvs/multidict/lib/python3.5/site-packages/coverage/control.py", line 784, in combine
INTERNALERROR>     self.get_data()
INTERNALERROR>   File "/home/andrew/.virtualenvs/multidict/lib/python3.5/site-packages/coverage/control.py", line 810, in get_data
INTERNALERROR>     self.collector.save_data(self.data)
INTERNALERROR>   File "/home/andrew/.virtualenvs/multidict/lib/python3.5/site-packages/coverage/collector.py", line 349, in save_data
INTERNALERROR>     covdata.add_arcs(abs_file_dict(self.data))
INTERNALERROR>   File "/home/andrew/.virtualenvs/multidict/lib/python3.5/site-packages/coverage/data.py", line 365, in add_arcs
INTERNALERROR>     raise CoverageException("Can't add arcs to existing line data")
INTERNALERROR> coverage.misc.CoverageException: Can't add arcs to existing line data
```

Aha, `coverage combine` doesn't work somehow.
Anyway, even without combining I have the same behavior (method definitions are not covered) on cython master too.

Steps to reproduce:
1. Clone https://github.com/aio-libs/multidict
2. Switch to `cython_coverage` branch
3. Run `pip install -r requirements-ci.txt`
4. Run `make cov-dev`, open prompted  html file.
Method definition lines from `multidict/_multidict.pyx` are not covered. 
5. For demonstrating issue with coverage files combining run `make cov-dev-full`.

Without having looked at the code, my guess is that we simply don't generate trace calls for function/method/class declarations. The whole tracing support works by explicitly calling the tracer before the C code that is generated for each line of user code that gets executed, and declarations aren't really considered user code. The trace call would need to get added to the instantiation of class objects and functions. AFAIR, we already trace the user code in the module init function, so this should generally be possible, although some C things might happen in an unexpected order in Cython.

Facing the same issue... Method definitions (def/cpdef/cdef) in cdef classes are not covered, when they should be.

Also found person facing the same issue on SO - http://stackoverflow.com/questions/34935796/how-to-use-coverage-analysis-with-cython

Same issue here. The methods' body shows up as green, but the signature is always red.
Please let me know if I can help further debug this issue.
Thanks
As written above, we don't generate tracing calls for the function signature line. Or, rather, if we do, then they are only executed at import time, not at call time. Probably easy to implement, but needs a test that makes sure it really behaves exactly like CPython.
I also spent few hours on figuring this out. In the end, I found [a _cython-users_ Google Groups message](https://groups.google.com/d/msg/cython-users/e80vdI9G_3w/NTQTF009BAAJ) which recommends **enabling `binding` directive**.

This seems to completely remove the function signature statements from the coverage and I can reach 100% coverage again.

Could anyone comment on this workaround? Is it safe to use?
(I should also add I only compile with `binding` directive for test and coverage, not the production)
The `binding` directive is generally a good idea, mainly because it improves the Python compatibility of Cython compiled functions. However, I would not recommend using it only for testing, since disabling it for production changes the behaviour there.
@scoder I don't fully grok coverage.py or Cython.Coverage, am trying to get a handle on them to make a PR.

Would patching `Plugin._parse_lines` to consider these lines not-executable be a viable solution to this problem?  Or would it be liable to cause more problems?

<b>Update</b> On trying this out I'm seeing weird results.  It looks like we need to patch the coverage results to think these lines _were_ executed, as opposed to thinking they were not executable.
> Would patching Plugin._parse_lines to consider these lines not-executable be a viable solution to this problem? Or would it be liable to cause more problems?

I don't know anything about the coverage.py or Cython.Coverage code, but in principle this sounds like the correct solution if the details are worked out.

The following report is an example of how I am seeing this problem manifest: https://codecov.io/gh/lenskit/lkpy/src/7d705b4b9a051bca66f6cdc1873d3ab4cecdb52b/lenskit/algorithms/_funksvd.pyx

`cdef` local variables with initializers are marked as hit. A `cdef` field declaration of a memory view is also being marked as hit.
> consider these lines not-executable

No, that sound wrong. They are executable, CPython counts them, too. Could someone find out if CPython counts them only at import time or also at call time?

If the latter, then the fix is to also generate a line trace event for the function call.
setting binding=True as @kysely suggested above didn't change anything for me

> Could someone find out if CPython counts them only at import time or also at call time?

How do we find out?
btw, i'm also seeing switch statements being flagged in red as not covered, but the body of the switch branches is green
https://codecov.io/gh/fonttools/openstep-plist/src/master/src/openstep_plist/_parser.pyx#L119...177
> switch statements

Yes, that's a known bug. Cython generates the initial `TraceLine` call in a place where it is not executed. PR welcome, as always.

> How do we find out?

Run an import with coverage reporting, look at the result. Exclude the import from coverage reporting, run a function, look at the result.

But, actually, I'd suggest looking at the CPython sources and finding out if a "call" trace event also triggers a "line" trace event. That would be the definite indicator.
I just did that. I created two python modules `a.py` and `b.py`. The former contains `import b`, and the latter defines a function `def f(): print('hello world')`.
When I run `coverage run a.py`, I can see that the signature of function `f` in the module `b.py` is green, whereas the body is red (as it was ony imported, not called).
If I add a `# pragma: no cover` next to `import b` inside `a.py` module, and then I add a new line where I call the function from b, `b.f()`, after I re-run `coverage run a.py` I can see the `import b` line is now grey (excluded), and in the module `b.py` I can see that both the function signature and its body are now green.

> I'd suggest looking at the CPython sources and finding out if a "call" trace event also triggers a "line" trace event. That would be the definite indicator.

right. That sounds more difficult, I'm not sure where to look..

Resolving patch:
diff --git a/Cython/Compiler/Nodes.py b/Cython/Compiler/Nodes.py
--- a/Cython/Compiler/Nodes.py
+++ b/Cython/Compiler/Nodes.py
@@ -2653,6 +2653,9 @@ def generate_argument_type_tests(self, code):
                 self.generate_arg_none_check(arg, code)
 
     def generate_execution_code(self, code):
+        if code.globalstate.directives['linetrace']:
+            code.mark_pos(self.pos)
+            code.putln("")  # generate line tracing code
         super(CFuncDefNode, self).generate_execution_code(code)
         if self.py_func_stat:
             self.py_func_stat.generate_execution_code(code)


Test patch:
diff --git a/tests/run/coverage_nogil.srctree b/tests/run/coverage_nogil.srctree
--- a/tests/run/coverage_nogil.srctree
+++ b/tests/run/coverage_nogil.srctree
@@ -25,19 +25,19 @@ plugins = Cython.Coverage
 # cython: linetrace=True
 # distutils: define_macros=CYTHON_TRACE=1 CYTHON_TRACE_NOGIL=1
 
-cdef int func1(int a, int b) nogil:
-    cdef int x                   #  5
-    with gil:                    #  6
-        x = 1                    #  7
-    cdef int c = func2(a) + b    #  8
-    return x + c                 #  9
+cdef int func1(int a, int b) nogil:  #  4
+    cdef int x                       #  5
+    with gil:                        #  6
+        x = 1                        #  7
+    cdef int c = func2(a) + b        #  8
+    return x + c                     #  9
 
 
-cdef int func2(int a) with gil:
+cdef int func2(int a) with gil:  # 12
     return a * 2                 # 13
 
 
-def call(int a, int b):
+def call(int a, int b):          # 16
     a, b = b, a                  # 17
     with nogil:                  # 18
         result = func1(b, a)     # 19
@@ -56,20 +56,18 @@ except ImportError:
 from coverage import coverage
 
 
-import coverage_test_nogil
-
-assert not any(coverage_test_nogil.__file__.endswith(ext)
-               for ext in '.py .pyc .pyo .pyw .pyx .pxi'.split()), \
-    coverage_test_nogil.__file__
-
+def run_coverage():
+    cov = coverage()
+    cov.start()
 
-def run_coverage(module):
+    import coverage_test_nogil as module
     module_name = module.__name__
     module_path = module_name + '.pyx'
-
-    cov = coverage()
-    cov.start()
+    assert not any(module.__file__.endswith(ext)
+                   for ext in '.py .pyc .pyo .pyw .pyx .pxi'.split()), \
+        module.__file__
     assert module.call(1, 2) == (1 * 2) + 2 + 1
+
     cov.stop()
 
     out = StringIO()
@@ -84,10 +82,10 @@ def run_coverage(module):
 
     executed = set(exec_lines) - set(missing_lines)
     # check that everything that runs with the gil owned was executed
-    assert all(line in executed for line in [13, 17, 18, 20]), '%s / %s' % (exec_lines, missing_lines)
+    assert all(line in executed for line in [12, 13, 16, 17, 18, 20]), '%s / %s' % (exec_lines, missing_lines)
     # check that everything that runs in nogil sections was executed
-    assert all(line in executed for line in [6, 7, 8, 9]), '%s / %s' % (exec_lines, missing_lines)
+    assert all(line in executed for line in [4, 6, 7, 8, 9]), '%s / %s' % (exec_lines, missing_lines)
 
 
 if __name__ == '__main__':
-    run_coverage(coverage_test_nogil)
+    run_coverage()