internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: twisted/twisted

Issue:
twisted uses deprecated three-argument form of generator.throw
**Describe the incorrect behavior you saw**
`src/twisted/python/failure.py` does `return g.throw(self.type, self.value, self.tb)`. This is deprecated in Python 3.12: https://github.com/python/cpython/issues/96348 , https://docs.python.org/3.12/whatsnew/3.12.html#deprecated . This is a particular problem for https://github.com/fedora-infra/fedora-messaging consumers - they print a deprecation warning about once a second.

**Describe how to cause this behavior**
Try using twisted with Python 3.12, in a way that hits this codepath (I'm not sure precisely how to do that other than "use a fedora-messaging consumer", sorry). You get a deprecation warning:

```
Oct 27 23:11:08 openqa-lab01.iad2.fedoraproject.org fedora-messaging[172628]: /usr/lib/python3.12/site-packages/twisted/python/failure.py:518: DeprecationWarning: the (type, exc, tb) signature of throw() is deprecated, use the single-arg signature instead.
Oct 27 23:11:08 openqa-lab01.iad2.fedoraproject.org fedora-messaging[172628]:   return g.throw(self.type, self.value, self.tb)
```

**Describe the correct behavior you'd like to see**
No deprecation warning.

**Testing environment**
Python 3.12 on Fedora 39, don't think anything else is relevant. The code is still using the deprecated form on current git trunk.

**Additional context**
I would send a pull request changing this, but I'm unclear on [whether it's safe to just use the modern one-argument form on Python 3.8](https://github.com/python/cpython/issues/96348#issuecomment-1783626480).



Discussion:

Resolving patch:
diff --git a/src/twisted/python/failure.py b/src/twisted/python/failure.py
--- a/src/twisted/python/failure.py
+++ b/src/twisted/python/failure.py
@@ -516,7 +516,7 @@ def throwExceptionIntoGenerator(self, g):
         """
         # Note that the actual magic to find the traceback information
         # is done in _findFailure.
-        return g.throw(self.type, self.value, self.tb)
+        return g.throw(self.value.with_traceback(self.tb))
 
     @classmethod
     def _findFailure(cls):


Test patch:
Repository: twisted/twisted

Issue:
Update ZshArgumentsGenerator for python 3.13
The arg parser works a bit different in 3.13


Discussion:

Resolving patch:
diff --git a/src/twisted/python/_shellcomp.py b/src/twisted/python/_shellcomp.py
--- a/src/twisted/python/_shellcomp.py
+++ b/src/twisted/python/_shellcomp.py
@@ -603,7 +603,9 @@ def getDescription(self, longname):
         obj = getattr(self.options, "opt_%s" % longMangled, None)
         if obj is not None:
             descr = descrFromDoc(obj)
-            if descr is not None:
+            # On Python3.13 we have an empty string instead of None,
+            # for missing description.
+            if descr:
                 return descr
 
         return longname  # we really ought to have a good description to use


Test patch:
Repository: twisted/twisted

Issue:
logging regression
As @om26er reports in https://github.com/twisted/twisted/pull/12040#issuecomment-1888181534 , the addition of parsing parens to match the documentation broke the subscripting part of the log-format protocol


Discussion:

Resolving patch:
diff --git a/src/twisted/logger/_format.py b/src/twisted/logger/_format.py
--- a/src/twisted/logger/_format.py
+++ b/src/twisted/logger/_format.py
@@ -201,6 +201,12 @@ def __init__(self, wrapped: object) -> None:
     def __getattr__(self, name: str) -> object:
         return keycall(name, self._wrapped.__getattribute__)
 
+    def __getitem__(self, name: str) -> object:
+        # The sub-object may not be indexable, but if it isn't, that's the
+        # caller's problem.
+        value = self._wrapped[name]  # type:ignore[index]
+        return PotentialCallWrapper(value)
+
     def __format__(self, format_spec: str) -> str:
         return self._wrapped.__format__(format_spec)
 


Test patch:
diff --git a/src/twisted/logger/test/test_format.py b/src/twisted/logger/test/test_format.py
--- a/src/twisted/logger/test/test_format.py
+++ b/src/twisted/logger/test/test_format.py
@@ -5,7 +5,7 @@
 Test cases for L{twisted.logger._format}.
 """
 
-from typing import AnyStr, Optional, cast
+from typing import AnyStr, Dict, Optional, cast
 
 try:
     from time import tzset
@@ -95,6 +95,22 @@ def where(self) -> str:
             "hello world", self.format("hello {what.where()}", what=World())
         )
 
+    def test_formatAttributeSubscript(self) -> None:
+        """
+        L{formatEvent} will format subscripts of attributes per PEP 3101.
+        """
+
+        class Example(object):
+            config: Dict[str, str] = dict(foo="bar", baz="qux")
+
+        self.assertEqual(
+            "bar qux",
+            self.format(
+                "{example.config[foo]} {example.config[baz]}",
+                example=Example(),
+            ),
+        )
+
     def test_formatEventNoFormat(self) -> None:
         """
         Formatting an event with no format.
Repository: twisted/twisted

Issue:
twisted's test suite fails on pypy 7.3.14 due to stack depth changes
This was reported in https://github.com/pypy/pypy/issues/4850 but is really a bug in Twisted, not PyPy, since PyPy is *fixing* a bug here to get closer to CPython's behavior.


Discussion:

Resolving patch:
diff --git a/src/twisted/internet/defer.py b/src/twisted/internet/defer.py
--- a/src/twisted/internet/defer.py
+++ b/src/twisted/internet/defer.py
@@ -17,7 +17,7 @@
 from contextvars import Context as _Context, copy_context as _copy_context
 from enum import Enum
 from functools import wraps
-from sys import exc_info
+from sys import exc_info, implementation
 from types import CoroutineType, GeneratorType, MappingProxyType, TracebackType
 from typing import (
     TYPE_CHECKING,
@@ -58,6 +58,9 @@
 _T = TypeVar("_T")
 _P = ParamSpec("_P")
 
+# See use in _inlineCallbacks for explanation and removal timeline.
+_oldPypyStack = _PYPY and implementation.version < (7, 3, 14)
+
 
 class AlreadyCalledError(Exception):
     """
@@ -2022,8 +2025,10 @@ def _inlineCallbacks(
             appCodeTrace = traceback.tb_next
             assert appCodeTrace is not None
 
-            if _PYPY:
-                # PyPy as of 3.7 adds an extra frame.
+            if _oldPypyStack:
+                # PyPy versions through 7.3.13 add an extra frame; 7.3.14 fixed
+                # this discrepancy with CPython.  This code can be removed once
+                # we no longer need to support PyPy 7.3.13 or older.
                 appCodeTrace = appCodeTrace.tb_next
                 assert appCodeTrace is not None
 


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -168,7 +168,7 @@ jobs:
           # extension.
           # There is very little PYPY specific code so there is not much to
           # gain from reporting coverage.
-          - python-version: 'pypy3.10-v7.3.13'
+          - python-version: 'pypy3.10-v7.3.14'
             tox-env: 'alldeps-nocov-posix'
             job-name: 'no-coverage'
             skip-coverage: yes
Repository: twisted/twisted

Issue:
`BufferingTLSTransport` has docstring formatting errors.
see https://docs.twisted.org/en/stable/api/twisted.protocols.tls.BufferingTLSTransport.html


Discussion:

Resolving patch:
diff --git a/src/twisted/protocols/tls.py b/src/twisted/protocols/tls.py
--- a/src/twisted/protocols/tls.py
+++ b/src/twisted/protocols/tls.py
@@ -764,19 +764,20 @@ def _get_default_clock() -> IReactorTime:
 class BufferingTLSTransport(TLSMemoryBIOProtocol):
     """
     A TLS transport implemented by wrapping buffering around a
-    ``TLSMemoryBIOProtocol``.
+    L{TLSMemoryBIOProtocol}.
 
-    Doing many small writes directly to a ``OpenSSL.SSL.Connection``, as
-    implemented in ``TLSMemoryBIOProtocol``, can add significant CPU and
+    Doing many small writes directly to a L{OpenSSL.SSL.Connection}, as
+    implemented in L{TLSMemoryBIOProtocol}, can add significant CPU and
     bandwidth overhead.  Thus, even when writing is possible, small writes will
     get aggregated and written as a single write at the next reactor iteration.
     """
 
-    # Note: An implementation based on composition would be nicer, but there's
-    # close integration between ``ProtocolWrapper`` subclasses like
-    # ``TLSMemoryBIOProtocol`` and the corresponding factory. Composition broke
-    # things like ``TLSMemoryBIOFactory.protocols`` having the correct
-    # instances, whereas subclassing makes that work.
+    # Implementation Note: An implementation based on composition would be
+    # nicer, but there's close integration between L{ProtocolWrapper}
+    # subclasses like L{TLSMemoryBIOProtocol} and the corresponding factory. An
+    # attempt to implement this with broke things like
+    # L{TLSMemoryBIOFactory.protocols} having the correct instances, whereas
+    # subclassing makes that work.
 
     def __init__(
         self,


Test patch:
Repository: twisted/twisted

Issue:
twisted.web.server.Request.defaultContentType is missing type annotation
MyPy assumes it's always `bytes`, but docs explain it can also be `None`, so really it should be `Optional[bytes]`.


Discussion:

Resolving patch:
diff --git a/src/twisted/web/server.py b/src/twisted/web/server.py
--- a/src/twisted/web/server.py
+++ b/src/twisted/web/server.py
@@ -101,8 +101,7 @@ class Request(Copyable, http.Request, components.Componentized):
         will be transmitted only over HTTPS.
     """
 
-    defaultContentType = b"text/html"
-
+    defaultContentType: Optional[bytes] = b"text/html"
     site = None
     appRootURL = None
     prepath: Optional[List[bytes]] = None


Test patch:
Repository: twisted/twisted

Issue:
Avoid encode/decode in chat.py for better readablity
As discussed in [this comment](https://github.com/twisted/twisted/pull/12070#discussion_r1442784443), it's better to use byte concat as mentioned in order of better readability in [docs/core/howto/listings/servers/chat.py:35](https://github.com/twisted/twisted/pull/12070/files/c59c93ec644a17e0f3a1752ca9ceca31a27a9f5e#diff-0923ff3db530a2e5d28ea8cc2b3a8f91f399792786772c541bf9edf7a0c50126)
```python
message = b'<' + self.name + b'> ' + message
```


Discussion:

Resolving patch:
diff --git a/docs/core/howto/listings/servers/chat.py b/docs/core/howto/listings/servers/chat.py
--- a/docs/core/howto/listings/servers/chat.py
+++ b/docs/core/howto/listings/servers/chat.py
@@ -32,9 +32,7 @@ def handle_GETNAME(self, name):
         self.state = "CHAT"
 
     def handle_CHAT(self, message):
-        message = f"<{self.name.decode('utf-8')}> {message.decode('utf-8')}".encode(
-            "utf-8"
-        )
+        message = b"<" + self.name + b"> " + message
         for name, protocol in self.users.items():
             if protocol != self:
                 protocol.sendLine(message)


Test patch:
Repository: twisted/twisted

Issue:
Remove isinstance assertions in Deferred
The Deferred implementation has a bunch of assertions about inputs. These assertions slow down code. We could remove them, relying on type checking to catch these problems.


Discussion:

Resolving patch:
diff --git a/src/twisted/internet/defer.py b/src/twisted/internet/defer.py
--- a/src/twisted/internet/defer.py
+++ b/src/twisted/internet/defer.py
@@ -522,9 +522,6 @@ def addCallbacks(
         if errbackKeywords is None:
             errbackKeywords = {}  # type: ignore[unreachable]
 
-        assert callable(callback)
-        assert callable(errback)
-
         self.callbacks.append(
             (
                 (callback, callbackArgs, callbackKeywords),
@@ -873,7 +870,6 @@ def callback(self, result: Union[_SelfResultT, Failure]) -> None:
         @raise AlreadyCalledError: If L{callback} or L{errback} has already been
             called on this L{Deferred}.
         """
-        assert not isinstance(result, Deferred)
         self._startRunCallbacks(result)
 
     def errback(self, fail: Optional[Union[Failure, BaseException]] = None) -> None:


Test patch:
Repository: angr/angr

Issue:
Explorer seems to not be removed
Hello,

So I was testing tracer on a non-CGC binary (`fauxware`), and got the `program did not behave correctly, expected only one path` error right after the first step (1). Printing out the simgr stashes gives me 0 `active` 0 `deadended` but 1 `found` (2), but we never use that `found` stash in tracer.

 Apparently we do use `Explorer` for Linux binaries in tracer to find out the entry point. But the `found` stash has indeed got unstashed right after `Explorer` returned.

From (1) and (2), I really think that `Explorer` is always used in `simgr.step`, despite the call to `remove_tech` in the end of `explore`.


Discussion:

Resolving patch:
diff --git a/angr/manager.py b/angr/manager.py
--- a/angr/manager.py
+++ b/angr/manager.py
@@ -118,6 +118,7 @@ def _ana_setstate(self, s):
     def copy(self, stashes=None):
         stashes = stashes if stashes is not None else self._copy_stashes(immutable=True)
         out = SimulationManager(self._project, stashes=stashes, hierarchy=self._hierarchy, immutable=self._immutable, resilience=self._resilience, save_unconstrained=self.save_unconstrained, save_unsat=self.save_unsat, errored=self.errored)
+        out._hooks_all = list(self._hooks_all)
         out._hooks_step = list(self._hooks_step)
         out._hooks_step_state = list(self._hooks_step_state)
         out._hooks_filter = list(self._hooks_filter)


Test patch:
Repository: angr/angr

Issue:
ARM jump sites missing
Hello,

![js_missing](https://user-images.githubusercontent.com/23738463/30420477-f4230df6-9939-11e7-91a8-da6a0e8dc752.png)

As you can see, a `ble` with **only one** out edge.... It is not a bug in angr-utils, since I've printed out all CFG edges and it's indeed missing...


Discussion:
Usually I blame VEX...

Joking aside, can you please try turning off the optimization?
Oh, I tried both with and without the optimization...
Can you please post the bytes of those two blocks here? Or put the binary here so we can debug it later. Thanks.
Wait, I found the bug. I submitted a PR for this but before the refactor and you didn't merge in...
OH

Well, time to rebase...
Should I send you a PR?
Of course.

Resolving patch:
diff --git a/angr/analyses/cfg/cfg_accurate.py b/angr/analyses/cfg/cfg_accurate.py
--- a/angr/analyses/cfg/cfg_accurate.py
+++ b/angr/analyses/cfg/cfg_accurate.py
@@ -1405,7 +1405,7 @@ def _post_handle_job_debug(self, job, successors):
         extra_info = job.extra_info
         successor_status = job.successor_status
 
-        func = self.project.loader.find_symbol(job.addr)
+        func = self.project.loader.find_symbol(job.func_addr)
         obj = self.project.loader.find_object_containing(job.addr)
         function_name = func.name if func is not None else None
         module_name = obj.provides if obj is not None else None
diff --git a/angr/analyses/cfg/cfg_base.py b/angr/analyses/cfg/cfg_base.py
--- a/angr/analyses/cfg/cfg_base.py
+++ b/angr/analyses/cfg/cfg_base.py
@@ -528,7 +528,7 @@ def _arm_thumb_filter_jump_successors(self, addr, successors, get_ins_addr, get_
         THUMB_BRANCH_INSTRUCTIONS = ('beq', 'bne', 'bcs', 'bhs', 'bcc', 'blo', 'bmi', 'bpl', 'bvs',
                                      'bvc', 'bhi', 'bls', 'bge', 'blt', 'bgt', 'ble', 'cbz', 'cbnz')
         for cs_insn in bb.capstone.insns:
-            if cs_insn.mnemonic in THUMB_BRANCH_INSTRUCTIONS:
+            if cs_insn.mnemonic.split('.')[0] in THUMB_BRANCH_INSTRUCTIONS:
                 can_produce_exits.add(cs_insn.address)
 
         successors_filtered = filter(lambda suc: get_ins_addr(suc) in can_produce_exits or


Test patch:
Repository: angr/angr

Issue:
Unsupported CCall armg_calculate_flags_nzcv
When running angr symbolic execution on 32-bit ARM Linux kernel binary, this error is very common:

For example, when doing:

```
proj.factory.sim_block(pg.active[i].state)
```

or 

```
pathgroup.step()
```

The error often occurs:

```
simuvex.vex.expressions.ccall | Unsupported CCall armg_calculate_flags_nzcv
```

or

```
angr/local/lib/python2.7/site-packages/simuvex/vex/expressions/ccall.py", line 39, in _execute

raise UnsupportedCCallError("Unsupported CCall %s" % self._expr.callee.name)

simuvex.s_errors.UnsupportedCCallError: Unsupported CCall armg_calculate_flags_nzcv
```



Discussion:
This ccall is not implemented. It'll be great if someone can implement it and give us a PR :-)

The whole ccall implementation is based on related code in VEX.

In case anyone else comes looking before this gets fixed, you can hack around this if necessary by adding `simuvex.o.BYPASS_UNSUPPORTED_IRCCALL` to the state options.


Resolving patch:
diff --git a/angr/engines/vex/ccall.py b/angr/engines/vex/ccall.py
--- a/angr/engines/vex/ccall.py
+++ b/angr/engines/vex/ccall.py
@@ -1375,7 +1375,7 @@ def armg_calculate_flag_v(state, cc_op, cc_dep1, cc_dep2, cc_dep3):
     l.error("Unknown cc_op %s (armg_calculate_flag_v)", cc_op)
     raise SimCCallError("Unknown cc_op %s" % cc_op)
 
-def armg_calculate_data_nzcv(state, cc_op, cc_dep1, cc_dep2, cc_dep3):
+def armg_calculate_flags_nzcv(state, cc_op, cc_dep1, cc_dep2, cc_dep3):
     # NOTE: adding constraints afterwards works here *only* because the constraints are actually useless, because we require
     # cc_op to be unique. If we didn't, we'd need to pass the constraints into any functions called after the constraints were
     # created.
@@ -1684,7 +1684,7 @@ def _get_flags(state):
     elif state.arch.name == 'AMD64':
         return amd64g_calculate_rflags_all(state, state.regs.cc_op, state.regs.cc_dep1, state.regs.cc_dep2, state.regs.cc_ndep)
     elif state.arch.name in ('ARMEL', 'ARMHF', 'ARM'):
-        return armg_calculate_data_nzcv(state, state.regs.cc_op, state.regs.cc_dep1, state.regs.cc_dep2, state.regs.cc_ndep)
+        return armg_calculate_flags_nzcv(state, state.regs.cc_op, state.regs.cc_dep1, state.regs.cc_dep2, state.regs.cc_ndep)
     elif state.arch.name == 'AARCH64':
         return arm64g_calculate_data_nzcv(state, state.regs.cc_op, state.regs.cc_dep1, state.regs.cc_dep2, state.regs.cc_ndep)
     else:


Test patch:
Repository: angr/angr

Issue:
CFG-Support for MachO
Hello, creating a dummy issue for missing CFG-Support for mach-o

```python
cfg_base:633++ 
        for b in binaries:
            if isinstance(b, ELF):
                # If we have sections, we get result from sections
                if not force_segment and b.sections:
                    # Get all executable sections
                    for section in b.sections:
                        if section.is_executable:
                            tpl = (section.min_addr, section.max_addr)
                            memory_regions.append(tpl)
                            ....
```
missing the appropriate handling for macho

```python
from cle import ELF, PE, MachO, Blob, TLSObject, ExternObject, KernelObject

if isinstance(b, MachO):
    # If we have sections, we get result from sections
    if not force_segment and b.segments:
        # Get all executable sections
        for seg in b.segments:
            if seg.is_executable:
                tpl = (seg.min_addr, seg.max_addr)
                memory_regions.append(tpl)
```

Pull-Request incoming


Discussion:

Resolving patch:
diff --git a/angr/analyses/cfg/cfg_base.py b/angr/analyses/cfg/cfg_base.py
--- a/angr/analyses/cfg/cfg_base.py
+++ b/angr/analyses/cfg/cfg_base.py
@@ -8,7 +8,7 @@
 
 import pyvex
 from claripy.utils.orderedset import OrderedSet
-from cle import ELF, PE, Blob, TLSObject, ExternObject, KernelObject
+from cle import ELF, PE, Blob, TLSObject, MachO, ExternObject, KernelObject
 
 from ...misc.ux import deprecated
 from ... import SIM_PROCEDURES
@@ -653,6 +653,16 @@ def _executable_memory_regions(self, binary=None, force_segment=False):
                         tpl = (section.min_addr, section.max_addr)
                         memory_regions.append(tpl)
 
+            elif isinstance(b, MachO):
+                if b.segments:
+                    # Get all executable segments
+                    for seg in b.segments:
+                        if seg.is_executable:
+                            # Take all sections from this segment (MachO style)
+                            for section in seg.sections:
+                                tpl = (section.min_addr, section.max_addr)
+                                memory_regions.append(tpl)
+
             elif isinstance(b, Blob):
                 # a blob is entirely executable
                 tpl = (b.min_addr, b.max_addr)


Test patch:
Repository: angr/angr

Issue:
DOWNSIZE_Z3 is not correctly added to options
This line doesn't include options which have numbers. https://github.com/angr/angr/blob/bcce17826b8b57b1fc924bc83eecb7c8e9ab0918/angr/sim_options.py#L293


Discussion:

Resolving patch:
diff --git a/angr/sim_options.py b/angr/sim_options.py
--- a/angr/sim_options.py
+++ b/angr/sim_options.py
@@ -290,7 +290,7 @@
 
 _g = globals().copy()
 for k, v in _g.items():
-    if all([ char in string.uppercase + "_" for char in k ]) and type(v) is str:
+    if all([ char in string.uppercase + "_" + string.digits for char in k ]) and type(v) is str:
         if k in ("UNKNOWN_FILES_HAVE_EOF", "CGC_ZERO_FILL_UNCONSTRAINED_MEMORY"):
             # UNKNOWN_FILES_HAVE_EOF == FILES_HAVE_EOF
             # CGC_ZERO_FILL_UNCONSTRAINED_MEMORY == ZERO_FILL_UNCONSTRAINED_MEMORY


Test patch:
Repository: angr/angr

Issue:
Difference between Dominator and Dominating Set
Code in https://github.com/angr/angr/blob/master/angr/analyses/veritesting.py#L578 may confuse the concepts of dominator and dominating set. To my best knowledge, the function which is needed here is `networkx.immediate_dominators()`


Discussion:
Confirmed that this is a bug. Thanks! Do you want to give us a PR, or should I do it?
just a one line fix, you please

Resolving patch:
diff --git a/angr/analyses/veritesting.py b/angr/analyses/veritesting.py
--- a/angr/analyses/veritesting.py
+++ b/angr/analyses/veritesting.py
@@ -575,7 +575,7 @@ def _post_dominate(reversed_graph, n1, n2):
         :returns bool:                           True/False.
         """
 
-        ds = networkx.dominating_set(reversed_graph, n1)
+        ds = networkx.immediate_dominators(reversed_graph, n1)
         return n2 in ds
 
     def _get_all_merge_points(self, cfg, graph_with_loops):


Test patch:
Repository: angr/angr

Issue:
Update reaching_definitions.py
Currently, the OP_BEFORE functionality in reaching definitions doesn't work, because it depends on the statement reaching observe function to be of type IMark, However, it's always filtered by light\engine.py


Discussion:
OMG... thanks for pointing this out.

This is not a right fix though. The correct fix should be removing the IMark filtering in `light/engine.py`.

Resolving patch:
diff --git a/angr/engines/light/engine.py b/angr/engines/light/engine.py
--- a/angr/engines/light/engine.py
+++ b/angr/engines/light/engine.py
@@ -67,8 +67,9 @@ def _process_Stmt(self):
             self.stmt_idx = stmt_idx
 
             if type(stmt) is pyvex.IRStmt.IMark:
+                # Note that we cannot skip IMarks as they are used later to trigger observation events
+                # The bug caused by skipping IMarks is reported at https://github.com/angr/angr/pull/1150
                 self.ins_addr = stmt.addr + stmt.delta
-                continue
 
             self._handle_Stmt(stmt)
 


Test patch: