eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: First of all I want to mention that I know this is a horrible idea and it shouldn't be done. My intention is mainly curiosity and learning the innards of Python, and how to 'hack' them. I was wondering whether it is at all possible to change what happens when we, for instance, use [] to create a list. Is there a way to modify how the parser behaves in order to, for instance, cause ["hello world"] to call print("hello world") instead of creating a list with one element? I've attempted to find any documentation or posts about this but failed to do so. Below is an example of replacing the built-in dict to instead use a custom class: from __future__ import annotations from typing import List, Any import builtins class Dict(dict): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.__dict__ = self def subset(self, keys: List[Any]) -> Dict: return Dict({key: self[key] for key in keys}) builtins.dict = Dict  When this module is imported, it replaces the dict built-in with the Dict class. However this only works when we directly call dict(). If we attempt to use {} it will fall back to the base dict built-in implementation: import new_dict a = dict({'a': 5, 'b': 8}) b = {'a': 5, 'b': 8} print(type(a)) print(type(b))  Yields: <class 'py_extensions.new_dict.Dict'> <class 'dict'> 

Answer: [] and {} are compiled to specific opcodes that specifically return a list or a dict, respectively. On the other hand list() and dict() compile to bytecodes that search global variables for list and dict and then call them as functions: import dis dis.dis(lambda:[]) dis.dis(lambda:{}) dis.dis(lambda:list()) dis.dis(lambda:dict())  returns (with some additional newlines for clarity):  3 0 BUILD_LIST 0 2 RETURN_VALUE 5 0 BUILD_MAP 0 2 RETURN_VALUE 7 0 LOAD_GLOBAL 0 (list) 2 CALL_FUNCTION 0 4 RETURN_VALUE 9 0 LOAD_GLOBAL 0 (dict) 2 CALL_FUNCTION 0 4 RETURN_VALUE  Thus you can overwrite what dict() returns simply by overwriting the global dict, but you can't overwrite what {} returns. These opcodes are documented here. If the BUILD_MAP opcode runs, you get a dict, no way around it. As an example, here is the implementation of BUILD_MAP in CPython, which calls the function _PyDict_FromItems. It doesn't look at any kind of user-defined classes, it specifically makes a C struct that represents a python dict. It is possible in at least some cases to manipulate the python bytecode at runtime. If you really wanted to make {} return a custom class, I suppose you could write some code to search for the BUILD_MAP opcode and replace it with the appropriate opcodes. Though those opcodes aren't the same size, so there's probably quite a few additional changes you'd have to make.
Question: I have a python script that loads a csv file using pandas, and then uses pgmpy to learn a bayesian network over the data. After learning the structure, I am drawing the graph using the function: nx.draw(graph_model, node_color='#00b4d9', with_labels=True)  This works perfectly in Ubuntu, However, it is throwing a StopIteration error in a virtual machine running Mac that I use to compile a Mac version. The error it is throwing is the following (I've removed the paths because it contains the name of the project and this is unpublished work): StopIteration: At: <path>/site-packages/matplotlib/bezier.py(352): split_path_inout <path>/site-packages/matplotlib/patches.py(2754): _shrink <path>/site-packages/matplotlib/patches.py(2771): _call_ <path>/site-packages/networkx/drawing/nx_pylab.py(794): _connectionstyle <path>/site-packages/matplotlib/patches.py(4453): _get_path_in_displaycoord <path>/site-packages/matplotlib/patches.py(4440): get_path <path>/site-packages/matplotlib/axes/_base.py(2376): _update_patch_limits <path>/site-packages/matplotlib/axes/_base.py(2358): add_patch <path>/site-packages/networkx/drawing/nx_pylab.py(867): _draw_networkx_edges_fancy_arrow_patch <path>/site-packages/networkx/drawing/nx_pylab.py(889): draw_networkx_edges <path>/site-packages/networkx/drawing/nx_pylab.py(334): draw_networkx <path>/site-packages/networkx/drawing/nx_pylab.py(120): draw <path>/bayesian_network/draw_model.py(7): <module>  I have checked that the learned graph has nodes and edges. If I try to draw a graph with only one node, it works. I have already upgraded all of my packages, including pgmpy, matplotlib and networkx. Could this problem be related to the code being executed in a virtual machine running Mac? I currently have no access to a real Mac machine to test it.

Answer: I finally solved it by adding the position as a circular layout. Looks like in the previous version, it automatically did this, but in the new version that was installed in the virtual machine don't. pos = nx.circular_layout(graph_model) nx.draw(graph_model, node_color='#00b4d9', pos=pos, with_labels=True)
Question: I'd like to construct an object that works like a random number generator, but generates numbers in a specified sequence. # a random number generator rng = lambda : np.random.randint(2,20)//2 # a non-random number generator def nrng(): numbers = np.arange(1,10.5,0.5) for i in range(len(numbers)): yield numbers[i] for j in range(10): print('random number', rng()) print('non-random number', nrng())  The issue with the code above that I cannot call nrng in the last line because it is a generator. I know that the most straightforward way to rewrite the code above is to simply loop over the non-random numbers instead of defining the generator. I would prefer getting the example above to work because I am working with a large chunk of code that include a function that accepts a random number generator as an argument, and I would like to add the functionality to pass non-random number sequences without rewriting the entire code. EDIT: I see some confusion in the comments. I am aware that python's random number generators generate pseudo-random numbers. This post is about replacing a pseudo-random-number generator by a number generator that generates numbers from a non-random, user-specified sequence (e.g., a generator that generates the number sequence 1,1,2,2,1,0,1 if I want it to).

Answer: Edit: The cleanest way to do this would be to use a lambda to wrap your call to next(nrng) as per great comment from @GACy20: def nrng_gen(): yield from range(10) nrng = nrng_gen() nrng_func = lambda: next(nrng) for i in range(10): print(nrng_func())  Original answer: If you want your object to keep state and look like a function, create a custom class with __call__ method. eg. class NRNG: def __init__(self): self.numbers = range(10) self.state = -1 def __call__(self): self.state += 1 return self.numbers[self.state] nrng = NRNG() for i in range(10): print(nrng())  However, I wouldn't recommend this unless absolutely necessary, as it obscures the fact that your nrng keeps a state (although technically, most rngs keep their state internally). It's best to just use a regular generator with yield by calling next on it or to write a custom iterator (also class-based). Those will work with things like for loops and other python tools for iteration (like the excellent itertools package).
Question: Today I by mistake inputted just a comma on an interactive session Input: ,  and I noticed strangely that it did not return an error but instead: Output ''  So I explored a bit this behaviour and tried some random stuff, and it seems like it creates tuples of strings, but it seems like these objects cannot be interacted with: , 'foo' bar 1 x  returns: ("'foo'", 'bar', '1', 'x')  Trying to assign those tuples or making some == checks doesn't really work but return errors. I couldn't find any answer or documentation about this behaviour. Anyone know what's happening here? EDIT: I am using Python 3.9.8 and running in VSCode interactive window with IPython. As someone pointed out in the comments this is not the behaviour when running from the terminal

Answer: This is an input transformation performed by the EscapedCommand class, specifically here. It's not part of autocall (details see below) which is handled by prefilter.AutoHandler. I couldn't find any public documentation on "escaped commands" and the class' docstring just mentions that it is a "transformer for escaped commands like %foo, !foo, or /foo". So I get the impression that the transformation for input like , a b is an (unintended) side effect of some other feature, as it's not publicly documented and doesn't seem to be of any use. We can request the current IPython shell by importing the corresponding module and then check what component modifies the input: In [1]: import IPython In [2]: shell = IPython.get_ipython() In [3]: %autocall Automatic calling is: Smart In [4]: shell.prefilter(',f a b') # autocall (note the function name 'f'), not applied since there is no callable `f` in the global namespace Out[4]: ',f a b' In [5]: f = lambda x,y: x+y In [6]: shell.prefilter(',f a b') # autocall (note the function name 'f'), now it works ------> f("a", "b") Out[6]: 'f("a", "b")' In [7]: shell.prefilter(', a b') # not identified as autocall --> remains unchanged Out[7]: ', a b' In [8]: shell.transform_cell(', a b') # however, it gets transformed by `EscapedCommand` Out[8]: '("a", "b")\n'  For autocall to work, we first have to activate it via the "magic" command %autocall. Also the indicated function name (f) must be present in the namespace and be callable. %quickref provides a brief overview of the autocall feature (scroll down to "Autocall"): Autocall: f 1,2 : f(1,2) # Off by default, enable with %autocall magic. /f 1,2 : f(1,2) (forced autoparen) ,f 1 2 : f("1","2") ;f 1 2 : f("1 2")
Question: New to python and pydantic, I come from a typescript background. I was wondering if you can inherit a generic class? In typescript the code would be as follows interface GenericInterface<T> { value: T } interface ExtendsGeneric<T> extends GenericInterface<T> { // inherit value from GenericInterface otherValue: string } const thing: ExtendsGeneric<Number> = { value: 1, otherValue: 'string' }  What I have been trying is something along the lines of #python3.9 from pydantic.generics import GenericModel from typing import TypeVar from typing import Generic T = TypeVar("T", int, str) class GenericField(GenericModel, Generic[T]): value: T class ExtendsGenericField(GenericField[T]): otherValue: str ExtendsGenericField[int](value=1, otherValue="other value")  And I get the error of TypeError: Too many parameters for ExtendsGenericField; actual 1, expected 0. This sort of checks out because in the Pydantic docs it explicitly states "In order to declare a generic model...Use the TypeVar instances as annotations where you will want to replace them..." The easy workaround is to make ExtendsGeneric inherit from GenericModel and have value in its own class definition, but I was trying to reuse classes. Is inheriting a value from a generic class possible?

Answer: Generics are a little weird in Python, and the problem is that ExtendsGenericField itself isn't declared as generic. To solve, just add Generic[T] as a super class of ExtendsGenericField: from pydantic.generics import GenericModel from typing import TypeVar from typing import Generic T = TypeVar("T", int, str) class GenericField(GenericModel, Generic[T]): value: T class ExtendsGenericField(GenericField[T], Generic[T]): otherValue: str ExtendsGenericField[int](value=1, otherValue="other value")
Question: I was looking through source and noticed that it references a variable environ in methods before its defined: def _createenviron(): if name == 'nt': # Where Env Var Names Must Be UPPERCASE def check_str(value): if not isinstance(value, str): raise TypeError("str expected, not %s" % type(value).__name__) return value encode = check_str decode = str def encodekey(key): return encode(key).upper() data = {} for key, value in environ.items(): data[encodekey(key)] = value else: # Where Env Var Names Can Be Mixed Case encoding = sys.getfilesystemencoding() def encode(value): if not isinstance(value, str): raise TypeError("str expected, not %s" % type(value).__name__) return value.encode(encoding, 'surrogateescape') def decode(value): return value.decode(encoding, 'surrogateescape') encodekey = encode data = environ return _Environ(data, encodekey, decode, encode, decode) # unicode environ environ = _createenviron() del _createenviron  So how does environ get setup? I cant seem to reason about where its initialized and declared so that _createenviron can use it?

Answer: TLDR search for from posix import * in os module content. The os module imports all public symbols from posix (Unix) or nt (Windows) low-level module at the beginning of os.py. posix exposes environ as a plain Python dict. os wraps it with _Environ dict-like object that updates environment variables on _Environ items changing.
Question: Here are two measurements: timeit.timeit('"toto"=="1234"', number=100000000) 1.8320042459999968 timeit.timeit('"toto"=="toto"', number=100000000) 1.4517491540000265  As you can see, comparing two strings that match is faster than comparing two strings with the same size that do not match. This is quite disturbing: During a string comparison, I believed that Python was testing strings character by character, so "toto"=="toto" should be longer to test than "toto"=="1234" as it requires four tests against one for the non-matching comparison. Maybe the comparison is hash-based, but in this case, timings should be the same for both comparisons. Why?

Answer: Combining my comment and the comment by @khelwood: TL;DR: When analysing the bytecode for the two comparisons, it reveals the 'time' and 'time' strings are assigned to the same object. Therefore, an up-front identity check (at C-level) is the reason for the increased comparison speed. The reason for the same object assignment is that, as an implementation detail, CPython interns strings which contain only 'name characters' (i.e. alpha and underscore characters). This enables the object's identity check.  Bytecode: import dis In [24]: dis.dis("'time'=='time'") 1 0 LOAD_CONST 0 ('time') # <-- same object (0) 2 LOAD_CONST 0 ('time') # <-- same object (0) 4 COMPARE_OP 2 (==) 6 RETURN_VALUE In [25]: dis.dis("'time'=='1234'") 1 0 LOAD_CONST 0 ('time') # <-- different object (0) 2 LOAD_CONST 1 ('1234') # <-- different object (1) 4 COMPARE_OP 2 (==) 6 RETURN_VALUE   Assignment Timing: The 'speed-up' can also be seen in using assignment for the time tests. The assignment (and compare) of two variables to the same string, is faster than the assignment (and compare) of two variables to different strings. Further supporting the hypothesis the underlying logic is performing an object comparison. This is confirmed in the next section. In [26]: timeit.timeit("x='time'; y='time'; x==y", number=1000000) Out[26]: 0.0745926329982467 In [27]: timeit.timeit("x='time'; y='1234'; x==y", number=1000000) Out[27]: 0.10328884399496019   Python source code: As helpfully provided by @mkrieger1 and @Masklinn in their comments, the source code for unicodeobject.c performs a pointer comparison first and if True, returns immediately. int _PyUnicode_Equal(PyObject *str1, PyObject *str2) { assert(PyUnicode_CheckExact(str1)); assert(PyUnicode_CheckExact(str2)); if (str1 == str2) { // <-- Here return 1; } if (PyUnicode_READY(str1) || PyUnicode_READY(str2)) { return -1; } return unicode_compare_eq(str1, str2); }   Appendix:  Reference answer nicely illustrating how to read the disassembled bytecode output. Courtesy of @Delgan Reference answer which nicely describes CPython's string interning. Coutresy of @ShadowRanger
Question: edit: I changed the title from complement to converse after the discussion below. In the operator module, the binary functions comparing objects take two parameters. But the contains function has them swapped. I use a list of operators, e.g. operator.lt, operator.ge. They take 2 arguments, a and b. I can say operator.lt(a, b) and it will tell me whether a is less than b. But with operator.contains, I want to know whether b contains a so I have to swap the arguments. This is a pain because I want a uniform interface, so I can have a user defined list of operations to use (I'm implementing something like Django QL). I know I could create a helper function which swaps the arguments: def is_contained_by(a, b): return operator.contains(b, a)  Is there a "standard" way to do it? Alternatively, I can implement everything backwards, except contains. So map lt to ge, etc, but that gets really confusing.

Answer: If either of them posts an answer, you should accept that, but between users @chepner and @khelwood, they gave you most of the answer. The complement of operator.contains would be something like operator.does_not_contain, so that's not what you're looking for exactly. Although I think a 'reflection' isn't quite what you're after either, since that would essentially be its inverse, if it were defined. At any rate, as @chepner points out, contains is not backwards. It just not the same as in, in would be is_contained_by as you defined it. Consider that a in b would not be a contains b, but rather b contains a, so the signature of operator.contains makes sense. It follows the convention of the function's stated infix operation being its name. I.e. (a < b) == operator.lt(a, b) and b contains a == operator.contains(b, a) == (a in b). (in a world where contains would be an existing infix operator) Although I wouldn't recommend it, because it may cause confusion with others reading your code and making the wrong assumptions, you could do something like: operator.in_ = lambda a, b: b.__contains__(a) # or operator.in_ = lambda a, b: operator.contains(b, a)  That would give you an operator.in_ that works as you expect (and avoids the in keyword), but at the cost of a little overhead and possible confusion. I'd recommend working with operator.contains instead.
Question: I have a customized built module, lets call it abc, and pip install /local_path/abc-0.1-py3-none-any.whl. Installation is correct, >>pip install dist/abc-0.1-py3-none-any.whl Processing ./dist/abc-0.1-py3-none-any.whl Successfully installed abc-0.1  but I could not import the module. After I ran ppip freeze list and found out the name of module in list is abc @ file:///local_path/abc-0.1-py3-none-any.whl.  my question is how could import the module? Thank you . ├── requirements.txt ├── setup.py ├── src │ ├── bin │ │ ├── __init__.py │ │ ├── xyz1.py │ │ ├── xyz2.py │ │ └── xyz3.py  here is my setup.py with open("requirements.txt") as f: install_requires = f.read() setup( name="abc", version="0.1", author="galaxyan", author_email="galaxyan@123.com", description="test whell framework", packages=find_packages(include=["src"]), zip_safe=False, install_requires=install_requires, )  ############ update ############ it does not work even change setup.py with open("requirements.txt") as f: install_requires = f.read() setup( name="abc", version="0.1", author="galaxyan", author_email="galaxyan@123.com", description="test whell framework", packages=find_packages(where="src"), package_dir={"": "src"}, zip_safe=False, install_requires=install_requires, ) 

Answer: The setup.py is wrong, which means you're building a wheel with no packages actually inside. Instead of setup( ... packages=find_packages(include=["src"]), ... )  Try this: setup( ... packages=find_packages(where="src"), package_dir={"": "src"}, ... )  See Testing & Packaging for more info.
Question: Apparently, Python 3.10 / 3.12 is going to deprecate / remove distutils (cpython/issues/92584). Unfortunately, I have not been able to find a replacement for the one and only function I am using from it; distutils.util.get_platform(). What is the replacement for this? Note that platform is NOT an answer. I need a function that returns the complete string that is used when building a binary wheel¹, e.g. macosx-12-x86_64. Note particularly that there appears to be platform-specific logic embedded in this (e.g. the only other way I know to get the macos version is with a macos-specific API). (¹ As noted in a comment, distutils.util.get_platform() is, strictly speaking, not that function. However, PEP 425 specifies that "the platform tag is simply distutils.util.get_platform() with all hyphens - and periods . replaced with underscore _." Ergo, it is straight-forward and platform-agnostic to derive the tag from distutils.util.get_platform(). An acceptable answer may therefore give an approved, public API which produces the platform tag directly, or a compatible replacement for distutils.util.get_platform().)

Answer: For your use-case, sysconfig has a replacement import sysconfig sysconfig.get_platform()  This is what the wheel project itself used as a replacement for distutils.util.get_platform() when removing distutils from the code in Replaced all uses of distutils with setuptools #428.
Question: While I am testing my API I recently started to get the error below.  if request is None: > builder = EnvironBuilder(*args, **kwargs) E TypeError: __init__() got an unexpected keyword argument 'as_tuple' /usr/local/lib/python3.7/site-packages/werkzeug/test.py:1081: TypeError  As I read from the documentation in the newer version of Werkzeug the as_tuple parameter is removed. Part of my test code is  from flask.testing import FlaskClient @pytest.fixture(name='test_client') def _test_client() -> FlaskClient: app = create_app() return app.test_client() class TestPeerscoutAPI: def test_should_have_access_for_status_page(self, test_client: FlaskClient): response = test_client.get('/api/status') assert _get_ok_json(response) == {"status": "OK"}  Any help would be greatly appreciated.

Answer: As of version 2.1.0, werkzeug has removed the as_tuple argument to Client. Since Flask wraps werkzeug and you're using a version that still passes this argument, it will fail. See the exact change on the GitHub PR here. You can take one of two paths to solve this:  Upgrade flask  Pin your werkzeug version   # in requirements.txt werkzeug==2.0.3
Question: I have the following structure for my project: project/ ├── backend │ ├── api_v1 │ ├── api_v2 │ └── api_v3 └── frontend  Each of the API dirs, api_v1, api_v2, and api_v3, have python files. I would like to run pre-commit for each of these directories only if there is a change in the code. For eg., I would like to run mypy -p api_v1 if there is a change in the directory api_v1. I'm aware of the keys files and types of the pre-commit, but I cannot figure out a way to run mypy as if it was running from the directory backend. Also, I cannot run mypy separately for api_v1, api_v2, or api_v3, when I have changes in more than 1 of these directories. Is it not possible or am

Answer: pre-commit operates on files so what you're trying to do isn't exactly supported but anything is possible. when not running on files you're going to take some efficiency concessions as you'll be linting much more often than you need to be here's a rough sketch for how you would do this: - repo: https://github.com/pre-commit/mirrors-mypy rev: ... hooks: - id: mypy pass_filenames: false # suppress the normal filename passing files: ^backend/api_v1/ # filter the files down to a specific subdirectory # pre-commit only supports running at the root of a repo since that's where # git hooks run. but it also allows running arbitrary code so you can # step outside of those bounds # note that `bash` will reduce your portability slightly entry: bash -c 'cd backend && mypy -p api_v1 "$@"' -- # and then repeat ... - id: mypy pass_filenames: false files: ^backend/api_v2/ entry: bash -c 'cd backend && mypy -p api_v2 "$@"' -- # etc.   disclaimer: I wrote pre-commit
Question: I have a 2x2 rotation matrix and several vectors stored in a Nx2 array. Is there a way to rotate them all (i.e. multiply them all by the rotation matrix) at once? I'm sure there is a numpy method for that, it's just not obvious. import numpy as np vectors = np.array( ( (1,1), (1,2), (2,2), (4,2) ) ) # 4 2D vectors ang = np.radians(30) m = np.array( ( (np.cos(ang), -np.sin(ang)), (np.sin(ang), np.cos(ang)) )) # 2x2 rotation matrix # rotate 1 vector: m.dot(vectors[0,:]) # rotate all vectors at once?? 

Answer: Because m has shape (2,2) and vectors has shape (4,2), you can simply do dots = vectors @ m.T  Then each row i contains the matrix-vector product m @ vectors[i, :].