eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Is it possible to drop terms with coefficients below a given, small number (say 1e-5) in a Sympy expression? I.e., such that 0.25 + 8.5*exp(-2.6*u) - 2.7e-17*exp(-2.4*u) + 1.2*exp(-0.1*u)  becomes 0.25 + 8.5*exp(-2.6*u) + 1.2*exp(-0.1*u)  for instance.

Answer: You can use a combination of coeffs, subs and Poly: u = Symbol('u') poly = Poly(0.25 + 8.5*exp(-2.6*u) - 2.7e-17*exp(-2.4*u) + 1.2*exp(-0.1*u)) threshold = 1e-5 to_remove = [abs(i) for i in poly.coeffs() if abs(i) < threshold] for i in to_remove: poly = poly.subs(i, 0)  poly no yields the following: 0.25 + 8.5*exp(-2.6*u) + 1.2*exp(-0.1*u)
Question: I have a dataframe that looks like this which contains all divisions and both conferences from 2000-2022. Tm Conference Division W-L%. Year Bills AFC East 0.813 2022 Dolphins AFC East 0.529 2022 Patriots AFC East 0.471 2022 Jets AFC East 0.412 2022 Cowboys NFC East 0.706 2022 Giants NFC East 0.559 2022 Eagles NFC East 0.824 2022 Commanders NFC East 0.500 2022  I want to groupby Team, Conference, and year, and create a new column called 'Division W-L%' which finds the average W-L% for every team in a particular division, conference, and year except for the team we are calculating it on. I know the formula to find the Division W-L% which is: df['Division_W-L%'] = (df['W-L%'].sum() - df['W-L%']) / (len(df) -1). This is what I want the dataframe to look like. For example, for the 'Bills' we would calculate the Division W-L% by doing (0.529 + 0.471 + 0.412)/3 since those 3 teams are in the same conference, division, and year. Tm Conference Division W-L%. Year Division W-L% Bills AFC East 0.813 2022 0.470667 Dolphins AFC East 0.529 2022 0.565333 Patriots AFC East 0.471 2022 0.584667 Jets AFC East 0.412 2022 0.604333 Cowboys NFC East 0.706 2022 0.627667 Giants NFC East 0.559 2022 0.676667 Eagles NFC East 0.824 2022 0.588333 Commanders NFC East 0.500 2022 0.696333  I tried doing what I described above, which is grouping by those three categories, and then applying that formula to the W-L% column, however I continue to receive errors. All help is appreciated!

Answer: You can use transform instead of apply. Compute the sum for the group and subtract the W-L%. of the current row then divide by the size of the group minus 1 (because you want to exclude the row itself): df['Division W-L%'] = (df.groupby(['Conference', 'Division', 'Year'])['W-L%.'] .transform(lambda x: (x.sum() - x) / (len(x) - 1)))  Output: >>> df Tm Conference Division W-L%. Year Division W-L% 0 Bills AFC East 0.813 2022 0.470667 1 Dolphins AFC East 0.529 2022 0.565333 2 Patriots AFC East 0.471 2022 0.584667 3 Jets AFC East 0.412 2022 0.604333 4 Cowboys NFC East 0.706 2022 0.627667 5 Giants NFC East 0.559 2022 0.676667 6 Eagles NFC East 0.824 2022 0.588333 7 Commanders NFC East 0.500 2022 0.696333
Question: I'm using a logging formatter to redact passwords. I want to write a test to confirm that the logging redactor is effective. In this example I simplified the code to redact "foo". With this redactor code in the my my_logger.py module (simplified redaction of a specific word): class RedactFoo: def __init__(self, base_formatter): self.base_formatter = base_formatter def format(self, record): msg = self.base_formatter.format(record) return msg.replace("foo", "<<REDACTED>>") def __getattr__(self, attr): return getattr(self.orig_formatter, attr)  Then I configure my logger and wrap the formatter for each handler: logging.config.dictConfig(logging_config) # Set the formatter on all handlers for h in logging.root.handlers: h.setFormatter(RedactFoo(h.formatter)) def get_logger(name): return logging.getLogger(name)  If I run: logger = my_logger.get_logger("Test") logger.error(f"This message has foo.")  the logger redacts the message. However, in Pytest test_my_logger.py: import my_logger def test_logging_redactor(caplog): logger = my_logger.get_logger("test") logger.error(f"This message has foo.") assert ""This message has <<REDACTED>>." in caplog.text  This test does not pass because the Pytest logging configuration overwrites my custom config. How can I use Pytest and the caplog fixture to perform this test? I have seen the unittest solution here: How to test specific logging formatting in Python (using `pytest` and `unittest`), but I'm interested in doing this with Pytest.

Answer: The caplog fixture works by injecting its own LogCaptureHandler to the logging framework configuration. That's how it is able to intercept log records and provide the events at caplog.text, caplog.records, caplog.record_tuples etc which the user can make assertions against. Note that it is capturing stdlib LogRecord instances, i.e. the log events have not been rendered yet and your custom format method has not been called. If you want the formatted text redacted during test as well, you'll have to apply it also to the test logging configuration i.e. after the caplog fixture has been entered: @pytest.fixture(autouse=True) def redact_caplog_handlers(caplog): caplog.handler.setFormatter(RedactFoo(caplog.handler.formatter))
Question: I have an issue with a 3d plot, at the moment of visualizing it. It appears without the z-axis label,  but when I set a longer title, it appears:  Is there any way to be able to "see" the z-axis label without modifying the title or another kind of solution to this issue? This is my code: mask1, mask2, mask3 shapes are (100,100) with values of 20, 32, 49 respectively fig = plt.figure(figsize=(12, 4)) ax = fig.add_subplot(projection='3d') x, y = np.meshgrid(np.arange(100)*20, np.arange(100)*20) s1 = ax.plot_surface(x, y, mask1*20, linewidth=0, label='Reflector 1') s1._facecolors2d = s1._facecolor3d s1._edgecolors2d = s1._edgecolor3d s2 = ax.plot_surface(x, y, mask2*20, linewidth=0, label='Reflector 2') s2._facecolors2d = s2._facecolor3d s2._edgecolors2d = s2._edgecolor3d s3 = ax.plot_surface(x, y, mask3*20, linewidth=0, label='Reflector 3') s3._facecolors2d = s3._facecolor3d s3._edgecolors2d = s3._edgecolor3d ax.set(xlim=[0, 2000], ylim=([0, 2000]), zlim=([1000, 0]), xlabel=(r' $x [m]$'), ylabel=(r'$y [m]$'), zlabel=(r'$z [m]$')) ax.legend() ax.set_title('a short title', fontsize=18) plt.show() 

Answer: You can zoom out a bit to make the label visible. E.g.: import matplotlib.pyplot as plt import numpy as np fig = plt.figure(figsize=(12, 4)) ax = fig.add_subplot(projection='3d') x, y = np.meshgrid(np.arange(100)*20, np.arange(100)*20) mask = np.ones(x.shape) s1 = ax.plot_surface(x, y, mask*400, linewidth=0, label='Reflector 1') s1._facecolors2d = s1._facecolor3d s1._edgecolors2d = s1._edgecolor3d s2 = ax.plot_surface(x, y, mask*700, linewidth=0, label='Reflector 2') s2._facecolors2d = s2._facecolor3d s2._edgecolors2d = s2._edgecolor3d s3 = ax.plot_surface(x, y, mask*1000, linewidth=0, label='Reflector 3') s3._facecolors2d = s3._facecolor3d s3._edgecolors2d = s3._edgecolor3d ax.set(xlim=[0, 2000], ylim=([0, 2000]), zlim=([1000, 0]), xlabel=(r' $x [m]$'), ylabel=(r'$y [m]$'), zlabel=(r'$z [m]$')) ax.legend() ax.set_title('a short title', fontsize=18) ax.set_box_aspect(aspect=None, zoom=0.8) plt.show()  Which creates this plot:
Question: I am training a neural network. Somewhere in my code base, I have a code snippet like the following: def foo(): d = {} with PIL.Image.open(img_path) as img: d["img"] = torchvision.transforms.functional.to_tensor(img) return d  This code doesn't cause any problems. However, when I run my program under strace, I see that there is a double call of the close syscall (below, I omit everything except relevant strace lines): openat(AT_FDCWD, "example.tif", O_RDONLY|O_CLOEXEC) = 3</tmp/example.tif> fstat(3</tmp/example.tif>, {st_mode=S_IFREG|0644, st_size=9274, ...}) = 0 lseek(3</tmp/example.tif>, 0, SEEK_CUR) = 0 lseek(3</tmp/example.tif>, 0, SEEK_SET) = 0 read(3</tmp/example.tif>, "II*\0\250#\0\0\200\0 P8$\26\r\7\204BaP\270d6\35\17\210DbQ8\244"..., 4096) = 4096 # here some more lseeks and reads are omitted mmap(NULL, 1765376, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7feb10bbd000 fcntl(3</tmp/example.tif>, F_DUPFD_CLOEXEC, 0) = 4</tmp/example.tif> lseek(4</tmp/example.tif>, 0, SEEK_SET) = 0 read(4</tmp/example.tif>, "II*\0\250#\0\0", 8) = 8 fstat(4</tmp/example.tif>, {st_mode=S_IFREG|0644, st_size=9274, ...}) = 0 mmap(NULL, 9274, PROT_READ, MAP_SHARED, 4</tmp/example.tif>, 0) = 0x7feb9e7a4000 munmap(0x7feb9e7a4000, 9274) = 0 close(4</tmp/example.tif>) = 0 close(4) = -1 EBADF (Bad file descriptor) close(3</tmp/example.tif>) = 0  I am worried about close(4) = -1 EBADF (Bad file descriptor). We can see that there is a duplicate call to the close syscall for some reason. The code works fine though. But still, does this necessarily mean that I have a bug in my code?  Upd: if anyone is interested, I have compiled a minimal self-contained example and reported the bug at https://github.com/python-pillow/Pillow/issues/7042

Answer: I would try to find out what causes this. In this particular example it seems OK, but there is room for very subtle errors. Imagine that you have a second thread, which open()s a file between the first and second close(4). It is likely, that this file descriptor get fd 4 assigned, and therefore will be closed immediately. This kind of error is a nightmare to debug.
Question: I understand that with openpyxl>=3.1 function save_virtual_workbook is gone and preferred solution is using NamedTemporaryFile. I want it to be done without writing to filesystem, only in memory, just like BytesIO did. Before update I had: wb = Workbook() populate_workbook(wb) return BytesIO(save_virtual_workbook(wb))  Now I need: wb = Workbook() populate_workbook(wb) tmpfile = NamedTemporaryFile() wb.save(tmpfile.name) return tmpfile  As in https://openpyxl.readthedocs.io/en/3.1/tutorial.html?highlight=save#saving-as-a-stream As I'm using django probably I could use InMemoryUploadedFile but I don't want to. How can I use BytesIO or something similar in elegant fashion ie. without hacks and additional packages?

Answer: You're so close, dude. Just pass a io.BytesIO buffer to the wb.save method. import openpyxl, io # Create an in-memory bytes buffer, which is a file-like object buffer = io.BytesIO() # Create a workbook, do the shit you gotta do wb = openpyxl.Workbook() # Save the workbook to the buffer, as if it were a file on disk opened for writing bytes wb.save(buffer) # Get all the bytes in the buffer. # It's functionally the same as buffer.seek(0) then buffer.read() buffer.getvalue()
Question: I cannot seem to figure out how to acquire an access token using MSAL. I’ve spend time reading the source code and Microsoft documentation to no avail. I’d like to use the PublicClientApplication to acquire the token. Even when running proof of concepts with the QuickStarts using ConfidentialClientApplication I seem to only get an ID_Token not an access token. Ultimately I’m trying to build a desktop/mobile app and want to be able to use MSAL for id token claims and access tokens for access to APIs. Also worth mentioning I want to implement this using AAD B2C Thanks!

Answer: To get your access use the acquire_token_silent or acquire_token_interactive methods of the PublicClientApplication class. Below is an example, replace the needed variables with your own. import msal app = msal.PublicClientApplication( "your_client_id", authority="https://yourtenant.b2clogin.com/yourtenant.onmicrosoft.com/B2C_1_signup_signin_policy", client_credential=None ) scopes = ["https://yourtenant.onmicrosoft.com/api/read"] result = app.acquire_token_interactive(scopes=scopes) print(result["access_token"])
Question: I am looking at a solution to an Advent of Code puzzle that stores coordinates as complex numbers:  heightmap = { complex(x, y): c for y, ln in enumerate(sys.stdin.read().strip().split("\n")) for x, c in enumerate(ln) }  Then accesses them later as follows: for xy, c in heightmap.items(): for d in (1, -1, 1j, -1j): if ord(heightmap.get(xy + d, "{")) <= ord(c) + 1: G.add_edge(xy, xy + d)  I can see that this code makes the 'get neighbors' line easy to write/think about, but I don't see that it is worth the added complexity (no pun intended). Can someone explain why it's useful to store the grid coordinates as complex numbers?

Answer: Yes, because it's easy/less to write and think about. Also means less opportunity for typos :-) I've been doing that for years, ever since I saw someone else do that. Usually not even typing the deltas explicitly but calculating them. I.e., instead of for d in (1, -1, 1j, -1j): use(z + d)  do: for i in range(4): use(z + 1j**i)  Possible alternatives when using separate x and y variables: for dx, dy in ((1, 0), (0, 1), (-1, 0), (0, -1)): use(x+dx, y+dy)  for x2, y2 in ((x+1, y), (x, y+1), (x-1, y), (x, y-1)): use(x2, y2)  Ermahgerd, so frustrating. I actually did make several typos while writing these :-) (Tests at Attempt This Online!)
Question: Consider the following example pd.set_option('display.width', 50) pl.DataFrame(data = np.random.randint(0,20, size = (10, 42)), columns = list('abcdefghijklmnopqrstuvwxyz123456789ABCDEFG')).to_pandas()   You can see how nicely the columns are formatted, breaking a line after column k so that the full dataframe is printed in chunks in the console. This was controlled by the pandas width argument above. I was not able to reproduce this behavior using polars and all the format options. I have tried tweaking all possible settings: pl.Config.set_tbl_cols(10) pl.Config.set_fmt_str_lengths(10) pl.Config.set_tbl_width_chars(70) pl.Config.set_tbl_rows(2) pl.Config.set_tbl_formatting('NOTHING') pl.Config.set_tbl_column_data_type_inline(True) pl.Config.set_tbl_dataframe_shape_below(True)  See below:  Any ideas? Thanks!

Answer: You can display all frame columns like so... with pl.Config() as cfg: cfg.set_tbl_cols(-1) print(df)  ...which will give you a good result on the given frame if you have sufficient terminal/console/output width available. If this isn't enough, I recommend making a feature request for this on the polars GitHub repository
Question: I have just installed pyswip and I was testing if it is working correctly but I always get this error: Assertion failed: 0, file /home/swipl/src/swipl-devel/src/pl-fli.c, line 2637 I was using this example code: from pyswip import Prolog prolog = Prolog() prolog.assertz("father(michael,john)")  I am using windows 10 python 3.11.1 and I have swipl in the path. I tested in WSL ubuntu and get the same error. Knowing why this error happen and solve it.

Answer: I faced a similar issue when getting a neurosymbolic model called DeepProbLog to run on my Windows. It'd be helpful to know which version of swipl and pyswip you have. I personally use  swipl version 8.4.2 for Windows 64 bits (https://www.swi-prolog.org/download/stable?show=all) \  fork the pyswip provided by the repo in DeepProbLog pip install git+https://github.com/ML-KULeuven/pyswip   If you want to use WSL Ubuntu, you can keep reading DeepProbLog's github page as they also provide commands on working with Prolog in Linux.
Question: In my python code, I import func... from sqlalchemy.sql.expression import func  Then, during my code I select data from a database table... select(func.max(MyTable.my_datetime))  ...where my_datetime is a DateTime data type... from sqlalchemy.types import DateTime my_datetime = Column('my_datetime', DateTime)  The code runs OK, but in the vscode editor I am getting the following error... func.max is not callable Pylint(E1102:not-callable) I don't want to ignore this if there is a genuine concern behind this Pylint error. Should I be concerned by this error or can I safely ignore it?

Answer: The Pylint error you have (func.max is not callable Pylint(E1102:not-callable)) is a false positive and you can ignore it in your case. Pylint is flagging func.max not callable because it can't analyze the func object statically and determine that it has a max method. You can use func: Callable  after importing .
Question: This happens whenever I ran any alembic command. I am using sqlalchemy version 2.0.3 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/__init__.py", line 8, in <module> from . import op # noqa File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/op.py", line 1, in <module> from .operations.base import Operations File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/operations/__init__.py", line 1, in <module> from .base import Operations, BatchOperations File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/operations/base.py", line 3, in <module> from .. import util File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/util/__init__.py", line 9, in <module> from .sqla_compat import ( # noqa File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/util/sqla_compat.py", line 8, in <module> from sqlalchemy.sql.expression import _BindParamClause ImportError: cannot import name '_BindParamClause' from 'sqlalchemy.sql.expression' (/home/***/anaconda3/lib/python3.8/site-packages/sqlalchemy/sql/expression.py) 

Answer: Solved after uninstalling alembic and reinstalling it afresh I ran: pip3 uninstall alembic pip3 install alembic
Question: The two following blocks of code should be equivalent: Block 1 array_1 = [[12, 15], [10, 1], [5, 13]] print(array_1) """ output: [[12, 15], [10, 1], [5, 13]] """ print(sorted(array_1)) """ output: [[5, 13], [10, 1], [12, 15]] """   Block 2 import numpy as np np_array_1 = np.array([[12, 15], [10, 1], [5, 13]]) print(np_array_1) """ output: [[12 15] [10 1] [ 5 13]] """ array_1 = np_array_1.tolist() print(array_1) """ output: [[12, 15], [10, 1], [5, 13]] """ print(sorted(array_1)) """ output: [[5, 13], [10, 1], [12, 15]] """   But, in PyCharm, if I use: sorted(np_array_1.tolist()) # SEE BLOCK 2  I receive this only warning, but apparently without any issue: Expected type 'Iterable[SupportsLessThan | Any]' (matched generic type 'Iterable[SupportsLessThanT]'), got 'object' instead  I would like to get clean code, thank you.

Answer: This is a bug in PyCharm. Please see: https://youtrack.jetbrains.com/issue/PY-45958/Type-error-sorting-Iterable-of-dataclassorderTrue-instances
Question: import sqlite3 sqlite3.Cursor.__base__ ## output: <class 'object'>  however, clearly, it inherited from Iterator, can see from the source code class Cursor(Iterator[Any]):  and  issubclass(sqlite3.Cursor, Iterator) # previous edit was saying iterable, sorry ## output: True  so, why does Python not properly display the parent class info? how to retrieve it properly?

Answer: The cursor object is defined in C as far as I can tell, so it doesn't have a parent class. As to why issubclass returns that, issubclass does essentially a "duck typing check" in some cases, as hinted at in the ABC docs:  Checking isinstance(obj, Iterable) detects classes that are registered as Iterable or that have an __iter__() method. . .  It's talking about isinstance there, but issubclass works the same. You can see this with a simple test: import typing class T: # Not an explicit subclass of "Iterable" def __iter__(self): pass print(issubclass(T, Iterable)) # True
Question: I am trying to understand threading in Python via this website. Here, it has the following code for a single producer/consumer-type problem: import random SENTINEL = object() def producer(pipeline): """Pretend we're getting a message from the network.""" for index in range(10): message = random.randint(1, 101) logging.info("Producer got message: %s", message) pipeline.set_message(message, "Producer") # Send a sentinel message to tell consumer we're done pipeline.set_message(SENTINEL, "Producer") def consumer(pipeline): """Pretend we're saving a number in the database.""" message = 0 while message is not SENTINEL: message = pipeline.get_message("Consumer") if message is not SENTINEL: logging.info("Consumer storing message: %s", message) if __name__ == "__main__": format = "%(asctime)s: %(message)s" logging.basicConfig(format=format, level=logging.INFO, datefmt="%H:%M:%S") # logging.getLogger().setLevel(logging.DEBUG) pipeline = Pipeline() with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor: executor.submit(producer, pipeline) executor.submit(consumer, pipeline) class Pipeline: """ Class to allow a single element pipeline between producer and consumer. """ def __init__(self): self.message = 0 self.producer_lock = threading.Lock() self.consumer_lock = threading.Lock() self.consumer_lock.acquire() def get_message(self, name): logging.debug("%s:about to acquire getlock", name) self.consumer_lock.acquire() logging.debug("%s:have getlock", name) message = self.message logging.debug("%s:about to release setlock", name) self.producer_lock.release() logging.debug("%s:setlock released", name) return message def set_message(self, message, name): logging.debug("%s:about to acquire setlock", name) self.producer_lock.acquire() logging.debug("%s:have setlock", name) self.message = message logging.debug("%s:about to release getlock", name) self.consumer_lock.release() logging.debug("%s:getlock released", name)  IIUC, the consumer lock is acquired and released in different threads (and the producer too). Is this possible? It feels like I'm missing something because then couldn't any thread release a lock held by another thread?

Answer: The answer is in the documentation:  The class implementing primitive lock objects. Once a thread has acquired a lock, subsequent attempts to acquire it block, until it is released; any thread may release it.  In fact this technique is frequently used to orchestrate the threads. If you look closely two Lock objects are created in the Pipeline class. The consumer_lock is acquired in initialization which means if two threads start concurrently, the consumer thread would hit its Lock object in self.consumer_lock.acquire() and gonna be blocked. It's gonna wait for producer thread to put something in there. There has to be something to be consumed after all. When producer puts something in, it releases the consumer_lock so that it can go ahead consumes it.
Question: I have a dataframe that looks like this:  And am running a logit model on fluid as dependent variable, and excluding vp and perip: model = smf.logit('''fluid ~ C(examq3_n, Treatment(reference = 2.0)) + C(pmhq3_n) + C(fluidq3_n) + C(mapq3_n, Treatment(reference = 3.0)) + C(examq6_n, Treatment(reference = 2.0)) + C(pmhq6_n) + C(fluidq6_n) + C(mapq6_n, Treatment(reference = 3.0)) + + C(case, Treatment(reference = 2))''', data = case1_2_vars).fit() print(model.summary())  I get the following results:  I am wondering if I need to add a constant to the data and if so, how? I've tried adding a column to the dataframe called const which equals 1, but when I then add const to the logit equation I get LinAlgError: Singular Matrix, and I don't know how to add it using smf.add_constant() because I have had to specify the categorical variables and their respective reference numbers in the equation, rather than defining x and y separately and simply inputting those into the smf.logit() call. My questions are: a) do I need to add a constant, and b) how? There are some links online that seem to imply it might not be necessary for a categorical variable-based logit model, but I would rather do it if it's best practice. I'm also wondering, does statsmodels automatically include a constant? Because Intercept is listed in the results.

Answer: If you use formulas, then the formula handling by patsy adds automatically a constant/intercept. (when using e.g. smf.logit or sm.Logit.from_formula) If you create a model without formula using numpy arrays or pandas DataFrame, then the exog is not changed by statsmodels, i.e. users needs to add a constant themselves. The helper function is sm.add_constant which adds a column of ones to the array or DataFrame. (when using e.g. sm.Logit(y, x))