eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am using a library that requires abydos/distance/_aline.py to be imported. But as can be seen from the source code, it uses the old np_float: 25 from numpy import float as np_float  which throws an error ImportError: cannot import name 'float' from 'numpy' (/usr/local/lib/python3.10/dist-packages/numpy/__init__.py)  I have seen this error before with old versions of packages, so upgrading them usually solves it. This is from the latest installation of the library (abydos==0.5.0). I don't really want to downgrade my numpy installation. Is there a way to get around this?

Answer: Found a github issue from abydos right after posting this. Installing v0.6.0b from source solves the issue: !pip install git+https://github.com/chrislit/abydos.git  Hope this helps anyone trying to use the library. The 0.5.0 version requirement seems to be everywhere.
Question: I have records in below format: | id | name | created | ----------------------------------------------- |1 | A |2024-04-10T02:49:47.327583-07:00| |2 | A |2024-04-01T02:49:47.327583-07:00| |3 | A |2024-03-01T02:49:47.327583-07:00| |4 | A |2024-02-01T02:49:47.327583-07:00| |5 | B |2024-02-01T02:49:47.327583-07:00|  Model: class Model1(model.Models): name = models.CharField(max_length=100) created = models.DateTimeField(auto_now_add=True)  I want to perform a group by in django with month from field created and get latest record from that month. Expected output: | id | name | created | ----------------------------------------------- |1 | A |2024-04-10T02:49:47.327583-07:00| |3 | A |2024-03-01T02:49:47.327583-07:00| |4 | A |2024-02-01T02:49:47.327583-07:00|  I am using django-cte to perform the above action from django.db.models.functions import DenseRank, ExtractMonth from django_cte import With m = Model1.objects.get(id=1) cte = With( Model1.objects.filter(name=m.name) rank=Window( expression=DenseRank(), partition_by=[ExtractMonth("created")], order_by=F("created").desc(), ) ) qs = cte.queryset().with_cte(cte).filter(rank=1)  But the above give error: qs = cte.queryset().with_cte(cte).filter(rank=1) ^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'QuerySet' object has no attribute 'with_cte'  Please help!

Answer: You need to mix in the CTEManager, otherwise you get a "vanilla" QuerySet: from django_cte import CTEManager class Model1(model.Models): name = models.CharField(max_length=100) created = models.DateTimeField(auto_now_add=True) objects = CTEManager()
Question: Imagine I have this table: Col-1 | Col-2 A | 2 A | 3 B | 1 B | 4 C | 7  Groupby on Col-1 with a sum aggregation on Col-2 will sum A to 5, B to 5, and C to 7. What I want to know is if there is a baked in feature that allows aggregation on a target value in a column and then groups all other entries into another bin. For example, if I wanted to groupby on Col-1 targeting A and grouping all other entries into a label named other, I would end up with A as 5 and Other as 12. Does that make sense? I know I could do some filtering sorcery and merging datasets back together, but figured there had to be a cleaner, more Pythonic way I am missing. I have tried going through the documentation, but nothing jumped out at me.

Answer: One solution is to make pd.Categorical from the Column 1 -> with two categories A for string A and Other for other strings. Then group by this categorical: tmp = pd.Categorical(df["Col1"], categories=["A", "Other"]).fillna("Other") out = df.groupby(tmp, observed=False)["Col2"].sum() print(out)  Prints: A 5 Other 12 Name: Col2, dtype: int64   Another solution, group by boolean mask: out = ( df.groupby(df["Col1"].eq("A"))["Col2"] .sum() .rename(index={True: "A", False: "Other"}) ) print(out)  Prints: Col1 Other 12 A 5 Name: Col2, dtype: int64
Question: I am using Python3.10 on Ubuntu 22.04.4, and I am trying to run code that I originally wrote on a Windows 11 machine. Whenever I run this script--main.py--it always stops at the import stages, and fails to import beautifulsoup4 and yfinance in particular: print("Starting imports.") import pandas print("Imported pandas...") from bs4 import BeautifulSoup print("Imported beautifulsoup4...") import yfinance as yf print("Imported yfinance...") # Rest of code...  When I run this script in the terminal, it only ever prints Starting imports. Imported pandas.... Following that print statement, it spits out Illegal instruction (core dumped): (myenv) minime@Shaguar:~/Coding/WebScraping$ python main.py Starting imports. Imported pandas... Illegal instruction (core dumped)  I have done some research into this "Illegal instruction" error and have come across failed imports of TensorFlow, numpy, and others because of incompatible cpu architecture, yet I do not know how it would apply to beautifulsoup4 in this situation. How could I successfully import beautifulsoup4 and yfinance without the Illegal instruction (core dumped) error? (Architecture output of lscpu): Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 36 bits physical, 48 bits virtual Byte Order: Little Endian 

Answer: I have solved it! After further inspection of /var/log/syslog--the Ubuntu error/warning logfile--I found that there was a problem with some etree thing: kernel: [ 3887.864653] traps: python3[5907] trap invalid opcode ip:74ab4c0807c0 sp:7fffee156560 error:0 in etree.cpython-310-x86_64-linux-gnu.so[74ab4c04e000+329000]  After some research on etree and what it is used for, I found that a library shared by both yfinance and beautifulsoup4 that runs on etree was the cause of the core dumpage: lxml. I looked at the lxml PyPI page and saw that the latest version (lxml 5.20) did not support CPU architectures that pre-dated 2011; my Ubuntu machine is from 2009. So, I uninstalled and downgraded my lxml version to v5.0 using pip and everything worked smoothly. Key Takeaway If you are having a similar issue, it is probably due to incompatible CPU architecture with regards to a library or a library's underlying dependencies. Check your system logfiles after a core dump. Do some research, especially on CPU compatability, on any specific keywords in that logfile--in my case it was etree. Lastly, try to downgrade the suspected problem library--in my case it was lxml--to a version that supports your architecture. Everything above is what worked for me.
Question: I have two data frame, I need to change column values of first data frame that are in list, using second data frame. df1 = pd.DataFrame({'title':['The Godfather','Fight Club','The Empire'], 'genre_ids':[[18, 80],[18],[12, 28, 878]]}) title genre_ids 0 The Godfather [18, 80] 1 Fight Club [18] 2 The Empire [12, 28, 878] df2 = pd.DataFrame({'id':[18,80,12,28,878,99],'name':['Action','Adventure','Adventure','Animation','Comedy','Documentary']}) id name 0 18 Action 1 80 Horror 2 12 Adventure 3 28 Animation 4 878 Comedy 5 99 Documentary  How can I assign genere_ids like this using df2 in python  title genre_ids 0 The Godfather [Action, Horror] 1 Fight Club [Action] 2 The Empire [Adventure, Animation, Comedy] 

Answer: You can also first map the genre IDs in df1 to their corresponding names using df2, and then replacing the genre IDs with the mapped names, like the following: import pandas as pd df1 = pd.DataFrame({'title':['The Godfather','Fight Club','The Empire'], 'genre_ids':[[18, 80],[18],[12, 28, 878]]}) df2 = pd.DataFrame({'id':[18,80,12,28,878,99],'name':['Action','Adventure','Adventure','Animation','Comedy','Documentary']}) genre_map = dict(zip(df2['id'], df2['name'])) df1['genre_ids'] = df1['genre_ids'].apply(lambda x: [genre_map[id] for id in x]) print(df1)  Output:  title genre_ids 0 The Godfather [Action, Adventure] 1 Fight Club [Action] 2 The Empire [Adventure, Animation, Comedy]
Question: Is this expected behaviour (and if so, can someone explain why)? This only happens when using dill, not pickle. from pathlib import Path import dill class MyClass: def __init__(self) -> None: pass path = Path('test/test.pkl') # create parent directory if it does not exist path.parent.mkdir(exist_ok=True) x = [ MyClass() ] dill.dump(x, path.open('wb')) y = dill.load(path.open('rb')) print(isinstance(x[0], MyClass)) # True print(isinstance(y[0], MyClass)) # False ???  I was expecting True.

Answer: The reason or this is that dill is pickling and re-creating the MyClass class object when deserializing your object. Hence MyClass (also x[0].__class__) is a different object compared to the deserialized y[0].__class__ object, which causes the isinstance check to fail against MyClass. print(id(MyClass)) # 140430969773264 print(id(x[0].__class__)) # same as above # 140430969773264 print(id(y[0].__class__)) # different # 140430969780544  By contrast, the stdlib pickle module will use a reference to the class instead, which results in the behavior you expect because it will import the class by reference rather than creating a new class when deserializing your object. To make dill use references, set the byref setting to True  with byref=True, dill to behave a lot more like pickle with certain objects (like modules) pickled by reference as opposed to attempting to pickle the object itself.  dill.settings['byref'] = True x = [ MyClass() ] dill.dump(x, path.open('wb')) y = dill.load(path.open('rb')) print(isinstance(x[0], MyClass)) # True print(isinstance(y[0], MyClass)) # True  Alternatively, you can just use pickle from the stdlib instead of dill: pickle.dump(x, path.open('wb')) y = pickle.load(path.open('rb')) print(isinstance(x[0], MyClass)) # True print(isinstance(y[0], MyClass)) # True
Question: I'm trying to run a model written in jax, https://github.com/lindermanlab/S5. However, I ran into some error that says  Traceback (most recent call last): File "/Path/run_train.py", line 101, in <module> train(parser.parse_args()) File "/Path/train.py", line 144, in train state = create_train_state(model_cls, File "/Path/train_helpers.py", line 135, in create_train_state params = variables["params"].unfreeze() AttributeError: 'dict' object has no attribute 'unfreeze'  I tried to replicate this error by import jax import jax.numpy as jnp import flax from flax import linen as nn model = nn.Dense(features=3) params = model.init(jax.random.PRNGKey(0), jnp.ones((1, 2))) params_unfrozen = flax.traverse_util.unfreeze(params)  And the error reads: Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'flax.traverse_util' has no attribute 'unfreeze'  I'm using: flax 0.7.4 jax 0.4.13 jaxlib 0.4.13+cuda12.cudnn89  I think this is an issue relating to the version of flax, but does anyone know what exactly is going on? Any help is appreciated. Let me know if you need any further information

Answer: unfreeze is a method of Flax's FrozenDict class: (See FrozenDict.unfreeze). It appears that you have passed a Python dict where a FrozenDict is expected. To fix this, you should ensure that variables['params'] is a FrozenDict, not a dict. Regarding the error in your attempted replication: flax.traverse_util does not define an unfreeze function, but this seems unrelated to the original problem.
Question: i am currently using odoo 16, where i have created a custom field and trying to make that field readonly. The readonly field should appear only if a non-administrator user is logged in This is my initial code:- <odoo> <data> <record id="product_template_only_form_view" model="ir.ui.view"> <field name="name">product.template.product.form</field> <field name="model">product.template</field> <field name="inherit_id" ref="product.product_template_only_form_view"/> <field name="arch" type="xml"> <xpath expr="//group[@name='group_general']" position="inside"> <field name="recommanded_price" string="Recommended Price" readonly="1" groups="base.group_system"/> </xpath> </field> </record> </data> </odoo>  What i tried? <odoo> <data> <record id="product_template_only_form_view" model="ir.ui.view"> <field name="name">product.template.product.form</field> <field name="model">product.template</field> <field name="inherit_id" ref="product.product_template_only_form_view"/> <field name="arch" type="xml"> <xpath expr="//group[@name='group_general']" position="inside"> <field name="recommanded_price" string="Recommended Price" attrs="{'readonly': [('groups', 'not in', ['base.group_system'])]}" /> </xpath> </field> </record> </data> </odoo> 

Answer: Try this: for base.group_user, the field will become read-only, and for base.group_system, it will become editable. <record id="product_template_only_form_view" model="ir.ui.view"> <field name="name">product.template.product.form</field> <field name="model">product.template</field> <field name="inherit_id" ref="product.product_template_only_form_view" /> <field name="groups_id" eval="[(6, 0, [ref('base.group_system') ])]" /> <field name="arch" type="xml"> <field name="name" position="attributes"> <attribute name="readonly">0</attribute> </field> </field> </record> <record id="product_template_only_form_view" model="ir.ui.view"> <field name="name">product.template.product.form</field> <field name="model">product.template</field> <field name="inherit_id" ref="product.product_template_only_form_view" /> <field name="groups_id" eval="[(6, 0, [ref('base.group_user') ])]" /> <field name="arch" type="xml"> <field name="name" position="attributes"> <attribute name="readonly">1</attribute> </field> </field> </record>
Question: On running: >>> import numpy as np >>> np.exp(1000) <stdin>:1: RuntimeWarning: overflow encountered in exp  Shows an overflow warning. But then why does the following not give an underflow warning? >>> np.exp(-100000) 0.0 

Answer: By default, underflow errors are ignored. The current settings can be checked as follows: print(np.geterr())  {'divide': 'warn', 'over': 'warn', 'under': 'ignore', 'invalid': 'warn'}  To issue a warning for underflows just like overflows, you can use np.seterr like this: np.seterr(under="warn") np.exp(-100000) # RuntimeWarning: underflow encountered in exp  Alternatively, you can use np.errstate like this: import numpy as np with np.errstate(under="warn"): np.exp(-100000) # RuntimeWarning: underflow encountered in exp
Question: I have a frame (events) which I want to join into another frame (fr), joining on Date and Symbol. There aren't necessarily any date overlaps. The date in events would match with the first occurrence only on the same or later date in fr, so if the event date is 2010-12-01, it would join on the same date or if not present then the next available date (2010-12-02). I've tried to do this using search_sorted and join_asof but I'd like to group by the Symbol column and also this isn't a proper join. This somewhat works for a single Symbol only. fr = pl.DataFrame( { 'Symbol': ['A']*5, 'Date': ['2010-08-29', '2010-09-01', '2010-09-05', '2010-11-30', '2010-12-02'], } ).with_columns(pl.col('Date').str.to_date('%Y-%m-%d')).with_row_index().set_sorted("Date") events = pl.DataFrame( { 'Symbol': ['A']*3, 'Earnings_Date': ['2010-06-01', '2010-09-01', '2010-12-01'], 'Event': [1, 4, 7], } ).with_columns(pl.col('Earnings_Date').str.to_date('%Y-%m-%d')).set_sorted("Earnings_Date") idx = fr["Date"].search_sorted(events["Earnings_Date"], "left") fr = fr.with_columns( pl.when( pl.col("index").is_in(idx) ) .then(True) .otherwise(False) .alias("Earnings") ) fr = fr.join_asof(events, by="Symbol", left_on="Date", right_on="Earnings_Date") fr = fr.with_columns( pl.when( pl.col("Earnings") == True ) .then(pl.col("Event")) .otherwise(False) .alias("Event") ) 

Answer: It sounds like you are on the right track using pl.DataFrame.join_asof. To group by the symbol the by parameter can be used. ( fr .join_asof( events, left_on="Date", right_on="Earnings_Date", by="Symbol", ) )  shape: (5, 5) ┌───────┬────────┬────────────┬───────────────┬───────┐ │ index ┆ Symbol ┆ Date ┆ Earnings_Date ┆ Event │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ u32 ┆ str ┆ date ┆ date ┆ i64 │ ╞═══════╪════════╪════════════╪═══════════════╪═══════╡ │ 0 ┆ A ┆ 2010-08-29 ┆ 2010-06-01 ┆ 1 │ │ 1 ┆ A ┆ 2010-09-01 ┆ 2010-09-01 ┆ 4 │ │ 2 ┆ A ┆ 2010-09-05 ┆ 2010-09-01 ┆ 4 │ │ 3 ┆ A ┆ 2010-11-30 ┆ 2010-09-01 ┆ 4 │ │ 4 ┆ A ┆ 2010-12-02 ┆ 2010-12-01 ┆ 7 │ └───────┴────────┴────────────┴───────────────┴───────┘  Now, I understand that you'd like each event to be matched at most once. I don't believe this is possible with join_asof alone. However, we can set all event rows that equal the previous row to Null. For this, an pl.when().then() construct can be used. ( fr .join_asof( events, left_on="Date", right_on="Earnings_Date", by="Symbol", ) .with_columns( pl.when( pl.col("Earnings_Date", "Event").is_first_distinct() ).then( pl.col("Earnings_Date", "Event") ).over("Symbol") ) )  shape: (5, 5) ┌───────┬────────┬────────────┬───────────────┬───────┐ │ index ┆ Symbol ┆ Date ┆ Earnings_Date ┆ Event │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ u32 ┆ str ┆ date ┆ date ┆ i64 │ ╞═══════╪════════╪════════════╪═══════════════╪═══════╡ │ 0 ┆ A ┆ 2010-08-29 ┆ 2010-06-01 ┆ 1 │ │ 1 ┆ A ┆ 2010-09-01 ┆ 2010-09-01 ┆ 4 │ │ 2 ┆ A ┆ 2010-09-05 ┆ null ┆ null │ │ 3 ┆ A ┆ 2010-11-30 ┆ null ┆ null │ │ 4 ┆ A ┆ 2010-12-02 ┆ 2010-12-01 ┆ 7 │ └───────┴────────┴────────────┴───────────────┴───────┘
Question: When we call vectorized function for some vector, for some reason it is called twice for the first vector element. What is the reason, and can we get rid of this strange effect (e.g. when this function needs to have some side effect, e.g. counts some sum etc) Example: import numpy @numpy.vectorize def test(x): print(x) test([1,2,3])  Result: 1 1 2 3  numpy 1.26.4

Answer: This is well defined in the vectorize documentation:  If otypes is not specified, then a call to the function with the first argument will be used to determine the number of outputs.  If you don't want this, you can define otypes: import numpy def test(x): print(x) test = numpy.vectorize(test, otypes=[float]) test([1,2,3]) 1 2 3  In any case, be aware that vectorize is just a convenience around a python loop. If you need to have side effects you might rather want to use an explicit python loop.
Question: I have stuck with a pretty simple problem - I can't communicate with process' stdout. The process is a simple stopwatch, so I'd be able to start it, stop and get current time. The code of stopwatch is: import argparse import time def main() -> None: parser = argparse.ArgumentParser() parser.add_argument('start', type=int, default=0) start = parser.parse_args().start while True: print(start) start += 1 time.sleep(1) if __name__ == "__main__": main()  And its manager is: import asyncio class RobotManager: def __init__(self): self.cmd = ["python", "stopwatch.py", "10"] self.robot = None async def start(self): self.robot = await asyncio.create_subprocess_exec( *self.cmd, stdout=asyncio.subprocess.PIPE, ) async def stop(self): if self.robot: self.robot.kill() stdout = await self.robot.stdout.readline() print(stdout) await self.robot.wait() self.robot = None async def main(): robot = RobotManager() await robot.start() await asyncio.sleep(3) await robot.stop() await robot.start() await asyncio.sleep(3) await robot.stop() asyncio.run(main())  But stdout.readline returns an empty byte string every time. When changing stdout = await self.robot.stdout.readline() to stdout, _ = await self.robot.communicate(), the result is still an empty byte string. When adding await self.robot.stdout.readline() to the end of the RobotManager.start method, it hangs forever. However, when removing stdout=asyncio.subprocess.PIPE and all readline calls, the subprocess prints to the terminal as expected. How do I read from the subprocess stdout correctly?

Answer: In this case "proc.communicate" cannot be used; it's not suitable for the purpose, since the OP wants to interrupt a running process. The sample code in the Python docs also shows how to directly read the piped stdout in these cases, so there is in principle nothing wrong with doing that. The main problem is that the stopwatch process is buffering the output. Try using as command: ["python", "-u", "stopwatch.py", "3"] For debugging it will also help to add some prints indicating when the robot started and ended. The following works for me: class RobotManager: def __init__(self): self.cmd = ["python", "-u", "stopwatch.py", "3"] self.robot = None async def start(self): print("======Starting======") self.robot = await asyncio.create_subprocess_exec( *self.cmd, stdout=asyncio.subprocess.PIPE, ) async def stop(self): if self.robot: self.robot.kill() stdout = await self.robot.stdout.readline() while stdout: print(stdout) stdout = await self.robot.stdout.readline() await self.robot.wait() print("======Terminated======") self.robot = None
Question: I'm using VSCode on Windows, with the GitBash as integrated terminal. When I launch the Python Debugger with default configurations, it works fine, and I get this command executed on the terminal: /usr/bin/env c:\\Users\\augus\\.Apps\\anaconda3\\envs\\muskit-env\\python.exe \ c:\\Users\\augus\\.vscode\\extensions\\ms-python.debugpy-2024.2.0-win32-x64\\bundled\\libs\\debugpy\\adapter/../..\\debugpy\\launcher \ 53684 -- E:\\muskit\\QuantumSoftwareTestingTools\\Muskit\\Muskit\\CommandMain.py  Notice the \\ in the file path. Again, above works just fine. The problem is when I add the args property to my launch.json configuration. launch.json { "configurations": [ { "name": "Python Debugger: Current File with Arguments", "type": "debugpy", "request": "launch", "program": "${file}", "console": "integratedTerminal", "args": "foo" } ] }  The following command is executed on the terminal: $ /usr/bin/env c:\Users\augus\.Apps\anaconda3\envs\muskit-env\python.exe \ c:\Users\augus\.vscode\extensions\ms-python.debugpy-2024.2.0-win32-x64\bundled\libs\debugpy\adapter/../..\debugpy\launcher \ 53805 -- E:\muskit\QuantumSoftwareTestingTools\Muskit\Muskit\CommandMain.py foo  /usr/bin/env: ‘c:Usersaugus.Appsanaconda3envsmuskit-envpython.exe’: No such file or directory  Notice that, instead of \\. it uses \, which causes the "No such file or directory". Is this a bug, or am I missing something?

Answer: Going through the issues on vscode-python repository, it is being mentioned in multiple issues that git bash is not officially supported. For example here:  Note Gitbash isn't supported by Python extension, so use Select default profile to switch to cmd or powershell if need be.  Possibly it is a bug and it will be better to use cmd or powershell, since you can run into issues in the future as well. Some related issues which mention the same  https://github.com/Microsoft/vscode-python/issues/3035 https://github.com/microsoft/vscode-python/issues/23008 https://github.com/microsoft/vscode-python/issues/22957
Question: How do I remove the gaps between the subplots on a mosaic? The traditional way does not work with mosaics: plt.subplots_adjust(wspace=0, hspace=0)  I also tried using gridspec_kw, but no luck. import matplotlib.pyplot as plt import numpy as np ax = plt.figure(layout="constrained").subplot_mosaic( """ abcde fghiX jklXX mnXXX oXXXX """, empty_sentinel="X", gridspec_kw={ "wspace": 0, "hspace": 0, }, ) for k,ax in ax.items(): print(ax) #ax.text(0.5, 0.5, k, transform=ax.transAxes, ha="center", va="center", fontsize=8, color="darkgrey") ax.set_xticklabels([]) ax.set_yticklabels([]) ax.tick_params(length = 0)  The code generates: 

Answer: This is not caused by subplot_mosaic but because a layout was specified. If you use constrained or tight layout, the layout manager will supersede the custom adjustments. Remove the layout manager and either method will work:   gridspec_kw fig = plt.figure() # without layout param ax = fig.subplot_mosaic( """ abcde fghiX jklXX mnXXX oXXXX """, empty_sentinel="X", gridspec_kw={ "wspace": 0, "hspace": 0, }, )   subplots_adjust fig = plt.figure() # without layout param ax = fig.subplot_mosaic( """ abcde fghiX jklXX mnXXX oXXXX """, empty_sentinel="X", ) fig.subplots_adjust(wspace=0, hspace=0)