eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Say I have two types (one of them generic) like this from typing import Generic, TypeVar T = TypeVar('T') class A(Generic[T]): pass class B: pass  And a union of A and B like this C = A|B  Or, in pre-Python-3.10/PEP 604-syntax: C = Union[A,B]  How do I have to change the definition of C, so that C is also generic? e.g. if an object is of type C[int], it is either  of type A[int] (type parameter is passed down) or of type B (type parameter is ignored) 

Answer: Rereading the mypy documentation I believe I have found my answer:  Type aliases can be generic. In this case they can be used in two ways: Subscripted aliases are equivalent to original types with substituted type variables, so the number of type arguments must match the number of free type variables in the generic type alias. Unsubscripted aliases are treated as original types with free variables replaced with Any  So, to answer my question: C = A[T]|B  should do the trick. And it does!
Question: So, I installed virtualenv in ubuntu terminal. I installed using the following commands: sudo apt install python3-virtualenv pip install virtualenv  But when I try creating a new virtualenv using: virtualenv -p python3 venv  I am getting the following error: AttributeError: module 'virtualenv.create.via_global_ref.builtin.cpython.mac_os' has no attribute 'CPython2macOsArmFramework'  How can I solve it?

Answer: You don't need to use virtualenv. You can use this: python3 -m venv ./some_env
Question: I have a list of names 'pattern' that I wish to match with strings in column 'url_text'. If there is a match i.e. True the name should be printed in a new column 'pol_names_block' and if False leave the row empty. pattern = '|'.join(pol_names_list) print(pattern) 'Jon Kyl|Doug Jones|Tim Kaine|Lindsey Graham|Cory Booker|Kamala Harris|Orrin Hatch|Bernie Sanders|Thom Tillis|Jerry Moran|Shelly Moore Capito|Maggie Hassan|Tom Carper|Martin Heinrich|Steve Daines|Pat Toomey|Todd Young|Bill Nelson|John Barrasso|Chris Murphy|Mike Rounds|Mike Crapo|John Thune|John. McCain|Susan Collins|Patty Murray|Dianne Feinstein|Claire McCaskill|Lamar Alexander|Jack Reed|Chuck Grassley|Catherine Masto|Pat Roberts|Ben Cardin|Dean Heller|Ron Wyden|Dick Durbin|Jeanne Shaheen|Tammy Duckworth|Sheldon Whitehouse|Tom Cotton|Sherrod Brown|Bob Corker|Tom Udall|Mitch McConnell|James Lankford|Ted Cruz|Mike Enzi|Gary Peters|Jeff Flake|Johnny Isakson|Jim Inhofe|Lindsey Graham|Marco Rubio|Angus King|Kirsten Gillibrand|Bob Casey|Chris Van Hollen|Thad Cochran|Richard Burr|Rob Portman|Jon Tester|Bob Menendez|John Boozman|Mazie Hirono|Joe Manchin|Deb Fischer|Michael Bennet|Debbie Stabenow|Ben Sasse|Brian Schatz|Jim Risch|Mike Lee|Elizabeth Warren|Richard Blumenthal|David Perdue|Al Franken|Bill Cassidy|Cory Gardner|Lisa Murkowski|Maria Cantwell|Tammy Baldwin|Joe Donnelly|Roger Wicker|Amy Klobuchar|Joel Heitkamp|Joni Ernst|Chris Coons|Mark Warner|John Cornyn|Ron Johnson|Patrick Leahy|Chuck Schumer|John Kennedy|Jeff Merkley|Roy Blunt|Richard Shelby|John Hoeven|Rand Paul|Dan Sullivan|Tim Scott|Ed Markey'  I am using the following code df['url_text'].str.contains(pattern) which results in True in case a name in 'pattern' is present in a row in column 'url_text' and False otherwise. With that I have tried the following code: df['pol_name_block'] = df.apply( lambda row: pol_names_list if df['url_text'].str.contains(pattern) in row['url_text'] else ' ', axis=1 )  I get the error: TypeError: 'in <string>' requires string as left operand, not Series 

Answer: From this toy Dataframe : >>> import pandas as pd >>> from io import StringIO >>> df = pd.read_csv(StringIO(""" ... id,url_text ... 1,Tim Kaine ... 2,Tim Kain ... 3,Tim ... 4,Lindsey Graham.com ... """), sep=',') >>> df id url_text 0 1 Tim Kaine 1 2 Tim Kain 2 3 Tim 3 4 Lindsey Graham.com  From pol_names_list, we build patterns by formating it like so : patterns = '(%s)' % '|'.join(pol_names_list)  Then, we can use the extract method to assign the value to the column pol_name_block to get the expected result : df['pol_name_block'] = df['url_text'].str.extract(patterns)  Output :  id url_text pol_name_block 0 1 Tim Kaine Tim Kaine 1 2 Tim Kain NaN 2 3 Tim NaN 3 4 Lindsey Graham.com Lindsey Graham
Question: Let's say I have an async function like this: async def foobar(argOne, argTwo, argThree): print(argOne, argTwo, argThree)  I want to make a decorator and use it on this function in a way that it wraps the above code in a try except statement like this: try: print(argOne, argTwo, argThree) except: print('Something went wrong.)  Is there any way to do this?

Answer: because wrapper called first, we should also define it as a async function: async def wrap(*arg, **kwargs): import asyncio def decorator(f): async def wrapper(*arg, **kwargs): try: await f(*arg, **kwargs) except Exception as e: print('Something went wrong.', e) return wrapper @decorator async def foobar(argOne, argTwo, argThree): print(argOne, argTwo, argThree) await asyncio.sleep(1) asyncio.run(foobar("a", "b", "c"))
Question: Today, I'm wondering what the difference between Conda in oneAPI and Conda in Anaconda is and how to use the oneAPI in the right way to get the maximum usage of the latest Intel Core gen 12. After installing oneAPI, they also contain conda. However, I cannot use this as a normal condition when: -It does not contain conda-build and several packages like normal conda in Anaconda. -Can not create as well as clone other environments from the "base" of Conda OneAPI. If I clone the "base" to the new one as conda create --name new_env --clone base and then activate the "new_env", I cannot use Conda anymore and it warns me like the conda does not exist. The warning is as below. 'conda' is not recognized as an internal or external command. operable program or batch file.   However, training any DNN model on conda oneAPI is faster than on conda in Anaconda by 30%, and it also has better performance in the data preprocessing tasks. I really want to always use the advantage of Python in the Conda OneAPI environment as normal Conda in Anaconda. So, how to merge them into one to make it easier to use, or how to fix the problem of Conda environment of oneAPI toolkit

Answer:  Conda executable in one api does not support all the features supported by conda in anaconda.  Conda executable in one api can be used to download both intel optimized packages as well as anaconda packages.  Conda executable in one api gives performance improvement for intel optimized packages.  Since setvars is not sourced you are getting this warning    'conda' is not recognized as an internal or external command. operable program or batch file.      Using Intel Conda Packages with Continuum's Python: If you want to install Intel packages into an environment with Continuum's python, do not add the "intel" channel to your configuration file because that will cause all your Continuum packages to be replaced with Intel builds, if available. Rather, specify the "intel" channel on the command line with "-c intel" parameter and the "--no-update-deps" flag to avoid switching other packages, such as python itself, to Intel's builds.  Use the following command to install intel optimized packages using conda executable in one api:     conda install "Package_name" -c intel --no-update-deps here Package_name can be(mkl,numpy..)    Available Intel packages can be viewed here: https://anaconda.org/intel/packages  Sample installation for intel optimized numpy package:    conda install numpy -c intel --no-update-deps
Question: I have a dataframe of stock prices: df = pd.DataFrame([100, 101, 99, 100,105,104,106], columns=['P'])  I would like to create a counter column, that counts either if the current price is higher than the previous row's price, BUT if the current price is lower than the previous row's price, only counts again, once that price is exceeded (like a watermark). Below is the desired column: df['counter'] = [np.nan, 1, 1, 1,2,2,3]  So the second row's price is 101 which exceeds 100, so the counter is 1, then the price drops to 99 and comes back to 100, but the counter is still 1, because we have not reached the 101 price (which is the watermark), then once we exceed 101 in row 4, with a price of 105, the counter goes to 2, then the price drops to 104 again, so we stay at 2, and then when it goes to 106 we increase the counter to 3.

Answer: Algorithm:  Find what current maximum previously observed value was at each row (inclusive of the current row).  See what the maximum previously observed value was for the preceding row.  Each time a difference exists between these two values, we know that a new water mark has been hit within the current row.  Calculate the cumulative sum of the number of times a new water mark has been hit.  df["current_observed_max"] = df["p"].cummax() df["previous_observed_max"] = df["current_observed_max"].shift(1) df["is_new_watermark"] =(df["current_observed_max"] != df["previous_observed_max"]).astype(int) df["counter"] = df["is_new_watermark"].cumsum()    With this you may need to subtract 1 depending on how you would like to handle the first observed number.
Question: I got this issue pip install easyocr  on python env import easyocr reader = easyocr.Reader(['en']) result = reader.readtext('./reports/dilate/NP6221833_126.png', workers=1)  finally Segmentation fault (core dumped) 

Answer: Solved downgrading to the nov 2021 version of opencv pip install opencv-python-headless==4.5.4.60
Question: I have a dataframe df:  2019 2020 2021 2022 A 1 10 15 15 31 2 5 4 7 9 3 0.3 0.4 0.4 0.7 4 500 600 70 90 B 1 10 15 15 31 2 5 4 7 9 3 0.3 0.4 0.4 0.7 4 500 600 70 90 C 1 10 15 15 31 2 5 4 7 9 3 0.3 0.4 0.4 0.7 4 500 600 70 90 D 1 10 15 15 31 2 5 4 7 9 3 0.3 0.4 0.4 0.7 4 500 600 70 90  I am trying to group by the level 1 index, 1, 2, 3, 4 and assign different aggregation functions for those 1, 2, 3, 4 indexes so that 1 is aggregated by sum, 2 by mean, and so on. So that the end result would look like this:  2019 2020 2021 2022 1 40 ... ... # sum 2 5 ... ... # mean 3 0.3 ... ... # mean 4 2000 ... ... # sum  I tried: df.groupby(level = 1).agg({'1':'sum', '2':'mean', '3':'sum', '4':'mean'})  But I get that none of 1, 2, 3, 4 are in columns which they are not, so I am not sure how should I proceed with this problem.

Answer: You could use apply with a custom function as follows: import numpy as np aggs = {1: np.sum, 2: np.mean, 3: np.mean, 4: np.sum} def f(x): func = aggs.get(x.name, np.sum) return func(x) df.groupby(level=1).apply(f)  The above code uses sum by default so 1 and 4 could be removed from aggs without any different results. In this way, only groups that should be handled differently from the rest need to be specified. Result:  2019 2020 2021 2022 1 40.0 60.0 60.0 124.0 2 5.0 4.0 7.0 9.0 3 0.3 0.4 0.4 0.7 4 2000.0 2400.0 280.0 360.0
Question: I was trying to train a model using tensorboard. While executing, I got this error: $ python train.py Traceback (most recent call last): File "train.py", line 6, in <module> from torch.utils.tensorboard import SummaryWriter File "C:\Users\91960\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\utils\tensorboard\__init__.py", line 4, in <module> LooseVersion = distutils.version.LooseVersion AttributeError: module 'setuptools._distutils' has no attribute 'version'.  I'm using python 3.8.9 64-bit & tensorflow with distutils already installed which is required by tensorboard. Why is this happening?

Answer: This command did the trick for me: python3 -m pip install setuptools==59.5.0 pip successfully installed this version: Successfully installed setuptools-60.1.0 instead of setuptools-60.2.0
Question: Computing nx.triangles(G) on an undirected graph with about 150 thousand nodes and 2 million edges, is currently very slow (on the scale of 80 hours). If the node degree distribution is highly skewed, is there any problem with counting triangles using the following procedure? import networkx as nx def largest_degree_node(G): # this was improved using suggestion by Stef in the comments return max(G.degree(), key=lambda x: x[1])[0] def count_triangles(G): G=G.copy() triangle_counts = 0 while len(G.nodes()): focal_node = largest_degree_node(G) triangle_counts += nx.triangles(G, nodes=[focal_node])[focal_node] G.remove_node(focal_node) return triangle_counts G = nx.erdos_renyi_graph(1000, 0.1) # compute triangles with nx triangles_nx = int(sum(v for k, v in nx.triangles(G).items()) / 3) # compute triangles iteratively triangles_iterative = count_triangles(G) # assertion passes assert int(triangles_nx) == int(triangles_iterative)  The assertion passes, but I am wary that there are some edge cases where this iterative approach will not work.

Answer: Assuming the graph is not directed (ie. G.is_directed() == False), the number of triangles can be efficiently found by finding nodes that are both neighbors of neighbors and direct neighbors of a same node. Pre-computing and pre-filtering the neighbors of nodes so that each triangle is counted only once helps to improve a lot the execution time. Here is the code: nodeNeighbours = { # The filtering of the set ensure each triangle is only computed once node: set(n for n in edgeInfos.keys() if n > node) for node, edgeInfos in G.adjacency() } triangleCount = sum( len(neighbours & nodeNeighbours[node2]) for node1, neighbours in nodeNeighbours.items() for node2 in neighbours )  The above code is about 12 times faster than the original iterative solution on the example graph. And up to 72 times faster on nx.erdos_renyi_graph(15000, 0.005).
Question: As a toy example, let's use the Fibonacci sequence: def fib(n: int) -> int: if n < 2: return 1 return fib(n - 2) + fib(n - 1)  Of course, this will hang the computer if we try to: print(fib(100))  So we decide to add memoization. To keep the logic of fib clear, we decide not to change fib and instead add memoization via a decorator: from typing import Callable from functools import wraps def remember(f: Callable[[int], int]) -> Callable[[int], int]: @wraps(f) def wrapper(n: int) -> int: if n not in wrapper.memory: wrapper.memory[n] = f(n) return wrapper.memory[n] wrapper.memory = dict[int, int]() return wrapper @remember def fib(n: int) -> int: if n < 2: return 1 return fib(n - 2) + fib(n - 1)  Now there is no problem if we: print(fib(100))  573147844013817084101  However, mypy complains that "Callable[[int], int]" has no attribute "memory", which makes sense, and usually I would want this complaint if I tried to access a property that is not part of the declared type... So, how should we use typing to indicate that wrapper, while a Callable, also has the property memory?

Answer: To describe something as "a callable with a memory attribute", you could define a protocol (Python 3.8+, or earlier versions with typing_extensions): from typing import Protocol class Wrapper(Protocol): memory: dict[int, int] def __call__(self, n: int) -> int: ...  In use, the type checker knows that a Wrapper is valid as a Callable[[int], int] and allows return wrapper as well as the assignment to wrapper.memory: from functools import wraps from typing import Callable, cast def remember(f: Callable[[int], int]) -> Callable[[int], int]: @wraps(f) def _wrapper(n: int) -> int: if n not in wrapper.memory: wrapper.memory[n] = f(n) return wrapper.memory[n] wrapper = cast(Wrapper, _wrapper) wrapper.memory = dict() return wrapper  Playground Unfortunately this requires wrapper = cast(Wrapper, _wrapper), which is not type safe - wrapper = cast(Wrapper, "foo") would also check just fine.
Question: I have styled a dataframe output and have gotten it to display how I want it in a Jupyter Notebook but I am having issues find a good way to save this as an image. I have tried https://pypi.org/project/dataframe-image/ but the way I have this working it seem to be a NoneType as it's a styler object and errors out when trying to use this library. This is just a snippet of the whole code, this is intended to loop through several 'col_names' and I want to save these as images (to explain some of the coding). import pandas as pd import numpy as np col_name = 'TestColumn' temp_df = pd.DataFrame({'TestColumn':['A','B','A',np.nan]}) t1 = (temp_df[col_name].fillna("Unknown").value_counts()/len(temp_df)*100).to_frame().reset_index() t1.rename(columns={'index':' '}, inplace=True) t1[' '] = t1[' '].astype(str) display(t1.style.bar(subset=[col_name], color='#5e81f2', vmax=100, vmin=0).set_table_attributes('style="font-size: 17px"').set_properties( **{'color': 'black !important', 'border': '1px black solid !important'} ).set_table_styles([{ 'selector': 'th', 'props': [('border', '1px black solid !important')] }]).set_properties( **{'width': '500px'}).hide_index().set_properties(subset=[" "], **{'text-align': 'left'}))  [OUTPUT] 

Answer: Was able to change how I was using dataframe-image on the styler object and got it working. Passing it into the export() function rather than calling it off the object directly seems to be the right way to do this. The .render() did get the HTML but was often losing much of the styling when converting it to image or when not viewed with Ipython HTML display. See comparision below.  Working Code: import pandas as pd import numpy as np import dataframe_image as dfi col_name = 'TestColumn' temp_df = pd.DataFrame({'TestColumn':['A','B','A',np.nan]}) t1 = (temp_df[col_name].fillna("Unknown").value_counts()/len(temp_df)*100).to_frame().reset_index() t1.rename(columns={'index':' '}, inplace=True) t1[' '] = t1[' '].astype(str) style_test = t1.style.bar(subset=[col_name], color='#5e81f2', vmax=100, vmin=0).set_table_attributes('style="font-size: 17px"').set_properties( **{'color': 'black !important', 'border': '1px black solid !important'} ).set_table_styles([{ 'selector': 'th', 'props': [('border', '1px black solid !important')] }]).set_properties( **{'width': '500px'}).hide_index().set_properties(subset=[" "], **{'text-align': 'left'}) dfi.export(style_test, 'successful_test.png')
Question: I have a base64-encoded public key in DER format. In Python, how can I convert it into a COSE key? Here is my failed attempt: from base64 import b64decode from cose.keys import CoseKey pubkeyder = "...==" decCborData.key = CoseKey.decode(b64decode(pubkeyder)) 

Answer: The posted key is an EC key for curve P-256 in X.509 format. With an ASN.1 parser (e.g. https://lapo.it/asn1js/) the x and y coordinates can be determined: x: 0x1AF1EA7FB498B65BDEBCEC80FE7A3E8B5FD67264B46CE60FD5B80FFA92538D39 y: 0x013A9422F9FEC87BAE35E56165F5AA2ACCC98A449984E94AF81FE6FD55B6BB14  Then the COSE key can be generated simply as follows: from cose.keys import EC2Key pub_x = bytes.fromhex('1AF1EA7FB498B65BDEBCEC80FE7A3E8B5FD67264B46CE60FD5B80FFA92538D39') pub_y = bytes.fromhex('013A9422F9FEC87BAE35E56165F5AA2ACCC98A449984E94AF81FE6FD55B6BB14') cose_pub_key = EC2Key(crv='P_256', x=pub_x, y=pub_y)  For details, s. the cose library documentation and RFC8152, CBOR Object Signing and Encryption (COSE), especially chapter 13.  The determination of the x and y coordinates can also be done programmatically, e.g. with PyCryptodome: from Crypto.PublicKey import ECC import base64 der = base64.b64decode('MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEGvHqf7SYtlvevOyA/no+i1/WcmS0bOYP1bgP+pJTjTkBOpQi+f7Ie6415WFl9aoqzMmKRJmE6Ur4H+b9Vba7FA==') key = ECC.import_key(der) pub_x = key.pointQ.x.to_bytes() pub_y = key.pointQ.y.to_bytes()
Question: I am learning C# and have been taking a lot of online courses. I am looking for a simpler/neater way to enumerate a list within a list. In python we can do something like this in just one line: newListofList=[[n,i] for n,i in enumerate([List1,List2,List3])]  Does it have to involve lambda and Linq in C#? if so, what would be the solution? I tried it with Dictionary in C# but my gut tells me this is not a perfect solution. List<List<string>> familyListss = new List<List<string>>(); familyListss.Add(new List<string> { "Mary", "Mary_sister", "Mary_father", "Mary_mother", "Mary_brother" }); familyListss.Add(new List<string> { "Peter", "Peter_sister", "Peter_father", "Peter_mother", "Peter_brother" }); familyListss.Add(new List<string> { "John", "John_sister", "John_father", "John_mother", "John_brother" }); Dictionary<int, List<string>> familyData = new Dictionary<int, List<string>>(); for (int i = 0; i < familyListss.Count; i++) { familyData.Add(i, familyListss[i]); } 

Answer: Just a constructor will be enough: List<List<string>> familyListss = new List<List<string>>() { new List<string> { "Mary", "Mary_sister", "Mary_father", "Mary_mother", "Mary_brother" }, new List<string> { "Peter", "Peter_sister", "Peter_father", "Peter_mother", "Peter_brother" }, new List<string> { "John", "John_sister", "John_father", "John_mother", "John_brother" } };  If you want to mimic enumerate you can use Linq, Select((value, index) => your lambda here): using System.Linq; ... var list = new List<string>() { "a", "b", "c", "d"}; var result = list .Select((value, index) => $"item[{index}] = {value}"); Console.Write(string.Join(Environment.NewLine, result));  Outcome: item[0] = a item[1] = b item[2] = c item[3] = d
Question: I'm python user learning R. Frequently, I need to check if columns of a dataframe contain NaN(s). In python, I can simply do import pandas as pd df = pd.DataFrame({'colA': [1, 2, None, 3], 'colB': ['A', 'B', 'C', 'D']}) df.isna().any()  giving me colA True colB False dtype: bool  In R I'm struggling to find an easy solution. People refer to some apply-like methods but that seems overly complex for such a primitive task. The closest solution I've found is this: library(tidyverse) df = data.frame(colA = c(1, 2, NA, 3), colB = c('A', 'B', 'C', 'D')) !complete.cases(t(df))  giving [1] TRUE FALSE  That's OKyish but I don't see the column names. If the dataframe has 50 columns I don't know which one has NaNs. Is there a better R solution?

Answer: You can use anyNA: Checks for NA in a vector df = data.frame(colA = c(1, 2, NA, 3), colB = c('A', 'B', 'C', 'D')) sapply(df, anyNA) colA colB TRUE FALSE  Edit jay.sf is right. This will check for NaNs. df = data.frame(colA = c(1, 2, NA, 3), colB = c('A', 'B', 'C', 'D')) anyNAN <- function(x) { any(is.nan(x)) } sapply(df, anyNAN)
Question: The author from python-onedrive python-onedrive warns that his library is achrived and obsoleted by official library from microsoft and refers to its git repo sdk What perplexes me is that the archived obsoleted library is being maintained while the official repo is dead since 6 years ago. What's going on? Thanks.

Answer: It appears the project was written before Microsoft authored their own SDK that solves the goal the creator originally had - namely that there was no Python SDK for OneDrive. Looking at the commit history, there's been no meaningful changes in six years. The only changes were typos in the documentation, which was cleaned up for long-term archival purposes. The author probably came to the conclusion that it's not worth maintaining a parallel SDK when an official one exists, which in theory should be maintained by the owner of OneDrive. Now, as to why Microsoft has not updated that SDK in six years is another issue entirely. There have been pull requests made to improve that library, but it's likely someone responsible for maintaining the repo needs to be contacted.