eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I would have hoped this works (in Python 3.6), class A: __hash__ = id A().__hash__()  but I get  TypeError: id() takes exactly one argument (0 given)  Surprisingly,  def my_id(self): return id(self) class A: __hash__ = my_id A().__hash__()  works as hoped.

Answer: id is of type builtin_function_or_method (it's a function that's built into the runtime - ), which for practical reasons (optimisation mainly) doesn't implement the descriptor protocol as a python function would, so A().__hash__ resolves to the id function itself, not to a method object wrapping the function. You'll observe the same behaviour with most builtin functions FWIW:  >>> type(len) <class 'builtin_function_or_method'> >>> len.__get__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'builtin_function_or_method' object has no attribute '__get__' >>> >>> type(all) <class 'builtin_function_or_method'> >>> all.__get__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'builtin_function_or_method' object has no attribute '__get__' >>> type(abs) <class 'builtin_function_or_method'> >>> abs.__get__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'builtin_function_or_method' object has no attribute '__get__' >>> type(isinstance) <class 'builtin_function_or_method'> >>> isinstance.__get__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'builtin_function_or_method' object has no attribute '__get__'  etc...
Question: Is there a format code to format -2.34 as '-2.3', but +2.34 as ' 2.3' (notice the leading space)? Basically show the negative sign but leave a space for positive sign.

Answer: Use " " (a space) to insert a space before positive numbers and a minus sign before negative numbers: txt = "The temperature is between {: } and {: } degrees celsius." print(txt.format(-3, 7))  answer : The temperature is between -3 and 7 degrees celsius.
Question: When I run the below code it loads the web, but then it leaves an error message, it is because in some section there is no data, can a conditional help?  I uploaded the code but I do not know where to act, i am new on Dash and my knowledge in javascript is limited. My main file import dash from dash.dependencies import Output, Input import dash_core_components as dcc import dash_html_components as html import plotly import random import plotly.graph_objs as go from collections import deque import sqlite3 import pandas as pd #popular topics: google, olympics, trump, gun, usa app = dash.Dash(__name__) app.layout = html.Div( [ html.H2('Live Twitter Sentiment'), dcc.Input(id='sentiment_term', value='trump', type='text'), dcc.Graph(id='live-graph', animate=False), dcc.Interval( id='graph-update', interval=1*1000 ), ] ) @app.callback(Output('live-graph', 'figure'), [Input(component_id='sentiment_term', component_property='value')]) def update_graph_scatter(sentiment_term): try: conn = sqlite3.connect('twitter.db') c = conn.cursor() df = pd.read_sql("SELECT * FROM sentiment WHERE tweet LIKE ? ORDER BY unix DESC LIMIT 1000", conn, params=('%' + sentiment_term + '%',)) df.sort_values('unix', inplace=True) df['sentiment_smoothed'] = df['sentiment'].rolling(int(len(df)/5)).mean() df.dropna(inplace=True) X = df.unix.values[-100:] Y = df.sentiment_smoothed.values[-100:] data = plotly.graph_objs.Scatter( x=X, y=Y, name='Scatter', mode= 'lines+markers' ) return {'data': [data],'layout' : go.Layout(xaxis=dict(range=[min(X),max(X)]), yaxis=dict(range=[min(Y),max(Y)]), title='Term: {}'.format(sentiment_term))} except Exception as e: with open('errors.txt','a') as f: f.write(str(e)) f.write('\n') if __name__ == '__main__': app.run_server(debug=True) 

Answer: Sometimes Dash can struggle if the prop being updated by a callback hasn't been initialized. In this case, the figure prop of the dcc.Graph was never declared. Setting an explicit empty value, such as figure={} is often enough to resolve this sort of error.
Question: I have a (really big) pandas Dataframe df: country age gender Brazil 10 F USA 20 F Brazil 10 F USA 20 M Brazil 10 M USA 20 M  I have another pandas Dataframe freq:  age gender counting 10 F 0 10 M 0 20 F 0  I wanna count the pair of values in freq when they occur in df:  age gender counting 10 F 2 10 M 1 20 F 1  I'm using this code, but it takes too long: for row in df.itertuples(index=False): freq.loc[np.all(freq['age','gender']==row[2:3],axis=1),'counting'] += 1  Is there a faster way to do that? Please note:  I have to use freq because not all combinations (as for instance 20 and M) are desired some columns in df may not be used counting counts how many times both values appear in each row freq may have more than 2 values to check for (this is just an small example) 

Answer: you can do it with inner merge to filter the combinations in df you don't want, then groupby age and gender and count the column counting. just reset_index to fit your expected output. freq = (df.merge(freq, on=['age', 'gender'], how='inner') .groupby(['age','gender'])['counting'].size() .reset_index()) print (freq) age gender counting 0 10 F 2 1 10 M 1 2 20 F 1  Depending on the number of combinations you don't want, it could be faster to groupby on df before doing the merge like: freq = (df.groupby(['age','gender']).size() .rename('counting').reset_index() .merge(freq[['age','gender']]) )
Question: I am new in work with JSON, so sorry in advance for the stupid question. I want to write JSON with the variable in the value field. It looks like this: def print_json(user_name): opened_json = open('way/to/json/file') tmp = json.load(opened_json) res = tmp(['path_to_folder'](user_name)) print(res) def main(user_name): print_json(user_name) main('user')  It is JSON: {"path_to_folder": "/Users/" + user_name + "/my_folder/"}  Awaiting for that output: /Users/user/my_folder/  Please, tell me if any solution here exists. Thanks in advance! EDIT: My problem, that I can't add variable to JSON correctly. It marked red. Wrong syntax, when I try to concat. 

Answer: What you want isn't directly possible in JSON, because it doesn't support "templating". One solution would be to use a templating language such as Jinja to write a JSON template, then load this file without the json library and fill in the values using Jinja, and finally use json.loads to load a dictionary from your rendered string. Your json-like file could look something like this: {"path_to_folder": "/Users/{{ user_name }}/my_folder/"}  Your Python code: import json from jinja2 import Environment, FileSystemLoader env = Environment( FileSystemLoader("path/to/template") ) template = env.get_template("template_filename.json") def print_json(username): return json.loads( template.render(user_name=username) ) ...   In fact, if this is a simple one-time thing, it might even be better to use Python's built-in templating. I would recommend old-style formatting in the case of JSON, because otherwise you'll have to escape a lot of braces: JSON file: {"path_to_folder": "/Users/%(user_name)s/my_folder/"}  "Rendering": with open("path/to/json") as f: rendered = json.loads(f.read() % {"user_name": username})
Question: I have a method inside a class to return a func which parameters may change. The Interface function accept two parameters, f and its args.I want to use mp.pool to accelerate it.However, it returns an error. from multiprocessing import Pool # from multiprocess import Pool # from pathos.multiprocessing import ProcessingPool as Pool import pickle import dill class Temp: def __init__(self, a): self.a = a def test(self): def test1(x): return self.a + x return test1 def InterfaceFunc(f, x): mypool = Pool(4) return list(mypool.map(f, x)) if __name__ == "__main__": t1 = Temp(1).test() x = [1, 2, 3, 1, 2] res1 = list(map(t1, x)) print(res1) res2 = InterfaceFunc(t1, x)  it raise the same error: AttributeError: Can't pickle local object 'Temp.test.<locals>.test1'  I have tried 3 method: What can multiprocessing and dill do together? Replace pickle in Python multiprocessing lib Python Multiprocessing Pool Map: AttributeError: Can't pickle local object Method 1, 2 :  from multiprocess import Pool from pathos.multiprocessing import ProcessingPool as Pool  It raise error:  File "E:\Users\ll\Anaconda3\lib\site-packages\dill\_dill.py", line 577, in _load_type return _reverse_typemap[name] KeyError: 'ClassType'  Method3 needs to change the code , however I cant simply move the func out of the class because I need f to be a parameter for the Interface. Do you have any suggestions? I'm an inexperienced newcomer. 

Answer: Python can't pickle the closure, but all you really need is something that you can call that retains state. The __call__ method makes a class instance callable, so use that from multiprocessing import Pool class TempTest1: def __init__(self, a): self.a = a def __call__(self, x): return self.a + x class Temp: def __init__(self, a): self.a = a def test(self): return TempTest1(self.a) def InterfaceFunc(f, x): mypool = Pool(4) return list(mypool.map(f, x)) if __name__ == "__main__": t1 = Temp(1).test() x = [1, 2, 3, 1, 2] res1 = list(map(t1, x)) print(res1) res2 = InterfaceFunc(t1, x) print(res2)
Question: According to the documentation the default value for lru_cache from functools is 128. But no unit is defined.   Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments. Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable. Distinct argument patterns may be considered to be distinct calls with separate cache entries. For example, f(a=1, b=2) and f(b=2, a=1) differ in their keyword argument order and may have two separate cache entries. If user_function is specified, it must be a callable. This allows the lru_cache decorator to be applied directly to a user function, leaving the maxsize at its default value of 128.  My question is there any unit like bits, bytes, megabytes attached to this or is this an arbitrary number that has no simple relationship with the used memory?

Answer: Short answer: It is the number of elements that are stored in the cache. We can look up the source code of the lru_cache [GitHub]. The code is rather complicated, but in a nutshell, line 619 already gives a clue:  full = (cache_len() >= maxsize) This specifies that the cache is full given that the cache_len() is greater than or equal to the maxsize. The cache_len is a function that returns the number of records in the dictionary, as we can see in the source code:  cache = {} hits = misses = 0 full = False cache_get = cache.get # bound method to lookup a key or return None cache_len = cache.__len__ # get cache size without calling len() The logic also each time branches when it adds a new record, in case the cache is full, it will "kick out" one of the elements:  if key in cache: # Getting here means that this same key was added to the # cache while the lock was released. Since the link # update is already done, we need only return the # computed result and update the count of misses. pass elif full: # Use the old root to store the new key and result. oldroot = root oldroot[KEY] = key oldroot[RESULT] = result # Empty the oldest link and make it the new root. # Keep a reference to the old key and old result to # prevent their ref counts from going to zero during the # update. That will prevent potentially arbitrary object # clean-up code (i.e. __del__) from running while we're # still adjusting the links. root = oldroot[NEXT] oldkey = root[KEY] oldresult = root[RESULT] root[KEY] = root[RESULT] = None # Now update the cache dictionary. del cache[oldkey] # Save the potentially reentrant cache[key] assignment # for last, after the root and links have been put in # a consistent state. cache[key] = oldroot else: # Put result in a new link at the front of the queue. last = root[PREV] link = [last, root, key, result] last[NEXT] = root[PREV] = cache[key] = link # Use the cache_len bound method instead of the len() function # which could potentially be wrapped in an lru_cache itself. full = (cache_len() >= maxsize)
Question: When I save my model I get the following error: --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-40-853303da8647> in <module>() 7 8 ----> 9 model.save(outdir+'model.h5') 10 11 5 frames /usr/local/lib/python3.6/dist-packages/h5py/_hl/group.py in __setitem__(self, name, obj) 371 372 if isinstance(obj, HLObject): --> 373 h5o.link(obj.id, self.id, name, lcpl=lcpl, lapl=self._lapl) 374 375 elif isinstance(obj, SoftLink): h5py/_objects.pyx in h5py._objects.with_phil.wrapper() h5py/_objects.pyx in h5py._objects.with_phil.wrapper() h5py/h5o.pyx in h5py.h5o.link() RuntimeError: Unable to create link (name already exists)  This does not happen when I use built-in layers to build my model or others user defined layers. This error arises only when I use this particular user defined layer: class MergeTwo(keras.layers.Layer): def __init__(self, nout, **kwargs): super(MergeTwo, self).__init__(**kwargs) self.nout = nout self.alpha = self.add_weight(shape=(self.nout,), initializer='zeros', trainable=True) self.beta = self.add_weight(shape=(self.nout,), initializer='zeros', trainable=True) def call(self, inputs): A, B = inputs result = keras.layers.add([self.alpha*A ,self.beta*B]) result = keras.activations.tanh(result) return result def get_config(self): config = super(MergeTwo, self).get_config() config['nout'] = self.nout return config  I read the Docs but nothing worked, I cannot figure out why. I am using Google Colab and Tensorflow version 2.2.0

Answer: I think the problem is that both of your weight variables have internally the same name, which should not happen, you can give them names with the name parameter to add_weight: self.alpha = self.add_weight(shape=(self.nout,), initializer='zeros', trainable=True, name="alpha") self.beta = self.add_weight(shape=(self.nout,), initializer='zeros', trainable=True, name="beta")  This should workaround the problem.
Question: I have the following command below: import pandas as pd import numpy as np from scipy import stats np.random.seed(12345) standarderrors1992 = stats.sem(np.random.normal(32000,200000,3650)) standarderrors1993 = stats.sem(np.random.normal(43000,100000,3650)) standarderrors1994 = stats.sem(np.random.normal(43500,140000,3650)) standarderrors1995 = stats.sem(np.random.normal(48000,70000,3650)) mean1992 = np.random.normal(32000,200000,3650).mean() mean1993 = np.random.normal(43000,100000,3650).mean() mean1994 = np.random.normal(43500,140000,3650).mean() mean1995 = np.random.normal(48000,70000,3650).mean()  Here, I have found both the mean and standard error for a set of randomly chosen values.  limit = 3000 dict = {mean1992:standarderrors1992,mean1993:standarderrors1993,mean1994:standarderrors1994,mean1995:standarderrors1995} for key,value in dict: if limit > (key+(1.96*value)): colour = 1 elif limit < (key+(1.96*value)): colour = 0 elif (limit !> (key+(1.96*value))) && (limit !< (key-(1.96*value))): colour = ((key+(1.96*value))-limit)/((key+(1.96*value))-(key-(1.96*value)))  Here, I am trying to put the values corresponding to the means and standard errors into a dictionary so that I can loop through both of them.  Ideally, I want to assign a particular value to the variable 'colour' depending on the values for the mean and standard error of a particular year. i.e. mean and SE for 1992  However, I keep getting the error: TypeError: cannot unpack non-iterable int object Coudld anyone let me know where I'm going wrong?

Answer: You need to iterate over dict.items() for this to work. for key,value in dict.items(): # do stuff here  I would advice against naming your variables dict which shadows the build in dict function though :)
Question: Given that, I have a dataframe as below: import pandas as pd import numpy as np dict = { "A": [[1,2,3,4],[3],[2,8,4],[5,8]] } dt = pd.DataFrame(dict)  I wish to have the Maximum and minimum of each row in column B. My favorite output is:  A B 0 [1, 2, 3, 4] [1,4] 1 [3] [3,3] 2 [2, 8, 4] [2,8] 3 [5, 8] [5,8]  What I already tried is the below code which does not work: dt["B"] =[np.min(dt.A), np.max(dt.A)] 

Answer: Like this: In [1592]: dt['B'] = dt.A.apply(lambda x: [min(x), max(x)]) In [1593]: dt Out[1593]: A B 0 [1, 2, 3, 4] [1, 4] 1 [3] [3, 3] 2 [2, 8, 4] [2, 8] 3 [5, 8] [5, 8]  As suggested by @Ch3steR, using map since it's faster: dt['B'] = dt.A.map(lambda x: [min(x), max(x)])
Question: I'm trying to convert the following Keras model code to pytorch, but am having problems dealing with padding='same'.   model = Sequential() model.add(Conv2D(64, (3, 3), input_shape=img_size)) model.add(BatchNormalization(axis=1)) model.add(Activation('relu')) model.add(Dropout(0.3)) model.add(Conv2D(64, (3, 3), padding='same')) model.add(BatchNormalization(axis=1)) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'))  Which produces the following summary:  Layer (type) Output Shape Param # ================================================================= conv2d_1 (Conv2D) (None, 30, 30, 64) 1792 _________________________________________________________________ batch_normalization_1 (Batch (None, 30, 30, 64) 120 _________________________________________________________________ activation_1 (Activation) (None, 30, 30, 64) 0 _________________________________________________________________ dropout_1 (Dropout) (None, 30, 30, 64) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 30, 30, 64) 36928 _________________________________________________________________ batch_normalization_2 (Batch (None, 30, 30, 64) 120 _________________________________________________________________ activation_2 (Activation) (None, 30, 30, 64) 0 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 15, 15, 64) 0 ================================================================= Total params: 38,960 Trainable params: 38,840 Non-trainable params: 120  Right now, I would write:  self.features = nn.Sequential( nn.Conv2d(3, 64, kernel_size=3, bias=False), nn.BatchNorm2d(64), nn.ReLU(inplace=True), nn.Dropout(0.3), nn.Conv2d(64, 64, kernel_size=3, padding = ? bias=False), nn.BatchNorm2d(64), nn.ReLU(inplace=True), nn.MaxPool2d(kernel_size=3, stride=2, padding = ?), )  Where padding should have numerical value. I was wondering if there is an easier way to calculate this since we're using padding='same'.  Also, the next line of the Keras model looks like: model.add(Conv2D(128, (3, 3), padding='same'))  So I really need to brush up on how to calculate padding, especially after stride too. From a rough eye only, is the padding 2?

Answer: W:input volume size F:kernel size S:stride P:amount of padding size of output volume = (W-F+2P)/S+1 e.g. input:7x7, kernel:3x3, stride:1, pad:0 output size = (7-3+2*0)/1+1 = 5 =>5x5
Question: When I am using a poetry command with Python 3.7, in my case: poetry export -f requirements.txt  I am getting the following error: Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to update them.  So far clear, but if I run poetry update it upgrades my dependencies, which is not what I want at this time for my project. If I run poetry lock instead, it still upgrades dependencies. How can I work around this?

Answer: UPDATE V2.0.0 (released date : 4 jan 2025) : The --no-update option does no longer exist (see V2 docs). The same documentation says:  By default, packages that have already been added to the lock file before will not be updated.  So now just use: poetry lock or downgrade you poetry version to use the solution below: Former answer (before Poetry V2.0.0 release): This is a known issue in Poetry. The issue is resolved, use: poetry lock --no-update. Old answer: There is a current workaround with the following commands: poetry add pathlib2 poetry remove pathlib2  Where pathlib2 is any library you don't already depend on and that has no dependencies on it's own, hence pathlib2. Using these commands will rewrite the lockfile hashes and resolve the file conflict without upgrading any of the other packages used in the project.
Question: I have the following pytest directory structure: system_tests/ ├── conftest ├── pytest.ini │ ├── suite_1/ │ └── test_A.py │ └── suite_2/ └── sub_suite_2a/ └── test_B.py  When each test method runs, a number of third-party libraries/processes generate artifacts in the current working directory.  When pytest is executed from the sub_suite folder (using CLI or IDE "play" button), the files are generated in the sub_suite folder, where I want them to be. However, when pytest is run from the system_tests folder to run all tests, all artifacts are created in the system_tests folder, which is not what I want.  Is there an easy way to force pytest to always use the test class folder as the working directory so I get the same results regardless of how or where I run a test from?

Answer: EDIT: Improved Solution Using monkeypatch as suggested by @Kound removes the boilerplate code to restore the cwd. You can also enable autouse to automatically apply this fixture to all test functions. Add the following fixture to conftest.py to change the cwd for all tests: @pytest.fixture(autouse=True) def change_test_dir(request, monkeypatch): monkeypatch.chdir(request.fspath.dirname)   request is a built-in pytest fixture fspath is the LocalPath to the test module being executed dirname is the directory of the test module  Any processes that are kicked off by the test will use the test case folder as their working directory and copy their logs, outputs, etc. there, regardless of where the test suite was executed. Original Solution The following function-level fixture will change to the test case directory, run the test (yield), then change back to the calling directory to avoid side-effects, as suggested by @hi2meuk: @pytest.fixture def change_test_dir(request): os.chdir(request.fspath.dirname) yield os.chdir(request.config.invocation_params.dir)   request.config.invocation_params.dir - the folder from which pytest was executed request.config.rootdir - pytest root, doesn't change based on where you run pytest. Not used here, but could be useful.
Question: I have a series of tools running locally and on Jenkins to check and format my Python code:  autoflake  isort  black  I use pyproject.toml file to configure black, isort with .isort.cfg and autoflake with command line parameters because I haven't found any support to configure it with a configuration file. Is there way to configure also isort and autoflake with pyproject.toml? I would like to have all tools configured with just a single file.

Answer: isort configuration can be found at https://pycqa.github.io/isort/docs/configuration/options.html In general, config params are separated by underscores. The example below will provide configuration that makes black and isort compatible, as discussed here https://copdips.com/2020/04/making-isort-compatible-with-black.html [tool.isort] multi_line_output = 3 line_length = 88 include_trailing_comma = true [tool.black] line_length = 88