eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm having trouble understanding the output of a piece of python code. mani=[] nima=[] for i in range(3) nima.append(i) mani.append(nima) print(mani)  The output is [[0,1,2], [0,1,2], [0,1,2]]  I can't for the life of me understand why it is not [[0], [0,1], [0,1,2]]  Any help much appreciated.

Answer: It's because when you append nima into mani, it isn't a copy of nima, but a reference to nima. So as nima changes, the reference at each location in mani, just points to the changed nima. Since nima ends up as [0, 1, 2], then each reference appended into mani, just refers to the same object.
Question: The Keras documentation of ImageDataGenerator class says—  width_shift_range: Float, 1-D array-like or int - float: fraction of total width, if < 1, or pixels if >= 1. - 1-D array-like: random elements from the array. - int: integer number of pixels from interval (-width_shift_range, +width_shift_range) - With width_shift_range=2 possible values are integers [-1, 0, +1], same as with width_shift_range=[-1, 0, +1], while with width_shift_range=1.0 possible values are floats in the interval [-1.0, +1.0).   height_shift_range: Float, 1-D array-like or int - float: fraction of total height, if < 1, or pixels if >= 1. - 1-D array-like: random elements from the array. - int: integer number of pixels from interval (-height_shift_range, +height_shift_range) - With height_shift_range=2 possible values are integers [-1, 0, +1], same as with height_shift_range=[-1, 0, +1], while with height_shift_range=1.0 possible values are floats in the interval [-1.0, +1.0).  I’m new in Keras and machine learning, and I just have started learning it. I am struggling to understand the documentation and use of these two arguments of Keras ImageDataGenerator class, named width_shift_range and height_shift_range. I have searched out a lot, but couldn't find any good documentation other than the official. What exactly do these two arguments do? When have to use them? This talk may seem inappropriate here, but since there is no discussion anywhere on the internet, I think it would be nice to have the discussion here. If anyone helps me understanding these, I would be grateful. Thank you very much.

Answer: These two argument used by ImageDataGenerator class Which use to preprocess image before feeding it into network. If you want to make your model more robust then small amount of data is not enough. That is where data augmentation come in handy. This are used to generate random data. width_shift_range: It actually shift the image to the left or right(horizontal shifts). If the value is float and <=1 it will take the percentage of total width as range. Suppose image width is 100px. if width_shift_range = 1.0 it will take -100% to +100% means -100px to +100px. It will shift image randomly between this range. Randomly selected positive value will shift the image to the right side and negative value will shift the image to the left side. We can also do this by selecting pixels. if we set width_shift_range = 100 it will have the same effect. More importantly integer value>=1 count pixel as range and float value<=1 count percentage of total width as range. Below images are for width_shift_range = 1.0.  height_shift_range: It works same as width_shift_range but shift vertically(up or down). Below images are for height_shift_range=0.2,fill_mode="constant"  fill_mode: It sets rules for newly shifted pixel in the input area. ## fill_mode: One of {"constant", "nearest", "reflect" or "wrap"}. ## Points outside the boundaries of the input are filled according to the given mode: ## "constant": kkkkkkkk|abcd|kkkkkkkk (cval=k) ## "nearest": aaaaaaaa|abcd|dddddddd ## "reflect": abcddcba|abcd|dcbaabcd ## "wrap": abcdabcd|abcd|abcdabcd  For more you can check this blog
Question: I am facing an issue where the sklearn set_config is failing. I am using Google Colab and also it is failing on Jupyter Notebook as well. Even the code when copied from https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_0_23_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-0-23-0-py also failing with same error. from sklearn import set_config set_config(display='diagram')  Error: TypeError: set_config() got an unexpected keyword argument 'display' Please suggest how to resolve this.

Answer: You need to upgrade scikit-learn on colab, its version is 'v0.22.2.post1', while display parameter in set_config() function was introduced in v0.23. !pip install --upgrade scikit-learn  Then, restart the runtime, display should work now.
Question: What's the difference? docs show nothing on this, and their help() is identical. Is there an object for which isinstance will fail with one but not other?

Answer: Back in 1994 I wasn't sure that we would always be using the same implementation type for lambda and def. That's all there is to it. It would be a pain to remove it, so we're just leaving it (it's only one line). If you want to add a note to the docs, feel free to submit a PR.
Question: I want to plot two charts side by side (and not one above the other) in Dash by Plotly. The tutorials did not have an example where the graphs are plotted side by side. I am writing the app.layout in the following way app.layout = html.Div(className = 'row', children= [ html.H1("Tips database analysis (First dashboard)"), dcc.Dropdown(id='d', options = col_options, value = 'Sun'), dcc.Graph(id="graph1"), dcc.Graph(id="graph2") ] )  but after this graph1 appears above graph2 rather than side by side

Answer: You can achieve this by wrapping the graphs in a div and adding display: inline-block css property to each of the graphs.  app.layout = html.Div(className='row', children=[ html.H1("Tips database analysis (First dashboard)"), dcc.Dropdown(), html.Div(children=[ dcc.Graph(id="graph1", style={'display': 'inline-block'}), dcc.Graph(id="graph2", style={'display': 'inline-block'}) ]) ])  Result EDIT I removed the display: flex property from the wrapping div, and instead added the display: inline-block property to each of the graphs. This makes it so the second graph does not get cut off, and will instead stack on smaller screens.
Question: I am plotting a dataframe that looks like this. Date 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 Date 01 Jan 12.896 13.353 12.959 13.011 13.073 12.721 12.643 12.484 12.876 13.102 02 Jan 12.915 13.421 12.961 13.103 13.125 12.806 12.644 12.600 12.956 13.075 03 Jan 12.926 13.379 13.012 13.116 13.112 12.790 12.713 12.634 12.959 13.176 04 Jan 13.051 13.414 13.045 13.219 13.051 12.829 12.954 12.724 13.047 13.187 05 Jan 13.176 13.417 13.065 13.148 13.115 12.874 12.956 12.834 13.098 13.123  The code for plotting is here. ice_data_dates.plot(figsize=(20,12), title='Arctic Sea Ice Extent', lw=3, fontsize=16, ax=ax, grid=True)  This plots a line plot for each of the years listed in the dataframe over each day in the year. However, I would like to make the line for 2020 much thicker than the others so it stands out more clearly. Is there a way to do that using this one line of code? Or do I need to manually plot all of the years such that I can control the thickness of each line separately? A current picture is attached, where the line thicknesses are all the same. 

Answer: You can iterate over the lines in the plot, which can be retrieved with ax.get_lines, and increase the width using set_linewidth if its label matches the value of interest: fig, ax = plt.subplots() df.plot(figsize=(20,12), title='Arctic Sea Ice Extent', lw=3, fontsize=16, ax=ax, grid=True) for line in ax.get_lines(): if line.get_label() == '2020': line.set_linewidth(15) plt.show()
Question: this is the settings.json file code  { "python.autoComplete.addBrackets": true, "python.linting.enabled": true, "python.pythonPath": "C:\\Program Files\\Python37\\python.exe", "python.jediEnabled": false, "python.languageServer": "Microsoft" }  in this "python.jediEnabled": false, showing error that Unknown Configuration Setting pleaase give the solution

Answer: With vscode-python's release on June 16th 2020 they removed the python.jediEnabled setting in favor for the python.languageServer setting. From the changelog:  Removed python.jediEnabled setting in favor of python.languageServer. Instead of "python.jediEnabled": true please use "python.languageServer": "Jedi". (#7010)
Question: Imagine one has two SQL tables objects_stock id | number  and objects_prop id | obj_id | color | weight  that should be joined on objects_stock.id=objects_prop.obj_id, hence the plain SQL-query reads select * from objects_prop join objects_stock on objects_stock.id = objects_prop.obj_id;  How can this query be performed with SqlAlchemy such that all returned columns of this join are accessible? When I execute query = session.query(ObjectsStock).join(ObjectsProp, ObjectsStock.id == ObjectsProp.obj_id) results = query.all()  with ObjectsStock and ObjectsProp the appropriate mapped classes, the list results contains objects of type ObjectsStock - why is that? What would be the correct SqlAlchemy-query to get access to all fields corresponding to the columns of both tables?

Answer: Just in case someone encounters a similar problem: the best way I have found so far is listing the columns to fetch explicitly,  query = session.query(ObjectsStock.id, ObjectsStock.number, ObjectsProp.color, ObjectsProp.weight).\ select_from(ObjectsStock).join(ObjectsProp, ObjectsStock.id == ObjectsProp.obj_id) results = query.all()  Then one can iterate over the results and access the properties by their original column names, e.g. for r in results: print(r.id, r.color, r.number)
Question: I want my bot to play a specific song when typing +playtest using already defined function (+play) but i got an error says   "Discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: 'Command' object is not callable"  an entire code work perfectly fine except for this command i wonder does ctx.invoke enable passing arguments? or i just missed something  here is my brief code import discord import wavelink from discord.ext import commands import asyncio from bs4 import BeautifulSoup import requests import datetime queue = [] class Bot(commands.Bot): def __init__(self): super(Bot, self).__init__(command_prefix=['+']) self.add_cog(Music(self)) async def on_ready(self): print(f'Logged in as {self.user.name} | {self.user.id}') class Music(commands.Cog): def __init__(self, bot): self.bot = bot if not hasattr(bot, 'wavelink'): self.bot.wavelink = wavelink.Client(bot=self.bot) self.bot.loop.create_task(self.start_nodes()) self.bot.remove_command("help") async def start_nodes(self): await self.bot.wait_until_ready() await self.bot.wavelink.initiate_node(host='127.0.0.1', port=80, rest_uri='http://127.0.0.1:80', password='testing', identifier='TEST', region='us_central') @commands.command(name='connect') async def connect_(self, ctx, *, channel: discord.VoiceChannel = None): @commands.command() async def help(self, ctx): @commands.command() async def play(self, ctx, *, query: str): @commands.command(aliases=['sc']) async def soundcloud(self, ctx, *, query: str): @commands.command() async def leave(self, ctx): @commands.command(aliases=['queue', 'q']) async def check_queue(self, ctx): @commands.command(aliases=['clearq', 'clearqueue']) async def clear_queue(self, ctx): @commands.command(aliases=['removequeue', 'removeq', 'req']) async def remove_queue(self, ctx, num: int): @commands.command() async def skip(self, ctx): @commands.command(aliases=['eq']) async def equalizer(self, ctx: commands.Context, *, equalizer: str): @commands.command() async def playtest(self,ctx): await ctx.invoke(self.play('hi')) bot = Bot() bot.run('sd') 

Answer: ctx.invoke does allow passing arguments, but they need to be handled in a different way to how you may be used to ( function(params) ) The parameters must be explicitly shown in the invoke (e.g. param = 'value') and the command must be a command object. This would be how you could invoke a command: @commands.command() async def playtest(self, ctx): await ctx.invoke(self.bot.get_command('play'), query='hi')
Question: Currently I am using the following code to make replacements which is a little cumbersome: df1['CompanyA'] = df1['CompanyA'].str.replace('.','') df1['CompanyA'] = df1['CompanyA'].str.replace('-','') df1['CompanyA'] = df1['CompanyA'].str.replace(',','') df1['CompanyA'] = df1['CompanyA'].str.replace('ltd','limited') df1['CompanyA'] = df1['CompanyA'].str.replace('&','and') df1['Address1A'] = df1['Address1A'].str.replace('.','') df1['Address1A'] = df1['Address1A'].str.replace('-','') df1['Address1A'] = df1['Address1A'].str.replace('&','and') df1['Address1A'].str.replace(r'\brd\b', 'road') df1['Address2A'] = df1['Address2A'].str.replace('.','') df1['Address2A'] = df1['Address2A'].str.replace('-','') df1['Address2A'] = df1['Address2A'].str.replace('&','and') df1['Address2A'].str.replace(r'\brd\b', 'road')  In order to make changing on the fly easier my ideal scenario would be something like: df1['CompanyA'] = df1['CompanyA'].str.replace(('&','and'), ('.', ''), ('-','')....) df1['Address1A'] = df1['Address1A'].str.replace(('&','and'), ('.', ''), ('-','')....) df1['Address2A'] = df1['Address2A'].str.replace(('&','and'), ('.', ''), ('-','')....)  This is so I could just input/change what I wanted to replace for a particular column without having to adjust multiple lines of code. Is this possible at all?

Answer: You can create a dictionary and pass it to the function replace() without needing to chain or name the function so many times. replacers = {',':'','.':'','-':'','ltd':'limited'} #etc.... df1['CompanyA'] = df1['CompanyA'].replace(replacers)
Question: Is it possible to reshape a np.array() and, in case of inconsistency of the new shape, fill the empty spaces with NaN? Ex: arr = np.array([1,2,3,4,5,6])  Target, for instance a 2x4 Matrix: [1 2 3 4] [5 6 NaN NaN]  I need this to bypass the error: ValueError: cannot reshape array of size 6 into shape (2,4)

Answer: We'll use np.pad first, then reshape: m, n = 2, 4 np.pad(arr.astype(float), (0, m*n - arr.size), mode='constant', constant_values=np.nan).reshape(m,n) array([[ 1., 2., 3., 4.], [ 5., 6., nan, nan]])  The assumption here is that arr is a 1D array. Add an assertion before this code to fail on unexpected cases.
Question: How do you get the name of a property in python? Any suggestions welcome. For functions and methods it is as simple as f.__name__. But properties do not have the __name__ attribute.

Answer: The property does not have a name, but you are probably really looking for the name of its fget attribute, which (ignoring any shuffling done after the fact) will be the name of the class attribute to which the property instance is bound. class A: @property def foo(self): return 3 assert A.foo.fget.__name__ == "foo"
Question: It is stated that strings are immutable objects, and when we make changes in that variable it actually creates a new string object. So I wanted to test this phenomenon with this piece of code: result_str = "" print("string 1 (unedited):", id(result_str)) for a in range(1,11): result_str = result_str + str(a) print(f"string {a+1}:", id(result_str))  And I got the following IDs: string 1 (unedited): 2386354993840 string 2: 2386357170336 string 3: 2386357170336 string 4: 2386357170336 string 5: 2386357170336 string 6: 2386357170336 string 7: 2386357170336 string 8: 2386357170336 string 9: 2386360410800 string 10: 2386360410800 string 11: 2386360410800  So, if each string is different from each other, then why do the strings 2-8 and 9-11 have the same ID? And, if somehow this question is explained why does the ID change at string 9 specifically?

Answer: The string you associate with result_str create reaches end of lifetime at the next assignment. Hence the possibility of duplicate id. Here's the doc  Return the “identity” of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value.
Question: The below prints False. Is this not how mocking works? I tried changing the path to the function, but it errors out, so the path seems correct. What am I missing? import pytest from deals.services.services import is_user_valid class TestApi: def test_api(self, mocker): mocker.patch('deals.services.services.is_user_valid', return_value=True) print(is_user_valid("sdfds", "sdfsdf")) 

Answer: The issue here is that you're essentially doing the following: from deals.services.services import is_user_valid import deals.services.services deals.services.services.is_user_valid = Mock(return_value=True) # call local is_user_valid  By importing the "terminal" symbol itself you've shorted any possibility of mocking, it's now a local reference, and so updating the "remote" reference will have no effect on the local version. Meaning you should keep a handle on the module itself, such that the relevant symbol gets resolved on each access: from deals.services import services def test_api(mocker): mocker.patch('deals.services.services.is_user_valid', return_value=True) print(services.is_user_valid("sdfds", "sdfsdf"))  should work better. This is also an issue with any module using such imports, they requiring patching the point of use rather than the point of definition because by the time the mock runs chances are the user module already has their copy. See the documentation for some more details.
Question: I have the following df  list_columns = ['A', 'B', 'C'] list_data = [ [1, '2', 3], [4, '4', 5], [1, '2', 3], [4, '4', 6] ] df = pd.DataFrame(columns=list_columns, data=list_data)  I want to check if multiple columns exist, and if not to create them.  Example: If B,C,D do not exist, create them(For the above df it will create only D column) I know how to do this with one column: if 'D' not in df: df['D']=0  Is there a way to test if all my columns exist, and if not create the one that are missing? And not to make an if for each column

Answer: Here loop is not necessary - use DataFrame.reindex with Index.union: cols = ['B','C','D'] df = df.reindex(df.columns.union(cols, sort=False), axis=1, fill_value=0) print (df) A B C D 0 1 2 3 0 1 4 4 5 0 2 1 2 3 0 3 4 4 6 0
Question: The following code: import concurrent.futures def worker(item, truefalse): print(item, truefalse) return item processed = [] with concurrent.futures.ThreadPoolExecutor() as pool: for res in pool.map(worker, [1,2,3], False): processed.append(res)  Yields an exception: TypeError: zip argument #2 must support iteration I also tried: for res in pool.map(worker, ([1,2,3], False)): Which yields: TypeError: worker() missing 1 required positional argument: 'truefalse' How does one pass multiple arguments to the function in calling ThreadPoolExecutor.map()?

Answer: If you're trying to call the worker function with 1, False, then 2, False, then 3, False, you need to extend your single False to an iterable of Falses at least as long as the other iterable. Two approaches that work:  Multiply a sequence: for res in pool.map(worker, [1,2,3], [False] * 3):  Use itertools.repeat to make it as long as needed (map will stop when the shortest iterable is exhausted). Add from itertools import repeat to the top of the file, then use: for res in pool.map(worker, [1,2,3], repeat(False)):   For the record, this is also how you'd do this with regular map, not just ThreadPoolExecutor.
Question: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\Anirudh\Documents\flask_app\connecting_to_database\application.py", line 2, in <module> from flask_sqlalchemy import SQLAlchemy File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\flask_sqlalchemy\__init__.py", line 18, in <module> import sqlalchemy File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\sqlalchemy\__init__.py", line 9, in <module> from .sql import ( File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\sqlalchemy\sql\__init__.py", line 8, in <module> from .expression import ( File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\sqlalchemy\sql\expression.py", line 34, in <module> from .visitors import Visitable File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\sqlalchemy\sql\visitors.py", line 28, in <module> from .. import util File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\sqlalchemy\util\__init__.py", line 8, in <module> from .compat import callable, cmp, reduce, \ File "C:\Users\Anirudh\AppData\Local\Programs\Python\Python38\lib\site-packages\sqlalchemy\util\compat.py", line 234, in <module> time_func = time.clock AttributeError: module 'time' has no attribute 'clock' 

Answer: The error occurs because in python 2, there is time.clock(), but in python 3, it has been replaced with time.perf_counter().  Just replace all the time.clock to time.perf_counter, and it should be fine. For more info: https://www.webucator.com/blog/2015/08/python-clocks-explained/
Question: I want to print a very very close-to-one float, truncating it to 2 decimal places without rounding, preferably with the least amount of code possible. a = 0.99999999999 print(f'{a:0.2f}')  Expected: 0.99 Actual: 1.00 

Answer: I don't think you need f-strings or math functions, if I understand you correctly. Plain old string manipulation should get you there: a = 0.987654321 print(str(a)[:4])  output: 0.98
Question: I am new to Pytest in python . I am facing a tricky scenario where I need to test for exit codes - exit(1) and exit(0) , using Pytest module. Below is the python program :  def sample_script(): count_file = 0 if count_file == 0: print("The count of files is zero") exit(1) else: print("File are present") exit(0)  Now I want to test the above program for exit codes, exit(1) and exit(0) . Using Pytest how we can frame the test code so that we can test or asset the exit code of the function sample_script ? Please help me.

Answer: Once you put the exit(1) inside the if block as suggested, you can test for SystemExit exception: from some_package import sample_script def test_exit(): with pytest.raises(SystemExit) as pytest_wrapped_e: sample_script() assert pytest_wrapped_e.type == SystemExit assert pytest_wrapped_e.value.code == 42  The example is taken from here: https://medium.com/python-pandemonium/testing-sys-exit-with-pytest-10c6e5f7726f UPDATE: Here's a complete working example you can copy/paste to test: import pytest def sample_func(): exit(1) def test_exit(): with pytest.raises(SystemExit) as e: sample_func() assert e.type == SystemExit assert e.value.code == 1 if __name__ == '__main__': test_exit()