eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I keep getting this error: HTTPConnectionPool(host='127.0.0.1', port=8001): Max retries exceeded with url: /api/v1/auth/sign_in (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f8cbdd430>: Failed to establish a new connection: [Errno 111] Connection refused')) I searched through the stackoverflow and couldn't find the solution that would help me. Here's my code example: host = 'http://127.0.0.1:8001' response = requests.request(method=request_data['method'], url=f'{host}/{settings.ACCOUNTS_API_PREFIX}{request_data["url"]}', json=json_data, params=params, headers=headers, )  Basically I'm trying to send a POST request to authenticate myself on the service, however I keep getting the above error. I have 2 containers - one is a web application (Django), another one is accounts that stores all details of the users to authenticate them. Both containers are up and running, I can open the website, I can open the API swagger for accounts, however I can't send the POST request and get any response. Containers settings as follows:  container_1: build: context: ./container_1 dockerfile: Dockerfile env_file: - '.env' stdin_open: true tty: true ports: - '8000:8000' expose: - 8000 volumes: - ./data:/data working_dir: /data command: [ "./start.sh" ] networks: - web container_2: context: ./container_2 dockerfile: Dockerfile env_file: 'accounts/settings/.env' stdin_open: true tty: true environment: - 'DJANGO_SETTINGS_MODULE=project.settings' expose: - 8000 ports: - "8001:8000" volumes: - ./data:/app networks: - web  Can someone assist me to figure it out?

Answer: Answer of @jordanm was right and it fixed my problem:  Change host = 'http://127.0.0.1:8001' to host = 'http://container_2:8000'
Question: I am trying to find out what the internal load factor is for the Python sets. For dictionary which uses a hash table with a load factor of 0.66 (2/3) is. The number of buckets start at 8 and when the 6th key is inserted the number of buckets increases to 16 The table below shows the shift in buckets.     bucket shift     8 5   16 10   32 21   64 42   128 85     This can be seen with de following Python code where the size of a dictionary and sets is shows with the getsizeof method: import sys d = {} s = set() for x in range(25): d[x] = 1 s.add(x) print(len(d), sys.getsizeof(d), sys.getsizeof(s))      # of elements memory used for dict memory used for sets     1 232 216   2 232 216   3 232 216   4 232 216   5 232 728   6 360 728   7 360 728   8 360 728   9 360 728   10 360 728   11 640 728   12 640 728   13 640 728   14 640 728   15 640 728   16 640 728   17 640 728   18 640 728   19 640 2264   20 640 2264   21 640 2264   22 1176 2264   23 1176 2264   24 1176 2264   25 1176 2264     The above table shows that the shift in the buckets correct is for the dictionary, but not for the sets. The memory in sets is different. I am trying to find out what the load factor is for a set. Is that also 2/3? Or am I doing something wrong with the code?

Answer: Currently, it's about 3/5. See the source: if ((size_t)so->fill*5 < mask*3) return 0; return set_table_resize(so, so->used>50000 ? so->used*2 : so->used*4);  fill is the number of occupied table cells (including "deleted entry" markers), and mask is 1 less than the total table capacity.
Question: i am currently learning about the assert statement in python and i cant seem to understand its main usage and what seperates it from simply raising an exception. if i wrote an if statement along with my condition and simply raised an exception if the condition is not met, how is that different from using the assert statement? def times_ten(number): return number * 100 result = times_ten(20) assert result == 200, 'Expected times_ten(20) to return 200, instead got ' + str(result)  to me both codes do the same thing def times_ten(number): return number * 100 result = times_ten(20) if result != 200: raise Exception('Expected times_ten(20) to return 200, instead got ' + str(result)) 

Answer: Not much. The documentation provides the equivalent if statements to an assert statement. assert expression  is the same as if __debug__: if not expression: raise AssertionError()  while assert expression1, expression2  is the same as if __debug__: if not expression1: raise AssertionError(expression2)  As you can see, it's an AssertionError, not a generic Exception, that is raised when the condition is false, and there is a guard that can be set to False on startup (using the -O option) to prevent the assertion from being checked at all.
Question: I have dataframe in pandas :- purchase_df. I want to convert it to sql table so I can perform sql query in pandas. I tried this method purchase_df.to_sql('purchase_df', con=engine, if_exists='replace', index=False)  It throw an error TypeError: __init__() got multiple values for argument 'schema'  I have dataframe name purchase_df and I need to perform sql query on it. I need to perform sql query on this dataframe like this ....engine.execute('''select * from purchase_df where condition'''). For this I need to convert dataframe into sql table as in our server pandas_sql is not installed only sql alchemy is installed. I ran this code in pycharm locally and it work perfectly fine but when i tried this in databrick notebook it is showing an error. Even though week ago it was running fine in databrick notebook too. Help me to fix this. note:- pandas version '1.3.4' Name: SQLAlchemy Version: 2.0.0

Answer: It seems that the version 2.0.0 (realeased on January 26, 2023) of SQLAlchemy is not compatible with earlier versions of pandas. I suggest you to upgrade your pandas version to the latest (version 1.5.3) with : pip install --upgrade pandas  Or: conda upgrade pandas
Question: Input data: s = {'111': {'name': 'john', 'exp': '1'}, '222': {'name': 'mia', 'exp': '1'}}  Code: import jmespath jmespath.search("(*)[?name=='john']", s)  Output: [{'name': 'john', 'exp': '1'}]  Output I want: [{'111': {'name': 'john', 'exp': '1'}}] 

Answer: Convert the dictionary to the list l1 = [{'key': k, 'value': v} for k, v in s.items()]  gives [{'key': '111', 'value': {'name': 'john', 'exp': '1'}}, {'key': '222', 'value': {'name': 'mia', 'exp': '1'}}]  Select the values where the attribute name is john l2 = jmespath.search('[?value.name == `john`]', l1)  gives [{'key': '111', 'value': {'name': 'john', 'exp': '1'}}]  Convert the list back to the dictionary s2 = dict([[i['key'], i['value']] for i in l2])  gives the expected result {'111': {'name': 'john', 'exp': '1'}}    Example of complete code for testing #!/usr/bin/python3 import jmespath s = {'111': {'name': 'john', 'exp': '1'}, '222': {'name': 'mia', 'exp': '1'}} # '333': {'name': 'john', 'exp': '1'}} l1 = [{'key': k, 'value': v} for k, v in s.items()] print(l1) l2 = jmespath.search('[?value.name == `john`]', l1) print(l2) s2 = dict([[i['key'], i['value']] for i in l2]) print(s2)
Question: In my GitHub CI I get errors like the one below since today: File "/home/runner/.local/lib/python3.8/site-packages/fb4/login_bp.py", line 12, in <module> from fb4.sqldb import db File "/home/runner/.local/lib/python3.8/site-packages/fb4/sqldb.py", line 8, in <module> db = SQLAlchemy() File "/home/runner/.local/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 758, in __init__ _include_sqlalchemy(self, query_class) File "/home/runner/.local/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 112, in _include_sqlalchemy for key in module.__all__: AttributeError: module 'sqlalchemy' has no attribute '__all__' CRITICAL: Exiting due to uncaught exception <class 'ImportError'>  without being aware of any significant commit that might cause this. My local tests and my Jenkins CI still work. I changed the matrix to stick to python 3.8 instead of also trying 3.9, 3.10 and 3.11 also taking into account that a similar problem in python 3.9 AttributeError: module 'posix' has no attribute '__all__' was due to missing 3.9 support. How can the above error could be debugged and mitigated? My assumption is that the problem is in the setup/environment or some strange behaviour change of GitHub actions, Python, pip or the test environment or whatever. I am a committer of the projects involved which are:  https://github.com/WolfgangFahl/pyOnlineSpreadSheetEditing and potentially https://github.com/WolfgangFahl/pyFlaskBootstrap4  Update: After following the suggestions by @snakecharmerb the logs Now show a version conflict RROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts The conflict is caused by: The user requested Flask~=2.0.2 bootstrap-flask 1.8.0 depends on Flask flask-dropzone 1.6.0 depends on Flask flask-login 0.6.2 depends on Flask>=1.0.4 flask-httpauth 1.0.0 depends on Flask flask-sqlalchemy 3.0.2 depends on Flask>=2.2  Which is interesting since i am trying to avoid the ~ notation ... and indeed it was a typo ... let's see whether the fix to upgrade Flask-SQLAlchemy>=3.0.2 works now. I have accepted the answer after setting the version as suggested. There are followup problems but the question is answered.

Answer: It seems the .__all__ attribute has been removed in the recently released SQLAlchemy 2.0. You may need to pin the SQLAlchemy version in your config somehow. Or ensure that you are using Flask-SQLAlchemy 3.0.2 or later, as this issue suggests that version has the required fix.
Question: The z-label does not show up in my figure. What is wrong? import matplotlib.pyplot as plt fig = plt.figure() ax = fig.add_subplot(111, projection='3d') ax.set_xlabel("x") ax.set_ylabel("y") ax.set_zlabel("z") plt.show()  Output Neither ax.set_zlabel("z") nor ax.set(zlabel="z") works. The x- and y-labels work fine.

Answer: That's a padding issue.  labelpadfloat The distance between the axis label and the tick labels. Defaults to rcParams["axes.labelpad"] (default: 4.0) = 4.  You can use matplotlib.axis.ZAxis.labelpad to adjust this value for the z-axis : import matplotlib.pyplot as plt fig = plt.figure() ax = fig.add_subplot(111, projection="3d") ax.set_xlabel("x") ax.set_ylabel("y") ax.set_zlabel("StackOverflow", rotation=90) ax.zaxis.labelpad=-0.7 # <- change the value here plt.show();  Output :
Question: I am using PyPDF2 to take an input PDF of any paper size and convert it to a PDF of A4 size with the input PDF scaled and fit in the centre of the output pdf. Here's an example of an input (convert to pdf with imagemagick convert image.png input.pdf), which can be of any dimensions:  And the expected output is:  I'm not a developer and my knowledge of python is basic but I have been trying to figure this out from the documentation, but haven't had much success. My latest attempt is as follows: from pypdf import PdfReader, PdfWriter, Transformation, PageObject from pypdf import PaperSize pdf_reader = PdfReader("input.pdf") page = pdf_reader.pages[0] writer = PdfWriter() A4_w = PaperSize.A4.width A4_h = PaperSize.A4.height # resize page2 to fit *inside* A4 h = float(page.mediabox.height) w = float(page.mediabox.width) print(A4_h, h, A4_w, w) scale_factor = min(A4_h / h, A4_w / w) print(scale_factor) transform = Transformation().scale(scale_factor, scale_factor).translate(0, A4_h / 3) print(transform.ctm) # page.scale_by(scale_factor) page.add_transformation(transform) # merge the pages to fit inside A4 # prepare A4 blank page page_A4 = PageObject.create_blank_page(width=A4_w, height=A4_h) page_A4.merge_page(page) print(page_A4.mediabox) writer.add_page(page_A4) writer.write("output.pdf")  Which gives this output:  I could be completely off track with my approach and it may be the inefficient way of doing it. I was hoping I would have a simple function in the package where I can define the output paper size and the scaling factor, similar to this.

Answer: You almost got it! The transformations are applied only to the content, but not to the boxes (mediabox/trimbox/cropbox/artbox/bleedbox). You need to adjust the cropbox: from pypdf.generic import RectangleObject page.cropbox = RectangleObject((0, 0, A4_w, A4_h))  Full script from pypdf import PdfReader, PdfWriter, Transformation, PageObject, PaperSize from pypdf.generic import RectangleObject reader = PdfReader("input.pdf") page = reader.pages[0] writer = PdfWriter() A4_w = PaperSize.A4.width A4_h = PaperSize.A4.height # resize page to fit *inside* A4 h = float(page.mediabox.height) w = float(page.mediabox.width) scale_factor = min(A4_h/h, A4_w/w) transform = Transformation().scale(scale_factor,scale_factor).translate(0, A4_h/3) page.add_transformation(transform) page.cropbox = RectangleObject((0, 0, A4_w, A4_h)) # merge the pages to fit inside A4 # prepare A4 blank page page_A4 = PageObject.create_blank_page(width = A4_w, height = A4_h) page.mediabox = page_A4.mediabox page_A4.merge_page(page) writer.add_page(page_A4) writer.write('output.pdf')
Question: pre-commit suddenly started to fail installing the isort hook in our builds today with the following error [INFO] Installing environment for https://github.com/pycqa/isort. [INFO] Once installed this environment will be reused. [INFO] This may take a few minutes... An unexpected error has occurred: CalledProcessError: command: ('/builds/.../.cache/pre-commit/repo0_h0f938/py_env-python3.8/bin/python', '-mpip', 'install', '.') return code: 1 expected return code: 0 [...] stderr: ERROR: Command errored out with exit status 1: [...] File "/tmp/pip-build-env-_3j1398p/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 40, in prepare_metadata_for_build_wheel poetry = Factory().create_poetry(Path(".").resolve(), with_groups=False) File "/tmp/pip-build-env-_3j1398p/overlay/lib/python3.8/site-packages/poetry/core/factory.py", line 57, in create_poetry raise RuntimeError("The Poetry configuration is invalid:\n" + message) RuntimeError: The Poetry configuration is invalid: - [extras.pipfile_deprecated_finder.2] 'pip-shims<=0.3.4' does not match '^[a-zA-Z-_.0-9]+$'  It seems to be related with poetry configuration..

Answer: Upgrading the hook to the freshly released isort 5.12.0 seems to be fixing the issue. Looking at the commit stack from isort repo, it sounds like recent version of Poetry had a breaking change incompatible with isort <= 5.11.4 (commit)
Question: I'm trying to implement type hinting for object unpacking. Here is what I have currently from typing import Tuple class A: def __init__(self, x: int, y: str): self.x = x self.y = y def astuple(self) -> Tuple[int, str]: return self.x, self.y # Need to annotate the return type of __iter__ def __iter__(self): return iter(self.astuple()) a = A(1, "a") # This cannot infer the type of x and y x, y = a reveal_type(x) reveal_type(y) # This infers the type of p and q as int and str respectively p, q = a.astuple() reveal_type(p) reveal_type(q)  prints $ mypy unpack_object.py unpack_object.py:20: note: Revealed type is "Any" unpack_object.py:21: note: Revealed type is "Any" unpack_object.py:24: note: Revealed type is "builtins.int" unpack_object.py:25: note: Revealed type is "builtins.str" Success: no issues found in 1 source file  However, I would like mypy to infer correct types for x, y (int, str). How can I achieve this?

Answer: There is no way to define your own heterogeneous iterable type in Python. Make A a subclass of NamedTuple instead. from typing import NamedTuple class A(NamedTuple): x: int y: str x, y = A(1, "a") reveal_type(x) # builtins.int reveal_type(y) # builtins.str
Question: I was watching Learn Python - Full Course for Beginners [Tutorial] on YouTube here. At timestamp 4:11:54 the tutor explains what a class function is, however from my background in object oriented programming using other languages I thought the correct term would be method? Now I am curious if there is a difference between a class function and method?

Answer: Method is the correct term for a function in a class. Methods and functions are pretty similar to each other. The only difference is that a method is called with an object and has the possibility to modify data of an object. Functions can modify and return data but they dont have an impact on objects. Edit : Class function and method both mean the same thing although saying class function is not the right way to say it.
Question: I am trying to add text labels to the top of a grouped/dodged bar plot using seaborn.objects. Here is a basic dodged bar plot: import seaborn.objects as so import pandas as pd dat = pd.DataFrame({'group':['a','a','b','b'], 'x':['1','2','1','2'], 'y':[3,4,1,2]}) (so.Plot(dat, x = 'x', y = 'y', color = 'group') .add(so.Bar(),so.Dodge()))   I can add text labels to the top of a non-dodged bar plot using so.Text(), no problem. (so.Plot(dat.query('group == "a"'), x = 'x', y = 'y', text = 'group') .add(so.Bar()) .add(so.Text({'va':'bottom'})))   However, when I combine dodging with text, the bars shrink and move far apart. (so.Plot(dat.query('group == "a"'), x = 'x', y = 'y', text = 'group') .add(so.Bar()) .add(so.Text({'va':'bottom'})))   This looks worse the more categories there are - in my actual application the bars have thinned out to single lines. Setting the gap parameter of so.Dodge() or the width parameter of so.Bar() doesn't seem to be capable of solving the problem (although either will alleviate it slightly if I'm not too picky). I'm guessing that the bar plot is using the so.Dodge() settings appropriate for text in order to figure out its own dodging, but that doesn't seem to be working right. Note that reversing the order I .add() the geometries doesn't seem to do anything. How can I avoid this?

Answer: This isn't ideal but can be worked around by assigning text only in the layer with the Text (or un-assigning it from the Bar layer) and restricting the variables used to compute the text dodge: ( so.Plot(dat, x='x', y='y', color='group') .add(so.Bar(), so.Dodge()) .add(so.Text({'va':'bottom'}), so.Dodge(by=["color"]), text='group') )
Question: I am working with a data frame that is written in wide format. Each book has a number of sales, but some quarters have null values because the book was not released before that quarter. import pandas as pd data = {'Book Title': ['A Court of Thorns and Roses', 'Where the Crawdads Sing', 'Bad Blood', 'Atomic Habits'], 'Metric': ['Book Sales','Book Sales','Book Sales','Book Sales'], 'Q1 2022': [100000,0,0,0], 'Q2 2022': [50000,75000,0,35000], 'Q3 2022': [25000,150000,20000,45000], 'Q4 2022': [25000,20000,10000,65000]} df1 = pd.DataFrame(data)   What I would like to do is create one field that identifies "ID of first available quarter" ("First Quarter ID"), and another that identifies "ID of quarter with maximum sales" ("Max Quarter ID"). Then I would like to show two fields with the sales in the first available quarter and the second available quarter.  Tips to go about this? Thank you!

Answer: Edit, updated approach making better use of groupby after melting #melt table to be long-form long_df1 = df1.melt( id_vars = ['Book Title','Metric'], value_name = 'Sales', var_name = 'Quarter', ) #remove rows that have 0 sales (could be dropna if null values used instead) long_df1 = long_df1[long_df1['Sales'].gt(0)] #groupby book title and find the first/max quarter/sales gb = long_df1.groupby('Book Title') first_df = gb[['Quarter','Sales']].first() max_df = long_df1.loc[gb['Sales'].idxmax(),['Book Title','Quarter','Sales']].set_index('Book Title') #concatenate the first/max dfs out_df = pd.concat( (first_df.add_prefix('First '),max_df.add_prefix('Max ')), axis=1 ).reset_index()  Output
Question: I am trying to assign a higher weight to one feature above others. Here is my code. ## Assign weight to High Net Worth feature cols = list(train_X.columns.values) # 0 - 1163 --Other Columns # 1164 --High Net Worth #Create an array of feature weights other_col_wt = [1]*1164 high_net_worth_wt = [5] feature_wt = other_col_wt + high_net_worth_wt feature_weights = np.array(feature_wt) # Initialize the XGBClassifier xgboost = XGBClassifier(subsample = 0.8, # subsample = 0.8 ideal for big datasets silent=False, # whether print messages during construction colsample_bytree = 0.4, # subsample ratio of columns when constructing each tree gamma=10, # minimum loss reduction required to make a further partition on a leaf node of the tree, regularisation parameter objective='binary:logistic', eval_metric = ["auc"], feature_weights = feature_weights ) # Hypertuning parameters lr = [0.1,1] # learning_rate = shrinkage for updating the rules ne = [100] # n_estimators = number of boosting rounds md = [3,4,5] # max_depth = maximum tree depth for base learners # Grid Search clf = GridSearchCV(xgboost,{ 'learning_rate':lr, 'n_estimators':ne, 'max_depth':md },cv = 5,return_train_score = False) # Fitting the model with the custom weights clf.fit(train_X,train_y, feature_weights = feature_weights) clf.cv_results_  I went through the documentation here and this Akshay Sehgal's stackoverflow response here for a similar question. But when I use the above code, I get below error?  Could anyone please help me where I am doing it wrong? Thanks.

Answer: I think you need to remove feature_weights from the init of XGBClassifier. At least, this works when I try your example.
Question: Only when I add the property filter_action="native" in a dash.DataTable in order to make it possible for the user to filter rows by column values I get an error that varies with the browser I run the webapp on:  Edge: Cannot read properties of undefined (reading 'placeholder_text') (This error originated from the built-in JavaScript code that runs Dash apps. Click to see the full stack trace or open your browser's console.) TypeError: Cannot read properties of undefined (reading 'placeholder_text') at s.value (http://localhost:8050/_dash-component-suites/dash/dash_table/async-table.js:2:236716)... Chrome: same as Edge Firefox: r is undefined (This error originated from the built-in JavaScript code that runs Dash apps. Click to see the full stack trace or open your browser's console.) value@http://localhost:8050/_dash-component-suites/dash/dash_table/async-table.js:2:236702...  Note that without setting that single property the app works perfectly. I terribly need the user to be able to filter rows by column values: what can I do to solve this issue? environment python 3.7, Flask 2.2.2, dash 2.8.0

Answer: I found a possible solution: that is giving also the columns attribute during DataTable declaration (no, giving just data is not sufficient): columns=[{"name": i, 'id': i} for i in df.columns]  The minimal code for making a DataTable with working filtering of rows based on column values is dash_table.DataTable( id='my_table_id', columns=[ {"name": i, 'id': i} for i in df.columns ], data=df.to_dict('records'), filter_action="native", )