eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Understanding that the below are not true constants, attempting to follow PEP 8 I'd like to make a "constant" in my @dataclass in Python 3.7. @dataclass class MyClass: data: DataFrame SEED = 8675309 # Jenny's Constant  My code used to be: class MyClass: SEED = 8675309 # Jenny's Constant def __init__(data): self.data = data  Are these two equivalent in there handling of the seed? Is the seed now part of the init/eq/hash? Is there a preferred style for these constants?

Answer: They are the same. dataclass ignores unannotated variables when determining what to use to generate __init__ et al. SEED is just an unhinted class attribute. If you want to provide a type hint for a class attribute, you use typing.ClassVar to specify the type, so that dataclass won't mistake it for an instance attribute. from dataclasses import dataclass from typing import ClassVar @dataclass class MyClass: data: DataFrame SEED: ClassVar[int] = 8675309
Question: The error code D401 for pydocstyle reads: First line should be in imperative mood. I often run into cases where I write a docstring, have this error thrown by my linter, and rewrite it -- but the two docstrings are semantically identical. Why is it important to have imperative mood for docstrings?

Answer: From the docstring of check_imperative_mood itself:   """D401: First line should be in imperative mood: 'Do', not 'Does'. [Docstring] prescribes the function or method's effect as a command: ("Do this", "Return that"), not as a description; e.g. don't write "Returns the pathname ...".   (We'll ignore the irony that this docstring itself would fail the test.)
Question: I have Python Enum class like this: from enum import Enum class Seniority(Enum): Intern = "Intern" Junior_Engineer = "Junior Engineer" Medior_Engineer = "Medior Engineer" Senior_Engineer = "Senior Engineer"  In MYSQL database, seniority ENUM column has values "Intern", "Junior Engineer", "Medior Engineer", "Senior Engineer". The problem is that I get an error: LookupError: "Junior Engineer" is not among the defined enum values  This error has occurred when I call query like: UserProperty.query.filter_by(full_name='John Doe').first()  seniority is enum property in the UserProperty model. class UserProperty(db.Model): ... seniority = db.Column(db.Enum(Seniority), nullable=True) ...  For this class I've defined Schema class using marshmallow Schema and EnumField from marshmallow_enum package: class UserPropertySchema(Schema): ... seniority = EnumField(Seniority, by_value=True) ...  What to do in this situation, because I can't define python class property name with space. How to force python to use values of defined properties instead of property names?

Answer: As Shenanigator stated in the comment of my question, we can use aliases to solve this problem. Seniority = Enum( value='Seniority', names=[ ('Intern', 'Intern'), ('Junior Engineer', 'Junior Engineer'), ('Junior_Engineer', 'Junior_Engineer'), ('Medior Engineer', 'Medior Engineer'), ('Medior_Engineer', 'Medior_Engineer'), ('Senior Engineer', 'Senior Engineer'), ('Senior_Engineer', 'Senior_Engineer') ] )
Question: I have a project that is fully annotated. Or at least I hope so, because it is entirely possible that there is a function or two somewhere in there that is missing type annotations. How can I find such functions (or any other blocks of code)?

Answer: You can use mypy for this. Just add some switches to the command call: $ mypy --disallow-untyped-calls --disallow-untyped-defs --disallow-incomplete-defs projectname  This will find you all untyped defines plus incomplete defines and also warns you if you call a untyped function. For further information have a look at the untyped-definitions-and-calls section of the mypy documentation.
Question: I have a class: class Holiday(ActivitiesBaseClass): """Holiday is an activity that involves taking time off work""" Hotel = models.CharField(max_length=255) """ Name of hotel """  I can print the class docstring by typing: print(Holiday.__doc__)  This outputs as: Holiday is an activity that involves taking time off work  How can I print the docstring for Hotel which is a Class attribute/element?  What I've tried print(Holiday.Hotel.__doc__)  returns:  A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.  I've also tried the help() function to no avail. N.B Sphinx seems to be able to extract the docstrings of class attributes and include them in readthedocs so I’m hoping there’s a way

Answer: Don't think it's possible to add docstring for specific field, but you can use field's help_text argument instead: Hotel = models.CharField(max_length=255, help_text="Name of hotel")
Question: The code I'm working on throws the error Unsupported operand type(s) for +: 'WindowsPath' and 'str'. I have tried many things, and none have fixed this (aside from removing the line with the error, but that's not helpful). For context, this script (when it's done) is supposed to:  Find a file (mp3) based on the ID you type in (in the directory specified in SongsPath.txt) Back it up Then replace it with another file (renamed to the previous file's name)  so that the program that fetches these files plays the new song instead of the old one, but can be restored to the original song at any time. (the songs are downloaded from newgrounds and saved by their newgrounds audio portal ID) I'm using Python 3.6.5 import os import pathlib from pathlib import Path nspt = open ("NewSongsPath.txt", "rt") nsp = Path (nspt.read()) spt = open("SongsPath.txt", "rt") sp = (Path(spt.read())) print("type the song ID:") ID = input() csp = str(path sp + "/" + ID + ".mp3") # this is the line throwing the error. sr = open(csp , "rb") sw = open(csp, "wb") print (sr.read()) 

Answer: What is happening is that you are using the "+" character to concatenate 2 different types of data Instead of using the error line: csp = str(path sp + "/" + ID + ".mp3")  Try to use it this way: csp = str(Path(sp)) fullpath = csp + "/" + ID + ".mp3"  Use the 'fullpath' variable to open file.
Question: I've got several virtual environments (dozens) lying on my disk made by the venv module of Python 3.6. Now I've upgraded to Ubuntu 19.10 in a haste and only afterwards noticed that 3.6 is not available at all for Ubuntu 19.10 from the generally acknowledged sources. I've managed to upgrade the Python versions of these virtual environments by locating bin/python3 under my home directory and running python3.7 -mvenv --upgrade on the containing folders.  Now, while python3.7 -mvenv --upgrade upgrades the Python in the virtual environment, it does nothing to reinstall my previous package versions in the lib/python3.7/site-packages under that venv. I guess I could have done this by installing Python 3.6, pip freezeing the requirements from the venv and then upgrading the venv to Python 3.7, pip install -ring - if only there was Python 3.6 install available for my new OS. Is there any other way to do this in a rather automated fashion (perhaps mainly the pip freeze using the old lib/python3.6 directory) without me having to install Python 3.6 from source, using conda or installing 3.6 from some random PPAs? I want to upgrade all environments en masse so that in the future when I need to do something with a random environment it would continue working with Python 3.7.

Answer: In your new 3.7 venv you should have pkg_resources available - setuptools is automatically installed when created. If not, just pip install setuptools. setuptools library code is actually what pip is vendoring to make pip freeze work. But you can just freeze it manually. # in 3.7 runtime... import pkg_resources old_site_dir = ".venv/lib/python3.6/site-packages/" working_set = pkg_resources.WorkingSet([old_site_dir]) for dist in working_set: print(dist.as_requirement())  You can throw that output in a requirements.txt file and likely have a working reconstructed site, no python3.6 runtime required. Note that this method may not be 100% foolproof, because it is possible for projects to declare separate dependency trees for python3.6 and python3.7 by using environment markers in their distribution metadata (see PEP 508). It is also possible that items installed in your 3.6 site do not support 3.7 at all. However it is pretty uncommon to see that in a minor version bump between 3.6 and 3.7, so just using the working set should be "good enough" in practice.
Question: I'm trying to write multiple lines of a string to a text file in Python3, but it only writes the single line. e.g Let's say printing my string returns this in the console; >> print(mylongstring) https://www.link1.com https://www.link2.com https://www.link3.com https://www.link4.com https://www.link5.com https://www.link6.com  And i go to write that into a text file f = open("temporary.txt","w+") f.write(mylongstring)  All that reads in my text file is the first link (link1.com) Any help? I can elaborate more if you want, it's my first post here after all.

Answer: Try closing the file: f = open("temporary.txt","w+") f.write(mylongstring) f.close()  If that doesn't work try using: f = open("temporary.txt","w+") f.writelines(mylongstring) f.close()  If that still doesn't work use: f = open("temporary.txt","w+") f.writelines([i + '\n' for i in mylongstring]) f.close()
Question: have a df with values df: 165 156 1 test greater 56gsa ------------------------------------- spin 201 2 normal lesser 12asgs pine 202 3 fast greater 5sasgs  required output: 0 1 2 3 4 5 ------------------------------------- 165 156 1 test greater 56gsa spin 201 2 normal lesser 12asgs pine 202 3 fast greater 5sasgs 

Answer: If DataFrame is created from file then header=None parameter is your friend: df = pd.read_csv(file, header=None)  pandas >= 2.0: If not then convert column to one row DataFrame and concatenate (append not working anymore) to original data: df = pd.concat([df.columns.to_frame().T, df]) df.columns = range(len(df.columns)) print (df) 0 1 2 3 4 5 0 165 156 1 test greater 56gsa 1 spin 201 2 normal lesser 12asgs 2 pine 202 3 fast greater 5sasgs  pandas < 2.0 (original answer): If not then convert column to one row DataFrame and DataFrame.append to original data: df = df.columns.to_frame().T.append(df, ignore_index=True) df.columns = range(len(df.columns)) print (df) 0 1 2 3 4 5 0 165 156 1 test greater 56gsa 1 spin 201 2 normal lesser 12asgs 2 pine 202 3 fast greater 5sasgs
Question: I get a case that when I tried to use np.max() in an empty numpy array it will report such error messages. # values is an empty numpy array here max_val = np.max(values)   ValueError: zero-size array to reduction operation maximum which has no identity  So the way I think to fix it is that I try to deal with the empty numpy array first before calling the np.max() like follows: # add some values as missing values on purposes. def deal_empty_np_array(a:np.array): if a.size == 0: a = np.append(a, [-999999, -999999]) return a values = deal_empty_np_array(values) max_val = np.max(values);  OR use the try catch way like this link.  So I am wondering if there is a better solution for this awkward case. Thanks in advance. PS: Sorry for not giving a clean description before.

Answer: In [3]: np.max([]) --------------------------------------------------------------------------- ... ValueError: zero-size array to reduction operation maximum which has no identity  But check the docs. In newer numpy ufunc like max take an initial parameter that lets you work with an empty array: In [4]: np.max([], initial=10) Out[4]: 10.0
Question: When present, what is the order that these elements should be declared in a Python module?  Hash bang (#!/usr/bin/env python) Encoding (# coding: utf-8) Future imports (from __future__ import unicode_literals, ...) Docstring  If declared last, will the docstring work in a call help(module)?

Answer:  Hash bang. The kernel literally looks at the first two bytes of the file to see if they are equal to #!, so it won't work otherwise.  Encoding. According to the Python Language Reference it must be "on the first or second line".  Docstring. According to PEP 257, a docstring is "a string literal that occurs as the first statement in a module, function, class, or method definition", so it cannot go after any import statements. You can see for yourself that help(module) no longer reports your docstring if you put it in a different place.  Future imports, because they cannot go before any of the above.
Question: Other similar answers are out of date or focus on a particular error and not the whole process. What is the full installation process of mod_wsgi into an Apache installation on Windows 10?

Answer:  Install Microsoft Visual C++ Build Tools: https://visualstudio.microsoft.com/visual-cpp-build-tools/  Point MOD_WSGI_APACHE_ROOTDIR to your installation (default is C:/Apache24). Use forward slashes: set MOD_WSGI_APACHE_ROOTDIR=C:/Users/me/apache  For PowerShell this would be (with backward slashes): $env:MOD_WSGI_APACHE_ROOTDIR="C:\Users\me\apache"   Install mod-wsgi package: pip install mod-wsgi  Note: Make sure that the python version you're using has the same architecture (32/64 bit) as your Apache version.  Get module configuration: mod_wsgi-express module-config   Copy the output of the previous command into your Apache's httpd.conf.   When you restart Apache, mod_wsgi will be loaded. Check out the quickstart Hello World example to test your mod_wsgi installation.
Question: As the question says, I have a series of strings like '2020-01-06T00:00:00.000Z'. How can I convert this series to datetime using Python? I prefer the method on pandas. If not is there any method to solve this task? Thank all. string '2020-01-06T00:00:00.000Z' convert to 2020-01-06 00:00:00 under datetime object 

Answer: With Python 3.7+, that can be achieved with datetime.fromisoformat() and some tweaking of the source string: from datetime import datetime datetime.fromisoformat('2020-01-06T00:00:00.000Z'[:-1] + '+00:00')  Output: datetime.datetime(2020, 1, 6, 0, 0, tzinfo=datetime.timezone.utc) And here is a more Pythonic way to achieve the same result: from datetime import datetime from datetime import timezone datetime.fromisoformat( '2020-01-06T00:00:00.000Z'[:-1] ).astimezone(timezone.utc)  Output: datetime.datetime(2020, 1, 6, 3, 0, tzinfo=datetime.timezone.utc) Finally, to format it as %Y-%m-%d %H:%M:%S, you can do: d = datetime.fromisoformat( '2020-01-06T00:00:00.000Z'[:-1] ).astimezone(timezone.utc) d.strftime('%Y-%m-%d %H:%M:%S')  Output: '2020-01-06 00:00:00'
Question: I am using Visual Studio Code (VSC) as my IDE. My computer just updated to Catalina 10.15.2 (19C57) and since the update, VSCode is no longer highlighting syntax errors. The extensions I have seem to be working and it recognizes my miniconda Python environment. Is there a solution for this yet? I was avoiding Catalina as I know it has caused a lot of issues but now that I was forced to install it I need a solution.  

Answer: In my case, the Catalina installation didn't remove my Python installation. After checking as suggested by @Brett Cannon in his comment, the update to Catalina uninstalled some extensions from VS Code. These are not available in the VS Code extension Marketplace anymore so there must be an issue regarding compatibility. I fixed it by doing the following:  Open the command palette (Command + Shift + P) Type Python: Select Linter and select pylint Select the Install with Conda option Restart VS Code  Now it's working correctly, though it's still not shown in my extensions section in VS Code. It's necessary to point out that you will have to install pylint in every Python environment you are using, in my case I have multiple Conda environments.
Question: I know how to set the legend location of matplotlib plot with plt.legend(loc='lower left'), however, I am plotting with pandas method df.plot() and need to set the legend location to 'lower left'. Does anyone know how to do it? Edited: I am actually looking for a way to do it through pandas' df.plot(), not via plt.legend(loc='lower left')

Answer: With pandas 1.5.3 you can chain legend() behind plot() see matplotlib. Example: matched.set_index( matched.index.date ).plot(kind='barh', stacked=True ).legend( bbox_to_anchor=(1.0, 1.0), fontsize='small', );
Question: I recently moved to a place with terrible internet connection. Ever since then I have been having huge issues getting my programming environments set up with all the tools I need - you don't realize how many things you need to download until each one of those things takes over a day. For this post I would like to try to figure out how to deal with this in pip. The Problem Almost every time I pip install something it ends out timing out somewhere in the middle. It takes many tries until I get lucky enough to have it complete without a time out. This happens with many different things I have tried, big or small. Every time an install fails the next time starts all over again from 0%, no matter how far I got before. I get something along the lines of pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.  What I want to happen Ideally I would like to either extend the definition of time pip uses before it declares a timeout or be able to disable the option of a timeout all together. I am not sure either of these are possible, so if anyone has any other solution for me that would be greatly appreciated as well. Other Information Not sure this helps any but what I found is that the only reliable way for me to download anything here is using torrents, as they do not restart a download once they lose connection, rather they always continue where they left off. If there is a way to use this fact in any way that would be nice too.

Answer: Use option --timeout <sec> to set socket time out. Also, as @Iain Shelvington mentioned, timeout = <sec> in pip configuration will also work. TIP: Every time you want to know something (maybe an option) about a command (tool), before googling, check the manual page of the command by using man <command> or use <command> --help or check that command's docs online will be very useful too.
Question: I have an Airflow operator which allows me to query Athena which accepts a Jinja templated file as the query input. Usually, I pass variables such as table/database names, etc to the template for create table and add partition statements. This works fine for defined strings. My task definition looks like this:  db = 'sample_db' table = 'sample_table' out = 's3://sample' p1='2020' p2='1' add_partition_task= AWSAthenaOperator( task_id='add_partition', query='add_partition.sql', params={'database': db, 'table_name': table, 'p1': p1 'p2': p2}, database=db, output_location=out )  The SQL file being templated looks like: ALTER TABLE {{ params.database }}.{{ params.table_name }} ADD IF NOT EXISTS PARTITION (partition1= '{{ params.p1 }}', partition2 = '{{ params.p2 }}')  This templating works fine. The extension to this is to allow 'partition1' and 'partition2' to be defined by a jinja templated variable containing an XCOM pull from an earlier task which converts a date into Financial Year and Period. Using date as the partition is a possibility but I am interested in whether params can be forced to accept Jinja templates. The code I would like to use looks like the following:  db = 'sample_db' table = 'sample_table' out = 's3://sample' p1 = '{{ task_instance.xcom_pull(task_ids="convert_to_partition", key="p1") }}' p2 = '{{ task_instance.xcom_pull(task_ids="convert_to_partition", key="p2") }}' add_partition_task= AWSAthenaOperator( task_id='add_partition', query='add_partition.sql', params={'database': db, 'table_name': table, 'p1': p1 'p2': p2}, database=db, output_location=out )  So now params.p1 and params.p2 contain a Jinja template. Obviously, params does not support jinja templating as the SQL rendered contains the string literal '{{ task_instance....' rather than the rendered XCOM value. Adding params to the template_fields in the operator implementation is not enough to force it to render the template. My operator only extends BaseOperator and uses an AthenaHook which extends AwsHook. Does anyone have some experience with passing templated variables in a params like structure or an alternative approach?

Answer: Since AWSAthenaOperator has both query as a templated field and accepts file extension .sql, you can include the jinja template in the files themselves. I modified your AWSAthenaOperator a bit to fit the example. add_partition_task= AWSAthenaOperator( task_id='add_partition', query='add_partition.sql', params={ 'database': db, 'table_name': table, } )  Here is what the add_partition.sql could look like. INSERT OVERWRITE TABLE {{ params.database }}.{{ params.table_name }} (day, month, year) SELECT * FROM db.table WHERE p1 = "{{ task_instance.xcom_pull(task_ids='convert_to_partition', key='p1') }}" AND p2 = "{{ task_instance.xcom_pull(task_ids='convert_to_partition', key='p2') }}" ;
Question: I have the following list: list = [1,1,2,2]. After applying the sample method (rd.sample(list, 3)) the, output is [1, 1, 2]. After applying the choices method (rd.choices(list, 3)), the output is: [2, 1, 2]. What is the difference between these two methods? When should one be preferred over the other?

Answer: The fundamental difference is that random.choices() will (eventually) draw elements at the same position (always sample from the entire sequence, so, once drawn, the elements are replaced - with replacement), while random.sample() will not (once elements are picked, they are removed from the population to sample, so, once drawn the elements are not replaced - without replacement). Note that here replaced (replacement) should be understood as placed back (placement back) and not as a synonym of substituted (and substitution). To better understand it, let's consider the following example: import random random.seed(0) ll = list(range(10)) print(random.sample(ll, 10)) # [6, 9, 0, 2, 4, 3, 5, 1, 8, 7] print(random.choices(ll, k=10)) # [5, 9, 5, 2, 7, 6, 2, 9, 9, 8]  As you can see, random.sample() does not produce repeating elements, while random.choices() does.  In your example, both methods have results with repeating values because you have repeating values in the original sequence, but, in the case of random.sample() those repeating values must come from different positions of the original input.  Eventually, you cannot sample() more than the size of the input sequence, while this is not an issue with choices(): # print(random.sample(ll, 20)) # ValueError: Sample larger than population or is negative print(random.choices(ll, k=20)) # [9, 3, 7, 8, 6, 4, 1, 4, 6, 9, 9, 4, 8, 2, 8, 5, 0, 7, 3, 8]  A more generic and theoretical discussion of the sampling process can be found on Wikipedia.
Question: However, I feel saving the figure with plotly.express is pretty tricky. How to save plotly.express or plotly plot into a individual html or static image file? Anyone can help?

Answer: Adding to @vestland 's answer about saving to HTML, another way to do it according to the documentation would be: import plotly.express as px # a sample scatter plot figure created fig = px.scatter(x=range(10), y=range(10)) fig.write_html("path/to/file.html")  You can read about it further (controlling size of the HTML file) here: Interactive HTML Export in Python