eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: after updating the Werkzeug version from 2.0.3 to 2.1.0, I keep getting errors every time I run the server, and here is the error log: Exception happened during processing of request from ('127.0.0.1', 44612) Traceback (most recent call last): File "/usr/lib/python3.8/socketserver.py", line 683, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python3.8/socketserver.py", line 360, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python3.8/socketserver.py", line 747, in __init__ self.handle() File "/home/oladhari/.virtualenvs/reachat/lib/python3.8/site-packages/werkzeug/serving.py", line 363, in handle super().handle() File "/usr/lib/python3.8/http/server.py", line 427, in handle self.handle_one_request() File "/usr/lib/python3.8/http/server.py", line 415, in handle_one_request method() File "/home/oladhari/.virtualenvs/reachat/lib/python3.8/site-packages/werkzeug/serving.py", line 243, in run_wsgi self.environ = environ = self.make_environ() File "/home/oladhari/.virtualenvs/reachat/lib/python3.8/site-packages/django_extensions/management/commands/runserver_plus.py", line 326, in make_environ del environ['werkzeug.server.shutdown'] KeyError: 'werkzeug.server.shutdown'  this exception keep appearing while incrementing by 2 ( ('127.0.0.1', 44612) -> ('127.0.0.1', 44628) and the server crash checking the changes log, I have found this detail: Remove previously deprecated code. #2276 Remove the non-standard shutdown function from the WSGI environ when running the development server. See the docs for alternatives.  here is the link to the changes log it asks to check the documentation for alternatives but can not find any please let me know how I would resolve this error, thank you NB: my python version is 3.8

Answer: Literally just ran into this today. According to their (git repo issue 1715) and assuming you are running runserver_plus, there are three options that worked for some users. The first worked for me:  Not altering your files and adding the option --keep-meta-shutdown. My full command looks like python manage.py runserver_plus --cert-file /path/to/cert.pem --key-file /path/to/key.pem --keep-meta-shutdown localhost:9000 Comment out open lines 325 and 326 under your runserver_plus.py Upgrading python to 3.10  Hope this helps!
Question: I have a situation where attrs class inherits from another class which attributes have default value. This raises ValueError. Here's an example: from attrs import define @define class A: a: int = 1 @define class B(A): b: int test = B(b=1) >>> ValueError: No mandatory attributes allowed after an attribute with a default value or factory. Attribute in question: Attribute(name='b', ...  How do I avoid this kind of behavior?

Answer: You're running into a limitation of Python. The __init__ you're asking attrs to write for you looks like this: def __init__(self, b=1, a): self.b = b self.a = a  which can't exist. You can work around that by either declaring class B or attribute b keyword-only: from attrs import define, field @define class A: a: int = 1 @define(kw_only=True) class B1(A): b: int @define class B2(A): b: int = field(kw_only=True) test1 = B1(b=1) test2 = B2(b=1)
Question: I come across this issue often, and I would be surprised if there wasn't some very simple and pythonic one-liner solution to it. Suppose I have a method or a function that takes a list or some other iterable object as an argument. I want for an operation to be performed once for each item in the object. Sometimes, only a single item (say, a float value) is passed to this function. In this situation, my for-loop doesn't know what to do. And so, I find myself peppering my code with the following snippet of code: from collections.abc import Sequence def my_function(value): if not isinstance(value, Sequence): value = [value] # rest of my function  This works, but it seems wasteful and not particularly legible. In searching StackOverflow I've also discovered that strings are considered sequences, and so this code could easily break given the wrong argument. It just doesn't feel like the right approach. I come from a MATLAB background, and this is neatly solved in that language since scalars are treated like 1x1 matrices. I'd expect, at the very least, for there to be a built-in, something like numpy's atleast_1d function, that automatically converts anything into an iterable if it isn't one.

Answer: The short answer is nope, there is no simple built-in. And yep, if you want str (or bytes or bytes-like stuff or whatever) to act as a scalar value, it gets uglier. Python expects callers to adhere to the interface contract; if you say you accept sequences, say so, and it's on the caller to wrap any individual arguments. If you must do this, there's two obvious ways to do it: First is to make your function accept varargs instead of a single argument, and leave it up to the caller to unpack any sequences, so you can always iterate the varargs received: def my_function(*values): for val in values: # Rest of function  A caller with individual items calls you with my_function(a, b), a caller with a sequence calls you with my_function(*seq). The latter does incur some overhead to unpack the sequence to a new tuple to be received by my_function, but in many cases this is fine. If that's not acceptable for whatever reason, the other solution is to roll your own "ensure iterable" converter function, following whatever rules you care about: from collections.abc import ByteString def ensure_iterable(obj): if isinstance(obj, (str, ByteString)): return (obj,) # Treat strings and bytes-like stuff as scalars and wrap try: iter(obj) # Simplest way to test if something is iterable is to try to make it an iterator except TypeError: return (obj,) # Not iterable, wrap else: return obj # Already iterable  which my_function can use with: def my_function(value): value = ensure_iterable(value)
Question: I am looking at lots of sentences and looking to extract the start and end indices of a word in a given sentence. For example, the input is as follows: "This is a sentence written in English by a native English speaker." And What I want is the span of the word 'English' which in this case is : (30,37) and (50, 57). Note: I was pointed to this answer (Get position of word in sentence with spacy) But this answer doesn't solve my problem. It can help me in getting the start character of the token but not the end index. All help appreciated

Answer: You can do this with re in pure python: s="This is a sentence written in english by a native English speaker." import re [(i.start(), i.end()) for i in re.finditer('ENGLISH', s.upper())] #output [(30, 37), (50, 57)]  You can do in spacy as well: import spacy nlp=spacy.load("en_core_web_sm") doc=nlp("This is a sentence written in english by a native English speaker.") for ent in doc.ents: if ent.text.upper()=='ENGLISH': print(ent.start_char,ent.end_char)
Question: I have been following the django-oAuth-toolkit documentation. In the Authorization Code step, I have registered an application as shown in the screenshot.  But then the next step is given like this:  To start the Authorization code flow go to this URL which is the same as shown below: http://127.0.0.1:8000/o/authorize/?response_type=code&client_id=vW1RcAl7Mb0d5gyHNQIAcH110lWoOW2BmWJIero8&redirect_uri=http://127.0.0.1:8000/noexist/callback  But when I replace my client id and ping that URL it redirects me to the following URL: http://localhost:8000/noexist/callback?error=invalid_request&error_description=Code+challenge+required. I have tried to google that error but it's such a common keyword that I am unable to find anything that is related to my issue. I am probably missing something obvious, I am new to Python and Django. Note: In the documentation screenshot there is one form field missing which is there in my local environment. It's the algorithm field. 

Answer: After debugging for so many hours I came to this, please include it in your settings.py file and it works. Maybe it is a bug since we defined our app as confidential with authorization_code grant type but oauth_provider is thinking it as public and trying to validate for pkce. OAUTH2_PROVIDER = { "PKCE_REQUIRED": False }
Question: When selecting data with xarray at x,y locations, I get data for any pair of x,y. I would like to have a 1-D array not a 2-D array from the selection. Is there an efficient way to do this? (For now I am doing it with a for-loop...) x = [x1,x2,x3,x4] y = [y1,y2,y3,y4] DS = 2-D array subset = Dataset.sel(longitude=x, latitude=y, method='nearest')  To rephrase, I would like to have the dataset at [x1,y1],[x2,y2],[x3,y3],[x4,y4] not at other location i.e. [x1,y2].

Answer: A list of points can be selected along multiple indices if the indexers are DataArrays with a common dimension. This will result in the array being reindexed along the indexers' common dimension. Straight from the docs on More Advanced Indexing: In [78]: da = xr.DataArray(np.arange(56).reshape((7, 8)), dims=['x', 'y']) In [79]: da Out[79]: <xarray.DataArray (x: 7, y: 8)> array([[ 0, 1, 2, 3, 4, 5, 6, 7], [ 8, 9, 10, 11, 12, 13, 14, 15], [16, 17, 18, 19, 20, 21, 22, 23], [24, 25, 26, 27, 28, 29, 30, 31], [32, 33, 34, 35, 36, 37, 38, 39], [40, 41, 42, 43, 44, 45, 46, 47], [48, 49, 50, 51, 52, 53, 54, 55]]) Dimensions without coordinates: x, y In [80]: da.isel(x=xr.DataArray([0, 1, 6], dims='z'), ....: y=xr.DataArray([0, 1, 0], dims='z')) ....: Out[80]: <xarray.DataArray (z: 3)> array([ 0, 9, 48]) Dimensions without coordinates: z  The indexing array can also be easily pulled out of a pandas DataFrame, with something like da.sel(longitude=df.longitude.to_xarray(), latitude=df.latitude.to_xarray()), which will result in the DataArray being reindexed by the DataFrame's index. So in your case, rather than selecting with the lists or arrays x, y, turn them into DataArrays with a common dim - let's call it location: x = xr.DataArray([x1,x2,x3,x4], dims=['location']) y = xr.DataArray([y1,y2,y3,y4], dims=['location'])  Now your selection will work as you hope: ds.sel(longitude=x, latitude=y, method='nearest')
Question: I've got a function of the form: def get_new_file(prefix: str) -> pathlib.Path: for i in itertools.count(0): p = pathlib.Path(f'{prefix}_{i}') if not p.is_file(): return p # This line is unreachable.  mypy understandably complains that the function is missing a return statement. Is there a way to mark a line so as to inform mypy that the line should be considered unreachable?

Answer: This was raised as an issue. The recommended solution is assert False. def get_new_file(prefix: str) -> pathlib.Path: for i in itertools.count(0): p = pathlib.Path(f'{prefix}_{i}') if not p.is_file(): return p assert False
Question: I would like to have a plot where the font are in "computer modern" (i.e. Latex style) but with x-ticks and y-ticks in bold. Due to the recent upgrade of matplotlib my previous procedure does not work anymore. This is my old procedure: plt.rc('font', family='serif',size=24) matplotlib.rc('text', usetex=True) matplotlib.rc('legend', fontsize=24) matplotlib.rcParams['text.latex.preamble'] = [r'\boldmath']  This is the output message: test_font.py:26: MatplotlibDeprecationWarning: Support for setting an rcParam that expects a str value to a non-str value is deprecated since 3.5 and support will be removed two minor releases later. matplotlib.rcParams['text.latex.preamble'] = [r'\boldmath']  I have decide that a possible solution could be to use the "computer modern" as font. This is my example: import matplotlib import matplotlib.pyplot as plt import numpy as np font = {'family' : 'serif', 'weight' : 'bold', 'size' : 12 } matplotlib.rc('font', **font) # Data for plotting t = np.arange(0.0, 2.0, 0.01) s = 1 + np.sin(2 * np.pi * t) fig, ax = plt.subplots(1,figsize=(9,6)) ax.plot(t, s) ax.set(xlabel='time (s) $a_1$', ylabel='voltage (mV)', title='About as simple as it gets, folks') ax.grid() fig.savefig("test.png") plt.show()  This is the result:  I am not able, however, to set-up in font the font style. I have tried to set the font family as "cmr10". This the code: font = {'family' : 'serif', 'weight' : 'bold', 'size' : 12, 'serif': 'cmr10' } matplotlib.rc('font', **font)  It seems that the "cmr10" makes disappear the bold option. Have I made some errors? Do you have in mind other possible solution? Thanks

Answer: You still can use your old procedure, but with a slight change. The MatplotlibDeprecationWarning that you get states that the parameter expects a str value but it's getting something else. In this case what is happening is that you are passing it as a list. Removing the brackets will do the trick: import matplotlib import matplotlib.pyplot as plt import numpy as np plt.rc('font', family='serif',size=24) matplotlib.rc('text', usetex=True) matplotlib.rc('legend', fontsize=24) matplotlib.rcParams['text.latex.preamble'] = r'\boldmath' # Data for plotting t = np.arange(0.0, 2.0, 0.01) s = 1 + np.sin(2 * np.pi * t) fig, ax = plt.subplots(1,figsize=(9,6)) ax.plot(t, s) ax.set(xlabel='time (s) $a_1$', ylabel='voltage (mV)', title='About as simple as it gets, folks') ax.grid() fig.savefig("test.png") plt.show()  The code above produces this plot without any errors:
Question: I need to install below multiple packages using conda. I am not sure what is conda-forge? some uses conda-forge, some doesn't use it. Is it possible to install them in one line without installing them one by one? Thanks conda install -c conda-forge dash-daq conda install -c conda-forge dash-core-components conda install -c conda-forge dash-html-components conda install -c conda-forge dash-bootstrap-components conda install -c conda-forge dash-table conda install -c plotly jupyter-dash 

Answer: Why some packages have to be installed through conda forge: Conda official repository only feature a few verified packages. A vast portion of python packages that are otherwise available through pip are installed through community led channel called conda-forge. You can visit their site to learn more about it. How to install multiple packages in a single line? The recommended way to install multiple packages is to create a .yml file and feed conda this. You can specify the version number for each package as well. The following example file can be fed to conda through conda install --file: appdirs=1.4.3 asn1crypto=0.24.0 ... zope=1.0 zope.interface=4.5.0  To specify different channel for each package in this environment.yml file, you can use the :: syntax. dependencies: - python=3.6 - bagit - conda-forge::beautifulsoup4
Question: I want to do a simple shap analysis and plot a shap.force_plot. I noticed that it works without any issues locally in a .ipynb file, but fails on Databricks with the following error message: Visualization omitted, Javascript library not loaded! Have you run `initjs()` in this notebook? If this notebook was from another user you must also trust this notebook (File -> Trust notebook). If you are viewing this notebook on github the Javascript has been stripped for security. If you are using JupyterLab this error is because a JupyterLab extension has not yet been written.  Code: import xgboost import shap shap.initjs() X, y = shap.datasets.boston() bst = xgboost.train({"learning_rate": 0.01}, xgboost.DMatrix(X, label=y), 100) explainer = shap.TreeExplainer(bst) shap_values = explainer.shap_values(X) shap.force_plot(explainer.expected_value, shap_values[0,:], X.iloc[0,:])  Is there any way to get the image plotting work on Databricks?

Answer: Let's try slightly different (matplotlib=True): import xgboost import shap X, y = shap.datasets.boston() bst = xgboost.train({"learning_rate": 0.01}, xgboost.DMatrix(X, label=y), 100) explainer = shap.TreeExplainer(bst) shap_values = explainer.shap_values(X) shap.force_plot( explainer.expected_value, shap_values[0,:], X.iloc[0,:], matplotlib=True # <-- )
Question: When I'm exploring data in an ad hoc way, I often have code like this: X = (adj_all.o.diff(1) / adj_none.o.diff(1)).diff(1) print(X[X > 0])  Is there a way to do this in a single line in an easy way? The following works but is verbose: (adj_all.o.diff(1) / adj_none.o.diff(1)).diff(1)[(adj_all.o.diff(1) / adj_none.o.diff(1)).diff(1) > 0]  I want something like this: (adj_all.o.diff(1) / adj_none.o.diff(1)).diff(1)[self > 0]  Note that this isn't production code. It is part of ad hoc exploration where iteration speed is important to results, which is why I wish to be able to do this common thing in a single line.

Answer: You can use pipe: (adj_all.o.diff(1) / adj_none.o.diff(1)).diff(1).pipe(lambda x: x[x>0])
Question: I have a text file like this: Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz Sclerophthora macrospora virus A Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz Influenza B virus RNA Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz Tomato mottle virus  And I need to get a csv file like this: Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus  Because later I want to use this like a tuple to find the compressed file, read it and get a final file with names like: Viruses/GCF_000837105.1/Tomato mottle virus.fna  I just need to learn how to do the first part of the problem. It could by with:  sed awk R Python  Any help would be very appreciated. This is hard for me to accomplish because the original filenames are very messed up. I have tried this: sed -z 's/\n/,/g;s/,$/\n/' multi_headers  However it put comma in all \n.

Answer: Using any awk in any shell on every Unix box and only storing 1 line at a time in memory so it'll work no matter how large your input file is: $ awk '{ORS=(NR%2 ? "," : RS)} 1' file Viruses/GCF_000820355.1_ViralMultiSegProj14361_genomic.fna.gz,Sclerophthora macrospora virus A Viruses/GCF_000820495.2_ViralMultiSegProj14656_genomic.fna.gz,Influenza B virus RNA Viruses/GCF_000837105.1_ViralMultiSegProj14079_genomic.fna.gz,Tomato mottle virus  There's a lot happening in a small amount of code above so here's an explanation:  ORS is the builtin variable containing the string to be printed at the end of each output record (record = line in this case), a newline by default. RS is the builtin variable containing the string (or regexp) that separates each input record, a newline by default. NR is the builtin variable containing the current record/line number so NR%2 is 1 for odd numbered records and 0 for even numbered. NR%2 ? "," : RS is a ternary expression resulting in , for odd numbered lines, \n (or whatever else you have set RS to, e.g. \r\n) for even numbered. 1 is a true condition which causes the default action of printing the current record to be executed.  So the above script says "if the current line number is odd print it with a , at the end, otherwise print it with a newline at the end", hence it's joining every pair of lines with a , between.
Question: I am quite new in building bots so I created a very simple Telegram bot and it works great but can't figure out how to make the bot send messages every n minutes or n hours when /start_auto command is initiated. I made a workaround with while loop but it looks stupid and during the loop users won't be able to interact with the bot about other topics. I want users to be able to start and stop this scheduled task by commands such as /start_auto and /stop_auto. I know there are many other answered questions related to this topic but none of them seem to be working with my code. import logging import os import time from telegram.ext import Updater, CommandHandler, MessageHandler, Filters logger = logging.getLogger(__name__) PORT = int(os.environ.get('PORT', '8443')) def start(update, context): """Sends a message when the command /start is issued.""" update.message.reply_text('Hi!') def help(update, context): """Sends a message when the command /help is issued.""" update.message.reply_text('Help!') def start_auto(update, context): """Sends a message when the command /start_auto is issued.""" n = 0 while n < 12: time.sleep(3600) update.message.reply_text('Auto message!') n += 1 def error(update, context): """Logs Errors caused by Updates.""" logger.warning('Update "%s" caused error "%s"', update, context.error) def main(): TOKEN = 'TOKEN_GOES_HERE' APP_NAME = 'https://my-tele-test.herokuapp.com/' updater = Updater(TOKEN, use_context=True) # Get the dispatcher to register handlers dp = updater.dispatcher dp.add_handler(CommandHandler("start", start)) dp.add_handler(CommandHandler("help", help)) dp.add_handler(CommandHandler("start_auto", start_auto)) # log all errors dp.add_error_handler(error) updater.start_webhook(listen="0.0.0.0", port=PORT, url_path=TOKEN, webhook_url=APP_NAME + TOKEN) updater.idle() if __name__ == '__main__': main() 

Answer: will post a solution which I found: def callback_auto_message(context): context.bot.send_message(chat_id='12345678', text='Automatic message!') def start_auto_messaging(update, context): chat_id = update.message.chat_id context.job_queue.run_repeating(callback_auto_message, 10, context=chat_id, name=str(chat_id)) # context.job_queue.run_once(callback_auto_message, 3600, context=chat_id) # context.job_queue.run_daily(callback_auto_message, time=datetime.time(hour=9, minute=22), days=(0, 1, 2, 3, 4, 5, 6), context=chat_id) def stop_notify(update, context): chat_id = update.message.chat_id context.bot.send_message(chat_id=chat_id, text='Stopping automatic messages!') job = context.job_queue.get_jobs_by_name(str(chat_id)) job[0].schedule_removal()  And in the main function I created a commands: dp.add_handler(CommandHandler("auto", start_auto_messaging)) dp.add_handler(CommandHandler("stop", stop_notify))