eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to run the example dash application but upon trying to run, the browser says it is refusing to connect. I have checked and Google Chrome has access through the firewall. The example code is:  import dash import dash_core_components as dcc import dash_html_components as html external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css'] app = dash.Dash(__name__, external_stylesheets=external_stylesheets) app.layout = html.Div(children=[ html.H1(children='Hello Dash'), html.Div(children=''' Dash: A web application framework for Python. '''), dcc.Graph( id='example-graph', figure={ 'data': [ {'x': [1, 2, 3], 'y': [4, 1, 2], 'type': 'bar', 'name': 'SF'}, {'x': [1, 2, 3], 'y': [2, 4, 5], 'type': 'bar', 'name': u'Montréal'}, ], 'layout': { 'title': 'Dash Data Visualization' } } ) ]) if __name__ == '__main__': app.run_server(debug=True)  Here is a picture of my browser:  Does anyone understand this?

Answer: First check if you are accessing the right port, the default one (usually) is 8050: http://localhost:8050/ Also, check if there is another Dash code running, it might be occupying the port. If it does not work, try determining the host as an argument in app.runserver(args), like this: app.run_server(host='0.0.0.0', debug=True)  You might also want to determine the port as an argument like this: app.run_server(host='0.0.0.0', port=8050, debug=True)
Question: I have a Dataframe  Date Category Sum 0 2019-06-03 "25M" 34 1 2019-06-03 "25M" 60 2 2019-06-03 "50M" 23 3 2019-06-04 "25M" 67 4 2019-06-05 "50M" -90 5 2019-06-05 "50M" 100 6 2019-06-06 "100M" 6 7 2019-06-07 "25M" -100 8 2019-06-08 "100M" 67 9 2019-06-09 "25M" 450 10 2019-06-10 "50M" 600 11 2019-06-11 "25M" -9 12 2019-07-12 "50M" 45 13 2019-07-13 "50M" 67 14 2019-07-14 "100M" 130 15 2019-07-14 "50M" 45 16 2019-07-15 "100M" 100 17 2019-07-16 "25M" -90 18 2019-07-17 "25M" 700 19 2019-07-18 "25M" -9  I want to create a plotly graph showing the addition of "Sum" for different "Category" on Every described date, but want to remove dates, if they don't have any data.  Code  df["Date"]=pd.to_datetime(df["Date"], format=("%Y%m%d")) df=df.sort_values(["Date","Category","Sum"],ascending=False) df=round(df.groupby(["Date","Category"]).agg({"Sum":"sum"}).reset_index(),1) fig = px.bar(df, x=df["Date"] , y='Sum',barmode="group",color="Category") fig.update_xaxes( rangeslider_visible=True, rangeselector=dict( buttons=list([ dict(count=1, label="day", step="day", stepmode="todate"), dict(count=24, label="montly", step="month", stepmode="todate"), dict(count=1, label="year", step="year", stepmode="todate"), dict(step="all") ]) )) fig.show()   I am getting graph like this but I want to remove the empty Dates from the plotly graph

Answer: I had the same problem with my graph. Just add the following to layout code: xaxis=dict(type = "category")  Note: I have used import plotly.graph_objs as go and NOT import plotly.express as px This worked for me. Hope it helps you too.
Question: I'm running on this issue for quite a while now. I set the writer as follows: writer = pd.ExcelWriter(arquivo+'.xlsx', engine = 'xlsxwriter', options = {'remove_timezone': True}) df.to_excel(writer, header = True, index = True)  This code is inside s function. The problem is every time I run the code, it gets information from the database, which contains two columns datetime64[ns, UTC] object with time zone info. But when the code to save to Excel runs I receive: ValueError: Excel does not support datetimes with timezones. Please ensure that datetimes are timezone unaware before writing to Excel.  I have already tried several things like 'dt.tz_convert', replace(tzinfo=None) and other solutions I have found here and around. The code runs without problem in my personal computer, my colleague at work with the same machine specs can run the code. Only in my machine it doesn't. I already reinstalled python and all the packages, including formatting the machine and nothing, the error persists. xlrd v1.1.0 xlsxwriter v1.0.4 python 3.7.4 pandas v0.25.1 If someone could bring some light into this issue I would much appreciate it. Thanks

Answer: What format is your timestamps in? I just had a similar problem. I was trying to save a data frame to Excel. However I was getting:  I checked my date format which was in this format '2019-09-01T00:00:00.000Z' This is a timestamp pandas._libs.tslibs.timestamps.Timestamp from pandas.to_datetime which includes a method date() that converted the date into a format "%Y-%m-%d" that was acceptable by excel So my code was something like: #Pseudo df['date'] = old_dates df['date'] = df['date'].apply(lambda a: pd.to_datetime(a).date()) # .date() removes timezone ...df.to_excel etc.
Question: I'm using numpy and mlrose, and all i have written so far is: import numpy as np import mlrose  However, when i run it, it comes up with an error message:  File "C:\Users\<my username>\AppData\Local\Programs\Python\Python38-32\lib\site-packages\mlrose\neural.py", line 12, in <module> from sklearn.externals import six ImportError: cannot import name 'six' from 'sklearn.externals' (C:\Users\<my username>\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\externals\__init__.py)  Any help on sorting this problem will be greatly appreciated.

Answer: Solution: The real answer is that the dependency needs to be changed by the mlrose maintainers. A workaround is: import six import sys sys.modules['sklearn.externals.six'] = six import mlrose
Question: I recently upgraded my OS to Ubuntu 20.04 LTS. Now when I try to import a library like Numpy in Python, I get the following error:  ImportError: libffi.so.6: cannot open shared object file: No such file or directory  I tried installing the libffi package, but apt can't locate it: sudo apt-get install libffi Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package libffi 

Answer: It seems like I fixed it. I could be wrong, but here is what I think happened:  Ubuntu 20.04 upgraded libffi6 to libffi7 Python is still looking for libffi6  What I did to fix it : Locate libffi.so.7 in your system $ find /usr/lib -name "libffi.so*"  Create a simlink named libffi.so.6 that points to libffi.so.7: sudo ln -s /usr/path/to/libffi.so.7 /usr/lib/path/to/libffi.so.6  UPDATE: As noted by many users, this fix could have unintended consequences. The better way to do it is to reinstall python as @amichaud explained. This should be used as a last resort IF you're not using pyenv/virtualenv/etc in which case removing python will cause a lot of dependencies to be removed as well.
Question: I made my project fine and when I run my server through a normal shell, it works. Now, I am trying to run my project through Git Bash. All the commands seem to work fine but when I do python manage.py runserver, it gets stuck on Watching for file changes with StatReloader  Apparently after that I go to localhost:8000 but that and my 127 port 8000 are not responding and show that there's nothing there. What is going on?

Answer: You may be missing the port binding, try to run python manage.py runserver 0.0.0.0:8000 to be sure that the app is running on localhost:8000
Question: So I'm using Python as a front end GUI that interacts with some C files for storage and memory management as a backend. Whenever the GUI's window is closed or exited, I call all the destructor methods for my allocated variables. Is there anyway to check memory leaks or availability, like a C Valgrind check, right before exiting the whole program to make sure there wasn't any memory leaks? Example exit: from tkinter import * root = Tk() # New GUI # some code here def destructorMethods: myFunctions.destructorLinkedList() # Destructor method of my allocated memory in my C file # Here is where I would want to run a Valgrind/Memory management check before closing root.destroy() # close the program root.protocol("WM_DELETE_WINDOW", destructorMethods) # When the close window option is pressed call destructorMethods function 

Answer: If you want to use Valgrind, then this readme might be helpful. Probably, this could be another good resource to make Valgrind friendly python and use it in your program. But if you consider something else like tracemalloc, then you can easily get some example usage of it here. The examples are pretty easy to interpret. For example according to their doc,  import tracemalloc tracemalloc.start() # ... run your application ... snapshot = tracemalloc.take_snapshot() top_stats = snapshot.statistics('lineno') print("[ Top 10 ]") for stat in top_stats[:10]: print(stat)  This will output something like.  <frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B <frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 >  You can either parse this to plot memory usage for your investigation or you may use the reference doc to get a more concrete idea. In this case your program could be something like the following:  from tkinter import * import tracemalloc root = Tk() # New GUI # some code here def destructorMethods: tracemalloc.start() myFunctions.destructorLinkedList() # Destructor method of my allocated memory in my C file # Here is where I would want to run a Valgrind/Memory management check before closing snapshot = tracemalloc.take_snapshot() top_stats = snapshot.statistics('lineno') print("[ Top 10 ]") for stat in top_stats[:10]: print(stat) root.destroy() # close the program root.protocol("WM_DELETE_WINDOW", destructorMethods)  Another option is, you can use a memory profiler to see memory usage at a variable time. The package is available here. After the installation of this package, you can probably use the following command in your script to get the memory usage over time in a png file.  mprof run --include-children python your_filename.py mprof plot --output timelyplot.png  or you may use different functions available on memory_profiler package according to your need. Maybe this tutorial can be an interesting one for you.
Question: I am stuck in django and would really appreciate it if someone could help me. I need to have an entry point for a 3rd party API. So I created a view and decorated it with @csrf_exempt Now the problem is I am not able to access any session variables I set before. edit - I set multiple session variables like user email to know if a user is already logged in. I was able to use the session before calling the 3rd party API. When the 3rd party API sends a response, they don't send CSRF token hence I have exempt that view from csrf. Once I receive a valid response I want to update my database. To do that, I need to know the email id of the user which I lost since I don't have session variables anymore.  ppConfirmPaymentProcess is another function that processes the POST data sent by this 3rd party API. Everything is working fine, csrf_exempt is also working fine but I can't do request.session["foo"] with this request. Can someone please help? @csrf_exempt def ppConfirmPayment(request): print(request.session, "=======================================") for key, value in request.session.items(): print('{} => {}'.format(key, value)) return ppConfirmPaymentProcess(request) 

Answer: I solved it using Django itself. No manipulation of session-id or interaction with the database. Step1: call 3rd party api  @login_required def thirdPartyAPICall(request): #do some stuff and send a request to 3rd party  Step2: receive a call-back from 3rd party in the view. Note how I put csrf_exempt and not login_required so that 3rd party can send a request to my application without CSRF token and session. It's like an entry point to my APP for them. In this callBackView do some action and check if this indeed is a valid response from the 3rd party or someone is trying to hack your system. E.g. check for CHECKSUM or TXNID etc and then create a response dictionary and sent another HTTP response to another resource using HttpResponseRedirect with-in my app and then I passed relevant GET parameter to it. This particular step restores my previous session and now I have the relevant data from the 3rd party to process the request I sent to them and I also got my session back in the request. @csrf_exempt def callBackView(request): if request.POST["CHECKSUM"] == myCalCulatedCheckSum: foo = True else: foo = False return HttpResponseRedirect("TEST.HTML" +"/" + str(foo))  I like this method the most because as I mentioned before, we don't need to store session, Django does it for us.
Question: I am using Flask,Python for my web application . The user will login and if the session time is more than 5 minutes then the app should come out and it should land on the login page. I tried some of the methods and I can see the session time out is happening but redirect to login page is not happening. @app.before_request def before_request(): "Session time out method" flask.session.permanent = True app.permanent_session_lifetime = datetime.timedelta(minutes=2) flask.session.modified = True flask.g.user = flask_login.current_user #return redirect(url_for('login'))  I have used before_request for seesion time out. I have referrred this link Flask logout if sessions expires if no activity and redirect for login page but I dont see any changes from what I have tried before and this code. I can see lot of stackoverflow questions over there for this topic and I couldnt find the solution. I have tried this link als Expire session in flask in ajax context But I am not sure what should I pass as a session and what qualifier I should return here? @mod.before_request def make_session_permanent(): if session_is_invalid(session): return redirect(url_for('logout')) def session_is_invalid(ses): # return your qualifier  if the previous method is correct can some one tell me what is the session and what qualifier I should return here? What I need is after session log out the page should automatically land on the login screen What is happening is session log out is happening but it's not redirecting to login page could some one help me in this?

Answer: When I read the documents of the Flask-Login package, i saw a few things. When creating your Flask application, you also need to create a login manager. login_manager = LoginManager()  A login_view variable in LoginManager class caught my attention. The details include the following explanation:   The name of the view to redirect to when the user needs to log in. (This can be an absolute URL as well, if your authentication machinery is external to your application.)  Actually, after you create a LoginManager object,  login_manager.login_view = 'your login view'  You should specify your login page. Finally, Once the actual application object has been created, you can configure it for login with: login_manager.init_app(app)  After doing these, any unauthorized calls to every method you use @login_required annotation will be sent to the page you have pointed out with login_view. I also developed a simple application. You can review this application from here. I tested it and working without any problems. I hope it helps you.
Question: I cloned a repository, installed pre-commit and was committing for the first time. This is the time when pre-commit packages actually get installed and setup. I faced the following issue.  [INFO] Installing environment for https://github.com/asottile/seed-isort-config. [INFO] Once installed this environment will be reused. [INFO] This may take a few minutes... An unexpected error has occurred: CalledProcessError: command: ('/home/roopak/.cache/pre-commit/repokb2ckm/py_env-python2.7/bin/python', u'/home/roopak/.cache/pre-commit/repokb2ckm/py_env-python2.7/bin/pip', 'install', '.') return code: 1 expected return code: 0 stdout: Processing /home/roopak/.cache/pre-commit/repokb2ckm stderr: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support ERROR: Package 'seed-isort-config' requires a different Python: 2.7.17 not in '>=3.6.1' 

Answer: The issue was that I have both Python2.7 and 3 installed. And my pre-commit was installed was using Python 2.7 as the default.   Solution 1: remove pre-commit from Python2.7 and add it to Python3. As per the creator of pre-commit - @anthony-sottile - it is better to use pre-commit with Python3. To do that we will have to uninstall pre-commit from Python2.7 and install it via Python3.  $ pip uninstall pre-commit # uninstall from Python2.7 $ pip3 install pre-commit # install with Python3   Solution 2: keeping pre-commit with Python2.7 (not recommended) To solve this I used default_language_version from the pre-commit documentation. Refer: https://pre-commit.com/#overriding-language-version By setting the default_language_version all hooks will use this particular version. If any particular hook needs to be overridden this property - language_version: - may be set on the hook.  Eg:- default_language_version: # force all unspecified python hooks to run python3 python: python3 repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.5.0 hooks: - id: trailing-whitespace name: trim trailing whitespace description: This hook trims trailing whitespace on files entry: trailing-whitespace-fixer - id: check-merge-conflict name: check for merge conflict description: Prevent accidentally commiting files with merge conflicts. language_version: python: python2.7  This example .pre-commit-config.yaml file set the default python version to Python 3. For the hook - check-merge-conflict it will use Python 2.7.
Question: i have the following dataframe: High Low Open Close Volume Adj Close year pct_day month day 1 1 NaN NaN NaN NaN NaN NaN 2010.0 0.000000 2 7869.853149 7718.482498 7779.655014 7818.089966 7.471689e+07 7818.089966 2010.0 0.007826 3 7839.965652 7719.758224 7775.396255 7777.940002 8.185879e+07 7777.940002 2010.0 0.002582 4 7747.175260 7624.540007 7691.152083 7686.288672 1.018877e+08 7686.288672 2010.0 -0.000744 5 7348.487095 7236.742135 7317.313616 7287.688546 1.035424e+08 7287.688546 2010.0 -0.002499 ... ... ... ... ... ... ... ... ... ... 12 27 7849.846680 7760.222526 7810.902051 7798.639258 4.678145e+07 7798.639258 2009.5 -0.000833 28 7746.209996 7678.152204 7713.497907 7710.449358 4.187133e+07 7710.449358 2009.5 0.000578 29 7357.001540 7291.827806 7319.393874 7338.938345 4.554891e+07 7338.938345 2009.5 0.003321 30 7343.726938 7276.871507 7322.123779 7302.545316 3.967812e+07 7302.545316 2009.5 -0.000312 31 NaN NaN NaN NaN NaN NaN 2009.5 0.000000  Since it is not clear from the above pasted dataframe, below is a snapshot:  The months are in 1,2 3 ... Is it possible to rename the month index to Jan Feb Mar format? Edit : I am having a hard time implementing the example by @ChihebNexus My code is as follows since it is a datetime : full_dates = pd.date_range(start, end) data = data.reindex(full_dates) data['year'] = data.index.year data['month'] = data.index.month data['week'] = data.index.week data['day'] = data.index.day data.set_index('month',append=True,inplace=True) data.set_index('week',append=True,inplace=True) data.set_index('day',append=True,inplace=True) df = data.groupby(['month', 'day']).mean() 

Answer: I would do it using calendar and pd.CategoricalDtype to ensure sorting works correctly. import pandas as pd import numpy as np import calendar #Create dummy dataframe dateindx = pd.date_range('2019-01-01', '2019-12-31', freq='D') df = pd.DataFrame(np.random.randint(0,1000, (len(dateindx), 5)), index=pd.MultiIndex.from_arrays([dateindx.month, dateindx.day]), columns=['High', 'Low','Open', 'Close','Volume']) #Use calendar library for abbreviations and order dd=dict((enumerate(calendar.month_abbr))) #rename level zero of multiindex df = df.rename(index=dd,level=0) #Create calendar month data type with order for sorting cal_dtype = pd.CategoricalDtype(list(calendar.month_abbr), ordered=True) #Change the dtype of the level zero index df.index = df1.index.set_levels(df.index.levels[0].astype(cal_dtype), level=0) df  Output:  High Low Open Close Volume Jan 1 501 720 671 943 586 2 410 67 207 945 284 3 473 481 527 415 852 4 157 809 484 592 894 5 294 38 458 62 945 ... ... ... ... ... ... Dec 27 305 354 347 0 726 28 764 987 564 260 72 29 730 151 846 137 118 30 999 399 634 674 81 31 347 980 441 600 676 [365 rows x 5 columns]
Question: Please forgive my ignorance. All I'm trying to do is add a squared term to my regression without going through the trouble of defining a new column in my dataframe. I'm using statsmodels.formula.api (as stats) because the format is similar to R, which I am more familiar with.  hours_model = stats.ols(formula='act_hours ~ h_hours + C(month) + trend', data = df).fit()  The above works as expected. hours_model = stats.ols(formula='act_hours ~ h_hours + h_hours**2 + C(month) + trend', data = df).fit()  This omits h_hours**2 and returns the same output as the line above.  I've also tried: h_hours^2, math.pow(h_hours,2), and poly(h_hours,2) All throw errors.  Any help would be appreciated.

Answer: You can try using I() like in R: import statsmodels.formula.api as smf np.random.seed(0) df = pd.DataFrame({'act_hours':np.random.uniform(1,4,100),'h_hours':np.random.uniform(1,4,100), 'month':np.random.randint(0,3,100),'trend':np.random.uniform(0,2,100)}) model = 'act_hours ~ h_hours + I(h_hours**2)' hours_model = smf.ols(formula = model, data = df) hours_model.exog[:5,] array([[ 1. , 3.03344961, 9.20181654], [ 1. , 1.81002392, 3.27618659], [ 1. , 3.20558207, 10.27575638], [ 1. , 3.88656564, 15.10539244], [ 1. , 1.74625943, 3.049422 ]])
Question: I want to use spacy's pretrained BERT model for text classification but I'm a little confused about cased/uncased models. I read somewhere that cased models should only be used when there is a chance that letter casing will be helpful for the task. In my specific case: I am working with German texts. And in German all nouns start with the capital letter. So, I think, (correct me if I'm wrong) that this is the exact situation where cased model must be used. (There is also no uncased model available for German in spacy).  But what must be done with data in this situation? Should I (while preprocessing train data) leave it as it is (by that I mean not using the .lower() function) or it doesn't make any difference?

Answer: As a non-German-speaker, your comment about nouns being uppercase does make it seem like case is more relevant for German than it might be for English, but that doesn't obviously mean that a cased model will give better performance on all tasks. For something like part-of-speech detection, case would probably be enormously helpful for the reason you describe, but for something like sentiment analysis, it's less clear whether the added complexity of having a much larger vocabulary is worth the benefits. (As a human, you could probably imagine doing sentiment analysis with all lowercase text just as easily.) Given that the only model available is the cased version, I would just go with that - I'm sure it will still be one of the best pretrained German models you can get your hands on. Cased models have separate vocab entries for differently-cased words (e.g. in english the and The will be different tokens). So yes, during preprocessing you wouldn't want to remove that information by calling .lower(), just leave the casing as-is.