eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I get the deprication warning, that Redis.hmset() is deprecated. Use Redis.hset() instead. However hset() takes a third parameter and I can't figure out what name is supposed to be. info = {'users': 10, "timestamp": datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')} r.hmset("myKey", info)  The above works, but this requires a first parameter called name. r.hset(name, "myKey", info)  Comparing the hset vs hmset in docs isn't clear to me.

Answer: You may execute multiple hset for each field/value pair in hmset. r.hset('myKey', 'users', 10) r.hset('myKey', 'timestamp', datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')) r.hset('myKey', 'yet-another-field', 'yet-another-value')   first parameter is the key name second parameter is the field name third parameter is the value of the field.
Question: The question is in the title: Is there a GO equivalent to python's virtualenv? What is the prefered work flow to start a new project?

Answer: Go modules, which are built into the tooling since Go 1.12 (or 1.11 with a special flag turned on). Create a directory outside of your GOPATH (i.e. basically anywhere), create a go.mod using go mod init (which gives your module a declared importpath), and start working. There's no need to "activate" an environment like with venv; the Go 1.12+ tooling will automatically work within the current module when one is detected, e.g. any go get will happen within module scope. Although the Go blog entry I linked to mostly focuses on creating a library within a module, which you may want to publish to allow using from elsehwere, this isn't a requirement. You can create a module to hold a program (package main) as well, and there is no need to publish it (although if you do, it becomes possible to use go get to install the program).
Question: From what I understood when doing research on the Python GIL, is that only one thread can be executed at the once (Whoever holds the lock). However, if that is true, then why would this code only take 3 seconds to execute, rather than 15 seconds? import threading import time def worker(): """thread worker function""" time.sleep(3) print 'Worker' for i in range(5): t = threading.Thread(target=worker) t.start()  By intuition about threads, I would have thought this would take 3 seconds, which it does. However after learning about the GIL and that one thread can be executing at once, now I look at this code and think, why does it not take 15 seconds?

Answer: Mario's answer is a good high level answer. If you're interested in some details of how this is implemented: in CPython, the implementation of time.sleep wraps its select system call with Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS: https://github.com/python/cpython/blob/7ba1f75f3f02b4b50ac6d7e17d15e467afa36aac/Modules/timemodule.c#L1880-L1882 These are macros that save and restore the thread state, while releasing/acquiring the GIL: https://github.com/python/cpython/blob/7c59d7c9860cdbaf4a9c26c9142aebd3259d046e/Include/ceval.h#L86-L94 https://github.com/python/cpython/blob/4c9ea093cd752a6687864674d34250653653f743/Python/ceval.c#L498
Question: I have a problem with loading the previously saved model. This is my save: def build_rnn_lstm_model(tokenizer, layers): model = tf.keras.Sequential([ tf.keras.layers.Embedding(len(tokenizer.word_index) + 1, layers,input_length=843), tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(layers, kernel_regularizer=l2(0.01), recurrent_regularizer=l2(0.01), bias_regularizer=l2(0.01))), tf.keras.layers.Dense(layers, activation='relu', kernel_regularizer=l2(0.01), bias_regularizer=l2(0.01)), tf.keras.layers.Dense(layers/2, activation='relu', kernel_regularizer=l2(0.01), bias_regularizer=l2(0.01)), tf.keras.layers.Dense(1, activation='sigmoid') ]) model.summary() model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy',f1,precision, recall]) print("Layers: ", len(model.layers)) return model model_path = str(Path(__file__).parents[2]) + os.path.sep + 'model' data_train_sequence, data_test_sequence, labels_train, labels_test, tokenizer = get_training_test_data_local() model = build_rnn_lstm_model(tokenizer, 32) model.fit(data_train_sequence, labels_train, epochs=num_epochs, validation_data=(data_test_sequence, labels_test)) model.save(model_path + os.path.sep + 'auditor_model', save_format='tf')  After this I can see that auditor_model is saved in model directory. now I would like to load this model with: model = tf.keras.models.load_model(model_path + os.path.sep + 'auditor_model')  but I get:  ValueError: Unable to restore custom object of type _tf_keras_metric currently. Please make sure that the layer implements get_configand from_config when saving. In addition, please use the custom_objects arg when calling load_model().  I have read about custom_objects in TensorFlow docs but I don't understand how to implement it while I use no custom layers but the predefined ones. Could anyone give me a hint how to make it work? I use TensorFlow 2.2 and Python3

Answer: Your example is missing the definition of f1, precision and recall functions. If the builtin metrics e.g. 'f1' (note it is a string) do not fit your usecase you can pass the custom_objects as follows: def f1(y_true, y_pred): return 1 model = tf.keras.models.load_model(path_to_model, custom_objects={'f1':f1})
Question: I have three indexes, all three of them share a particular key-value pair. When I do a blanket search with the api "http://localhost:9200/_search" using the request body {"query":{ "query_string": { "query":"city*" } } }  It is only returning results from two of the indexes. I tried using the same request body by altering the url to search only in that missed index "http://localhost:9200/index_name/_search" and that's working. Am I missing anything here? The code for inserting all three indexes follow the same procedure and I used elasticsearch-py to ingest the data. I'm using the GET HTTP method and also tried the POST HTTP method. Both returns the same results. Elasticsearch version is 7.6.0. Results for specific index search is like the one below { "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 1.0, "hits": [ { "_index": "index_name", "_type": "meta", "_id": "LMRqDnIBh5wU6Ax_YsOD", "_score": 1.0, "_source": { "table_schema": "test_table", "table_name": "citymaster_old" } } ] } } 

Answer: The reason might be that you haven't provided the size parameter in the query. This limits the result count to 10 by default. Out of all the results the top 10 might be from the two index even thought the match is present in third index as well. This in turn giving the perception that result from third index are not being returned. Try adding size parameter. { "query": { "query_string": { "query": "city*" } }, "size": 20 }  You can figure out the number of documents that matched the query by the total key in response "total": { "value": 1, "relation": "eq" }
Question: So, I want to split a given dataframe into two dataframes based on an if condition for a particular column. I am currently achieving this by iterating over the whole dataframe two times. Please suggest some ways to improve this.  player score dan 10 dmitri 45 darren 15 xae12 40  Like in the above dataframe, I want to split the df into two such that one df contains rows with the players scoring less than 15 and the other df contains the remaining rows. I want to this with one iteration only. (Also, if the answer can be generic for n dfs, it will help me a lot) Thanks in advance.

Answer: IICU Use boolean select m=df.score>15 Lessthan15=df[~m] Morethan15=df[m]  Morethan15  LessThan15
Question: I want to use a semaphore with a gather() to limit api calls. I think I have to use create_task() but I obtain a runtime error: "RuntimeError: await wasn't used with future". How can I fix it?  Here is the code: import asyncio # pip install git+https://github.com/sammchardy/python-binance.git@00dc9a978590e79d4aa02e6c75106a3632990e8d from binance import AsyncClient async def catch_up_aggtrades(client, symbols): tasks = asyncio.create_task(get_historical_aggtrades(client, symbol) for symbol in symbols) sem = asyncio.Semaphore(1) async with sem: await asyncio.gather(*tasks) async def get_historical_aggtrades(client, symbol): async for trade in client.aggregate_trade_iter(symbol, '1 day ago UTC'): print(f"symbol {symbol}") async def main(): client = await AsyncClient.create() symbols = ['BTCUSDT', 'ETHUSDT', 'BNBUSDT'] await catch_up_aggtrades(client, symbols) if __name__ == "__main__": loop = asyncio.get_event_loop() loop.run_until_complete(main()) 

Answer: A sempahore limiting a resource usage is actually a very simple concept. It is similar to counting free parking lots. (-1 when a car enters, +1 when it leaves). When the counter drops to zero, a queue of waiting cars starts to build. That means:  one semaphore per resource initial value = upper limit of concurrent resource users each resource usage is guarded by async with sem:  The existing code: sem = asyncio.Semaphore(1) async with sem: await asyncio.gather(*tasks)  limits the use of asyncio.gather to 1 task gathering at a time. It does not limit the tasks, just their gathering. Since the gather is called just once anyway, the semaphore does not change anything. Your program might be changed to (including the issue resolved in comments): LIMIT = 1 async def catch_up_aggtrades(client, symbols): sem = asyncio.Semaphore(LIMIT) tasks = [asyncio.create_task(get_historical_aggtrades(client, symbol, sem)) for symbol in symbols] await asyncio.gather(*tasks) async def get_historical_aggtrades(client, symbol, sem): async with sem: async for trade in client.aggregate_trade_iter(symbol, '1 day ago UTC'): print(f"symbol {symbol}")
Question: I'm using windows10 64-bit, I downloaded Python 3.8.1 for windows x86-64. But when I type "python" in cmd, the output says "win32". Where does that come from? Or is that normal? C:\>python Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>>  New in Python, please could anyone help?

Answer: tl;dr: "Win32" is still the common name of the Windows API, regardless of whether you're using it on a 32-bit or 64-bit machine. Background The Windows API was once called WinAPI, and the earliest versions ran on 16-bit computers. When they started to make versions for 32-bit computers, they had to modify a bunch of the API's functions and how some parameters were passed in window messages. This was largely due to Win16 having used some tricks to save memory. But the names of these functions and messages were mostly unchanged, so it was often necessary to distinguish between the 16- and 32-bit versions when, for example, you're trying to figure out what information the WPARAM of a window message is carrying. The term "WinAPI" gave way to "Win16" and "Win32", and eventually Win16 was left behind. When Windows built versions for 64-bit computers, the API did not have to undergo the same kinds of changes (at the source level). So there was no need to change the term. Python's version line simply means that this version is made to run on Windows.
Question: I have a Django project running on my local machine with dev server manage.py runserver and I'm trying to run it with Uvicorn before I deploy it in a virtual machine. So in my virtual environment I installed uvicorn and started the server, but as you can see below it fails to find Django static css files. (envdev) user@lenovo:~/python/myproject$ uvicorn myproject.asgi:application --port 8001 Started server process [17426] Waiting for application startup. ASGI 'lifespan' protocol appears unsupported. Application startup complete. Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) INFO: 127.0.0.1:45720 - "GET /admin/ HTTP/1.1" 200 OK Not Found: /static/admin/css/base.css Not Found: /static/admin/css/base.css INFO: 127.0.0.1:45720 - "GET /static/admin/css/base.css HTTP/1.1" 404 Not Found Not Found: /static/admin/css/dashboard.css Not Found: /static/admin/css/dashboard.css INFO: 127.0.0.1:45724 - "GET /static/admin/css/dashboard.css HTTP/1.1" 404 Not Found Not Found: /static/admin/css/responsive.css Not Found: /static/admin/css/responsive.css INFO: 127.0.0.1:45726 - "GET /static/admin/css/responsive.css HTTP/1.1" 404 Not Found  Uvicorn has an option --root-path so I tried to specify the directory where these files are located but there is still the same error (path is correct). How can I solve this issue?

Answer: When not running with the built-in development server, you'll need to either  use whitenoise which does this as a Django/WSGI middleware (my recommendation) use the classic staticfile deployment procedure which collects all static files into some root and a static file server is expected to serve them. Uvicorn doesn't seem to support static file serving, so you might need something else too (see e.g. https://www.uvicorn.org/deployment/#running-behind-nginx). (very, very unpreferably!) have Django serve static files like it does in dev
Question: Over the weekends I have upgraded my Ubuntu to 20.04, and I tried creating virtualenvironment with python 3.8.2, and pip install requirements.txt. In requirement.txt, I am installing some code from private gitlab repositories.  Previously, if I do pip freeze, I was able to see all packages name and version (formatted as package_name == version. However, if I do pip freeze, now I see something like this... pkg1 @ file:///tmp/tmp44ir_jik pkg2 @ file:///tmp/tmp5pijtzbq ...  (pkg1 and pkg2 are both from pip installing private git repo) I would like to somehow display the version, but don't know how to. I mean, pip list does show the version, but I am writing a script and would like to use pip freeze for it. How can I get pip freeze to show how it use to before (with the format as pkg_name==pkg_version)? Thanks in advance.

Answer: You can use pip list --format=freeze instead.
Question: I have a pandas dataframe that is datetime indexed and it looks like this:  Datetime 2020-05-11 14:00:00-03:00 0.097538 2020-05-11 14:30:00-03:00 -0.083788 2020-05-11 15:00:00-03:00 -0.074128 2020-05-11 15:30:00-03:00 0.059725 2020-05-11 16:00:00-03:00 0.041369 2020-05-11 16:30:00-03:00 0.034388 2020-05-12 10:00:00-03:00 0.006814 2020-05-12 10:30:00-03:00 -0.005308 2020-05-12 11:00:00-03:00 -0.036952 2020-05-12 11:30:00-03:00 -0.070307 2020-05-12 12:00:00-03:00 0.102004 2020-05-12 12:30:00-03:00 -0.139317 2020-05-12 13:00:00-03:00 -0.167589 2020-05-12 13:30:00-03:00 -0.179942 2020-05-12 14:00:00-03:00 0.182351 2020-05-12 14:30:00-03:00 -0.160736 2020-05-12 15:00:00-03:00 -0.150033 2020-05-12 15:30:00-03:00 -0.141862 2020-05-12 16:00:00-03:00 -0.121372 2020-05-12 16:30:00-03:00 -0.095990 Name: result_col, dtype: float64  My need is to mark the rows where it changes signal, from negative to positive and vice-versa. Any thoughts on how to achieve it? Edit: I need +1 on the cross up and -1 on the cross down.

Answer: Let us try  import numpy as np np.sign(data).diff().ne(0)
Question: I have a python script with the following code: print("Hello Deno")  I want to run this python script (test.py) from test.ts using Deno. This is the code in test.ts so far: const cmd = Deno.run({cmd: ["python3", "test.py"]});  How can I get the output, of the python script in Deno?

Answer: To execute a python script from Deno you need to use Deno.Command const command = new Deno.Command('python3', { args: [ "test.py" ], }); const { code, stdout, stderr } = await command.output(); console.log(new TextDecoder().decode(stdout)); console.log(new TextDecoder().decode(stderr));   Old answer: Deno.run is now deprecated in favor of Deno.Command Deno.run returns an instance of Deno.Process. In order to get the output use .output(). Don't forget to pass stdout/stderr options if you want to read the contents. // --allow-run const cmd = Deno.run({ cmd: ["python3", "test.py"], stdout: "piped", stderr: "piped" }); const output = await cmd.output() // "piped" must be set const outStr = new TextDecoder().decode(output); const error = await cmd.stderrOutput(); const errorStr = new TextDecoder().decode(error); cmd.close(); // Don't forget to close it console.log(outStr, errorStr);   If you don't pass stdout property you'll get the output directly to stdout  const p = Deno.run({ cmd: ["python3", "test.py"] }); await p.status(); // output to stdout "Hello Deno" // calling p.output() will result in an Error p.close()   You can also send the output to a File // --allow-run --allow-read --allow-write const filepath = "/tmp/output"; const file = await Deno.open(filepath, { create: true, write: true }); const p = Deno.run({ cmd: ["python3", "test.py"], stdout: file.rid, stderr: file.rid // you can use different file for stderr }); await p.status(); p.close(); file.close(); const fileContents = await Deno.readFile(filepath); const text = new TextDecoder().decode(fileContents); console.log(text)   In order to check status code of the process you need to use .status() const status = await cmd.status() // { success: true, code: 0, signal: undefined } // { success: false, code: number, signal: number }   In case you need to write data to stdin you can do it like this: const p = Deno.run({ cmd: ["python", "-c", "import sys; assert 'foo' == sys.stdin.read();"], stdin: "piped", }); // send other value for different status code const msg = new TextEncoder().encode("foo"); const n = await p.stdin.write(msg); p.stdin.close() const status = await p.status(); p.close() console.log(status)   You'll need to run Deno with: --allow-run flag in order to use Deno.run
Question: Trying to debug a pytest unit test gives me  exec code in self.locals SyntaxError: Missing parentheses in call to 'exec'  on very simple code. What could be causing it?

Answer: Don't have a package/directory/file/module named code in your code, because it conflicts with pytest. Changing to src solved this. I found the answer here:  it turned out to be a conflict with my own python module called 'code' and one in use by the debugger. I changed my module name and the debugger began working. This article pointed me to the solution: https://superuser.com/questions/1385995/my-pycharm-run-is-working-but-debugging-is-failing  This took me a while to find, so I thought I'd post it here for easy googling.
Question: More generally, how to change the color values for a subset of the boxes properties in a seaborn boxplot? Be that the median, the whiskers, or such. I'm particularly interested in how to change the median value, as I have to create plots which have a dark colour and the median line can't be seen against it. Here's some example code: import matplotlib.pyplot as plt import seaborn as sns fig, ax1 = plt.subplots(figsize=(15,5) colours = ["#000184", "#834177"] sns.set_palette(sns.color_palette(colours)) tips = sns.load_dataset("tips") sns.boxplot(x="day", y="total_bill", hue="smoker", data=tips, ax=ax1)  Which creates:  As can be seen, it's quite hard to see the median line against the blue here. Note - I don't want to change the entire boxplot colour, just a subsection of the lines (in this case the median). Also - the option of changing it for a particular group (in this case the Yes smokers) is needed, as the colour might not apply to both groups.

Answer: Update: the old version of this answer used the index in the generated lines to guess which line of the plot corresponds to a median line. The new version attaches a special label to each of the means, that can be retrieved later to filter out only those lines. You can use medianprops to change the color of the median line. However, there only will be one color for all boxplots. If you need to assign individual colors, you can loop through the generated lines. This makes the code more robust towards different options and possible future changes. ax1.get_lines() gives a list of all Line2D objects created by the boxplot. There will be a Line2D object for each of the elements that make up a box: 4 for the whiskers (2 vertical and 2 horizontal lines), 1 for the median and also 1 for the outliers (it is a Line2D that shows markers without connecting lines). In case showmeans=True there will also be a Line2D with a marker for the mean. So in that case there would be 7 Line2Ds per box, in this case 6. Note that a boxplot also accepts a parameter notch=True which shows a confidence interval around the median. In the case of the current plot, most of the confidence intervals are larger than the quartiles, which create some flipped notches. Depending on your real application, such notches can help to accentuate the median. import matplotlib.pyplot as plt import seaborn as sns fig, ax1 = plt.subplots(figsize=(15,5)) colours = ["#000184", "#834177"] sns.set_palette(sns.color_palette(colours)) tips = sns.load_dataset("tips") sns.boxplot(x="day", y="total_bill", hue="smoker", data=tips, ax=ax1, medianprops={'color': 'red', 'label': '_median_'}) median_colors = ['orange', 'yellow'] median_lines = [line for line in ax1.get_lines() if line.get_label() == '_median_'] for i, line in enumerate(median_lines): line.set_color(median_colors[i % len(median_colors)]) plt.show()
Question: I'm trying to emulate Chrome for iPhone X with Selenium emulation and Python, as follow: from selenium import webdriver mobile_emulation = { "deviceName": "iphone X" } chrome_options = webdriver.ChromeOptions() chrome_options.add_experimental_option("mobileEmulation", mobile_emulation) driver = webdriver.Chrome(r'C:\Users\Alex\PythonDev\chromedriver') driver.get('https://www.google.com')  However, nothing happens: my page is still a normal browser page, and I don't see it as a mobile page. What is missing or wrong in my code?

Answer: You might have found an answer by now, but here's a general one: In your code example, your driver has no chance to know that you want it to emulate another device. Here's full working code: from selenium import webdriver mobile_emulation = { "deviceName": "your device" } chrome_options = webdriver.ChromeOptions() chrome_options.add_experimental_option("mobileEmulation", mobile_emulation) driver = webdriver.Chrome(options=chrome_options) #sometimes you have to insert your execution path driver.get('https://www.google.com')  Make sure that Chrome supports your device and your device name is spelled correctly.