eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to add a very simple line across a bar chart in plotly and am struggling to to this. My dataframe contains one column with bins and the other with returnsdata and can be copied here: {'bins': {0: '(-23.077, 25.877]', 1: '(25.877, 34.666]', 2: '(34.666, 42.552]', 3: '(42.552, 46.044]', 4: '(46.044, 49.302]', 5: '(49.302, 52.746]', 6: '(52.746, 57.075]', 7: '(57.075, 62.349]', 8: '(62.349, 69.171]', 9: '(69.171, 90.975]'}, 'returns': {0: 0.39754, 1: 0.6817, 2: -0.1918399999999998, 3: -0.44406, 4: -0.6611199999999998, 5: -0.0742857142857142, 6: 0.25304, 7: 0.4166, 8: 0.97648, 9: 0.0539999999999999}}  I have created a plotly bar chart from this using the following code: fig = px.bar(dfs, x='bins', y='returns') fig.show()   I want to add a constant line across the bar chart that represents a benchmark score and have looked at this: Plotly: How to add trendline to a bar chart? The methods seem to be depreciated and I can't seem to find any way to do this. The benchmark list is this: [0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858, 0.14080542857142858]  I want it to look this this (the line is suppose to be straight, apologies for the terrible paint job edit) Would anyone know how to do this? 

Answer: You can use the Plotly's horizontal and vertical shapes to add a horizontal line: fig.add_hline(y=0.14080542857142858)
Question: My Initial import looks like this and this code block runs fine. # Libraries to help with reading and manipulating data import numpy as np import pandas as pd # Libraries to help with data visualization import matplotlib.pyplot as plt import seaborn as sns sns.set() # Removes the limit for the number of displayed columns pd.set_option("display.max_columns", None) # Sets the limit for the number of displayed rows pd.set_option("display.max_rows", 200) # to split the data into train and test from sklearn.model_selection import train_test_split # to build linear regression_model from sklearn.linear_model import LinearRegression # to check model performance from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score  But when I try to following command I get the error ModuleNotFoundError: No module named 'sklearn.externals.joblib' I tried to use !pip to install all the modules and other suggestions for this error it didnt work. This is google colab so not sure what I am missing from mlxtend.feature_selection import SequentialFeatureSelector as SFS 

Answer: For the second part you can do this to fix it, I copied the rest of your code as well, and added the bottom part. # Libraries to help with reading and manipulating data import numpy as np import pandas as pd # Libraries to help with data visualization import matplotlib.pyplot as plt import seaborn as sns sns.set() # Removes the limit for the number of displayed columns pd.set_option("display.max_columns", None) # Sets the limit for the number of displayed rows pd.set_option("display.max_rows", 200) # to split the data into train and test from sklearn.model_selection import train_test_split # to build linear regression_model from sklearn.linear_model import LinearRegression # to check model performance from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score # I changed this part !pip install mlxtend import joblib import sys sys.modules['sklearn.externals.joblib'] = joblib from mlxtend.feature_selection import SequentialFeatureSelector as SFS  it works for me.
Question: The docs say:  Data Matrix used in XGBoost. DMatrix is an internal data structure that is used by XGBoost, which is optimized for both memory efficiency and training speed. You can construct DMatrix from multiple different sources of data.  I get this bit but what's the difference/use of DMatrix instead of a Pandas Dataframe?

Answer: When using the XGBoost Python package you can choose between two different APIs to train your model. XGB's own Learning API and the Scikit-Learn API. When using the Scikit-Learn API data is passed to the model as numpy array or pandas dataframes. When using the Learning API data is passed using the DMatrix. Have a look at the python examples, to see both APIs used. Basically you already found the "use of DMatrix instead of a Pandas Dataframe" in the docs: It is a data structure the XGBoost developers created for "memory efficiency and training speed" with their machine learning library.
Question: I wrote below code using PyTorch and ran into the runtime error: tns = torch.tensor([1,0,1]) tns.mean() --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-666-194e5ab56931> in <module> ----> 1 tns.mean() RuntimeError: mean(): input dtype should be either floating point or complex dtypes. Got Long instead.  However, if I change the tensor to float, the error goes away: tns = torch.tensor([1.,0,1]) tns.mean() --------------------------------------------------------------------------- tensor(0.6667)  My question is why the error happens. The data type of the first tensor is int64 instead of Long, why does PyTorch take it as Long?

Answer: This is because torch.int64 and torch.long both refer to the same data type, of 64-bit signed integers. See here for an overview of all data types.
Question: I want to download a txt and pdf files to a specific folder. But it just downloads them on another folder. The website http://demo.automationtesting.in/FileDownload.html. Is there something wrong with the code or I didn't put the correct location of the folder? import time from selenium import webdriver from selenium.webdriver.chrome.options import Options chromeOptions=Options() chromeOptions.add_experimental_option("prefs", {"download.default_dictionary": "C:\DownloadedAutomationFiles"}) driver=webdriver.Chrome(executable_path="D:\ChromeDriverExtracted\chromedriver", chrome_options=chromeOptions) driver.get("http://demo.automationtesting.in/FileDownload.html") driver.maximize_window() driver.find_element_by_id("textbox").send_keys("testing") driver.find_element_by_id("createTxt").click() #generate file button driver.find_element_by_id("link-to-download").click() #dowload link #Download PDF FILE driver.find_element_by_id("pdfbox").send_keys("testing download text file") driver.find_element_by_id("createPdf").click() #generate file button driver.find_element_by_id("pdf-link-to-download").click() #dowload link time.sleep(2) driver.close() 

Answer: To download the required file within Automation Demo Site to a specific folder using Selenium, ChromeDriver and google-chrome you need to pass the preference "download.default_directory" along with the value (location of the directory) through add_experimental_option() and you can use the following solution: Code Block: from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.service import Service from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC options = Options() options.add_argument("start-maximized") options.add_experimental_option("excludeSwitches", ["enable-automation"]) options.add_experimental_option('useAutomationExtension', False) options.add_experimental_option("prefs", { "download.default_directory": r"C:\Data_Files\output_files" }) s = Service('C:\\BrowserDrivers\\chromedriver.exe') driver = webdriver.Chrome(service=s, options=options) driver.get("http://demo.automationtesting.in/FileDownload.html") driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Download")))) WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "textbox"))).send_keys("testing") WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "createTxt"))).click() WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "link-to-download"))).click()  Snapshot:
Question: This is a follow-up to my last post. The code works without any errors and can calculate the next best move. I've been looking into how to incorporate transposition tables and move ordering into my negamax function to make it run faster and more accurately, but it seems somewhat difficult and advanced for a beginner like myself. You can find my code here. While researching on the chess programming wiki I found some sample code for transposition tables: def negamax(node, depth, alpha, beta, color): alphaOrig = alpha ## Transposition Table Lookup; node is the lookup key for ttEntry ttEntry = transpositionTableLookup(node) if ttEntry.is_valid is True and ttEntry.depth >= depth: if ttEntry.flag == EXACT : return ttEntry.value if ttEntry.flag == LOWERBOUND: alpha = max(alpha, ttEntry.value) if ttEntry.flag == UPPERBOUND: beta = min(beta, ttEntry.value) if alpha >= beta: return ttEntry.value if depth == 0 or node is terminal_node: return color* heuristic_value_of_node childNodes = domove(node) childNodes = orderMoves(childNodes) bestValue = -99999 for child in childNodes: bestValue = max(bestValue, -negamax(child, depth - 1, -beta, -alpha, -color)) alpha = max(alpha, bestValue) if alpha >= beta: break ##Transposition Table Store; node is the lookup key for ttEntry ttEntry.value = bestValue if bestValue <= alphaOrig: ttEntry.flag = UPPERBOUND if bestValue >= beta: ttEntry.flag = LOWERBOUND else: ttEntry.flag = EXACT ttEntry.depth = depth transpositionTableStore(node, ttEntry) return bestValue  I tried making a few modifications to integrate it into my code, but I didn't get any results out of it. I've also seen something about storing hash keys with a Zobrist key for the positions, but I didn't understand well how it worked so I dropped the idea. Currently somewhat stuck with these problems and don't know what the next step is.

Answer: To use transposition tables you "need" to use Zorbrist hashing. The hashing gives each position an (almost) unique code that you store in the transposition table along with its evaluation. Then, to explain it easily, if the current position you are searching is found in your transposition table you won't have to evaluate it again, you just use the stored value. Zorbrist keys are a nightmare to get right and very hard to debug. If it helps you can check out my implementation in the Endamat Chess Engine, but since you might have a different approach it might be easier to just read on how Zorbrist keys works and try to get it right for your implementation.
Question: How would one go about importing only a specific class from a Python module using its path? I need to import a specific class from a Python file using the file path. I have no control over the file and its completely outside of my package. file.py: class Wanted(metaclass=MyMeta): ... class Unwanted(metaclass=MyMeta): ...  The metaclass implementation is not relevant here. However, I will point out that it's part of my package and I have full control over it. Import example: spec = importlib.util.spec_from_file_location(name='Wanted', location="path_to_module/mudule.py") module = importlib.util.module_from_spec(spec) spec.loader.exec_module(module)  This works, and Wanted is imported. The problem is that Unwanted is also imported. In fact, as long as there is any string value given for name (including an empty string), both Wanted and Unwanted are imported from the module. This has the same effect as in the example before, where both Wanted and Unwanted are imported: importlib.util.spec_from_file_location(name='random string', location="path_to_module/mudule.py")  I'm not looking for a specific solution using importlib; any reasonable way will do. I will point out that I don't have a need of using the class when it's imported, I only need the import to happen and my metaclass will take care of the rest.

Answer: If I am not mistaken, the name parameter is just used to name the module you are importing. But, more importantly, when you are importing any module, you are executing the whole file, which means that in your case both of these classes will be created. It would not matter whether you wrote from file import Wanted, import file or used any other form of the import statement.  A Python program is constructed from code blocks. A block is a piece of Python program text that is executed as a unit. The following are blocks: a module, a function body, and a class definition.  Source: https://docs.python.org/3/reference/executionmodel.html#structure-of-a-program
Question: I would like to remove all characters from a first string s1 exactly the number of times they appear in another string s2, i.e. if s1 = "AAABBBCCCCCCD" and s2 = "ABBCCC" then the result should be s = "AABCCCD". (The order of the characters in the resulting string is actually irrelevant but it's a plus if it can be preserved.) The following rather crude code can do this: def reduce_string(s1, s2): s = s1 for c in s2: if c in s: s = s.replace(c, "", 1) return(s) # examples reduce_string("AAABBBCCCCCCD", "ABBCCC") reduce_string("AAABBBCCCCCCD", "ABBCCCE")  My question is, can the same be achieved by clever use of some built-in function or at least in a more elegant way? Thank you for all your answers!

Answer: You can use counter objects. Subtract one against the other and join the remaining elements together. from collections import Counter s1 = "AAABBBCCCCCCD" s2 = "ABBCCC" counter = Counter(s1) counter.subtract(Counter(s2)) result = ''.join(counter.elements()) print(result)  AABCCCD  As a one-liner: print(''.join((Counter(s1) - Counter(s2)).elements()))
Question: Let's say I have an enum class Color(Enum): RED = "RED" GREEN = "GREEN" BLUE = "BLUE"  I wanted to create a ColorDict class that works as a native python dictionary but only takes the Color enum or its corresponding string value as key. d = ColorDict() # I want to implement a ColorDict class such that ... d[Color.RED] = 123 d["RED"] = 456 # I want this to override the previous value d[Color.RED] # ==> 456 d["foo"] = 789 # I want this to produce an KeyError exception  What's the "pythonic way" of implementing this ColorDict class? Shall I use inheritance (overriding python's native dict) or composition (keep a dict as a member)?

Answer: A simple solution would be to slightly modify your Color object and then subclass dict to add a test for the key. I would do something like this: class Color(Enum): RED = "RED" GREEN = "GREEN" BLUE = "BLUE" @classmethod def is_color(cls, color): if isinstance(color, cls): color=color.value if not color in cls.__members__: return False else: return True class ColorDict(dict): def __setitem__(self, k, v): if Color.is_color(k): super().__setitem__(Color(k), v) else: raise KeyError(f"Color {k} is not valid") def __getitem__(self, k): if isinstance(k, str): k = Color(k.upper()) return super().__getitem__(k) d = ColorDict() d[Color.RED] = 123 d["RED"] = 456 d[Color.RED] d["foo"] = 789  In the Color class, I have added a test function to return True or False if a color is/isn't in the allowed list. The upper() function puts the string in upper case so it can be compared to the pre-defined values. Then I have subclassed the dict object to override the __setitem__ special method to include a test of the value passed, and an override of __getitem__ to convert any key passed as str into the correct Enum. Depending on the specifics of how you want to use the ColorDict class, you may need to override more functions. There's a good explanation of that here: How to properly subclass dict and override __getitem__ & __setitem__
Question: Within my code, I am attempting to gather the Application Default Credentials from the associated service account in Cloud Build: from google.auth import default credentials, project_id = default()  This works fine in my local space because I have set the environment variable GOOGLE_APPLICATION_CREDENTIALS appropriately. However, when this line is executed (via a test step in my build configuration) within Cloud Build, the following error is raised: google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started  This is confusing me because, according to the docs:  By default, Cloud Build uses a special service account to execute builds on your behalf. This service account is called the Cloud Build service account and it is created automatically when you enable the Cloud Build API in a Google Cloud project. Read Here   If the environment variable GOOGLE_APPLICATION_CREDENTIALS isn't set, ADC uses the service account that is attached to the resource that is running your code. Read Here  So why is the default call not able to access the Cloud Build service account credentials?

Answer: There is a trick: you have to define the network to use in your Docker build. Use the parameter --network=cloudbuild, like that steps: - name: gcr.io/cloud-builders/docker entrypoint: 'docker' args: - build - '--no-cache' - '--network=cloudbuild' - '-t' - '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA' - . - '-f' - 'Dockerfile' ...  You can find the documentation here
Question: Say I have a Numpy array of N = 10 random float numbers: import numpy as np np.random.seed(99) N = 10 arr = np.random.uniform(0., 10., size=(N,)) print(arr) out[1]: [6.72278559 4.88078399 8.25495174 0.31446388 8.08049963 5.6561742 2.97622499 0.46695721 9.90627399 0.06825733]  I want to find all unique pairs of numbers that are not different from each other more than a tolerance tol = 1. (i.e. absolute difference <= 1). Specifically, I want to get all unique pairs of indexes. The indexes of each close pair should be sorted, and all close pairs should be sorted by the first index. I managed to write the following working code: def all_close_pairs(arr, tol=1.): res = set() for i, x1 in enumerate(arr): for j, x2 in enumerate(arr): if i == j: continue if np.isclose(x1, x2, rtol=0., atol=tol): res.add(tuple(sorted([i, j]))) res = np.array(list(res)) return res[res[:,0].argsort()] print(all_close_pairs(arr, tol=1.)) out[2]: [[1 5] [2 4] [3 7] [3 9] [7 9]]  However, in reality I have an array of N = 1000 numbers, and my code becomes extremely slow due to the nested for loops. I believe there are much more efficient ways to do this with Numpy vectorization. Does anyone know the fastest way to do this in Numpy?

Answer: This is a solution with pure numpy operations. It seems pretty fast on my machine, but I don't know what kind of speed we're looking for. def all_close_pairs(arr, tol=1.): N = arr.shape[0] # get indices in the array to consider using meshgrid pair_coords = np.array(np.meshgrid(np.arange(N), np.arange(N))).T # filter out pairs so we get indices in increasing order pair_coords = pair_coords[pair_coords[:, :, 0] < pair_coords[:, :, 1]] # compare indices in your array for closeness is_close = np.isclose(arr[pair_coords[:, 0]], arr[pair_coords[:, 1]], rtol=0, atol=tol) return pair_coords[is_close, :]
Question: Lately in PyCharm (I don't know which version started it, I'm currently running 2021.2.3 Pro), I'm getting warnings that don't make sense. For example, this snippet: d = {1: 2, 3: 4, 5: 6} for v in d.values(): print(v)  Triggers the following warning:  Expected type 'collections.Iterable', got 'ValuesView' instead  In the above snippet, replacing values() with keys() gives a similar warning. BTW, the return value of d.values() is dict_values and not ValuesView: type(d.values()) <class 'dict_values'>  Why does PyCharm give me this warning, when this has always been the correct way to iterate over dictionary keys/values? It might be a bug in PyCharm, but maybe I'm missing something.  EDIT: Even the sample code at https://docs.python.org/3.8/library/stdtypes.html#dict-views gets this warning, see screenshot.

Answer: This is a regression of a known bug in the PyCharm 2021.2.3 linter, see PY-41457 on the JetBrains bug tracker. It doesn't happen for me using the immediately previous PyCharm 2021.2.2 Pro version. The solution, for now, is to report the regression on the JetBrains bug tracker and wait for a fix. There's nothing wrong with your code.
Question: In Perl I can do something like this to dynamically create an array based on some conditions: my @arr = ( $foo_defined ? 'foo' : (), $bar_defined ? 'bar' : (), $baz_defined ? 'baz' : (), );  This creates an array of up to 3 elements based on the values of the variables. What would be the closest alternative to this in Python?

Answer: A close literal translation might be arr = list(chain( ['foo'] if foo_defined else [], ['bar'] if bar_defined else [], ['baz'] if baz_defined else [], ))  a if b else c is the Python equivalent of b ? a : c. chain concatenates multiple iterable values into a single sequence, similar to how ( (), (), () ) builds an array consisting of array elements (rather than nesting the arrays). list turns the chain object into an actual list. The differing semantics of a Perl array and a Python list make it difficult to do exactly the same thing as tersely as Perl does. Python lacks the ability to have an expression evaluate to nothing to simulate the Perl array handling in your example. (Or put another way, [ [], [], [] ] in Python creates a list containing 3 empty lists, rather than a single empty list.)  Patrick Artner's answer takes advantage of the fact that a list comprehension can selectively include values from one iterable into the list under construction. A combination of his and my answers might look like arr = [s for s in ['foo' if foo_defined else None, 'bar' if bar_defined else None, 'baz' if baz_defined else None] if s is not None]  The difference, though, is that a full list is built first, then a second, final list is built from the first one.  Rolv Apneseth's answer dispenses with a single expression, building the result from an initially empty list and conditionally adding values to it one item at a time. It's terse, but the simulation of Perl-style statement modifiers isn't really idiomatic in Python. Instead, one would just use if statements: arr = [] if foo_defined: arr.append("foo") if bar_defined: arr.append("bar") if baz_defined: arr.append("baz")
Question: I need to write a function that receives an non-negative integer and returns: [] for n=0 [[]] for n=1 [[],[[]]] for n=2 [[],[[]],[[],[[]]]] for n=3  And so on. For n, we will receive an n sized list, so that in index i there will be all the i-1 elements from the list. I don't know how to explain that better, English isn't my first language. I'm not allowed to use list slicing or loops and I'm supposed to create deep copies of each list, without the copy module. I'm not allowed to let 2 different lists or indexes point to the same list in memory. This is what I tried: def list_seq(x, outer_list=[]): if x == 0: return [] outer_list.append(list_seq(x-1,outer_list)) return outer_list  And the output for print(list_seq(2)) is [[], [...]].

Answer: If you can't use loops, you can use the following: def recursive_list(n): if n == 0: return [] else: return recursive_list(n-1) + [recursive_list(n-1)]  EDIT You can do the following if you want to use append: def recursive_list(n: int) -> list: if n: result = recursive_list(n-1) result.append(recursive_list(n-1)) return result return []  NOTE as pointed out in the comments, caching introduces some reference issues, so I have removed the cached versions.