eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I want to parse emails in python through the Outlook application. Running this code I get only a few of my emails. import win32com.client outlook = win32com.client.Dispatch('outlook.application') mapi = outlook.GetNamespace("MAPI") inbox= mapi.GetDefaultFolder(6) messages= inbox.Items for i in messages: message=i.subject print(message)  I have tried changing the Default Folder and it happens everywhere. What have I done wrong?

Answer: Emails were not visible beacuse they were on the server! File-> Account Settings-> Account Settings...-> double click on your Exchange account-> set the Mail to keep offline slider to: All. Found it here !
Question: I came across an interview question that went like this:  There are factories in an area which produce a pollutive gas and filters are to be installed at each factory to reduce the pollution. Each filter installed would half the pollution in that factory. Each factory can have multiple filters. There is a list of N integers representing the level of pollution in each of the N factories in the area. Find the minimum number of filters needed to half the overall pollution. E.g. - Let [3, 5, 6, 1, 18] be the list of pollution levels in 5 factories  Overall pollution = 3+5+6+1+18 = 33 (target is 33/2 = 16.5)  Install a filter in factory given by index=4 -- > pollution levels will be [3, 5, 6, 1, 9]  Install a filter in factory given by index=4 -- > pollution levels will be [3, 5, 6, 1, 4.5]  Install a filter in factory given by index=2 -- > pollution levels will be [3, 5, 3, 1, 4.5]  Need 3 filters minimum to half the overall pollution.   N is an integer within the range [1....30,000]. Each element in the list is an integer within the range [0....70,000]  The solution I came up with for this was simple: Find the max in the list and half in every time until the sum is <=target def solution(A): total = sum(A) target = total/2 count = 0 while total>target: count+=1 max_p = max(A) total-= max_p/2 A.remove(max_p) A.append(max_p/2) return count  This works well, except that the time complexity seems to be O(N^2). Can someone please suggest an approach to solve this with less time complexity (preferably O(N))?

Answer: Maybe you could utilize a max heap to retrieve the worst factory more efficiently than you are right now, i.e., using a heap would allow for an O(N log N) solution: import heapq def filters_required(factories: list[int]) -> int: """Returns minimum filters required to halve pollution.""" current_pollution = sum(factories) goal_pollution = current_pollution / 2 filters = 0 factory_pollution_max_heap = [-p for p in factories] heapq.heapify(factory_pollution_max_heap) while current_pollution > goal_pollution: worst_factory = heapq.heappop(factory_pollution_max_heap) pollution = worst_factory / 2 current_pollution += pollution # Use += since pollution will be a negative number. heapq.heappush(factory_pollution_max_heap, pollution) print('DEBUG:', [-p for p in factory_pollution_max_heap], current_pollution) filters += 1 return filters def main() -> None: print(f'{filters_required(factories=[3, 5, 6, 1, 18]) = }') if __name__ == '__main__': main()  Output: DEBUG: [9.0, 6, 3, 1, 5] 24.0 DEBUG: [6, 5, 3, 1, 4.5] 19.5 DEBUG: [5, 4.5, 3, 1, 3.0] 16.5 filters_required(factories=[3, 5, 6, 1, 18]) = 3
Question: My while loop prints nothing when it is running. import os import time place = 10 running = True def Write(): j = 1 for i in range(place - 1): print("-", end = "") j += 1 print("a", end = "") for k in range(10 - j): print("-", end = "") while running: Write() time.sleep(5) if place > 1: place -= 1 os.system("clear")  When there is just the print and a time.sleep, the while loop works. while running: print("Looping...") time.sleep(5)  When there is the function and a time.sleep, the code doesn't work. while running: Write() time.sleep(5)  Please tell me how to fix this.

Answer: I got puzzled by what you discovered, and now found a solution to this interesting behavior; you can use flush=True parameter to force flushing: import os import time place = 10 running = True def Write(): j = 1 for i in range(place - 1): print("-", end = "") j += 1 print("a", end = "", flush=True) for k in range(10 - j): print("-", end = "", flush=True) while running: Write() time.sleep(1) if place > 1: place -= 1 os.system("clear")   Whether the output is buffered is usually determined by file, but if the flush keyword argument is true, the stream is forcibly flushed.—https://docs.python.org/3/library/functions.html#print  Alternatively, (i) putting print() (without end) at the end of Write, or (ii) making Write to return a string (not print inside the function) and printing the string outside the function (in the while loop) seems to work. Green Cloak Guy's solution in the comment section, i.e., sys.stdout.flush() works too. It seems to me that end='' makes python or console reluctant to show the characters eagerly (in some cases), waiting for a line to end.
Question: I am trying to create a Python pip package. This works also well. I can successfully upload and download the package and use it in the Python code. What I can't do is to use the Python package via the command line. In another StackOverflow post I found the link to a tutorial. I tried to follow it. Obviously I made a mistake. Can you guys help me please ? Installation of the package via pip here you can see that the installation worked. Unfortunately, not the whole script fit on the image. Pip does not find the package. Unfortunately, I can't embed the images directly, so I'll just embed them as links. I have created a simple Python package. It represents here only an example. Here you can see the structure of the folder Riffecs | .gitignore | .pylintrc | LICENSE | README.md | requirements.txt | setup.py | | \---riffecs __init__.py __main__.py  Here are the basic files shown. main.py from . import hello_world if __name__ == '__main__': hello_world()  and init.py def hello_world(): print("Hello world")  In the following you can see the "setup.py". I am of the opinion that I have followed the instructions. But obviously I made a mistake somewhere. Can you please help me to correct this mistake. import io import os import setuptools def read_description(): url = "README.md" """ Read and Return the description """ return io.open(os.path.join(os.path.dirname(__file__), url), encoding="utf-8").read() def def_requirements(): """ Check PIP Requirements """ with open('requirements.txt', encoding='utf-8') as file_content: pip_lines = file_content.read().splitlines() return pip_lines setuptools.setup( name="riffecs", version='0.0.3', description='test', entry_points={'console_scripts': ['hello-world=riffecs:hello_world',]}, long_description=read_description(), long_description_content_type="text/markdown", license="MIT", keywords="test - riffecs", url="https://github.com/Riffecs/riffecs", packages=["riffecs"], install_requires=def_requirements(), python_requires=">=3.6", classifiers=[ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", ], ) 

Answer: In your setup.py file you have this line... entry_points={'console_scripts': ['hello-world=riffecs:hello_world',]},  This is the entry point to calling you package via command line. This configuration is asking the entry point to be hello-world, which I tried and it runs fine. In your image however you run riffecx which is not configured as an entrypoint to the package. If you wanted the entrypoint to be riffecx. change the line to: entry_points={'console_scripts': ['riffecx=riffecs:hello_world']},  Hope this helped.
Question: I am maintaining a python package in which I did some restructuring. Now, I want to support clients who still do from my_package.old_subpackage.foo import Foo instead of the new from my_package.new_subpackage.foo import Foo, without explicitly reintroducing many files that do the forwarding. (old_subpackage still exists, but no longer contains foo.py.) I have learned that there are "loaders" and "finders", and my impression was that I should implement a loader for my purpose, but I only managed to implement a finder so far: RENAMED_PACKAGES = { 'my_package.old_subpackage.foo': 'my_package.new_subpackage.foo', } # TODO: ideally, we would not just implement a "finder", but also a "loader" # (using the importlib.util.module_for_loader decorator); this would enable us # to get module contents that also pass identity checks class RenamedFinder: @classmethod def find_spec(cls, fullname, path, target=None): renamed = RENAMED_PACKAGES.get(fullname) if renamed is not None: sys.stderr.write( f'WARNING: {fullname} was renamed to {renamed}; please adapt import accordingly!\n') return importlib.util.find_spec(renamed) return None sys.meta_path.append(RenamedFinder())  https://docs.python.org/3.5/library/importlib.html#importlib.util.module_for_loader and related functionality, however, seem to be deprecated. I know it's not a very pythonic thing I am trying to achieve, but I would be glad to learn that it's achievable.

Answer: On import of your package's __init__.py, you can place whatever objects you want into sys.modules, the values you put in there will be returned by import statements: from . import new_package from .new_package import module1, module2 import sys sys.modules["my_lib.old_package"] = new_package sys.modules["my_lib.old_package.module1"] = module1 sys.modules["my_lib.old_package.module2"] = module2  If someone now uses import my_lib.old_package or import my_lib.old_package.module1 they will obtain a reference to my_lib.new_package.module1. Since the import machinery already finds the keys in the sys.modules dictionary, it never even begins looking for the old files. If you want to avoid importing all the submodules immediately, you can emulate a bit of lazy loading by placing a module with a __getattr__ in sys.modules: from types import ModuleType import importlib import sys class LazyModule(ModuleType): def __init__(self, name, mod_name): super().__init__(name) self.__mod_name = name def __getattr__(self, attr): if "_lazy_module" not in self.__dict__: self._lazy_module = importlib.import(self.__mod_name, package="my_lib") return self._lazy_module.__getattr__(attr) sys.modules["my_lib.old_package"] = LazyModule("my_lib.old_package", "my_lib.new_package")
Question: I have defined two classes. A Bookshelf class and a Book class and have defined each with its own methods and type hints. When I run the below code in VS Code using python 3.10 it comes up with the following error: class Bookshelf: SyntaxError: Invalid syntax  Which is referring to the init of the BookShelf class below. Can any of you spot the issue? class Bookshelf: def __init__(self, books: list[Book]): self.books = books def __str__(self) -> str: return f"Bookshelf with {len(self.books)}" class Book: def __init__(self, name: str, page_count: int): self.name=name self.page_count = page_count 

Answer: It is not a SyntaxError, It is a NameError because Book class is not defined yet when you are using it in your type hints. 1. First solution is moving the definition of Book class before the BookShelf. 2. Second solution is use the string instead of the book itself: def __init__(self, books: list["Book"]):  I think in Python 3.11, They will allow to use it as it is. Evaluation of type annotations are gonna be postponed: https://peps.python.org/pep-0563/ 3. third solution: If you want to have this now, you can import: from __future__ import annotations  then your code is going to work.
Question: My model uses LGBMClassifier. I'd like to use Shap (Shapley) to interpret features. However, Shap gave me errors on categorical features. For example, I have a feature "Smoker" and its values include "Yes" and "No". I got an error from Shap: ValueError: could not convert string to float: 'Yes'.  Am I missing any settings? BTW, I know that I could use one-hot encoding to convert categorical features but I don't want to, since LGBMClassifier can handle categorical features without one-hot encoding. Here's the sample code: (shap version is 0.40.0, lightgbm version is 3.3.2) import pandas as pd from lightgbm import LGBMClassifier #My version is 3.3.2 import shap #My version is 0.40.0 #The training data X_train = pd.DataFrame() X_train["Age"] = [50, 20, 60, 30] X_train["Smoker"] = ["Yes", "No", "No", "Yes"] #Target: whether the person had a certain disease y_train = [1, 0, 0, 0] #I did convert categorical features to the Category data type. X_train["Smoker"] = X_train["Smoker"].astype("category") #The test data X_test = pd.DataFrame() X_test["Age"] = [50] X_test["Smoker"] = ["Yes"] X_test["Smoker"] = X_test["Smoker"].astype("category") #the classifier clf = LGBMClassifier() clf.fit(X_train, y_train) predicted = clf.predict(X_test) #shap explainer = shap.TreeExplainer(clf) #I see this setting from google search but it did not really help explainer.model.original_model.params = {"categorical_feature":["Smoker"]} shap_values = explainer(X_train) #the error came out here: ValueError: could not convert string to float: 'Yes' 

Answer: Let's try slightly different: from lightgbm import LGBMClassifier import shap X_train = pd.DataFrame({ "Age": [50, 20, 60, 30], "Smoker": ["Yes", "No", "No", "Yes"]} ) X_train["Smoker"] = X_train["Smoker"].astype("category") y_train = [1, 0, 0, 0] X_test = pd.DataFrame({"Age": [50], "Smoker": ["Yes"]}) X_test["Smoker"] = X_test["Smoker"].astype("category") clf = LGBMClassifier(verbose=-1).fit(X_train, y_train) predicted = clf.predict(X_test) print("Predictions:", predicted) exp = shap.TreeExplainer(clf) sv = exp.shap_values(X_train) # <-- here print(f"Expected values: {exp.expected_value}") print(f"SHAP values for 0th data point: {sv[1][0]}")   Predictions: [0] Expected values: [1.0986122886681098, -1.0986122886681098] SHAP values for 0th data point: [0. 0.]  Note, you don't need to tinker with explainer.model.original_model.params as it gives you non-intended public access to the model's params, which are already set for you by virtue of training model.
Question: Some uncontroversial background experimentation up front: import inspect def func(foo, bar): pass print(inspect.signature(func)) # Prints "(foo, bar)" like you'd expect def decorator(fn): def _wrapper(baz, *args, *kwargs): fn(*args, **kwargs) return _wrapper wrapped = decorator(func) print(inspect.signature(wrapped)) # Prints "(baz, *args, **kwargs)" which is totally understandable  The Question How can implement my decorator so that print(inspect.signature(wrapped)) spits out "(baz, foo, bar)"? Can I build _wrapper dynamically somehow by adding the arguments of whatever fn is passed in, then gluing baz on to the list? The answer is NOT def decorator(fn): @functools.wraps(fn) def _wrapper(baz, *args, *kwargs): fn(*args, **kwargs) return _wrapper  That give "(foo, bar)" again - which is totally wrong. Calling wrapped(foo=1, bar=2) is a type error - "Missing 1 required positional argument: 'baz'" I don't think it's necessary to be this pedantic, but def decorator(fn): def _wrapper(baz, foo, bar): fn(foo=foo, bar=bar) return _wrapper  Is also not the answer I'm looking for - I'd like the decorator to work for all functions.

Answer: You can use __signature__ (PEP) attribute to modify returned signature of wrapped object. For example: import inspect def func(foo, bar): pass def decorator(fn): def _wrapper(baz, *args, **kwargs): fn(*args, **kwargs) f = inspect.getfullargspec(fn) fn_params = [] if f.args: for a in f.args: fn_params.append( inspect.Parameter(a, inspect.Parameter.POSITIONAL_OR_KEYWORD) ) if f.varargs: fn_params.append( inspect.Parameter(f.varargs, inspect.Parameter.VAR_POSITIONAL) ) if f.varkw: fn_params.append( inspect.Parameter(f.varkw, inspect.Parameter.VAR_KEYWORD) ) _wrapper.__signature__ = inspect.Signature( [ inspect.Parameter("baz", inspect.Parameter.POSITIONAL_OR_KEYWORD), *fn_params, ] ) return _wrapper wrapped = decorator(func) print(inspect.signature(wrapped))  Prints: (baz, foo, bar)   If the func is: def func(foo, bar, *xxx, **yyy): pass  Then print(inspect.signature(wrapped)) prints: (baz, foo, bar, *xxx, **yyy)
Question: To create a default bigquery client I use: from google.cloud import bigquery client = bigquery.Client()  This uses the (default) credentials available in the environment. But how I see then which (default) service account is used?

Answer: While you can interrogate the credentials directly (be it json keys, metadata server, etc), I have occasionally found it valuable to simply query bigquery using the SESSION_USER() function. Something quick like this should suffice: client = bigquery.Client() query_job = client.query("SELECT SESSION_USER() as whoami") results = query_job.result() for row in results: print("i am {}".format(row.whoami))
Question: I am using event bridge to trigger a lambda function at 8 everyday to perform some ETL operations. At times i receive requests to trigger the lambda manually ondemand. How can i achieve that using the same lambda function.

Answer: There are many ways to run the Lambda on demand like running it in the AWS Console, connecting it to an API Gateway and triggering it from the API Gateway etc. But the easiest way is to use Lambda URLs It will give you an URL that you can envoke that will run the Lambda.
Question: I'm tending to collect my GPU status during my python code is running. I need to import nvidia_smi in my code to do this. but even by installing it by pip install nvidia_smi hit this error: No module named 'nvidia_smi' Any Idea?

Answer: I found the answer here just pip install nvidia-ml-py3: and: import nvidia_smi nvidia_smi.nvmlInit() handle = nvidia_smi.nvmlDeviceGetHandleByIndex(0)
Question: I want to get url by name specified in urls.py. Like {% url 'some_name' %} in template, but in Python. My urls.py: urlpatterns = [ ... path('admin_section/transactions/', AdminTransactionsView.as_view(), name='admin_transactions'), ... ]  I want to something like: Input: url('admin_transactions') Output: '/admin_section/transactions/'  I know about django.urls.reverse function, but its argument must be View, not url name

Answer: Django has the reverse() utility function for this. Example from the docs: given the following url: from news import views path('archive/', views.archive, name='news-archive')  you can use the following to reverse the URL: from django.urls import reverse reverse('news-archive')  The documentation goes further into function's use, so I suggest reading it.
Question: How to format 1.3435434533e-8 into 1.34e-8 in python? Keep only two digits. The round() method will round this number to zero.

Answer: The "g" formatting suffix on string mini-format language, used both by f-strings, and the .format method will do that: In [1]: a = 1.34434325435e-8 In [2]: a Out[2]: 1.34434325435e-08 In [4]: f"{a:.03g}" Out[4]: '1.34e-08' # in contrast with: In [5]: f"{a:.03f}" Out[5]: '0.000'
Question: Problem Statement I would like to apply a list of functions fs = [ f, g, h ] sequentially to a string text=' abCdEf ' Something like f( g( h( text) ) ). This could easily be accomplished with the following code: # initial text text = ' abCDef ' # list of functions to apply sequentially fs = [str.rstrip, str.lstrip, str.lower] for f in fs: text = f(text) # expected result is 'abcdef' with spaces stripped, and all lowercase print(text)  Using functools.reduce It seems that functools.reduce should do the job here, since it "consumes" the list of functions at each iteration. from functools import reduce # I know `reduce` requires two arguments, but I don't even know # which one to chose as text of function from the list reduce(f(text), fs) # first interaction should call y = str.rstrip(' abCDef ') --> ' abCDef' # next iterations fails, because tries to call ' abCDef'() -- as a function  Unfortunately, this code doesn't work, since each iteration returns a string istead of a function, and fails with TypeError : 'str' object is not callable. QUESTION: Is there any solution using map, reduce or list comprehension to this problem?

Answer: reduce can take three arguments: reduce(function, iterable, initializer)  What are these three arguments in general?  function is a function of two arguments. Let's call these two arguments t and f. the first argument, t, will start as initializer; then will continue as the return value of the previous call of function. the second argument, f, is taken from iterable.  What are these three arguments in our case?  the iterable is your list of function; the second argument f is going to be one of the functions; the first argument t must be the text; the initializer must be the initial text; the return of function must be the resulting text; function(t, f) must be f(t).  Finally: from functools import reduce # initial text text = ' abCDef ' # list of functions to apply sequentially fs = [str.rstrip, str.lstrip, str.lower] result = reduce(lambda t,f: f(t), fs, text) print(repr(result)) # 'abcdef'
Question: I have a 2D numpy array from which I want to extract multiple sets of rows/ columns. # img is 2D array img = np.arange(25).reshape(5,5) array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]])  I know the syntax to extract one set of row/ column. The following will extract the first 4 rows and the 3rd and 4th column as shown below img[0:4, 2:4] array([[ 2, 3], [ 7, 8], [12, 13], [17, 18]])  However, what is the syntax if I want to extract multiple sets of rows and/or columns? I tried the following but it leads to an invalid syntax error img[[0,2:4],2]  The output that I am looking for from the above command is array([[ 2], [12], [17]])  I tried searching for this but it only leads to results for one set of rows/ columns or extracting discrete rows/ columns which I know how to do, like using np.ix. For context, the 2D array that I am actually dealing with has the dimensions ~800X1200, and from this array I want to extract multiple ranges of rows and columns in one go. So something like img[[0:100, 120:210, 400, 500:600], [1:450, 500:550, 600, 700:950]].

Answer: IIUC, you can use numpy.r_ to generate the indices from the slice: img[np.r_[0,2:4][:,None],2]  output: array([[ 2], [12], [17]])  intermediates: np.r_[0,2:4] # array([0, 2, 3]) np.r_[0,2:4][:,None] # variant: np.c_[np.r_[0,2:4]] # array([[0], # [2], # [3]])
Question: Given a CuPy array a, there are two ways to get a numpy array from it: a.get() and cupy.asnumpy(a). Is there any practical difference between them? import cupy as cp a = cp.random.randint(10, size=(4,5,6,7)) b = a.get() c = cp.asnumpy(a) assert type(b) == type(c) and (b == c).all() 

Answer: cp.asnumpy is a wrapper calling ndarray.get. You can see that in the code of cp.asnumpy: def asnumpy(a, stream=None, order='C', out=None): """Returns an array on the host memory from an arbitrary source array. Args: a: Arbitrary object that can be converted to :class:`numpy.ndarray`. stream (cupy.cuda.Stream): CUDA stream object. If it is specified, then the device-to-host copy runs asynchronously. Otherwise, the copy is synchronous. Note that if ``a`` is not a :class:`cupy.ndarray` object, then this argument has no effect. order ({'C', 'F', 'A'}): The desired memory layout of the host array. When ``order`` is 'A', it uses 'F' if ``a`` is fortran-contiguous and 'C' otherwise. out (numpy.ndarray): The output array to be written to. It must have compatible shape and dtype with those of ``a``'s. Returns: numpy.ndarray: Converted array on the host memory. """ if isinstance(a, ndarray): return a.get(stream=stream, order=order, out=out) elif hasattr(a, "__cuda_array_interface__"): return array(a).get(stream=stream, order=order, out=out) else: temp = _numpy.asarray(a, order=order) if out is not None: out[...] = temp else: out = temp return out  As you can see (both in the documentation and in the code), cp.asnumpy supports more input types than just CuPy arrays. It supports inputs that are CUDA objects with the __cuda_array_interface__ attribute and any objects that can be actually converted to a Numpy array. This includes Numpy arrays themselves and iterables (eg. list, generators, etc.).