eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I want to set a parameter for a python script by using the parameter field in PyCharm. My config:  But the command in the Run console is: python3 path_to_script.py '{app_id: picoballoon_network, dev_id: ferdinand_8c ... and so on and not: python3 path_to_script.py '{"app_id": "picoballoon_network", "dev_id": "ferdinand_8c" ... and so on Basically, it deletes all " in the parameter. Does anyone know how to turn this off? My PyCharm version is: PyCharm 2020.3.1 (Professional Edition) Build #PY-203.6682.86, built on January 4, 2021 Runtime version: 11.0.9.1+11-b1145.37 amd64 VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o. Windows 10 10.0 

Answer: To avoid the quotation marks being deleted notice the rules to writing parameters that contain quotation marks.  Run/Debug Configuration: Python Configuration tab When specifying the script parameters, follow these rules:  Use spaces to separate individual script parameters.  Script parameters containing spaces should be delimited with double quotes, for example, some" "param or "some param".  If script parameter includes double quotes, escape the double quotes with backslashes, for example:   -s"main.snap_source_dirs=[\"pcomponents/src/main/python\"]" -s"http.cc_port=8189" -s"backdoor.port=9189" -s"main.metadata={\"location\": \"B\", \"language\": \"python\", \"platform\": \"unix\"}"   The case in the question would be a single parameter, lets apply the rules to the example: '{"app_id": "picoballoon_network", "dev_id": "ferdinand_8c"'  Because it's a single parameter containing spaces it has to be surounded by quotation marks.  Since the content of the parameter also contains quotation marks they must be escaped using a backslash \. So applying the parameter formatting rules gives:   "'{\"app_id\": \"picoballoon_network\", \"dev_id\": \"ferdinand_8c\"}'"  (Side note): In the example the parameter was surrounded by Apostrophes, this may be unnecessary and will probably have to be stripped later in your Python code (the below example uses the strip method).  You can test it with this simple script: import sys import ast your_dictionary = ast.literal_eval(sys.argv[1].strip("'"))  (Side note): Your example parameter is a string containing a Python dictionary, there are several ways to convert it, in the example I included the highest voted answer from this question: "Convert a String representation of a Dictionary to a dictionary?" A screenshot showing the parameter and test code in use:
Question: I've looked through the examples out there and don't seem to find one that fits. Looking to unzip a file in-memory to a temporary directory using Python mkdtemp(). Something like this feels intuitive, but I can't find the correct syntax: import zipfile import tempfile zf = zipfile.Zipfile('incoming.zip') with tempfile.mkdtemp() as tempdir: zf.extractall(tempdir) # do stuff on extracted files  But this results in: AttributeError Traceback (most recent call last) <ipython-input-5-af39c866a2ba> in <module> 1 zip_file = zipfile.ZipFile('incoming.zip') 2 ----> 3 with tempfile.mkdtemp() as tempdir: 4 zip_file.extractall(tempdir) AttributeError: __enter__ 

Answer: I already mentioned in my comment why the code that you wrote doesn't work. .mkdtemp() returns just a path as a string, but what you really want to have is a context manager. You can easily fix that by using the the correct function .TemporaryDirectory()  This function securely creates a temporary directory using the same rules as mkdtemp(). The resulting object can be used as a context manager (see Examples). On completion of the context or destruction of the temporary directory object the newly created temporary directory and all its contents are removed from the filesystem.   zf = zipfile.ZipFile('incoming.zip') with tempfile.TemporaryDirectory() as tempdir: zf.extractall(tempdir)  This alone would work
Question: Lets say I have the following dataframe: df1 = pd.DataFrame(data = [1,np.nan,np.nan,1,1,np.nan,1,1,1], columns = ['X'], index = ['a', 'a', 'a', 'b', 'b', 'b', 'c', 'c', 'c']) print(df1) X a 1.0 a NaN a NaN b 1.0 b 1.0 b NaN c 1.0 c 1.0 c 1.0  I want to keep only the indices which have 2 or more non-NaN entries. In this case, the 'a' entries only have one non-NaN value, so I want to drop it and have my result be:  X b 1.0 b 1.0 b NaN c 1.0 c 1.0 c 1.0  What is the best way to do this? Ideally I want something that works with Dask too, although usually if it works with Pandas it also works in Dask.

Answer: Let us try filter out = df.groupby(level=0).filter(lambda x : x.isna().sum()<=1) X b 1.0 b 1.0 b NaN c 1.0 c 1.0 c 1.0  Or we do isin df[df.index.isin(df.isna().sum(level=0).loc[lambda x : x['X']<=1].index)] X b 1.0 b 1.0 b NaN c 1.0 c 1.0 c 1.0
Question: With matplotlib I could do this: import numpy as np import matplotlib.pyplot as plt fig, axs = plt.subplots(1, 2, figsize=(10, 5)) fig.patch.set_facecolor('white') axs[0].bar(x=[0,1,2], height=[1,2,3]) axs[1].plot(np.random.randint(1, 10, 50)) axs[0].set_title('BARPLOT') axs[1].set_title('WOLOLO')   With plotly I know only one way to set subplot titles -- at the creation: fig = make_subplots(rows=1, cols=2, subplot_titles=('title1', 'title2')) Is it possible to set subplots title after creation? Maybe, through getting access to original Axes class of matplotlib (I read that plotly.python based on seaborn which is based on matplotlib)? Or through fig.layout? Please use this code: import numpy as np import plotly.graph_objects as go from plotly.subplots import make_subplots fig = make_subplots(rows=1, cols=2) fig.add_trace(go.Bar(y=[1, 2, 3]), row=1, col=1) fig.add_trace(go.Scatter(y=np.random.randint(1, 10, 50)), row=1, col=2)  

Answer: The following code does the trick you want. import numpy as np import plotly.graph_objects as go from plotly.subplots import make_subplots fig = make_subplots(rows=1, cols=2, subplot_titles=("Plot 1", "Plot 2")) fig.add_trace(go.Bar(y=[1, 2, 3]), row=1, col=1) fig.add_trace(go.Scatter(y=np.random.randint(1, 10, 50)), row=1, col=2) fig.layout.annotations[1].update(text="Stackoverflow") fig.show()  I got this idea from https://community.plotly.com/t/subplot-title-alignment/33210/2 when it's described how to access each plot's annotation.
Question: I'm trying to replicate this command with the Docker SDK for Python: docker build -f path/to/dockerfile/Dockerfile.name -t image:version path/to/context/.  path/to/dockerfile and path/to/context are different paths, ie: /opt/project/dockerfile and /opt/project/src/app/. The directory structure is the following: opt ├── project │ ├── dockerfile │ │ └── Dockerfile.name │ └── src │ └── app │ └── target │ └── app-0.0.1-SNAPSHOT.jar └── script.py  The command is working correctly from the CLI, but I'm not able to make it work with the SDK. From the documentation, the images build method has the following parameters:  path (str) – Path to the directory containing the Dockerfile (I'm guessing this is the context) custom_context (bool) – Optional if using fileobj fileobj – A file object to use as the Dockerfile. (Or a file-like object)  When I use the method like this: client.images.build( path = path_to_context, fileobj=open(path_to_file, 'rb'), custom_context=True, tag='image:version' )  I get this error: Traceback (most recent call last): File "script.py", line 33, in <module> client.images.build( File "/Library/Python/3.8/site-packages/docker/models/images.py", line 298, in build raise BuildError(last_event or 'Unknown', result_stream) docker.errors.BuildError: {'message': 'unexpected EOF'}  The content of the Dockerfile is the following: FROM openjdk:16-alpine COPY target/app-0.0.1-SNAPSHOT.jar app.jar CMD ["java", "-jar", "-Dspring.profiles.active=docker", "/app.jar"]  but I'm guessing the error is not due to that for the command correctly works with the CLI, it's only breaking with the SDK. Am I doing something wrong? Thanks! Edit: Simply removing custom_context=True does not solve the problem, for the context and build paths are different. In fact it causes another error, relative to the fact that the file does not exist in the current path: Traceback (most recent call last): File "script.py", line 33, in <module> client.images.build( File "/Library/Python/3.8/site-packages/docker/models/images.py", line 287, in build raise BuildError(chunk['error'], result_stream) docker.errors.BuildError: COPY failed: file not found in build context or excluded by .dockerignore: stat target/app-0.0.1-SNAPSHOT.jar: file does not exist 

Answer: I removed custom_context=True, and the problem went away. EDIT: Using your project tree: import docker client = docker.from_env() client.images.build( path = './project/src/app/target/', dockerfile = '../../../Dockerfile/Dockerfile.name', tag='image:version', )
Question: I'm having trouble finding a good way to find all negative entries in a column and move them up the column, summing them up with the existing entry (i.e. subtracting the negative entry from the present entry) until all values are positive. It is important that there are no negative values for the final dataframe & that all previously negative entries = 0. Also, the table is repeating, which means that I need to aggregate the results both based on the ID as well as on the entries (only do summations of entries of the same ID). Based on an already presented table here: Present:     ID Date Entries     1 2013 100   1 2014 0   1 2015 60   1 2016 -30   1 2017 0   1 2018 50   1 2019 0   1 2020 -20   2 2013 100   2 2014 0   2 2015 60   2 2016 -30   2 2017 0   2 2018 50   2 2019 0   2 2020 -20     Desired:     ID Date Entries     1 2013 100   1 2014 0   1 2015 30   1 2016 0   1 2017 0   1 2018 30   1 2019 0   1 2020 0   2 2013 100   2 2014 0   2 2015 30   2 2016 0   2 2017 0   2 2018 30   2 2019 0   2 2020 0    

Answer: You can try reverse cumsum after creating a group, then mask: s = df['Entries'].gt(0).cumsum() u= df['Entries'][::-1].groupby(s).cumsum().mask(df['Entries'].le(0),0) out = df.assign(New_Entries=u) # you can assign to the original column too.   print(out) ID Date Entries New_Entries 0 1 2013 100 100 1 1 2014 0 0 2 1 2015 60 30 3 1 2016 -30 0 4 1 2017 0 0 5 1 2018 50 30 6 1 2019 0 0 7 1 2020 -20 0 8 2 2013 100 100 9 2 2014 0 0 10 2 2015 60 30 11 2 2016 -30 0 12 2 2017 0 0 13 2 2018 50 30 14 2 2019 0 0 15 2 2020 -20 0
Question: I have the following code: s = [1,2,3] t = reversed(s) for i in t: print(i) # output: 3,2,1  If I pop one element from s (original), then the t (reversed) is emptied: s = [1,2,3] t = reversed(s) s.pop() for i in t: print(i) # expected output: 2, 1 # actual output (nothing):  Why does this happen?

Answer: Taking a look at the cpython code on GitHub, we can get some intuition as to why it no longer works. The iterator that is returned essentially requires knowing the position of the last index and the length of the array. If the size of the array is changed, the iterator will no longer work. Test 1: Increasing the array length This will not produce the correct results either, but the iterator does run: s = [1,2,3] t = reversed(s) s.append(4) for i in t: print(i) # output: [3, 2, 1]  Test 2: Decreasing, then increasing the length s = [1,2,3] t = reversed(s) s.pop() s.append(4) for i in t: print(i) # output: [4, 2, 1]  It still works! So there's an internal check to see whether or not the last index is still valid, and if it is, it's a simple for loop down to index 0. If it doesn't work, the iterator returns empty.
Question: I have a simple class below: class MyClass(int): def __index__(self): return 1  According to operator.index documentation:  operator.index(a) Return a converted to an integer. Equivalent to a.__index__()  But when I use operator.index with MyClass instance, I got 100 instead of 1 (I am getting 1 if I use a.__index__()). Why is that? >>> a = MyClass(100) >>> >>> import operator >>> print(operator.index(a)) 100 >>> print(a.__index__()) 1 

Answer: This is because your type is an int subclass. __index__ will not be used because the instance is already an integer. That much is by design, and unlikely to be considered a bug in CPython. PyPy behaves the same. In _operator.c: static PyObject * _operator_index(PyObject *module, PyObject *a) /*[clinic end generated code: output=d972b0764ac305fc input=6f54d50ea64a579c]*/ { return PyNumber_Index(a); }  Note that operator.py Python code is not used generally, this code is only a fallback in the case that compiled _operator module is not available. That explains why the result a.__index__() differs. In abstract.c, cropped after the relevant PyLong_Check part: /* Return an exact Python int from the object item. Raise TypeError if the result is not an int or if the object cannot be interpreted as an index. */ PyObject * PyNumber_Index(PyObject *item) { PyObject *result = _PyNumber_Index(item); if (result != NULL && !PyLong_CheckExact(result)) { Py_SETREF(result, _PyLong_Copy((PyLongObject *)result)); } return result; } ... /* Return a Python int from the object item. Can return an instance of int subclass. Raise TypeError if the result is not an int or if the object cannot be interpreted as an index. */ PyObject * _PyNumber_Index(PyObject *item) { PyObject *result = NULL; if (item == NULL) { return null_error(); } if (PyLong_Check(item)) { Py_INCREF(item); return item; /* <---- short-circuited here */ } ... }  The documentation for operator.index is inaccurate, so this may be considered a minor documentation issue: >>> import operator >>> operator.index.__doc__ 'Same as a.__index__()'  So, why isn't __index__ considered for integers? The probable answer is found in PEP 357, under the discussion section titled Speed:  Implementation should not slow down Python because integers and long integers used as indexes will complete in the same number of instructions. The only change will be that what used to generate an error will now be acceptable.  We do not want to slow down the most common case for slicing with integers, having to check for an nb_index slot every time.
Question: I am iterating through my text file, but when I use the read() function, the loop iterates through the letters instead of the sentences. with the following code: for question in questions: # voor elke question moet er door alle lines geiterate worden print(f"Question: {question}") f = open("glad.txt", "r") text = f.read() # text = text.replace("\n", ". ") # text = text.replace(". .", "") # text = text.replace(".. ", ". ") # text = text.replace(".", ".\n") #text = text.strip(".. ") # test = text.replace('[bewerken | brontekst bewerken]', "") # output = re.sub(r'\[\d+\]', '', test) for line in text: text = str(line) #het antwoord moet een string zijn #encoding met tokenizen van de zinnen print(text)  The output is:  But when I remove the f.read() I receive the expected out:  I need to use the read() function, otherwise I cannot use the replace() function. Does anyone how to solve this issue?

Answer: Using text = f.read(), you are getting the whole text file into text. When you iterate over a string in Python, it gives you one character per iteration. Since you want to continue using .read(), use splitlines(): text = f.read().splitlines()  Now, text is a list which you can freely iterate the same way you are already doing: for line in text:
Question: I am trying to create a simple abstract base class Abstract that along with its own methods provides the methods of two others abstract base classes: Publisher and Subscriber. When I try to initialize the concrete class Concrete, built on Abstract I get this error: Cannot create a consistent method resolution order (MRO) for bases ABC, Publisher, Subscriber. What is the right way to do it? from abc import ABC, abstractmethod class Publisher(ABC): subscribers = set() def register(self, obj): self.subscribers.add(obj) def unregister(self, obj): self.subscribers.remove(obj) def dispatch(self, event): print("dispatching", event) class Subscriber(ABC): @abstractmethod def handle_event(self, event): raise NotImplementedError class Abstract(ABC, Publisher, Subscriber): @abstractmethod def do_something(self, event): raise NotImplementedError class Concrete(Abstract): def handle_event(self, event): print("handle_event") def do_something(self, event): print("do_something") c = Concrete() 

Answer: Abstract classes don't have to have abc.ABC in their list of bases. They have to have abc.ABCMeta (or a descendant) as their metaclass, and they have to have at least one abstract method (or something else that counts, like an abstract property), or they'll be considered concrete. (Publisher has no abstract methods, so it's actually concrete.) Inheriting from ABC is just a way to get ABCMeta as your class's metaclass, for people more comfortable with inheritance than metaclasses, but it's not the only way. You can also inherit from another class with ABCMeta as its metaclass, or specify metaclass=ABCMeta explicitly.  In your case, inheriting from Publisher and Subscriber will already set Abstract's metaclass to ABCMeta, so inheriting from ABC is redundant. Remove ABC from Abstract's base class list, and everything should work. Alternatively, if you really want ABC in there for some reason, you can move it to the end of the base class list, which will resolve the MRO conflict - putting it first says you want ABC methods to override methods from the other classes, which conflicts with the fact that the other classes are subclasses of ABC.
Question: Is there an idiomatic way to mask elements of an array in vanilla Python 3? For example: a = [True, False, True, False] b = [2, 3, 5, 7] b[a]  I was hoping b[a] would return [2, 5], but I get an error:  TypeError: list indices must be integers or slices, not list  In R, this works as I expected (using c() instead of [] to create the lists). I know NumPy has MaskedArray that can do this, I'm looking for an idiomatic way to do this in plain vanilla Python. Of course, I could use a loop and iterate through the mask list and the element list, but I'm hoping there's a more efficient way to mask elements using a higher level abstraction.

Answer: You can use itertools.compress: >>> from itertools import compress >>> a = [True, False, True, False] >>> b = [2, 3, 5, 7] >>> list(compress(b, a)) [2, 5]  Refer "itertools.compress()" document for more details
Question: I have a dataframe including a datetime column for date and a column for hour. like this: min hour date 0 0 2020-12-01 1 5 2020-12-02 2 6 2020-12-01  I need a datetime column including both date and hour. like this : min hour date datetime 0 0 2020-12-01 2020-12-01 00:00:00 0 5 2020-12-02 2020-12-02 05:00:00 0 6 2020-12-01 2020-12-01 06:00:00  How can I do it?

Answer: You could also try using apply and np.timedelta64: df['datetime'] = df['date'] + df['hour'].apply(lambda x: np.timedelta64(x, 'h')) print(df)  Output:  min hour date datetime 0 0 0 2020-12-01 2020-12-01 00:00:00 1 1 5 2020-12-02 2020-12-02 05:00:00 2 2 6 2020-12-01 2020-12-01 06:00:00
Question: I have: class A: a=1 b=2  I want to make as setattr(A,'c')  then all objects that I create it from class A has c attribute. i did not want to use inheritance

Answer: There're two ways of setting an attribute to your class; First, by using setattr(class, variable, value) Code Syntax setattr(A,'c', 'c') print(dir(A))  OUTPUT You can see the structure of the class A within attributes ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'a', 'b', 'c'] [Program finished]   Second, you can do it simply by assigning the variable Code Syntax A.d = 'd' print(dir(A))  OUTPUT You can see the structure of the class A within attributes ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'a', 'b', 'c', 'd'] [Program finished]
Question: The Python API doesn't give much more information other than that the seed= parameter is passed to numpy.random.seed:  seed (int) – Seed used to generate the folds (passed to numpy.random.seed).  But what features of xgboost use numpy.random.seed?  Running xgboost with all default settings still produces the same performance even when altering the seed. I have already been able to verify colsample_bytree does so; different seeds yield different performance. I have been told it is also used by subsample and the other colsample_* features, which seems plausible since any form of sampling requires randomness.  What other features of xgboost rely on numpy.random.seed?

Answer: Boosted trees are grown sequentially, with tree growth within one iteration being distributed among threads. To avoid overfitting, randomness is induced through the following params:  colsample_bytree colsample_bylevel colsample_bynode subsample (note the *sample* pattern) shuffle in CV folder creation for cross validation  In addition, you may encounter non-determinism, not controlled by random state, in the following places:  [GPU] histogram building is not deterministic due to the nonassociative aspect of floating point summation.   Using gblinear booster with shotgun updater is nondeterministic as it uses Hogwild algorithm   when using GPU ranking objective, the result is not deterministic due to the non-associative aspect of floating point summation.  Comment Re: how you know this? For this to know it's helpful:  To be aware of how trees are grown: Demystify Modern Gradient Boosting Trees (references may be also helpful)  Scanning documentation full text for the terms of interest: random, sample, deterministic, determinism etc.  Lastly (firstly?), knowing why you need sampling and similar cases from counterparts like bagged trees (RANDOM FORESTS by Leo Breiman) and neural networks (Deep learning with Python by François Chollet, chapter on overfitting) may also be helpful.
Question: I have a list of strings and I need to create a function that prints out n elements of the list each time it is run. For instance: book1 = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o']  Expected output first time I run the function if n = 5: a b c d e  second time: f g h i j  I tried this: def print_book(book): printed = book while printed != []: for i in range(0, len(book), 5): new_list = (book[i:i+5]) for el in new_list: print(el) break del(printed[i:i+10])  And I get either the entire list printed out, or I end up printing first n elements each time I run the function. If this question has already been asked, please point it out to me, I would really apreciate it. Thanks!

Answer: I guess you can try the following user function which applied to iterator book def print_book(book): cnt = 0 while cnt < 5: try: print(next(book)) except StopIteration: print("You have reached the end!") break cnt += 1  such that >>> bk1 = iter(book1) >>> print_book(bk1) a b c d e >>> print_book(bk1) f g h i j >>> print_book(bk1) k l m n o >>> print_book(bk1) You have reached the end!