eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Pydantic copies a model when passing it to the constructor of another model. This fails: from pydantic import BaseModel class Child(BaseModel): pass class Parent(BaseModel): child: Child child = Child() parent = Parent(child=child) assert parent.child is child # Fails  It seems child is copied when passing it to the parent's constructor and therefore the identities of child and parent.child are not the same. I would like to have them to be the same as I need to modify child's attributes later and the changes should be seen in parent.child. How do I make Pydantic not copy the child?

Answer: I found the answer myself. Seems this was an issue but it was fixed in a PR by creating a config option copy_on_model_validation. If this option is set to False for the child, then the child is not copied in the construction. This does not copy the child: from pydantic import BaseModel class Child(BaseModel): class Config: copy_on_model_validation = False class Parent(BaseModel): child: Child child = Child() parent = Parent(child=child) assert parent.child is child # Passes
Question: I believe the following code s = ''' ... .o. ... ''' print(s.splitlines())  should print ['', '...', '.o.', '...']  Indeed, this is the case when Python is executed normally (example run on Wandbox is here). But the reality is ruthless (as usual); Google Colaboratory prints a result without "triple dots":  I also tried the same code with a locally installed Jupyter (Python 3.7.13, Jupyter notebook 6.4.12, IPython 7.34.0) and it gave me the same result as Google Colaboratory. Does anyone know what causes this deletion of the triple dots?

Answer: Google collab interprets ... as part of the prompt. You can change the prompt to some other string and the result will be as you expected : import sys sys.ps2 = '<<<' # default value is ... s = ''' ... .o. ... ''' print(s.splitlines()) ['', '...', '.o.', '...']  EDIT: As @user2357112 pointed out in the comments and in their answer changing the prompt does not affect this. Here it seemed to work because adding more lines to the beginning of the cell make Ipython interpreter think they are no longer part of the prompt. You can change your string to '\n...\n.o.\n...' as a workaround.
Question: I have a dataframe: pd.DataFrame(columns=['a','b'],data=[[3,4], [5,5],[9,3],[1,2],[9,9],[6,5],[6,5],[6,5],[6,5], [6,5],[6,5],[6,5],[6,5],[6,5],[6,5],[6,5],[6,5]])  I want to insert two empty rows after every third row so the resulting output looks like that:  a b 0 3.0 4.0 1 5.0 5.0 2 9.0 3.0 3 NaN NaN 4 NaN NaN 5 1.0 2.0 6 9.0 9.0 7 6.0 5.0 8 NaN NaN 9 NaN NaN 10 6.0 5.0 11 6.0 5.0 12 6.0 5.0 13 NaN NaN 14 NaN NaN 15 6.0 5.0 16 6.0 5.0 17 6.0 5.0 18 NaN NaN 19 NaN NaN 20 6.0 5.0 21 6.0 5.0 22 6.0 5.0 23 NaN NaN 24 NaN NaN 25 6.0 5.0 26 6.0 5.0  I tried a number of things but didn't get any closer to the desired output.

Answer: The following should scale well with the size of the DataFrame since it doesn't iterate over the rows and doesn't create intermediate DataFrames. import pandas as pd df = pd.DataFrame(columns=['a','b'],data=[[3,4], [5,5],[9,3],[1,2],[9,9],[6,5],[6,5],[6,5],[6,5], [6,5],[6,5],[6,5],[6,5],[6,5],[6,5],[6,5],[6,5]]) def add_empty_rows(df, n_empty, period): """ adds 'n_empty' empty rows every 'period' rows to 'df'. Returns a new DataFrame. """ # to make sure that the DataFrame index is a RangeIndex(start=0, stop=len(df)) # and that the original df object is not mutated. df = df.reset_index(drop=True) # length of the new DataFrame containing the NaN rows len_new_index = len(df) + n_empty*(len(df) // period) # index of the new DataFrame new_index = pd.RangeIndex(len_new_index) # add an offset (= number of NaN rows up to that row) # to the current df.index to align with new_index. df.index += n_empty * (df.index .to_series() .groupby(df.index // period) .ngroup()) # reindex by aligning df.index with new_index. # Values of new_index not present in df.index are filled with NaN. new_df = df.reindex(new_index) return new_df  Tests: # original df >>> df a b 0 3 4 1 5 5 2 9 3 3 1 2 4 9 9 5 6 5 6 6 5 7 6 5 8 6 5 9 6 5 10 6 5 11 6 5 12 6 5 13 6 5 14 6 5 15 6 5 16 6 5 # add 2 empty rows every 3 rows >>> add_empty_rows(df, 2, 3) a b 0 3.0 4.0 1 5.0 5.0 2 9.0 3.0 3 NaN NaN 4 NaN NaN 5 1.0 2.0 6 9.0 9.0 7 6.0 5.0 8 NaN NaN 9 NaN NaN 10 6.0 5.0 11 6.0 5.0 12 6.0 5.0 13 NaN NaN 14 NaN NaN 15 6.0 5.0 16 6.0 5.0 17 6.0 5.0 18 NaN NaN 19 NaN NaN 20 6.0 5.0 21 6.0 5.0 22 6.0 5.0 23 NaN NaN 24 NaN NaN 25 6.0 5.0 26 6.0 5.0 # add 5 empty rows every 4 rows >>> add_empty_rows(df, 5, 4) a b 0 3.0 4.0 1 5.0 5.0 2 9.0 3.0 3 1.0 2.0 4 NaN NaN 5 NaN NaN 6 NaN NaN 7 NaN NaN 8 NaN NaN 9 9.0 9.0 10 6.0 5.0 11 6.0 5.0 12 6.0 5.0 13 NaN NaN 14 NaN NaN 15 NaN NaN 16 NaN NaN 17 NaN NaN 18 6.0 5.0 19 6.0 5.0 20 6.0 5.0 21 6.0 5.0 22 NaN NaN 23 NaN NaN 24 NaN NaN 25 NaN NaN 26 NaN NaN 27 6.0 5.0 28 6.0 5.0 29 6.0 5.0 30 6.0 5.0 31 NaN NaN 32 NaN NaN 33 NaN NaN 34 NaN NaN 35 NaN NaN 36 6.0 5.0
Question: In answering this question, I stumbled across some unexpected behavior: from typing import List, Iterable class Name: def __init__(self, name: str): self.name = name def generator(lst: List[Name]) -> Iterable[str]: lst_copy = lst.copy() for obj in lst_copy: yield obj.name  When modifying the list that is passed to the generator, even though a copy is made, changes to the original list are still reflected: lst = [Name("Tom"), Name("Tommy")] gen = generator(lst) lst[0] = Name("Andrea") for name in gen: print(name)  Output: Andrea Tommy  Simply returning a generator expression works as expected: def generator(lst: List[Name]) -> Iterable[str]: return (obj.name for obj in lst.copy())  Output: Tom Tommy  Why doesn't the lst.copy() in the first generator function work as expected?

Answer: I think the behavior is best understood with the addition of some extra print statements: def generator(lst: List[Name]) -> Iterable[str]: print("Creating list copy...") lst_copy = lst.copy() print("Created list copy!") for obj in lst_copy: yield obj.name lst = [Name("Tom"), Name("Tommy")] print("Starting assignment...") gen = generator(lst) print("Assignment complete!") print("Modifying list...") lst[0] = Name("Andrea") print("Modification complete!") for name in gen: print(name)  Notice that the copy does not happen at assignment time -- it happens after the list is modified! Starting assignment... Assignment complete! Modifying list... Modification complete! Creating list copy... Created list copy! Andrea Tommy  Nothing in the generator's body is executed until the for loop attempts to extract an element. Since this extraction attempt occurs after the list is mutated, the mutation is reflected in the results from the generator.
Question: I recently switched from using simple forms in Django to model forms. I am now trying to use a select field in my form that has set field names (eg:Europe,North America,South America...) I thought I would just add a select input type to the for fields but it shows up as just a regular text input. The select field is supposed to be "continent". Does anyone know how to to d this?? class TripForm(forms.ModelForm): # city = forms.TextInput() # country = forms.TimeField() # description = forms.Textarea() # photo = forms.ImageField() class Meta: model = Trip fields = ('city', 'country', 'continent', 'description', 'photo') widget = { 'city': forms.TextInput(attrs={'class': 'form-control'}), 'country': forms.TextInput(attrs ={'class': 'form-control'}), 'continent': forms.SelectMultiple(attrs= {'class':'form-control'}), 'description': forms.TextInput(attrs = {'class': 'form-control'}), # 'photo': forms.ImageField(attrs = {'class': 'form-control'}), } [![enter image description here][1]][1] This is the form continent should show as a select. 

Answer: Here you can use ModelChoiceField provided by django forms. Define a Country model containing all the countries, and use that as a QuerySet here. It will be also be dynamic that can be changed later. from django import forms from .models import Trip, Country class TripForm(forms.ModelForm): class Meta: model = Trip fields = ['name', 'country', 'start_date', 'end_date'] country = forms.ModelChoiceField( queryset=Country.objects.all(), to_field_name='name', required=True, widget=forms.Select(attrs={'class': 'form-control'}) )
Question: I want to run a very simple python script (or any language) which collects data from an api at regular 5 minute intervals and saves the data. I need this process to run for a whole week - day and night. I can't keep my laptop on all week so I guess I will need this to run on some kind of server. What is the cheapest and easiest way for me to do something like this? This is an incredibly simple script so I don't want to spend ages setting up a complicated server which is ultimately overkill for a task like this. Pseudocode: # Every 5 minutes for a whole week: data = call_api(url) write_csv(data) 

Answer: You're going to need a cloud server as it's the cheapest option if u can't setup a system at home. I'll 2nd what furas said about pythonanywhere.com. I use their $5 a month plan to run simple tasks like this. You can cancel anytime so if it's just 1 week then you pay $5 for your account, that's it. There is no messing around with setup either, just upload your main.py and schedule as task in the dashboard.
Question: I am deploying a web app on Cloud Run using the automated Cloud Build "Buildpack" option (as explained here); hence not having to create a Docker File. I would like to deploy the app using python-3.8.12 and buildpacks. How can I specify that?

Answer: There’s no need to specify it. The builder attempts to autodetect the language of your source code and Python 3.7+ is one of the supported languages. You may try this Python buildpacks sample. You can also find out deeper information about buildpacks in this presentation. Update As an alternative, you can also manually specify which buildpack to use, thereby skipping the auto-detection step. As described in this answer, you can also specify it in the project.toml the following way: [[build.env]] name = "GOOGLE_RUNTIME_VERSION" value = "3.8.6"  This would be the same as what you'd put in the .python-version, but would take precedence over it.
Question: I am trying to measure code coverage by my pytest tests. I tried following the quick start guide of coverage (https://coverage.readthedocs.io/en/6.4.1/) When I run my test with the following command, everything seems fine coverage run -m pytest tests/  ===================================== test session starts ====================================== platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0 rootdir: /home/arnaud/Documents/Github/gotcha collected 4 items tests/preprocessing/test_preprocessing.py .... [100%] ====================================== 4 passed in 0.30s =======================================  However, when I try to access the report with either of those commands, coverage report coverage html  I get the following message: No source for code: '<project_directory>/config-3.py'.  I did not find an appropriate solution to this problem so far

Answer: It is possible to ignore errors using the command coverage html -i  which solved my issue
Question: I am getting the following FutureWarning in my Python code: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. Right now I was using the append function, in various parts of my code, to add rows to an existing DataFrame. Example 1: init_hour = pd.to_datetime('00:00:00') orig_hour = init_hour+timedelta(days=1) while init_hour < orig_hour: row = {'Hours': init_hour.time()} df = df.append(row, ignore_index = True) init_hour = init_hour + timedelta(minutes=60)  Example 2: row2 = {'date': tmp_date, 'false_negatives': fn, 'total': total} df2 = df2.append(row2, ignore_index = True)  How could I solve this in a simple way without modifying much of the code before the sections above?

Answer: Use pd.concat instead of append. Preferably on a bunch of rows at the same time. Also, use date_range and timedelta_range whenever possible. Example 1: # First transformation init_hour = pd.to_datetime('00:00:00') orig_hour = init_hour+timedelta(days=1) rows = [] while init_hour < orig_hour: rows.append({'Hours': init_hour.time()}) init_hour = init_hour + timedelta(minutes=60) df = pd.concat([df, pd.DataFrame(rows)], axis=0, ignore_index=True)  # Second transformation - just construct it without loop hours = pd.Series(pd.date_range("00:00:00", periods=24, freq="H").time, name="Hours") # Then insert/concat hours into your dataframe.  Example 2 Don't see the context so it's harder to know what's appropriate. Either of these two # Alternative 1 row2 = {'date': tmp_date, 'false_negatives': fn, 'total': total} row2df = pd.DataFrame.from_records([row2]) df2 = pd.concat([df2, row2df], ignore_index=True, axis=0) # how to handle index depends on context # Alternative 2 # assuming integer monotonic index - assign a new row with loc new_index = len(df2.index) df2.loc[new_index] = row2  Note: there's a reason append is deprecated. Using it row by row leads to very slow code. So it's important that we do more than just a local translation of the code to not waste computer time when the analysis runs. Pandas authors are trying to get us to understand and appreciate concatenating or building dataframes using more efficient means, i.e. not row-by-row. Combining many rows into one DataFrame and concatenating bigger dataframes together is the way to go.
Question: I get the message stdin is not a tty when I run the command below in my terminal. psql -U postgres kdc < kdc.psql  kdc is the database and kdc.psql is the psql file with commands to populate the database. I am in the directory that holds the psql file.

Answer: I am not sure what causes that message (it does not happen here), but you should be able to avoid it using the -f option: psql -U postgres -d kdc -f kdc.psql
Question: I am trying to load a Keras model on aws server with the following command import tensorflow_hub as hub from keras.models import load_model model = load_model(model_path, custom_objects={'KerasLayer': hub.KerasLayer})  but its giving an error FileNotFoundError: Op type not registered 'RegexSplitWithOffsets' in binary running on ip-10-0-xx-xxx.us-east-2.compute.internal. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed. You may be trying to load on a different device from the computational device. Consider setting the `experimental_io_device` option in `tf.saved_model.LoadOptions` to the io_device such as '/job:localhost'.  The exact same code was working the other day but this time it's giving an error. I double-checked if the model file is still there on the specified path. Also, the file is not modified so there is no doubt of the file being corrupt or anything. I guess the error has to do something with aws, but I don't how to resolve it. I tried finding any solution on the web but it wasn't helpful.

Answer: importing tensorflow_text resolve this issue
Question: I am new to building directed graphs with networkx and I'm trying to work out how to compare two graphs. More specifically, how to tell if a smaller graph is a subgraph (unsure of the exact terminology) of a larger graph As an example, assume I have the following directed graph:  I would like to be able to check whether a series of smaller graphs are sub-graphs of this initial graph. Returning a True value if they are (graph B), and False if they are not (graph C):  Graph B = Sub-graph of Graph A  Graph C != Sub-graph of Graph A Example Code import networkx A = nx.DiGraph() A.add_edges_from([('A','B'),('B','C'),('C','A')]) nx.draw_networkx(A) B = nx.DiGraph() B.add_edges_from([('A','B')]) nx.draw_networkx(B) C = nx.DiGraph() C.add_edges_from([('A','B'),('A','C')]) nx.draw_networkx(C)  I've had a look through the documentation and cannot seem to find what I need. An alternative I have been considering is to represent the nodes as a sequence of strings, and then searching for each substring in the main graphs string sequence - however, I can't imagine this is the most effecient/effective/stable way to solve the problem.

Answer: You are looking for a subgraph isomorphisms. nx.isomorphism.DiGraphMatcher(A, B).subgraph_is_isomorphic() # True nx.isomorphism.DiGraphMatcher(A, C).subgraph_is_isomorphic() # False  Note that the operation can be slow for large graphs, as the problem is NP-complete.
Question: I'm trying to generate two numpy arrays from one. One which is a slice slice of an original array, and another which represents the indexes which can be used to look up the values produced. The best way I can explain this is by example: import numpy as np original = np.array([ [5, 3, 7, 3, 2], [8, 4, 22, 6, 4], ]) sliced_array = original[:,::3] indices_of_slice = None # here, what command to use? for val, idx in zip(np.nditer(sliced_array), np.nditer(indices_of_slice)): # I want indices_of_slice to behave the following way: assert val == original[idx], "Error. This implementation is not correct. "  Ultimately what I'm aiming for is an array which I can iterate through with nditer, and a corresponding array indices_of_slices, which returns the original lookup indices (i,j,...). Then, the value of the sliced array should be equal to value of the original array in index (i,j,...). Main question is: Can I return both the new sliced array as well as the indices of the values when slicing a numpy array? Hopefully all is clear! Edit: Here are the expected printouts of the two arrays: # print(sliced_array) # >>> [[5 3] # >>> [8 6]] # expected result of # print(indices_of_slice) # >>> [[(0 0) (0 3)] # >>> [(1 0) (1 3)]] 

Answer: You can use numpy's slice np.s_[] with a tiny bit of gymnastics to get the indices you are looking for: slc = np.s_[:, ::3] shape = original.shape ix = np.unravel_index(np.arange(np.prod(shape)).reshape(shape)[slc], shape) >>> ix (array([[0, 0], [1, 1]]), array([[0, 3], [0, 3]])) >>> original[ix]￼ array([[5, 3], [8, 6]]) >>> original[slc] array([[5, 3], [8, 6]])  Note that this works with slices that have some reverse direction: slc = np.s_[:, ::-2] # ... (as above) >>> ix (array([[0, 0, 0], [1, 1, 1]]), array([[4, 2, 0], [4, 2, 0]])) >>> np.array_equal(original[ix], original[slc]) True
Question: I wanted to have no errors while using VSCode Pylance type checker. How to type the axs correctly in the following code: import matplotlib.pyplot as plt fig, axs = plt.subplots(2, 2)  In the image below, you can see that Pylance on VSCode is detecting an error. 

Answer: It turns out that strongly typing the axs variable is not straightforward at all and requires to understant well how to type np.ndarray. See this question and this question for more details. The simplest and most powerful solution is to wrap numpy.ndarray with ' characters, in order to avoid the infamous TypeError: 'numpy._DTypeMeta' object is not subscriptable when Python tries to interpret the [] in the expression. An example: import matplotlib.pyplot as plt import numpy as np import numpy.typing as npt import seaborn as sns from typing import cast, Type, Sequence import typing sns.set() # Some example data to display x = np.linspace(0, 2 * np.pi, 400) y = np.sin(x ** 2) fig, axs = plt.subplots( 2, 2, figsize=(12, 10) # set graph size ) # typechecking operation NDArrayOfAxes: typing.TypeAlias = 'np.ndarray[Sequence[Sequence[plt.Axes]], np.dtype[np.object_]]' axs = cast(np.ndarray, axs) axs[0, 0].plot(x, y) axs[0, 0].set_title("main") axs[1, 0].plot(x, y**2) axs[1, 0].set_title("shares x with main") axs[1, 0].sharex(axs[0, 0]) axs[0, 1].plot(x + 1, y + 1) axs[0, 1].set_title("unrelated") axs[1, 1].plot(x + 2, y + 2) axs[1, 1].set_title("also unrelated") fig.tight_layout()  Which is well detected by Pylance and runs correctly:
Question: I have two lists, one with word and another one with word type like this ['fish', 'Robert', 'dog', 'ball', 'cat', 'dog', 'Robert'] ['animal', 'person', 'animal', 'object', 'animal', 'animal', 'person']  I need to remove duplicates in the first list and to remove the type at the same index of the word removed. In the end I need something like this: ['fish', 'Robert', 'dog', 'ball', 'cat'] ['animal', 'person', 'animal', 'object', 'animal']  How can I do that?

Answer: just do a loop and use the zip function Script # Input Vars a = ['fish', 'Robert', 'dog', 'ball', 'cat', 'dog', 'Robert'] b = ['animal', 'person', 'animal', 'object', 'animal', 'animal', 'person'] # Output Vars c = [] d = [] # Process for e, f in zip(a, b): if(e not in c): c.append(e) d.append(f) # Print Output print(c) print(d)  Output ['fish', 'Robert', 'dog', 'ball', 'cat'] ['animal', 'person', 'animal', 'object', 'animal']
Question: I am trying to get used with new python's features (dataclasses). I am trying to initialize variables and I get error: raise ValueError(f'mutable default {type(f.default)} for field ' ValueError: mutable default <class 'dict'> for field headers is not allowed: use default_factory  My code: @dataclass class Application(): __config = ConfigParser() __config.read('mydb.ini') __host: str = __config.get('db','dbhost') __user: str = __config.get('db','dbuser') __password: str = __config.get('db','dbpw') __database: str = __config.get('db','database') url: str = "https://xxxx.domain.com/" headers: str = {'X-ApiKeys':'accessKey=xxxxxxx;secretKey=xxxxx','Content-Type': 'application/json'} def main(self): print(self.__host,self.__user,self.__password, self.__database) app = Application() if __name__=="__main__": app.main()  What's the proper way to initialize dictionaries?

Answer: Dataclasses have a few useful things for defining complex fields. The one that you need is called field. This one has the argument default_factory that needs to receive callable, and there is where lambda comes to the rescue. So using this above, code that will work looks like (just part with dict): from dataclasses import field from typing import Dict @dataclass class Application(): ... headers: Dict[str, str] = field( default_factory=lambda: {'X-ApiKeys':'accessKey=xxxxxxx;secretKey=xxxxx','Content-Type': 'application/json'} )