eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I create a list by appending dictionary entries containing display_name, browse_name and node_id of OPCUA server nodes. When I print the list, all the elements are on one line. I have no idea why. Please help! # Code for inserting elements into the Listbox def display_nodes(self, nodes_list): # Code zum Anzeigen der Nodes im GUI i = 0 print(nodes_list,sep=" ") display_text="" self.nodes_listbox.delete(0,tk.END) for node in nodes_list: display_text=str(node) self.nodes_listbox.insert(tk.END,display_text) pass # Code for reading nodes and adding them to the List def read_nodes(self,node): # Code zum Lesen der Nodes vom Server for childId in node.get_children(): ch = self.client.get_node(childId) print(ch.get_node_class()) if ch.get_node_class() == ua.NodeClass.Variable: #if str(ch.get_browse_name()).find("QualifiedName(1:") != -1: if (str(ch.nodeid)).find("ns=1")!= -1: node_data={ 'display_name':ch.get_display_name(), 'browse_name':ch.get_browse_name(), 'node_id':str(ch.nodeid), } print(node_data) self.nodes_list.append(node_data) else: #if str(ch.get_browse_name).find("QualifiedName(1:") != -1: node_data={ 'display_name':ch.get_display_name(), 'browse_name':ch.get_browse_name(), 'node_id':str(ch.nodeid), #"data_type":ch.get_data_type_as_variant_type(), } print(node_data) self.read_nodes(ch) return[self.nodes_list] 

Answer:  When i print the list, all the elements are in one line.  The problem can be fixed by using asterik(*) Change this: self.nodes_listbox.insert(tk.END,display_text)  to: self.nodes_listbox.insert(tk.END, *display_text)
Question: When I try to save a Python list in a csv, the csv have the items that I want to save separated by each character. I have a list like this with links: links = ['https://www.portalinmobiliario.com/MLC-2150551226-departamento-los-talaveras-id-117671-_JM#position=1&search_layout=grid&type=item&tracking_id=01bab66e-7cd3-43ce-b3d7-8389260b443d', 'https://www.portalinmobiliario.com/MLC-2148268902-departamento-los-espinos-id-116373-_JM#position=2&search_layout=grid&type=item&tracking_id=01bab66e-7cd3-43ce-b3d7-8389260b443d']  Im trying to save this to a csv with this code: with open('links.csv', 'w', newline='') as f: writer = csv.writer(f) writer.writerows(links)  The result I get from the list is the link in a row but each character in a column.  How can I get the links separated by rows but in the same column? 

Answer: writer.writerows expects the parameter to be an iterable of row lists (quoting the docs: "A row must be an iterable of strings or numbers for Writer objects"); right now it's interpreting your link strings as rows of 1-character columns (since a string is indeed an iterable of strings). In short, you'll need to wrap each link in a list (an 1-tuple would do too), e.g. with a generator: writer.writerows([link] for link in links)
Question: I have several CSVfiles with the same structure:  data_product_1.csv data_product_2.csv data_product_3.csv etc.  It is clear to me that to obtain a dataframe with all the data concatted together with polars, I can do something like: import polars as pl df = pl.read_csv("data_*.csv")  What I would like to do is to add an extra column to the final dataframe containing the name of the product, e.g.     data value product_code     2000-01-01 1 product_1   2000-01-02 2 product_1   2000-01-01 3 product_2   2000-01-02 4 product_2   2000-01-01 5 product_3     I'm aware I can load the files one by one, add the extra column and concat them together afterwards but I was wondering if I'm missing some other way to take advantage of polars performances here.

Answer: It seems you're wanting the filename added as a column, e.g. duckdb.sql(""" from read_csv_auto('data_*.csv', filename = true) """)  ┌────────────┬───────┬────────────────────┐ │ data │ value │ filename │ │ date │ int64 │ varchar │ ├────────────┼───────┼────────────────────┤ │ 2000-01-01 │ 1 │ data_product_1.csv │ │ 2000-01-02 │ 2 │ data_product_1.csv │ │ 2000-01-01 │ 3 │ data_product_2.csv │ │ 2000-01-02 │ 4 │ data_product_2.csv │ │ 2000-01-01 │ 4 │ data_product_3.csv │ │ 2000-01-02 │ 5 │ data_product_3.csv │ └────────────┴───────┴────────────────────┘  This has been requested a few times but is yet to be added to Polars: https://github.com/pola-rs/polars/issues/9096 You can replace read_csv with scan_csv which delays reading the file and returns a LazyFrame instead. The frames can be combined with concat which (by default) "computes" LazyFrames in parallel. from pathlib import Path # lazyframes csvs = [ pl.scan_csv(f).with_columns(product_code=pl.lit(f.name)) for f in Path().glob("data_*.csv") ] # inputs are read in parallel df = pl.concat(csvs).collect()  shape: (6, 3) ┌────────────┬───────┬────────────────────┐ │ data ┆ value ┆ product_code │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ str │ ╞════════════╪═══════╪════════════════════╡ │ 2000-01-01 ┆ 1 ┆ data_product_1.csv │ │ 2000-01-02 ┆ 2 ┆ data_product_1.csv │ │ 2000-01-01 ┆ 3 ┆ data_product_2.csv │ │ 2000-01-02 ┆ 4 ┆ data_product_2.csv │ │ 2000-01-01 ┆ 4 ┆ data_product_3.csv │ │ 2000-01-02 ┆ 5 ┆ data_product_3.csv │ └────────────┴───────┴────────────────────┘
Question: I am following Hands On Machine Learning Book by Aurélien Géron and running in to the following error. Code: y_train_large = (y_train.astype("int") >= 7) y_train_odd = (y_train.astype("int") % 2 == 1) y_multilabel = np.c_[y_train_large, y_train_odd] #model knn_clf = KNeighborsClassifier() knn_clf.fit(X_train, y_multilabel) y_train_knn_pred = cross_val_predict(knn_clf, X_train, y_multilabel, cv=3)  The last line produces the following error: { AttributeError: 'Flags' object has no attribute 'c_contiguous'" }  Since I am following the book, I expected this code to work. I have tried solutions from Google Bard and Claude AI chatbots but with no success.

Answer: There seems to be a bug report for this in Scikit-learn 1.3.0 (although it seems to have been fixed in the nightly builds). Try downgrading to version 1.2.2: pip uninstall scikit-learn pip install scikit-learn==1.2.2
Question: Sorry if my title is a bit crypt, but this is the problem I have a list of dict data = [{"a": 1, "b": 2},{"a": 3, "b": 4} ] and a list with keys = ["a","b"] I want in a template do this: for dat in data: <tr> for k in keys: <th> dat[k]</th> </tr>  to get this: <tr> <th>1</th> <th>2</th> </tr> <tr> <th>3</th> <th>4</th> </tr> 

Answer: Use one of these solutions if you want to keep the order given by the keys list. Result will be different with keys = ["a", "b"] VS keys = ["b", "a"].  Solution 1 - Prepare data in the view Process the data in the view. Create a list of list instead of dictionary to keep the order of your keys list. def home(request): data = [{"a": 1, "b": 2},{"a":3, "b":4}] data_to_render = [ [] for _i in range(len(data)) ] keys = ['a', 'b'] for i in range(len(data)): for k in keys: data_to_render[i] += [data[i].get(k)] context = { "data_to_render": data_to_render } return render(request, 'index.html', context)  And your template index.html may look like: <table> {% for l in data_to_render %} <tr> {% for value in l %} <th> {{ value }} </th> {% endfor %} </tr> {% endfor %} </table>  Solution 2 - Use a custom tag Your view might look like… def home(request): context = { "data" : [{"a": 1, "b": 2},{"a":3, "b":4}], "keys": ["a","b"] } return render(request, 'index.html', context)  Create a custom tag getval() at templatetags/extras.py: from django import template register = template.Library() @register.simple_tag def getval(dic, key): return dic.get(key)  Finally, your template might look like: {% load extras %} <table> {% for d in data %} <tr> {% for k in keys %} <th> {% getval d k %} </th> {% endfor %} </tr> {% endfor %} </table>
Question: I want to apply a function to a numpy array, which goes through infinity to arrive at the correct values: def relu(x): odds = x / (1-x) lnex = np.log(np.exp(odds) + 1) return lnex / (lnex + 1) x = np.linspace(0,1,10) np.where(x==1,1,relu(x))  correctly computes array([0.40938389, 0.43104202, 0.45833921, 0.49343414, 0.53940413, 0.60030842, 0.68019731, 0.77923729, 0.88889303, 1. ])  but also issues warnings: 3478817693.py:2: RuntimeWarning: divide by zero encountered in divide odds = x / (1-x) 3478817693.py:4: RuntimeWarning: invalid value encountered in divide return lnex / (lnex + 1)  How do I avoid the warnings? Please note that performance is of critical importance here, so I would rather avoid creating intermediate arrays.

Answer: Another possible solution, based on np.divide, to avoid division by zero. This solution is inspired by @hpaulj's comment. def relu(x): odds = np.divide(x, 1-x, out=np.zeros_like(x), where=x!=1) lnex = np.log(np.exp(odds) + 1) return lnex / (lnex + 1) x = np.linspace(0,1,10) np.where(x==1,1,relu(x))  Output: array([0.40938389, 0.43104202, 0.45833921, 0.49343414, 0.53940413, 0.60030842, 0.68019731, 0.77923729, 0.88889303, 1. ])
Question: I am quite new to the programming and Im struggling to this matter. Any help is appreciated! I have a dataframe of stocks including the prices and the signal if it will be up (1) or down (-1). I want to count the sequence of repetition into another column 'count'. So, when there is a sequence of 1,1,1; then the count will be 1,2,3. If its -1,-1,-1; then the count will be 1,2,3 too. Additionally, when a threshold value reaches 5, the counting resets. Doesn't matter if it's 1 or -1. So, what I have is:  price sign 0 13 1 1 12 1 2 11 -1 3 12 -1 4 13 1 5 14 1 6 14 1 7 14 1 8 14 1 9 14 1 10 14 1 . . .  And what I want is:  price sign count 0 13 1 1 1 12 1 2 2 11 -1 1 3 12 -1 2 4 13 1 1 5 14 1 2 6 14 1 3 7 14 1 4 8 14 1 5 9 14 1 1 10 14 1 2 . . .  I already have this code in normal python code. But I cannot do this in Pandas Vectorized Operation! Help me, please!

Answer: Use GroupBy.cumcount by consecutive values of sign with modulo 5: df['count'] = df.groupby(df['sign'].ne(df['sign'].shift()).cumsum()).cumcount() % 5 + 1 print (df) price sign count 0 13 1 1 1 12 1 2 2 11 -1 1 3 12 -1 2 4 13 1 1 5 14 1 2 6 14 1 3 7 14 1 4 8 14 1 5 9 14 1 1 10 14 1 2  Detail: print (df.assign(consecutive=df['sign'].ne(df['sign'].shift()).cumsum(), counter=df.groupby(df['sign'].ne(df['sign'].shift()).cumsum()).cumcount(), count = df.groupby(df['sign'].ne(df['sign'].shift()).cumsum()).cumcount() % 5 + 1)) price sign consecutive counter count 0 13 1 1 0 1 1 12 1 1 1 2 2 11 -1 2 0 1 3 12 -1 2 1 2 4 13 1 3 0 1 5 14 1 3 1 2 6 14 1 3 2 3 7 14 1 3 3 4 8 14 1 3 4 5 9 14 1 3 5 1 10 14 1 3 6 2
Question: Looks like numpy is using deprecated function DataFrame.swapaxes in fromnumeric.py. Anaconda3\lib\site-packages\numpy\core\fromnumeric.py:59: FutureWarning: 'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead. return bound(*args, **kwds)  I am getting this warning from the following line of code in Jupyter Notebook: train, val, test = np.split(df.sample(frac=1), [int(0.8*len(df)), int(0.9*len(df))])  This is the structure of the dataframe I am using:  What exactly is raising this warning and what should I change in my code to get rid of this Warning? I also found that this is currently an open issue of numpy in github. It will be great if anybody could help. Thanks in advance.

Answer: According to the numpy issue on github, this "bug" will not be fixed in numpy. The official statement is that np.split should not be used to split pandas DataFrames anymore. Instead, iloc should be used to split DataFrames as it is described in this answer. As it looks that you are splitting the DataFrame for machine learning reasons, please be aware that there are built-in methods in many machine learning libraries doing the test/train(/validation) split for you. Here the example of sklearn.
Question: In the context of exporting pytorch code to ONNX, I get this warning: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. Here is the offending line: text_lengths = torch.tensor([text_inputs.shape[1]]).long().to(text_inputs.device)  text_inputs is a torch.Tensor of shape torch.Size([1, 81]) And the warning is spot on and cannot be ignored, because the shape of text_inputs is supposed to be dynamic. I need text_lengths to be a torch.Tensor that contains the number 81 coming from the shape of text_inputs. The "offending line" from above succeeds in doing that, but we actually make a round trip from pytorch to a Python int and back to pytorch, because the elements in the torch.Size objects are Python ints. This is (1) somewhat weird, (2) probably inefficient in terms of GPU -> CPU -> GPU and, as stated above, an actual problem in the ONNX exporting context. Is there some other way how I can use a tensor's shape in torch computations, without "leaving" the torch world?

Answer: EDIT As was pointed out in the comments, the original answer was incorrect. Using torch.tensor will lead to a constant value in the exported graph. When tracing, outputs of torch.tensor.shape and torch.tensor.size should return tensors (instead of python integers), so the code above should export as intended just with text_lengths = text_inputs.shape[1]  returning a LongTensor with the correct value. Seems that the code doesn't behave as intended because of the brackets in the constructor. The shape inside of [text_inputs.shape[1]] gets interpreted as an integer inside of a python list, and consequently saved as a constant during the trace. Dropping the extra brackets when creating the tensor will export correctly as a scalar defined at runtime by the shape of text_inputs . text_lengths = torch.LongTensor(text_inputs.size(1))
Question: Given the documentation of nixtla y dont find any way to compute the prediction intervals for insample prediction (training data) but just for future predicitons. I put an example of what I can achieve but just to predict (future). from statsforecast.models import SeasonalExponentialSmoothing, ADIDA, ARIMA from statsforecast.utils import ConformalIntervals # Create a list of models and instantiation parameters intervals = ConformalIntervals(h=24, n_windows=2) models = [ SeasonalExponentialSmoothing(season_length=24,alpha=0.1, prediction_intervals=intervals), ADIDA(prediction_intervals=intervals), ARIMA(order=(24,0,12), season_length=24, prediction_intervals=intervals), ] sf = StatsForecast( df=train, models=models, freq='H', ) levels = [80, 90] # confidence levels of the prediction intervals forecasts = sf.forecast(h=24, level=levels) forecasts = forecasts.reset_index() forecasts.head()  So my goal will be to do something like:  forecasts = sf.forecast(df_x, level=levels)  So we can have any prediction intervals in the training set.

Answer: You can access the in-sample forecast with a conformal prediction interval using the forecast_fitted_values method.  Your selected models need to support in-sample fitted values. From your example code, SeasonalExponentialSmoothing and ADIDA didn't support in-sample fitted values currently. You can find the list of supported models from official documentation You need to specify the fitted=True argument in the forecast step. Then, you can access the in-sample forecast with a conformal prediction interval using the forecast_fitted_values method.  import pandas as pd from statsforecast import StatsForecast from statsforecast.models import SeasonalExponentialSmoothing, ADIDA, ARIMA from statsforecast.utils import ConformalIntervals train = pd.read_csv('https://auto-arima-results.s3.amazonaws.com/M4-Hourly.csv') test = pd.read_csv('https://auto-arima-results.s3.amazonaws.com/M4-Hourly-test.csv').rename(columns={'y': 'y_test'}) n_series = 1 uids = train['unique_id'].unique()[:n_series] # select first n_series of the dataset train = train.query('unique_id in @uids') test = test.query('unique_id in @uids') # Create a list of models and instantiation parameters intervals = ConformalIntervals(h=24, n_windows=2) models = [ ARIMA(order=(24,0,12), season_length=24, prediction_intervals=intervals), ] sf = StatsForecast( df=train, models=models, freq='H', n_jobs=-1 ) levels = [80, 90] # confidence levels of the prediction intervals forecasts = sf.forecast(h=24, level=levels, fitted=True) # Add fitted=True to store in-sample predictions. insample_forecasts = sf.forecast_fitted_values() # Access insample predictions
Question: Problem summary Whenever I try to create a plot with plotly express 5.18.0 containing a subplot and axes labels that are not numbers, I only get labels for the first subplot subsequent subplots show empty axis labels. How can I ensure that all subplots show their respective axes labels, even if they contain strings? Example data import numpy as np import pandas as pd import plotly.express as px N = 100 food = ["Dim sum", "Noodles", "Burger", "Pizza", "Pancake"] drink = ["Beer", "Wine", "Soda", "Water", "Fruit juice", "Coffee", "Tea"] df = pd.DataFrame( { "age": np.random.randint(8, 99, N), "favourite_food": np.random.choice(food, N, replace=True), "favourite_drink": np.random.choice(drink, N, replace=True), "max_running_speed": np.random.random(N)*20, "number_of_bicycles": np.random.randint(0, 5, N) } ) df.age.replace({range(0, 19): "Kid", range(19, 100): "Adult"}, inplace=True)  Random 5 rows:      age favourite_food favourite_drink max_running_speed number_of_bicycles     0 Adult Dim sum Wine 8.57536 2   65 Kid Pizza Water 9.45698 1   57 Kid Pancake Beer 11.1445 0   84 Adult Dim sum Soda 8.80699 0   45 Adult Pizza Fruit juice 17.7258 4    Demonstration of problem If I now create a figure with two subplots:  First subplot contains the distribution of the max. running speed (a number) Second subplot contains the distribution of the number of bicycles (a number)  For convenience I use the facet_col argument in combination with the wide-form support of plotly express and the formatting updates I found in this related Q&A): px.histogram( df, x=["max_running_speed", "number_of_bicycles"], facet_col="variable", color="age", barmode="group", histnorm="percent", text_auto=".2r", ).update_xaxes(matches=None, showticklabels=True).update_yaxes(matches=None, showticklabels=True)   All works as it should ✅: I get separate ranges x- and y-axes and I get separate labels on the x-axes. Now I do the same, but for the columns with text data: px.histogram( df, x=["favourite_food", "favourite_drink"], facet_col="variable", color="age", barmode="group", histnorm="percent", text_auto=".2r", ).update_xaxes(matches=None, showticklabels=True).update_yaxes(matches=None, showticklabels=True)   Now there's a problem ❌: The x-axis of the right plot does not show the names of the favourite drinks. What I've tried I checked the underlying data JSON object, as I noticed that when I hover over the bars of the right plot, the "value" field is empty:  But when I inspect the JSON object in the .data key of the figure, I see that x-values are present for both histograms: 

Answer: One of the plotly contributors suggested I use the .update_traces(bingroup=None) on my figure. This indeed shows the missing categories on the right plot and is a viable workaround for now.
Question: I am new to python programming so excuse me if the question may seem silly or trivial. So for a certain function I need to do a check in case x is a vector and (in that case) it must be a column vector. However I wanted to make it so that the user could also pass it a row vector and turn it into a column vector. However at this point how do I make it not do the check if x is a scalar quantity. I attach a code sketch to illustrate the problem. Thanks in advance to whoever responds import numpy as np x = np.arange(80, 130, 10) # x: float = 80.0 print('array before:\n', x) if x is not np.array: x = np.array(x).reshape([len(x), 1]) print('array after:\n', x) 

Answer: You can check the type using if not isinstance(x, np.ndarray). After that check you can check the number of dimension of the array and convert to a columnar array. In the code below, first if block checks and converts to an array. Then we get how many dimensions are missing. I.e. a scalar value is missing 2 dimensions, a flat list is missing 1 dimension. Finally we iterate over the number of missing dimensions up-converting the array's dimensions with each step. def to_column_array(x): if not isinstance(x, np.ndarray): x = np.array(x) missing_dims = 2 - x.ndim if missing_dims < 0: raise ValueError('You array has too many dimensions') for _ in range(missing_dims): x = x.reshape(-1, 1) return x  So if a user has just a number, this convert it to an array of size (1, 1). to_column_array(10) # returns: array([[10]])  Passing in a list or list of lists converts to an array as well. to_column_array([3, 6, 9]) # returns: array([[3], [6], [9]]) to_column_array([[1, 2], [3, 4], [5, 6]) # returns: array([[1, 2], [3, 4], [5, 6]])
Question: My old code concats some dataframes, some may be empty. I now receive two future warnings regarding this. My goal is to have the old logic but without any warnings. Mainly I need to retain all the column names without empty rows. I wrote the code below (fixed 1 of the 2 warning) but it still gives me FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. import io import pandas as pd df_list = ['RevisionTime,Data,2019/Q2,2019/Q3,2019/Q4\r\n', 'RevisionTime,Data,2019/Q3\r\n2019-08-17,10.5,10.5\r\n', 'RevisionTime,Data,2019/Q3\r\n2019-09-18 08:10:00,51.0,51.0\r\n', 'RevisionTime,Data,2019/Q3\r\n2019-10-18 08:10:00,111.5,111.5\r\n', 'RevisionTime,Data,2019/Q3,2019/Q4\r\n2019-11-15 22:31:00,182.0,111.5,70.5\r\n'] # list with dataframes df_list = [pd.read_csv(io.StringIO(df)) for df in df_list] # to avoid 'The behaviour of array concatenation with empty entries is deprecated.' # and to retain all column names # https://stackoverflow.com/questions/63970182/concat-list-of-dataframes-containing-empty-dataframes for i, df in enumerate(df_list): col_length = len(df.columns) template = pd.DataFrame(data=[[pd.NA] * col_length], columns=df.columns) df_list[i] = df if not df.empty else template # res_df = pd.concat(df_list) # warning here res_df = res_df.dropna(how='all') # remove empty rows print(res_df)  Output dataframe should look like:  RevisionTime Data 2019/Q2 2019/Q3 2019/Q4 0 2019-08-17 10.5 NaN 10.5 NaN 0 2019-09-18 08:10:00 51.0 NaN 51.0 NaN 0 2019-10-18 08:10:00 111.5 NaN 111.5 NaN 0 2019-11-15 22:31:00 182.0 NaN 111.5 70.5  Basically, help me fix the code to remove the warning.

Answer: You can create Index with all possible column names and then reindex the final dataframe: # create `column_names` index with all posible column names column_names = pd.Index([]) for df in df_list: column_names = column_names.union(df.columns) res_df = pd.concat([df for df in df_list if not df.empty]) # reindex the final dataframe (this adds NaN to missing columns) res_df = res_df.reindex(column_names, axis=1) print(res_df)  Prints:  2019/Q2 2019/Q3 2019/Q4 Data RevisionTime 0 NaN 10.5 NaN 10.5 2019-08-17 0 NaN 51.0 NaN 51.0 2019-09-18 08:10:00 0 NaN 111.5 NaN 111.5 2019-10-18 08:10:00 0 NaN 111.5 70.5 182.0 2019-11-15 22:31:00