eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am looking for the idiomatic and fast python solution for the following problem. Input is a list of sets. For example, 3 sets of strings. [ {a, b, c, d, e}, {a, c, e, f, g}, {e, f, a, d, l} ]  I would like to find all choices of string combinations so that there is only one element in combination per set. For example, these are "mappings" that show at what list positions these strings occur: a -> 0, 1, 2 b -> 0 c -> 0, 1 d -> 0, 2 e -> 0, 1, 2 f -> 1, 2 g -> 1 l -> 2  So the correct solution is the following list of sets a b, g, l b, f e c, l d, g  Here are some examples of incorrect solutions: a, b # incorrect because more than one element (2) from set 0 are used b, l # incorrect because less than one element (0) from set 1 is used  I tried naive recurrent solution that adds one element at a time from the row and then checks remaining rows if they met requirements or not. It was extremely slow (probably because I overused list concatenations, copies, etc). I am looking for the solution that will work relatively fast (less than 10 s) for 100 rows of sets where each of them has 50 elements in average. As noted in the comments this problem may not be solvable in finite time with these constraints. In that case I am still interested in the python solution that will work with weaker constraints (e.g. alphabet size 30, average set size 10, number of sets 50).

Answer: I would use a SAT solver like z3 for this. import z3 sets = [ {"a", "b", "c", "d", "e"}, {"a", "c", "e", "f", "g"}, {"e", "f", "a", "d", "l"} ] alphabet = set.union(*sets) zvars = {w: z3.Bool(w) for w in alphabet} sol = z3.Solver() for s in sets: # Exactly one in each set. sol.add(z3.PbEq([(zvars[w], True) for w in s], 1)) # Iterate over all solutions. while sol.check() == z3.sat: model = sol.model() print({w for w, v in zvars.items() if bool(model[v])}) # Prevent same solution being returned. trues = [v for v in zvars.values() if bool(model[v])] falses = [z3.Not(v) for v in zvars.values() if not bool(model[v])] sol.add(z3.Not(z3.And(trues + falses)))  Solution: {'e'} {'g', 'd'} {'g', 'b', 'l'} {'b', 'f'} {'c', 'l'} {'a'}
Question: How can I remove the play and stop buttons and just keep the slider? import plotly.express as px import pandas as pd df = pd.DataFrame(dict(x=[0, 1, 0, 1], y=[0, 1, 1, 0], z=[0, 0, 1, 1])) px.line(df, "x", "y", animation_frame="z")  

Answer: fig = px.line(df, "x", "y", animation_frame="z") fig["layout"].pop("updatemenus") fig.show()
Question: I'm using seaborn to create the following chart:  I'd like to customize the colors that are generated by hue , preferably to set the order of the colors as Blue, Green, Yellow, Red. I have tried passing a color or a list of colors to the color argument in sns.barplot however it yields either gradients of the color or an error. Please Advise. You can use the following code to reproduce the chart: import matplotlib.pyplot as plt import seaborn as sns import pandas as pd df = pd.DataFrame({'Early': {'A': 686, 'B': 533, 'C': 833, 'D': 625, 'E': 820}, 'Long Overdue': {'A': 203, 'B': 237, 'C': 436, 'D': 458, 'E': 408}, 'On Time': {'A': 881, 'B': 903, 'C': 100, 'D': 53, 'E': 50}, 'Overdue': {'A': 412, 'B': 509, 'C': 813, 'D': 1046, 'E': 904}}) df_long = df.unstack().to_frame(name='value') df_long = df_long.swaplevel() df_long.reset_index(inplace=True) df_long.columns = ['group', 'status', 'value'] df_long['status'] = pd.Categorical(df_long['status'], ["Early", "On Time", "Overdue", "Long Overdue"]) df_long = df_long.sort_values("status") fig, ax = plt.subplots(figsize=(18.5, 10.5)) g = sns.barplot(data=df_long, x='group', y='value', hue='status', ax=ax) for bar in g.patches: height = bar.get_height() ax.text(bar.get_x() + bar.get_width() / 2., 0.5 * height, int(height), ha='center', va='center', color='white') plt.xticks(fontsize=12) plt.legend(loc='upper left', prop={'size': 14}) ax.xaxis.label.set_visible(False) ax.axes.get_yaxis().set_visible(False) plt.show() 

Answer: The hue variable of seaborn.barplot() is mapped via palette:  palette: palette name, list, or dict Colors to use for the different levels of the hue variable. Should be something that can be interpreted by seaborn.color_palette(), or a dictionary mapping hue levels to matplotlib colors.  So to customize your hue colors,  either define a color list: palette = ['tab:blue', 'tab:green', 'tab:orange', 'tab:red']   or a hue-color dictionary: palette = { 'Early': 'tab:blue', 'On Time': 'tab:green', 'Overdue': 'tab:orange', 'Long Overdue': 'tab:red', }    And pass that to palette: sns.barplot(data=df_long, x='group', y='value', hue='status', palette=palette, ax=ax)
Question: I am trying to train PeleeNet pytorch and got the following error  train.py line 80  pelee_voc train configuration 

Answer: Turning the shuffle parameter off in the dataloader solved it. Got the answer form here.
Question: To customize the styles of the boxplot displayed inside a violinplot, on could try to plot a boxplot in front of a violinplot. However this does not seem to work as it is always displayed behind the violinplot when using seaborn. When using seaborn + matplotlib this works (but only for a single category): import matplotlib.pyplot as plt import seaborn as sns import numpy as np df=pd.DataFrame(np.random.rand(10,2)).melt(var_name='group') fig, axes = plt.subplots() # Seaborn violin plot sns.violinplot(y=df[df['group']==0]['value'], color="#af52f4", inner=None, linewidth=0, saturation=0.5) # Normal boxplot has full range, same in Seaborn boxplot axes.boxplot(df[df['group']==0]['value'], whis='range', positions=np.array([0]), showcaps=False,widths=0.06, patch_artist=True, boxprops=dict(color="indigo", facecolor="indigo"), whiskerprops=dict(color="indigo", linewidth=2), medianprops=dict(color="w", linewidth=2 )) axes.set_xlim(-1,1) plt.show()   However when using only seaborn to be able to plot across multiple categories, ordering is always wrong: sns.violinplot(data=df, x='group', y='value', color="#af52f4", inner=None, linewidth=0, saturation=0.5) sns.boxplot(data=df, x='group', y='value', saturation=0.5) plt.show()   Even when trying to fix this with zorder this does not work.

Answer: The zorder parameter of sns.boxplot only affects the lines of the boxplot, but not the rectangular box. One possibility is to access these boxes afterwards; they form the list of artists in ax.artists. Setting their zorder=2 will put them in front of the violins while still being behind the other boxplot lines. In the comments, @mwaskom, noted a better way. sns.boxplot delegates all parameters it doesn't recognize via **kwargs to ax.boxplot. One of these is boxprops with the properties of the box rectangle. So, boxprops={'zorder': 2} would change the zorder of only the box. Here is an example: import matplotlib.pyplot as plt import seaborn as sns import pandas as pd import numpy as np df = pd.DataFrame(np.random.rand(10, 2)).melt(var_name='group') ax = sns.violinplot(data=df, x='group', y='value', color="#af52f4", inner=None, linewidth=0, saturation=0.5) sns.boxplot(data=df, x='group', y='value', saturation=0.5, width=0.4, palette='rocket', boxprops={'zorder': 2}, ax=ax) plt.show()   Here is another example, using the tips dataset: tips = sns.load_dataset('tips') ax = sns.violinplot(data=tips, x='day', y='total_bill', palette='turbo', inner=None, linewidth=0, saturation=0.4) sns.boxplot(x='day', y='total_bill', data=tips, palette='turbo', width=0.3, boxprops={'zorder': 2}, ax=ax)
Question: I'm working with PyCharm 2019 and Django, in Windows 10 in a project that I haven't opened in a year. The Project files window is showing up as yellow, which seems new. What does this mean and how to I get the files to appear as white.  

Answer: What the yellow background usually means is that the files are excluded form the project (it can also mean the files are "read-only"). This might happen for several reasons, the .idea folder might have broken and you need to delete it and recreate the project. If your project is installed in a venv sometimes the source files are marked read-only (which means the source files being edited are the versions installed in the venv). So here it gets complicated because it can depend on the specifics of the project itself. My usual steps for this problem are:  Close and reopen the project.  See if marking one of the directories as sources root changes the file color in the project tree. (Files might have been marked as excluded from the project for whatever reason.)  Just to help diagnosing the issue, open a search and go to costum scopes, see what scope those directories are associated with.  Check if file permissions are read-only. This can happen if you logged into PyCharm (or the OS) with a user account that doesn't have editing permissions on those files.  Delete the .idea folder (so the IDE recreates it) and create a new project with those files. (Remember to make a backup copy.)
Question: I have to use selenium and proxy with authentication. I have a few constraints  I can't use selenium-wire (only pure selenium allowed) I have to use headless mode (e.g. chrome_options.add_argument("--headless"))  I read this answer Python proxy authentication through Selenium chromedriver but it doesn't work for headless mode. Is it possible to use proxy with authentication in my case? (browser(Chrome, Firefox) is not important)  I need python function to create selenium webdriver object authenticate to proxy

Answer: you can't because you need a GUI to handle it with selenium in your case so I would recommend using a virtual display like Xvfb display server You can use PyVirtualDisplay (a Python wrapper for Xvfb) to run headless. for Linux sudo apt-get install firefox xvfb  install virtual display for python pip install pyvirtualdisplay  then from pyvirtualdisplay import Display from selenium import webdriver display = Display(visible=0, size=(800, 600)) display.start() # now Firefox will run in a virtual display. # you will not see the browser. browser = webdriver.Firefox() browser.get('http://www.google.com') print browser.title browser.quit() display.stop()  in this case, you do not need to add options.add_argument("--headless") argument and you can follow the answers commented above as a solution or doing it your way but I think this is the best solution for using pure selenium for proxy
Question: I've got a pandas dataframe called data_clean. It looks like this:  I want to convert it to a Spark dataframe, so I use the createDataFrame() method: sparkDF = spark.createDataFrame(data_clean) However, that seems to drop the index column (the one that has the names ali, anthony, bill, etc) from the original dataframe. The output of sparkDF.printSchema() sparkDF.show()  is root |-- transcript: string (nullable = true) +--------------------+ | transcript| +--------------------+ |ladies and gentle...| |thank you thank y...| | all right thank ...| | | |this is dave he t...| | | | ladies and gen...| | ladies and gen...| |armed with boyish...| |introfade the mus...| |wow hey thank you...| |hello hello how y...| +--------------------+  The docs say createDataFrame() can take a pandas.DataFrame as an input. I'm using Spark version '3.0.1'. Other questions on SO related to this don't mention this problem of the index column disappearing:  This one about converting Pandas to Pyspark doesn't mention this issue of the index column disappearing. Same with this one And this one relates to data dropping during conversion, but is more about window functions.  I'm probably missing something obvious, but how do I get to keep the index column when I convert from a pandas dataframe to a PySpark dataframe?

Answer: Spark DataFrame has no concept of index, so if you want to preserve it, you have to assign it to a column first using reset_index in a pandas dataframe You can also use inplace to avoid additional memory overhead while resting the index df.reset_index(drop=False,inplace=True) sparkDF = sqlContext.createDataFrame(df)
Question: I'm trying to classify text data into multiple classes. I'd like to perform cross-validation to compare several models with sample weights. With each model, I can put a parameter like this. all_together = y_train.to_numpy() unique_classes = np.unique(all_together) c_w = class_weight.compute_class_weight('balanced', unique_classes, all_together) clf = MultinomialNB().fit(X_train_tfidf, y_train, sample_weight=[c_w[i] for i in all_together])  It doesn't seem cross_val_score() allows parameter about sample_weight. How can I do this with cross validation? models = [ RandomForestClassifier(n_estimators=200, max_depth=3, random_state=0), LinearSVC(), MultinomialNB(), LogisticRegression(random_state=0), ] all_together = y_train.to_numpy() unique_classes = np.unique(all_together) c_w = class_weight.compute_class_weight('balanced', unique_classes, all_together) CV = 5 cv_df = pd.DataFrame(index=range(CV * len(models))) entries = [] for model in models: model_name = model.__class__.__name__ f1_micros = cross_val_score(model, X_tfidf, y_train, scoring='f1_micro', cv=CV) for fold_idx, f1_micro in enumerate(f1_micros): entries.append((model_name, fold_idx, f1_micro)) cv_df_women = pd.DataFrame(entries, columns=['model_name', 'fold_idx', 'f1_micro']) 

Answer: cross_val_score has a parameter called fit_params which accepts a dictionary of parameters (keys) and values to pass to the fit() method of the estimator. In your case, you can do cross_val_score(model, X_tfidf, y_train, scoring='f1_micro', cv=CV, fit_params={'sample_weight': [c_w[i] for i in all_together]})
Question: from ReadWriteMemory import ReadWriteMemory rwm = ReadWriteMemory() process = rwm.get_process_by_name("javaw.exe") process.open() module_base = 0x6FBB0000 static_address_offset = 0x007FD7C0 static_address = module_base + static_address_offset pitch_pointer = process.get_pointer(static_address, offsets=[0xB8, 0x1C8, 0x1C8, 0x1D0, 0x178, 0xAC, 0x8C]) camera_pitch = process.read(pitch_pointer) print(camera_pitch)  I am trying to get the camera pitch using a pointer I got in cheat engine, and the script works fine but the camera pitch is a float value, while process.read(pitch_pointer) returns an int, and that for example sets camera_pitch to 1108138163 instead of 35.2. Can't find how to get a float instead anywhere.

Answer: You can use the struct module. Something like this: >>> import struct >>> i_value = 1108138163 >>> struct.unpack("@f", struct.pack("@I", i_value))[0] 35.21162033081055  That is, you convert your integer to a 4-byte array, and then you convert that to a float. struct.unpack always returns a tuple, in this case of a single value, so use [0] to get to it.
Question: I'm doing a Seaborn lineplot for longitudinal data which is grouped by "Subscale" using hue and by "Item" using style. Here is my code (I hope this is understandable also without data): ax = sns.lineplot(data = df, x = 'Week', y = 'Value', style = 'Item', hue = 'Subscale', palette = 'colorblind', markers = True) plt.legend(bbox_to_anchor = (1.03, 1.02), fontsize = 10)  Whicht gets me this plot:  What I want is to combine the sublegends so that it only shows the legend for "Item" but the items are colored according to "Subscale", kind of like this:  I've failed to create this, so if any of you could help, this would be highly appreciated! Thanks :)

Answer: If I understand correctly, all items of a certain type have the same subscale. So, you already have (or can create) a dictionary that maps an item type to the corresponding subscale. Seaborn creates following labels for the legend:  'Subscale' for a subtitle each of the subscales 'Item' for a second subtitle each of the items  Each label corresponds to a "handle" identifying how the marker looks like. The following code:  finds the index of 'Item' to be able to split the arrays of labels and handles extracts the colors of the 'Subscale's applies these colors to the item markers only uses the items for the legend  import matplotlib.pyplot as plt import seaborn as sns import pandas as pd import numpy as np # first create some random data similar to the description np.random.seed(123) items = ['01', '04', '05', '06', '07', '10', '11', '13'] N = len(items) subscale_dict = {'01': 'A', '04': 'C', '05': 'C', '06': 'C', '07': 'A', '10': 'B', '11': 'B', '13': 'A'} df = pd.DataFrame({'Week': np.tile(np.arange(7), N), 'Value': np.random.rand(7 * N), 'Item': np.repeat(items, 7)}) df['Subscale'] = df['Item'].apply(lambda i: subscale_dict[i]) df['Subscale'] = pd.Categorical(df['Subscale']) # creates a fixed order # create the line plot as before ax = sns.lineplot(data=df, x='Week', y='Value', style='Item', hue='Subscale', palette='colorblind', markers=True) # create a dictionary mapping the subscales to their color handles, labels = ax.get_legend_handles_labels() index_item_title = labels.index('Item') color_dict = {label: handle.get_color() for handle, label in zip(handles[1:index_item_title], labels[1:index_item_title])} # loop through the items, assign color via the subscale of the item idem for handle, label in zip(handles[index_item_title + 1:], labels[index_item_title + 1:]): handle.set_color(color_dict[subscale_dict[ label]]) # create a legend only using the items ax.legend(handles[index_item_title + 1:], labels[index_item_title + 1:], title='Item', bbox_to_anchor=(1.03, 1.02), fontsize=10) plt.tight_layout() plt.show()
Question: I trained a simple model with Keras/TF2.5 and saved it as saved model. tf.saved_model.save(my_model,'/path/to/model')  If I examine it via saved_model_cli show --dir /path/to/model --tag_set serve --signature_def serving_default  I get these outputs/names: inputs['conv2d_input'] tensor_info: dtype: DT_FLOAT shape: (-1, 32, 32, 1) name: serving_default_conv2d_input:0 outputs['dense'] tensor_info: dtype: DT_FLOAT shape: (-1, 2) name: StatefulPartitionedCall:0  The names serving_default_conv2d_input and StatefulPartitionedCall can actually be used for inference. I want to extract them using python API. If I query it by loading the model: >>> m=tf.saved_model.load('/path/to/model') >>> m.signatures['serving_default'].inputs[0].name 'conv2d_input:0' >>> m.signatures['serving_default'].outputs[0].name 'Identity:0'  I get entirely different names. Questions:  How can I extract these names serving_default_conv2d_input and StatefulPartitionedCall from python API? Alternatively how can I define/fix the names when I call tf.saved_model.save? What does :0 mean?  And side question: How do you handle deployment TF model to production via SavedModel?

Answer: The input/output tensor names displayed by saved_model_cli can be extracted as follows: from tensorflow.python.tools import saved_model_utils saved_model_dir = '/path/to/model' tag_set = 'serve' signature_def_key = 'serving_default' # 1. Load MetaGraphDef with saved_model_utils meta_graph_def = saved_model_utils.get_meta_graph_def(saved_model_dir, tag_set) # 2. Get input signature names input_signatures = list(meta_graph_def.signature_def[signature_def_key].inputs.values()) input_names = [signature.name for signature in input_signatures] print(input_names) # ['serving_default_conv2d_input:0'] # 3. Get output signature names output_signatures = list(meta_graph_def.signature_def[signature_def_key].outputs.values()) output_names = [signature.name for signature in output_signatures] print(output_names) # ['StatefulPartitionedCall:0']  Regarding the meaning of :0, op_name:0 means "the tensor that is the 0th output of an operation called op_name." So you might use …:1 to get the output of an operation with multiple outputs, but many operations are single-output so you'll always use …:0 for them (source: @mrry's comment).
Question: I'm trying to customize django's AbstractUser. When I try to reset username to None, I get the following exception: "django.core.exceptions.FieldDoesNotExist: User has no field named 'username'".  Here is my code: class UserManager(BaseUserManager): use_in_migrations = True def _create_user(self, email, password, **extra_fields): if not email: raise ValueError("L'adresse e-mail donnée doit etre definie") email = self.normalize_email(email) user = self.model(email=email, **extra_fields) user.set_password(password) user.save(using=self._db) return user def create_user(self, email, password, **extra_fields): extra_fields.setdefault("is_staff", False) extra_fields.setdefault("is_superuser", False) return self._create_user(email, password, **extra_fields) def create_superuser(self, email, password, **extra_fields): extra_fields.setdefault("is_staff", True) extra_fields.setdefault("is_superuser", True) if extra_fields.get("is_staff") is not True: raise ValueError("Superuser must have is_staff=True") if extra_fields.get("is_superuser") is not True: raise ValueError("Superuser must have is_superuser=True") return self._create_user(email, password, **extra_fields) class User(AbstractUser): username = None email = models.EmailField('email adress', unique=True) telephone = models.CharField(max_length=20) REQUIRED_FIELDS = []  What can I do to solve this problem?

Answer: You haven't set a value for USERNAME_FIELD in your code. This must be set to a field that uniquely identifies a user instance. AbstractUser sets this to 'username' and hence you are getting the error. You can set this to 'email' to solve your problem: class User(AbstractUser): username = None email = models.EmailField('email adress', unique=True) telephone = models.CharField(max_length=20) # set below value USERNAME_FIELD = 'email' REQUIRED_FIELDS = []
Question: I would like to separate my panels because the titles superposed. ADX title is in RSI panel. I tried with tight_layout=True but still the same. My code:  ap0 = [ mpf.make_addplot(df['sma_200'],color='#FF0000', panel=2), mpf.make_addplot(df['sma_50'],color='#ffa500', panel=2), mpf.make_addplot(df['sma_20'],color='#00FF00', panel=2), mpf.make_addplot(df['rsi'],color='#ffa500', panel=0, title="RSI"), mpf.make_addplot(df['hline_30'], panel=0), mpf.make_addplot(df['hline_70'], panel=0), mpf.make_addplot(df['adx'],color='#0000FF', panel=1, secondary_y=False, title="ADX"), mpf.make_addplot(df['-di'],color='#FF0000', panel=1, secondary_y=False), mpf.make_addplot(df['+di'],color='#32cd32', panel=1, secondary_y=False), mpf.make_addplot(df['hline_25'], panel=1, secondary_y=False) ] fig, axlist = mpf.plot( df, panel_ratios=(.05, .05, .2, .05), type="hollow_candle", yscale='log', volume=True, title="{} - {}".format(ticker, interval), style=mpf_style, figsize=(12.8, 10), returnfig=True, closefig=True, addplot=ap0, main_panel=2, volume_panel=3, num_panels=4, )  

Answer: Mplfinance subplots do not have a function to adjust the graph spacing, so there is a way to fit them into matplotlib subplots and make them spaced. We recommend using the y-axis label instead of the title, which is a feature only available in Mplfinance. import yfinance as yf import mplfinance as mpf import matplotlib.pyplot as plt df = yf.download("AAPL", start="2021-01-01", end="2021-07-01") df['sam_20'] = df['Close'].rolling(20).mean() df['sam_50'] = df['Close'].rolling(50).mean() fig = mpf.figure(style='yahoo',figsize=(7,8)) ax1 = fig.add_subplot(4,1,1, sharex=ax4) ax2 = fig.add_subplot(4,1,2, sharex=ax4) ax3 = fig.add_subplot(4,1,3, sharex=ax4) ax4 = fig.add_subplot(4,1,4) ap0 = [ mpf.make_addplot(df['sam_20'], color='#00FF00', panel=0, title='sma_20', ax=ax1), mpf.make_addplot(df['sam_50'], color='#FFa500', panel=1, title='sma_50', ax=ax2) ] mpf.plot(df,ax=ax3, addplot=ap0, volume=ax4, ) fig.subplots_adjust(hspace=0.2) ax1.tick_params(labelbottom=False) ax2.tick_params(labelbottom=False) ax3.tick_params(labelbottom=False) ax3.yaxis.set_label_position('left') ax3.yaxis.tick_left()   Another solution ap0 = [ mpf.make_addplot(df['sam_20'], color='#00FF00', panel=0, ylabel='sma_20'), mpf.make_addplot(df['sam_50'], color='#FFa500', panel=1, ylabel='sma_50') ]   @Daniel Goldfarlb solution ap0 = [ mpf.make_addplot(df['sam_20'], color='#00FF00', panel=0, title='sma_20', ylim=(110,140)), mpf.make_addplot(df['sam_50'], color='#FFa500', panel=1, title='sma_50') ] mpf.plot( df, panel_ratios=(2, 1, 3, 1), type="hollow_candle", yscale='log', volume=True, style='yahoo', figsize=(12.8, 10), addplot=ap0, main_panel=2, volume_panel=3, num_panels=4, )