eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Consider the example of a typed decorator bound to certain classes. import unittest from typing import * T = TypeVar("T", bound=unittest.TestCase) def decorate(func: Callable[[T], None]) -> Callable[[T], None]: def decorated_function(self: T) -> None: return func(self) return decorated_function  Now I even have a generator that creates these decorators and want to shorthand these decorators. What type do I to the variables variables storing the decorator (simplified example omitting the generator). my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate  This works, but is clunky. So the question is: How can I alias this type to avoid having to write the the full signature?  Things that don't work: TD = Callable[[Callable[[T], None]], Callable[[T], None]] my_decorate: TD[T] = decorator_variable  Gives the error error: Type variable "mypytest.T" is unbound note: (Hint: Use "Generic[T]" or "Protocol[T]" base class to bind "T" inside a class) note: (Hint: Use "T" in function signature to bind "T" inside a function)  In contrast, I can use TD[T] as argument type for a function. Just using my_decorate: TD = ... yields a --strict error error: Missing type parameters for generic type "TD"  And it no longer detects wrong applications of my_decorate.

Answer: As in many cases, when Callable is too limited use a Protocol instead: class TD(Protocol): """Type of any callable `(T -> None) -> (T -> None)` for all `T`""" def __call__(self, __original: Callable[[T], None]) -> Callable[[T], None]: ...  TD is not a generic type and thus does not need "filling in" a type variable. It can be used directly as an annotation: my_decorate: TD = decorate  Notably, TD.__call__ is still a generic callable even though TD is not generic. Its T is filled by the context of each call as desired.
Question: I have a df: date category subcategory order_id product_id branch 2021-05-04 A aa 10 5 web 2021-06-04 A dd 10 2 web 2021-05-06 B aa 18 3 shop 2021-07-06 A aa 50 10 web 2021-07-06 C cc 10 15 web 2021-07-05 A ff 101 30 shop 2021-10-04 D aa 100 15 shop  I am trying to answer a question which items categories and subcategories are bought together per branch type weekly. I am thinking of grouping the order_ids and aggregating the category & subcategory to a list like so: a = (df.set_index('date') .groupby(['order_id','branch']) .resample('W-MON', label = 'left') .agg({'category':list, 'subcategory':list}))  Which returns :  category subcategory order_id branch date [A, A, A] [aa, dd, aa] 10 web 2021-05-04 ... ... 18 shop ... 50 web 100 web 101 shop  I am trying to build a structure which would show the frequency of each variation of the categories and subcategories bought each week per branch, something similar to this: branch date 2021-05-04 2021-05-011 ... web category 3, [A, A, A] 2, [A, A] 2, [A, A, B, B] subcategory 5, [aa, dd, aa] 4, [dd, aa] 1, [dd] shop category 3, [A, A, A] 2, [A, A] 2, [A, A, B, B] subcategory 5, [aa, dd, aa] 4, [dd, aa] 1, [dd]  Where the number before the list denotes the number of times a certain combinations of categories and subcategories were bought in the same order. I am unsure how to achieve such a structure or a similar one that would show the weekly combination frequencies by branch. The order of the product_id in the order does not matter as the final basket is the same. So the goal is to see the frequency of categories, subcategories & product_ids bought in the same order weekly. So if 2 different orders have the same products, the aggregated result would show 2, [A,B] [aa, bb] [5, 2] where the lists hold category, subcategory & product_id combinations.

Answer: This is what you need: import pandas as pd import numpy as np from datetime import timedelta from datetime import datetime as dt # df=pd.read_excel('demo.xlsx') df['date']=pd.to_datetime(df['date']) df['date']=df['date'].dt.strftime('%Y-%m-%d') df['date']=pd.to_datetime(df['date']) df['year_week'] = df['date'].dt.strftime('%Y_%U') df['orderid_year_week']=df['order_id'].astype(str)+'_'+df['year_week'] df=df.sort_values(['category', 'subcategory','product_id'], ascending=[True, True,True]) a = (df.set_index('orderid_year_week') .groupby(['year_week','order_id'],sort=False) .agg({'category':list, 'subcategory':list,'product_id':list})).reset_index() a['category'] =a['category'].astype(str) a['subcategory'] =a['subcategory'].astype(str) a['product_id'] =a['product_id'].astype(str) df=pd.pivot_table(a,index=['year_week','category','subcategory','product_id'],values='product_id',aggfunc='count').reset_index() df.rename({'order_id':'count'},axis=1,inplace=True)  The output looks like this (I have added a few more entries on top of the sample that you provided):  Some things in your explanations are not crystal clear. But let me know if this fully answers your question.
Question: After the deprecation of my discovery url, I had to make some change on my code and now I get this error.  googleapiclient.errors.HttpError: <HttpError 400 when requesting https://mybusinessbusinessinformation.googleapis.com/v1/accounts/{*accountid*}/locations?filter=locationKey.placeId%3{*placeid*}&readMask=paths%3A+%22locations%28name%29%22%0A&alt=json returned "Request contains an invalid argument.". Details: "[{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'field': 'read_mask', 'description': 'Invalid field mask provided'}]}]">  I am trying to use this end point accounts.locations.list  I'm using :  python 3.8 google-api-python-client 2.29.0  My current code look likes : from google.protobuf.field_mask_pb2 import FieldMask googleAPI = GoogleAPI.auth_with_credentials(client_id=config.GMB_CLIENT_ID, client_secret=config.GMB_CLIENT_SECRET, client_refresh_token=config.GMB_REFRESH_TOKEN, api_name='mybusinessbusinessinformation', api_version='v1', discovery_service_url="https://mybusinessbusinessinformation.googleapis.com/$discovery/rest") field_mask = FieldMask(paths=["locations(name)"]) outputLocation = googleAPI.service.accounts().locations().list(parent="accounts/{*id*}", filter="locationKey.placeId=" + google_place_id, readMask=field_mask ).execute()  From the error, i tried a lot of fieldmask path and still don't know what they want. I've tried things like location.name, name, locations.name, locations.location.name and it did'nt work. I also try to pass the readMask params without use the FieldMask class with passing a string and same problem. So if someone know what is the format of the readMask they want it will be great for me ! . Can help: https://www.youtube.com/watch?v=T1FUDXRB7Ns https://developers.google.com/google-ads/api/docs/client-libs/python/field-masks

Answer: You have not set the readMask correctly. I have done a similar task in Java and Google returns the results. readMask is a String type, and what I am going to provide you in the following line includes all fields. You can omit anyone which does not serve you. I am also writing the request code in Java, maybe it can help you better to convert into Python. String readMask="storeCode,regularHours,name,languageCode,title,phoneNumbers,categories,storefrontAddress,websiteUri,regularHours,specialHours,serviceArea,labels,adWordsLocationExtensions,latlng,openInfo,metadata,profile,relationshipData,moreHours"; MyBusinessBusinessInformation.Accounts.Locations.List request= mybusinessaccountLocations.accounts().locations().list(accountName).setReadMask(readMask); ListLocationsResponse Response = request.execute(); List<Location>= Response.getLocations(); while (Response.getNextPageToken() != null) { locationsList.setPageToken(Response.getNextPageToken()); Response=request.execute(); locations.addAll(Response.getLocations()); }  -- about the question in the comment you have asked this is what I have for placeId:
Question: I have the following working code. I need to add a percentage column to monitor changes. I dont know much on how to do it in pandas. I need ideas on what part needs to be modified. import pandas as pd dl = [] with open('sampledata.txt') as f: for line in f: parts = line.split() # Cleaning data here.. Conversions to int/float etc, if not parts[3][:2].startswith('($'): parts.insert(3,'0') if len(parts) > 5: temp = ' '.join(parts[4:]) parts = parts[:4] + [temp] parts[1] = int(parts[1]) parts[2] = float(parts[2].replace(',', '')) parts[3] = float(parts[3].strip('($)')) dl.append(parts) headers = ['col1', 'col2', 'col3', 'col4', 'col5'] df = pd.DataFrame(dl,columns=headers) df = df.groupby(['col1','col5']).sum().reset_index() df = df.sort_values('col2',ascending=False) df['col4'] = '($' + df['col4'].astype(str) + ')' df = df[headers] print(df)  sampledata.txt #-- Sample Data Source file alpha 1 54,00.01 ABC DSW2S bravo 3 500,000.00 ACDEF charlie 1 27,722.29 ($250.45) DGAS-CAS delta 2 11 ($10) SWSDSASS-CCSSW echo 5 143,299.00 ($101) ACS34S1 lima 6 45.00181 ($38.9) FGF5GGD-DDD falcon 3 0.1234 DSS2SFS3 echo 8 145,300 ($125.01) ACS34S1 charlie 10 252,336,733.383 ($492.06) DGAS-CAS romeo 12 980 ASDS SSSS SDSD falcon 5 9.19 DSS2SFS3  Current Output: #-- working result  col1 col2 col3 col4 col5 4 echo 13 2.885990e+05 ($226.01) ACS34S1 7 romeo 12 9.800000e+02 ($0.0) ASDS SSSS SDSD 2 charlie 11 2.523645e+08 ($742.51) DGAS-CAS 5 falcon 8 9.313400e+00 ($0.0) DSS2SFS3 6 lima 6 4.500181e+01 ($38.9) FGF5GGD-DDD 1 bravo 3 5.000000e+05 ($0.0) ACDEF 3 delta 2 1.100000e+01 ($10.0) SWSDSASS-CCSSW 0 alpha 1 5.400010e+03 ($0.0) ABC DSW2S  Improved Output: #-- with Additional Column for %  col1 col2 col3 col4 col5 col6 4 echo 13 2.885990e+05 ($226.01) ACS34S1 60% #-- (5 + 8) = 13 7 romeo 12 9.800000e+02 ($0.0) ASDS SSSS SDSD 0% 2 charlie 11 2.523645e+08 ($742.51) DGAS-CAS 900% #-- (1 + 10) = 11 5 falcon 8 9.313400e+00 ($0.0) DSS2SFS3 66.67% #-- (3 + 5) = 8 6 lima 6 4.500181e+01 ($38.9) FGF5GGD-DDD 0% 1 bravo 3 5.000000e+05 ($0.0) ACDEF 0% 3 delta 2 1.100000e+01 ($10.0) SWSDSASS-CCSSW 0% 0 alpha 1 5.400010e+03 ($0.0) ABC DSW2S 0% 

Answer: You can add the following lines just after your code: The function compute_percentage() is using the list variable dl. def compute_percentage(row): vl = [float(parts[1]) for parts in dl if parts[0] == row['col1']] i = round(100. * (vl[-1]-vl[0])/vl[0] if vl[0] != 0 else 0, 2) if float(int(i)) == i: i = int(i) return str(i) + '%' df['col6'] = df.apply(compute_percentage, axis=1)  Output:  col1 col2 col3 col4 col5 col6 4 echo 13 2.885990e+05 ($226.01) ACS34S1 60% 7 romeo 12 9.800000e+02 ($0.0) ASDS SSSS SDSD 0% 2 charlie 11 2.523645e+08 ($742.51) DGAS-CAS 900% 5 falcon 8 9.313400e+00 ($0.0) DSS2SFS3 66.67% 6 lima 6 4.500181e+01 ($38.9) FGF5GGD-DDD 0% 1 bravo 3 5.000000e+05 ($0.0) ACDEF 0% 3 delta 2 1.100000e+01 ($10.0) SWSDSASS-CCSSW 0% 0 alpha 1 5.400010e+03 ($0.0) ABC DSW2S 0%
Question: I am trying to run this github repo found at this link: https://github.com/HowieMa/DeepSORT_YOLOv5_Pytorch After installing the requirements via pip install -r requirements.txt. I am running this in a python 3.8 virtual environment, on a dji manifold 2g which runs on an Nvidia jetson tx2. The following is the terminal output. $ python main.py --cam 0 --display Namespace(agnostic_nms=False, augment=False, cam=0, classes=[0], conf_thres=0.5, config_deepsort='./configs/deep_sort.yaml', device='', display=True, display_height=600, display_width=800, fourcc='mp4v', frame_interval=2, img_size=640, input_path='input_480.mp4', iou_thres=0.5, save_path='output/', save_txt='output/predict/', weights='yolov5/weights/yolov5s.pt') Initialize DeepSORT & YOLO-V5 Using CPU Using webcam 0 Traceback (most recent call last): File "main.py", line 259, in <module> with VideoTracker(args) as vdo_trk: File "main.py", line 53, in __init__ cfg.merge_from_file(args.config_deepsort) File "/home/dji/Desktop/targetTrackers/howieMa/DeepSORT_YOLOv5_Pytorch/utils_ds/parser.py", line 23, in merge_from_file self.update(yaml.load(fo.read())) TypeError: load() missing 1 required positional argument: 'Loader'  I have found some suggestions on github, such as in here TypeError: load() missing 1 required positional argument: 'Loader' in Google Colab, which suggests to change yaml.load to yaml.safe_load This is the code block to modify:  class YamlParser(edict): """ This is yaml parser based on EasyDict. """ def __init__(self, cfg_dict=None, config_file=None): if cfg_dict is None: cfg_dict = {} if config_file is not None: assert(os.path.isfile(config_file)) with open(config_file, 'r') as fo: cfg_dict.update(yaml.load(fo.read())) super(YamlParser, self).__init__(cfg_dict) def merge_from_file(self, config_file): with open(config_file, 'r') as fo: self.update(yaml.load(fo.read())) def merge_from_dict(self, config_dict): self.update(config_dict)  However, changing yaml.load into yaml.safe_load leads me to this error instead  $ python main.py --cam 0 --display Namespace(agnostic_nms=False, augment=False, cam=0, classes=[0], conf_thres=0.5, config_deepsort='./configs/deep_sort.yaml', device='', display=True, display_height=600, display_width=800, fourcc='mp4v', frame_interval=2, img_size=640, input_path='input_480.mp4', iou_thres=0.5, save_path='output/', save_txt='output/predict/', weights='yolov5/weights/yolov5s.pt') Initialize DeepSORT & YOLO-V5 Using CPU Using webcam 0 Done.. Camera ... Done. Create output file output/results.mp4 Illegal instruction (core dumped)  Has anyone encountered anything similar ? Thank you !

Answer: Try this: yaml.load(fo.read(), Loader=yaml.FullLoader)  It seems that pyyaml>=5.1 requires a Loader argument.
Question: I want to deconstruct a pandas DataFrame, using column headers as a new data-column and create a list with all combinations of the row index and columns. Easier to show than explain: index_col = ["store1", "store2", "store3"] cols = ["January", "February", "March"] values = [[2,3,4],[5,6,7],[8,9,10]] df = pd.DataFrame(values, index=index_col, columns=cols)  From this DataFrame I wish to get the following list: [['store1', 'January', 2], ['store1', 'February', 3], ['store1', 'March', 4], ['store2', 'January', 5], ['store2', 'February', 6], ['store2', 'March', 7], ['store3', 'January', 8], ['store3', 'February', 9], ['store3', 'March', 10]]  Is there a convenient way to do this?

Answer: df.unstack().swaplevel().reset_index().values.tolist() #OR df.reset_index().melt(id_vars="index").values.tolist() # [['store1', 'January', 2], # ['store2', 'January', 5], # ['store3', 'January', 8], # ['store1', 'February', 3], # ['store2', 'February', 6], # ['store3', 'February', 9], # ['store1', 'March', 4], # ['store2', 'March', 7], # ['store3', 'March', 10]]  With following, the order of elements will match the output in the question. df.transpose().unstack().reset_index().values.tolist() # [['store1', 'January', 2], # ['store1', 'February', 3], # ['store1', 'March', 4], # ['store2', 'January', 5], # ['store2', 'February', 6], # ['store2', 'March', 7], # ['store3', 'January', 8], # ['store3', 'February', 9], # ['store3', 'March', 10]]
Question: I am training a model in pytorch for which I have made a class like so: from torch import nn class myNN(nn.Module): def __init__(self, dense1=128, dense2=64, dense3=32, ...): self.MLP = nn.Sequential( nn.Linear(dense1, dense2), nn.ReLU(), nn.Linear(dense2, dense3), nn.ReLU(), nn.Linear(dense3, 1) ) ...  In order to save it I am using: torch.save(model.state_dict(), checkpoint_model_path)  and to load it I am using: model = myNN() # or with specified parameters model.load_state_dict(torch.load(model_file))  However, in order for this method to work I have to use the right values in myNN()'s constructor. That means that I would need to somehow remember or store which parameters (layer sizes) I have used in each case in order to properly load different models. Is there a flexible way to save/load models in pytorch where I would also read the size of the layers? E.g. by loading a myNN() object directly or somehow reading the layer sizes from the saved pickle file? I am hesitant to try the second method in Best way to save a trained model in PyTorch? due to the warnings mentioned there. Is there a better way to achieve what I want?

Answer: Indeed serializing the whole Python is quite a drastic move. Instead, you can always add user-defined items in the saved file: you can save the model's state along with its class parameters. Something like this would work:  First save your arguments in the instance such that we can serialize them when saving the model: class myNN(nn.Module): def __init__(self, dense1=128, dense2=64, dense3=32): super().__init__() self.kwargs = {'dense1': dense1, 'dense2': dense2, 'dense3': dense3} self.MLP = nn.Sequential( nn.Linear(dense1, dense2), nn.ReLU(), nn.Linear(dense2, dense3), nn.ReLU(), nn.Linear(dense3, 1))   We can save the parameters of the model along with its initializer arguments: >>> torch.save([model.kwargs, model.state_dict()], path)   Then load it: >>> kwargs, state = torch.load(path) >>> model = myNN(**kwargs) >>> model.load_state_dict(state) <All keys matched successfully>
Question: I was looking to visualise a categorical variable as marker style in seaborn stripplot, but it does not seem to be possible easily. Is there an easy way to do this. For example, I'm trying to run this code tips = sns.load_dataset("tips") sns.stripplot(x="day", y="total_bill", hue="time", style="sex", jitter=True, data=tips)  which fails. An alternative is to use relplot which does provide the option but has no way to insert jitter which makes the visualisation less nice. sns.relplot(x="day", y="total_bill", hue="time", data=tips, style="sex")  works providing this  Is there any way of doing this using stripplot/catplot/swarmplot? EDIT: This question is related. However the solution there does not seem to allow generation of a legend for size (and is quite dated).

Answer: sns.relplot is a figure-level function which relies on the axes-level function sns.scatterplot. sns.scatterplot has a parameter x_jitter which unfortunately currently has no effect (seaborn 0.11.2). You can mimic the functionality by grasping the positions of the points, add some random jitter and assigning these positions again. Here is an example: from matplotlib import pyplot as plt import seaborn as sns import numpy as np tips = sns.load_dataset("tips") ax = sns.scatterplot(x="day", y="total_bill", hue="time", data=tips, style="sex") for points in ax.collections: vertices = points.get_offsets().data if len(vertices) > 0: vertices[:, 0] += np.random.uniform(-0.3, 0.3, vertices.shape[0]) points.set_offsets(vertices) xticks = ax.get_xticks() ax.set_xlim(xticks[0] - 0.5, xticks[-1] + 0.5) # the limits need to be moved to show all the jittered dots sns.move_legend(ax, bbox_to_anchor=(1.01, 1.02), loc='upper left') # needs seaborn 0.11.2 sns.despine() plt.tight_layout() plt.show()   With sns.relplot you could iterate through all the subplots: g = sns.relplot(x="day", y="total_bill", hue="time", data=tips, style="sex") for ax in g.axes.flat: for points in ax.collections: vertices = points.get_offsets().data if len(vertices) > 0: vertices[:, 0] += np.random.uniform(-0.3, 0.3, vertices.shape[0]) points.set_offsets(vertices) xticks = ax.get_xticks() ax.set_xlim(xticks[0] - 0.5, xticks[-1] + 0.5) # the limits need to be moved to show all the jittered dots plt.show()
Question: Hi everyone I have the following dataframe and I want to create heatmap from it with the following code.  plt.figure(figsize=(10,10)) g = sns.heatmap( top_5_stations_hourly_total_traffic_by_time, square=True, cbar_kws={'fraction' : 0.01}, cmap='OrRd', linewidth=1 ) g.set_xticklabels(top_5_stations_hourly_total_traffic_by_time.STATION, rotation=45, horizontalalignment='right') g.set_yticklabels(top_5_stations_hourly_total_traffic_by_time.TIME, rotation=45, horizontalalignment='right')  But it gives the following error :    ValueError: could not convert string to float: '125 ST'   

Answer: pivot your data before calling heatmap: df_heatmap = df.pivot("STATION", "TIME", "HOURLY_TOTAL_TRAFFIC") >>> sns.heatmap(df_heatmap)
Question: I'd like to update more than one objects at same time, when the register date achieve more than 6 days: The Idea is update all issue_status from 'On Going' to 'Pending' for each objects Is it necessary iterate it? Below is my current code and error: models.py class MaintenanceIssue(models.Model): issue_status = models.CharField(max_length=30, choices=[('pending', 'Pending'), ('on going', 'On going'), ('done', 'Done')]) register_dt = models.DateTimeField(blank=True, null=True) @property def pending_issue(self): issue_time_diff = (datetime.now() - self.register_dt).days return issue_time_diff  views.py: on_going_issues = MaintenanceIssue.objects.get(issue_status='On Going') if on_going_issues.pending_issue > 6: on_going_issues.issue_status = 'Pending' on_going_issues.save()   get() returned more than one MaintenanceIssue -- it returned 61! 

Answer: at: on_going_issues = MaintenanceIssue.objects.get(issue_status='On Going') if on_going_issues.pending_issue > 6: on_going_issues.issue_status = 'Pending' on_going_issues.save()  should filter by the field and then loop through each on_going_issues = MaintenanceIssue.objects.filter(issue_status='On Going') for one in on_going_issues: if one.pending_issue > 6: one.issue_status = "Pending" one.save()
Question: Right to the point, here below is the sample code which will raise error in PyCharm: list1 = [0] * 5 list1[0] = '' list2 = [0 for n in range(5)] list2[0] = ''  PyCharm then return an error on both line 2 and line 4 as below: Unexpected type(s):(int, str)Possible type(s):(SupportsIndex, int)(slice, Iterable[int])  Running the code would not cause any error, but PyCharm keep raising the above message when I am coding. Why PyCharm would give this error and how can I solve this with cleanest code?

Answer: In your case PyCharm sees you first line and thinks that the type of list is List[int]. I mean it is a list of integers. You may tell that you list is not int-typed and can accept any value this way: from typing import Any, List list1: List[Any] = [0] * 5 list1[0] = ''  I used typing module just to explain the idea. Here is a simple way to announce that list1 is just a list of anything: list1: list = [0] * 5 list1[0] = ''  Also consider to use as strict typing as possible. It may help you to prevent bugs. If you need both strings and ints then use this: from typing import List, Union list1: List[Union[int, str]] = [0] * 5 # Starting from Python 3.10 you may use List[int|str] instead  Docs here: https://docs.python.org/3/library/typing.html
Question: Currently I'm using command plt.errorbar(X,Y,yerr=myYerr, fmt="o", alpha=0.5,capsize=4)  and I get default marker colours:  But what should I do to force matplotlib to be more colorblind-friendly?

Answer: According to this [1] you can use the predefined colorblind style. It should be as simple as: import matplotlib.pyplot as plt plt.style.use('tableau-colorblind10')  [1] https://matplotlib.org/stable/users/prev_whats_new/whats_new_2.2.html#new-style-colorblind-friendly-color-cycle Check out the last image from this link as well: https://matplotlib.org/stable/gallery/style_sheets/style_sheets_reference.html
Question: While we are trying to execute with python 3.6.8 version getting below module error from wtforms.compat import string_types, text_type ModuleNotFoundError: No module named 'wtforms.compat'  when i tried installing or upgrading wtforms still it shows the same error Can any one pls suggest

Answer: Noticed this error today while running our Airflow 1.10.12 builds: from wtforms.compat import text_type ModuleNotFoundError: No module named 'wtforms.compat'  Apparently, the issue has to do with the latest version of wtforms released yesterday (3.0.0). We managed to get around it by pinning it to the previous version: wtforms==2.3.3. Editing just to add more information: compat.py was completely removed once support for Python < 3.6 was dropped (see PR). If you are running Python >= 3.6, you can also work with the latest wtforms by simply using str instead of text_type and string_types, since those were just aliases: if sys.version_info[0] >= 3: text_type = str string_types = (str,) izip = zip  And the imports should not be needed anymore. If running Python < 3.6, you may need to stick with wtforms<=2.3.3.