eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I've got a pytest test that's parametrized with a @pytest.mark.parametrize decorator using a custom function load_test_cases() that loads the test cases from a yaml file. class SelectTestCase: def __init__(self, test_case): self.select = test_case['select'] self.expect = test_case['expect'] def __str__(self): # also tried __repr__() # Attempt to print the 'select' attribute in "pytest -v" output return self.select def load_test_cases(path): with open(path, "rt") as f: test_cases = yaml.safe_load_all(f) return [ SelectTestCase(test_case) for test_case in test_cases ] @pytest.mark.parametrize("test_case", load_test_cases("tests/select-test-cases.yaml")) def test_select_prefixes(test_case): # .. run the test  It works well except that the tests when run with pytest -v are displayed with test_case0, test_case1, etc parameters. tests/test_resolver.py::test_select_prefixes[test_case0] PASSED [ 40%] tests/test_resolver.py::test_select_prefixes[test_case1] PASSED [ 60%] tests/test_resolver.py::test_select_prefixes[test_case2] PASSED [ 80%] tests/test_resolver.py::test_select_prefixes[test_case3] PASSED [100%]  I would love to see the select attribute displayed instead, e.g. tests/test_resolver.py::test_select_prefixes["some query"] PASSED [ 40%] tests/test_resolver.py::test_select_prefixes["another query"] PASSED [ 60%]  I tried to add __str__() and __repr__() methods to the SelectTestCase class but it didn't make any difference. Any idea how to do it?

Answer: You can define how your parametrized test names look using the ids parameter. This can be a list of strings, or a function that takes the current parameter as argument and returns the ID to be shown in the test name. So, in your case it is sufficient to use str as that function, as you have already implemented __str__ for the parameters (which are of type SelectTestCase). If you just write: @pytest.mark.parametrize("test_case", load_test_cases("tests/select-test-cases.yaml"), ids=str) def test_select_prefixes(test_case): # .. run the test  you will get the desired behavior, e.g. tests/test_resolver.py::test_select_prefixes[some query] PASSED [ 40%] tests/test_resolver.py::test_select_prefixes[another query] PASSED [ 60%]  apart from the apostrophes (which you can add by adapting __str__ accordingly).
Question: While adding loss in Pytorch, I have the same function in torch.nn.Functional as well as in torch.nn. what is the difference ?  torch.nn.CrossEntropyLoss() and torch.nn.functional.cross_entropy 

Answer: Putting same text from PyTorch discussion forum @Alban D has given answer to similar question. F.cross entropy vs torch.nn.Cross_Entropy_Loss  There isn’t much difference for losses. The main difference between the nn.functional.xxx and the nn.Xxx is that one has a state and one does not. This means that for a linear layer for example, if you use the functional version, you will need to handle the weights yourself (including passing them to the optimizer or moving them to the gpu) while the nn.Xxx version will do all of that for you with .parameters() or .to(device). For loss functions, as no parameters are needed (in general), you won’t find much difference. Except for example, if you use cross entropy with some weighting between your classes, using the nn.CrossEntropyLoss() module, you will give your weights only once while creating the module and then use it. If you were using the functional version, you will need to pass the weights every single time you will use it.
Question: I have managed to extract images from several PDF pages with the below code, but the resolution is quite low. Is there a way to adjust that? import fitz pdffile = "C:\\Users\\me\\Desktop\\myfile.pdf" doc = fitz.open(pdffile) for page_index in range(doc.pageCount): page = doc.loadPage(page_index) pix = page.getPixmap() output = "image_page_" + str(page_index) + ".jpg" pix.writePNG(output)  I have also tried using the code here and updated if pix.n < 5" to "if pix.n - pix.alpha < 4 but this didn't output any images in my case.

Answer: As stated in this issue for PyMuPDF, you have to use a matrix: issue on Github. The example given is: zoom = 2 # zoom factor mat = fitz.Matrix(zoom, zoom) pix = page.getPixmap(matrix = mat, <...>)  Indicated in the issue is also that the default resolution is 72 dpi if you don't use a matrix which likely explains your getting low resolution.
Question: Some hooks can take a while to run, and I would like to run those before I push, but not before each particular commit (for example, pylint can be a bit slow). I've seen the following:  Question: Using hooks at different stages mesos-commits mailing list archives Feature request: pre-commit or pre-push only hooks  But it's still not clear to be how I'm supposed to set this up. Here is what I have tried: default_stages: [commit] repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v3.1.0 hooks: - id: end-of-file-fixer - id: trailing-whitespace - repo: https://github.com/psf/black rev: 19.10b0 hooks: - id: black stages: [push]  From that I'm expecting the first couple of hooks to run before a commit (which they do), but I'm expecting black to run before pushing, which it doesn't. To test that I have created the following file: """This is a docstring.""" print('this should be formatted')  Which is certainly not being formatted by black.

Answer: your configuration is correct, except that the whitespace hooks in pre-commit/pre-commit-hooks set stages themselves so they won't be affected by default_stages adjusting your configuration slightly: repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v3.1.0 hooks: - id: end-of-file-fixer stages: [commit] - id: trailing-whitespace stages: [commit] - repo: https://github.com/psf/black rev: 19.10b0 hooks: - id: black stages: [push]  next you'll need to make sure both of the hook scripts are installed You can install both the pre-commit and pre-push commit at the same time using: pre-commit install --hook-type pre-commit --hook-type pre-push  or you can run them separately: pre-commit install # installs .git/hooks/pre-commit pre-commit install --hook-type pre-push # installs .git/hooks/pre-push  note that the second command comes directly from the documentation on using pre-push  disclaimer: I'm the author of pre-commit and pre-commit-hooks
Question: I used namedtuples for immutable data structures until I came across dataclasses, which I prefer in my use-cases (not relevant to the question). Now I learned that they are not immutable! At least not strictly speaking. setattr(frozen_dc_obj, "prop", "value") raises an exception. ok. But why does object.__setattr__(frozen_dc_obj,..) work? Compared to namedtuple, where it raises an exception! from collections import namedtuple from dataclasses import dataclass NTTest = namedtuple("NTTest", "id") nttest = NTTest(1) setattr(nttest, "id", 2) # Exception object.__setattr__(nttest, "id", 2) # Exception @dataclass(frozen=True) class DCTest: id: int dctest = DCTest(1) setattr(dctest, "id", 2) # Exception object.__setattr__(dctest, "id", 2) # WORKS 

Answer: namedtuple defines __slots__ = () and hence you can't set any attribute (it doesn't have a __dict__). Frozen dataclasses on the other hand perform a manual check in their __setattr__ method and raise an exception if it's a frozen instance. Compare the following: >>> class Foo: ... __slots__ = () ... >>> f = Foo() >>> f.__dict__ # doesn't exist, so object.__setattr__ won't work Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'Foo' object has no attribute '__dict__' >>> @dataclass(frozen=True) ... class Bar: ... pass ... >>> b = Bar() >>> b.__dict__ # this exists, so object.__setattr__ works {}
Question: As we know, optional arguments must be at the end of the arguments list, like below: def func(arg1, arg2, ..., argN=default) I saw some exceptions in the PyTorch package. For example, we can find this issue in torch.randint. As it is shown, it has a leading optional argument in its positional arguments! How could be possible? Docstring: randint(low=0, high, size, \*, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor  How can we define a function in a similar way as above?

Answer: A single function is not allowed to have only leading optional parameters:  8.6. Function definitions [...] If a parameter has a default value, all following parameters up until the “*” must also have a default value — this is a syntactic restriction that is not expressed by the grammar.  Note this excludes keyword-only parameters, which never receive arguments by position.  If desired, one can emulate such behaviour by manually implementing the argument to parameter matching. For example, one can dispatch based on arity, or explicitly match variadic arguments. def leading_default(*args): # match arguments to "parameters" *_, low, high, size = 0, *args print(low, high, size) leading_default(1, 2) # 0, 1, 2 leading_default(1, 2, 3) # 1, 2, 3  A simple form of dispatch achieves function overloading by iterating signatures and calling the first matching one. import inspect class MatchOverload: """Overload a function via explicitly matching arguments to parameters on call""" def __init__(self, base_case=None): self.cases = [base_case] if base_case is not None else [] def overload(self, call): self.cases.append(call) return self def __call__(self, *args, **kwargs): failures = [] for call in self.cases: try: inspect.signature(call).bind(*args, **kwargs) except TypeError as err: failures.append(str(err)) else: return call(*args, **kwargs) raise TypeError(', '.join(failures)) @MatchOverload def func(high, size): print('two', 0, high, size) @func.overload def func(low, high, size): print('three', low, high, size) func(1, 2, size=3) # three 1 2 3 func(1, 2) # two 0 1 2 func(1, 2, 3, low=4) # TypeError: too many positional arguments, multiple values for argument 'low'
Question: I am using a Mac and programming with Python on VS Code. After installing pylint, I had a bunch of warnings and messages. How do I disable these? I know about adding some lines to the pylintrc file, but I don't know where to find it or how to create it on a Mac.

Answer: Fully disable the linting Here is a link that explain how to do it : Disable Linting on VsCode. To do so, type Command + Shift + P (or Ctrl + Shift + P on PC) in VsCode. This will open a command prompt at the top of the window. Then type the command Python: Enable Linting, and select off. Another option is to choose no linter. To do so, open the command prompt with Command + Shift + P (or Ctrl + Shift + P on PC), type Python: Select Linter, and choose the option Disable Linting.  Disable warnings, but keep errors : If you want to keep the errors, but disable only the warnings, you can also configure pylint directly from VsCode. Go to the menu File -> Preferences -> Settings (Or open directly with Command + , or Ctrl + ,). Then in the search box at the top of the window, search for pylint Args. Click on the button Add item and add the line --disable=W.
Question: I made a plot that looks like this  I want to turn off the ticklabels along the y axis. And to do that I am using plt.tick_params(labelleft=False, left=False)  And now the plot looks like this. Even though the labels are turned off the scale 1e67 still remains.  Turning off the scale 1e67 would make the plot look better. How do I do that?

Answer:  seaborn is used to draw the plot, but it's just a high-level API for matplotlib.  The functions called to remove the y-axis labels and ticks are matplotlib methods.   After creating the plot, use .set(). .set(yticklabels=[]) should remove tick labels.  This doesn't work if you use .set_title(), but you can use .set(title='') Do not use sns.boxplot(...).set(xticklabels=[]) because, while this works, the object type is changed from matplotlib.axes._axes.Axes for sns.boxplot(...), to list.   .set(ylabel=None) should remove the axis label. .tick_params(left=False) will remove the ticks. Similarly, for the x-axis: How to remove or hide x-axis labels from a plot Tested in python 3.11, pandas 1.5.2, matplotlib 3.6.2, seaborn 0.12.1  Example 1 import seaborn as sns import matplotlib.pyplot as plt # load data exercise = sns.load_dataset('exercise') pen = sns.load_dataset('penguins') # create figures fig, ax = plt.subplots(2, 1, figsize=(8, 8)) # plot data g1 = sns.boxplot(x='time', y='pulse', hue='kind', data=exercise, ax=ax[0]) g2 = sns.boxplot(x='species', y='body_mass_g', hue='sex', data=pen, ax=ax[1]) plt.show()   Remove Labels fig, ax = plt.subplots(2, 1, figsize=(8, 8)) g1 = sns.boxplot(x='time', y='pulse', hue='kind', data=exercise, ax=ax[0]) g1.set(yticklabels=[]) # remove the tick labels g1.set(title='Exercise: Pulse by Time for Exercise Type') # add a title g1.set(ylabel=None) # remove the axis label g2 = sns.boxplot(x='species', y='body_mass_g', hue='sex', data=pen, ax=ax[1]) g2.set(yticklabels=[]) g2.set(title='Penguins: Body Mass by Species for Gender') g2.set(ylabel=None) # remove the y-axis label g2.tick_params(left=False) # remove the ticks plt.tight_layout() plt.show()   Example 2 import numpy as np import matplotlib.pyplot as plt import pandas as pd # sinusoidal sample data sample_length = range(1, 1+1) # number of columns of frequencies rads = np.arange(0, 2*np.pi, 0.01) data = np.array([(np.cos(t*rads)*10**67) + 3*10**67 for t in sample_length]) df = pd.DataFrame(data.T, index=pd.Series(rads.tolist(), name='radians'), columns=[f'freq: {i}x' for i in sample_length]) df.reset_index(inplace=True) # plot fig, ax = plt.subplots(figsize=(8, 8)) ax.plot('radians', 'freq: 1x', data=df) # or skip the previous two lines and plot df directly # ax = df.plot(x='radians', y='freq: 1x', figsize=(8, 8), legend=False)   Remove Labels # plot fig, ax = plt.subplots(figsize=(8, 8)) ax.plot('radians', 'freq: 1x', data=df) # or skip the previous two lines and plot df directly # ax = df.plot(x='radians', y='freq: 1x', figsize=(8, 8), legend=False) ax.set(yticklabels=[]) # remove the tick labels ax.tick_params(left=False) # remove the ticks
Question: I want to create a Pydantic model in which there is a list field, which left uninitialized has a default value of an empty list. Is there an idiomatic way to do this? For Python's built-in dataclass objects you can use field(default_factory=list), however in my own experiments this seems to prevent my Pydantic models from being pickled. A naive implementation might be, something like this: from pydantic import BaseModel class Foo(BaseModel): defaulted_list_field: Sequence[str] = [] # Bad!  But we all know not to use a mutable value like the empty-list literal as a default. So what's the correct way to give a Pydantic list-field a default value?

Answer: For pydantic you can use mutable default value, like: class Foo(BaseModel): defaulted_list_field: List[str] = [] f1, f2 = Foo(), Foo() f1.defaulted_list_field.append("hey!") print(f1) # defaulted_list_field=['hey!'] print(f2) # defaulted_list_field=[]  It will be handled correctly (deep copy) and each model instance will have its own empty list. See "Fields with non-hashable default values" from the documentation.  Pydantic also has default_factory parameter. In the case of an empty list, the result will be identical, it is rather used when declaring a field with a default value, you may want it to be dynamic (i.e. different for each model). from typing import List from pydantic import BaseModel, Field from uuid import UUID, uuid4 class Foo(BaseModel): defaulted_list_field: List[str] = Field(default_factory=list) uid: UUID = Field(default_factory=uuid4)
Question: I'm using SQLAlchemy to manage a database and I'm trying to delete all rows that contain duplicates. The table has an id (primary key) and domain name. Example: ID| Domain 1 | example-1.com 2 | example-2.com 3 | example-1.com In this case I want to delete 1 instance of example-1.com. Sometimes I will need to delete more than 1 but in general the database should not have a domain more than once and if it does, only the first row should be kept and the others should be deleted.

Answer: Assuming your model looks something like this: import sqlalchemy as sa from sqlalchemy import orm Base = orm.declarative_base() class Domain(Base): __tablename__ = 'domain_names' id = sa.Column(sa.Integer, primary_key=True) domain = sa.Column(sa.String)  Then you can delete the duplicates like this: # Create a query that identifies the row for each domain with the lowest id inner_q = session.query(sa.func.min(Domain.id)).group_by(Domain.domain) aliased = sa.alias(inner_q) # Select the rows that do not match the subquery q = session.query(Domain).filter(~Domain.id.in_(aliased)) # Delete the unmatched rows (SQLAlchemy generates a single DELETE statement from this loop) for domain in q: session.delete(domain) session.commit() # Show remaining rows for domain in session.query(Domain): print(domain) print()  If you are not using the ORM, the core equivalent is: meta = sa.MetaData() domains = sa.Table('domain_names', meta, autoload=True, autoload_with=engine) inner_q = sa.select([sa.func.min(domains.c.id)]).group_by(domains.c.domain) aliased = sa.alias(inner_q) with engine.connect() as conn: conn.execute(domains.delete().where(~domains.c.id.in_(aliased)))  This answer is based on the SQL provided in this answer. There are other ways of deleting duplicates, which you can see in the other answers on the link, or by googling "sql delete duplicates" or similar.
Question: I want to format y-axis labels in a seaborn FacetGrid plot, with a number of decimals, and/or with some text added. import seaborn as sns import matplotlib.pyplot as plt sns.set(style="ticks") exercise = sns.load_dataset("exercise") g = sns.catplot(x="time", y="pulse", hue="kind", col="diet", data=exercise) #g.xaxis.set_major_formatter(ticker.FuncFormatter(lambda x, pos: '{:,.2f}'.format(x) + 'K')) #g.set(xticks=['a','try',0.5]) g.yaxis.set_major_formatter(ticker.FuncFormatter(lambda x, pos: '{:,.2f}'.format(x) + 'K')) plt.show()  Inspired from How to format seaborn/matplotlib axis tick labels from number to thousands or Millions? (125,436 to 125.4K) ax.xaxis.set_major_formatter(ticker.FuncFormatter(lambda x, pos: '{:,.2f}'.format(x) + 'K'))  It results in the following error. AttributeError: 'FacetGrid' object has no attribute 'xaxis'

Answer:  xaxis and yaxis are attributes of the plot axes, for a seaborn.axisgrid.FacetGrid type.  In the linked answer, the type is matplotlib.axes._subplots.AxesSubplot   p in the lambda expression is the tick label number. seaborn: Building structured multi-plot grids matplotlib: Creating multiple subplots Tested and working with the following versions:  matplotlib v3.3.4 seaborn v0.11.1    import pandas as pd import seaborn as sns import matplotlib.pyplot as plt import matplotlib.ticker as tkr sns.set(style="ticks") # load data exercise = sns.load_dataset("exercise") # plot data g = sns.catplot(x="time", y="pulse", hue="kind", col="diet", data=exercise) # format the labels with f-strings for ax in g.axes.flat: ax.yaxis.set_major_formatter(tkr.FuncFormatter(lambda y, p: f'{y:.2f}: Oh baby, baby')) ax.xaxis.set_major_formatter(tkr.FuncFormatter(lambda x, p: f'{x}: Is that your best'))    As noted in a comment by Patrick FitzGerald, the following code, without using tkr.FuncFormatter, also works to generate the previous plot. See matplotlib.axis.Axis.set_major_formatter  # format the labels with f-strings for ax in g.axes.flat: ax.yaxis.set_major_formatter(lambda y, p: f'{y:.2f}: Oh baby, baby') ax.xaxis.set_major_formatter(lambda x, p: f'{x}: Is that your best')
Question: I am trying to plot boxplots in seaborn whose widths depend upon the log of the value of x-axis. I am creating the list of widths and passing it to the widths=widths parameter of seaborn.boxplot. However, I am getting that raise ValueError(datashape_message.format("widths")) ValueError: List of boxplot statistics and `widths` values must have same the length  When I debugged and checked there is just one dict in boxplot statistics, whereas I have 8 boxplots. Cannot Exactly figure out where the problem lies.  I am using pandas data frame and seaborn for plotting.

Answer: Seaborn's boxplot doesn't seem to understand the widths= parameter. Here is a way to create a boxplot per x value via matplotlib's boxplot which does accept the width= parameter. The code below supposes the data is organized in a panda's dataframe. from matplotlib import pyplot as plt import numpy as np import pandas as pd import seaborn as sns df = pd.DataFrame({'x': np.random.choice([1, 3, 5, 8, 10, 30, 50, 100], 500), 'y': np.random.normal(750, 20, 500)}) xvals = np.unique(df.x) positions = range(len(xvals)) plt.boxplot([df[df.x == xi].y for xi in xvals], positions=positions, showfliers=False, boxprops={'facecolor': 'none'}, medianprops={'color': 'black'}, patch_artist=True, widths=[0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]) means = [np.mean(df[df.x == xi].y) for xi in xvals] plt.plot(positions, means, '--k*', lw=2) # plt.xticks(positions, xvals) # not needed anymore, as the xticks are set by the swarmplot sns.swarmplot('x', 'y', data=df) plt.show()   A related question asked how to set the box's widths depending on group size. The widths can be calculated as some maximum width multiplied by each group's size compared to the size of the largest group. from matplotlib import pyplot as plt import numpy as np import pandas as pd import seaborn as sns y_true = np.random.normal(size=100) y_pred = y_true + np.random.normal(size=100) df = pd.DataFrame({'y_true': y_true, 'y_pred': y_pred}) df['y_true_bin'] = pd.cut(df['y_true'], range(-3, 4)) sns.set() fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5)) sns.boxplot(x='y_true_bin', y='y_pred', data=df, color='lightblue', ax=ax1) bins, groups = zip(*df.groupby('y_true_bin')['y_pred']) lengths = np.array([len(group) for group in groups]) max_width = 0.8 ax2.boxplot(groups, widths=max_width * lengths / lengths.max(), patch_artist=True, boxprops={'facecolor': 'lightblue'}) ax2.set_xticklabels(bins) ax2.set_xlabel('y_true_bin') ax2.set_ylabel('y_pred') plt.tight_layout() plt.show()
Question: I currently have a program that generates a .docx document using the python-docx library. Upon completing the building of the .docx file I save it into a Bytestream as so file_stream = io.BytesIO() document.save(file_stream) file_stream.seek(0)  Now, I need to convert this word document into a PDF. I have looked at a few different libraries for conversion such as docx2pdf or even doing it manually using comtypes as so import sys import os import comtypes.client wdFormatPDF = 17 in_file = "Input_file_path.docx" out_file = "output_file_path.pdf" word = comtypes.client.CreateObject('Word.Application') doc = word.Documents.Open(in_file) doc.SaveAs(out_file, FileFormat=wdFormatPDF) doc.Close() word.Quit()  The problem is, I need to do this conversion in memory and cannot physically save the DOCX or the PDF to the machine. Every converter I've seen requires a filepath to the physical document on the machine and I do not have that. Is there a way I can convert the DOCX filestream into a PDF stream just in memory? Thanks

Answer: This method is a little convoluted, but it works entirely in memory, and you get the option to add custom CSS to style the final document. Convert the DOCX bytestream to HTML using mammoth, and the resulting HTML to PDF using pdfkit. Here's an example # create a dummy docx file from docx import Document document = Document() document.add_paragraph('Lorem ipsum dolor sit amet.') # create a bytestream import io file_stream = io.BytesIO() document.save(file_stream) file_stream.seek(0) # convert the docx to html import mammoth result = mammoth.convert_to_html(file_stream) # >>> result.value # >>> '<p>Lorem ipsum dolor sit amet.</p>' # convert html to pdf import pdfkit pdf = pdfkit.from_string(result.value)  If you want to output the stream to a file, just do with open('test.pdf','wb') as file: file.write(pdf)