eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm working on creating assignments for students using Jupyter Notebooks. My goal is to generate different versions of the same Notebook to distribute to tutors and students. I need certain cells, such as those containing exercise solutions, to be included only in the tutor's version, while other cells should be exclusive to the student's version. Sometimes, I want to remove entire cells, and other times, just the output. I believe this can be achieved by tagging cells and using nbconvert to save the Notebook (.ipynb) into another .ipynb file. A similar method was suggested for converting .ipynb to HTML here. I'm interested in hearing if anyone has experience, or creative new ideas, in creating different versions of a single Jupyter Notebook for various audiences. My project is quite extensive, so I need an automated solution. While I could manually create different versions by copying and pasting if I only had a few Notebooks, this approach isn’t feasible for a larger scale. Following initial comments, I tried assigning the tag "remove_cell" to some of the Notebook cells (example notebook here) and then removing them using the suggestion of this answer jupyter nbconvert nbconvert-example.ipynb --TagRemovePreprocessor.remove_cell_tags='{"remove_cell"}'  However, the tagged cells are not removed when saving to Jupyter Notebook with the --to notebook option. They are only removed with the --to html option.

Answer: This is all built in to nbconvert. You can just do: # ensure all output has been produced jupyter nbconvert tutor-source-only.ipynb --to notebook \ --execute --output tutor-with-output.ipynb # remove cells that you want the students to figure out on their own and # optionally remove the output of some/all cells, so the students have to # run the notebook jupyter nbconvert tutor-with-output.ipynb \ --to notebook \ --output student.ipynb \ --TagRemovePreprocessor.enabled True \ --TagRemovePreprocessor.remove_cell_tags remove_cell \ --TagRemovePreprocessor.remove_all_outputs_tags remove_output  I would advise against using TagRemovePreprocessor.remove_input_tags as the source is kept in the file, but the GUI is told not to show it. For any cells where you want to show the students what the output should look like, but not how to produce it, I would recommend writing your own preprocessor that takes the cell, strips the input, and converts the cell to a markdown cell, copying across the output. That way the "output" remains when the students run the notebook. You can find more on how to add and run your own preprocessors here: https://nbconvert.readthedocs.io/en/latest/nbconvert_library.html#Custom-Preprocessors
Question: Trying to find the fastest method to initiate a 1D numpy array of True values. %timeit -n 100000 -r 30 np.ones(10000, dtype=bool) returns 750 ns ± 35.7 ns whereas %timeit -n 100000 -r 30 ~np.zeros(10000, dtype=bool) returns 682 ns ± 7.47 ns Behaviour probably depends on the array size but is there a general rule of thumb to which one to choose? Are there other faster methods?

Answer: np.frombuffer(bytearray().ljust(10000, b"\x01"), dtype=bool) is faster than empty or full. Benchmark results for NumPy 1.22.4: $ python3 -m timeit -s 'import numpy as np' 'np.ones(10000, dtype=bool)' 100000 loops, best of 5: 2.12 usec per loop $ python3 -m timeit -s 'import numpy as np' '~np.zeros(10000, dtype=bool)' 200000 loops, best of 5: 1.66 usec per loop $ python3 -m timeit -s 'import numpy as np' 'np.full(10000, True)' 100000 loops, best of 5: 2.01 usec per loop $ python3 -m timeit -s 'import numpy as np' 'a = np.empty(10000, dtype=bool); a[:] = True' 200000 loops, best of 5: 1.25 usec per loop $ python3 -m timeit -s 'import numpy as np' 'np.frombuffer(bytearray().ljust(10000, b"\x01"), dtype=bool)' 500000 loops, best of 5: 636 nsec per loop  This (ab)uses the fact that bytearray.ljust calls memset under the hood, which means that the bytearray can be initialized very quickly. The rest is just overhead from constructing the bytearray and the NumPy array.
Question: I want to write function in python when given float or string for example 0.003214 to return me 0.003215, but if I give it 0.00321 to return 0.003211, it will apply on other floats also like 0.0000324 -> 0.00003241. What is wrong with my solution and how do I fix it? def add_one_at_last_digit(v): after_comma = Decimal(v).as_tuple()[-1]*-1 add = Decimal(1) / Decimal(10**after_comma) return Decimal(v) + add 

Answer: Here is a working solution from decimal import Decimal def add_one_at_last_digit(v: float | str) -> float: d = Decimal(str(v)) decimal_places = abs(d.as_tuple().exponent) add = Decimal(f"1e-{decimal_places}") result = (d + add).quantize(Decimal(f"1e-{decimal_places}")) return float(result)  The key ideas are:  d = Decimal(str(v)) gives the a Decimal object with the exact number of decimal places. The string conversion does the trick here. For example  >>> Decimal(213.4) Decimal('213.400000000000005684341886080801486968994140625') >>> Decimal("213.4") Decimal('213.4')  You did not do this in your solution which is why your after_comma is wrong.  abs(d.as_tuple().exponent) gives the exact number of decimal places. (d + add).quantize(Decimal(f"1e-{decimal_places}")) again quantizes it to remove imprecision.
Question: I have been asked to use the following set of column indices:  y = np.array([3, 0, 4, 1]) to turn into 1 all the elements in the following matrix: x = np.zeros(shape = (4, 5))  that have y as starting column and rows given by the position of y. Just to be clear. The final result has to be the following: [[0. 0. 0. 1. 1.] [1. 1. 1. 1. 1.] [0. 0. 0. 0. 1.] [0. 1. 1. 1. 1.]]  For example: y[0] = 3, then row 0, columns 3 and 4 need to be equal to 1. I did it like this: for (idx, num) in enumerate(y): x[idx, num:] = 1 Can this result be written differently and/or improved by using other Numpy functions (for example, using vectorization)?

Answer: Lots of ways of doing this. For example, since x is basically a boolean mask, you can compute a mask and turn it into whatever type you want: x = (np.arange(5) < y[:, None]).astype(float)  You might also use np.where to avoid the conversion: x = np.where(np.arange(5) < y[:, None], 1.0, 0.0)
Question: I am trying to assess whether the lips of a person are moving too much while the mouth is closed (to conclude it is chewing). The mouth closed part is done without any issue, but when I try to assess the lip movement through landmarks (dlib) there seems to be a problem with the last landmark of the mouth. Inspired by the mouth example (https://github.com/mauckc/mouth-open/blob/master/detect_open_mouth.py#L17), I wrote the following function: def lips_aspect_ratio(shape): # grab the indexes of the facial landmarks for the lip (mStart, mEnd) = (61, 68) lip = shape[mStart:mEnd] print(len(lip)) # compute the euclidean distances between the two sets of # vertical lip landmarks (x, y)-coordinates # to reach landmark 68 I need to get lib[7] not lip[6] (while I get lip[7] I get IndexOutOfBoundError) A = dist.euclidean(lip[1], lip[6]) # 62, 68 B = dist.euclidean(lip[3], lip[5]) # 64, 66 # compute the euclidean distance between the horizontal # lip landmark (x, y)-coordinates C = dist.euclidean(lip[0], lip[4]) # 61, 65 # compute the lip aspect ratio mar = (A + B) / (2.0 * C) # return the lip aspect ratio return mar  The landmark of the lips are (61, 68), when I extract the lip as lip = shape[61:68] and try to access the last landmark as lip[7] I get the following error: IndexError: index 7 is out of bounds for axis 0 with size 7  Why is that? and How to get the last landmark of the lip/face

Answer: lip = shape[61:68]  The slices exclude the end element. So you got 7 elements: 61,62,63,64, 65,66,67. And len(lip) == 7 confirms that. If there truly are 8 points for the lip shape, and they include element 68, the slice should be: lip = shape[61:69] assert(len(range(61, 69) == 8) assert(len(lip) == 8)  Note that if shape is sufficiently long, then len(shape[a:b:c]) == len(range(a, b, c)). That's because the range function acts like taking a slice out of an infinitely long list of integers (with caveats). So, the problem is a classic off-by-one, not much to do with AI/image analysis :)
Question: I am practicing crawling web, and yesterday I had an unexpected correct result which I dont think it should be work. I used soup.find(id=i) to find the attribute key i, I though i must be string, but when I passed a tuple - which is first element of tuple is string that is key, and I was surprise when it still ran correct result. let say '01' is the key of attribute, the code below had exact result with id='01' tup = ('01', 'Revenue') acc = soup.find(id=tup).text.strip().split('\n') Who has experience on this matter, please help me to explain? Thank you so much. What I tried: tup = ('01', 'Revenue') acc = soup.find(id=tup).text.strip().split('\n')  I expect the KeyError because I passed a tuple instead of a string to id.

Answer: I searched the BeautifulSoup source code And found 3 occurrences where it checks if something is a tuple. I haven't went to the whole chain of calls, but it seems to me that whenever you pass tuples or lists as arguments, BeautifulSoup will turn it into a space-separated string, and will further checks for every values. I think this part is the actual unpacking / conversion: From source code  def _attr_value_as_string(self, value, default=None): """Force an attribute value into a string representation. A multi-valued attribute will be converted into a space-separated stirng. """ value = self.get(value, default) if isinstance(value, list) or isinstance(value, tuple): value =" ".join(value) return value
Question: I have created the following pandas dataframe: ds = {'col1':[1,2,2,3,4,5,5,6,7,8]} df = pd.DataFrame(data=ds)  The dataframe looks like this: print(df) col1 0 1 1 2 2 2 3 3 4 4 5 5 6 5 7 6 8 7 9 8  I have then created a new field, called newCol, which has been defined as follows: def criteria(row): if((row['col1'] > 0) & (row['col1'] <= 2)): return "A" elif((row['col1'] > 2) & (row['col1'] <= 3)): return "B" else: return "C" df['newCol'] = df.apply(criteria, axis=1)  The new dataframe looks like this: print(df) col1 newCol 0 1 A 1 2 A 2 2 A 3 3 B 4 4 C 5 5 C 6 5 C 7 6 C 8 7 C 9 8 C  Is there a possibility to create a dictionary like this: dict = { '0 <= 2' : "A", '2 <= 3' : "B", 'Else' : "C" }  And then apply it to the dataframe: df['newCol'] = df['col1'].map(dict)  ? Can anyone help me please?

Answer: Yes, you could do this with IntervalIndex: dic = {(0, 2): 'A', (2, 3): 'B', } other = 'C' bins = pd.IntervalIndex.from_tuples(dic) labels = list(dic.values()) df['newCol'] = (pd.Series(labels, index=bins) .reindex(df['col1']).fillna(other) .tolist() )  But given your example, it seems more straightforward to go with cut: df['newCol'] = pd.cut(df['col1'], bins=[0, 2, 3, np.inf], labels=['A', 'B', 'C'])  Output:  col1 newCol 0 1 A 1 2 A 2 2 A 3 3 B 4 4 C 5 5 C 6 5 C 7 6 C 8 7 C 9 8 C  If you insist on your original dictionary format, you could convert using: dic = {'0 <= 2' : "A", '2 <= 3' : "B", 'Else' : "C" } dic2 = {tuple(map(int, k.split(' <= '))): v for k, v in dic.items() if k != 'Else'} # {(0, 2): 'A', (2, 3): 'B'} other = dic['Else']
Question: In Python, I can use vars() to dynamically set class attributes class A: vars()['x'] = 1 print(A.x) # Prints 1  However, the documentation for vars() states  Without an argument, vars() acts like locals(). Note, the locals dictionary is only useful for reads since updates to the locals dictionary are ignored.  This indicates updates to vars() shouldn't have any effect, but they clearly do based on the example above. The documentation for locals() similarly states  Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.  This is slightly different from the var() documentation, as it states updates may not affect local variables, while the vars() documentation claims it doesn't affect local variables at all. Is it safe to keep relying on using vars() to set class attributes? Or am I relying on unspecified behavior by doing this?

Answer: In the very early versions of Python, all namespaces, whether in functions, classes or modules, were all implemented as a dictionary. But for performance reasons, since Python 1.0, the implementation of function namespaces was changed to use the STORE_FAST and LOAD_FAST bytecodes to access local variables as indices in an array instead. A call to locals() from a function returns a dictionary that is created on the fly from the array of local variables. This means that the dictionary returned by locals() and the actual array storing local variables may get out of sync with each other. Writes to the locals() dictionary may not show up as modifications to local variables. And writes to local variables may not be reflected in the locals() dictionary obtained earlier. This is why you see the notes in the documentation of locals() and vars() that updates to the dictionary returned by the call may not affect the actual local variables. However, the STORE_FAST and LOAD_FAST bytecodes were implemented only for functions, not classes or modules, which is why you are observing the behavior that changes made to the dictionary returned by a call to locals()/vars() from a class are reflected in the resulting namespace of the class. So even though the behavior is technically undocumented, it is something that has not changed since the beginning of Python. Moreover, since Python 3.13, with the implementation of PEP-667, access to the f_locals attribute of a frame now returns a proxy mapping to the array of local variables, guaranteeing that writes to f_locals can now be reflected in actual local variables, and vice versa. For backwards compatibility though, a call to locals() from a function returns a snapshot of f_locals rather than the proxy mapping itself. But this is a clear sign that Python developers intend to improve the consistency between locals() and the local namespace rather than the other way around, so I believe you can indeed keep relying on using locals()/vars() to set class attributes for the foreseeable future.
Question: I've just started using VSCode (moving over from Spyder) and I regularly run lines/selections from my code in the terminal. When I have a terminal open, shift + enter runs my selection in a new terminal and is incredibly slow. The below takes about 5 seconds to send  print(1)  If i copy and paste it in the new terminal, it runs instantly. Why is it slow? I want to continue using shift+enter to run my selection. Why does it need to open a new terminal anyway? It can run in the existing terminal, where i start with py  which begins a python interactive session.

Answer: As for the delay, according to this post on GitHub, the issue seems to be with the latest version (v2024.16.0) of the Python extension for VS Code. Downgrading it (and restarting the app) seems to do the trick for the time being.
Question: numpy.random.uniform returns a double between the specified low and high bounds. I can use np.finfo(np.double) to get the min and max representable number, but if I use those values in numpy.random.uniform I get an error. import numpy as np info = np.finfo(np.double) np.random.uniform(low=info.min, high=info.max)   OverflowError: Range exceeds valid bounds  Can someone shed light on what is going wrong here? What are the actual supported limits for low and high? I tried getting the bounds for a double and inputting them to a rng intended for generating doubles, I expected these would be compatible. Halving the bounds does work: np.random.uniform(low=info.min/2, high=info.max/2)  So it suggests that either uniform cannot actually generate any double, or that the bounds returned by finfo are not inclusive.

Answer: uniform actually performs low + (high-low) * np.random.random(). However high - low is not possible: info.max-info.min # RuntimeWarning: overflow encountered in scalar subtract info.max-info.min  You found the limits of uniform. The difference between low and high should not exceed abs(info.max). np.random.uniform(low=0, high=info.max) # valid np.random.uniform(low=info.min, high=0) # valid np.random.uniform(low=info.min/2, high=info.max/2) # valid rng.uniform(low=info.min*0.25, high=info.max*0.75) # valid rng.uniform(low=info.min*0.501, high=info.max*0.501) # OverflowError
Question: I am trying to display a folium map in a card in a shiny for Python app using the code below (Reproducible example): from shiny import App, ui, render, reactive import folium app_ui = ui.page_fluid( ui.tags.style( """ /* apply css to control height of some UI widgets */ .map-container { height: 700px !important; width: 100%; overflow: hidden; } .map-container > * { height: 100% !important; } """ ), ui.column(6, ui.navset_card_tab( ui.nav_panel("Map", ui.div( ui.output_ui("map"), class_="map-container" ) ), ui.nav_panel("Data", ui.p( ui.output_table("data_table") ) ) ) ) ) def server(input, output, session): @output @render.ui def map(): m = folium.Map(location=[51.509865, -0.118092], zoom_start=12) return ui.HTML(m._repr_html_()) @output @render.table def data_table(): # Your data table rendering logic here pass app = App(app_ui, server)  I want the element containing the map to fill the height of my screen but my the map itself is refusing to fill the whole height of the card. It is only occupying the upper half of the card area. In Shiny for R, I had achieved the same result using the as_fill_carrier() function from the bslib library like this:  card( style = "height: 80vh;", full_screen = TRUE, #card_header("Map"), card_body( tmapOutput("map") %>% withSpinner(type = 6, color = "#30804e") %>% as_fill_carrier() ) ) )  What can I do?

Answer: This is an issue with folium, where m._repr_html_() causes the map to be embedded within a div which has padding-bottom: 60%; by default. This has to be overwritten if you want to use the whole space. Below is an example where this is done by just replacing the css with more suitable one, in particular, I remove the padding-bottom: 60%; and apply height: 100%; which yields the desired filling layout. from shiny import App, ui, render import folium app_ui = ui.page_fluid( ui.tags.style( """ /* apply css to control height of some UI widgets */ .map-container { height: 700px !important; width: 100%; overflow: hidden; } .map-container > * { height: 100% !important; } """ ), ui.column(6, ui.navset_card_tab( ui.nav_panel("Map", ui.div( ui.output_ui("map"), class_="map-container" ) ), ui.nav_panel("Data", ui.p( ui.output_table("data_table") ) ) ) ) ) def server(input, output, session): @output @render.ui def map(): m = folium.Map(location=[51.509865, -0.118092], zoom_start=12) m = m._repr_html_() m = m.replace( '<div style="width:100%;"><div style="position:relative;width:100%;height:0;padding-bottom:60%;">', '<div style="width:100%; height:100%;"><div style="position:relative;width:100%;height:100%;>', 1) return ui.HTML(m) @output @render.table def data_table(): # Your data table rendering logic here pass app = App(app_ui, server)
Question: I want to add Snowflake hybrid tables to my database schema using SQLAlchemy. Per this issue, support for hybrid tables is not implemented in the Snowflake SQLAlchemy official dialect. Is there a way for me to customize the CREATE TABLE ... statement generated by SQLAlchemy so that it actually generates CREATE HYBRID TABLE ... for a given table? Consider the example code below: from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column class Base(DeclarativeBase): pass class MyTable(Base): id: Mapped[int] = mapped_column(primary_key=True) Base.metadata.create_all(bind=...)  Running it will emit the following SQL statement: CREATE TABLE mytable ( id INTEGER NOT NULL AUTOINCREMENT, CONSTRAINT pk_mytable PRIMARY KEY (id) )  and I would like a way to have the following instead: CREATE HYBRID TABLE mytable ( id INTEGER NOT NULL AUTOINCREMENT, CONSTRAINT pk_mytable PRIMARY KEY (id) ) 

Answer: You can pass prefixes to a table. from sqlalchemy import create_engine from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column class Base(DeclarativeBase): ... class SomeTable(Base): __tablename__ = "some_table" __table_args__ = {"prefixes": ["HYBRID"]} id: Mapped[int] = mapped_column(primary_key=True) engine = create_engine("sqlite:///test.sqlite", echo=True) Base.metadata.create_all(engine)  SQL Emitted CREATE HYBRID TABLE some_table ( id INTEGER NOT NULL, PRIMARY KEY (id) )  I used SQLite because I do not have access to Snowflake but it should work all the same in an actual Snowflake instance.
Question: Let's say I have the following DataFrame: df = pl.DataFrame({ 'values': [[0, 1], [9, 8]], 'qc_flags': [["", "X"], ["T", ""]] })  I only want to keep my values if the corresponding qc_flag equals "". Does anyone know the correct way to go about this? I've tried something like this: filtered = df.with_columns( pl.col("values").list.eval( pl.element().filter( pl.col("qc_flags").list.eval( pl.element() == "" ) ) ) )  I would expect to get 'values': [[0], [8]], but then I just end up with this error:  ComputeError: named columns are not allowed in `list.eval`; consider using `element` or `col("")` 

Answer:  pl.Expr.list.eval to evaluate expression within list. pl.arg_where() to find indexes where value is "". pl.Expr.list.gather() to take sublist by indexes.  df.with_columns( filtered = pl.col.values.list.gather( pl.col.qc_flags.list.eval(pl.arg_where(pl.element() == "")) ) )  shape: (2, 3) ┌───────────┬───────────┬───────────┐ │ values ┆ qc_flags ┆ filtered │ │ --- ┆ --- ┆ --- │ │ list[i64] ┆ list[str] ┆ list[i64] │ ╞═══════════╪═══════════╪═══════════╡ │ [0, 1] ┆ ["", "X"] ┆ [0] │ │ [9, 8] ┆ ["T", ""] ┆ [8] │ └───────────┴───────────┴───────────┘
Question: I'm trying to set a conditional statmeent in sympy so that when the iterable is not equal to a certain number (30) in this case, it returns 0. My code is: def formula(address): address0 = 30 logic = 3 * x # Define the symbol and the summation range x = symbols('x') expr = Piecewise( (x + logic, logic == address0), # When x equals certain_integer, only add logic ((x + logic) - (x + logic), logic!=address0) # For all other values of x, apply full logic (x + logic - x + logic) ) #If -x + logic only do it to the ones that aren't address0 total = summation(expr(x, 0, 089237316195423570985008687907836)) total2 = total - address0 print(f'{total}\n{total2}') return total2  As you can see in my code, in the expr variable I set x+logic when logic is 30 and when the other logicsaren't 30 it returns 0 since it subtracts it. The code is returning 0 regardless of what I do and I don't know why. Can someone help me?

Answer: The == operator will instantly evaluate to True or False based on structure of the objects. Since 3*x is not 30 the equality evaluates to False and that term of the Piecewise is ignored. Use the following instead: ... from sympy import Eq expr = Piecewise( (x + logic, Eq(logic,address0)), (0, True)) # (x+logic)-(x-logic)->0 if x and logic are finite  Your use of summation seems strange, too. How about: >>> piecewise_fold(Sum(expr,(x,0,100))).doit() Piecewise((20200, Eq(x, 10)), (0, True))  But I am not really sure what you are trying to do with summation.