eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Python: 3.12.2, OS: MacOS 13.6.6 When I specify a filename to shelve.open, it appends .db to the filename: % ls % python Python 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import shelve >>> with shelve.open("myfile"): ... pass ... >>> quit() % ls myfile.db  Furthermore, if I attempt to open an existing file as "myfile.db" (with the extension), I get the following error: % python Python 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import shelve >>> with shelve.open("myfile.db"): ... pass ... Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shelve.py", line 243, in open return DbfilenameShelf(filename, flag, protocol, writeback) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/shelve.py", line 227, in __init__ Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/dbm/__init__.py", line 89, in open raise error[0]("db type could not be determined") dbm.error: db type could not be determined  Opening the existing file as simply "myfile" with no extension works fine, however.  How do I prevent shelve.open from appending ".db" to the filename? Why can't I open existing databases if I specify their ".db" extension?  Neither of these issues happen on Python 3.10.12 on Ubuntu 22, so I'm not sure if it's a Python version thing, or a platform thing.

Answer:  Why can't I open existing databases if I specify their ".db" extension?  After scrying shelve source code it could be unveiled that shelve.open("myfile") does result in calling dbm.open(filename, 'c') After scrying dbm source code it could be unveiled that this does depend on dbm.whichdb, where following line could be found f = io.open(filename + b".db", "rb")  therefore if you attempt to do shelve.open("myfile.db") whichdb would be looking for myfile.db.db.  How do I prevent shelve module from appending ".db" to filename?  If you would do that this would most likely cause dbm.whichdb to malfunction for ndbm and dumbdm as it rely on extensions to detect nature of system used, as unveiled in comments # Check for ndbm first -- this has a .pag and a .dir file # Check for dumbdbm next -- this has a .dir and a .dat file  whilst for sqlite3 and gnu it should would work independently from extensions as detection rely on headers (see source code linked above for details).
Question: I encounter problems trying to fill all null values on a specific column of a data frame. Here is an example of dataframe and my expected outcome. Example data frame: Column_1 Column_2 F A None B None None G C None None None D H D  I want to get the first value from the column 1 to all null value from column 2 Expected Outcome: Column_1 Column_2 F A None B None G #First value from the left column G C None H #First value from the left column None D H D  I'm getting error when I try this code. df['Colunmn_2'].ffill(df.loc[df['Column_1'].first_valid_index(), 'Column_1'],inplace=True) Thanks in advance!

Answer: You can combine fillna on Column_2 and bfill on Column_1: df['Column_2'] = df['Column_2'].fillna(df['Column_1'].bfill())  Output:  Column_1 Column_2 0 F A 1 NaN B 2 NaN G 3 G C 4 NaN H 5 NaN D 6 H D  Intermediates:  Column_1 Column_2 col1_bfill col2_fillna 0 F A F A 1 NaN B G B 2 NaN NaN G ------> G 3 G C G C 4 NaN NaN H ------> H 5 NaN D H D 6 H D H D
Question: Here is an example code: while True: # here is a comment pass  Here is my Python and Nuitka version: D:\test>py -m nuitka --version 2.6.8 Commercial: None Python: 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 22:45:29) [MSC v.1916 32 bit (Intel)] Flavor: CPython Official Executable: ~\AppData\Local\Programs\Python\Python38-32\python.exe OS: Windows Arch: x86 WindowsRelease: 10 Version C compiler: ~\AppData\Local\Nuitka\Nuitka\Cache\DOWNLO~1\gcc\x86\14.2.0posix-19.1.1-12.0.0-msvcrt-r2\mingw32\bin\gcc.exe (gcc 14.2.0).  I ran the command py -m nuitka --standalone --onefile test.py and finally got test.exe. Then I ran test.exe and pressed Ctrl+C in the windows terminal. I got a traceback like this: Traceback (most recent call last): File "C:\Users\User\AppData\Local\Temp\ONEFIL~3\test.py", line 1, in <module> while True: # here is a comment KeyboardInterrupt ^C  I found the source code even with the comment. But most people say that programs compiled by Nuitka is irreversible. So what's wrong?

Answer: In the Nuitka page you can read that  Nuitka Standard The standard edition bundles your code, dependencies and data into a single executable if you want. It also does acceleration, just running faster in the same environment, and can produce extension modules as well. It is freely distributed under the Apache license.   Nuitka Commercial The commercial edition additionally protects your code, data and outputs, so that users of the executable cannot access these. This a private repository of plugins that you pay to get access to. Additionally, you can purchase priority support.  So to encrypt all traceback outputs you have to buy the Commercial version. In this Nuitka Commercial you can see the features only Nuitka Commercial offers.  Onefile:  Creates a single executable file. When this executable runs, it extracts its contents to a temporary directory. If the source code is within the folder where the onefile is placed, then Nuitka can find it and display the source code in tracebacks.  Standalone:  Creates a directory containing the executable and all its dependencies. By default this would create a dist folder, but the source code is not included in the folder. Because of that Nuitka can't display the source code in tracebacks.  For Nuitka to display source code lines within tracebacks, the original Python source files must be present in the same location when the compiled program is executed.
Question: I changed my code base to always import the python datetime module like this: import datetime as dt  instead of using import datetime  or from datetime import datetime  And we had both those in the codebase! It's confusing because you can't know at this point what datetime can do, if it is the module or the class. See also this blog post by Adam Johnson: https://adamj.eu/tech/2019/09/12/how-i-import-pythons-datetime-module/ What I try to do is create a rule for the ruff linter that enforces this import style. There is tidy-imports but I can't get it to work.

Answer: You should be able to lint code using from datetime import datetime syntax to suggest replacing it with import datetime as dt with the TOML configuration: [tool.ruff.lint] # Add the ICN rules to any others you have selected. select = ["E4", "E7", "E9", "F", "ICN"] [tool.ruff.lint.flake8-import-conventions] banned-from = ["datetime"] [tool.ruff.lint.flake8-import-conventions.extend-aliases] "datetime" = "dt"  Which, for the code: import datetime from datetime import datetime as dto import datetime as dt print(datetime, dto, dt)  Outputs the errors: `datetime` should be imported as `dt` (ICN001) [Ln 1, Col 8] Members of `datetime` should not be imported explicitly (ICN003) [Ln2, Col 1]  fiddle See unconventional-import-alias (ICN001) and banned-import-from (ICN003)
Question: Here is the data import polars as pl from datetime import datetime df = pl.DataFrame( { "time": [ datetime(2021, 2, 1), datetime(2021, 4, 2), datetime(2021, 5, 4), datetime(2021, 6, 6), datetime(2021, 6, 8), datetime(2021, 7, 10), datetime(2021, 8, 18), datetime(2021, 9, 20), ], "groups": ["A", "B", "A", "B","A","B","A","B"], "values": [0, 1, 2, 3,4,5,6,7], } )  The upsampling and the testing: ( df .upsample( time_column="time", every="1d", group_by="groups", maintain_order=True ) .group_by('groups') .agg(pl.col('time').diff().max()) )  shape: (3, 2) ┌────────┬──────────────┐ │ groups ┆ time │ │ --- ┆ --- │ │ str ┆ duration[μs] │ ╞════════╪══════════════╡ │ A ┆ 92d │ │ null ┆ 2d │ │ B ┆ 72d │ └────────┴──────────────┘  The diff is not 1 day as I would expect. Is this a bug, or am I doing something wrong?

Answer: It is due to the group columns resulting in null - which is a bug.  https://github.com/pola-rs/polars/issues/15530  upsample itself is implemented as a datetime_range and join  https://github.com/pola-rs/polars/blob/a4fbc9453cacb7e7e5cc476b30a98845aaa5f506/crates/polars-time/src/upsample.rs#L203  Which you could do manually as a workaround. (df.group_by("groups") .agg(pl.datetime_range(pl.col("time").first(), pl.col("time").last())) .explode("time") .join(df, on=["groups", "time"], how="left") .group_by("groups") .agg(pl.col("time").diff().max()) )  shape: (2, 2) ┌────────┬──────────────┐ │ groups ┆ time │ │ --- ┆ --- │ │ str ┆ duration[μs] │ ╞════════╪══════════════╡ │ A ┆ 1d │ │ B ┆ 1d │ └────────┴──────────────┘
Question: I have a CSV (or rather TSV) I got from stripping the header off a gVCF with bcftools view foo.g.vcf -H > foo.g.vcf.csv  A head gives me this, so everything looks like expected so far chr1H 1 . T <*> 0 . END=1000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 1001 . T <*> 0 . END=1707 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 1708 . C <*> 0 . END=1763 GT:GQ:MIN_DP:PL 0/0:6:2:0,6,59 chr1H 1764 . T <*> 0 . END=2000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 2001 . A <*> 0 . END=3000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 3001 . G <*> 0 . END=4000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 4001 . T <*> 0 . END=5000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 5001 . T <*> 0 . END=6000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 6001 . A <*> 0 . END=7000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0 chr1H 7001 . G <*> 0 . END=8000 GT:GQ:MIN_DP:PL 0/0:1:0:0,0,0  When I know try to read the file as a dataframe in a Jupyter Notebook like this import polars as pl df = pl.read_csv("foo.g.vcf.csv", has_header=False, new_columns=["CHROM", "POS", "ID", "REF", "ALT", "QUAL", "FILTER", "INFO", "FORMAT", "SAMPLE"], separator="\t")  I get a compute error "Original error: remaining bytes non-empty". However, when I do import pandas as pd import polars as pl df = pd.read_csv("foo.g.vcf.csv", header=None, sep="\t", names=["CHROM", "POS", "ID", "REF", "ALT", "QUAL", "FILTER", "INFO", "FORMAT", "SAMPLE"]) df = pl.DataFrame(df)  every works as intended. Why can I read with pandas without problems and convert to polars, but not read with polars directly? The other VCF I want to compare with, which I stripped the same way, works with polars.

Answer: Looks like you might have empty trailing spaces. Hence the error: Original error: remaining bytes non-empty  Polars is stricter than pandas on file formatting. Pandas will infer formatting but Polars will not. You can use this command to remove empty lines and white spaces: sed -i '/^\s*$/d' foo.g.vcf.csv  But I recommend you tell Polars to infer the schema from the whole file instead with: infer_schema_length=None  Or you can tell Polars to ignore parsing errors (I do not recommend this but it is an option) with: ignore_errors=True
Question: I have a MongoDB document like: [ { _id: ObjectId('67cfd69ba3e561d35ee57f51'), created_at: ISODate('2025-03-11T06:22:19.044Z'), conversation: [ { id: '67cfd6c1a3e561d35ee57f53', feedback: { liked: false, disliked: true, copied: true, created_at: ISODate('2025-03-11T06:27:48.634Z') } }, { id: '67cfd77fa3e561d35ee57f54', feedback: { liked: true, disliked: false, copied: false, created_at: ISODate('2025-03-11T06:28:25.099Z') } }, { id: '67d009f1a3e561d35ee57f5a', feedback: null }, { id: '67d009f8a3e561d35ee57f5b', feedback: null } ] }, { _id: ObjectId('67d00aeaa3e561d35ee57f5d'), created_at: ISODate('2025-03-11T10:05:30.848Z'), conversation: [ { id: '67d00af7a3e561d35ee57f5f', feedback: null }, { id: '67d00afaa3e561d35ee57f60', feedback: null } ] } ]  Where the main document has a conversation subdocument, I want to know how many likes, dislikes and copied data in each year. I tried to get year from the conversation.feedback.created_at using $dateToString operator. pipeline = [ { '$match': { 'conversation.feedback.copied': True } }, { '$group': { '_id': { '$dateToString': { 'format': '%Y', 'date': '$conversation.feedback.created_at' } }, 'total_copied': { '$sum': 1 } } } ]  But it gives an error:  OperationFailure: PlanExecutor error during aggregation :: caused by :: can't convert from BSON type array to Date, full error: {'ok': 0.0, 'errmsg': "PlanExecutor error during aggregation :: caused by :: can't convert from BSON type array to Date", 'code': 16006, 'codeName': 'Location16006'}  What I am expecting out as: { "2025": { "total_liked": 1, "total_disliked": 1, "total_copied": 1 } }  How to convert the DateTime object to year and combine the total counts for 3 parameters?

Answer:  You need the $unwind stage to deconstruct the conversation array before grouping by conversation.feedback.created_at.  Note that, in your sample data, there is possibly the conversation.feedback is null. Hence you should remove those unwinded document with conversation.feedback is null.  For calculating the sum based on the boolean value, you can work with $cond to add 1 when the value is true.  If you are looking for the generated output with key-value pair, you may look for $replaceRoot and $arrayToObject to convert list of objects to key-value pair.   db.collection.aggregate([ { "$match": { "conversation.feedback.copied": true } }, { "$unwind": "$conversation" }, { "$match": { "conversation.feedback": { "$ne": null } } }, { "$group": { "_id": { "$dateToString": { "format": "%Y", "date": "$conversation.feedback.created_at" } }, "total_copied": { "$sum": { $cond: [ { $eq: [ "$conversation.feedback.copied", true ] }, 1, 0 ] } }, "total_liked": { "$sum": { "$cond": [ { "$eq": [ "$conversation.feedback.liked", true ] }, 1, 0 ] } }, "total_disliked": { "$sum": { "$cond": [ { "$eq": [ "$conversation.feedback.disliked", true ] }, 1, 0 ] } } } }, { "$replaceRoot": { "newRoot": { "$arrayToObject": [ [ { "k": "$_id", "v": { "total_copied": "$total_copied", "total_liked": "$total_liked", "total_disliked": "$total_disliked" } } ] ] } } } ])  Demo @ Mongo Playground
Question: I am working with Polars and need to ensure that my dataset contains all possible combinations of unique values in certain index columns. If a combination is missing in the original data, it should be filled with null. Currently, I use the following approach with sequential cross joins: def ensure_uniform(df: pl.DataFrame, index_cols: Sequence[str]) -> pl.DataFrame: # Quick exit if len(index_cols) == 1: return df # Get unique values of the first index column uniform_df = df.select(index_cols[0]).unique(maintain_order=True) # Cross join with other unique index columns for i in range(1, len(index_cols)): unique_index_values = df.select(index_cols[i]).unique(maintain_order=True) uniform_df = uniform_df.join(unique_index_values, how="cross") # Left join with the original DataFrame to preserve existing values return uniform_df.join(df, on=index_cols, how="left")  Here is an example: df = pl.from_repr(''' ┌─────┬─────┬─────┬───────┐ │ g1 ┆ g2 ┆ g3 ┆ value │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═══════╡ │ A ┆ 1 ┆ 1 ┆ 10 │ │ A ┆ 1 ┆ 2 ┆ 20 │ │ B ┆ 2 ┆ 1 ┆ 30 │ │ B ┆ 2 ┆ 2 ┆ 40 │ └─────┴─────┴─────┴───────┘ ''') uniform_df = ensure_uniform(df, index_cols=["g1", "g2", "g3"]) print(uniform_df) # ┌─────┬─────┬─────┬───────┐ # │ g1 ┆ g2 ┆ g3 ┆ value │ # │ --- ┆ --- ┆ --- ┆ --- │ # │ str ┆ i64 ┆ i64 ┆ i64 │ # ╞═════╪═════╪═════╪═══════╡ # │ A ┆ 1 ┆ 1 ┆ 10 │ # │ A ┆ 1 ┆ 2 ┆ 20 │ # │ A ┆ 2 ┆ 1 ┆ null │ # │ A ┆ 2 ┆ 2 ┆ null │ # │ B ┆ 1 ┆ 1 ┆ null │ # │ B ┆ 1 ┆ 2 ┆ null │ # │ B ┆ 2 ┆ 1 ┆ 30 │ # │ B ┆ 2 ┆ 2 ┆ 40 │ # └─────┴─────┴─────┴───────┘  Any suggestions for making this more graceful and efficient? Edit: @Dean MacGregor & @orlp Thank you for your answers. All approaches show comparable performance (+/-10%), with @Dean MacGregor's proposal consistently being slightly faster than the others. After testing on multiple setups, I found that the main bottleneck seems to be the final join process, combining the uniform multiindex with the original dataset, rather than building the Cartesian product beforehand. This suggests that speed and peak memory consumption remain similar regardless of how the Cartesian product is computed, especially as dataset sizes grow. Both proposals enable lazy operations, which could be useful depending on the use case. Since there is no dedicated method for computing the Cartesian product, all options seem to be valid.

Answer: I think your approach is sensible, it can however be done lazily: from functools import reduce def ensure_uniform(df: pl.DataFrame, index_cols: Sequence[str]) -> pl.DataFrame: if len(index_cols) == 1: return df lf = df.lazy() uniques = [lf.select(col).unique(maintain_order=True) for col in index_cols] product = reduce(lambda a, b: a.join(b, how="cross"), uniques) out = product.join(lf, on=index_cols, how="left", maintain_order="left") return out.collect()
Question: I want to add columns in my df with values based on the sample list in one column and the next column headers as sample numbers. In detail: based on the 11 column, I want to add 3 columns designed as 11_1, 11_2 and 11_3 with values according to the sample list in the 11 and then the same for 00. My tiny part of input data: df_matrix_data = {'11': [['P4-1', 'P4-2', 'P4-3'], ['P4-1', 'P4-3', 'P4-4']], '00': [['P4-4', 'P4-6', 'P4-7',], ['P4-2', 'P4-5', 'P4-7']], 'P4-1': [1, 2], 'P4-2': [6, 8], 'P4-3': [5, 2], 'P4-4': [2, 3], 'P4-5': [np.nan, 2], 'P4-6': [6, np.nan], 'P4-7': [3, 2]} df_matrix = pd.DataFrame.from_dict(df_matrix_data)  will look like this:  11 00 P4-1 P4-2 P4-3 P4-4 P4-5 P4-6 P4-7 0 [P4-1, P4-2, P4-3] [P4-4, P4-6, P4-7] 1 6 5 2 NaN 6.0 3 1 [P4-1, P4-3, P4-4] [P4-2, P4-5, P4-7] 2 8 2 3 2.0 NaN 2  and desired output should look like this:  11 00 P4-1 P4-2 P4-3 P4-4 P4-5 P4-6 P4-7 11_1 11_2 11_3 00_1 00_2 00_3 0 [P4-1, P4-2, P4-3] [P4-4, P4-6, P4-7] 1 6 5 2 NaN 6.0 3 1 6 5 2 6 3 1 [P4-1, P4-3, P4-4] [P4-2, P4-5, P4-7] 2 8 2 3 2.0 NaN 2 2 2 3 8 2 2  Any ideas on how to perform it?

Answer: Another possible solution: df_matrix.assign( **{f"{k}_{i+1}": df_matrix.apply( lambda row: row[row[k][i]], axis=1) for k in ['11', '00'] for i in range(3)})  It uses a dictionary comprehension within assign, iterating over each key (e.g., '11') and list index (0-2), then generates columns like 11_1 by mapping the list's element (e.g., row['11'][0]) to its corresponding value in the row via lambda.  To avoid the inefficient apply: df_matrix.assign( **{f"{k}_{i+1}": df_matrix.values[ np.arange(len(df_matrix)), df_matrix.columns.get_indexer(df_matrix[k].str[i])] for k in ['11', '00'] for i in range(3)})  It uses index.get_indexer to convert column names to numeric indices. Output:  11 00 P4-1 P4-2 P4-3 P4-4 P4-5 P4-6 \ 0 [P4-1, P4-2, P4-3] [P4-4, P4-6, P4-7] 1 6 5 2 NaN 6.0 1 [P4-1, P4-3, P4-4] [P4-2, P4-5, P4-7] 2 8 2 3 2.0 NaN P4-7 11_1 11_2 11_3 00_1 00_2 00_3 0 3 1 6 5 2 6.0 3 1 2 2 2 3 8 2.0 2
Question: I have recently switched from using mypy to pyright for static type checking. mypy has a useful feature whereby it can be configured to warn you if it detects comments instructing it to ignore certain errors which would not actually be raised (described here: https://stackoverflow.com/a/65581907/7256443). Does anyone know if pyright has a similar feature? I can't see any reference to it in their documentation. Ideally, it would apply to the pyright specific ignore comments (# pyright: ignore [errorCode]) as well as the generic # type: ignore.

Answer: Pyright has reportUnnecessaryTypeIgnoreComment, which flags unnecessary # type: ignore comments as errors. To enable this check, add the following to your pyrightconfig.json: { "reportUnnecessaryTypeIgnoreComment": "error" }  Source: Pyright Documentation - Type Check Diagnostics Settings  reportUnnecessaryTypeIgnoreComment [boolean or string, optional]: Generate or suppress diagnostics for a # type: ignore or # pyright: ignore comment that would have no effect if removed. The default value for this setting is "none".
Question: I have a dataframe that contains a product name, question, and answers. I would like to process the dataframe and transform it into a JSON format. Each product should have nested sections for questions and answers. My dataframe: import polars as pl df = pl.DataFrame({ "Product": ["X", "X", "Y", "Y"], "Question": ["Q1", "Q2", "Q3", "Q4"], "Anwers": ["A1", "A2", "A3", "A4"], })  Desired Output: { "faqByCommunity": { "id": 5, "communityName": "name", "faqList": [ { "id": 1, "product": "X", "faqs": [ { "id": 1, "question": "Q1", "answer": "A1" }, { "id": 2, "question": "Q2", "answer": "A2" } ] }, { "id": 2, "product": "Y", "faqs": [ { "id": 1, "question": "Q3", "answer": "A3" }, { "id": 2, "question": "Q4", "answer": "A4" } ] } ] } }  Since the first part it's static , i think i could append it to the file before and after polars writes to it (Like my other question ). However, im not sure how can i work with the nested part

Answer: You could do some of the reshaping in Polars first. faq_list = ( df.group_by("product", maintain_order=True) .agg(faqs=pl.struct(pl.int_range(pl.len()).alias("id") + 1, pl.exclude("product"))) .with_row_index("id", offset=1) #.to_struct() #.to_list() )  shape: (2, 3) ┌─────┬─────────┬────────────────────────────────┐ │ id ┆ product ┆ faqs │ │ --- ┆ --- ┆ --- │ │ u32 ┆ str ┆ list[struct[3]] │ ╞═════╪═════════╪════════════════════════════════╡ │ 1 ┆ X ┆ [{1,"Q1","A1"}, {2,"Q2","A2"}] │ │ 2 ┆ Y ┆ [{1,"Q3","A3"}, {2,"Q4","A4"}] │ └─────┴─────────┴────────────────────────────────┘  With the to_struct/list uncommented: [{'id': 1, 'product': 'X', 'faqs': [{'id': 1, 'question': 'Q1', 'answer': 'A1'}, {'id': 2, 'question': 'Q2', 'answer': 'A2'}]}, {'id': 2, 'product': 'Y', 'faqs': [{'id': 1, 'question': 'Q3', 'answer': 'A3'}, {'id': 2, 'question': 'Q4', 'answer': 'A4'}]}]  You could then add the static parts and pretty-print it with json.dumps print( json.dumps({ "faqByCommunity": { "id": 5, "communityName": "name", "faqList": faq_list } }, indent=4) )  You could also add the static parts with Polars if you really wanted to. print( json.dumps( (df.group_by("product", maintain_order=True) .agg( faqs = pl.struct( pl.int_range(pl.len()).alias("id") + 1, pl.exclude("product") ) ) .with_row_index("id", offset=1) .select( pl.struct( faqByCommunity = pl.struct( id = 5, communityName = pl.lit("name"), faqList = pl.struct(pl.all()).implode() ) ) ) .item() ), indent = 4 ) )