eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: My goal is to perform tracing of the whole process of my application through several component. I am using GCP and Pub/Sub message queue to communicate information between components (developped in Python). I am currently trying to keep the same root trace between component A and component B by creating a new span as a child span of my root trace. Here is a small diagram: Component A ---> Pub/Sub message ---> component B (create the root trace) (contain information) (new span for root trace)  I have a given trace_id and span_id of my parent that I can transmit through Pub/Sub but I can't figure out how to declare a new span as a child of this last. All I managed to do is to link a new trace to the parent one but it is not the behavior I am looking for. Has someone already tried to do something like that ? Regards,

Answer: It's called trace context propagation and there are multiple formats such w3c trace context, jaeger, b3 etc... https://github.com/open-telemetry/opentelemetry-specification/blob/b46bcab5fb709381f1fd52096a19541370c7d1b3/specification/context/api-propagators.md#propagators-distribution. You will have to use one of the propagator's inject/extract methods for this. Here is the simple example using W3CTraceContext propagator. from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import (BatchSpanProcessor, ConsoleSpanExporter) from opentelemetry.trace.propagation.tracecontext import \ TraceContextTextMapPropagator trace.set_tracer_provider(TracerProvider()) trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(ConsoleSpanExporter())) tracer = trace.get_tracer(__name__) prop = TraceContextTextMapPropagator() carrier = {} # Injecting the context into carrier and send it over with tracer.start_as_current_span("first-span") as span: prop.inject(carrier=carrier) print("Carrier after injecting span context", carrier) # Extracting the remote context from carrier and starting a new span under same trace. ctx = prop.extract(carrier=carrier) with tracer.start_as_current_span("next-span", context=ctx): pass
Question: I'm trying to change some PyTorch code so that it can run on the CPU. The model was trained with torch.nn.DataParallel() so when I load the pre-trained model and try using it I must use nn.DataParallel() which I am currently doing like this: device = torch.device("cuda:0") net = nn.DataParallel(net, device_ids=[0]) net.load_state_dict(torch.load(PATH)) net.to(device)  However after I switched my torch device to cpu like this: device = torch.device('cpu') net = nn.DataParallel(net, device_ids=[0]) net.load_state_dict(torch.load(PATH)) net.to(device)  I got this error: File "C:\My\Program\win-py362-venv\lib\site-packages\torch\nn\parallel\data_parallel.py", line 156, in forward "them on device: {}".format(self.src_device_obj, t.device)) RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu  I'm assuming that it's still looking for CUDA because that's what device_ids is set to but is there a way to make it use the CPU? This post from the PyTorch repo makes me think that I can but it doesn't explain how. If not is there any other way to use a model trained with DataParallel on your CPU?

Answer: When you use torch.nn.DataParallel() it implements data parallelism at the module level. According to the doc:  The parallelized module must have its parameters and buffers on device_ids[0] before running this DataParallel module.  So even though you are doing .to(torch.device('cpu')) it is still expecting to pass the data to a GPU. However since DataParallel is a container you can bypass it and get just the original module by doing this: net = net.module.to(device)  Now it will access the original module you defined before you applied the DataParallel container.
Question: Is there a way to make a rule mark longer without disrupting the axes of a chart? If I have this: random.seed(0) df = pd.DataFrame({'x':[i for i in range(1,21)],'y':random.sample(range(1,50), 20)}) chart = alt.Chart(df).mark_area().encode(x='x',y='y') ruler = alt.Chart(pd.DataFrame({'x':[5]})).mark_rule().encode(x='x') chart+ruler   But I want this   : 

Answer: You can set an explicit y-domain and then set clip=False inside mark_rule, but you also need to define the y-range of the rule since the default is to stretch over the entire plot: import altair as alt import pandas as pd import random random.seed(0) df = pd.DataFrame({'x':[i for i in range(1,21)],'y':random.sample(range(1,50), 20)}) chart = alt.Chart(df).mark_area().encode(x='x', y=alt.Y('y', scale=alt.Scale(domain=(0, 50)))) ruler = alt.Chart(pd.DataFrame({'x':[5], 'y': [-10], 'y2': [50]})).mark_rule(clip=False, fill='black').encode(x='x', y='y', y2='y2') chart+ruler
Question: I have a pandas dataframe which looks like this :  Un_ID P_ID segment 0 Q8TDU6 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) 1 P63092 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) 2 Q8TDU6 7cfm 1( 22- 41), 2( 51- 72), 3( 86- 108)  I want to split the third column'segment' into three columns i.e TM,starting,ending  Un_ID P_ID segment TM starting ending 0 Q8TDU6 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) TM1 16 41 1 P63092 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) TM1 16 41 2 Q8TDU6 7cfm 1( 22- 41), 2( 51- 72), 3( 86- 108) TM1 22 41 0 Q8TDU6 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) TM2 51 73 1 P63092 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) TM2 51 73 2 Q8TDU6 7cfm 1( 22- 41), 2( 51- 72), 3( 86- 108) TM2 51 72 0 Q8TDU6 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) TM3 86 108 1 P63092 7bw0 1( 16- 41), 2( 51- 73), 3( 86- 108) TM3 86 108 2 Q8TDU6 7cfm 1( 22- 41), 2( 51- 72), 3( 86- 108) TM3 86 108  I tried following code df[['TM','starting','ending']] = df.segment.apply(lambda x: pd.Series(str(x).split(","))  But,I am not sure how to manipulate the above code to get the dataframe as I want..

Answer: Try: import re r = re.compile(r"(\d+)\(\s*(\d+)-\s*(\d+)\)") df["segment"] = df["segment"].apply(lambda x: r.findall(x)) df = df.explode("segment") df[["TM", "starting", "ending"]] = df.pop("segment").apply(pd.Series) df = df.sort_values(by="TM") df["TM"] = "TM" + df["TM"].astype(str) print(df)  Prints:  Un_ID P_ID TM starting ending 0 Q8TDU6 7bw0 TM1 16 41 1 P63092 7bw0 TM1 16 41 2 Q8TDU6 7cfm TM1 22 41 0 Q8TDU6 7bw0 TM2 51 73 1 P63092 7bw0 TM2 51 73 2 Q8TDU6 7cfm TM2 51 72 0 Q8TDU6 7bw0 TM3 86 108 1 P63092 7bw0 TM3 86 108 2 Q8TDU6 7cfm TM3 86 108
Question: I am trying to use a simple pipeline offline. I am only allowed to download files directly from the web. I went to https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english/tree/main and downloaded all the files in a local folder C:\\Users\\me\\mymodel However, when I tried to load the model I get a strange error from transformers import pipeline classifier = pipeline(task= 'sentiment-analysis', model= "C:\\Users\\me\\mymodel", tokenizer = "C:\\Users\\me\\mymodel") ValueError: unable to parse C:\Users\me\mymodel\modelcard.json as a URL or as a local path  What is the issue here?

Answer: the solution was slightly indirect:  load the model on a computer with internet access save the model with save_pretrained() transfer the folder obtained above to the offline machine and point its path in the pipeline call  The folder will contain all the expected files.
Question: For instance I have a list A: A = [100, 200, 300, 200, 400, 500, 600, 400, 700, 200, 500, 800]  And I have list B: B = [100, 200, 200, 500, 600, 200, 500]  I need to identify the index of elements in B with comparison to A I have tried: list_index = [A.index(i) for i in B]  It returns: [0, 1, 1, 5, 6, 1, 5]  But what I need is: [0, 1, 3, 5, 6, 9, 10]  How can I solve it?

Answer: You can iterate through the enumeration of A to keep track of the indices and yield the values where they match: A = [100,200,300,200,400,500,600,400,700,200,500,800] B = [100,200,200,500,600,200,500] def get_indices(A, B): a_it = enumerate(A) for n in B: for i, an in a_it: if n == an: yield i break list(get_indices(A, B)) # [0, 1, 3, 5, 6, 9, 10]  This avoids using index() multiple times.
Question: We are injecting tracing information into request headers of all the http request calls in our API client library which is implemented based on urllib3 def _init_jaeger_tracer(): '''Jaeger tracer initialization''' config = Config( config={ 'sampler': { 'type': 'const', 'param': 1, }, }, service_name="session" ) return config.new_tracer() class APIObject(rest.APIObject): '''Class for injecting traces into urllib3 request headers''' def __init__(self, configuration): print("RESTClientObject child class called####") self._tracer = None super().__init__(configuration) self._tracer = _init_jaeger_tracer() # pylint: disable=W0221 def request(self, method, url, *args, **kwargs): lower_method = method.lower() with self._tracer.start_active_span('requests.{}'.format(lower_method)) as scope: span = scope.span span.set_tag(tags.SPAN_KIND, tags.SPAN_KIND_RPC_CLIENT) span.set_tag(tags.COMPONENT, 'request') span.set_tag(tags.HTTP_METHOD, lower_method) span.set_tag(tags.HTTP_URL, url) headers = kwargs.setdefault('headers', {}) self._tracer.inject(span.context, Format.HTTP_HEADERS, headers) return r  After such implementation, urllib3 request headers look like this, headers = { 'Content-Type': 'application/json', 'User-Agent': 'API-Generator/1.0.0/python', # ** 'uber-trace-id': '30cef3e816482516:1a4fed2c4863b2f6:0:1' # ** }  Now we have to obtain trace-id from the injected request header. In python "requests" library we have "response.request.headers" attribute to return request headers In urllib3 i could not find a way return request headers for further processing Could any of you share some thoughts Now from this request headers, we need to fetch the 'uber-trace-id' and its value for further building URL.

Answer: The documentation of urllib3.response.HTTPResponse says it's:  Backwards-compatible with http.client.HTTPResponse [...]  That's stdlib class which has getheader method described as:  Return the value of the header name, or default if there is no header matching name. If there is more than one header with the name name, return all of the values joined by ', '. If default is any iterable other than a single string, its elements are similarly returned joined by commas.  Hence in your case it's something like this. import urllib3 http = urllib3.PoolManager() response = http.request('GET', 'https://python.org/') response.getheader('uber-trace-id', 'UNKNOWN')  Update urllib3 doesn't have HTTP request object representation. An HTTP request is represented in the formal arguments of RequestMethods.request . Hence, you need to store the tracing identifier yourself in your request override after it was injected by jaeger_client. Storing it as an attribute of the response object is probably a good idea: from jaeger_client.constants import TRACE_ID_HEADER class APIObject(rest.APIObject): def request(self, *args, **kwargs) -> urllib3.response.BaseHTTPResponse: ... r._request_tracing_id = headers[TRACE_ID_HEADER] return r
Question: I have the following sample data stored in Excel file     CLAIM CODE1 AGE DATE     7538 359 71 28/11/2019   7538 359 71 28/11/2019   540 428 73 16/10/2019   540 428 73 16/10/2019   605 1670 40 04/12/2019   740 134 55 24/12/2019     When importing to my Jupyter Notebook using the pandas.read_excel API, the date field does not get properly formated: excel = pd.read_excel('Libro.xlsx')   Then I am getting the DATE field different as I have it formatted in the excel file. What argument should I apply to read_excel in order to display the DATE column formatted as I have it in the excel file? .info() method, outputs the column as int64  I already tried using the pd.to_datetime function, but I am getting strange results:  Find the sample excel file I am using for my project in the following link sample_raw_data Here is some code that can be used to reproduce the DataFrame that is read in from excel: excel = pd.DataFrame({ 'CLAIM': {0: 7538, 1: 7538, 2: 540, 3: 540, 4: 4605, 5: 1740, 6: 7605}, 'CODE1': {0: 359, 1: 359, 2: 428, 3: 428, 4: 1670, 5: 134, 6: 415}, 'AGE': {0: 71, 1: 71, 2: 73, 3: 73, 4: 40, 5: 55, 6: 56}, 'DATE': {0: 43797, 1: 43797, 2: 43754, 3: 43754, 4: 43803, 5: 43823, 6: 43818} }) 

Answer: To convert this Excel Date into datetime64[ns] use to_datetime to get unit in days with offset from origin '1899-12-30': excel = pd.read_excel('Libro.xlsx') excel['DATE'] = pd.to_datetime(excel['DATE'], unit='d', origin='1899-12-30')  excel:  CLAIM CODE1 AGE DATE 0 7538 359 71 2019-11-28 1 7538 359 71 2019-11-28 2 540 428 73 2019-10-16 3 540 428 73 2019-10-16 4 4605 1670 40 2019-12-04 5 1740 134 55 2019-12-24 6 7605 415 56 2019-12-19  info:  # Column Non-Null Count Dtype --- ------ -------------- ----- 0 CLAIM 7 non-null int64 1 CODE1 7 non-null int64 2 AGE 7 non-null int64 3 DATE 7 non-null datetime64[ns]  See Why is 1899-12-30 the zero date in Access / SQL Server instead of 12/31? for more information about why this is the base date.  A converter for DATE can also be used with read_excel: excel = pd.read_excel( 'Libro.xlsx', converters={ 'DATE': lambda x: pd.to_datetime(x, unit='d', origin='1899-12-30') } )  info:  # Column Non-Null Count Dtype --- ------ -------------- ----- 0 CLAIM 7 non-null int64 1 CODE1 7 non-null int64 2 AGE 7 non-null int64 3 DATE 7 non-null datetime64[ns]
Question: I have a file that has thousands of accession numbers: and looks like this.. >NC_033829.1 Kallithea virus isolate DrosEU46_Kharkiv_2014, complete genome AGTCAGCAACGTCGATGTGGCGTACAATTTCTTGATTACATTTTTGTTCCTAACAAAATGTTGATATACT >NC_020414.2 Escherichia phage UAB_Phi78, complete genome TAGGCGTGTGTCAGGTCTCTCGGCCTCGGCCTCGCCGGGATGTCCCCATAGGGTGCCTGTGGGCGCTAGG  If want to split this to multiple files with one accession number each then I can use the following code awk -F '|' '/^>/ {F=sprintf("%s.fasta",$2); print > F;next;} {print >> F;}' < yourfile.fa  I have a file with thousands of accession numbers (aka >NC_*) and want to split it such as each files contains ~ 5000 accession numbers. Since I am new to awk/bash/python i struggle to find a neat solution Any idea or comment are appreciated

Answer: It wasn't clear from your question that an "accession number" is unique per input block (don't assume the people reading your question know anything about your domain - it's all just lines of text to us). It would have been clearer if you had phrased your question to just say you want 5000 new-line-separated blocks per output file rather than 5000 accession numbers. Having seen the answer you posted, it's now clear that this is what you should be using: awk -v RS= -v ORS='\n\n' ' (NR%5000) == 1 { close(out); out="myseq"(++n_seq)".fa" } { print > out } ' my_sequences.fa
Question: When I try to compute d_j(x), defined below, the algorithm based on Numpy results in unexpected values. I believe it has something to do with numerical precision, but I'm not sure how to solve this. The function is:  where  and  The code fails when j>10. For example, when j=16, the function d_j(x) returns wrong values from around x=1, while the expected result is a smooth, almost periodic curve. Graph for 0<x<1.5:  The code is: #%% import numpy as np import matplotlib.pyplot as plt #%% L = 1.5 # Length [m] def eta(j): if j == 1: return 1.875 / L if j > 1: return (j - 0.5) * np.pi / L def D(etaj): etajL = etaj * L return (np.cos(etajL) + np.cosh(etajL)) / (np.sin(etajL) - np.sinh(etajL)) def d(x, etaj): etajx = etaj * x return np.sin(etajx) - np.sinh(etajx) + D(etaj) * (np.cos(etajx) - np.cosh(etajx)) #%% aux = np.linspace(0, L, 2000) plt.plot(aux, d(aux, eta(16))) plt.show() 

Answer: TL;DR: The problem comes from numerical instabilities. First of all, here is a simplified code on which the exact same problem appear (with different values): x = np.arange(0, 50, 0.1) plt.plot(np.sin(x) - np.sinh(x) - np.cos(x) + np.cosh(x)) plt.show()  Here is another example where the problem does not appear: x = np.arange(0, 50, 0.1) plt.plot((np.sin(x) - np.cos(x)) + (np.cosh(x) - np.sinh(x))) plt.show()  While the two code are mathematically equivalent with real numbers, they are not equivalent because of fixed-size floating-point precision. Indeed, np.sinh(x) and np.cosh(x) result both in huge values when x is big compared to np.sin(x) and np.cos(x). Unfortunately, when two fixed-size floating-point numbers are added together, there is a loss of precision. The loss of precision can be huge (if not critical) when the order of magnitude of the added numbers are very different. For example, in Python and on a mainstream platform (so with IEEE-754 64-bit floating-point numbers), 0.1 + 1e20 == 1e20 is true due to the limited precision of the number representation. Thus (0.1 + 1e20) - 1e20 == 0.0 is also true, while 0.1 + (1e20 - 1e20) == 0.0 is not true (the resulting value is 0.1). The floating-point addition is neither associative nor commutative. In this specific case, the accuracy can reach a threshold where there is not significant number anymore in the result. For more information about floating-point precision, please read this post. The point is you should be very careful when you subtract floating-point numbers. A good solution is to put parenthesis so that added/subtracted values have the same order of magnitude. Variable-sized and higher precision help a bit too. However, the best solution is to analyses the numerical stability of your algorithm. For example, studying condition number of the numerical operations used in your algorithm is a good start. Here, a relatively good solution is just to use the second code instead of the first.
Question: I am working through "Deep Learning for Coders with fastai & Pytorch". Chapter 4 introduces the autograd function from the PyTorch library on a trivial example. x = tensor([3.,4.,10.]).requires_grad_() def f(q): return sum(q**2) y = f(x) y.backward()  My question boils down to this: the result of y = f(x) is tensor(125., grad_fn=AddBackward0), but what does that even mean? Why would I sum the values of three completely different inputs? I get that using .backward() in this case is shorthand for .backward(tensor[1.,1.,1.]) in this scenario, but I don't see how summing 3 unrelated numbers in a list helps get the gradient for anything. What am I not understanding? I'm not looking for a grad-level explanation here. The subtitle for the book I'm using is AI Applications Without a Ph.D. My experience with gradients is from school is that I should be getting a FUNCTION back, but I understand that isn't the case with Autograd. A graph of this short example would be helpful, but the ones I see online usually include too many parameters or weights and biases to be useful, my mind gets lost in the paths.

Answer: TLDR; the derivative of a sum of functions is the sum of their derivatives Let x be your input vector made of x_i (where i in [0,n]), y = x**2 and L = sum(y_i). You are looking to compute dL/dx, a vector of the same size as x whose components are the dL/dx_j (where j in [0,n]). For j in [0,n], dL/dx_j is simply dy_j/dx_j (derivative of the sum is the sum of derivates and only one of them is different to zero), which is d(x_j**2)/dx_j, i.e. 2*x_j. Therefore, dL/dx = [2*x_j where j in [0,n]]. This is the result you get in x.grad when either computing the gradient of x as: y = f(x) y.backward()  or the gradient of each components of x separately: y = x**2 y.backward(torch.ones_like(x))
Question: I'm try to create a service role for AWS CodeBuild. I can create a role like this: from aws_cdk import aws_iam as iam role = iam.Role( self, 'CodebuildServiceRole', assumed_by=iam.ServicePrincipal('codebuild.amazonaws.com'), max_session_duration=cdk.Duration.hours(1), )  Now I need to attach the Amazon-provided AWSCodeBuildAdminAccess policy to the role. How can I do this using the CDK?

Answer: You can get access to the policy like this: AWSCodeBuildAdminAccess = iam.ManagedPolicy.from_aws_managed_policy_name('AWSCodeBuildAdminAccess')  And attach it to your role like this: role.add_managed_policy(AWSCodeBuildAdminAccess)
Question: I like to merge or combine two dataframes of different size df1 and df2, based on a range of dates, for example: df1: Date Open High Low 2021-07-01 8.43 8.44 8.22 2021-07-02 8.36 8.4 8.28 2021-07-06 8.22 8.23 8.06 2021-07-07 8.1 8.19 7.98 2021-07-08 8.07 8.1 7.91 2021-07-09 7.97 8.11 7.92 2021-07-12 8 8.2 8 2021-07-13 8.15 8.18 8.06 2021-07-14 8.18 8.27 8.12 2021-07-15 8.21 8.26 8.06 2021-07-16 8.12 8.23 8.07  df2: Day of month Revenue Earnings 01 45000 4000 07 43500 5000 12 44350 6000 15 39050 7000  results should be something like this: combination: Date Open High Low Earnings 2021-07-01 8.43 8.44 8.22 4000 2021-07-02 8.36 8.4 8.28 4000 2021-07-06 8.22 8.23 8.06 4000 2021-07-07 8.1 8.19 7.98 5000 2021-07-08 8.07 8.1 7.91 5000 2021-07-09 7.97 8.11 7.92 5000 2021-07-12 8 8.2 8 6000 2021-07-13 8.15 8.18 8.06 6000 2021-07-14 8.18 8.27 8.12 6000 2021-07-15 8.21 8.26 8.06 7000 2021-07-16 8.12 8.23 8.07 7000  The Earnings column is merged based on a range of date, how can I do this in python pandas?

Answer: Try merge_asof #df1.date=pd.to_datetime(df1.date) df1['Day of month'] = df1.Date.dt.day out = pd.merge_asof(df1, df2, on ='Day of month', direction = 'backward') out Out[213]: Date Open High Low Day of month Revenue Earnings 0 2021-07-01 8.43 8.44 8.22 1 45000 4000 1 2021-07-02 8.36 8.40 8.28 2 45000 4000 2 2021-07-06 8.22 8.23 8.06 6 45000 4000 3 2021-07-07 8.10 8.19 7.98 7 43500 5000 4 2021-07-08 8.07 8.10 7.91 8 43500 5000 5 2021-07-09 7.97 8.11 7.92 9 43500 5000 6 2021-07-12 8.00 8.20 8.00 12 44350 6000 7 2021-07-13 8.15 8.18 8.06 13 44350 6000 8 2021-07-14 8.18 8.27 8.12 14 44350 6000 9 2021-07-15 8.21 8.26 8.06 15 39050 7000 10 2021-07-16 8.12 8.23 8.07 16 39050 7000
Question: I have got a dataframe like this:  part part_parent 0 part1 NaN 1 part2 part1 2 part3 part2 3 part4 part3 4 part5 part2  I need to add an additional column hierarchy like this:  part part_parent hierarchy 0 part1 NaN part1 1 part2 part1 part1/part2/ 2 part3 part2 part1/part2/part3/ 3 part4 part3 part1/part2/part3/part4 4 part5 part2 part1/part2/part5  Dict to create input/output dataframes: from numpy import nan df1 = pd.DataFrame({'part': {0: 'part1', 1: 'part2', 2: 'part3', 3: 'part4', 4: 'part5'}, 'part_parent': {0: nan, 1: 'part1', 2: 'part2', 3: 'part3', 4: 'part2'}}) df2 = pd.DataFrame({'part': {0: 'part1', 1: 'part2', 2: 'part3', 3: 'part4', 4: 'part5'}, 'part_parent': {0: nan, 1: 'part1', 2: 'part2', 3: 'part3', 4: 'part2'}, 'hierarchy': {0: 'part1', 1: 'part1/part2/', 2: 'part1/part2/part3/', 3: 'part1/part2/part3/part4', 4: 'part1/part2/part5'}})  NOTE: I've seen a couple of threads related to NetworkX to solve this issue but I'm not able to do so. Any help is appreciated.

Answer: Here is a solution using networkx. It treats nan as the root node, and finds the shortest path to each node based on that. import networkx as nx def find_path(net, source, target): # Adjust this as needed (in case multiple paths are present) # or error handling in case a path doesn't exist path = nx.shortest_path(net, source, target) return "/".join(list(path)[1:]) net = nx.from_pandas_edgelist(df1, "part", "part_parent") df1["hierarchy"] = [find_path(net, nan, node) for node in df1["part"]]    part part_parent hierarchy 0 part1 NaN part1 1 part2 part1 part1/part2 2 part3 part2 part1/part2/part3 3 part4 part3 part1/part2/part3/part4 4 part5 part2 part1/part2/part5  The formatting of the path is contrived for this example, if more robust error-handling or multiple path formatting is needed, the path finder will have to be adjusted.
Question: I have a Pandas Dataframe : A || B || C x1 x [x,y] x2 a [b,c,d]  and I am trying to make a dictionary to that looks like: {x1: {B : x, c : [x,y]}, x2: {B: a, C:[b,c,d}}  I have tried the to_dict function but that changes the entire dataframe into a dictionary. I am kind of lost on how to iterate onto the first column and make it the key and the rest of the df a dictionary as the value of that key.

Answer: Try: x = df.set_index("A").to_dict("index") print(x)  Prints: {'x1': {'B': 'x', 'C': ['x', 'y']}, 'x2': {'B': 'a', 'C': ['b', 'c', 'd']}}