Title: Pyrthon script to calculate two rows together from same column based on a match between same rows in two different columns
Tags: python,pandas,calculation

Question: I want to create a Python script to calculate a new column, based on subtracting two values from same column in two different rows.
The two rows used for the calculation should be defined by being a match in values of two other columns. So, to specify and give an example:




Id
Tag
Amount




1
2
3.75


2
xxx
15


3
4
4


4
xxx
14


5
6
5


6
xxx
15.5




The above table is an example of what I have right now.
The below table is including the column that I would like to create. For me, it does not matter if 'NaN or 0' is in the specified row or the row afterwards:




Id
Tag
Amount
NewColumn




1
2
3.75
NaN or 0 or simply the value from Amount


2
xxx
15
11.25


3
4
4
NaN or 0 or simply the value from Amount


4
xxx
14
10


5
6
5
NaN or 0 or simply the value from Amount


6
xxx
15.5
10.5




So here, the value of NewColumn in the second row is equal to 11.25, because the following conditions are met:

The value of the column 'Id' is equal to the value in the column 'Tag'.
Therefore, the NewColumn should take the value of the column 'Amount' in row the bigger number and subtract it by the value in the row with the smaller number.
This means that the calculation is 15-3.75 = 11.25.

To give some context, the value in 'Amount' in row 2 is with VAT included. The value in the row before of the same column is the VAT by itself. The Id is the Transaction ID, and the Tag column are used to link together the VAT transaction the correct corresponding full transaction.
I have tried to use ChatGPT to solve this issue, but can not seem to fully solve it. Here is what I have so far:
import pandas as pd

# Load the dataset into a pandas dataframe
df = pd.read_csv('path/to/dataset.csv')

# Define the name of the column to fetch data from
other_column_name = 'other_column_name'

# Iterate over each row in the dataframe
for index, row in df.iterrows():
    # Fetch data from another row and column based on an exact match
    search_value = row['column_name']
    matching_row = df.loc[df['column_name'] == search_value]
    if len(matching_row) == 1:
        other_column_data = matching_row[other_column_name].values[0]
    else:
        other_column_data = None

    # Use the fetched data to calculate a new column
    if other_column_data is not None:
        new_column_data = row['existing_column'] + other_column_data
    else:
        new_column_data = None

    # Add the new column to the dataframe
    if new_column_data is not None:
        df.at[index, 'new_column'] = new_column_data

# Save the updated dataset to a new CSV file
df.to_csv('path/to/new_dataset.csv', index=False)

Which simply outputs a combination of the values in Tag and Id.

Answer: Since I am unable to edit my question, I would like to contribute with this update to make my second table readable.




Id
Tag
Amount
NewColumn




1
2
3.75
NaN or 0 or simply the value from Amount


2
xxx
15
11.25


3
4
4
NaN or 0 or simply the value from Amount


4
xxx
14
10


5
6
5
NaN or 0 or simply the value from Amount


6
xxx
15.5
10.5




I should also add, that I can not simply apply a singular VAT percent rate for these transactions, as the transactions differ in their VAT.
Also, it is not to be expected a "perfect" relationship here where the corresponding rows will be right after each other.