Question: Suppose I have sample = {"Location+Type": ["A", "A", "A", "A", "B", "B", "B", "B"], "Year": ["2010", "2011", "2012", "2013", "2010", "2011", "2012", "2013"], "Price": [1, 2, 1, 3, 1, 1, 1, 1]} df_sample = pd.DataFrame(data=sample) df_sample['pct_pop'] = df_sample['Price'].pct_change() df_sample.head(8)  From say 2012, I want to make a change in the dataframes values following the pct_chage() values. In other words, if the percentage change is greater than 30 percent than I want to average the previous values with respect to the Location+Type column starting from the year of 2012. Thus the new dataframe would look like this sample = {"Location+Type": ["A", "A", "A", "A", "B", "B", "B", "B"], "Year": ["2010", "2011", "2012", "2013", "2010", "2011", "2012", "2013"], "Price": [1, 2, 1.5, 1.33, 1, 1, 1, 1]} df_sample = pd.DataFrame(data=sample) df_sample['pct_pop'] = df_sample['Price'].pct_change() df_sample.head(8)  Still having some issues following the answers for my particular data which is instead of changing from 2012 its 2023, here is the shared drive https://drive.google.com/drive/u/0/folders/11dh05Dq_VNfUNgALeMg7MI1keVQdN0ZN And here is the code modified df_complete['Year'] = df_complete['Year'].astype(int) # just in case if "Year" holds strings def calculate_new_price(row, df): current_index = row.name # row number will be used to check if value is in the first of the DF # Check if row is not the first if current_index > 0: # assign previous pct_pop value to variable: previous_pct_pop previous_pct_pop = df.loc[current_index - 1, 'pct_pop'] # Check if the year is 2023 or later and if the difference in pct_pop is 0.3 or more if row['Year'] >= 2023 and abs(row['pct_pop'] - previous_pct_pop) >= 0.3: # Filter previous years previous_data = df[(df['Year'] < row['Year']) & (df['Location+Type'] == row['Location+Type'])] # Calculate average if row above is not empty if not previous_data.empty: return previous_data['Median_Home_Value_prediction'].mean() else: return row['Median_Home_Value_prediction'] # if pct_pop change was smaller than 0.3 or year is older than 2012, - use default price. else: return row['Median_Home_Value_prediction'] # if index 0 (first row) use value from first row of col: Price return None # Apply the function to create the new_price column df_complete['Median_Home_Value_prediction_new'] = df_complete.apply(lambda row: calculate_new_price(row, df_complete), axis=1) 

Answer: Pandas Style We have 4 steps here that can be done using typical pandas tools:  Sort data. Find where the percent change exceeds the limit. Calculate cumulative means. Replace the data with average values by masking.  # Load data and define key variables data = pd.DataFrame({ 'Location+Type': ['A', 'A', 'A', 'A', 'B', 'B', 'B', 'B'], 'Year': [2010, 2011, 2012, 2013, 2010, 2011, 2012, 2013], 'Price': [1, 2, 1, 3, 1, 1, 1, 1], }) group = 'Location+Type' inner_index = 'Year' value = 'Price' index_start = 2012 # apply to index >= this one value_pct_change_upper_limit = 0.3 # update values where percent change >= this one # Sort and group data df = data.sort_values([group, inner_index]) by_groups = df.groupby(group, as_index=False) # Find where abs(percent change) is greater than or equal to change limit pct_change_ge_limit = (by_groups[value].pct_change() .abs().ge(value_pct_change_upper_limit) .rename('pct_change')) # Find cumulative mean by groups shifted one step forward # to the place where they can be used cum_mean = (by_groups[value].expanding().mean() .groupby(level=0).shift().bfill() # shift the means to replace the values following them .reset_index(level=0, drop=True) # drop group names returend at the level 0 .rename('cum_mean')) # Replace values with cumulative means # where inner group index (Year) starts from a given value # and abs(percent change) >= percent change limit df[value] = df[value].mask( df[inner_index].ge(index_start) & pct_change_ge_limit, other=cum_mean )  NumPy/Numba Acceleration To make it faster, we can calculate whatever is needed on the fly with help of NumPy and Numba with some modifications in the algorithm:  Sort data by group (e.g. Location+Type) and inner group index (e.g. Year). Find group start/end points. Iterate over every group starting from the end until the group index limit. Calculate the average only if needed.  import numpy as np from numba import njit def get_bounds(series): '''Return the starting points of consecutive values and put `len(series)` at the end, where `series` is a properly sorted `pandas.Series` ''' grouper = series.ne(series.shift()) size = grouper.sum() + 1 bounds = np.empty(size, dtype=int) bounds[:-1] = np.nonzero(grouper)[0] bounds[-1] = len(series) return bounds @njit def modify_excessive_values(bounds, years, values, year_limit, pct_limit): '''Replace the values with averages of their predecessors if their percent change exceeds pct_limit and the corresponding year is more or equal the year_limit ''' out = values.copy().astype('float') # use float to avoid truncating means for integer values for left, right in zip(bounds[:-1], bounds[1:]): # iterate over groups ind = right - 1 while years[ind] >= year_limit and ind > left: # iterate inside the group starting from its end if abs(out[ind]/out[ind-1] - 1) >= pct_limit: out[ind] = out[left:ind].mean() ind -= 1 return out  Working with the original data Here's an example of the original data: csv_data = '''\ Location+Type,Year,Tract_number,Median_Home_Value_prediction,pct_pop "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2015,5069000102,112000,0.3365155131264916 "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2016,5069000102,117811,0.0518839285714285 "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2017,5069000102,122031,0.0358200847119538 "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2018,5069000102,127031,0.0409731953356113 "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2019,5069000102,132779,0.045248797537609 "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2020,5069000102,145703,0.0973346688858931 "Census Tract 1.02, Jefferson County, Arkansas: Summary level: 140, state:05> county:069> tract:000102",2021,5069000102,166769,0.1445817862363849 "Census Tract 1.02; Jefferson County; Arkansas: Summary level: 140, state:05> county:069> tract:000102",2023,5069000102,168596,0.594531611608196 "Census Tract 1.02; Jefferson County; Arkansas: Summary level: 140, state:05> county:069> tract:000102",2024,5069000102,213385,0.2656597148863364 "Census Tract 1.02; Jefferson County; Arkansas: Summary level: 140, state:05> county:069> tract:000102",2025,5069000102,176771,-0.1715863204676784 "Census Tract 1.02; Jefferson County; Arkansas: Summary level: 140, state:05> county:069> tract:000102",2026,5069000102,181905,0.0290384486578858 "Census Tract 1.02; Jefferson County; Arkansas: Summary level: 140, state:05> county:069> tract:000102",2027,5069000102,291646,0.6032906623467567 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2010,18003003400,86600,-0.7280150753768844 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2011,18003003400,85500,-0.0127020785219399 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2012,18003003400,85900,0.0046783625730995 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2013,18003003400,85400,-0.0058207217694994 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2014,18003003400,83900,-0.0175644028103044 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2015,18003003400,83200,-0.0083432657926102 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2016,18003003400,87700,0.0540865384615385 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2017,18003003400,90800,0.0353477765108323 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2018,18003003400,100400,0.1057268722466959 "Census Tract 34.00, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2019,18003003400,103100,0.0268924302788844 "Census Tract 34, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2020,18003003400,111500,-0.3719003776540518 "Census Tract 34, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2021,18003003400,117700,0.0556053811659191 "Census Tract 34, Allen County, Indiana: Summary level: 140, state:18> county:003> tract:003400",2022,18003003400,132200,0.1231945624468988 "Census Tract 34; Allen County; Indiana: Summary level: 140, state:18> county:003> tract:003400",2023,18003003400,186965,-0.4049759667529261 "Census Tract 34; Allen County; Indiana: Summary level: 140, state:18> county:003> tract:003400",2024,18003003400,209587,0.1209947984468349 "Census Tract 34; Allen County; Indiana: Summary level: 140, state:18> county:003> tract:003400",2025,18003003400,186934,-0.1080818221957703 "Census Tract 34; Allen County; Indiana: Summary level: 140, state:18> county:003> tract:003400",2026,18003003400,194739,0.0417513977349723 "Census Tract 34; Allen County; Indiana: Summary level: 140, state:18> county:003> tract:003400",2027,18003003400,200658,0.0303972448096778 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2010,26049003400,113600,-0.2685125563425627 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2011,26049003400,173800,0.5299295774647887 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2012,26049003400,164000,-0.0563866513233601 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2013,26049003400,115300,-0.2969512195121951 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2014,26049003400,50000,-0.5663486556808326 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2016,26049003400,13900,-0.722 "Census Tract 34, Genesee County, Michigan: Summary level: 140, state:26> county:049> tract:003400",2022,26049003400,9999,-0.2806474820143885 "Census Tract 34; Genesee County; Michigan: Summary level: 140, state:26> county:049> tract:003400",2023,26049003400,64772,-0.5894961662501869 "Census Tract 34; Genesee County; Michigan: Summary level: 140, state:26> county:049> tract:003400",2025,26049003400,89406,0.3803308197295025 "Census Tract 34; Genesee County; Michigan: Summary level: 140, state:26> county:049> tract:003400",2026,26049003400,61398,-0.3132673555695764 "Census Tract 34; Genesee County; Michigan: Summary level: 140, state:26> county:049> tract:003400",2027,26049003400,37238,-0.3935019397012603 "Census Tract 113.02, Jefferson County, Texas: Summary level: 140, state:48> county:245> tract:011302",2018,48245011302,154081,0.4454127579737335 "Census Tract 113.02, Jefferson County, Texas: Summary level: 140, state:48> county:245> tract:011302",2019,48245011302,160861,0.0440028296804926 '''  There you can find all the records for 4 selected Tract_number values representing some oddities of data:  Location+Type can have up to 3 variants for one Tract_number, which complicate grouping data by them. Especially note the difference in the use of commas and semicolons, which can be difficult to pick out visually. So maybe it's better to use Tract_number for grouping, not Location+Type. There are missing years in the middle, making pct_change difficult to interpret (e.g. missing 2022 where Tract_number is 5069000102). Far not all years are represented for Tract_number, sometimes there are only 2 records in a group. The minimum year found is 2010. However, the initial year in the group may be different from 2010, which may complicate the interpretation of the mean values. All of the above oddities are not rare exceptions, but are widely represented in the data (e.g. 7% of all Tract_number have missing years in the middle).  Anyway, let's do the calculations on the data example with 2020 as a starting year and grouping the data by Tract_number: import pandas as pd from io import StringIO # Load data and define key variables file = StringIO(csv_data) df = pd.read_csv(file) group = 'Tract_number' inner_index = 'Year' value = 'Median_Home_Value_prediction' index_start = 2020 # inner_index >= index_start pct_change_upper_limit = 0.3 # pct_change >= pct_change_upper_limit # Modify data df.sort_values([group, inner_index], inplace=True) df['pct_change'] = df.groupby(group)[value].pct_change() df[f'New_{value}'] = modify_excessive_values( get_bounds(df[group]), df[inner_index].values, df[value].values, index_start, pct_change_upper_limit ) result = df[[group, inner_index, value, f'New_{value}', 'pct_change']] result.columns = ['Tract', 'Year', 'Value', 'New_Value', 'pct_change']  >>> print(result) Tract Year Value New_Value pct_change 0 5069000102 2015 112000 112000.000000 NaN 1 5069000102 2016 117811 117811.000000 0.051884 2 5069000102 2017 122031 122031.000000 0.035820 3 5069000102 2018 127031 127031.000000 0.040973 4 5069000102 2019 132779 132779.000000 0.045249 5 5069000102 2020 145703 145703.000000 0.097335 6 5069000102 2021 166769 166769.000000 0.144582 7 5069000102 2023 168596 168596.000000 0.010955 8 5069000102 2024 213385 213385.000000 0.265659 9 5069000102 2025 176771 176771.000000 -0.171587 10 5069000102 2026 181905 181905.000000 0.029043 11 5069000102 2027 291646 151343.727273 0.603287 12 18003003400 2010 86600 86600.000000 NaN 13 18003003400 2011 85500 85500.000000 -0.012702 14 18003003400 2012 85900 85900.000000 0.004678 15 18003003400 2013 85400 85400.000000 -0.005821 16 18003003400 2014 83900 83900.000000 -0.017564 17 18003003400 2015 83200 83200.000000 -0.008343 18 18003003400 2016 87700 87700.000000 0.054087 19 18003003400 2017 90800 90800.000000 0.035348 20 18003003400 2018 100400 100400.000000 0.105727 21 18003003400 2019 103100 103100.000000 0.026892 22 18003003400 2020 111500 111500.000000 0.081474 23 18003003400 2021 117700 117700.000000 0.055605 24 18003003400 2022 132200 132200.000000 0.123195 25 18003003400 2023 186965 96453.846154 0.414259 26 18003003400 2024 209587 209587.000000 0.120996 27 18003003400 2025 186934 186934.000000 -0.108084 28 18003003400 2026 194739 194739.000000 0.041753 29 18003003400 2027 200658 200658.000000 0.030395 30 26049003400 2010 113600 113600.000000 NaN 31 26049003400 2011 173800 173800.000000 0.529930 32 26049003400 2012 164000 164000.000000 -0.056387 33 26049003400 2013 115300 115300.000000 -0.296951 34 26049003400 2014 50000 50000.000000 -0.566349 35 26049003400 2016 13900 13900.000000 -0.722000 36 26049003400 2022 9999 9999.000000 -0.280647 37 26049003400 2023 64772 91514.142857 5.477848 38 26049003400 2025 89406 88171.375000 0.380319 39 26049003400 2026 61398 88308.555556 -0.313268 40 26049003400 2027 37238 85617.500000 -0.393498 41 48245011302 2018 154081 154081.000000 NaN 42 48245011302 2019 160861 160861.000000 0.044003