Add an average callable parameter to sklearn.neighbors.KNeighborsRegressor

### Describe the workflow you want to enable

For some algorithms of kNN it is desired to use other calculations at the end of this method besides using the mean. In some cases using a geometric average or arithmetic-geometric average is needed instead of the regular average.

### Describe your proposed solution

Add a parameter to the KNeighborsRegressor class, for computing the average at the end of the predict() method - something like:

```
# Use the default way to compute the average with a new parameter and string value of 'mean' which would be the default

KNeighborsRegressor(n_neighbors=5, metric=manhattan_distance, weights='uniform', average='mean')
```


```
# Allow callables for the new average parameter...

def geometric_mean(l):
    k=1
    for i in l:
        k*=i
        
    return pow(k, 1/len(l))

KNeighborsRegressor(n_neighbors=5, metric=manhattan_distance, weights='uniform', average=geometric_mean)





# Allow callables for the new average parameter...

def arithmetic_geometric_mean(l):
    tolerance=1e-10
    a0 = mean(l)
    g0 = geometric_mean(l)
    
    an, gn = (a0 + g0) / 2.0, math.sqrt(a0 * g0)
    while abs(an - gn) > tolerance:
        an, gn = (an + gn) / 2.0, math.sqrt(an * gn)
    
    return an

KNeighborsRegressor(n_neighbors=5, metric=manhattan_distance, weights='uniform', average=arithmetic_geometric_mean)
```

### Describe alternatives you've considered, if relevant

Not use scikit-learn's KNeighborsRegressor and build my own:

```
#
# Here avg_func is the callable method to compute the average
#
def kNN_regressor(q, R, truth, k, avg_func=my_mean, distance_func=euclidean_distance, **kwargs):
    idx = [0]*k
    dist = [0]*k
    
    for i in range(k):
        idx[i] = i
        if 'minkowski_param' in kwargs:
            mink_p = kwargs['minkowski_param']
            dist[i] = distance_func(q, R[i], mink_p)
        else:
            dist[i] = distance_func(q, R[i])
        
    max_idx = dist.index(max(dist))
        
    for i in range(k, len(R)):
        if 'minkowski_param' in kwargs:
            mink_p = kwargs['minkowski_param']
            d = distance_func(q, R[i], mink_p)
        else:
            d = distance_func(q, R[i])

        if d < dist[max_idx]:
            dist[max_idx] = d
            idx[max_idx] = i
            max_idx = dist.index(max(dist))
    
    t = []
    for i in range(k):
        t.append(truth[idx[i]])
        
    return avg_func(t)
```

### Additional context

_No response_

COMMENT:
One reference is here: https://aip.scitation.org/doi/10.1063/1.5011683
[https://aip.scitation.org/action/showOpenGraphArticleImage?doi=10.1063/1.5011683&id=images/medium/1.5011683.figures.online.highlight_f1.jpg]<https://aip.scitation.org/doi/10.1063/1.5011683>
Geometric k-nearest neighbor estimation of entropy and mutual information: Chaos: An Interdisciplinary Journal of Nonlinear Science: Vol 28, No 3 - Scitation<https://aip.scitation.org/doi/10.1063/1.5011683>
Nonparametric estimation of mutual information is used in a wide range of scientific problems to quantify dependence between variables. The k-nearest neighbor (knn) methods are consistent, and therefore expected to work well for a large sample size.These methods use geometrically regular local volume elements.
aip.scitation.org

________________________________
From: Thomas J. Fan ***@***.***>
Sent: Monday, February 28, 2022 10:31 AM
To: scikit-learn/scikit-learn ***@***.***>
Cc: Jennings, Mr. Ray, III ***@***.***>; Mention ***@***.***>
Subject: Re: [scikit-learn/scikit-learn] Add an average callable parameter to sklearn.neighbors.KNeighborsRegressor (Issue #22626)


@leo-ware<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fleo-ware&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pnbueqdpjey7bwu1G2724G9uaZajG%2FgyjRbtJhbqDUI%3D&reserved=0> Maintainers need to decide if the feature should be included first. This issue<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fscikit-learn%2Fscikit-learn%2Fissues%2F21350&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=q39MGAfCfGNGiGkcIHapiHF08v77DaJCLmcK3nDYZmo%3D&reserved=0> is a good one to get started with.

@raymondj-pace<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fraymondj-pace&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iXVg0sZuLGCLCA4Uv%2BuogAVRLfM0tQjK4B%2B%2Frl%2Bi7d0%3D&reserved=0> Do you have references or papers where they show it is useful to perform different averaging strategies for KNN? We need to make sure the new feature fits scikit-learn's inclusion criterion<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fscikit-learn.org%2Fstable%2Ffaq.html%23what-are-the-inclusion-criteria-for-new-algorithms&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DfUihu%2B5KlA2PSso7RClm5lOXWrGijRgilbXn83JUrw%3D&reserved=0>

—
Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fscikit-learn%2Fscikit-learn%2Fissues%2F22626%23issuecomment-1054375090&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5K%2BSLk8yK90uDCWIl9fvitjlhjFs%2BKWsHYaCz%2FY8fmA%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWSHG5U3SFLLLGD3RGNNZGLU5OIMFANCNFSM5PPOZOIA&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WSvq38doWDIleJ%2F%2FDv1fg1JRFxe5%2FlL7s4aAkbkJTjg%3D&reserved=0>.
Triage notifications on the go with GitHub Mobile for iOS<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2FPd8N%2Fz2gxA2EDvLzR4EKf3xJbTcCtFH4eykPwUCZrA%3D&reserved=0> or Android<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7C9c1d84bb34534c7e907408d9facf5b83%7C0799c53eca9a49e88901064a6412a41d%7C0%7C0%7C637816590768356726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mMplaAtxxg30YNX3MGxZmlg%2BUI8LncRMRy4jmWH1ryQ%3D&reserved=0>.
You are receiving this because you were mentioned.Message ID: ***@***.***>

Labels: New Feature, module:neighbors, Needs Decision - Include Feature