Bug: ProxyUserAuthenticationFilter add properties 'hadoop.security.impersonation.provider.class'  to enable  load custom ImpersonationProvider class when start namenode
Status: Open | Priority: Major | Resolution: 
Created: 16/Jan/23 10:06

h3. h3.  the phenomenon

I made a custom  ImpersonationProvider class and configured in core-site.xml
{code:none}
    <property>
      <name>hadoop.security.impersonation.provider.class</name>
      <value>org.apache.hadoop.security.authorize.MyImpersonationProvider</value>
    </property>
{code}
 

{color:#ff0000}However, when  start namenode, MyImpersonationProvider could't be load automatically, but DefaultImpersonationProvider is loaded.{color}

When execute the following command, custom ImpersonationProvider could be load.
{code:java}
bin/hdfs dfsadmin -refreshSuperUserGroupsConfiguration{code}
h3. h3. what I see else

custom ImpersonationProvider was load in org.apache.hadoop.security.authorize.ProxyUsers#refreshSuperUserGroupsConfiguration
through the property "hadoop.security.impersonation.provider.class"

[https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ProxyUsers.java#L70]
{code:java}
public static void refreshSuperUserGroupsConfiguration(Configuration conf,
    String proxyUserPrefix) {
  Preconditions.checkArgument(proxyUserPrefix != null && 
      !proxyUserPrefix.isEmpty(), "prefix cannot be NULL or empty");
  // sip is volatile. Any assignment to it as well as the object's state
  // will be visible to all the other threads. 
  ImpersonationProvider ip = getInstance(conf);
  ip.init(proxyUserPrefix);
  sip = ip;
  ProxyServers.refresh(conf);
} 


private static ImpersonationProvider getInstance(Configuration conf) {
  Class<? extends ImpersonationProvider> clazz =
      conf.getClass(
          CommonConfigurationKeysPublic.HADOOP_SECURITY_IMPERSONATION_PROVIDER_CLASS,
          DefaultImpersonationProvider.class, ImpersonationProvider.class);
  return ReflectionUtils.newInstance(clazz, conf);
}{code}
 

when namenode start, refreshSuperUserGroupsConfiguration was called in ProxyUserAuthenticationFilter,

[https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilter.java#L56]
{code:java}
  public void init(FilterConfig filterConfig) throws ServletException {
    Configuration conf = getProxyuserConfiguration(filterConfig);
    ProxyUsers.refreshSuperUserGroupsConfiguration(conf, PROXYUSER_PREFIX);
    super.init(filterConfig);
  }
{code}
here is the stack trace
{code:none}
init:70, DefaultImpersonationProvider (org.apache.hadoop.security.authorize)
refreshSuperUserGroupsConfiguration:77, ProxyUsers (org.apache.hadoop.security.authorize)
init:56, ProxyUserAuthenticationFilter (org.apache.hadoop.security.authentication.server)
initialize:140, FilterHolder (org.eclipse.jetty.servlet)
lambda$initialize$0:731, ServletHandler (org.eclipse.jetty.servlet)
accept:-1, 1541075662 (org.eclipse.jetty.servlet.ServletHandler$$Lambda$36)
forEachRemaining:948, Spliterators$ArraySpliterator (java.util)
forEachRemaining:742, Streams$ConcatSpliterator (java.util.stream)
forEach:580, ReferencePipeline$Head (java.util.stream)
initialize:755, ServletHandler (org.eclipse.jetty.servlet)
startContext:379, ServletContextHandler (org.eclipse.jetty.servlet)
doStart:910, ContextHandler (org.eclipse.jetty.server.handler)
doStart:288, ServletContextHandler (org.eclipse.jetty.servlet)
start:73, AbstractLifeCycle (org.eclipse.jetty.util.component)
start:169, ContainerLifeCycle (org.eclipse.jetty.util.component)
doStart:117, ContainerLifeCycle (org.eclipse.jetty.util.component)
doStart:97, AbstractHandler (org.eclipse.jetty.server.handler)
start:73, AbstractLifeCycle (org.eclipse.jetty.util.component)
start:169, ContainerLifeCycle (org.eclipse.jetty.util.component)
doStart:117, ContainerLifeCycle (org.eclipse.jetty.util.component)
doStart:97, AbstractHandler (org.eclipse.jetty.server.handler)
start:73, AbstractLifeCycle (org.eclipse.jetty.util.component)
start:169, ContainerLifeCycle (org.eclipse.jetty.util.component)
start:423, Server (org.eclipse.jetty.server)
doStart:110, ContainerLifeCycle (org.eclipse.jetty.util.component)
doStart:97, AbstractHandler (org.eclipse.jetty.server.handler)
doStart:387, Server (org.eclipse.jetty.server)
start:73, AbstractLifeCycle (org.eclipse.jetty.util.component)
start:1276, HttpServer2 (org.apache.hadoop.http)
start:170, NameNodeHttpServer (org.apache.hadoop.hdfs.server.namenode)
startHttpServer:954, NameNode (org.apache.hadoop.hdfs.server.namenode)
initialize:765, NameNode (org.apache.hadoop.hdfs.server.namenode)
<init>:1020, NameNode (org.apache.hadoop.hdfs.server.namenode)
<init>:995, NameNode (org.apache.hadoop.hdfs.server.namenode)
createNameNode:1769, NameNode (org.apache.hadoop.hdfs.server.namenode)
main:1834, NameNode (org.apache.hadoop.hdfs.server.namenode)
{code}
 
{color:#ff0000}but the filterConfig in ProxyUserAuthenticationFilter did't contains properties ''hadoop.security.impersonation.provider.class''{color}
filterConfig in ProxyUserAuthenticationFilter is controled by ProxyUserAuthenticationFilterInitializer or AuthFilterInitializer
filterConfig only put property which start with "hadoop.proxyuser", but not put "hadoop.security.impersonation.provider.class"
{code:java}
  protected Map<String, String> createFilterConfig(Configuration conf) {
    Map<String, String> filterConfig = AuthenticationFilterInitializer
        .getFilterConfigMap(conf, configPrefix);
    //Add proxy user configs
    for (Map.Entry<String, String> entry : conf.getPropsWithPrefix(
        ProxyUsers.CONF_HADOOP_PROXYUSER).entrySet()) {
      filterConfig.put("proxyuser" + entry.getKey(), entry.getValue());
    }
    return filterConfig;
  }
{code}
[https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authentication/server/ProxyUserAuthenticationFilterInitializer.java#L46]

[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/AuthFilterInitializer.java#L46]

it leads to custome ImpersonationProvider can't be load during namenode start.