Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.0.20.205.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block. | Major | hdfs-client, namenode | Ruyue Ma | Cosmin Lehene | | | Append 0.20: New semantics for recoverLease | Major | . | Hairong Kuang | Hairong Kuang | | | Changes to balancer bandwidth should not require datanode restart. | Major | balancer & mover, datanode | Eric Payne | Eric Payne | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | In HDFS, sync() not yet guarantees data available to the new readers | Blocker | . | Tsz Wo Nicholas Sze | dhruba borthakur | | | HDFS 20 append: Lightweight NameNode operation to trigger lease recovery | Major | namenode | Hairong Kuang | Hairong Kuang | | | Support HTTP REST in HttpServer | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | | | Make RPC to have an option to timeout | Major | ipc | Hairong Kuang | John George | | | Backport MAPREDUCE-220 to Hadoop 20 security branch | Major | . | Jonathan Eagles | Amar Kamat | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Allow a mechanism for an application to detect that datanode(s) have died in the write pipeline | Major | hdfs-client | dhruba borthakur | dhruba borthakur | | | Remove unnecessary sleep after failure in nextBlockOutputStream | Major | hdfs-client | Todd Lipcon | Todd Lipcon | | | DFSClient should log exception when block recovery fails | Trivial | hdfs-client | Todd Lipcon | Todd Lipcon | | | 0.20 append: Block receiver should not log "rewind" packets at INFO level | Minor | datanode | Todd Lipcon | Todd Lipcon | | | Allow hflush/sync to occur in parallel with new writes to the file | Major | hdfs-client | dhruba borthakur | Todd Lipcon | | | HDFS 20 append: Disallow pipeline recovery if a file is already being lease recovered | Major | . | Hairong Kuang | Hairong Kuang | | | backport HADOOP-7008 and HADOOP-7042 to branch-0.20-security | Minor | test | Thomas Graves | Thomas Graves | | | Back-port HADOOP-7110 to 0.20-security | Major | . | Sherry Chen | Sherry Chen | | | Add support for throwing UnknownHostException when a host doesn't resolve | Major | . | Jeffrey Naisbitt | Jeffrey Naisbitt | | | Make the distributed cache delete entires using LRU priority | Major | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | | | Reducers are always counted having "pending tasks" even if they can't be scheduled yet because not enough of their mappers have completed | Major | . | Sherry Chen | Sherry Chen | | | RPC client should deal with the IP address changes | Minor | ipc | Kihwal Lee | Kihwal Lee | | | Backport trunk fairscheduler to

branch | Major | contrib/fair-share | Matei Zaharia | Matei Zaharia | | | improve config generator to allow users to specify proxy user, turn append on or off, turn webhdfs on or off | Major | conf | Arpit Gupta | Arpit Gupta | | | improve the hadoop-setup-conf.sh to read in the hbase user and setup the configs | Major | conf | Arpit Gupta | Arpit Gupta | | | provide a small validation script that smoke tests the installed cluster | Major | . | Arpit Gupta | Arpit Gupta | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | ConcurrentModificationException in invalidateCorruptReplicas() | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | | | Pipeline close hangs if one of the datanode is not responsive. | Blocker | datanode, hdfs-client | Tsz Wo Nicholas Sze | Hairong Kuang | | | NetUtils.connect should check that it hasn't connected a socket to itself | Major | util | Todd Lipcon | Todd Lipcon | | | In 0.20, move blocks being written into a blocksBeingWritten directory | Blocker | . | Raghu Angadi | dhruba borthakur | | | completeFile does not check lease ownership | Blocker | namenode | Todd Lipcon | Todd Lipcon | | | 0.20-append: stallReplicationWork should be volatile | Major | namenode | Todd Lipcon | Todd Lipcon | | | 0.20: Lease expiration should recover single files, not entire lease holder | Major | . | Todd Lipcon | sam rash | | | DFSOutputStream socket leak when cannot connect to DataNode | Major | . | Zheng Shao | Zheng Shao | | | DataBlockScanner throws NPE when updated before initialized | Major | datanode | Todd Lipcon | Todd Lipcon | | | IPC leaks call parameters when exceptions thrown | Blocker | . | Todd Lipcon | Todd Lipcon | | | DFSClient receives out of order packet ack | Major | datanode, hdfs-client | Hairong Kuang | Hairong Kuang | | | TestHdfsProxy is failing | Major | contrib/hdfsproxy | Eli Collins | Todd Lipcon | | | saveNamespace race can corrupt the edits log | Blocker | namenode | dhruba borthakur | Eli Collins | | | Thousand of CLOSE\_WAIT socket | Major | hdfs-client | Dennis Cheung | Bharath Mundlapudi | | | HdfsProxyTests fails when the -Dtest.build.dir and -Dbuild.test is set | Major | build | Giridharan Kesavan | Giridharan Kesavan | | | back-port MAPREDUCE-2238 to 0.20-security | Major | . | Sherry Chen | Sherry Chen | | | Job should fail if a reduce task can't be scheduled anywhere | Major | . | Todd Lipcon | Robert Joseph Evans | | | map tasks timeout during sorting | Major | . | Gianmarco De Francisci Morales | Anupam Seth | | | Bug in INodeDirectory#computeContentSummary warning | Minor | namenode | Michael Noll | Michael Noll | | | DiskChecker#mkdirsWithExistsAndPermissionCheck may return true even when the dir is not created | Minor | datanode | Eli Collins | Eli Collins | | | tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed | Major | tasktracker | Thomas Graves | Thomas Graves | | | NN fails to start if it encounters an empty or malformed fstime file | Major | namenode | Aaron T. Myers | Aaron

Myers | | | Jobsplits with random hostnames can make the queue unusable | Major | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | | | DN web-UI doesn't work with paths that contain html | Minor | datanode | Eli Collins | Eli Collins | | | Jira for YDH bug 2854624 | Major | tasktracker | Eli Collins | Kihwal Lee | | | merge hadoop archive goodness from trunk to .20 | Major | . | John George | John George | | | Enable jsvc to work with Hadoop RPM package | Major | build | Eric Yang | Eric Yang | | | TestRenameWhileOpen fails in branch-0.20-security | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | | | Make merged protocol changes from 0.20-append to 0.20-security compatible with previous releases. | Major | datanode, hdfs-client, namenode | Suresh Srinivas | Suresh Srinivas | | | Inconsistent API JobClient.getQueueAclsForCurrentUser | Major | client | Joep Rottinghuis | Joep Rottinghuis | | | LinuxTaskController does not work when JniBasedUnixGroupsNetgroupMapping or JniBasedUnixGroupsMapping is enabled | Major | task-controller | Kihwal Lee | Kihwal Lee | | | /etc/profile.d does not exist on Debian | Major | scripts | Eric Yang | Eric Yang | | | Improve hadoop setup conf script to setup secure Hadoop cluster | Major | scripts | Eric Yang | Eric Yang | | | Allow overwrite of HADOOP\CLASSPATH and HADOOP\OPTS | Major | scripts | Eric Yang | Eric Yang | | | Potential resource leaks in HadoopServer.java, RunOnHadoopWizard.java and Environment.java | Major | contrib/eclipse-plugin, contrib/streaming | Devaraj K | Devaraj K | | | hftp throws NPE if security is not enabled on remote cluster | Critical | . | Daryn Sharp | Owen O'Malley | | | After NameNode restart , Clients can not read partial files even after client invokes Sync. | Major | datanode, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | | | Hdfs compilation fails | Major | hdfs-client | Abhijit Suresh Shingate | Abhijit Suresh Shingate | | | HDFS-2284 introduced 2 findbugs warnings on trunk | Major | . | Ivan Kelly | Tsz Wo Nicholas Sze | | | TestDelegationToken is failing in 205 | Major | . | Owen O'Malley | Owen O'Malley | | | Fair scheduler configuration file is not bundled in RPM | Major | build | Eric Yang | Eric Yang | | | In mapred-site.xml, stream.tmpdir is mapped to ${mapred.temp.dir} which is undeclared. | Major | conf | Ramya Sunil | Eric Yang | | | hadoop-metrics2.properties should have a property \*.period set to a default value foe metrics | Major | conf | Arpit Gupta | Eric Yang | | | log4j.properties should be added to the hadoop conf on deploy | Major | conf | Arpit Gupta | Eric Yang | | | Fix the delegation token tests to use the new style renewers | Blocker | security | Owen O'Malley | Owen O'Malley | | | TestSleepJob fails | Blocker | test | Arun C Murthy | Arun C Murthy | | | HTTP auth tests requiring Kerberos infrastructure are not disabled on branch-0.20-security | Blocker | security | Aaron T. Myers | Jitendra Nath Pandey | | | TestKerberosName fails in 20 branch. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | | | TestSleepJob and TestHdfsProxy broken after HDFS-2284 | Blocker | build | Kihwal Lee | Tsz Wo Nicholas Sze | | | Fuse-DFS fails to build on Hadoop 20.203.0 | Blocker | fuse-dfs, libhdfs | Charles Earl | Kihwal Lee | | | log4j.properties templates does not define

| Major | conf | Ramya Sunil | Ramya Sunil | | | NPE found in Datanode log while Disk failed during different HDFS operation | Major | datanode | Rajit Saha | Jonathan Eagles | | | TestFileAppend4 fails in 0.20.205 branch | Blocker | hdfs-client | Suresh Srinivas | Suresh Srinivas | | | Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh | Major | contrib/vaidya | vitthal (Suhas) Gogate | | | | Commands using webhdfs and hftp print unnecessary debug information on the console with security enabled | Major | webhdfs | Arpit Gupta | Arpit Gupta | | | defaults created for web keytab and principal, these properties should not have defaults | Major | . | Arpit Gupta | Tsz Wo Nicholas Sze | | | hftp is broken | Critical | namenode | Rajit Saha | Jitendra Nath Pandey | | | hdfs-site.xml template has properties that are not used in 20 | Minor | . | Arpit Gupta | Arpit Gupta | | | 0.20: Block lost when multiple DNs trying to recover it to different genstamps | Critical | . | Todd Lipcon | Todd Lipcon | | | 0.20: DNs should interrupt writers at start of recovery | Blocker | datanode | Todd Lipcon | Todd Lipcon | | | 20 append: Blocks recovered on startup should be treated with lower priority during block synchronization | Critical | datanode | Todd Lipcon | Todd Lipcon | | | hadoop deb pkg should take a diff group id | Major | . | Giridharan Kesavan | Eric Yang | | | Set default hdfs, mapred uid, and hadoop group gid for RPM packages | Major | . | Eric Yang | Eric Yang | | | to fix hadoop config template | Major | . | Giridharan Kesavan | Eric Yang | | | jobhistory server and secondarynamenode should have init.d script | Major | scripts | Eric Yang | Eric Yang | | | Calling hadoop cli inside mapreduce job leads to errors | Major | contrib/streaming | Eric Yang | Eric Yang | | | see log4j Error when running mr jobs and certain dfs calls | Major | conf | Arpit Gupta | Eric Yang | | | hadoop-env.sh generated from templates has duplicate info | Major | conf | Arpit Gupta | Arpit Gupta | | | log4j.properties is missing properties for security audit and hdfs audit should be changed to info | Minor | conf | Arpit Gupta | Arpit Gupta | | | config generator does not update the properties file if on exists already | Critical | conf | Arpit Gupta | Eric Yang | | | NPE when the default filesystem's uri has no authority | Major | namenode | Rajit Saha | Daryn Sharp | | | Dist with hftp is failing again | Critical | namenode | Rajit Saha | Daryn Sharp | | | DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append | Blocker | hdfs-client | stack | stack | | | dfs.web.authentication.kerberos.principal expects the full hostname and does not replace \_HOST with the hostname | Major | . | Arpit Gupta | Jitendra Nath Pandey | | | hadoop dfs command with webhdfs fails on secure hadoop | Critical | webhdfs | Arpit Gupta | Jitendra Nath Pandey | | | hadoop-setup-conf.sh should put proxy user info into the core-site.xml | Major | . | Giridharan Kesavan | Arpit Gupta | | | Fix renewal of dfs delegation tokens | Major | . | Daryn Sharp | Owen O'Malley | | | wordcount, sort etc on har files fails with NPE | Major |

| John George | John George | | | TestMapredGroupMappingServiceRefresh and TestRefreshUserMappings fail after HADOOP-7625 | Blocker | security, test | Kihwal Lee | Jitendra Nath Pandey | | | with webhdfs enabled in secure mode the auth to local mappings are not being respected. | Major | webhdfs | Arpit Gupta | Jitendra Nath Pandey | | | Blocks are considered "complete" prematurely after commitBlockSynchronization or DN restart | Major | datanode, hdfs-client, namenode | Todd Lipcon | Todd Lipcon | | | Binary layout does not put share/hadoop/contrib/\*.jar into the class path | Major | scripts | Eric Yang | Eric Yang | | | TestFileAppend4 and TestMultiThreadedSync fail on 20.append and 20-security. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | 0.20 append: Add test for appendFile() race solved in HDFS-142 | Major | . | Todd Lipcon | Todd Lipcon | | | TestDFSConcurrentFileOperations broken in 0.20-appendj | Major | test | Todd Lipcon | Todd Lipcon | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Fix write pipeline READ\_TIMEOUT | Major | datanode, hdfs-client | Kan Zhang | Kan Zhang | | | Concurrent readers hit ChecksumExceptions if following a writer to very end of file | Blocker | datanode | Todd Lipcon | sam rash | | | Write Http access to HDFS | Major | . | Sanjay Radia | Tsz Wo Nicholas Sze | | | Read access to HDFS using HTTP REST | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Configuration option to enable/disable webhdfs. | Major | webhdfs | Jitendra Nath Pandey | Jitendra Nath Pandey | | | Provide authentication to webhdfs using SPNEGO | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Support getFileBlockLocations and getDelegationToken in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | MR-2413 improvements | Major | tasktracker | Eli Collins | Eli Collins | | | webhdfs: support case insensitive query parameter names | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | webhdfs throws a npe when ugi is null from getDelegationToken | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | | | Support getContentSummary and getFileChecksum in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | webhdfs api's should return a root element in the json response | Critical | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | | | The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | webhdfs liststatus json response is not correct | Major | webhdfs | Arpit Gupta | Suresh Srinivas | | | Support delegation token renewal in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Standardize the value of token service | Major | . | Daryn Sharp | Daryn Sharp | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Port 0.20-append changes onto 0.20-security-203 | Major | . | Andrew Purtell | |