Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.0.16.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | credits page should have more information | Major | documentation | Doug Cutting | Doug Cutting | | | admins should be able to finalize namenode upgrades without running the cluster | Critical | . | Owen O'Malley | Konstantin Shvachko | | | Datanode should support block replacement | Major | . | Hairong Kuang | Hairong Kuang | | | Change FileSystem API to support access control. | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Provide a simple login implementation | Major | fs | Tsz Wo Nicholas Sze | Hairong Kuang | | | RPC Support for user permissions and authentication. | Major | ipc | Tsz Wo Nicholas Sze | Raghu Angadi | | | Rebalance data blocks when new data nodes added or data nodes become full | Major | . | Hairong Kuang | Hairong Kuang | | | need 'doc' target that runs forrest | Major | build | Doug Cutting | Doug Cutting | | | Map-side joins on sorted, equally-partitioned datasets | Major | . | Chris Douglas | Chris Douglas | | | Support permission information in FileStatus | Major | fs | Tsz Wo Nicholas Sze | Raghu Angadi | | | Shell commands to access and modify file permissions | Major | fs | Raghu Angadi | Raghu Angadi | | | resource management proviosioning for Hadoop | Major | . | Pete Wyckoff | Hemanth Yamijala | | | adding user info to file | Major | fs | Kurtis Heimerl | Tsz Wo Nicholas Sze | | | HDFS should be capable of limiting the total number of inodes in the system | Major | . | Sameer Paranjpye | dhruba borthakur | | | Provide an option to get job status for all jobs run by or submitted to a job tracker | Major | . | Hemanth Yamijala | Amareshwari Sriramadasu | | | Additional Instrumentation for NameNode, RPC Layer and JMX support | Major | . | Sanjay Radia | Sanjay Radia | | | Trash and permissions don't mix | Major | . | Robert Chansler | Doug Cutting | | | Periodic verification at the Datanode | Major | . | Raghu Angadi | Raghu Angadi | | | No-permission-checking mode for smooth transition to 0.16's permissions features. | Major | . | Sanjay Radia | Hairong Kuang | | | SequenceFileAsBinaryInputFormat | Major | . | Chris Douglas | Chris Douglas | | | Get representative hprof information from tasks | Major | . | Owen O'Malley | Owen O'Malley | | |

FileSystem#getHomeDirectory() method | Major | fs | Doug Cutting | Doug Cutting | | | DFS User Guide | Major | documentation | Raghu Angadi | Raghu Angadi | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Allow adding additional datanodes to MiniDFSCluster | Major | test | Hairong Kuang | Hairong Kuang | | | fsck should verify block placement | Major | . | dhruba borthakur | Konstantin Shvachko | | | Link-ify the Pending/Running/Complete/Killed tasks/task-attempts on jobdetails.jsp | Major | . | Arun C Murthy | Amar Kamat | | | Redesign of Eclipse plug-in interface with Hadoop | Major | . | Christophe Taton | Christophe Taton | | | Ability to run a script when a task fails to capture stack traces | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | | | Log counters in job history | Minor | . | Albert Chern | Owen O'Malley | | | Clarify requirements for Hadoop in overview.html | Minor | . | Jim Kellerman | Jim Kellerman | | | ability to add dependencies to a job after construction | Major | . | Adrian Woodhead | Adrian Woodhead | | | dynamically change log levels | Major | util | dhruba borthakur | Tsz Wo Nicholas Sze | | | Remove developer-centric requirements from overview.html | Major | documentation | Arun C Murthy | Jim Kellerman | | | Configuring different number of mappers and reducers per TaskTracker | Major | . | Koji Noguchi | Amareshwari Sriramadasu | | | Add pipes sort example | Major | . | Owen O'Malley | Owen O'Malley | | | Remove deprecated code in Configuration/JobConf | Major | conf | Arun C Murthy | Arun C Murthy | | | Add "-text" command to FsShell to decode SequenceFile to stdout | Minor | fs | Chris Douglas | Chris Douglas | | | the heartbeat and task event queries interval should be set dynamically by the JobTracker | Major | . | Owen O'Malley | Amareshwari Sriramadasu | | | Remove deprecated contrib/abacus code | Major | . | Doug Cutting | Enis Soztutar | | | FSEditLog.logEdit(byte op, Writable w1, Writable w2) should accept variable numbers of Writable, instead of two. | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Server ports: to roll or not to roll. | Major | conf | Konstantin Shvachko | Konstantin Shvachko | | | Lease holder information should be passed in ClientProtocol.abandonBlock(...) | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | IPC server should write repsonses asynchronously | Major | ipc | Doug Cutting | dhruba borthakur | | | If HDFS is going to throw an exception "File does not exist" it should include the name of the file | Minor | . | Jim Kellerman | Jim Kellerman | | | Add a 'forrest.home' property for the 'docs' target in build.xml | Minor | documentation | Arun C Murthy | Arun C Murthy | | | Pure name-node benchmarks. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | FileInputFormat computeSplitSize() method, change visibility to protected and make it a member method | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | | | TextOutputFormat should special case Text | Major | . | Owen O'Malley | Owen O'Malley | | | Reduce frequency of Counter updates in the task tracker status | Major |

| Amareshwari Sriramadasu | Amareshwari Sriramadasu | | | Improve the utilization of shuffle copier threads | Major | . | Devaraj Das | Amar Kamat | | | Document the user-controls for intermediate/output compression via forrest | Major | documentation | Arun C Murthy | Arun C Murthy | | | add support for native library toDistributedCache | Major | . | Alejandro Abdelnur | Arun C Murthy | | | General example for modeling m/r load in Java | Minor | test | Chris Douglas | Chris Douglas | | | remove use of 'magic number' in build.xml | Trivial | build | Hrishikesh | Hrishikesh | | | JobControl classes should use interfaces rather than implemenations | Minor | . | Adrian Woodhead | Adrian Woodhead | | | enable hdfs permission checking by default | Major | . | Doug Cutting | Doug Cutting | | | Speculative execution should be allowed for reducers only | Critical | . | Srikanth Kakani | Amareshwari Sriramadasu | | | User permissions for Map/Reduce | Major | . | Raghu Angadi | Hairong Kuang | | | Handle map output buffers better | Major | . | Devaraj Das | Amar Kamat | | | Test permissions related shell commands with DFS | Major | . | Raghu Angadi | Raghu Angadi | | | Persisting completed jobs status | Critical | . | Alejandro Abdelnur | Alejandro Abdelnur | | | Logging version number (and compiled date) at STARTUP\_MSG | Trivial | . | Koji Noguchi | Arun C Murthy | | | Add support for simulated Data Nodes - helpful for testing and performance benchmarking of the Name Node without having a large cluster | Minor | . | Sanjay Radia | Sanjay Radia | | | FSNamesystem.startFile() javadoc is inconsistent | Minor | . | Konstantin Shvachko | Konstantin Shvachko | | | Remove the DFS Client disk-based cache | Major | . | dhruba borthakur | dhruba borthakur | | | ant target without source and docs | Major | build | Gautam Kowshik | Hrishikesh | | | WritableUtils.clone should take Configuration rather than JobConf | Minor | io | stack | stack | | | add SequenceFile.createWriter() method that takes block size as parameter | Minor | io | Alejandro Abdelnur | Alejandro Abdelnur | | | Additional metrics & jmx beans and cleanup to use the recent metrics libraries | Major | . | Sanjay Radia | Sanjay Radia | | | Add option to disable nagles algorithm in the IPC Server | Major | ipc | Clint Morgan | Clint Morgan | | | need FileSystem#globStatus method | Major | fs | Doug Cutting | Hairong Kuang | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | RawLocalFileStatus is causing Path problems | Major | fs | Dennis Kubes | | | | value for mapred.tasktracker.tasks.maximum taken from jobtracker, not tasktracker | Major | . | Michael Bieniosek | Michael Bieniosek | | | locking for the ReflectionUtils.logThreadInfo is too conservative | Major | util | Owen O'Malley | Amareshwari Sriramadasu | | |

script fails if HADOOP\PID\DIR doesn't exist | Major | scripts | Arun C Murthy | Michael Bieniosek | | | The file used to localize job.xml should be closed. | Minor | . | Amar Kamat | Amar Kamat | | | File handles for log files are still open in case of jobs with 0 maps | Major | . | Amar Kamat | Amar Kamat | | | TestMiniMRMapRedDebugScript fails due to a missing file | Blocker | . | Devaraj Das | Arun C Murthy | | | Multiple caheArchive does not work in Hadoop streaming | Critical | . | Milind Bhandarkar | Lohit Vijayarenu | | | StreamXmlRecordReader throws java.io.IOException: Mark/reset exception in hadoop 0.14 | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | | | Configuration XML bug: comments inside values | Critical | conf | Eelco Lempsink | Rajagopal Natarajan | | | Streaming does not handle invalid -inputformat (typo by users for example) | Minor | . | Lohit Vijayarenu | Lohit Vijayarenu | | | FileSyste.globPaths does not validate the return list of Paths | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | | | Unexpected IOException in DFSOutputStream.close() | Major | . | Raghu Angadi | Raghu Angadi | | | Remove unused ClientFinalizer in DFSClient | Major | . | Philippe Gassmann | Chris Douglas | | | DFSTestUtil.waitReplication does not wait. | Major | . | Raghu Angadi | Raghu Angadi | | | Name-node should be able to close empty files. | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Jobs using LocalJobRunner + JobControl fails | Critical | . | Johan Oskarsson | Doug Cutting | | | clover description attribute suppresses all other targets in -projecthelp | Trivial | build | Chris Douglas | Chris Douglas | | | java.lang.ArithmeticException: / by zero in ChecksumFileSystem.open | Critical | fs | Michael Bieniosek | Michael Bieniosek | | | Job UI doesnot show running tasks and complete tasks correctly. | Major | . | Amareshwari Sriramadasu | Amar Kamat | | | findbugs currently fails due to hadoop-streaming having moved | Major | build | Adrian Woodhead | stack | | | MapWritable.readFields needs to clear internal hash else instance accumulates entries forever | Major | io | stack | stack | | | some reducer stuck at copy phase and progress extremely slowly | Critical | . | Runping Qi | Amar Kamat | | | TestRecordMR and TestAggregates fail once in a while | Major | . | Devaraj Das | Adrian Woodhead | | | Erroneous detection of corrupted file when namenode fails to allocate any datanodes for newly allocated block | Major | . | dhruba borthakur | dhruba borthakur | | | TestBlockReplacement unit test failed. | Major | . | Raghu Angadi | Hairong Kuang | | | SecondaryNamenode process should exit if it encounters Runtime exceptions | Major | . | dhruba borthakur | | | | TestBlockReplacement occasionally get into an infinite loop | Major | . | Hairong Kuang | Hairong Kuang | | | mapred.tasktracker.tasks.maximum is completely ignored | Blocker |

| Owen O'Malley | Amareshwari Sriramadasu | | | Lots of unit tests fail on Windows with exception: Login failed: CreateProcess | Blocker | . | Mukund Madhugiri | Tsz Wo Nicholas Sze | | | Unit tests fail if there is another instance of Hadoop | Major | test | Raghu Angadi | Konstantin Shvachko | | | chmod in ant package target fails | Major | build | Adrian Woodhead | Adrian Woodhead | | | build does not fail when libhdfs build fails | Minor | . | Nigel Daley | Nigel Daley | | | PendingReplicationMonitor thread received exception. java.lang.InterruptedException | Major | . | Owen O'Malley | dhruba borthakur | | | Result of HashFunction.hash() contains all identical values | Minor | . | Andrzej Bialecki | Jim Kellerman | | | JobTracker.close() prints stack traces for exceptions that are not errors | Minor | . | Jim Kellerman | Jim Kellerman | | | The sort example shouldn't override the number of maps | Major | . | Owen O'Malley | Owen O'Malley | | | Word count example is spending 24% of the time in incrCounter | Major | . | Owen O'Malley | Owen O'Malley | | | Jobs fail because job.xml exists | Major | . | Johan Oskarsson | Johan Oskarsson | | | MapFile.get on HDFS in TRUNK is WAY!!! slower than 0.15.x | Blocker | io | stack | stack | | | Running 'ant docs tar' includes src/docs/build in the resulting tar file | Minor | build | Nigel Daley | Nigel Daley | | | wordcount-simple example gives ParseException with examples configuration file | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | | | Change documentation in cluster\setup.html and mapred\tutorial.html post HADOOP-1274 | Major | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | | | Remove AC\LIB\CHECK from src/native/configure.ac to ensure libhadoop.so doesn't have a dependency on libz.so/liblzo.so | Major | native | Arun C Murthy | Arun C Murthy | | | Reduce tasks fail too easily because of repeated fetch failures | Blocker | . | Christian Kunz | Amar Kamat | | | Mappers fail easily due to repeated failures | Blocker | . | Srikanth Kakani | Amar Kamat | | | Eclipse plug-in build.xml issue | Trivial | build | Christophe Taton | Christophe Taton | | | Unit test fails on Windows: TestCopyFiles.testCopyFromLocalToLocal | Blocker | fs | Mukund Madhugiri | Raghu Angadi | | | REST Insert / Select | Critical | . | Billy Pearson | Bryan Duxbury | | | ConcurrentModificationException in org.apache.hadoop.ipc.Server.Responder | Major | ipc | Devaraj Das | dhruba borthakur | | | Free up the buffers (input and error) while executing a shell command before waiting for it to finish. | Major | . | Amar Kamat | Amar Kamat | | | HADOOP-2344 introduced a javadoc warning | Major | documentation | Arun C Murthy | Arun C Murthy | | | Unit test failed: org.apache.hadoop.fs.TestLocalFileSystemPermission.testLocalFSsetOwner | Critical | fs | Mukund Madhugiri | Raghu Angadi | | | Unit test fails on Windows: TestDFSShell.testFilePermissions | Blocker | . | Mukund Madhugiri | Raghu Angadi | | | Remove support for deprecated mapred.child.heap.size and indentation fix in TaskRunner.java | Minor |

| Arun C Murthy | Arun C Murthy | | | error stream handling in Shell executor | Blocker | util | Raghu Angadi | Raghu Angadi | | | Use exit code to detect normal errors while excuting 'ls' in Local FS | Blocker | fs | Raghu Angadi | Raghu Angadi | | | TextInputFormat is slow compared to reading files. | Major | . | Owen O'Malley | Owen O'Malley | | | TestHDFSServerPorts fails. | Major | test | Raghu Angadi | Nigel Daley | | | make build process compatible with Ant 1.7.0 | Major | build | Nigel Daley | Hrishikesh | | | Speculative map tasks aren't getting killed although the TIP completed | Critical | . | Arun C Murthy | Arun C Murthy | | | javac generates a warning in test/o.a.h.io.FileBench | Trivial | test | Chris Douglas | Chris Douglas | | | Potential Eclipse plug-in UI loop when editing location parameters | Minor | contrib/eclipse-plugin | Christophe Taton | Christophe Taton | | | NNBench should periodically report its progress | Major | test | Hairong Kuang | Hairong Kuang | | | TestNNThroughput should not use a fixed namenode port | Major | . | Hairong Kuang | Konstantin Shvachko | | | Set +x on contrib/\/bin/\ in packaged tar bundle | Major | scripts | stack | stack | | | leading slash in mapred.task.tracker.report.bindAddress | Major | conf | Konstantin Shvachko | Konstantin Shvachko | | | 'bin/hadoop fs -help' does not list file permissions commands. | Trivial | fs | Raghu Angadi | Raghu Angadi | | | dfs web interfaces should run as a configurable user account | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Fix Datanode transfer thread logging | Minor | . | Dennis Kubes | dhruba borthakur | | | NPE in TaskLog.java | Trivial | . | Michael Bieniosek | Michael Bieniosek | | | hadoop dfs -copyToLocal creates zero byte files, when source file does not exists | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | | | Incrementing user counters should count as progress | Blocker | . | Owen O'Malley | Devaraj Das | | | BasicTypeSorterBase.compare calls progress on each compare | Major | . | Owen O'Malley | Amar Kamat | | | The ReplicationMonitor sleep period should be configurable | Blocker | . | dhruba borthakur | dhruba borthakur | | | hdfs does not honor dfs.du.reserved setting | Critical | . | Joydeep Sen Sarma | Hairong Kuang | | | Add rat target to build | Major | build | Nigel Daley | Hrishikesh | | | The commands in DFSAdmin should require admin privilege | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Revert change to fsck made as part of permissions implementation | Blocker | . | Robert Chansler | Tsz Wo Nicholas Sze | | | 1707 added errant INFO-level logging to DFSClient | Blocker |

| stack | stack | | | Lzo compression compresses each write from TextOutputFormat | Major | io, native | Chris Douglas | Chris Douglas | | | Some junit tests fail with the exception: All datanodes are bad. Aborting... | Major | . | Hairong Kuang | dhruba borthakur | | | NullPointerException in FSNamesystem due to getDatanode() return value is not checked | Major | . | Konstantin Shvachko | dhruba borthakur | | | MultiFileSplitInputFormat always returns 1 split when avgLengthPerSplit \> Integer.MAX\_VALUE | Blocker | . | Frdric Bertin | Enis Soztutar | | | RawLocalFileStatus is badly handling URIs | Major | fs | Frdric Bertin | Doug Cutting | | | SortValidator broken with fully-qualified working directories | Blocker | test | Doug Cutting | Arun C Murthy | | | Fix permission issues for HftpFileSystem | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | | Pipes C++ task does not die even if the Java tasks die | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | | | Unit test fails on Windows: rg.apache.hadoop.dfs.TestDecommission | Blocker | . | Mukund Madhugiri | dhruba borthakur | | | Namenode performance degradation over time | Blocker | . | Christian Kunz | Raghu Angadi | | | Update HOD in Hadoop 0.16 | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | | | Unit test fails on Windows: org.apache.hadoop.dfs.TestDatanodeDeath | Blocker | . | Mukund Madhugiri | dhruba borthakur | | | Reducers stuck in shuffle | Blocker | . | Amareshwari Sriramadasu | Arun C Murthy | | | Hadoop 2367- Does not respect JobConf.getProfileEnabled() | Blocker | . | Clint Morgan | Amareshwari Sriramadasu | | | docs link to lucene.apache.org | Major | documentation | Doug Cutting | Doug Cutting | | | ab{5}.gz should not be treated as an illegal glob | Blocker | fs | Hairong Kuang | Hairong Kuang | | | HADOOP-2185 breaks compatibility with hadoop-0.15.0 | Blocker | conf | Arun C Murthy | Owen O'Malley | | | Modify HOD to work with changes mentioned in HADOOP-2404 | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | | | dfs fsck extremely slow, dfs ls times out | Blocker | . | Christian Kunz | Tsz Wo Nicholas Sze | | | DFSIO write performance benchmark shows a regression | Blocker | . | Mukund Madhugiri | dhruba borthakur | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Re-write NNBench to use MapReduce | Major | test | Mukund Madhugiri | Mukund Madhugiri | | | Representative mix of jobs for large cluster throughput benchmarking | Major | test | Chris Douglas | Runping Qi | | | Micro-benchmark to measure read/write times through InputFormats | Major | fs, test | Chris Douglas | Chris Douglas | | | Restore the old NN Bench that was replaced by a MR NN Bench | Major | . | Sanjay Radia | Sanjay Radia | | | Test HDFS File Permissions | Major | test | Hairong Kuang | Hairong Kuang | | | create a micro-benchmark for measure local-file versus hdfs read | Major | . | Owen O'Malley | Owen O'Malley |