Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.0.13.0.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Contribute some code helping implement map/reduce apps for joining data from multiple sources | Major | . | Runping Qi | Runping Qi | | | DFS Upgrade Proposal | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | Specify a junit test timeout in build.xml files | Minor | build | Nigel Daley | Nigel Daley | | | A method to get the InputSplit from a Mapper | Major | . | Owen O'Malley | Owen O'Malley | | | Job completion notification to a job configured URL | Major | . | Alejandro Abdelnur | | | | Hadoop should support reduce none option | Major | . | Runping Qi | Runping Qi | | | Make Hadoop Abacus work with Hadoop Streaming | Major | . | Runping Qi | Runping Qi | | | allow a different comparator for grouping keys in calls to reduce | Major | . | Owen O'Malley | Tahir Hashmi | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Add more statistics in the web-ui to do with tasks | Major | . | Devaraj Das | Devaraj Das | | | Add maxmemory="256m" in the junit call of build-contrib.xml | Major | build | Philippe Gassmann | | | | Optimize readFields and write methods in record I/O | Major | record | Milind Bhandarkar | Milind Bhandarkar | | | Improve error message for 0 datanode case | Major | . | Owen O'Malley | dhruba borthakur | | | Add a closeAll() static method to FileSystem | Minor | . | Philippe Gassmann | | | | Code for toString in code generated by Record I/O Compiler can be generic | Minor | record | Milind Bhandarkar | Milind Bhandarkar | | | Pull the NullOutputFormat into the lib package | Major | . | Owen O'Malley | Owen O'Malley | | | InMemoryFileSystem uses synchronizedtMaps with maps that are locked anyways | Minor | fs | Owen O'Malley | Owen O'Malley | | | map output should not do block level compression | Major | . | Runping Qi | Arun C Murthy | | | Tools to analyze and debug namenode on a production cluster | Major | . | dhruba borthakur | dhruba borthakur | | | Namenode should use single map for block to its meta data. | Major | . | Raghu Angadi | Raghu Angadi | | | AlreadyBeingCreatedException detail message could contain more useful info | Minor | . | Nigel Daley | Konstantin Shvachko | | | DFS Scalabilty: Improve name node performance by adding a hostname to datanodes map | Major | . | Hairong Kuang | Hairong Kuang | | | LineRecordWriter should not always insert tab char between key and value | Major | . | Runping Qi | Runping Qi | | | When RPC call fails then log call message detail | Minor | ipc | Nigel Daley | Doug Cutting | | | Remove the MustangFile class from streaming and promote the chmod into FileUtils | Major |

| Owen O'Malley | Owen O'Malley | | | the first step for streaming clean up | Major | . | Runping Qi | Runping Qi | | | re-indent all code | Minor | . | Doug Cutting | Doug Cutting | | | Reduce shuffle and merge should be done a child JVM | Major | . | Owen O'Malley | Devaraj Das | | | need code review guidelines | Major | build | Nigel Daley | Nigel Daley | | | Fix unchecked warnings | Major | . | Tom White | Tom White | | | Speculative Execution and output of Reduce tasks | Major | . | Arun C Murthy | Arun C Murthy | | | clean up the protocol between stream mapper/reducer and the framework | Major | . | Runping Qi | Runping Qi | | | Move Hadoop Abacus to hadoop.mapred.lib | Major | . | Runping Qi | | | | MAX\TASK\FAILURES should be configurable | Major | . | Christian Kunz | Devaraj Das | | | Tighten generic Class restrictions in JobConf.java | Minor | . | Michael Bieniosek | Michael Bieniosek | | | Hadoop Streaming code clean up | Major | . | Runping Qi | Runping Qi | | | Hadoop should allow a configurable percentage of failed map tasks before declaring a job failed. | Major | . | Christian Kunz | Arun C Murthy | | | retry logic when dfs exist or open fails temporarily, e.g because of timeout | Major | . | Christian Kunz | Hairong Kuang | | | TaskTracker expiry interval is not configurable | Major | . | Alejandro Abdelnur | Arun C Murthy | | | Return the RunningJob from JobClient.runJob | Major | . | Owen O'Malley | Owen O'Malley | | | Randomize the fetch of map outputs | Major | . | Arun C Murthy | Arun C Murthy | | | FSError encountered by one running task should not be fatal to other tasks on that node | Major | . | Devaraj Das | Arun C Murthy | | | need improved release process | Major | build | Doug Cutting | Doug Cutting | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | TestReplication fails because DFS does not guarantee all the replicas are placed when a file is closed | Major | . | Hairong Kuang | Hairong Kuang | | | MiniDFSCluster exists a race condition that lead to data node resources are not properly released | Major | test | Hairong Kuang | Hairong Kuang | | | RPC$VersionMismatch exception is not fatal to JobTracker | Major | . | Nigel Daley | Tahir Hashmi | | | dfsclient logging messages should have appropriate log levels | Major | . | dhruba borthakur | dhruba borthakur | | | StatusHttpServer assumes that resources for /static are in files | Major | . | Benjamin Reed | | | | Divide-by-zero exception in chooseTarget | Major | . | dhruba borthakur | dhruba borthakur | | | Remove 'port rolling' from Mini{DFS\|MR}Cluster | Major | test | Arun C Murthy | Arun C Murthy | | | DataNode and FSNamesystem don't shutdown cleanly | Major |

| Nigel Daley | Konstantin Shvachko | | | "Reduce input records" counter name is misleading | Major | . | David Bowen | David Bowen | | | TestReplicationPolicy doesn't use port 0 for the NameNode | Major | test | Owen O'Malley | Owen O'Malley | | | CopyFiles skips src files of s3 urls | Minor | util | stack | | | | JobTracker WebUI "Map input records" a little off. | Trivial | . | Koji Noguchi | David Bowen | | | NullPointer Exception in org.apache.hadoop.dfs.NameNode.isDir on namenode restart | Major | . | dhruba borthakur | dhruba borthakur | | | ConcurrentModificationException in JobHistory | Major | . | Nigel Daley | Tahir Hashmi | | | dfs shell enhancements | Minor | . | Yoram Arnon | dhruba borthakur | | | Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect | Major | . | Igor Bolotin | | | | In SortValidator, the check for whether a file belongs to sort-input or sort-output dir is weak | Major | . | Devaraj Das | Arun C Murthy | | | JAVA\_PLATFORM with spaces (i.e. Mac OS X-ppc-32) breaks bin/hadoop script | Major | scripts | Andrzej Bialecki | | | | NullPointerException in MiniDFSCluster | Major | . | Konstantin Shvachko | Hairong Kuang | | | Ganglia metrics reporting is misconfigured | Minor | metrics | Michael Bieniosek | | | | DFS Scalability: avoid scanning entire list of datanodes in getAdditionalBlocks | Major | . | dhruba borthakur | dhruba borthakur | | | streaming PipeMapRed prints system info to stderr | Trivial | . | Koji Noguchi | Koji Noguchi | | | UpgradeUtilities should use MiniDFSCluster to start and stop NameNode/DataNodes | Major | test | Nigel Daley | Nigel Daley | | | DFS Scalability: high cpu usage in addStoredBlock | Major | . | dhruba borthakur | Raghu Angadi | | | DFS Scalability: high CPU usage in choosing replication targets and file open | Major | . | dhruba borthakur | Hairong Kuang | | | In TaskTracker the access to RunningJob object is not synchronized in one place | Major | . | Devaraj Das | Devaraj Das | | | Spurious progress messages should be discarded after a task is done | Major | . | Devaraj Das | Devaraj Das | | | ipc.client.timeout of 2000ms for test cases seems too small; causes too many timeouts and leads to hung test cases | Major | test | Arun C Murthy | Arun C Murthy | | | Remove deprecated constructor and unused static members in DataNode class | Major | . | Konstantin Shvachko | Konstantin Shvachko | | | exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required | Major | . | dhruba borthakur | Hairong Kuang | | | the output of the map is not type checked against the specified types | Major | . | Owen O'Malley | Tahir Hashmi | | | Still seeing some unexpected 'No space left on device' exceptions | Major | . | Raghu Angadi | Raghu Angadi | | | Re-factor InputFormat/RecordReader related classes | Major |

| Runping Qi | Runping Qi | | | bin/hadoop script clobbers CLASSPATH | Minor | scripts | Michael Bieniosek | Doug Cutting | | | maps\_running metric is only updated at the end of the task | Minor | metrics | Michael Bieniosek | David Bowen | | | NNBench generates millions of NotReplicatedYetException in Namenode log | Major | . | Nigel Daley | dhruba borthakur | | | Classes in src/test/testjar need package name | Trivial | test | Jim Kellerman | Jim Kellerman | | | ClientProtocol.versionID should be 11 | Major | . | Konstantin Shvachko | dhruba borthakur | | | Null PointerException in processReport when namenode is restarted | Major | . | dhruba borthakur | dhruba borthakur | | | stop-dfs.sh incorrectly specifies slaves file for stopping datanode | Minor | . | Michael Bieniosek | dhruba borthakur | | | ConcurrentModificationException and NPE in JobControl | Minor | . | Johan Oskarsson | Johan Oskarsson | | | Dfs image loading and edits loading creates multiple instances of DatanodeDescriptor for the same datanode | Major | . | Hairong Kuang | Hairong Kuang | | | TestCheckpoint test case doesn't wait for MiniDFSCluster to be active | Trivial | test | Nigel Daley | Nigel Daley | | | streaming hang. (PipeMapRed$MROutputThread gone) | Major | . | Koji Noguchi | Koji Noguchi | | | job notification property in hadoop-default.xml is misspelled | Trivial | . | Alejandro Abdelnur | | | | Reduce task hang failing in MapOutputCopier.copyOutput | Major | . | Koji Noguchi | Tahir Hashmi | | | Do not count lost tasktracker against the job | Major | . | Arun C Murthy | Arun C Murthy | | | The StreamBaseRecordReader is unable to log record data that's not UTF-8 | Minor | . | Gautam Kowshik | Arun C Murthy | | | file corruption detected because dfs client does not use replica blocks for checksum file | Major | . | dhruba borthakur | Hairong Kuang | | | list of completed jobs purges jobs based on submission not on completion age | Major | . | Alejandro Abdelnur | Arun C Murthy | | | Reduce hang on huge map output | Major | . | Hairong Kuang | Arun C Murthy | | | stderr from streaming skipped after first 20 lines. | Minor | . | Koji Noguchi | Koji Noguchi | | | datanode sending block reports to namenode once every second | Major | . | dhruba borthakur | dhruba borthakur | | | Fix unchecked warnings in main Hadoop code under Java 6. | Major | test | Tom White | Tom White | | | Once RPC.stopClient has been called, RPC can not be used again | Minor | ipc | stack | stack | | | Fix the per-job tasktracker 'blacklist' | Major | . | Arun C Murthy | Arun C Murthy | | | Extract InnerClasses from FSNamesystem into separate classes | Major | . | dhruba borthakur | dhruba borthakur | | | S3 listSubPaths bug | Critical | fs | Mike Smith | | | | heartbeat monitor thread goes away | Blocker | . | dhruba borthakur | dhruba borthakur | | | Tasktracker blacklist leads to hung jobs in single-node cluster | Critical | . | Arun C Murthy | Arun C Murthy | | | Decommission fails if a block that needs replication has only one replica | Major | . | dhruba borthakur | | | | "Browse the filesystem" link pointing to a dead data-node | Major |

| Konstantin Shvachko | Enis Soztutar | | | Datanode should periodically do a disk check | Blocker | . | Hairong Kuang | Hairong Kuang | | | Disk problems should be handled better by the MR framework | Major | . | Devaraj Das | Devaraj Das | | | Fix unchecked warnings in aggregate code | Major | . | Tom White | Tom White | | | Name-node falls into infinite loop trying to remove a dead node. | Blocker | . | Konstantin Shvachko | Hairong Kuang | | | DistributedFileSystem doesn't close the RawDistributedFileSystem on close. | Blocker | . | Owen O'Malley | Hairong Kuang | | | Shuffle started taking a very long time after the HADOOP-1176 fix | Blocker | . | Devaraj Das | Devaraj Das | | | Checksum object does not get restored to the old state in retries when handle ChecksumException | Blocker | . | Hairong Kuang | Hairong Kuang | | | The open method of FSNamesystem should be synchronized | Blocker | . | Hairong Kuang | Hairong Kuang | | | Null pointer dereference of nodeInfo in FSNamesystem.removeDatanode(DatanodeID) | Blocker | . | Nigel Daley | dhruba borthakur | | | Null pointer dereference of paths in FsShell.dus(String) | Blocker | fs | Nigel Daley | Hairong Kuang | | | seek call ignores result of skipBytes(int) | Blocker | . | Nigel Daley | Hairong Kuang | | | ValueHistogram.addNextValue(Object) ignores return value of String.substring(int, int) | Blocker | . | Nigel Daley | Runping Qi | | | waitForCompletion() calls Thread.sleep() with a lock held | Blocker | . | Nigel Daley | Owen O'Malley | | | Inconsistent synchronization of 3 fields in JobInProgress.java | Blocker | . | Nigel Daley | Owen O'Malley | | | Inconsistent synchronization of TaskTracker fields | Blocker | . | Nigel Daley | Owen O'Malley | | | seek calls in 3 io classes ignore result of skipBytes(int) | Blocker | io | Nigel Daley | Hairong Kuang | | | Possible Null Pointer Dereference in taskdetails.jsp | Major | . | Devaraj Das | Devaraj Das | | | MD5Hash has a bad hash function | Major | io | Owen O'Malley | Owen O'Malley | | | The constructor of Path should not take an empty string as a parameter | Blocker | . | Hairong Kuang | Hairong Kuang | | | Failed tasks not killing job | Blocker | . | Nigel Daley | Arun C Murthy | | | Typo in GzipCodec.createInputStream - bufferSize | Blocker | io | Espen Amble Kolstad | Espen Amble Kolstad | | | FileSystem.globPaths should not create a Path from an empty string | Blocker | fs | Hairong Kuang | Hairong Kuang | | | AlreadyBeingCreatedException from task retries | Blocker | . | Nigel Daley | Hairong Kuang | | | dfs upgrade/downgrade problems | Blocker | . | Owen O'Malley | Konstantin Shvachko | | | Sporadic unit test failures (TestMiniMRClasspath, TestMiniMRLocalFS, TestMiniMRDFSCaching) | Blocker | . | Nigel Daley | Arun C Murthy | | | Map tasks can't timeout for failing to call progress | Blocker | . | Owen O'Malley | Arun C Murthy | | | map output transfers of more than 2^31 bytes output are failing | Blocker | . | Christian Kunz | Owen O'Malley |