A method of decoding video data, the method comprising: receiving a current block of video data encoded with a merge mode, the current block in a current picture, wherein in merge mode one motion vector candidate from a motion vector candidate list is selected to be used as the motion information for inter prediction of the current block; constructing a motion vector candidate list of merge candidates for the current block of video data, the motion vector candidate list comprising a number of spatial merge candidates, the spatial merge candidates comprising motion information from spatially neighboring blocks of the current block, wherein a spatially neighboring block of the current block is a block in the current picture neighboring the current block, wherein the number of spatial merge candidates is greater than 5, the constructing comprising: analyzing in the current picture spatially neighboring blocks to determine if the spatially neighboring blocks include motion information, wherein the number of analyzed spatially neighboring blocks is based on the size of the current block and is greater than 5; for each analyzed spatially neighboring block determined to include motion information using the motion information in a spatial merge candidate in the constructed motion vector candidate list; using a histogram of motion vector information for the spatially neighboring blocks, as a motion vector distribution of the spatially neighboring blocks, and a weight assigned to the motion information to determine based on the frequency of occurrence of the associated motion vectors and the weight the order of the spatial merge candidates in the motion vector candidate list wherein assigning a weight to each motion information for the neighboring blocks which include motion information comprises assigning a weight to a particular motion vector proportional to the size of the neighboring block that includes that motion information, wherein if two or more candidates have the same number of occurrences in the histogram, a predetermined order is used; determining a current motion vector from the motion vector candidate list; and decoding the current block of video data using the current motion vector. receiving a current block of video data encoded with a merge mode, the current block in a current picture, wherein in merge mode one motion vector candidate from a motion vector candidate list is selected to be used as the motion information for inter prediction of the current block; constructing a motion vector candidate list of merge candidates for the current block of video data, the motion vector candidate list comprising a number of spatial merge candidates, the spatial merge candidates comprising motion information from spatially neighboring blocks of the current block, wherein a spatially neighboring block of the current block is a block in the current picture neighboring the current block, wherein the number of spatial merge candidates is greater than 5, the constructing comprising: analyzing in the current picture spatially neighboring blocks to determine if the spatially neighboring blocks include motion information, wherein the number of analyzed spatially neighboring blocks is based on the size of the current block and is greater than 5; for each analyzed spatially neighboring block determined to include motion information using the motion information in a spatial merge candidate in the constructed motion vector candidate list; using a histogram of motion vector information for the spatially neighboring blocks, as a motion vector distribution of the spatially neighboring blocks, and a weight assigned to the motion information to determine based on the frequency of occurrence of the associated motion vectors and the weight the order of the spatial merge candidates in the motion vector candidate list wherein assigning a weight to each motion information for the neighboring blocks which include motion information comprises assigning a weight to a particular motion vector proportional to the size of the neighboring block that includes that motion information, wherein if two or more candidates have the same number of occurrences in the histogram, a predetermined order is used; analyzing in the current picture spatially neighboring blocks to determine if the spatially neighboring blocks include motion information, wherein the number of analyzed spatially neighboring blocks is based on the size of the current block and is greater than 5; for each analyzed spatially neighboring block determined to include motion information using the motion information in a spatial merge candidate in the constructed motion vector candidate list; using a histogram of motion vector information for the spatially neighboring blocks, as a motion vector distribution of the spatially neighboring blocks, and a weight assigned to the motion information to determine based on the frequency of occurrence of the associated motion vectors and the weight the order of the spatial merge candidates in the motion vector candidate list wherein assigning a weight to each motion information for the neighboring blocks which include motion information comprises assigning a weight to a particular motion vector proportional to the size of the neighboring block that includes that motion information, wherein if two or more candidates have the same number of occurrences in the histogram, a predetermined order is used; determining a current motion vector from the motion vector candidate list; and decoding the current block of video data using the current motion vector.