Program: 1381
Project: 1381
Buggy source:
/** 
 * Encode the sp length idx and, if necessary, the run length. <p/> Start with the 64 bit hash as x. <p/> Find all the bits that belong in the first sp (roughly 25) bits. (This is idx') Get rid of the first p (roughly 18) bits of those. (Those were idx (not prime)) <p/> If all the remaining bits are zero then we are going to need to find and encode the full run length of leading zeros, but this only happens once in 2 ^ (sp - p) or roughly 2 ^ 7 times. <p/> If at least one of them is not zero, then since the run length is determined by bits after p and the idx' contains the first (sp - p) bits of those, then just by putting idx' in the encoding, we will also be giving it all the information it needs to find the run length. <p/> The relationship looks like this: <p/> *******************************************************   <- hashed length of bits | p bits = idx ||     look for leading zeros here     | |      sp bits = idx'     | | all 0s? | <p/> If we have idx', we could theoretically scan it (as per zeroTest) when unencoding and therefore know whether to look for the extra run length information. However, for now we have followed the authors of the paper and put this information in a flag bit at the end of the encoding. <p/> Based on this flag, we know whether to adjust for the missing run length bits. We could also use this flag to compress all the zeros in "| all 0s? |", but saving a byte or so once in 128 times is less than the 120 bits spent on the flag elsewhere. Of course, after compression, the losses are not quite so large either way. <p/> The encoding scheme now looks like: <p/> ********************************* <- smaller length of bits (half, but not to scale) | empty ||       sp bits     ||F| |  p bits  || has 1 ||0| <p/> <p/> or if the run length was needed (ie 'all 0s?' was indeed all 0s): <p/> * |      sp bits    || run len ||F| |  p bits ||  0s  |           |1| <p/> <p/> The other notable encoding feature is the inversion of the run length, which just lets the lists be sorted in a convenient way. (Could alternatively sort in reverse, and use descending deltas).
 * @param x  the hash bits
 * @param p  the 'normal' mode precision
 * @param sp the 'sparse' mode precision
 * @return the encoded data as an integer
 */
private int encodeHash(long x,int p,int sp){
  int idx=(int)(x >>> (64 - sp));
  int zeroTest=idx << ((32 - sp) + p);
  if (zeroTest == 0) {
    final int runLength=Long.numberOfLeadingZeros((x << this.p) | (1 << (this.p - 1))) + 1;
    int invrl=runLength ^ 63;
    return idx << 6 | invrl << 1 | 1;
  }
 else {
    return idx << 1;
  }
}

Fixed version (ground truth):
/** 
 * Encode the sp length idx and, if necessary, the run length. <p/> Start with the 64 bit hash as x. <p/> Find all the bits that belong in the first sp (roughly 25) bits. (This is idx') Get rid of the first p (roughly 18) bits of those. (Those were idx (not prime)) <p/> If all the remaining bits are zero then we are going to need to find and encode the full run length of leading zeros, but this only happens once in 2 ^ (sp - p) or roughly 2 ^ 7 times. <p/> If at least one of them is not zero, then since the run length is determined by bits after p and the idx' contains the first (sp - p) bits of those, then just by putting idx' in the encoding, we will also be giving it all the information it needs to find the run length. <p/> The relationship looks like this: <p/> *******************************************************   <- hashed length of bits | p bits = idx ||     look for leading zeros here     | |      sp bits = idx'     | | all 0s? | <p/> If we have idx', we could theoretically scan it (as per zeroTest) when unencoding and therefore know whether to look for the extra run length information. However, for now we have followed the authors of the paper and put this information in a flag bit at the end of the encoding. <p/> Based on this flag, we know whether to adjust for the missing run length bits. We could also use this flag to compress all the zeros in "| all 0s? |", but saving a byte or so once in 128 times is less than the 120 bits spent on the flag elsewhere. Of course, after compression, the losses are not quite so large either way. <p/> The encoding scheme now looks like: <p/> ********************************* <- smaller length of bits (half, but not to scale) | empty ||       sp bits     ||F| |  p bits  || has 1 ||0| <p/> <p/> or if the run length was needed (ie 'all 0s?' was indeed all 0s): <p/> * |      sp bits    || run len ||F| |  p bits ||  0s  |           |1| <p/> <p/> The other notable encoding feature is the inversion of the run length, which just lets the lists be sorted in a convenient way. (Could alternatively sort in reverse, and use descending deltas).
 * @param x  the hash bits
 * @param p  the 'normal' mode precision
 * @param sp the 'sparse' mode precision
 * @return the encoded data as an integer
 */
private int encodeHash(long x,int p,int sp){
  int idx=(int)(x >>> (64 - sp));
  int zeroTest=0;
  if (p < sp) {
    zeroTest=idx << ((32 - sp) + p);
  }
  if (zeroTest == 0) {
    final int runLength=Long.numberOfLeadingZeros((x << this.p) | (1 << (this.p - 1))) + 1;
    int invrl=runLength ^ 63;
    return idx << 6 | invrl << 1 | 1;
  }
 else {
    return idx << 1;
  }
}