When you come out of the loop, just subtract nNulls from i, so you'd have
the same array 0 4 4 10 10 16, but you'd pass through how many values are
in the array to your serialization function (instead of assuming that the
entire array of positions is being used).

We can't store trailing nulls, it'll mess things up.


On Fri, Feb 21, 2014 at 10:54 AM, ramkrish86 <notifications@github.com>wrote:

> In
> phoenix-core/src/main/java/org/apache/phoenix/expression/ArrayConstructorExpression.java:
>
> > +            int noOfElements =  children.size();
> > +            int elementLength = 0;
> > +            byteStream = new TrustedByteArrayOutputStream(estimatedSize);
> > +            oStream = new DataOutputStream(byteStream);
> > +            for (int i = position >= 0 ? position : 0; i < elements.length; i++) {
> > +                Expression child = children.get(i);
> > +                if (!child.evaluate(tuple, ptr)) {
> > +                    if (tuple != null && !tuple.isImmutable()) {
> > +                        if (position >= 0) position = i;
> > +                        return false;
> > +                    }
> > +                } else {
> > +                    // track the offset position here from the size of the byteStream
> > +                    if (!baseType.isFixedWidth()) {
> > +                        offset = byteStream.size();
> > +                        offsetPos[i] = offset;
>
> Take this case
> abc, null, bcd, null, null, b
> The offset for this would be as per the above logic where we get the
> offset both for nulls and non nulls
> 0 4 4 10 10 10
> Now while deserialization i know there are 6 elements and always we need
> to compare successive two elements to know the length
> For the first element it would do 4 - 0 = 4
> The next element would mean null 4 -4 = 0 (fine no problem)
> For the next element it is now 10 -4 = 6 (but this is wrong) because we
> already have the seperator byte added and the null value counter along with
> a seperator. So we need to create a logic to manipulate this. I have done
> that. Not a problem. So i would track nulls here and add two bytes to the
> currOff while reading the next element.
> The same happens for the last element, because we cannot compare last with
> anyother element we would know its offset and if there was a null
> previously then we need to adjust its offset to read the exact element.
> Now take a case where there are trailing nulls
>
> abc, null, bcd, null, ced, null. The offset array would be
> 0 4 4 10 10 16
> In this case
> For the first element 4 - 0 = 0
> second element = 4 - 4 = 0 (so null)
> 3rd element = 10 - 4 = 6 (but i have applied logic to skip the seperator
> byte) so i am able to read this
> 4th element = 10 -10 = 0
> 5th element = 16 - 10 = 6 (adjust offset)
> 6th element is actually a null. But how do i know that? Because we have
> only the last elements offset in hand with us. And using that we cannot
> infer the presence of null. Am i missing something here.
>
> I tried changing the logic of how we add the offset but that is again not
> easy in deserialization. That is why I thought better to write the number
> of trailing nulls.
>
> The logic to deal with byte buffer should actually know how many nulls are
> there, in that how many are repeating for us to fix the exact bytebuffer
> size. That again needs some tweak, but i have used this logic to find out
> the trailing nulls.
>
> --
> Reply to this email directly or view it on GitHub<https://github.com/apache/incubator-phoenix/pull/8/files#r9956836>
> .
>