sorry spoke to soon.
when you add a new element that has not been observed before and the
structure is at capacity it kicks out the element with the lowest count. so
the data structure is sensitive to insertion order and structural equality
will not work. all that can be shown is that two probabilistic data
structures are consistent with each other.


On Mon, Feb 10, 2014 at 9:08 PM, Koert Kuipers <koert@tresata.com> wrote:

> (left consistentWith right) && (right consistentWith left) works.
>
> Doing the test based on equality also works, with equality based on
> counters and m.
> I chose not to implement equality since i thought it is not very
> meaningful for a probabilistic data structure, however it is easy to add
> (to the sealed abstract class I guess). Would you prefer that?
> On Feb 10, 2014 6:58 PM, "P. Oscar Boykin" <notifications@github.com>
> wrote:
>
>> In
>> algebird-test/src/test/scala/com/twitter/algebird/StreamSummaryTest.scala:
>>
>> > +
>> > +import org.scalacheck.Arbitrary
>> > +import org.scalacheck.Properties
>> > +import org.scalacheck.Gen
>> > +import org.scalacheck.Gen.frequency
>> > +
>> > +object StreamSummaryLaws extends Properties("StreamSummary") {
>> > +  import BaseProperties._
>> > +
>> > +  implicit val ssMonoid = new StreamSummaryMonoid
>> > +  // need a non-uniform distro
>> > +  implicit val ssGen: Arbitrary[com.twitter.algebird.StreamSummary[Int]] = Arbitrary {
>> > +    for (key: Int <- frequency((1 to 10).map{ x => (x * x, x : Gen[Int]) }: _*)) yield StreamSummary(4, key)
>> > +  }
>> > +
>> > +  property("StreamSummary is a Monoid") = commutativeMonoidLawsEq[StreamSummary[Int]] { (left, right) => left consistentWith right }
>>
>> hmmm. At least we need (left consistentWith right) && (right
>> consistentWith left).
>>
>> I am a bit nervous here. I tend to think the heavy hitters in sketch map
>> was a mistake, and we should have kept it associative. Can you expand on
>> your thoughts there about these not being structurally equal?
>>
>> --
>> Reply to this email directly or view it on GitHub<https://github.com/twitter/algebird/pull/259/files#r9608530>
>> .
>>
>