I think that's the right thing. At least they can grep the logs to see that
happened. Otherwise they get zero notification that these events were lost.
If it happens too much, I think we have an issue.


On Fri, Oct 25, 2013 at 1:41 PM, ianoc <notifications@github.com> wrote:

> In
> summingbird-builder/src/main/scala/com/twitter/summingbird/scalding/VersionedState.scala:
>
> >    def begin: PrepareState[Interval[Timestamp]] = new VersionedPrepareState
> >
> >    private class VersionedPrepareState extends PrepareState[Interval[Timestamp]] {
> > +    def newestCompleted: Option[BatchID] =
> > +      meta.versions.map { vers =>
> > +        val thisMeta = meta(vers)
> > +        thisMeta
> > +          .get[String]
> > +          .flatMap { str => ScalaTry(BatchID(str)) } match {
> > +            case Success(batchID) => Some(batchID)
> > +            case Failure(ex) =>
> > +              logger.warn("Path: {} missing or corrupt completion file. Ignoring and trying previous",
>
> Will this not result in spurious error logging if the older data doesn't
> have this file but the latest does ok?
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/twitter/summingbird/pull/337/files#r7227745>
> .
>



-- 
Oscar Boykin :: @posco :: http://twitter.com/posco