We can also hide a bit of the type verbosity for common cases, eg working
with strings. So, we can do:

trait StringSlab[+I <: StringAnnotation] extends
Slab[String,StringAnnotation,I]

trait StringAnalysisFunction[I<:StringAnnotation,O<:StringAnnotation]
extends (Slab[String,StringAnnotation,I] =>
Slab[String,StringAnnotation,StringAnnotation with I with O])
ao

Which then makes it possible to define Tokenizer this way:

trait Tokenizer[I <: Sentence] extends StringAnalysisFunction[I, Token] {
  def apply(slab: StringSlab[I]) =
    // the [Token] is required because of
https://issues.scala-lang.org/browse/SI-7647
    slab.++[Token](slab.iterator[Sentence].flatMap(sentence =>
      "\\p{L}+|\\p{P}+|\\p{N}+".r.findAllMatchIn(sentence.in(slab).content).map(m
=>
        Token(sentence.begin + m.start, sentence.begin + m.end))))
}

Rather than this way:

trait Tokenizer[I <: Sentence] extends AnalysisFunction[String,
StringAnnotation, I, Token] {
  def apply(slab: Slab[String, StringAnnotation, I]) =
    // the [Token] is required because of
https://issues.scala-lang.org/browse/SI-7647
    slab.++[Token](slab.iterator[Sentence].flatMap(sentence =>
      "\\p{L}+|\\p{P}+|\\p{N}+".r.findAllMatchIn(sentence.in(slab).content).map(m
=>
        Token(sentence.begin + m.start, sentence.begin + m.end))))
}




On Mon, Jul 22, 2013 at 9:42 AM, Ryan LeCompte <notifications@github.com>wrote:

> In src/main/scala/chalk/slab/AnalysisFunction.scala:
>
> > @@ -19,30 +19,41 @@ object StringIdentityAnalyzer extends AnalysisFunction[String, StringAnnotation,
> >  /**
> >    * A simple regex sentence segmenter.
> >    */
> > -trait SentenceSegmenter extends AnalysisFunction[String, StringAnnotation, StringAnnotation, Sentence] {
> > -  def apply(slab: Slab[String, StringAnnotation, StringAnnotation]) =
> > -    slab ++ "[^\\s.!?]+[^.!?]+[.!?]".r.findAllMatchIn(slab.content).map(m => Sentence(m.start, m.end))
> > +trait SentenceSegmenter[I <: StringAnnotation] extends AnalysisFunction[String, StringAnnotation, I, Sentence] {
>
> You're correct, in that case I think the only way to do it is the current
> approach. Thanks!
> … <#14006d520196277c_>
> On Jul 22, 2013, at 10:39 AM, bethard <notifications@github.com> wrote:
> In src/main/scala/chalk/slab/AnalysisFunction.scala: > @@ -19,30 +19,41 @@
> object StringIdentityAnalyzer extends AnalysisFunction[String,
> StringAnnotation, > /** > * A simple regex sentence segmenter. > */ >
> -trait SentenceSegmenter extends AnalysisFunction[String, StringAnnotation,
> StringAnnotation, Sentence] { > - def apply(slab: Slab[String,
> StringAnnotation, StringAnnotation]) = > - slab ++
> "[^\\s.!?]+[^.!?]+[.!?]".r.findAllMatchIn(slab.content).map(m =>
> Sentence(m.start, m.end)) > +trait SentenceSegmenter[I <: StringAnnotation]
> extends AnalysisFunction[String, StringAnnotation, I, Sentence] { I like
> the readability in your proposal, but I think we'd lose the ability to use
> a regular Function1 as an analyzer, no? How would you write trait
> AnalysisFunction[C,B,I<:B,O<:B] extends (Slab[C,B,I] => Slab[C,B,B with I
> with O]) in your proposal? — Reply to this email directly or view it on
> GitHub.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/scalanlp/chalk/pull/20/files#r5320125>
> .
>



-- 
Jason Baldridge
Associate Professor, Dept. of Linguistics, UT Austin
Co-founder & Chief Scientist, People Pattern
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge