Ok, base-uri(.) it is.

Yes I found the performance issue 5 mins ago actually and committed a
patch. Whenever px:mediatype-detect encountered an unknown filetype, it
tried to load it into memory. And I suppose there must've been some big
binary file around /d:fileset/d:file[1000] in my test fileset, because
that's where it crashed. The intention was to only load XML files to check
their namespace; not *all* files. So I've changed that and now it works
fine. A fileset at about 2500 files still clocks in at around 35 seconds
though (with 20-30 of the files loaded from disk), and I think that could
be further improved if we were able to parse only the opening tag of the
root element instead of loading the entire XML file into memory. Really big
XML files might still crash the step though (think gigabytes). Or if the
*.xml file isn't actually XML.

Something good came of it though; px:fileset-filter is much more efficient
(and the code is simpler as well) than before.

Jostein


On 14 February 2013 11:07, Romain Deltour <notifications@github.com> wrote:

> In fileset-utils/src/main/resources/xml/xproc/fileset-filter.xpl:
>
> >
> > +    <p:choose>
> > +        <p:when test="$href=''">
> > +            <p:identity/>
> > +        </p:when>
> > +        <p:otherwise>
> > +            <p:delete>
> > +                <p:with-option name="match" select="concat(&quot;//d:file[not(resolve-uri(@href,'&quot;,$base,&quot;')=resolve-uri('&quot;,$resolved-href,&quot;'))]&quot;)"/>
>
> I don't think base-uri(.) and $base are necessarily the same, nothing
> really prevents us to add "out of base" resources to a file set in some
> tweaked use case, so I'd be more confortable with using base-uri(.), which
> is correct 100% of the case.
>
> I don't think that particular point would be a performance issue (since
> the XPath processor has direct access to the node's base URI as part of the
> XDM).
>
> Interesting about the performance issues though. Do you know what's the
> bottleneck ?
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/daisy-consortium/pipeline-modules-common/pull/1/files#r3010582>.
>
>