Don't use xmltodict. Use locally stored files. I've contacted the Data/API
team about that.


On Thu, Mar 20, 2014 at 2:09 PM, tristaneuan <notifications@github.com>wrote:

> In wikia_dstk/identify/services.py:
>
> > @@ -0,0 +1,204 @@
> > +import nltk
>
> Apologies in advance for the long comment.
>
> I'm not sure if you addressed the missing description_txt, headline_txt,
> and sitename_txt parses - the XML resulting from feeding these Solr fields
> through the parser was stored locally on nlp-s1 the first time around, but
> we never stored it on S3. This identify algorithm needs to extract NPs from
> these fields in order to work. Are you taking issue with using data cached
> locally on nlp-s1, or simply with the use of xmltodict to interpret it?
>
> Additionally, performing the query "headline_txt:*" on the xwiki core
> yields far fewer results than expected. Are the aforementioned 3 fields out
> of date?
>
> To be honest, porting the deprecated phrases_for_wiki_field service over
> just for this seems like more trouble than it's worth. I'm tempted to leave
> it out of the algorithm, but be aware that the results are going to be
> different, and I can't guarantee that it'll be as effective.
>
> --
> Reply to this email directly or view it on GitHub<https://github.com/Wikia/data-science-toolkit/pull/2/files#r10816466>
> .
>