HTML Parsing Cleanup
State: closed

- Can pass `Flavor` to HTML parse (closes #4594 and #5130). Instead of #5131
- Cleanup API for HTML parsers

I added new (public class) `Flavor` which users can subclass. Currently the expectation is to pass in the class (not an instance of the class)

CC @cpcloud @jtratner @jreback 


Labels:
name: API Design
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
can't add strict=False, that's pretty much the same as supporting a lxml-liberal parser. better to factor out the parser as you suggested.

@jtratner No more strict

It would be great to get this in for v13. If that is feasible and what I have looks reasonable let me know and I will write up docs, etc.

@jtratner Anything else you want me to change before rebase?

I'll take a look after you rebase and make the changes.

I am rebasing and adding release notes now. I think I made the requested changes.

@jtratner @cpcloud What about this?

Bump?

Bump
@cpcloud @jtratner @jreback 

Any way to get this in for v13?

I will try to take a look over the weekend, but I'm hesitant to make
changes to this tricky-ish part of the code right before a release.

@cpcloud Changes pushed.

@cpcloud Any other issues you see?

Is this pull request planned to be merged? Thanks

Yes, probably in the next month or so...I just need to find a bit of time to dot T's and cross I's :)

Any updates on this?

I hope that this PR is still an acceptable addition to pandas even given the other discussions we have had regarding feature creep.

I feel this represents only a minor change to pandas itself but allows quite a lot of user extensibility.

I would say remove the flavor stuff and it's okay by me (which just leaves the IO change). @y-p?

The flavor addition is was makes this PR useful.
On Jan 24, 2014 12:12 PM, "Phillip Cloud" notifications@github.com wrote:

> I would say remove the flavor stuff and it's okay by me (which just leaves
> the IO change). @y-p https://github.com/y-p?
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/5395#issuecomment-33256543
> .

As I said, I don't think the flavor addition is feature creep. It just
empowers the user.
On Jan 24, 2014 12:20 PM, "Alex Rothberg" agrothberg@gmail.com wrote:

> The flavor addition is was makes this PR useful.
> On Jan 24, 2014 12:12 PM, "Phillip Cloud" notifications@github.com
> wrote:
> 
> > I would say remove the flavor stuff and it's okay by me (which just
> > leaves the IO change). @y-p https://github.com/y-p?
> > 
> > —
> > Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/pull/5395#issuecomment-33256543
> > .

<del>So would a shark with a freakin' laser, but that's out of scope too</del>

@cancan101 , we undertsand. You have lots of good ideas on how `read_html` could
be made more featureful. But, we want to keep read_html small and are fairly happy with the
scope it currently has. I again invite you to start your own project that takes these
ideas and builds a useful, focused tool to serve users that need more powerful HTML/pandas
capabilities.

We'll be more then glad to include it on the new ["pandas ecosystem"](http://pandas.pydata.org/pandas-docs/dev/ecosystem.html) section of the docs.

@cpcloud what are we doing with this?

closing in favor of not having sharks with laser beams

@cancan101 as noted above....we REALLY appreciate all the effort you have made towards making pandas better. This would be a nice separate project. Clearly if we need hooks to sub-class (the parsing engine wrapper) would be in favor of that.

Indeed. @cancan101 You have a lot of good ideas about making a more full featured version of this .... keep your ideas coming.

Since the new html parsing library does not yet exist, it would be great to have some means of (ie place to record) collecting feature requests and ideas beforehand . This applies to not just html parsing but also to other feature requests for which their addition to pandas would be feature creep.

In this specific case, I would want to avoid duplicating existing code so at some point it will make sense to see what hooks would be needed allow a user to tie in an external library.

I believe the wiki is public, so you could create an Enhancements wanted page?

Could (some of) this functionality be considered for a pandas-data(?) subproject (allowing for additional library dependencies)?

sure 

@gliptak stepping up to manage????