joomla/joomla-cms — pull request #5340 by chrisdavenport (2014-12-07 16:46:31+00:00)
Title: Improved HTML parser for Smart Search

This PR fixes several known issues with the HTML parser used by Smart Search. Most of the issues result from breaking the input string into 2Kb chunks to improve performance when saving large articles.

In particular

1. Correctly removes HTML tags even when part of the tag (usually a tag which has attributes) crosses the 2Kb boundary.
2. Correctly removes script blocks even when the block crosses the 2Kb boundary.
3. Removes embedded style blocks.
4. Also incorporates the fix from PR #5206 from @smanzi.

To test this PR

1. Make sure that Smart Search is setup and working.
2. Make sure you can save embedded JavaScript and stylesheet blocks: you might need to adjust your input filter settings and/or change your default editor to "none" in order to do this.
3. Create an article and enter some text then save it.
4. Check that the words in the article have been indexed (the quickest way to do this is to use the auto-completion feature on the front-end search forms).
5. Check that none of the HTML tag names or any words contained in attributes of the HTML tags, have been indexed.
6. Try to embed some JavaScript and check that none of the words contained in the JavaScript code have been indexed. You don't need to use real JavaScript to do this; just use some dummy text.
7. Try to embed some stylesheet declarations and check that none of the stylesheet keywords have been indexed. You don't need to use real CSS to do this; just use some dummy text.
8. If you're really clever you can try the same tests with the words or embedded blocks crossing a 2Kb boundary.