contao/import — Issue #546: Large pages are not indexed for search engine
Author: importer
Created: 2011-11-29 16:26:22+00:00
State: open
Labels: Defect, framework

Hi,
I'm working on my third Typolight-site with some larger dan average pages (~400 kB text). The problem is that these are completely skipped by the search engine. The limit is about 16 kB per page.

The cause seems to be the preg_replace expressions in search.php (lines 97-99, 115-116). The resulting variables ($strContent, $strHead, $strBody) are empty as the page size exceeds ~16 kB.
Adding preg_last_error() to the code resulted in several 'PREG_BACKTRACK_LIMIT_ERROR'. A solution for this is setting 'pcre.backtrack_limit' in php.ini to a much higher value than the default 100000 (1000000 worked for me). However, I can change this on my local server, not at the provider's server.

Preg_replace itself is limited only by internal memory and I found several references of success with much bigger files but not complex pattern.

I hope you can work out a solution for this.

Greetings
Pieter

TL is really a great product. Well done.

[Download the attachments](http://www.contao.org/tl_files/tickets/546.zip "Download the attachments of this ticket")
Related issues: #2520, #2949

--- _Originally created by pdreef on February 26th, 2009, at 12:55am (ID 546)_