l3pp4rd/DoctrineExtensions — Issue #217: Searchable extension
Author: comfortablynumb
Created: 2011-12-30 20:13:42+00:00
State: open

Hi all,

I started to work yesterday on this. I want to create a simple, but flexible, way to search, without the need of using a real search engine. What I need is:

- A way to configure which fields are indexed, stored or both. Stored values are useful to use them directly in a search result list, without the need of making a new query for every object in the result.
- A way to configure how are the values processed at index time, query time or both (With "processed" I mean how are they tokenized, filtered, or something like that).
- A SearchRepository, which have a simple "search" method that accepts an array of conditions, limit parameters, etc.
- This array of conditions should be really flexible, allowing me to create a complex query with something like:

```php
$conditions = array(':OR:' => array(
'myField' => 'myValue',
':>=:' => array(
'myField2' => 'myValue2'),
':ALL_FIELDS:' => 'myValue3',
':OR:' => array(
'myField3' => 'myValue3',
'myField4' => 'myValue4'
)));
```

:KEYWORD: would be the format for special keywords that produce different additions to the final WHERE clause. With the example it should create something like (this is just an example. The real condition would have to take into account the different tokens of each value):

```php
WHERE ((o.field = 'myField' AND o.value = 'myValue') OR (o.field = 'myField2' AND o.value >= 'myValue2') OR (o.value = 'myValue3') OR ((o.field = 'myField3' AND o.value = 'myValue3') OR (o.field = 'myField4' AND o.value = 'myValue4')))
```

With something like this, it would be really easy to create forms that are able to create dynamic queries.

Another feature of this extension is that you'll be able to search for entities of all classes, or specific types of classes (using something like :CLASSES: to limit the classes to search for).

Now, the problems I'm having right now:

How to persist the indexed and stored data? I'm working on the ORM right now and I have something like this:

StoredObject (represents an object). Schema:

- id
- objectId
- class
- data (type: array. it's used to save the stored values)
- tokens (all the tokens for all the values of the object. I've just added this to configure the cascade={"remove"} functionality)

IndexedToken (represent a token for a value of a given field). Schema:

- id
- storedObject
- field
- token

StoredObject::data member is useful because at the moment of returning results from a search, the only thing you'd need to do is to access the stored values in this way:

```php
foreach ($results as $result) {
echo $result['data']['myStoredField'];
echo $result['data']['myStoredField2'];
}
```

The problem with this schema is that I can't find a way to order the results! I'm sure this schema is not the best way to do it. So I'd like to hear your opinions and advices on this guys.

So far, I have worked on the configuration and indexation of data. Configuration should be like:

```php
class Article
{
// More properties..

/**
* @ORM\Column(name="title", type="string")
* @Gedmo\Searchable(indexed=true, stored=true, processors={
* @Gedmo\SearchableProcessor(class="Gedmo\Searchable\Processor\Filter\TrimFilter"),
* @Gedmo\SearchableProcessor(class="Gedmo\Searchable\Processor\Tokenizer\DelimiterTokenizer")
* })
*/
private $title;
```

As you can see, you can use your own processors just defining your class. SearchableProcessor accepts an aditional option "parameters" which are aditional parameters for the constructor of the Processor.

Well, that's all for now. I wanted to work in some extensions and "Searchable" was the first in my wishlist. I'd really like to hear your opinions and advices about this guys.

Cheers.