elasticsearch/elasticsearch — Issue #1551: Additions to the River API
Author: skade
Created: 2011-12-20 21:17:00+00:00
State: open

(This is somewhat related to this couchdb-river-related issue):

https://github.com/elasticsearch/elasticsearch-river-couchdb/issues/3

I'd like to suggest some additional actions for the river API:

1) stop: Allow Rivers to instruct ES to stop themselves. This allows to write Rivers that do one-shot tasks (like migrations etc.) and indicate to ES once they are done. Currently, this can be done by letting the river delete itself using the client passed to itself, but this is more of a hack.
2) restart: Allow Rivers to flag themselves for restart (close -> start). This would allow rivers to "fail early". In the issue given above, I do have the impression that the CouchDB river actually fails at reconstructing its proper runtime state. It could be better to just let the river fail and shut down properly and then let ES restart it, which would make it much easier to assess in which state the river actually was when failing. Also, it takes the burden of implementing full crash-safety from the river, as implementations can decide to handle specific failures by requesting a restart.
3) store/get state: Allow the river to persist its state through a blessed API. At the moment, this can be done by writing directly to the river index (e.g. /_river/myriver/_mystate), but keeping this consistent across rivers might be worthwhile. This comes in handy when using restart strategies as in 2).

If there is interest in these changes, I would try to implement them myself.