elasticsearch/logstash-forwarder — pull request #181 by a86c6f7964 (2014-03-23 04:25:08+00:00)
Title: Stop harvesting file iff at the end of a file and the name changed

I am not sure if this is the best way to deal with this or not, but at least to get a conversation started I thought that it would be a good idea to stop harvesting a file that has its name changed. Most likely it was log rotated. But here is my reasoning.

If configuration says watch files for A.log that means that we do not want to watch files for A-hour0.log or A-hour1.log just A.log
If A.log gets rotated to A-hour0.log then the application will stop writing to it and currently we will keep it around for 24 hours before releasing the handle. This seems rather clumsy, and could lead to LEGIT really quiet logs getting released.

Now if for whatever reason we would like to have A.log and A-hour*.log then that would be part of the config. This would probably be terrible because we would get all events written to A.log and then re get all of the events when it was rotated to A-hour0.log, but that is what we asked for. A.log would be one Harvester instance and A-hour0.log would be another Harvester instance.

Enter my proposal. If at any point the name of the file changes, then we do not want it anymore. We can figure out if the name of the file changes by just re reading the file stat of the original file name. If we did still want it, the config would be setup to read the new file anyhow.