[SPARK-3154][STREAMING] Make FlumePollingInputDStream shutdown cleaner.

Currently lot of errors get thrown from Avro IPC layer when the dstream
or sink is shutdown. This PR cleans it up. Some refactoring is done in the
receiver code to put all of the RPC code into a single Try and just recover
from that. The sink code has also been cleaned up.
The thing is this project does not depend on spark-core, its standalone. So it actually makes sense to add scala-library directly. However, the current code surprisingly compiles (both sbt and maven) even if we dont have scala-library as the dependency. Probably because scala compiler automatically brings in scala. 

I am of the opinion that we should add this back in, but ensure that this has the same version of scala as the spark core. Can you take a look at this, @srowen .