Fix errors in usages of try_compile

* There's no OPENCV_BINARY_DIR variable;
* No need to append CMakeFiles/CMakeTmp, as CMake does it for you;
* Output variables are unused;
* Wrong usage of CMAKE_FLAGS;
* Small quoting and style issues.
Merge pull request #1324 from SpecLad:try-compile