github/linguist — pull request #1378 by mkrautz (2014-07-13 15:45:38+00:00)
Title: Allow Qt translation files with the .ts extension to be detected as XML instead of TypeScript

Hi,

Since shortly after the release of TypeScript, the main Mumble repository (https://github.com/mumble-voip/mumble) has been detected as mostly TypeScript code by linguist. But It's not TypeScript. It's Qt-style C++.

I've gotten tired of seeing that in our repo header, and took a stab at fixing it.

The actual changes I've done are:

- Added `.ts` as a file extension for XML in `languages.yml` (this is in addition to the `.ts` file extension for TypeScript, which is already in the list)
- Copied the Qt translation file at `samples/XML/pt_BR.xml` file to `samples/XML/pt_BR.ts`.
- Ran `bundle exec rake tests`

After doing this, I had an issue where the sample file at "samples/TypeScript/empty.ts" would be misdetected/undetected.

This was because of https://github.com/mkrautz/linguist/compare/fix-qt-ts-detection?expand=1#diff-5565bb45d402d49dbf0d584129df265dL128 - which returns nil for empty files. I fixed that by removing the empty check.

Once the empty file was able to be detected as *something*, it would be detected as XML by the classifier (lib/linguist/classifier.rb), causing two tests to fail:

test_language(TestBlob) [/home/mkrautz/linguist/test/test_blob.rb:437]:
TypeScript/empty.ts.
<"TypeScript"> expected but was
<"XML">.

This is obviously because the `empty.ts` file in the TypeScript directory was detected as XML by lib/linguist/classifier.rb.

The other failing test was test_classifier.rb:

test_classify_ambiguous_languages(TestClassifier) [/home/mkrautz/linguist/test/test_classifier.rb:63]:
/home/mkrautz/linguist/samples/TypeScript/empty.ts
[["XML", -4.060443010546419], ["TypeScript", -5.66988092298052]].
<"TypeScript"> expected but was
<"XML">.

The test_blob.rb failure, I was able to "fix" by enabling the heuristic module by:
- Setting ACTIVE to true
- Fixing the `find_by_heuristics` method to correctly return the results of the calls to the `disambiguate_...` methods by making the subsequent `if` statements in `find_by_heuristics` into `elsif`s.

This causes the `samples/TypeScript/empty.ts` file to correctly identified as TypeScript. (At least, I think it makes sense to have an empty .ts file be identified as TypeScript over identifying it as XML -- and the old? heuristics code seems to agree with that thought.)

At this point, the test_classifier.rb failure was still present. In this changeset, I've silenced the failure by skipping the test if the sample file ends with "`empty.ts`".

Now, this is obviously not something you can ship (I think?). But I'm hoping that this PR can get the ball rolling on getting a cleaner fix. So: take the contents of this PR with a grain of salt. It's just there to get the discussion for a proper fix started, and to better identify (via code/patches) the problems I've faced in "fixing" this issue.

I'm happy to implement any suggested fixes, if needed.

Thanks for the consideration!