Repository: owncloud/ocis
Pull request #2723 (2021-11-03 17:38:40)

Allow rendering a text preview using custom fonts
<!--
Thanks for submitting a change to ownCloud!

This is the bug tracker for the oCIS component. Find other components at https://github.com/owncloud/core/blob/master/.github/CONTRIBUTING.md#guidelines

For fixing potential security issues please see https://owncloud.org/security/

To make it possible for us to get your change reviewed and merged please carefully fill out the requested information below.

Please note that any kind of change needs first be submitted to the master branch which holds the next version of oCIS.

Please set the following labels:

- Set label "Status:Needs-Review" for review or "Status:In-Progress" if the PR still has open tasks.
- Assignment: assign to self
- Reviewers: pick at least one
-->

## Description
Allow usage of custom fonts for the text previews so non-latin chars (or chars not supported by the "goregular" font) can be shown

## Related Issue
https://github.com/owncloud/ocis/issues/2620

## Motivation and Context
Files containing non-latin can have a proper preview

## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
- test environment:
- test case 1:
- test case 2:
- ...

## Screenshots (if appropriate):

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Technical debt
- [ ] Tests only (no source changes)

## Checklist:
<!-- Tick the checkboxes when done. -->
<!-- Raise documentation ticket in owncloud.github.io/ -->
- [ ] Code changes
- [ ] Unit tests added
- [ ] Acceptance tests added
- [ ] Documentation ticket raised: <link> 

### Expected usage

Start the thumbnails service using the `THUMBNAILS_TXT_FONTMAP_FILE` pointing to a json file, for example `THUMBNAILS_TXT_FONTMAP_FILE=/myfonts/fontMap.json`. Note that all the fonts referred by the font map file must be downloaded manually and be present in the FS.

The json file must contain a json object with the following info:
* A "fontMap" key, which contains a map of script => font_file
* A "defaultFont" key, which contains the default font to be used in case the target script isn't found in the fontMap

An example of font map file can be the following
```
{
        "fontMap": {
                "Arabic": "NotoSansArabic-Regular.ttf",
                "Devanagari": "NotoSansDevanagari-Regular.ttf",
                "Latin": "NotoSans-Regular.ttf",
                "Hangul": "NotoSansMonoCJKkr-Regular.otf",
                "Han": "NotoSansMonoCJKsc-Regular.otf",
                "Hiragana": "NotoSansMonoCJKjp-Regular.otf",
                "Katakana": "NotoSansMonoCJKjp-Regular.otf"
        },
        "defaultFont": "NotoSans-Regular.ttf"
}
```
A full list of available scripts can be found in https://cs.opensource.google/go/go/+/refs/tags/go1.17.2:src/unicode/tables.go;l=3655

In case the target script isn't mapped, the default font will be used. The character of that script will likely appear as a white box instead of the expected character.

The font map contains paths to the font files. The paths can be absolute or relative as shown below:
```
{
        "fontMap": {
                "Arabic": "NotoSansArabic-Regular.ttf",
                "Devanagari": "NotoSansDevanagari-Regular.ttf",
                "Latin": "/customFonts/test001.ttf",
                "Hangul": "NotoSansMonoCJKkr-Regular.otf",
                "Han": "NotoSansMonoCJKsc-Regular.otf",
                "Hiragana": "NotoSansMonoCJKjp-Regular.otf",
                "Katakana": "NotoSansMonoCJKjp-Regular.otf"
        },
        "defaultFont": "/customFonts/test001.ttf"
}
```
If the path is absolute, the font will be loaded directly from that path.
If the path is relative, it will be relative to the directory where the font map file is placed. For example, if we assume that the "/fontMaps/fontMap1.json" file contains the content above, the "NotoSans*" paths will refer to "/fontMaps/NotoSans*"
This allows the usage of self-contained packages of fonts that are ready to be used. The thumbnails service would just need to point to the font map file contained in the package.

### Known issues

* Arabic (and likely any right-to-left) script won't be rendered properly. Glyphs will be printed left-to-right
* Glyphs that might require repositioning or that are a combination of 2 or more glyphs, won't be rendered properly. This has been noticed with the Devanagari script such as `विश्वस्त हुनुहोस् तपाइँको`. Glyphs will be printed separately one after another.
* Punctuation symbols might not be rendered properly in some circumstances. This usually involves using multiple scripts such as in `附加到电子邮件的未版本化文档以及安全性有问题的公共云中的影子 IT。`. In the example, the last character is used in chinese or japanese languages, and latin fonts (such as NotoSans-Regular.ttf) doesn't contain that char. The problem is that the char is wrongly interpreted as latin because it's preceded by a latin char.
* Text file MUST be encoded in UTF-8. Text encoded in any other format will show random chars.

### To Do List
* Include tests
* Check possible concurrency problems (docs say that the `font.Face` isn't thread-safe)

update-docs[bot]: Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a [changelog](https://github.com/owncloud/ocis/blob/master/changelog/README.md) item based on your changes.

ownclouders: <details><summary>:boom: Acceptance tests <strong>Web-Tests-ocis-ocis-storage-1</strong> failed. The build is cancelled...</summary>



jvillafanez: We'll leave the known issues aside.

Both first and second issues are too complex to tackle on our own just for a text preview. It involves a text shaping mechanism which isn't present in the "golang.org/x/image" package, and I didn't found alternative packages (maybe https://github.com/benoitkugler/textlayout).
Note that this goes beyond the initial thought of the problem. It isn't "just" draw the chars from right to left, or reposition a char, but substituting a combination of glyphs for a different one.
ownclouders: <details><summary>:boom: Acceptance tests <strong>accountsUITests</strong> failed. The build is cancelled...</summary>



ownclouders: <details><summary>:boom: Acceptance tests <strong>Web-Tests-ocis-ocis-storage-1</strong> failed. The build is cancelled...</summary>



sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [17 Code Smells](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL)

[![41.1%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/40-16px.png '41.1%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list) [41.1% Coverage](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list)  
[![4.9%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/5-16px.png '4.9%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list) [4.9% Duplication](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list)


sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [6 Code Smells](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL)

[![42.3%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/40-16px.png '42.3%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list) [42.3% Coverage](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list)  
[![0.7%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.7%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list) [0.7% Duplication](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list)


ownclouders: :boom: Acceptance test [<strong>localApiTests-apiArchiver-ocis</strong>](https://drone.owncloud.com/owncloud/ocis/7773/21/1) failed. Further test are cancelled...
<!-- id: pr-2723 -->

sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL)

[![43.5%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/40-16px.png '43.5%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list) [43.5% Coverage](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list)  
[![0.6%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.6%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list)


sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL)

[![43.7%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/40-16px.png '43.7%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list) [43.7% Coverage](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list)  
[![0.6%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.6%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list)


sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=owncloud_ocis&pullRequest=2723&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL) [5 Code Smells](https://sonarcloud.io/project/issues?id=owncloud_ocis&pullRequest=2723&resolved=false&types=CODE_SMELL)

[![43.7%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/40-16px.png '43.7%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list) [43.7% Coverage](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_coverage&view=list)  
[![0.6%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.6%')](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list) [0.6% Duplication](https://sonarcloud.io/component_measures?id=owncloud_ocis&pullRequest=2723&metric=new_duplicated_lines_density&view=list)



Patch:
diff --git a/thumbnails/pkg/config/config.go b/thumbnails/pkg/config/config.go
index 60350383497..5126a2d60d7 100644
--- a/thumbnails/pkg/config/config.go
+++ b/thumbnails/pkg/config/config.go
@@ -66,6 +66,7 @@ type Thumbnail struct {
 	CS3AllowInsecure    bool              `ocisConfig:"cs3_allow_insecure"`
 	RevaGateway         string            `ocisConfig:"reva_gateway"`
 	WebdavNamespace     string            `ocisConfig:"webdav_namespace"`
+	FontMapFile         string            `ocisConfig:"font_map_file"`
 }
 
 // New initializes a new configuration with or without defaults.
diff --git a/thumbnails/pkg/config/mappings.go b/thumbnails/pkg/config/mappings.go
index 3a75427c061..6f6ef0d2fa1 100644
--- a/thumbnails/pkg/config/mappings.go
+++ b/thumbnails/pkg/config/mappings.go
@@ -91,6 +91,10 @@ func structMappings(cfg *Config) []shared.EnvBinding {
 			EnvVars:     []string{"THUMBNAILS_GRPC_NAMESPACE"},
 			Destination: &cfg.Server.Namespace,
 		},
+		{
+			EnvVars:     []string{"THUMBNAILS_TXT_FONTMAP_FILE"},
+			Destination: &cfg.Thumbnail.FontMapFile,
+		},
 		{
 			EnvVars:     []string{"THUMBNAILS_FILESYSTEMSTORAGE_ROOT"},
 			Destination: &cfg.Thumbnail.FileSystemStorage.RootDirectory,
diff --git a/thumbnails/pkg/preprocessor/fontloader.go b/thumbnails/pkg/preprocessor/fontloader.go
new file mode 100644
index 00000000000..3053abaed9b
--- /dev/null
+++ b/thumbnails/pkg/preprocessor/fontloader.go
@@ -0,0 +1,188 @@
+package preprocessor
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"time"
+
+	"github.com/owncloud/ocis/ocis-pkg/sync"
+	"golang.org/x/image/font"
+	"golang.org/x/image/font/gofont/goregular"
+	"golang.org/x/image/font/opentype"
+)
+
+// FontMap maps a script with the target font to be used for that script
+// It also uses a DefaultFont in case there isn't a matching script in the map
+//
+// For cases like Japanese where multiple scripts are used, we rely on the text
+// analyzer to use the script which is unique to japanese (Hiragana or Katakana)
+// even if it has to overwrite the "official" detected script (Han). This means
+// that "Han" should be used just for chinese while "Hiragana" and "Katakana"
+// should be used for japanese
+type FontMap struct {
+	FontMap     map[string]string `json:"fontMap"`
+	DefaultFont string            `json:"defaultFont"`
+}
+
+// It contains the location of the loaded file (in FLoc) and the FontMap loaded
+// from the file
+type FontMapData struct {
+	FMap *FontMap
+	FLoc string
+}
+
+// It contains the location of the font used, and the loaded face (font.Face)
+// ready to be used
+type LoadedFace struct {
+	FontFile string
+	Face     font.Face
+}
+
+// Represents a FontLoader. Use the "NewFontLoader" to get a instance
+type FontLoader struct {
+	faceCache   sync.Cache
+	fontMapData *FontMapData
+	faceOpts    *opentype.FaceOptions
+}
+
+// Create a new FontLoader based on the fontMapFile. The FaceOptions will
+// be the same for all the font loaded by this instance.
+// Note that only the fonts described in the fontMapFile will be used.
+//
+// The fontMapFile has the following structure
+//	{
+//		"fontMap": {
+//			"Han": "packaged/myFont-CJK.otf",
+//			"Arabic": "packaged/myFont-Arab.otf",
+//			"Latin": "/fonts/regular/myFont.otf"
+//		}
+//		"defaultFont": "/fonts/regular/myFont.otf"
+//	}
+//
+// The fontMapFile contains paths to where the fonts are located in the FS.
+// Absolute paths can be used as shown above. If a relative path is used,
+// it will be relative to the fontMapFile location. This should make the
+// packaging easier since all the fonts can be placed in the same directory
+// where the fontMapFile is, or in inner directories.
+func NewFontLoader(fontMapFile string, faceOpts *opentype.FaceOptions) (*FontLoader, error) {
+	fontMap := &FontMap{}
+
+	if fontMapFile != "" {
+		file, err := os.Open(fontMapFile)
+		if err != nil {
+			return nil, err
+		}
+		defer file.Close()
+
+		parser := json.NewDecoder(file)
+		if err = parser.Decode(fontMap); err != nil {
+			return nil, err
+		}
+	}
+
+	return &FontLoader{
+		faceCache: sync.NewCache(5),
+		fontMapData: &FontMapData{
+			FMap: fontMap,
+			FLoc: fontMapFile,
+		},
+		faceOpts: faceOpts,
+	}, nil
+}
+
+// Load and return the font face to be used for that script according to the
+// FontMap set when the FontLoader was created. If the script doesn't have
+// an associated font, a default font will be used. Note that the default font
+// might not be able to handle properly the script
+func (fl *FontLoader) LoadFaceForScript(script string) (*LoadedFace, error) {
+	var parsedFont *opentype.Font
+	var parsingError error
+
+	fontFile := fl.fontMapData.FMap.DefaultFont
+	if val, ok := fl.fontMapData.FMap.FontMap[script]; ok {
+		fontFile = val
+	}
+
+	if fontFile != "" && !filepath.IsAbs(fontFile) {
+		fontFile = filepath.Join(filepath.Dir(fl.fontMapData.FLoc), fontFile)
+	}
+
+	// if the face for the script isn't cached, load the font file and create a new face
+	cachedFace := fl.faceCache.Load(fontFile)
+	if cachedFace != nil {
+		return cachedFace.V.(*LoadedFace), nil
+	}
+
+	if fontFile == "" {
+		parsedFont, parsingError = opentype.Parse(goregular.TTF)
+		if parsingError != nil {
+			return nil, parsingError
+		}
+	} else {
+		// opentype.ParseReaderAt seems to require to keep the file opened
+		// so read the font file into memory
+		data, err := os.ReadFile(fontFile)
+		if err != nil {
+			return nil, err
+		}
+		parsedFont, parsingError = opentype.Parse(data)
+		if parsingError != nil {
+			return nil, parsingError
+		}
+	}
+
+	face, err := opentype.NewFace(parsedFont, fl.faceOpts)
+	if err != nil {
+		return nil, err
+	}
+
+	loadedFace := &LoadedFace{
+		FontFile: fontFile,
+		Face:     face,
+	}
+	fl.faceCache.Store(fontFile, loadedFace, time.Now().Add(10*time.Minute))
+	return loadedFace, nil
+}
+
+func (fl *FontLoader) GetFaceOptSize() float64 {
+	return fl.faceOpts.Size
+}
+
+func (fl *FontLoader) GetFaceOptDPI() float64 {
+	return fl.faceOpts.DPI
+}
+
+func (fl *FontLoader) GetScriptList() []string {
+	fontMap := fl.fontMapData.FMap.FontMap
+
+	arePresent := map[string]bool{
+		"Common":    false,
+		"Inherited": false,
+	}
+	listSize := len(fontMap)
+
+	for key := range arePresent {
+		if _, inFontMap := fontMap[key]; inFontMap {
+			arePresent[key] = true
+		} else {
+			listSize++
+		}
+	}
+
+	keys := make([]string, listSize)
+
+	i := 0
+	for k := range fontMap {
+		keys[i] = k
+		i++
+	}
+
+	for script, isPresent := range arePresent {
+		if !isPresent {
+			keys[i] = script
+			i++
+		}
+	}
+	return keys
+}
diff --git a/thumbnails/pkg/preprocessor/preprocessor.go b/thumbnails/pkg/preprocessor/preprocessor.go
index b0d22d8e3ec..f521967d57b 100644
--- a/thumbnails/pkg/preprocessor/preprocessor.go
+++ b/thumbnails/pkg/preprocessor/preprocessor.go
@@ -2,21 +2,17 @@ package preprocessor
 
 import (
 	"bufio"
-	"github.com/golang/freetype"
-	"github.com/golang/freetype/truetype"
-	"github.com/pkg/errors"
-	"golang.org/x/image/font"
-	"golang.org/x/image/font/gofont/goregular"
 	"image"
 	"image/draw"
 	"io"
+	"math"
 	"mime"
 	"strings"
-)
 
-const (
-	fontSize         = 12
-	spacing  float64 = 1.5
+	"github.com/pkg/errors"
+	"golang.org/x/image/font"
+	"golang.org/x/image/font/opentype"
+	"golang.org/x/image/math/fixed"
 )
 
 type FileConverter interface {
@@ -33,73 +29,160 @@ func (i ImageDecoder) Convert(r io.Reader) (image.Image, error) {
 	return img, nil
 }
 
-type TxtToImageConverter struct{}
+type TxtToImageConverter struct {
+	fontLoader *FontLoader
+}
 
 func (t TxtToImageConverter) Convert(r io.Reader) (image.Image, error) {
 	img := image.NewRGBA(image.Rect(0, 0, 640, 480))
-	draw.Draw(img, img.Bounds(), image.White, image.Point{}, draw.Src)
-
-	c := freetype.NewContext()
-	// Ignoring the error since we are using the embedded Golang font.
-	// This shouldn't return an error.
-	f, _ := truetype.Parse(goregular.TTF)
-	c.SetFont(f)
-	c.SetFontSize(fontSize)
-	c.SetClip(img.Bounds())
-	c.SetDst(img)
-	c.SetSrc(image.Black)
-	c.SetHinting(font.HintingFull)
-	pt := freetype.Pt(10, 10+int(c.PointToFixed(fontSize)>>6))
+
+	imgBounds := img.Bounds()
+	draw.Draw(img, imgBounds, image.White, image.Point{}, draw.Src)
+
+	fontSizeAsInt := int(math.Ceil(t.fontLoader.GetFaceOptSize()))
+	margin := 10
+	minX := fixed.I(imgBounds.Min.X + margin)
+	maxX := fixed.I(imgBounds.Max.X - margin)
+	maxY := fixed.I(imgBounds.Max.Y - margin)
+	initialPoint := fixed.P(imgBounds.Min.X+margin, imgBounds.Min.Y+margin+fontSizeAsInt)
+	canvas := &font.Drawer{
+		Dst: img,
+		Src: image.Black,
+		Dot: initialPoint,
+	}
+
+	scriptList := t.fontLoader.GetScriptList()
+	textAnalyzer := NewTextAnalyzer(scriptList)
+	taOpts := AnalysisOpts{
+		UseMergeMap: true,
+		MergeMap:    DefaultMergeMap,
+	}
 
 	scanner := bufio.NewScanner(r)
+Scan: // Label for the scanner loop, so we can break it easily
 	for scanner.Scan() {
 		txt := scanner.Text()
-		cs := chunks(txt, 80)
-		for _, s := range cs {
-			_, err := c.DrawString(strings.TrimSpace(s), pt)
-			if err != nil {
-				return nil, err
+		height := fixed.I(fontSizeAsInt) // reset to default height
+
+		textResult := textAnalyzer.AnalyzeString(txt, taOpts)
+		textResult.MergeCommon(DefaultMergeMap)
+
+		for _, sRange := range textResult.ScriptRanges {
+			targetFontFace, _ := t.fontLoader.LoadFaceForScript(sRange.TargetScript)
+			// if the target script is "_unknown" it's expected that the loaded face
+			// uses the default font
+			faceHeight := targetFontFace.Face.Metrics().Height
+			if faceHeight > height {
+				height = faceHeight
+			}
+
+			canvas.Face = targetFontFace.Face
+			initialByte := sRange.Low
+			for _, sRangeSpace := range sRange.Spaces {
+				if canvas.Dot.Y > maxY {
+					break Scan
+				}
+				drawWord(canvas, textResult.Text[initialByte:sRangeSpace], minX, maxX, height, maxY, true)
+				initialByte = sRangeSpace
 			}
-			pt.Y += c.PointToFixed(fontSize * spacing)
-			if pt.Y.Round() >= img.Bounds().Dy() {
-				return img, scanner.Err()
+			if initialByte <= sRange.High {
+				// some bytes left to be written
+				if canvas.Dot.Y > maxY {
+					break Scan
+				}
+				drawWord(canvas, textResult.Text[initialByte:sRange.High+1], minX, maxX, height, maxY, len(sRange.Spaces) > 0)
 			}
 		}
 
+		canvas.Dot.X = minX
+		canvas.Dot.Y += height.Mul(fixed.Int26_6(1<<6 + 1<<5)) // height * 1.5
+
+		if canvas.Dot.Y > maxY {
+			break
+		}
 	}
 	return img, scanner.Err()
 }
 
-// Code from https://stackoverflow.com/a/61469854
-// Written By Igor Mikushkin
-func chunks(s string, chunkSize int) []string {
-	if chunkSize >= len(s) {
-		return []string{s}
-	}
-	var chunks []string
-	chunk := make([]rune, chunkSize)
-	length := 0
-	for _, r := range s {
-		chunk[length] = r
-		length++
-		if length == chunkSize {
-			chunks = append(chunks, string(chunk))
-			length = 0
+// Draw the word in the canvas. The mixX and maxX defines the drawable range
+// (X axis) where the word can be drawn (in case the word is too big and doesn't
+// fit in the canvas), and the incY defines the increment in the Y axis if we
+// need to draw the word in a new line
+//
+// Note that the word will likely start with a white space char
+func drawWord(canvas *font.Drawer, word string, minX, maxX, incY, maxY fixed.Int26_6, goToNewLine bool) {
+	bbox, _ := canvas.BoundString(word)
+	if bbox.Max.X <= maxX {
+		// word fits in the current line
+		canvas.DrawString(word)
+	} else {
+		// word doesn't fit -> retry in a new line
+		trimmedWord := strings.TrimSpace(word)
+		oldDot := canvas.Dot
+
+		canvas.Dot.X = minX
+		canvas.Dot.Y += incY
+		bbox2, _ := canvas.BoundString(trimmedWord)
+		if goToNewLine && bbox2.Max.X <= maxX {
+			if canvas.Dot.Y > maxY {
+				// Don't draw if we're over the Y limit
+				return
+			}
+			canvas.DrawString(trimmedWord)
+		} else {
+			// word doesn't fit in a new line -> draw as many chars as possible
+			canvas.Dot = oldDot
+			for _, char := range trimmedWord {
+				charBytes := []byte(string(char))
+				bbox3, _ := canvas.BoundBytes(charBytes)
+				if bbox3.Max.X > maxX {
+					canvas.Dot.X = minX
+					canvas.Dot.Y += incY
+					if canvas.Dot.Y > maxY {
+						// Don't draw if we're over the Y limit
+						return
+					}
+				}
+				canvas.DrawBytes(charBytes)
+			}
 		}
 	}
-	if length > 0 {
-		chunks = append(chunks, string(chunk[:length]))
-	}
-	return chunks
 }
 
-func ForType(mimeType string) FileConverter {
+func ForType(mimeType string, opts map[string]interface{}) FileConverter {
 	// We can ignore the error here because we parse it in IsMimeTypeSupported before and if it fails
 	// return the service call. So we should only get here when the mimeType parses fine.
 	mimeType, _, _ = mime.ParseMediaType(mimeType)
 	switch mimeType {
 	case "text/plain":
-		return TxtToImageConverter{}
+		fontFileMap := ""
+		fontFaceOpts := &opentype.FaceOptions{
+			Size:    12,
+			DPI:     72,
+			Hinting: font.HintingNone,
+		}
+
+		if optedFontFileMap, ok := opts["fontFileMap"]; ok {
+			if stringFontFileMap, ok := optedFontFileMap.(string); ok {
+				fontFileMap = stringFontFileMap
+			}
+		}
+
+		if optedFontFaceOpts, ok := opts["fontFaceOpts"]; ok {
+			if typedFontFaceOpts, ok := optedFontFaceOpts.(*opentype.FaceOptions); ok {
+				fontFaceOpts = typedFontFaceOpts
+			}
+		}
+
+		fontLoader, err := NewFontLoader(fontFileMap, fontFaceOpts)
+		if err != nil {
+			// if couldn't create the FontLoader with the specified fontFileMap,
+			// try to use the default font
+			fontLoader, _ = NewFontLoader("", fontFaceOpts)
+		}
+		return TxtToImageConverter{
+			fontLoader: fontLoader,
+		}
 	default:
 		return ImageDecoder{}
 	}
diff --git a/thumbnails/pkg/preprocessor/textanalyzer.go b/thumbnails/pkg/preprocessor/textanalyzer.go
new file mode 100644
index 00000000000..8dddfa642ab
--- /dev/null
+++ b/thumbnails/pkg/preprocessor/textanalyzer.go
@@ -0,0 +1,309 @@
+package preprocessor
+
+import (
+	"unicode"
+	"unicode/utf8"
+)
+
+// Default list of scripts to be analyzed within the string.
+//
+// Scripts that aren't present in the list will be considered as part
+// of the last "known" script. For example, if "Avestan" script (which isn't
+// present) is preceeded by "Arabic" script, then the "Avestan" script will
+// be considered as "Arabic"
+//
+// Punctuation symbols are usually considered part of the "Common" script
+var DefaultScripts = []string{
+	"Arabic",
+	"Common",
+	"Devanagari",
+	"Han",
+	"Hangul",
+	"Hiragana",
+	"Inherited",
+	"Katakana",
+	"Latin",
+}
+
+// Convenient map[string]map[string]string type used to merge multiple
+// scripts into one. This is mainly used for japanese language which uses
+// "Han", "Hiragana" and "Katakana" scripts.
+//
+// The map contains the expected previous script as first key, the expected
+// current script as second key, and the resulting script (if both keys
+// match) as value
+type MergeMap map[string]map[string]string
+
+// The default mergeMap containing info for the japanese scripts
+var DefaultMergeMap = MergeMap{
+	"Han": map[string]string{
+		"Hiragana": "Hiragana",
+		"Katakana": "Katakana",
+	},
+	"Hiragana": map[string]string{
+		"Han":      "Hiragana",
+		"Katakana": "Hiragana",
+	},
+	"Katakana": map[string]string{
+		"Han":      "Katakana",
+		"Hiragana": "Hiragana",
+	},
+}
+
+// Analysis options.
+type AnalysisOpts struct {
+	UseMergeMap bool
+	MergeMap    MergeMap
+}
+
+// A script range. The range should be attached to a string which could contain
+// multiple scripts. The "TargetScript" will go from bytes "Low" to "High"
+// (both inclusive), and contains a "RuneCount" number of runes or chars
+// (mostly for debugging purposes).
+// The Space contains the bytes (inside the range) that are considered as
+// white space.
+type ScriptRange struct {
+	Low, High    int
+	Spaces       []int
+	TargetScript string
+	RuneCount    int
+}
+
+// The result of a text analysis. It contains the analyzed text, a list of
+// script ranges (see the ScriptRange type) and a map containing how many
+// runes have been detected for a particular script.
+type TextAnalysis struct {
+	ScriptRanges []ScriptRange
+	RuneCount    map[string]int
+	Text         string
+}
+
+// The TextAnalyzer object contains private members. It should be created via
+// "NewTextAnalyzer" function.
+type TextAnalyzer struct {
+	scripts         map[string]*unicode.RangeTable
+	scriptListCache []string
+}
+
+// Create a new TextAnalyzer. A list of scripts must be provided.
+// You can use the "DefaultScripts" variable for a default list,
+// although it doesn't contain all the available scripts.
+// See the unicode.Scripts variable (in the unicode package) for a
+// full list. Note that using invalid scripts will cause an undefined
+// behavior
+func NewTextAnalyzer(scriptList []string) TextAnalyzer {
+	scriptRanges := make(map[string]*unicode.RangeTable, len(scriptList))
+	for _, script := range scriptList {
+		scriptRanges[script] = unicode.Scripts[script]
+	}
+	return TextAnalyzer{
+		scripts:         scriptRanges,
+		scriptListCache: scriptList,
+	}
+}
+
+// Analyze the target string using the specified options.
+// A TextAnalysis will be returned with the result of the analysis.
+func (ta *TextAnalyzer) AnalyzeString(word string, opts AnalysisOpts) TextAnalysis {
+	analysis := TextAnalysis{
+		ScriptRanges: []ScriptRange{},
+		RuneCount:    make(map[string]int),
+		Text:         word,
+	}
+
+	if len(word) < 1 {
+		return analysis
+	}
+
+	firstRune, runeLen := utf8.DecodeRuneInString(word)
+
+	lastRange := &ScriptRange{
+		Low:          0,
+		Spaces:       make([]int, 0),
+		TargetScript: ta.chooseScriptFor(firstRune),
+	}
+	firstRuneIsWhiteSpace := unicode.Is(unicode.White_Space, firstRune)
+	if firstRuneIsWhiteSpace {
+		lastRange.Spaces = append(lastRange.Spaces, 0)
+	}
+
+	runeCount := 1
+	for wordIndex, char := range word[runeLen:] {
+		wordIndex += runeLen // shifted from the original string
+		script := ta.chooseScriptFor(char)
+
+		isWhiteSpace := unicode.Is(unicode.White_Space, char)
+		if script != lastRange.TargetScript {
+			if mapScript, isOk := ta.getMergeMapValue(opts, lastRange.TargetScript, script); isOk {
+				lastRange.TargetScript = mapScript
+				if isWhiteSpace {
+					// TODO: Check if this is dead code.
+					// whitespace should be part of the "Common" script, and the Common
+					// script shouldn't be part of a mergeMap
+					lastRange.Spaces = append(lastRange.Spaces, wordIndex)
+				}
+				runeCount++
+				continue
+			}
+
+			lastRange.High = wordIndex - 1
+			lastRange.RuneCount = runeCount
+			analysis.ScriptRanges = append(analysis.ScriptRanges, *lastRange)
+			if _, exists := analysis.RuneCount[lastRange.TargetScript]; !exists {
+				analysis.RuneCount[lastRange.TargetScript] = 0
+			}
+			analysis.RuneCount[lastRange.TargetScript] += runeCount
+			lastRange = &ScriptRange{
+				Low:          wordIndex,
+				Spaces:       make([]int, 0),
+				TargetScript: script,
+			}
+			runeCount = 0
+		}
+		runeCount++
+		if isWhiteSpace {
+			lastRange.Spaces = append(lastRange.Spaces, wordIndex)
+		}
+	}
+
+	// close the last range
+	lastRange.High = len(word) - 1
+	lastRange.RuneCount = runeCount
+	analysis.RuneCount[lastRange.TargetScript] += runeCount
+	analysis.ScriptRanges = append(analysis.ScriptRanges, *lastRange)
+
+	return analysis
+}
+
+func (ta *TextAnalyzer) chooseScriptFor(char rune) string {
+	script := "_unknown"
+	for scriptIndex, scriptFound := range ta.scriptListCache {
+		// if we can't match with a known script, do nothing and jump to the next char
+		if unicode.Is(ta.scripts[scriptFound], char) {
+			if scriptIndex > 3 {
+				// we might expect more chars with the same script
+				// so move the script first to match it faster next time
+				ta.reorderScriptList(scriptFound)
+			}
+			return scriptFound
+		}
+	}
+	return script
+}
+
+// Reorder the scriptListCache in the TextAnalyzer in order to speed up
+// the next script searches. A "Latin" script is expected to be surrounded
+// by "Latin" chars, although "Common" script chars might be present too
+func (ta *TextAnalyzer) reorderScriptList(matchedScript string) {
+	for index, script := range ta.scriptListCache {
+		if script == matchedScript {
+			if index != 0 {
+				// move the script to the first position for a faster matching
+				newList := append([]string{script}, ta.scriptListCache[:index]...)
+				ta.scriptListCache = append(newList, ta.scriptListCache[index+1:]...)
+			}
+			// if index == 0 there is nothing to do: the element is already the first
+			break
+		}
+	}
+}
+
+// Get the value from the merge map based on the previous and current scripts.
+// The information about using the merge map and the actual merge map will be
+// gotten from the AnalysisOpts passed as parameter
+func (ta *TextAnalyzer) getMergeMapValue(opts AnalysisOpts, previous, current string) (string, bool) {
+	if opts.UseMergeMap {
+		// This option mainly target japanese chars; multiple scripts can be used
+		// in the same piece of text (Han, Hiragana and Katakana)
+		// Instead of starting a new range, adjust the target script of the last range
+		if expCurrent, currentOk := opts.MergeMap[previous]; currentOk {
+			if expFinal, finalOk := expCurrent[current]; finalOk {
+				return expFinal, finalOk
+			}
+		}
+	}
+	return "", false
+}
+
+// Change the "Common" script to the one used in the previous script range.
+// The ranges will be readjusted and merged if they're adjacent.
+// This naive approach should be good enough for normal use cases
+//
+// The MergeMap is needed in case of the japanese language: the ranges
+// "Han"-"Common"-"Katakana" might be replaced to "Han"-"Hiragana"-"Katakana"
+// However, the ranges should be merged together into a big "Hiragana" range.
+// If the MergeMap isn't needed, use an empty one
+func (tr *TextAnalysis) MergeCommon(mergeMap MergeMap) {
+	var finalRanges []ScriptRange
+
+	if len(tr.ScriptRanges) < 1 {
+		// no ranges -> nothing to do
+		return
+	}
+
+	previousRange := &ScriptRange{}
+	*previousRange = tr.ScriptRanges[0]
+	for _, sRange := range tr.ScriptRanges[1:] {
+		if previousRange.TargetScript == sRange.TargetScript {
+			previousRange.High = sRange.High
+			previousRange.Spaces = append(previousRange.Spaces, sRange.Spaces...)
+			previousRange.RuneCount += sRange.RuneCount
+		} else if sRange.TargetScript == "Common" || sRange.TargetScript == "Inherited" {
+			// new range will be absorbed into the previous one
+			previousRange.High = sRange.High
+			previousRange.Spaces = append(previousRange.Spaces, sRange.Spaces...)
+			previousRange.RuneCount += sRange.RuneCount
+			tr.RuneCount[previousRange.TargetScript] += sRange.RuneCount
+			tr.RuneCount[sRange.TargetScript] -= sRange.RuneCount
+		} else if previousRange.TargetScript == "Common" || previousRange.TargetScript == "Inherited" {
+			// might happen if the text starts with a Common script
+			previousRange.High = sRange.High
+			previousRange.Spaces = append(previousRange.Spaces, sRange.Spaces...)
+			tr.RuneCount[sRange.TargetScript] += previousRange.RuneCount
+			tr.RuneCount[previousRange.TargetScript] -= previousRange.RuneCount
+			previousRange.RuneCount += sRange.RuneCount
+			previousRange.TargetScript = sRange.TargetScript
+		} else {
+			if mapScript, isOk := tr.getMergeMapValue(mergeMap, previousRange.TargetScript, sRange.TargetScript); isOk {
+				if sRange.TargetScript == mapScript {
+					// the previous range has changed the target script
+					tr.RuneCount[previousRange.TargetScript] -= previousRange.RuneCount
+					tr.RuneCount[sRange.TargetScript] += previousRange.RuneCount
+				} else {
+					// new range has been absorbed
+					tr.RuneCount[sRange.TargetScript] -= sRange.RuneCount
+					tr.RuneCount[previousRange.TargetScript] += sRange.RuneCount
+				}
+				previousRange.TargetScript = mapScript
+				previousRange.High = sRange.High
+				previousRange.Spaces = append(previousRange.Spaces, sRange.Spaces...)
+				previousRange.RuneCount += sRange.RuneCount
+				continue
+			}
+			finalRanges = append(finalRanges, *previousRange)
+			*previousRange = sRange
+		}
+	}
+
+	finalRanges = append(finalRanges, *previousRange)
+	tr.ScriptRanges = finalRanges
+	delete(tr.RuneCount, "Common")
+	delete(tr.RuneCount, "Inherited")
+	for index, rCount := range tr.RuneCount {
+		if rCount == 0 {
+			delete(tr.RuneCount, index)
+		}
+	}
+}
+
+func (tr *TextAnalysis) getMergeMapValue(mMap MergeMap, previous, current string) (string, bool) {
+	// This option mainly target japanese chars; multiple scripts can be used
+	// in the same piece of text (Han, Hiragana and Katakana)
+	// Instead of starting a new range, adjust the target script of the last range
+	if expCurrent, currentOk := mMap[previous]; currentOk {
+		if expFinal, finalOk := expCurrent[current]; finalOk {
+			return expFinal, finalOk
+		}
+	}
+	return "", false
+}
diff --git a/thumbnails/pkg/service/v0/service.go b/thumbnails/pkg/service/v0/service.go
index b5388f84461..8914aebbceb 100644
--- a/thumbnails/pkg/service/v0/service.go
+++ b/thumbnails/pkg/service/v0/service.go
@@ -41,6 +41,9 @@ func NewService(opts ...Option) v0proto.ThumbnailServiceHandler {
 		cs3Source:    options.CS3Source,
 		logger:       logger,
 		cs3Client:    options.CS3Client,
+		preprocessorOpts: PreprocessorOpts{
+			TxtFontFileMap: options.Config.Thumbnail.FontMapFile,
+		},
 	}
 
 	return svc
@@ -48,13 +51,18 @@ func NewService(opts ...Option) v0proto.ThumbnailServiceHandler {
 
 // Thumbnail implements the GRPC handler.
 type Thumbnail struct {
-	serviceID       string
-	webdavNamespace string
-	manager         thumbnail.Manager
-	webdavSource    imgsource.Source
-	cs3Source       imgsource.Source
-	logger          log.Logger
-	cs3Client       gateway.GatewayAPIClient
+	serviceID        string
+	webdavNamespace  string
+	manager          thumbnail.Manager
+	webdavSource     imgsource.Source
+	cs3Source        imgsource.Source
+	logger           log.Logger
+	cs3Client        gateway.GatewayAPIClient
+	preprocessorOpts PreprocessorOpts
+}
+
+type PreprocessorOpts struct {
+	TxtFontFileMap string
 }
 
 // GetThumbnail retrieves a thumbnail for an image
@@ -114,7 +122,10 @@ func (g Thumbnail) handleCS3Source(ctx context.Context, req *v0proto.GetThumbnai
 		return nil, merrors.InternalServerError(g.serviceID, "could not get image from source: %s", err.Error())
 	}
 	defer r.Close() // nolint:errcheck
-	pp := preprocessor.ForType(sRes.GetInfo().GetMimeType())
+	ppOpts := map[string]interface{}{
+		"fontFileMap": g.preprocessorOpts.TxtFontFileMap,
+	}
+	pp := preprocessor.ForType(sRes.GetInfo().GetMimeType(), ppOpts)
 	img, err := pp.Convert(r)
 	if img == nil || err != nil {
 		return nil, merrors.InternalServerError(g.serviceID, "could not get image")
@@ -188,7 +199,10 @@ func (g Thumbnail) handleWebdavSource(ctx context.Context, req *v0proto.GetThumb
 		return nil, merrors.InternalServerError(g.serviceID, "could not get image from source: %s", err.Error())
 	}
 	defer r.Close() // nolint:errcheck
-	pp := preprocessor.ForType(sRes.GetInfo().GetMimeType())
+	ppOpts := map[string]interface{}{
+		"fontFileMap": g.preprocessorOpts.TxtFontFileMap,
+	}
+	pp := preprocessor.ForType(sRes.GetInfo().GetMimeType(), ppOpts)
 	img, err := pp.Convert(r)
 	if img == nil || err != nil {
 		return nil, merrors.InternalServerError(g.serviceID, "could not get image")


Test patch:
diff --git a/thumbnails/pkg/preprocessor/textanalyzer_test.go b/thumbnails/pkg/preprocessor/textanalyzer_test.go
new file mode 100644
index 00000000000..91f98024779
--- /dev/null
+++ b/thumbnails/pkg/preprocessor/textanalyzer_test.go
@@ -0,0 +1,614 @@
+package preprocessor
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+var (
+	inputs = [18]string{
+		"basic latin",
+		"trailing tab	",
+		"Small text. \"$\", \"£\" and \"¥\" are currencies.",
+		"latin with 🖖",
+		"기본 한국어",
+		"基本的な日本語",
+		"ウーロン茶",
+		"私はエンジニアです",
+		"ティー私はエンジニアです",
+		"私はエンジニアです ティー",
+		"आधारभूत देवनागरी",
+		"mixed 언어 传入 🚀!",
+		"/k͜p/",
+		// ä and a + ¨
+		"ä ä",
+		"базовый русский", // cyrillic script isn't part of our default
+		"latin русский",   // latin + cyrillic (cyrillic not supported)
+		" space justified ",
+		"",
+	}
+)
+
+func TestAnalyzeString(t *testing.T) {
+	defaultOpts := AnalysisOpts{
+		UseMergeMap: true,
+		MergeMap:    DefaultMergeMap,
+	}
+
+	tables := []struct {
+		input string
+		opts  AnalysisOpts
+		eOut  TextAnalysis
+	}{
+		{
+			input: inputs[0],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 10, Spaces: []int{5}, TargetScript: "Latin", RuneCount: 11},
+				},
+				RuneCount: map[string]int{
+					"Latin": 11,
+				},
+				Text: inputs[0],
+			},
+		},
+		{
+			input: inputs[1],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 12, Spaces: []int{8, 12}, TargetScript: "Latin", RuneCount: 13},
+				},
+				RuneCount: map[string]int{
+					"Latin": 13,
+				},
+				Text: inputs[1],
+			},
+		},
+		{
+			input: inputs[2],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 45, Spaces: []int{5, 11, 16, 21, 25, 30, 34}, TargetScript: "Latin", RuneCount: 44},
+				},
+				RuneCount: map[string]int{
+					"Latin": 44,
+				},
+				Text: inputs[2],
+			},
+		},
+		{
+			input: inputs[3],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 14, Spaces: []int{5, 10}, TargetScript: "Latin", RuneCount: 12},
+				},
+				RuneCount: map[string]int{
+					"Latin": 12,
+				},
+				Text: inputs[3],
+			},
+		},
+		{
+			input: inputs[4],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 15, Spaces: []int{6}, TargetScript: "Hangul", RuneCount: 6},
+				},
+				RuneCount: map[string]int{
+					"Hangul": 6,
+				},
+				Text: inputs[4],
+			},
+		},
+		{
+			input: inputs[5],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 20, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 7},
+				},
+				RuneCount: map[string]int{
+					"Hiragana": 7,
+				},
+				Text: inputs[5],
+			},
+		},
+		{
+			input: inputs[6],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 14, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 5},
+				},
+				RuneCount: map[string]int{
+					"Katakana": 5,
+				},
+				Text: inputs[6],
+			},
+		},
+		{
+			input: inputs[7],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 26, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 9},
+				},
+				RuneCount: map[string]int{
+					"Hiragana": 9,
+				},
+				Text: inputs[7],
+			},
+		},
+		{
+			input: inputs[8],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 35, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 12},
+				},
+				RuneCount: map[string]int{
+					"Hiragana": 12,
+				},
+				Text: inputs[8],
+			},
+		},
+		{
+			input: inputs[9],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 36, Spaces: []int{27}, TargetScript: "Hiragana", RuneCount: 13},
+				},
+				RuneCount: map[string]int{
+					"Hiragana": 13,
+				},
+				Text: inputs[9],
+			},
+		},
+		{
+			input: inputs[10],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 45, Spaces: []int{21}, TargetScript: "Devanagari", RuneCount: 16},
+				},
+				RuneCount: map[string]int{
+					"Devanagari": 16,
+				},
+				Text: inputs[10],
+			},
+		},
+		{
+			input: inputs[11],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 5, Spaces: []int{5}, TargetScript: "Latin", RuneCount: 6},
+					ScriptRange{Low: 6, High: 12, Spaces: []int{12}, TargetScript: "Hangul", RuneCount: 3},
+					ScriptRange{Low: 13, High: 24, Spaces: []int{19}, TargetScript: "Han", RuneCount: 5}, // 🚀 and ! are "Common" script and will be merged with "Han"
+				},
+				RuneCount: map[string]int{
+					"Latin":  6,
+					"Hangul": 3,
+					"Han":    5,
+				},
+				Text: inputs[11],
+			},
+		},
+		{
+			input: inputs[12],
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 5, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+				},
+				RuneCount: map[string]int{
+					"Latin": 5,
+				},
+				Text: inputs[12],
+			},
+		},
+		{
+			input: inputs[13], // ä and a + ¨
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 5, Spaces: []int{2}, TargetScript: "Latin", RuneCount: 4},
+				},
+				RuneCount: map[string]int{
+					"Latin": 4,
+				},
+				Text: inputs[13],
+			},
+		},
+		{
+			input: inputs[14], // cyrillic script isn't part of our default
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 28, Spaces: []int{14}, TargetScript: "_unknown", RuneCount: 15},
+				},
+				RuneCount: map[string]int{
+					"_unknown": 15,
+				},
+				Text: inputs[14],
+			},
+		},
+		{
+			input: inputs[15], // latin + cyrillic (cyrillic script isn't part of our default)
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 5, Spaces: []int{5}, TargetScript: "Latin", RuneCount: 6},
+					ScriptRange{Low: 6, High: 19, Spaces: []int{}, TargetScript: "_unknown", RuneCount: 7},
+				},
+				RuneCount: map[string]int{
+					"Latin":    6,
+					"_unknown": 7,
+				},
+				Text: inputs[15],
+			},
+		},
+		{
+			input: inputs[16], // latin + cyrillic (cyrillic script isn't part of our default)
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 16, Spaces: []int{0, 6, 16}, TargetScript: "Latin", RuneCount: 17},
+				},
+				RuneCount: map[string]int{
+					"Latin": 17,
+				},
+				Text: inputs[16],
+			},
+		},
+		{
+			input: inputs[17], // latin + cyrillic (cyrillic script isn't part of our default)
+			opts:  defaultOpts,
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{},
+				RuneCount:    map[string]int{},
+				Text:         inputs[17],
+			},
+		},
+	}
+
+	for _, table := range tables {
+		testname := fmt.Sprintf("Analyzing \"%s\" string", table.input)
+		t.Run(testname, func(t *testing.T) {
+			ta := NewTextAnalyzer(DefaultScripts)
+			result := ta.AnalyzeString(table.input, table.opts)
+			if table.opts.UseMergeMap {
+				result.MergeCommon(table.opts.MergeMap)
+			} else {
+				result.MergeCommon(MergeMap{})
+			}
+			assert.Equal(t, table.eOut, result)
+		})
+	}
+}
+
+func TestAnalyzeStringRaw(t *testing.T) {
+	tables := []struct {
+		input string
+		eOut  TextAnalysis
+	}{
+		{
+			input: inputs[0],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 4, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+					ScriptRange{Low: 5, High: 5, Spaces: []int{5}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 6, High: 10, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+				},
+				RuneCount: map[string]int{
+					"Latin":  10,
+					"Common": 1,
+				},
+				Text: inputs[0],
+			},
+		},
+		{
+			input: inputs[1],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 7, Spaces: []int{}, TargetScript: "Latin", RuneCount: 8},
+					ScriptRange{Low: 8, High: 8, Spaces: []int{8}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 9, High: 11, Spaces: []int{}, TargetScript: "Latin", RuneCount: 3},
+					ScriptRange{Low: 12, High: 12, Spaces: []int{12}, TargetScript: "Common", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Latin":  11,
+					"Common": 2,
+				},
+				Text: inputs[1],
+			},
+		},
+		{
+			input: inputs[2],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 4, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+					ScriptRange{Low: 5, High: 5, Spaces: []int{5}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 6, High: 9, Spaces: []int{}, TargetScript: "Latin", RuneCount: 4},
+					ScriptRange{Low: 10, High: 21, Spaces: []int{11, 16, 21}, TargetScript: "Common", RuneCount: 11}, // £ takes 2 bytes
+					ScriptRange{Low: 22, High: 24, Spaces: []int{}, TargetScript: "Latin", RuneCount: 3},
+					ScriptRange{Low: 25, High: 30, Spaces: []int{25, 30}, TargetScript: "Common", RuneCount: 5}, // ¥ takes 2 bytes
+					ScriptRange{Low: 31, High: 33, Spaces: []int{}, TargetScript: "Latin", RuneCount: 3},
+					ScriptRange{Low: 34, High: 34, Spaces: []int{34}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 35, High: 44, Spaces: []int{}, TargetScript: "Latin", RuneCount: 10},
+					ScriptRange{Low: 45, High: 45, Spaces: []int{}, TargetScript: "Common", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Latin":  25,
+					"Common": 19,
+				},
+				Text: inputs[2],
+			},
+		},
+		{
+			input: inputs[3],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 4, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+					ScriptRange{Low: 5, High: 5, Spaces: []int{5}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 6, High: 9, Spaces: []int{}, TargetScript: "Latin", RuneCount: 4},
+					ScriptRange{Low: 10, High: 14, Spaces: []int{10}, TargetScript: "Common", RuneCount: 2},
+				},
+				RuneCount: map[string]int{
+					"Latin":  9,
+					"Common": 3,
+				},
+				Text: inputs[3],
+			},
+		},
+		{
+			input: inputs[4],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 5, Spaces: []int{}, TargetScript: "Hangul", RuneCount: 2},
+					ScriptRange{Low: 6, High: 6, Spaces: []int{6}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 7, High: 15, Spaces: []int{}, TargetScript: "Hangul", RuneCount: 3},
+				},
+				RuneCount: map[string]int{
+					"Hangul": 5,
+					"Common": 1,
+				},
+				Text: inputs[4],
+			},
+		},
+		{
+			input: inputs[5],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 8, Spaces: []int{}, TargetScript: "Han", RuneCount: 3},
+					ScriptRange{Low: 9, High: 11, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 1},
+					ScriptRange{Low: 12, High: 20, Spaces: []int{}, TargetScript: "Han", RuneCount: 3},
+				},
+				RuneCount: map[string]int{
+					"Hiragana": 1,
+					"Han":      6,
+				},
+				Text: inputs[5],
+			},
+		},
+		{
+			input: inputs[6],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 2, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 1},
+					ScriptRange{Low: 3, High: 5, Spaces: []int{}, TargetScript: "Common", RuneCount: 1}, // ー U+30FC (KATAKANA-HIRAGANA PROLONGED SOUND MARK) seems to be counted as Common
+					ScriptRange{Low: 6, High: 11, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 2},
+					ScriptRange{Low: 12, High: 14, Spaces: []int{}, TargetScript: "Han", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Katakana": 3,
+					"Common":   1,
+					"Han":      1,
+				},
+				Text: inputs[6],
+			},
+		},
+		{
+			input: inputs[7],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 2, Spaces: []int{}, TargetScript: "Han", RuneCount: 1},
+					ScriptRange{Low: 3, High: 5, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 1},
+					ScriptRange{Low: 6, High: 20, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 5},
+					ScriptRange{Low: 21, High: 26, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 2},
+				},
+				RuneCount: map[string]int{
+					"Han":      1,
+					"Hiragana": 3,
+					"Katakana": 5,
+				},
+				Text: inputs[7],
+			},
+		},
+		{
+			input: inputs[8],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 5, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 2},
+					ScriptRange{Low: 6, High: 8, Spaces: []int{}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 9, High: 11, Spaces: []int{}, TargetScript: "Han", RuneCount: 1},
+					ScriptRange{Low: 12, High: 14, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 1},
+					ScriptRange{Low: 15, High: 29, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 5},
+					ScriptRange{Low: 30, High: 35, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 2},
+				},
+				RuneCount: map[string]int{
+					"Han":      1,
+					"Hiragana": 3,
+					"Katakana": 7,
+					"Common":   1,
+				},
+				Text: inputs[8],
+			},
+		},
+		{
+			input: inputs[9],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 2, Spaces: []int{}, TargetScript: "Han", RuneCount: 1},
+					ScriptRange{Low: 3, High: 5, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 1},
+					ScriptRange{Low: 6, High: 20, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 5},
+					ScriptRange{Low: 21, High: 26, Spaces: []int{}, TargetScript: "Hiragana", RuneCount: 2},
+					ScriptRange{Low: 27, High: 27, Spaces: []int{27}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 28, High: 33, Spaces: []int{}, TargetScript: "Katakana", RuneCount: 2},
+					ScriptRange{Low: 34, High: 36, Spaces: []int{}, TargetScript: "Common", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Han":      1,
+					"Hiragana": 3,
+					"Katakana": 7,
+					"Common":   2,
+				},
+				Text: inputs[9],
+			},
+		},
+		{
+			input: inputs[10],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 20, Spaces: []int{}, TargetScript: "Devanagari", RuneCount: 7},
+					ScriptRange{Low: 21, High: 21, Spaces: []int{21}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 22, High: 45, Spaces: []int{}, TargetScript: "Devanagari", RuneCount: 8},
+				},
+				RuneCount: map[string]int{
+					"Devanagari": 15,
+					"Common":     1,
+				},
+				Text: inputs[10],
+			},
+		},
+		{
+			input: inputs[11],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 4, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+					ScriptRange{Low: 5, High: 5, Spaces: []int{5}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 6, High: 11, Spaces: []int{}, TargetScript: "Hangul", RuneCount: 2},
+					ScriptRange{Low: 12, High: 12, Spaces: []int{12}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 13, High: 18, Spaces: []int{}, TargetScript: "Han", RuneCount: 2},
+					ScriptRange{Low: 19, High: 24, Spaces: []int{19}, TargetScript: "Common", RuneCount: 3},
+				},
+				RuneCount: map[string]int{
+					"Latin":  5,
+					"Hangul": 2,
+					"Han":    2,
+					"Common": 5,
+				},
+				Text: inputs[11],
+			},
+		},
+		{
+			input: inputs[12],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 0, Spaces: []int{}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 1, High: 1, Spaces: []int{}, TargetScript: "Latin", RuneCount: 1},
+					ScriptRange{Low: 2, High: 3, Spaces: []int{}, TargetScript: "Inherited", RuneCount: 1},
+					ScriptRange{Low: 4, High: 4, Spaces: []int{}, TargetScript: "Latin", RuneCount: 1},
+					ScriptRange{Low: 5, High: 5, Spaces: []int{}, TargetScript: "Common", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Latin":     2,
+					"Common":    2,
+					"Inherited": 1,
+				},
+				Text: inputs[12],
+			},
+		},
+		{
+			input: inputs[13], // ä and a + ¨
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 1, Spaces: []int{}, TargetScript: "Latin", RuneCount: 1},
+					ScriptRange{Low: 2, High: 2, Spaces: []int{2}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 3, High: 3, Spaces: []int{}, TargetScript: "Latin", RuneCount: 1},
+					ScriptRange{Low: 4, High: 5, Spaces: []int{}, TargetScript: "Inherited", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Latin":     2,
+					"Common":    1,
+					"Inherited": 1,
+				},
+				Text: inputs[13],
+			},
+		},
+		{
+			input: inputs[14], // cyrillic script isn't part of our default
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 13, Spaces: []int{}, TargetScript: "_unknown", RuneCount: 7},
+					ScriptRange{Low: 14, High: 14, Spaces: []int{14}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 15, High: 28, Spaces: []int{}, TargetScript: "_unknown", RuneCount: 7},
+				},
+				RuneCount: map[string]int{
+					"_unknown": 14,
+					"Common":   1,
+				},
+				Text: inputs[14],
+			},
+		},
+		{
+			input: inputs[15], // latin + cyrillic (cyrillic script isn't part of our default)
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 4, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+					ScriptRange{Low: 5, High: 5, Spaces: []int{5}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 6, High: 19, Spaces: []int{}, TargetScript: "_unknown", RuneCount: 7},
+				},
+				RuneCount: map[string]int{
+					"Latin":    5,
+					"Common":   1,
+					"_unknown": 7,
+				},
+				Text: inputs[15],
+			},
+		},
+		{
+			input: inputs[16],
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{
+					ScriptRange{Low: 0, High: 0, Spaces: []int{0}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 1, High: 5, Spaces: []int{}, TargetScript: "Latin", RuneCount: 5},
+					ScriptRange{Low: 6, High: 6, Spaces: []int{6}, TargetScript: "Common", RuneCount: 1},
+					ScriptRange{Low: 7, High: 15, Spaces: []int{}, TargetScript: "Latin", RuneCount: 9},
+					ScriptRange{Low: 16, High: 16, Spaces: []int{16}, TargetScript: "Common", RuneCount: 1},
+				},
+				RuneCount: map[string]int{
+					"Latin":  14,
+					"Common": 3,
+				},
+				Text: inputs[16],
+			},
+		},
+		{
+			input: inputs[17], // empty string
+			eOut: TextAnalysis{
+				ScriptRanges: []ScriptRange{},
+				RuneCount:    map[string]int{},
+				Text:         inputs[17],
+			},
+		},
+	}
+
+	for _, table := range tables {
+		testname := fmt.Sprintf("Raw-Analyzing \"%s\" string", table.input)
+		t.Run(testname, func(t *testing.T) {
+			ta := NewTextAnalyzer(DefaultScripts)
+			result := ta.AnalyzeString(table.input, AnalysisOpts{})
+
+			assert.Equal(t, table.eOut, result)
+		})
+	}
+}