Presented at IMUG SF 2019-08-07
Abstract

Critical Values for I18n
Testing
Tex Texin
Chief Globalization Architect
XenCraft

In this session, we recommend specific data values thatare likely to identify internationalization problems insoftware intended for global markets.
Based on years of global software experience, these data
values are useful for functional or linguistic QA tests ofinternationalized software. In the last session, data value
recommendations included character encoding, postal
address, locale and other data types typically used insoftware and will trigger common internationalization
problems. This presentation will offer specific test
suggestions.

3

Copyright © 2019 Tex Texin. All rights reserved.

Assumptions: QA you already know

Test Results Possibilities

Knowledge and practice of traditional i18n functional QA
techniques are assumed- Functional vs. Linguistic QA- Pseudolocalization (Psèüdõlõcâlîzâtîõn)- Testing with different locale settings- Testing with native software (Operating System,
Browsers, 3rd party software, etc.) and devices

Possibilities

True

False

Positive

True Positive

False Positive

Negative

True Negative

False Negative

Results

True

False

+ (Tests Fail)

Actual Bug(s)Imagined Bug(s)- (Tests Pass)

Quality Assured

Hidden Bug(s)True +
You earned
your money- Security and other aspects are not addressed$
Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

5True -

False +The team did Investigation
well!
time down thedrain

Copyright © 2019 Tex Texin. All rights reserved.

False Customer dissatisfaction,
corporate embarassment,
heads may roll

7

1

Presented at IMUG SF 2019-08-07
Test Results Possibilities

Volume testing

Possibilities

True

False

Positive

True Positive

False Positive

Negative

True Negative

False Negative

Results

True

False

+ (Tests Fail)

Actual Bug(s)Imagined Bug(s)- (Tests Pass)

Quality Assured

Hidden Bug(s)It is a mistaken statistical premise that testing many
random values will uncover problems.
- Automated testing can exercise many values, but repeats
them each test session.
- Problem values are not randomly distributed.
- Testing without knowledge of the underlying architectureoften misses problem areas.
- Testing hits only a small % of all possible values

Of course, assuring quality assumes adequate test coverage.

-Unicode

Often, bugs can remain hidden despite voluminous testing.

- 150 Scripts- 87,887 Ideographs- 137,929 Characters- 19,024,409,041 Combinations of 2 characters- Bazillions of Character Combinations for a string of N characters.

Voluminous testing gives false confidence that coverage isadequate and quality is assured.

Copyright © 2019 Tex Texin. All rights reserved.

8

Copyright © 2019 Tex Texin. All rights reserved.

Large test data sets hide problems

How to adequately test large data sets?

Big (random) data sets miss key problems

E.g. Text, Date-time, Arithmetic, Identifiers- Volume testing creates the incorrect impressionthat the software is robust

Solutions?
- Test every possible value and permutation- At least once, possibly over a long period- Don’t test same values repeatedly

Numerous examples of catastrophic errors- User, random and other testing- Use architecture knowledge to test risky areas- Intel FPU arithmetic error- DG systems won’t boot on a certain date
- Text corruption/Mojibake- Limits, Boundaries, Syntax, Error cases- Test with values that have found problems before- There isn’t one answer short of testing all cases,
and repeating in different sequences

False Negatives can be consequential- To the business- And often easily experienced by users
Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

10- Testing with problem-finding values is a minimum
11

Copyright © 2019 Tex Texin. All rights reserved.

122

Presented at IMUG SF 2019-08-07
Goal: Critical Values For I18n Testing

Text and Unicode

Goal: Test values that provoke problems orraise issues for global software-Unicode is a large character set-137,000+ characters & millions of variants- Use real-world business values where possible- Including combining characters- Varying length characters and strings- Domains: Text, Date-Time, Identifiers (email, URL)

- Avoid exotic or contrived examples

Copyright © 2019 Tex Texin. All rights reserved.

-Many characters are similar visually orfunctionally, and distinct in unexpected ways
-Explicit and implicit rules-Locale-based behaviors-Continually evolving

13

Copyright © 2019 Tex Texin. All rights reserved.

Wide Loads: Full Width Characters

The Invisible Man and Invisible GirlASCII (et al) characters are duplicatedWhite space (space, tab, return, et al) often
needs trimming or other special handling

Full Width:！＂＃＄％＆＇（）＊＋，－．／０１２３４５６７８９：；＜＝＞？
＠ＡＢＣＤＥＦＧＨＩＪＫＬＭＮＯＰＱＲＳＴＵＶＷＸＹＺ［＼］＾＿｀ａｂ
ｃｄｅｆｇｈｉｊｋｌｍｎｏｐｑｒｓｔｕｖｗｘｙｚ｛｜｝～ASCII: !"#$%&'()*+,-./0123456789:;< = >?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijkl
mnopqrstuvwxyz{|}~- Full Width characters may be syntax equivalents.
- Inconsistent across products, even where standards exist
SELECT * FROM Employees WHERE LastName='龍' AND(FirstName='陳元' OR FirstName='Jackie')

Tex Texin, XenCraft, I18nGuy

1. No-Break Space (U+00A0, &nbsp;) often missed- Is Group separator in some locales (e.g. 1 234 567,89)

2. Full width (ideographic) space (U+3000)
3. Other spaces En Space U+2002
Em Space U+2003
Three-Per-Em Space U+2004
Four-Per-Em Space U+2005
Six-Per-Em Space U+2006
Figure Space U+2007
Punctuation Space U+2008
Thin Space U+2009
Hair Space U+200A
Zero Width Space U+200B- Test Case: Use full width (Ideographic) "*" andother characters in keywords and operators

Copyright © 2019 Tex Texin. All rights reserved.

14

15

Copyright © 2019 Tex Texin. All rights reserved.

16

3

Presented at IMUG SF 2019-08-07
The Invisible Man and Invisible GirlThe usual suspects- Test Data NBSP, Full Width Space

Unicode Storage Test Values

Although No-Break Space is not usually typed in by users, it isoften copy/paste into fields.

Background- UTF-8 characters can be 1-4 bytes (1-4 octets)- UTF-16 characters can be 1-2 words (1-2 16-bit units)- U+10000-U+1FFFFF are 4 bytes or 2 words

If NBSP or Full Width Space are not trimmed, searches can fail,or duplicate entries can occur
Duplicate Entries

StringTypical software text storage errors

String+NBSP String+FWSP NBSP+String- Assume 1 character =1 byte or 1 word
- Assume 1 UTF-8 character <= 3 bytes- Assume maximum length string will not occur and less is adequate- Assume text operations do not change length (e.g. case)- Memory allocation or string indexing “off by one” errors- Inconsistent definitions of maximum length (e.g. UI <> DB)- Performance degrades or severe memory leaks with long strings

Search for "String", won't match"String+NBSP".
Create User "String+NBSP" won't fail (asexpected) if "String" already exists.

Copyright © 2019 Tex Texin. All rights reserved.

17

Copyright © 2019 Tex Texin. All rights reserved.

18

Unicode Text Examples

Unicode Storage Test Values- One byte
“A”
- Two Bytes“É” or Three bytes “E” + “ ́”- Three Bytes“€”- Four Bytes“𠜎”
Test various length characters and strings to findmistakes in string indexing, line wrapping, etc.

Maximum length strings using maximum lengthcharacters (Supplementary characters > U+FFFF)
Frequently used 4-byte Supplementary characters- U+2070E, U+20731, U+20779, U+20C53, U+20C78, U+20C96,
U+20CCF, U+20CD5, U+20D15 and more

- See i18nguy.com/unicode/supplementary-test.html- Provokes memory overruns, off by one errors,
errors where the validation rules are incorrect
Test1: create maximum length strings with them

- E.g. wrapping a line in the middle of a character.
- E.g. Family Emoji: 15 Characters for a single emojiMan, Dark Skin Tone, Joiner, Bald, Joiner,
Woman, Med. Skin Tone, Joiner, Red Hair, Joiner,
Girl, Light Skin Tone, Joiner,
Boy, Medium Light Skin Tone

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

19

Copyright © 2019 Tex Texin. All rights reserved.

20

4

Presented at IMUG SF 2019-08-07
Unicode Storage Test Values

Case rules

Test character substitution near boundaries

This building is for eels congress. (congrès vs congres)

4 Characters
4 Bytes

A

A

A

A

4 Characters
4 BytesA

A

4 Characters
5 Bytes

A

A

A

É

4 Characters
5 BytesA

É

A

Palais des congrès

AA

BackgroundA- Case character maps can be more complex than English
Test Case

Rationale

1. Fill field to maximum length withASCII characters
2. Replace one character with a
Supplementary character

Test that field supports full count ofcharacters not bytes
OR
If counting bytes, that validation
prevents overruns

Typical software errors- Assuming that string length doesn’t change- Assuming that case rules are reversible

Failure cases

1. Fill field to maximum length-1 (or -2, Verify that validation prevents overruns-3) with ASCII characters
2. Replace one character with a
Supplementary character- Upper(ß) => “SS”
- Length(eßen) = 4
21

Copyright © 2019 Tex Texin. All rights reserved.

Lower(SS) => “ss”
Length(Upper(eßen)) = 5
22

Copyright © 2019 Tex Texin. All rights reserved.

Case rules

Case rules

Example real world failure"ß" U+00DF is a useful character for i18n testing- Case-insensitive compare used to determine if
the record changed and should be saved

Tests for poor case mappings assumptions- If (Upper(CurrentVal) <> Upper(InputVal)) save-changes;
Test Data- Fails if the User edits the field and the change is not saved
Description

CurrentVal

InputVal

ResultLocale = fr_CA"résumé""resume"

SUCCESS!
"RÉSUMÉ"<>"RESUME"

Locale = fr_FR"résumé""resume"FAIL!
"RESUME"="RESUME"

Different lowercase
characters with the same
uppercase"essen""eßen"FAIL!
"ESSEN"="ESSEN"

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

Assumption

Failure Case

case rules arereversible

Upper(ß) => “SS”
Lower(SS) => “ss”

ß (U+00DF)

string length doesn’tchange with case

Length(eßen) = 4
X = mem(4)Length(Upper(eßen)) = 5 X = Upper(“eßen”)

Unique strings remainß (U+00DF) unique with casechanges

23

Realityß (U+00DF)

Copyright © 2019 Tex Texin. All rights reserved.

Upper(eßen)=ESSEN
Upper(essen)=ESSEN

X=Upper(Lower(X))

X=“eßen”
Y=“essen”
X<>Y
Upper(X)=Upper(Y)245Presented at IMUG SF 2019-08-07
Example Tests For Case Rules

Locale-dependent case rules

Test for uppercasing overrunning memory

Case mappings are locale-dependent- Turkish is special as it changes case of ASCII characters

Data Case 1: Use maximum length run of "ß"Language

Case Rules

Notes- "ßßßßß" 5 chars become 10: "SSSSSSSSSS"

EnglishUpper(i) = I

Dotted i uppercases to Dotless I

Data Case 2: Edit a maximum length string,
replace one character with "ß"TurkishUpper(i) = İ
Upper(ı) = I

Dotted i uppercases to Dotted İ
Dotless ı uppercases to Dotless I

Recommended: Test with Turkish locale- Even if you don't localize for or plan to support Turkey

Copyright © 2019 Tex Texin. All rights reserved.

25

Test Data

Assumption

Failure Casei (U+0069)
I (U+0049)
İ (U+0130)
ı (U+0131)

Case rules for ASCII characters
do not change

Upper(exit)<>EXIT

26

Copyright © 2019 Tex Texin. All rights reserved.

Syntax Character Tests
Software uses many programming languages andprotocols. Each has its own syntax with "special" charactersand escapes.
Note: A syntax can rely on a sequence of characters.
Note: A syntax can rely on pairings of characters.
e.g. " " or /* */Text me!
Character Tests

Language /ProtocolExample Syntax
Characters

escapeHTML, XML

&<>'"

&

URL

?&#+/.:=%%

Javascript' " \ // /* */\\uCopyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

Examples&amp;&#x20AC;

tex.com?a=san+jos%C3%A8+%26+CA\n \\\u20AC

28

6

Presented at IMUG SF 2019-08-07
Syntax Character Tests

Syntax Character Tests

Potential Failure cases

Storage Formats- Syntax characters are not escaped- Syntax characters are escaped twice- Syntax characters used in 2 or more languages,
are escaped and unescaped in different orders

Logic Stack

Output Formats
URL

Java

Text

SQL Database
PHP

Problems can occur because data comes fromdifferent sources, goes thru different code paths,
and is intended for different uses.

Properties
Files- E.g. Text in properties files destined for HTML vs e-mailvs javascript alert vs URL, et al

HTML +
Javascript

Regex

E-mail- Each has different escape requirements

Libraries
29

Copyright © 2019 Tex Texin. All rights reserved.

30

Copyright © 2019 Tex Texin. All rights reserved.

Syntax Character Tests

ISO-8859-1 vs Windows-1252

Recommendation: Test with syntaxsignificant characters everywhere&+/\"'()?.#@_-~
Escape/Unescape example Ampersand "&"Very common error: incorrect charset nameCharacterHTML Escape

URI Escape

&

&amp;

%26

- Can result in white box or ?

Step 1

Step 2

Encode M&M HTML followed by URI

M&amp;M

M%26amp;M

URI first

M&amp;M

M&M

Decode M%26amp;M HTML first

M%26amp;MM&amp;M

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

Background- Convert "ISO-8859-1" from/to Unicode is incorrect

Example: M&M

Decode M%26amp;M- For CSV, HTML, XML, e-mail, and other fileformats, DB, HTTP and other protocols

OE
0x8C

White
Box

ISO-8859-1?

Windows-1252

31

Copyright © 2019 Tex Texin. All rights reserved.

00
8C

Control
Code
U+008C

OE
U+015233

7

Presented at IMUG SF 2019-08-07
ISO-8859-1

Windows 1252

Every code point is used. C1 control codes are legacy of
mainframe and minicomputers

C1 control codes replaced by characters. Some code
points are not (yet) assigned (81, 8D, 8F, 90, 9D)

80-9F

C1 Control Codes

Copyright © 2019 Tex Texin. All rights reserved.

34

35

Copyright © 2019 Tex Texin. All rights reserved.

ISO-8859-1 vs Windows-1252

Double Decode Detector

Test Recommendations

Common error is double conversion to UTF-8- Use characters in 0x80-0x9F (128-159)- Often data goes in and out without error being noticed- Dashes: – —- Punctuation: Smart quotes ‘’“”- Ellipses …
- Left/Right Single Pointing Quotation Marks: ‹ ›- Characters: OE Ligature "Œ œ", Trademark "™"Start

1st time

Most
characters
succeed1. Œ easy to use in text data & pseudolocalization
2. Use Ÿ and ÿ for case tests-

Convert Windows-1252 to UTF-8

ÿ U+00FF is in both encodings
Ÿ U+0178 0x9F (159) is only in Windows-1252

Test Data
showing error

2nd time

Convert Back to Windows-1252

1st time

Âƒ‚ƒ
Â
Âƒ
Âƒ
0xC3 0x83
0xC3 0xC3 0x83
0xC3 0x83
0xC2 0x83
Error 0x81
ÁÂ?
doesn't0xC1 0xC3 0x81
exist

2nd timeÂ
0xC3- Recommend Test Character Á (U+00C1) A-acute- Or others with byte value of 81, 8D, 8F, 90, 9D

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

36

Copyright © 2019 Tex Texin. All rights reserved.

37

8

Presented at IMUG SF 2019-08-07
The Date-Time Continuum
Time zone affects date as well as time
- Often overlooked, but important to test
- Use data that crosses datelines, dates, months- Use time zones with offsets other than 00

From time out of mind

UTC-13Date-time Tests

Yesterday
Feb 27
Dec 30Feb 28/29
Feb 28
Dec 31

UTC

UTC+0530

UTC+1225 Hour span
+ 2 days

Midnight
Feb 28
Dec 31
Mar 1
Feb 29
Jan 1

India

Tomorrow
Feb 29Next Year
Mar 2
Mar 1
Jan 2

Copyright © 2019 Tex Texin. All rights reserved.

The Date-Time Continuum

The Date-Time Continuum

Daylight Savings-Time zones change-Test changing periods of Daylight Savings- Test gaps in single time zone
- Test gaps crossing multiple time zones

39- How do you find dates that need updating?

- E.g. PST-BST, PST-BDT, PDT-BST, PDT-BDT
2 amBack 1 hr
1 am

1:30 am exists twice on this date.
Which do you convert to UTC?
1:30 PST or 1:30 PDT?

Forward 1 hr
3 am

2:30 am never occurs on thisdate. Do you detect the error?

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

40

Copyright © 2019 Tex Texin. All rights reserved.

41

9

Presented at IMUG SF 2019-08-07
Identify yourself! Names, Postal Addresses
Vast differences in conventions- Standards are not consistent- Due to differences in applications and use cases

Recommended Tests- Country Code: UK

Knock Knock! Who’s there?

- ISO 3166 code is GB

Postal code is UK

Source of errors- ISO Code GB causes mail to be rejected.
- Postal Code UK will fail validations using ISO Codes

Identity Tests- DB & XML Identifiers should distinguish them
- NOT <Country Code> but <PostalCountryCode> <ISOCountryCode>- Countries with no post (zip) code
- Ireland, Somalia, Syria (Although these may be changing)
Copyright © 2019 Tex Texin. All rights reserved.

Identify yourself! Names, Postal Addresses

E-mailField sizes are often inadequate

E-mail tests- Longest Surname MacGhilleseatheanaich- Longest Street Name (Poland)- Test syntax characters in email

Dwudziestego Pierwszego Praskiego Pulku Piechoty imienia Dzieci
Warszawy- Test maximum size (254 bytes)- Test International domain names in mail- "Téx" <Tex+1@xencraft.com>- Long Street Name with no spaces
Carl-Philipp-Emanuel-Bach-Straße Frankfurt (Oder), Germany- TEX@globalização.biz (Need your own domain)- tex@xn--globalizao-n5a1c.biz- Verify equivalence for search- Longest City Name (Anglesey, Wales 58 characters)- Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

43

44

Copyright © 2019 Tex Texin. All rights reserved.

45

10

Presented at IMUG SF 2019-08-07
Web Address / International Domain Names

Web Addresses

Test Cases for International Domain Names (IDN)

Test Cases- Max length (255-2048)- International pathwww.xencraft.com/globalização/globalização

- Mobile and Personal device browsers have lower limits- http://thelongestlistofthelongeststuffatthelongestdomainnameatlonglast.com- http://www.thequickbrownfoxjumpsoveralazydog.com- Verify links in non-UTF-8 pages first convert to UTF-8
Ç is %C3%A7 not %C7- Syntax characters, escapes (&, /, + # ? . % :)- International TLD, domain and subdomain names- International query and fragment, including non-UTF-8
www.xencraft.com/globalização/?a=çç#ã%C3%A7- http://www.xn--globalizao-n5a1c.biz- http://www.globalização.biz- http://globalização.globalização.biz- Testing subdomains are useful if you don't have an IDN- Test search equivalence of international and ASCII-fied names- Evaluate display- Test international domain names as parameters in URLs
- google.com/search?q=http%3A%2F%2Fglobaliza%C3%A7%C3%A3o.biz
Copyright © 2019 Tex Texin. All rights reserved.

46

Questions?

47

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin
Tex is an industry thought leader specializing in business and software
globalization services. His expertise includes global product strategy,
Unicode and internationalization architecture, and cost-effective
implementation and testing. Over the past two decades, Tex has creatednumerous global products, led internationalization development teams,
and guided companies in taking business to new regional markets.
Tex is a contributor to internationalization standards for software and on
the Web.
Tex is a popular speaker at conferences around the world and provides
on-site training on Unicode, internationalization, and globalization QA
worldwide.
Tex is the author of the popular, instructional web site www.I18nGuy.com
Tex is founder and Chief Globalization Architect for XenCraft. XenCraft
provides global business consulting and software design,
implementation, test and training services on globalization productstrategy and software internationalization architecture.

Copyright © 2019 Tex Texin. All rights reserved.

Tex Texin, XenCraft, I18nGuy

48

Copyright © 2019 Tex Texin. All rights reserved.

“XenCraft”, “TexTexin” and “I18nGuy”
are Trademarks of Tex Texin.

11