Issue 1:
by travis.cardwell:

The gofmt command does not take into account the width of runes when calculating the
alignment for comments. I ran into the issue in a program that has rune constants, some
of which are single-width and some of which are double-width, but the issue is more
apparent in the following (contrived) example:

What does 'go version' print?

$ go version
go version go1.2.1 linux/amd64

What steps will reproduce the problem?

$ cat fmttest.go
package fmttest

const (
msgEn = "This is a test." // English
msgJa = "これはテストです。" // Japanese
)

$ gofmt -d fmttest.go
diff fmttest.go gofmt/fmttest.go
--- /tmp/gofmt819271649 2014-03-06 14:13:28.000000000 +0900
+++ /tmp/gofmt002870220 2014-03-06 14:13:28.000000000 +0900
@@ -1,6 +1,6 @@
package fmttest

const (
- msgEn = "This is a test." // English
- msgJa = "これはテストです。" // Japanese
+ msgEn = "This is a test." // English
+ msgJa = "これはテストです。" // Japanese
)

The issue is not clear in this web interface [with my font settings] because spaces do
not have the same constant width as non-spaces, but it is clear if you copy the above
example into a file and run the gofmt command in a terminal.

What happened?

The alignment for comments is calculated based on the maximum width of code in the
block. Currently, all runes are assumed to have a width of 1. With tabwidth=8, the
width of the msgEn code is 33 columns, and the width of the msgJa code is incorrectly
calculated as 27 columns. Comments are therefore calculated to start on column 35. The
msgEn comment is offset from the code by one space, and the msgJa comment is offset from
the code by 7 spaces. The result is not aligned properly. The original alignment was
expected to be correct.

What should have happened instead?

Instead, the width of runes should be taken into account when calculating alignment.
With tabwidth=8, the width of the msgJa code should be calculated as 36 columns, as all
9 runes in the string are double-width. Comments should then be calculated to start on
column 38, resulting in the original alignment.

Please provide any additional information below.

Note that the length of a rune (in bytes) is unrelated to the width of a rune (in
columns). When testing, I highly recommend including troublesome cases. For example,
the following rune is commonly used in Japanese yet causes issues in some software:

http://decodeunicode.org/u+203B

Issue 2:
What steps will reproduce the problem?
Issue:
gofmt, or text.tabwriter, assumes that all Unicode code points occupy exactly one column
in editors or on terminals. That assumption is not correct because most (but not all)
Chinese/Japanese/Korean characters, emojis, "fullwidth" Latin characters, etc,
occupy two columns. As a result gofmt formats Go code like this.

var Countries = map[string]string{
"アメリカ合衆国": "United States of America",
"日本": "Japan",
"ドイツ": "Germany",
"フランス": "France",
"ポーランド": "Poland",
}

As you can see the column of the map value is misaligned. You cannot fix this by hand
because gofmt would reformat it for you in the wrong way if you do that. That's annoying.

In Unicode, there's a zero column character (ZERO WIDTH SPACE; U+200B). SOFT HYPHEN
(U+00AD) may be displayed as a hyphen at the end of a line but may be zero-width in
other places, depending on your display environment. These chracters also affect the
column layout.

What is the expected output? What do you see instead?
Proposal:
Unicode Standard Annex #11 gives the definition of column width for characters in the
legacy East Asian character sets. I propose to add the East Asian Width property to the
unicode package, so that we can get the column width for a CJK character. East Asian
Fullwidth and East Asian Wide characters should be treated as two column by tabwriter.

(Note: East Asian Ambiguous characters need to be treated as one column. They are
treated as two columns only in East Asian display environment. The character set
contains Cyrillic characters and others which we would never want to handle as two
column.)

Because the Annex #11 does not say anything about characters that are not in the legacy
East Asian character sets, we need additional rules for characters not in CJK character
sets but in Unicode. I propose this simple rule:

- ZERO WIDTH SPACE is 0 column
- Emojis are 2 columns
- Other code points, including U+0000, SOFT HYPHEN, and all control characters, are 1 column

This additional rule will be implemented to an unexported function in text.tabwriter.

Caveats:
I deliberately avoid defining the generic "wcswidth" function to determine the
column width for a string in the standard library. That function can never be defined in
the right way because there's no standard for it. Also it'd be hard to get a reasonable
definition for characters with odd semantics, such as SOFT HYPHEN.