opened
alkorang
username_0
```
julia> versioninfo()
Julia Version 0.7.0-DEV.5219
Commit b65712285a* (2018-05-25 14:37 UTC)
Platform Info:
  OS: Windows (i686-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz
  WORD_SIZE: 32
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)

julia> display("δ"); println("δ")
"��"
δ

julia> display("한"); println("한")
"���"
한

julia> display("aδ한a"); println("aδ한a")
"a�����a"
aδ한a

julia>
```
display() problem printing multi-bytes characters
issue
created
digital-carver
username_1
Can't reproduce on my few weeks old build: 

```
julia> versioninfo()
Julia Version 0.7.0-DEV.5039
Commit 6a2313094d* (2018-05-08 06:05 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM)2 Duo CPU     P8600  @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, penryn)

julia> display("கககபோ"); println("கககபோ");
"கககபோ"
கககபோ

julia> display("한"); println("한")
"한"
한

julia>  display("aδ한a"); println("aδ한a")
"aδ한a"
aδ한a

julia>
```
comment
created
nalimilan
username_2
Probably a Windows-specific issue, as I cannot reproduce it either on Linux with a build from yesterday. What's your terminal app? Could you try with another one?
comment
created
alkorang
username_0
I use CMD, PowerShell, and mintty installed along with "Git for Windows". With mintty, no problem.

By the way it was not a problem with Julia 0.6.2 in any terminals.
```
julia> display("δ"); println("δ")
"δ"
δ

julia> display("한"); println("한")
"한"
한

julia> display("aδ한a"); println("aδ한a")
"aδ한a"
aδ한a

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17* (2017-12-13 18:08 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, ivybridge)

julia>
```
comment
created
alkorang
username_0
https://github.com/JuliaLang/julia/blob/deaefef5e34855e42270c8f257802db413b22cc6/base/multimedia.jl#L194-L205

By default `display(x)` uses `TextDisplay` and it call `show()` function.
```
julia> display("δ"); show("δ")
"��"
"��"
julia> display("한"); display("한")
"���"
"���"

julia> display("aδ한a"); show("aδ한a")
"a�����a"
"a�����a"
julia>
```
comment
created
alkorang
username_0
Write `String` UTF-8 bytes directrly works:
https://github.com/JuliaLang/julia/blob/2256c6baccfbc3ce34956cc72b6d01d3f40904f5/base/strings/io.jl#L149-L152

Writing `Char` code has a problem on Windows:
https://github.com/JuliaLang/julia/blob/2256c6baccfbc3ce34956cc72b6d01d3f40904f5/base/io.jl#L565-L573

`write()` calls `unsafe_write` with `String` which copies the bytes into `io`, while it writes single byte at a time with `Char`.
```
julia> A = unsafe_wrap(Array{UInt8,1}, pointer("한"), (sizeof("한"),))
3-element Array{UInt8,1}:
 0xed
 0x95
 0x9c

julia> write(stdout, A);
한
julia> for c in A; write(stdout, c); end
���
julia>
```
comment
created
vtjnash
username_3
We should probably consider fixing this is in libuv. It explicitly mutates the stream of utf-8 bytes when writing to a Console (into utf-16). When emitting a truncated trailing character, we should perhaps buffer it instead until the rest of the character (or the start of a new character) is written.
comment
created
StefanKarpinski
username_4
That does seem like a cleaner and more general way to fix this.
comment
created
davidanthoff
username_5
I think this should be assigned to the 0.7 milestone, i.e. I think this needs to be fixed before a release and should be treated as blocking. As far as I can tell all the ``show`` methods for all the tables in the various packages like DataFrames.jl are broken on Window because of this, and having those work properly seems really important for a release.
comment
created
vtjnash
username_3
Yes, that's essentially what the regression tag means
comment
created
davidanthoff
username_5
Oh, so the 65 open issues that have the ``regression` tag all need to be fixed before 0.7 can ship?
comment
created
JeffBezanson
username_6
No. "Regression" does not mean the same thing as "release blocker", it just means something is worse than before. A 10% drop in performance would be a regression, but probably not release-blocking.
comment
created
davidanthoff
username_5
So, I think this issue should be classified as "release blocker", and you guys can figure out what combination of labels/milestone signals that :)
comment
created
bkamins
username_7
What would you say about such an approach?

```
const CHAR_BUF = Vector{UInt8}(undef, 4)
const PTR_CHAR_BUF = pointer(CHAR_BUF)

function write(io::IO, c::Char)
    u = bswap(reinterpret(UInt32, c))
    n = UInt(1)
    while true
        @inbounds CHAR_BUF[n] = u % UInt8
        (u >>= 8) == 0 && return unsafe_write(io, PTR_CHAR_BUF, n)
        n += 1
    end
end
```

Not sure if `PTR_CHAR_BUF` is needed to be defined for performance reasons (maybe `pointer(CHAR_BUF)` in the body of the function is equivalent)?
comment
created
bkamins
username_7
I see I have made a suggestion in parallel to @username_6 commit - I will check which is faster.
comment
created
bkamins
username_7
Maybe in @username_6 PR you can avoid `u = fill(hton(v))` by making `u` a global constant and replacing it with `u[] = hton(v)`?
comment
created
JeffBezanson
username_6
Ha, amazing timing. I've been habitually avoiding something like that since it's not thread-safe, but we can't do I/O from multiple threads yet anyway so maybe it's ok for now.
comment
created
bkamins
username_7
Actually - after writing it I thought about threads also. You should create a global `Tuple` containing the `CHAR_BUF` individually allocated for each thread. We can do it, because number of threads is fixed in Julia on startup. This way you can make it threadsafe and avoid false sharing / NUMA issues (`CHAR_BUF` would have to be allocated on startup individually by each thread).
comment
created
JeffBezanson
username_6
Oh wait, I don't think that works either because multiple tasks can be trying to write at the same time, even with one thread.
comment
closed
JeffBezanson
username_6
issue