sharjeelyunus/github-issues-dataset huggingface dataset:
Repository: go
Issue: encoding/gob: should report if a name is registered or not.
Labels: NeedsInvestigation

by **brady@orchestrate.io**:

<pre>It would be nice to have a logical function to go with gob.Register() that reports back
if an object is actually registered. This would help establish if encoding of a given
object is actually going to be readable on the other end or not.

I attached a terrible, poorly written and completely untested conceptual patch to show
my intentions.</pre>


Attachments:
1. <a href="https://storage.googleapis.com/go-attachment/8779/0/gob.patch">gob.patch</a> (1806 bytes)
Repository: go
Issue: cmd/cgo: C streams are not flushed at program exit
Labels: NeedsInvestigation

by **kalyanakrishna**:

<pre>What does 'go version' print?
go version go1.3.1 linux/amd64

What steps reproduce the problem?
Take the first example on this page:
<a href="https://code.google.com/p/go-wiki/wiki/cgo">https://code.google.com/p/go-wiki/wiki/cgo</a>

Add 

func main() { Example() }
remove &quot;\n&quot; from cs

build and run


What happened?
Nothing prints

What should have happened instead?
it should have printed &quot;Hello from stdio&quot; without a trailing newline


Additional detail:
If &quot;\n&quot; appears in the printf argument anywhere, it will print till \n and
discard everything after \n

ex:

if you change cs to &quot;Hello from stdio \n and go &quot;, it will print 

&quot;Hello from stdio&quot;

this will happen even if you modify printf directly

ex:
if printf is modified to 

        printf(&quot;%s 123 &quot;, s);

it will print till the last &quot;\n&quot; it encounters and discard the rest. so if s
was &quot;hello from stdio\n&quot;, it will print &quot;hello from stdio&quot; and
discard 123


If possible, include a link to a program on play.golang.org.

import &quot;C&quot; wont work on play.golang.org</pre>
Repository: go
Issue: cmd/link: invalid dwarf frames
Labels: NeedsInvestigation

<pre>What does 'go version' print?
go version go1.3 darwin/amd64

What steps reproduce the problem?
If possible, include a link to a program on play.golang.org.

1. Compile test.go <a href="http://play.golang.org/p/8G9JQFMlmY">http://play.golang.org/p/8G9JQFMlmY</a>
2. Run lldb test
3. b main.foo
4. run

What happened?
lldb segfaulted

What should have happened instead?
no segfault

Please provide any additional information below.
It looks to me like the go compiler is generating invalid dwarf frames. I can't really
follow the code in dwarf.c to tell what's going on. Here's what dwarfdump shows:

0x00000014: FDE
        length: 0x00000020
   CIE_pointer: 0x00000000
    start_addr: 0x0000000000002000 main.foo
    range_size: 0x0000000000000060 (end_addr = 0x0000000000002060)
  Instructions: 0x0000000000002000: CFA=rsp+8     rip=[rsp]
                DW_CFA_advance_loc (25)
                DW_CFA_def_cfa_offset_sf (rax, 8)
                0x0000000000002019: CFA=rsp+8     rip=[rsp]
                DW_CFA_advance_loc (55)
                DW_CFA_def_cfa_offset_sf (rax, 24)
                0x0000000000002050: CFA=rsp+24    rip=[rsp+16]
                DW_CFA_advance_loc (16)
                DW_CFA_def_cfa_offset_sf (rax, 8)
                DW_CFA_nop
                DW_CFA_nop
                DW_CFA_nop
                0x0000000000002060: CFA=rsp+8     rip=[rsp]

If I'm reading the dissasembly right, I think the rows should be

0x0000000000002000: CFA=rsp+8     rip=[rsp]
0x0000000000002019: CFA=rsp+24     rip=[rsp+16]
0x0000000000002050: CFA=rsp+8    rip=[rsp]

Also, dwarfdump --verify complains because the row for 0x2060 is beyond the address
range of the function.

Here's the disassembled function:
test`main.foo at test.go:3:
test[0x2000]:  movq   %gs:0x8a0, %rcx
test[0x2009]:  cmpq   (%rcx), %rsp
test[0x200c]:  ja     0x2015                    ; main.foo + 21 at test.go:3
test[0x200e]:  callq  0x21710                   ; runtime.morestack8_noctxt at
asm_amd64.s:495
test[0x2013]:  jmp    0x2000                    ; main.foo at test.go:3
test[0x2015]:  subq   $0x10, %rsp
test[0x2019]:  leaq   0x2b800, %rbx
test[0x2021]:  leaq   (%rsp), %rbp
test[0x2025]:  movq   %rbp, %rdi
test[0x2028]:  movq   %rbx, %rsi
test[0x202b]:  movsq  (%rsi), %es:(%rdi)
test[0x202d]:  movsq  (%rsi), %es:(%rdi)
test[0x202f]:  callq  0x10770                   ; runtime.printstring at print.c:373
test[0x2034]:  callq  0x107e0                   ; runtime.printsp at print.c:384
test[0x2039]:  movq   0x18(%rsp), %rbx
test[0x203e]:  movq   %rbx, (%rsp)
test[0x2042]:  callq  0x10620                   ; runtime.printint at print.c:340
test[0x2047]:  callq  0x10820                   ; runtime.printnl at print.c:390
test[0x204c]:  addq   $0x10, %rsp
test[0x2050]:  retq
test[0x2051]:  addb   %al, (%rax)
test[0x2053]:  addb   %al, (%rax)
test[0x2055]:  addb   %al, (%rax)
test[0x2057]:  addb   %al, (%rax)
test[0x2059]:  addb   %al, (%rax)
test[0x205b]:  addb   %al, (%rax)
test[0x205d]:  addb   %al, (%rax)</pre>
Repository: go
Issue: cmd/go: when compiling with gccgo uses go tool's ReleaseTags, not gccgo's
Labels: NeedsInvestigation

<pre>What steps reproduce the problem?

1. Install a version of go (such as 1.3) and a version of gccgo that implements a
different version of the language (such as gccgo 4.9, which implements go 1.2)
2. Try to build a package that uses the go1.3 build tag to conditionally support go 1.3
features (for example, <a href="https://github.com/docker/docker/tree/master/pkg/pools">https://github.com/docker/docker/tree/master/pkg/pools</a> )

What happened?

The go tool attempts to compile the file with the go1.3 built tag with gccgo, and this
fails.

What should have happened instead?

It should have compiled the file with the !go1.3 build tag instead.

Please provide any additional information below.

The cause for this is pretty clear: the go tool uses the ReleaseTags from the go/build
package of the standard library that the go tool was built against, which is not
necessarily the same as the one the compiler being invoked.  This could be fixed by
compiling a little program that prints out go/build.Default.ReleaseTags and using those
in the go tool

There are other places where go tool and gccgo compiler have different views of things,
but this is the one that is biting me today.</pre>
Repository: go
Issue: cmd/compile: confusing error messages when a type is repeated
Labels: Suggested

<pre>Code:
<a href="http://play.golang.org/p/MoahP8X1aD">http://play.golang.org/p/MoahP8X1aD</a>

If a type is defined twice in a given package, this can lead to unfortunate interface
assignment errors. As an example, in the code above, &quot;type Foo int&quot; exists
twice. The error message is

prog.go:9: Foo redeclared in this block
    previous declaration at prog.go:3
prog.go:18: cannot convert Bar literal (type Bar) to type Fooer:
    Bar does not implement Fooer (wrong type for Foo method)
        have Foo() Foo
        want Foo() Foo

In this particular case it's fine because the &quot;Foo redeclared&quot; error comes
first, but in my actual code the interface conversion error was returned first and then
there were more errors before the redeclaration error.</pre>
Repository: go
Issue: crypto/x509: no support for parsing encrypted PKCS8 private keys
Labels: FeatureRequest

by **alex.gaynor**:

<pre>Right now only un-encrypted keys are supported, it would be good if encrypted ones were
as well.</pre>
Repository: go
Issue: time: mockable time support
Labels: NeedsInvestigation

<pre>With <a href="https://github.com/jmhodges/clock">https://github.com/jmhodges/clock</a> and the broader <a href="https://github.com/benbjohnson/clock">https://github.com/benbjohnson/clock</a> API,
there's a desire for the timing systems in Go to be fakeable[1]. Using a fake clock
instead of time.Now directly is useful when testing code that stores timestamps, caches
data, etc.

The benbjohnson clock package attempts to make Ticker and Timer calls (as well as their
related AfterFunc and Sleep methods) work against a fake time that can be set and
updated in tests. However, it relies on some micro-sleeps and runtime.Gosched calls that
are obviously going to be flaky. But there is a desire to able to test code that uses
Tickers and Timers, not by adjusting the durations they work for (which can induce flaky
testing), but by adjusting when they think they need to wake up.

To do that, we seem to need more runtime magic to help developers out. In fact, it might
be best if a clock package like these lived in the stdlib so that it could be tied more
carefully and thoughtfully to the scheduler.

 [1] or &quot;mockable&quot;, whatever language you prefer. The summary of this issue is to distinguish it from <a href="https://golang.org/issue/5356">issue #5356</a>.</pre>
Repository: go
Issue: net: support for android without cgo
Labels: help wanted,OS-Android

It is possible to build Android OS binaries (not apps) without cgo. Those apps will fail
because the Android OS does not define /etc/resolve.conf. Some more details on #8020 and
the linked CL.
Repository: go
Issue: os/user: android support
Labels: OS-Android

<pre>Android OS does not support getpwuid_r. There does not appear to be a C/C++ interface
for this, we may have to call into Java to get some notion of users.

As most apps don't make heavy use of OS-level user information, this is a low priority.</pre>
Repository: go
Issue: runtime: consider using SetFileIoOverlappedRange
Labels: Performance,OS-Windows

<pre>SetFileIoOverlappedRange should improve performance/scalability of networking apps on
windows:
<a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa365540(v=vs.85).aspx">http://msdn.microsoft.com/en-us/library/windows/desktop/aa365540(v=vs.85).aspx</a>
I have a prototype patch:
<a href="https://golang.org/cl/12562049/">https://golang.org/cl/12562049/</a>
but did not benchmark it. Need to benchmark it and decide whether it is worth it.</pre>
Repository: go
Issue: runtime: efficient access to thread-local data
Labels: Performance

<pre>Currently we have 3 performance issues with accesses to thread-local data (g/m/p):
1. Accesses require non-inlinable function calls.
2. The only thread-local var is now g, while most frequently accesses data is in m. So
most of the accesses has an additional indirection.
3. We do lots of duplicate loads of g/m.

We need to:
1. Make the thread-local var m (instead of g).
2. Move stack guard of the current g into m (that's the only hot data in g).
3. Declare runtime.curm variable in runtime, teach the compiler to recognize it and turn
into tls access.
4. Teach compiler to not do unnecessary duplicate loads of curm (like in
<a href="https://golang.org/issue/4946)">https://golang.org/issue/4946)</a>.</pre>
Repository: go
Issue: runtime: don't pass *Type to new 
Labels: Performance

<pre>Currently we pass a pointer to Type descriptor to new. It preserves alive all methods of
allocated types, as Type contains Method array. So linker can't strip them.
We should pass a special object that contains only necessary information to new:
1. size
2. size class (needs to be ignored for arrays with more than one element)
2. name string
3. embed GC pointer bitmask or a pointer to satellite GC bitmask (if both are zero, then
it's noptr type)</pre>
Repository: go
Issue: runtime: don't do unnecessary loads on malloc fast path
Labels: Performance

<pre>Currently we load runtime.memstats.heap_alloc and runtime.MemProfileRate on every
malloc. Both are not strictly necessary and bring 2 unnecessary cache lines into cache.

Here are proof-of-concept CLs:
<a href="https://golang.org/cl/129810043/">https://golang.org/cl/129810043/</a>
<a href="https://golang.org/cl/128100043/">https://golang.org/cl/128100043/</a>

Ideally we trigger GC directly from mcacheRefill, but it's not trivial to implement.</pre>
Repository: go
Issue: runtime: remove unnecessary allocations in convT2E
Labels: Performance

<pre>This was at lengths discussed in:
<a href="https://golang.org/issue/8405">https://golang.org/issue/8405</a>
and previously on golang-dev:
<a href="https://groups.google.com/d/msg/golang-dev/pwUh0BVFpY0/zqJInvU3NkQJ">https://groups.google.com/d/msg/golang-dev/pwUh0BVFpY0/zqJInvU3NkQJ</a>

Namely, we should allocate heap block for scalars iff the scalar look like a pointer
into heap (otherwise GC will ignore it anyway).

This will allow to have 1-bit/word GC pointer type info *and* don't allocate additional
memory for scalars in interfaces in most cases.</pre>
Repository: go
Issue: cmd/go: critical path scheduling
Labels: ToolSpeed

<pre>Currently go tool schedules work in topological order, this leads to suboptimal
parallelization. It's possible that towards end, very few work items are available
leading to idle processors.

Critical path scheduling gives much better results in this respect:
<a href="http://en.wikipedia.org/wiki/Critical_path_method">http://en.wikipedia.org/wiki/Critical_path_method</a>

Here is a prototype CL:
<a href="https://golang.org/cl/13235046/diff/3001/src/cmd/go/build.go">https://golang.org/cl/13235046/diff/3001/src/cmd/go/build.go</a>
I've observed up to 2x 'go test -a std' speedup on a 32-way machine.

We can also assign meaningful weights to work nodes: e.g. test &gt; link &gt; compile.
It would also be super useful to somehow assign weights to test nodes proportional to
test run time (e.g. runtime tests take significantly more than than most packages). This
will have significant effect even on low-core machines (as runtime tests will start
first).</pre>
Repository: go
Issue: cmd/compile: recognize case <-time.After(x)
Labels: Performance,compiler/runtime

<pre>Compiler can recognize the common:
   case &lt;-time.After(x):
construct and lower it into:
   runtime.selectrecvafter(x)
Runtime can handle this case much more efficiently: no need to create Timer, chan and
arm timers; it just needs to do a timed park of the goroutine until x. As a bonus, the
timer does not leak.</pre>
Repository: go
Issue: runtime: fine-grained locking in select
Labels: Performance,early-in-cycle,Scalability

<pre>Fine-grained locking in select provides substantial speedups:

benchmark                            old ns/op     new ns/op     delta
BenchmarkSelectUncontended           213           188           -11.74%
BenchmarkSelectUncontended-2         110           97.2          -11.64%
BenchmarkSelectUncontended-4         56.2          49.5          -11.92%
BenchmarkSelectUncontended-8         29.7          25.9          -12.79%
BenchmarkSelectUncontended-16        17.3          16.4          -5.20%
BenchmarkSelectUncontended-32        13.7          11.4          -16.79%
BenchmarkSelectSyncContended         795           710           -10.69%
BenchmarkSelectSyncContended-2       946           710           -24.95%
BenchmarkSelectSyncContended-4       1791          1187          -33.72%
BenchmarkSelectSyncContended-8       1749          637           -63.58%
BenchmarkSelectSyncContended-16      2109          563           -73.30%
BenchmarkSelectSyncContended-32      3165          1172          -62.97%
BenchmarkSelectAsyncContended        231           208           -9.96%
BenchmarkSelectAsyncContended-2      671           209           -68.85%
BenchmarkSelectAsyncContended-4      395           215           -45.57%
BenchmarkSelectAsyncContended-8      451           240           -46.78%
BenchmarkSelectAsyncContended-16     486           233           -52.06%
BenchmarkSelectAsyncContended-32     861           395           -54.12%
BenchmarkSelectNonblock              126           36.0          -71.43%
BenchmarkSelectNonblock-2            155           17.6          -88.65%
BenchmarkSelectNonblock-4            285           8.45          -97.04%
BenchmarkSelectNonblock-8            212           4.41          -97.92%
BenchmarkSelectNonblock-16           180           2.44          -98.64%
BenchmarkSelectNonblock-32           395           1.63          -99.59%
BenchmarkSelectProdCons              1140          926           -18.77%
BenchmarkSelectProdCons-2            653           545           -16.54%
BenchmarkSelectProdCons-4            519           352           -32.18%
BenchmarkSelectProdCons-8            667           372           -44.23%
BenchmarkSelectProdCons-16           751           431           -42.61%
BenchmarkSelectProdCons-32           1142          583           -48.95%

Implementation:
<a href="https://golang.org/cl/112990043/">https://golang.org/cl/112990043/</a></pre>
Repository: go
Issue: runtime: lock-free channels
Labels: Performance

<pre>Here is design doc:
<a href="https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW4cSv1wy5hC0ApeGMV9s/pub">https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW4cSv1wy5hC0ApeGMV9s/pub</a>

Here is the implementation:
<a href="https://golang.org/cl/12544043/">https://golang.org/cl/12544043/</a>

I've benchmarked the change of a real app where chans were abandoned in favor of other
sync primitives due to perf reasons:
<a href="https://groups.google.com/d/msg/golang-dev/0IElw_BbTrk/cGHMdNoHGQEJ">https://groups.google.com/d/msg/golang-dev/0IElw_BbTrk/cGHMdNoHGQEJ</a>
and the change provides 23% end-to-end speedup making chans significantly faster than
the other synchronization.</pre>
Repository: go
Issue: runtime: throttle goroutines on async chans 
Labels: Performance

<pre>There is a common trick for reduction of contention on sync primitives. Unfortunately it
does not have a common name, but see mutexWaiterShift handling in sync/mutex.go for an
example. We can use it for async channels to reduce contention as well. Namely:

- add a waiterAwake flag to Hchan
- when a goroutine sends to async chan and there is a parked receiver, if
waiterAwake=false unpark it and set waiterAwake, otherwise do nothing
- if the woken receiver discovers empty queue, it resets waiterAwake before parking again
- if the woken receiver successfully dequeues from the queue, it does:
    if queue empty || no parked receivers {
      waiterAwake = false
    } else {
      unpark one parked receiver
      // responsibility to manage waiterAwake and subsequent parked receivers
      // transfers to that next recevier
    }

This algorithm ensures that there is at most 1 excessive receiver loafs about producing
unnecessary contention. When this receivers goes away, it wakes up another receiver. And
so on.</pre>
Repository: go
Issue: runtime: squeeze Hchan into 64 bytes 
Labels: Performance

<pre>Currently Hchan is 12 words (on 64-bit machine). It would be nice to squeeze it into 8
words. First it's just memory consumption reduction, and second it will be exactly cache
line (and due to malloc alignment, it will be also cache line aligned) which will
speedup contented synchronization - ping-pong only one line instead of two + no false
sharing.

elemtype goes away as it is not needed.
dataqsiz is a derivative of qcount/sendx/recvx, so it just goes away as well.
recvq/sendq can be reduced to 1 pointer to a head of doubly-linked cycle of waiters.
This gives exactly 8 words.

FTR, here is a CL that lowers OCAP/OLEN for chans into runtime calls (necessary to
remove dataqsiz):
<a href="https://golang.org/cl/54570046/">https://golang.org/cl/54570046/</a></pre>
Repository: go
Issue: runtime: special case chan struct{}
Labels: Performance

<pre>chan struct{} is commonly used for semaphores.
We can special case it in runtime as atomic counter. Then common case (of no work
limiting) will be just atomicinc/atomicdec.</pre>
Repository: go
Issue: runtime: make chan-based generators faster
Labels: Performance

<pre>In some cases parallelism is not needed/desirable -- fine-grained chan-based
generators/iterators, or &quot;coroutine-like&quot; approach when you have several
independent stacks and manually switch between them.
Sync channels substantially limit parallelism, to the point where pipeline-like
parallelization does not work. But they do not eliminate it entirely, as after an
operation both producer and consumer are runnable. Current implementation handles this
&quot;contact bounce&quot; under-parallelization very inefficiently -- goroutines are
constantly scheduled and de-scheduled, threads are constantly parked and unparked, work
migrate between threads, etc.

The idea is to limit parallelism on sync channels even more aggressively. So that
fine-grained communication over sync channels can be very efficient. But w/o eliminating
concurrency nor affecting semantics.

Implementation:
1. Each P has an additional local work queue, let's call it Domain. Work form Domain
can't be stolen by other P's. Domain is has maximum size of 4 goroutines.
2. When a goroutine unblocks another goroutine on a sync chan, it adds the other
goroutine to own Domain (Domain of the current P) instead of putting it to the normal
work queue. This effectively limits parallelism. We can use Domains in other situations,
but this is outside of context of this bug.
3. When a goroutine blocks, P schedules one of the goroutines from Domain (if it's not
empty). In some circumstances (e.g. a goroutine is preempted due to time slice
exhaustion, or blocks in syscall) P can decide to just move all goroutines in the Domain
to normal work queue.
Then, *before* a goroutine blocks on a sync chan it saves Hchan and elem pointers in G
and switches to another goroutine in Domain *once*. Or if it sees that a goroutine in
the Domain want to accomplish a pairing operation on the same chan, it executes the
operation with that other goroutine.

This allows to implement tight communication over sync channels with the cost of just
several function calls and w/o several degradation with GOMAXPROCS&gt;1. Communication
domains form and break up dynamically. Program semantics are not affected.

Here is a prototype implementation:
<a href="https://golang.org/cl/74180043/">https://golang.org/cl/74180043/</a>

I've benchmarked it on a real application where chan-based generators were abounded due
to forementioned reasons:
<a href="https://groups.google.com/d/msg/golang-dev/0IElw_BbTrk/cGHMdNoHGQEJ">https://groups.google.com/d/msg/golang-dev/0IElw_BbTrk/cGHMdNoHGQEJ</a>
The change provides 17.5x speedup (and it's not the limit as runtime.switchto can be
significantly more efficient).</pre>
Repository: go
Issue: cmd/link: implement external linking support for ppc64/linux
Labels: NeedsInvestigation

<pre>We need to fix the 9l linker to provide external linking support, so that
-linkmode=external works on power64.

This code is currently on the dev.power64 branch and should come into tip after the 1.4
release is out.  This means writing elfreloc1, elfsetupplt, adddynsym, adddynrela, etc.,
in cmd/9l/asm.c and doing whatever else is necessary so that (cd misc/cgo/test
&amp;&amp; go test -linkmode=external) passes for both big- and little- endian power64.</pre>
Repository: go
Issue: math, crypto, hash: Write power64 versions of assembler routines where applicable
Labels: NeedsInvestigation

<pre>We want to write power64 versions of the assembler routines in the math package, in
several crypto packages, and in the hash/crc32 package.  For each of those packages
where we have .s files, we should either write a power64 version or determine that the
Go version is as good as we can write in assembler.</pre>