Repository: fatedier/frp (language: go)

Issue:
代理端持续登录不上，导致frps疑似出现内存泄漏
### Bug Description

frp客户端一直在尝试登录到frps服务端，frps服务端显示登录成功，但是客户端持续显示登录失败。frps服务端缓慢增长直至触发oom。此时连接代理不可用，kill客户端后代理注册恢复正常。

### frpc Version

0.48.0

### frps Version

0.48.0

### System Architecture

amd64

### Configurations

[common]
server_addr = x.x.x.x
server_port = 5000
tls_enable = true
tcp_mux = true

[api]
local_port = 443
type = tcp
local_ip = 127.0.0.1
[web]
local_port = 80
type = tcp
local_ip = 127.0.0.1

### Logs

客户端反复出现：
```
[I] [service.go:214] [d3e3d2791485238d] try to reconnect to server...
[W] [service.go:221] [d3e3d2791485238d] reconnect to server error: i/o deadline reached, wait 20s for another retry
```
服务端：反复出现 Replaced by client ， 无proxy注册成功信息
```
19:27:57 [I] [service.go:500] [d3e3d2791485238d] client login info: ip [x.x.x.x:7602] version [0.48.0] hostname [] os [linux] arch [amd64]
19:27:57 [I] [control.go:280] [d3e3d2791485238d] Replaced by client [d3e3d2791485238d]
19:28:27 [I] [control.go:280] [d3e3d2791485238d] Replaced by client [d3e3d2791485238d]
19:28:27 [I] [service.go:500] [d3e3d2791485238d] client login info: ip [x.x.x.x:61302] version [0.48.0] hostname [] os [linux] arch [amd64]
```

### Steps to reproduce

目前无发现复现手段

### Affected area

- [ ] Docs
- [ ] Installation
- [X] Performance and Scalability
- [ ] Security
- [ ] User Experience
- [ ] Test and Release
- [ ] Developer Infrastructure
- [ ] Client Plugin
- [ ] Server Plugin
- [ ] Extensions
- [ ] Others


Discussion / hints:
看不出问题
有没有可能是这里持续阻塞，导致登录请求超时，所以客户端是显示超时，服务端不停产生goroutine导致内存上涨。
![image](https://github.com/fatedier/frp/assets/44562010/9cc17924-fff1-4773-8ac3-1d42ee362a19)

> 看不出问题

```
	if oldCtl := svr.ctlManager.Add(loginMsg.RunID, ctl); oldCtl != nil {
		oldCtl.allShutdown.WaitDone()
	} else {
       	        time.Sleep(60*time.Second)
        }
        
	ctl.Start()
```
这样就复现了相同的日志情况
通过 pprof_enable 配置为 true 可以开启 prof 接口，然后查看一些相关的内存信息
主要是本地如何构造这个场景l来测试
> 主要是本地如何构造这个场景l来测试

![image](https://github.com/fatedier/frp/assets/44562010/80414cf2-fa66-43be-b769-c0d3ab9972b9)
客户端目前超时默认是10s，服务端在LOGIN的过程处理需要在10s内完成，如果超过时间就会复现此问题。在Login的过程目前看 svr.pluginManager.Login 插件调用是同步的，我在服务端有使用Login插件。且在插件接口 sleep 60 再次复现了问题。

服务端出现一个日志：[control.go:315] [f038074a851789aa] write message to control connection error: stream closed
问题可能出现在这里，，
![image](https://github.com/fatedier/frp/assets/44562010/ebb60a4e-34ca-43ac-82ab-50ae313e25df)
ctl.sendCh的大小为10，此时设置了poolCount 为20，ctl.writer内出现异常退出，导致 ctl.sendCh <- &msg.ReqWorkConn{} 持续阻塞，导致后续的login全部阻塞在等待这个control退出。
怀疑是 sendCh 和 readCh 的值没有与 poolCount 设置保持一致导致的
测试将 sendCh 和 readCh 设置大小为 poolCount  时，sleep后代理可以正常注册，control stop正常。
![image](https://github.com/fatedier/frp/assets/44562010/dc8f5711-1c5b-4f50-b45d-94d8c048bab9)

看起来这两个问题组合起来，确实会触发这个 bug

Fix patch:
diff --git a/Release.md b/Release.md
index ceaa7b02d58..8b59bafa523 100644
--- a/Release.md
+++ b/Release.md
@@ -1,3 +1,7 @@
 ### Features
 
 * Adds a `completion` command for shell completions.
+
+### Fixes
+
+* fix a goroutine leak issue caused by Login plugin timeout.
diff --git a/server/control.go b/server/control.go
index 9764b7c1485..abf17ab3b6b 100644
--- a/server/control.go
+++ b/server/control.go
@@ -29,7 +29,6 @@ import (
 
 	"github.com/fatedier/frp/pkg/auth"
 	"github.com/fatedier/frp/pkg/config"
-	"github.com/fatedier/frp/pkg/consts"
 	pkgerr "github.com/fatedier/frp/pkg/errors"
 	"github.com/fatedier/frp/pkg/msg"
 	plugin "github.com/fatedier/frp/pkg/plugin/server"
@@ -55,13 +54,14 @@ func NewControlManager() *ControlManager {
 	}
 }
 
-func (cm *ControlManager) Add(runID string, ctl *Control) (oldCtl *Control) {
+func (cm *ControlManager) Add(runID string, ctl *Control) (old *Control) {
 	cm.mu.Lock()
 	defer cm.mu.Unlock()
 
-	oldCtl, ok := cm.ctlsByRunID[runID]
+	var ok bool
+	old, ok = cm.ctlsByRunID[runID]
 	if ok {
-		oldCtl.Replaced(ctl)
+		old.Replaced(ctl)
 	}
 	cm.ctlsByRunID[runID] = ctl
 	return
@@ -141,14 +141,13 @@ type Control struct {
 	// replace old controller instantly.
 	runID string
 
-	// control status
-	status string
-
 	readerShutdown  *shutdown.Shutdown
 	writerShutdown  *shutdown.Shutdown
 	managerShutdown *shutdown.Shutdown
 	allShutdown     *shutdown.Shutdown
 
+	started bool
+
 	mu sync.RWMutex
 
 	// Server configuration information
@@ -187,7 +186,6 @@ func NewControl(
 		portsUsedNum:    0,
 		lastPing:        time.Now(),
 		runID:           loginMsg.RunID,
-		status:          consts.Working,
 		readerShutdown:  shutdown.New(),
 		writerShutdown:  shutdown.New(),
 		managerShutdown: shutdown.New(),
@@ -208,11 +206,19 @@ func (ctl *Control) Start() {
 		Error:   "",
 	}
 	_ = msg.WriteMsg(ctl.conn, loginRespMsg)
+	ctl.mu.Lock()
+	ctl.started = true
+	ctl.mu.Unlock()
 
 	go ctl.writer()
-	for i := 0; i < ctl.poolCount; i++ {
-		ctl.sendCh <- &msg.ReqWorkConn{}
-	}
+	go func() {
+		for i := 0; i < ctl.poolCount; i++ {
+			// ignore error here, that means that this control is closed
+			_ = errors.PanicToError(func() {
+				ctl.sendCh <- &msg.ReqWorkConn{}
+			})
+		}
+	}()
 
 	go ctl.manager()
 	go ctl.reader()
@@ -418,6 +424,14 @@ func (ctl *Control) stoper() {
 
 // block until Control closed
 func (ctl *Control) WaitClosed() {
+	ctl.mu.RLock()
+	started := ctl.started
+	ctl.mu.RUnlock()
+
+	if !started {
+		ctl.allShutdown.Done()
+		return
+	}
 	ctl.allShutdown.WaitDone()
 }
 
diff --git a/server/service.go b/server/service.go
index 5c93b667a86..002468814d1 100644
--- a/server/service.go
+++ b/server/service.go
@@ -552,7 +552,7 @@ func (svr *Service) RegisterControl(ctlConn net.Conn, loginMsg *msg.Login) (err
 
 	ctl := NewControl(ctx, svr.rc, svr.pxyManager, svr.pluginManager, svr.authVerifier, ctlConn, loginMsg, svr.cfg)
 	if oldCtl := svr.ctlManager.Add(loginMsg.RunID, ctl); oldCtl != nil {
-		oldCtl.allShutdown.WaitDone()
+		oldCtl.WaitClosed()
 	}
 
 	ctl.Start()


Test patch: