Repository: kubernetes-sigs/gateway-api

Issue:
GEP: Backend Protocol Selection
/kind gep

This is an attempt to separate out the backend protocol selection from [GEP-1282](https://github.com/kubernetes-sigs/gateway-api/issues/1282). This is similar to how TLS use-cases are being split out into [GEP-1897](https://github.com/kubernetes-sigs/gateway-api/issues/1897)


<!-- Please only use this template for submitting enhancement requests -->

**What would you like to be added**:

The ability to express the protocols supported by backends.

**Why this is needed**:

Not all implementation support automatic protocol selection. Even in some cases protocols are disabled without an explicit opt-in (ie. websockets). Thus application developers need the ability specify the protocol of their applications.



Discussion:
@dprotaso: /assign @dprotaso @dprotaso: For those following GEP PR is out - https://github.com/kubernetes-sigs/gateway-api/pull/1979@shaneutt: Since the GEP already exists and _because_ it's a GEP it seems reasonable to consider this accepted:

/triage accepted

However, it doesn't seem clear to me that this is something we _must_ sort out prior to GA (though I invite arguments to the contrary) so at least for now we wont consider this a blocker for GA and it will have a lower priority than GA-oriented work:

/priority backlog@dprotaso: @shaneutt the GA backlog has #1282 -  you could say this GEP and #1897 are part the fallout of that original GEP
@shaneutt: ACK

Ultimately it's just not clear that this is something we _must_ have for GA :thinking: 

Since you're working on it anyway, let's go ahead and track it on the board. But we should probably take another look at what it's gonna take to implement and do some planning and prioritization then.@robscott: Leaving this open until this GEP either graduates to standard or is removed from the API.

/kind gep
/reopen@k8s-ci-robot: @robscott: Reopened this issue.

<details>

In response to [this](https://github.com/kubernetes-sigs/gateway-api/issues/1911#issuecomment-1836963386):

>Leaving this open until this GEP either graduates to standard or is removed from the API.
>
>/kind gep
>/reopen


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>@k8s-triage-robot: This issue has not been updated in over 1 year, and should be re-triaged.

You can:
- Confirm that this issue is still relevant with `/triage accepted` (org members only)
- Close this issue with `/close`

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted@k8s-triage-robot: The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Mark this issue as fresh with `/remove-lifecycle stale`
- Close this issue with `/close`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/lifecycle stale

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/@aviallon: /close
Implemented by https://github.com/kubernetes-sigs/gateway-api/pull/1979@k8s-ci-robot: @aviallon: Closing this issue.

<details>

In response to [this](https://github.com/kubernetes-sigs/gateway-api/issues/1911#issuecomment-2748596744):

>/close
>Implemented by https://github.com/kubernetes-sigs/gateway-api/pull/1979


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>

Solution patch:
diff --git a/conformance/base/manifests.yaml b/conformance/base/manifests.yaml
index 361797c6..7f6754c9 100644
--- a/conformance/base/manifests.yaml
+++ b/conformance/base/manifests.yaml
@@ -100,6 +100,11 @@ spec:
     appProtocol: kubernetes.io/h2c
     port: 8081
     targetPort: 3001
+  - name: third-port
+    protocol: TCP
+    appProtocol: kubernetes.io/ws
+    port: 8082
+    targetPort: 3000
 ---
 apiVersion: apps/v1
 kind: Deployment
diff --git a/conformance/echo-basic/echo-basic.go b/conformance/echo-basic/echo-basic.go
index ac0e568c..465e3031 100644
--- a/conformance/echo-basic/echo-basic.go
+++ b/conformance/echo-basic/echo-basic.go
@@ -22,6 +22,7 @@ import (
 	"encoding/json"
 	"encoding/pem"
 	"fmt"
+	"io"
 	"net/http"
 	"os"
 	"regexp"
@@ -31,6 +32,7 @@ import (
 
 	"golang.org/x/net/http2"
 	"golang.org/x/net/http2/h2c"
+	"golang.org/x/net/websocket"
 )
 
 // RequestAssertions contains information about the request and the Ingress
@@ -100,6 +102,7 @@ func main() {
 	httpMux.HandleFunc("/health", healthHandler)
 	httpMux.HandleFunc("/status/", statusHandler)
 	httpMux.HandleFunc("/", echoHandler)
+	httpMux.Handle("/ws", websocket.Handler(wsHandler))
 	httpHandler := &preserveSlashes{httpMux}
 
 	errchan := make(chan error)
@@ -130,6 +133,13 @@ func main() {
 	}
 }
 
+func wsHandler(ws *websocket.Conn) {
+	fmt.Println("established websocket connection", ws.RemoteAddr())
+	// Echo websocket frames from the connection back to the client
+	// until io.EOF
+	io.Copy(ws, ws)
+}
+
 func healthHandler(w http.ResponseWriter, r *http.Request) {
 	w.WriteHeader(200)
 	w.Write([]byte(`OK`))
diff --git a/conformance/utils/suite/features.go b/conformance/utils/suite/features.go
index 063920f6..dc6e8511 100644
--- a/conformance/utils/suite/features.go
+++ b/conformance/utils/suite/features.go
@@ -136,6 +136,9 @@ const (
 
 	// This option indicates support for HTTPRoute with a backendref with an appProtocol 'kubernetes.io/h2c'
 	SupportHTTPRouteBackendProtocolH2C SupportedFeature = "HTTPRouteBackendProtocolH2C"
+
+	// This option indicates support for HTTPRoute with a backendref with an appProtoocol 'kubernetes.io/ws'
+	SupportHTTPRouteBackendProtocolWebSocket SupportedFeature = "HTTPRouteBackendProtocolWebSocket"
 )
 
 // HTTPRouteExtendedFeatures includes all the supported features for HTTPRoute
@@ -171,6 +174,7 @@ const (
 var HTTPRouteExperimentalFeatures = sets.New(
 	SupportHTTPRouteDestinationPortMatching,
 	SupportHTTPRouteBackendProtocolH2C,
+	SupportHTTPRouteBackendProtocolWebSocket,
 )
 
 // -----------------------------------------------------------------------------



Test patch:
diff --git a/conformance/tests/httproute-backend-protocol-ws.go b/conformance/tests/httproute-backend-protocol-ws.go
new file mode 100644
index 00000000..e05cda86
--- /dev/null
+++ b/conformance/tests/httproute-backend-protocol-ws.go
@@ -0,0 +1,109 @@
+/*
+Copyright 2023 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tests
+
+import (
+	"bytes"
+	"fmt"
+	"testing"
+	"time"
+
+	"golang.org/x/net/websocket"
+	"k8s.io/apimachinery/pkg/types"
+
+	"sigs.k8s.io/gateway-api/conformance/utils/http"
+	"sigs.k8s.io/gateway-api/conformance/utils/kubernetes"
+	"sigs.k8s.io/gateway-api/conformance/utils/suite"
+)
+
+func init() {
+	ConformanceTests = append(ConformanceTests,
+		HTTPRouteBackendProtocolWebSocket,
+	)
+}
+
+var HTTPRouteBackendProtocolWebSocket = suite.ConformanceTest{
+	ShortName:   "HTTPRouteBackendProtocolWebSocket",
+	Description: "A HTTPRoute with a BackendRef that has an appProtocol kubernetes.io/ws should be functional",
+	Features: []suite.SupportedFeature{
+		suite.SupportGateway,
+		suite.SupportHTTPRoute,
+		suite.SupportHTTPRouteBackendProtocolWebSocket,
+	},
+	Manifests: []string{
+		"tests/httproute-backend-protocol-ws.yaml",
+	},
+	Test: func(t *testing.T, suite *suite.ConformanceTestSuite) {
+		ns := "gateway-conformance-infra"
+		routeNN := types.NamespacedName{Name: "backend-protocol-ws", Namespace: ns}
+		gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns}
+		gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN)
+
+		threshold := suite.TimeoutConfig.RequiredConsecutiveSuccesses
+		maxTimeToConsistency := suite.TimeoutConfig.MaxTimeToConsistency
+
+		t.Run("websocket connection should reach backend", func(t *testing.T) {
+			http.AwaitConvergence(t, threshold, maxTimeToConsistency, func(elapsed time.Duration) bool {
+				origin := fmt.Sprintf("ws://gateway/%s", t.Name())
+				remote := fmt.Sprintf("ws://%s/ws", gwAddr)
+
+				ws, err := websocket.Dial(remote, "", origin)
+				if err != nil {
+					t.Log("failed to dial", err)
+					return false
+				}
+				defer ws.Close()
+
+				// Send text frame
+				var (
+					textMessage = "Websocket Support!"
+					textReply   string
+				)
+				if err := websocket.Message.Send(ws, textMessage); err != nil {
+					t.Log("failed to send text frame", err)
+					return false
+				}
+				if err := websocket.Message.Receive(ws, &textReply); err != nil {
+					t.Log("failed to receive text frame", err)
+					return false
+				}
+				if textMessage != textReply {
+					t.Logf("unexpected reply - want: %s got: %s", textMessage, textReply)
+					return false
+				}
+
+				// Send byte frame
+				var (
+					binaryMessage = []byte{1, 2, 3, 4, 5, 6, 7}
+					binaryReply   []byte
+				)
+				if err := websocket.Message.Send(ws, binaryMessage); err != nil {
+					t.Log("failed to send binary frame", err)
+					return false
+				}
+				if err := websocket.Message.Receive(ws, &binaryReply); err != nil {
+					t.Log("failed to receive binary frame", err)
+				}
+				if !bytes.Equal(binaryMessage, binaryReply) {
+					t.Logf("unexpected reply - want: %#v got: %#v", binaryMessage, binaryReply)
+					return false
+				}
+				return true
+			})
+		})
+	},
+}
diff --git a/conformance/tests/httproute-backend-protocol-ws.yaml b/conformance/tests/httproute-backend-protocol-ws.yaml
new file mode 100644
index 00000000..22cd52ac
--- /dev/null
+++ b/conformance/tests/httproute-backend-protocol-ws.yaml
@@ -0,0 +1,18 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: backend-protocol-ws
+  namespace: gateway-conformance-infra
+spec:
+  parentRefs:
+  - name: same-namespace
+  rules:
+  - backendRefs:
+    # This points to a Service with the following ServicePort
+    # - name: third-port
+    #   appProtocol: kubernetes.io/ws
+    #   protocol: TCP
+    #   port: 8082
+    #   targetPort: 3000
+    - name: infra-backend-v1
+      port: 8082