Repository: traefik/traefik (language: go)

Issue:
AWS ECS Service Connect - Go runtime error: index out of range
### Welcome!

- [X] Yes, I've searched similar issues on [GitHub](https://github.com/traefik/traefik/issues) and didn't find any.
- [X] Yes, I've searched similar issues on the [Traefik community forum](https://community.traefik.io) and didn't find any.

### What did you do?

AWS ECS Service Connect feature for ECS was recently released (Nov 2022). It provides service mesh like communication between containers by injecting a sidecar (Envoy) into the task definition automatically. 

Have done the following;

1. deployed ECS Cluster (with ec2 instances)
2. deployed Traefik Proxy as service (with ALB fronting it) & dashboard is working
3. deployed Nginx as service (bridge mode so that it maps a dynamic port on host, to the mapped container port 80, either Service Connect enabled)

The Nginx service deploys and runs. Has one task with two containers (nginx, and the Envoy sidecar). Traefik however starts throwing errors when trying to interrogate this new service it found and cannot register it. 

Interestingly, if I disable ECS Service Connect for the service, Traefik successfully registers the service with the Proxy and all works as expected.

This is a problem though as I need ECS Service Connect to work. So I have started debugging this a little, but I do not know GO or how to develop/test/debug with this language. So I am hoping all the information I put in here might provide a quick fix for someone that can edit and test the functionality.

# Error logged by Traefik & stack trace


```log
Error in Go routine: runtime error: index out of range [0] with length 0

Stack: goroutine 47 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:24 +0x65
github.com/traefik/traefik/v2/pkg/safe.defaultRecoverGoroutine({0x42db560?, 0xc000829d40})
	github.com/traefik/traefik/v2/pkg/safe/routine.go:66 +0xa5
github.com/traefik/traefik/v2/pkg/safe.OperationWithRecover.func1.1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:74 +0x35
panic({0x42db560, 0xc000829d40})
	runtime/panic.go:884 +0x212
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).listInstances(0xc000820630, {0x52a21f0, 0xc0009eb4d0}, 0xc000a7e5b8)
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:334 +0x1ae8
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).loadConfiguration(0xc000efbd20?, {0x52a21f0, 0xc0009eb4d0}, 0x0?, 0x445dee0?)
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:196 +0x31
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).Provide.func1.1()
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:172 +0x267
github.com/traefik/traefik/v2/pkg/safe.OperationWithRecover.func1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:78 +0x63
github.com/cenkalti/backoff/v4.RetryNotifyWithTimer(0xc000c99460, {0x52818f0, 0xc000c26000}, 0xc001129f20, {0x0?, 0x0?})
	github.com/cenkalti/backoff/v4@v4.1.3/retry.go:55 +0x12a
github.com/cenkalti/backoff/v4.RetryNotify(...)
	github.com/cenkalti/backoff/v4@v4.1.3/retry.go:34
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).Provide.func1({0x52a2148?, 0xc000a64040})
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:186 +0x337
github.com/traefik/traefik/v2/pkg/safe.(*Pool).GoCtx.func1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:36 +0x66
github.com/traefik/traefik/v2/pkg/safe.GoWithRecover.func1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:59 +0x5b
created by github.com/traefik/traefik/v2/pkg/safe.GoWithRecover
	github.com/traefik/traefik/v2/pkg/safe/routine.go:53 +0x72
```



Most interesting bits;
* Index 0 out of range - sounds like an array has no items.
* Line 334 of ecs.go - appears to be the offending line.


# Looking at https://github.com/traefik/traefik/blob/454f552691015d5f01041f61ae0cca0cff5d5d5d/pkg/provider/ecs/ecs.go#L334

```
317				if len(task.Attachments) != 0 {
318					var ports []portMapping
319					for _, mapping := range containerDefinition.PortMappings {
320						if mapping != nil {
321							protocol := "TCP"
322							if aws.StringValue(mapping.Protocol) == "udp" {
323								protocol = "UDP"
324							}
325
326							ports = append(ports, portMapping{
327								hostPort:      aws.Int64Value(mapping.HostPort),
328								containerPort: aws.Int64Value(mapping.ContainerPort),
329								protocol:      protocol,
330							})
331						}
332					}
333					mach = &machine{
334						privateIP:    aws.StringValue(container.NetworkInterfaces[0].PrivateIpv4Address),
335						ports:        ports,
336						state:        aws.StringValue(task.LastStatus),
337						healthStatus: aws.StringValue(task.HealthStatus),
338					}
```

Line 334 appears to look for the 1st element (0) of container.NetworkInterfaces. 
Suspect this does not exist. Let's look at the containers within the task.



# Describing the containers within task for this service

aws ecs describe-tasks --cluster ecs-cluster --tasks  arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task/ecs-cluster/0e66e90e04694eba9c1b0593264a220b


```json
{
    "tasks": [
        {
            "attachments": [
                {
                    "id": "e74dcf96-ad50-48d1-99ad-5d181070d0e9",
                    "type": "ServiceConnect",
                    "status": "ATTACHED",
                    "details": [
                        {
                            "name": "ContainerName",
                            "value": "ecs-service-connect-9qpWT"
                        }
                    ]
                }
            ],
            "attributes": [
                {
                    "name": "ecs.cpu-architecture",
                    "value": "x86_64"
                }
            ],
            "availabilityZone": "ap-southeast-2a",
            "clusterArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:cluster/ecs-cluster",
            "connectivity": "CONNECTED",
            "connectivityAt": "2023-01-11T11:06:24.787000+11:00",
            "containerInstanceArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:container-instance/ecs-cluster/63e1895b9c2d4d528932c6ab9cdb7a01",
            "containers": [
                {
                    "containerArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:container/ecs-cluster/0e66e90e04694eba9c1b0593264a220b/ad26a2c7-a1bd-4ee0-b3c0-83d7dffade89",
                    "taskArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task/ecs-cluster/0e66e90e04694eba9c1b0593264a220b",
                    "name": "result",
                    "image": "nginx:latest",
                    "runtimeId": "fd236fa02ef67a4c00ea6a30a0c50365c50a259135905fdd3cd4e866df24c001",
                    "lastStatus": "RUNNING",
                    "networkBindings": [
                        {
                            "bindIP": "0.0.0.0",
                            "containerPort": 80,
                            "hostPort": 49160,
                            "protocol": "tcp"
                        }
                    ],
                    "networkInterfaces": [],
                    "healthStatus": "UNKNOWN",
                    "cpu": "0"
                },
                {
                    "containerArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:container/ecs-cluster/0e66e90e04694eba9c1b0593264a220b/bf930cda-c08c-4168-b825-a21cbd932781",
                    "taskArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task/ecs-cluster/0e66e90e04694eba9c1b0593264a220b",
                    "name": "ecs-service-connect-9qpWT",
                    "runtimeId": "6f8dab6af4db5dcb37a26686b79174e904fb912674238541be85a662cc85cd04",
                    "lastStatus": "RUNNING",
                    "networkBindings": [
                        {
                            "bindIP": "0.0.0.0",
                            "containerPort": 10000,
                            "hostPort": 49159,
                            "protocol": "tcp"
                        }
                    ],
                    "networkInterfaces": [],
                    "healthStatus": "HEALTHY"
                }
            ],
            "cpu": "512",
            "createdAt": "2023-01-11T11:06:24.787000+11:00",
            "desiredStatus": "RUNNING",
            "enableExecuteCommand": false,
            "group": "service:voting-result",
            "healthStatus": "UNKNOWN",
            "lastStatus": "RUNNING",
            "launchType": "EC2",
            "memory": "1024",
            "overrides": {
                "containerOverrides": [
                    {
                        "name": "result"
                    },
                    {
                        "name": "ecs-service-connect-9qpWT",
                        "environment": [
                            {
                                "name": "ENVOY_CONCURRENCY",
                                "value": "2"
                            },
                            {
                                "name": "ENVOY_ADMIN_MODE",
                                "value": "UDS"
                            },
                            {
                                "name": "ENABLE_STATS_SNAPSHOT",
                                "value": "true"
                            },
                            {
                                "name": "APPMESH_METRIC_EXTENSION_VERSION",
                                "value": "1"
                            },
                            {
                                "name": "APPNET_ENVOY_RESTART_COUNT",
                                "value": "3"
                            },
                            {
                                "name": "HC_DISCONNECTED_TIMEOUT_S",
                                "value": "10800"
                            },
                            {
                                "name": "APPNET_AGENT_ADMIN_MODE",
                                "value": "UDS"
                            },
                            {
                                "name": "APPMESH_RESOURCE_ARN",
                                "value": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task-set/ecs-cluster/voting-result/ecs-svc/2777190142156290852"
                            }
                        ]
                    }
                ],
                "inferenceAcceleratorOverrides": []
            },
            "pullStartedAt": "2023-01-11T11:07:11.690000+11:00",
            "pullStoppedAt": "2023-01-11T11:07:19.151000+11:00",
            "startedAt": "2023-01-11T11:07:40.235000+11:00",
            "startedBy": "ecs-svc/2777190142156290852",
            "tags": [],
            "taskArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task/ecs-cluster/0e66e90e04694eba9c1b0593264a220b",
            "taskDefinitionArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task-definition/main-apps-nonprod-devtest-apse2-ecs-aws-fixed-application-voting-com-ResultTaskDef-8fRjBzpOir0x:1",
            "version": 5
        }
    ],
    "failures": []
}
```


Neither of these 2 containers have network interfaces attached in their containers. The **attachments** element details the 
'ServiceConnect' attachment type - which I guess handles network interfaces differently.

Now, I just don't know what the fix is as I'm not familiar with either the codebase, or GO. 
I suspect that a condition should determine if this is **ServiceConnect**, and use the Container Instance IP. 


# Disabling ECS Service Connect

If I disable Service Connect, you can see in the following task description that the 'Attachments' element is no has items - the ECS provider code therefor uses a different branch of code that does not look at the **networkInterfaces** element.
And Traefik correctly finds the service.

```json

{
    "tasks": [
        {
            "attachments": [],
            "attributes": [
                {
                    "name": "ecs.cpu-architecture",
                    "value": "x86_64"
                }
            ],
            "availabilityZone": "ap-southeast-2a",
            "clusterArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:cluster/ecs-cluster",
            "connectivity": "CONNECTED",
            "connectivityAt": "2023-01-11T12:26:27.016000+11:00",
            "containerInstanceArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:container-instance/ecs-cluster/63e1895b9c2d4d528932c6ab9cdb7a01",
            "containers": [
                {
                    "containerArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:container/ecs-cluster/6420359f07064d10aaab512e073adc09/dec1a481-519f-4b3b-8b3a-04165dec6ca0",
                    "taskArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task/ecs-cluster/6420359f07064d10aaab512e073adc09",
                    "name": "result",
                    "image": "nginx:latest",
                    "runtimeId": "cce09bb997da47e3945960a8f371bf36e66e51fb495e755d9ef91591c68a0e0c",
                    "lastStatus": "RUNNING",
                    "networkBindings": [
                        {
                            "bindIP": "0.0.0.0",
                            "containerPort": 80,
                            "hostPort": 49161,
                            "protocol": "tcp"
                        }
                    ],
                    "networkInterfaces": [],
                    "healthStatus": "UNKNOWN",
                    "cpu": "0"
                }
            ],
            "cpu": "512",
            "createdAt": "2023-01-11T12:26:27.016000+11:00",
            "desiredStatus": "RUNNING",
            "enableExecuteCommand": false,
            "group": "service:voting-result",
            "healthStatus": "UNKNOWN",
            "lastStatus": "RUNNING",
            "launchType": "EC2",
            "memory": "1024",
            "overrides": {
                "containerOverrides": [
                    {
                        "name": "result"
                    }
                ],
                "inferenceAcceleratorOverrides": []
            },
            "pullStartedAt": "2023-01-11T12:26:40.803000+11:00",
            "pullStoppedAt": "2023-01-11T12:26:42.673000+11:00",
            "startedAt": "2023-01-11T12:26:42.794000+11:00",
            "startedBy": "ecs-svc/6087098654864739701",
            "tags": [],
            "taskArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task/ecs-cluster/6420359f07064d10aaab512e073adc09",
            "taskDefinitionArn": "arn:aws:ecs:ap-southeast-2:REDACTEDACCID:task-definition/main-apps-nonprod-devtest-apse2-ecs-aws-fixed-application-voting-com-ResultTaskDef-8fRjBzpOir0x:1",
            "version": 2
        }
    ],
    "failures": []
}


```


Hopefully this might shed some light on a fix that needs to be made.






### What did you see instead?

Traffic Proxy logged the following;


```log
Error in Go routine: runtime error: index out of range [0] with length 0

Stack: goroutine 47 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:24 +0x65
github.com/traefik/traefik/v2/pkg/safe.defaultRecoverGoroutine({0x42db560?, 0xc000829d40})
	github.com/traefik/traefik/v2/pkg/safe/routine.go:66 +0xa5
github.com/traefik/traefik/v2/pkg/safe.OperationWithRecover.func1.1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:74 +0x35
panic({0x42db560, 0xc000829d40})
	runtime/panic.go:884 +0x212
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).listInstances(0xc000820630, {0x52a21f0, 0xc0009eb4d0}, 0xc000a7e5b8)
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:334 +0x1ae8
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).loadConfiguration(0xc000efbd20?, {0x52a21f0, 0xc0009eb4d0}, 0x0?, 0x445dee0?)
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:196 +0x31
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).Provide.func1.1()
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:172 +0x267
github.com/traefik/traefik/v2/pkg/safe.OperationWithRecover.func1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:78 +0x63
github.com/cenkalti/backoff/v4.RetryNotifyWithTimer(0xc000c99460, {0x52818f0, 0xc000c26000}, 0xc001129f20, {0x0?, 0x0?})
	github.com/cenkalti/backoff/v4@v4.1.3/retry.go:55 +0x12a
github.com/cenkalti/backoff/v4.RetryNotify(...)
	github.com/cenkalti/backoff/v4@v4.1.3/retry.go:34
github.com/traefik/traefik/v2/pkg/provider/ecs.(*Provider).Provide.func1({0x52a2148?, 0xc000a64040})
	github.com/traefik/traefik/v2/pkg/provider/ecs/ecs.go:186 +0x337
github.com/traefik/traefik/v2/pkg/safe.(*Pool).GoCtx.func1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:36 +0x66
github.com/traefik/traefik/v2/pkg/safe.GoWithRecover.func1()
	github.com/traefik/traefik/v2/pkg/safe/routine.go:59 +0x5b
created by github.com/traefik/traefik/v2/pkg/safe.GoWithRecover
	github.com/traefik/traefik/v2/pkg/safe/routine.go:53 +0x72
```



### What version of Traefik are you using?

Version:      2.9.5
Codename:     banon
Go version:   go1.19.3
Built:        2022-11-17T15:04:26Z
OS/Arch:      linux/amd64


### What is your environment & configuration?

```yaml

################################################################
# Global configuration
################################################################
global:
  checkNewVersion: true
  sendAnonymousUsage: false

################################################################
# EntryPoints configuration
################################################################
entryPoints:
  web:
    address: :80
  websecure:
    address: :443
  traefik:
    address: :8080

################################################################
# Traefik logs configuration
################################################################
log:
  level: DEBUG

accessLog:

api:
  insecure: true
  dashboard: true

################################################################
# Ping configuration
################################################################
ping:
  entryPoint: traefik

################################################################
# Docker configuration backend
################################################################
providers:
  ecs:
    autoDiscoverClusters: true
    exposedByDefault: false
    # constraints: "Label(`a.label.name`,`foo`)"
    refreshSeconds: 15
    region: ap-southeast-2


```


### If applicable, please paste the log output in DEBUG level

_No response_


Discussion / hints:

Fix patch:
diff --git a/pkg/provider/ecs/ecs.go b/pkg/provider/ecs/ecs.go
index 1958723dba..8def19a85e 100644
--- a/pkg/provider/ecs/ecs.go
+++ b/pkg/provider/ecs/ecs.go
@@ -315,6 +315,11 @@ func (p *Provider) listInstances(ctx context.Context, client *awsClient) ([]ecsI
 
 				var mach *machine
 				if len(task.Attachments) != 0 {
+					if len(container.NetworkInterfaces) == 0 {
+						logger.Errorf("Skip container %s: no network interfaces", aws.StringValue(container.Name))
+						continue
+					}
+
 					var ports []portMapping
 					for _, mapping := range containerDefinition.PortMappings {
 						if mapping != nil {
@@ -378,7 +383,7 @@ func (p *Provider) listInstances(ctx context.Context, client *awsClient) ([]ecsI
 
 				extraConf, err := p.getConfiguration(instance)
 				if err != nil {
-					log.FromContext(ctx).Errorf("Skip container %s: %w", getServiceName(instance), err)
+					logger.Errorf("Skip container %s: %w", getServiceName(instance), err)
 					continue
 				}
 				instance.ExtraConf = extraConf


Test patch: