Change:
I enabled the on EKS 1.27.4 using Ubuntu 20.04.6. When I try to query the kubelet logs or log files from the system I get the error "input contains unsupported characters" The systemd kubelet service runs as a snap called and I think that might be the source of this problem (running the kubelet in a snap), but I'm not sure if it's intended to work. The fact that I couldn't see /var/log file query was surprising and the error message is misleading which is why I'm opening an issue here. I followed the same configuration steps on Amazon Linux 2 and it works as expected. Log query should work on Ubuntu or provide a clearer error message. This should be reproducible on any Kubernetes cluster running Ubuntu with Kubelet in a snap, but this is what I did. Install to make cluster creation and karpenter setup easier) Install Karpenter as a node autoscaler. This will automatically create a provisioner and AWSNodeTemplate objects. Modify the AWSNodeTemplate with userdata that can customize Ubuntu with log query enabled Deploy a workload that uses hostname spread to get more nodes. When the new Ubuntu node is created you can log into the node with session manager. I like to use You can verify that the kubelet is running and logging is enabled Now you can try to query logs This works with AL2. You can test with a different AWSNodeTemplate that uses AL2 instead <details</details<detailsEKS (but should happen on any provider) </details<details</details<details</details<detailscontainerd </details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
it's always best to talk to the product support before opening the upstream ticket (this repository in this case). the k8s docs do not have a guide for running the kubelet as a snap. it could be an entirely snap related bug, indeed. please communicate with EKS and the owners of the snap package. if they confirm a kubernetes/kubernetes bug, you can reopen this ticket. thanks
/sig node /triage needs-information /close to discuss with other users:
Closing this issue. <detailsIn response to : /sig node /triage needs-information /close </details
After digging into the error a bit more I found the problem was actually the regex match for the service name. I opened to fix it.

Discussion:
// The set of known safe characters to pass to journalctl / GetWinEvent flags - only add to this list if the // character cannot be used to create invalid sequences. This is intended as a broad defense against malformed // input that could cause an escape. <del> reServiceNameUnsafeCharacters = regexp.MustCompile(`[^a-zA-Z-_0-9@]+`) </del> <ins> reServiceNameUnsafeCharacters = regexp.MustCompile(`[^a-zA-Z-_.:0-9@]+`) </ins> ) // journalServer returns text output from the OS specific service logger to view

var ( service1 = "svc1" service2 = "svc2" <ins> service3 = "svc.foo" service4 = "svc_foo" service5 = "svc@foo" service6 = "svc:foo" </ins> invalid1 = "svcn" invalid2 = "svc.foon" )

}{ {name: "one service", services: []string{service1}}, {name: "two services", services: []string{service1, service2}}, <ins> {name: "dot service", services: []string{service3}}, {name: "underscore service", services: []string{service4}}, {name: "at service", services: []string{service5}}, {name: "colon service", services: []string{service6}}, </ins> {name: "invalid service new line", services: []string{invalid1}, wantErr: true}, {name: "invalid service with dot", services: []string{invalid2}, wantErr: true}, {name: "long service", services: []string{strings.Repeat(service1, 100)}, wantErr: true}, <del> {name: "max number of services", services: []string{service1, service2, service1, service2, service1}, wantErr: true}, </del> <ins> {name: "max number of services", services: []string{service1, service2, service3, service4, service5}, wantErr: true}, </ins> } for _, tt := range tests { errs := validateServices(tt.services)