opened
arthurlogilab
username_0
<!--- Provide a general summary of the issue in the Title above -->

Maybe related to https://github.com/openfaas/faasd/issues/87  but since that issue is more about memory requests and limits, I'll open a new one

## Current Behaviour

On our kubernetes cluster where openfaas is used as well as other deployments, we've had the following problem : some nodes had a very high load, while other were very low (very visible when sending a bunch of CI jobs using gitlab-runner). Investigating we found that some pods had a cpu request of 1, essentially "reserving" a whole CPU for themselves. Some test functions such as `certinfo` where in this case. 

## Expected Behaviour

I'm not sure, but I would say that having no cpu.requests would be a good default. But not an expert, maybe a low value (50m) to at least avoid overstacking them ? 

## Possible Solution

Being able to set the value in a deployment of openfaas ? Having a default value that is a bit saner than `1` ? Searching through the documentation couldn't find mention of this, so if it can be configured maybe we can improve the documentation ?  

## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
1. deploy openfaas on kubernetes
2.  deploy some functions
3. extract cpu.requests : 

```
kubectl get pod --all-namespaces -o="custom-columns=NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeName,CPU:.spec.containers[*].resources.requests.cpu" | grep -v '<none>'
[snip]
certinfo-77d58489ff-2xqwc                    Running     node1                    1
[snip] 
```


## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->

## Your Environment

* OS and architecture: kubernetes v1.18.9
* openfaas : openfaas/faas-netes:0.9.13 openfaas/gateway:0.18.3
functions deployed with openfaas on kubernets have cpu.request set to 1 
issue
created
alexellis
username_1
Since we don't know each other yet ( welcome :) ), and you're new to the community, could you say who "we" is?

Under `Steps to Reproduce (for bugs)` - please provide the exact instructions used to reproduce the issue that you are facing.

I'd also encourage you to get the Pod and Deployment spec and to share that with us, so that we can verify whether OpenFaaS is setting this value. I doubt that an entire CPU is being requested on all functions, otherwise we would have ben made aware of this earlier, however I'll give you the benefit of the doubt, if you can provide the evidence in the issue, we can also look into it.

I would also encourage you to tell us about anything else you have in the cluster like VPA, Goldilocks and any operators that may be altering deployments. Which cloud / K8s service is this?
comment
created
LucasRoesler
username_2
Just to followup on this, we definitely only set the CPU request _if_ a value is provided in the function spec

https://github.com/openfaas/faas-netes/blob/fe8e309696ce16ad58ce037d41b14adc4e650492/pkg/handlers/deploy.go#L379

we otherwise leave the CPU requests empty
comment
created
alexellis
username_1
```bash
k3d cluster create
arkade install openfaas

faas-cli new --lang python3 py-test --prefix username_12
faas-cli up -f py-test.yml 
```

Then:

```
kubectl get pod -l faas_function=py-test -n openfaas-fn -o yaml --export
 
 spec:
    containers:
    - env:
      - name: fprocess
        value: python3 index.py
      image: username_12/py-test:latest
      imagePullPolicy: Always
      livenessProbe:
        failureThreshold: 3
        httpGet:
          path: /_/health
          port: 8080
          scheme: HTTP
        initialDelaySeconds: 2
        periodSeconds: 2
        successThreshold: 1
        timeoutSeconds: 1
      name: py-test
      ports:
      - containerPort: 8080
        name: http
        protocol: TCP
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /_/health
          port: 8080
          scheme: HTTP
        initialDelaySeconds: 2
        periodSeconds: 2
        successThreshold: 1
        timeoutSeconds: 1
      resources: {}
      securityContext:
        readOnlyRootFilesystem: false
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
      - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        name: default-token-lkl5j
        readOnly: true
    dnsPolicy: ClusterFirst
    enableServiceLinks: false
    nodeName: k3d-k3s-default-server-0
    priority: 0
    restartPolicy: Always
    schedulerName: default-scheduler
    securityContext: {}
    serviceAccount: default
    serviceAccountName: default
    terminationGracePeriodSeconds: 30
    tolerations:
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
    volumes:
    - name: default-token-lkl5j
      secret:
        defaultMode: 420
        secretName: default-token-lkl5j

```

I can't see any CPU requests in the pod spec at all.

Here is a Store function:

```
  spec:
    containers:
    - image: functions/nodeinfo-http:latest
      imagePullPolicy: Always
      livenessProbe:
        failureThreshold: 3
        httpGet:
          path: /_/health
          port: 8080
          scheme: HTTP
        initialDelaySeconds: 2
        periodSeconds: 2
        successThreshold: 1
        timeoutSeconds: 1
      name: nodeinfo
      ports:
      - containerPort: 8080
        name: http
        protocol: TCP
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /_/health
          port: 8080
          scheme: HTTP
        initialDelaySeconds: 2
        periodSeconds: 2
        successThreshold: 1
        timeoutSeconds: 1
      resources: {}
      securityContext:
        readOnlyRootFilesystem: false
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
      - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        name: default-token-lkl5j
        readOnly: true
    dnsPolicy: ClusterFirst
    enableServiceLinks: false
    nodeName: k3d-k3s-default-server-0
    priority: 0
    restartPolicy: Always
    schedulerName: default-scheduler
    securityContext: {}
    serviceAccount: default
    serviceAccountName: default
    terminationGracePeriodSeconds: 30
    tolerations:
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
    volumes:
    - name: default-token-lkl5j
      secret:
        defaultMode: 420
        secretName: default-token-lkl5j
```

I would ask your administrator to investigate the cluster to see if they have something running that is overriding or mutating the requests on objects.
comment
closed
alexellis
username_1
issue
created
alexellis
username_1
Closing as technical support, which we can help you with, but via Slack or [support](https://www.openfaas.com/support/)

Feel free to comment again if you have the info we asked for in the thread.
comment
created
arthurlogilab
username_0
Hey @username_1, first off thanks for taking the time to take a look at this issue. I was suprised too, but hopefully investigating this we can understand it (and make sure it's not openfaas doing this...)

The 'we' is Logilab https://www.logilab.fr/ We operate two kubernetes cluster in-house, and use  third on managed by OVHCloud. In this use case, we've seen this on our internal kubernetes cluster (debian based, deployed using kubeadm). We don't have any "VPA, Goldilocks" operating there. We have a rancher that "looking" at our kubernetes, but as far as I know it doesn't modify deployments and pods. 

Here is the pod's yaml : https://gist.github.com/username_0/2c675f3c0d2b1ab012f5db9ef588d64f the request cpu is here : https://gist.github.com/username_0/2c675f3c0d2b1ab012f5db9ef588d64f#file-pod-yaml-L164-L170

But copying this I find mention of the LimitRanger (just read the documentation https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/admission_control_limit_range.md) which is probably the problem here. It seems to have been set on the openfaas-fn name space with bad values. I'm investigating with the other admins of the kubernetes cluster to see if they understand where this came from...
comment