[k8s] VPA (Vertical Pod Autoscaler) 설치 및 실습

VPA (Vertical Pod Autoscaler) 는 Pod의 리소스를 감시하고, pod 의 resource가 부족한 경우 pod를 Restart 하며 pod의 resource를 증가시킵니다. Kubernetes VPA는 pods에 대한 CPU 및 Memory 예약을 자동으로 조정하여 애플리케이션의 크기를 적절하게 조정할 수 있게 지원합니다. 클러스터의 리소스 사용률을 개선하고 다른 pod를 위한 CPU 및 Memory를 확보할 수 있습니다.

 

 

직접 k8s 클러스터에 VPA를 설치하고 테스트 하는것을 해당 글에서 다루어 봅니다.

 

1.Metric Server 설치

kubectl apply -f \
https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

2.에러 발생시 해결 방법

[에러 1]

( Readiness probe failed: HTTP probe fail )

kubectl edit deploy -n kube-system metrics-server

Deployment / metrics-server 에서 아래의 사진과 같이 --kubelet-insecure-tls args 를 추가해주면 해결가능하다.

 

 

[에러 2]

( Error from server (ServiceUnavailable): the server is currently unable to
 handle the request (get nodes.metrics.k8s.io)

kubectl edit deploy -n kube-system metrics-server

위의 명령어로 metric server의 deploy의 yaml 파일로 들어갑니다.
spec.template.spec  부분에 hostNetwork: true 를 추가해준다. (약 88번째 줄)

 

정상적으로 metric 서버를 설치하여 top 명령어를 사용할수 있게 됩니다.

 

3. Vertical Pod Autoscaler 설치

kevin@k8s-master:~$ helm repo add cowboysysop https://cowboysysop.github.io/charts/
"cowboysysop" has been added to your repositories

kevin@k8s-master:~$ helm -n kube-system upgrade -install vertical-pod-autoscaler cowboysysop/vertical-pod-autoscaler
Release "vertical-pod-autoscaler" does not exist. Installing it now.
NAME: vertical-pod-autoscaler
LAST DEPLOYED: Wed May  3 12:38:32 2023
NAMESPACE: stall
STATUS: deployed
REVISION: 1
NOTES:
** Please be patient while the chart is being deployed **

1. Get the application status by running the command:

$ helm status vertical-pod-autoscaler --namespace stall

정상적으로 ‘stall’ namespace에 vpa 가 deploy 된 것을 확인할 수 있습니다.

kevin@k8s-master:~$ kubectl get crd

 

4. Vertical Pod Autoscaler 테스트

evin@k8s-master:~$ kubectl create ns hamster
namespace/hamster created
kevin@k8s-master:~$ mkdir vpa-test
kevin@k8s-master:~$ cd vpa-test/
kevin@k8s-master:~/vpa-test$
kevin@k8s-master:~/vpa-test$ vi hamster.yaml

hamster namespace를 생성하고 hamster.yaml 파일을 생성합니다.

 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hamster
  namespace: hamster
spec:
  selector:
    matchLabels:
      app: hamster
  replicas: 2
  template:
    metadata:
      labels:
        app: hamster
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534 # nobody
      containers:
        - name: hamster
          image: registry.k8s.io/ubuntu-slim:0.1
          resources:
            requests:
              cpu: 100m
              memory: 50Mi
          command: ["/bin/sh"]
          args:
          - "-c"
          - "while true; do timeout 0.5s yes >/dev/null; sleep 0.5s; done"

cpu 100m과 memory 50 Mi 로 2개의 pod를 생성합니다.

kevin@k8s-master:~/vpa-test$ kubectl apply -f hamster.yaml
deployment.apps/hamster created

 

사용량을 kubectl top 명령어로 리소스 사용량을 확인합니다.

kevin@k8s-master:~/vpa-test$ kubectl top po -n hamster
NAME                       CPU(cores)   MEMORY(bytes)
hamster-65cd4dd797-8zf6w   495m         0Mi
hamster-65cd4dd797-z6ggn   502m         0Mi

5. VPA 추가

kevin@k8s-master:~/vpa-test$ vi hamster-vpa.yaml

hamster-vpa.yaml 파일을 만들어 다음 내용을 추가합니다.

apiVersion: "autoscaling.k8s.io/v1"
kind: VerticalPodAutoscaler
metadata:
name: hamster-vpa
namespace: hamster
spec:
# recommenders field can be unset when using the default recommender.
# When using an alternative recommender, the alternative recommender’s name
# can be specified as the following in a list.
# recommenders:
# — name: ‘alternative’
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: hamster
  resourcePolicy:
    containerPolicies:
    - containerName: '*'
      minAllowed:
        cpu: 100m
        memory: 50Mi
      maxAllowed:
        cpu: 1
        memory: 500Mi
      controlledResources: ["cpu", "memory"]

vpa를 생성합니다.

kevin@k8s-master:~/vpa-test$ kubectl apply -f hamster-vpa.yaml
verticalpodautoscaler.autoscaling.k8s.io/hamster-vpa created

생성이 잘 되었는지 확인합니다.

1~2분 안에 Recommender가 작동합니다.. 그 후 update 작업을 확인합니다.

kevin@k8s-master:~$ kubectl -n hamster  get pod hamster-65cd4dd797-cpqn7 -o yaml | grep -i cpu -A5

kevin@k8s-master:~$ kubectl -n hamster  get pod hamster-65cd4dd797-cpqn7 -o yaml | grep -i memory -A5

새로운 pod의 value를 check 합니다.

-w 옵션으로 vpa에 의해 동작하는 deploy의 모습을 관찰 할 수 있습니다.

kevin@k8s-master:~$ kubectl get po -n hamster -w
NAME                       READY   STATUS        RESTARTS   AGE
hamster-65cd4dd797-5625r   1/1     Terminating   0          97s
hamster-65cd4dd797-7f78c   1/1     Running       0          37s
hamster-65cd4dd797-jf84l   0/1     Pending       0          0s
hamster-65cd4dd797-jf84l   0/1     Pending       0          0s
hamster-65cd4dd797-jf84l   0/1     ContainerCreating   0          0s
hamster-65cd4dd797-5625r   1/1     Terminating         0          2m
hamster-65cd4dd797-jf84l   0/1     ContainerCreating   0          1s
hamster-65cd4dd797-5625r   0/1     Terminating         0          2m1s
hamster-65cd4dd797-5625r   0/1     Terminating         0          2m1s
hamster-65cd4dd797-5625r   0/1     Terminating         0          2m1s
hamster-65cd4dd797-jf84l   1/1     Running             0          2s

5. VPA 확인해보기

kevin@k8s-master:~/vpa-test$ kubectl describe -n hamster vpa hamster-vpa
Name:         hamster-vpa
Namespace:    hamster
Labels:       <none>
Annotations:  <none>
API Version:  autoscaling.k8s.io/v1
Kind:         VerticalPodAutoscaler
Metadata:
  Creation Timestamp:  2023-05-03T03:54:40Z
  Generation:          16
  Managed Fields:
    API Version:  autoscaling.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:resourcePolicy:
          .:
          f:containerPolicies:
        f:targetRef:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2023-05-03T03:54:10Z
    API Version:  autoscaling.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:recommendation:
          .:
          f:containerRecommendations:
    Manager:         recommender
    Operation:       Update
    Time:            2023-05-03T04:09:16Z
  Resource Version:  488592
  UID:               8c80d95e-4cd0-4d7f-9b07-1902a25ac9aa
Spec:
  Resource Policy:
    Container Policies:
      Container Name:  *
      Controlled Resources:
        cpu
        memory
      Max Allowed:
        Cpu:     1
        Memory:  500Mi
      Min Allowed:
        Cpu:     100m
        Memory:  50Mi
  Target Ref:
    API Version:  apps/v1
    Kind:         Deployment
    Name:         hamster
Status:
  Conditions:
    Last Transition Time:  2023-05-03T03:55:16Z
    Status:                True
    Type:                  RecommendationProvided
  Recommendation:
    Container Recommendations:
      Container Name:  hamster
      Lower Bound:
        Cpu:     507m
        Memory:  262144k
      Target:
        Cpu:     587m
        Memory:  262144k
      Uncapped Target:
        Cpu:     587m
        Memory:  262144k
      Upper Bound:
        Cpu:     1
        Memory:  500Mi
Events:          <none>