본문 바로가기
Kubernetes (k8s)

[k8s] migrate calico to cilium

by moveho 2023. 6. 12.

[설치 참조 문서]

https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/

https://cilium.io/blog/2020/10/06/skybet-cilium-migration/

https://kubernetes.io/ko/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy/

https://github.com/microservices-demo/microservices-demo

 

[Calico 삭제]

calico 관련 daemonset, deploy, po 등을 삭제합니다.

kevin@k8s-master:~$ kubectl get po,ds,deploy -n kube-system
NAME                                     READY   STATUS    RESTARTS       AGE
pod/coredns-787d4945fb-csdcq             1/1     Running   42 (11m ago)   156d
pod/coredns-787d4945fb-gvdxt             1/1     Running   42 (11m ago)   156d
pod/etcd-k8s-master                      1/1     Running   45 (11m ago)   156d
pod/kube-apiserver-k8s-master            1/1     Running   45 (11m ago)   156d
pod/kube-controller-manager-k8s-master   1/1     Running   53 (11m ago)   156d
pod/kube-proxy-cskgw                     1/1     Running   20 (11m ago)   156d
pod/kube-proxy-npp69                     1/1     Running   20 (11m ago)   156d
pod/kube-proxy-zv2k9                     1/1     Running   44 (11m ago)   156d
pod/kube-scheduler-k8s-master            1/1     Running   52 (11m ago)   156d
pod/metrics-server-b76787867-xmkqn       1/1     Running   7 (10m ago)    47d

NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/kube-proxy   3         3         3       3            3           kubernetes.io/os=linux   156d

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/coredns          2/2     2            2           156d
deployment.apps/metrics-server   1/1     1            1           55d

[Cilium 설치]

kevin@k8s-master:~/cilium$ curl -LO https://raw.githubusercontent.com/cilium/cilium/1.13.3/Documentation/installation/kind-config.yaml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    397      0 --:--:-- --:--:-- --:--:--   397

kevin@k8s-master:~/cilium$ ls
cilium.yaml  kind-config.yaml

kevin@k8s-master:~/cilium$ CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)

kevin@k8s-master:~/cilium$ CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}kevin@k8s-master:~/cilium$ CLI_ARCH=amd64
kevin@k8s-master:~/cilium$ if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
kevin@k8s-master:~/cilium$ curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 29.3M  100 29.3M    0     0  7701k      0  0:00:03  0:00:03 --:--:-- 8835k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    92  100    92    0     0    138      0 --:--:-- --:--:-- --:--:--     0
kevin@k8s-master:~/cilium$ sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
cilium-linux-amd64.tar.gz: OK
kevin@k8s-master:~/cilium$ sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
[sudo] password for kevin:
cilium

install 명령어를 사용합니다.

kevin@k8s-master:~/cilium$ cilium install
ℹ️  Using Cilium version 1.13.3
🔮 Auto-detected cluster name: kubernetes
🔮 Auto-detected datapath mode: tunnel
🔮 Auto-detected kube-proxy has been installed
ℹ️  helm template --namespace kube-system cilium cilium/cilium --version 1.13.3 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vxlan
ℹ️  Storing helm values file in kube-system/cilium-cli-helm-values Secret
🔑 Created CA in secret cilium-ca
🔑 Generating certificates for Hubble...
🚀 Creating Service accounts...
🚀 Creating Cluster roles...
🚀 Creating ConfigMap for Cilium version 1.13.3...
🚀 Creating Agent DaemonSet...
🚀 Creating Operator Deployment...
⌛ Waiting for Cilium to be installed and ready...

✅ Cilium was successfully installed! Run 'cilium status' to view installation health
kevin@k8s-master:~/cilium$ cilium status
    /¯¯\
 /¯¯\__/¯¯\    Cilium:             OK
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
 \__/¯¯\__/    Hubble Relay:       disabled
    \__/       ClusterMesh:        disabled

Deployment        cilium-operator    Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet         cilium             Desired: 3, Ready: 3/3, Available: 3/3
Containers:       cilium-operator    Running: 1
                  cilium             Running: 3

[Cilium 설치 확인]

kevin@k8s-master:~/cilium$ kubectl get po -n kube-system
NAME                                 READY   STATUS    RESTARTS       AGE
cilium-8rh46                         1/1     Running   0              96s
cilium-k76cq                         1/1     Running   0              96s
cilium-operator-58bf55d99b-6nmgv     1/1     Running   0              96s
cilium-wb8pd                         1/1     Running   0              96s
coredns-787d4945fb-h8twb             1/1     Running   0              33s
coredns-787d4945fb-qf8ct             1/1     Running   0              24s
etcd-k8s-master                      1/1     Running   45 (18m ago)   156d
kube-apiserver-k8s-master            1/1     Running   45 (18m ago)   156d
kube-controller-manager-k8s-master   1/1     Running   53 (18m ago)   156d
kube-proxy-cskgw                     1/1     Running   20 (18m ago)   156d
kube-proxy-npp69                     1/1     Running   20 (18m ago)   156d
kube-proxy-zv2k9                     1/1     Running   44 (18m ago)   156d
kube-scheduler-k8s-master            1/1     Running   52 (18m ago)   156d
metrics-server-b76787867-xmkqn       1/1     Running   7 (17m ago)    47d
kevin@k8s-master:~/cilium$ kubectl get po,ds -n kube-system
NAME                                     READY   STATUS    RESTARTS       AGE
pod/cilium-8rh46                         1/1     Running   0              107s
pod/cilium-k76cq                         1/1     Running   0              107s
pod/cilium-operator-58bf55d99b-6nmgv     1/1     Running   0              107s
pod/cilium-wb8pd                         1/1     Running   0              107s
pod/coredns-787d4945fb-h8twb             1/1     Running   0              44s
pod/coredns-787d4945fb-qf8ct             1/1     Running   0              35s
pod/etcd-k8s-master                      1/1     Running   45 (18m ago)   156d
pod/kube-apiserver-k8s-master            1/1     Running   45 (18m ago)   156d
pod/kube-controller-manager-k8s-master   1/1     Running   53 (18m ago)   156d
pod/kube-proxy-cskgw                     1/1     Running   20 (18m ago)   156d
pod/kube-proxy-npp69                     1/1     Running   20 (18m ago)   156d
pod/kube-proxy-zv2k9                     1/1     Running   44 (18m ago)   156d
pod/kube-scheduler-k8s-master            1/1     Running   52 (18m ago)   156d
pod/metrics-server-b76787867-xmkqn       1/1     Running   7 (17m ago)    47d

NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/cilium       3         3         3       3            3           kubernetes.io/os=linux   107s
daemonset.apps/kube-proxy   3         3         3       3            3           kubernetes.io/os=linux   156d

[테스트]

1.노드 Ready 상태 확인

kevin@k8s-master:~/cilium$ kubectl get no
NAME         STATUS   ROLES           AGE    VERSION
k8s-master   Ready    control-plane   156d   v1.26.0
k8s-node1    Ready    <none>          156d   v1.26.0
k8s-node2    Ready    <none>          156d   v1.26.0

2.테스트 pod 생성합니다.

kevin@k8s-master:~/cilium$ kubectl run nginx-pod --image nginx
pod/nginx-pod created
kevin@k8s-master:~/cilium$ kubectl get po
NAME        READY   STATUS    RESTARTS   AGE
nginx-pod   1/1     Running   0          4m54s

pod 정상 running 상태를 확인합니다.

 

3. k8s CNI 플러그인 확인

total 12
drwx------ 2 root root   90  6월 12 09:21 ./
drwx------ 3 root root   19  1월  6 10:28 ../
-rw-r--r-- 1 root root  145  6월 12 09:21 05-cilium.conf
-rw-r--r-- 1 root root  661  6월 12 09:03 10-calico.conflist.cilium_bak
-rw------- 1 root root 2712  6월 12 09:03 calico-kubeconfig
kevin@k8s-master:~$ ls /opt/cni/bin
bandwidth  calico       cilium-cni  firewall  host-device  install  loopback  portmap  sbr     tuning
bridge     calico-ipam  dhcp        flannel   host-local   ipvlan   macvlan   ptp      static

이로서 cilium CNI 사용하는 k8s cluster 구성되었습니다.

댓글