By

欢迎转载，请支持原创，保留原文链接:blog.ilibrary.me

Mac上搭建

k3s不能直接跑在Mac上，在Mac上可以通过K3D搭建，把k3s跑在容器里。 具体步骤参考Kubernetes + Rancher Cluster Manager 2.6 on your macOS laptop with k3d/k3s in 5 min

步骤:

  1. brew install k3d kubectl helm
  2. Use k3d to spin up a single-node Kubernetes cluster (using the k3s distro) 
    k3d cluster create k3d-rancher \
 --api-port 6550 \
 --servers 1 \
 --image rancher/k3s:v1.20.10-k3s1 \
 --port 443:443@loadbalancer \
 --wait --verbose
  3. Use helm to bootstrap cert-manager then Rancher to the cluster 
    ### Install cert-manager with helm
helm repo add jetstack https://charts.jetstack.io
helm repo update
kubectl create namespace cert-manager
helm install cert-manager jetstack/cert-manager \
   --namespace cert-manager \
   --version v1.5.3 \
   --set installCRDs=true --wait --debug
kubectl -n cert-manager rollout status deploy/cert-manager
date
### Install the helm repos for rancher
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
helm repo update
kubectl create namespace cattle-system
helm install rancher rancher-latest/rancher \
   --namespace cattle-system \
   --version=2.6.1 \
   --set hostname=rancher.localhost \
   --set bootstrapPassword=congratsthanandayme \
   --wait --debug
kubectl -n cattle-system rollout status deploy/rancher
kubectl -n cattle-system get all,ing
date
  4. Use the Rancher GUI to observe the cluster. browse to https://rancher.localhost. If you see this screen where you can’t manually accept the risk, you may have to use the thisisunsafe chrome trick. You do this trick by clicking on the red warning triangle and literally typing “thisisunsafe” (believe me!)
    1. bootstrap password: congratsthanandayme
    2. bootstrap password还可以通过docker命令获取: docker logs container-id 2>&1 | grep "Bootstrap Password:"
    3. 或者通过helm获取: kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='\n'
  6. 可以通过.localhost来访问127.0.0.1
  7. brew install k3d kubectl helm
  8. k3d version
  9. k3d cluster create devcluster

在linux上搭建

  1. 搭建云服务, ubuntu系统.
  2. 安装docker: apt install docker.io
  3. 安装k3s master node: 
     curl -sfL https://get.k3s.io | sh -s - --docker # 自动安装， 并且以docker为容器运行时，默认为containerd.

 sudo k3s kubectl get nodes # confirm it is running
 cat /var/lib/rancher/k3s/server/node-token # 获取master node token

    K3s 包含并默认为containerd， 一个行业标准的容器运行时。要使用 Docker 而不是 containerd,可以用下面两种方法:

    1. 使用--docker选项安装K3s: curl -sfL https://get.k3s.io | sh -s - --docker
    2. 在 K3s 节点上安装 Docker。可以使用 Rancher 的一个Docker 安装脚本来安装 Docker：curl https://releases.rancher.com/install-docker/19.03.sh | sh
    3. 参考使用 Docker 作为容器运行时. 卸载k3s: /usr/local/bin/k3s-uninstall.sh
  4. 查看containers: ` k3s ctr task ls`
  5. 安装K3S worker nodes: 需要用到上一步在master node获取到的master node token. 
    curl -sfL http://get.k3s.io | K3S_URL=https://<master_IP>:6443 K3S_TOKEN=<join_token> sh -s - --docker # 用docker作为运行时。默认是 containerd作为运行时.
  6. install helm: 
    curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh

    Notes: helm包下载很慢，容易断，如果脚本安装不了，就本地下载helm tar.gz, 然后scp拷贝到服务器上，然后解压安装，官方有文档:

    tar -zxvf helm-v3.0.0-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/helm
  7. 更新helm repo 
       helm repo add stable https://charts.helm.sh/stable
   helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
   helm repo update
  8. (optional)安装 cert-manager, kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml
    1. kubectl create namespace cattle-system && kubectl create namespace cert-manager
  9. 通过helm安装rancher:
   # 注意rancher.localhost, localhost后缀是可以自动解析到本地127.0.0.1的
   helm install rancher rancher-stable/rancher \
   --namespace cattle-system \
   --set hostname=rancher.localhost \
   --set bootstrapPassword=verySecretivePasswordIndeed \
   --set ingress.tls.source=secret

或者

   helm install rancher rancher-latest/rancher --namespace cattle-system --set hostname=k3s-rancher.localhost
   kubectl -n cattle-system rollout status deploy/rancher
   kubectl -n cattle-system get deploy rancher
  1. 获取setup link: echo https://rancher.mydomain.com/dashboard/?setup=$(kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='')
  2. helm版本rancher获取rancher登录密码: k3s kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='\n'
  3. rancher.ilibrary.me账号密码: admin/BtfEQI1E7yajt9lx

在Rancher里创建cluster

参考multiple k8s cluster management with rancher k3s lightweight k8s cluster for edge

  1. 按照说明里做。
  2. 做了以后可能要等1个多小时cluster才会从reconciling变成active.
  3. 变成active以后可以按页面的说明部署一个hello world玩玩。

Notes

  1. Error: INSTALLATION FAILED: repo rancher-stable not found
    1. helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
  2. Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "http://localhost:8080/version": dial tcp [::1]:8080: connect: connection refused
    1. export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
  3. Error: INSTALLATION FAILED: chart requires kubeVersion: < 1.25.0-0 which is incompatible with Kubernetes v1.25.6+k3s1
    1. 安装指定版本: curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.23.16+k3s1 sh -
    2. 如果curl命令超时，可以手动下载k3s binary, 放/usr/local/bin目录下, 然后再运行上面的curl命令，脚本会跳过k3s下载步骤，走后面的配置流程。
  4. rainbow部署以后通过ingress发布，https://ranxxx.ixxx.me/rainbow不工作。
    1. chart里面的host写的不对，改成目标host就好了。
  5. Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: resource mapping not found for name: "rancher" namespace: "" from "": no matches for kind "Issuer" in version "cert-manager.io/v1" ensure CRDs are installed first

手动打包部署

  1. 创建chart: helm create mychart
  2. 打包: helm package mychart 或者指定输出的版本helm package mychart --version=1.0.1
  3. 上传mychart-1.0.1.tgz到服务器: scp mychart-1.0.1.tgz root@xxxx.com:~/
  4. ssh到服务器, 安装chart: 
    helm install mychart ./mychart-1.0.1.tgz
# or
helm install -n my-app --namespace myspace mychart-1.0.1.tgz
# or
tar -zxvf mychart-1.0.1.tgz
helm install -n my-app --namespace myspace mychart/
  5. 如果有在线tgz包，也可以在命令行直接安装在线包: helm install jenkins https://example.com/charts/jenkins-1.2.3.tgz

自动打包部署

参考Rancher官方CI/CD流水线文档

  1. 进入Rancher CD页面
  2. 点击git repos -> Create,
  3. 填名称
  4. 填git repo地址: git@github.com:xxxxx/myk8s.git, xxxxx/myk8s代表个人github repo.
    1. Rancher有个官方example: https://github.com/rancher/pipeline-example-go.git
  5. 手动生成ssh key pair, 把private/pub key填入rancher
  6. 把pub key设置到repo的deploy keys里面去。

Rancher管理

删除Rancher

  1. 列出所有安装好的App: helm list -A
  2. 删除指定namespace下面的指定App: helm uninstall rancher -n cattle-system

删除k3d

  1. k3d是一个binary, 它会管理所有k3s相关的容器.
  2. which k3d, 查看k3d的路径.
  3. k3d cluster delete -a, 删除所有k3d创建的资源.

scripts

k3d

  1. k3d cluster list
  2. k3d cluster delete -a
  3. k3d cluster create devcluster
  4. k3d cluster create devhacluster –servers 3 –agents 1
  5. k3d cluster delete devhacluster

k3s

  1. kubectl get nodes

Helm

  1. helm repo add jetstack https://charts.jetstack.io
  2. helm repo update
  4. 安装的时候监控日志直到安装完成: 
    helm install cert-manager jetstack/cert-manager \
 --namespace cert-manager \
 --version v1.5.3 \
 --set installCRDs=true --wait --debug

kubectl

  1. kubectl create namespace cert-manager
  2. kubectl -n cert-manager rollout status deploy/cert-manager
  3. kubectl cluster-info
  4. kubectl get po -o wide -A

Ref

  1. https://computingforgeeks.com/install-kubernetes-on-ubuntu-using-k3s/
  2. https://selectfrom.dev/multiple-k8s-cluster-management-with-rancher-k3s-lightweight-k8s-cluster-for-edge-and-eea1f71175d0
  3. K8s - Kubernetes重要概念介绍（Cluster、Master、Node、Pod、Controller、Service、Namespace）
  4. Kubernetes指南
  5. 单机部署k3s, 提供了一个sh, k3s-helm-install.sh
  6. Autok3s, Rancher 中国团队推出了一款针对 K3s 的效率提升工具：AutoK3s。只需要输入一行命令，即可快速创建 K3s 集群并添加指定数量的 master 节点和 worker 节点.
  7. How to install rancher on k3s
  8. 【大强哥-k8s从入门到放弃12】Deployment资源详解, 对K8S基本概念讲得很清楚。
  9. Kubernetes + Rancher Cluster Manager 2.6 on your macOS laptop with k3d/k3s in 5 min
  10. Provision K3S cluster on mac using K3D
  11. 打包Helm Chart
  12. Rancher CI/CD 官方流水线