google云部署 k3s 集群

一、前言
二、基础环境部署
- 1）前期准备
- 2）部署k3s

一、前言

K3s 是轻量级的 Kubernetes。K3s 易于安装，仅需要 Kubernetes 内存的一半，所有组件都在一个小于 100 MB 的二进制文件中。

它适用于：

Edge
IoT
CI
Development
ARM
嵌入 K8s
不去学习那么多的 Kubernetes 高深知识也能上手使用

官方文档：https://docs.k3s.io/zh/quick-start

k3s version: 1.27.6

helm version: 3.14.2

cert-manager: 1.11.1

操作系统：centos-8

二、基础环境部署

1）前期准备

1、更新系统

yum update

2、修改主机名,在所有节点进行对应设置

hostnamectl set-hostname k3s-master

hostnamectl set-hostname k3s-node1

hostnamectl set-hostname k3s-node2

3、开启 IP 地址转发

echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

echo "net.ipv4.conf.all.proxy_arp = 1" >> /etc/sysctl.conf

sysctl -p /etc/sysctl.conf

2）、部署k3s

1、server节点

## 安装k3s

##国内
curl –sfL      https://rancher-mirror.oss-cn-beijing.aliyuncs.com/k3s/k3s-install.sh |      INSTALL_K3S_MIRROR=cn sh -s -      --system-default-registry "registry.cn-hangzhou.aliyuncs.com"      --write-kubeconfig ~/.kube/config      --write-kubeconfig-mode 600

##国外
curl -sfL https://get.k3s.io |    sh -s -       --write-kubeconfig ~/.kube/config      --write-kubeconfig-mode 600

将/etc/rancher/k3s/k3s.yaml中ip改为server节点public_ip,单节点不用修改

##部署cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.crds.yaml
 
 
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.14.4    --set "ingressShim.defaultIssuerName=letsencrypt-prod,ingressShim.defaultIssuerKind=ClusterIssuer"

部署issuer

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: zszxingchenid@gmail.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik

kubectl apply -f production-issuer.yaml

##获取 K3S_TOKEN
 cat /var/lib/rancher/k3s/server/node-token

2、agent节点

curl -sfL https://get.k3s.io | K3S_URL= ${K3S_URL} K3S_TOKEN=$ {K3S_TOKEN} sh -

curl -sfL https://get.k3s.io | K3S_URL=https://34.92.*.*:6443  K3S_TOKEN=K10161398e6e303dc0880e9085b187c9ca05e4f3418a89be7d5328874c31202fd04::server:8ef5ef25f273051d5a730118161f2801 sh -

curl -sfL https://get.k3s.io | K3S_URL= ${K3S_URL} K3S_TOKEN=$ {K3S_TOKEN} sh -

curl -sfL https://get.k3s.io | K3S_URL=https://34.92.*.*:6443  K3S_TOKEN=K10161398e6e303dc0880e9085b187c9ca05e4f3418a89be7d5328874c31202fd04::server:8ef5ef25f273051d5a730118161f2801 sh -

##免密传输
1、server节点生成密钥

ssh-keygen -t rsa -b 4096

2、将公钥复制到agent的.ssh/authorized_keys文件中,过程中需要输入agent节点登陆密码

ssh-copy-id root@<agent-ip>

3、复制~/.kube/config文件到agent
scp -r ~/.kube/config root@<agent-ip>:~/.kube

### 3）、验证集群可用性
```sh
## server 节点执行
helm install nginx-test ./nginx-test

kubectl get certificate

4）、部署gitlab-runner

1、安装gitlab-runner

1、Simply download one of the binaries for your system:

# Linux x86-64
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"

# Linux x86
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-386"

# Linux arm
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-arm"

# Linux arm64
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-arm64"

# Linux s390x
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-s390x"

# Linux ppc64le
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-ppc64le"

# Linux x86-64 FIPS Compliant
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64-fips"

2、Give it permissions to execute:

sudo chmod +x /usr/local/bin/gitlab-runner

3、Create a GitLab CI user:

sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

4、Install and run as service:

sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start

5、add gitlab-runner to the sudoers file

sudo visudo
gitlab-runner ALL=(ALL) NOPASSWD: ALL

6、检测gitlab-runner通过helm部署项目时连通性

Error: Kubernetes cluster unreachable: Get "https://ip:6443/version": tls: failed to verify certificate: x509: certificate signed by unknown authority

sudo groupadd k3s
sudo usermod -a -G k3s gitlab-runner
sudo chown gitlab-runner:k3s /etc/rancher/k3s/k3s.yaml

sudo usermod -aG root gitlab-runner

sudo chmod 600 /etc/rancher/k3s/k3s.yaml
sudo gitlab-runner restart

2、docker镜像权限授权

vi ~/.docker/config.json
{
    "auths": {
        "registry.cn-shenzhen.aliyuncs.com": {
            "username": "zszxcken",
            "password": "zszxc123@",
            "auth": "enN6eGNrZW46enN6eGMxMjNA"
        },
	 "registry.cn-hangzhou.aliyuncs.com": {
            "username": "zszweb",
            "password": "zsz127428",
            "auth": "enN6d2ViOnpzejEyNzQyOA=="
        }
    }
}

sudo systemctl restart docker

sudo usermod -a -G docker gitlab-runner

5）、补充

1、为主节点添加污点（Taint），此情况主要用于主节点内存不够的情况下

kubectl taint nodes <server-node-name> key=value:NoSchedule

在这里，是你的主节点的名称，key 和 value 是你自定义的污点键和值。例如：

kubectl taint nodes k3s-master  role=master:NoSchedule

2、为主节点解除污点

kubectl taint nodes --all node-role.kubernetes.io/(角色)-

3、安装cert-manager常见报错

1、“Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "https://127.0.0.1:6443/version": tls: failed to verify certificate: x509: certificate signed by unknown authority

2、"Error: Kubernetes cluster unreachable: Get "https://127.0.0.1:6443/version": dial tcp 127.0.0.1:6443: connect: no route to host"

vi ~/.zshrc

添加 export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

source ~/.zshrc
chmod 600 /etc/rancher/k3s/k3s.yaml

google云部署 k3s 集群

一、前言

二、基础环境部署

1）前期准备

1、更新系统

2、修改主机名,在所有节点进行对应设置

3、开启 IP 地址转发

2）、部署k3s

1、server节点

2、agent节点

curl -sfL https://get.k3s.io | K3S_URL={K3S_URL} K3S_TOKEN={K3S_TOKEN} sh -

curl -sfL https://get.k3s.io | K3S_URL={K3S_URL} K3S_TOKEN={K3S_TOKEN} sh -

4）、部署gitlab-runner

1、安装gitlab-runner

2、docker镜像权限授权

5）、补充

1、为主节点添加污点（Taint），此情况主要用于主节点内存不够的情况下

2、为主节点解除污点

3、安装cert-manager常见报错

1、“Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "https://127.0.0.1:6443/version": tls: failed to verify certificate: x509: certificate signed by unknown authority

2、"Error: Kubernetes cluster unreachable: Get "https://127.0.0.1:6443/version": dial tcp 127.0.0.1:6443: connect: no route to host"

curl -sfL https://get.k3s.io | K3S_URL= ${K3S_URL} K3S_TOKEN=$ {K3S_TOKEN} sh -

curl -sfL https://get.k3s.io | K3S_URL= ${K3S_URL} K3S_TOKEN=$ {K3S_TOKEN} sh -