一、kubekey方式文章参考链接GitHubhttps://github.com/kubesphere/kubekeykubespherehttps://get-images.kubesphere.io微信公众号https://mp.weixin.qq.com/s/HacD-QPS-7TzlQsDlKnj8Q云原生测试补给站下载kubekeycurl-sfLhttps://get-kk.kubesphere.io|VERSIONvx.xx.xxsh-# x.xx.xx为版本号由于下载网站再国外国内网络下载可能会很慢或者下载不成功可以采用下面方式下载① 官方方案个人感觉不好用exportKKZONEcn# 设置变量curl-sfLhttps://get-kk.kubesphere.io|VERSIONvx.xx.xxsh-# x.xx.xx为版本号② 个人推荐方案手动下载后上传到服务器kubekey下载地址https://github.com/kubesphere/kubekey/releasestarzxvf kubekey-vx.xx.xx-*.tar.gz# 解压软件包./kk version# 查看版本生成配置信息kubekey版本低于4.0./kk version --show-supported-k8s./kk create config --with-kubernetes vx.xx.xx# x.xx.xx替换为你需要安装的k8s版本生成 config-sample.yaml 文件文件内容大致如下(重点修改标颜色部分内容name、ip地址、用户名和密码)apiVersion:kubekey.kubesphere.io/v1alpha2kind:Clustermetadata:name:samplespec:hosts:-{name:Master01,address:192.168.0.6,internalAddress:192.168.0.6,user:root,password:QAZ2wsx}-{name:node01,address:192.168.0.144,internalAddress:192.168.0.144,user:root,password:QAZ2wsx}-{name:node02,address:192.168.0.121,internalAddress:192.168.0.121,user:root,password:QAZ2wsx}roleGroups:etcd:-Master01control-plane:-Master01worker:-node01-node02controlPlaneEndpoint:## Internal loadbalancer for apiservers# internalLoadbalancer: haproxydomain:lb.kubesphere.localaddress:port:6443kubernetes:version:v1.33.4clusterName:cluster.localautoRenewCerts:truecontainerManager:containerdetcd:type:kubekeynetwork:plugin:calicokubePodsCIDR:10.233.64.0/18kubeServiceCIDR:10.233.0.0/18## multus support. https://github.com/k8snetworkplumbingwg/multus-cnimultusCNI:enabled:falseregistry:privateRegistry:namespaceOverride:registryMirrors:[]insecureRegistries:[]addons:[]kubekey版本高于4.0./kk create inventory-o.会生成 inventory.yaml 文件内容如下(重点修改标颜色部分内容)apiVersion:kubekey.kubesphere.io/v1kind:Inventorymetadata:name:defaultspec:hosts:# your can set all nodes here. or set nodes on special groups.master01:# 主机名connector:tyep:sshhost:192.168.0.6port:22user:rootpassword:QAZ2wsxinternal_ipv4:192.168.0.6node01:# 主机名connector:type:sshhost:192.168.0.144port:22user:rootpassword:QAZ2wsxinternal_ipv4:192.168.0.144node02:# 主机名connector:type:sshhost:192.168.0.121port:22user:rootpassword:QAZ2wsxinternal_ipv4:192.168.0.121groups:# all kubernetes nodes.k8s_cluster:groups:-kube_control_plane-kube_worker# control_plane nodeskube_control_plane:hosts:-master01# worker nodeskube_worker:hosts:-master01-node01-node02# etcd nodes when etcd_deployment_type is externaletcd:hosts:-master01# image_registry:# hosts:# - localhost# nfs nodes for registry storage. and kubernetes nfs storage# nfs:# hosts:# - localhost./kk create config --with-kubernetesKubernetes version-o.会生成 config-.yaml 文件内容如下:apiVersion:kubekey.kubesphere.io/v1kind:Configspec:# If set to cn, online downloads will prioritize domestic sources when available.zone:cn## 此处很重要如果填会导致你的镜像下载不成功kubernetes:# Specify the Kubernetes version to be installed.kube_version:v1.34.3# Specify the Helm version to be installed.helm_version:v3.18.5# Tag for the sandbox (pause) image used by pods.sandbox_image:tag:3.10.1control_plane_endpoint:# Supported HA types: local, kube-vip, haproxy.# If set to local, configure local hostname resolution as follows:# - Control-plane nodes: 127.0.0.1 .kubernetes.control_plane_endpoint.host# - Worker nodes: .init_kubernetes_node .kubernetes.control_plane_endpoint.hosttype:local# Kube-vip image tag used for static pod deployment (when type is kube-vip).kube_vip:image:tag:v0.7.2# HAProxy image tag used for static pod deployment (when type is haproxy).haproxy:image:tag:2.9.6-alpineetcd:# Specify the etcd version to be installed.etcd_version:v3.6.5image_registry:# Image registry type to install. Supported: harbor, docker-registry.# Leave empty to skip installation (assuming an existing registry is available).type:# auth:# # Address of the private image registry. Format: host:port/project# # The port (if present) is used as http_port when plain_httptrue,# # or as https_port when plain_httpfalse.# registry: dockerhub.kubekey.local# # Use plain HTTP (no TLS). Default is false.# plain_http: false# username: admin# password: Harbor12345# Specify a VIP for image registry high availability. If set, enables HA.ha_vip:# Image Registry - High Availability # keepalived image tag for load balancing when multiple registry nodes exist.# keepalived_version: 2.3.4# Image Registry - Harbor # Image registry port settings. Usually you do not need to set them manually;# they are derived from image_registry.auth.registry and plain_http.# http_port: 80# https_port: 443# Harbor image tag (only valid if type is harbor).#harbor_version: v2.10.2# docker-compose binary#dockercompose_version: v2.20.3# Image Registry - Docker Registry # Docker Registry image tag (only valid if type is docker-registry).# docker_registry_version: 2.8.3cri:# Container runtime type. Supported: containerd, docker.container_manager:containerd# CRI Tool # crictl binary version.crictl_version:v1.34.0# Docker Runtime # Docker binary version.#docker_version: 25.0.5# cridockerd version (required for Kubernetes 1.24).# cridockerd_version: v0.3.21# Containerd Runtime # containerd binary version (active only when container_manager is containerd).# containerd_version: v1.7.13# runc binary version (active only when container_manager is containerd).# runc_version: v1.1.12cni:# CNI plugin type. Supported: calico, cilium, flannel, hybridnet, kubeovn, othertype:calico# Multi-CNI # Multi-CNI type. Supported: multus, spiderpool, none.multi_cni:none# Multi-CNI - Multus # Image tag for Multus (configure if needed).# multus:# image:# tag: v4.2.4# Multi-CNI - Spiderpool # Spiderpool version (configure if needed).# spiderpool_version: v1.1.1# Calico CNI # Calico version (effective only when type is calico).calico_version:v3.31.3# Cilium CNI # Cilium version (effective only when type is cilium).# cilium_version: 1.19.1# Flannel CNI # Flannel version (effective only when type is flannel).#flannel_version: v0.27.4# Kube-OVN CNI # Kube-OVN version (effective only when type is kubeovn).#kubeovn_version: v1.15.0# Hybridnet CNI # Hybridnet version (effective only when type is hybridnet).#hybridnet_version: 0.6.8storage_class:# Storage Class Configuration # Local/OpenEBS Storage Class # Local storage class settings.local:enabled:true# Enable the local storage class.default:true# Set as the default storage class.# openebs/dynamic-localpv-provisioner helm chart version.localpv_provisioner_version:4.4.0# NFS Storage Class nfs:enabled:false# Enable the NFS storage class.# nfs-provisioner Helm chart version.#nfs_provisioner_version: 4.0.18dns:# CoreDNS settings.coredns:image:tag:v1.12.1# NodeLocalDNS settings.nodelocaldns:enabled:trueimage:tag:1.26.4# the external images will to add in packages初始化OS依赖kubekey版本低于4.0./kk init os-fconfig-sample.yamlkubekey版本高于4.0./kk init os-iinventory.yaml-cconfig-Kubernetes version.yaml创建集群kubekey版本低于4.0./kk create cluster-fconfig-sample.yamlkubekey版本高于4.0./kk create cluster-iinventory.yaml-cconfig-Kubernetes version.yaml验证集群kubectl get nodes -o wide kubectl get pods _Anode节点状态均为Ready为正常pod状态均为Running为正常二、kubespray方式文章参考链接roc 云原生https://imroc.cc/kubernetes/deploy/kubespray/install准备工作下载kubespraygitclone--depth1https://github.com/kubernetes-sigs/kubespray.git正常国内采用上面命令下载是不成功的建议去github下载后上传到服务器下载地址https://github.com/kubernetes-sigs/kubespray/releases# zip包解压unzipkubespray-kubespray version.zip# tar包解压tra zxvf kubespray-kubespray version.zip安装依赖cdkubespray-kubespray versionpip3install-rrequirements.txt这里很容易出现下面的这种报错这种报错原因是你的操作系统版本太低kubespray版本太高的问题换个高版本操作系统或者低版本kubespray就可以了复制配置文件cp-rfpinventory/sample inventory/mycluster配置文件结构如下cdinventory tree.├── local│ ├── group_vars - …/sample/group_vars│ └── hosts.ini├── mycluster│ ├── group_vars│ │ ├── all│ │ │ ├── all.yml│ │ │ ├── aws.yml│ │ │ ├── azure.yml│ │ │ ├── containerd.yml│ │ │ ├── coreos.yml│ │ │ ├── cri-o.yml│ │ │ ├── docker.yml│ │ │ ├── etcd.yml│ │ │ ├── gcp.yml│ │ │ ├── hcloud.yml│ │ │ ├── huaweicloud.yml│ │ │ ├── oci.yml│ │ │ ├── offline.yml│ │ │ ├── openstack.yml│ │ │ ├── upcloud.yml│ │ │ └── vsphere.yml│ │ └── k8s_cluster│ │ ├── addons.yml│ │ ├── k8s-cluster.yml│ │ ├── k8s-net-calico.yml│ │ ├── k8s-net-cilium.yml│ │ ├── k8s-net-custom-cni.yml│ │ ├── k8s-net-flannel.yml│ │ ├── k8s-net-kube-ovn.yml│ │ ├── k8s-net-kube-router.yml│ │ ├── k8s-net-macvlan.yml│ │ └── kube_control_plane.yml│ └── inventory.ini└── sample├── group_vars │ ├── all │ │ ├── all.yml │ │ ├── aws.yml │ │ ├── azure.yml │ │ ├── containerd.yml │ │ ├── coreos.yml │ │ ├── cri-o.yml │ │ ├── docker.yml │ │ ├── etcd.yml │ │ ├── gcp.yml │ │ ├── hcloud.yml │ │ ├── huaweicloud.yml │ │ ├── oci.yml │ │ ├── offline.yml │ │ ├── openstack.yml │ │ ├── upcloud.yml │ │ └── vsphere.yml │ └── k8s_cluster │ ├── addons.yml │ ├── k8s-cluster.yml │ ├── k8s-net-calico.yml │ ├── k8s-net-cilium.yml │ ├── k8s-net-custom-cni.yml │ ├── k8s-net-flannel.yml │ ├── k8s-net-kube-ovn.yml │ ├── k8s-net-kube-router.yml │ ├── k8s-net-macvlan.yml │ └── kube_control_plane.yml └── inventory.ini10 directories, 55 files修改配置文件基本配置修改配置文件inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml# 选择网络插件支持 cilium, calico, weave 和 flannelkube_network_plugin: calico# 设置 Service 网段kube_service_addresses:10.233.0.0/18# 设置 Pod 网段kube_pods_subnet:10.233.64.0/18# 支持 docker, crio 和 containerd推荐 containerd.container_manager: containerd# 是否开启 kata containerskata_containers_enabled:false# 是否开启自动更新证书推荐开启。auto_renew_certificates:true准备机器列表配置文件inventory/mycluster/inventory.ini[all]master01ansible_host192.168.0.6 node01ansible_host192.168.0.144 node02ansible_host192.168.0.121[all:vars]ansible_userrootansible_passwordQAZ2wsx[kube_control_plane]master01[etcd]master01[kube_node]master01 node01 node02[calico_rr][k8s_cluster:children]kube_control_plane kube_node calico_rr使用密码登录需要安装sshpass才能正常执行旧版本系统sudoyum-yinstallsshpass 新版本系统sudodnf-yinstallsshpasssudoapt-yinstallsshpass国内离线包准备如果可以正常下载国外镜像此节点可忽略下载镜像及软件包生成依赖文件和镜像列表cdcontrib/offlinebashgenerate_list.sh生成之后正常可以看到下内容files.list 是依赖文件的列表images.list 是依赖镜像的列表下载静态文件wget-x-Ptemp/files-itemp/files.list设置下载服务为了方便安装的时候可以通过自己的服务进行下载安装nginx旧版本系统sudoyum-yinstallnginx 新版本系统sudodnf-yinstallnginx设置nginx配置文件配置文件目录**/etc/nginx/conf.d/download.conf **没有这个文件可以创建这个文件server{# 监听端口如果 80 被占用或没有公网 IP可以改为 8080 等高位端口listen80;# 如果有域名填域名没有域名填 localhost 或您的内网 IPserver_name localhost;# 定义下载路径访问 http://IP/download/ 即可看到文件列表location /download/{# 【核心】指定本地文件的绝对路径# Windows 用户注意路径请使用正斜杠 / 或双反斜杠 \\例如D:/my_share_files/ 或 D:\\my_share_files\\# Linux 用户注意路径末尾必须带斜杠例如/data/my_share_files/alias/root/kubespray/contrib/offline/temp/files/;# 【核心】开启目录浏览功能像 FTP 一样显示文件列表autoindex on;autoindex_exact_size off;# 显示文件大小KB/MB而不是精确的字节数autoindex_localtime on;# 显示文件时间为服务器本地时间# 解决中文文件名/目录名乱码问题charset utf-8;# 大文件下载优化防止下载大文件时内存占用过高或超时sendfile on;tcp_nopush on;tcp_nodelay on;keepalive_timeout300;# 增加超时时间防止大文件下载中断}}启动nginxnginx-t# 检测配置文件内容是否正确systemctl restart nginx修改配置文件配置文件路径inventory/mycluster/group_vars/all/offline.yml可以按照以下修改具体版本不同需要的内容不通# 可以修改成自己的镜像仓库registry_host:swr.cn-north-4.myhuaweicloud.com/ddn-k8s/# 修改成自己nginx服务器的IP地址目录files_repo:http://IP/download/### If using CentOS, RedHat, AlmaLinux or Fedora# yum_repo: http://myinternalyumrepo### If using Debian# debian_repo: http://myinternaldebianrepo### If using Ubuntu# ubuntu_repo: http://myinternalubunturepo## Container Registry overrides# 参考以下内容修改如果你有镜像仓库可以略微修改下kube_image_repo:{{ registry_host/registry.k8s.io }}gcr_image_repo:{{ registry_host/gcr.io }}github_image_repo:{{ registry_host/ghcr.io }}docker_image_repo:{{ registry_host/docker.io }}quay_image_repo:{{ registry_host/quay.io }}## Kubernetes componentskubeadm_download_url:{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubeadmkubectl_download_url:{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubectlkubelet_download_url:{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubelet## Two options - Override entire repository or override only a single binary.## [Optional] 1 - Override entire binary repository# github_url: https://my_github_proxy# dl_k8s_io_url: https://my_dl_k8s_io_proxy# storage_googleapis_url: https://my_storage_googleapi_proxy# get_helm_url: https://my_helm_sh_proxy## [Optional] 2 - Override a specific binary## CNI Pluginscni_download_url:{{ files_repo }}/github.com/containernetworking/plugins/releases/download/v{{ cni_version }}/cni-plugins-linux-{{ image_arch }}-v{{ cni_version }}.tgz## cri-toolscrictl_download_url:{{ files_repo }}/github.com/kubernetes-sigs/cri-tools/releases/download/v{{ crictl_version }}/crictl-v{{ crictl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz## [Optional] etcd: only if you use etcd_deploymenthostetcd_download_url:{{ files_repo }}/github.com/etcd-io/etcd/releases/download/v{{ etcd_version }}/etcd-v{{ etcd_version }}-linux-{{ image_arch }}.tar.gz# [Optional] Calico: If using Calico network plugincalicoctl_download_url:{{ files_repo }}/github.com/projectcalico/calico/releases/download/v{{ calico_ctl_version }}/calicoctl-linux-{{ image_arch }}# [Optional] Calico with kdd: If using Calico network plugin with kdd datastorecalico_crds_download_url:{{ files_repo }}/github.com/projectcalico/calico/archive/v{{ calico_version }}.tar.gz# [Optional] Cilium: If using Cilium network pluginciliumcli_download_url:{{ files_repo }}/github.com/cilium/cilium-cli/releases/download/v{{ cilium_cli_version }}/cilium-linux-{{ image_arch }}.tar.gz# [Optional] helm: only if you set helm_enabled: truehelm_download_url:{{ files_repo }}/get.helm.sh/helm-v{{ helm_version }}-linux-{{ image_arch }}.tar.gz# [Optional] crun: only if you set crun_enabled: truecrun_download_url:{{ files_repo }}/github.com/containers/crun/releases/download/{{ crun_version }}/crun-{{ crun_version }}-linux-{{ image_arch }}# [Optional] kata: only if you set kata_containers_enabled: truekata_containers_download_url:{{ files_repo }}/github.com/kata-containers/kata-containers/releases/download/{{ kata_containers_version }}/kata-static-{{ kata_containers_version }}-{{ image_arch }}.tar.xz# [Optional] cri-dockerd: only if you set container_manager: dockercri_dockerd_download_url:{{ files_repo }}/github.com/Mirantis/cri-dockerd/releases/download/v{{ cri_dockerd_version }}/cri-dockerd-{{ cri_dockerd_version }}.{{ image_arch }}.tgz# [Optional] runc: if you set container_manager to containerd or criorunc_download_url:{{ files_repo }}/github.com/opencontainers/runc/releases/download/v{{ runc_version }}/runc.{{ image_arch }}# [Optional] cri-o: only if you set container_manager: crio# crio_download_base: download.opensuse.org/repositories/devel:kubic:libcontainers:stable# crio_download_crio: http://{{ crio_download_base }}:/cri-o:/crio_download_url:{{ files_repo }}/storage.googleapis.com/cri-o/artifacts/cri-o.{{ image_arch }}.v{{ crio_version }}.tar.gzskopeo_download_url:{{ files_repo }}/github.com/lework/skopeo-binary/releases/download/v{{ skopeo_version }}/skopeo-linux-{{ image_arch }}# [Optional] containerd: only if you set container_runtime: containerdcontainerd_download_url:{{ files_repo }}/github.com/containerd/containerd/releases/download/v{{ containerd_version }}/containerd-{{ containerd_version }}-linux-{{ image_arch }}.tar.gznerdctl_download_url:{{ files_repo }}/github.com/containerd/nerdctl/releases/download/v{{ nerdctl_version }}/nerdctl-{{ nerdctl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz# [Optional] runsc,containerd-shim-runsc: only if you set gvisor_enabled: truegvisor_runsc_download_url:{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/runscgvisor_containerd_shim_runsc_download_url:{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/containerd-shim-runsc-v1部署集群# 回到kubespray主目录再执行下面命令ansible-playbook\-iinventory/mycluster/inventory.ini\cluster.yml验证集群kubectl get nodes -o wide kubectl get pods _Anode节点状态均为Ready为正常pod状态均为Running为正常
快速部署K8S集群
一、kubekey方式文章参考链接GitHubhttps://github.com/kubesphere/kubekeykubespherehttps://get-images.kubesphere.io微信公众号https://mp.weixin.qq.com/s/HacD-QPS-7TzlQsDlKnj8Q云原生测试补给站下载kubekeycurl-sfLhttps://get-kk.kubesphere.io|VERSIONvx.xx.xxsh-# x.xx.xx为版本号由于下载网站再国外国内网络下载可能会很慢或者下载不成功可以采用下面方式下载① 官方方案个人感觉不好用exportKKZONEcn# 设置变量curl-sfLhttps://get-kk.kubesphere.io|VERSIONvx.xx.xxsh-# x.xx.xx为版本号② 个人推荐方案手动下载后上传到服务器kubekey下载地址https://github.com/kubesphere/kubekey/releasestarzxvf kubekey-vx.xx.xx-*.tar.gz# 解压软件包./kk version# 查看版本生成配置信息kubekey版本低于4.0./kk version --show-supported-k8s./kk create config --with-kubernetes vx.xx.xx# x.xx.xx替换为你需要安装的k8s版本生成 config-sample.yaml 文件文件内容大致如下(重点修改标颜色部分内容name、ip地址、用户名和密码)apiVersion:kubekey.kubesphere.io/v1alpha2kind:Clustermetadata:name:samplespec:hosts:-{name:Master01,address:192.168.0.6,internalAddress:192.168.0.6,user:root,password:QAZ2wsx}-{name:node01,address:192.168.0.144,internalAddress:192.168.0.144,user:root,password:QAZ2wsx}-{name:node02,address:192.168.0.121,internalAddress:192.168.0.121,user:root,password:QAZ2wsx}roleGroups:etcd:-Master01control-plane:-Master01worker:-node01-node02controlPlaneEndpoint:## Internal loadbalancer for apiservers# internalLoadbalancer: haproxydomain:lb.kubesphere.localaddress:port:6443kubernetes:version:v1.33.4clusterName:cluster.localautoRenewCerts:truecontainerManager:containerdetcd:type:kubekeynetwork:plugin:calicokubePodsCIDR:10.233.64.0/18kubeServiceCIDR:10.233.0.0/18## multus support. https://github.com/k8snetworkplumbingwg/multus-cnimultusCNI:enabled:falseregistry:privateRegistry:namespaceOverride:registryMirrors:[]insecureRegistries:[]addons:[]kubekey版本高于4.0./kk create inventory-o.会生成 inventory.yaml 文件内容如下(重点修改标颜色部分内容)apiVersion:kubekey.kubesphere.io/v1kind:Inventorymetadata:name:defaultspec:hosts:# your can set all nodes here. or set nodes on special groups.master01:# 主机名connector:tyep:sshhost:192.168.0.6port:22user:rootpassword:QAZ2wsxinternal_ipv4:192.168.0.6node01:# 主机名connector:type:sshhost:192.168.0.144port:22user:rootpassword:QAZ2wsxinternal_ipv4:192.168.0.144node02:# 主机名connector:type:sshhost:192.168.0.121port:22user:rootpassword:QAZ2wsxinternal_ipv4:192.168.0.121groups:# all kubernetes nodes.k8s_cluster:groups:-kube_control_plane-kube_worker# control_plane nodeskube_control_plane:hosts:-master01# worker nodeskube_worker:hosts:-master01-node01-node02# etcd nodes when etcd_deployment_type is externaletcd:hosts:-master01# image_registry:# hosts:# - localhost# nfs nodes for registry storage. and kubernetes nfs storage# nfs:# hosts:# - localhost./kk create config --with-kubernetesKubernetes version-o.会生成 config-.yaml 文件内容如下:apiVersion:kubekey.kubesphere.io/v1kind:Configspec:# If set to cn, online downloads will prioritize domestic sources when available.zone:cn## 此处很重要如果填会导致你的镜像下载不成功kubernetes:# Specify the Kubernetes version to be installed.kube_version:v1.34.3# Specify the Helm version to be installed.helm_version:v3.18.5# Tag for the sandbox (pause) image used by pods.sandbox_image:tag:3.10.1control_plane_endpoint:# Supported HA types: local, kube-vip, haproxy.# If set to local, configure local hostname resolution as follows:# - Control-plane nodes: 127.0.0.1 .kubernetes.control_plane_endpoint.host# - Worker nodes: .init_kubernetes_node .kubernetes.control_plane_endpoint.hosttype:local# Kube-vip image tag used for static pod deployment (when type is kube-vip).kube_vip:image:tag:v0.7.2# HAProxy image tag used for static pod deployment (when type is haproxy).haproxy:image:tag:2.9.6-alpineetcd:# Specify the etcd version to be installed.etcd_version:v3.6.5image_registry:# Image registry type to install. Supported: harbor, docker-registry.# Leave empty to skip installation (assuming an existing registry is available).type:# auth:# # Address of the private image registry. Format: host:port/project# # The port (if present) is used as http_port when plain_httptrue,# # or as https_port when plain_httpfalse.# registry: dockerhub.kubekey.local# # Use plain HTTP (no TLS). Default is false.# plain_http: false# username: admin# password: Harbor12345# Specify a VIP for image registry high availability. If set, enables HA.ha_vip:# Image Registry - High Availability # keepalived image tag for load balancing when multiple registry nodes exist.# keepalived_version: 2.3.4# Image Registry - Harbor # Image registry port settings. Usually you do not need to set them manually;# they are derived from image_registry.auth.registry and plain_http.# http_port: 80# https_port: 443# Harbor image tag (only valid if type is harbor).#harbor_version: v2.10.2# docker-compose binary#dockercompose_version: v2.20.3# Image Registry - Docker Registry # Docker Registry image tag (only valid if type is docker-registry).# docker_registry_version: 2.8.3cri:# Container runtime type. Supported: containerd, docker.container_manager:containerd# CRI Tool # crictl binary version.crictl_version:v1.34.0# Docker Runtime # Docker binary version.#docker_version: 25.0.5# cridockerd version (required for Kubernetes 1.24).# cridockerd_version: v0.3.21# Containerd Runtime # containerd binary version (active only when container_manager is containerd).# containerd_version: v1.7.13# runc binary version (active only when container_manager is containerd).# runc_version: v1.1.12cni:# CNI plugin type. Supported: calico, cilium, flannel, hybridnet, kubeovn, othertype:calico# Multi-CNI # Multi-CNI type. Supported: multus, spiderpool, none.multi_cni:none# Multi-CNI - Multus # Image tag for Multus (configure if needed).# multus:# image:# tag: v4.2.4# Multi-CNI - Spiderpool # Spiderpool version (configure if needed).# spiderpool_version: v1.1.1# Calico CNI # Calico version (effective only when type is calico).calico_version:v3.31.3# Cilium CNI # Cilium version (effective only when type is cilium).# cilium_version: 1.19.1# Flannel CNI # Flannel version (effective only when type is flannel).#flannel_version: v0.27.4# Kube-OVN CNI # Kube-OVN version (effective only when type is kubeovn).#kubeovn_version: v1.15.0# Hybridnet CNI # Hybridnet version (effective only when type is hybridnet).#hybridnet_version: 0.6.8storage_class:# Storage Class Configuration # Local/OpenEBS Storage Class # Local storage class settings.local:enabled:true# Enable the local storage class.default:true# Set as the default storage class.# openebs/dynamic-localpv-provisioner helm chart version.localpv_provisioner_version:4.4.0# NFS Storage Class nfs:enabled:false# Enable the NFS storage class.# nfs-provisioner Helm chart version.#nfs_provisioner_version: 4.0.18dns:# CoreDNS settings.coredns:image:tag:v1.12.1# NodeLocalDNS settings.nodelocaldns:enabled:trueimage:tag:1.26.4# the external images will to add in packages初始化OS依赖kubekey版本低于4.0./kk init os-fconfig-sample.yamlkubekey版本高于4.0./kk init os-iinventory.yaml-cconfig-Kubernetes version.yaml创建集群kubekey版本低于4.0./kk create cluster-fconfig-sample.yamlkubekey版本高于4.0./kk create cluster-iinventory.yaml-cconfig-Kubernetes version.yaml验证集群kubectl get nodes -o wide kubectl get pods _Anode节点状态均为Ready为正常pod状态均为Running为正常二、kubespray方式文章参考链接roc 云原生https://imroc.cc/kubernetes/deploy/kubespray/install准备工作下载kubespraygitclone--depth1https://github.com/kubernetes-sigs/kubespray.git正常国内采用上面命令下载是不成功的建议去github下载后上传到服务器下载地址https://github.com/kubernetes-sigs/kubespray/releases# zip包解压unzipkubespray-kubespray version.zip# tar包解压tra zxvf kubespray-kubespray version.zip安装依赖cdkubespray-kubespray versionpip3install-rrequirements.txt这里很容易出现下面的这种报错这种报错原因是你的操作系统版本太低kubespray版本太高的问题换个高版本操作系统或者低版本kubespray就可以了复制配置文件cp-rfpinventory/sample inventory/mycluster配置文件结构如下cdinventory tree.├── local│ ├── group_vars - …/sample/group_vars│ └── hosts.ini├── mycluster│ ├── group_vars│ │ ├── all│ │ │ ├── all.yml│ │ │ ├── aws.yml│ │ │ ├── azure.yml│ │ │ ├── containerd.yml│ │ │ ├── coreos.yml│ │ │ ├── cri-o.yml│ │ │ ├── docker.yml│ │ │ ├── etcd.yml│ │ │ ├── gcp.yml│ │ │ ├── hcloud.yml│ │ │ ├── huaweicloud.yml│ │ │ ├── oci.yml│ │ │ ├── offline.yml│ │ │ ├── openstack.yml│ │ │ ├── upcloud.yml│ │ │ └── vsphere.yml│ │ └── k8s_cluster│ │ ├── addons.yml│ │ ├── k8s-cluster.yml│ │ ├── k8s-net-calico.yml│ │ ├── k8s-net-cilium.yml│ │ ├── k8s-net-custom-cni.yml│ │ ├── k8s-net-flannel.yml│ │ ├── k8s-net-kube-ovn.yml│ │ ├── k8s-net-kube-router.yml│ │ ├── k8s-net-macvlan.yml│ │ └── kube_control_plane.yml│ └── inventory.ini└── sample├── group_vars │ ├── all │ │ ├── all.yml │ │ ├── aws.yml │ │ ├── azure.yml │ │ ├── containerd.yml │ │ ├── coreos.yml │ │ ├── cri-o.yml │ │ ├── docker.yml │ │ ├── etcd.yml │ │ ├── gcp.yml │ │ ├── hcloud.yml │ │ ├── huaweicloud.yml │ │ ├── oci.yml │ │ ├── offline.yml │ │ ├── openstack.yml │ │ ├── upcloud.yml │ │ └── vsphere.yml │ └── k8s_cluster │ ├── addons.yml │ ├── k8s-cluster.yml │ ├── k8s-net-calico.yml │ ├── k8s-net-cilium.yml │ ├── k8s-net-custom-cni.yml │ ├── k8s-net-flannel.yml │ ├── k8s-net-kube-ovn.yml │ ├── k8s-net-kube-router.yml │ ├── k8s-net-macvlan.yml │ └── kube_control_plane.yml └── inventory.ini10 directories, 55 files修改配置文件基本配置修改配置文件inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml# 选择网络插件支持 cilium, calico, weave 和 flannelkube_network_plugin: calico# 设置 Service 网段kube_service_addresses:10.233.0.0/18# 设置 Pod 网段kube_pods_subnet:10.233.64.0/18# 支持 docker, crio 和 containerd推荐 containerd.container_manager: containerd# 是否开启 kata containerskata_containers_enabled:false# 是否开启自动更新证书推荐开启。auto_renew_certificates:true准备机器列表配置文件inventory/mycluster/inventory.ini[all]master01ansible_host192.168.0.6 node01ansible_host192.168.0.144 node02ansible_host192.168.0.121[all:vars]ansible_userrootansible_passwordQAZ2wsx[kube_control_plane]master01[etcd]master01[kube_node]master01 node01 node02[calico_rr][k8s_cluster:children]kube_control_plane kube_node calico_rr使用密码登录需要安装sshpass才能正常执行旧版本系统sudoyum-yinstallsshpass 新版本系统sudodnf-yinstallsshpasssudoapt-yinstallsshpass国内离线包准备如果可以正常下载国外镜像此节点可忽略下载镜像及软件包生成依赖文件和镜像列表cdcontrib/offlinebashgenerate_list.sh生成之后正常可以看到下内容files.list 是依赖文件的列表images.list 是依赖镜像的列表下载静态文件wget-x-Ptemp/files-itemp/files.list设置下载服务为了方便安装的时候可以通过自己的服务进行下载安装nginx旧版本系统sudoyum-yinstallnginx 新版本系统sudodnf-yinstallnginx设置nginx配置文件配置文件目录**/etc/nginx/conf.d/download.conf **没有这个文件可以创建这个文件server{# 监听端口如果 80 被占用或没有公网 IP可以改为 8080 等高位端口listen80;# 如果有域名填域名没有域名填 localhost 或您的内网 IPserver_name localhost;# 定义下载路径访问 http://IP/download/ 即可看到文件列表location /download/{# 【核心】指定本地文件的绝对路径# Windows 用户注意路径请使用正斜杠 / 或双反斜杠 \\例如D:/my_share_files/ 或 D:\\my_share_files\\# Linux 用户注意路径末尾必须带斜杠例如/data/my_share_files/alias/root/kubespray/contrib/offline/temp/files/;# 【核心】开启目录浏览功能像 FTP 一样显示文件列表autoindex on;autoindex_exact_size off;# 显示文件大小KB/MB而不是精确的字节数autoindex_localtime on;# 显示文件时间为服务器本地时间# 解决中文文件名/目录名乱码问题charset utf-8;# 大文件下载优化防止下载大文件时内存占用过高或超时sendfile on;tcp_nopush on;tcp_nodelay on;keepalive_timeout300;# 增加超时时间防止大文件下载中断}}启动nginxnginx-t# 检测配置文件内容是否正确systemctl restart nginx修改配置文件配置文件路径inventory/mycluster/group_vars/all/offline.yml可以按照以下修改具体版本不同需要的内容不通# 可以修改成自己的镜像仓库registry_host:swr.cn-north-4.myhuaweicloud.com/ddn-k8s/# 修改成自己nginx服务器的IP地址目录files_repo:http://IP/download/### If using CentOS, RedHat, AlmaLinux or Fedora# yum_repo: http://myinternalyumrepo### If using Debian# debian_repo: http://myinternaldebianrepo### If using Ubuntu# ubuntu_repo: http://myinternalubunturepo## Container Registry overrides# 参考以下内容修改如果你有镜像仓库可以略微修改下kube_image_repo:{{ registry_host/registry.k8s.io }}gcr_image_repo:{{ registry_host/gcr.io }}github_image_repo:{{ registry_host/ghcr.io }}docker_image_repo:{{ registry_host/docker.io }}quay_image_repo:{{ registry_host/quay.io }}## Kubernetes componentskubeadm_download_url:{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubeadmkubectl_download_url:{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubectlkubelet_download_url:{{ files_repo }}/dl.k8s.io/release/v{{ kube_version }}/bin/linux/{{ image_arch }}/kubelet## Two options - Override entire repository or override only a single binary.## [Optional] 1 - Override entire binary repository# github_url: https://my_github_proxy# dl_k8s_io_url: https://my_dl_k8s_io_proxy# storage_googleapis_url: https://my_storage_googleapi_proxy# get_helm_url: https://my_helm_sh_proxy## [Optional] 2 - Override a specific binary## CNI Pluginscni_download_url:{{ files_repo }}/github.com/containernetworking/plugins/releases/download/v{{ cni_version }}/cni-plugins-linux-{{ image_arch }}-v{{ cni_version }}.tgz## cri-toolscrictl_download_url:{{ files_repo }}/github.com/kubernetes-sigs/cri-tools/releases/download/v{{ crictl_version }}/crictl-v{{ crictl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz## [Optional] etcd: only if you use etcd_deploymenthostetcd_download_url:{{ files_repo }}/github.com/etcd-io/etcd/releases/download/v{{ etcd_version }}/etcd-v{{ etcd_version }}-linux-{{ image_arch }}.tar.gz# [Optional] Calico: If using Calico network plugincalicoctl_download_url:{{ files_repo }}/github.com/projectcalico/calico/releases/download/v{{ calico_ctl_version }}/calicoctl-linux-{{ image_arch }}# [Optional] Calico with kdd: If using Calico network plugin with kdd datastorecalico_crds_download_url:{{ files_repo }}/github.com/projectcalico/calico/archive/v{{ calico_version }}.tar.gz# [Optional] Cilium: If using Cilium network pluginciliumcli_download_url:{{ files_repo }}/github.com/cilium/cilium-cli/releases/download/v{{ cilium_cli_version }}/cilium-linux-{{ image_arch }}.tar.gz# [Optional] helm: only if you set helm_enabled: truehelm_download_url:{{ files_repo }}/get.helm.sh/helm-v{{ helm_version }}-linux-{{ image_arch }}.tar.gz# [Optional] crun: only if you set crun_enabled: truecrun_download_url:{{ files_repo }}/github.com/containers/crun/releases/download/{{ crun_version }}/crun-{{ crun_version }}-linux-{{ image_arch }}# [Optional] kata: only if you set kata_containers_enabled: truekata_containers_download_url:{{ files_repo }}/github.com/kata-containers/kata-containers/releases/download/{{ kata_containers_version }}/kata-static-{{ kata_containers_version }}-{{ image_arch }}.tar.xz# [Optional] cri-dockerd: only if you set container_manager: dockercri_dockerd_download_url:{{ files_repo }}/github.com/Mirantis/cri-dockerd/releases/download/v{{ cri_dockerd_version }}/cri-dockerd-{{ cri_dockerd_version }}.{{ image_arch }}.tgz# [Optional] runc: if you set container_manager to containerd or criorunc_download_url:{{ files_repo }}/github.com/opencontainers/runc/releases/download/v{{ runc_version }}/runc.{{ image_arch }}# [Optional] cri-o: only if you set container_manager: crio# crio_download_base: download.opensuse.org/repositories/devel:kubic:libcontainers:stable# crio_download_crio: http://{{ crio_download_base }}:/cri-o:/crio_download_url:{{ files_repo }}/storage.googleapis.com/cri-o/artifacts/cri-o.{{ image_arch }}.v{{ crio_version }}.tar.gzskopeo_download_url:{{ files_repo }}/github.com/lework/skopeo-binary/releases/download/v{{ skopeo_version }}/skopeo-linux-{{ image_arch }}# [Optional] containerd: only if you set container_runtime: containerdcontainerd_download_url:{{ files_repo }}/github.com/containerd/containerd/releases/download/v{{ containerd_version }}/containerd-{{ containerd_version }}-linux-{{ image_arch }}.tar.gznerdctl_download_url:{{ files_repo }}/github.com/containerd/nerdctl/releases/download/v{{ nerdctl_version }}/nerdctl-{{ nerdctl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz# [Optional] runsc,containerd-shim-runsc: only if you set gvisor_enabled: truegvisor_runsc_download_url:{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/runscgvisor_containerd_shim_runsc_download_url:{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/containerd-shim-runsc-v1部署集群# 回到kubespray主目录再执行下面命令ansible-playbook\-iinventory/mycluster/inventory.ini\cluster.yml验证集群kubectl get nodes -o wide kubectl get pods _Anode节点状态均为Ready为正常pod状态均为Running为正常
![[光学原理与应用-521]：对光的错误理解与纠偏](http://pic.xiahunao.cn/yaotu/[光学原理与应用-521]：对光的错误理解与纠偏)
