搜索到
152
篇与
的结果
-
kubeadm 部署k8s 1.30 一、k8s包yum源介绍二、准备工作准备3台机器修改好网络改为固定IPcd /etc/NetworkManager/system-connections/ cp /etc/NetworkManager/system-connections/ens160.nmconnection /etc/NetworkManager/system-connections/ens160.nmconnection.backup vi ens160.nmconnection TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=static DEFROUTE=yes NAME=ens33 DEVICE=ens33 ONBOOT=yes #这个可以让开机不用nmcli IPADDR=192.168.110.97 GATEWAY=192.168.110.1 NETSTAT=255.255.255.0 DNS1=8.8.8.8 DNS2=192.168.110.1 sudo systemctl restart NetworkManager nmcli conn up ens33修改主机名及解析（三台节点）# 1、修改主机名 hostnamectl set-hostname k8s-master-01 hostnamectl set-hostname k8s-node-01 hostnamectl set-hostname k8s-node-02 # 2、三台机器添加host解析 cat >> /etc/hosts << "EOF" 192.168.110.97 k8s-master-01 m1 192.168.110.213 k8s-node-01 n1 192.168.110.2 k8s-node-02 n2 EOF关闭一些服务（三台节点）# 1、关闭selinux sed -i 's#enforcing#disabled#g' /etc/selinux/config setenforce 0 # 2、禁用防火墙，网络管理，邮箱 systemctl disable --now firewalld NetworkManager postfix # 3、关闭swap分区 swapoff -a # 注释swap分区 cp /etc/fstab /etc/fstab_bak sed -i '/swap/d' /etc/fstabsshd服务优化# 1、加速访问 sed -ri 's@^#UseDNS yes@UseDNS no@g' /etc/ssh/sshd_config sudo sed -ri 's@^#?\s*GSSAPIAuthentication\s+yes@GSSAPIAuthentication no@gi' /etc/ssh/sshd_config grep ^UseDNS /etc/ssh/sshd_config grep ^GSSAPIAuthentication /etc/ssh/sshd_config systemctl restart sshd # 2、密钥登录（主机点做）:为了让后续一些远程拷贝操作更方便 ssh-keygen ssh-copy-id -i root@k8s-master-01 ssh-copy-id -i root@k8s-node-01 ssh-copy-id -i root@k8s-node-02 #连接测试 [root@m01 ~]# ssh 172.16.1.7 Last login: Tue Nov 24 09:02:26 2020 from 10.0.0.1 [root@web01 ~]#6.增大文件标识符数量（退出当前会话立即生效）cat > /etc/security/limits.d/k8s.conf <<EOF * soft nofile 65535 * hard nofile 131070 EOF ulimit -Sn ulimit -Hn所有节点配置模块自动加载，此步骤不做的话（kubeadm init时会直接失败）modprobe br_netfilter modprobe ip_conntrack cat >>/etc/rc.sysinit<<EOF #!/bin/bash for file in /etc/sysconfig/modules/*.modules ; do [ -x $file ] && $file done EOF echo "modprobe br_netfilter" >/etc/sysconfig/modules/br_netfilter.modules echo "modprobe ip_conntrack" >/etc/sysconfig/modules/ip_conntrack.modules chmod 755 /etc/sysconfig/modules/br_netfilter.modules chmod 755 /etc/sysconfig/modules/ip_conntrack.modules lsmod | grep br_netfilter 同步集群时间# =====================》chrony服务端：服务端我们可以自己搭建，也可以直接用公网上的时间服务器，所以是否部署服务端看你自己 # 1、安装 yum -y install chrony # 2、修改配置文件 mv /etc/chrony.conf /etc/chrony.conf.bak cat > /etc/chrony.conf << EOF server ntp1.aliyun.com iburst minpoll 4 maxpoll 10 server ntp2.aliyun.com iburst minpoll 4 maxpoll 10 server ntp3.aliyun.com iburst minpoll 4 maxpoll 10 server ntp4.aliyun.com iburst minpoll 4 maxpoll 10 server ntp5.aliyun.com iburst minpoll 4 maxpoll 10 server ntp6.aliyun.com iburst minpoll 4 maxpoll 10 server ntp7.aliyun.com iburst minpoll 4 maxpoll 10 driftfile /var/lib/chrony/drift makestep 10 3 rtcsync allow 0.0.0.0/0 local stratum 10 keyfile /etc/chrony.keys logdir /var/log/chrony stratumweight 0.05 noclientlog logchange 0.5 EOF # 4、启动chronyd服务 systemctl restart chronyd.service # 最好重启，这样无论原来是否启动都可以重新加载配置 systemctl enable chronyd.service systemctl status chronyd.service # =====================》chrony客户端：在需要与外部同步时间的机器上安装，启动后会自动与你指定的服务端同步时间 # 下述步骤一次性粘贴到每个客户端执行即可 # 1、安装chrony yum -y install chrony # 2、需改客户端配置文件 mv /etc/chrony.conf /etc/chrony.conf.bak cat > /etc/chrony.conf << EOF server 192.168.110.97 iburst driftfile /var/lib/chrony/drift makestep 10 3 rtcsync local stratum 10 keyfile /etc/chrony.key logdir /var/log/chrony stratumweight 0.05 noclientlog logchange 0.5 EOF # 3、启动chronyd systemctl restart chronyd.service systemctl enable chronyd.service systemctl status chronyd.service # 4、验证 chronyc sources -v更新基础yum源（三台机器）# 1、清理 rm -rf /etc/yum.repos.d/* yum remove epel-release -y rm -rf /var/cache/yum/x86_64/6/epel/ # 2、安装阿里的base与epel源 curl -s -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo curl -s -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo yum clean all yum makecache # 或者用华为的也行 # curl -o /etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo # yum install -y https://repo.huaweicloud.com/epel/epel-release-latest-7.noarch.rpm更新基础yum源（三台机器）# 1、清理 rm -rf /etc/yum.repos.d/* yum remove epel-release -y rm -rf /var/cache/yum/x86_64/6/epel/ # 2、安装阿里的base与epel源 curl -s -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo curl -s -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo yum clean all yum makecache # 或者用华为的也行 # curl -o /etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo # yum install -y https://repo.huaweicloud.com/epel/epel-release-latest-7.noarch.rpm更新系统软件（排除内核） yum update -y --exclud=kernel*安装基础常用软件yum -y install expect wget jq psmisc vim net-tools telnet yum-utils device-mapper-persistent-data lvm2 git ntpdate chrony bind-utils rsync unzip git更新内核（docker对系统内核要求比较高，最好使用4.4+）主节点操作wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-5.4.274-1.el7.elrepo.x86_64.rpm wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.274-1.el7.elrepo.x86_64.rpm for i in n1 n2 ; do scp kernel-lt-* $i:/root; done 补充：如果下载的慢就从网盘里拿吧 链接：https://pan.baidu.com/s/1gVyeBQsJPZjc336E8zGjyQ 提取码：Egon 三个节点操作 #安装 yum localinstall -y /root/kernel-lt* #调到默认启动 grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg #查看当前默认启动的内核 grubby --default-kernel #重启系统 reboot三个节点安装IPVS# 1、安装ipvsadm等相关工具 yum -y install ipvsadm ipset sysstat conntrack libseccomp # 2、配置加载 cat > /etc/sysconfig/modules/ipvs.modules <<"EOF" #!/bin/bash ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack" for kernel_module in ${ipvs_modules}; do /sbin/modinfo -F filename ${kernel_module} > /dev/null 2>&1 if [ $? -eq 0 ]; then /sbin/modprobe ${kernel_module} fi done EOF chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep ip_vs三台机器修改内核参数cat > /etc/sysctl.d/k8s.conf << EOF net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 fs.may_detach_mounts = 1 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100 fs.file-max=52706963 fs.nr_open=52706963 net.ipv4.tcp_keepalive_time = 600 net.ipv4.tcp.keepaliv.probes = 3 net.ipv4.tcp_keepalive_intvl = 15 net.ipv4.tcp.max_tw_buckets = 36000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp.max_orphans = 327680 net.ipv4.tcp_orphan_retries = 3 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.ip_conntrack_max = 65536 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.top_timestamps = 0 net.core.somaxconn = 16384 EOF # 立即生效 sysctl --system三、 安装containerd(三台节点都要做)自Kubernetes1.24以后，K8S就不再原生支持docker了我们都知道containerd来自于docker，后被docker捐献给了云原生计算基金会(我们安装docker当然会一并安装上containerd)安装方法：centos的libseccomp的版本为2.3.1，不满足containerd的需求，需要下载2.4以上的版本即可，我这里部署2.5.1版本。 rpm -e libseccomp-2.5.1-1.el8.x86_64 --nodeps rpm -ivh libseccomp-2.5.1-1.e18.x8664.rpm #官网已经gg了，不更新了，请用阿里云 # wget http://rpmfind.net/linux/centos/8-stream/Base0s/x86 64/0s/Packages/libseccomp-2.5.1-1.el8.x86_64.rpm wget https://mirrors.aliyun.com/centos/8/BaseOS/x86_64/os/Packages/libseccomp-2.5.1-1.el8.x86_64.rpm cd /root/rpms sudo yum localinstall libseccomp-2.5.1-1.el8.x86_64.rpm -y #yum libseccomp-2.5.1-1.el8.x86_64.rpm -y rpm -qa | grep libseccomp 安装方式一:(基于阿里云的源)推荐用这种方式，安装的是#1、卸载之前的 yum remove docker docker-ce containerd docker-common docker-selinux docker-engine -y #2、准备repo cd /etc/yum.repos.d/ wget http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo # 3、安装 yum install containerd* -y配置# 1、配置 mkdir -pv /etc/containerd containerd config default > /etc/containerd/config.toml #为containerd生成配置文件 #2、替换默认pause镜像地址:这一步非常非常非常非常重要 grep sandbox_image /etc/containerd/config.toml sed -i 's/registry.k8s.io/registry.cn-hangzhou.aliyuncs.com\/google_containers/' /etc/containerd/config.toml grep sandbox_image /etc/containerd/config.toml #请务必确认新地址是可用的: sandbox_image="registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6" #3、配置systemd作为容器的cgroup driver grep SystemdCgroup /etc/containerd/config.toml sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/' /etc/containerd/config.toml grep SystemdCgroup /etc/containerd/config.toml # 4、配置加速器(必须配置，否则后续安装cni网络插件时无法从docker.io里下载镜像) #参考:https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration #添加 config_path="/etc/containerd/certs.d" sed -i 's/config_path\ =.*/config_path = \"\/etc\/containerd\/certs.d\"/g' /etc/containerd/config.tomlmkdir -p /etc/containerd/certs.d/docker.io cat>/etc/containerd/certs.d/docker.io/hosts.toml << EOF server ="https://docker.io" [host."https ://dockerproxy.com"] capabilities = ["pull","resolve"] [host."https://docker.m.daocloud.io"] capabilities = ["pull","resolve"] [host."https://docker.chenby.cn"] capabilities = ["pull","resolve"] [host."https://registry.docker-cn.com"] capabilities = ["pull","resolve" ] [host."http://hub-mirror.c.163.com"] capabilities = ["pull","resolve" ] EOF#5、配置containerd开机自启动 #5.1 启动containerd服务并配置开机自启动 systemctl daemon-reload && systemctl restart containerd systemctl enable --now containerd #5.2 查看containerd状态 systemctl status containerd #5.3查看containerd的版本 ctr version-------------------------配置docker（下述内容不用操作，因为k8s1.30直接对接containerd） # 1、配置docker # 修改配置：驱动与kubelet保持一致，否则会后期无法启动kubelet cat > /etc/docker/daemon.json << "EOF" { "exec-opts": ["native.cgroupdriver=systemd"], "registry-mirrors":["https://reg-mirror.qiniu.com/"] } EOF # 2、重启docker systemctl restart docker.service systemctl enable docker.service # 3、查看验证 [root@k8s-master-01 ~]# docker info |grep -i cgroup Cgroup Driver: systemd Cgroup Version: 1四、 安装k8s官网:https://kubernetes.io/zh-cn/docs/reference/setup-tools/kubeadm/kubeadm-init/1、三台机器准备k8s源cat > /etc/yum.repos.d/kubernetes.repo <<"EOF" [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/rpm/ enabled=1 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/rpm/repodata/repomd.xml.key EOF #参考:https://developer.aliyun.com/mirror/kubernetes/setenforce yum install -y kubelet-1.30* kubeadm-1.30* kubectl-1.30* systemctl enable kubelet && systemctl start kubelet && systemctl status kubelet2、主节点操作（node节点不执行）初始化master节点(仅在master节点上执行) #可以kubeadm config images list查看 [root@k8s-master-01 ~]# kubeadm config images list registry.k8s.io/kube-apiserver:v1.30.0 registry.k8s.io/kube-controller-manager:v1.30.0 registry.k8s.io/kube-scheduler:v1.30.0 registry.k8s.io/kube-proxy:v1.30.0 registry.k8s.io/coredns/coredns:v1.11.1 registry.k8s.io/pause:3.9 registry.k8s.io/etcd:3.5.12-0kubeadm config print init-defaults > kubeadm.yamlvi kubeadm.yaml apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.110.97 #这里要改为控制节点 bindPort: 6443 nodeRegistration: criSocket: unix:///var/run/containerd/containerd.sock imagePullPolicy: IfNotPresent name: k8s-master-01 #这里要修改 taints: null --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta3 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: {} etcd: local: dataDir: /var/lib/etcd imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers #要去阿里云创建仓库 kind: ClusterConfiguration kubernetesVersion: 1.30.3 networking: dnsDomain: cluster.local serviceSubnet: 10.96.0.0/12 podSubnet: 10.244.0.0/16 #添加这行 scheduler: {} #在最后插入以下内容 --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: ipvs --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd部署K8Skubeadm init --config=kubeadm.yaml --ignore-preflight-errors=SystemVerification --ignore-preflight-errors=Swap部署网络插件下载网络插件wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml[root@k8s-master-01 ~]# grep -i image kube-flannel.yml image: docker.io/flannel/flannel:v0.25.5 image: docker.io/flannel/flannel-cni-plugin:v1.5.1-flannel1 image: docker.io/flannel/flannel:v0.25.5 改为下面 要去阿里云上面构建自己的镜像[root@k8s-master-01 ~]# grep -i image kube-flannel.yml image: registry.cn-guangzhou.aliyuncs.com/xingcangku/cccc:0.25.5 image: registry.cn-guangzhou.aliyuncs.com/xingcangku/ddd:1.5.1 image: registry.cn-guangzhou.aliyuncs.com/xingcangku/cccc:0.25.5 部署在master上即可kubectl apply -f kube-flannel.yml kubectl delete -f kube-flannel.yml #这个是删除网络插件的查看状态kubectl -n kube-flannel get pods kubectl -n kube-flannel get pods -w [root@k8s-master-01 ~]# kubectl get nodes # 全部ready [root@k8s-master-01 ~]# kubectl -n kube-system get pods # 两个coredns的pod也都ready部署kubectl命令提示（在所有节点上执行）yum install bash-completion* -y kubectl completion bash > ~/.kube/completion.bash.inc echo "source '$HOME/.kube/completion.bash.inc'" >> $HOME/.bash_profile source $HOME/.bash_profile排错解决方法：===========================================部署遇到问题之后，铲掉环境重新部署 # 在master节点上 kubeadm reset -f # 在所有节点包括master节点在内上执行如下命令 cd /tmp # 有时候在当前目录下可能与要卸载的包重名的而导致卸载报错，可以切个目录 rm -rf ~/.kube/ rm -rf /etc/kubernetes/ rm -rf /etc/cni rm -rf /opt/cni rm -rf /var/lib/etcd rm -rf /var/etcd rm -rf /run/flannel rm -rf /opt/cni rm -rf /etc/cni/net.d rm -rf /run/xtables.lock systemctl stop kubelet yum remove kube* -y for i in `df |grep kubelet |awk '{print $NF}'`;do umount -l $i ;done # 先卸载所有kubelet挂载否则下条命令无法删除 rm -rf /var/lib/kubelet rm -rf /etc/systemd/system/kubelet.service.d rm -rf /etc/systemd/system/kubelet.service rm -rf /usr/bin/kube* iptables -F reboot # 重新启动，从头再来 # 第一步：在所有节点执行 yum install -y kubelet-1.30* kubeadm-1.30* kubectl-1.30* systemctl enable kubelet && systemctl start kubelet && systemctl status kubelet # 第二步：只在master节点上执行 [root@k8s-master-01 ~]# kubeadm init --config=kubeadm.yaml --ignore-preflight-errors=SystemVerification --ignore-preflight-errors=Swap # 第三步：部署网络插件 kubectl apply -f kube-flannel.yml kubectl delete -f kube-flannel.yml mkdir -p /etc/containerd/certs.d/registry.aliyuncs.com tee /etc/containerd/certs.d/registry.aliyuncs.com/hosts.toml <<EOF server = "https://registry.aliyuncs.com" [host."https://registry.aliyuncs.com"] capabilities = ["pull", "resolve"] EOF -
-
operator开发 mysql一主多从 CRD | | | 定义出/创建出 | | ↓ CR，即resource type ----------------》受自定义的控制器watch监听并控制 | | | 定义出/创建出 | | ↓ 一条具体的resource 实现的功能： 1. 支持一主多从 采用GID的自动备份 2. 支持主从的自动选举切换 3. 支持在线扩容 副本不足时会自动拉起 4. 支持就绪探针的检测 5. .........一、go环境准备wget https://golang.google.cn/dl/go1.22.5.linux-amd64.tar.gz tar zxvf go1.22.5.linux-amd64.tar.gz mv go /usr/local/ cat >> /etc/profile << 'EOF' export GOROOT=/usr/local/go export PATH=$PATH:$GOROOT/bin EOF source /etc/profile go version #查看是否生效 # 设置go代理 # 1、也可以用全球cdn加速 export GOPROXY=https://goproxy.cn,direct go env -w GOPROXY=https://goproxy.cn,direct二、安装kubebuilder框架# 1、下载最新版本的kubebuilder(下载慢的话你就手动下载然后上传) wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v4.1.1/kubebuilder_linux_amd64 chmod +x kubebuilder && mv kubebuilder /usr/local/bin/ $ kubebuilder version #安装必要工具 sudo apt update && sudo apt install -y make sudo apt install -y build-essential git curl 三、初始化项目# 创建项目 mkdir -p /src/application-operator cd /src/application-operator go mod init application-operator kubebuilder init --domain=egonlin.com --owner egonlin # 创建api $ kubebuilder create api --group apps --version v1 --kind Application # 设定的kind的首字母必须大写 Create Resource [y/n] y Create Controller [y/n] y # --kind Application，指定你要创建的resource type的名字，注意首字母必须大写#项目地址直接拉取 https://gitee.com/axzys/mysqlcluster-operator/tree/slave/四、可以先在本地测试执行# 一、修改文件：文件utils.go #1、文件开头增加导入："k8s.io/client-go/tools/clientcmd" 删除导入："k8s.io/client-go/rest" #2、方法execCommandOnPod修改 config, err := clientcmd.BuildConfigFromFlags("", KubeConfigPath) // 打开注释 // config, err := rest.InClusterConfig() // 加上注释 # 二、mysqlcluster_controller.go修改 const ( ...... KubeConfigPath = "/root/.kube/config" // 打开注释 ...... ) # 并且确保宿主机上存在/root/.kube/config # 测试yaml apiVersion: apps.egonlin.com/v1 kind: MysqlCluster metadata: name: mysqlcluster-sample labels: app.kubernetes.io/name: mysql-operator app.kubernetes.io/managed-by: kustomize spec: image: registry.cn-shanghai.aliyuncs.com/egon-k8s-test/mysql:5.7 replicas: 4 masterService: master-service slaveService: slave-service storage: storageClassName: "local-path" size: 1Gi resources: requests: cpu: "500m" memory: "1Gi" limits: cpu: "1" memory: "2Gi" livenessProbe: initialDelaySeconds: 30 timeoutSeconds: 5 tcpSocket: port: 3306 先执行make install 然后执行 make run 然后创建测试pod创建测试功能正常以后。可以把控制器放进k8s里面。五、以容器形式部署controller如果想要部署在k8s里面需要把上面修改的配置还原回去。# dockerfile文件中的FROM镜像无法拉取，要换成自己的 $ vi Dockerfile # FROM golang:1.22 AS builder FROM registry.cn-hangzhou.aliyuncs.com/egon-k8s-test/golang:1.22 AS builder #FROM gcr.io/distroless/static:nonroot FROM registry.cn-shanghai.aliyuncs.com/egon-k8s-test/static:nonroot #并且构建过程中需要执行go mod download，默认从国外源下载非常慢需要再该命令前设置好环境变量 # 在go mod download前设置好环境变量 ENV GOPROXY=https://mirrors.aliyun.com/goproxy/,direct RUN go mod download 然后构建 docker 镜像make docker-build IMG=mysql-operator-master:v0.01 #然后启动推上阿里云仓库# 使用 docker 镜像, 部署 controller 到 k8s 集群，会部署成一个deployment make deploy IMG=registry.cn-guangzhou.aliyuncs.com/xingcangku/bendi:v0.8#查询: 默认在system名称空间下 [root@master01 mysql-operator-master]# kubectl get namespace NAME STATUS AGE application-operator-system Active 3d default Active 23d kube-flannel Active 23d kube-node-lease Active 23d kube-public Active 23d kube-system Active 23d monitor Active 22d system Active 36s [root@master01 mysql-operator-master]# kubectl -n system get api/ cmd/ Dockerfile .git/ .golangci.yml go.sum internal/ PROJECT test/ bin/ config/ .dockerignore .gitignore go.mod hack/ Makefile README.md test.yaml [root@master01 mysql-operator-master]# kubectl -n system get deployments.apps NAME READY UP-TO-DATE AVAILABLE AGE controller-manager 1/1 1 1 52s [root@master01 mysql-operator-master]# kubectl -n controller-manager get pods No resources found in controller-manager namespace. [root@master01 mysql-operator-master]# kubectl delete -f ./config/samples/apps_v1_mysqlcluster.yaml Error from server (NotFound): error when deleting "./config/samples/apps_v1_mysqlcluster.yaml": mysqlclusters.apps.egonlin.com "mysqlcluster-sample" not found [root@master01 mysql-operator-master]# kubectl apply -f ./config/samples/apps_v1_mysqlcluster.yaml mysqlcluster.apps.egonlin.com/mysqlcluster-sample created [root@master01 mysql-operator-master]# kubectl -n controller-manager get pods No resources found in controller-manager namespace. [root@master01 mysql-operator-master]# kubectl get pods -n system NAME READY STATUS RESTARTS AGE controller-manager-5699b5b476-4ngwd 1/1 Running 0 3m3s# 如果发现pod没有起来可能是存储的问题。项目来面有个文件local-path-provisioner-0.0.29 进入然后再进入deploy这个文件 [root@master01 deploy]# kubectl apply -f local-path-storage.yaml namespace/local-path-storage created serviceaccount/local-path-provisioner-service-account created role.rbac.authorization.k8s.io/local-path-provisioner-role created clusterrole.rbac.authorization.k8s.io/local-path-provisioner-role created rolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created clusterrolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created deployment.apps/local-path-provisioner created storageclass.storage.k8s.io/local-path created configmap/local-path-config created [root@master01 deploy]# kubectl get pods NAME READY STATUS RESTARTS AGE axing-zzz-7d5cb7df74-4lbqn 1/1 Running 6 (31m ago) 16d mysql-01 1/1 Running 0 7m50s mysql-02 1/1 Running 0 40s mysql-03 0/1 ContainerCreating 0 30s [root@master01 deploy]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE mysql-01 Bound pvc-c4ffa04d-78bc-44e5-9948-8dd23e8197d4 1Gi RWO local-path <unset> 8m4s mysql-02 Bound pvc-9870b7dc-274f-48d9-ab9c-12fdad4ab267 1Gi RWO local-path <unset> 8m4s mysql-03 Bound pvc-517035dc-ec28-4733-8d8d-244cce025604 1Gi RWO local-path <unset> 8m4s [root@master01 mysql-operator-master]# kubectl get pod -n system 'NAME READY STATUS RESTARTS AGE controller-manager-5699b5b476-4ngwd 1/1 Running 0 103m [root@master01 mysql-operator-master]# [root@master01 mysql-operator-master]# kubectl -n system get deployments.apps NAME READY UP-TO-DATE AVAILABLE AGE controller-manager 1/1 1 1 103m # 可以看日志的情况 [root@master01 mysql-operator-master]# kubectl -n system logs -f controller-manager-5699b5b476-4ngwd正常最后是会一直更新日志{lamp/}最后问题总结# 启动operator的时候第三个pod无法拉起，一直pending，查看 [root@k8s-node-01 ~]# kubectl describe pod mysql-03 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 11m (x3 over 17m) default-scheduler 0/3 nodes are available: 1 Insufficient cpu, 1 node(s) had untolerated taint {node.kubernetes.io/disk-pressure: }, 2 Insufficient memory. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod. Warning FailedScheduling 89s (x2 over 6m30s) default-scheduler 0/3 nodes are available: 3 node(s) had untolerated taint {node.kubernetes.io/unreachable: }. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling. [root@k8s-node-01 ~]## 报错磁disk磁盘资源不足，因为我们用的存储卷是local-path-storage，所以会有卷亲和，msyql-03固定调度到卷所在的节点，卷所在的节点为k8s-node-01节点，通过查看也能分析出来 [root@k8s-node-01 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mysql-01 1/1 Running 0 18m 10.244.0.103 k8s-master-01 <none> <none> mysql-02 1/1 Running 0 18m 10.244.2.184 k8s-node-02 <none> <none> mysql-03 0/1 Pending 0 18m <none> <none> <none> <none> # 于是去k8s-node-01节点上查看，发现磁盘空间确实占满了，如下先尝试把该节点的一些安装包，/tmp目录，yum缓存，/var/log都清理掉 kubelet的日志轮转也设置了 [root@k8s-node-01 ~]# cat /var/lib/kubelet/kubeadm-flags.env KUBELET_KUBEADM_ARGS="--container-runtimeendpoint=unix:///var/run/containerd/containerd.sock --pod-infra-containerimage=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9 --containerlog-max-files=2 --container-log-max-size='1Ki'" # 注意：--container-log-max-files=2必须大于1，不能小于或等于1，否则无法启动# go build缓存（/root/.cache）还是别清了，否则make run或花很久时间 # 并且把一些没有用的镜像也清理掉 docker system prune -a nerdctl system prune -a # 作用解释： system prune：这个命令用于清理 Docker 系统，删除不再使用的容器、镜像、网络等资源。 -a（--all）：此选项会使命令删除所有未使用的镜像，而不仅仅是无标签的镜像。 运行 docker/nerdctl system prune -a 后，系统会问你是否确认要删除这些资源。确认后，Docker会清理掉停止的容器、未使用的镜像和网络，从而释放磁盘空间。发现空间得到了一定程度的释放查看已删除但仍被占用的文件 当一个文件被删除后，如果有进程仍然在使用它，那么这个文件所占用的空间并不会立即被释放。文件 系统的空间使用会显示为已用，但 du 无法检测到这些被删除的文件。 检测被删除但仍然占用的文件 可以使用 lsof 来列出所有仍然被进程占用但已删除的文件。 lsof | grep deleted 如果发现某些文件已经被删除，但仍然被进程占用，可以通过重启相应的进程来释放这些文件占用的空间。 发现一堆这种文件查找该进程，发现就是一个裸启动的mysql进程，无用，可以kill杀掉kill -9 1100
-
gitee使用 1. mkdir mysqlcluster-operator #创建一个名为 mysqlcluster-operator 的目录。 2. cd mysqlcluster-operator #进入刚刚创建的 mysqlcluster-operator 目录。 3. git init #初始化一个新的 Git 仓库，将当前目录标记为一个 Git 仓库以便进行版本控制。 4. touch README.md #创建一个名为 README.md 的文件，通常用于项目的介绍或说明。 5. git add README.md #将 README.md 文件添加到 Git 的暂存区，准备提交到仓库。 6. git commit -m "first commit" #提交暂存区中的文件，并添加提交信息 "first commit"。 7. git remote add origin https://gitee.com/axzys/mysqlcluster-operator.git #将远程仓库的地址（在 Gitee 上）与当前的本地仓库关联，命名为 origin。 8. git push -u origin "master" #将本地仓库的 master 分支推送到远程仓库的 origin，并设置跟踪（-u 参数表示在未来自动与这个远程分支同步）。mkdir webhook cd webhook git init touch README.md git add README.md git commit -m "first commit" git remote add origin https://gitee.com/axzys/webhook.git git push -u origin "master"[root@master01 mysqlcluster-operator]# git push -u origin "master" Username for 'https://gitee.com': 13143087657 Password for 'https://13143087657@gitee.com': Counting objects: 3, done. Delta compression using up to 8 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 1.64 KiB | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: Powered by GITEE.COM [1.1.5] remote: Set trace flag 7e4992ae To https://gitee.com/axzys/mysqlcluster-operator.git * [new branch] master -> master Counting objects：统计需要推送的对象数目。 Compressing objects：压缩要推送的对象。 Writing objects：将对象写入远程仓库。 remote: Powered by GITEE.COM：表示已经成功连接到 Gitee 并完成操作。 [new branch] master -> master：本地的 master 分支已经成功推送到远程仓库的 master 分支。git add . #添加修改 git commit -m "Your commit message" #提交修改 git push origin master #推送到远程仓库 #推送到其他分支 git checkout -b <branch_name> #创建并切换到新分支 git push origin <branch_name> #推送到新分支 #回滚到之前的提交并推送（如果需要重置远程仓库的状态） git reset --hard abc1234 #回滚到某个提交（假设提交ID为 abc1234） git push origin master --force #强制推送到远程仓库
-
安装EFK 一、 准备规划（1）先创建一个名称空间，后续日志相关组件都安装到该名称空间下 kubectl create ns logging（2）环境准备ElasticSearch 安装有最低安装要求，如果安装后 Pod 无法正常启动，请检查是否符合最低要求的配置，要求如下：建议：每台机器cpu调成4c内存>=4G我的是3台master节点，一个node节点，每台都可以参与调度，每台机器都是4G注意：如果使用vmware workstation做实验，你的虚拟机是开机状态，最大把虚拟机内存调大到3G，想要调的大一些需要关机后才能调的更大（3）部署规划ES集群有三种角色组成，详细描述与规划如下二、为ES准备持久化存储为了能够持久化Elasticsearch的数据，需要准备一个存储，此处我们使用NFS类型的 StorageClass ，如果你是线上环境建议使用 Local PV 或者 Ceph RBD（1） 安装nfs服务找一台机器192.168.110.101安装nfs服务端systemctl stop firewalld.service systemctl disable firewalld.service # 服务端软件安装: 192.168.71.101 yum install -y nfs-utils rpcbind # 安装nfs-utils和rpcbind两个包 # 创建共享目录 mkdir -p /data/nfs chmod 755 /data/nfs # 配置共享目录 cat > /etc/exports <<EOF /data/nfs *(rw,sync,no_root_squash) EOF *：表示任何人都有权限连接，当然也可以是一个网段，一个 IP，也可以是域名 rw：读写的权限 sync：表示文件同时写入硬盘和内存 no_root_squash：当登录 NFS 主机使用共享目录的使用者是 root 时，其权限将被转换成为匿名使用者，通常它的 UID 与 GID，都会变成 nobody 身份 # 启动nfs服务 systemctl start rpcbind.service systemctl enable rpcbind systemctl status rpcbind systemctl start nfs systemctl enable nfs systemctl status nfs # 如下显示则ok $ rpcinfo -p|grep nfs 100003 3 tcp 2049 nfs 100003 4 tcp 2049 nfs 100227 3 tcp 2049 nfs_acl 100003 3 udp 2049 nfs 100227 3 udp 2049 nfs_acl客户端软件安装(在所有node节点安装)yum install -y nfs-utils你可以在客户端宿主机上验证下能不能用，能用的话我们一会再用到pod的pv上➜ showmount -e 192.168.71.101 Export list for 192.168.71.101: /data/nfs * ➜ mount -t nfs 192.168.71.101:/data/nfs /mnt #可千万别挂载到/opt下哈，/opt/cni/bin放着网络插件呢 ➜ touch /mnt/a.txt # 成功后，另外一台客户端上挂载查看验证（2） 搭建StorageClass+NFS官网：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner➜ helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/ ➜ helm upgrade --install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner --set nfs.server=192.168.110.101 --set nfs.path=/data/nfs --set storageClass.defaultClass=true -n kube-system # 镜像下载不了的话可以用自己制作的镜像 registry.cn-shanghai.aliyuncs.com/egon-k8s-test/nfs-subdir-external-provisioner:v4.0.2查看helm -n kube-system list查看nfs_provider的pod$ kubectl -n kube-system get pods |grep nfs nfs-subdir-external-provisioner-5c7dc6cd57-4mrmx 1/1 Running 0 18s查看sc（已经设置为默认的了）➜ kubectl -n kube-system get sc nfs-client NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION nfs-client (default) cluster.local/nfs-subdir-external-provisioner Delete三、为ES准备证书文件由于 ElasticSearch 7.x 版本默认安装了 X-Pack 插件，需要我们配置一些安全证书文件。（1）生成证书文件# 运行容器生成证书，containerd下面用nerdctl $ mkdir -p /logging/elastic-certs $ nerdctl run --name elastic-certs \ -v /logging/elastic-certs:/app \ -it \ -w /app \ registry.cn-guangzhou.aliyuncs.com/xingcangku/oooooo:1.0 \ /bin/sh -c " elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass '' && \ elasticsearch-certutil cert --name security-master --dns security-master \ --ca /app/elastic-stack-ca.p12 --pass '' --ca-pass '' --out /app/elastic-certificates.p12 " # 删除容器 $ nerdctl rm -f elastic-certs # 将 pcks12（Public Key Cryptography Standards #12） 文件中的证书和私钥提取出来，并保存为 PEM 格式的文件 #$ cd /logging/elastic-certs #$ cd elastic-certs && openssl pkcs12 -nodes -passin pass:'' -in elastic-certificates.p12 -out elastic-certificate.pem（2）添加证书到 Kubernetes# 添加证书 $ cd /logging/elastic-certs $ kubectl create secret -n logging generic elastic-certs --from-file=elasticcertificates.p12 # 设置集群用户名密码，用户名为elastic，密码为egon666 $ kubectl create secret -n logging generic elastic-auth --fromliteral=username=elastic --from-literal=password=egon666四、安装ES集群首先添加 ELastic 的 Helm 仓库：$ helm repo add elastic https://helm.elastic.co $ helm repo updateElaticSearch 安装需要安装三次，分别安装 Master、Data、Client 节点，Master 节点负责集群间的管理工作；Data 节点负责存储数据；Client 节点负责代理 ElasticSearch Cluster 集群，负载均衡。首先使用 helm pull 拉取 Chart 并解压：$ helm pull elastic/elasticsearch --untar --version 7.17.3 $ cd elasticsearch在 Chart 目录下面创建用于 Master 节点安装配置的 values 文件：(默认自带的values.yaml不用管，我们不用它)# 创建一个新文件：values-master.yaml，内容如下 ## 设置集群名称 clusterName: 'elasticsearch' ## 设置节点名称 nodeGroup: 'master' ## 设置角色 roles: master: 'true' ingest: 'false' data: 'false' # ============ 镜像配置 ============ ## 指定镜像与镜像版本 image: 'registry.cn-hangzhou.aliyuncs.com/egon-k8s-test/elasticsearch' # 可以用自己的镜像，你的镜像仓库必须是公开的 # image: 'elasticsearch' imageTag: '7.17.3' imagePullPolicy: 'IfNotPresent' ## 副本数 # replicas: 3 # 测试环境资源有限，所以设置为1 replicas: 1 # ============ 资源配置 ============ ## JVM 配置参数 esJavaOpts: '-Xmx1g -Xms1g' ## 部署资源配置(生产环境要设置大些) resources: requests: cpu: '2000m' memory: '2Gi' limits: cpu: '2000m' memory: '2Gi' ## 数据持久卷配置 persistence: enabled: true ## 存储数据大小配置 volumeClaimTemplate: storageClassName: nfs-client accessModes: ['ReadWriteOnce'] resources: requests: storage: 5Gi # ============ 安全配置 ============ ## 设置协议，可配置为 http、https protocol: http ## 证书挂载配置，这里我们挂入上面创建的证书 secretMounts: - name: elastic-certs secretName: elastic-certs path: /usr/share/elasticsearch/config/certs defaultMode: 0755 ## Elasticsearch 配置 esConfig: elasticsearch.yml: | xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 # xpack.security.http.ssl.enabled: true # xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 # xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 ## 环境变量配置，这里引入上面设置的用户名、密码 secret 文件 extraEnvs: - name: ELASTIC_USERNAME valueFrom: secretKeyRef: name: elastic-auth key: username - name: ELASTIC_PASSWORD valueFrom: secretKeyRef: name: elastic-auth key: password # ============ 调度配置 ============ ## 设置调度策略 ## - hard：只有当有足够的节点时 Pod 才会被调度，并且它们永远不会出现在同一个节点上 ## - soft：尽最大努力调度 antiAffinity: 'soft' # tolerations: # - operator: "Exists" # 容忍全部污点 # 创建新文件：values-client.yaml # ============ 设置集群名称 ============ ## 设置集群名称 clusterName: 'elasticsearch' ## 设置节点名称 nodeGroup: 'client' ## 设置角色 roles: master: 'false' ingest: 'false' data: 'false' # ============ 镜像配置 ============ ## 指定镜像与镜像版本 image: 'registry.cn-hangzhou.aliyuncs.com/egon-k8s-test/elasticsearch' # 可以用自己的镜像 # image: 'elasticsearch' imageTag: '7.17.3' ## 副本数 # 测试环境资源有限，所以设置为1吧 replicas: 1 # ============ 资源配置 ============ ## JVM 配置参数 esJavaOpts: '-Xmx1g -Xms1g' ## 部署资源配置(生产环境一定要设置大些) resources: requests: cpu: '1000m' memory: '2Gi' limits: cpu: '1000m' memory: '2Gi' ## 数据持久卷配置 persistence: enabled: false # ============ 安全配置 ============ ## 设置协议，可配置为 http、https protocol: http ## 证书挂载配置，这里我们挂入上面创建的证书 secretMounts: - name: elastic-certs secretName: elastic-certs path: /usr/share/elasticsearch/config/certs ## 自定义配置文件 elasticsearch.yml esConfig: elasticsearch.yml: | xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 # xpack.security.http.ssl.enabled: true # xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 # xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 ## 环境变量配置，引入上面设置的用户名、密码 secret 文件 extraEnvs: - name: ELASTIC_USERNAME valueFrom: secretKeyRef: name: elastic-auth key: username - name: ELASTIC_PASSWORD valueFrom: secretKeyRef: name: elastic-auth key: password # ============ Service 配置 ============ service: type: NodePort nodePort: '30200'我的集群资源不足，我将上述三个yaml副本都设置成了1现在用上面的 values 文件来安装：（切换到elasticsearch的chart目录下）[root@master01 ~]# cd /logging/elasticsearch/ [root@master01 /logging/elasticsearch]# ls Chart.yaml Makefile templates values-data.yaml values.yaml examples README.md values-client.yaml values-master.yaml # --------------->>>>>>>>>>> 注意install指定的release名字要不同哦 # helm install 你起的release名 你的chart包的路径 -f xxx.yaml --namespace yyy # 如果是升级安装则用：helm upgrade --install 你起的release名 你的chart包的路径 -f values-master.yaml --namespace logging . cd elasticsearch/ # 安装 master 节点 helm install es-master ./ -f values-master.yaml --namespace logging # 安装 data 节点 helm install es-data ./ -f values-data.yaml --namespace logging # 安装 client 节点 helm install es-client ./ -f values-client.yaml --namespace logging # 升级操作 #$ helm upgrade --install es-master -f values-master.yaml --namespace logging . #$ helm upgrade --install es-data -f values-data.yaml --namespace logging . #$ helm upgrade --install es-client -f values-client.yaml --namespace logging .如果你的es-master设置的replicas副本数为3，那么在安装 Master 节点后 Pod 启动时候会抛出异 常，就绪探针探活失败， $ kubectl -n logging describe pod elasticsearch-master-0 Warning Unhealthy 61s kubelet Readiness probe failed: Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" ) Cluster is not yet ready (request params: "wait_for_status=green&timeout=1s" ) 这是个正常现象。在执行安装 Data 节点后 Master 节点 Pod 就会恢复正常。 [root@master01 /logging/elasticsearch]# kubectl -n logging get pods -w NAME READY STATUS RESTARTS AGE elasticsearch-master-0 1/1 Running 0 2m17s elasticsearch-master-1 1/1 Running 0 2m13s elasticsearch-master-2 1/1 Running 0 2m20s 此外，如果因为资源原因导致出现pending状态的pod，请根据describe信息扩容cpu或内存资源查看master（建议3副本，但是我的资源有限，我就启了一个副本）[root@master01 /logging/elasticsearch]# kubectl -n logging get pods -w NAME READY STATUS RESTARTS AGE elasticsearch-client-0 1/1 Running 0 7m17s elasticsearch-data-0 1/1 Running 0 7m21s elasticsearch-master-0 1/1 Running 0 9m16s一会访问es就用用该svc[root@master01 harbor]# kubectl -n logging get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE elasticsearch-client NodePort 10.111.16.229 <none> 9200:30200/TCP,9300:30969/TCP 23h五、安装Kibanahelm pull elastic/kibana --untar --version 7.17.3 # 下载并解压chart包 cd kibana创建用于安装 Kibana 的 values 文件：# 创建全新文件：values-prod.yaml # ============ 镜像配置 ============ ## 指定镜像与镜像版本 image: 'registry.cn-hangzhou.aliyuncs.com/egon-k8s-test/kibana' # image: 'docker.elastic.co/kibana/kibana' imageTag: '7.17.3' imagePullPolicy: "IfNotPresent" # ============ ElasticSearch 配置 ============ ## 配置 ElasticSearch 地址，主要使用 es-client 的 SVC elasticsearchHosts: 'http://elasticsearch-client:9200' # ============ 环境变量配置 ============ ## 引入用户名、密码的 secret 文件 extraEnvs: - name: 'ELASTICSEARCH_USERNAME' valueFrom: secretKeyRef: name: elastic-auth key: username - name: 'ELASTICSEARCH_PASSWORD' valueFrom: secretKeyRef: name: elastic-auth key: password # ============ 资源配置 ============ resources: requests: cpu: '500m' memory: '1Gi' limits: cpu: '500m' memory: '1Gi' # ============ Kibana 参数配置 ============ ## 添加语言配置，设置 Kibana 为中文 kibanaConfig: kibana.yml: | i18n.locale: "zh-CN" server.publicBaseUrl: "http://192.168.110.101:30601" # 将地址改为你访问 Kibana 的地址，不能以 / 结尾 # ============ Service 配置 ============ service: type: NodePort nodePort: '30601'部署helm install kibana -f values-prod.yaml --namespace logging .部署完后查看[root@master01 /]# kubectl -n logging get pods -w NAME READY STATUS RESTARTS AGE elasticsearch-client-0 0/1 Running 0 2m3s elasticsearch-data-0 1/1 Running 0 4m35s elasticsearch-master-0 1/1 Running 0 4m45s elasticsearch-client-0 1/1 Running 0 2m20s上面我们安装 Kibana 的时候指定了 30601 的 NodePort 端口，所以我们可以从任意节点http://IP:30601 来访问 Kibana。我们可以看到会跳转到登录页面，让我们输出用户名、密码，这里我们输入上面配置的用户名elastic 、密码 egon666 进行登录。登录成功后进入如下所示的 Kibana 主页(点击自己浏览)：六、安装Fluentd来作为日志收集工具（1）安装fluentd要想在k8s每个物理节点都能采集到数据，们可以直接用 DasemonSet 控制器来部署 Fluentd 应用，确保在集群中的每个节点上始终运行一个 Fluentd 容器可以直接使用 Helm 来进行一键安装，为了能够了解更多实现细节，我们这里还是采用手动方法来进行安装。官网部署参考：https://docs.fluentd.org/container-deployment/kubernetes。
