etcd支持通过TLS协议的加密通讯,在实际企业生产环境中,出于安全规范要求,建议开启TLS加密。TLS通道可以用于加密内部的集群通讯,也可以用于加密客户端请求。
etcd的TLS有两对,一对是etcd和client端的TLS配置。一对是etcd之间的peer的TLS配置。有很多方式可以创建CA证书和私钥,其中比较流行的有两种
openssl cfssl
我们目前规划集群信息如下:
192.168.1.76 etcd1 192.168.1.77 etcd2 192.168.1.79 etcd3
官方文档推荐使用cfssl生成证书,所以我们这里今天使用cfssl进行加密,具体操作如下
首先我们需要下载cfssl和csffljson两个工具
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.3/cfssl_1.6.3_linux_amd64 wget https://github.com/cloudflare/cfssl/releases/download/v1.6.3/cfssljson_1.6.3_linux_amd64 mv cfssl_1.6.3_linux_amd64 /usr/bin/cfssl mv cfssljson_1.6.3_linux_amd64 /usr/bin/cfssljson chmod +x /usr/bin/{cfssl,cfssljson} cfssl version Version: 1.6.3 Runtime: go1.18
生成etcd证书,自签证书办法机构(CA)
#生成CA证书配置 cat >ca-config.json<<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "www": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF #生成CA证书请求配置 cat >ca-csr.json<<EOF { "CN": "etcd CA", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF
生成证书,会生成ca.pem和ca-key.pem文件
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
使用自签额CA签发etcd https证书
#生成证书请求配置 cat >server-csr.json<<EOF { "CN": "etcd", "hosts": [ "127.0.0.1", "192.168.1.76", "192.168.1.77", "192.168.1.79" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] } EOF
生成证书,会生成server.pem和server-key.pem文件
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
在每个etc集群下创建配置和存储,配置目录下再创建pki证书目录
mkdir -p /etc/etcd/pki mkdir -p /data/etcd/
在etcd1上拷贝证书到/etc/etcd/pki/目录
cp ca* server* /etc/etcd/pki/
同时也将这个证书打包分发到其他etd集群节点
cd /etc/etcd tar -zcvf pki.tar.gz ./pki scp pki.tar.gz root@192.168.1.77:/root/ scp pki.tar.gz root@192.168.1.79:/root/
然后在每个etcd集群节点把证书存放到指定位置
tar -zxvf pki.tar.gz -C /etc/etcd/
接着我们在每个etcd集群节点上配置etcd的配置文件,其中192.168.1.76为我etcd1的IP,其他节点改成其他节点的IP,ETCD_NAME名字也根据节点情况进行修改
cat /etc/etcd/etcd.conf #[Member] ETCD_NAME="etcd1" # 确保集群中命名唯一 ETCD_DATA_DIR="/data/etcd" # ETCD存储目录 ETCD_LISTEN_PEER_URLS="https://192.168.1.76:2380" # 修改为本机IP ETCD_LISTEN_CLIENT_URLS="https://192.168.1.76:2379" # 修改为本机IP #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.76:2380" # 修改为本机IP ETCD_ADVERTISE_CLIENT_URLS="https://192.168.1.76:2379" # 修改为本机IP ETCD_INITIAL_CLUSTER="etcd1=https://192.168.1.76:2380,etcd2=https://192.168.1.77:2380,etcd3=https://192.168.1.79:2380" # 集群中三个etcd节点IP ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" # 所有节点token一致 ETCD_INITIAL_CLUSTER_STATE="new"
参数说明
ETCD_NAME:节点名称,集群中唯一 ETCD_DATA_DIR:数据目录 ETCD_LISTEN_PEER_URLS:集群通信监听地址 ETCD_LISTEN_CLIENT_URLS:客户端访问监听地址 ETCD_INITIAL_ADVERTISE_PEERURLS:集群通告地址 ETCD_ADVERTISE_CLIENT_URLS:客户端通告地址 ETCD_INITIAL_CLUSTER:集群节点地址 ETCD_INITIALCLUSTER_TOKEN:集群Token ETCD_INITIALCLUSTER_STATE:加入集群的当前状态,new是新集群,existing表示加入已有集群
添加etcd启动服务文件,每个节点都均要添加
cat >/usr/lib/systemd/system/etcd.service<<EOF [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/etc/etcd/etcd.conf ExecStart=/usr/local/etcd/etcd \ --cert-file=/etc/etcd/pki/server.pem \ --key-file=/etc/etcd/pki/server-key.pem \ --peer-cert-file=/etc/etcd/pki/server.pem \ --peer-key-file=/etc/etcd/pki/server-key.pem \ --trusted-ca-file=/etc/etcd/pki/ca.pem \ --peer-trusted-ca-file=/etc/etcd/pki/ca.pem \ --logger=zap Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF
设置开机启动,并启动etcd服务
systemctl daemon-reload systemctl enable etcd systemctl start etcd
查看集群成员
etcdctl --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/server.pem --key=/etc/etcd/pki/server-key.pem --endpoints="https://192.168.1.76:2379,https://192.168.1.77:2379,https://192.168.1.79:2379" member list -w table
查看集群状态
etcdctl --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/server.pem --key=/etc/etcd/pki/server-key.pem --endpoints="https://192.168.1.76:2379,https://192.168.1.77:2379,https://192.168.1.79:2379" endpoint status --cluster -w table