1.简介

CDH集群使用 Kerberos 作为用户和服务的强身份验证和身份传播的基础。Kerberos可以将认证的密钥放置在集群中可靠的节点上，集群运行时，集群内的节点使用密钥得到认证，认证通过后节点才提供服务。没有事先得到密钥信息的节点，无法与集群内部节点通信。通过这种方式来防止恶意用户伪装集群节点或超级用户篡改Hadoop集群信息，从而确保Hadoop集群的可靠性和安全性。

Kerberos在CDH集群中作用是认证“自己”是自己，然后在整个 Hadoop 集群中传播该身份。完成此操作后，这些用户可以访问资源（例如文件或目录）或与集群交互（如运行 MapReduce 作业）。除了用户之外，Hadoop 集群资源本身（例如主机和服务）需要相互进行身份验证，以避免潜在的恶意系统或守护程序 “冒充” 受信任的集群组件来获取数据访问权限。

2.Kerberos安装

2.1 角色分配

2.2 kerberos相关安装包下载

1. 集群能够联网

yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2.集群不能联网

需要在一台能联网的服务器上安装yum-plugin-downloadonly插件，然后下载kerberos对应的软件包，命令如下

yum install -y --downloadonly --downloaddir=/kerberos/  krb5-server krb5-libs krb5-auth-dialog krb5-workstation

--downloadonly参数会同时将所依赖的rpm一起下载下来，然后上传到集群服务器，就可以通过rpm -ivh命令或者yum localinstall命令安装。

[root@cdh-001 ~]# yum localinstall -y --nogpgcheck \
krb5-libs-1.15.1-50.el7.x86_64.rpm \
krb5-server-1.15.1-50.el7.x86_64.rpm \
krb5-workstation-1.15.1-50.el7.x86_64.rpm \
libevent-2.0.21-4.el7.x86_64.rpm \
libkadm5-1.15.1-50.el7.x86_64.rpm \
libverto-libevent-0.2.5-4.el7.x86_64.rpm

2.3 配置文件修改

修改krb5.conf文件

[root@cdh-001 kerberos]# vim /etc/krb5.conf
[root@cdh-001 kerberos]# cat /etc/krb5.conf
# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
 default_realm = BI.COM
 #default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 BI.COM = {
  kdc = cdh-001
  admin_server = cdh-001
 }

[domain_realm]
# .example.com = EXAMPLE.COM
# example.com = EXAMPLE.COM

修改kadm5.acl文件

[root@cdh-001 kerberos]# vim /var/kerberos/krb5kdc/kadm5.acl
[root@cdh-001 kerberos]# cat /var/kerberos/krb5kdc/kadm5.acl

*/admin@BI.COM        *

修改kdc.conf配置文件

[root@cdh-001 kerberos]# vim /var/kerberos/krb5kdc/kdc.conf
[root@cdh-001 kerberos]# cat /var/kerberos/krb5kdc/kdc.conf

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 BI.COM = {
  #master_key_type = aes256-cts
  max_renewable_life = 7d 0h 0m 0s
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

2.4 创建kerberos数据库

[root@cdh-001 kerberos]# vim /var/kerberos/krb5kdc/kdc.conf
[root@cdh-001 kerberos]# cat /var/kerberos/krb5kdc/kdc.conf

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 BI.COM = {
  #master_key_type = aes256-cts
  max_renewable_life = 7d 0h 0m 0s
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

2.5创建Kerberos管理员账号

[root@cdh-001 kerberos]# /usr/sbin/kadmin.local -q "addprinc admin/admin"
Authenticating as principal root/admin@BI.COM with password.
WARNING: no policy specified for admin/admin@BI.COM; defaulting to no policy
Enter password for principal "admin/admin@BI.COM": 
Re-enter password for principal "admin/admin@BI.COM": 
Principal "admin/admin@BI.COM" created.

账户为admin，配置时密码一定要记住、

2.6 配置开机自启动

[root@cdh-001 kerberos]# systemctl enable krb5kdc
Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service.
[root@cdh-001 kerberos]# systemctl enable kadmin
Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service.
[root@cdh-001 kerberos]# systemctl start krb5kdc
[root@cdh-001 kerberos]# systemctl start kadmin

现在KDC已经在工作了。这两个daemons将会在后台运行，可以查看它们的日志文件

（ /var/log/krb5kdc.log 和 /var/log/kadmind.log)

2.7 测试Kerberos管理员账号

[root@cdh-001 kerberos]# kinit admin/admin@BI.COM
Password for admin/admin@BI.COM: 
[root@cdh-001 kerberos]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin/admin@BI.COM

Valid starting       Expires              Service principal
03/22/2021 10:58:48  03/23/2021 10:58:48  krbtgt/BI.COM@BI.COM
        renew until 03/29/2021 10:58:48

2.8 Kerberos客户端安装

注：KDC所在节点客户端已经安装，只在剩余节点安装即可

[root@cdh-006 kerberos]# yum localinstall -y --nogpgcheck \
> krb5-libs-1.15.1-50.el7.x86_64.rpm \
> krb5-workstation-1.15.1-50.el7.x86_64.rpm \
> libkadm5-1.15.1-50.el7.x86_64.rpm
.............................
Complete!
  # 查看安装情况
[root@cdh-006 kerberos]# rpm -qa | grep krb5
krb5-libs-1.15.1-50.el7.x86_64
krb5-workstation-1.15.1-50.el7.x86_64

2.9 KDC服务器安装OpenLDAP客户端

[root@cdh-001 kerberos]# yum localinstall -y --nogpgcheck \
> openldap-2.4.44-22.el7.x86_64.rpm \
> openldap-clients-2.4.44-22.el7.x86_64.rpm

2.10 配置文件分发

把krb5.conf分发到各个节点上

[root@cdh-001 kerberos]# scp -P18822 -r /etc/krb5.conf root@192.168.xxx.xxx:/etc/
root@192.168.xxx.xxx's password: 
krb5.conf 

2.11 为CM创建kerberos账号

[root@cdh-001 kerberos]# kadmin.local
Authenticating as principal admin/admin@BI.COM with password.
kadmin.local:  addprinc cloudera-scm/admin@BI.COM
WARNING: no policy specified for cloudera-scm/admin@BI.COM; defaulting to no policy
Enter password for principal "cloudera-scm/admin@BI.COM": 
Re-enter password for principal "cloudera-scm/admin@BI.COM": 
Principal "cloudera-scm/admin@BI.COM" created.
kadmin.local:  exit

账号：cloudera-scm/admin，密码: 123,后续配置会用。

查看是否创建成功

[root@cdh-001 kerberos]# kadmin.local -q "list_principals"
Authenticating as principal admin/admin@BI.COM with password.
K/M@BI.COM
admin/admin@BI.COM
cloudera-scm/admin@BI.COM
kadmin/admin@BI.COM
kadmin/changepw@BI.COM
kadmin/cdh-001@BI.COM
kiprop/cdh-001@BI.COM
krbtgt/BI.COM@BI.COM

2.12 在CM管理页面启用Kerberos

进入Cloudera Manager的管理->安全界面，选择启用Kerberos

配置引导的欢迎页由于已进行过配置，全部打钩，点击继续，配置KDC相关的信息。

进入管理krb5.conf页面，不做任何变动（不让CM管理krb5.conf）,继续

进入 KDC Account Manager 凭证页面，输入用户名和密码（cloudera-scm/admin，123）

一直点击继续直到完成。

集群重启完成，点击“继续”，点击“完成”，至此已成功配置并启用Kerberos。

3.可能的报错

3.1 kinit: Client not found in Kerberos database while getting initial credentials

# kinit登录到该账号
[root@cdh-001 kerberos]# kinit cloudera-scm/admin@BI.COM
Password for cloudera-scm/admin@BI.COM:
# klist查看当前账号的状态，没问题
[root@cdh-001 kerberos]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: cloudera-scm/admin@BI.COM
Valid starting Expires Service principal
03/21/2021 19:03:31 03/22/2021 19:03:31 krbtgt/BI.COM@BI.COM
renew until 03/28/2021 19:03:31
## 列出所有用户也有。。。
[root@cdh-001 kerberos]# kadmin.local -q "list_principals"
Authenticating as principal cloudera-scm/admin@BI.COM with password.
K/M@BI.COM
admin/admin@BI.COM
cloudera-scm/admin@BI.COM
kadmin/admin@BI.COM
kadmin/cdh-001@BI.COM
kadmin/changepw@BI.COM
kiprop/cdh-001@BI.COM
krbtgt/BI.COM@BI.COM

排查发现是前面KDC Account Manager 凭证页面输入的用户名为cloudera-scm，应改为cloudera-scm/admin。

3.2 Kafka MorrirMaker数据同步中断

该集群提前配置了Mirror Maker ，用于同步另一个集群kafka的数据，但是配置好kerberos后数据中断。

查看kafka的broker log（路径/var/log/kafka/kafka-broker-cdh-001.log）

发现有如下报错

2021-03-22 13:09:45,304 WARN org.apache.kafka.clients.NetworkClient: [Controller id=217, targetBrokerId=218] Connection to node 218 (cdh-002/192.168.xxx.xxx:9092) could not be established. Broker may not 
be available.
2021-03-22 13:09:45,305 WARN kafka.controller.RequestSendThread: [RequestSendThread controllerId=217] Controller 217's connection to broker cdh-002:9092 (id: 218 rack: null) was unsuccessful
java.io.IOException: Connection to cdh-002:9092 (id: 218 rack: null) failed.
        at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:71)
        at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:282)
        at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:236)
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
2021-03-22 13:09:45,375 WARN org.apache.kafka.clients.NetworkClient: [Controller id=217, targetBrokerId=216] Connection to node 216 (cdh-003/192.168.xxx.xxx:9092) could not be established. Broker may not 
be available.

进入CM kafka的配置页面修改如下配置

再次查看kafka topic数据恢复正常。

3.3 Impala无法正常启动

Impala开启了Kerberos之后无法正常启动

在管理-kerberos-Kerberos加密类型中添加如下

des3-hmac-sha1
arcfour-hmac
des-hmac-sha1
des-cbc-md5
des-cbc-crc

重新部署客户端，重启集群，查看，如果仍然异常退出，将以下包拷贝包Impala相关节点。安装，重启Impala服务。

rpm -ivh cyrus-sasl-devel-2.1.26-23.el7.x86_64.rpm --nodeps
rpm -ivh  cyrus-sasl-gssapi-2.1.26-23.el7.x86_64.rpm --nodeps
rpm -ivh cyrus-sasl-plain-2.1.26-23.el7.x86_64.rpm --nodeps

再次在CM查看Impala，运行正常。