云原生HBaseonk8s编排部署讲解与实战操作

　　一、概述
　　HBase   是一个面向列式存储的分布式数据库，其设计思想来源于 Google 的 BigTable 论文。HBase 底层存储基于 HDFS 实现，集群的管理基于 ZooKeeper 实现。HBase 良好的分布式架构设计为海量数据的快速存储、随机访问提供了可能，基于数据副本机制和分区机制可以轻松实现在线扩容、缩容和数据容灾，是大数据领域中 Key-Value 数据结构存储最常用的数据库方案。
　　官方文档：https://hbase.apache.org/book.html GitHub地址：https://github.com/apache/hbase 关于更多hbase的介绍，也可以参考我这篇文章：列式存储的分布式数据库——HBase（环境部署）
　　因为hbase依赖于HDFS存储，关于HDFS容器化部署可以参考我以下文章：【云原生】Hadoop on k8s 环境部署
　　但是生产环境一般使用高可用的Hadoop的，关于高可用部署可参考我这篇文章：【云原生】Hadoop HA on k8s 环境部署
　　hbase依赖于Zookeeper，zookeeper容器化部署可以参考：【云原生】zookeeper + kafka on k8s 环境部署
　　二、开始编排部署（非高可用HDFS）
　　地址：https://artifacthub.io/packages/helm/hbase/hbase 1）下载chart 包helm repo add hbase https://itboy87.github.io/bigdata-charts/  # hbase version 2.4.13 helm pull hbase/hbase --version 0.1.72）构建镜像
　　在下面连接hadoop高可用会重新构建镜像，这里就不重新构建镜像了，只是把远程的包推送到本地harbor仓库 docker pull ghcr.io/fleeksoft/hbase/hbase-base:2.4.13.2  # tag docker tag ghcr.io/fleeksoft/hbase/hbase-base:2.4.13.2 myharbor.com/bigdata/hbase-base:2.4.13.2  # push docker push myharbor.com/bigdata/hbase-base:2.4.13.23）修改yaml编排（非高可用HDFS）hbase/values.yaml  image:   repository: myharbor.com/bigdata/hbase-base   tag: 2.4.13.2   pullPolicy: IfNotPresent  ...  conf:   hadoopUserName: admin   hbaseSite:     hbase.rootdir:  ＂hdfs://hadoop-hadoop-hdfs-nn.hadoop:9000/hbase＂     hbase.zookeeper.quorum:  ＂zookeeper.zookeeper:2181＂  ...  hbase:   master:     replicas: 2    regionServer:     replicas: 2  # 禁用内部的hadoop hadoop:   enabled: false  # 禁用内部的zookeeper zookeeper:   enabled: falsehbase/templates/hbase-configmap.yaml  if [ {{ .Values.hadoop.enabled }} = true ];then   NAMENODE_URL={{- printf ＂http://%s-hadoop-hdfs-nn:9870/index.html＂ .Release.Name }} else   hadoop_url={{ index .Values.conf.hbaseSite ＂hbase.rootdir＂ }}   hadoop_url=`echo $hadoop_url|awk -F ＂/＂ ＂{print $3}＂|awk -F＂:＂ ＂{print $1}＂`   NAMENODE_URL=http://${hadoop_url}:9870/index.html fi4）开始部署# 先检查语法 helm lint ./hbase  # 开始安装 helm install hbase ./hbase -n hbase --create-namespace
　　NOTES NAME: hbase LAST DEPLOYED: Sat Nov  5 15:44:14 2022 NAMESPACE: hbase STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: 1. You can get an HBASE Shell by running this command:    kubectl exec -n hbase -it hbase-hbase-master-0 -- hbase shell  2. Inspect hbase master service ports with:    kubectl exec -n hbase describe service hbase-hbase-master  3. Create a port-forward to the hbase manager UI:    kubectl port-forward -n hbase svc/hbase-hbase-master 16010:16010     Then open the ui in your browser:     open http://localhost:16010  4. Create a port-forward to the hbase thrift manager UI:    kubectl port-forward -n hbase svc/hbase-hbase-master 9095:9095     Then open the ui in your browser:     open http://localhost:9095
　　HDFS
　　查看 kubectl get pods,svc -n hbase -owide
　　5）测试验证
　　测试主备切换，重启当前active master pod kubectl delete pod hbase-hbase-master-0 -n hbase
　　主备能正常切换
　　6）卸载helm uninstall hbase -n hbase # delete ns  kubectl delete ns hbase --force三、开始编排部署（高可用 HDFS）1）下载chart 包helm repo add hbase https://itboy87.github.io/bigdata-charts/  # hbase version 2.4.13 helm pull hbase/hbase --version 0.1.72）构建镜像
　　这里是基于上面的镜像进行构建，只是把hadoop打包到镜像中，主要用的hadoop配置文件是 core-site.yaml  ，hdfs-site.yaml
　　Dockerfile FROM myharbor.com/bigdata/hbase-base:2.4.13.2  RUN mkdir -p /opt/apache  ENV HADOOP_VERSION=3.3.2  ADD hadoop-${HADOOP_VERSION}.tar.gz /opt/apache  ENV HADOOP_HOME=/opt/apache/hadoop  RUN ln -s /opt/apache/hadoop-${HADOOP_VERSION} $HADOOP_HOME  ENV HADOOP_CONF_DIR=${HADOOP_HOME}/et/hadoop  ENV PATH=${HADOOP_HOME}/bin:$PATH
　　开始构建 docker build -t myharbor.com/bigdata/hbase-hdfs-ha:2.4.13.2 . --no-cache  ### 参数解释 # -t：指定镜像名称 # . ：当前目录Dockerfile # -f：指定Dockerfile路径 #  --no-cache：不缓存  # 推送到harbor docker push myharbor.com/bigdata/hbase-hdfs-ha:2.4.13.23）修改配置hbase-hdfs-ha/values.yaml  image:   repository: myharbor.com/bigdata/hbase-hdfs-ha   tag: 2.4.13.2   pullPolicy: IfNotPresent  ...  conf:   hadoopUserName: admin   hbaseSite:     hbase.rootdir: ＂hdfs://myhdfs/hbase＂     hbase.zookeeper.quorum: ＂zookeeper.zookeeper:2181＂hbase-hdfs-ha/templates/hbase-configmap.yaml  if [ {{ .Values.hadoop.enabled }} = true ];then   NAMENODE_URL={{- printf ＂http://%s-hadoop-hdfs-nn:9870/index.html＂ .Release.Name }} else   NAMENODE_URL=http://hadoop-ha-hadoop-hdfs-nn-1.hadoop-ha:9870:9870/index.html fi# 先检查语法 helm lint ./hbase-hdfs-ha  # 开始安装 helm install hbase-hdfs-ha ./hbase-hdfs-ha -n hbase-hdfs-ha --create-namespace
　　NOTES NAME: hbase-hdfs-ha LAST DEPLOYED: Sat Nov  5 17:23:20 2022 NAMESPACE: hbase-hdfs-ha STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: 1. You can get an HBASE Shell by running this command:    kubectl exec -n hbase-hdfs-ha -it hbase-hdfs-ha-hbase-master-0 -- hbase shell  2. Inspect hbase master service ports with:    kubectl exec -n hbase-hdfs-ha describe service hbase-hdfs-ha-hbase-master  3. Create a port-forward to the hbase manager UI:    kubectl port-forward -n hbase-hdfs-ha svc/hbase-hdfs-ha-hbase-master 16010:16010     Then open the ui in your browser:     open http://localhost:16010  4. Create a port-forward to the hbase thrift manager UI:    kubectl port-forward -n hbase-hdfs-ha svc/hbase-hdfs-ha-hbase-master 9095:9095     Then open the ui in your browser:     open http://localhost:9095
　　HDFS
　　查看 kubectl get pods,svc -n hbase-hdfs-ha
　　5）测试验证
　　测试主备切换，重启当前active master pod kubectl delete pod hbase-hbase-master-0 -n hbase
　　主备能正常切换
　　6）卸载helm uninstall hbase-hdfs-ha -n hbase-hdfs-ha # delete ns  kubectl delete ns hbase-hdfs-ha --force
　　git 地址：https://gitee.com/hadoop-bigdata/hbase-on-k8s
　　HBase on k8s 编排部署讲解与实战操作就先到这里了，有疑问的小伙伴欢迎留言或私信，后续会持续更新 【云原生+大数据】 相关的文章，请小伙伴耐心等待~