1、前言
ELK由Elastaicsearch 、 Logstash 和 Kibana 组合，如下图所示，Logstash 从各种不同的数据源收集数据，通过内置的管道对输入的数据进行加工。最终，这些数据会被存储到 Elastaicsearch 中供 Kibana 完成数据可视化。
2、创建Docker自定义网络 docker network create esnet
3、Elasticsearch 安装 镜像拉取 docker pull elasticsearch:7.6.2Docker运行 docker run -d --name elasticsearch --net esnet -p 9200:9200 -p 9300:9300 -e ＂discovery.type=single-node＂ elasticsearch:7.6.2配置 elasticsearch.yml //添加 http.cors.enabled: true http.cors.allow-origin: ＂*＂ http.cors.allow-headers: Authorization xpack.security.enabled: true xpack.security.transport.ssl.enabled: true //配置完成后，重启elasticsearch容器 docker exec -it elasticsearch bash 执行 bin/elasticsearch-setup-passwords interactive [root@node01 elasticsearch-7.7.0]# bin/elasticsearch-setup-passwords interactive future versions of Elasticsearch will require Java 11; your Java version from [/opt/app/jdk1.8.0_181/jre] does not meet this requirement Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user. You will be prompted to enter passwords as the process progresses. Please confirm that you would like to continue [y/N]y Enter password for [elastic]: Reenter password for [elastic]: Enter password for [apm_system]: Reenter password for [apm_system]: Enter password for [kibana]: Reenter password for [kibana]: Enter password for [logstash_system]: Reenter password for [logstash_system]: Enter password for [beats_system]: Reenter password for [beats_system]: Enter password for [remote_monitoring_user]: Reenter password for [remote_monitoring_user]: Changed password for user [apm_system] Changed password for user [kibana] Changed password for user [logstash_system] Changed password for user [beats_system] Changed password for user [remote_monitoring_user] Changed password for user [elastic]
4、Kibana 安装 镜像拉取 docker pull kibana:7.6.2docker run -d --name kibana --net esnet -p 5601:5601 kibana：7.6.2配置修改kibana yml server.name: kibana server.host: ＂0＂ elasticsearch.hosts: [ ＂http://192.168.50.16:9200＂ ] xpack.monitoring.ui.container.elasticsearch.enabled: true i18n.locale: ＂zh-CN＂ elasticsearch.username: ＂elastic＂ elasticsearch.password: ＂123456＂
5、Logstash 安装 镜像拉取 docker pull logstash:7.6.2docker run -it -d -p 5044:5044 --name logstash --net esnet logstash:7.6.2配置修改logstash yml //屏蔽掉 #http.host: ＂0.0.0.0＂ #xpack.monitoring.elasticsearch.hosts: [ ＂http://elasticsearch:9200＂ ] //宿主拷贝到logstash容器 docker cp mysql-connector-java-8.0.23.jar logstash:/usr/share/logstash/config docker cp logstash-user.conf logstash:/usr/share/logstash/pipeline logstash-user.conf修改配置 input { stdin { } jdbc { type => ＂usertrack＂ #注意mysql连接地址一定要用ip，不能使用localhost等 jdbc_connection_string => ＂jdbc:mysql://192.168.19.113:3306/test_shop?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&useSSL=false＂ jdbc_user => ＂root＂ jdbc_password => ＂123456＂ #这个jar包的地址是容器内的地址 jdbc_driver_library => ＂/usr/share/logstash/config/mysql-connector-java-8.0.23.jar＂ jdbc_driver_class => ＂com.mysql.jdbc.Driver＂ jdbc_paging_enabled => ＂true＂ statement => ＂SELECT * FROM t_user_login＂ schedule => ＂* * * * *＂ jdbc_default_timezone => ＂Asia/Shanghai＂ jdbc_page_size => ＂500＂ record_last_run => true #use_column_value => true clean_run => false } } output { stdout { codec => json_lines } elasticsearch { #注意mysql连接地址一定要用ip，不能使用localhost等 hosts => ＂192.168.50.16:9200＂ index => ＂user-%{type}-%{+YYYY.MM.dd}＂ document_id => ＂%{id}＂ user => ＂elastic＂ password => ＂123456＂ #document_type => ＂_doc＂ } } //重启logstash (docker restart logstash) 查看日志 docker logs –f logstash