搭建Spring Boot + ELK日志平台，实现可视化日志监控

1.原因每次用tail -f在服务器上翻日志太麻烦线上问题排查时有时找不到日志。本文将带你从零开始搭建一套企业级的ELK日志平台让你的日志管理变得简单、高效、可视化。传统日志管理的痛点# 登录服务器ssh userprod-server# 查看日志tail -f /var/log/app/app.log# 搜索错误grep ERROR /var/log/app/app.log# 跨服务器查日志噩梦开始...痛点总结日志分散在多台服务器排查问题需要来回切换 搜索能力有限复杂的查询难以实现无法可视化分析只能看原始文本日志文件轮转后历史日志难以追溯2.分析功能介绍ElasticsearchES分布式搜索和分析引擎负责日志的存储和索引Logstash-日志采集管道支持从多种来源采集、过滤、转换数据Kibana-可视化平台提供图表、仪表盘、日志浏览界面3.解决3.1 版本选择本文使用ELK 7.17.15这是7.x系列的最后一个稳定版本兼容性好社区文档丰富。组件版本说明Elasticsearch7.17.15长期支持版本Kibana7.17.15与ES版本保持一致Logstash7.17.15与ES版本保持一致Spring Boot3.x使用logback-logstash-encoder3.2 macOS 安装使用二进制包安装一般不管win10\mac\linux,都是官网、华为云、阿里云、清华云等镜像搜索下载。备注我的是mac安装包如果不能下载留言我上传或者发送。# 创建安装目录mkdir -p ~/elk cd ~/elk# 下载 Elasticsearch (Apple Silicon 版本)curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.15-darwin-aarch64.tar.gz# 下载 Kibanacurl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-7.17.15-darwin-x86_64.tar.gz# 下载 Logstashcurl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-7.17.15-darwin-x86_64.tar.gz# 解压for file in *.tar.gz; do tar -xzf $file; done# 重命名mv elasticsearch-7.17.15 elasticsearchmv kibana-7.17.15-darwin-x86_64 kibanamv logstash-7.17.15-darwin-x86_64 logstash3.3 启动3.3.1 启动ES:# 进入 ES 目录并后台启动 cd ~/elk/elasticsearch ./bin/elasticsearch -dlocalhost:90023.3.2 启动kibanacd ~/elk/kibananohup ./bin/kibana ~/elk/logs/kibana.log 21 localhost:56013.3.3 启动logstash先建测试包config目录下新建 springboot-logstash.confinput { file { path /tmp/spring-logs/*.log codec json start_position beginning sincedb_path /dev/null } } output { elasticsearch { hosts [localhost:9200] index springboot-logs-%{YYYY.MM.dd} } stdout { codec rubydebug } }启动命令 logstash -f /path/to/config.conf4. springboot 项目关联4.1 新建项目 elk_dmpom.xml?xml version1.0 encodingUTF-8? project xmlnshttp://maven.apache.org/POM/4.0.0 xmlns:xsihttp://www.w3.org/2001/XMLSchema-instance xsi:schemaLocationhttp://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd modelVersion4.0.0/modelVersion parent groupIdorg.springframework.boot/groupId artifactIdspring-boot-starter-parent/artifactId version3.5.13/version relativePath/ !-- lookup parent from repository -- /parent groupIdcom.dp.ccrr/groupId artifactIdelk_dm/artifactId version0.0.1-SNAPSHOT/version nameelk_dm/name descriptionelk_dm/description url/ licenses license/ /licenses developers developer/ /developers scm connection/ developerConnection/ tag/ url/ /scm properties java.version17/java.version /properties dependencies dependency groupIdorg.springframework.boot/groupId artifactIdspring-boot-starter-web/artifactId /dependency dependency groupIdorg.projectlombok/groupId artifactIdlombok/artifactId /dependency dependency groupIdnet.logstash.logback/groupId artifactIdlogstash-logback-encoder/artifactId version7.4/version /dependency dependency groupIdorg.springframework.boot/groupId artifactIdspring-boot-starter-test/artifactId scopetest/scope /dependency /dependencies build plugins plugin groupIdorg.springframework.boot/groupId artifactIdspring-boot-maven-plugin/artifactId /plugin /plugins /build /projectresource :logback-spring.xml?xml version1.0 encodingUTF-8? configuration springProperty scopecontext nameappName sourcespring.application.name defaultValueapp/ !-- JSON 格式输出到文件 -- appender nameJSON_FILE classch.qos.logback.core.rolling.RollingFileAppender file/tmp/spring-logs/${appName}.log/file !-- 添加滚动策略同时解决 TriggeringPolicy 警告 -- rollingPolicy classch.qos.logback.core.rolling.TimeBasedRollingPolicy fileNamePattern/tmp/spring-logs/${appName}.%d{yyyy-MM-dd}.log/fileNamePattern maxHistory7/maxHistory /rollingPolicy encoder classnet.logstash.logback.encoder.LogstashEncoder customFields{service:${appName},environment:dev}/customFields /encoder /appender !-- 控制台输出开发环境 -- appender nameCONSOLE classch.qos.logback.core.ConsoleAppender encoder pattern%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n/pattern /encoder /appender !-- 开发环境输出到控制台 -- springProfile namedev root levelINFO appender-ref refCONSOLE/ /root /springProfile !-- 生产/测试环境输出 JSON 到文件 -- springProfile nameprod,test,default root levelINFO appender-ref refJSON_FILE/ /root /springProfile /configurationELKController.javapackage com.dp.ccrr.elk_dm.controller; import lombok.extern.slf4j.Slf4j; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; Slf4j RestController RequestMapping(/elk) public class ELKController { RequestMapping(/) public String index() { log.info(------------ELKController.index); return Hello World!; } }TraceIdFilter.javapackage com.dp.ccrr.elk_dm.filter; import jakarta.servlet.*; import org.slf4j.MDC; import org.springframework.stereotype.Component; import java.io.IOException; import java.util.UUID; Component public class TraceIdFilter implements Filter { Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { MDC.put(traceId, UUID.randomUUID().toString().replace(-, )); chain.doFilter(request, response); MDC.clear(); } }启动项目http://localhost:8081/elk/5. 测试http://localhost:9200/test-index/_search?prettyhttp://localhost:5601/新建信息仓库输入springboot-logs*然后在discover查看搜索条件service elk_dmtoday然后就能看到测试数据了。Done.