Linux下Golang日志管理优化方案

1. 日志框架选择与配置优化

推荐日志库

• Zap (Uber开发): 高性能结构化日志库

  import "go.uber.org/zap"
  
  logger, _ := zap.NewProduction()
  defer logger.Sync()
  logger.Info("This is a structured log",
  zap.String("key", "value"),
  zap.Int("count", 1))
  

• Logrus: 功能丰富，兼容标准库

  import log "github.com/sirupsen/logrus"
  
  log.SetFormatter(&log.JSONFormatter{})
  log.WithFields(log.Fields{
  "event": "event",
  "topic": "topic",
  }).Info("This is a structured log")
  

配置优化建议

• 生产环境使用NewProduction()而非NewDevelopment()
• 合理设置日志级别(DEBUG/INFO/WARN/ERROR)
• 启用异步日志写入减少I/O阻塞

2. 日志轮转与归档策略

使用Lumberjack实现日志轮转

import "gopkg.in/natefinch/lumberjack.v2"

logger := &lumberjack.Logger{
    Filename:   "/var/log/myapp/app.log",
    MaxSize:    100, // MB
    MaxBackups: 5,
    MaxAge:     30,  // days
    Compress:   true,
}
log.SetOutput(logger)

结合logrotate系统工具

/var/log/myapp/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 0644 appuser appgroup
    sharedscripts
    postrotate
        systemctl reload myapp > /dev/null
    endscript
}

3. 结构化日志与上下文

结构化日志实践

// Zap示例
logger.Info("User logged in",
    zap.String("username", username),
    zap.String("ip", ip),
    zap.Time("time", time.Now()),
    zap.Duration("latency", latency))

// Logrus示例
log.WithFields(log.Fields{
    "username": username,
    "ip":       ip,
    "time":     time.Now(),
    "latency":  latency,
}).Info("User logged in")

请求上下文传递

func handler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    logger := zap.L().With(
        zap.String("requestID", middleware.GetRequestID(ctx)),
        zap.String("path", r.URL.Path),
    )
    // 使用logger记录日志
}

4. 日志收集与分析系统集成

与ELK/EFK集成

• 配置Filebeat收集日志
• 使用Fluentd/Fluent Bit处理日志
• 输出到Elasticsearch并通过Kibana展示

直接输出到日志服务

// 输出到Syslog
func setupSyslog() {
    log.SetFormatter(&log.JSONFormatter{})
    hook, err := logrus_syslog.NewSyslogHook("udp", "localhost:514", syslog.LOG_INFO, "myapp")
    if err == nil {
        log.AddHook(hook)
    }
}

5. 性能优化技巧

1. 避免频繁字符串拼接:

   // 不好
   log.Info(fmt.Sprintf("User %s logged in from %s", user, ip))
   
   // 推荐
   log.Info("User logged in", "user", user, "ip", ip)
   

2. 采样高频日志:

   if rand.Intn(100) == 0 { // 1%采样率
      log.Debug("Debug info", fields...)
   }
   

3. 异步日志写入:

   // Zap异步核心配置
   zap.New(zapcore.NewCore(
      zapcore.NewJSONEncoder(encoderCfg),
      zapcore.AddSync(asyncWriter),
      zap.InfoLevel,
   ))
   

6. 监控与告警

1. 日志错误率监控:

   # 统计ERROR级别日志
   grep -c '"level":"ERROR"' /var/log/myapp/app.log
   

2. Prometheus集成:

   var errorCounter = prometheus.NewCounter(prometheus.CounterOpts{
      Name: "app_error_logs_total",
      Help: "Total number of ERROR logs",
   })
   
   func logError(msg string) {
      errorCounter.Inc()
      log.Error(msg)
   }
   

7. 安全考虑

1. 敏感信息过滤:

   func sanitize(input string) string {
      return regexp.MustCompile(`(?i)(password|token|key)=[^&\s]+`).
          ReplaceAllString(input, "$1=[REDACTED]")
   }
   

2. 日志文件权限:

   chmod 640 /var/log/myapp/*.log
   chown appuser:appgroup /var/log/myapp/*.log
   

通过以上优化措施，可以构建一个高性能、易维护且安全的Golang日志管理系统，满足生产环境的需求。