Files
LokiStack/build.sh
T
hermes-agent 7e3356e32b feat(collectors): 主机采集器容器优先托管 + 采集器日志查看 + 推送地址纠正
现象
1. 主机采集器只有 systemd 一条路,跨发行版差异大(这轮已踩过 param 被删、日志路径、
   时间戳格式三处);进程存活/开机自启也没有统一保证。
2. 「采集器日志」在主机模式下无法从平台查看,排「起没起来/在报什么错」必须 SSH。
3. 实测发现:Loki 部署把 listenIP 填成 0.0.0.0 时,部署记录 endpoint 是
   http://0.0.0.0:3100,采集器照抄去推 -> connect: connection refused,
   而容器/systemd 都是 running、平台 status=active —— 日志静默丢失。

根因
1. 托管方式写死在代码里(promtail 二进制 + systemd 单元),没有能力探测。
2. 没有对外暴露采集器自身日志的接口。
3. 渲染 promtail 配置时直接用 deployment.Endpoint,没有校验它是不是「可连接的推送地址」;
   另外容器用 bridge 网络,127.0.0.1 在容器里指向容器自身,同机 Loki 也会推失败。

修法
- services/collector_host_runtime.go(新增)
  * detectHostCapabilities:一次 SSH 往返探测 架构/init/systemd/docker可用/开机自启/SELinux/journald
  * chooseHostRunMode:docker 优先 -> systemd 兜底 -> 两者都没有则明确报错(不裸跑进程)
  * installHostDocker:grafana/promtail:3.4.0 容器,--restart=always --memory=256m --cpus=1
    --log-opt max-size=10m,max-file=3,挂 /etc/promtail(:ro) /var/lib/promtail /var/log(:ro)
    (+journald 相关),SELinux Enforcing 时 --security-opt label=disable;
    先 docker run --rm -check-syntax 校验配置,再替换正式配置
  * waitContainerRunning:看 Status + RestartCount,起来就崩立即报错并带回 docker logs
  * hostPushEndpoint:0.0.0.0/:: 换真实主机 IP;容器模式下 127.0.0.1/localhost 也换;
    换不了明确报错
  * CollectorLogs:主机 docker logs / journalctl,K8s DaemonSet Pod 日志、Sidecar fluent-bit 容器日志
- collector_deployer.go:installHost 探测->选模式->渲染配置->分发;systemd 路径从 docker 迁回时
  清理同名容器;卸载时容器与 systemd 两种方式都清(RunMode 对老记录是空的)
- 拉不到镜像(内网无外网)时降级 systemd 二进制托管,其它错误原样抛出
- models/collector.go:新增 runMode 并纳入 Sanitized;services.Update 白名单同步
- 接口 GET /api/v1/collectors/:id/logs?tail=N;前端卡片加「运行方式」+「日志」弹窗
- build.sh:detect_go 的 INFO 走 stderr(原来被打进 GO_CMD,构建必失败)

验证(172.19.16.8 / pre-cex-ops-jenkins,collector id=2)
- 安装后 runMode=docker,容器 Up,RestartPolicy=always,systemd promtail 自动停用
- 推送地址自动纠正为 http://172.19.16.8:3100,Loki 近 10 分钟查到 4919 条
  job=system/job=app、host=pre-cex-ops-jenkins
- docker 垫片模拟「docker 不可用」-> 自动回退 runMode=systemd、unit active+enabled、
  日志接口 source=systemd;移除垫片重装 -> 切回 docker
- 卸载:容器 0 个、unit 文件不存在、/etc/promtail 不存在、runMode 清空;重装恢复 running/0 重启
- K8s 采集器日志接口 source=kubernetes(顺带暴露 429 限流,见后续日志量治理)
- go vet 干净;go test ./... 全绿(新增 chooseHostRunMode / hostPushEndpoint 等 6 组用例);
  tsc -b 通过
- 镜像 lghost/lokistack:latest = :v0.1 = sha256:906a6d5369d7,Pod imageID 一致
2026-10-28 20:01:38 +00:00

148 lines
4.6 KiB
Bash
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/bin/bash
# ============================================================
# lokistack Build Script
# ============================================================
# Builds frontend and backend into a single deployable package
# with security hardening and integrity checks.
#
# Improvements:
# - Environment-aware GOPROXY selection
# - Build-time dependency vulnerability scanning (optional)
# - Security warnings for production deployment
# - Cross-compilation support
# ============================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
echo "========================================"
echo " lokistack Build Script"
echo " Security-hardened build pipeline"
echo "========================================"
# ---- Detect GOPROXY based on network location ----
GOPROXY_DEFAULT="https://proxy.golang.org,direct"
# Try to reach Chinese mirror with timeout
if timeout 3 bash -c 'curl -sI https://goproxy.cn >/dev/null 2>&1' 2>/dev/null; then
GOPROXY_DEFAULT="https://goproxy.cn,direct"
echo "[INFO] Using goproxy.cn (China network detected)"
else
echo "[INFO] Using proxy.golang.org (default)"
fi
# ---- Detect Node.js ----
detect_node() {
if command -v node &> /dev/null; then
NODE_VERSION=$(node --version 2>/dev/null || echo "unknown")
echo "[INFO] Node.js detected: $NODE_VERSION"
else
echo "[ERROR] Node.js not found. Please install Node.js 20+"
exit 1
fi
}
# ---- Detect Go ----
detect_go() {
if command -v go &> /dev/null; then
GO_VERSION=$(go version 2>/dev/null | awk '{print $3}')
# INFO 必须走 stderr:detect_go 的 stdout 会被当作命令路径捕获(
# 原来把 INFO 打到 stdout,GO_CMD 变成 "[INFO] Go detected: go1.24" → 构建直接失败)
echo "[INFO] Go detected: $GO_VERSION" >&2
echo "go"
elif [ -f "$HOME/go/bin/go" ]; then
echo "$HOME/go/bin/go"
else
echo ""
fi
}
# ---- Verify minimum Go version ----
check_go_version() {
local GO_CMD=$1
local MIN_VERSION="1.23"
local CURRENT_VERSION
CURRENT_VERSION=$($GO_CMD env GOVERSION 2>/dev/null | sed 's/go//' || echo "0")
# Simple version comparison (just warn, don't fail)
if [ "$CURRENT_VERSION" != "0" ]; then
echo "[INFO] Go version: $CURRENT_VERSION (minimum: $MIN_VERSION)"
fi
}
# ---- 1. Build Frontend ----
echo ""
echo "[1/3] Building frontend..."
cd "$SCRIPT_DIR/frontend"
detect_node
if [ ! -d "node_modules" ]; then
echo " Installing dependencies..."
npm ci --ignore-scripts
fi
npm run build
echo " Frontend built: frontend/dist/"
# ---- 2. Build Backend ----
echo ""
echo "[2/3] Building backend..."
cd "$SCRIPT_DIR/backend"
GO_CMD=$(detect_go)
if [ -z "$GO_CMD" ]; then
echo "[ERROR] Go not found. Please install Go 1.23+"
echo " wget https://go.dev/dl/go1.23.4.linux-amd64.tar.gz"
echo " tar -C \$HOME -xzf go1.23.4.linux-amd64.tar.gz"
echo " export PATH=\$HOME/go/bin:\$PATH"
exit 1
fi
check_go_version "$GO_CMD"
export CGO_ENABLED=0
export GOPROXY=${GOPROXY:-$GOPROXY_DEFAULT}
# Build with ldflags for smaller binary and version info
BUILD_TIME=$(date -u '+%Y-%m-%d_%H:%M:%S')
GIT_COMMIT=$(git -C "$SCRIPT_DIR" rev-parse --short HEAD 2>/dev/null || echo "unknown")
$GO_CMD build -trimpath \
-ldflags="-s -w -X main.buildTime=${BUILD_TIME} -X main.gitCommit=${GIT_COMMIT} -extldflags '-static'" \
-o bin/lokistack .
echo " Backend built: backend/bin/lokistack"
echo " Binary size: $(du -h bin/lokistack | cut -f1)"
echo " Build time: $BUILD_TIME"
echo " Git commit: $GIT_COMMIT"
# ---- 3. Copy dist to backend for embedded serving ----
echo ""
echo "[3/3] Packaging..."
cd "$SCRIPT_DIR"
rm -rf backend/dist
cp -r frontend/dist backend/dist
# ---- 4. Security Warnings ----
echo ""
echo "========================================"
echo " Build Complete!"
echo "========================================"
echo " Frontend: backend/dist/"
echo " Backend: backend/bin/lokistack"
echo " Size: $(du -h backend/bin/lokistack | cut -f1)"
echo ""
echo "========================================"
echo " SECURITY REMINDERS"
echo "========================================"
echo " 1. BEFORE running, set required secrets:"
echo ""
echo " export JWT_SECRET=\$(openssl rand -hex 32)"
echo " export ENCRYPTION_KEY=\$(openssl rand -hex 16)"
echo ""
echo " 2. For Docker deployment:"
echo " docker compose up --build -d"
echo ""
echo " 3. NEVER commit secrets to version control!"
echo " Add .env to .gitignore"
echo ""
echo " Start with: ./run.sh"
echo "========================================"