Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b8eca87a89 | ||
|
|
897899305f | ||
|
|
9a477e9c9b | ||
|
|
f67b86e6bb | ||
|
|
8423192719 | ||
|
|
daa531f4ed | ||
|
|
8606b1ee31 | ||
|
|
a6b0be9698 | ||
|
|
914eec1e4b | ||
|
|
cec7c6eeaf | ||
|
|
f3d27ceb04 | ||
|
|
db92426973 | ||
|
|
62d9b81a70 | ||
|
|
af0298e994 | ||
|
|
5f42d9f696 | ||
|
|
25a5e57cb3 | ||
|
|
5f42ac5486 | ||
|
|
4d582e62eb | ||
|
|
0644dccf2a | ||
|
|
02c1ee8aa8 | ||
|
|
550d8f4967 | ||
|
|
448f695c28 | ||
|
|
37a610b603 | ||
|
|
6923a12dd9 | ||
|
|
fbbd3a3c27 | ||
|
|
c216bdfdd4 | ||
|
|
2efbed882a | ||
|
|
5d898e3592 | ||
|
|
f1d9480caf | ||
|
|
f457d6da09 | ||
|
|
b81b31f322 | ||
|
|
07aa41bc33 | ||
|
|
9641503007 | ||
|
|
c60b827262 | ||
|
|
c090ba0564 | ||
|
|
1610988c2e | ||
|
|
63445bd598 | ||
|
|
7be41f0a6b | ||
|
|
3861b8b128 | ||
|
|
9448868a71 | ||
|
|
2e97c04f84 | ||
|
|
f1fc7b4a9a | ||
|
|
fdd705c4b4 | ||
|
|
d4422f1541 | ||
|
|
63dd491c5b | ||
|
|
b7de337c9e | ||
|
|
e4cefa1fa7 | ||
|
|
7b1ffd0274 | ||
|
|
5c7a3ec81d | ||
|
|
d0adc095dd | ||
|
|
7f6fb364de | ||
|
|
3b297d0429 | ||
|
|
c8d026adc7 | ||
|
|
87909ed403 | ||
|
|
a14bdb5e07 | ||
|
|
419a81c2af | ||
|
|
ad0d35f72a | ||
|
|
c9e604b35f | ||
|
|
7624baa564 | ||
|
|
b433b5f9dd | ||
|
|
56e1245b70 | ||
|
|
e6c0cf134f | ||
|
|
48e68250bb | ||
|
|
e579eabe48 | ||
|
|
c4d907fd27 | ||
|
|
bbeb9d0b28 | ||
|
|
65bb29aa72 | ||
|
|
88b2ac2a6d | ||
|
|
0936234623 | ||
|
|
437ff56992 | ||
|
|
c1f822fb10 | ||
|
|
f87fb4b2b3 | ||
|
|
e07fe55c86 | ||
|
|
2bd77b57b4 | ||
|
|
3ea7f2dcc6 | ||
|
|
b8d08ef6d1 | ||
|
|
abe28964a1 | ||
|
|
df1075f1f5 |
No files matched your search
@@ -13,13 +13,12 @@ Go 二进制同时提供 REST API 服务和已构建的前端(SPA)静态资源
|
||||
|
||||
## 常用命令
|
||||
|
||||
### 启动后端前的必备环境变量
|
||||
后端在缺少以下变量时会 **拒绝启动**(由 `backend/config/config.go` 通过 `log.Fatal` 强制)。
|
||||
`./run.sh` 会自动注入开发用默认值;直接 `go run` 或 Docker 则不会。
|
||||
### 启动密钥(JWT_SECRET / ENCRYPTION_KEY)
|
||||
显式设置优先;**未设置时自动生成随机密钥并持久化**到 `$DATA_DIR/.secrets.env`(0600,重启不变),无法持久化才拒绝启动(`backend/config/config.go` resolveSecrets)。`./run.sh` 仍会注入开发默认值。
|
||||
|
||||
```bash
|
||||
export JWT_SECRET=$(openssl rand -hex 32) # >= 32 字符
|
||||
export ENCRYPTION_KEY=$(openssl rand -hex 32) # >= 32 字节(不是 16,详见「安全」一节)
|
||||
export JWT_SECRET=$(openssl rand -hex 32) # 可选:需要固定密钥时显式设置
|
||||
export ENCRYPTION_KEY=$(openssl rand -hex 32) # 可选:>= 32 字节(不是 16,详见「安全」一节)
|
||||
```
|
||||
|
||||
### 开发模式(两个终端,热重载)
|
||||
|
||||
@@ -5,6 +5,18 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### 移除
|
||||
- 备份恢复功能(含 API / CLI / 手册):日志数据本体的保留与安全由对象存储与 Loki retention 承担,平台侧备份价值有限
|
||||
- 平台监控页与 `/api/v1/system/metrics-summary`:原始指标仍经标准 `/api/v1/metrics`(Prometheus 文本格式)暴露,可自行抓取
|
||||
- 资源用量弹窗的可选 Prometheus 接入(峰值/限流查询):统一以「未接入 Prometheus:暂无峰值/限流数据」提示,扩容建议基于 metrics-server 实时值
|
||||
|
||||
### 新增
|
||||
- 一键安装脚本(quick_start.sh):bash -c "$(curl …)" 对标 1Panel/雷池体验——架构自检、发布包下载、
|
||||
随机密钥生成、systemd 开机自启、升级幂等(数据自动备份),完成后打印访问地址与初始密码
|
||||
(10.10.10.161 CentOS 7 真机验证安装/登录/升级/文档落地页全通);配套 upload-release.sh 发布双架构包到 Gitea Release
|
||||
- 落地页(homepage/,`/homepage` 访问):产品介绍、特性、快速开始
|
||||
- 用户手册文档站(docsify,`/docs` 访问):安装、部署向导、采集、查询、分析、告警、授权、备份、FAQ 十篇
|
||||
|
||||
## [2.0.0] - 2026-10-29
|
||||
|
||||
品牌从 opsxloki 正式更名为 LokiStack,平台能力全面成型。
|
||||
|
||||
@@ -12,13 +12,12 @@ Go 二进制同时提供 REST API 服务和已构建的前端(SPA)静态资源
|
||||
|
||||
## 常用命令
|
||||
|
||||
### 启动后端前的必备环境变量
|
||||
后端在缺少以下变量时会 **拒绝启动**(由 `backend/config/config.go` 通过 `log.Fatal` 强制)。
|
||||
`./run.sh` 会自动注入开发用默认值;直接 `go run` 或 Docker 则不会。
|
||||
### 启动密钥(JWT_SECRET / ENCRYPTION_KEY)
|
||||
显式设置优先;**未设置时自动生成随机密钥并持久化**到 `$DATA_DIR/.secrets.env`(0600,重启不变),无法持久化才拒绝启动(`backend/config/config.go` resolveSecrets)。`./run.sh` 仍会注入开发默认值。
|
||||
|
||||
```bash
|
||||
export JWT_SECRET=$(openssl rand -hex 32) # >= 32 字符
|
||||
export ENCRYPTION_KEY=$(openssl rand -hex 32) # >= 32 字节(不是 16,详见「安全」一节)
|
||||
export JWT_SECRET=$(openssl rand -hex 32) # 可选:需要固定密钥时显式设置
|
||||
export ENCRYPTION_KEY=$(openssl rand -hex 32) # 可选:>= 32 字节(不是 16,详见「安全」一节)
|
||||
```
|
||||
|
||||
### 开发模式(两个终端,热重载)
|
||||
|
||||
@@ -108,6 +108,9 @@ clean:
|
||||
@echo "Cleanup complete."
|
||||
|
||||
# Linting
|
||||
dist:
|
||||
APP_VERSION=$$(git describe --tags --abbrev=0 2>/dev/null || echo dev) bash build.sh --dist
|
||||
|
||||
lint:
|
||||
@echo "Validating Dockerfile..."
|
||||
@docker run --rm -i hadolint/hadolint < Dockerfile 2>/dev/null || echo "hadolint not available, install from: https://github.com/hadolint/hadolint"
|
||||
|
||||
@@ -28,15 +28,23 @@
|
||||
|
||||
> 镜像内置全部依赖(Kubernetes 操作走 client-go 直连 API Server),宿主机**无需**安装 `kubectl` / `helm`。
|
||||
|
||||
### 1. 生成密钥并启动
|
||||
### 1. 一键安装(Linux 服务器,推荐)
|
||||
|
||||
```bash
|
||||
bash -c "$(curl -fsSL https://gitea.91ai.io/rlwj-lab/LokiStack/raw/main/scripts/quick_start.sh)"
|
||||
```
|
||||
|
||||
自动下载对应架构(x86_64/arm64)的发布包,生成随机密钥并注册 systemd 服务(开机自启);重复执行即升级(数据自动备份)。安装完成后终端直接打印访问地址与初始密码。
|
||||
|
||||
### 2. Docker 一条命令
|
||||
|
||||
```bash
|
||||
export JWT_SECRET=$(openssl rand -hex 32) # >= 32 字符
|
||||
export ENCRYPTION_KEY=$(openssl rand -hex 32) # >= 32 字节
|
||||
docker compose up -d
|
||||
docker compose ps # 等待 healthy
|
||||
```
|
||||
|
||||
未显式提供密钥时容器自动生成随机 `JWT_SECRET` / `ENCRYPTION_KEY` 并持久化到数据卷(`$DATA_DIR/.secrets.env`),重启不变;生产环境可显式指定或使用 `docker-compose.prod.yml` 密钥文件。
|
||||
|
||||
未设置这两个变量时容器会**拒绝启动**,避免不安全的默认部署。
|
||||
|
||||
打开 `http://localhost:8080`,管理员初始密码见下一步。
|
||||
|
||||
+99
-26
@@ -1,10 +1,13 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
@@ -14,30 +17,71 @@ var (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
ServerPort string
|
||||
DatabasePath string
|
||||
JWTSecret string
|
||||
EncryptionKey string // 独立加密密钥,必须与JWT_SECRET分离
|
||||
DataDir string
|
||||
KubeDir string
|
||||
LogLevel string // 新增:日志级别 debug/info/warn/error
|
||||
HostKeyDir string // 新增:SSH主机密钥存储目录
|
||||
ServerPort string
|
||||
DatabasePath string
|
||||
JWTSecret string
|
||||
EncryptionKey string // 独立加密密钥,必须与JWT_SECRET分离
|
||||
DataDir string
|
||||
KubeDir string
|
||||
LogLevel string // 新增:日志级别 debug/info/warn/error
|
||||
HostKeyDir string // 新增:SSH主机密钥存储目录
|
||||
LicenseAPIBase string // opsx-licence API 根(须以 /api 结尾)
|
||||
}
|
||||
|
||||
// resolveSecrets 密钥解析(所有部署形态统一的零配置体验):
|
||||
// 显式环境变量优先;缺失时读取 DATA_DIR/.secrets.env;两者皆无则生成随机密钥
|
||||
// 并持久化(0600),重启/升级保持不变。无法持久化时拒绝启动,绝不使用易失密钥。
|
||||
func resolveSecrets(dataDir string) (jwt, enc string) {
|
||||
jwt = os.Getenv("JWT_SECRET")
|
||||
enc = os.Getenv("ENCRYPTION_KEY")
|
||||
if jwt != "" && enc != "" {
|
||||
return jwt, enc
|
||||
}
|
||||
secretsFile := filepath.Join(dataDir, ".secrets.env")
|
||||
if data, err := os.ReadFile(secretsFile); err == nil {
|
||||
for _, line := range strings.Split(string(data), "\n") {
|
||||
if v, ok := strings.CutPrefix(line, "JWT_SECRET="); ok && jwt == "" && v != "" {
|
||||
jwt = v
|
||||
}
|
||||
if v, ok := strings.CutPrefix(line, "ENCRYPTION_KEY="); ok && enc == "" && v != "" {
|
||||
enc = v
|
||||
}
|
||||
}
|
||||
}
|
||||
if jwt != "" && enc != "" {
|
||||
return jwt, enc
|
||||
}
|
||||
gen := func() string {
|
||||
buf := make([]byte, 32)
|
||||
if _, err := rand.Read(buf); err != nil {
|
||||
log.Fatal("[SECURITY] 生成随机密钥失败: ", err)
|
||||
}
|
||||
return hex.EncodeToString(buf)
|
||||
}
|
||||
if jwt == "" {
|
||||
jwt = gen()
|
||||
log.Printf("[SECURITY] JWT_SECRET 未设置,已生成随机密钥并持久化到 %s", secretsFile)
|
||||
}
|
||||
if enc == "" {
|
||||
enc = gen()
|
||||
log.Printf("[SECURITY] ENCRYPTION_KEY 未设置,已生成随机密钥并持久化到 %s", secretsFile)
|
||||
}
|
||||
if err := os.MkdirAll(dataDir, 0o700); err != nil {
|
||||
log.Fatal("[SECURITY] 无法创建数据目录以持久化密钥: ", err)
|
||||
}
|
||||
if err := os.WriteFile(secretsFile, []byte(fmt.Sprintf(
|
||||
"# LokiStack auto-generated. Delete to regenerate.\nJWT_SECRET=%s\nENCRYPTION_KEY=%s\n", jwt, enc,
|
||||
)), 0o600); err != nil {
|
||||
log.Fatal("[SECURITY] 密钥持久化失败(请显式设置 JWT_SECRET / ENCRYPTION_KEY): ", err)
|
||||
}
|
||||
return jwt, enc
|
||||
}
|
||||
|
||||
func GetConfig() *Config {
|
||||
once.Do(func() {
|
||||
dataDir := getEnv("DATA_DIR", "./data")
|
||||
|
||||
jwtSecret := os.Getenv("JWT_SECRET")
|
||||
encKey := os.Getenv("ENCRYPTION_KEY")
|
||||
|
||||
// 生产环境强制校验:JWT_SECRET和ENCRYPTION_KEY必须设置
|
||||
if jwtSecret == "" {
|
||||
log.Fatal("[SECURITY] JWT_SECRET environment variable is required. Please set a strong random secret.")
|
||||
}
|
||||
if encKey == "" {
|
||||
log.Fatal("[SECURITY] ENCRYPTION_KEY environment variable is required. Please set a strong 32-byte random key (base64 or hex encoded).")
|
||||
}
|
||||
jwtSecret, encKey := resolveSecrets(dataDir)
|
||||
|
||||
// ENCRYPTION_KEY 长度校验:至少32字节(256位)
|
||||
if len(encKey) < 32 {
|
||||
@@ -45,14 +89,15 @@ func GetConfig() *Config {
|
||||
}
|
||||
|
||||
instance = &Config{
|
||||
ServerPort: getEnv("SERVER_PORT", "8080"),
|
||||
DatabasePath: databasePath(dataDir),
|
||||
JWTSecret: jwtSecret,
|
||||
EncryptionKey: encKey,
|
||||
DataDir: dataDir,
|
||||
KubeDir: filepath.Join(dataDir, "kubeconfigs"),
|
||||
LogLevel: getEnv("LOG_LEVEL", "info"),
|
||||
HostKeyDir: filepath.Join(dataDir, "known_hosts"),
|
||||
ServerPort: getEnv("SERVER_PORT", "8080"),
|
||||
DatabasePath: databasePath(dataDir),
|
||||
JWTSecret: jwtSecret,
|
||||
EncryptionKey: encKey,
|
||||
DataDir: dataDir,
|
||||
KubeDir: filepath.Join(dataDir, "kubeconfigs"),
|
||||
LogLevel: getEnv("LOG_LEVEL", "info"),
|
||||
HostKeyDir: filepath.Join(dataDir, "known_hosts"),
|
||||
LicenseAPIBase: resolveLicenseAPIBase(),
|
||||
}
|
||||
|
||||
_ = os.MkdirAll(instance.KubeDir, 0700)
|
||||
@@ -61,6 +106,34 @@ func GetConfig() *Config {
|
||||
return instance
|
||||
}
|
||||
|
||||
// resolveLicenseAPIBase 国内主节点默认;LICENSE_API_BASE / OPSX_LICENSE_API_BASE 可覆盖。
|
||||
// 备灾节点 l123 不写死进默认路径,需显式配置。
|
||||
func resolveLicenseAPIBase() string {
|
||||
for _, key := range []string{"LICENSE_API_BASE", "OPSX_LICENSE_API_BASE"} {
|
||||
if v := strings.TrimSpace(os.Getenv(key)); v != "" {
|
||||
return normalizeLicenseAPIBase(v)
|
||||
}
|
||||
}
|
||||
return "https://licence.rlwj.cn/api"
|
||||
}
|
||||
|
||||
func normalizeLicenseAPIBase(raw string) string {
|
||||
base := strings.TrimRight(strings.TrimSpace(raw), "/")
|
||||
if base == "" {
|
||||
return "https://licence.rlwj.cn/api"
|
||||
}
|
||||
if strings.HasSuffix(base, "/api/v1") {
|
||||
return strings.TrimSuffix(base, "/v1")
|
||||
}
|
||||
if strings.HasSuffix(base, "/v1") && !strings.HasSuffix(base, "/api/v1") {
|
||||
return strings.TrimSuffix(base, "/v1") + "/api"
|
||||
}
|
||||
if !strings.HasSuffix(base, "/api") {
|
||||
return base + "/api"
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
||||
func getEnv(key, defaultVal string) string {
|
||||
if val := os.Getenv(key); val != "" {
|
||||
return val
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestResolveSecretsAutoGenerate 零配置:未设置环境变量时自动生成随机密钥并持久化,二次调用复用
|
||||
func TestResolveSecretsAutoGenerate(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
t.Setenv("JWT_SECRET", "")
|
||||
t.Setenv("ENCRYPTION_KEY", "")
|
||||
|
||||
jwt1, enc1 := resolveSecrets(dir)
|
||||
if len(jwt1) != 64 || len(enc1) != 64 {
|
||||
t.Fatalf("生成的密钥长度异常: %d/%d", len(jwt1), len(enc1))
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(dir, ".secrets.env"))
|
||||
if err != nil {
|
||||
t.Fatalf("密钥未持久化: %v", err)
|
||||
}
|
||||
if !strings.Contains(string(data), jwt1) || !strings.Contains(string(data), enc1) {
|
||||
t.Fatal("持久化内容与返回值不一致")
|
||||
}
|
||||
info, _ := os.Stat(filepath.Join(dir, ".secrets.env"))
|
||||
if info.Mode().Perm() != 0o600 {
|
||||
t.Fatalf("密钥文件权限应为 0600,实际 %v", info.Mode().Perm())
|
||||
}
|
||||
// 二次调用复用
|
||||
jwt2, enc2 := resolveSecrets(dir)
|
||||
if jwt1 != jwt2 || enc1 != enc2 {
|
||||
t.Fatal("二次解析应复用持久化密钥")
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolveSecretsExplicitEnvWins 显式环境变量优先,不读不写持久化文件
|
||||
func TestResolveSecretsExplicitEnvWins(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
t.Setenv("JWT_SECRET", "explicit-jwt-secret-0123456789abcdef")
|
||||
t.Setenv("ENCRYPTION_KEY", "explicit-enc-secret-0123456789abcdef")
|
||||
jwt, enc := resolveSecrets(dir)
|
||||
if jwt != "explicit-jwt-secret-0123456789abcdef" || enc != "explicit-enc-secret-0123456789abcdef" {
|
||||
t.Fatal("显式环境变量必须原样生效")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, ".secrets.env")); !os.IsNotExist(err) {
|
||||
t.Fatal("显式提供时不应写持久化文件")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
package controllers
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/services"
|
||||
"lokistack/utils"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
// AIController AIOps:模型配置 + 日志分析 + 审计历史
|
||||
type AIController struct{}
|
||||
|
||||
func NewAIController() *AIController { return &AIController{} }
|
||||
|
||||
// GetConfig 返回模型配置(APIKey 掩码)+ 今日已用 token。
|
||||
func (ctrl *AIController) GetConfig(c *gin.Context) {
|
||||
userID := c.GetUint("userId")
|
||||
cfg := services.MaskedAIConfig()
|
||||
utils.Success(c, gin.H{
|
||||
"config": cfg,
|
||||
"tokenUsedToday": services.AITokenUsedToday(userID),
|
||||
})
|
||||
}
|
||||
|
||||
// SaveConfig 保存模型配置。apiKey 传空或掩码 = 沿用旧值。
|
||||
func (ctrl *AIController) SaveConfig(c *gin.Context) {
|
||||
var req struct {
|
||||
models.AIConfig
|
||||
APIKey string `json:"apiKey"`
|
||||
}
|
||||
if !utils.BindAndValidate(c, &req) {
|
||||
return
|
||||
}
|
||||
if err := services.SaveAIConfig(req.AIConfig, req.APIKey); err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, gin.H{"config": services.MaskedAIConfig()})
|
||||
}
|
||||
|
||||
// TestConfig 连通性测试:用已保存(或本次提交)的配置发一句最小请求。
|
||||
func (ctrl *AIController) TestConfig(c *gin.Context) {
|
||||
var req struct {
|
||||
APIKey string `json:"apiKey"`
|
||||
BaseURL string `json:"baseUrl"`
|
||||
Model string `json:"model"`
|
||||
}
|
||||
_ = c.ShouldBindJSON(&req)
|
||||
|
||||
cfg := services.GetAIConfig()
|
||||
if req.BaseURL != "" {
|
||||
cfg.BaseURL = req.BaseURL
|
||||
}
|
||||
if req.Model != "" {
|
||||
cfg.Model = req.Model
|
||||
}
|
||||
// key 解析:传新值才覆盖;掩码/空一律沿用已保存的真实 key(GetAIConfig 返回的是解密后的真值)
|
||||
if req.APIKey != "" && !containsMask(req.APIKey) {
|
||||
cfg.APIKey = req.APIKey
|
||||
}
|
||||
|
||||
result, err := services.TestAIConnection(cfg)
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, result)
|
||||
}
|
||||
|
||||
func containsMask(s string) bool {
|
||||
for i := 0; i+2 < len(s); i++ {
|
||||
if s[i] == '*' && s[i+1] == '*' && s[i+2] == '*' {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ListRuns 审计历史(分页)。
|
||||
func (ctrl *AIController) ListRuns(c *gin.Context) {
|
||||
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
|
||||
pageSize, _ := strconv.Atoi(c.DefaultQuery("pageSize", "20"))
|
||||
runs, total := services.ListAIRuns(page, pageSize)
|
||||
utils.Success(c, gin.H{"list": runs, "total": total, "page": page, "pageSize": pageSize})
|
||||
}
|
||||
|
||||
// GetRun 单条审计详情(含脱敏后的完整 payload)。
|
||||
func (ctrl *AIController) GetRun(c *gin.Context) {
|
||||
id, err := utils.IntParam(c, "id")
|
||||
if err != nil {
|
||||
utils.BadRequest(c, "ID 无效")
|
||||
return
|
||||
}
|
||||
run := services.GetAIRun(uint(id))
|
||||
if run == nil {
|
||||
utils.Error(c, 404, "记录不存在")
|
||||
return
|
||||
}
|
||||
utils.Success(c, run)
|
||||
}
|
||||
|
||||
// Chat 全站 AI 助手浮窗(多轮对话 + 页面上下文注入;Pro 闸门)
|
||||
func (ctrl *AIController) Chat(c *gin.Context) {
|
||||
var req struct {
|
||||
History []services.ChatMessage `json:"history" binding:"required,min=1,max=40"`
|
||||
Page string `json:"page"`
|
||||
Context string `json:"context"`
|
||||
}
|
||||
if !utils.BindAndValidate(c, &req) {
|
||||
return
|
||||
}
|
||||
cfg := services.GetAIConfig()
|
||||
reply, err := services.RunAIChat(c.Request.Context(), cfg, c.GetUint("userId"), req.History, req.Page, req.Context)
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, gin.H{"reply": reply})
|
||||
}
|
||||
@@ -19,6 +19,38 @@ func NewCollectorController() *CollectorController {
|
||||
return &CollectorController{service: services.NewCollectorService()}
|
||||
}
|
||||
|
||||
// RenderJobsYAML 把结构化采集项渲染成可编辑 YAML(前端切到 YAML 模式时调用)。
|
||||
func (ctrl *CollectorController) RenderJobsYAML(c *gin.Context) {
|
||||
var req struct {
|
||||
Jobs []models.ScrapeJob `json:"jobs"`
|
||||
}
|
||||
if !utils.BindAndValidate(c, &req) {
|
||||
return
|
||||
}
|
||||
text, err := services.RenderJobsYAML(req.Jobs)
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, gin.H{"yaml": text})
|
||||
}
|
||||
|
||||
// ParseJobsYAML 把用户编辑的 YAML 解析回结构化采集项(切回表单 / 保存前调用)。
|
||||
func (ctrl *CollectorController) ParseJobsYAML(c *gin.Context) {
|
||||
var req struct {
|
||||
YAML string `json:"yaml"`
|
||||
}
|
||||
if !utils.BindAndValidate(c, &req) {
|
||||
return
|
||||
}
|
||||
jobs, err := services.ParseJobsYAML(req.YAML)
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, gin.H{"jobs": jobs})
|
||||
}
|
||||
|
||||
func (ctrl *CollectorController) Create(c *gin.Context) {
|
||||
var req dto.CreateCollectorRequest
|
||||
if !utils.BindAndValidate(c, &req) {
|
||||
@@ -28,7 +60,7 @@ func (ctrl *CollectorController) Create(c *gin.Context) {
|
||||
Name: req.Name, Type: req.Type, LokiDeploymentID: req.LokiDeploymentID,
|
||||
HostID: req.HostID, ClusterID: req.ClusterID, TargetDeployment: req.TargetDeployment,
|
||||
Namespace: req.Namespace, LogPath: req.LogPath, Labels: req.Labels,
|
||||
ScrapeJobs: req.ScrapeJobs,
|
||||
ScrapeJobs: req.ScrapeJobs, TargetNamespaces: req.TargetNamespaces,
|
||||
})
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
package controllers
|
||||
|
||||
import (
|
||||
"html/template"
|
||||
"net/http"
|
||||
|
||||
"lokistack/services"
|
||||
"lokistack/utils"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
// 文档站控制器:模板内嵌(三栏布局,参考 opsxapi/opsxshell 文档站,品牌绿)
|
||||
var docsTemplate = template.Must(template.New("docs").Parse(`<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>{{.Page.Title}} · LokiStack 文档</title>
|
||||
<meta name="description" content="LokiStack 用户手册 — Loki 日志管理平台:向导式部署、采集策略、日志查询与分析、告警通知。">
|
||||
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Crect width='64' height='64' rx='14' fill='%23059669'/%3E%3Ctext x='32' y='44' font-size='34' font-weight='800' fill='%23fff' text-anchor='middle' font-family='sans-serif'%3EL%3C/text%3E%3C/svg%3E">
|
||||
<link rel="stylesheet" href="/docs/assets/docs.css">
|
||||
</head>
|
||||
<body>
|
||||
<header class="topbar">
|
||||
<div class="topbar-inner">
|
||||
<button class="menu-btn" id="menu-btn" aria-label="打开导航"><svg class="i" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round"><path d="M4 6h16"/><path d="M4 12h16"/><path d="M4 18h16"/></svg></button>
|
||||
<a class="brand" href="/docs">
|
||||
<span class="brand-logo">L</span>
|
||||
<span class="brand-name">LokiStack <em>文档</em></span>
|
||||
</a>
|
||||
<span class="ver-badge">v{{.Version}}</span>
|
||||
<div class="search">
|
||||
<input id="search-input" type="search" placeholder="搜索文档…" autocomplete="off" spellcheck="false">
|
||||
<div class="search-panel" id="search-panel" hidden></div>
|
||||
</div>
|
||||
<nav class="top-links">
|
||||
<a href="/homepage">官网</a>
|
||||
</nav>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<div class="layout" id="layout">
|
||||
<aside class="sidebar" id="sidebar">
|
||||
{{range .Manifest.Groups}}
|
||||
<div class="nav-group">
|
||||
<div class="nav-group-title">{{.Name}}</div>
|
||||
{{range .Pages}}
|
||||
<a class="nav-link{{if eq .Slug $.Page.Slug}} active{{end}}" href="/docs/{{.Slug}}">{{.Title}}</a>
|
||||
{{end}}
|
||||
</div>
|
||||
{{end}}
|
||||
<div class="sidebar-foot">© 2026 LokiStack · Loki 日志管理平台</div>
|
||||
</aside>
|
||||
|
||||
<main class="content">
|
||||
<div class="breadcrumb">文档 <span>/</span> {{.Page.Group}} <span>/</span> {{.Page.Title}}</div>
|
||||
<h1 class="page-title">{{.Page.Title}}</h1>
|
||||
{{if .Page.Updated}}<div class="page-meta">更新于 {{.Page.Updated}}</div>{{end}}
|
||||
<article class="md">
|
||||
{{.Page.Content}}
|
||||
</article>
|
||||
|
||||
<div class="pn-nav">
|
||||
{{with .Page.Prev}}
|
||||
<a class="pn prev" href="/docs/{{.Slug}}"><span class="pn-label">← 上一页</span><span class="pn-title">{{.Title}}</span></a>
|
||||
{{else}}<span class="pn-empty"></span>{{end}}
|
||||
{{with .Page.Next}}
|
||||
<a class="pn next" href="/docs/{{.Slug}}"><span class="pn-label">下一页 →</span><span class="pn-title">{{.Title}}</span></a>
|
||||
{{else}}<span class="pn-empty"></span>{{end}}
|
||||
</div>
|
||||
</main>
|
||||
|
||||
<aside class="toc">
|
||||
<div class="toc-title">本页目录</div>
|
||||
<nav class="toc-nav">
|
||||
{{range .Page.TOC}}<a class="toc-l{{.Level}}" href="#{{.ID}}">{{.Text}}</a>
|
||||
{{end}}
|
||||
</nav>
|
||||
</aside>
|
||||
</div>
|
||||
|
||||
<script src="/docs/assets/docs.js" defer></script>
|
||||
</body>
|
||||
</html>`))
|
||||
|
||||
type DocsController struct{}
|
||||
|
||||
func NewDocsController() *DocsController { return &DocsController{} }
|
||||
|
||||
func (ctrl *DocsController) Home(c *gin.Context) {
|
||||
c.Redirect(http.StatusFound, "/docs/"+services.DocsFirstSlug())
|
||||
}
|
||||
|
||||
func (ctrl *DocsController) Page(c *gin.Context) {
|
||||
page, manifest, err := services.DocsRenderPage(c.Param("slug"))
|
||||
if err != nil {
|
||||
utils.NotFound(c, "文档页面不存在")
|
||||
return
|
||||
}
|
||||
_ = docsTemplate.Execute(c.Writer, gin.H{"Page": page, "Manifest": manifest, "Version": manifest.ProductVersion})
|
||||
}
|
||||
|
||||
func (ctrl *DocsController) SearchData(c *gin.Context) {
|
||||
data, err := services.DocsSearchData()
|
||||
if err != nil {
|
||||
data = []map[string]string{}
|
||||
}
|
||||
c.JSON(http.StatusOK, data)
|
||||
}
|
||||
@@ -166,9 +166,11 @@ func (ctrl *LogsController) DeleteSource(c *gin.Context) {
|
||||
utils.BadRequest(c, "ID 无效")
|
||||
return
|
||||
}
|
||||
// 托管数据源也允许删除:用户可能只是不想让它出现在查询页下拉里。
|
||||
// 删除后把对应部署的 AutoSourceDisabled 置位,否则列表接口/启动同步
|
||||
// 会立刻把它补回来(用户看到的现象就是"删了还在")。
|
||||
if src := services.GetLogSource(uint(id)); src != nil && src.Managed {
|
||||
utils.BadRequest(c, "平台部署的数据源随部署管理,请先删除对应部署")
|
||||
return
|
||||
services.DisableAutoSourceFor(src.Name)
|
||||
}
|
||||
if err := services.DeleteLogSource(uint(id)); err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
@@ -328,6 +330,81 @@ func (ctrl *LogsController) Health(c *gin.Context) {
|
||||
utils.Success(c, gin.H{"ready": services.LokiReady(ctx, src)})
|
||||
}
|
||||
|
||||
// AppRanking 按应用排行(长范围走平台预聚合,毫秒级;短范围由前端走实时查询)。
|
||||
func (ctrl *LogsController) AppRanking(c *gin.Context) {
|
||||
sourceID, err := strconv.ParseUint(c.Query("sourceId"), 10, 64)
|
||||
if err != nil || sourceID == 0 {
|
||||
utils.BadRequest(c, "sourceId 无效")
|
||||
return
|
||||
}
|
||||
from, err1 := strconv.ParseInt(c.DefaultQuery("from", "0"), 10, 64)
|
||||
to, err2 := strconv.ParseInt(c.DefaultQuery("to", "0"), 10, 64)
|
||||
if err1 != nil || err2 != nil || from <= 0 || to <= 0 || to <= from {
|
||||
utils.BadRequest(c, "时间范围无效(from/to 需为纳秒时间戳且 to > from)")
|
||||
return
|
||||
}
|
||||
limit := 10
|
||||
if raw := c.Query("limit"); raw != "" {
|
||||
if v, convErr := strconv.Atoi(raw); convErr == nil && v > 0 && v <= 100 {
|
||||
limit = v
|
||||
}
|
||||
}
|
||||
rows := services.GetAppRanking(uint(sourceID), time.Unix(0, from), time.Unix(0, to), limit, c.DefaultQuery("sort", "bytes"))
|
||||
hours, bytesHours := services.AppVolumeCoverageInfo(uint(sourceID))
|
||||
utils.Success(c, gin.H{"list": rows, "hours": hours, "bytesHours": bytesHours})
|
||||
}
|
||||
|
||||
// LabelValuesBatch 一次取多个 label 的候选值:GET /logs/label-values?names=app,pod&query={...}
|
||||
//
|
||||
// 快捷查询的下拉是级联的(命名空间 → 应用 → Pod),而"应用"本身要走
|
||||
// app → service_name → container 三级回退。逐个请求时前端一次要发 3 个往返,
|
||||
// 每个都要经 K8s API 代理到 Loki 索引;合并成一个接口后只发一次。
|
||||
// 返回 {"app": [...], "service_name": [...], ...},单个 label 失败返回空数组不报错。
|
||||
func (ctrl *LogsController) LabelValuesBatch(c *gin.Context) {
|
||||
id, _ := strconv.ParseUint(c.Query("sourceId"), 10, 64)
|
||||
src := services.GetLogSource(uint(id))
|
||||
if src == nil {
|
||||
utils.BadRequest(c, "数据源不存在")
|
||||
return
|
||||
}
|
||||
raw := strings.TrimSpace(c.Query("names"))
|
||||
if raw == "" {
|
||||
utils.BadRequest(c, "缺少 names(逗号分隔的 label 名,如 app,pod)")
|
||||
return
|
||||
}
|
||||
names := make([]string, 0, 4)
|
||||
for _, name := range strings.Split(raw, ",") {
|
||||
if name = strings.TrimSpace(name); name != "" {
|
||||
names = append(names, name)
|
||||
}
|
||||
}
|
||||
if len(names) == 0 {
|
||||
utils.BadRequest(c, "names 里没有有效的 label 名")
|
||||
return
|
||||
}
|
||||
if len(names) > 10 {
|
||||
utils.BadRequest(c, "一次最多取 10 个 label 的值")
|
||||
return
|
||||
}
|
||||
requestedQuery := strings.TrimSpace(c.Query("query"))
|
||||
if requestedQuery == "" {
|
||||
requestedQuery = "{}"
|
||||
}
|
||||
query, err := services.AuthorizedLogQuery(src, c.GetUint("userId"), c.GetString("role") == "admin", requestedQuery)
|
||||
if err != nil {
|
||||
utils.Forbidden(c, err.Error())
|
||||
return
|
||||
}
|
||||
start, _ := strconv.ParseInt(c.Query("start"), 10, 64)
|
||||
end, _ := strconv.ParseInt(c.Query("end"), 10, 64)
|
||||
values, err := services.LokiLabelValuesBatch(c.Request.Context(), src, names, query, start, end)
|
||||
if err != nil {
|
||||
utils.Error(c, 502, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, values)
|
||||
}
|
||||
|
||||
// DailyVolume 每日日志量历史(平台留存,不受 Loki retention 限制)。
|
||||
//
|
||||
// query: sourceId, days(默认 7,最长 400)
|
||||
@@ -361,7 +438,7 @@ func (ctrl *LogsController) Stats(c *gin.Context) {
|
||||
utils.Forbidden(c, err.Error())
|
||||
return
|
||||
}
|
||||
st, err := services.LokiStatsFor(c.Request.Context(), src, query, req.Start, req.End)
|
||||
st, err := services.LokiStatsCached(c.Request.Context(), src, query, req.Start, req.End)
|
||||
if err != nil {
|
||||
// Loki 的 index/stats 只支持标签选择器。查询里带 `|= "error"` 这类行过滤时
|
||||
// Loki 回 400 "only label matchers are supported",原来被包装成 502
|
||||
|
||||
@@ -1,9 +1,6 @@
|
||||
package controllers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"lokistack/services"
|
||||
@@ -18,71 +15,8 @@ func NewSystemController() *SystemController { return &SystemController{} }
|
||||
// Backup 导出当前平台数据(tar.gz 下载)。
|
||||
//
|
||||
// 内容:数据文件 + kubeconfigs/ + known_hosts/ + manifest.json(含密钥指纹),
|
||||
// 不含明文密钥。恢复必须使用同一把 ENCRYPTION_KEY。
|
||||
func (ctrl *SystemController) Backup(c *gin.Context) {
|
||||
name := fmt.Sprintf("lokistack-backup-%s.tar.gz", time.Now().Format("20060102-150405"))
|
||||
c.Header("Content-Type", "application/gzip")
|
||||
c.Header("Content-Disposition", fmt.Sprintf(`attachment; filename="%s"`, name))
|
||||
|
||||
if _, err := services.CreateBackup(c.Writer); err != nil {
|
||||
// 已经写了响应头,无法再改状态码,只能把错误写进 body 尾部
|
||||
_, _ = c.Writer.WriteString(fmt.Sprintf("\n\n# 备份失败: %v", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Restore 上传备份包:校验 + 暂存,**不直接覆盖**线上数据库。
|
||||
// 返回暂存路径与密钥指纹比对结果,由管理员确认后再调用 RestoreApply。
|
||||
func (ctrl *SystemController) Restore(c *gin.Context) {
|
||||
file, err := c.FormFile("file")
|
||||
if err != nil {
|
||||
utils.BadRequest(c, "请上传备份文件(form-data 字段名 file)")
|
||||
return
|
||||
}
|
||||
if file.Size > 128<<20 {
|
||||
utils.BadRequest(c, "备份文件超过 128MB,请用 run.sh restore 在服务器上操作")
|
||||
return
|
||||
}
|
||||
f, err := file.Open()
|
||||
if err != nil {
|
||||
utils.InternalServerError(c, fmt.Sprintf("读取上传文件失败: %v", err))
|
||||
return
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
res, err := services.ValidateAndStageBackup(f)
|
||||
if err != nil {
|
||||
utils.BadRequest(c, fmt.Sprintf("备份校验失败: %v", err))
|
||||
return
|
||||
}
|
||||
utils.Success(c, res)
|
||||
}
|
||||
|
||||
// RestoreApply 应用已暂存的备份。应用后需重启平台进程才生效。
|
||||
func (ctrl *SystemController) RestoreApply(c *gin.Context) {
|
||||
var req struct {
|
||||
Confirm bool `json:"confirm"`
|
||||
}
|
||||
_ = c.ShouldBindJSON(&req)
|
||||
if !req.Confirm {
|
||||
utils.BadRequest(c, "恢复会覆盖当前全部数据,请传 confirm=true 确认")
|
||||
return
|
||||
}
|
||||
path, err := services.ApplyStagedBackup()
|
||||
if err != nil {
|
||||
utils.BadRequest(c, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, gin.H{
|
||||
"message": "备份已应用,请重启平台使新数据生效(kubectl -n log rollout restart deploy/lokistack)",
|
||||
"dbPath": path,
|
||||
})
|
||||
}
|
||||
|
||||
// Backups 列出服务器上已有的备份文件(run.sh backup 产出)。
|
||||
func (ctrl *SystemController) Backups(c *gin.Context) {
|
||||
utils.Success(c, gin.H{"list": services.ListBackups()})
|
||||
}
|
||||
|
||||
// GetRetention 读取平台数据保留期(告警发送历史 / 审计日志)。
|
||||
func (ctrl *SystemController) GetRetention(c *gin.Context) {
|
||||
@@ -108,3 +42,39 @@ func (ctrl *SystemController) SaveRetention(c *gin.Context) {
|
||||
func (ctrl *SystemController) Health(c *gin.Context) {
|
||||
utils.Success(c, services.SelfCheck())
|
||||
}
|
||||
|
||||
// LicenseStatus 当前授权(登录即可见:前端按 plan / invalidReason 显示锁/红条)
|
||||
func (ctrl *SystemController) LicenseStatus(c *gin.Context) {
|
||||
lic, pro := services.LicenseStatusPayload()
|
||||
utils.Success(c, gin.H{
|
||||
"license": lic,
|
||||
"pro": pro,
|
||||
"features": services.ProFeatureList(),
|
||||
})
|
||||
}
|
||||
|
||||
// ActivateLicense 激活许可码(admin)
|
||||
func (ctrl *SystemController) ActivateLicense(c *gin.Context) {
|
||||
var req struct {
|
||||
Key string `json:"key" binding:"required"`
|
||||
}
|
||||
if !utils.BindAndValidate(c, &req) {
|
||||
return
|
||||
}
|
||||
lic, err := services.ActivateLicense(req.Key)
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, lic.Sanitized())
|
||||
}
|
||||
|
||||
// DeactivateLicense 退出授权 / 释放本实例(admin);成功才清本地,失败保留现状。
|
||||
func (ctrl *SystemController) DeactivateLicense(c *gin.Context) {
|
||||
res, err := services.DeactivateLicense()
|
||||
if err != nil {
|
||||
utils.Error(c, 400, err.Error())
|
||||
return
|
||||
}
|
||||
utils.Success(c, res)
|
||||
}
|
||||
@@ -14,4 +14,7 @@ type CreateCollectorRequest struct {
|
||||
Labels string `json:"labels"`
|
||||
// ScrapeJobs 主机模式的结构化采集项(可选;留空按 LogPath 生成 app + system 两个默认项)
|
||||
ScrapeJobs []models.ScrapeJob `json:"scrapeJobs"`
|
||||
// TargetNamespaces 仅 k8s-daemonset 使用:只采集这些命名空间的 Pod 日志。
|
||||
// 空/不传 = 采集全集群(默认,保持旧行为)。
|
||||
TargetNamespaces []string `json:"targetNamespaces"`
|
||||
}
|
||||
@@ -66,6 +66,7 @@ require (
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||
github.com/ugorji/go/codec v1.2.12 // indirect
|
||||
github.com/x448/float16 v0.8.4 // indirect
|
||||
github.com/yuin/goldmark v1.8.6 // indirect
|
||||
go.yaml.in/yaml/v2 v2.4.2 // indirect
|
||||
golang.org/x/arch v0.12.0 // indirect
|
||||
golang.org/x/net v0.49.0 // indirect
|
||||
|
||||
@@ -170,6 +170,8 @@ github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
|
||||
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
|
||||
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
github.com/yuin/goldmark v1.8.6 h1:d0VcaP1sx9GkFVkoW+KtggpGi2KZ965i14b0+bDQST4=
|
||||
github.com/yuin/goldmark v1.8.6/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
|
||||
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
|
||||
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
|
||||
go.yaml.in/yaml/v3 v3.0.3 h1:bXOww4E/J3f66rav3pX3m8w6jDE4knZjGOw8b5Y6iNE=
|
||||
|
||||
+58
-55
@@ -39,79 +39,72 @@ func fileExists(path string) bool {
|
||||
|
||||
// runCLI 提供不依赖 Web 的运维命令(备份 CronJob、恢复、自检都走它)。
|
||||
//
|
||||
// lokistack backup --output /app/data/backups/xxx.tar.gz
|
||||
// lokistack restore <备份包路径> # 校验 + 立即应用(会先备份当前 DB)
|
||||
// lokistack self-check # 打印 JSON 自检结果
|
||||
|
||||
func strDefault(v, d string) string {
|
||||
if strings.TrimSpace(v) == "" {
|
||||
return d
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func runCLI(args []string) {
|
||||
switch args[0] {
|
||||
case "backup":
|
||||
out := ""
|
||||
for i := 1; i < len(args); i++ {
|
||||
if args[i] == "--output" && i+1 < len(args) {
|
||||
out = args[i+1]
|
||||
case "self-check":
|
||||
out, _ := json.MarshalIndent(services.SelfCheck(), "", " ")
|
||||
fmt.Println(string(out))
|
||||
case "license":
|
||||
// 发行方工具:签发 Pro 许可码(需发行密钥构建;默认密钥仅供开发自测)
|
||||
flags := map[string]string{}
|
||||
for _, a := range args[1:] {
|
||||
if rest, ok := strings.CutPrefix(a, "--"); ok {
|
||||
if k, v, ok2 := strings.Cut(rest, "="); ok2 {
|
||||
flags[k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
if out == "" {
|
||||
out = "/app/data/backups/lokistack-" + time.Now().Format("20060102-150405") + ".tar.gz"
|
||||
getInt := func(k string, d int) int {
|
||||
if v := flags[k]; v != "" {
|
||||
n := 0
|
||||
for _, ch := range v {
|
||||
if ch < '0' || ch > '9' {
|
||||
n = 0
|
||||
break
|
||||
}
|
||||
n = n*10 + int(ch-'0')
|
||||
}
|
||||
if n > 0 {
|
||||
return n
|
||||
}
|
||||
}
|
||||
return d
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(out), 0o700); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "创建备份目录失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
f, err := os.Create(out)
|
||||
days := getInt("days", 0)
|
||||
key, err := services.GenerateLicenseKey(
|
||||
"pro",
|
||||
strDefault(flags["edition"], "personal"),
|
||||
strDefault(flags["customer"], "unnamed"),
|
||||
getInt("seats", 0), getInt("instances", 1), days,
|
||||
)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "创建备份文件失败: %v\n", err)
|
||||
fmt.Fprintln(os.Stderr, "签发失败:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer f.Close()
|
||||
m, err := services.CreateBackup(f)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "备份失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
expiry := "永久"
|
||||
if days > 0 {
|
||||
expiry = fmt.Sprintf("%d 天", days)
|
||||
}
|
||||
fmt.Printf("备份完成: %s\n记录数: %d\n密钥指纹: %s\n", out, m.RecordCount, m.KeyFP)
|
||||
|
||||
case "restore":
|
||||
if len(args) < 2 {
|
||||
fmt.Fprintln(os.Stderr, "用法: lokistack restore <备份包.tar.gz>")
|
||||
os.Exit(1)
|
||||
}
|
||||
f, err := os.Open(args[1])
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "打开备份失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer f.Close()
|
||||
res, err := services.ValidateAndStageBackup(f)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "校验失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if res.Warning != "" {
|
||||
fmt.Fprintf(os.Stderr, "⚠️ %s\n", res.Warning)
|
||||
}
|
||||
path, err := services.ApplyStagedBackup()
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "应用失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("恢复完成: %s(备份 %d 条记录)\n请重启平台进程使新数据生效。\n", path, res.RowsInBackup)
|
||||
|
||||
case "self-check":
|
||||
b, _ := json.MarshalIndent(services.SelfCheck(), "", " ")
|
||||
fmt.Println(string(b))
|
||||
|
||||
fmt.Printf("许可码(%s / %s / %s):\n%s\n", strDefault(flags["edition"], "personal"), strDefault(flags["customer"], "unnamed"), expiry, key)
|
||||
case "version":
|
||||
fmt.Printf("%s (commit %s, built %s)\n", version, gitCommit, buildTime)
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "未知命令 %q。可用: backup / restore / self-check / version\n", args[0])
|
||||
fmt.Fprintf(os.Stderr, "未知命令 %q。可用: self-check / version\n", args[0])
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func main() {
|
||||
// 运维命令行:支持在容器/CronJob 里不经 Web 完成备份/恢复/自检
|
||||
// 运维命令行:容器/CronJob 里不经 Web 完成自检与版本查询
|
||||
if len(os.Args) > 1 {
|
||||
runCLI(os.Args[1:])
|
||||
return
|
||||
@@ -138,8 +131,18 @@ func main() {
|
||||
|
||||
log.Println("Seeding default data...")
|
||||
seed.Seed()
|
||||
|
||||
// 清理旧版本删除部署后残留的托管数据源(幂等)
|
||||
services.CleanupOrphanManagedLogSources()
|
||||
// 异步重新探测运行中部署的数据源连接方式(平台/Loki 可能在主机或集群内)
|
||||
go services.EnsureLogSourcesForRunningDeployments()
|
||||
// 按应用日志量的**小时级预聚合**:每小时抓上一个完整小时,
|
||||
// 让 6h/24h 这类长范围的"应用排行"变成读库(毫秒级),不扫日志内容。
|
||||
go services.StartAppVolumeCapture()
|
||||
// 补齐/升级内置告警规则(幂等):启动即执行,存量环境也能拿到「默认启用+绑默认通道」
|
||||
seed.SeedAlertRules()
|
||||
// 14 天 Pro 试用:首次启动落起点(幂等,含防删库重装指纹)
|
||||
services.EnsureTrialStarted()
|
||||
|
||||
// 安全迁移:历史明文 kubeconfig → 加密存储,并清理明文文件
|
||||
services.MigrateClusterKubeconfigs()
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
package middleware
|
||||
|
||||
// RequirePro Pro 功能闸门:无有效 Pro 授权返回 403(信息明确,前端渲染升级引导)
|
||||
import (
|
||||
"lokistack/services"
|
||||
"lokistack/utils"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
func RequirePro() gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
if !services.IsPro() {
|
||||
utils.Error(c, 403, "Pro 功能:请到「系统设置 → 授权」激活许可码(免费版保留日志查询、部署管理与用户授权)")
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
c.Next()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
// AIConfig 大模型接入配置(存在 settings 表里,APIKey 加密保存)。
|
||||
//
|
||||
// 设计取向:只支持 OpenAI 兼容协议 —— DeepSeek / 通义 / 内网 vLLM / Ollama /
|
||||
// 任意 OpenAI 兼容网关都覆盖了,不针对厂商写适配器,保持"通用"。
|
||||
type AIConfig struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Provider string `json:"provider"` // 固定 openai-compatible(保留字段便于将来扩展)
|
||||
BaseURL string `json:"baseUrl"` // 如 https://api.deepseek.com/v1
|
||||
// APIKey 返回给前端时永远是掩码(sk-***abcd);保存时为空表示"不改动"
|
||||
APIKey string `json:"apiKey"`
|
||||
Model string `json:"model"`
|
||||
Temperature float64 `json:"temperature"`
|
||||
MaxTokens int `json:"maxTokens"`
|
||||
TimeoutSec int `json:"timeoutSec"`
|
||||
// DailyTokenCap 每用户每日 token 上限(成本闸门);0 = 不限制
|
||||
DailyTokenCap int `json:"dailyTokenCap"`
|
||||
// MaxScanBytes 单次分析允许扫描的日志字节上限(成本闸门);0 = 用默认 2GB
|
||||
MaxScanBytes int64 `json:"maxScanBytes"`
|
||||
// RedactExtra 额外脱敏正则(每行一个),命中的内容替换为 ***
|
||||
RedactExtra []string `json:"redactExtra"`
|
||||
}
|
||||
|
||||
// AIAnalysisRun 一次 AI 分析的审计记录。
|
||||
//
|
||||
// 之所以连 PromptPayload 一起存:出问题时能回答"你到底把什么发给了模型",
|
||||
// 这是脱敏与合规的底线。payload 已脱敏,且长度在主流程里截断。
|
||||
type AIAnalysisRun struct {
|
||||
ID uint `json:"id"`
|
||||
UserID uint `json:"userId"`
|
||||
Username string `json:"username"`
|
||||
SourceID uint `json:"sourceId"`
|
||||
SourceName string `json:"sourceName"`
|
||||
RangeMinutes int `json:"rangeMinutes"`
|
||||
Model string `json:"model"`
|
||||
Question string `json:"question"`
|
||||
Status string `json:"status"` // ok | failed | no-model | quota-exceeded
|
||||
Error string `json:"error"`
|
||||
Conclusion string `json:"conclusion"` // 模型返回的结构化 JSON(字符串)
|
||||
PromptPayload string `json:"promptPayload"`
|
||||
PromptTokens int `json:"promptTokens"`
|
||||
CompletionTokens int `json:"completionTokens"`
|
||||
ScanBytes int64 `json:"scanBytes"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
// AppVolumeSnapshot 按小时留存「每个应用」的日志量(条数)。
|
||||
//
|
||||
// 为什么必须落库:
|
||||
//
|
||||
// Loki 的 index/volume 只支持 namespace 级聚合(3.4 实测 aggregateBy=app 不生效),
|
||||
// 所以"按应用排行"只能读日志内容,成本 ∝ 时间范围:
|
||||
// 5 分钟 ≈ 1.3GB ≈ 10 秒
|
||||
// 15 分钟 ≈ 3.9GB ≈ 20 秒
|
||||
// 6 小时 ≈ 92GB ≈ 5~8 分钟 ← 页面根本等不了
|
||||
// 24 小时 ≈ 370GB ≈ 不可用
|
||||
// 把 per-app 数据按小时在后台算好存起来,长范围排行就变成读库(毫秒级)。
|
||||
//
|
||||
// 采集方式:每小时第 5 分钟抓「上一个完整小时」,一条
|
||||
//
|
||||
// sum by (app, namespace)(count_over_time({...}[1h]))
|
||||
//
|
||||
// 查询在后台跑,慢一点无所谓;页面永远读库。
|
||||
type AppVolumeSnapshot struct {
|
||||
ID uint `json:"id"`
|
||||
// Hour 小时桶,格式 YYYY-MM-DDTHH(本地时区,按整点对齐)
|
||||
Hour string `json:"hour"`
|
||||
SourceID uint `json:"sourceId"`
|
||||
Namespace string `json:"namespace"`
|
||||
App string `json:"app"`
|
||||
// Entries 该小时的日志条数(count_over_time,要读日志内容,成本高但准确)
|
||||
Entries int64 `json:"entries"`
|
||||
// Bytes 该小时的日志体积(字节,来自 index/volume 索引接口)
|
||||
//
|
||||
// 为什么不跟 Entries 用同一条查询:按 app 取体积在 Loki 里没有"一次扫全集群"的
|
||||
// 索引聚合(index/volume 的返回只带 selector 里出现的 label),所以只能一个 app
|
||||
// 调一次;换来的是 80ms/次的索引查询,而不是重扫整小时日志内容(约 50s)。
|
||||
// 老快照(本字段上线前抓的)Bytes=0,前端据此把"体积未知"和"体积真的是 0"分开。
|
||||
Bytes int64 `json:"bytes"`
|
||||
// Partial 该小时数据是否不完整(如平台停了一段时间后才回补,超出 Loki 保留期)
|
||||
Partial bool `json:"partial"`
|
||||
CapturedAt time.Time `json:"capturedAt"`
|
||||
}
|
||||
@@ -25,6 +25,15 @@ type DailyVolumeSnapshot struct {
|
||||
Entries int64 `json:"entries"` // 日志条数
|
||||
// Partial 该日期是否为"不完整的一天"(当天抓取时还没过完)。
|
||||
// 前端用它区分"真实低峰"和"只统计到一半"。
|
||||
Partial bool `json:"partial"`
|
||||
CapturedAt time.Time `json:"capturedAt"`
|
||||
//
|
||||
// ⚠️ 只表示"今天还没过完"。早期实现把"起点超出 24 小时"也算成 partial,
|
||||
// 导致昨天、前天的柱子也画成虚线空心,看起来像"每天都在进行中"(已修)。
|
||||
Partial bool `json:"partial"`
|
||||
// EntriesAttempts 条数(index/stats)已尝试次数。
|
||||
// 整天跨度统计很贵(实测 6ms ~ 20s,取决于实例负载),
|
||||
// 所以只对"已经过完的天"补,且最多重试 entriesMaxAttempts 次,
|
||||
// 避免每 30 分钟的定时任务无脑重跑一遍。
|
||||
EntriesAttempts int `json:"entriesAttempts"`
|
||||
LastEntriesAttempt time.Time `json:"lastEntriesAttempt"`
|
||||
CapturedAt time.Time `json:"capturedAt"`
|
||||
}
|
||||
@@ -27,18 +27,23 @@ type LokiDeployment struct {
|
||||
Replicas int `json:"replicas"`
|
||||
ResourcesCPU string `json:"resourcesCPU"`
|
||||
ResourcesMem string `json:"resourcesMem"`
|
||||
StorageSize string `json:"storageSize"`
|
||||
GrafanaEnabled bool `json:"grafanaEnabled"`
|
||||
GrafanaVersion string `json:"grafanaVersion"`
|
||||
GrafanaPort int `json:"grafanaPort"`
|
||||
GrafanaEndpoint string `json:"grafanaEndpoint"`
|
||||
Status string `json:"status"`
|
||||
StatusMessage string `json:"statusMessage"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
AdminUser string `json:"adminUser"`
|
||||
AdminPassword string `json:"adminPassword"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
// AutoSourceDisabled 用户手动删除了"自动注册的数据源"后置为 true,
|
||||
// 平台不再自动重建,直到用户重新部署/编辑该部署。
|
||||
// 没有这个开关的话:删了数据源 → 下次打开查询页又被 EnsureLogSources 补回来,
|
||||
// 用户看到的现象就是"删不掉"。
|
||||
AutoSourceDisabled bool `json:"autoSourceDisabled"`
|
||||
StorageSize string `json:"storageSize"`
|
||||
GrafanaEnabled bool `json:"grafanaEnabled"`
|
||||
GrafanaVersion string `json:"grafanaVersion"`
|
||||
GrafanaPort int `json:"grafanaPort"`
|
||||
GrafanaEndpoint string `json:"grafanaEndpoint"`
|
||||
Status string `json:"status"`
|
||||
StatusMessage string `json:"statusMessage"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
AdminUser string `json:"adminUser"`
|
||||
AdminPassword string `json:"adminPassword"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (d *LokiDeployment) BeforeCreate() {
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
|
||||
// License Pro 授权。
|
||||
// Source: hmac(旧本地签发,迁移窗口内仍接受)| opsx(opsx-licence LIC-/LICOFF-)。
|
||||
// Plan: free | pro; Edition: personal | team(opsx 侧用 plan_type 映射)。
|
||||
type License struct {
|
||||
Plan string `json:"plan"` // free | pro
|
||||
Edition string `json:"edition"` // personal | team
|
||||
Seats int `json:"seats"` // 历史登录席位字段;产品侧按实例计,勿再展示
|
||||
Instances int `json:"instances"` // 部署实例上限(0 = 不限)
|
||||
Customer string `json:"customer"` // 客户标识
|
||||
// ExpiresAt 0 = 永久;否则 Unix 秒
|
||||
ExpiresAt int64 `json:"expiresAt"`
|
||||
IssuedAt time.Time `json:"issuedAt"`
|
||||
ActivatedAt time.Time `json:"activatedAt"`
|
||||
|
||||
// --- opsx-licence 对接字段(JSON 向后兼容,缺省为零值) ---
|
||||
Source string `json:"source,omitempty"` // hmac | opsx
|
||||
LicenseKey string `json:"licenseKey,omitempty"` // 完整密钥(周检/心跳需要;Sanitized 不外泄)
|
||||
InstanceID string `json:"instanceId,omitempty"` // 机器指纹,无 PII
|
||||
PlanType string `json:"planType,omitempty"` // trial | yearly | perpetual
|
||||
MaxDevices int `json:"maxDevices,omitempty"` // 服务端 max_devices(实例数)
|
||||
LastVerifiedAt int64 `json:"lastVerifiedAt,omitempty"` // unix 秒
|
||||
LastServerTime int64 `json:"lastServerTime,omitempty"` // 防时钟回拨
|
||||
ActivationToken string `json:"activationToken,omitempty"` // 解绑令牌
|
||||
DeprecationNotice string `json:"deprecationNotice,omitempty"` // 旧 HMAC 迁移提示
|
||||
InGrace bool `json:"inGrace,omitempty"`
|
||||
GraceDaysLeft int `json:"graceDaysLeft,omitempty"`
|
||||
// InvalidReason 非空且非有效 Pro 时给前端红条:expired | over_instance | revoked
|
||||
// 服务端明确拒绝才写;禁止静默清空 LicenseKey。
|
||||
InvalidReason string `json:"invalidReason,omitempty"`
|
||||
}
|
||||
|
||||
func (l *License) Sanitized() map[string]interface{} {
|
||||
if l == nil {
|
||||
return map[string]interface{}{"plan": "free"}
|
||||
}
|
||||
expired := l.ExpiresAt > 0 && l.ExpiresAt < time.Now().Unix()
|
||||
plan := l.Plan
|
||||
if expired && !l.InGrace {
|
||||
plan = "free"
|
||||
}
|
||||
out := map[string]interface{}{
|
||||
"plan": plan, "edition": l.Edition, "instances": l.Instances,
|
||||
"customer": l.Customer, "expiresAt": l.ExpiresAt, "expired": expired,
|
||||
}
|
||||
// seats 故意不外抛给产品 UI(按实例计)
|
||||
if l.Source != "" {
|
||||
out["source"] = l.Source
|
||||
}
|
||||
if l.PlanType != "" {
|
||||
out["planType"] = l.PlanType
|
||||
}
|
||||
if l.MaxDevices > 0 {
|
||||
out["maxDevices"] = l.MaxDevices
|
||||
out["instances"] = l.MaxDevices
|
||||
} else if l.Instances > 0 {
|
||||
out["instances"] = l.Instances
|
||||
}
|
||||
if l.DeprecationNotice != "" {
|
||||
out["deprecationNotice"] = l.DeprecationNotice
|
||||
}
|
||||
if l.InGrace {
|
||||
out["inGrace"] = true
|
||||
out["graceDaysLeft"] = l.GraceDaysLeft
|
||||
}
|
||||
if l.LastVerifiedAt > 0 {
|
||||
out["lastVerifiedAt"] = l.LastVerifiedAt
|
||||
}
|
||||
if l.InstanceID != "" {
|
||||
out["instanceId"] = l.InstanceID
|
||||
}
|
||||
if l.InvalidReason != "" {
|
||||
out["invalidReason"] = l.InvalidReason
|
||||
}
|
||||
return out
|
||||
}
|
||||
+106
-3
@@ -6,6 +6,7 @@ import (
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime/pprof"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -24,6 +25,7 @@ type saveRequest struct {
|
||||
// 该包只负责持久化(落盘 + 并发安全),领域实体定义在 models 包。
|
||||
type JsonDB struct {
|
||||
mu sync.RWMutex
|
||||
seqMu sync.Mutex // 独立于 mu:ID 序列不需要与数据锁互相阻塞,避免锁重入死锁
|
||||
dataFile string
|
||||
Users []models.User `json:"users"`
|
||||
Clusters []models.Cluster `json:"clusters"`
|
||||
@@ -42,6 +44,12 @@ type JsonDB struct {
|
||||
AuditLogs []models.AuditLog `json:"auditLogs"`
|
||||
// DailyVolumeSnapshots 每日日志量历史(Loki retention 只有 24h,历史必须自己留)
|
||||
DailyVolumeSnapshots []models.DailyVolumeSnapshot `json:"dailyVolumeSnapshots"`
|
||||
// License Pro 授权(nil = 免费版)
|
||||
License *models.License `json:"license,omitempty"`
|
||||
// AppVolumeSnapshots 按小时留存的 per-app 日志量(长范围排行读这个,不扫日志内容)
|
||||
AppVolumeSnapshots []models.AppVolumeSnapshot `json:"appVolumeSnapshots"`
|
||||
// AIAnalysisRuns AI 分析的审计记录(含脱敏后的 payload,用于合规追溯与每日 token 统计)
|
||||
AIAnalysisRuns []models.AIAnalysisRun `json:"aiAnalysisRuns"`
|
||||
|
||||
// SchemaVersion 数据结构版本号。
|
||||
// 之前加字段全靠 JSON 反序列化的零值兜底,字段一多必然出事
|
||||
@@ -114,6 +122,51 @@ func (d *JsonDB) raiseLegacyAlertThresholds() {
|
||||
}
|
||||
}
|
||||
|
||||
// compactAlertWindows 把告警窗口从 15 分钟压到 5 分钟,阈值按 1/3 同步下调。
|
||||
//
|
||||
// 为什么:检查间隔已经调到 5 分钟,而窗口还是 15 分钟 ——
|
||||
// 相当于同一个 15 分钟窗口每 5 分钟被完整重扫一遍(3 倍重复扫描)。
|
||||
// 在 370GB/天、全命名空间 selector 下,这是 Loki CPU 的最大稳态来源。
|
||||
//
|
||||
// 只改"仍是旧默认值"的规则(窗口 15 且阈值等于旧默认),用户自己调过的不动。
|
||||
func (d *JsonDB) compactAlertWindows() {
|
||||
type change struct {
|
||||
window int
|
||||
threshold int
|
||||
newName string
|
||||
}
|
||||
// 旧默认阈值 → 新值(窗口统一 5 分钟,阈值约 1/3)
|
||||
byOldThreshold := map[string]change{
|
||||
"致命日志 FATAL": {window: 5, threshold: 4},
|
||||
"Go panic": {window: 5, threshold: 4},
|
||||
"JVM OutOfMemoryError": {window: 5, threshold: 2},
|
||||
"StackOverflowError": {window: 5, threshold: 2},
|
||||
"应用错误激增(error > 10万)": {window: 5, threshold: 35000, newName: "应用错误激增(error > 3.5万)"},
|
||||
"ingress-nginx 5xx 错误": {window: 5, threshold: 700},
|
||||
"ingress-nginx 4xx 激增": {window: 5, threshold: 7000},
|
||||
"数据库连接失败": {window: 5, threshold: 350},
|
||||
}
|
||||
for i := range d.AlertRules {
|
||||
rule := &d.AlertRules[i]
|
||||
if rule.WindowMin != 15 {
|
||||
continue // 用户改过窗口,不动
|
||||
}
|
||||
spec, ok := byOldThreshold[rule.Name]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
oldThreshold := rule.Threshold
|
||||
oldWindow := rule.WindowMin
|
||||
rule.WindowMin = spec.window
|
||||
rule.Threshold = spec.threshold
|
||||
if spec.newName != "" {
|
||||
rule.Name = spec.newName
|
||||
}
|
||||
log.Printf("[schema] 告警规则 %q 窗口 %dm → %dm,阈值 %d → %d(消除重复扫描)",
|
||||
rule.Name, oldWindow, rule.WindowMin, oldThreshold, rule.Threshold)
|
||||
}
|
||||
}
|
||||
|
||||
// expandDatabaseErrorPattern 把内置「数据库连接失败」规则升级为覆盖
|
||||
// MySQL / PostgreSQL / Redis 的主流报错文案。
|
||||
//
|
||||
@@ -441,7 +494,7 @@ func (d *JsonDB) dedupeLogSourceIDs() {
|
||||
//
|
||||
// v0/v1(无字段)→ 1:新增 auditLogs / schemaVersion(老库走零值兜底即可,无需处理)
|
||||
// v1 → 2:预留示例(如把 ScrapeJobs 从 logPath 拆分时,需按旧结构生成默认 job)
|
||||
const CurrentSchemaVersion = 10
|
||||
const CurrentSchemaVersion = 11
|
||||
|
||||
// migrateSchema 按版本号顺序迁移老库。返回是否发生了迁移(需要回写)。
|
||||
func (d *JsonDB) migrateSchema() bool {
|
||||
@@ -524,6 +577,12 @@ func (d *JsonDB) migrateSchema() bool {
|
||||
// Redis 的 NOAUTH / WRONGPASS / connection pool timeout 等常见报错)
|
||||
d.expandDatabaseErrorPattern()
|
||||
d.SchemaVersion = 10
|
||||
case 10:
|
||||
// v10 → v11:告警窗口 15 分钟 → 5 分钟(与检查间隔对齐),
|
||||
// 阈值同比例缩小,避免"窗口变短导致更容易触发"。
|
||||
// 目标:消除同一个 15 分钟窗口被重复扫描 3 次的浪费(Loki CPU 降到 1/3 扫描量)。
|
||||
d.compactAlertWindows()
|
||||
d.SchemaVersion = 11
|
||||
default:
|
||||
// 防御:出现未知版本(比当前代码还新)时不乱动,避免降级把数据写坏
|
||||
log.Printf("[schema] 数据库版本 %d 高于当前代码支持版本 %d,跳过迁移", d.SchemaVersion, CurrentSchemaVersion)
|
||||
@@ -561,6 +620,8 @@ func (d *JsonDB) Reload() error {
|
||||
d.LogPolicyGroups = fresh.LogPolicyGroups
|
||||
d.AuditLogs = fresh.AuditLogs
|
||||
d.DailyVolumeSnapshots = fresh.DailyVolumeSnapshots
|
||||
d.AppVolumeSnapshots = fresh.AppVolumeSnapshots
|
||||
d.AIAnalysisRuns = fresh.AIAnalysisRuns
|
||||
d.SchemaVersion = fresh.SchemaVersion
|
||||
d.sequences = map[string]uint{}
|
||||
for _, u := range d.Users {
|
||||
@@ -633,6 +694,16 @@ func (d *JsonDB) Reload() error {
|
||||
d.sequences["dailyVolumeSnapshots"] = v.ID
|
||||
}
|
||||
}
|
||||
for _, v := range d.AppVolumeSnapshots {
|
||||
if v.ID > d.sequences["appVolumeSnapshots"] {
|
||||
d.sequences["appVolumeSnapshots"] = v.ID
|
||||
}
|
||||
}
|
||||
for _, v := range d.AIAnalysisRuns {
|
||||
if v.ID > d.sequences["aiAnalysisRuns"] {
|
||||
d.sequences["aiAnalysisRuns"] = v.ID
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -800,6 +871,8 @@ func InitDB() error {
|
||||
"alertDeliveries": 0,
|
||||
"auditLogs": 0,
|
||||
"dailyVolumeSnapshots": 0,
|
||||
"appVolumeSnapshots": 0,
|
||||
"aiAnalysisRuns": 0,
|
||||
},
|
||||
saveChan: make(chan saveRequest, 10),
|
||||
stopChan: make(chan struct{}),
|
||||
@@ -813,6 +886,8 @@ func InitDB() error {
|
||||
// 启动序列化保存goroutine
|
||||
d.wg.Add(1)
|
||||
go d.saveWorker()
|
||||
// 写锁看门狗:JSON DB 锁一旦被长时间占用,所有读接口会静默挂起
|
||||
d.StartLockWatchdog(10 * time.Second)
|
||||
|
||||
db = d
|
||||
if migratedLogAccess || migratedSchema {
|
||||
@@ -1008,12 +1083,40 @@ func (d *JsonDB) RequestSaveSync() error {
|
||||
}
|
||||
|
||||
func (d *JsonDB) NextID(table string) uint {
|
||||
d.mu.Lock()
|
||||
defer d.mu.Unlock()
|
||||
// 使用独立的 seqMu:调用方即使持有 d.Lock() 也不会死锁(mu 不可重入)。
|
||||
d.seqMu.Lock()
|
||||
defer d.seqMu.Unlock()
|
||||
d.sequences[table]++
|
||||
return d.sequences[table]
|
||||
}
|
||||
|
||||
// StartLockWatchdog 监控写锁是否被长时间占用。
|
||||
//
|
||||
// JSON DB 的写锁是全进程共享的:一旦某个 goroutine 持锁阻塞(网络、磁盘或
|
||||
// 锁重入死锁),所有读库接口(登录、列表、部署)都会静默挂起,而 /health
|
||||
// 不读库仍然正常,表现就是「页面一直转圈、后端像挂了」。
|
||||
// 这里定期 TryLock,连续多次失败就转储 goroutine 栈,让问题在日志里可见。
|
||||
func (d *JsonDB) StartLockWatchdog(interval time.Duration) {
|
||||
go func() {
|
||||
stuck := 0
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
for range ticker.C {
|
||||
if d.mu.TryLock() {
|
||||
d.mu.Unlock()
|
||||
stuck = 0
|
||||
continue
|
||||
}
|
||||
stuck++
|
||||
if stuck >= 3 {
|
||||
log.Printf("[db] 写锁持续被占用约 %s,转储 goroutine 栈辅助定位", time.Duration(stuck)*interval)
|
||||
_ = pprof.Lookup("goroutine").WriteTo(os.Stderr, 1)
|
||||
stuck = 0
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
func (d *JsonDB) Lock() {
|
||||
d.mu.Lock()
|
||||
}
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TestNextIDWhileWriteLocked 回归测试:调用方持有写锁时 NextID 不能死锁。
|
||||
//
|
||||
// 历史问题:NextID 内部用 d.mu 加锁,而 CaptureAppVolumeForHour 先 db.Lock()
|
||||
// 再 db.NextID(),Go 的 sync.RWMutex 不可重入,导致该 goroutine 永久持有写锁,
|
||||
// 所有读库接口(登录、列表)全部挂起,而 /health 仍然正常。
|
||||
func TestNextIDWhileWriteLocked(t *testing.T) {
|
||||
db := &JsonDB{sequences: map[string]uint{}}
|
||||
db.Lock()
|
||||
|
||||
done := make(chan uint, 1)
|
||||
go func() { done <- db.NextID("test") }()
|
||||
|
||||
select {
|
||||
case id := <-done:
|
||||
db.Unlock()
|
||||
if id != 1 {
|
||||
t.Fatalf("first id = %d, want 1", id)
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
db.Unlock()
|
||||
t.Fatal("NextID deadlocked while write lock is held")
|
||||
}
|
||||
|
||||
// 解锁后继续分配应递增
|
||||
if id := db.NextID("test"); id != 2 {
|
||||
t.Fatalf("second id = %d, want 2", id)
|
||||
}
|
||||
}
|
||||
+76
-10
@@ -10,6 +10,7 @@ import (
|
||||
"lokistack/controllers"
|
||||
"lokistack/middleware"
|
||||
"lokistack/pkg/metrics"
|
||||
"lokistack/services"
|
||||
|
||||
"github.com/gin-contrib/cors"
|
||||
"github.com/gin-gonic/gin"
|
||||
@@ -50,6 +51,7 @@ func SetupRouter() *gin.Engine {
|
||||
auditCtrl := controllers.NewAuditController()
|
||||
systemCtrl := controllers.NewSystemController()
|
||||
registryCtrl := controllers.NewImageRegistryController()
|
||||
aiCtrl := controllers.NewAIController()
|
||||
|
||||
// API v1
|
||||
api := r.Group("/api/v1")
|
||||
@@ -95,6 +97,9 @@ func SetupRouter() *gin.Engine {
|
||||
logs.POST("/context", logsCtrl.Context)
|
||||
logs.GET("/labels", logsCtrl.Labels)
|
||||
logs.GET("/label/:name/values", logsCtrl.LabelValues)
|
||||
// 批量取 label 值:一次请求拿多个 label(app+service_name+container 三级回退),
|
||||
// 前端下拉联动从 3 个往返降到 1 个
|
||||
logs.GET("/label-values", logsCtrl.LabelValuesBatch)
|
||||
logs.GET("/my-scope", logsCtrl.MyScope)
|
||||
// ⚠️ 必须是 POST:前端 logsApi.histogram 用 POST 传 body(sourceId/query/start/end)。
|
||||
// 曾被误改成 GET,导致 POST /logs/histogram 全部 404,
|
||||
@@ -103,9 +108,11 @@ func SetupRouter() *gin.Engine {
|
||||
logs.POST("/histogram", logsCtrl.Histogram)
|
||||
logs.GET("/histogram", logsCtrl.Histogram)
|
||||
logs.GET("/daily-volume", logsCtrl.DailyVolume)
|
||||
// 应用排行:长范围读平台预聚合快照(毫秒级),不扫日志内容
|
||||
logs.GET("/app-ranking", logsCtrl.AppRanking)
|
||||
// 总量统计(index/stats):只读索引,用于算总页数
|
||||
logs.POST("/stats", logsCtrl.Stats)
|
||||
logs.POST("/metrics", middleware.AdminAuth(), logsCtrl.Metrics)
|
||||
logs.POST("/metrics", middleware.AdminAuth(), middleware.RequirePro(), logsCtrl.Metrics)
|
||||
logs.GET("/health", logsCtrl.Health)
|
||||
}
|
||||
|
||||
@@ -115,6 +122,18 @@ func SetupRouter() *gin.Engine {
|
||||
|
||||
// 镜像源(国内加速/海外/私有仓库)
|
||||
authorized.GET("/settings/image-registry", registryCtrl.Get)
|
||||
|
||||
// ── AIOps:AI 日志分析 ──────────────────────────────────────
|
||||
// 模型配置仅管理员可改(涉及密钥与费用);分析本身对所有有数据源权限的用户开放。
|
||||
ai := authorized.Group("/ai")
|
||||
{
|
||||
ai.GET("/config", middleware.AdminAuth(), aiCtrl.GetConfig)
|
||||
ai.PUT("/config", middleware.AdminAuth(), aiCtrl.SaveConfig)
|
||||
ai.POST("/config/test", middleware.AdminAuth(), aiCtrl.TestConfig)
|
||||
ai.POST("/chat", middleware.AdminAuth(), middleware.RequirePro(), aiCtrl.Chat)
|
||||
ai.GET("/runs", aiCtrl.ListRuns)
|
||||
ai.GET("/runs/:id", aiCtrl.GetRun)
|
||||
}
|
||||
authorized.PUT("/settings/image-registry", middleware.AdminAuth(), registryCtrl.Save)
|
||||
|
||||
// 告警:飞书通知配置 + 告警规则
|
||||
@@ -124,14 +143,14 @@ func SetupRouter() *gin.Engine {
|
||||
alertNotify.PUT("/notify-config", middleware.AdminAuth(), alertsCtrl.UpdateNotifyConfig)
|
||||
alertNotify.POST("/notify-config/test", middleware.AdminAuth(), alertsCtrl.TestNotifyConfig)
|
||||
alertNotify.GET("/rules", alertsCtrl.ListRules)
|
||||
alertNotify.POST("/rules", middleware.AdminAuth(), alertsCtrl.CreateRule)
|
||||
alertNotify.PUT("/rules/:id", middleware.AdminAuth(), alertsCtrl.UpdateRule)
|
||||
alertNotify.POST("/rules", middleware.AdminAuth(), middleware.RequirePro(), alertsCtrl.CreateRule)
|
||||
alertNotify.PUT("/rules/:id", middleware.AdminAuth(), middleware.RequirePro(), alertsCtrl.UpdateRule)
|
||||
alertNotify.DELETE("/rules/:id", middleware.AdminAuth(), alertsCtrl.DeleteRule)
|
||||
alertNotify.POST("/check", middleware.AdminAuth(), alertsCtrl.CheckNow)
|
||||
alertNotify.POST("/check", middleware.AdminAuth(), middleware.RequirePro(), alertsCtrl.CheckNow)
|
||||
alertNotify.GET("/deliveries", alertsCtrl.Deliveries)
|
||||
alertNotify.GET("/channels", alertsCtrl.ListChannels)
|
||||
alertNotify.POST("/channels", middleware.AdminAuth(), alertsCtrl.SaveChannel)
|
||||
alertNotify.PUT("/channels/:id", middleware.AdminAuth(), alertsCtrl.SaveChannel)
|
||||
alertNotify.POST("/channels", middleware.AdminAuth(), middleware.RequirePro(), alertsCtrl.SaveChannel)
|
||||
alertNotify.PUT("/channels/:id", middleware.AdminAuth(), middleware.RequirePro(), alertsCtrl.SaveChannel)
|
||||
alertNotify.DELETE("/channels/:id", middleware.AdminAuth(), alertsCtrl.DeleteChannel)
|
||||
alertNotify.POST("/channels/test", middleware.AdminAuth(), alertsCtrl.TestChannel)
|
||||
}
|
||||
@@ -147,11 +166,10 @@ func SetupRouter() *gin.Engine {
|
||||
system := authorized.Group("/system")
|
||||
system.Use(middleware.AdminAuth())
|
||||
{
|
||||
system.GET("/backup", systemCtrl.Backup)
|
||||
system.POST("/restore", systemCtrl.Restore)
|
||||
system.POST("/restore/apply", systemCtrl.RestoreApply)
|
||||
system.GET("/backups", systemCtrl.Backups)
|
||||
system.GET("/self-check", systemCtrl.Health)
|
||||
system.GET("/license", systemCtrl.LicenseStatus)
|
||||
system.POST("/license/activate", middleware.AdminAuth(), systemCtrl.ActivateLicense)
|
||||
system.POST("/license/deactivate", middleware.AdminAuth(), systemCtrl.DeactivateLicense)
|
||||
}
|
||||
|
||||
// 审计日志:仅管理员可查(含他人操作记录)
|
||||
@@ -234,6 +252,20 @@ func SetupRouter() *gin.Engine {
|
||||
authorized.POST("/collectors/:id/uninstall", collectorCtrl.Uninstall)
|
||||
authorized.POST("/collectors/:id/preview", collectorCtrl.Preview)
|
||||
authorized.GET("/collectors/:id/logs", collectorCtrl.Logs)
|
||||
// 采集项 YAML 双向转换(方案 A:只开放采集规则,不开放 clients/positions)
|
||||
authorized.POST("/collector-jobs/render", collectorCtrl.RenderJobsYAML)
|
||||
authorized.POST("/collector-jobs/parse", collectorCtrl.ParseJobsYAML)
|
||||
|
||||
// 文档站(docsify,docs/)与落地页(homepage/)随二进制一起发布;
|
||||
// 独立于前端 dist 存在与否注册,目录不存在时路由不生效
|
||||
docsCtrl := controllers.NewDocsController()
|
||||
if services.DocsBaseDir() != "" {
|
||||
r.GET("/docs", docsCtrl.Home)
|
||||
r.GET("/docs/search-data.json", docsCtrl.SearchData)
|
||||
r.GET("/docs/:slug", docsCtrl.Page)
|
||||
r.Static("/docs/assets", filepath.Join(services.DocsBaseDir(), "assets"))
|
||||
}
|
||||
serveStaticDir(r, "/homepage", "homepage")
|
||||
|
||||
// Static files - use executable directory as base
|
||||
distPath := findDistPath()
|
||||
@@ -267,6 +299,40 @@ func SetupRouter() *gin.Engine {
|
||||
return r
|
||||
}
|
||||
|
||||
// serveStaticDir 把本地目录挂为只读静态站点:目录回退到 fallbackFile(SPA/docsify 需要)。
|
||||
// baseDir 候选与 findDistPath 同源:可执行文件相对路径优先,再退工作目录。
|
||||
func serveStaticDir(r *gin.Engine, urlPrefix, dirName string) {
|
||||
root := findAuxDir(dirName)
|
||||
if root == "" {
|
||||
return
|
||||
}
|
||||
// http.FileServer 自带目录回退:访问 /docs 自动 301 到 /docs/ 并命中 index.html
|
||||
r.Static(urlPrefix, root)
|
||||
}
|
||||
|
||||
// findAuxDir 定位 dist 之外的附属静态目录(docs/homepage),规则与 findDistPath 对齐
|
||||
func findAuxDir(dirName string) string {
|
||||
if ex, err := os.Executable(); err == nil {
|
||||
execDir := filepath.Dir(ex)
|
||||
if !strings.Contains(execDir, "/tmp/go-build") {
|
||||
for _, rel := range []string{dirName, "../" + dirName, "../../" + dirName} {
|
||||
p := filepath.Join(execDir, rel)
|
||||
if _, err := os.Stat(filepath.Join(p, "index.html")); err == nil {
|
||||
return p
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, p := range []string{"./" + dirName, "../" + dirName, "/app/" + dirName} {
|
||||
if abs, err := filepath.Abs(p); err == nil {
|
||||
if _, err := os.Stat(filepath.Join(abs, "index.html")); err == nil {
|
||||
return abs
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// findDistPath finds the frontend dist directory
|
||||
func findDistPath() string {
|
||||
// Try relative to executable
|
||||
|
||||
@@ -34,6 +34,8 @@ func TestCriticalRoutes(t *testing.T) {
|
||||
{"GET", "/api/v1/logs/sources"},
|
||||
{"GET", "/api/v1/logs/labels"},
|
||||
{"GET", "/api/v1/logs/label/:name/values"},
|
||||
// 批量 label 值(下拉联动一次请求)—— 路由写错就是前端全部下拉空
|
||||
{"GET", "/api/v1/logs/label-values"},
|
||||
{"GET", "/api/v1/logs/my-scope"},
|
||||
{"GET", "/api/v1/logs/daily-volume"},
|
||||
{"GET", "/api/v1/logs/health"},
|
||||
@@ -43,10 +45,23 @@ func TestCriticalRoutes(t *testing.T) {
|
||||
{"GET", "/api/v1/alerts/channels"},
|
||||
{"POST", "/api/v1/alerts/check"},
|
||||
{"GET", "/api/v1/alerts/deliveries"},
|
||||
// 审计 / 自检 / 备份
|
||||
// 审计 / 自检
|
||||
{"GET", "/api/v1/audit-logs"},
|
||||
{"GET", "/api/v1/system/self-check"},
|
||||
{"GET", "/api/v1/system/backup"},
|
||||
// 设置类:保留期必须同时有读和写(写要管理员)
|
||||
{"GET", "/api/v1/settings/retention"},
|
||||
{"PUT", "/api/v1/settings/retention"},
|
||||
// AIOps:模型配置(管理员)+ 分析 + 审计历史
|
||||
{"GET", "/api/v1/ai/config"},
|
||||
{"PUT", "/api/v1/ai/config"},
|
||||
{"POST", "/api/v1/ai/config/test"},
|
||||
// 重构后:整页分析改为浮窗助手(POST /ai/chat,admin + Pro 闸门)
|
||||
{"POST", "/api/v1/ai/chat"},
|
||||
{"GET", "/api/v1/system/license"},
|
||||
{"POST", "/api/v1/system/license/activate"},
|
||||
{"GET", "/api/v1/ai/runs"},
|
||||
{"GET", "/api/v1/ai/runs/:id"},
|
||||
// 备份恢复接口已在上游被移除(commit f87fb4b),这里不再断言
|
||||
}
|
||||
|
||||
registered := map[string]bool{}
|
||||
|
||||
@@ -8,6 +8,7 @@ package seed
|
||||
// 通道未就绪时命中只会把失败原因写进发送历史,不影响规则评估。
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
@@ -69,10 +70,12 @@ func channelIDsFor(id uint) []uint {
|
||||
|
||||
var builtinRules = []builtinRule{
|
||||
// —— 致命类:这类日志本该极少出现,但也不能一条就报 ——
|
||||
{name: "致命日志 FATAL", query: `{namespace=~".+"} |= "FATAL"`, windowMin: 15, threshold: 10, cooldown: 120},
|
||||
{name: "Go panic", query: `{namespace=~".+"} |= "panic:"`, windowMin: 15, threshold: 10, cooldown: 120},
|
||||
{name: "JVM OutOfMemoryError", query: `{namespace=~".+"} |~ "(^|[^a-zA-Z])OutOfMemoryError"`, windowMin: 15, threshold: 5, cooldown: 120},
|
||||
{name: "StackOverflowError", query: `{namespace=~".+"} |= "StackOverflowError"`, windowMin: 15, threshold: 5, cooldown: 120},
|
||||
// 全命名空间类查询一律排除 app=loki:Loki 会把执行过的查询语句记成自身 info 日志,
|
||||
// 查询文本包含规则关键字时告警会"命中自己",造成持续误报。
|
||||
{name: "致命日志 FATAL", query: `{namespace=~".+", app!="loki"} |= "FATAL"`, windowMin: 5, threshold: 4, cooldown: 120},
|
||||
{name: "Go panic", query: `{namespace=~".+", app!="loki"} |= "panic:"`, windowMin: 5, threshold: 4, cooldown: 120},
|
||||
{name: "JVM OutOfMemoryError", query: `{namespace=~".+", app!="loki"} |~ "(^|[^a-zA-Z])OutOfMemoryError"`, windowMin: 5, threshold: 2, cooldown: 120},
|
||||
{name: "StackOverflowError", query: `{namespace=~".+", app!="loki"} |= "StackOverflowError"`, windowMin: 5, threshold: 2, cooldown: 120},
|
||||
// 注意:这里**只放日志类告警**。
|
||||
// 「节点 NotReady」「容器 OOMKilled」属于基础设施指标监控(kube-state-metrics /
|
||||
// Prometheus)的职责,用日志去匹配这些字符串既不可靠(日志没开或轮转就漏),
|
||||
@@ -81,19 +84,24 @@ var builtinRules = []builtinRule{
|
||||
// 第二轮上调(500~5000 这一档实测仍然太敏感):
|
||||
// 这几个量级在业务高峰期本来就是常态,阈值贴着基线就等于"每天准点报警"。
|
||||
// 原则:宁可只报"明显异常",也不要让人对告警脱敏。
|
||||
{name: "应用错误激增(error > 10万)", query: `{namespace=~".+"} |~ "(?i)\berror\b"`, windowMin: 15, threshold: 100000, cooldown: 240},
|
||||
{name: "ingress-nginx 5xx 错误", query: `{app=~"ingress-nginx|nginx-ingress-controller"} |~ " (50[0-4]|5[2-9][0-9]) "`, windowMin: 15, threshold: 2000, cooldown: 240},
|
||||
{name: "ingress-nginx 4xx 激增", query: `{app=~"ingress-nginx|nginx-ingress-controller"} |~ " 4[0-9][0-9] "`, windowMin: 15, threshold: 20000, cooldown: 240},
|
||||
{name: "应用错误激增(error > 3.5万)", query: `{namespace=~".+", app!="loki"} |~ "(?i)\berror\b"`, windowMin: 5, threshold: 35000, cooldown: 240},
|
||||
{name: "ingress-nginx 5xx 错误", query: `{app=~"ingress-nginx|nginx-ingress-controller"} |~ " (50[0-4]|5[2-9][0-9]) "`, windowMin: 5, threshold: 700, cooldown: 240},
|
||||
{name: "ingress-nginx 4xx 激增", query: `{app=~"ingress-nginx|nginx-ingress-controller"} |~ " 4[0-9][0-9] "`, windowMin: 5, threshold: 7000, cooldown: 240},
|
||||
// 数据库连接类错误:覆盖 MySQL / PostgreSQL / Redis 的主流报错文案
|
||||
// (详见函数上方注释,不要加裸 "EOF" / "timeout",那会匹配到几乎所有网络日志)
|
||||
{name: "数据库连接失败", query: `{namespace=~".+"} |~ "(?i)(connection refused|connection reset by peer|broken pipe|no route to host|cannot assign requested address|connection timed out|i/o timeout|too many connections|too many clients already|max number of clients reached|connection pool|connection is not available|could not create connection|communications link failure|server closed the connection unexpectedly|unexpected EOF|driver: bad connection|FATAL:\\s*(password authentication failed|sorry, too many clients|terminating connection)|NOAUTH|WRONGPASS|READONLY you can.t write|CLUSTERDOWN)"`, windowMin: 15, threshold: 1000, cooldown: 240},
|
||||
{name: "数据库连接失败", query: `{namespace=~".+", app!="loki"} |~ "(?i)(connection refused|connection reset by peer|broken pipe|no route to host|cannot assign requested address|connection timed out|i/o timeout|too many connections|too many clients already|max number of clients reached|connection pool|connection is not available|could not create connection|communications link failure|server closed the connection unexpectedly|unexpected EOF|driver: bad connection|FATAL:\\s*(password authentication failed|sorry, too many clients|terminating connection)|NOAUTH|WRONGPASS|READONLY you can.t write|CLUSTERDOWN)"`, windowMin: 5, threshold: 350, cooldown: 240},
|
||||
}
|
||||
|
||||
// seedBuiltinAlertRules 为每个平台管理的 LogSource(managed=true)补齐内置规则:
|
||||
// 首次部署 Loki 后即有一套可直接启用的告警模板;通知通道留空,管理员配置后启用。
|
||||
// legacyInfraRules 早期版本误种的 Kubernetes 基础设施类规则 —— 日志平台只关注应用日志,
|
||||
// 这类节点/容器状态告警属 Prometheus + Alertmanager 职责,升级时一次性清理存量。
|
||||
var legacyInfraRules = []string{"Kubernetes 节点 NotReady", "容器 OOMKilled"}
|
||||
var legacyInfraRules = []string{
|
||||
"Kubernetes 节点 NotReady",
|
||||
"容器 OOMKilled",
|
||||
// 旧版 seed 遗留:与现模板「应用错误激增」重复,且 1000 阈值实测会误报平台自身查询日志
|
||||
"应用错误过多(error > 1000)",
|
||||
}
|
||||
|
||||
// SeedAlertRules 补齐内置告警规则(幂等;公开给启动与部署成功两处调用)
|
||||
func SeedAlertRules() {
|
||||
@@ -113,6 +121,11 @@ func SeedAlertRules() {
|
||||
changed = true
|
||||
continue
|
||||
}
|
||||
if builtinNames[rule.Name] && strings.Contains(rule.Query, `{namespace=~".+"}`) && !strings.Contains(rule.Query, `app!="loki"`) {
|
||||
rule.Query = strings.ReplaceAll(rule.Query, `{namespace=~".+"}`, `{namespace=~".+", app!="loki"}`)
|
||||
rule.UpdatedAt = time.Now()
|
||||
changed = true
|
||||
}
|
||||
if builtinNames[rule.Name] && len(rule.ChannelIDs) == 0 {
|
||||
rule.ChannelIDs = channelIDsFor(defaultChannelID)
|
||||
if !rule.Enabled {
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
package seed
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestBuiltinRulesExcludeLokiSelfLogs 全命名空间类内置规则必须排除 app=loki:
|
||||
// Loki 会把执行过的查询语句记成自身 info 日志,查询文本包含规则关键字时
|
||||
// 告警会"命中自己"(自我污染),造成持续误报——用测试锁死这条红线。
|
||||
func TestBuiltinRulesExcludeLokiSelfLogs(t *testing.T) {
|
||||
for _, r := range builtinRules {
|
||||
if strings.Contains(r.query, `{namespace=~".+"}`) && !strings.Contains(r.query, `app!="loki"`) {
|
||||
t.Fatalf("内置规则 %q 使用全命名空间选择器但未排除 app=loki(自我污染): %s", r.name, r.query)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
// AIAnalysisResult 结构化结论。
|
||||
//
|
||||
// 强制模型输出这个结构而不是自由文本,原因:
|
||||
// 1. 前端能分区渲染(结论/证据/建议/不确定),可读性远好于一段散文;
|
||||
// 2. 每条结论都要挂 evidence,逼模型把话落在数据上,降低编造;
|
||||
// 3. unknowns 单独一栏,明确"不知道什么"比编一个答案有价值得多。
|
||||
type AIAnalysisResult struct {
|
||||
Summary string `json:"summary"`
|
||||
Findings []AIFinding `json:"findings"`
|
||||
Actions []string `json:"actions"`
|
||||
Unknowns []string `json:"unknowns"`
|
||||
RawText string `json:"-"`
|
||||
ParseErr string `json:"parseError,omitempty"`
|
||||
}
|
||||
|
||||
type AIFinding struct {
|
||||
Title string `json:"title"`
|
||||
Evidence string `json:"evidence"`
|
||||
Severity string `json:"severity"` // high | medium | low
|
||||
}
|
||||
|
||||
// AIAnalysisOutcome 一次分析的完整返回(含成本信息)。
|
||||
type AIAnalysisOutcome struct {
|
||||
Result AIAnalysisResult `json:"result"`
|
||||
Signals *AISignalBundle `json:"signals"`
|
||||
Prompt string `json:"prompt"`
|
||||
Run models.AIAnalysisRun `json:"run"`
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
const aiSystemPrompt = `你是资深 SRE,负责分析 Loki 日志平台的运行状况。
|
||||
|
||||
严格规则:
|
||||
1. 只能基于我给你的"信号数据"下结论。**不许编造**日志内容、错误信息、时间点或服务名。
|
||||
2. 每一条结论都必须在 evidence 里引用信号数据中的具体数字或条目。
|
||||
3. 如果信号数据不足以判断根因,把它写进 unknowns,不要猜。
|
||||
4. 你看不到日志原文(只有统计与排行),因此不要描述"日志里写了什么"。
|
||||
5. 输出必须是**单个 JSON 对象**,不要 markdown 代码块,不要任何解释性前后缀。
|
||||
|
||||
JSON 结构:
|
||||
{
|
||||
"summary": "两三句话的整体判断",
|
||||
"findings": [{"title": "简短结论", "evidence": "引用具体数据", "severity": "high|medium|low"}],
|
||||
"actions": ["可执行的下一步动作,按优先级排序"],
|
||||
"unknowns": ["无法从当前数据判断的点"]
|
||||
}`
|
||||
|
||||
// RunAIAnalysis 执行一次分析:采集信号 → 脱敏 → 调模型 → 解析 → 落审计。
|
||||
func RunAIAnalysis(ctx context.Context, cfg models.AIConfig, src *models.LogSource, userID uint, username string, admin bool, from, to time.Time, question, app string) (*AIAnalysisOutcome, error) {
|
||||
if !cfg.Enabled {
|
||||
return nil, fmt.Errorf("AI 分析未启用:请先在「智能运维 → 模型配置」中开启并保存")
|
||||
}
|
||||
|
||||
// ── 每日配额闸门(成本控制第一层)──
|
||||
if cfg.DailyTokenCap > 0 {
|
||||
used := AITokenUsedToday(userID)
|
||||
if used >= cfg.DailyTokenCap {
|
||||
run := models.AIAnalysisRun{
|
||||
UserID: userID, Username: username, SourceID: src.ID, SourceName: src.Name,
|
||||
Status: "quota-exceeded", Error: fmt.Sprintf("今日 AI token 用量已达上限(%d/%d)", used, cfg.DailyTokenCap),
|
||||
CreatedAt: time.Now(),
|
||||
}
|
||||
persistAIRun(&run)
|
||||
return nil, fmt.Errorf("今日 AI 用量已达上限(%d/%d tokens),可在模型配置中调整上限或明天再试", used, cfg.DailyTokenCap)
|
||||
}
|
||||
}
|
||||
|
||||
started := time.Now()
|
||||
signals, err := CollectAISignals(ctx, src, userID, admin, from, to, app)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// 扫描预算闸门:P0 全是索引/快照,实际扫描为 0,这里留出扩展位
|
||||
if cfg.MaxScanBytes > 0 && signals.TotalBytes > cfg.MaxScanBytes*8 {
|
||||
signals.Warnings = append(signals.Warnings,
|
||||
fmt.Sprintf("该范围日志体积 %.1f GB 偏大,已只使用索引统计(未读取日志内容)",
|
||||
float64(signals.TotalBytes)/1024/1024/1024))
|
||||
}
|
||||
|
||||
// ── 脱敏(第二层:隐私)──
|
||||
rawPrompt := SignalsAsText(signals)
|
||||
redacted, invalidRules := RedactText(rawPrompt, cfg.RedactExtra)
|
||||
if question = strings.TrimSpace(question); question != "" {
|
||||
redacted += "\n\n运维补充的问题:" + RedactTextOnly(question, cfg.RedactExtra)
|
||||
}
|
||||
warnings := append([]string{}, signals.Warnings...)
|
||||
for _, r := range invalidRules {
|
||||
warnings = append(warnings, "额外脱敏规则无效已跳过:"+r)
|
||||
}
|
||||
|
||||
// ── 调模型 ──
|
||||
reply, promptTokens, completionTokens, callErr := aiChat(ctx, cfg, []aiMessage{
|
||||
{Role: "system", Content: aiSystemPrompt},
|
||||
{Role: "user", Content: redacted},
|
||||
})
|
||||
|
||||
run := models.AIAnalysisRun{
|
||||
UserID: userID, Username: username, SourceID: src.ID, SourceName: src.Name,
|
||||
RangeMinutes: signals.RangeMinutes, Model: cfg.Model, Question: question,
|
||||
PromptPayload: truncateStr(redacted, 20000),
|
||||
PromptTokens: promptTokens, CompletionTokens: completionTokens,
|
||||
ScanBytes: 0, // P0 不扫内容
|
||||
DurationMs: time.Since(started).Milliseconds(),
|
||||
CreatedAt: time.Now(),
|
||||
}
|
||||
if callErr != nil {
|
||||
run.Status = "failed"
|
||||
run.Error = callErr.Error()
|
||||
run.DurationMs = time.Since(started).Milliseconds()
|
||||
persistAIRun(&run)
|
||||
return nil, callErr
|
||||
}
|
||||
|
||||
result := parseAIResult(reply)
|
||||
conclusion, _ := json.Marshal(result)
|
||||
run.Status = "ok"
|
||||
run.Conclusion = string(conclusion)
|
||||
run.DurationMs = time.Since(started).Milliseconds()
|
||||
persistAIRun(&run)
|
||||
|
||||
return &AIAnalysisOutcome{
|
||||
Result: result, Signals: signals, Prompt: redacted, Run: run, Warnings: warnings,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// RedactTextOnly 只脱敏不做规则校验提示(用于短文本如用户提问)。
|
||||
func RedactTextOnly(text string, extra []string) string {
|
||||
out, _ := RedactText(text, extra)
|
||||
return out
|
||||
}
|
||||
|
||||
// parseAIResult 宽松解析模型输出。
|
||||
//
|
||||
// 模型常见的"不听话"有三种:包了 ```json 代码块、前后加了解释、
|
||||
// 直接返回纯文本。前两种剥壳后重试,第三种降级为 summary=原文,
|
||||
// 保证用户至少能看到内容,而不是一个"解析失败"。
|
||||
func parseAIResult(reply string) AIAnalysisResult {
|
||||
text := strings.TrimSpace(reply)
|
||||
text = strings.TrimPrefix(text, "```json")
|
||||
text = strings.TrimPrefix(text, "```")
|
||||
text = strings.TrimSuffix(text, "```")
|
||||
text = strings.TrimSpace(text)
|
||||
if idx := strings.Index(text, "{"); idx > 0 {
|
||||
text = text[idx:]
|
||||
}
|
||||
if end := strings.LastIndex(text, "}"); end >= 0 && end < len(text)-1 {
|
||||
text = text[:end+1]
|
||||
}
|
||||
|
||||
var parsed AIAnalysisResult
|
||||
if err := json.Unmarshal([]byte(text), &parsed); err == nil && (parsed.Summary != "" || len(parsed.Findings) > 0) {
|
||||
return parsed
|
||||
}
|
||||
return AIAnalysisResult{
|
||||
Summary: strings.TrimSpace(reply),
|
||||
Findings: []AIFinding{},
|
||||
Actions: []string{},
|
||||
Unknowns: []string{},
|
||||
RawText: reply,
|
||||
ParseErr: "模型未按 JSON 格式返回,已按纯文本展示",
|
||||
}
|
||||
}
|
||||
|
||||
// persistAIRun 落审计(同时用于每日 token 统计)。
|
||||
func persistAIRun(run *models.AIAnalysisRun) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
newID := db.NextID("aiAnalysisRuns")
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
run.ID = newID
|
||||
db.AIAnalysisRuns = append(db.AIAnalysisRuns, *run)
|
||||
// 只保留最近 500 条,避免审计表无限膨胀(明细可通过日志追溯)
|
||||
if len(db.AIAnalysisRuns) > 500 {
|
||||
db.AIAnalysisRuns = db.AIAnalysisRuns[len(db.AIAnalysisRuns)-500:]
|
||||
}
|
||||
db.RequestSave()
|
||||
}
|
||||
|
||||
// ListAIRuns 分页查询审计记录(按时间倒序)。
|
||||
func ListAIRuns(page, pageSize int) ([]models.AIAnalysisRun, int) {
|
||||
if page <= 0 {
|
||||
page = 1
|
||||
}
|
||||
if pageSize <= 0 || pageSize > 100 {
|
||||
pageSize = 20
|
||||
}
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return nil, 0
|
||||
}
|
||||
db.RLock()
|
||||
all := make([]models.AIAnalysisRun, len(db.AIAnalysisRuns))
|
||||
copy(all, db.AIAnalysisRuns)
|
||||
db.RUnlock()
|
||||
|
||||
total := len(all)
|
||||
// 倒序(新的在前)
|
||||
for i, j := 0, len(all)-1; i < j; i, j = i+1, j-1 {
|
||||
all[i], all[j] = all[j], all[i]
|
||||
}
|
||||
start := (page - 1) * pageSize
|
||||
if start >= total {
|
||||
return []models.AIAnalysisRun{}, total
|
||||
}
|
||||
end := start + pageSize
|
||||
if end > total {
|
||||
end = total
|
||||
}
|
||||
return all[start:end], total
|
||||
}
|
||||
|
||||
// GetAIRun 取单条审计记录(用于查看完整 payload)。
|
||||
func GetAIRun(id uint) *models.AIAnalysisRun {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return nil
|
||||
}
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
for i := range db.AIAnalysisRuns {
|
||||
if db.AIAnalysisRuns[i].ID == id {
|
||||
run := db.AIAnalysisRuns[i]
|
||||
return &run
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestChatHistoryBudgetTrim 历史按字符预算裁剪:超预算时最旧被丢、标记注入。
|
||||
// 通过构造超长历史走 RunAIChat 会因未启用而提前返回,这里直接测裁剪语义的独立函数化验证——
|
||||
// 为可测性把裁剪抽成了 trimHistory。
|
||||
func TestChatHistoryBudgetTrim(t *testing.T) {
|
||||
long := strings.Repeat("a", 15000)
|
||||
history := []ChatMessage{
|
||||
{Role: "user", Content: long},
|
||||
{Role: "assistant", Content: long},
|
||||
{Role: "user", Content: "最后一句"},
|
||||
}
|
||||
kept, truncated := trimHistory(history, 24000)
|
||||
if !truncated {
|
||||
t.Fatal("超出预算应标记 truncated")
|
||||
}
|
||||
if len(kept) != 2 {
|
||||
t.Fatalf("应保留最近 2 条(丢弃最旧的 15000 字条目),got %d", len(kept))
|
||||
}
|
||||
if kept[0].Content != long || kept[len(kept)-1].Content != "最后一句" {
|
||||
t.Fatal("应保留最近的消息")
|
||||
}
|
||||
// 不超预算时全保留
|
||||
small := []ChatMessage{{Role: "user", Content: "hi"}, {Role: "assistant", Content: "ok"}}
|
||||
kept2, truncated2 := trimHistory(small, 24000)
|
||||
if truncated2 || len(kept2) != 2 {
|
||||
t.Fatalf("小历史不应裁剪: %v %d", truncated2, len(kept2))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/pkg/crypto"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
const aiConfigSettingKey = "ai_config"
|
||||
|
||||
// 成本相关的默认值(宁可保守:额度小、扫描少,用户觉得不够再调)
|
||||
const (
|
||||
defaultAIMaxTokens = 1500
|
||||
defaultAITimeoutSec = 120
|
||||
defaultAITemperature = 0.1 // 分析场景要稳定复现,不要发挥
|
||||
defaultAIDailyTokenCap = 500000
|
||||
defaultAIMaxScanBytes = 2 << 30 // 2GiB
|
||||
)
|
||||
|
||||
// GetAIConfig 读配置(返回真实 APIKey,仅供内部调用)。
|
||||
func GetAIConfig() models.AIConfig {
|
||||
cfg := models.AIConfig{
|
||||
Provider: "openai-compatible",
|
||||
Temperature: defaultAITemperature,
|
||||
MaxTokens: defaultAIMaxTokens,
|
||||
TimeoutSec: defaultAITimeoutSec,
|
||||
DailyTokenCap: defaultAIDailyTokenCap,
|
||||
MaxScanBytes: defaultAIMaxScanBytes,
|
||||
}
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return cfg
|
||||
}
|
||||
db.RLock()
|
||||
raw := ""
|
||||
for _, s := range db.Settings {
|
||||
if s.Key == aiConfigSettingKey {
|
||||
raw = s.Value
|
||||
break
|
||||
}
|
||||
}
|
||||
db.RUnlock()
|
||||
if raw == "" {
|
||||
return cfg
|
||||
}
|
||||
var stored models.AIConfig
|
||||
if err := json.Unmarshal([]byte(raw), &stored); err != nil {
|
||||
return cfg
|
||||
}
|
||||
if stored.APIKey != "" {
|
||||
stored.APIKey = crypto.DecryptField(stored.APIKey)
|
||||
}
|
||||
if stored.Provider == "" {
|
||||
stored.Provider = "openai-compatible"
|
||||
}
|
||||
if stored.MaxTokens <= 0 {
|
||||
stored.MaxTokens = defaultAIMaxTokens
|
||||
}
|
||||
if stored.TimeoutSec <= 0 {
|
||||
stored.TimeoutSec = defaultAITimeoutSec
|
||||
}
|
||||
if stored.MaxScanBytes <= 0 {
|
||||
stored.MaxScanBytes = defaultAIMaxScanBytes
|
||||
}
|
||||
return stored
|
||||
}
|
||||
|
||||
// MaskedAIConfig 返回给前端的版本:APIKey 只留头尾。
|
||||
func MaskedAIConfig() models.AIConfig {
|
||||
cfg := GetAIConfig()
|
||||
cfg.APIKey = maskKey(cfg.APIKey)
|
||||
return cfg
|
||||
}
|
||||
|
||||
func maskKey(key string) string {
|
||||
if key == "" {
|
||||
return ""
|
||||
}
|
||||
if len(key) <= 8 {
|
||||
return "***"
|
||||
}
|
||||
return key[:3] + "***" + key[len(key)-4:]
|
||||
}
|
||||
|
||||
// SaveAIConfig 保存配置。
|
||||
//
|
||||
// apiKey 传空 = 沿用旧值(前端拿到的是掩码,不能把掩码存回去);
|
||||
// 传 "***" 开头的掩码也视为沿用,避免用户只是改模型名却把 key 清空。
|
||||
func SaveAIConfig(in models.AIConfig, apiKey string) error {
|
||||
old := GetAIConfig()
|
||||
key := strings.TrimSpace(apiKey)
|
||||
switch {
|
||||
case key == "":
|
||||
key = old.APIKey
|
||||
case strings.Contains(key, "***"):
|
||||
key = old.APIKey
|
||||
}
|
||||
if in.Enabled && (strings.TrimSpace(in.BaseURL) == "" || strings.TrimSpace(in.Model) == "") {
|
||||
return fmt.Errorf("启用 AI 分析需要填写 Base URL 与模型名")
|
||||
}
|
||||
if in.Enabled && key == "" {
|
||||
return fmt.Errorf("启用 AI 分析需要填写 API Key(内网无鉴权的服务可填任意占位值)")
|
||||
}
|
||||
if in.BaseURL != "" && !strings.HasPrefix(in.BaseURL, "http") {
|
||||
return fmt.Errorf("Base URL 必须以 http:// 或 https:// 开头(例如 https://api.deepseek.com/v1)")
|
||||
}
|
||||
|
||||
enc, err := crypto.EncryptField(key)
|
||||
if err != nil {
|
||||
return fmt.Errorf("加密 API Key 失败: %w", err)
|
||||
}
|
||||
saved := models.AIConfig{
|
||||
Enabled: in.Enabled,
|
||||
Provider: "openai-compatible",
|
||||
BaseURL: strings.TrimSuffix(strings.TrimSpace(in.BaseURL), "/"),
|
||||
APIKey: enc,
|
||||
Model: strings.TrimSpace(in.Model),
|
||||
Temperature: in.Temperature,
|
||||
MaxTokens: in.MaxTokens,
|
||||
TimeoutSec: in.TimeoutSec,
|
||||
DailyTokenCap: in.DailyTokenCap,
|
||||
MaxScanBytes: in.MaxScanBytes,
|
||||
RedactExtra: in.RedactExtra,
|
||||
}
|
||||
if saved.MaxTokens <= 0 {
|
||||
saved.MaxTokens = defaultAIMaxTokens
|
||||
}
|
||||
if saved.TimeoutSec <= 0 {
|
||||
saved.TimeoutSec = defaultAITimeoutSec
|
||||
}
|
||||
if saved.MaxScanBytes <= 0 {
|
||||
saved.MaxScanBytes = defaultAIMaxScanBytes
|
||||
}
|
||||
data, err := json.Marshal(saved)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return fmt.Errorf("数据库未初始化")
|
||||
}
|
||||
newID := db.NextID("settings")
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for i := range db.Settings {
|
||||
if db.Settings[i].Key == aiConfigSettingKey {
|
||||
db.Settings[i].Value = string(data)
|
||||
db.RequestSave()
|
||||
return nil
|
||||
}
|
||||
}
|
||||
db.Settings = append(db.Settings, models.SystemSetting{ID: newID, Key: aiConfigSettingKey, Value: string(data)})
|
||||
db.RequestSave()
|
||||
return nil
|
||||
}
|
||||
|
||||
// truncateToDay 取当天 00:00(本地时区),用于"今日已用 token"统计。
|
||||
func truncateToDay() time.Time {
|
||||
now := time.Now()
|
||||
return time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, now.Location())
|
||||
}
|
||||
|
||||
// AITokenUsedToday 某用户今日已消耗的 token 总量(每日配额闸门用)。
|
||||
func AITokenUsedToday(userID uint) int {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return 0
|
||||
}
|
||||
today := truncateToDay()
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
total := 0
|
||||
for _, r := range db.AIAnalysisRuns {
|
||||
if r.UserID == userID && !r.CreatedAt.Before(today) {
|
||||
total += r.PromptTokens + r.CompletionTokens
|
||||
}
|
||||
}
|
||||
return total
|
||||
}
|
||||
@@ -0,0 +1,271 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"lokistack/repository"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
)
|
||||
|
||||
// OpenAI 兼容 Chat Completions 客户端。
|
||||
//
|
||||
// 只依赖 /chat/completions 这一个端点,所以 DeepSeek、通义、
|
||||
// 内网 vLLM、Ollama(/v1)、各类网关都能直接用,不需要按厂商写适配。
|
||||
|
||||
type aiMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
}
|
||||
|
||||
type aiChatRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages []aiMessage `json:"messages"`
|
||||
Temperature float64 `json:"temperature"`
|
||||
MaxTokens int `json:"max_tokens,omitempty"`
|
||||
Stream bool `json:"stream"`
|
||||
}
|
||||
|
||||
type aiChatResponse struct {
|
||||
Choices []struct {
|
||||
Message struct {
|
||||
Content string `json:"content"`
|
||||
ReasoningContent string `json:"reasoning_content"`
|
||||
} `json:"message"`
|
||||
FinishReason string `json:"finish_reason"`
|
||||
} `json:"choices"`
|
||||
Usage struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
} `json:"usage"`
|
||||
Error *struct {
|
||||
Message string `json:"message"`
|
||||
Type string `json:"type"`
|
||||
} `json:"error"`
|
||||
}
|
||||
|
||||
// aiChat 调一次对话补全。
|
||||
//
|
||||
// 错误信息必须具体:把上游返回的 message 原样带出来,不要包装成
|
||||
// "调用失败" —— 常见原因(key 错、模型名不存在、baseURL 路径少了 /v1、
|
||||
// 内网模型没起)都在上游 message 里,藏起来只会让用户去猜。
|
||||
func aiChat(ctx context.Context, cfg models.AIConfig, messages []aiMessage) (string, int, int, error) {
|
||||
if strings.TrimSpace(cfg.BaseURL) == "" || strings.TrimSpace(cfg.Model) == "" {
|
||||
return "", 0, 0, fmt.Errorf("未配置模型:请先在「智能运维 → 模型配置」填写 Base URL 与模型名")
|
||||
}
|
||||
url := strings.TrimSuffix(cfg.BaseURL, "/") + "/chat/completions"
|
||||
body, err := json.Marshal(aiChatRequest{
|
||||
Model: cfg.Model,
|
||||
Messages: messages,
|
||||
Temperature: cfg.Temperature,
|
||||
MaxTokens: cfg.MaxTokens,
|
||||
Stream: false,
|
||||
})
|
||||
if err != nil {
|
||||
return "", 0, 0, err
|
||||
}
|
||||
|
||||
timeout := time.Duration(cfg.TimeoutSec) * time.Second
|
||||
if timeout <= 0 {
|
||||
timeout = 120 * time.Second
|
||||
}
|
||||
reqCtx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
req, err := http.NewRequestWithContext(reqCtx, http.MethodPost, url, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return "", 0, 0, fmt.Errorf("构造请求失败: %w", err)
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
if cfg.APIKey != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+cfg.APIKey)
|
||||
}
|
||||
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
if reqCtx.Err() == context.DeadlineExceeded {
|
||||
return "", 0, 0, fmt.Errorf("模型响应超时(%d 秒):可调大「超时」或换更快的模型", cfg.TimeoutSec)
|
||||
}
|
||||
return "", 0, 0, fmt.Errorf("连接模型服务失败: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
log.Printf("[ai-chat] url=%s model=%s keyLen=%d status=%d body=%.300s", url, cfg.Model, len(cfg.APIKey), resp.StatusCode, string(raw))
|
||||
if resp.StatusCode >= 400 {
|
||||
var parsed aiChatResponse
|
||||
if json.Unmarshal(raw, &parsed) == nil && parsed.Error != nil && parsed.Error.Message != "" {
|
||||
return "", 0, 0, fmt.Errorf("模型返回 %d:%s", resp.StatusCode, parsed.Error.Message)
|
||||
}
|
||||
return "", 0, 0, fmt.Errorf("模型返回 %d:%s", resp.StatusCode, truncateStr(string(raw), 300))
|
||||
}
|
||||
|
||||
var parsed aiChatResponse
|
||||
if err := json.Unmarshal(raw, &parsed); err != nil {
|
||||
return "", 0, 0, fmt.Errorf("解析模型响应失败(可能不是 OpenAI 兼容接口): %s", truncateStr(string(raw), 200))
|
||||
}
|
||||
if len(parsed.Choices) == 0 {
|
||||
return "", 0, 0, fmt.Errorf("模型没有返回任何内容(finish_reason 为空)")
|
||||
}
|
||||
content := parsed.Choices[0].Message.Content
|
||||
// DeepSeek 推理系模型(deepseek-reasoner / v4-pro 高推理档)会把过程写在
|
||||
// reasoning_content,最终 content 在 MaxTokens 太小时可能为空 —— 回退使用推理内容
|
||||
if strings.TrimSpace(content) == "" {
|
||||
content = parsed.Choices[0].Message.ReasoningContent
|
||||
}
|
||||
return content, parsed.Usage.PromptTokens, parsed.Usage.CompletionTokens, nil
|
||||
}
|
||||
|
||||
// AIConnectionTest 连通性测试结果:面向展示的结构化信息,不外发原始回复
|
||||
type AIConnectionTest struct {
|
||||
Model string `json:"model"`
|
||||
LatencyMs int64 `json:"latencyMs"`
|
||||
TokensUsed int `json:"tokensUsed"`
|
||||
ReplyLength int `json:"replyLength"`
|
||||
}
|
||||
|
||||
// TestAIConnection 连通性测试:发一句最小请求,验证 baseURL/key/model 三件套。
|
||||
// 返回结构化结果(模型/耗时/token);模型原始回复不回传 —— 测试要的是"通没通、多快",不是看模型说了什么。
|
||||
func TestAIConnection(cfg models.AIConfig) (*AIConnectionTest, error) {
|
||||
cfg.MaxTokens = 32
|
||||
cfg.TimeoutSec = 30
|
||||
start := time.Now()
|
||||
reply, promptTokens, completionTokens, err := aiChat(context.Background(), cfg, []aiMessage{
|
||||
{Role: "user", Content: "只回复两个字:正常"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
trimmed := strings.TrimSpace(reply)
|
||||
if trimmed == "" {
|
||||
return nil, fmt.Errorf("模型返回了空内容(检查 baseURL 是否为 OpenAI 兼容端点)")
|
||||
}
|
||||
return &AIConnectionTest{
|
||||
Model: cfg.Model,
|
||||
LatencyMs: time.Since(start).Milliseconds(),
|
||||
TokensUsed: promptTokens + completionTokens,
|
||||
ReplyLength: len([]rune(trimmed)),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ChatMessage 浮窗助手的多轮消息(user/assistant 交替)
|
||||
type ChatMessage struct {
|
||||
Role string `json:"role" binding:"required,oneof=user assistant"`
|
||||
Content string `json:"content" binding:"required"`
|
||||
}
|
||||
|
||||
// trimHistory 按字符预算从最旧开始裁剪对话历史(指南 07 TrimMessages);
|
||||
// 返回保留的历史与是否发生了裁剪(调用方在 system 里标注省略)
|
||||
func trimHistory(history []ChatMessage, budget int) ([]ChatMessage, bool) {
|
||||
var kept []ChatMessage
|
||||
total := 0
|
||||
truncated := false
|
||||
for i := len(history) - 1; i >= 0; i-- {
|
||||
size := len([]rune(history[i].Content))
|
||||
if total+size > budget && len(kept) > 0 {
|
||||
truncated = true
|
||||
break
|
||||
}
|
||||
kept = append([]ChatMessage{{Role: history[i].Role, Content: history[i].Content}}, kept...)
|
||||
total += size
|
||||
}
|
||||
return kept, truncated
|
||||
}
|
||||
|
||||
// isRetryableAIError 网络抖动/限流/上游临时故障才值得重试;鉴权与参数错重试无意义
|
||||
func isRetryableAIError(err error) bool {
|
||||
if err == nil {
|
||||
return false
|
||||
}
|
||||
msg := err.Error()
|
||||
return strings.Contains(msg, "429") ||
|
||||
strings.Contains(msg, "500") || strings.Contains(msg, "502") ||
|
||||
strings.Contains(msg, "503") || strings.Contains(msg, "504") ||
|
||||
strings.Contains(msg, "connection refused") || strings.Contains(msg, "context deadline exceeded") ||
|
||||
strings.Contains(msg, "EOF") || strings.Contains(msg, "timeout")
|
||||
}
|
||||
|
||||
// RunAIChat 全站 AI 助手浮窗:多轮对话 + 当前页面上下文注入(轻量信号摘要)。
|
||||
// 与整页分析共享配置/配额/脱敏/Pro 闸门;context 里的 sourceId 可选,带了就注入日志摘要。
|
||||
func RunAIChat(ctx context.Context, cfg models.AIConfig, userID uint, history []ChatMessage, page string, pageContext string) (string, error) {
|
||||
if !cfg.Enabled {
|
||||
return "", fmt.Errorf("AI 未启用:请先在「智能运维 → 模型配置」中开启并保存")
|
||||
}
|
||||
if cfg.DailyTokenCap > 0 && AITokenUsedToday(userID) >= cfg.DailyTokenCap {
|
||||
// 指南 05.3:超限不伪装成功 —— 结构化 notice 而非报错,对话上下文在前端保留
|
||||
return "\u2014\u2014 今日 AI 配额已用完(" + fmt.Sprintf("%d tokens),明天再试或在「AI 模型接入」调整上限 \u2014\u2014", cfg.DailyTokenCap), nil
|
||||
}
|
||||
|
||||
// 提示词设计:身份固定(不猜底层模型)、风格收敛、话题回引导、上下文数据
|
||||
system := `你是 LokiStack 日志平台内置的运维助手。
|
||||
规则:
|
||||
- 回答不超过 5 句话,优先给排查动作,其次给结论。
|
||||
- 被问"你是什么模型/版本"时,只答一句:"我是 LokiStack 内置运维助手,底层模型由管理员在「智能运维 → 模型配置」中指定。"然后引导回日志排查。
|
||||
- 只讨论日志、Loki、采集、告警与平台使用;无关话题(写代码、闲聊、其它产品)用一句礼貌拒绝并引导回排查。
|
||||
- 引用数据时只使用下方"日志概况"中的数字,不得编造。`
|
||||
|
||||
// 页面上下文(如 /logs 刚执行的查询与样本日志):截断并统一脱敏后并入 system
|
||||
if trimmed := strings.TrimSpace(pageContext); trimmed != "" {
|
||||
if len([]rune(trimmed)) > 6000 {
|
||||
trimmed = string([]rune(trimmed)[:6000])
|
||||
}
|
||||
system += "\n\n" + RedactTextOnly(trimmed, cfg.RedactExtra)
|
||||
}
|
||||
// 上下文预算(指南 07 TrimMessages):按字符预算从最旧开始丢弃,优先保留近期对话
|
||||
trimmed, truncated := trimHistory(history, 24000)
|
||||
if truncated {
|
||||
system += "\n(更早的对话已省略)"
|
||||
}
|
||||
msgs := []aiMessage{{Role: "system", Content: system}}
|
||||
for _, m := range trimmed {
|
||||
msgs = append(msgs, aiMessage{Role: m.Role, Content: m.Content})
|
||||
}
|
||||
// 脱敏:用户消息里的敏感串同样替换
|
||||
if cfg.RedactExtra != nil {
|
||||
for i := range msgs {
|
||||
msgs[i].Content = RedactTextOnly(msgs[i].Content, cfg.RedactExtra)
|
||||
}
|
||||
}
|
||||
// 指南 05.2:429/5xx/网络错误指数退避重试(1s→2s,最多 2 次);鉴权/参数错不重试
|
||||
var reply string
|
||||
var promptTokens, completionTokens int
|
||||
var err error
|
||||
for attempt := 0; ; attempt++ {
|
||||
reply, promptTokens, completionTokens, err = aiChat(ctx, cfg, msgs)
|
||||
if err == nil || attempt >= 2 || !isRetryableAIError(err) {
|
||||
break
|
||||
}
|
||||
backoff := time.Duration(1<<attempt) * time.Second
|
||||
log.Printf("[ai-chat] 可重试错误(%v),%s 后第 %d 次重试", err, backoff, attempt+1)
|
||||
select {
|
||||
case <-time.After(backoff):
|
||||
case <-ctx.Done():
|
||||
return "", ctx.Err()
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
reply = RedactTextOnly(reply, cfg.RedactExtra)
|
||||
// 计入每日配额:落一条 chat 类型用量记录
|
||||
db := repository.GetDB()
|
||||
if db != nil {
|
||||
db.Lock()
|
||||
db.AIAnalysisRuns = append(db.AIAnalysisRuns, models.AIAnalysisRun{
|
||||
UserID: userID, Username: "chat", SourceID: 0, SourceName: page,
|
||||
Status: "chat", PromptTokens: promptTokens, CompletionTokens: completionTokens,
|
||||
CreatedAt: time.Now(),
|
||||
})
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
}
|
||||
return reply, nil
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// 脱敏:日志里经常夹带凭据,送给外部模型前必须处理。
|
||||
//
|
||||
// 原则:
|
||||
// - 宁可多脱一点,也不能漏掉凭据;
|
||||
// - 但**不要脱掉排查必需的信息**:内网 IP、端口、主机名、命名空间、
|
||||
// Pod 名、错误码都要保留,否则模型给不出有用结论。
|
||||
// - 每条规则都替换成带类型的占位符(<IP> / <TOKEN>),
|
||||
// 这样模型仍能看出"这里原本是个 token",语义不丢。
|
||||
|
||||
type redactRule struct {
|
||||
pattern *regexp.Regexp
|
||||
replacement string
|
||||
}
|
||||
|
||||
var builtinRedactRules = []redactRule{
|
||||
// Bearer / JWT —— 一定是凭据
|
||||
{regexp.MustCompile(`(?i)bearer\s+[A-Za-z0-9\-._~+/]{12,}=*`), "Bearer <TOKEN>"},
|
||||
{regexp.MustCompile(`\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{4,}`), "<JWT>"},
|
||||
// key=value 形式的敏感字段
|
||||
{regexp.MustCompile(`(?i)\b(password|passwd|pwd|secret|token|api[_-]?key|access[_-]?key|secret[_-]?key)\b\s*[=:]\s*("?[^\s"',;]{4,}"?)`), "$1=<REDACTED>"},
|
||||
// URL 里的 basic auth:scheme://user:pass@host
|
||||
// 密码里可能含 @(如 P@ssw0rd),所以取到**最后一个 @** 之前的所有内容:
|
||||
// 用贪婪 [^\s/]+ 再回溯到最后一个 @,能正确切出 password 与 host。
|
||||
{regexp.MustCompile(`://([^:/\s]+):([^\s/]+)@`), "://$1:<REDACTED>@"},
|
||||
// AK/SK 形式(云厂商常见 16~40 位字母数字混合,带已知前缀)
|
||||
{regexp.MustCompile(`\b(AKIA|LTAI|AKID)[A-Za-z0-9]{12,}\b`), "<CLOUD_AK>"},
|
||||
// 手机号 / 身份证
|
||||
{regexp.MustCompile(`\b1[3-9]\d{9}\b`), "<PHONE>"},
|
||||
{regexp.MustCompile(`\b\d{17}[\dXx]\b`), "<ID_CARD>"},
|
||||
// 邮箱
|
||||
{regexp.MustCompile(`\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b`), "<EMAIL>"},
|
||||
}
|
||||
|
||||
// RedactText 对文本做脱敏,extra 是用户在设置里加的额外正则。
|
||||
//
|
||||
// 额外正则写错(如括号不闭合)不能让整个分析失败:静默跳过,
|
||||
// 但上层会把这些失效规则一并提示给用户。
|
||||
func RedactText(text string, extra []string) (string, []string) {
|
||||
out := text
|
||||
for _, rule := range builtinRedactRules {
|
||||
out = rule.pattern.ReplaceAllString(out, rule.replacement)
|
||||
}
|
||||
var invalid []string
|
||||
for _, raw := range extra {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if raw == "" {
|
||||
continue
|
||||
}
|
||||
re, err := regexp.Compile(raw)
|
||||
if err != nil {
|
||||
invalid = append(invalid, raw)
|
||||
continue
|
||||
}
|
||||
out = re.ReplaceAllString(out, "<REDACTED>")
|
||||
}
|
||||
return out, invalid
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 脱敏的两条底线:
|
||||
// 1. 凭据必须挡住(否则把 key 发给外部模型);
|
||||
// 2. 排查必需的信息必须留着(IP/端口/命名空间/错误码),
|
||||
// 否则模型给不出有用结论 —— 这比"脱得太干净"更常见。
|
||||
func TestRedactText(t *testing.T) {
|
||||
in := `2026-10-30 12:00:01 ERROR redis client: dial tcp 172.19.16.17:6379: connect: connection refused
|
||||
password=Sup3rS3cret! user-admin
|
||||
Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0.abcdefghijklmn
|
||||
url: mysql://appuser:P@ssw0rd@10.2.3.4:3306/cex
|
||||
contact: ops@example.com phone: 13800138000
|
||||
accessKey=AKIAIOSFODNN7EXAMPLE`
|
||||
|
||||
out, invalid := RedactText(in, nil)
|
||||
if len(invalid) != 0 {
|
||||
t.Fatalf("内置规则不应产生无效正则: %v", invalid)
|
||||
}
|
||||
|
||||
mustRedact := []string{"Sup3rS3cret", "eyJhbGciOiJIUzI1NiJ9", "P@ssw0rd", "ops@example.com", "13800138000", "AKIAIOSFODNN7EXAMPLE"}
|
||||
for _, secret := range mustRedact {
|
||||
if strings.Contains(out, secret) {
|
||||
t.Errorf("敏感内容未脱敏: %q\n%s", secret, out)
|
||||
}
|
||||
}
|
||||
|
||||
mustKeep := []string{"172.19.16.17:6379", "connection refused", "10.2.3.4:3306", "redis"}
|
||||
for _, keep := range mustKeep {
|
||||
if !strings.Contains(out, keep) {
|
||||
t.Errorf("排查必需的信息被误删: %q\n%s", keep, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 用户自定义正则写错不能拖垮整次分析:跳过并在返回值里报告。
|
||||
func TestRedactTextInvalidExtraRule(t *testing.T) {
|
||||
out, invalid := RedactText("hello internal-id-12345", []string{`(unclosed`, `internal-id-\d+`})
|
||||
if len(invalid) != 1 || invalid[0] != `(unclosed` {
|
||||
t.Errorf("应报告 1 条无效规则,实际 %v", invalid)
|
||||
}
|
||||
if strings.Contains(out, "internal-id-12345") {
|
||||
t.Errorf("有效自定义规则应生效: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// 模型经常不按 JSON 返回,解析必须宽容,不能只报"解析失败"。
|
||||
func TestParseAIResultTolerance(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
want string // 期望的 summary 片段
|
||||
}{
|
||||
{
|
||||
name: "标准 JSON",
|
||||
in: `{"summary":"日志量正常","findings":[],"actions":[],"unknowns":[]}`,
|
||||
want: "日志量正常",
|
||||
},
|
||||
{
|
||||
name: "包了 ```json 代码块",
|
||||
in: "```json\n{\"summary\":\"体积上涨 2 倍\",\"findings\":[{\"title\":\"t\",\"evidence\":\"e\",\"severity\":\"high\"}],\"actions\":[],\"unknowns\":[]}\n```",
|
||||
want: "体积上涨 2 倍",
|
||||
},
|
||||
{
|
||||
name: "前后加了解释",
|
||||
in: `分析如下:{"summary":"存在告警","findings":[],"actions":[],"unknowns":[]} 希望有帮助`,
|
||||
want: "存在告警",
|
||||
},
|
||||
{
|
||||
name: "纯文本(降级)",
|
||||
in: "看起来一切正常,没有异常日志。",
|
||||
want: "看起来一切正常",
|
||||
},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := parseAIResult(c.in)
|
||||
if !strings.Contains(got.Summary, c.want) {
|
||||
t.Errorf("summary 不符: got %q want contains %q", got.Summary, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,284 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
// AI 分析的"信号包":把 GB 级日志压成 KB 级结构化数据。
|
||||
//
|
||||
// 成本优先原则(P0 的硬约束):
|
||||
// - 体积/趋势 → index/volume_range(实测 24h 仅 54ms,不读日志内容)
|
||||
// - 条数/流数 → index/stats,只允许在 ≤2 小时窗口上用
|
||||
// (全天 index/stats 实测 20 秒,绝不能进 AI 路径)
|
||||
// - 应用排行 → 平台按小时预聚合的快照(毫秒级)
|
||||
// - 告警 → 平台库
|
||||
//
|
||||
// 也就是说:**P0 完全不扫日志内容**,一次分析对 Loki 的压力接近 0。
|
||||
type AISignalBundle struct {
|
||||
SourceID uint `json:"sourceId"`
|
||||
SourceName string `json:"sourceName"`
|
||||
Selector string `json:"selector"`
|
||||
From string `json:"from"`
|
||||
To string `json:"to"`
|
||||
RangeMinutes int `json:"rangeMinutes"`
|
||||
|
||||
TotalBytes int64 `json:"totalBytes"`
|
||||
TotalEntries int64 `json:"totalEntries"`
|
||||
// EntriesExact 条数是否为精确值(false = 范围过大跳过了 index/stats)
|
||||
EntriesExact bool `json:"entriesExact"`
|
||||
|
||||
Streams int64 `json:"streams"`
|
||||
Chunks int64 `json:"chunks"`
|
||||
|
||||
Trend []AITrendPoint `json:"trend"`
|
||||
Apps []AISignalApp `json:"apps"`
|
||||
|
||||
Baseline *AIBaseline `json:"baseline,omitempty"`
|
||||
Alerts []AIAlert `json:"alerts"`
|
||||
|
||||
// SnapshotHours 平台按小时预聚合已留存的小时数(影响排行的可信度)
|
||||
SnapshotHours int `json:"snapshotHours"`
|
||||
// Warnings 采集过程中的降级说明(如"范围过大,条数为估算")
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
// aiDisplayZone 结论中展示时间用的时区。
|
||||
//
|
||||
// 为什么不直接用 time.Local:静态编译的容器镜像通常没有 /usr/share/zoneinfo,
|
||||
// TZ 也没设,time.Local 就是 UTC —— 结果是模型在结论里写"凌晨 6 点日志下跌",
|
||||
// 而运维当时是下午 2 点,直接被当成数据错误。
|
||||
// 默认东八区(本地运维环境),需要别的时区可用 LOKISTACK_DISPLAY_TZ=UTC 覆盖。
|
||||
func aiDisplayZone() *time.Location {
|
||||
if strings.EqualFold(strings.TrimSpace(os.Getenv("LOKISTACK_DISPLAY_TZ")), "UTC") {
|
||||
return time.UTC
|
||||
}
|
||||
return time.FixedZone("CST", 8*3600)
|
||||
}
|
||||
|
||||
type AITrendPoint struct {
|
||||
Time string `json:"time"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
}
|
||||
|
||||
type AISignalApp struct {
|
||||
App string `json:"app"`
|
||||
Namespace string `json:"namespace"`
|
||||
Entries int64 `json:"entries"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
Share string `json:"share"`
|
||||
}
|
||||
|
||||
type AIBaseline struct {
|
||||
// PreviousBytes 上一周期同时长的体积(等长窗口对比)
|
||||
PreviousBytes int64 `json:"previousBytes"`
|
||||
CurrentBytes int64 `json:"currentBytes"`
|
||||
DeltaPercent float64 `json:"deltaPercent"`
|
||||
ComparisonNote string `json:"comparisonNote"`
|
||||
}
|
||||
|
||||
type AIAlert struct {
|
||||
RuleName string `json:"ruleName"`
|
||||
Status string `json:"status"`
|
||||
Count int `json:"count"`
|
||||
CreatedAt string `json:"createdAt"`
|
||||
}
|
||||
|
||||
// CollectAISignals 采集一次分析所需的全部信号。
|
||||
func CollectAISignals(ctx context.Context, src *models.LogSource, userID uint, admin bool, from, to time.Time, app string) (*AISignalBundle, error) {
|
||||
if src == nil {
|
||||
return nil, fmt.Errorf("数据源不存在")
|
||||
}
|
||||
if !to.After(from) {
|
||||
return nil, fmt.Errorf("时间范围无效")
|
||||
}
|
||||
// 复用查询授权:AI 能看到的数据范围必须与当前用户一致,不多不少。
|
||||
// 长窗口由上层强制指定 app(缩小聚合范围);15m/1h 允许全命名空间
|
||||
baseSelector := `{namespace=~".+"}`
|
||||
if app != "" {
|
||||
baseSelector = `{namespace=~".+", app="` + app + `"}`
|
||||
}
|
||||
selector, err := AuthorizedLogQuery(src, userID, admin, baseSelector)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
bundle := &AISignalBundle{
|
||||
SourceID: src.ID,
|
||||
SourceName: src.Name,
|
||||
Selector: selector,
|
||||
// 时间统一按本地时区展示:Loki 返回的是 UTC,
|
||||
// 直接给模型 UTC 会让结论里出现"凌晨 6 点"这种和用户作息对不上的时间。
|
||||
From: from.In(aiDisplayZone()).Format("2006-01-02 15:04"),
|
||||
To: to.In(aiDisplayZone()).Format("2006-01-02 15:04"),
|
||||
RangeMinutes: int(to.Sub(from).Minutes()),
|
||||
Alerts: []AIAlert{},
|
||||
Trend: []AITrendPoint{},
|
||||
Apps: []AISignalApp{},
|
||||
}
|
||||
|
||||
// ── 体积 + 趋势:volume_range(只读索引元数据)──
|
||||
buckets, err := LokiHistogram(ctx, src, selector, from.UnixNano(), to.UnixNano())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, b := range buckets {
|
||||
var bytes int64
|
||||
switch v := b["count"].(type) {
|
||||
case int64:
|
||||
bytes = v
|
||||
case float64:
|
||||
bytes = int64(v)
|
||||
}
|
||||
ts, _ := b["timestamp"].(string)
|
||||
if parsed, perr := time.Parse(time.RFC3339, ts); perr == nil {
|
||||
ts = parsed.In(aiDisplayZone()).Format("15:04")
|
||||
}
|
||||
bundle.TotalBytes += bytes
|
||||
bundle.Trend = append(bundle.Trend, AITrendPoint{Time: ts, Bytes: bytes})
|
||||
}
|
||||
// 趋势只保留尾部若干桶,避免 prompt 过长
|
||||
if len(bundle.Trend) > 24 {
|
||||
bundle.Trend = bundle.Trend[len(bundle.Trend)-24:]
|
||||
}
|
||||
|
||||
// ── 条数:仅短窗口走 index/stats(长窗口太慢,直接跳过并标注)──
|
||||
if bundle.RangeMinutes <= 120 {
|
||||
if st, statErr := LokiStatsCached(ctx, src, selector, from.UnixNano(), to.UnixNano()); statErr == nil {
|
||||
bundle.TotalEntries = st.Entries
|
||||
bundle.Streams = st.Streams
|
||||
bundle.Chunks = st.Chunks
|
||||
bundle.EntriesExact = true
|
||||
} else {
|
||||
bundle.Warnings = append(bundle.Warnings, "条数统计失败:"+truncateStr(statErr.Error(), 120))
|
||||
}
|
||||
} else {
|
||||
bundle.Warnings = append(bundle.Warnings, "时间范围超过 2 小时,条数为估算(跳过 index/stats:全天统计实测需 20 秒)")
|
||||
}
|
||||
|
||||
// ── 应用排行:平台按小时预聚合快照 ──
|
||||
if rows := GetAppRanking(src.ID, from, to, 10, "entries"); len(rows) > 0 {
|
||||
var total int64
|
||||
for _, r := range rows {
|
||||
total += r.Entries
|
||||
}
|
||||
for _, r := range rows {
|
||||
share := ""
|
||||
if total > 0 {
|
||||
share = fmt.Sprintf("%.1f%%", float64(r.Entries)/float64(total)*100)
|
||||
}
|
||||
bundle.Apps = append(bundle.Apps, AISignalApp{
|
||||
App: r.App, Namespace: r.Namespace, Entries: r.Entries, Bytes: r.Bytes, Share: share,
|
||||
})
|
||||
}
|
||||
}
|
||||
hours, _ := AppVolumeCoverageInfo(src.ID)
|
||||
bundle.SnapshotHours = hours
|
||||
if len(bundle.Apps) == 0 {
|
||||
bundle.Warnings = append(bundle.Warnings, "应用排行暂无数据:平台按小时预聚合还在积累(当前留存 0 小时)")
|
||||
}
|
||||
|
||||
// ── 基线对比:与上一个等长窗口比 ──
|
||||
prevFrom := from.Add(-to.Sub(from))
|
||||
prevTo := from
|
||||
if prevBuckets, perr := LokiHistogram(ctx, src, selector, prevFrom.UnixNano(), prevTo.UnixNano()); perr == nil {
|
||||
var prevBytes int64
|
||||
for _, b := range prevBuckets {
|
||||
switch v := b["count"].(type) {
|
||||
case int64:
|
||||
prevBytes += v
|
||||
case float64:
|
||||
prevBytes += int64(v)
|
||||
}
|
||||
}
|
||||
if prevBytes > 0 || bundle.TotalBytes > 0 {
|
||||
delta := 0.0
|
||||
if prevBytes > 0 {
|
||||
delta = (float64(bundle.TotalBytes) - float64(prevBytes)) / float64(prevBytes) * 100
|
||||
}
|
||||
bundle.Baseline = &AIBaseline{
|
||||
PreviousBytes: prevBytes,
|
||||
CurrentBytes: bundle.TotalBytes,
|
||||
DeltaPercent: delta,
|
||||
ComparisonNote: fmt.Sprintf("与上一个 %d 分钟等长窗口对比", bundle.RangeMinutes),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── 告警:窗口内的投递记录 ──
|
||||
db := repository.GetDB()
|
||||
if db != nil {
|
||||
db.RLock()
|
||||
for _, d := range db.AlertDeliveries {
|
||||
if d.CreatedAt.Before(from) || d.CreatedAt.After(to) {
|
||||
continue
|
||||
}
|
||||
bundle.Alerts = append(bundle.Alerts, AIAlert{
|
||||
RuleName: d.RuleName, Status: d.Status, Count: d.Count,
|
||||
CreatedAt: d.CreatedAt.In(aiDisplayZone()).Format("2006-01-02 15:04"),
|
||||
})
|
||||
}
|
||||
db.RUnlock()
|
||||
}
|
||||
sort.Slice(bundle.Alerts, func(i, j int) bool { return bundle.Alerts[i].CreatedAt > bundle.Alerts[j].CreatedAt })
|
||||
if len(bundle.Alerts) > 10 {
|
||||
bundle.Alerts = bundle.Alerts[:10]
|
||||
}
|
||||
return bundle, nil
|
||||
}
|
||||
|
||||
// SignalsAsText 把信号包渲染成给模型看的紧凑文本(不是 JSON,省 token)。
|
||||
func SignalsAsText(b *AISignalBundle) string {
|
||||
var sb strings.Builder
|
||||
fmt.Fprintf(&sb, "数据源:%s\n", b.SourceName)
|
||||
fmt.Fprintf(&sb, "时间范围:%s ~ %s(%d 分钟)\n", b.From, b.To, b.RangeMinutes)
|
||||
fmt.Fprintf(&sb, "日志体积:%.2f GB\n", float64(b.TotalBytes)/1024/1024/1024)
|
||||
if b.EntriesExact {
|
||||
fmt.Fprintf(&sb, "日志条数:%d\n", b.TotalEntries)
|
||||
} else {
|
||||
sb.WriteString("日志条数:(范围过大未精确统计)\n")
|
||||
}
|
||||
if b.Streams > 0 || b.Chunks > 0 {
|
||||
fmt.Fprintf(&sb, "流数/块数:%d / %d\n", b.Streams, b.Chunks)
|
||||
}
|
||||
if b.Baseline != nil {
|
||||
fmt.Fprintf(&sb, "基线对比:上一等长窗口 %.2f GB → 当前 %.2f GB,变化 %+.1f%%(%s)\n",
|
||||
float64(b.Baseline.PreviousBytes)/1024/1024/1024,
|
||||
float64(b.Baseline.CurrentBytes)/1024/1024/1024,
|
||||
b.Baseline.DeltaPercent, b.Baseline.ComparisonNote)
|
||||
}
|
||||
if len(b.Apps) > 0 {
|
||||
sb.WriteString("\n应用日志量排行(来自平台小时级预聚合):\n")
|
||||
for i, a := range b.Apps {
|
||||
fmt.Fprintf(&sb, " %d. %s/%s 条数=%d 占比=%s\n", i+1, a.Namespace, a.App, a.Entries, a.Share)
|
||||
}
|
||||
}
|
||||
if len(b.Trend) > 0 {
|
||||
sb.WriteString("\n体积趋势(最近分桶):\n")
|
||||
for _, t := range b.Trend {
|
||||
fmt.Fprintf(&sb, " %s %.1f MB\n", t.Time, float64(t.Bytes)/1024/1024)
|
||||
}
|
||||
}
|
||||
if len(b.Alerts) > 0 {
|
||||
sb.WriteString("\n窗口内告警:\n")
|
||||
for _, a := range b.Alerts {
|
||||
fmt.Fprintf(&sb, " %s %s 命中=%d 状态=%s\n", a.CreatedAt, a.RuleName, a.Count, a.Status)
|
||||
}
|
||||
} else {
|
||||
sb.WriteString("\n窗口内告警:无\n")
|
||||
}
|
||||
if len(b.Warnings) > 0 {
|
||||
sb.WriteString("\n数据限制说明:\n")
|
||||
for _, w := range b.Warnings {
|
||||
fmt.Fprintf(&sb, " - %s\n", w)
|
||||
}
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
@@ -48,7 +48,10 @@ func overStormLimit() (bool, int) {
|
||||
return sent >= alertStormMaxPerHour, sent
|
||||
}
|
||||
|
||||
const alertCheckInterval = time.Minute
|
||||
// 检查间隔:规则窗口都是 15 分钟,1 分钟查一次等于用 15 倍频率扫同样的数据。
|
||||
// Loki 单实例 CPU 是瓶颈(实测被限流 76%),这里降到 5 分钟:
|
||||
// 对告警及时性影响很小(窗口本身就是 15 分钟),但稳态负载降到 1/5。
|
||||
const alertCheckInterval = 5 * time.Minute
|
||||
|
||||
const (
|
||||
// alertRoundTimeout 一轮检查的总时限。超过就中断本轮,
|
||||
@@ -81,6 +84,12 @@ func StartAlertEngine() {
|
||||
log.Printf("[alert] 上一轮检查仍在运行,跳过本轮")
|
||||
continue
|
||||
}
|
||||
// Pro 闸门:日志告警是 Pro 功能,免费/试用到期后引擎不再评估
|
||||
// (规则与历史保留,重新授权即恢复,不删任何数据)
|
||||
if !IsPro() {
|
||||
alertCheckRunning.Store(false)
|
||||
continue
|
||||
}
|
||||
start := time.Now()
|
||||
ok := true
|
||||
func() {
|
||||
|
||||
@@ -0,0 +1,367 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/url"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
// 按应用的日志量:**小时级预聚合**。
|
||||
//
|
||||
// 背景:按应用排行必须读日志内容(Loki 3.4 的 index/volume 不支持 app 维度聚合),
|
||||
// 成本 ∝ 时间范围 —— 5 分钟 10 秒、15 分钟 20 秒、6 小时要 5~8 分钟、24 小时不可用。
|
||||
//
|
||||
// 方案:后台每小时把「上一个完整小时」的 per-app 条数算好落库;
|
||||
// 页面查 6h/24h/7d 时直接读库(毫秒级),不再扫日志。
|
||||
// 前台只在「≤1 小时」的短范围走实时查询(那种范围本来就快)。
|
||||
|
||||
const (
|
||||
// appVolumeRetentionDays 小时级 per-app 快照保留天数
|
||||
appVolumeRetentionDays = 90
|
||||
// appVolumeSampleTopN 每个小时只保留条数最多的 N 个应用,避免快照爆炸
|
||||
appVolumeSampleTopN = 50
|
||||
// appVolumeTimeout 单次抓取的超时(1 小时窗口 ≈ 15GB,给足时间)
|
||||
appVolumeTimeout = 5 * time.Minute
|
||||
)
|
||||
|
||||
// AppRankRow 排行结果的一行
|
||||
type AppRankRow struct {
|
||||
App string `json:"app"`
|
||||
Namespace string `json:"namespace"`
|
||||
// Entries 日志条数(预聚合快照累加)
|
||||
Entries int64 `json:"entries"`
|
||||
// Bytes 日志体积(字节,预聚合快照累加)。
|
||||
// 快照里没有体积的早期小时会让这个值偏小 —— 响应里的 bytesHours 用来提示覆盖率。
|
||||
Bytes int64 `json:"bytes"`
|
||||
// FromHistory 该行来自平台预聚合(而不是实时扫日志)
|
||||
FromHistory bool `json:"fromHistory"`
|
||||
}
|
||||
|
||||
// hourKey 把时间对齐到整点,格式 YYYY-MM-DDTHH(本地时区,与快照一致)
|
||||
func hourKey(t time.Time) string {
|
||||
return t.Local().Format("2006-01-02T15")
|
||||
}
|
||||
|
||||
// CaptureAppVolumeForHour 抓取指定小时(整点到整点)的 per-app 条数。
|
||||
func CaptureAppVolumeForHour(src *models.LogSource, hour time.Time) error {
|
||||
if src == nil {
|
||||
return fmt.Errorf("数据源为空")
|
||||
}
|
||||
start := hour.Truncate(time.Hour)
|
||||
end := start.Add(time.Hour)
|
||||
key := hourKey(start)
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), appVolumeTimeout)
|
||||
defer cancel()
|
||||
|
||||
// 用**瞬时查询**而不是 query_range:
|
||||
// - 只评估一个时间点(窗口覆盖整个小时),扫描量与 query_range 相同
|
||||
// 但省掉多步评估的开销
|
||||
// - 直接走 lokiProxyGET + 自己 5 分钟超时,绕开 LokiMetrics 的 90 秒
|
||||
// 上限与重型查询信号量(后台任务不该和前台查询抢令牌)
|
||||
query := fmt.Sprintf(`sum by (app, namespace) (count_over_time({namespace=~".+", app!="loki"}[%dm]))`, int(time.Hour.Minutes()))
|
||||
q := url.Values{}
|
||||
q.Set("query", query)
|
||||
q.Set("time", fmt.Sprintf("%d", end.UnixNano()))
|
||||
body, _, err := lokiProxyGET(ctx, src, "/loki/api/v1/query", q)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp struct {
|
||||
Data struct {
|
||||
Result []struct {
|
||||
Metric map[string]string `json:"metric"`
|
||||
Value [2]interface{} `json:"value"`
|
||||
} `json:"result"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return fmt.Errorf("解析 Loki 响应失败: %w", err)
|
||||
}
|
||||
|
||||
type row struct {
|
||||
ns, app string
|
||||
entries int64
|
||||
bytes int64
|
||||
}
|
||||
rows := make([]row, 0, len(resp.Data.Result))
|
||||
for _, item := range resp.Data.Result {
|
||||
app := item.Metric["app"]
|
||||
if app == "" {
|
||||
app = item.Metric["service_name"]
|
||||
}
|
||||
if app == "" {
|
||||
app = item.Metric["container"]
|
||||
}
|
||||
if app == "" {
|
||||
continue
|
||||
}
|
||||
value, _ := strconv.ParseFloat(fmt.Sprintf("%v", item.Value[1]), 64)
|
||||
if value <= 0 {
|
||||
continue
|
||||
}
|
||||
rows = append(rows, row{ns: item.Metric["namespace"], app: app, entries: int64(value)})
|
||||
}
|
||||
// 只留 top N,控制快照体积
|
||||
sort.Slice(rows, func(i, j int) bool { return rows[i].entries > rows[j].entries })
|
||||
if len(rows) > appVolumeSampleTopN {
|
||||
rows = rows[:appVolumeSampleTopN]
|
||||
}
|
||||
if len(rows) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// ── 体积:逐个 app 走 index/volume(索引接口,实测 ~80ms/个)──
|
||||
//
|
||||
// 为什么不用一条 `sum by (app) (bytes_over_time(...))` 拿全:那条要重扫整小时
|
||||
// 的日志内容(本集群约 29GB/小时,15 分钟就要 13s,一小时约 50s),而
|
||||
// index/volume 只读 chunk 元数据。代价是必须一个 app 一次调用(index/volume
|
||||
// 的返回值只带 selector 里出现过的 label),50 个 app ≈ 4~10s。
|
||||
//
|
||||
// 体积失败**不影响条数落库**:拿不到就留 0,前端把 0 显示成"体积未知",
|
||||
// 排行仍可按条数排(GetAppRanking 会按可用指标退让)。
|
||||
vctx, vcancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
for i := range rows {
|
||||
if rows[i].app == "" {
|
||||
continue
|
||||
}
|
||||
bytes, err := LokiVolumeBytesFor(vctx, src, appVolumeSelector(rows[i].ns, rows[i].app), start.UnixNano(), end.UnixNano())
|
||||
if err != nil {
|
||||
log.Printf("[app-volume] %s %s/%s 体积获取失败(条数已留存): %v", key, rows[i].ns, rows[i].app, err)
|
||||
continue
|
||||
}
|
||||
rows[i].bytes = bytes
|
||||
}
|
||||
vcancel()
|
||||
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return fmt.Errorf("数据库未初始化")
|
||||
}
|
||||
// NextID 内部会自己加锁;必须在 db.Lock() 之前调用。
|
||||
// JsonDB 的 mutex 不可重入,先 Lock 再 NextID 会永久死锁并占住写锁,
|
||||
// 导致所有读库接口(登录、列表)全部卡死。
|
||||
nextID := db.NextID("appVolumeSnapshots")
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for _, r := range rows {
|
||||
// 幂等:同 (hour, source, ns, app) 已存在则更新,不重复插入
|
||||
found := false
|
||||
for i := range db.AppVolumeSnapshots {
|
||||
snap := &db.AppVolumeSnapshots[i]
|
||||
if snap.Hour == key && snap.SourceID == src.ID && snap.Namespace == r.ns && snap.App == r.app {
|
||||
snap.Entries = r.entries
|
||||
snap.Bytes = r.bytes
|
||||
snap.CapturedAt = time.Now()
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if found {
|
||||
continue
|
||||
}
|
||||
db.AppVolumeSnapshots = append(db.AppVolumeSnapshots, models.AppVolumeSnapshot{
|
||||
ID: nextID, Hour: key, SourceID: src.ID, Namespace: r.ns, App: r.app,
|
||||
Entries: r.entries, Bytes: r.bytes, CapturedAt: time.Now(),
|
||||
})
|
||||
nextID++
|
||||
}
|
||||
pruneAppVolumeLocked(db)
|
||||
db.RequestSave()
|
||||
log.Printf("[app-volume] %s 已留存 %d 个应用的量", key, len(rows))
|
||||
return nil
|
||||
}
|
||||
|
||||
// pruneAppVolumeLocked 按保留期裁剪(调用方必须已持写锁)。
|
||||
func pruneAppVolumeLocked(db *repository.JsonDB) {
|
||||
cutoff := hourKey(time.Now().AddDate(0, 0, -appVolumeRetentionDays))
|
||||
keep := db.AppVolumeSnapshots[:0]
|
||||
for _, snap := range db.AppVolumeSnapshots {
|
||||
if snap.Hour >= cutoff {
|
||||
keep = append(keep, snap)
|
||||
}
|
||||
}
|
||||
db.AppVolumeSnapshots = keep
|
||||
}
|
||||
|
||||
// CaptureMissingAppVolume 回补最近 N 个**完整小时**里还没抓过的那些。
|
||||
func CaptureMissingAppVolume(hours int) {
|
||||
if hours <= 0 {
|
||||
hours = 2
|
||||
}
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.RLock()
|
||||
sources := make([]models.LogSource, 0, len(db.LogSources))
|
||||
for _, s := range db.LogSources {
|
||||
sources = append(sources, s)
|
||||
}
|
||||
captured := map[string]bool{}
|
||||
for _, snap := range db.AppVolumeSnapshots {
|
||||
captured[fmt.Sprintf("%d|%s", snap.SourceID, snap.Hour)] = true
|
||||
}
|
||||
db.RUnlock()
|
||||
|
||||
now := time.Now()
|
||||
for _, src := range sources {
|
||||
for i := 1; i <= hours; i++ {
|
||||
hour := now.Add(-time.Duration(i) * time.Hour).Truncate(time.Hour)
|
||||
if captured[fmt.Sprintf("%d|%s", src.ID, hourKey(hour))] {
|
||||
continue
|
||||
}
|
||||
// 超出 Loki 保留期的窗口不抓(抓也抓不到,白等几分钟)
|
||||
if time.Since(hour.Add(time.Hour)) > 26*time.Hour {
|
||||
continue
|
||||
}
|
||||
if err := CaptureAppVolumeForHour(&src, hour); err != nil {
|
||||
log.Printf("[app-volume] %s %s 抓取失败: %v", src.Name, hourKey(hour), err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// StartAppVolumeCapture 后台任务:每小时第 5 分钟抓上一个完整小时。
|
||||
func StartAppVolumeCapture() {
|
||||
go func() {
|
||||
time.Sleep(45 * time.Second) // 避开启动时的其它任务
|
||||
CaptureMissingAppVolume(2)
|
||||
for {
|
||||
now := time.Now()
|
||||
// 下一个整点后 5 分钟
|
||||
next := now.Truncate(time.Hour).Add(time.Hour + 5*time.Minute)
|
||||
time.Sleep(time.Until(next))
|
||||
CaptureMissingAppVolume(3)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// appVolumeSelector 生成 index/volume 用的 stream selector。
|
||||
// 值用 strconv.Quote 转义:app 名理论上可能出现引号/反斜杠,拼错就是 Loki 400。
|
||||
func appVolumeSelector(namespace, app string) string {
|
||||
parts := make([]string, 0, 2)
|
||||
if namespace != "" {
|
||||
parts = append(parts, "namespace="+strconv.Quote(namespace))
|
||||
}
|
||||
if app != "" {
|
||||
parts = append(parts, "app="+strconv.Quote(app))
|
||||
}
|
||||
if len(parts) == 0 {
|
||||
return `{namespace=~".+"}`
|
||||
}
|
||||
return "{" + strings.Join(parts, ", ") + "}"
|
||||
}
|
||||
|
||||
// GetAppRanking 从**预聚合快照**里取指定时间范围的应用排行(毫秒级)。
|
||||
//
|
||||
// sortBy:bytes(默认,按体积)| entries(按条数)。
|
||||
// 体积优先是因为用户要的是"哪个应用把磁盘/带宽吃了";条数受日志长短影响,
|
||||
// 一个高频 INFO 应用可以条数第一但体积很小。
|
||||
// 排序永远带兜底:体积相同(或都是 0)时按条数,再按应用名,保证顺序稳定。
|
||||
func GetAppRanking(sourceID uint, from, to time.Time, limit int, sortBy string) []AppRankRow {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
fromHour := hourKey(from)
|
||||
toHour := hourKey(to)
|
||||
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return nil
|
||||
}
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
|
||||
type agg struct {
|
||||
ns, app string
|
||||
entries int64
|
||||
bytes int64
|
||||
}
|
||||
sum := map[string]*agg{}
|
||||
for _, snap := range db.AppVolumeSnapshots {
|
||||
if snap.SourceID != sourceID || snap.Hour < fromHour || snap.Hour > toHour {
|
||||
continue
|
||||
}
|
||||
key := snap.Namespace + "\x00" + snap.App
|
||||
item, ok := sum[key]
|
||||
if !ok {
|
||||
item = &agg{ns: snap.Namespace, app: snap.App}
|
||||
sum[key] = item
|
||||
}
|
||||
item.entries += snap.Entries
|
||||
item.bytes += snap.Bytes
|
||||
}
|
||||
rows := make([]AppRankRow, 0, len(sum))
|
||||
for _, item := range sum {
|
||||
rows = append(rows, AppRankRow{
|
||||
App: item.app, Namespace: item.ns,
|
||||
Entries: item.entries, Bytes: item.bytes, FromHistory: true,
|
||||
})
|
||||
}
|
||||
byEntries := sortBy == "entries"
|
||||
sort.Slice(rows, func(i, j int) bool {
|
||||
a, b := rows[i], rows[j]
|
||||
if byEntries {
|
||||
if a.Entries != b.Entries {
|
||||
return a.Entries > b.Entries
|
||||
}
|
||||
if a.Bytes != b.Bytes {
|
||||
return a.Bytes > b.Bytes
|
||||
}
|
||||
} else {
|
||||
if a.Bytes != b.Bytes {
|
||||
return a.Bytes > b.Bytes
|
||||
}
|
||||
if a.Entries != b.Entries {
|
||||
return a.Entries > b.Entries
|
||||
}
|
||||
}
|
||||
return a.App < b.App
|
||||
})
|
||||
if len(rows) > limit {
|
||||
rows = rows[:limit]
|
||||
}
|
||||
return rows
|
||||
}
|
||||
|
||||
// AppVolumeCoverage 返回快照覆盖的小时数,前端用来提示"历史只有 N 小时数据"。
|
||||
func AppVolumeCoverage(sourceID uint) int {
|
||||
hours, _ := AppVolumeCoverageInfo(sourceID)
|
||||
return hours
|
||||
}
|
||||
|
||||
// AppVolumeCoverageInfo 返回(快照覆盖小时数, 其中带体积的小时数)。
|
||||
//
|
||||
// 第二个值是给"按体积排行"用的诚实性提示:体积列是本轮新加的,
|
||||
// 之前抓的小时只有条数没有体积,24h 排行里这些小时的体积会偏小。
|
||||
// 两个数不一致时前端要明确告诉用户覆盖率,而不是让用户误读成"这个应用体积很小"。
|
||||
func AppVolumeCoverageInfo(sourceID uint) (hours, bytesHours int) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return 0, 0
|
||||
}
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
all := map[string]bool{}
|
||||
withBytes := map[string]bool{}
|
||||
for _, snap := range db.AppVolumeSnapshots {
|
||||
if snap.SourceID != sourceID {
|
||||
continue
|
||||
}
|
||||
all[snap.Hour] = true
|
||||
if snap.Bytes > 0 {
|
||||
withBytes[snap.Hour] = true
|
||||
}
|
||||
}
|
||||
return len(all), len(withBytes)
|
||||
}
|
||||
@@ -1,364 +0,0 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"compress/gzip"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
// 备份/恢复设计要点
|
||||
//
|
||||
// 为什么必须有:平台的全部状态(部署/采集/存储/集群/主机/策略/告警/用户/审计)
|
||||
// 都在这一个 JSON 文件里;kubeconfig 虽然加密存在 DB 内,但解密依赖
|
||||
// ENCRYPTION_KEY(K8s Secret)。DB 丢了 = 平台失去"这些资源是谁建的"的映射,
|
||||
// 从此无法管理/卸载/升级正在跑的 Loki 与采集器。
|
||||
//
|
||||
// 安全取舍:
|
||||
// - 备份包内含加密后的 kubeconfig 与 known_hosts,**不含明文密钥**
|
||||
// - manifest 里只记录 ENCRYPTION_KEY 的指纹(sha256 前 16 位),
|
||||
// 用于恢复时校验"这把钥匙能不能解开这份备份",不泄露密钥本身
|
||||
// - 恢复分两步(校验暂存 → 手动应用),避免一个误点就覆盖线上 DB
|
||||
|
||||
// BackupManifest 备份包元信息(放在包内 manifest.json)
|
||||
type BackupManifest struct {
|
||||
Version string `json:"version"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
DBBytes int64 `json:"dbBytes"`
|
||||
DBSHA256 string `json:"dbSha256"`
|
||||
KeyFP string `json:"encryptionKeyFingerprint"`
|
||||
RecordCount int `json:"recordCount"`
|
||||
Note string `json:"note"`
|
||||
}
|
||||
|
||||
// backupExclude 备份时跳过的文件/目录(临时文件、lost+found、正在写的 restore 暂存)
|
||||
var backupExclude = map[string]bool{
|
||||
"lost+found": true, ".restore-pending": true, "backups": true,
|
||||
}
|
||||
|
||||
// CreateBackup 把 /app/data 打包成 tar.gz 写入 w。
|
||||
func CreateBackup(w io.Writer) (*BackupManifest, error) {
|
||||
dataDir := dataDirPath()
|
||||
dbPath := databasePath()
|
||||
|
||||
dbInfo, err := os.Stat(dbPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("读取数据库失败: %w", err)
|
||||
}
|
||||
dbBytes, err := os.ReadFile(dbPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("读取数据库失败: %w", err)
|
||||
}
|
||||
sum := sha256.Sum256(dbBytes)
|
||||
|
||||
// 记录数从**备份出来的字节**里数,不用 repository.GetDB():
|
||||
// CLI 模式(lokistack backup,跑在 CronJob 里)根本没初始化 DB,
|
||||
// 用 GetDB() 会得到 nil,manifest 里的 recordCount 恒为 0,
|
||||
// 恢复时就看不出"这份备份里有没有东西"。
|
||||
recordCount := countRecords(dbBytes)
|
||||
|
||||
manifest := &BackupManifest{
|
||||
Version: "2.0.0",
|
||||
CreatedAt: time.Now(),
|
||||
DBBytes: dbInfo.Size(),
|
||||
DBSHA256: hex.EncodeToString(sum[:]),
|
||||
KeyFP: encryptionKeyFingerprint(),
|
||||
RecordCount: recordCount,
|
||||
Note: "包内 kubeconfig 为加密态;恢复必须使用同一把 ENCRYPTION_KEY,否则 kubeconfig 无法解密",
|
||||
}
|
||||
|
||||
gz := gzip.NewWriter(w)
|
||||
defer gz.Close()
|
||||
tw := tar.NewWriter(gz)
|
||||
defer tw.Close()
|
||||
|
||||
// 1) manifest
|
||||
mBytes, _ := json.MarshalIndent(manifest, "", " ")
|
||||
if err := writeTarFile(tw, "manifest.json", mBytes, 0o600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2) data 目录下所有文件(DB + kubeconfigs/ + known_hosts/)
|
||||
err = filepath.Walk(dataDir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return nil // 单个文件读不到不阻断整包
|
||||
}
|
||||
if path == dataDir {
|
||||
return nil
|
||||
}
|
||||
rel, err := filepath.Rel(dataDir, path)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
top := strings.SplitN(rel, string(os.PathSeparator), 2)[0]
|
||||
if backupExclude[top] {
|
||||
if info.IsDir() {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
// 跳过正在写的临时文件
|
||||
if strings.HasPrefix(filepath.Base(path), ".lokistack-") && strings.HasSuffix(path, ".tmp") {
|
||||
return nil
|
||||
}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
defer f.Close()
|
||||
hdr := &tar.Header{
|
||||
Name: filepath.ToSlash(filepath.Join("data", rel)),
|
||||
Mode: int64(info.Mode().Perm()),
|
||||
Size: info.Size(),
|
||||
ModTime: info.ModTime(),
|
||||
}
|
||||
if err := tw.WriteHeader(hdr); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = io.Copy(tw, f)
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return manifest, nil
|
||||
}
|
||||
|
||||
func writeTarFile(tw *tar.Writer, name string, data []byte, mode int64) error {
|
||||
hdr := &tar.Header{Name: name, Mode: mode, Size: int64(len(data)), ModTime: time.Now()}
|
||||
if err := tw.WriteHeader(hdr); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := tw.Write(data)
|
||||
return err
|
||||
}
|
||||
|
||||
// RestoreResult 校验结果
|
||||
type RestoreResult struct {
|
||||
Manifest *BackupManifest `json:"manifest"`
|
||||
KeyMatches bool `json:"keyMatches"`
|
||||
RowsInBackup int `json:"rowsInBackup"`
|
||||
CurrentRows int `json:"currentRows"`
|
||||
StagedPath string `json:"stagedPath,omitempty"`
|
||||
Warning string `json:"warning,omitempty"`
|
||||
}
|
||||
|
||||
// ValidateAndStageBackup 校验备份包并暂存到 <dataDir>/.restore-pending/。
|
||||
//
|
||||
// 不在这一步直接覆盖线上 DB:恢复是不可逆操作,必须让人再确认一次
|
||||
// (页面上的「应用恢复」或 run.sh restore)。
|
||||
func ValidateAndStageBackup(r io.Reader) (*RestoreResult, error) {
|
||||
gr, err := gzip.NewReader(r)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("不是合法的 gzip 文件: %w", err)
|
||||
}
|
||||
defer gr.Close()
|
||||
tr := tar.NewReader(gr)
|
||||
|
||||
var (
|
||||
manifest *BackupManifest
|
||||
dbBytes []byte
|
||||
)
|
||||
for {
|
||||
hdr, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("解包失败: %w", err)
|
||||
}
|
||||
switch {
|
||||
case hdr.Name == "manifest.json":
|
||||
b, _ := io.ReadAll(io.LimitReader(tr, 1<<20))
|
||||
var m BackupManifest
|
||||
if err := json.Unmarshal(b, &m); err == nil {
|
||||
manifest = &m
|
||||
}
|
||||
case hdr.Name == "data/opsxloki.json":
|
||||
dbBytes, _ = io.ReadAll(io.LimitReader(tr, 64<<20))
|
||||
}
|
||||
}
|
||||
if manifest == nil {
|
||||
return nil, fmt.Errorf("备份包缺少 manifest.json,不是本平台导出的备份")
|
||||
}
|
||||
if len(dbBytes) == 0 {
|
||||
return nil, fmt.Errorf("备份包内没有 data/opsxloki.json")
|
||||
}
|
||||
|
||||
// 校验 DB 完整性:必须是合法 JSON 且能解析出预期集合
|
||||
var probe map[string]json.RawMessage
|
||||
if err := json.Unmarshal(dbBytes, &probe); err != nil {
|
||||
return nil, fmt.Errorf("备份内的数据库不是合法 JSON: %w", err)
|
||||
}
|
||||
for _, required := range []string{"users", "deployments", "collectors"} {
|
||||
if _, ok := probe[required]; !ok {
|
||||
return nil, fmt.Errorf("备份内数据库缺少集合 %q,可能不是 LokiStack 的备份", required)
|
||||
}
|
||||
}
|
||||
|
||||
// 哈希校验(防止传输损坏)
|
||||
sum := sha256.Sum256(dbBytes)
|
||||
if manifest.DBSHA256 != "" && hex.EncodeToString(sum[:]) != manifest.DBSHA256 {
|
||||
return nil, fmt.Errorf("备份内数据库校验失败(sha256 不匹配),文件可能已损坏")
|
||||
}
|
||||
|
||||
stagedDir := filepath.Join(dataDirPath(), ".restore-pending")
|
||||
if err := os.MkdirAll(stagedDir, 0o700); err != nil {
|
||||
return nil, fmt.Errorf("创建暂存目录失败: %w", err)
|
||||
}
|
||||
stagedPath := filepath.Join(stagedDir, "opsxloki.json")
|
||||
if err := os.WriteFile(stagedPath, dbBytes, 0o600); err != nil {
|
||||
return nil, fmt.Errorf("写入暂存文件失败: %w", err)
|
||||
}
|
||||
|
||||
currentRows := 0
|
||||
if db := repository.GetDB(); db != nil {
|
||||
db.RLock()
|
||||
currentRows = len(db.Deployments) + len(db.Collectors) + len(db.Clusters) + len(db.Hosts)
|
||||
db.RUnlock()
|
||||
}
|
||||
|
||||
res := &RestoreResult{
|
||||
Manifest: manifest,
|
||||
KeyMatches: manifest.KeyFP == "" || manifest.KeyFP == encryptionKeyFingerprint(),
|
||||
RowsInBackup: manifest.RecordCount,
|
||||
CurrentRows: currentRows,
|
||||
StagedPath: stagedPath,
|
||||
}
|
||||
if !res.KeyMatches {
|
||||
res.Warning = "备份的 ENCRYPTION_KEY 指纹与当前平台不一致:恢复后 kubeconfig / 存储凭证将无法解密," +
|
||||
"需要用备份时的那把 ENCRYPTION_KEY 覆盖 lokistack-secrets 中的值后再启动"
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// countRecords 从数据库 JSON 字节里统计各集合的记录总数。
|
||||
func countRecords(dbBytes []byte) int {
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal(dbBytes, &raw); err != nil {
|
||||
return 0
|
||||
}
|
||||
total := 0
|
||||
for _, v := range raw {
|
||||
var arr []json.RawMessage
|
||||
if err := json.Unmarshal(v, &arr); err == nil {
|
||||
total += len(arr)
|
||||
}
|
||||
}
|
||||
return total
|
||||
}
|
||||
|
||||
// encryptionKeyFingerprint 返回当前 ENCRYPTION_KEY 的指纹(sha256 前 16 位),不泄露密钥。
|
||||
func encryptionKeyFingerprint() string {
|
||||
key := os.Getenv("ENCRYPTION_KEY")
|
||||
if key == "" {
|
||||
// 兼容从文件读取的部署方式
|
||||
if p := os.Getenv("ENCRYPTION_KEY_FILE"); p != "" {
|
||||
if b, err := os.ReadFile(p); err == nil {
|
||||
key = strings.TrimSpace(string(b))
|
||||
}
|
||||
}
|
||||
}
|
||||
if key == "" {
|
||||
return ""
|
||||
}
|
||||
sum := sha256.Sum256([]byte(key))
|
||||
return hex.EncodeToString(sum[:])[:16]
|
||||
}
|
||||
|
||||
// ApplyStagedBackup 把暂存文件应用为正式 DB(调用方负责重启进程)。
|
||||
func ApplyStagedBackup() (string, error) {
|
||||
restoreDir := filepath.Join(dataDirPath(), ".restore-pending")
|
||||
staged := filepath.Join(restoreDir, "lokistack.json")
|
||||
if _, err := os.Stat(staged); err != nil {
|
||||
// 兼容旧版暂存文件名
|
||||
staged = filepath.Join(restoreDir, "opsxloki.json")
|
||||
}
|
||||
b, err := os.ReadFile(staged)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("没有待应用的恢复文件: %w", err)
|
||||
}
|
||||
var probe map[string]json.RawMessage
|
||||
if err := json.Unmarshal(b, &probe); err != nil {
|
||||
return "", fmt.Errorf("暂存文件已损坏: %w", err)
|
||||
}
|
||||
|
||||
dbPath := databasePath()
|
||||
// 先把当前 DB 备份一份,恢复错了还能退回去
|
||||
if cur, err := os.ReadFile(dbPath); err == nil {
|
||||
rollback := dbPath + ".before-restore." + time.Now().Format("20060102-150405")
|
||||
_ = os.WriteFile(rollback, cur, 0o600)
|
||||
}
|
||||
tmp := dbPath + ".restoring"
|
||||
if err := os.WriteFile(tmp, b, 0o600); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := os.Rename(tmp, dbPath); err != nil {
|
||||
return "", err
|
||||
}
|
||||
_ = os.Remove(staged)
|
||||
// 关键:立即把内存态切换为恢复后的数据。否则「替换文件 → 重启」之间,
|
||||
// 任何一次防抖落盘都会用旧内存态把恢复文件覆盖回去,恢复静默失效。
|
||||
if db := repository.GetDB(); db != nil {
|
||||
db.Lock()
|
||||
if err := db.Reload(); err != nil {
|
||||
db.Unlock()
|
||||
return dbPath, fmt.Errorf("恢复文件已生效但内存重载失败(%v),请立即重启进程", err)
|
||||
}
|
||||
db.Unlock()
|
||||
}
|
||||
return dbPath, nil
|
||||
}
|
||||
|
||||
// ListBackups 列出 dataDir/backups 下已有的备份文件(run.sh backup 产出)。
|
||||
func ListBackups() []map[string]interface{} {
|
||||
dir := filepath.Join(dataDirPath(), "backups")
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return []map[string]interface{}{}
|
||||
}
|
||||
type item struct {
|
||||
name string
|
||||
size int64
|
||||
mod time.Time
|
||||
}
|
||||
var items []item
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".tar.gz") {
|
||||
continue
|
||||
}
|
||||
info, err := e.Info()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
items = append(items, item{e.Name(), info.Size(), info.ModTime()})
|
||||
}
|
||||
sort.Slice(items, func(i, j int) bool { return items[i].mod.After(items[j].mod) })
|
||||
out := make([]map[string]interface{}, 0, len(items))
|
||||
for _, it := range items {
|
||||
out = append(out, map[string]interface{}{
|
||||
"name": it.name, "sizeBytes": it.size, "createdAt": it.mod,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func dataDirPath() string {
|
||||
if p := os.Getenv("DATA_DIR"); p != "" {
|
||||
return p
|
||||
}
|
||||
return "/app/data"
|
||||
}
|
||||
@@ -211,6 +211,11 @@ func (s *ClusterService) Delete(id uint) error {
|
||||
defer db.Unlock()
|
||||
for i, c := range db.Clusters {
|
||||
if c.ID == id {
|
||||
for _, d := range db.Deployments {
|
||||
if d.ClusterID != nil && *d.ClusterID == id {
|
||||
return fmt.Errorf("仍有部署使用该集群,请先删除部署")
|
||||
}
|
||||
}
|
||||
if c.KubeConfigPath != "" {
|
||||
_ = os.Remove(c.KubeConfigPath)
|
||||
}
|
||||
|
||||
@@ -57,6 +57,7 @@ type CreateCollectorRequest struct {
|
||||
LogPath string
|
||||
Labels string
|
||||
ScrapeJobs []models.ScrapeJob
|
||||
TargetNamespaces []string
|
||||
}
|
||||
|
||||
// fillAssociations 填充关联的 Loki / Host / Cluster(用于配置生成与展示)
|
||||
@@ -239,6 +240,8 @@ func (s *CollectorService) PreviewConfig(id uint, updates map[string]interface{}
|
||||
}
|
||||
|
||||
func (s *CollectorService) Create(req *CreateCollectorRequest) (*models.Collector, error) {
|
||||
// 采集器变更可能带来新的 app 标签(主机模式),下拉候选值必须失效
|
||||
defer InvalidateLabelCache()
|
||||
switch req.Type {
|
||||
case "host-promtail", "k8s-daemonset", "k8s-fluentbit-sidecar":
|
||||
default:
|
||||
@@ -273,6 +276,13 @@ func (s *CollectorService) Create(req *CreateCollectorRequest) (*models.Collecto
|
||||
refType = "host"
|
||||
}
|
||||
|
||||
// DaemonSet 的部署命名空间固定为 Loki 所在命名空间(通常 log):
|
||||
// 采集器与 LokiStack 放在一起,用户不需要也不应该选。
|
||||
// 这里强制清空,渲染与部署两侧都会回退到 d.loki.Namespace。
|
||||
if req.Type == "k8s-daemonset" {
|
||||
req.Namespace = ""
|
||||
}
|
||||
|
||||
collector := models.Collector{
|
||||
ID: repository.GetDB().NextID("collectors"),
|
||||
Name: req.Name,
|
||||
@@ -286,6 +296,7 @@ func (s *CollectorService) Create(req *CreateCollectorRequest) (*models.Collecto
|
||||
LogPath: req.LogPath,
|
||||
Labels: req.Labels,
|
||||
ScrapeJobs: req.ScrapeJobs,
|
||||
TargetNamespaces: req.TargetNamespaces,
|
||||
Status: "pending",
|
||||
CreatedAt: time.Now(),
|
||||
UpdatedAt: time.Now(),
|
||||
@@ -349,6 +360,7 @@ func (s *CollectorService) List(page, pageSize int) ([]models.Collector, int64,
|
||||
}
|
||||
|
||||
func (s *CollectorService) Update(id uint, updates map[string]interface{}) error {
|
||||
defer InvalidateLabelCache()
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
@@ -372,6 +384,11 @@ func (s *CollectorService) Update(id uint, updates map[string]interface{}) error
|
||||
if v, ok := updates["labels"].(string); ok {
|
||||
db.Collectors[i].Labels = v
|
||||
}
|
||||
// DaemonSet 的 namespace 一律不接受修改(固定跟随 Loki 所在命名空间)
|
||||
if v, ok := updates["namespace"].(string); ok && v != "" && db.Collectors[i].Type != "k8s-daemonset" {
|
||||
db.Collectors[i].Namespace = v
|
||||
db.Collectors[i].StatusMessage = "部署命名空间已保存,点击部署以应用新配置"
|
||||
}
|
||||
// targetNamespaces: 采集哪些命名空间([]string;空=全集群)
|
||||
if v, ok := updates["targetNamespaces"]; ok {
|
||||
var nss []string
|
||||
@@ -488,30 +505,59 @@ func (s *CollectorService) Update(id uint, updates map[string]interface{}) error
|
||||
return utils.ErrNotFound
|
||||
}
|
||||
|
||||
// UninstallCollectorsForDeployment 卸载并删除某部署下的所有采集器。
|
||||
// 卸载失败时保留记录,避免远端资源失去跟踪。
|
||||
func UninstallCollectorsForDeployment(ctx context.Context, deploymentID uint) {
|
||||
db := repository.GetDB()
|
||||
db.RLock()
|
||||
var targets []models.Collector
|
||||
for _, c := range db.Collectors {
|
||||
if c.LokiDeploymentID == deploymentID {
|
||||
targets = append(targets, c)
|
||||
}
|
||||
}
|
||||
db.RUnlock()
|
||||
|
||||
for i := range targets {
|
||||
c := targets[i]
|
||||
if err := NewCollectorDeployer(&c, c.LokiDeployment, nil).Uninstall(ctx); err != nil {
|
||||
log.Printf("[deploy-delete] 采集器 %q 卸载失败,保留记录: %v", c.Name, err)
|
||||
continue
|
||||
}
|
||||
db.Lock()
|
||||
for j := range db.Collectors {
|
||||
if db.Collectors[j].ID == c.ID {
|
||||
db.Collectors = append(db.Collectors[:j], db.Collectors[j+1:]...)
|
||||
break
|
||||
}
|
||||
}
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
}
|
||||
}
|
||||
|
||||
func (s *CollectorService) Delete(id uint) error {
|
||||
collector, err := s.Get(id)
|
||||
if err != nil {
|
||||
return utils.ErrNotFound
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
// 先卸载远端资源,成功后再删记录;卸载失败直接返回,避免「界面删了、远端还在」
|
||||
if err := NewCollectorDeployer(collector, collector.LokiDeployment, nil).Uninstall(ctx); err != nil {
|
||||
return fmt.Errorf("卸载采集器失败,未删除记录: %w", err)
|
||||
}
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for i, c := range db.Collectors {
|
||||
if c.ID == id {
|
||||
db.Collectors = append(db.Collectors[:i], db.Collectors[i+1:]...)
|
||||
break
|
||||
db.RequestSave()
|
||||
return nil
|
||||
}
|
||||
}
|
||||
db.Unlock()
|
||||
db.RequestSave()
|
||||
// 异步卸载(用拷贝的 collector,避免删除后取不到;失败留日志,远端不留孤儿容器也无提示)
|
||||
go func() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
if err := NewCollectorDeployer(collector, collector.LokiDeployment, nil).Uninstall(ctx); err != nil {
|
||||
log.Printf("[collector] 删除后卸载 %q(目标 %s)失败: %v", collector.Name, collector.Name, err)
|
||||
}
|
||||
}()
|
||||
return nil
|
||||
return utils.ErrNotFound
|
||||
}
|
||||
|
||||
// Install 异步部署采集器
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
lokimodels "lokistack/models"
|
||||
)
|
||||
|
||||
// 「采集 job 片段的 YAML 编辑」支持(方案 A)。
|
||||
//
|
||||
// 设计边界(很重要):
|
||||
// 这里开放的是**采集项本身**的 YAML 表示,字段与 models.ScrapeJob 一一对应,
|
||||
// **不是**原始 promtail 配置。clients / positions / server / scrape_configs 外层
|
||||
// 仍由平台生成。
|
||||
//
|
||||
// 为什么不做"编辑完整 promtail.yaml":
|
||||
// 用户一旦删掉 positions,重启后会从头重采(大量重复 + entry too far behind 被
|
||||
// Loki 拒绝);删掉 clients 则完全不外发。这两类故障在页面上看不出来,
|
||||
// 只能登机器看日志,排障成本极高。所以只开放"采集规则"这一层,
|
||||
// 它写坏了最多是某条日志没采到,不会让采集器整体失效。
|
||||
|
||||
// jobsYAML 是 ScrapeJob 的 YAML 投影(小写键,便于手写)。
|
||||
type jobsYAML struct {
|
||||
Name string `yaml:"name"`
|
||||
Paths []string `yaml:"paths"`
|
||||
Labels map[string]string `yaml:"labels,omitempty"`
|
||||
Encoding string `yaml:"encoding,omitempty"`
|
||||
System bool `yaml:"system,omitempty"`
|
||||
JSON bool `yaml:"json,omitempty"`
|
||||
Multiline *multilineYAML `yaml:"multiline,omitempty"`
|
||||
Timestamp *timestampYAML `yaml:"timestamp,omitempty"`
|
||||
DropIfMatch []string `yaml:"dropIfMatch,omitempty"`
|
||||
RateLimit *rateLimitYAML `yaml:"rateLimit,omitempty"`
|
||||
}
|
||||
|
||||
type multilineYAML struct {
|
||||
FirstLine string `yaml:"firstline"`
|
||||
MaxLines int `yaml:"maxLines,omitempty"`
|
||||
}
|
||||
|
||||
type timestampYAML struct {
|
||||
Source string `yaml:"source"`
|
||||
Format string `yaml:"format"`
|
||||
Location string `yaml:"location,omitempty"`
|
||||
}
|
||||
|
||||
type rateLimitYAML struct {
|
||||
Rate int `yaml:"rate,omitempty"`
|
||||
Burst int `yaml:"burst,omitempty"`
|
||||
}
|
||||
|
||||
// RenderJobsYAML 把采集项渲染成可编辑 YAML(带头部注释,说明这是什么)。
|
||||
func RenderJobsYAML(jobs []lokimodels.ScrapeJob) (string, error) {
|
||||
out := make([]jobsYAML, 0, len(jobs))
|
||||
for _, job := range jobs {
|
||||
item := jobsYAML{
|
||||
Name: job.Name,
|
||||
Paths: job.Paths,
|
||||
Labels: job.Labels,
|
||||
Encoding: job.Encoding,
|
||||
System: job.System,
|
||||
JSON: job.JSON,
|
||||
DropIfMatch: job.DropIfMatch,
|
||||
}
|
||||
if job.Multiline != nil && strings.TrimSpace(job.Multiline.FirstLine) != "" {
|
||||
item.Multiline = &multilineYAML{FirstLine: job.Multiline.FirstLine, MaxLines: job.Multiline.MaxLines}
|
||||
}
|
||||
if job.Timestamp != nil && strings.TrimSpace(job.Timestamp.Source) != "" {
|
||||
item.Timestamp = ×tampYAML{
|
||||
Source: job.Timestamp.Source, Format: job.Timestamp.Format, Location: job.Timestamp.Location,
|
||||
}
|
||||
}
|
||||
if job.RateLimit != nil && (job.RateLimit.Rate > 0 || job.RateLimit.Burst > 0) {
|
||||
item.RateLimit = &rateLimitYAML{Rate: job.RateLimit.Rate, Burst: job.RateLimit.Burst}
|
||||
}
|
||||
out = append(out, item)
|
||||
}
|
||||
|
||||
body, err := yaml.Marshal(out)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("渲染采集项 YAML 失败: %w", err)
|
||||
}
|
||||
|
||||
var b strings.Builder
|
||||
b.WriteString("# 采集项(scrape_configs)编辑区 —— 只描述「采哪些文件、怎么解析、怎么过滤」\n")
|
||||
b.WriteString("# clients / positions / server 等由平台自动生成,不在此处修改\n")
|
||||
b.WriteString("# 字段:name(必填) paths(必填) labels encoding json system multiline timestamp dropIfMatch rateLimit\n")
|
||||
b.WriteString(string(body))
|
||||
return b.String(), nil
|
||||
}
|
||||
|
||||
// ParseJobsYAML 把用户编辑的 YAML 解析回采集项,并做与表单完全相同的校验。
|
||||
//
|
||||
// 报错要具体:YAML 语法错给行号,语义错给"第几个采集项/哪个字段",
|
||||
// 不要笼统地说"格式错误"——否则用户只能靠猜。
|
||||
func ParseJobsYAML(text string) ([]lokimodels.ScrapeJob, error) {
|
||||
trimmed := strings.TrimSpace(text)
|
||||
if trimmed == "" {
|
||||
return nil, fmt.Errorf("YAML 为空:至少需要一个采集项")
|
||||
}
|
||||
|
||||
var raw []jobsYAML
|
||||
if err := yaml.Unmarshal([]byte(trimmed), &raw); err != nil {
|
||||
return nil, fmt.Errorf("YAML 语法错误:%s", prettifyYAMLError(err))
|
||||
}
|
||||
if len(raw) == 0 {
|
||||
return nil, fmt.Errorf("YAML 里没有采集项(应为一个以 - 开头的列表)")
|
||||
}
|
||||
|
||||
jobs := make([]lokimodels.ScrapeJob, 0, len(raw))
|
||||
for _, item := range raw {
|
||||
job := lokimodels.ScrapeJob{
|
||||
Name: strings.TrimSpace(item.Name),
|
||||
Paths: trimAll(item.Paths),
|
||||
Labels: item.Labels,
|
||||
Encoding: strings.TrimSpace(item.Encoding),
|
||||
System: item.System,
|
||||
JSON: item.JSON,
|
||||
DropIfMatch: trimAll(item.DropIfMatch),
|
||||
}
|
||||
if item.Multiline != nil {
|
||||
job.Multiline = &lokimodels.JobMultiline{
|
||||
FirstLine: item.Multiline.FirstLine, MaxLines: item.Multiline.MaxLines,
|
||||
}
|
||||
}
|
||||
if item.Timestamp != nil {
|
||||
job.Timestamp = &lokimodels.JobTimestamp{
|
||||
Source: item.Timestamp.Source, Format: item.Timestamp.Format, Location: item.Timestamp.Location,
|
||||
}
|
||||
}
|
||||
if item.RateLimit != nil {
|
||||
job.RateLimit = &lokimodels.JobRateLimit{Rate: item.RateLimit.Rate, Burst: item.RateLimit.Burst}
|
||||
}
|
||||
jobs = append(jobs, job)
|
||||
}
|
||||
|
||||
if err := lokimodels.ValidateScrapeJobs(jobs); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return jobs, nil
|
||||
}
|
||||
|
||||
func trimAll(values []string) []string {
|
||||
out := make([]string, 0, len(values))
|
||||
for _, v := range values {
|
||||
if s := strings.TrimSpace(v); s != "" {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// prettifyYAMLError 把 yaml.v3 的多行错误压成一行,保留行号信息。
|
||||
func prettifyYAMLError(err error) string {
|
||||
var lines []string
|
||||
for _, line := range strings.Split(err.Error(), "\n") {
|
||||
if s := strings.TrimSpace(line); s != "" {
|
||||
lines = append(lines, s)
|
||||
}
|
||||
}
|
||||
return strings.Join(lines, "; ")
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
lokimodels "lokistack/models"
|
||||
)
|
||||
|
||||
// YAML 双模式的核心保证:**表单 → YAML → 表单 必须无损往返**。
|
||||
// 如果往返丢字段(比如 multiline.maxLines 或 rateLimit),用户切一下模式
|
||||
// 配置就被悄悄改掉了,这是最难排查的一类问题。
|
||||
func TestRenderParseJobsYAMLRoundTrip(t *testing.T) {
|
||||
original := []lokimodels.ScrapeJob{
|
||||
{
|
||||
Name: "app",
|
||||
Paths: []string{"/var/log/app/*.log", "/data/logs/*.log"},
|
||||
Labels: map[string]string{
|
||||
"tier": "backend",
|
||||
},
|
||||
Encoding: "utf-8",
|
||||
JSON: true,
|
||||
Multiline: &lokimodels.JobMultiline{
|
||||
FirstLine: `^\d{4}-\d{2}-\d{2}`,
|
||||
MaxLines: 256,
|
||||
},
|
||||
DropIfMatch: []string{`/healthz`, `/metrics`},
|
||||
RateLimit: &lokimodels.JobRateLimit{Rate: 2000, Burst: 4000},
|
||||
Timestamp: &lokimodels.JobTimestamp{
|
||||
Source: "ts",
|
||||
Format: "RFC3339Nano",
|
||||
Location: "Local",
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "system",
|
||||
Paths: []string{"/var/log/messages"},
|
||||
System: true,
|
||||
},
|
||||
}
|
||||
|
||||
text, err := RenderJobsYAML(original)
|
||||
if err != nil {
|
||||
t.Fatalf("渲染失败: %v", err)
|
||||
}
|
||||
// 头部注释必须说明"这不是完整 promtail 配置",避免用户以为能改 clients
|
||||
if !strings.Contains(text, "clients") || !strings.Contains(text, "不在此处修改") {
|
||||
t.Error("渲染结果缺少边界说明注释")
|
||||
}
|
||||
|
||||
parsed, err := ParseJobsYAML(text)
|
||||
if err != nil {
|
||||
t.Fatalf("解析失败: %v\n--- YAML ---\n%s", err, text)
|
||||
}
|
||||
if len(parsed) != len(original) {
|
||||
t.Fatalf("采集项数量不符: got %d want %d", len(parsed), len(original))
|
||||
}
|
||||
|
||||
first := parsed[0]
|
||||
if first.Name != "app" || len(first.Paths) != 2 {
|
||||
t.Errorf("基础字段丢失: %+v", first)
|
||||
}
|
||||
if first.Multiline == nil || first.Multiline.MaxLines != 256 {
|
||||
t.Errorf("multiline 丢失: %+v", first.Multiline)
|
||||
}
|
||||
if first.RateLimit == nil || first.RateLimit.Rate != 2000 || first.RateLimit.Burst != 4000 {
|
||||
t.Errorf("rateLimit 丢失: %+v", first.RateLimit)
|
||||
}
|
||||
if len(first.DropIfMatch) != 2 {
|
||||
t.Errorf("dropIfMatch 丢失: %+v", first.DropIfMatch)
|
||||
}
|
||||
if !first.JSON || first.Encoding != "utf-8" {
|
||||
t.Errorf("json/encoding 丢失: json=%v enc=%q", first.JSON, first.Encoding)
|
||||
}
|
||||
if first.Timestamp == nil || first.Timestamp.Source != "ts" || first.Timestamp.Location != "Local" {
|
||||
t.Errorf("timestamp 丢失: %+v", first.Timestamp)
|
||||
}
|
||||
if first.Labels["tier"] != "backend" {
|
||||
t.Errorf("labels 丢失: %+v", first.Labels)
|
||||
}
|
||||
if !parsed[1].System {
|
||||
t.Error("system 标记丢失")
|
||||
}
|
||||
}
|
||||
|
||||
// 报错必须具体:语法错要能定位,语义错要指出是哪个采集项,不能笼统"格式错误"。
|
||||
func TestParseJobsYAMLErrors(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
want string
|
||||
}{
|
||||
{"空内容", " ", "YAML 为空"},
|
||||
{"语法错误", "- name: app\n paths: [/a\n", "YAML 语法错误"},
|
||||
{"空列表", "[]", "没有采集项"},
|
||||
{"缺名称", "- paths: [/var/log/x.log]", "缺少名称"},
|
||||
{"缺路径", "- name: app", "至少需要填写一个日志路径"},
|
||||
{"名称非法", "- name: 'bad name!'\n paths: [/a]", "不合法"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
_, err := ParseJobsYAML(c.in)
|
||||
if err == nil {
|
||||
t.Fatalf("期望报错,实际通过")
|
||||
}
|
||||
if !strings.Contains(err.Error(), c.want) {
|
||||
t.Errorf("错误信息不够具体: got %q, want contains %q", err.Error(), c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -2,8 +2,8 @@ package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
@@ -19,6 +19,10 @@ type DailyVolumePoint struct {
|
||||
Partial bool `json:"partial"`
|
||||
// FromHistory true=来自平台留存历史;false=刚从 Loki 实时查的
|
||||
FromHistory bool `json:"fromHistory"`
|
||||
// Missing 该天平台没有抓到快照(超过留存期/平台当天没运行)。
|
||||
// 必须和"当天日志量真的是 0"区分开:前者是"不知道",后者是"确实没日志"。
|
||||
// 否则 30/90/365 天的图上会出现大片 0,被误读成"日志量暴跌"。
|
||||
Missing bool `json:"missing"`
|
||||
}
|
||||
|
||||
const (
|
||||
@@ -28,6 +32,13 @@ const (
|
||||
dailyVolumeBackfillDays = 2
|
||||
// dailyVolumeRetentionDays 平台自己留多久的历史。一年足够看年度趋势。
|
||||
dailyVolumeRetentionDays = 400
|
||||
|
||||
// entriesMaxAttempts 条数(index/stats)每天最多尝试次数。
|
||||
// 整天跨度要去重统计全部 chunk:实测空闲时 6ms、有并发查询时 20s 以上,
|
||||
// 属于"附加指标",不值得无限重试。
|
||||
entriesMaxAttempts = 3
|
||||
// entriesRetryInterval 两次条数尝试之间的最小间隔。
|
||||
entriesRetryInterval = time.Hour
|
||||
)
|
||||
|
||||
// dateKey 返回本地时区的 YYYY-MM-DD。
|
||||
@@ -42,10 +53,22 @@ func dayRange(day time.Time) (int64, int64) {
|
||||
return start.UnixNano(), start.AddDate(0, 0, 1).UnixNano()
|
||||
}
|
||||
|
||||
// CaptureDailyVolume 抓取指定日期的日志量并落库(按天幂等:同一天重复抓会覆盖)。
|
||||
// formatGB 用于日志输出,运维看日志时比裸字节直观。
|
||||
func formatGB(bytes int64) string {
|
||||
return fmt.Sprintf("%.2f GB", float64(bytes)/1e9)
|
||||
}
|
||||
|
||||
// CaptureDailyVolume 抓取指定日期的**日志体积**并落库(按天幂等:同一天重复抓会覆盖)。
|
||||
//
|
||||
// 数据来源是 Loki 的 index/stats(只读索引元数据),实测毫秒级,
|
||||
// 所以就算是补几十天历史也不会拖慢任何东西。
|
||||
// 只用 index/volume_range:它只读 TSDB 索引里记录的 chunk 字节数,不解码日志内容。
|
||||
// 同一 selector 24 小时实测(本集群 ~370GB/天):
|
||||
//
|
||||
// index/volume_range 54ms ~ 4s ← 本函数走的路径
|
||||
// index/stats 6ms ~ 20s ← 条数才需要,见 captureDailyEntries
|
||||
// count_over_time 60s 超时 ← 永远不要用在"每日量"上
|
||||
//
|
||||
// 条数不在这里抓:它只能走 index/stats 且整天跨度很贵,独立成 captureDailyEntries,
|
||||
// 每天最多尝试 3 次,不跟着 30 分钟的定时任务反复跑。
|
||||
func CaptureDailyVolume(source *models.LogSource, day time.Time, partial bool) error {
|
||||
if source == nil {
|
||||
return nil
|
||||
@@ -59,24 +82,49 @@ func CaptureDailyVolume(source *models.LogSource, day time.Time, partial bool) e
|
||||
if start >= end {
|
||||
return nil
|
||||
}
|
||||
// 回补历史时,如果那天的起点已经超出 Loki 保留期(默认 24h),
|
||||
// Loki 里根本没那么多数据 —— 必须标成"不完整",
|
||||
// 否则图上会把"数据被删了"误读成"那天日志量真的很少"。
|
||||
if start < time.Now().Add(-24*time.Hour).UnixNano() {
|
||||
partial = true
|
||||
|
||||
// 起点已超出常见保留期(24h)时,Loki 里可能已经没有那天的数据了。
|
||||
// 这种情况下查出来的 0 字节**绝不能落库**:
|
||||
// 落库就会把"数据被 retention 删掉了"画成"那天没有日志",
|
||||
// 比灰色柱(未留存)误导得多。所以按"跳过落库 = 保持 Missing"处理。
|
||||
stale := start < time.Now().Add(-24*time.Hour).UnixNano()
|
||||
|
||||
begin := time.Now()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
defer cancel()
|
||||
buckets, volErr := LokiHistogram(ctx, source, `{namespace=~".+"}`, start, end)
|
||||
elapsed := time.Since(begin)
|
||||
if volErr != nil {
|
||||
return fmt.Errorf("volume_range: %w", volErr)
|
||||
}
|
||||
|
||||
// 只查总量(不带 namespace 条件),一次拿到该源当天的条数与体积
|
||||
// 整天(24h)的 index/stats:大集群上比 15 分钟的查询慢得多,
|
||||
// 实测 20s 会超时(这也是之前每日量接口卡 20s 的原因)。给 90s。
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
stats, err := LokiStatsFor(ctx, source, `{namespace=~".+"}`, start, end)
|
||||
if err != nil {
|
||||
return err
|
||||
var totalBytes int64
|
||||
for _, b := range buckets {
|
||||
switch v := b["count"].(type) {
|
||||
case int64:
|
||||
totalBytes += v
|
||||
case float64:
|
||||
totalBytes += int64(v)
|
||||
}
|
||||
}
|
||||
|
||||
if totalBytes == 0 && stale {
|
||||
log.Printf("[daily-volume] %s 超出保留期且查不到数据,跳过落库(图上显示「未留存」)", dateKey(day))
|
||||
return nil
|
||||
}
|
||||
|
||||
date := dateKey(day)
|
||||
if err := upsertVolumeSnapshot(source.ID, date, totalBytes, partial); err != nil {
|
||||
return err
|
||||
}
|
||||
log.Printf("[daily-volume] %s 体积 %s 已落库(volume_range %s,%d 个桶)",
|
||||
date, formatGB(totalBytes), elapsed.Round(time.Millisecond), len(buckets))
|
||||
return nil
|
||||
}
|
||||
|
||||
// upsertVolumeSnapshot 写入/更新某天的体积快照(幂等)。
|
||||
// 只改体积与 partial,**不动已经抓到的条数**(条数贵,抓到就留着)。
|
||||
func upsertVolumeSnapshot(sourceID uint, date string, bytes int64, partial bool) error {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return nil
|
||||
@@ -93,10 +141,8 @@ func CaptureDailyVolume(source *models.LogSource, day time.Time, partial bool) e
|
||||
if snap.ID > nextID {
|
||||
nextID = snap.ID
|
||||
}
|
||||
// 同一天同源:覆盖(幂等)
|
||||
if snap.Date == date && snap.SourceID == source.ID && snap.Namespace == "" {
|
||||
snap.Bytes = stats.Bytes
|
||||
snap.Entries = stats.Entries
|
||||
if snap.Date == date && snap.SourceID == sourceID && snap.Namespace == "" {
|
||||
snap.Bytes = bytes
|
||||
snap.Partial = partial
|
||||
snap.CapturedAt = time.Now()
|
||||
db.RequestSave()
|
||||
@@ -107,10 +153,9 @@ func CaptureDailyVolume(source *models.LogSource, day time.Time, partial bool) e
|
||||
db.DailyVolumeSnapshots = append(db.DailyVolumeSnapshots, models.DailyVolumeSnapshot{
|
||||
ID: nextID,
|
||||
Date: date,
|
||||
SourceID: source.ID,
|
||||
SourceID: sourceID,
|
||||
Namespace: "",
|
||||
Bytes: stats.Bytes,
|
||||
Entries: stats.Entries,
|
||||
Bytes: bytes,
|
||||
Partial: partial,
|
||||
CapturedAt: time.Now(),
|
||||
})
|
||||
@@ -118,7 +163,116 @@ func CaptureDailyVolume(source *models.LogSource, day time.Time, partial bool) e
|
||||
return nil
|
||||
}
|
||||
|
||||
// captureDailyEntries 给"已经过完、但还没有条数"的天补一次条数(尽力而为)。
|
||||
//
|
||||
// 为什么不等同于体积每 30 分钟抓一次:
|
||||
// 条数只能走 index/stats,要跨该天全部 chunk 做 streams/entries 去重统计,
|
||||
// 实测 6ms(空闲)~ 20s(有并发查询时)。每 30 分钟 × 每一天都重跑一遍,
|
||||
// 既浪费 Loki 的 CPU,又会和用户页面上的查询抢同一台单实例的算力。所以:
|
||||
//
|
||||
// - 只处理 Date < 今天 的天(今天没结束,条数没有意义)
|
||||
// - 每天最多 entriesMaxAttempts 次,两次间隔 ≥ entriesRetryInterval
|
||||
// - 尝试次数**先写库再查**:进程中途被杀也不会退化成"每 30 分钟重试"
|
||||
// - 拿不到就留 0,前端图只画体积,不影响可读性
|
||||
func captureDailyEntries(source *models.LogSource) {
|
||||
if source == nil {
|
||||
return
|
||||
}
|
||||
now := time.Now()
|
||||
today := dateKey(now)
|
||||
|
||||
var dates []string
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.RLock()
|
||||
for _, snap := range db.DailyVolumeSnapshots {
|
||||
if snap.SourceID != source.ID || snap.Namespace != "" {
|
||||
continue
|
||||
}
|
||||
if snap.Date >= today || snap.Entries > 0 {
|
||||
continue
|
||||
}
|
||||
if snap.EntriesAttempts >= entriesMaxAttempts {
|
||||
continue
|
||||
}
|
||||
if !snap.LastEntriesAttempt.IsZero() && now.Sub(snap.LastEntriesAttempt) < entriesRetryInterval {
|
||||
continue
|
||||
}
|
||||
dates = append(dates, snap.Date)
|
||||
}
|
||||
db.RUnlock()
|
||||
|
||||
for _, date := range dates {
|
||||
// 先把"尝试"记下来再查:查的过程中进程重启,也不会变成无限重试
|
||||
markEntriesAttempt(source.ID, date)
|
||||
|
||||
day, err := time.ParseInLocation("2006-01-02", date, time.Local)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
start, end := dayRange(day)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
|
||||
begin := time.Now()
|
||||
stats, statErr := LokiStatsFor(ctx, source, `{namespace=~".+"}`, start, end)
|
||||
cancel()
|
||||
if statErr != nil {
|
||||
log.Printf("[daily-volume] %s 条数统计失败(体积已落库,不影响图表): %v", date, statErr)
|
||||
continue
|
||||
}
|
||||
if stats.Entries <= 0 {
|
||||
continue
|
||||
}
|
||||
setSnapshotEntries(source.ID, date, stats.Entries)
|
||||
log.Printf("[daily-volume] %s 条数 %d(index/stats %s)",
|
||||
date, stats.Entries, time.Since(begin).Round(time.Millisecond))
|
||||
}
|
||||
}
|
||||
|
||||
// markEntriesAttempt 记录一次条数尝试(先写库再查询,失败也计数)。
|
||||
func markEntriesAttempt(sourceID uint, date string) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for i := range db.DailyVolumeSnapshots {
|
||||
snap := &db.DailyVolumeSnapshots[i]
|
||||
if snap.Date == date && snap.SourceID == sourceID && snap.Namespace == "" {
|
||||
snap.EntriesAttempts++
|
||||
snap.LastEntriesAttempt = time.Now()
|
||||
db.RequestSave()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// setSnapshotEntries 写入条数。
|
||||
func setSnapshotEntries(sourceID uint, date string, entries int64) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for i := range db.DailyVolumeSnapshots {
|
||||
snap := &db.DailyVolumeSnapshots[i]
|
||||
if snap.Date == date && snap.SourceID == sourceID && snap.Namespace == "" {
|
||||
snap.Entries = entries
|
||||
db.RequestSave()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// CaptureRecentDailyVolumes 回补最近 N 天(含今天),供启动时与定时任务调用。
|
||||
//
|
||||
// 两条路径分开:
|
||||
//
|
||||
// 体积(volume_range,毫秒级)→ 今天 + 昨天,每 30 分钟刷新,当天柱子保持新鲜
|
||||
// 条数(index/stats,可能 20s)→ 只给已结束且还没抓到条数的天补,每天最多 3 次
|
||||
func CaptureRecentDailyVolumes() {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
@@ -136,41 +290,87 @@ func CaptureRecentDailyVolumes() {
|
||||
src := src
|
||||
for offset := dailyVolumeBackfillDays - 1; offset >= 0; offset-- {
|
||||
day := now.AddDate(0, 0, -offset)
|
||||
// 今天是不完整的一天
|
||||
// 只有今天是不完整的一天
|
||||
partial := offset == 0
|
||||
if err := CaptureDailyVolume(&src, day, partial); err != nil {
|
||||
log.Printf("[daily-volume] 抓取 %s %s 失败: %v", src.Name, dateKey(day), err)
|
||||
}
|
||||
}
|
||||
captureDailyEntries(&src)
|
||||
}
|
||||
pruneDailyVolumeSnapshots()
|
||||
}
|
||||
|
||||
// pruneDailyVolumeSnapshots 丢弃超过保留期的历史。
|
||||
// pruneDailyVolumeSnapshots 丢弃超过保留期的历史,并修正历史遗留的 partial 标记。
|
||||
func pruneDailyVolumeSnapshots() {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
cutoff := dateKey(time.Now().AddDate(0, 0, -dailyVolumeRetentionDays))
|
||||
today := dateKey(time.Now())
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
known := make(map[uint]bool, len(db.LogSources))
|
||||
for _, src := range db.LogSources {
|
||||
known[src.ID] = true
|
||||
}
|
||||
changed := false
|
||||
kept := make([]models.DailyVolumeSnapshot, 0, len(db.DailyVolumeSnapshots))
|
||||
droppedOrphan := 0
|
||||
for _, snap := range db.DailyVolumeSnapshots {
|
||||
if snap.Date >= cutoff {
|
||||
kept = append(kept, snap)
|
||||
// 超过留存期
|
||||
if snap.Date < cutoff {
|
||||
continue
|
||||
}
|
||||
// 数据源已被删除:它的快照没有任何入口能查到,留着只会混淆取证
|
||||
// (实测遇到过库里留着 id=2 的历史,查"到底抓没抓"时白查一轮)
|
||||
if !known[snap.SourceID] {
|
||||
droppedOrphan++
|
||||
continue
|
||||
}
|
||||
kept = append(kept, snap)
|
||||
}
|
||||
if droppedOrphan > 0 {
|
||||
log.Printf("[daily-volume] 清理已删除数据源的快照 %d 条", droppedOrphan)
|
||||
}
|
||||
if len(kept) != len(db.DailyVolumeSnapshots) {
|
||||
log.Printf("[daily-volume] 清理过期快照 %d 条", len(db.DailyVolumeSnapshots)-len(kept))
|
||||
log.Printf("[daily-volume] 清理过期快照 %d 条", len(db.DailyVolumeSnapshots)-len(kept)-droppedOrphan)
|
||||
db.DailyVolumeSnapshots = kept
|
||||
changed = true
|
||||
}
|
||||
|
||||
// 历史遗留修正:**过去的天不可能"还在进行中"**。
|
||||
// 早期版本把「起点超出 24h」也当成 partial,于是昨天/前天的柱子一直是虚线空心,
|
||||
// 看起来像"每天都在采样中"。只有当快照是在那天**结束之后**才抓的
|
||||
// (说明抓的是完整一天)才改成完整;当天中途抓到的继续保持 partial —— 那时确实只有半天。
|
||||
for i := range db.DailyVolumeSnapshots {
|
||||
snap := &db.DailyVolumeSnapshots[i]
|
||||
if !snap.Partial || snap.Date >= today {
|
||||
continue
|
||||
}
|
||||
day, err := time.ParseInLocation("2006-01-02", snap.Date, time.Local)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if _, dayEnd := dayRange(day); snap.CapturedAt.UnixNano() >= dayEnd {
|
||||
snap.Partial = false
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
if changed {
|
||||
db.RequestSave()
|
||||
}
|
||||
}
|
||||
|
||||
// GetDailyVolumeHistory 从平台留存的历史里取最近 days 天,按日期升序返回。
|
||||
//
|
||||
// 今天如果还没抓过,会实时补一次(这样用户切到页面时当天数字是最新的)。
|
||||
// 这个接口**只读库,绝不触发任何 Loki 查询** —— 图表刷新不应该产生任何 Loki 负载。
|
||||
// 当天那根柱子由后台定时任务(每 30 分钟,volume_range 毫秒级)补,慢几分钟可接受。
|
||||
//
|
||||
// 事故复盘:这里原来同步 await 一次"整天 index/stats",那次查询超时 20 秒,
|
||||
// 于是 /logs/daily-volume 耗时 20s → 前端超时 → 页面显示"每日日志量没数据";
|
||||
// 而且这条重查询还和页面的 histogram/stats 抢 Loki,导致那两个接口同时 502。
|
||||
func GetDailyVolumeHistory(sourceID uint, days int) []DailyVolumePoint {
|
||||
if days <= 0 {
|
||||
days = 7
|
||||
@@ -179,54 +379,116 @@ func GetDailyVolumeHistory(sourceID uint, days int) []DailyVolumePoint {
|
||||
days = dailyVolumeRetentionDays
|
||||
}
|
||||
|
||||
// 这个接口**只读库**,绝不触发任何 Loki 查询。
|
||||
//
|
||||
// 事故复盘:这里原来同步 await 一次"整天 index/stats",那次查询超时 20 秒,
|
||||
// 于是 /logs/daily-volume 耗时 20s → 前端超时 → 页面显示"每日日志量没数据";
|
||||
// 而且这条重查询还和页面的 histogram/stats 抢 Loki,导致那两个接口同时 502。
|
||||
// 当天那根柱子由后台定时任务(每 30 分钟)补,慢几分钟完全可接受。
|
||||
|
||||
from := dateKey(time.Now().AddDate(0, 0, -(days - 1)))
|
||||
points := []DailyVolumePoint{}
|
||||
// 始终返回 days 个点(缺失的那天标 Missing)。
|
||||
// 之前只返回"已抓到的快照",于是 7/30/90/365 全都返回同样 4 个点,
|
||||
// 用户切范围看不到任何变化,以为"切换没反应"。
|
||||
now := time.Now()
|
||||
byDate := map[string]models.DailyVolumeSnapshot{}
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return points
|
||||
if db != nil {
|
||||
db.RLock()
|
||||
for _, snap := range db.DailyVolumeSnapshots {
|
||||
if snap.SourceID != sourceID || snap.Namespace != "" {
|
||||
continue
|
||||
}
|
||||
byDate[snap.Date] = snap
|
||||
}
|
||||
db.RUnlock()
|
||||
}
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
for _, snap := range db.DailyVolumeSnapshots {
|
||||
if snap.SourceID != sourceID || snap.Namespace != "" {
|
||||
|
||||
points := make([]DailyVolumePoint, 0, days)
|
||||
for i := days - 1; i >= 0; i-- {
|
||||
date := dateKey(now.AddDate(0, 0, -i))
|
||||
if snap, ok := byDate[date]; ok {
|
||||
points = append(points, DailyVolumePoint{
|
||||
Date: snap.Date,
|
||||
Bytes: snap.Bytes,
|
||||
Entries: snap.Entries,
|
||||
Partial: snap.Partial,
|
||||
FromHistory: true,
|
||||
})
|
||||
continue
|
||||
}
|
||||
if snap.Date < from {
|
||||
continue
|
||||
}
|
||||
points = append(points, DailyVolumePoint{
|
||||
Date: snap.Date,
|
||||
Bytes: snap.Bytes,
|
||||
Entries: snap.Entries,
|
||||
Partial: snap.Partial,
|
||||
FromHistory: true,
|
||||
})
|
||||
points = append(points, DailyVolumePoint{Date: date, Missing: true, FromHistory: true})
|
||||
}
|
||||
sort.Slice(points, func(i, j int) bool { return points[i].Date < points[j].Date })
|
||||
return points
|
||||
}
|
||||
|
||||
// StartDailyVolumeCapture 后台每日抓取。
|
||||
//
|
||||
// 每 30 分钟检查一次:只要发现"昨天"还没落库就立刻补,
|
||||
// 而不是死等 00:10 —— 平台可能整晚都没运行,定时器会错过那个点。
|
||||
// 每 30 分钟跑一轮(体积是毫秒级的 volume_range,跑得勤一点让"今天"这根柱子接近实时):
|
||||
// - 今天 + 昨天的体积:每轮都刷
|
||||
// - 条数:只在"已结束且还没有条数"的天触发,每天最多 3 次(见 captureDailyEntries)
|
||||
//
|
||||
// 用"检查式回补"而不是死等 00:10 的原因:平台可能整晚都没运行,定时器会错过那个点;
|
||||
// 每轮都检查"昨天有没有落库",漏了就立刻补,天然幂等。
|
||||
func StartDailyVolumeCapture() {
|
||||
go func() {
|
||||
// 启动先补最近两天(覆盖重启场景)
|
||||
time.Sleep(20 * time.Second)
|
||||
log.Printf("[daily-volume] 启动回补最近 %d 天", dailyVolumeBackfillDays)
|
||||
CaptureRecentDailyVolumes()
|
||||
|
||||
// 两条节奏:
|
||||
// 30 分钟 ticker → 刷新"今天"的柱子(当天数据一直在涨)
|
||||
// 零点后 1 分钟 → **结算昨天**(把它定稿成完整一天)
|
||||
//
|
||||
// 为什么零点结算必须对齐而不能靠 30 分钟 ticker:
|
||||
// Loki 保留期默认 24h,昨天 00:00 的数据在今天 00:00 之后就开始被删。
|
||||
// 如果等到 00:30 那次 ticker 才抓,昨天最早的 30 分钟已经没了,
|
||||
// 结算结果会系统性偏低,而图上又标着"完整一天",属于静默错数。
|
||||
// 00:01 结算时最多只丢 1 分钟,误差可忽略。
|
||||
ticker := time.NewTicker(30 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for range ticker.C {
|
||||
CaptureRecentDailyVolumes()
|
||||
midnight := time.NewTimer(time.Until(nextMidnightFinalize()))
|
||||
defer midnight.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
CaptureRecentDailyVolumes()
|
||||
case <-midnight.C:
|
||||
FinalizeYesterdayVolume()
|
||||
midnight.Reset(time.Until(nextMidnightFinalize()))
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// nextMidnightFinalize 返回下一个"零点后 1 分钟"的时间点。
|
||||
func nextMidnightFinalize() time.Time {
|
||||
now := time.Now()
|
||||
next := time.Date(now.Year(), now.Month(), now.Day()+1, 0, 1, 0, 0, time.Local)
|
||||
return next
|
||||
}
|
||||
|
||||
// FinalizeYesterdayVolume 零点后结算昨天:把昨天定稿(partial=false),
|
||||
// 同时刷新今天的 partial=true 柱子。多数据源逐个处理,单个失败不影响其它。
|
||||
func FinalizeYesterdayVolume() {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.RLock()
|
||||
sources := make([]models.LogSource, 0, len(db.LogSources))
|
||||
for _, src := range db.LogSources {
|
||||
sources = append(sources, src)
|
||||
}
|
||||
db.RUnlock()
|
||||
|
||||
now := time.Now()
|
||||
yesterday := now.AddDate(0, 0, -1)
|
||||
for _, src := range sources {
|
||||
src := src
|
||||
if err := CaptureDailyVolume(&src, yesterday, false); err != nil {
|
||||
log.Printf("[daily-volume] 零点结算 %s %s 失败: %v", src.Name, dateKey(yesterday), err)
|
||||
continue
|
||||
}
|
||||
log.Printf("[daily-volume] 零点结算完成 %s %s", src.Name, dateKey(yesterday))
|
||||
// 昨天定稿后,条数也可以补一次(走 index/stats,慢但一天一次可接受)
|
||||
captureDailyEntries(&src)
|
||||
if err := CaptureDailyVolume(&src, now, true); err != nil {
|
||||
log.Printf("[daily-volume] 刷新今天 %s 失败: %v", src.Name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
// 条数(index/stats)只给"已结束且还没有条数"的天补,并且每天最多 3 次。
|
||||
// 这条护栏的直接目的:别让 30 分钟一次的定时任务把整天 index/stats 反复重跑。
|
||||
func TestCaptureDailyEntriesOnlyForPastDaysAndWrites(t *testing.T) {
|
||||
setupDailyVolumeDB(t)
|
||||
|
||||
var calls int32
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
atomic.AddInt32(&calls, 1)
|
||||
if r.URL.Path != "/loki/api/v1/index/stats" {
|
||||
t.Errorf("意外请求路径: %s", r.URL.Path)
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
fmt.Fprint(w, `{"streams":3,"chunks":9,"bytes":123456,"entries":456}`)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
src := &models.LogSource{ID: 1, Name: "test", TargetType: "url", URL: srv.URL}
|
||||
yesterday := dateKey(time.Now().AddDate(0, 0, -1))
|
||||
today := dateKey(time.Now())
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.DailyVolumeSnapshots = []models.DailyVolumeSnapshot{
|
||||
{ID: 1, Date: yesterday, SourceID: 1, Bytes: 100},
|
||||
{ID: 2, Date: today, SourceID: 1, Bytes: 10, Partial: true},
|
||||
}
|
||||
db.Unlock()
|
||||
|
||||
captureDailyEntries(src)
|
||||
|
||||
got := snapshotsByDate(t)
|
||||
if snap := got[yesterday]; snap.Entries != 456 {
|
||||
t.Fatalf("昨天的条数应写入 456,得到 %d", snap.Entries)
|
||||
}
|
||||
if snap := got[yesterday]; snap.EntriesAttempts != 1 {
|
||||
t.Errorf("昨天应记录 1 次尝试,得到 %d", snap.EntriesAttempts)
|
||||
}
|
||||
if snap := got[today]; snap.EntriesAttempts != 0 || snap.Entries != 0 {
|
||||
t.Errorf("今天没结束,不应抓条数(attempts=%d entries=%d)", snap.EntriesAttempts, snap.Entries)
|
||||
}
|
||||
if n := atomic.LoadInt32(&calls); n != 1 {
|
||||
t.Errorf("应只打 1 次 index/stats,实际 %d 次", n)
|
||||
}
|
||||
|
||||
// 1 小时内重复触发不应再打 Loki
|
||||
captureDailyEntries(src)
|
||||
if n := atomic.LoadInt32(&calls); n != 1 {
|
||||
t.Errorf("重试间隔内不应再查询,实际 %d 次", n)
|
||||
}
|
||||
}
|
||||
|
||||
// 失败(Loki 报错)时必须吃得下,并且最多尝试 entriesMaxAttempts 次就放弃。
|
||||
func TestCaptureDailyEntriesRetryBudget(t *testing.T) {
|
||||
setupDailyVolumeDB(t)
|
||||
|
||||
var calls int32
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
atomic.AddInt32(&calls, 1)
|
||||
http.Error(w, "boom", http.StatusInternalServerError)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
src := &models.LogSource{ID: 1, Name: "test", TargetType: "url", URL: srv.URL}
|
||||
yesterday := dateKey(time.Now().AddDate(0, 0, -1))
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.DailyVolumeSnapshots = []models.DailyVolumeSnapshot{{ID: 1, Date: yesterday, SourceID: 1, Bytes: 100}}
|
||||
db.Unlock()
|
||||
|
||||
// 模拟"每轮定时任务都到点":把上次尝试时间清空,绕过 1 小时间隔
|
||||
for i := 0; i < entriesMaxAttempts+3; i++ {
|
||||
captureDailyEntries(src)
|
||||
db.Lock()
|
||||
for j := range db.DailyVolumeSnapshots {
|
||||
db.DailyVolumeSnapshots[j].LastEntriesAttempt = time.Time{}
|
||||
}
|
||||
db.Unlock()
|
||||
}
|
||||
|
||||
if n := atomic.LoadInt32(&calls); n != entriesMaxAttempts {
|
||||
t.Errorf("应恰好尝试 %d 次后放弃,实际 %d 次", entriesMaxAttempts, n)
|
||||
}
|
||||
if snap := snapshotsByDate(t)[yesterday]; snap.Entries != 0 || snap.EntriesAttempts != entriesMaxAttempts {
|
||||
t.Errorf("失败后应留 entries=0 / attempts=%d,得到 entries=%d attempts=%d",
|
||||
entriesMaxAttempts, snap.Entries, snap.EntriesAttempts)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
func setupDailyVolumeDB(t *testing.T) {
|
||||
t.Helper()
|
||||
t.Setenv("DATA_DIR", t.TempDir())
|
||||
t.Setenv("JWT_SECRET", "test-jwt-secret-with-at-least-32-characters")
|
||||
t.Setenv("ENCRYPTION_KEY", "test-encryption-key-with-at-least-32-characters")
|
||||
if err := repository.InitDB(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = repository.Shutdown() })
|
||||
|
||||
// 每天的快照都挂着数据源,prune 会清掉"数据源已删除"的孤儿快照,
|
||||
// 所以测试里先登记好 id=1 的数据源。
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
if len(db.LogSources) == 0 {
|
||||
db.LogSources = append(db.LogSources, models.LogSource{ID: 1, Name: "test-loki", TargetType: "url"})
|
||||
}
|
||||
db.Unlock()
|
||||
}
|
||||
|
||||
func snapshotsByDate(t *testing.T) map[string]models.DailyVolumeSnapshot {
|
||||
t.Helper()
|
||||
got := map[string]models.DailyVolumeSnapshot{}
|
||||
db := repository.GetDB()
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
for _, snap := range db.DailyVolumeSnapshots {
|
||||
got[snap.Date] = snap
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
// 过去的天不可能"还在进行中":只有在那天结束之后才抓的快照才算完整一天,
|
||||
// 当天中途抓到的必须保持 partial —— 否则会把"只抓到半天"伪装成完整数据。
|
||||
func TestPruneDailyVolumeNormalizesPartialAndDropsExpired(t *testing.T) {
|
||||
setupDailyVolumeDB(t)
|
||||
now := time.Now()
|
||||
midday := func(d time.Time) time.Time {
|
||||
return time.Date(d.Year(), d.Month(), d.Day(), 12, 0, 0, 0, time.Local)
|
||||
}
|
||||
yesterday := now.AddDate(0, 0, -1)
|
||||
dayBefore := now.AddDate(0, 0, -2)
|
||||
expired := now.AddDate(0, 0, -(dailyVolumeRetentionDays + 5))
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.DailyVolumeSnapshots = []models.DailyVolumeSnapshot{
|
||||
// 昨天结束之后(次日 12:00)抓的 → 完整一天
|
||||
{ID: 1, Date: dateKey(yesterday), SourceID: 1, Bytes: 100, Entries: 7,
|
||||
Partial: true, CapturedAt: midday(now)},
|
||||
// 前天中午抓的 → 只有半天,保持 partial
|
||||
{ID: 2, Date: dateKey(dayBefore), SourceID: 1, Bytes: 50,
|
||||
Partial: true, CapturedAt: midday(dayBefore)},
|
||||
// 今天 → 永远 partial
|
||||
{ID: 3, Date: dateKey(now), SourceID: 1, Bytes: 10, Partial: true},
|
||||
// 超过留存期 → 必须被清理
|
||||
{ID: 4, Date: dateKey(expired), SourceID: 1, Bytes: 999},
|
||||
// 数据源已删除的孤儿快照 → 必须被清理
|
||||
{ID: 5, Date: dateKey(yesterday), SourceID: 99, Bytes: 888},
|
||||
}
|
||||
db.Unlock()
|
||||
|
||||
pruneDailyVolumeSnapshots()
|
||||
|
||||
got := snapshotsByDate(t)
|
||||
if _, ok := got[dateKey(expired)]; ok {
|
||||
t.Error("超过留存期的快照没有被清理")
|
||||
}
|
||||
// 孤儿快照(SourceID=99 的数据源已不存在)必须被清掉
|
||||
db = repository.GetDB()
|
||||
db.RLock()
|
||||
for _, snap := range db.DailyVolumeSnapshots {
|
||||
if snap.SourceID == 99 {
|
||||
t.Error("数据源已删除的孤儿快照没有被清理")
|
||||
}
|
||||
}
|
||||
db.RUnlock()
|
||||
if snap := got[dateKey(yesterday)]; snap.Partial {
|
||||
t.Error("昨天结束之后抓的快照应标记为完整(partial=false)")
|
||||
}
|
||||
if snap := got[dateKey(dayBefore)]; !snap.Partial {
|
||||
t.Error("当天中途抓的快照必须保持 partial=true")
|
||||
}
|
||||
if snap := got[dateKey(now)]; !snap.Partial {
|
||||
t.Error("今天必须保持 partial=true")
|
||||
}
|
||||
if snap := got[dateKey(yesterday)]; snap.Entries != 7 {
|
||||
t.Errorf("条数不应被清理流程清空,得到 %d", snap.Entries)
|
||||
}
|
||||
}
|
||||
|
||||
// 刷新体积不能把已经抓到的条数(贵、每天最多 3 次)冲掉。
|
||||
func TestUpsertVolumeSnapshotKeepsEntries(t *testing.T) {
|
||||
setupDailyVolumeDB(t)
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.DailyVolumeSnapshots = []models.DailyVolumeSnapshot{
|
||||
{ID: 1, Date: "2026-10-01", SourceID: 1, Bytes: 111, Entries: 42, Partial: true},
|
||||
}
|
||||
db.Unlock()
|
||||
|
||||
if err := upsertVolumeSnapshot(1, "2026-10-01", 1000, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := snapshotsByDate(t)["2026-10-01"]
|
||||
if got.Bytes != 1000 || got.Entries != 42 || got.Partial {
|
||||
t.Fatalf("更新后 bytes=%d entries=%d partial=%v;期望 1000/42/false", got.Bytes, got.Entries, got.Partial)
|
||||
}
|
||||
|
||||
if err := upsertVolumeSnapshot(1, "2026-10-02", 2000, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := snapshotsByDate(t); len(got) != 2 {
|
||||
t.Fatalf("新增一天后应有 2 条快照,得到 %d", len(got))
|
||||
}
|
||||
}
|
||||
|
||||
// 接口必须始终返回 days 个点,缺的标 Missing("不知道"),绝不伪造 0("确实没日志")。
|
||||
func TestGetDailyVolumeHistoryReturnsAllDaysWithMissing(t *testing.T) {
|
||||
setupDailyVolumeDB(t)
|
||||
now := time.Now()
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.DailyVolumeSnapshots = []models.DailyVolumeSnapshot{
|
||||
{ID: 1, Date: dateKey(now), SourceID: 1, Bytes: 123, Partial: true},
|
||||
{ID: 2, Date: dateKey(now), SourceID: 2, Bytes: 999}, // 别的数据源,不应混进来
|
||||
}
|
||||
db.Unlock()
|
||||
|
||||
points := GetDailyVolumeHistory(1, 7)
|
||||
if len(points) != 7 {
|
||||
t.Fatalf("期望 7 个点,得到 %d", len(points))
|
||||
}
|
||||
today := points[len(points)-1]
|
||||
if today.Missing || today.Bytes != 123 || !today.Partial {
|
||||
t.Errorf("今天应命中快照:missing=%v bytes=%d partial=%v", today.Missing, today.Bytes, today.Partial)
|
||||
}
|
||||
for i := 0; i < len(points)-1; i++ {
|
||||
if !points[i].Missing || points[i].Bytes != 0 {
|
||||
t.Errorf("%s 应标记 Missing 且 bytes=0,得到 missing=%v bytes=%d",
|
||||
points[i].Date, points[i].Missing, points[i].Bytes)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package services
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"net"
|
||||
"regexp"
|
||||
"strconv"
|
||||
@@ -228,6 +229,13 @@ func (s *DeploymentService) List(page, pageSize int) ([]models.LokiDeployment, i
|
||||
}
|
||||
|
||||
func (s *DeploymentService) Update(id uint, updates map[string]interface{}) error {
|
||||
// 用户编辑部署 = 明确表达"这个部署我还在用",重新允许自动注册数据源。
|
||||
// (对应地,用户在数据源列表里删掉自动源后不会再被补回来,直到这里重置)
|
||||
defer func() {
|
||||
if d, err := s.Get(id); err == nil && d != nil {
|
||||
EnableAutoSourceFor(d.Name)
|
||||
}
|
||||
}()
|
||||
if value, exists := updates["retentionDays"]; exists {
|
||||
days, ok := value.(float64)
|
||||
if !ok || days < 1 || days != float64(int(days)) {
|
||||
@@ -311,22 +319,45 @@ func (s *DeploymentService) Update(id uint, updates map[string]interface{}) erro
|
||||
}
|
||||
|
||||
func (s *DeploymentService) Delete(id uint) error {
|
||||
// 部署删除会连带清理数据源,候选值失效
|
||||
defer InvalidateLabelCache()
|
||||
deployment, err := s.Get(id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if deployment.Mode == "host-docker" {
|
||||
if deployment.Status == "deploying" {
|
||||
return fmt.Errorf("部署正在进行中,请等待完成后删除")
|
||||
}
|
||||
if deployment.Status == "deploying" {
|
||||
return fmt.Errorf("部署正在进行中,请等待完成后删除")
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
// 先卸载该部署下的采集器:它们依赖这个 Loki 实例,留着会指向不存在的服务
|
||||
UninstallCollectorsForDeployment(ctx, id)
|
||||
|
||||
switch deployment.Mode {
|
||||
case "host-docker":
|
||||
if deployment.Status != "pending" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||
defer cancel()
|
||||
if err := deleteDockerLoki(ctx, deployment); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case "k8s-yaml":
|
||||
// 停止/重启走 CleanupLoki 时保留 PVC;这里是显式删除,连数据卷一起清理
|
||||
CleanupLoki(ctx, deployment)
|
||||
if deployment.Cluster != nil {
|
||||
ns := deployment.Namespace
|
||||
if ns == "" {
|
||||
ns = "log"
|
||||
}
|
||||
if err := deleteK8sResource(ctx, deployment.Cluster, "persistentvolumeclaim", ns, "loki-data"); err != nil {
|
||||
log.Printf("[deploy-delete] 清理 PVC %s/loki-data 失败: %v", ns, err)
|
||||
}
|
||||
}
|
||||
case "host-binary":
|
||||
CleanupLoki(ctx, deployment)
|
||||
}
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
removedName := ""
|
||||
|
||||
@@ -4,9 +4,6 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -19,7 +16,7 @@ import (
|
||||
// ResourceUsage 是页面上「资源」弹框的数据。
|
||||
//
|
||||
// 三种部署模式都能采集:
|
||||
// - k8s-yaml : StatefulSet resources + metrics-server(实时用量)+ 可选 Prometheus(峰值/限流)
|
||||
// - k8s-yaml : StatefulSet resources + metrics-server(实时用量)
|
||||
// - host-docker : docker stats / docker inspect(容器限额未设则按宿主机内存做参考)
|
||||
// - host-binary : ps 采集进程 CPU%/RSS(无 cgroup 限额)
|
||||
//
|
||||
@@ -45,11 +42,6 @@ type ResourceUsage struct {
|
||||
// 限额是否真的配置了(docker 未加 --memory 时为 false,页面不画百分比)
|
||||
LimitsConfigured bool `json:"limitsConfigured"`
|
||||
|
||||
// 可选(需 Prometheus)
|
||||
CPUPeakMilli *int64 `json:"cpuPeakMilli,omitempty"`
|
||||
MemPeakBytes *int64 `json:"memPeakBytes,omitempty"`
|
||||
ThrottlePercent *float64 `json:"throttlePercent,omitempty"`
|
||||
|
||||
// 采集受限说明(如缺 metrics.k8s.io 权限、未装 metrics-server、docker 无历史数据)
|
||||
MetricsNote string `json:"metricsNote,omitempty"`
|
||||
|
||||
@@ -148,7 +140,6 @@ func (s *DeploymentService) k8sUsage(ctx context.Context, d *models.LokiDeployme
|
||||
usage.MetricsNote = "无法解析 metrics-server 返回"
|
||||
}
|
||||
|
||||
s.applyPrometheus(usage)
|
||||
usage.buildHints()
|
||||
return usage, nil
|
||||
}
|
||||
@@ -167,29 +158,6 @@ func explainMetricsError(err error) string {
|
||||
}
|
||||
}
|
||||
|
||||
// applyPrometheus 补 24h 峰值与 CPU 限流率(未配置 Prometheus 地址则跳过)。
|
||||
func (s *DeploymentService) applyPrometheus(u *ResourceUsage) {
|
||||
base := strings.TrimSpace(os.Getenv("LOKI_USAGE_PROMETHEUS_URL"))
|
||||
if base == "" || u.Pod == "" {
|
||||
if u.MetricsNote == "" {
|
||||
u.MetricsNote = "未配置 Prometheus,暂无 24h 峰值与限流率"
|
||||
}
|
||||
return
|
||||
}
|
||||
sel := fmt.Sprintf(`pod=%q,namespace=%q,container="loki"`, u.Pod, u.Namespace)
|
||||
if v, ok := promQueryValue(base, fmt.Sprintf(`max_over_time(sum(rate(container_cpu_usage_seconds_total{%s}[5m]))[24h:1m])`, sel)); ok {
|
||||
milli := int64(v * 1000)
|
||||
u.CPUPeakMilli = &milli
|
||||
}
|
||||
if v, ok := promQueryValue(base, fmt.Sprintf(`max_over_time(sum(container_memory_working_set_bytes{%s})[24h:1m])`, sel)); ok {
|
||||
b := int64(v)
|
||||
u.MemPeakBytes = &b
|
||||
}
|
||||
if v, ok := promQueryValue(base, fmt.Sprintf(`100 * sum(rate(container_cpu_cfs_throttled_periods_total{pod=%q,namespace=%q}[15m])) / sum(rate(container_cpu_cfs_periods_total{pod=%q,namespace=%q}[15m]))`, u.Pod, u.Namespace, u.Pod, u.Namespace)); ok {
|
||||
u.ThrottlePercent = &v
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- host-docker ----------
|
||||
|
||||
func (s *DeploymentService) dockerUsage(ctx context.Context, d *models.LokiDeployment) (*ResourceUsage, error) {
|
||||
@@ -245,7 +213,7 @@ func (s *DeploymentService) dockerUsage(ctx context.Context, d *models.LokiDeplo
|
||||
usage.MetricsNote = "docker stats 读取失败:" + err.Error()
|
||||
}
|
||||
if usage.MetricsNote == "" {
|
||||
usage.MetricsNote = "Docker 部署无历史峰值;如需 24h 峰值请接入 cAdvisor/Prometheus"
|
||||
usage.MetricsNote = "未接入 Prometheus:暂无峰值/限流数据"
|
||||
}
|
||||
|
||||
usage.buildHints()
|
||||
@@ -287,14 +255,9 @@ func (s *DeploymentService) processUsage(ctx context.Context, d *models.LokiDepl
|
||||
// ---------- 共用:扩容建议 ----------
|
||||
|
||||
func (u *ResourceUsage) buildHints() {
|
||||
// 峰值需 Prometheus,平台不接入:扩容建议基于 metrics-server 实时值
|
||||
cpuRef := u.CPUCurrentMilli
|
||||
if u.CPUPeakMilli != nil && *u.CPUPeakMilli > cpuRef {
|
||||
cpuRef = *u.CPUPeakMilli
|
||||
}
|
||||
memRef := u.MemCurrentBytes
|
||||
if u.MemPeakBytes != nil && *u.MemPeakBytes > memRef {
|
||||
memRef = *u.MemPeakBytes
|
||||
}
|
||||
|
||||
var notes []string
|
||||
if u.CPULimitMilli > 0 {
|
||||
@@ -319,9 +282,6 @@ func (u *ResourceUsage) buildHints() {
|
||||
notes = append(notes, "内存偏高")
|
||||
}
|
||||
}
|
||||
if u.ThrottlePercent != nil && *u.ThrottlePercent >= 10 {
|
||||
notes = append(notes, fmt.Sprintf("CPU 限流 %.0f%%", *u.ThrottlePercent))
|
||||
}
|
||||
if !u.Ready {
|
||||
notes = append(notes, "实例未就绪")
|
||||
}
|
||||
@@ -337,40 +297,6 @@ func (u *ResourceUsage) buildHints() {
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- Prometheus ----------
|
||||
|
||||
// promQueryValue 查 Prometheus 瞬时值;未配置或失败返回 ok=false(页面降级)。
|
||||
func promQueryValue(base, expr string) (float64, bool) {
|
||||
client := &http.Client{Timeout: 8 * time.Second}
|
||||
u := strings.TrimSuffix(base, "/") + "/api/v1/query?query=" + url.QueryEscape(expr)
|
||||
resp, err := client.Get(u)
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 400 {
|
||||
return 0, false
|
||||
}
|
||||
var payload struct {
|
||||
Data struct {
|
||||
Result []struct {
|
||||
Value [2]interface{} `json:"value"`
|
||||
} `json:"result"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if json.NewDecoder(resp.Body).Decode(&payload) != nil || len(payload.Data.Result) == 0 {
|
||||
return 0, false
|
||||
}
|
||||
if len(payload.Data.Result[0].Value) < 2 {
|
||||
return 0, false
|
||||
}
|
||||
raw, ok := payload.Data.Result[0].Value[1].(string)
|
||||
if !ok {
|
||||
return 0, false
|
||||
}
|
||||
return parseFloatSafe(raw), true
|
||||
}
|
||||
|
||||
// ---------- 单位解析 ----------
|
||||
|
||||
// parseQuantityMilli 解析 CPU 用量/限额为毫核。
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
package services
|
||||
|
||||
// 用户手册文档站:docs/user-guide/*.md 按 manifest.json 分组,
|
||||
// 服务端 goldmark 渲染 + 三栏布局(侧栏/内容/本页目录)+ 顶栏搜索。
|
||||
// 布局与交互参考 opsxapi/opsxshell 文档站,换 LokiStack 品牌绿。
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"html/template"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
htmlRenderer "github.com/yuin/goldmark/renderer/html"
|
||||
)
|
||||
|
||||
// DocsManifest docs/user-guide/manifest.json
|
||||
type DocsManifest struct {
|
||||
ProductVersion string `json:"productVersion"`
|
||||
Groups []struct {
|
||||
Name string `json:"name"`
|
||||
Pages []struct {
|
||||
File string `json:"file"`
|
||||
Slug string `json:"slug"`
|
||||
Title string `json:"title"`
|
||||
Updated string `json:"updated"`
|
||||
} `json:"pages"`
|
||||
} `json:"groups"`
|
||||
}
|
||||
|
||||
// DocsPage 渲染一页所需的数据
|
||||
type DocsPage struct {
|
||||
Title string
|
||||
Group string
|
||||
Updated string
|
||||
Slug string
|
||||
Content template.HTML
|
||||
TOC []DocsTOCItem
|
||||
Prev *DocsNav
|
||||
Next *DocsNav
|
||||
}
|
||||
|
||||
type DocsTOCItem struct {
|
||||
Level int
|
||||
ID string
|
||||
Text string
|
||||
}
|
||||
|
||||
type DocsNav struct {
|
||||
Slug string
|
||||
Title string
|
||||
}
|
||||
|
||||
var (
|
||||
docsManifestCache *DocsManifest
|
||||
docsBasePath string
|
||||
docsMD = goldmark.New(goldmark.WithExtensions(extension.Table), goldmark.WithParserOptions(parser.WithAutoHeadingID()), goldmark.WithRendererOptions(htmlRenderer.WithUnsafe()))
|
||||
docsTOCPattern = regexp.MustCompile(`<h([23]) id="([^"]+)"[^>]*>([^<]+)</h[23]>`)
|
||||
)
|
||||
|
||||
// DocsBaseDir 定位 docs 根(含 user-guide/manifest.json);找不到返回空
|
||||
func DocsBaseDir() string {
|
||||
if docsBasePath != "" {
|
||||
return docsBasePath
|
||||
}
|
||||
for _, p := range []string{"docs", "../docs", "../../docs"} {
|
||||
abs, err := filepath.Abs(p)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(abs, "user-guide", "manifest.json")); err == nil {
|
||||
docsBasePath = abs
|
||||
return abs
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// DocsLoadManifest 解析(并缓存)manifest
|
||||
func DocsLoadManifest() (*DocsManifest, error) {
|
||||
if docsManifestCache != nil {
|
||||
return docsManifestCache, nil
|
||||
}
|
||||
base := DocsBaseDir()
|
||||
if base == "" {
|
||||
return nil, fmt.Errorf("docs 目录不存在")
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(base, "user-guide", "manifest.json"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m DocsManifest
|
||||
if err := json.Unmarshal(data, &m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
docsManifestCache = &m
|
||||
return docsManifestCache, nil
|
||||
}
|
||||
|
||||
// DocsFlat 顺序展开全部页面(slug 定位与上下页)
|
||||
func DocsFlat(m *DocsManifest) []struct {
|
||||
Group string
|
||||
File string
|
||||
Slug string
|
||||
Title string
|
||||
Updated string
|
||||
} {
|
||||
var out []struct {
|
||||
Group string
|
||||
File string
|
||||
Slug string
|
||||
Title string
|
||||
Updated string
|
||||
}
|
||||
for _, g := range m.Groups {
|
||||
for _, p := range g.Pages {
|
||||
out = append(out, struct {
|
||||
Group string
|
||||
File string
|
||||
Slug string
|
||||
Title string
|
||||
Updated string
|
||||
}{g.Name, p.File, p.Slug, p.Title, p.Updated})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// DocsRenderPage 渲染指定 slug;含 TOC 与上一页/下一页
|
||||
func DocsRenderPage(slug string) (*DocsPage, *DocsManifest, error) {
|
||||
m, err := DocsLoadManifest()
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
flat := DocsFlat(m)
|
||||
idx := -1
|
||||
for i, p := range flat {
|
||||
if p.Slug == slug {
|
||||
idx = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if idx < 0 {
|
||||
return nil, nil, fmt.Errorf("页面不存在")
|
||||
}
|
||||
cur := flat[idx]
|
||||
data, err := os.ReadFile(filepath.Join(DocsBaseDir(), "user-guide", cur.File))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
var sb strings.Builder
|
||||
if err := docsMD.Convert(data, &sb); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
html := sb.String()
|
||||
// 页面头已渲染标题,剥掉正文首个 H1 避免重复
|
||||
if i := strings.Index(html, "<h1"); i >= 0 {
|
||||
if j := strings.Index(html[i:], "</h1>"); j >= 0 {
|
||||
html = html[:i] + html[i+j+len("</h1>"):]
|
||||
}
|
||||
}
|
||||
|
||||
page := &DocsPage{Title: cur.Title, Group: cur.Group, Updated: cur.Updated, Slug: cur.Slug, Content: template.HTML(html)}
|
||||
for _, m2 := range docsTOCPattern.FindAllStringSubmatch(html, -1) {
|
||||
page.TOC = append(page.TOC, DocsTOCItem{Level: int(m2[1][0] - '0'), ID: m2[2], Text: m2[3]})
|
||||
}
|
||||
if idx > 0 {
|
||||
page.Prev = &DocsNav{Slug: flat[idx-1].Slug, Title: flat[idx-1].Title}
|
||||
}
|
||||
if idx < len(flat)-1 {
|
||||
page.Next = &DocsNav{Slug: flat[idx+1].Slug, Title: flat[idx+1].Title}
|
||||
}
|
||||
return page, m, nil
|
||||
}
|
||||
|
||||
// DocsSearchData 顶栏搜索索引(标题 + 纯文本摘要)
|
||||
func DocsSearchData() ([]map[string]string, error) {
|
||||
m, err := DocsLoadManifest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []map[string]string
|
||||
for _, p := range DocsFlat(m) {
|
||||
data, err := os.ReadFile(filepath.Join(DocsBaseDir(), "user-guide", p.File))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
text := regexp.MustCompile(`<[^>]+>`).ReplaceAllString(string(data), " ")
|
||||
text = strings.Join(strings.Fields(text), " ")
|
||||
if len(text) > 4000 {
|
||||
text = text[:4000]
|
||||
}
|
||||
out = append(out, map[string]string{"slug": p.Slug, "title": p.Title, "group": p.Group, "text": text})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// DocsFirstSlug 首页重定向目标
|
||||
func DocsFirstSlug() string {
|
||||
m, err := DocsLoadManifest()
|
||||
if err != nil || len(m.Groups) == 0 || len(m.Groups[0].Pages) == 0 {
|
||||
return "readme"
|
||||
}
|
||||
return m.Groups[0].Pages[0].Slug
|
||||
}
|
||||
@@ -186,6 +186,16 @@ func (s *HostService) Delete(id uint) error {
|
||||
defer db.Unlock()
|
||||
for i, h := range db.Hosts {
|
||||
if h.ID == id {
|
||||
for _, d := range db.Deployments {
|
||||
if d.HostID != nil && *d.HostID == id {
|
||||
return fmt.Errorf("仍有部署使用该主机,请先删除部署")
|
||||
}
|
||||
}
|
||||
for _, c := range db.Collectors {
|
||||
if c.HostID != nil && *c.HostID == id {
|
||||
return fmt.Errorf("仍有采集器使用该主机,请先删除采集器")
|
||||
}
|
||||
}
|
||||
db.Hosts = append(db.Hosts[:i], db.Hosts[i+1:]...)
|
||||
db.RequestSave()
|
||||
return nil
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"lokistack/config"
|
||||
)
|
||||
|
||||
// instance_id 是部署实例指纹:持久化到 DATA_DIR/instance.id,不含邮箱/姓名等 PII。
|
||||
// 对齐 OpsxShell device_id 语义,供 opsx-licence 按实例占座。
|
||||
|
||||
var (
|
||||
instanceIDOnce sync.Once
|
||||
instanceIDVal string
|
||||
)
|
||||
|
||||
// GetOrCreateInstanceID 返回稳定实例 ID(机器/安装指纹)。
|
||||
func GetOrCreateInstanceID() string {
|
||||
instanceIDOnce.Do(func() {
|
||||
cfg := config.GetConfig()
|
||||
path := filepath.Join(cfg.DataDir, "instance.id")
|
||||
if raw, err := os.ReadFile(path); err == nil {
|
||||
id := strings.TrimSpace(string(raw))
|
||||
if id != "" {
|
||||
instanceIDVal = id
|
||||
return
|
||||
}
|
||||
}
|
||||
buf := make([]byte, 16)
|
||||
if _, err := rand.Read(buf); err != nil {
|
||||
instanceIDVal = "lokistack-fallback-instance"
|
||||
return
|
||||
}
|
||||
id := "ls-" + hex.EncodeToString(buf)
|
||||
_ = os.MkdirAll(cfg.DataDir, 0o700)
|
||||
_ = os.WriteFile(path, []byte(id+"\n"), 0o600)
|
||||
instanceIDVal = id
|
||||
})
|
||||
return instanceIDVal
|
||||
}
|
||||
@@ -0,0 +1,355 @@
|
||||
package services
|
||||
|
||||
// opsx-licence 客户端:activate / verify / heartbeat。
|
||||
// 默认 Base = https://licence.rlwj.cn/api;product_code = lokistack。
|
||||
// 响应若带 Ed25519 sig 则验签(公钥与 OpsxShell 同源)。
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/ed25519"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/config"
|
||||
)
|
||||
|
||||
const (
|
||||
licenceProductCode = "lokistack"
|
||||
licenceHTTPTimeout = 15 * time.Second
|
||||
// 与 OpsxShell / opsx-licence OFFLINE_LICENSE_SEED 派生公钥一致
|
||||
offlineLicensePublicKeyHex = "108375d485c58a1a738f93c2cbb838f76331de981ee5da13d90a1f280672131d"
|
||||
offlineToleranceSecs = int64(30 * 86400)
|
||||
weeklyVerifySecs = int64(7 * 86400)
|
||||
hmacDeprecationNotice = "当前为旧版本地 HMAC 许可码,将逐步停用。请联系发卡方换发 LIC- / LICOFF-(opsx-licence)授权码;迁移窗口内旧码仍有效,不会静默失效。"
|
||||
)
|
||||
|
||||
type licenceRemoteInfo struct {
|
||||
Valid bool
|
||||
LicenseKey string
|
||||
ProductCode string
|
||||
Features []string
|
||||
Status string
|
||||
ExpiresAtUnix int64
|
||||
MaxDevices int
|
||||
PlanType string
|
||||
InGrace bool
|
||||
GraceDaysLeft int
|
||||
ServerTime int64
|
||||
ActivationToken string
|
||||
Message string
|
||||
}
|
||||
|
||||
func licenceAPIBases() []string {
|
||||
primary := strings.TrimRight(config.GetConfig().LicenseAPIBase, "/")
|
||||
bases := []string{primary}
|
||||
// 仅当主节点不是 l123 时把 l123 作备灾(网络失败才切)
|
||||
backup := "https://l123.91ai.io/api"
|
||||
if primary != backup && !strings.Contains(primary, "l123.91ai.io") {
|
||||
bases = append(bases, backup)
|
||||
}
|
||||
return bases
|
||||
}
|
||||
|
||||
func isOpsxLicenseKey(key string) bool {
|
||||
k := strings.TrimSpace(key)
|
||||
return strings.HasPrefix(k, "LIC-") || strings.HasPrefix(k, "LICOFF-")
|
||||
}
|
||||
|
||||
func isOfflineLicenseKey(key string) bool {
|
||||
return strings.HasPrefix(strings.TrimSpace(key), "LICOFF-")
|
||||
}
|
||||
|
||||
func licenceHTTPClient() *http.Client {
|
||||
return &http.Client{Timeout: licenceHTTPTimeout}
|
||||
}
|
||||
|
||||
func postLicenceJSON(path string, body any) (map[string]any, error) {
|
||||
payload, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var lastErr error
|
||||
for _, base := range licenceAPIBases() {
|
||||
url := base + path
|
||||
req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", "LokiStack/1")
|
||||
resp, err := licenceHTTPClient().Do(req)
|
||||
if err != nil {
|
||||
lastErr = fmt.Errorf("授权服务不可达: %w", err)
|
||||
continue
|
||||
}
|
||||
raw, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
_ = resp.Body.Close()
|
||||
if readErr != nil {
|
||||
lastErr = readErr
|
||||
continue
|
||||
}
|
||||
var envelope map[string]any
|
||||
if err := json.Unmarshal(raw, &envelope); err != nil {
|
||||
lastErr = fmt.Errorf("授权服务响应无法解析 (%s)", resp.Status)
|
||||
continue
|
||||
}
|
||||
return envelope, nil
|
||||
}
|
||||
if lastErr == nil {
|
||||
lastErr = fmt.Errorf("无可用授权节点")
|
||||
}
|
||||
return nil, lastErr
|
||||
}
|
||||
|
||||
func envelopeCode(m map[string]any) int {
|
||||
switch v := m["code"].(type) {
|
||||
case float64:
|
||||
return int(v)
|
||||
case int:
|
||||
return v
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
func envelopeMessage(m map[string]any) string {
|
||||
if s, ok := m["message"].(string); ok {
|
||||
return s
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func envelopeData(m map[string]any) map[string]any {
|
||||
if d, ok := m["data"].(map[string]any); ok {
|
||||
return d
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func jsonString(m map[string]any, keys ...string) string {
|
||||
for _, k := range keys {
|
||||
if s, ok := m[k].(string); ok && s != "" {
|
||||
return s
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func jsonInt(m map[string]any, keys ...string) int {
|
||||
for _, k := range keys {
|
||||
switch v := m[k].(type) {
|
||||
case float64:
|
||||
return int(v)
|
||||
case int:
|
||||
return v
|
||||
case json.Number:
|
||||
n, _ := v.Int64()
|
||||
return int(n)
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func jsonInt64(m map[string]any, keys ...string) int64 {
|
||||
for _, k := range keys {
|
||||
switch v := m[k].(type) {
|
||||
case float64:
|
||||
return int64(v)
|
||||
case int64:
|
||||
return v
|
||||
case int:
|
||||
return int64(v)
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func jsonBool(m map[string]any, keys ...string) bool {
|
||||
for _, k := range keys {
|
||||
if b, ok := m[k].(bool); ok {
|
||||
return b
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func parseRemoteInfo(data map[string]any) licenceRemoteInfo {
|
||||
features := []string{}
|
||||
if arr, ok := data["features"].([]any); ok {
|
||||
for _, item := range arr {
|
||||
if s, ok := item.(string); ok {
|
||||
features = append(features, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
return licenceRemoteInfo{
|
||||
Valid: jsonBool(data, "valid"),
|
||||
LicenseKey: jsonString(data, "license_key", "licenseKey"),
|
||||
ProductCode: jsonString(data, "product_code", "productCode"),
|
||||
Features: features,
|
||||
Status: jsonString(data, "status"),
|
||||
ExpiresAtUnix: jsonInt64(data, "expires_at_unix", "expiresAtUnix"),
|
||||
MaxDevices: jsonInt(data, "max_devices", "maxDevices"),
|
||||
PlanType: jsonString(data, "plan_type", "planType"),
|
||||
InGrace: jsonBool(data, "in_grace", "inGrace"),
|
||||
GraceDaysLeft: jsonInt(data, "grace_days_left", "graceDaysLeft"),
|
||||
ServerTime: jsonInt64(data, "server_time", "serverTime"),
|
||||
ActivationToken: jsonString(data, "activation_token", "activationToken"),
|
||||
Message: jsonString(data, "message"),
|
||||
}
|
||||
}
|
||||
|
||||
func verifyLicenceResponseSig(licenseKey string, data map[string]any) error {
|
||||
sigHex := jsonString(data, "sig")
|
||||
if sigHex == "" {
|
||||
// 开发环境可能未配 OFFLINE_LICENSE_SEED;生产应有签名
|
||||
return nil
|
||||
}
|
||||
pub, err := hex.DecodeString(offlineLicensePublicKeyHex)
|
||||
if err != nil || len(pub) != ed25519.PublicKeySize {
|
||||
return fmt.Errorf("离线验签公钥配置错误")
|
||||
}
|
||||
sig, err := hex.DecodeString(sigHex)
|
||||
if err != nil {
|
||||
return fmt.Errorf("授权响应签名无效")
|
||||
}
|
||||
valid := jsonBool(data, "valid")
|
||||
status := jsonString(data, "status")
|
||||
expiresUnix := jsonInt64(data, "expires_at_unix", "expiresAtUnix")
|
||||
serverTime := jsonInt64(data, "server_time", "serverTime")
|
||||
inGrace := jsonBool(data, "in_grace", "inGrace")
|
||||
maxDevices := jsonInt(data, "max_devices", "maxDevices")
|
||||
planType := jsonString(data, "plan_type", "planType")
|
||||
msg := "v1|" + licenseKey + "|" +
|
||||
strconv.FormatBool(valid) + "|" +
|
||||
status + "|" +
|
||||
strconv.FormatInt(expiresUnix, 10) + "|" +
|
||||
strconv.FormatInt(serverTime, 10) + "|" +
|
||||
strconv.FormatBool(inGrace) + "|" +
|
||||
strconv.Itoa(maxDevices) + "|" +
|
||||
planType
|
||||
if !ed25519.Verify(ed25519.PublicKey(pub), []byte(msg), sig) {
|
||||
return fmt.Errorf("授权响应签名校验失败(疑似假服务器)")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func remoteActivateLicence(licenseKey, deviceID string) (*licenceRemoteInfo, error) {
|
||||
body := map[string]string{
|
||||
"license_key": licenseKey,
|
||||
"device_id": deviceID,
|
||||
"device_name": "LokiStack",
|
||||
}
|
||||
env, err := postLicenceJSON("/v1/license/activate", body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if envelopeCode(env) != 0 {
|
||||
msg := envelopeMessage(env)
|
||||
if msg == "" {
|
||||
msg = "激活失败"
|
||||
}
|
||||
return nil, fmt.Errorf("%s", msg)
|
||||
}
|
||||
data := envelopeData(env)
|
||||
if err := verifyLicenceResponseSig(licenseKey, data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
info := parseRemoteInfo(data)
|
||||
if info.LicenseKey == "" {
|
||||
info.LicenseKey = licenseKey
|
||||
}
|
||||
if !info.Valid && info.Status == "" {
|
||||
// activate payload 可能不带 valid,视为成功
|
||||
info.Valid = true
|
||||
}
|
||||
return &info, nil
|
||||
}
|
||||
|
||||
func remoteVerifyLicence(licenseKey, deviceID string) (*licenceRemoteInfo, error) {
|
||||
body := map[string]string{
|
||||
"license_key": licenseKey,
|
||||
"device_id": deviceID,
|
||||
}
|
||||
env, err := postLicenceJSON("/v1/license/verify", body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("network:%w", err)
|
||||
}
|
||||
// verify 可能是裸对象或 envelope
|
||||
var data map[string]any
|
||||
if _, ok := env["valid"]; ok {
|
||||
data = env
|
||||
} else if envelopeCode(env) == 0 {
|
||||
data = envelopeData(env)
|
||||
} else if c := envelopeCode(env); c != 0 {
|
||||
msg := envelopeMessage(env)
|
||||
if msg == "" {
|
||||
msg = "验证失败"
|
||||
}
|
||||
return nil, fmt.Errorf("rejected:%s", msg)
|
||||
} else {
|
||||
data = env
|
||||
}
|
||||
info := parseRemoteInfo(data)
|
||||
if info.LicenseKey == "" {
|
||||
info.LicenseKey = licenseKey
|
||||
}
|
||||
if !info.Valid {
|
||||
msg := info.Message
|
||||
if msg == "" {
|
||||
msg = envelopeMessage(env)
|
||||
}
|
||||
if msg == "" {
|
||||
msg = "许可证无效"
|
||||
}
|
||||
return nil, fmt.Errorf("rejected:%s", msg)
|
||||
}
|
||||
if err := verifyLicenceResponseSig(licenseKey, data); err != nil {
|
||||
return nil, fmt.Errorf("rejected:%v", err)
|
||||
}
|
||||
return &info, nil
|
||||
}
|
||||
|
||||
func remoteHeartbeatLicence(licenseKey, deviceID string) error {
|
||||
body := map[string]string{
|
||||
"license_key": licenseKey,
|
||||
"device_id": deviceID,
|
||||
}
|
||||
env, err := postLicenceJSON("/v1/license/heartbeat", body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if envelopeCode(env) != 0 {
|
||||
return fmt.Errorf("%s", envelopeMessage(env))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func remoteDeactivateLicence(licenseKey, deviceID, activationToken string) error {
|
||||
body := map[string]string{
|
||||
"license_key": licenseKey,
|
||||
"device_id": deviceID,
|
||||
}
|
||||
if strings.TrimSpace(activationToken) != "" {
|
||||
body["activation_token"] = strings.TrimSpace(activationToken)
|
||||
}
|
||||
env, err := postLicenceJSON("/v1/license/deactivate", body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if envelopeCode(env) != 0 {
|
||||
msg := envelopeMessage(env)
|
||||
if msg == "" {
|
||||
msg = "解绑失败"
|
||||
}
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package services
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestIsOpsxLicenseKey(t *testing.T) {
|
||||
cases := []struct {
|
||||
key string
|
||||
want bool
|
||||
}{
|
||||
{"LIC-ABC", true},
|
||||
{"LICOFF-1.xxx", true},
|
||||
{" LIC-x ", true},
|
||||
{"not-a-key", false},
|
||||
{"", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := isOpsxLicenseKey(c.key); got != c.want {
|
||||
t.Fatalf("%q: got %v want %v", c.key, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeDoesNotBreakHMAC(t *testing.T) {
|
||||
key, err := GenerateLicenseKey("pro", "personal", "mig-test", 1, 1, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if isOpsxLicenseKey(key) {
|
||||
t.Fatal("HMAC key must not be classified as opsx")
|
||||
}
|
||||
lic, err := VerifyLicenseKey(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if lic.Source != "hmac" || lic.DeprecationNotice == "" {
|
||||
t.Fatalf("HMAC path must set source+deprecation: %+v", lic)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeLicenseAPIBaseViaConfigHelper(t *testing.T) {
|
||||
// smoke: constants present
|
||||
if licenceProductCode != "lokistack" {
|
||||
t.Fatal(licenceProductCode)
|
||||
}
|
||||
if weeklyVerifySecs != 7*86400 {
|
||||
t.Fatal(weeklyVerifySecs)
|
||||
}
|
||||
if offlineToleranceSecs != 30*86400 {
|
||||
t.Fatal(offlineToleranceSecs)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"crypto/ed25519"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type offlineClaims struct {
|
||||
V int `json:"v"`
|
||||
ProductCode string `json:"pid"`
|
||||
DeviceID string `json:"did"`
|
||||
Email string `json:"email,omitempty"`
|
||||
PlanType string `json:"pt"`
|
||||
Iat int64 `json:"iat"`
|
||||
Exp int64 `json:"exp"`
|
||||
MaxDevices int `json:"md"`
|
||||
}
|
||||
|
||||
func verifyOfflineSerial(serial, deviceID string) (*offlineClaims, error) {
|
||||
trimmed := strings.TrimSpace(serial)
|
||||
body, ok := strings.CutPrefix(trimmed, "LICOFF-1.")
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("不是合法的离线序列号(需 LICOFF-1. 前缀)")
|
||||
}
|
||||
idx := strings.LastIndex(body, ".")
|
||||
if idx <= 0 {
|
||||
return nil, fmt.Errorf("离线序列号格式错误")
|
||||
}
|
||||
payloadB64, sigB64 := body[:idx], body[idx+1:]
|
||||
sigBytes, err := base64.RawURLEncoding.DecodeString(sigB64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("离线序列号签名无法解码")
|
||||
}
|
||||
pub, err := hex.DecodeString(offlineLicensePublicKeyHex)
|
||||
if err != nil || len(pub) != ed25519.PublicKeySize {
|
||||
return nil, fmt.Errorf("离线验签公钥配置错误")
|
||||
}
|
||||
if !ed25519.Verify(ed25519.PublicKey(pub), []byte(payloadB64), sigBytes) {
|
||||
return nil, fmt.Errorf("离线序列号签名校验失败")
|
||||
}
|
||||
payload, err := base64.RawURLEncoding.DecodeString(payloadB64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("离线序列号载荷无法解码")
|
||||
}
|
||||
var claims offlineClaims
|
||||
if err := json.Unmarshal(payload, &claims); err != nil {
|
||||
return nil, fmt.Errorf("离线序列号载荷无法解析")
|
||||
}
|
||||
if claims.V != 0 && claims.V != 1 {
|
||||
return nil, fmt.Errorf("不支持的离线证版本")
|
||||
}
|
||||
if claims.ProductCode != licenceProductCode {
|
||||
return nil, fmt.Errorf("离线证产品不匹配(需要 %s)", licenceProductCode)
|
||||
}
|
||||
if claims.DeviceID != deviceID {
|
||||
return nil, fmt.Errorf("离线证与当前实例不匹配")
|
||||
}
|
||||
if claims.PlanType != "trial" && claims.PlanType != "yearly" {
|
||||
return nil, fmt.Errorf("离线证计划无效(仅体验/包年)")
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
if claims.Exp <= now {
|
||||
return nil, fmt.Errorf("离线许可证已过期")
|
||||
}
|
||||
if claims.Exp-claims.Iat > 365*86400+3600 {
|
||||
return nil, fmt.Errorf("离线许可证有效期非法")
|
||||
}
|
||||
return &claims, nil
|
||||
}
|
||||
@@ -0,0 +1,472 @@
|
||||
package services
|
||||
|
||||
// Pro 授权双路径:
|
||||
// 1) opsx-licence: LIC- / LICOFF- → activate|verify|heartbeat(默认 https://licence.rlwj.cn/api)
|
||||
// 2) 旧 HMAC 本地码: 迁移窗口内仍接受,并附带 DeprecationNotice(禁止静默失效)
|
||||
// 发行旧码: `lokistack license --plan=pro ...`(见 licenseIssueKey);新码由授权后台签发。
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
// licenseIssueKey 发行密钥(正式发行用 -ldflags 覆盖;默认值为开发/试用签名)
|
||||
var licenseIssueKey = "lokistack-issue-key-override-in-release-build"
|
||||
|
||||
// Pro 功能清单(免费版可用功能见 router 的 RequirePro 挂载点)
|
||||
var proFeatures = []string{"log_analysis", "log_alerts", "ai_ops", "audit_log"}
|
||||
|
||||
type licensePayload struct {
|
||||
Plan string `json:"plan"`
|
||||
Edition string `json:"edition"`
|
||||
Seats int `json:"seats"`
|
||||
Instances int `json:"instances"`
|
||||
Customer string `json:"customer"`
|
||||
ExpiresAt int64 `json:"expiresAt"`
|
||||
IssuedAt int64 `json:"issuedAt"`
|
||||
}
|
||||
|
||||
// GenerateLicenseKey 签发许可码(发行方工具;days=0 表示永久) — 旧 HMAC 路径,迁移期保留。
|
||||
func GenerateLicenseKey(plan, edition, customer string, seats, instances, days int) (string, error) {
|
||||
if plan != "pro" {
|
||||
return "", fmt.Errorf("许可码只用于签发 pro;免费版无需许可")
|
||||
}
|
||||
if edition != "personal" && edition != "team" {
|
||||
return "", fmt.Errorf("edition 仅支持 personal / team")
|
||||
}
|
||||
p := licensePayload{
|
||||
Plan: plan, Edition: edition, Seats: seats, Instances: instances,
|
||||
Customer: customer, IssuedAt: time.Now().Unix(),
|
||||
}
|
||||
if days > 0 {
|
||||
p.ExpiresAt = time.Now().AddDate(0, 0, days).Unix()
|
||||
}
|
||||
body, _ := json.Marshal(p)
|
||||
mac := hmac.New(sha256.New, []byte(licenseIssueKey))
|
||||
mac.Write(body)
|
||||
sig := mac.Sum(nil)
|
||||
return base64.RawURLEncoding.EncodeToString(append(sig[:16], body...)), nil
|
||||
}
|
||||
|
||||
// VerifyLicenseKey 校验旧 HMAC 许可码,返回许可内容;伪造/篡改/过期均报错
|
||||
func VerifyLicenseKey(key string) (*models.License, error) {
|
||||
raw, err := base64.RawURLEncoding.DecodeString(strings.TrimSpace(key))
|
||||
if err != nil || len(raw) <= 16 {
|
||||
return nil, fmt.Errorf("许可码格式无效")
|
||||
}
|
||||
sig, body := raw[:16], raw[16:]
|
||||
mac := hmac.New(sha256.New, []byte(licenseIssueKey))
|
||||
mac.Write(body)
|
||||
if !hmac.Equal(sig, mac.Sum(nil)[:16]) {
|
||||
return nil, fmt.Errorf("许可码签名校验失败(伪造或被篡改)")
|
||||
}
|
||||
var p licensePayload
|
||||
if err := json.Unmarshal(body, &p); err != nil {
|
||||
return nil, fmt.Errorf("许可码内容无效")
|
||||
}
|
||||
if p.Plan != "pro" {
|
||||
return nil, fmt.Errorf("许可码不是 Pro 版")
|
||||
}
|
||||
if p.ExpiresAt > 0 && p.ExpiresAt < time.Now().Unix() {
|
||||
return nil, fmt.Errorf("许可码已过期")
|
||||
}
|
||||
return &models.License{
|
||||
Plan: p.Plan, Edition: p.Edition, Seats: p.Seats, Instances: p.Instances,
|
||||
Customer: p.Customer, ExpiresAt: p.ExpiresAt,
|
||||
IssuedAt: time.Unix(p.IssuedAt, 0), ActivatedAt: time.Now(),
|
||||
Source: "hmac", DeprecationNotice: hmacDeprecationNotice,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func editionFromPlanType(planType string, maxDevices int) string {
|
||||
if planType == "trial" || maxDevices <= 1 {
|
||||
return "personal"
|
||||
}
|
||||
return "team"
|
||||
}
|
||||
|
||||
func applyRemoteInfo(lic *models.License, key string, instanceID string, info *licenceRemoteInfo) {
|
||||
lic.Plan = "pro"
|
||||
lic.Source = "opsx"
|
||||
lic.LicenseKey = key
|
||||
lic.InstanceID = instanceID
|
||||
lic.PlanType = info.PlanType
|
||||
lic.MaxDevices = info.MaxDevices
|
||||
if info.MaxDevices > 0 {
|
||||
lic.Instances = info.MaxDevices
|
||||
}
|
||||
lic.Edition = editionFromPlanType(info.PlanType, info.MaxDevices)
|
||||
lic.ExpiresAt = info.ExpiresAtUnix
|
||||
lic.InGrace = info.InGrace
|
||||
lic.GraceDaysLeft = info.GraceDaysLeft
|
||||
lic.LastVerifiedAt = time.Now().Unix()
|
||||
if info.ServerTime > 0 {
|
||||
lic.LastServerTime = info.ServerTime
|
||||
}
|
||||
if info.ActivationToken != "" {
|
||||
lic.ActivationToken = info.ActivationToken
|
||||
}
|
||||
lic.DeprecationNotice = ""
|
||||
lic.InvalidReason = ""
|
||||
if lic.ActivatedAt.IsZero() {
|
||||
lic.ActivatedAt = time.Now()
|
||||
}
|
||||
if lic.IssuedAt.IsZero() {
|
||||
lic.IssuedAt = time.Now()
|
||||
}
|
||||
}
|
||||
|
||||
// ActivateLicense 激活: LIC-/LICOFF- 走 opsx-licence;其余走旧 HMAC(带迁移提示)。
|
||||
func ActivateLicense(key string) (*models.License, error) {
|
||||
key = strings.TrimSpace(key)
|
||||
if key == "" {
|
||||
return nil, fmt.Errorf("请输入许可码")
|
||||
}
|
||||
instanceID := GetOrCreateInstanceID()
|
||||
|
||||
if isOfflineLicenseKey(key) {
|
||||
claims, err := verifyOfflineSerial(key, instanceID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lic := &models.License{
|
||||
Plan: "pro", Source: "opsx", LicenseKey: key, InstanceID: instanceID,
|
||||
PlanType: claims.PlanType, MaxDevices: claims.MaxDevices, Instances: claims.MaxDevices,
|
||||
Edition: editionFromPlanType(claims.PlanType, claims.MaxDevices),
|
||||
ExpiresAt: claims.Exp, IssuedAt: time.Unix(claims.Iat, 0), ActivatedAt: time.Now(),
|
||||
LastVerifiedAt: time.Now().Unix(), Customer: claims.Email, InvalidReason: "",
|
||||
}
|
||||
return persistLicense(lic)
|
||||
}
|
||||
|
||||
if isOpsxLicenseKey(key) {
|
||||
info, err := remoteActivateLicence(key, instanceID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// 激活后再 verify 一次拿齐 expires/plan
|
||||
if v, vErr := remoteVerifyLicence(key, instanceID); vErr == nil {
|
||||
if v.ExpiresAtUnix > 0 {
|
||||
info.ExpiresAtUnix = v.ExpiresAtUnix
|
||||
}
|
||||
if v.PlanType != "" {
|
||||
info.PlanType = v.PlanType
|
||||
}
|
||||
if v.MaxDevices > 0 {
|
||||
info.MaxDevices = v.MaxDevices
|
||||
}
|
||||
info.InGrace = v.InGrace
|
||||
info.GraceDaysLeft = v.GraceDaysLeft
|
||||
if v.ServerTime > 0 {
|
||||
info.ServerTime = v.ServerTime
|
||||
}
|
||||
}
|
||||
lic := &models.License{ActivatedAt: time.Now(), IssuedAt: time.Now()}
|
||||
applyRemoteInfo(lic, key, instanceID, info)
|
||||
_ = remoteHeartbeatLicence(key, instanceID) // best-effort
|
||||
return persistLicense(lic)
|
||||
}
|
||||
|
||||
// 旧 HMAC 路径:迁移窗口内保留
|
||||
lic, err := VerifyLicenseKey(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lic.Source = "hmac"
|
||||
lic.DeprecationNotice = hmacDeprecationNotice
|
||||
lic.InvalidReason = ""
|
||||
lic.InstanceID = instanceID
|
||||
lic.LicenseKey = key
|
||||
lic.LastVerifiedAt = time.Now().Unix()
|
||||
return persistLicense(lic)
|
||||
}
|
||||
|
||||
// DeactivateResult 退出授权结果。OfflineLocalOnly 表示仅解本机(LICOFF-),后台名额可能仍占。
|
||||
type DeactivateResult struct {
|
||||
OfflineLocalOnly bool `json:"offlineLocalOnly"`
|
||||
Message string `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
// DeactivateLicense 用户主动退出授权 / 释放本实例。
|
||||
// - 在线 LIC-:先 POST /v1/license/deactivate(带 activation_token),成功才清本地;失败保留现状。
|
||||
// - 离线 LICOFF-:只解本机绑定,并提示后台名额可能仍占。
|
||||
// - 旧 HMAC:仅清本地(无远端座位)。
|
||||
// 禁止在校验失败路径静默清码;本函数仅用户确认后调用。
|
||||
func DeactivateLicense() (*DeactivateResult, error) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return nil, fmt.Errorf("数据库未就绪")
|
||||
}
|
||||
db.RLock()
|
||||
lic := db.License
|
||||
db.RUnlock()
|
||||
if lic == nil {
|
||||
return nil, fmt.Errorf("当前未激活授权")
|
||||
}
|
||||
// 允许对仍存证的 Pro(含宽限)或刚降级但仍占远端席位的 opsx 证做主动释放
|
||||
key := strings.TrimSpace(lic.LicenseKey)
|
||||
if lic.Plan != "pro" && key == "" {
|
||||
return nil, fmt.Errorf("当前未激活授权")
|
||||
}
|
||||
|
||||
token := lic.ActivationToken
|
||||
instanceID := lic.InstanceID
|
||||
if instanceID == "" {
|
||||
instanceID = GetOrCreateInstanceID()
|
||||
}
|
||||
|
||||
result := &DeactivateResult{}
|
||||
|
||||
if key != "" && isOfflineLicenseKey(key) {
|
||||
clearStoredLicense()
|
||||
result.OfflineLocalOnly = true
|
||||
result.Message = "已解除本机离线授权。后台名额可能仍占用,如需释放请联系运营处理。"
|
||||
return result, nil
|
||||
}
|
||||
|
||||
if key != "" && isOpsxLicenseKey(key) {
|
||||
if err := remoteDeactivateLicence(key, instanceID, token); err != nil {
|
||||
// 授权服务不返回 activation_token(激活响应无该字段),本机拿不到解绑凭证时
|
||||
// 远端必然拒绝——不能把用户锁死在"永远解不了绑"里。降级为仅清本机:
|
||||
// 本机回免费版(用户诉求达成),远端座位由服务端到期/管理员释放。
|
||||
clearStoredLicense()
|
||||
result.OfflineLocalOnly = true
|
||||
result.Message = "已退出授权,本机已回到免费版。远端席位未能自动释放(服务未下发解绑凭证),如需释放请到授权平台管理或联系运营。"
|
||||
return result, nil
|
||||
}
|
||||
clearStoredLicense()
|
||||
result.Message = "已退出授权并释放本实例"
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// HMAC 或无 key 的本地 Pro
|
||||
clearStoredLicense()
|
||||
result.Message = "已清除本机授权"
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func clearStoredLicense() {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
db.License = nil
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
}
|
||||
|
||||
func persistLicense(lic *models.License) (*models.License, error) {
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.License = lic
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
return lic, nil
|
||||
}
|
||||
|
||||
// RefreshOpsxLicenseIfDue 周检:距上次验证超过 7 天则联网 verify;网络失败保留 30 天离线宽限。
|
||||
// 服务端明确拒绝则降级为 free 并写 InvalidReason;保留 LicenseKey(禁静默清码);旧 HMAC 不联网。
|
||||
func RefreshOpsxLicenseIfDue() {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.RLock()
|
||||
lic := db.License
|
||||
db.RUnlock()
|
||||
if lic == nil || lic.Source != "opsx" || lic.LicenseKey == "" {
|
||||
return
|
||||
}
|
||||
if isOfflineLicenseKey(lic.LicenseKey) {
|
||||
return // 离线证本地自验即可
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
if lic.LastVerifiedAt > 0 && now-lic.LastVerifiedAt < weeklyVerifySecs {
|
||||
return
|
||||
}
|
||||
instanceID := lic.InstanceID
|
||||
if instanceID == "" {
|
||||
instanceID = GetOrCreateInstanceID()
|
||||
}
|
||||
info, err := remoteVerifyLicence(lic.LicenseKey, instanceID)
|
||||
if err != nil {
|
||||
msg := err.Error()
|
||||
if strings.HasPrefix(msg, "rejected:") {
|
||||
// 明确拒绝:降级为 free,保留 LicenseKey(禁止静默清码)
|
||||
reason := classifyRejectReason(strings.TrimPrefix(msg, "rejected:"))
|
||||
db.Lock()
|
||||
if db.License != nil {
|
||||
db.License.Plan = "free"
|
||||
db.License.InvalidReason = reason
|
||||
db.RequestSave()
|
||||
}
|
||||
db.Unlock()
|
||||
return
|
||||
}
|
||||
// 网络失败:30 天离线容忍,由 CurrentLicense 判断
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
if db.License != nil {
|
||||
applyRemoteInfo(db.License, lic.LicenseKey, instanceID, info)
|
||||
db.RequestSave()
|
||||
}
|
||||
db.Unlock()
|
||||
_ = remoteHeartbeatLicence(lic.LicenseKey, instanceID)
|
||||
}
|
||||
|
||||
// StoredLicense 库内原始许可(含已降级/过期记录);勿与 CurrentLicense 混淆。
|
||||
func StoredLicense() *models.License {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return nil
|
||||
}
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
return db.License
|
||||
}
|
||||
|
||||
// classifyRejectReason 把授权服务拒绝文案映射为 UI 红条类别。
|
||||
func classifyRejectReason(msg string) string {
|
||||
lower := strings.ToLower(msg)
|
||||
over := (strings.Contains(lower, "device") && (strings.Contains(lower, "limit") || strings.Contains(lower, "full") || strings.Contains(lower, "exceed"))) ||
|
||||
strings.Contains(lower, "seat") ||
|
||||
(strings.Contains(lower, "instance") && (strings.Contains(lower, "limit") || strings.Contains(lower, "exceed") || strings.Contains(lower, "full"))) ||
|
||||
(strings.Contains(msg, "实例") && (strings.Contains(msg, "上限") || strings.Contains(msg, "满") || strings.Contains(msg, "超"))) ||
|
||||
(strings.Contains(msg, "席位") && (strings.Contains(msg, "满") || strings.Contains(msg, "超") || strings.Contains(msg, "上限"))) ||
|
||||
(strings.Contains(msg, "设备") && (strings.Contains(msg, "满") || strings.Contains(msg, "超") || strings.Contains(msg, "上限")))
|
||||
if over {
|
||||
return "over_instance"
|
||||
}
|
||||
if strings.Contains(lower, "expir") || strings.Contains(msg, "过期") {
|
||||
return "expired"
|
||||
}
|
||||
return "revoked"
|
||||
}
|
||||
|
||||
func deriveInvalidReason(lic *models.License) string {
|
||||
if lic == nil {
|
||||
return ""
|
||||
}
|
||||
if lic.InvalidReason != "" {
|
||||
return lic.InvalidReason
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
if lic.ExpiresAt > 0 && lic.ExpiresAt < now && !lic.InGrace {
|
||||
return "expired"
|
||||
}
|
||||
if lic.Source == "opsx" && lic.LicenseKey != "" && !isOfflineLicenseKey(lic.LicenseKey) {
|
||||
last := lic.LastVerifiedAt
|
||||
if last == 0 && !lic.ActivatedAt.IsZero() {
|
||||
last = lic.ActivatedAt.Unix()
|
||||
}
|
||||
if last > 0 && now-last > offlineToleranceSecs {
|
||||
return "expired"
|
||||
}
|
||||
}
|
||||
if lic.Plan == "free" && lic.LicenseKey != "" {
|
||||
return "revoked"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// LicenseStatusPayload 给前端:始终带回库存证的 sanitized + 是否有效 Pro。
|
||||
// 无效时带 invalidReason(expired|over_instance|revoked),且不清 LicenseKey。
|
||||
// instanceId 始终返回本机指纹(DATA_DIR/instance.id),供离线 LICOFF- 绑码展示。
|
||||
func LicenseStatusPayload() (map[string]interface{}, bool) {
|
||||
RefreshOpsxLicenseIfDue()
|
||||
current := CurrentLicense()
|
||||
stored := StoredLicense()
|
||||
instanceID := GetOrCreateInstanceID()
|
||||
EnsureTrialStarted()
|
||||
trialDaysLeft := 0
|
||||
if TrialActive() {
|
||||
trialDaysLeft = TrialRemainingDays()
|
||||
}
|
||||
if stored == nil {
|
||||
plan := "free"
|
||||
if trialDaysLeft > 0 {
|
||||
plan = "trial"
|
||||
}
|
||||
return map[string]interface{}{
|
||||
"plan": plan, "instanceId": instanceID,
|
||||
"trial": map[string]interface{}{"active": trialDaysLeft > 0, "daysLeft": trialDaysLeft},
|
||||
}, trialDaysLeft > 0
|
||||
}
|
||||
out := stored.Sanitized()
|
||||
out["instanceId"] = instanceID
|
||||
if current != nil {
|
||||
delete(out, "invalidReason")
|
||||
return out, true
|
||||
}
|
||||
reason := deriveInvalidReason(stored)
|
||||
if reason != "" {
|
||||
out["invalidReason"] = reason
|
||||
}
|
||||
out["plan"] = "free"
|
||||
if trialDaysLeft > 0 {
|
||||
out["plan"] = "trial"
|
||||
}
|
||||
out["trial"] = map[string]interface{}{"active": trialDaysLeft > 0, "daysLeft": trialDaysLeft}
|
||||
return out, trialDaysLeft > 0
|
||||
}
|
||||
|
||||
// CurrentLicense 当前有效许可(nil = 免费版)
|
||||
func CurrentLicense() *models.License {
|
||||
db := repository.GetDB()
|
||||
if db == nil || db.License == nil {
|
||||
return nil
|
||||
}
|
||||
lic := db.License
|
||||
if lic.Plan != "pro" {
|
||||
return nil
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
|
||||
// opsx 在线证:超过 30 天未成功联网 → 降级
|
||||
if lic.Source == "opsx" && lic.LicenseKey != "" && !isOfflineLicenseKey(lic.LicenseKey) {
|
||||
last := lic.LastVerifiedAt
|
||||
if last == 0 {
|
||||
last = lic.ActivatedAt.Unix()
|
||||
}
|
||||
if last > 0 && now-last > offlineToleranceSecs {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
if lic.ExpiresAt > 0 && lic.ExpiresAt < now {
|
||||
if lic.InGrace {
|
||||
return lic // 服务端宽限内仍可用
|
||||
}
|
||||
return nil
|
||||
}
|
||||
// 旧 HMAC:补迁移提示(已激活历史数据)
|
||||
if lic.Source == "" || lic.Source == "hmac" {
|
||||
if lic.DeprecationNotice == "" {
|
||||
lic.DeprecationNotice = hmacDeprecationNotice
|
||||
}
|
||||
if lic.Source == "" {
|
||||
lic.Source = "hmac"
|
||||
}
|
||||
}
|
||||
return lic
|
||||
}
|
||||
|
||||
// IsPro 是否为有效 Pro:正式授权优先;无授权时 14 天试用窗口内等效 Pro
|
||||
func IsPro() bool {
|
||||
if CurrentLicense() != nil {
|
||||
return true
|
||||
}
|
||||
return TrialActive()
|
||||
}
|
||||
|
||||
// ProFeatureList Pro 功能标识(给前端展示)
|
||||
func ProFeatureList() []string { return proFeatures }
|
||||
@@ -0,0 +1,166 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"lokistack/config"
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
func withTempDB(t *testing.T) {
|
||||
t.Helper()
|
||||
t.Setenv("DATA_DIR", t.TempDir())
|
||||
t.Setenv("JWT_SECRET", "test-jwt-secret-with-at-least-32-characters")
|
||||
t.Setenv("ENCRYPTION_KEY", "test-encryption-key-with-at-least-32-characters")
|
||||
if err := repository.InitDB(); err != nil {
|
||||
t.Fatalf("InitDB: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = repository.Shutdown() })
|
||||
}
|
||||
|
||||
func TestDeactivateLicenseHMACClearsLocal(t *testing.T) {
|
||||
withTempDB(t)
|
||||
key, err := GenerateLicenseKey("pro", "personal", "c", 1, 1, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := ActivateLicense(key); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if CurrentLicense() == nil {
|
||||
t.Fatal("expected pro before deactivate")
|
||||
}
|
||||
res, err := DeactivateLicense()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.OfflineLocalOnly {
|
||||
t.Fatal("HMAC should not be offlineLocalOnly")
|
||||
}
|
||||
if CurrentLicense() != nil || StoredLicense() != nil {
|
||||
t.Fatal("expected local license cleared")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeactivateLicenseOfflineLocalOnly(t *testing.T) {
|
||||
withTempDB(t)
|
||||
instanceID := GetOrCreateInstanceID()
|
||||
// 无法签发真 LICOFF- 无私钥;直接写入库存模拟离线证
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.License = &models.License{
|
||||
Plan: "pro", Source: "opsx", LicenseKey: "LICOFF-TEST-FAKE",
|
||||
InstanceID: instanceID, ActivatedAt: time.Now(), IssuedAt: time.Now(),
|
||||
}
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
|
||||
res, err := DeactivateLicense()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !res.OfflineLocalOnly {
|
||||
t.Fatal("expected offlineLocalOnly")
|
||||
}
|
||||
if !strings.Contains(res.Message, "后台") && !strings.Contains(strings.ToLower(res.Message), "ops") {
|
||||
// 中文提示
|
||||
if res.Message == "" {
|
||||
t.Fatal("expected offline note message")
|
||||
}
|
||||
}
|
||||
if StoredLicense() != nil {
|
||||
t.Fatal("expected cleared")
|
||||
}
|
||||
}
|
||||
|
||||
// 远端解绑失败时不再报错卡死(用户无法自愈),降级为清本机 + offlineLocalOnly 标记;
|
||||
// 授权服务不返回 activation_token 的现实下,"保留本地等重试"等于永远解不了绑。
|
||||
func TestDeactivateLicenseOnlineKeepsStateOnRemoteFailure(t *testing.T) {
|
||||
withTempDB(t)
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"code": 1, "message": "解绑校验失败"})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := config.GetConfig()
|
||||
prev := cfg.LicenseAPIBase
|
||||
cfg.LicenseAPIBase = srv.URL
|
||||
defer func() { cfg.LicenseAPIBase = prev }()
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.License = &models.License{
|
||||
Plan: "pro", Source: "opsx", LicenseKey: "LIC-KEEP-ON-FAIL",
|
||||
InstanceID: GetOrCreateInstanceID(), ActivationToken: "tok",
|
||||
ActivatedAt: time.Now(), IssuedAt: time.Now(), LastVerifiedAt: time.Now().Unix(),
|
||||
}
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
|
||||
res, err := DeactivateLicense()
|
||||
if err != nil {
|
||||
t.Fatalf("远端失败应降级成功而非报错: %v", err)
|
||||
}
|
||||
if !res.OfflineLocalOnly {
|
||||
t.Fatal("远端失败降级时应标记 OfflineLocalOnly")
|
||||
}
|
||||
if StoredLicense() != nil {
|
||||
t.Fatal("降级后本机授权应已清除")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeactivateLicenseOnlineSuccessClears(t *testing.T) {
|
||||
withTempDB(t)
|
||||
var gotBody map[string]any
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/v1/license/deactivate" {
|
||||
t.Errorf("path %s", r.URL.Path)
|
||||
}
|
||||
_ = json.NewDecoder(r.Body).Decode(&gotBody)
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"code": 0, "message": "ok"})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
cfg := config.GetConfig()
|
||||
prev := cfg.LicenseAPIBase
|
||||
cfg.LicenseAPIBase = srv.URL
|
||||
defer func() { cfg.LicenseAPIBase = prev }()
|
||||
|
||||
id := GetOrCreateInstanceID()
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
db.License = &models.License{
|
||||
Plan: "pro", Source: "opsx", LicenseKey: "LIC-OK",
|
||||
InstanceID: id, ActivationToken: "act-token",
|
||||
ActivatedAt: time.Now(), IssuedAt: time.Now(), LastVerifiedAt: time.Now().Unix(),
|
||||
}
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
|
||||
res, err := DeactivateLicense()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.OfflineLocalOnly {
|
||||
t.Fatal("online should not set offlineLocalOnly")
|
||||
}
|
||||
if gotBody["license_key"] != "LIC-OK" || gotBody["device_id"] != id || gotBody["activation_token"] != "act-token" {
|
||||
t.Fatalf("bad deactivate body: %#v", gotBody)
|
||||
}
|
||||
if StoredLicense() != nil {
|
||||
t.Fatal("expected cleared after success")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeactivateLicenseEmpty(t *testing.T) {
|
||||
withTempDB(t)
|
||||
if _, err := DeactivateLicense(); err == nil {
|
||||
t.Fatal("expected error when no license")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package services
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestClassifyRejectReason(t *testing.T) {
|
||||
cases := []struct {
|
||||
msg string
|
||||
want string
|
||||
}{
|
||||
{"device limit exceeded", "over_instance"},
|
||||
{"设备席位已满", "over_instance"},
|
||||
{"已超过实例上限", "over_instance"},
|
||||
{"license expired", "expired"},
|
||||
{"许可码已过期", "expired"},
|
||||
{"revoked by admin", "revoked"},
|
||||
{"许可证无效", "revoked"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := classifyRejectReason(tc.msg); got != tc.want {
|
||||
t.Fatalf("classifyRejectReason(%q)=%q want %q", tc.msg, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSanitizedHidesSeatsAndKeepsInvalidReason(t *testing.T) {
|
||||
// 通过 models 包测 Sanitized:此处仅确保 classify 与 payload 路径可编译。
|
||||
_ = classifyRejectReason("x")
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TestLicenseIssueVerifyRoundTrip 签发 → 校验 → 激活全链路
|
||||
func TestLicenseIssueVerifyRoundTrip(t *testing.T) {
|
||||
key, err := GenerateLicenseKey("pro", "personal", "test-customer", 10, 1, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lic, err := VerifyLicenseKey(key)
|
||||
if err != nil {
|
||||
t.Fatalf("有效许可码校验失败: %v", err)
|
||||
}
|
||||
if lic.Plan != "pro" || lic.Edition != "personal" || lic.Customer != "test-customer" {
|
||||
t.Fatalf("许可内容不符: %+v", lic)
|
||||
}
|
||||
if lic.ExpiresAt != 0 {
|
||||
t.Fatalf("永久许可不应有到期时间")
|
||||
}
|
||||
}
|
||||
|
||||
// TestLicenseTamperedRejected 篡改许可码必须被拒
|
||||
func TestLicenseTamperedRejected(t *testing.T) {
|
||||
key, _ := GenerateLicenseKey("pro", "team", "c", 5, 3, 0)
|
||||
tampered := key[:len(key)-4] + "AAAA"
|
||||
if _, err := VerifyLicenseKey(tampered); err == nil {
|
||||
t.Fatal("篡改的许可码不应通过校验")
|
||||
}
|
||||
if _, err := VerifyLicenseKey("not-a-license"); err == nil {
|
||||
t.Fatal("非法字符串不应通过校验")
|
||||
}
|
||||
}
|
||||
|
||||
// TestLicenseExpiredRejected 过期许可拒收
|
||||
func TestLicenseExpiredRejected(t *testing.T) {
|
||||
key, err := GenerateLicenseKey("pro", "team", "c", 5, 3, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// days=1 尚未过期,先签一个已过期的:手工构造已过期载荷不可行(签名),
|
||||
// 改为校验 CurrentLicense 对过期值的回落 —— 这里至少验证短期许可可签可验
|
||||
if _, err := VerifyLicenseKey(key); err != nil {
|
||||
t.Fatalf("未过期的短期许可应可校验: %v", err)
|
||||
}
|
||||
_ = time.Now()
|
||||
}
|
||||
|
||||
// TestLicenseFreePlanRejected 免费计划不签发
|
||||
func TestLicenseFreePlanRejected(t *testing.T) {
|
||||
if _, err := GenerateLicenseKey("free", "personal", "c", 0, 0, 0); err == nil {
|
||||
t.Fatal("免费计划不应签发许可码")
|
||||
}
|
||||
if !strings.Contains("", "") {
|
||||
t.Fatal("unreachable")
|
||||
}
|
||||
}
|
||||
@@ -367,11 +367,103 @@ func LokiLabels(ctx context.Context, src *models.LogSource, start, end int64) ([
|
||||
}
|
||||
|
||||
// LokiLabelValues 标签值列表(query 只取 {...} selector 部分)
|
||||
// labelValuesDefaultWindow 是 label 值下拉框的默认时间窗。
|
||||
// 只影响"没显式给时间范围"的调用(下拉框),不影响带范围的查询。
|
||||
const labelValuesDefaultWindow = time.Hour
|
||||
|
||||
// ── label 候选值的短缓存 ────────────────────────────────────────────
|
||||
// 快捷 LogQL 的下拉框在「切数据源 / 切命名空间 / 开关面板」时都会重新请求,
|
||||
// 而值本身的时效性要求很低(新上线的 app 晚 1 分钟出现完全可接受)。
|
||||
// 缓存后重复请求从 25~40ms 降到微秒级,体感是"打开就有"。
|
||||
const (
|
||||
// 30 秒:新上线的 app / 新副本要尽快出现在下拉里,
|
||||
// 但也不必每点一次都打 Loki(重复请求间隔通常远小于 30 秒)。
|
||||
labelCacheTTL = 30 * time.Second
|
||||
labelCacheMax = 500
|
||||
)
|
||||
|
||||
type labelCacheEntry struct {
|
||||
values []string
|
||||
expiresAt time.Time
|
||||
}
|
||||
|
||||
var (
|
||||
labelCacheMu sync.Mutex
|
||||
labelCache = map[string]labelCacheEntry{}
|
||||
)
|
||||
|
||||
func labelCacheKey(src *models.LogSource, name, selector string, start, end int64) string {
|
||||
if src == nil {
|
||||
return ""
|
||||
}
|
||||
// 时间窗按 5 分钟取整:避免"每次请求差几秒"导致缓存全不命中
|
||||
bucket := int64(5 * time.Minute)
|
||||
return fmt.Sprintf("%d|%s|%s|%d|%d", src.ID, name, selector, start/bucket, end/bucket)
|
||||
}
|
||||
|
||||
func labelCacheGet(key string) ([]string, bool) {
|
||||
if key == "" {
|
||||
return nil, false
|
||||
}
|
||||
labelCacheMu.Lock()
|
||||
defer labelCacheMu.Unlock()
|
||||
if entry, ok := labelCache[key]; ok && time.Now().Before(entry.expiresAt) {
|
||||
return entry.values, true
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
func labelCachePut(key string, values []string) {
|
||||
if key == "" {
|
||||
return
|
||||
}
|
||||
labelCacheMu.Lock()
|
||||
defer labelCacheMu.Unlock()
|
||||
if len(labelCache) >= labelCacheMax {
|
||||
now := time.Now()
|
||||
for k, v := range labelCache {
|
||||
if now.After(v.expiresAt) {
|
||||
delete(labelCache, k)
|
||||
}
|
||||
}
|
||||
if len(labelCache) >= labelCacheMax {
|
||||
labelCache = map[string]labelCacheEntry{}
|
||||
}
|
||||
}
|
||||
labelCache[key] = labelCacheEntry{values: values, expiresAt: time.Now().Add(labelCacheTTL)}
|
||||
}
|
||||
|
||||
// InvalidateLabelCache 清空 label 候选值缓存。
|
||||
//
|
||||
// 什么时候必须调:平台侧发生了"会改变候选值集合"的变更 ——
|
||||
// 采集器新增/修改/卸载(主机模式下 app 标签来自采集器配置)、
|
||||
// 部署删除、数据源增删改。这些场景下等 30 秒 TTL 自然过期会让用户
|
||||
// 觉得"刚配好的 app 在下拉里找不到"。
|
||||
func InvalidateLabelCache() {
|
||||
labelCacheMu.Lock()
|
||||
labelCache = map[string]labelCacheEntry{}
|
||||
labelCacheMu.Unlock()
|
||||
}
|
||||
|
||||
func LokiLabelValues(ctx context.Context, src *models.LogSource, name, selector string, start, end int64) ([]string, error) {
|
||||
// 缓存按调用方传入的原始 start/end 计算 key(下方默认窗口不影响 key 的稳定性)
|
||||
cacheKey := labelCacheKey(src, name, selector, start, end)
|
||||
if cached, ok := labelCacheGet(cacheKey); ok {
|
||||
return cached, nil
|
||||
}
|
||||
q := url.Values{}
|
||||
if s := extractStreamSelector(selector); s != "" && s != "{}" {
|
||||
q.Set("query", s)
|
||||
}
|
||||
// 不传时间范围时 Loki 默认扫**最近 6 小时**的索引 —— 在 370GB/天的集群上
|
||||
// 实测要 10~21 秒(下拉框"一直加载中"的直接原因)。
|
||||
// 下拉框只需要"现在有哪些 app/pod 在打日志",给 1 小时足够,索引量降到 1/6。
|
||||
// 调用方显式传了时间范围则完全尊从,不受影响。
|
||||
if start <= 0 && end <= 0 {
|
||||
now := time.Now()
|
||||
start = now.Add(-labelValuesDefaultWindow).UnixNano()
|
||||
end = now.UnixNano()
|
||||
}
|
||||
if end > 0 {
|
||||
q.Set("end", strconv.FormatInt(end, 10))
|
||||
}
|
||||
@@ -393,9 +485,113 @@ func LokiLabelValues(ctx context.Context, src *models.LogSource, name, selector
|
||||
if resp.Data == nil {
|
||||
resp.Data = []string{}
|
||||
}
|
||||
labelCachePut(cacheKey, resp.Data)
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
// LokiLabelValuesBatch 一次取多个 label 的候选值。
|
||||
//
|
||||
// 前端快捷查询要对 app / service_name / container 做逐级回退(哪个有值用哪个),
|
||||
// 原来最多并发 3 个请求、每个请求各走一次 Loki 索引查询 + 一次 kube-apiserver
|
||||
// service proxy;合并成本接口后前端只发一次,后端内部并发(上限 4)。
|
||||
//
|
||||
// 语义:单个 label 失败不影响其它 label(返回空数组),**全部失败**才报错 ——
|
||||
// 否则某个 label 在特定 selector 下索引异常,整个下拉框都会变成"加载失败"。
|
||||
func LokiLabelValuesBatch(ctx context.Context, src *models.LogSource, names []string, selector string, start, end int64) (map[string][]string, error) {
|
||||
out := make(map[string][]string, len(names))
|
||||
var (
|
||||
mu sync.Mutex
|
||||
wg sync.WaitGroup
|
||||
firstErr error
|
||||
okCount int
|
||||
seen = make(map[string]bool, len(names))
|
||||
)
|
||||
sem := make(chan struct{}, 4)
|
||||
for _, raw := range names {
|
||||
name := strings.TrimSpace(raw)
|
||||
if name == "" || seen[name] {
|
||||
continue
|
||||
}
|
||||
seen[name] = true
|
||||
out[name] = []string{}
|
||||
wg.Add(1)
|
||||
go func(labelName string) {
|
||||
defer wg.Done()
|
||||
sem <- struct{}{}
|
||||
defer func() { <-sem }()
|
||||
values, err := LokiLabelValues(ctx, src, labelName, selector, start, end)
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if err != nil {
|
||||
if firstErr == nil {
|
||||
firstErr = err
|
||||
}
|
||||
return
|
||||
}
|
||||
okCount++
|
||||
out[labelName] = values
|
||||
}(name)
|
||||
}
|
||||
wg.Wait()
|
||||
if okCount == 0 && firstErr != nil {
|
||||
return out, firstErr
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// LokiVolumeBytesFor 用 index/volume 读某个 selector 的日志体积(字节)。
|
||||
//
|
||||
// 这是 Loki 的**索引接口**:只读 TSDB 里记录的 chunk 字节数,不解码日志内容。
|
||||
// 实测单个 app、一小时窗口约 80ms(对比 bytes_over_time 同窗口 13s/15 分钟、
|
||||
// 约 50s/小时 —— 因为它要把整小时的日志解压出来数行字节)。
|
||||
//
|
||||
// 代价:index/volume 返回的 metric 只包含 selector 里出现过的 label,
|
||||
// 所以"按 app 取体积"没法一条查询拿全,必须一个 app 调一次(见 app_volume.go)。
|
||||
// aggregateBy=series 让返回值带上 selector 里的全部 label,便于调用方核对。
|
||||
func LokiVolumeBytesFor(ctx context.Context, src *models.LogSource, selector string, start, end int64) (int64, error) {
|
||||
if end <= 0 {
|
||||
end = time.Now().UnixNano()
|
||||
}
|
||||
if start <= 0 {
|
||||
start = end - int64(time.Hour)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
|
||||
defer cancel()
|
||||
|
||||
q := url.Values{}
|
||||
q.Set("query", selector)
|
||||
q.Set("start", strconv.FormatInt(start, 10))
|
||||
q.Set("end", strconv.FormatInt(end, 10))
|
||||
q.Set("aggregateBy", "series")
|
||||
body, _, err := lokiProxyGET(ctx, src, "/loki/api/v1/index/volume", q)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var resp struct {
|
||||
Data struct {
|
||||
Result []struct {
|
||||
Metric map[string]string `json:"metric"`
|
||||
Value [2]interface{} `json:"value"`
|
||||
} `json:"result"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return 0, fmt.Errorf("解析 volume 响应失败: %w", err)
|
||||
}
|
||||
var total int64
|
||||
for _, item := range resp.Data.Result {
|
||||
switch v := item.Value[1].(type) {
|
||||
case float64:
|
||||
total += int64(v)
|
||||
case string:
|
||||
if n, convErr := strconv.ParseFloat(v, 64); convErr == nil {
|
||||
total += int64(n)
|
||||
}
|
||||
}
|
||||
}
|
||||
return total, nil
|
||||
}
|
||||
|
||||
// extractStreamSelector 从 LogQL 中提取 {...} 部分(labels API 仅接受 stream selector)
|
||||
func extractStreamSelector(query string) string {
|
||||
i := strings.Index(query, "{")
|
||||
@@ -688,6 +884,61 @@ func LokiMetrics(ctx context.Context, src *models.LogSource, query string, start
|
||||
return series, nil
|
||||
}
|
||||
|
||||
// ── 总条数/总体积的短缓存 ─────────────────────────────────────────────
|
||||
// /logs/stats 走 index/stats,实测 5 分钟窗口要 1.2 秒(60 分钟更慢)。
|
||||
// 分析页每次进页面、每次切时间范围都会打它,而这类聚合值的可容忍陈旧度
|
||||
// 与指标查询一致(分钟级),所以用同样的 90s TTL + 2 分钟取整。
|
||||
const (
|
||||
statsCacheTTL = 90 * time.Second
|
||||
statsCacheMax = 200
|
||||
)
|
||||
|
||||
type statsCacheEntry struct {
|
||||
stats *LokiStats
|
||||
expiresAt time.Time
|
||||
}
|
||||
|
||||
var (
|
||||
statsCacheMu sync.Mutex
|
||||
statsCache = map[string]statsCacheEntry{}
|
||||
)
|
||||
|
||||
// LokiStatsCached 带短缓存的总量统计。
|
||||
func LokiStatsCached(ctx context.Context, src *models.LogSource, query string, start, end int64) (*LokiStats, error) {
|
||||
key := ""
|
||||
if src != nil {
|
||||
bucket := int64(metricCacheBucket)
|
||||
key = fmt.Sprintf("%d|%s|%d|%d", src.ID, query, start/bucket, end/bucket)
|
||||
statsCacheMu.Lock()
|
||||
if entry, ok := statsCache[key]; ok && time.Now().Before(entry.expiresAt) {
|
||||
statsCacheMu.Unlock()
|
||||
return entry.stats, nil
|
||||
}
|
||||
statsCacheMu.Unlock()
|
||||
}
|
||||
|
||||
st, err := LokiStatsFor(ctx, src, query, start, end)
|
||||
if err != nil || key == "" {
|
||||
return st, err
|
||||
}
|
||||
|
||||
statsCacheMu.Lock()
|
||||
if len(statsCache) >= statsCacheMax {
|
||||
now := time.Now()
|
||||
for k, v := range statsCache {
|
||||
if now.After(v.expiresAt) {
|
||||
delete(statsCache, k)
|
||||
}
|
||||
}
|
||||
if len(statsCache) >= statsCacheMax {
|
||||
statsCache = map[string]statsCacheEntry{}
|
||||
}
|
||||
}
|
||||
statsCache[key] = statsCacheEntry{stats: st, expiresAt: time.Now().Add(statsCacheTTL)}
|
||||
statsCacheMu.Unlock()
|
||||
return st, nil
|
||||
}
|
||||
|
||||
// lokiMetricsUncached 真正打 Loki 的实现(不带缓存)。
|
||||
func lokiMetricsUncached(ctx context.Context, src *models.LogSource, query string, start, end int64) ([]MetricSeries, error) {
|
||||
if end <= 0 {
|
||||
@@ -758,9 +1009,12 @@ func pickHistogramStep(spanNano int64) string {
|
||||
return "1m"
|
||||
case span <= 6*time.Hour:
|
||||
return "5m"
|
||||
case span <= 24*time.Hour:
|
||||
case span <= 12*time.Hour:
|
||||
return "15m"
|
||||
case span <= 7*24*time.Hour:
|
||||
// 12 小时以上统一 1 小时一桶。
|
||||
// 24 小时原来是 15m(96 根柱子):横轴挤成一片、且每根柱子只有十来分钟的数据,
|
||||
// 看不出趋势;1h → 24 根,和"按小时看日志量"的心智一致。
|
||||
return "1h"
|
||||
default:
|
||||
return "6h"
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
@@ -148,6 +149,7 @@ func AuthorizedLogQuery(src *models.LogSource, userID uint, admin bool, query st
|
||||
}
|
||||
|
||||
func CreateLogSource(s *models.LogSource) (*models.LogSource, error) {
|
||||
defer InvalidateLabelCache()
|
||||
if err := encryptSourceCredentials(s); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -211,6 +213,7 @@ func validSourceHeader(name string) bool {
|
||||
}
|
||||
|
||||
func UpdateLogSource(id uint, patch map[string]interface{}) error {
|
||||
defer InvalidateLabelCache()
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
@@ -326,6 +329,7 @@ func ValidateSourceAccess(access []models.LogSourceAccess) error {
|
||||
}
|
||||
|
||||
func DeleteLogSource(id uint) error {
|
||||
defer InvalidateLabelCache()
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
@@ -433,12 +437,94 @@ func CleanupManagedLogSource(name string) {
|
||||
}
|
||||
}
|
||||
|
||||
// CleanupOrphanManagedLogSources 启动时清理孤儿托管数据源:
|
||||
// 部署记录已删除、或所属集群已删除的 managed 源。
|
||||
//
|
||||
// 为什么需要:CleanupManagedLogSource 只在「删除部署」的代码路径里执行;
|
||||
// 如果部署是在旧版本(没有清理逻辑)删掉的,或删除过程中进程退出,
|
||||
// 托管源就会永久残留,在数据源列表里指向一个不存在的实例。
|
||||
func CleanupOrphanManagedLogSources() {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
names := make(map[string]bool, len(db.Deployments))
|
||||
for _, d := range db.Deployments {
|
||||
names[d.Name] = true
|
||||
}
|
||||
clusters := make(map[uint]bool, len(db.Clusters))
|
||||
for _, c := range db.Clusters {
|
||||
clusters[c.ID] = true
|
||||
}
|
||||
kept := make([]models.LogSource, 0, len(db.LogSources))
|
||||
removed, removedDefault := 0, false
|
||||
for _, s := range db.LogSources {
|
||||
orphan := false
|
||||
if s.Managed {
|
||||
if !names[s.Name] {
|
||||
orphan = true
|
||||
}
|
||||
if s.ClusterID != nil && !clusters[*s.ClusterID] {
|
||||
orphan = true
|
||||
}
|
||||
}
|
||||
if orphan {
|
||||
removed++
|
||||
if s.IsDefault {
|
||||
removedDefault = true
|
||||
}
|
||||
continue
|
||||
}
|
||||
kept = append(kept, s)
|
||||
}
|
||||
if removed > 0 {
|
||||
if removedDefault && len(kept) > 0 {
|
||||
kept[0].IsDefault = true
|
||||
}
|
||||
db.LogSources = kept
|
||||
db.RequestSave()
|
||||
log.Printf("[startup] 清理 %d 个孤儿托管数据源", removed)
|
||||
}
|
||||
db.Unlock()
|
||||
}
|
||||
|
||||
// AutoDetectLogSource 探测数据源的可用连接方式,返回最终选择(cluster-svc / url)。
|
||||
//
|
||||
// 交付环境组合很多:平台可能在主机或集群内,Loki 也可能在主机或集群内。
|
||||
// cluster-svc 走 kube-apiserver 的 service proxy,不依赖集群 DNS;url 直连
|
||||
// 则需要平台能访问到该地址。这里按「代理优先、URL 兜底」实测,避免按部署
|
||||
// 模式硬编码导致某种组合连不上。
|
||||
func AutoDetectLogSource(ctx context.Context, src *models.LogSource) string {
|
||||
if src.ClusterID != nil {
|
||||
probe := *src
|
||||
probe.TargetType = "cluster-svc"
|
||||
if _, status, err := lokiProxyGET(ctx, &probe, "/ready", nil); err == nil && status == 200 {
|
||||
src.TargetType = "cluster-svc"
|
||||
return "cluster-svc"
|
||||
}
|
||||
}
|
||||
if src.URL != "" {
|
||||
probe := *src
|
||||
probe.TargetType = "url"
|
||||
if _, status, err := lokiProxyGET(ctx, &probe, "/ready", nil); err == nil && status == 200 {
|
||||
src.TargetType = "url"
|
||||
return "url"
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// EnsureLogSourceForDeployment 部署成功后自动注册查询数据源(host→url;k8s→cluster-svc)。
|
||||
// 已存在同名数据源时:平台托管的自动刷新连接信息,用户手填的不覆盖。
|
||||
func EnsureLogSourceForDeployment(d *models.LokiDeployment) {
|
||||
if d == nil || d.Endpoint == "" {
|
||||
return
|
||||
}
|
||||
// 用户主动删过这个自动数据源 → 不再补回来(重新部署/编辑部署会重置该开关)
|
||||
if d.AutoSourceDisabled {
|
||||
return
|
||||
}
|
||||
svc := "loki"
|
||||
port := 3100
|
||||
target := &models.LogSource{
|
||||
@@ -454,8 +540,18 @@ func EnsureLogSourceForDeployment(d *models.LokiDeployment) {
|
||||
// 这里换成主机真实 IP,平台才能连上。
|
||||
target.URL = reachableEndpoint(d)
|
||||
target.ClusterID = nil
|
||||
} else {
|
||||
// k8s 部署:集群内 Service 地址也记下来,作为 URL 直连候选
|
||||
target.URL = d.Endpoint
|
||||
}
|
||||
|
||||
// 实测可用性后决定连接方式:主机/集群内平台、主机/集群内 Loki 都兼容
|
||||
detectCtx, cancel := context.WithTimeout(context.Background(), 6*time.Second)
|
||||
if method := AutoDetectLogSource(detectCtx, target); method != "" {
|
||||
log.Printf("[log-source] %q 自动选择连接方式: %s", target.Name, method)
|
||||
}
|
||||
cancel()
|
||||
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
for i := range db.LogSources {
|
||||
@@ -481,6 +577,42 @@ func EnsureLogSourceForDeployment(d *models.LokiDeployment) {
|
||||
_, _ = CreateLogSource(target)
|
||||
}
|
||||
|
||||
// DisableAutoSourceFor 标记某个部署不再自动注册数据源。
|
||||
//
|
||||
// 触发场景:用户在「日志查询 → 数据源」里删掉了平台自动创建的数据源。
|
||||
// 不置位的话,EnsureLogSourcesForRunningDeployments 会立刻把它补回来。
|
||||
func DisableAutoSourceFor(name string) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for i := range db.Deployments {
|
||||
if db.Deployments[i].Name == name && !db.Deployments[i].AutoSourceDisabled {
|
||||
db.Deployments[i].AutoSourceDisabled = true
|
||||
log.Printf("[log-source] %q 的自动数据源已被用户删除,不再自动重建(重新部署可恢复)", name)
|
||||
db.RequestSave()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// EnableAutoSourceFor 重新允许自动注册(部署被编辑/重新部署时调用)。
|
||||
func EnableAutoSourceFor(name string) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for i := range db.Deployments {
|
||||
if db.Deployments[i].Name == name && db.Deployments[i].AutoSourceDisabled {
|
||||
db.Deployments[i].AutoSourceDisabled = false
|
||||
db.RequestSave()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// EnsureLogSourcesForRunningDeployments 补齐运行中部署的数据源,避免查询页默认没有数据源。
|
||||
func EnsureLogSourcesForRunningDeployments() {
|
||||
db := repository.GetDB()
|
||||
|
||||
@@ -69,8 +69,12 @@ compactor:
|
||||
|
||||
limits_config:
|
||||
retention_period: {{ .RetentionPeriod }}h
|
||||
# 单实例 Loki 的并发护栏:默认 32 高得离谱,一条重查询就能把单进程打满
|
||||
max_query_parallelism: 8
|
||||
# 单实例 Loki 的并发护栏:默认 32 高得离谱,一条重查询就能把单进程打满。
|
||||
# 并行度**自动等于部署的 CPU 核数**(见 queryParallelism()):
|
||||
# 高于核数只会让多个子查询互相抢 CPU——实测 4 核配 6/8 路并行时
|
||||
# 25%~76% 的调度周期处于限流状态,查询耗时并没有变快,反而 label values 要 20 秒。
|
||||
# 所以这个值不需要单独暴露配置:用户在「组件部署」改 CPU,这里自动跟随。
|
||||
max_query_parallelism: {{ .QueryParallelism }}
|
||||
|
||||
# ── 查询结果缓存(官方推荐;此前完全没配,重复查询要重算一遍)───────
|
||||
# align_queries_with_step 让不同时间范围的同类查询对齐到同一 step,
|
||||
@@ -113,6 +117,34 @@ analytics:
|
||||
reporting_enabled: false
|
||||
`
|
||||
|
||||
// queryParallelism 把部署的 CPU 核数换算成查询并行度。
|
||||
//
|
||||
// 为什么不用独立配置项:并行度只有"跟核数对齐"一个正确值。
|
||||
// 单独暴露一个配置,用户改了 CPU 忘了改并行度,就会出现
|
||||
// "4 核配 8 路并行 → 限流 76%"这种自己坑自己的组合。
|
||||
// 这里直接跟随 CPU,用户只需要在「组件部署」里改 CPU。
|
||||
func queryParallelism(resourcesCPU string) int {
|
||||
value := strings.TrimSpace(resourcesCPU)
|
||||
value = strings.TrimSuffix(value, "m") // 容忍 "4000m" 这种写法
|
||||
cores, err := strconv.Atoi(value)
|
||||
if err != nil || cores <= 0 {
|
||||
return defaultQueryParallelism
|
||||
}
|
||||
if strings.HasSuffix(strings.TrimSpace(resourcesCPU), "m") {
|
||||
cores = (cores + 999) / 1000 // 毫核向上取整到核
|
||||
}
|
||||
if cores < 2 {
|
||||
return 2
|
||||
}
|
||||
// 上限 12:再高单实例 Loki 只会互相抢 CPU,且并发护栏本身还有全局闸门
|
||||
if cores > 12 {
|
||||
return 12
|
||||
}
|
||||
return cores
|
||||
}
|
||||
|
||||
const defaultQueryParallelism = 4
|
||||
|
||||
type lokiConfigData struct {
|
||||
AuthEnabled bool
|
||||
ReplicationFactor int
|
||||
@@ -123,12 +155,14 @@ type lokiConfigData struct {
|
||||
CompactorStoreType string
|
||||
// CompactorDeleteWorkers 过期 chunk 删除并发,按对象存储类型取默认值
|
||||
CompactorDeleteWorkers int
|
||||
RingInstanceAddr string // 单副本 127.0.0.1;多副本 0.0.0.0(须可被对端寻址)
|
||||
RingKVStore string // 单副本 inmemory;多副本 memberlist
|
||||
MemberlistBlock string // 多副本时非空,注入 memberlist 顶级配置
|
||||
RulerStorageBlock string // 对象存储时非空,注入 ruler.storage.s3/gcs/azure 嵌套配置
|
||||
MultiReplica bool
|
||||
FilesystemStorage bool
|
||||
// QueryParallelism 查询并行度,自动等于部署 CPU 核数(2~12)
|
||||
QueryParallelism int
|
||||
RingInstanceAddr string // 单副本 127.0.0.1;多副本 0.0.0.0(须可被对端寻址)
|
||||
RingKVStore string // 单副本 inmemory;多副本 memberlist
|
||||
MemberlistBlock string // 多副本时非空,注入 memberlist 顶级配置
|
||||
RulerStorageBlock string // 对象存储时非空,注入 ruler.storage.s3/gcs/azure 嵌套配置
|
||||
MultiReplica bool
|
||||
FilesystemStorage bool
|
||||
}
|
||||
|
||||
func (d *LokiDeployer) generateLokiConfig() string {
|
||||
@@ -189,6 +223,7 @@ func (d *LokiDeployer) generateLokiConfig() string {
|
||||
RulerStoreType: rulerStore,
|
||||
CompactorStoreType: compactorStore,
|
||||
CompactorDeleteWorkers: compactorDeleteWorkers(d.deployment.StorageConfig),
|
||||
QueryParallelism: queryParallelism(d.deployment.ResourcesCPU),
|
||||
RingInstanceAddr: ringAddr,
|
||||
RingKVStore: ringStore,
|
||||
MemberlistBlock: memberlistBlock,
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
package services
|
||||
|
||||
import "testing"
|
||||
|
||||
// 查询并行度必须跟部署的 CPU 核数对齐。
|
||||
//
|
||||
// 事故背景:曾经写死 8,而部署只有 4 核 → 多个子查询互相抢 CPU,
|
||||
// 实测 76% 的调度周期被限流,连 label values 都要 20 秒。
|
||||
// 现在改成从 resourcesCPU 推导,用户在界面改 CPU 时并行度自动跟随。
|
||||
func TestQueryParallelism(t *testing.T) {
|
||||
cases := []struct {
|
||||
cpu string
|
||||
want int
|
||||
}{
|
||||
{"4", 4},
|
||||
{"8", 8},
|
||||
{"2", 2},
|
||||
{"1", 2}, // 低于 2 核按 2 兜底
|
||||
{"16", 12}, // 上限 12
|
||||
{"", 4}, // 空值走默认
|
||||
{"abc", 4}, // 非法值走默认
|
||||
{"4000m", 4}, // 毫核写法
|
||||
{"500m", 2}, // 0.5 核 → 向上取整 1 → 不低于 2
|
||||
{"6000m", 6},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := queryParallelism(c.cpu); got != c.want {
|
||||
t.Errorf("queryParallelism(%q) = %d, want %d", c.cpu, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 回归:DaemonSet 模式的"只采集这些命名空间"必须真正渲染进 promtail 配置,
|
||||
// 否则前端多选形同虚设(用户选了 astralx,实际还是全集群在采)。
|
||||
func TestDaemonsetTargetNamespacesFilter(t *testing.T) {
|
||||
base := func(ns []string) string {
|
||||
return PromtailDaemonsetYAML("log", "http://loki.log.svc:3100", "c1", "", ns, "", nil)
|
||||
}
|
||||
|
||||
// 1) 空 = 全集群:不应出现 namespaces 白名单
|
||||
all := base(nil)
|
||||
if strings.Contains(all, "namespaces:") {
|
||||
t.Error("targetNamespaces 为空时不应生成命名空间白名单(应为全集群采集)")
|
||||
}
|
||||
|
||||
// 2) 指定一个命名空间:必须出现白名单
|
||||
one := base([]string{"astralx"})
|
||||
if !strings.Contains(one, "namespaces:") {
|
||||
t.Fatal("targetNamespaces=[astralx] 时没有生成 namespaces 白名单")
|
||||
}
|
||||
if !strings.Contains(one, "astralx") {
|
||||
t.Error("白名单里没有 astralx")
|
||||
}
|
||||
|
||||
// 3) 多个命名空间都要在
|
||||
multi := base([]string{"astralx", "default"})
|
||||
for _, want := range []string{"astralx", "default"} {
|
||||
if !strings.Contains(multi, want) {
|
||||
t.Errorf("多选时白名单缺少 %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,14 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"lokistack/config"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
@@ -263,6 +267,11 @@ func fileDBStats(path string) string {
|
||||
count("deployments"), count("collectors"), count("auditLogs"))
|
||||
}
|
||||
|
||||
// dataDirPath 数据目录绝对路径(自检磁盘空间等场景使用)
|
||||
func dataDirPath() string {
|
||||
return config.GetConfig().DataDir
|
||||
}
|
||||
|
||||
// diskUsage 返回剩余与总空间(字节)
|
||||
func diskUsage(path string) (free, total uint64, err error) {
|
||||
var st syscall.Statfs_t
|
||||
@@ -271,3 +280,18 @@ func diskUsage(path string) (free, total uint64, err error) {
|
||||
}
|
||||
return st.Bavail * uint64(st.Bsize), st.Blocks * uint64(st.Bsize), nil
|
||||
}
|
||||
|
||||
// encryptionKeyFingerprint ENCRYPTION_KEY 指纹(SHA256 前 16 位,自检展示用,不泄露密钥)
|
||||
func encryptionKeyFingerprint() string {
|
||||
key := config.GetConfig().EncryptionKey
|
||||
if key == "" {
|
||||
if b, err := os.ReadFile(filepath.Join(config.GetConfig().DataDir, ".encryption-key")); err == nil {
|
||||
key = strings.TrimSpace(string(b))
|
||||
}
|
||||
}
|
||||
if key == "" {
|
||||
return ""
|
||||
}
|
||||
sum := sha256.Sum256([]byte(key))
|
||||
return hex.EncodeToString(sum[:])[:16]
|
||||
}
|
||||
@@ -151,6 +151,11 @@ func (s *StorageService) Delete(id uint) error {
|
||||
defer db.Unlock()
|
||||
for i, st := range db.Storages {
|
||||
if st.ID == id {
|
||||
for _, d := range db.Deployments {
|
||||
if d.StorageConfigID == id {
|
||||
return fmt.Errorf("仍有部署使用该存储配置,请先删除部署或改用其他存储")
|
||||
}
|
||||
}
|
||||
db.Storages = append(db.Storages[:i], db.Storages[i+1:]...)
|
||||
db.RequestSave()
|
||||
return nil
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
package services
|
||||
|
||||
// 14 天 Pro 试用:首次启动落一个安装时间戳,窗口内等效 Pro(无需授权码)。
|
||||
// 滥用防护(多层,删库重装只能重置最外层):
|
||||
// 1. DATA_DIR/.trial-start(0600)——用户可删,快但易防
|
||||
// 2. 库内 settings.trial_started_at——随 lokistack.json 走,删库重装会重置
|
||||
// 3. 主机指纹(机器级):/etc/machine-id 优先(容器内不稳定)、回退 os.Hostname+CPU 数+
|
||||
// DATA_DIR 所在磁盘标识——重装容器/换目录仍命中同一指纹则不重发试用
|
||||
// 指纹哈希只存本地(与远端无关),换机器必然新试用(接受:试用成本有限,Pro 功能
|
||||
// 本身零边际成本,追杀到 BIOS 级指纹得不偿失)。
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"os"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"lokistack/config"
|
||||
"lokistack/models"
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
const trialDays = 14
|
||||
|
||||
// TrialRemainingDays 剩余试用天数(0=无试用;>0=试用中)。trialStart==0 表示从未启动过计时。
|
||||
func TrialRemainingDays() int {
|
||||
start := trialStartAt()
|
||||
if start == 0 {
|
||||
return trialDays // 尚未计时:首查即激活,调用方应先 ensureTrialStarted
|
||||
}
|
||||
days := int((trialStartExpiry(start) - time.Now().Unix()) / 86400)
|
||||
if days < 0 {
|
||||
days = 0
|
||||
}
|
||||
return days
|
||||
}
|
||||
|
||||
// TrialActive 试用是否生效(Pro 等效)
|
||||
func TrialActive() bool {
|
||||
start := trialStartAt()
|
||||
if start == 0 {
|
||||
return true // 未计时视为可用(首次调用会落时间戳)
|
||||
}
|
||||
return time.Now().Unix() < trialStartExpiry(start)
|
||||
}
|
||||
|
||||
func trialStartExpiry(start int64) int64 { return start + int64(trialDays)*86400 }
|
||||
|
||||
// EnsureTrialStarted 首次启动时落试用起点(幂等)。三层防线:文件 > 库 > 指纹。
|
||||
// 指纹命中"该机器已用过试用"则不重发(防删库重装刷试用)。
|
||||
func EnsureTrialStarted() {
|
||||
if trialStartAt() != 0 {
|
||||
return
|
||||
}
|
||||
now := time.Now().Unix()
|
||||
// 库内 + 文件双写(任一存在即以最早的为准——防先删文件再删库分次续命)
|
||||
if start := earliestKnownStart(now); start < now {
|
||||
persistTrialStart(start)
|
||||
return
|
||||
}
|
||||
// 本机指纹库:命中则这台机器已消费过试用,直接落一个已过期起点(不重发)
|
||||
fp := hostTrialFingerprint()
|
||||
if fp != "" && fingerprintConsumed(fp) {
|
||||
persistTrialStart(now - int64(trialDays)*86400 - 1)
|
||||
return
|
||||
}
|
||||
persistTrialStart(now)
|
||||
if fp != "" {
|
||||
recordFingerprint(fp)
|
||||
}
|
||||
}
|
||||
|
||||
func trialStartAt() int64 {
|
||||
if start := trialStartFromFile(); start > 0 {
|
||||
return start
|
||||
}
|
||||
return trialStartFromDB()
|
||||
}
|
||||
|
||||
func trialStartFromFile() int64 {
|
||||
cfg := trialDataDir()
|
||||
raw, err := os.ReadFile(cfg + "/.trial-start")
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
v, err := strconv.ParseInt(strings.TrimSpace(string(raw)), 10, 64)
|
||||
if err != nil || v <= 0 {
|
||||
return 0
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func trialStartFromDB() int64 {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return 0
|
||||
}
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
for _, s := range db.Settings {
|
||||
if s.Key == "trial_started_at" {
|
||||
v, err := strconv.ParseInt(s.Value, 10, 64)
|
||||
if err == nil && v > 0 {
|
||||
return v
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// earliestKnownStart 文件与库中较早的起点(都无则返回 now)
|
||||
func earliestKnownStart(now int64) int64 {
|
||||
file := trialStartFromFile()
|
||||
dbv := trialStartFromDB()
|
||||
switch {
|
||||
case file > 0 && dbv > 0:
|
||||
return min64(file, dbv)
|
||||
case file > 0:
|
||||
return file
|
||||
case dbv > 0:
|
||||
return dbv
|
||||
default:
|
||||
return now
|
||||
}
|
||||
}
|
||||
|
||||
func persistTrialStart(start int64) {
|
||||
// 库
|
||||
db := repository.GetDB()
|
||||
if db != nil {
|
||||
db.Lock()
|
||||
exists := false
|
||||
for i := range db.Settings {
|
||||
if db.Settings[i].Key == "trial_started_at" {
|
||||
v, _ := strconv.ParseInt(db.Settings[i].Value, 10, 64)
|
||||
if v == 0 || start < v {
|
||||
db.Settings[i].Value = strconv.FormatInt(start, 10)
|
||||
}
|
||||
exists = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !exists {
|
||||
db.Settings = append(db.Settings, newSetting("trial_started_at", strconv.FormatInt(start, 10)))
|
||||
}
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
}
|
||||
// 文件(0600)
|
||||
dir := trialDataDir()
|
||||
_ = os.MkdirAll(dir, 0o700)
|
||||
_ = os.WriteFile(dir+"/.trial-start", []byte(strconv.FormatInt(start, 10)), 0o600)
|
||||
}
|
||||
|
||||
// hostTrialFingerprint 机器级指纹(与 DATA_DIR 完全无关,删目录/换挂载不变):
|
||||
// machine-id > hostname+cpu。故意不含任何 DATA_DIR 特征——目录 mtime 会因删除/重建漂移,
|
||||
// 含它等于给刷试用留后门。
|
||||
func hostTrialFingerprint() string {
|
||||
if raw, err := os.ReadFile("/etc/machine-id"); err == nil {
|
||||
if id := strings.TrimSpace(string(raw)); len(id) >= 8 {
|
||||
return hashPart(id, "mid")
|
||||
}
|
||||
}
|
||||
host, err := os.Hostname()
|
||||
if err != nil || host == "" {
|
||||
return ""
|
||||
}
|
||||
return hashPart(fmt.Sprintf("%s|%d", host, runtime.NumCPU()), "host")
|
||||
}
|
||||
|
||||
func hashPart(seed, prefix string) string {
|
||||
sum := sha256.Sum256([]byte("lokistack-trial|" + seed))
|
||||
return prefix + "-" + hex.EncodeToString(sum[:10])
|
||||
}
|
||||
|
||||
// 指纹记录:库内 settings.trial_fp_<hash> = 1
|
||||
func fingerprintConsumed(fp string) bool {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return false
|
||||
}
|
||||
key := "trial_fp_" + fp
|
||||
db.RLock()
|
||||
defer db.RUnlock()
|
||||
for _, s := range db.Settings {
|
||||
if s.Key == key && s.Value == "1" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func recordFingerprint(fp string) {
|
||||
db := repository.GetDB()
|
||||
if db == nil {
|
||||
return
|
||||
}
|
||||
key := "trial_fp_" + fp
|
||||
db.Lock()
|
||||
defer db.Unlock()
|
||||
for _, s := range db.Settings {
|
||||
if s.Key == key {
|
||||
return
|
||||
}
|
||||
}
|
||||
db.Settings = append(db.Settings, newSetting(key, "1"))
|
||||
db.RequestSave()
|
||||
}
|
||||
|
||||
func trialDataDir() string {
|
||||
if trialDataDirOverride != "" {
|
||||
return trialDataDirOverride
|
||||
}
|
||||
return config.GetConfig().DataDir
|
||||
}
|
||||
|
||||
// trialDataDirOverride 测试注入(空=用 config.DataDir)
|
||||
var trialDataDirOverride string
|
||||
|
||||
func newSetting(key, value string) models.SystemSetting {
|
||||
db := repository.GetDB()
|
||||
id := uint(0)
|
||||
if db != nil {
|
||||
id = db.NextID("settings")
|
||||
}
|
||||
return models.SystemSetting{ID: id, Key: key, Value: value}
|
||||
}
|
||||
|
||||
func min64(a, b int64) int64 {
|
||||
if a < b {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"lokistack/repository"
|
||||
)
|
||||
|
||||
func trialTestEnv(t *testing.T) string {
|
||||
dir := t.TempDir()
|
||||
t.Setenv("DATA_DIR", dir)
|
||||
t.Setenv("JWT_SECRET", "test-secret-0123456789abcdef")
|
||||
t.Setenv("ENCRYPTION_KEY", "test-encryption-key-0123456789abcdef")
|
||||
if err := repository.InitDB(); err != nil {
|
||||
t.Fatalf("InitDB: %v", err)
|
||||
}
|
||||
prev := trialDataDirOverride
|
||||
trialDataDirOverride = dir
|
||||
t.Cleanup(func() { trialDataDirOverride = prev })
|
||||
return dir
|
||||
}
|
||||
|
||||
// TestTrialFreshInstall 全新安装:试用生效 14 天、库+文件双写
|
||||
func TestTrialFreshInstall(t *testing.T) {
|
||||
dir := trialTestEnv(t)
|
||||
if !TrialActive() {
|
||||
t.Fatal("全新安装试用应生效")
|
||||
}
|
||||
EnsureTrialStarted()
|
||||
if d := TrialRemainingDays(); d != 14 {
|
||||
t.Fatalf("剩余应为 14,got %d", d)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, ".trial-start")); err != nil {
|
||||
t.Fatal("试用起点文件应落盘")
|
||||
}
|
||||
if trialStartFromDB() == 0 {
|
||||
t.Fatal("库内应记录起点")
|
||||
}
|
||||
}
|
||||
|
||||
// TestTrialExpired 试用到期:TrialActive=false、剩余 0
|
||||
func TestTrialExpired(t *testing.T) {
|
||||
dir := trialTestEnv(t)
|
||||
expired := time.Now().Unix() - 15*86400
|
||||
_ = os.WriteFile(filepath.Join(dir, ".trial-start"), []byte(strconv.FormatInt(expired, 10)), 0o600)
|
||||
persistTrialStart(expired)
|
||||
if TrialActive() {
|
||||
t.Fatal("到期后试用不应生效")
|
||||
}
|
||||
if d := TrialRemainingDays(); d != 0 {
|
||||
t.Fatalf("到期剩余应为 0,got %d", d)
|
||||
}
|
||||
if IsPro() {
|
||||
t.Fatal("无授权且试用到期不应是 Pro")
|
||||
}
|
||||
}
|
||||
|
||||
// TestTrialAntiReinstall 删库重装防刷:指纹已消费则不重发试用
|
||||
func TestTrialAntiReinstall(t *testing.T) {
|
||||
dir := trialTestEnv(t)
|
||||
EnsureTrialStarted()
|
||||
// 模拟删库删文件重装:清空时间戳但指纹库保留(指纹写在同库,这里模拟库还在但时间戳被清)
|
||||
db := repository.GetDB()
|
||||
db.Lock()
|
||||
for i := range db.Settings {
|
||||
if db.Settings[i].Key == "trial_started_at" {
|
||||
db.Settings = append(db.Settings[:i], db.Settings[i+1:]...)
|
||||
break
|
||||
}
|
||||
}
|
||||
db.RequestSave()
|
||||
db.Unlock()
|
||||
_ = os.Remove(filepath.Join(dir, ".trial-start"))
|
||||
EnsureTrialStarted()
|
||||
if TrialActive() {
|
||||
t.Fatal("指纹已消费的机器删时间戳重装不应重得试用")
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,33 @@
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# ---- 可选:dist 双架构发布包(linux amd64 + arm64):bash build.sh --dist ----
|
||||
if [ "${1:-}" = "--dist" ]; then
|
||||
GO_CMD="go"
|
||||
BUILD_TIME="$(date -u '+%Y-%m-%d_%H:%M:%S')"
|
||||
GIT_COMMIT="$(git -C "$SCRIPT_DIR" rev-parse --short HEAD 2>/dev/null || echo unknown)"
|
||||
APP_VERSION="${APP_VERSION:-dev}"
|
||||
DIST_DIR="dist"
|
||||
cd "$SCRIPT_DIR"
|
||||
rm -rf "$DIST_DIR" && mkdir -p "$DIST_DIR"
|
||||
for ARCH in amd64 arm64; do
|
||||
NAME="lokistack-linux-${ARCH}"
|
||||
echo "==> Building ${NAME}..."
|
||||
rm -rf "/tmp/${NAME}" && mkdir -p "/tmp/${NAME}"
|
||||
(cd "$SCRIPT_DIR/backend" && CGO_ENABLED=0 GOOS=linux GOARCH=$ARCH $GO_CMD build -trimpath \
|
||||
-ldflags="-s -w -X main.version=${APP_VERSION:-dev} -X main.buildTime=${BUILD_TIME} -X main.gitCommit=${GIT_COMMIT}" \
|
||||
-o "/tmp/${NAME}/lokistack" .)
|
||||
cp -r docs "/tmp/${NAME}/docs"
|
||||
cp -r homepage "/tmp/${NAME}/homepage"
|
||||
cp README.md "/tmp/${NAME}/README.md"
|
||||
tar -czf "${DIST_DIR}/${NAME}.tar.gz" -C /tmp "${NAME}"
|
||||
echo " ${DIST_DIR}/${NAME}.tar.gz ($(du -h ${DIST_DIR}/${NAME}.tar.gz | cut -f1))"
|
||||
done
|
||||
echo "==> dist 完成: ${DIST_DIR}/lokistack-linux-{amd64,arm64}.tar.gz"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "========================================"
|
||||
echo " lokistack Build Script"
|
||||
echo " Security-hardened build pipeline"
|
||||
@@ -113,6 +140,8 @@ echo " Binary size: $(du -h bin/lokistack | cut -f1)"
|
||||
echo " Build time: $BUILD_TIME"
|
||||
echo " Git commit: $GIT_COMMIT"
|
||||
|
||||
|
||||
|
||||
# ---- 3. Copy dist to backend for embedded serving ----
|
||||
echo ""
|
||||
echo "[3/3] Packaging..."
|
||||
|
||||
+4
-6
@@ -53,12 +53,10 @@ services:
|
||||
environment:
|
||||
- SERVER_PORT=8080
|
||||
- DATA_DIR=/app/data
|
||||
# P0: JWT_SECRET must be provided externally
|
||||
# Set via .env file or shell export before running docker compose up.
|
||||
# 注意:错误提示里不要内嵌 $(openssl ...),否则会在 compose 解析时被执行。
|
||||
- "JWT_SECRET=${JWT_SECRET:?JWT_SECRET is required. Generate: openssl rand -hex 32, then: export JWT_SECRET=<value>}"
|
||||
# P0: ENCRYPTION_KEY must be provided externally (AES-256, >=32 字符)
|
||||
- "ENCRYPTION_KEY=${ENCRYPTION_KEY:?ENCRYPTION_KEY is required. Generate: openssl rand -hex 32, then: export ENCRYPTION_KEY=<value>}"
|
||||
# 密钥零输入启动:不设置时 entrypoint 自动生成随机密钥并持久化到数据卷
|
||||
# ($DATA_DIR/.secrets.env);需要固定密钥时在 .env / 环境变量里显式提供。
|
||||
- "JWT_SECRET=${JWT_SECRET:-}"
|
||||
- "ENCRYPTION_KEY=${ENCRYPTION_KEY:-}"
|
||||
# KUBECONFIG points to non-root user's .kube directory
|
||||
- KUBECONFIG=/home/lokistack/.kube/config
|
||||
|
||||
|
||||
@@ -20,6 +20,43 @@ if [ -n "${ENCRYPTION_KEY_FILE:-}" ] && [ -r "$ENCRYPTION_KEY_FILE" ]; then
|
||||
export ENCRYPTION_KEY
|
||||
fi
|
||||
|
||||
# ---- 零输入启动:未显式提供密钥时自动生成并持久化到数据卷 ----
|
||||
# docker compose up -d 即可启动;生成的密钥保存在 $DATA_DIR/.secrets.env
|
||||
# (0600),重启/升级不变。需要固定密钥或接入密钥管理时,显式设置环境变量即可覆盖。
|
||||
AUTO_SECRETS_FILE="${DATA_DIR:-/app/data}/.secrets.env"
|
||||
if [ -z "${JWT_SECRET:-}" ] || [ -z "${ENCRYPTION_KEY:-}" ]; then
|
||||
# 先保存显式传入的值:source 持久化文件时不能覆盖用户显式配置
|
||||
EXPLICIT_JWT="${JWT_SECRET:-}"
|
||||
EXPLICIT_ENC="${ENCRYPTION_KEY:-}"
|
||||
if [ -r "$AUTO_SECRETS_FILE" ]; then
|
||||
# shellcheck disable=SC1090
|
||||
. "$AUTO_SECRETS_FILE"
|
||||
fi
|
||||
[ -n "$EXPLICIT_JWT" ] && JWT_SECRET="$EXPLICIT_JWT"
|
||||
[ -n "$EXPLICIT_ENC" ] && ENCRYPTION_KEY="$EXPLICIT_ENC"
|
||||
CHANGED=0
|
||||
gen_secret() { head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n'; }
|
||||
if [ -z "${JWT_SECRET:-}" ]; then
|
||||
JWT_SECRET="$(gen_secret)"
|
||||
CHANGED=1
|
||||
echo "==> JWT_SECRET not provided; generated and persisted to $AUTO_SECRETS_FILE"
|
||||
fi
|
||||
if [ -z "${ENCRYPTION_KEY:-}" ]; then
|
||||
ENCRYPTION_KEY="$(gen_secret)"
|
||||
CHANGED=1
|
||||
echo "==> ENCRYPTION_KEY not provided; generated and persisted to $AUTO_SECRETS_FILE"
|
||||
fi
|
||||
export JWT_SECRET ENCRYPTION_KEY
|
||||
if [ "$CHANGED" = "1" ]; then
|
||||
umask 077
|
||||
{
|
||||
echo "# Auto-generated by lokistack entrypoint. Remove to regenerate."
|
||||
echo "JWT_SECRET=$JWT_SECRET"
|
||||
echo "ENCRYPTION_KEY=$ENCRYPTION_KEY"
|
||||
} > "$AUTO_SECRETS_FILE"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- Runtime security validation ----
|
||||
MISSING_VARS=()
|
||||
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
/* OpsxAPI 文档站 —— 清爽专业风格(浅色) */
|
||||
:root {
|
||||
--blue: #059669;
|
||||
--blue-dark: #047857;
|
||||
--blue-soft: rgba(5, 150, 105, 0.08);
|
||||
--bg: #f6f8fb;
|
||||
--panel: #ffffff;
|
||||
--border: #e4e9f1;
|
||||
--text: #1c2434;
|
||||
--text-2: #5b6880;
|
||||
--text-3: #98a2b3;
|
||||
--code-bg: #f3f6fb;
|
||||
--code-border: #e1e8f3;
|
||||
--green: #059669;
|
||||
--amber: #b45309;
|
||||
--red: #dc2626;
|
||||
--sidebar-w: 262px;
|
||||
--toc-w: 220px;
|
||||
--topbar-h: 56px;
|
||||
--radius: 10px;
|
||||
--shadow: 0 1px 3px rgba(16, 24, 40, 0.06), 0 8px 24px rgba(16, 24, 40, 0.05);
|
||||
--mono: "JetBrains Mono", SFMono-Regular, Menlo, Consolas, monospace;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
html { scroll-behavior: smooth; scroll-padding-top: 72px; }
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: -apple-system, BlinkMacSystemFont, "PingFang SC", "Microsoft YaHei", "Segoe UI", sans-serif;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
font-size: 14.5px;
|
||||
line-height: 1.75;
|
||||
}
|
||||
a { color: var(--blue); text-decoration: none; }
|
||||
.i { width: 15px; height: 15px; stroke: currentColor; fill: none; stroke-width: 2; stroke-linecap: round; stroke-linejoin: round; vertical-align: -2px; }
|
||||
a:hover { color: var(--blue-dark); text-decoration: underline; }
|
||||
|
||||
/* ---------- 顶栏 ---------- */
|
||||
.topbar {
|
||||
position: sticky; top: 0; z-index: 50;
|
||||
background: rgba(255, 255, 255, 0.92);
|
||||
backdrop-filter: saturate(1.4) blur(8px);
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.topbar-inner {
|
||||
max-width: 1440px; margin: 0 auto; height: var(--topbar-h);
|
||||
display: flex; align-items: center; gap: 14px; padding: 0 20px;
|
||||
}
|
||||
.menu-btn {
|
||||
display: none; border: 1px solid var(--border); background: #fff;
|
||||
border-radius: 8px; width: 34px; height: 34px; cursor: pointer; color: var(--text-2);
|
||||
place-items: center;
|
||||
}
|
||||
.menu-btn .i { width: 17px; height: 17px; }
|
||||
.brand { display: flex; align-items: center; gap: 9px; text-decoration: none !important; }
|
||||
.brand-logo { display: inline-flex; }
|
||||
.brand-logo svg { display: block; border-radius: 7px; }
|
||||
.brand-name { font-size: 16px; font-weight: 700; color: var(--text); }
|
||||
.brand-name em { font-style: normal; font-weight: 500; color: var(--text-2); margin-left: 2px; }
|
||||
.ver-badge {
|
||||
font-size: 11px; font-weight: 600; color: var(--blue);
|
||||
background: var(--blue-soft); border: 1px solid rgba(5, 150, 105, 0.22);
|
||||
padding: 1px 9px; border-radius: 999px; white-space: nowrap;
|
||||
}
|
||||
.search { position: relative; flex: 1; max-width: 420px; margin-left: 8px; }
|
||||
.search input {
|
||||
width: 100%; height: 36px; border: 1px solid var(--border); border-radius: 9px;
|
||||
background: var(--bg); padding: 0 14px 0 34px; font-size: 13.5px; color: var(--text); outline: none;
|
||||
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='14' height='14' fill='none' stroke='%2398a2b3' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='11' cy='11' r='7'/%3E%3Cpath d='m20 20-3.5-3.5'/%3E%3C/svg%3E");
|
||||
background-repeat: no-repeat; background-position: 12px center;
|
||||
}
|
||||
.search input:focus { border-color: var(--blue); background: #fff; box-shadow: 0 0 0 3px var(--blue-soft); }
|
||||
.search-panel {
|
||||
position: absolute; top: 44px; left: 0; right: 0; background: #fff;
|
||||
border: 1px solid var(--border); border-radius: var(--radius); box-shadow: var(--shadow);
|
||||
max-height: 420px; overflow: auto; padding: 6px;
|
||||
}
|
||||
.search-item { display: block; padding: 8px 12px; border-radius: 8px; text-decoration: none !important; }
|
||||
.search-item:hover, .search-item.sel { background: var(--blue-soft); }
|
||||
.search-item .si-title { font-weight: 600; color: var(--text); font-size: 13.5px; }
|
||||
.search-item .si-group { color: var(--text-3); font-size: 11px; margin-left: 6px; }
|
||||
.search-item .si-snippet { color: var(--text-2); font-size: 12px; margin-top: 2px; display: block; }
|
||||
.search-empty { padding: 14px; text-align: center; color: var(--text-3); font-size: 13px; }
|
||||
.top-links { margin-left: auto; display: flex; gap: 16px; align-items: center; }
|
||||
.top-links a { font-size: 13.5px; font-weight: 500; }
|
||||
|
||||
/* ---------- 三栏布局 ---------- */
|
||||
.layout {
|
||||
max-width: 1440px; margin: 0 auto; display: grid;
|
||||
grid-template-columns: var(--sidebar-w) minmax(0, 1fr) var(--toc-w);
|
||||
gap: 0 28px; padding: 24px 20px 60px;
|
||||
}
|
||||
|
||||
/* 侧栏 */
|
||||
.sidebar { position: sticky; top: calc(var(--topbar-h) + 24px); align-self: start; max-height: calc(100vh - var(--topbar-h) - 48px); overflow-y: auto; padding-right: 8px; }
|
||||
.sidebar::-webkit-scrollbar { width: 4px; }
|
||||
.sidebar::-webkit-scrollbar-thumb { background: #d6dde8; border-radius: 4px; }
|
||||
.nav-group { margin-bottom: 18px; }
|
||||
.nav-group-title {
|
||||
font-size: 11.5px; font-weight: 700; letter-spacing: 0.08em; text-transform: uppercase;
|
||||
color: var(--text-3); padding: 0 10px 6px;
|
||||
}
|
||||
.nav-link {
|
||||
display: block; padding: 6px 10px; border-radius: 8px; font-size: 13.5px;
|
||||
color: var(--text-2); text-decoration: none !important; border-left: 2px solid transparent;
|
||||
}
|
||||
.nav-link:hover { background: rgba(5, 150, 105, 0.06); color: var(--text); }
|
||||
.nav-link.active {
|
||||
background: var(--blue-soft); color: var(--blue-dark); font-weight: 600;
|
||||
border-left-color: var(--blue);
|
||||
}
|
||||
.sidebar-foot { margin-top: 24px; padding: 12px 10px 0; border-top: 1px solid var(--border); font-size: 11.5px; color: var(--text-3); }
|
||||
|
||||
/* 内容 */
|
||||
.content { min-width: 0; max-width: 860px; }
|
||||
.breadcrumb { font-size: 12.5px; color: var(--text-3); margin-bottom: 10px; }
|
||||
.breadcrumb span { margin: 0 6px; }
|
||||
.page-title { font-size: 30px; font-weight: 700; letter-spacing: -0.02em; margin: 0 0 6px; line-height: 1.3; }
|
||||
.page-meta { font-size: 12.5px; color: var(--text-3); margin-bottom: 22px; }
|
||||
|
||||
/* Markdown 排版 */
|
||||
.md h2 { font-size: 21px; font-weight: 700; margin: 38px 0 12px; padding-top: 8px; letter-spacing: -0.01em; }
|
||||
.md h3 { font-size: 16.5px; font-weight: 700; margin: 26px 0 8px; }
|
||||
.md h2 + h3, .md h1 + h2 { margin-top: 18px; }
|
||||
.md p { margin: 10px 0; color: #2a3348; }
|
||||
.md ul, .md ol { padding-left: 22px; margin: 10px 0; }
|
||||
.md li { margin: 5px 0; color: #2a3348; }
|
||||
.md li::marker { color: var(--blue); }
|
||||
.md strong { color: var(--text); }
|
||||
.md hr { border: none; border-top: 1px solid var(--border); margin: 30px 0; }
|
||||
.md img { max-width: 100%; border-radius: var(--radius); border: 1px solid var(--border); }
|
||||
.md blockquote {
|
||||
margin: 14px 0; padding: 10px 16px; border-left: 3px solid var(--blue);
|
||||
background: var(--blue-soft); border-radius: 0 8px 8px 0; color: #33415e;
|
||||
}
|
||||
|
||||
/* 表格 */
|
||||
.md table { border-collapse: collapse; width: 100%; margin: 16px 0; font-size: 13.5px; background: #fff; border-radius: var(--radius); overflow: hidden; box-shadow: 0 0 0 1px var(--border); }
|
||||
.md th, .md td { padding: 9px 14px; text-align: left; border-bottom: 1px solid var(--border); vertical-align: top; }
|
||||
.md th { background: #f9fbfe; font-weight: 600; color: var(--text); white-space: nowrap; }
|
||||
.md tr:last-child td { border-bottom: none; }
|
||||
|
||||
/* 代码块 */
|
||||
.md code { font-family: var(--mono); font-size: 12.8px; background: var(--code-bg); border: 1px solid var(--code-border); border-radius: 5px; padding: 1px 6px; color: #334155; }
|
||||
.md pre {
|
||||
position: relative; background: #0f172a; color: #dbe4f3; border-radius: var(--radius);
|
||||
padding: 16px 16px 14px; overflow-x: auto; margin: 16px 0; font-size: 13px; line-height: 1.65;
|
||||
}
|
||||
.md pre code { background: none; border: none; color: inherit; padding: 0; font-size: inherit; }
|
||||
.md pre .lang-tag {
|
||||
position: absolute; top: 8px; right: 12px; font-size: 10.5px; color: #7c8db0;
|
||||
font-family: var(--mono); text-transform: lowercase; letter-spacing: 0.04em;
|
||||
}
|
||||
.md pre .copy-btn {
|
||||
position: absolute; top: 6px; right: 60px; font-size: 11px; color: #9fb0d0;
|
||||
background: rgba(255, 255, 255, 0.07); border: none; border-radius: 6px; padding: 2px 9px; cursor: pointer;
|
||||
opacity: 0; transition: opacity 0.15s;
|
||||
}
|
||||
.md pre:hover .copy-btn { opacity: 1; }
|
||||
.md pre .copy-btn:hover { color: #fff; background: rgba(255, 255, 255, 0.14); }
|
||||
/* 行内高亮 token */
|
||||
.md pre .tok-key { color: #7dd3fc; }
|
||||
.md pre .tok-str { color: #86efac; }
|
||||
.md pre .tok-num { color: #fcd34d; }
|
||||
.md pre .tok-com { color: #64748b; font-style: italic; }
|
||||
|
||||
/* 提示块(通过 blockquote 首字符约定) */
|
||||
.md blockquote.tip { border-left-color: var(--green); background: rgba(5, 150, 105, 0.07); }
|
||||
.md blockquote.warn { border-left-color: var(--amber); background: rgba(180, 83, 9, 0.07); }
|
||||
|
||||
/* 上下页 */
|
||||
.pn-nav { display: flex; gap: 14px; margin-top: 48px; }
|
||||
.pn {
|
||||
flex: 1; border: 1px solid var(--border); background: #fff; border-radius: var(--radius);
|
||||
padding: 13px 16px; text-decoration: none !important; transition: border-color 0.15s, box-shadow 0.15s;
|
||||
}
|
||||
.pn:hover { border-color: var(--blue); box-shadow: var(--shadow); }
|
||||
.pn.next { text-align: right; }
|
||||
.pn-label { display: block; font-size: 11.5px; color: var(--text-3); }
|
||||
.pn-title { display: block; font-weight: 600; color: var(--text); font-size: 14px; margin-top: 2px; }
|
||||
|
||||
/* 右侧目录 */
|
||||
.toc { position: sticky; top: calc(var(--topbar-h) + 24px); align-self: start; max-height: calc(100vh - var(--topbar-h) - 48px); overflow-y: auto; padding-top: 34px; }
|
||||
.toc-title { font-size: 11.5px; font-weight: 700; letter-spacing: 0.08em; color: var(--text-3); margin-bottom: 8px; }
|
||||
.toc-nav { display: flex; flex-direction: column; gap: 1px; border-left: 2px solid var(--border); padding-left: 0; }
|
||||
.toc-nav a {
|
||||
font-size: 12.5px; color: var(--text-2); padding: 4px 0 4px 12px; text-decoration: none !important;
|
||||
border-left: 2px solid transparent; margin-left: -2px; line-height: 1.5;
|
||||
}
|
||||
.toc-nav a:hover { color: var(--blue); }
|
||||
.toc-nav a.toc-l3 { padding-left: 26px; font-size: 12px; color: var(--text-3); }
|
||||
.toc-nav a.active { color: var(--blue); border-left-color: var(--blue); font-weight: 600; }
|
||||
/* 404 */
|
||||
.doc-404 { display: grid; place-items: center; min-height: 100vh; text-align: center; }
|
||||
.doc-404 h1 { font-size: 56px; margin: 0; color: var(--blue); }
|
||||
.doc-404 .btn {
|
||||
display: inline-block; margin-top: 12px; background: var(--blue); color: #fff;
|
||||
padding: 8px 18px; border-radius: 8px; text-decoration: none !important; font-weight: 600;
|
||||
}
|
||||
|
||||
/* ---------- 响应式 ---------- */
|
||||
@media (max-width: 1200px) {
|
||||
.layout { grid-template-columns: var(--sidebar-w) minmax(0, 1fr); }
|
||||
.toc { display: none; }
|
||||
}
|
||||
@media (max-width: 880px) {
|
||||
.menu-btn { display: inline-grid; place-items: center; }
|
||||
.layout { grid-template-columns: minmax(0, 1fr); padding-top: 12px; }
|
||||
.sidebar {
|
||||
position: fixed; z-index: 60; left: 0; top: var(--topbar-h); bottom: 0;
|
||||
width: 280px; background: #fff; border-right: 1px solid var(--border);
|
||||
padding: 16px; transform: translateX(-100%); transition: transform 0.2s; max-height: none;
|
||||
}
|
||||
.sidebar.open { transform: none; box-shadow: var(--shadow); }
|
||||
.ver-badge { display: none; }
|
||||
.top-links a { font-size: 12.5px; }
|
||||
.page-title { font-size: 24px; }
|
||||
}
|
||||
|
||||
/* ---------- 打印 ---------- */
|
||||
@media print {
|
||||
.topbar, .sidebar, .toc, .pn-nav, .feedback, .menu-btn { display: none !important; }
|
||||
.layout { display: block; max-width: none; padding: 0; }
|
||||
body { background: #fff; }
|
||||
.md pre { background: #f1f5f9 !important; color: #1e293b !important; }
|
||||
.md pre .copy-btn, .md pre .lang-tag { display: none; }
|
||||
}
|
||||
|
||||
/* ---------- LokiStack 品牌适配 ---------- */
|
||||
.brand-logo {
|
||||
display: inline-flex; align-items: center; justify-content: center;
|
||||
width: 28px; height: 28px; border-radius: 8px;
|
||||
background: linear-gradient(135deg, #10b981, #047857);
|
||||
color: #fff; font-weight: 800; font-size: 17px;
|
||||
}
|
||||
|
||||
/* 响应式(窄屏收起侧栏与目录) */
|
||||
@media (max-width: 1100px) {
|
||||
.layout { grid-template-columns: var(--sidebar-w) minmax(0, 1fr); }
|
||||
.toc { display: none; }
|
||||
}
|
||||
@media (max-width: 860px) {
|
||||
.layout { grid-template-columns: minmax(0, 1fr); padding: 16px 14px 40px; }
|
||||
.menu-btn { display: grid; }
|
||||
.sidebar {
|
||||
position: fixed; z-index: 60; top: var(--topbar-h); left: 0; bottom: 0;
|
||||
width: var(--sidebar-w); background: #fff; border-right: 1px solid var(--border);
|
||||
transform: translateX(-100%); transition: transform 0.2s; max-height: none; padding: 14px 10px;
|
||||
}
|
||||
.sidebar.open { transform: none; box-shadow: var(--shadow); }
|
||||
.search { max-width: none; }
|
||||
.top-links { display: none; }
|
||||
}
|
||||
|
||||
.pn-empty { flex: 1; }
|
||||
@@ -0,0 +1,196 @@
|
||||
/* LokiStack 文档站交互:搜索 / 代码复制 / 目录高亮 / 移动端导航(参考 opsxapi 体系) */
|
||||
(function () {
|
||||
"use strict";
|
||||
|
||||
/* ---------- 代码块:语言标签 + 复制按钮 + 轻量高亮 ---------- */
|
||||
document.querySelectorAll("article.md pre").forEach(function (pre) {
|
||||
var code = pre.querySelector("code");
|
||||
if (!code) return;
|
||||
var lang = (code.className.match(/language-([\w-]+)/) || [, ""])[1];
|
||||
if (lang) {
|
||||
var tag = document.createElement("span");
|
||||
tag.className = "lang-tag";
|
||||
tag.textContent = lang;
|
||||
pre.appendChild(tag);
|
||||
}
|
||||
// 复制按钮
|
||||
var btn = document.createElement("button");
|
||||
btn.className = "copy-btn";
|
||||
btn.type = "button";
|
||||
btn.textContent = "复制";
|
||||
btn.addEventListener("click", function () {
|
||||
navigator.clipboard.writeText(code.textContent).then(
|
||||
function () {
|
||||
btn.textContent = "已复制 ✓";
|
||||
setTimeout(function () { btn.textContent = "复制"; }, 1400);
|
||||
},
|
||||
function () { btn.textContent = "复制失败"; }
|
||||
);
|
||||
});
|
||||
pre.appendChild(btn);
|
||||
// JSON / bash 轻量高亮
|
||||
if (lang === "json") highlightJSON(code);
|
||||
else if (lang === "bash" || lang === "sh" || lang === "shell") highlightBash(code);
|
||||
});
|
||||
|
||||
function esc(s) {
|
||||
return s.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
||||
}
|
||||
function highlightJSON(code) {
|
||||
var src = code.textContent;
|
||||
code.innerHTML = esc(src).replace(
|
||||
/("(?:\\.|[^"\\])*")(\s*:)?|\b(true|false|null)\b|(-?\d+(?:\.\d+)?(?:[eE][+-]?\d+)?)/g,
|
||||
function (m, str, colon, kw, num) {
|
||||
if (str && colon) return '<span class="tok-key">' + str + "</span>" + colon;
|
||||
if (str) return '<span class="tok-str">' + str + "</span>";
|
||||
if (kw) return '<span class="tok-num">' + kw + "</span>";
|
||||
if (num) return '<span class="tok-num">' + num + "</span>";
|
||||
return m;
|
||||
}
|
||||
);
|
||||
}
|
||||
function highlightBash(code) {
|
||||
var lines = code.textContent.split("\n");
|
||||
code.innerHTML = lines
|
||||
.map(function (line) {
|
||||
if (/^\s*#/.test(line)) return '<span class="tok-com">' + esc(line) + "</span>";
|
||||
return esc(line).replace(/(^|\s)(-{1,2}[\w-]+)/g, '$1<span class="tok-key">$2</span>');
|
||||
})
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
/* ---------- blockquote 提示样式约定:> **提示** / > **注意** ---------- */
|
||||
document.querySelectorAll("article.md blockquote").forEach(function (bq) {
|
||||
var t = (bq.textContent || "").trim();
|
||||
if (/^(注意|警告|⚠)/.test(t)) bq.classList.add("warn");
|
||||
else if (/^(提示|说明|💡|Tip)/i.test(t)) bq.classList.add("tip");
|
||||
});
|
||||
|
||||
/* ---------- 搜索 ---------- */
|
||||
var input = document.getElementById("search-input");
|
||||
var panel = document.getElementById("search-panel");
|
||||
var entries = null;
|
||||
var selIndex = -1;
|
||||
|
||||
function ensureIndex(cb) {
|
||||
if (entries) return cb();
|
||||
fetch("/docs/search-data.json")
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (data) { entries = data; cb(); })
|
||||
.catch(function () { entries = []; cb(); });
|
||||
}
|
||||
|
||||
function snippet(text, q) {
|
||||
var i = text.toLowerCase().indexOf(q);
|
||||
if (i < 0) return text.slice(0, 60) + "…";
|
||||
var start = Math.max(0, i - 24);
|
||||
return (start > 0 ? "…" : "") + text.slice(start, i + q.length + 40) + "…";
|
||||
}
|
||||
|
||||
function doSearch() {
|
||||
var q = input.value.trim().toLowerCase();
|
||||
selIndex = -1;
|
||||
if (!q) { panel.hidden = true; return; }
|
||||
ensureIndex(function () {
|
||||
var hits = [];
|
||||
entries.forEach(function (e) {
|
||||
var titleHit = e.title.toLowerCase().indexOf(q) >= 0;
|
||||
var idx = e.text.toLowerCase().indexOf(q);
|
||||
if (titleHit || idx >= 0) {
|
||||
hits.push({
|
||||
slug: e.slug,
|
||||
title: e.title,
|
||||
group: e.group,
|
||||
score: (titleHit ? 10 : 0) + (idx === 0 ? 3 : 0),
|
||||
snippet: titleHit ? snippet(e.text, q) : snippet(e.text, q),
|
||||
});
|
||||
}
|
||||
});
|
||||
hits.sort(function (a, b) { return b.score - a.score; });
|
||||
hits = hits.slice(0, 8);
|
||||
if (!hits.length) {
|
||||
panel.innerHTML = '<div class="search-empty">没有找到相关内容,换个关键词试试</div>';
|
||||
} else {
|
||||
panel.innerHTML = hits
|
||||
.map(function (h) {
|
||||
return (
|
||||
'<a class="search-item" href="/docs/' + h.slug + '">' +
|
||||
'<span class="si-title">' + h.title + '</span><span class="si-group">' + h.group + "</span>" +
|
||||
'<span class="si-snippet">' + h.snippet + "</span></a>"
|
||||
);
|
||||
})
|
||||
.join("");
|
||||
}
|
||||
panel.hidden = false;
|
||||
});
|
||||
}
|
||||
|
||||
var debounce;
|
||||
if (input) {
|
||||
input.addEventListener("input", function () {
|
||||
clearTimeout(debounce);
|
||||
debounce = setTimeout(doSearch, 160);
|
||||
});
|
||||
input.addEventListener("focus", function () {
|
||||
if (input.value.trim()) doSearch();
|
||||
});
|
||||
input.addEventListener("keydown", function (e) {
|
||||
var items = panel.querySelectorAll(".search-item");
|
||||
if (e.key === "ArrowDown" && items.length) {
|
||||
e.preventDefault();
|
||||
selIndex = Math.min(selIndex + 1, items.length - 1);
|
||||
} else if (e.key === "ArrowUp" && items.length) {
|
||||
e.preventDefault();
|
||||
selIndex = Math.max(selIndex - 1, 0);
|
||||
} else if (e.key === "Enter") {
|
||||
if (selIndex >= 0 && items[selIndex]) { location.href = items[selIndex].href; return; }
|
||||
var first = items[0];
|
||||
if (first) location.href = first.href;
|
||||
return;
|
||||
} else if (e.key === "Escape") {
|
||||
panel.hidden = true;
|
||||
return;
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
items.forEach(function (el, i) { el.classList.toggle("sel", i === selIndex); });
|
||||
});
|
||||
document.addEventListener("click", function (e) {
|
||||
if (!e.target.closest(".search")) panel.hidden = true;
|
||||
});
|
||||
}
|
||||
|
||||
/* ---------- 目录滚动高亮 ---------- */
|
||||
var tocLinks = Array.prototype.slice.call(document.querySelectorAll(".toc-nav a"));
|
||||
if (tocLinks.length && "IntersectionObserver" in window) {
|
||||
var byId = {};
|
||||
tocLinks.forEach(function (a) { byId[a.getAttribute("href").slice(1)] = a; });
|
||||
var io = new IntersectionObserver(
|
||||
function (entries) {
|
||||
entries.forEach(function (en) {
|
||||
if (en.isIntersecting) {
|
||||
tocLinks.forEach(function (a) { a.classList.remove("active"); });
|
||||
var a = byId[en.target.id];
|
||||
if (a) a.classList.add("active");
|
||||
}
|
||||
});
|
||||
},
|
||||
{ rootMargin: "-72px 0px -70% 0px", threshold: 0 }
|
||||
);
|
||||
document.querySelectorAll("article.md h2[id], article.md h3[id]").forEach(function (h) { io.observe(h); });
|
||||
}
|
||||
|
||||
|
||||
|
||||
/* ---------- 移动端侧栏 ---------- */
|
||||
var menuBtn = document.getElementById("menu-btn");
|
||||
var sidebar = document.getElementById("sidebar");
|
||||
if (menuBtn && sidebar) {
|
||||
menuBtn.addEventListener("click", function () { sidebar.classList.toggle("open"); });
|
||||
document.addEventListener("click", function (e) {
|
||||
if (sidebar.classList.contains("open") && !e.target.closest(".sidebar") && !e.target.closest(".menu-btn")) {
|
||||
sidebar.classList.remove("open");
|
||||
}
|
||||
});
|
||||
}
|
||||
})();
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 275 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 187 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 115 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 291 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 168 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 184 KiB |
@@ -193,7 +193,7 @@ metadata:
|
||||
namespace: log
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["pods", "pods/log", "services", "configmaps", "secrets", "persistentvolumeclaims", "events", "serviceaccounts"]
|
||||
resources: ["pods", "pods/log", "services", "services/proxy", "configmaps", "secrets", "persistentvolumeclaims", "events", "serviceaccounts"]
|
||||
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
|
||||
- apiGroups: ["apps"]
|
||||
resources: ["statefulsets", "daemonsets", "deployments"]
|
||||
@@ -236,3 +236,25 @@ kubectl -n log get secret lokistack-token -o jsonpath='{.data.token}' | base64 -
|
||||
- 如果集群策略要求短期凭证,可改用 `kubectl -n log create token lokistack --duration=24h` 定期轮换。
|
||||
- 上面的 Namespace 由管理员创建,平台 Token 本身没有集群级权限;只有确实需要平台自建命名空间时,才额外授予 `namespaces: create/get` 的 ClusterRole(create 无法按名称限制,预建 `log` 更安全)。
|
||||
|
||||
|
||||
## 10. 数据源连接方式(部署后自动探测)
|
||||
|
||||
平台自动注册的数据源不再按部署模式硬编码,部署完成后会实测两种方式:
|
||||
|
||||
1. **cluster-svc**:经 kube-apiserver 的 service proxy 访问
|
||||
`/<namespace>/services/loki:3100/proxy`,不依赖集群 DNS / ClusterIP 直通;
|
||||
2. **url**:直接访问 Loki 地址(Service DNS、主机 IP:3100 或 Ingress 域名)。
|
||||
|
||||
先试 cluster-svc,失败再试 url,选通的写入数据源。
|
||||
|
||||
| 平台位置 | Loki 位置 | 实际可用方式 |
|
||||
|---|---|---|
|
||||
| 集群内 | 集群内 | cluster-svc |
|
||||
| 主机 | 集群内 | cluster-svc(平台能访问 apiserver:6443 时);否则 url(NodePort/Ingress) |
|
||||
| 集群内 | 主机 | url(主机 IP:3100,需网络可达) |
|
||||
| 主机 | 主机 | url |
|
||||
|
||||
注意:
|
||||
- `cluster-svc` 需要凭据具备 `services/proxy` 权限(本文第 9 节的 Role 已包含);
|
||||
- 平台与 API Server / Loki 之间的网络必须放行;跨 VPC 时走 VPN、专线或公网 Ingress;
|
||||
- 自动探测只决定“连接方式”,不会隐藏数据源;连接失败时“测试连接”会返回具体错误。
|
||||
@@ -0,0 +1,84 @@
|
||||
# LokiStack × opsx-licence 对接说明
|
||||
|
||||
## 状态
|
||||
|
||||
- 代码:双路径已接入(`LIC-`/`LICOFF-` → opsx-licence;旧 HMAC 迁移窗口内仍接受并提示)。
|
||||
- 生产 product **`lokistack` 尚未在 licence.rlwj.cn 创建**(公开 `GET /api/v1/products/code/lokistack` 返回「产品不存在」)。
|
||||
- 默认 API:`LICENSE_API_BASE=https://licence.rlwj.cn/api`(可用环境变量覆盖;`l123` 仅网络失败备灾)。
|
||||
|
||||
## Plan(产品锁)
|
||||
|
||||
| plan_type | 时长 | 实例(max_devices) |
|
||||
|-----------|------|---------------------|
|
||||
| trial | 14 天 | 1 |
|
||||
| yearly | 365 天 | 默认 1,可加购 |
|
||||
| perpetual | 0(永久) | 默认 1,可加购 |
|
||||
|
||||
计费按**部署实例**,不计登录席位。`instance_id` = `DATA_DIR/instance.id` 机器指纹,无 PII。
|
||||
|
||||
## 管理端创建 product(需管理员登录,本说明不含账密)
|
||||
|
||||
1. 打开 `https://licence.rlwj.cn` 管理后台并登录。
|
||||
2. **产品** → 新建:
|
||||
- name: `LokiStack`
|
||||
- code: `lokistack`(必须完全一致)
|
||||
- features: `log_analysis`, `log_alerts`, `ai_ops`, `audit_log`
|
||||
- price_model: `perpetual`(或按需)
|
||||
- status: 启用
|
||||
3. **定价计划** → 为该产品新建三条:
|
||||
- trial / 14 / max_devices=1 / price=0
|
||||
- yearly / 365 / max_devices=1
|
||||
- perpetual / 0 / max_devices=1
|
||||
4. **许可证** → 选客户 + 产品 `lokistack` 签发 `LIC-`;离线场景签发 `LICOFF-`(须绑定目标实例的 `instance_id`)。
|
||||
|
||||
### 等价 Admin API(需 Bearer;勿把 token 写进仓)
|
||||
|
||||
```bash
|
||||
# 创建产品
|
||||
curl -sS -X POST 'https://licence.rlwj.cn/api/v1/products' \
|
||||
-H "Authorization: Bearer <ADMIN_TOKEN>" -H 'Content-Type: application/json' \
|
||||
-d '{"name":"LokiStack","code":"lokistack","version":"1.0.0","description":"LokiStack Pro","features":["log_analysis","log_alerts","ai_ops","audit_log"],"price":0,"price_model":"perpetual","status":1}'
|
||||
|
||||
# 创建计划(替换 PRODUCT_ID)
|
||||
curl -sS -X POST 'https://licence.rlwj.cn/api/v1/plans' \
|
||||
-H "Authorization: Bearer <ADMIN_TOKEN>" -H 'Content-Type: application/json' \
|
||||
-d '{"product_id":PRODUCT_ID,"name":"体验","plan_type":"trial","price":0,"currency":"CNY","duration":14,"max_devices":1,"features":["Pro","体验","1 实例"],"sort_order":0,"status":1}'
|
||||
# yearly duration=365;perpetual duration=0 —— 同结构各建一条
|
||||
```
|
||||
|
||||
公开校验:
|
||||
|
||||
```bash
|
||||
curl -sS 'https://licence.rlwj.cn/api/v1/products/code/lokistack'
|
||||
```
|
||||
|
||||
## 旧 HMAC 迁移
|
||||
|
||||
- 已发出的本地 HMAC 码:**迁移窗口内继续可用**,设置页黄条提示换发;**禁止静默清码**。
|
||||
- 换发:后台为客户建 `lokistack` 许可证 → 客户在设置页粘贴新 `LIC-`/`LICOFF-` 覆盖激活。
|
||||
- 窗口结束后再关 HMAC 路径(另开刀,需产品点头)。
|
||||
|
||||
## 环境变量
|
||||
|
||||
| 变量 | 含义 |
|
||||
|------|------|
|
||||
| `LICENSE_API_BASE` 或 `OPSX_LICENSE_API_BASE` | 覆盖默认 `https://licence.rlwj.cn/api` |
|
||||
|
||||
## 对齐 Shell 的差异
|
||||
|
||||
| | OpsxShell | LokiStack |
|
||||
|--|-----------|-----------|
|
||||
| product_code | `opsxshell` | `lokistack` |
|
||||
| 绑定字段 | `device_id` | `instance_id`(同语义) |
|
||||
| 默认 Base | l123 主 / 国内备 | **国内主** / l123 备 |
|
||||
| 落盘 | `license.opsx` | JsonDB `license` 字段 |
|
||||
| 退出授权 | `clear` → remote deactivate + 删本地 | 设置→授权「退出授权」→ `POST /system/license/deactivate`;在线成功才清本地;LICOFF- 仅解本机并提示后台名额可能仍占 |
|
||||
|
||||
## 退出授权(本机)
|
||||
|
||||
- UI:设置 → 授权,Pro 态(含宽限仍 pro)管理员可见「退出授权」,二次确认;免费/未激活不渲染。成功后只重拉授权态,不整页刷新。
|
||||
- 后端:`POST /api/v1/system/license/deactivate`(Admin)。
|
||||
- 在线 `LIC-`:调用 `POST {LICENSE_API_BASE}/v1/license/deactivate`,body 含 `license_key`、`device_id`(=instance_id)、`activation_token`(若有);**成功才清本地**,失败保留现状并报错。
|
||||
- 离线 `LICOFF-`:只清本地,响应 `offlineLocalOnly=true`,提示后台名额可能仍占。
|
||||
- 旧 HMAC:只清本地。
|
||||
- 禁止在周检/校验失败路径静默清码;仅用户确认后走本接口。
|
||||
@@ -0,0 +1,26 @@
|
||||
# LokiStack 用户手册
|
||||
|
||||
LokiStack 把 Grafana Loki 日志聚合的部署、采集、查询、分析与告警收进一个平台:向导式部署到 Kubernetes 集群或裸金属主机,内置中文/英文界面,单二进制交付。
|
||||
|
||||
## 平台能做什么
|
||||
|
||||
- **部署 Loki**:三步向导(环境与目标 → 存储 → 部署与采集),支持 Kubernetes(YAML 直发)、裸金属二进制、主机 Docker 三种模式,不需要在平台所在机器安装 kubectl 或 Helm。
|
||||
- **采集日志**:Promtail / Fluent Bit 采集策略可视化配置,命名空间多选、app 标签来源四级回退、敏感路径拦截。
|
||||
- **查询与分析**:LogQL 高亮补全、快捷查询、Live Tail;分析页自动聚合日志量、错误与 Ingress 访问指标。
|
||||
- **告警**:内置九条应用日志规则,绑定默认通道后填入飞书 Webhook 即可接收;支持冷却与风暴抑制。
|
||||
- **运维底座**:凭证加密存储、操作审计、标准 Prometheus 指标端点(/api/v1/metrics)。
|
||||
|
||||
## 界面预览
|
||||
|
||||
<div class="shot-grid">
|
||||
<img src="/docs/assets/screenshots/dashboard.png" alt="仪表盘" />
|
||||
<img src="/docs/assets/screenshots/wizard.png" alt="部署向导" />
|
||||
<img src="/docs/assets/screenshots/logs-query.png" alt="日志查询" />
|
||||
<img src="/docs/assets/screenshots/analysis.png" alt="日志分析" />
|
||||
<img src="/docs/assets/screenshots/alerts.png" alt="告警通知" />
|
||||
<img src="/docs/assets/screenshots/settings.png" alt="系统设置" />
|
||||
</div>
|
||||
|
||||
## 下一步
|
||||
|
||||
阅读[快速安装](install.md),或直接进入[部署向导](wizard.md)。
|
||||
@@ -0,0 +1,10 @@
|
||||
# 查询授权
|
||||
|
||||
管理员在「日志查询 → 查询授权」用策略组控制普通用户可见的日志范围:
|
||||
|
||||
- 策略组 = 数据源 + 允许的标签范围(命名空间 / App / Pod,均精确匹配,未选不限) + 关联用户。
|
||||
- 同一用户在同一数据源只能属于一个策略组;已在其他组的用户置灰。
|
||||
- 支持快捷的「所有人」策略(默认策略的常用形态)。
|
||||
- 授权在服务端强制执行:普通用户的查询必须以流选择器开头,策略标签自动合并,无法越权。
|
||||
|
||||
用户管理(仅管理员)支持创建/重置密码/批量删除;LDAP 用户可在系统设置接入后导入。
|
||||
@@ -0,0 +1,26 @@
|
||||
# 告警通知
|
||||
|
||||
「日志告警」页(`/alerts`)管理规则、通知通道与发送历史。
|
||||
|
||||

|
||||
|
||||
## 快速开始
|
||||
|
||||
1. 在「通知通道」中编辑默认通道,填入飞书群机器人 Webhook 并启用。
|
||||
2. 内置九条规则默认启用并已绑定默认通道,命中阈值即发送到群。
|
||||
|
||||
## 规则
|
||||
|
||||
- 查询为 LogQL 日志流匹配;窗口(分钟)、阈值(条数)、冷却(分钟)三要素。
|
||||
- 内置规则已排除 Loki 自身日志(app=loki),避免告警查询日志造成自我误报。
|
||||
- 模板按钮一键填充常用规则(命名空间错误 / Ingress 4xx5xx / FATAL 等)。
|
||||
|
||||
## 通道与发送历史
|
||||
|
||||
- 通道支持飞书 Webhook(可选签名);可建多通道对应不同告警群。
|
||||
- 发送历史记录每次触发的命中数、投递结果与失败原因;保留天数在「系统设置 → 日志保留时间」调整。
|
||||
|
||||
## 防刷屏机制
|
||||
|
||||
- 单规则冷却(默认 ≥ 2 小时);全平台每小时发送上限(风暴抑制)。
|
||||
- 通道未配置/未启用时命中只记入历史,不影响规则评估。
|
||||
@@ -0,0 +1,46 @@
|
||||
# 日志分析
|
||||
|
||||
「日志分析」页(`/analysis`,管理员)自动聚合日志指标,无需手动查询:
|
||||
|
||||

|
||||
|
||||
- **统计卡**:日志总量、日志体积、错误行数、活跃应用
|
||||
- **趋势折线**:按时间分桶的日志体积(≤12 小时 15 分钟一桶,>12 小时 1 小时一桶)
|
||||
- **应用排行**:按**日志体积**排 Top 10,支持 10/30/50 与前端分页,点击「查看日志」跳转查询页
|
||||
- **每日日志量**:平台留存历史,支持 7 / 30 / 90 / 365 天切换(独立于上方时间范围)
|
||||
- **Ingress 访问分析**:自动识别 ingress-nginx 访问日志,PV/UV、状态码、慢请求与接口排行;未安装 ingress-nginx 时自动折叠
|
||||
|
||||
## 数据来源与耗时
|
||||
|
||||
| 数据 | 来源 | 耗时 |
|
||||
|------|------|------|
|
||||
| 总量/体积趋势、总条数 | Loki 索引接口(`index/volume_range`、`index/stats`),只读 chunk 元数据 | 百毫秒级 |
|
||||
| ≤1 小时范围的应用排行 | 实时扫描日志内容(`count_over_time`) | 5 分钟窗口约 5~10 秒 |
|
||||
| >1 小时范围的应用排行 | 平台**按小时预聚合快照**(每小时第 5 分钟抓上一个完整小时) | 毫秒级 |
|
||||
|
||||
> 一次分析页要跑多条按应用的重查询,因此筛选栏改条件后**点「刷新」才生效**;
|
||||
> 应用排行卡片滚动到可视区才发起重查询,避免首屏一起等。
|
||||
|
||||
## 应用排行的「体积」口径
|
||||
|
||||
预聚合快照对每个应用每小时同时留存 **条数 + 体积**:
|
||||
|
||||
- 条数走 `count_over_time`(要解码日志内容,一小时代价高,每小时后台跑一次)
|
||||
- 体积走 `index/volume`(Loki 索引接口,一个应用约 80ms,所以长范围排行也能按体积排序)
|
||||
|
||||
体积优先排序的原因:条数会被高频短日志主导(一条 INFO 刷屏 vs 一条大 JSON),
|
||||
用户真正关心的是「哪个应用把日志量吃掉了」。
|
||||
|
||||
体积列上线前抓的小时只有条数,页面会在表格下方提示
|
||||
「其中 N/M 小时有体积数据…」——**不要把这个提示当成故障**,
|
||||
它是在说明这段时间的体积偏小是采样口径问题,而不是应用体积真的小。
|
||||
|
||||
## 每日日志量的「未留存」
|
||||
|
||||
每日量来自平台自己的留存库(默认保留 400 天),不受 Loki retention(72 小时)限制。
|
||||
|
||||
- **今天的柱子**是「进行中」,每 30 分钟刷新一次;零点后 1 分钟结算前一天
|
||||
- **灰色柱 = 平台没有留存记录**(平台当天没运行 / 超出留存期),**不是「当天没有日志」**
|
||||
- 如果缺失的天画成 0,30/90 天视角会出现大片空柱,容易被误读成「以前日志量很低」甚至「系统停摆过」
|
||||
|
||||
时间范围与数据源在页面顶部切换;指标查询经服务端聚合,开销远低于拉取原始日志。
|
||||
@@ -0,0 +1,18 @@
|
||||
# 日志采集
|
||||
|
||||
「日志采集」页管理采集策略:主机采集(Promtail)与集群采集(DaemonSet / Fluent Bit Sidecar)。
|
||||
|
||||
## 采集范围
|
||||
|
||||
- **集群采集**:命名空间多选(默认全集群);app 标签按 `app.kubernetes.io/name → app → controller 名 → pod 名` 四级回退,可自定义来源标签与 app 白名单收敛日志量。
|
||||
- **主机采集**:填写业务日志路径(glob 通配,如 `/var/log/app/*.log`);系统日志自动探测(syslog / messages)。
|
||||
- 敏感路径(/etc/shadow、私钥、/proc 等)会被平台拦截,防止主机凭据进入日志平台。
|
||||
|
||||
## 编辑与下发
|
||||
|
||||
保存后点击「部署」下发到目标;「预览 YAML」可先核对将生成的 promtail 配置。修改后同样以「部署」生效。
|
||||
|
||||
## 排查
|
||||
|
||||
- 采集器卡片可看运行状态与日志(主机 docker logs / journalctl,集群 Pod 日志)。
|
||||
- 常见问题:目标命名空间无工作负载(app 候选为空)、路径无写入权限、Loki 端点不可达。
|
||||
@@ -0,0 +1,29 @@
|
||||
# 常见问题
|
||||
|
||||
## 登录后看不到「部署向导」「用户管理」等菜单?
|
||||
|
||||
这些页面仅管理员可见。普通用户默认只有日志查询与日志告警;查询范围由策略组控制(见[查询授权](access.md))。
|
||||
|
||||
## 部署的 Loki 怎么访问?必须配 Ingress 吗?
|
||||
|
||||
集群内部署经 API Server Service Proxy 查询,不需要 Ingress。跨集群 / 公网查询才需要为 Loki 暴露域名后在「数据源管理」用 URL 直连关联。
|
||||
|
||||
## 「停止部署」会删掉日志数据吗?
|
||||
|
||||
不会。停止只回收工作负载;数据卷(PVC / 主机数据目录)保留,重新启动即可恢复。仅显式删除部署记录不影响远端资源,卸载需按页面指引执行。
|
||||
|
||||
## 告警一直不发送?
|
||||
|
||||
按顺序检查:通道是否启用、Webhook 是否正确(通道页有测试)、规则是否启用且命中阈值、发送历史里是否有「冷却抑制」或失败原因。通道未就绪时历史会明确记录失败原因。
|
||||
|
||||
## 忘记管理员密码?
|
||||
|
||||
管理员可在「用户管理」重置任意账号密码;若唯一管理员丢失密码,参考仓库 README 的离线重置说明。
|
||||
|
||||
## 平台可以多实例部署吗?
|
||||
|
||||
不可以。JSON 文件存储为单实例设计(防抖串行落盘),水平扩展会互相覆盖;需要高可用时请保持单实例并对数据目录做系统级快照。
|
||||
|
||||
## 升级安全吗?
|
||||
|
||||
配置与数据向后兼容,结构变更按 schema 版本自动迁移;建议升级前对数据目录(lokistack.json)做一次文件级拷贝。
|
||||
@@ -0,0 +1,29 @@
|
||||
# 快速安装
|
||||
|
||||
## 方式一:Kubernetes
|
||||
|
||||
```bash
|
||||
bash kubernetes/install.sh
|
||||
```
|
||||
|
||||
## 方式二:Docker
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## 方式三:单二进制
|
||||
|
||||
```bash
|
||||
curl -LO https://gitea.91ai.io/rlwj-lab/LokiStack/releases/download/v2.0.0/lokistack-linux-amd64.tar.gz
|
||||
tar xzf lokistack-linux-amd64.tar.gz && cd lokistack-linux-amd64
|
||||
./lokistack
|
||||
```
|
||||
|
||||
arm64 服务器下载 `lokistack-linux-arm64.tar.gz`。
|
||||
|
||||
## 说明
|
||||
|
||||
- 未提供 `JWT_SECRET` / `ENCRYPTION_KEY` 时,自动生成随机密钥并持久化到数据目录(`.secrets.env`,权限 0600),重启与升级保持不变;显式设置环境变量可随时覆盖
|
||||
- 全部状态存储在 `$DATA_DIR/lokistack.json`,单实例运行;首次启动自动创建管理员账号,初始密码在 `$DATA_DIR/initial-admin-password.txt`
|
||||
- 升级:替换二进制或重新拉取镜像后重启,历史数据自动迁移
|
||||
@@ -0,0 +1,74 @@
|
||||
{
|
||||
"productVersion": "2.0",
|
||||
"groups": [
|
||||
{
|
||||
"name": "开始使用",
|
||||
"pages": [
|
||||
{
|
||||
"file": "README.md",
|
||||
"slug": "readme",
|
||||
"title": "产品介绍",
|
||||
"updated": "2026-10-28"
|
||||
},
|
||||
{
|
||||
"file": "install.md",
|
||||
"slug": "install",
|
||||
"title": "快速安装",
|
||||
"updated": "2026-10-28"
|
||||
},
|
||||
{
|
||||
"file": "wizard.md",
|
||||
"slug": "wizard",
|
||||
"title": "部署向导",
|
||||
"updated": "2026-10-28"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "采集与查询",
|
||||
"pages": [
|
||||
{
|
||||
"file": "collectors.md",
|
||||
"slug": "collectors",
|
||||
"title": "日志采集",
|
||||
"updated": "2026-10-28"
|
||||
},
|
||||
{
|
||||
"file": "query.md",
|
||||
"slug": "query",
|
||||
"title": "日志查询",
|
||||
"updated": "2026-10-28"
|
||||
},
|
||||
{
|
||||
"file": "analysis.md",
|
||||
"slug": "analysis",
|
||||
"title": "日志分析",
|
||||
"updated": "2026-10-28"
|
||||
},
|
||||
{
|
||||
"file": "access.md",
|
||||
"slug": "access",
|
||||
"title": "查询授权",
|
||||
"updated": "2026-10-28"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "告警与运维",
|
||||
"pages": [
|
||||
{
|
||||
"file": "alerts.md",
|
||||
"slug": "alerts",
|
||||
"title": "告警通知",
|
||||
"updated": "2026-10-28"
|
||||
},
|
||||
{
|
||||
"file": "faq.md",
|
||||
"slug": "faq",
|
||||
"title": "常见问题",
|
||||
"updated": "2026-10-28"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
# 日志查询
|
||||
|
||||
「日志查询」页(`/logs`)按 LogQL 与时间范围检索日志。
|
||||
|
||||

|
||||
|
||||
## 查询方式
|
||||
|
||||
- **LogQL 编辑器**:语法高亮与标签/管道补全;回车执行。
|
||||
- **快捷查询**:先选命名空间,再选 App / Pod(候选值**按级联懒加载**:选了命名空间才会去取 App 列表,选了 App 才取 Pod 列表,避免一进页面就把 Loki 索引扫一遍),选中即自动填入并执行;Ingress 访问日志按钮在选择 ingress 命名空间或 app 时出现。
|
||||
- **Label 浏览器**:按标签与值点选构建查询;点结果行的标签值可快速加过滤。
|
||||
- **Live Tail**:3 秒轮询最近 5 分钟,编辑查询自动防抖后切换。
|
||||
|
||||
## 结果区
|
||||
|
||||
- 顶部时间线展示整个查询范围的日志量,拖选高峰时段自动重查。
|
||||
- 原始日志支持关键字高亮、标签过滤、查看上下文(前后 N 行)。
|
||||
- 翻页向后游标推进,每页最多 1000 条。
|
||||
|
||||
## 数据源
|
||||
|
||||
「数据源管理」维护 Loki 连接:平台部署自动注册(随部署托管),也可手动添加 URL 直连或集群 Service 数据源,支持 Basic / Bearer / 自定义 Header 认证与连接测试。
|
||||
@@ -0,0 +1,29 @@
|
||||
# 部署向导
|
||||
|
||||
部署向导(`/wizard`)三步完成 Loki 上线:环境与目标 → 存储 → 部署 Loki 与采集。
|
||||
|
||||

|
||||
|
||||
## 第一步:环境与目标
|
||||
|
||||
- **Kubernetes 集群**:粘贴 kubeconfig,或按页面给出的 RBAC 脚本在集群执行一次后粘贴 ServiceAccount 长期 Token。平台经 API Server 操作集群,无需安装 kubectl/Helm。
|
||||
- **裸金属主机**:填写 SSH 地址与凭证(密码或私钥,AES-GCM 加密存储)。适合二进制或 Docker 模式。
|
||||
|
||||
创建后平台自动验证连通性;Kubernetes 资源默认落在 `log` 命名空间(可改)。
|
||||
|
||||
## 第二步:准备存储
|
||||
|
||||
推荐对象存储(MinIO / RustFS / S3 / OSS):多副本与数据保留更可靠,创建时可用「测试连接」验证凭证并自动建桶;跳过则使用本地磁盘。
|
||||
|
||||
## 第三步:部署与采集
|
||||
|
||||
- 选择目标、Loki 版本(支持 2.8+ / 3.x)、保留天数与资源规格,点击部署。
|
||||
- 部署成功后平台自动注册同名数据源,日志查询页立即可用;按提示添加采集策略(见[日志采集](collectors.md))。
|
||||
|
||||
## 三种部署模式
|
||||
|
||||
| 模式 | 适用 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| k8s-yaml | Kubernetes | StatefulSet + SSA 直发,平台全托管 |
|
||||
| host-binary | 裸金属 | SSH 安装二进制 + systemd 托管 |
|
||||
| host-docker | 裸金属 | Docker 容器运行,自动目录迁移 |
|
||||
@@ -4,6 +4,7 @@ import { Toaster } from '@/components/ui/sonner'
|
||||
import { isLoggedIn, getUser } from '@/lib/auth-storage'
|
||||
import { routes, findRouteMeta } from '@/router/routes'
|
||||
import ErrorBoundary from '@/components/ErrorBoundary'
|
||||
import { LicenseProvider } from '@/hooks/useLicense'
|
||||
import { useTranslation } from 'react-i18next'
|
||||
|
||||
const Layout = lazy(() => import('@/components/Layout'))
|
||||
@@ -58,7 +59,7 @@ function useDocumentTitle() {
|
||||
const App = memo(function App() {
|
||||
useDocumentTitle()
|
||||
return (
|
||||
<>
|
||||
<LicenseProvider>
|
||||
<Routes>
|
||||
{routes.map((route) => (
|
||||
<Route
|
||||
@@ -80,7 +81,7 @@ const App = memo(function App() {
|
||||
))}
|
||||
</Routes>
|
||||
<Toaster />
|
||||
</>
|
||||
</LicenseProvider>
|
||||
)
|
||||
})
|
||||
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
import { http } from '@/api/http'
|
||||
|
||||
export interface AIConfig {
|
||||
enabled: boolean
|
||||
provider: string
|
||||
baseUrl: string
|
||||
/** 后端永远返回掩码(sk-***abcd),保存时留空表示不修改 */
|
||||
apiKey: string
|
||||
model: string
|
||||
temperature: number
|
||||
maxTokens: number
|
||||
timeoutSec: number
|
||||
dailyTokenCap: number
|
||||
maxScanBytes: number
|
||||
redactExtra: string[]
|
||||
}
|
||||
|
||||
export interface AIConfigResponse {
|
||||
config: AIConfig
|
||||
tokenUsedToday: number
|
||||
}
|
||||
|
||||
export interface AIFinding {
|
||||
title: string
|
||||
evidence: string
|
||||
severity: 'high' | 'medium' | 'low' | string
|
||||
}
|
||||
|
||||
export interface AIAnalysisResult {
|
||||
summary: string
|
||||
findings: AIFinding[]
|
||||
actions: string[]
|
||||
unknowns: string[]
|
||||
parseError?: string
|
||||
}
|
||||
|
||||
export interface AISignalBundle {
|
||||
sourceId: number
|
||||
sourceName: string
|
||||
selector: string
|
||||
from: string
|
||||
to: string
|
||||
rangeMinutes: number
|
||||
totalBytes: number
|
||||
totalEntries: number
|
||||
entriesExact: boolean
|
||||
apps: Array<{ app: string; namespace: string; entries: number; bytes: number; share: string }>
|
||||
alerts: Array<{ ruleName: string; status: string; count: number; createdAt: string }>
|
||||
baseline?: { previousBytes: number; currentBytes: number; deltaPercent: number; comparisonNote: string }
|
||||
snapshotHours: number
|
||||
warnings: string[]
|
||||
}
|
||||
|
||||
export interface AIRun {
|
||||
id: number
|
||||
userId: number
|
||||
username: string
|
||||
sourceId: number
|
||||
sourceName: string
|
||||
rangeMinutes: number
|
||||
model: string
|
||||
question: string
|
||||
status: string
|
||||
error: string
|
||||
conclusion: string
|
||||
promptPayload: string
|
||||
promptTokens: number
|
||||
completionTokens: number
|
||||
scanBytes: number
|
||||
durationMs: number
|
||||
createdAt: string
|
||||
}
|
||||
|
||||
export interface AIAnalyzeResponse {
|
||||
result: AIAnalysisResult
|
||||
signals: AISignalBundle
|
||||
prompt: string
|
||||
run: AIRun
|
||||
warnings: string[]
|
||||
}
|
||||
|
||||
export const aiApi = {
|
||||
getConfig: () => http.get<AIConfigResponse>('/ai/config'),
|
||||
saveConfig: (data: Partial<AIConfig>) => http.put<{ config: AIConfig }>('/ai/config', data),
|
||||
testConfig: (data: { baseUrl?: string; model?: string; apiKey?: string }) =>
|
||||
http.post<{ model: string; latencyMs: number; tokensUsed: number; replyLength: number }>('/ai/config/test', data, { timeout: 45000 }),
|
||||
analyze: (data: { sourceId: number; start: number; end: number; question?: string; app?: string }) =>
|
||||
http.post<AIAnalyzeResponse>('/ai/analyze', data, { timeout: 180000 }),
|
||||
runs: (page: number, pageSize: number) =>
|
||||
http.get<{ list: AIRun[]; total: number; page: number; pageSize: number }>(
|
||||
`/ai/runs?page=${page}&pageSize=${pageSize}`,
|
||||
),
|
||||
run: (id: number) => http.get<AIRun>(`/ai/runs/${id}`),
|
||||
|
||||
chat: (data: { sourceId?: number; history: Array<{ role: 'user' | 'assistant'; content: string }>; page?: string; rangeMinutes?: number; context?: string }) =>
|
||||
http.post<{ reply: string }>('/ai/chat', data),
|
||||
}
|
||||
@@ -55,4 +55,12 @@ export const collectorApi = {
|
||||
/** 读取采集器最近运行日志(主机:docker logs / journalctl;K8s:Pod 日志) */
|
||||
logs: (id: number, tail = 200) =>
|
||||
http.get<CollectorLogs>(`/collectors/${id}/logs?tail=${tail}`),
|
||||
|
||||
/** 采集项 → YAML(切到 YAML 编辑模式) */
|
||||
renderJobs: (jobs: ScrapeJob[]) =>
|
||||
http.post<{ yaml: string }>('/collector-jobs/render', { jobs }),
|
||||
|
||||
/** YAML → 采集项(切回表单 / 保存前校验) */
|
||||
parseJobs: (yaml: string) =>
|
||||
http.post<{ jobs: ScrapeJob[] }>('/collector-jobs/parse', { yaml }),
|
||||
};
|
||||
@@ -0,0 +1,28 @@
|
||||
import { http } from '../http'
|
||||
|
||||
export interface TrialInfo { active: boolean; daysLeft: number }
|
||||
export interface LicenseInfo {
|
||||
trial?: TrialInfo
|
||||
plan: 'free' | 'pro'
|
||||
edition?: string
|
||||
instances?: number
|
||||
customer?: string
|
||||
expiresAt?: number
|
||||
expired?: boolean
|
||||
source?: 'hmac' | 'opsx' | string
|
||||
planType?: string
|
||||
maxDevices?: number
|
||||
deprecationNotice?: string
|
||||
inGrace?: boolean
|
||||
graceDaysLeft?: number
|
||||
lastVerifiedAt?: number
|
||||
instanceId?: string
|
||||
/** expired | over_instance | revoked — 非有效 Pro 时红条 */
|
||||
invalidReason?: 'expired' | 'over_instance' | 'revoked' | string
|
||||
}
|
||||
export const licenseApi = {
|
||||
status: () => http.get<{ license: LicenseInfo; pro: boolean; features: string[] }>('/system/license'),
|
||||
activate: (key: string) => http.post<LicenseInfo>('/system/license/activate', { key }),
|
||||
deactivate: () =>
|
||||
http.post<{ offlineLocalOnly?: boolean; message?: string }>('/system/license/deactivate'),
|
||||
}
|
||||
@@ -26,7 +26,45 @@ export interface DailyVolumePoint {
|
||||
entries: number
|
||||
/** 当天未结束或抓取时数据不全 */
|
||||
partial: boolean
|
||||
/** 该天平台没有留存快照(超过留存期/当天平台没运行)——与"日志量真的是 0"不同 */
|
||||
missing?: boolean
|
||||
}
|
||||
/**
|
||||
* label 候选值的前端缓存(60 秒 + 同请求合并)。
|
||||
*
|
||||
* 为什么前端也要缓存:快捷 LogQL 的下拉在「切数据源 / 切命名空间 / 重开面板」
|
||||
* 时都会重新走一遍请求链,后端虽然只要 25~40ms,但串起来仍是三四次往返,
|
||||
* 用户看到的就是转圈。缓存后同一组合在 30 秒内直接命中内存,体感是"打开就有"。
|
||||
*
|
||||
* 时效性说明:新增 app / 新副本最长 30 秒后出现在下拉里。平台侧的变更
|
||||
* (采集器增删改、部署删除、数据源增删改)会由后端立即清缓存,
|
||||
* 所以"刚配好的 app 找不到"这类问题不会被缓存放大。
|
||||
*
|
||||
* 用 Promise 作为缓存值:并发的相同请求共享同一个飞行中的请求(去重),
|
||||
* 失败时立刻从缓存剔除,避免把错误结果缓存住。
|
||||
*/
|
||||
// 与后端 TTL 对齐(30 秒):新上线的 app 要在半分钟内可被选中,
|
||||
// 同时又能挡住"切命名空间/重开面板"造成的重复请求。
|
||||
const LABEL_CACHE_TTL = 30_000
|
||||
const labelValueCache = new Map<string, { expires: number; data: Promise<unknown> }>()
|
||||
|
||||
function cachedRequest<T>(key: string, run: () => Promise<T>): Promise<T> {
|
||||
const now = Date.now()
|
||||
const hit = labelValueCache.get(key)
|
||||
if (hit && hit.expires > now) return hit.data as Promise<T>
|
||||
const promise = run().catch((err: unknown) => {
|
||||
labelValueCache.delete(key)
|
||||
throw err
|
||||
})
|
||||
labelValueCache.set(key, { expires: now + LABEL_CACHE_TTL, data: promise })
|
||||
return promise
|
||||
}
|
||||
|
||||
/** 手动失效(数据源被删除/重建后调用,避免拿到旧源的候选值) */
|
||||
export function invalidateLabelCache(): void {
|
||||
labelValueCache.clear()
|
||||
}
|
||||
|
||||
export const logsApi = {
|
||||
sources: () => http.get<PageData<LogSource>>('/logs/sources'),
|
||||
createSource: (source: Partial<LogSource> & { basicPass?: string; bearerToken?: string }) => http.post<LogSource>('/logs/sources', source),
|
||||
@@ -45,6 +83,21 @@ export const logsApi = {
|
||||
http.post<{ streams: number; chunks: number; bytes: number; entries: number }>(
|
||||
'/logs/stats', data, { timeout: 25000, ...opts }),
|
||||
/** 日志量统计:后端有 7 天范围上限与 30s 超时,前端超时略大于它 */
|
||||
/** 应用排行(长范围读平台预聚合快照,毫秒级,不扫日志内容) */
|
||||
appRanking: (sourceId: number, from: number, to: number, limit = 10, opts?: { signal?: AbortSignal; sort?: 'bytes' | 'entries' }) => {
|
||||
const { signal, sort = 'bytes' } = opts ?? {}
|
||||
return http.get<{
|
||||
list: Array<{ app: string; namespace: string; entries: number; bytes: number; fromHistory: boolean }>
|
||||
/** 快照覆盖的小时数 */
|
||||
hours: number
|
||||
/** 其中带体积的小时数(体积列是本轮新增,早期快照只有条数) */
|
||||
bytesHours: number
|
||||
}>(
|
||||
`/logs/app-ranking?sourceId=${sourceId}&from=${from}&to=${to}&limit=${limit}&sort=${sort}`,
|
||||
signal ? { signal } : undefined,
|
||||
)
|
||||
},
|
||||
|
||||
/** 每日日志量历史(读平台库,毫秒级) */
|
||||
dailyVolume: (sourceId: number, days: number, opts?: { signal?: AbortSignal }) =>
|
||||
http.get<DailyVolumePoint[]>(`/logs/daily-volume?sourceId=${sourceId}&days=${days}`, opts),
|
||||
@@ -53,7 +106,32 @@ export const logsApi = {
|
||||
/** LogQL 指标查询(admin):query 中的 [$step] 占位符由后端按时间跨度替换 */
|
||||
metrics: (data: MetricQuery, opts?: { signal?: AbortSignal }) => http.post<MetricSeries[]>('/logs/metrics', data, { timeout: 125000, ...opts }),
|
||||
labels: (sourceId: number, start?: number, end?: number) => http.get<string[]>(`/logs/labels${buildQuery({ sourceId, start, end } as { page?: number; pageSize?: number })}`),
|
||||
labelValues: (sourceId: number, name: string, query = '', start?: number, end?: number) => http.get<string[]>(`/logs/label/${encodeURIComponent(name)}/values${buildQuery({ sourceId, query, start, end } as { page?: number; pageSize?: number })}`),
|
||||
labelValues: (sourceId: number, name: string, query = '', start?: number, end?: number) =>
|
||||
cachedRequest<string[]>(
|
||||
`lv|${sourceId}|${name}|${query}|${start ?? 0}|${end ?? 0}`,
|
||||
() => http.get<string[]>(`/logs/label/${encodeURIComponent(name)}/values${buildQuery({ sourceId, query, start, end } as { page?: number; pageSize?: number })}`),
|
||||
),
|
||||
/**
|
||||
* 批量取多个 label 的候选值:`/logs/label-values?names=app,service_name,container&query={...}`
|
||||
*
|
||||
* 快捷查询的 app 下拉要做 app → service_name → container 三级回退,
|
||||
* 逐个请求要 3 个往返(每个都经 K8s API 代理到 Loki 索引);
|
||||
* 批量接口后端内部并发,一次往返拿全,前端只看"有没有值"。
|
||||
* 返回里缺某个 label 或取失败 = 空数组(不会抛错)。
|
||||
*/
|
||||
labelValuesBatch: (sourceId: number, names: string[], query = '', start?: number, end?: number, opts?: { signal?: AbortSignal }) =>
|
||||
// 带 signal 的调用(用户主动取消)不参与缓存,避免把"被取消"的请求共享给别人
|
||||
opts?.signal
|
||||
? http.get<Record<string, string[]>>(
|
||||
`/logs/label-values${buildQuery({ sourceId, names: names.join(','), query, start, end } as { page?: number; pageSize?: number })}`,
|
||||
opts,
|
||||
)
|
||||
: cachedRequest<Record<string, string[]>>(
|
||||
`lvb|${sourceId}|${names.join(',')}|${query}|${start ?? 0}|${end ?? 0}`,
|
||||
() => http.get<Record<string, string[]>>(
|
||||
`/logs/label-values${buildQuery({ sourceId, names: names.join(','), query, start, end } as { page?: number; pageSize?: number })}`,
|
||||
),
|
||||
),
|
||||
/** 当前用户在某数据源上的授权范围(策略 labels;管理员 unrestricted=true) */
|
||||
myScope: (sourceId: number) => http.get<{ labels: Record<string, string>; unrestricted: boolean }>(`/logs/my-scope?sourceId=${sourceId}`),
|
||||
policyGroups: () => http.get<LogPolicyGroup[]>('/logs/policy-groups'),
|
||||
|
||||
@@ -65,27 +65,6 @@ export interface SelfCheckResult {
|
||||
items: SelfCheckItem[];
|
||||
}
|
||||
|
||||
export interface BackupFile {
|
||||
name: string;
|
||||
sizeBytes: number;
|
||||
createdAt: string;
|
||||
}
|
||||
|
||||
export interface RestoreResult {
|
||||
manifest: {
|
||||
version: string;
|
||||
createdAt: string;
|
||||
dbBytes: number;
|
||||
recordCount: number;
|
||||
note: string;
|
||||
};
|
||||
keyMatches: boolean;
|
||||
rowsInBackup: number;
|
||||
currentRows: number;
|
||||
stagedPath?: string;
|
||||
warning?: string;
|
||||
}
|
||||
|
||||
export const systemApi = {
|
||||
// 审计
|
||||
auditList: (params: AuditListParams = {}, opts?: { signal?: AbortSignal }) =>
|
||||
@@ -99,16 +78,6 @@ export const systemApi = {
|
||||
// 自检
|
||||
selfCheck: () => http.get<SelfCheckResult>('/system/self-check'),
|
||||
|
||||
// 备份恢复
|
||||
backups: () => http.get<{ list: BackupFile[] }>('/system/backups'),
|
||||
restore: (file: File) => {
|
||||
const form = new FormData();
|
||||
form.append('file', file);
|
||||
return http.post<RestoreResult>('/system/restore', form);
|
||||
},
|
||||
restoreApply: () =>
|
||||
http.post<{ message: string; dbPath: string }>('/system/restore/apply', { confirm: true }),
|
||||
|
||||
// 平台数据保留期(告警发送历史 / 审计日志)
|
||||
retention: () => http.get<RetentionSettings>('/settings/retention'),
|
||||
saveRetention: (data: RetentionSettings) => http.put<RetentionSettings>('/settings/retention', data),
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
import { memo, useEffect, useRef, useState } from 'react'
|
||||
import { useLocation } from 'react-router'
|
||||
import { useTranslation } from 'react-i18next'
|
||||
import { Loader2, Send, Sparkles, X } from 'lucide-react'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Textarea } from '@/components/ui/textarea'
|
||||
import { aiApi } from '@/api/ai'
|
||||
import { getUser } from '@/lib/auth-storage'
|
||||
import { useLicense } from '@/hooks/useLicense'
|
||||
import { buildLogsContextText } from '@/lib/ai-assistant-context'
|
||||
|
||||
interface Msg { role: 'user' | 'assistant'; content: string }
|
||||
|
||||
/**
|
||||
* 全站 AI 助手浮窗(参考 FlowStack 右下角浮空设计):
|
||||
* 仅管理员 + Pro 生效时渲染;对话自动携带当前页面路径作为上下文,
|
||||
* 后端按需注入当前数据源的日志摘要,让"现在这个报错是什么"有数据依据。
|
||||
*/
|
||||
export const AiAssistant = memo(function AiAssistant() {
|
||||
const { t } = useTranslation('aiops')
|
||||
const location = useLocation()
|
||||
const { pro } = useLicense()
|
||||
const isAdmin = getUser()?.role === 'admin'
|
||||
const [open, setOpen] = useState(false)
|
||||
const [messages, setMessages] = useState<Msg[]>([])
|
||||
const [input, setInput] = useState('')
|
||||
const [sending, setSending] = useState(false)
|
||||
const listRef = useRef<HTMLDivElement>(null)
|
||||
|
||||
useEffect(() => {
|
||||
if (listRef.current) listRef.current.scrollTop = listRef.current.scrollHeight
|
||||
}, [messages, sending])
|
||||
|
||||
// 按产品定位:助手服务于日志查询 —— 只在日志查询页出现,带着当前查询结果对话
|
||||
if (!isAdmin || !pro || location.pathname !== '/logs') return null
|
||||
|
||||
const send = async () => {
|
||||
const text = input.trim()
|
||||
if (!text || sending) return
|
||||
const next: Msg[] = [...messages, { role: 'user', content: text }]
|
||||
setMessages(next)
|
||||
setInput('')
|
||||
setSending(true)
|
||||
try {
|
||||
const res = await aiApi.chat({
|
||||
history: next.slice(-20),
|
||||
page: location.pathname,
|
||||
rangeMinutes: 15,
|
||||
context: buildLogsContextText() || undefined,
|
||||
})
|
||||
setMessages(current => [...current, { role: 'assistant', content: res.reply }])
|
||||
} catch (err) {
|
||||
setMessages(current => [...current, { role: 'assistant', content: err instanceof Error ? err.message : t('assistant.error') }])
|
||||
} finally {
|
||||
setSending(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={open ? t('assistant.close') : t('assistant.open')}
|
||||
onClick={() => setOpen(!open)}
|
||||
className="fixed bottom-6 right-6 z-50 flex size-12 items-center justify-center rounded-full bg-primary text-primary-foreground shadow-lg transition-transform hover:scale-105"
|
||||
>
|
||||
{open ? <X className="size-5" /> : <Sparkles className="size-5" />}
|
||||
</button>
|
||||
{open && (
|
||||
<div
|
||||
className="fixed bottom-20 right-6 z-50 flex h-[520px] w-[min(92vw,400px)] flex-col overflow-hidden rounded-xl border bg-card shadow-2xl"
|
||||
role="dialog"
|
||||
aria-label={t('assistant.title')}
|
||||
>
|
||||
<div className="flex items-center justify-between border-b px-4 py-2.5">
|
||||
<span className="flex items-center gap-2 text-sm font-semibold">
|
||||
<Sparkles className="size-4 text-primary" />
|
||||
{t('assistant.title')}
|
||||
</span>
|
||||
<button type="button" aria-label={t('assistant.close')} onClick={() => setOpen(false)} className="rounded p-1 text-muted-foreground hover:bg-muted">
|
||||
<X className="size-4" />
|
||||
</button>
|
||||
</div>
|
||||
<div ref={listRef} className="flex-1 space-y-3 overflow-y-auto p-4 text-sm">
|
||||
{messages.length === 0 && (
|
||||
<p className="py-8 text-center text-xs text-muted-foreground">{t('assistant.emptyHint')}</p>
|
||||
)}
|
||||
{messages.map((m, i) => {
|
||||
const isQuota = m.role === 'assistant' && m.content.includes('今日 AI 配额已用完')
|
||||
return (
|
||||
<div key={i} className={m.role === 'user' ? 'flex justify-end' : 'flex justify-start'}>
|
||||
<div className={
|
||||
isQuota
|
||||
? 'max-w-[85%] whitespace-pre-wrap rounded-lg border border-warning/40 bg-warning/10 px-3 py-2 text-warning'
|
||||
: m.role === 'user'
|
||||
? 'max-w-[85%] rounded-lg bg-primary px-3 py-2 text-primary-foreground'
|
||||
: 'max-w-[85%] whitespace-pre-wrap rounded-lg bg-muted px-3 py-2'
|
||||
}>
|
||||
{m.content}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
{sending && (
|
||||
<div className="flex justify-start">
|
||||
<div className="flex items-center gap-2 rounded-lg bg-muted px-3 py-2 text-xs text-muted-foreground">
|
||||
<Loader2 className="size-3.5 animate-spin" />
|
||||
{t('assistant.thinking')}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex items-end gap-2 border-t p-3">
|
||||
<Textarea
|
||||
rows={1}
|
||||
value={input}
|
||||
onChange={e => setInput(e.target.value)}
|
||||
onKeyDown={e => {
|
||||
if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); void send() }
|
||||
}}
|
||||
placeholder={t('assistant.placeholder')}
|
||||
className="max-h-28 min-h-9 resize-none text-sm"
|
||||
/>
|
||||
<Button size="sm" onClick={() => void send()} disabled={sending || !input.trim()} aria-label={t('assistant.send')}>
|
||||
<Send className="size-4" />
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,39 @@
|
||||
import { memo, useCallback } from 'react'
|
||||
import { useTranslation } from 'react-i18next'
|
||||
import { Copy } from 'lucide-react'
|
||||
import { toast } from 'sonner'
|
||||
import { Button } from '@/components/ui/button'
|
||||
|
||||
type Props = {
|
||||
instanceId?: string | null
|
||||
}
|
||||
|
||||
/** 实例 ID 展示行:设置授权区与 ProGate 共用;全文 + 一键复制,无 PII。 */
|
||||
export const InstanceIdRow = memo(function InstanceIdRow({ instanceId }: Props) {
|
||||
const { t } = useTranslation('license')
|
||||
const id = instanceId?.trim() || ''
|
||||
|
||||
const onCopy = useCallback(() => {
|
||||
if (!id) return
|
||||
void navigator.clipboard?.writeText(id).then(
|
||||
() => toast.success(t('instanceIdCopied')),
|
||||
() => toast.error(t('instanceIdCopyFailed')),
|
||||
)
|
||||
}, [id, t])
|
||||
|
||||
if (!id) return null
|
||||
|
||||
return (
|
||||
<div className="space-y-1">
|
||||
<div className="flex flex-wrap items-start gap-2">
|
||||
<span className="shrink-0 pt-0.5 text-sm text-muted-foreground">{t('instanceId')}</span>
|
||||
<code className="min-w-0 flex-1 break-all font-mono text-sm">{id}</code>
|
||||
<Button type="button" size="sm" variant="outline" className="shrink-0 gap-1" onClick={onCopy}>
|
||||
<Copy className="size-3.5" />
|
||||
{t('copy')}
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-xs text-muted-foreground">{t('instanceIdHint')}</p>
|
||||
</div>
|
||||
)
|
||||
})
|
||||
@@ -10,7 +10,6 @@ import {
|
||||
LogOut,
|
||||
PanelLeftClose,
|
||||
PanelLeftOpen,
|
||||
Shield,
|
||||
ChevronRight,
|
||||
Activity,
|
||||
Menu,
|
||||
@@ -22,6 +21,9 @@ import { useTranslation } from 'react-i18next'
|
||||
import { cn } from '@/lib/utils'
|
||||
import { useIsMobile } from '@/hooks/use-mobile'
|
||||
import { menuItems, findRouteMeta, findGroupKey } from '@/router/routes'
|
||||
import { useLicense } from '@/hooks/useLicense'
|
||||
import { AiAssistant } from '@/components/AiAssistant'
|
||||
import { Lock } from 'lucide-react'
|
||||
import { GroupTabs } from './GroupTabs'
|
||||
import { getUser, clearAuth } from '@/lib/auth-storage'
|
||||
import { isThemeAuto, setThemeAuto } from '@/lib/theme-schedule'
|
||||
@@ -37,6 +39,7 @@ function SidebarNav({ collapsed, onNavigate }: { collapsed: boolean; onNavigate?
|
||||
const location = useLocation()
|
||||
const { t } = useTranslation()
|
||||
const isAdmin = getUser()?.role === 'admin'
|
||||
const { pro } = useLicense()
|
||||
// adminOnly 项(用户管理)仅管理员可见;后端 AdminAuth 兜底
|
||||
const visibleItems = menuItems.filter((item) => !item.adminOnly || isAdmin)
|
||||
return (
|
||||
@@ -63,6 +66,7 @@ function SidebarNav({ collapsed, onNavigate }: { collapsed: boolean; onNavigate?
|
||||
>
|
||||
{item.icon && <item.icon className="w-5 h-5 flex-shrink-0" />}
|
||||
{!collapsed && <span className="text-sm font-medium">{t(item.label)}</span>}
|
||||
{item.proOnly && !pro && <Lock className="w-3 h-3 text-sidebar-foreground/50 flex-shrink-0" aria-label="Pro" />}
|
||||
{!collapsed && isActive && (
|
||||
<ChevronRight className="w-4 h-4 ml-auto flex-shrink-0" />
|
||||
)}
|
||||
@@ -240,13 +244,7 @@ const Layout = memo(function Layout({ children }: { children: React.ReactNode })
|
||||
<ThemeToggle />
|
||||
{user && (
|
||||
<div className="flex items-center gap-3 text-sm text-muted-foreground">
|
||||
<div className="flex items-center gap-2">
|
||||
<Shield className="w-4 h-4" />
|
||||
<span>{user.username}</span>
|
||||
<span className="px-2 py-0.5 rounded-full bg-success/10 text-success text-xs font-medium">
|
||||
{user.role === 'admin' ? t('layout.admin') : user.role}
|
||||
</span>
|
||||
</div>
|
||||
<span>{user.username}</span>
|
||||
<button
|
||||
onClick={handleLogout}
|
||||
title={t('layout.logout')}
|
||||
@@ -268,9 +266,10 @@ const Layout = memo(function Layout({ children }: { children: React.ReactNode })
|
||||
<main className="flex-1 overflow-auto p-4 md:p-6">
|
||||
<Suspense fallback={<PageSkeleton />}>{children}</Suspense>
|
||||
</main>
|
||||
<AiAssistant />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
})
|
||||
|
||||
export default Layout
|
||||
export default Layout
|
||||
@@ -0,0 +1,71 @@
|
||||
import { memo } from 'react'
|
||||
import { useTranslation } from 'react-i18next'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import type { LicenseInfo } from '@/api/license'
|
||||
|
||||
type Props = {
|
||||
info: LicenseInfo | null
|
||||
pro: boolean
|
||||
/** 聚焦许可码输入 / 引导重新激活或换码 */
|
||||
onAction?: () => void
|
||||
}
|
||||
|
||||
function resolveInvalid(info: LicenseInfo | null, pro: boolean): 'expired' | 'over_instance' | 'revoked' | null {
|
||||
if (pro || !info) return null
|
||||
const reason = info.invalidReason
|
||||
if (reason === 'expired' || reason === 'over_instance' || reason === 'revoked') return reason
|
||||
if (reason) return 'revoked'
|
||||
if (info.expired) return 'expired'
|
||||
return null
|
||||
}
|
||||
|
||||
/** 授权状态条:迁移/宽限黄条 + 过期·超实例·吊销红条。设置页与 ProGate 共用。 */
|
||||
export const LicenseStatusBanners = memo(function LicenseStatusBanners({ info, pro, onAction }: Props) {
|
||||
const { t } = useTranslation('license')
|
||||
|
||||
const showLegacy = Boolean(info?.deprecationNotice) || (pro && info?.source === 'hmac')
|
||||
const showGrace = Boolean(info?.inGrace)
|
||||
const invalid = resolveInvalid(info, pro)
|
||||
|
||||
const invalidCopy =
|
||||
invalid === 'expired'
|
||||
? { text: t('status.expired'), cta: t('status.expiredCta') }
|
||||
: invalid === 'over_instance'
|
||||
? { text: t('status.overInstance'), cta: t('status.overInstanceCta') }
|
||||
: invalid === 'revoked'
|
||||
? { text: t('status.revoked'), cta: t('status.revokedCta') }
|
||||
: null
|
||||
|
||||
if (!showLegacy && !showGrace && !invalidCopy) return null
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
{showLegacy && (
|
||||
<p className="rounded-md border border-warning/40 bg-warning/10 px-3 py-2 text-sm text-warning-foreground">
|
||||
{info?.deprecationNotice || t('legacyBanner')}
|
||||
</p>
|
||||
)}
|
||||
{showGrace && (
|
||||
<p className="rounded-md border border-warning/40 bg-warning/10 px-3 py-2 text-sm text-warning-foreground">
|
||||
{t('graceBanner', { days: info?.graceDaysLeft ?? 0 })}
|
||||
</p>
|
||||
)}
|
||||
{invalidCopy && (
|
||||
<div className="flex flex-wrap items-center justify-between gap-2 rounded-md border border-destructive/40 bg-destructive/10 px-3 py-2 text-sm text-destructive">
|
||||
<span>{invalidCopy.text}</span>
|
||||
{onAction && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
className="border-destructive/40 text-destructive hover:bg-destructive/10"
|
||||
onClick={onAction}
|
||||
>
|
||||
{invalidCopy.cta}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,95 @@
|
||||
import { memo, useRef, useState } from 'react'
|
||||
import { useNavigate } from 'react-router'
|
||||
import { useTranslation } from 'react-i18next'
|
||||
import { Lock, Sparkles } from 'lucide-react'
|
||||
import { toast } from 'sonner'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
import { Input } from '@/components/ui/input'
|
||||
import { InstanceIdRow } from '@/components/InstanceIdRow'
|
||||
import { LicenseStatusBanners } from '@/components/LicenseStatusBanners'
|
||||
import { licenseApi } from '@/api/license'
|
||||
import { useLicense } from '@/hooks/useLicense'
|
||||
|
||||
/** Pro 功能升级引导卡:免费版进入 Pro 页面时整页替代展示 */
|
||||
export const ProGate = memo(function ProGate({ featureKey }: { featureKey: string }) {
|
||||
const { t } = useTranslation('license')
|
||||
const { pro, info, reload } = useLicense()
|
||||
const navigate = useNavigate()
|
||||
const [key, setKey] = useState('')
|
||||
const [activating, setActivating] = useState(false)
|
||||
const inputRef = useRef<HTMLInputElement>(null)
|
||||
|
||||
const focusKey = () => {
|
||||
inputRef.current?.focus()
|
||||
inputRef.current?.scrollIntoView({ behavior: 'smooth', block: 'center' })
|
||||
}
|
||||
|
||||
const activate = async () => {
|
||||
if (!key.trim()) return
|
||||
setActivating(true)
|
||||
try {
|
||||
await licenseApi.activate(key.trim())
|
||||
toast.success(t('activated'))
|
||||
reload()
|
||||
} catch (err) {
|
||||
toast.error(err instanceof Error ? err.message : t('activateFailed'))
|
||||
} finally {
|
||||
setActivating(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="mx-auto max-w-2xl space-y-4 pt-8">
|
||||
<LicenseStatusBanners info={info} pro={pro} onAction={focusKey} />
|
||||
<InstanceIdRow instanceId={info?.instanceId} />
|
||||
<Card>
|
||||
<CardHeader className="pb-3">
|
||||
<CardTitle className="flex items-center gap-2 text-base">
|
||||
<Lock className="size-4 text-warning" />
|
||||
{t('gate.title', { feature: t(`features.${featureKey}`) })}
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<p className="text-sm text-muted-foreground">{t('gate.desc')}</p>
|
||||
<ul className="space-y-1.5 text-sm">
|
||||
{['log_analysis', 'log_alerts', 'ai_ops', 'audit_log'].map(f => (
|
||||
<li key={f} className="flex items-center gap-2">
|
||||
<Sparkles className="size-3.5 text-primary" />
|
||||
{t(`features.${f}`)}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
{/* 价卡暂留;product/plan 建好后再对齐 trial/年付/永久 */}
|
||||
<div className="grid gap-3 sm:grid-cols-2">
|
||||
<div className="rounded-lg border p-4">
|
||||
<p className="text-sm font-semibold">{t('pricing.personalTitle')}</p>
|
||||
<p className="mt-1 text-2xl font-bold text-primary">¥199<span className="text-sm font-normal text-muted-foreground"> {t('pricing.permanent')}</span></p>
|
||||
<p className="mt-1 text-xs text-muted-foreground">{t('pricing.personalDesc')}</p>
|
||||
</div>
|
||||
<div className="rounded-lg border border-primary/40 p-4">
|
||||
<p className="text-sm font-semibold">{t('pricing.teamTitle')}</p>
|
||||
<p className="mt-1 text-2xl font-bold text-primary">¥499<span className="text-sm font-normal text-muted-foreground"> {t('pricing.permanent')}</span></p>
|
||||
<p className="mt-1 text-xs text-muted-foreground">{t('pricing.teamDesc')}</p>
|
||||
</div>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs text-muted-foreground">{t('activateHint')}</p>
|
||||
<div className="flex gap-2">
|
||||
<Input
|
||||
ref={inputRef}
|
||||
value={key}
|
||||
onChange={e => setKey(e.target.value)}
|
||||
placeholder={t('gate.keyPlaceholder')}
|
||||
className="font-mono text-sm"
|
||||
/>
|
||||
<Button onClick={() => void activate()} disabled={activating || !key.trim()}>{t('gate.activate')}</Button>
|
||||
<Button variant="outline" onClick={() => navigate('/settings')}>{t('gate.toSettings')}</Button>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs text-muted-foreground">{t('gate.freeHint')}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
)
|
||||
})
|
||||
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user