feat: opsx-loki 全面升级 — 前端重构/用户与LDAP/部署向导/日志查询
基于 WEB-UI-Development-Guide 标准的整体升级,并更名 LokiDeploy → opsx-loki。 前端(Gin+React+shadcn/ui): - 按 02/07 章重构:页面域化拆分(无超 400 行文件)、PageContainer/ListQuery、 反馈分级(成功静默/失败 toast/危险确认)、401 单飞刷新+队列重放 - 暗色模式(next-themes)+ 语义 Token(--primary 品牌色/success/warning/info) - 部署向导(状态推导四步:环境→存储→Loki→采集,内嵌表单闭环) - 日志查询页:LogQL 标签构建器/拖拽直方图/日志上下文/Live Tail/查询历史 - 按用户+标签策略的查询授权(服务端强制合并策略) 后端(Go+Gin+JSON DB): - 简单用户管理(仅 admin,无 RBAC)+ TokenVersion 会话吊销 + 登录 IP 限流 - LDAP 接入:配置(AES 加密存储/hasPassword 脱敏)/测试连接/批量导入/登录 - 存储配置测试连接(SigV4 零依赖)+ 桶不存在自动创建 - Loki 三模式部署修复:SSH 密钥认证、3.x 配置模板、ruler 对象存储嵌套块、 凭证解密、孤儿 deploying 状态恢复、Grafana 部署(可选默认关)+面板导入 - DaemonSet 采集纯 YAML(去 helm 依赖)+ DNS 自愈回退 ClusterIP + 集群名标签 - 日志查询代理:cluster-svc 经 kube-apiserver proxy/url 直连 + cursor 翻页 - 统一 log 命名空间;部署就绪探活;Grafana 访问地址自动记录 修复:run.sh 增量编译检测、Sanitized 脱敏补全、LokiDeployment Update 白名单、 健康检查 CreatedAt 零值等 10+ 存量 bug。 数据兼容:DB 沿用旧 lokideploy.json(新名优先);localStorage 一次性迁移; HKDF 加密前缀保留(已加密凭证不受影响)。
This commit is contained in:
logan committed
1 parent
327e282356
commit
f9c6c1d826
204 files changed
+14107
-9103
No files matched your search
@@ -10,8 +10,8 @@ import (
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"lokideploy/models"
|
||||
"lokideploy/pkg/crypto"
|
||||
"opsxloki/models"
|
||||
"opsxloki/pkg/crypto"
|
||||
)
|
||||
|
||||
// LokiDeployer 负责 Loki 的三种部署模式：k8s-helm / k8s-yaml / host-binary
|
||||
@@ -34,16 +34,30 @@ func (d *LokiDeployer) Deploy(ctx context.Context) error {
|
||||
d.log("warn", err.Error(), "")
|
||||
}
|
||||
|
||||
var err error
|
||||
switch d.deployment.Mode {
|
||||
case "k8s-helm":
|
||||
return d.deployHelm(ctx)
|
||||
err = d.deployHelm(ctx)
|
||||
case "k8s-yaml":
|
||||
return d.deployYaml(ctx)
|
||||
err = d.deployYaml(ctx)
|
||||
case "host-binary":
|
||||
return d.deployBinary(ctx)
|
||||
err = d.deployBinary(ctx)
|
||||
default:
|
||||
return fmt.Errorf("unsupported deployment mode: %s", d.deployment.Mode)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Loki 就绪后,按需同环境部署 Grafana(数据源自动指向本 Loki)
|
||||
if d.deployment.GrafanaEnabled {
|
||||
g := &GrafanaDeployer{deployment: d.deployment, log: d.log}
|
||||
if gerr := g.Deploy(ctx); gerr != nil {
|
||||
d.log("error", fmt.Sprintf("Grafana deployment failed: %v", gerr), "")
|
||||
return fmt.Errorf("grafana deployment failed: %w", gerr)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ==================== Helm 部署 (grafana/loki v6+) ====================
|
||||
@@ -169,12 +183,7 @@ func (d *LokiDeployer) deployBinary(ctx context.Context) error {
|
||||
return fmt.Errorf("please select target host")
|
||||
}
|
||||
host := d.deployment.Host
|
||||
password := host.Password
|
||||
if strings.HasPrefix(password, "ENC:") {
|
||||
password, _ = DecryptCred(password)
|
||||
}
|
||||
|
||||
client, err := NewSSHClient(host.HostIP, host.Port, host.User, password)
|
||||
client, err := NewSSHClientWithAuth(HostSSHAuth(host))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -696,7 +705,6 @@ common:
|
||||
filesystem:
|
||||
chunks_directory: /var/lib/loki/chunks
|
||||
rules_directory: /var/lib/loki/rules
|
||||
admin_api_directory: /var/lib/loki/admin
|
||||
replication_factor: {{ .ReplicationFactor }}
|
||||
ring:
|
||||
instance_addr: {{ .RingInstanceAddr }}
|
||||
@@ -730,20 +738,11 @@ ingester:
|
||||
chunk_target_size: 1048576
|
||||
chunk_encoding: snappy
|
||||
max_chunk_age: 2h
|
||||
max_transfer_retries: 0
|
||||
autoforget_unhealthy: true
|
||||
|
||||
querier:
|
||||
max_concurrent: 10
|
||||
query_ingesters_within: 2h
|
||||
engine:
|
||||
timeout: 3m
|
||||
max_look_back_period: 30s
|
||||
|
||||
query_frontend:
|
||||
max_outstanding_per_tenant: 4096
|
||||
compress_responses: true
|
||||
scheduler_address: ""
|
||||
|
||||
query_scheduler:
|
||||
max_outstanding_requests_per_tenant: 4096
|
||||
@@ -786,6 +785,9 @@ schema_config:
|
||||
ruler:
|
||||
storage:
|
||||
type: {{ .RulerStoreType }}
|
||||
{{- if .RulerStorageBlock }}
|
||||
{{ .RulerStorageBlock }}
|
||||
{{- end }}
|
||||
{{- if eq .RulerStoreType "local" }}
|
||||
local:
|
||||
directory: /var/lib/loki/rules
|
||||
@@ -802,12 +804,7 @@ frontend:
|
||||
max_outstanding_per_tenant: 4096
|
||||
compress_responses: true
|
||||
|
||||
frontend_worker:
|
||||
parallelism: 10
|
||||
|
||||
table_manager:
|
||||
retention_deletes_enabled: true
|
||||
retention_period: {{ .RetentionPeriod }}h
|
||||
frontend_worker: {}
|
||||
|
||||
chunk_store_config:
|
||||
chunk_cache_config:
|
||||
@@ -831,6 +828,7 @@ type lokiConfigData struct {
|
||||
RingInstanceAddr string // 单副本 127.0.0.1；多副本 0.0.0.0（须可被对端寻址）
|
||||
RingKVStore string // 单副本 inmemory；多副本 memberlist
|
||||
MemberlistBlock string // 多副本时非空，注入 memberlist 顶级配置
|
||||
RulerStorageBlock string // 对象存储时非空，注入 ruler.storage.s3/gcs/azure 嵌套配置
|
||||
}
|
||||
|
||||
func (d *LokiDeployer) generateLokiConfig() string {
|
||||
@@ -870,7 +868,7 @@ func (d *LokiDeployer) generateLokiConfig() string {
|
||||
ringStore = "memberlist"
|
||||
ns := d.deployment.Namespace
|
||||
if ns == "" {
|
||||
ns = "loki"
|
||||
ns = "log"
|
||||
}
|
||||
memberlistBlock = fmt.Sprintf(`memberlist:
|
||||
join_members:
|
||||
@@ -892,6 +890,7 @@ func (d *LokiDeployer) generateLokiConfig() string {
|
||||
RingInstanceAddr: ringAddr,
|
||||
RingKVStore: ringStore,
|
||||
MemberlistBlock: memberlistBlock,
|
||||
RulerStorageBlock: d.buildRulerStorageBlock(),
|
||||
}
|
||||
|
||||
tmpl, err := template.New("loki-config").Parse(lokiConfigTemplate)
|
||||
@@ -962,6 +961,63 @@ func (d *LokiDeployer) buildStorageConfigBlock() string {
|
||||
}
|
||||
}
|
||||
|
||||
// buildRulerStorageBlock 生成 ruler.storage 下与存储类型同名的嵌套配置
|
||||
// (Loki 3.x 的 ruler 存储客户端不回退读取 storage_config,缺少此块会以空 bucket 启动失败)
|
||||
func (d *LokiDeployer) buildRulerStorageBlock() string {
|
||||
if d.deployment.StorageConfig == nil {
|
||||
return ""
|
||||
}
|
||||
sc := d.deployment.StorageConfig
|
||||
switch sc.StorageType {
|
||||
case "s3":
|
||||
endpoint := sc.Endpoint
|
||||
if endpoint == "" {
|
||||
endpoint = fmt.Sprintf("s3.%s.amazonaws.com", sc.Region)
|
||||
}
|
||||
insecure := "false"
|
||||
if sc.Insecure {
|
||||
insecure = "true"
|
||||
}
|
||||
return fmt.Sprintf(` s3:
|
||||
s3: s3://%s/%s
|
||||
endpoint: %s
|
||||
region: %s
|
||||
access_key_id: %s
|
||||
secret_access_key: %s
|
||||
insecure: %s
|
||||
s3forcepathstyle: true`, sc.Bucket, sc.PathPrefix, endpoint, sc.Region, sc.AccessKey, sc.SecretKey, insecure)
|
||||
case "oss":
|
||||
return fmt.Sprintf(` s3:
|
||||
s3: s3://%s/%s
|
||||
endpoint: %s
|
||||
access_key_id: %s
|
||||
secret_access_key: %s
|
||||
s3forcepathstyle: true`, sc.Bucket, sc.PathPrefix, sc.Endpoint, sc.AccessKey, sc.SecretKey)
|
||||
case "minio":
|
||||
insecure := "false"
|
||||
if sc.Insecure {
|
||||
insecure = "true"
|
||||
}
|
||||
return fmt.Sprintf(` s3:
|
||||
s3: s3://%s/%s
|
||||
endpoint: %s
|
||||
access_key_id: %s
|
||||
secret_access_key: %s
|
||||
insecure: %s
|
||||
s3forcepathstyle: true`, sc.Bucket, sc.PathPrefix, sc.Endpoint, sc.AccessKey, sc.SecretKey, insecure)
|
||||
case "gcs":
|
||||
return fmt.Sprintf(` gcs:
|
||||
bucket_name: %s`, sc.Bucket)
|
||||
case "azure":
|
||||
return fmt.Sprintf(` azure:
|
||||
container_name: %s
|
||||
account_name: %s
|
||||
account_key: %s`, sc.Bucket, sc.AccessKey, sc.SecretKey)
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// generateLokiConfigFallback 当模板解析失败时的降级配置
|
||||
func (d *LokiDeployer) generateLokiConfigFallback(retentionHours int) string {
|
||||
return fmt.Sprintf(`auth_enabled: %v
|
||||
@@ -996,8 +1052,6 @@ ingester:
|
||||
max_chunk_age: 2h
|
||||
querier:
|
||||
max_concurrent: 10
|
||||
query_frontend:
|
||||
max_outstanding_per_tenant: 4096
|
||||
compactor:
|
||||
working_directory: /var/lib/loki/compactor
|
||||
retention_enabled: true
|
||||
@@ -1033,7 +1087,7 @@ func (d *LokiDeployer) effectiveNamespace() string {
|
||||
if d.deployment.Namespace != "" {
|
||||
return d.deployment.Namespace
|
||||
}
|
||||
return "loki"
|
||||
return "log"
|
||||
}
|
||||
|
||||
func (d *LokiDeployer) effectiveReleaseName() string {
|
||||
@@ -1075,6 +1129,11 @@ func (d *LokiDeployer) log(level, message, detail string) {
|
||||
// ==================== 清理 / 卸载 ====================
|
||||
|
||||
func CleanupLoki(ctx context.Context, deployment *models.LokiDeployment) {
|
||||
// 同环境部署的 Grafana 一并清理
|
||||
if deployment.GrafanaEnabled {
|
||||
CleanupGrafana(ctx, deployment)
|
||||
}
|
||||
|
||||
if deployment.Mode == "k8s-helm" && deployment.Cluster != nil {
|
||||
kubeconfig, err := resolveKubeconfig(deployment.Cluster)
|
||||
if err != nil {
|
||||
@@ -1089,7 +1148,7 @@ func CleanupLoki(ctx context.Context, deployment *models.LokiDeployment) {
|
||||
env := append(os.Environ(), fmt.Sprintf("KUBECONFIG=%s", kubeFile))
|
||||
ns := deployment.Namespace
|
||||
if ns == "" {
|
||||
ns = "loki"
|
||||
ns = "log"
|
||||
}
|
||||
name := deployment.Name
|
||||
if name == "" {
|
||||
@@ -1121,7 +1180,7 @@ func CleanupLoki(ctx context.Context, deployment *models.LokiDeployment) {
|
||||
env := append(os.Environ(), fmt.Sprintf("KUBECONFIG=%s", kubeFile))
|
||||
ns := deployment.Namespace
|
||||
if ns == "" {
|
||||
ns = "loki"
|
||||
ns = "log"
|
||||
}
|
||||
|
||||
// 删除所有相关资源
|
||||
@@ -1140,11 +1199,7 @@ func CleanupLoki(ctx context.Context, deployment *models.LokiDeployment) {
|
||||
|
||||
if deployment.Mode == "host-binary" && deployment.Host != nil {
|
||||
host := deployment.Host
|
||||
password := host.Password
|
||||
if strings.HasPrefix(password, "ENC:") {
|
||||
password, _ = DecryptCred(password)
|
||||
}
|
||||
client, _ := NewSSHClient(host.HostIP, host.Port, host.User, password)
|
||||
client, _ := NewSSHClientWithAuth(HostSSHAuth(host))
|
||||
if client != nil {
|
||||
defer client.Close()
|
||||
client.Execute(ctx, "systemctl stop loki 2>/dev/null || true")
|
||||
|
||||
Reference in new issue
Block a user