审计发现 5 处硬伤(/metrics 空壳、零备份、prod 双副本数据分裂、零审计、资源偏紧),
本轮全部落地。
=== P0-1 真实 /metrics(原来是空壳)===
原实现只返回一行注释 `# LokiStack Metrics`,注释却写着"可被 Prometheus 抓取",
平台挂了/DB 写不进去/告警引擎卡死都只能靠用户反馈。
现接入 prometheus/client_golang,四层指标:
HTTP : lokistack_http_requests_total / _duration_seconds / _in_flight_requests
存储 : lokistack_db_size_bytes / _db_save_duration_seconds / _db_save_total / _db_records
业务 : lokistack_deployments{status,mode} / lokistack_collectors{status,type,run_mode}
后台任务: lokistack_alert_evaluations_total / _duration_seconds / lokistack_loki_queries_total
+ go_* / process_*(GoCollector、ProcessCollector)
关键点:path 标签用 c.FullPath() 路由模板,不用原始 URL —— 否则 117 个 Deployment
会把 Prometheus 基数炸掉。指标定义下沉 pkg/metrics 叶子包,避开
services ↔ repository 循环依赖。
=== P0-2 备份与恢复(原来 0 处代码)===
平台全部状态在 /app/data/opsxloki.json(51KB)+ kubeconfigs/ + known_hosts/,
DB 丢了 = 失去"这些资源是谁建的"的映射,从此无法管理/卸载/升级在跑的 Loki。
CLI : lokistack backup --output <path> / restore <pkg> / self-check / version
API : GET /system/backup(下载)/ POST /system/restore(校验暂存)/
POST /system/restore/apply(确认应用)/ GET /system/backups
K8s : kubernetes/pdb-backup.yaml —— 每日 03:30 CronJob,保留最近 7 份
安全设计:
- manifest.json 记录 ENCRYPTION_KEY 的 sha256 前 16 位**指纹**,用于恢复时
校验"这把钥匙能不能解开这份备份",不泄露密钥本身
- 恢复分两步(校验暂存 → 确认应用),应用前自动把当前 DB 备份为
opsxloki.json.before-restore.<时间戳>
- 包内不含明文密钥(kubeconfig 本就是加密态)
=== P0-3 prod compose 双副本 → 单副本 ===
lokistack-1 / lokistack-2 各挂独立数据卷 + nginx 轮询 = 数据分裂:
页面刷新两次可能看到不同结果,写操作只落一个副本。不是高可用。
已改单实例(nginx 仅做反代),文件顶部与 nginx.conf 都写明原因。
=== P0-4 操作审计日志(借鉴 /usr/local/src/aiops 的字段口径)===
aiops 是每个 controller 各写 recordXxxAudit();本平台有 60+ 写接口,
逐个埋点必然漏,漏掉的那个恰好最危险(删集群、改存储凭证)。
改为 **gin 中间件统一记录** + handler 通过 SetAudit 补充业务语义:
models/audit.go : 字段对齐 aiops,另加 Method/Path/Before/After
services/audit.go : 写入 + 90 天/2 万条自动裁剪 + 脱敏 + diff 生成
middleware/audit.go : 自动识别 module/action/resource,跳过只读接口
controllers/audit.go: 列表/详情/统计(仅 admin)
敏感字段(password/secretKey/privateKey/token)写入前统一替换为 ***。
=== P1 稳定性 ===
- 平台资源 1C/1Gi → 4C/4Gi 限(500m/512Mi 请求):JSON DB 全量读写 +
每分钟告警评估 + 批量发布并发,1Gi 会 OOM
- terminationGracePeriodSeconds: 30 —— 收到 SIGTERM 要等 DB 完整落盘,
否则 kubelet SIGKILL 会把 JSON 写一半(全库丢失)
- liveness timeoutSeconds 10s(默认 1s 写盘时会被误杀)
- PDB minAvailable: 0 —— 让 kubectl drain 正确等待重建,不卡维护
- DB schemaVersion + migrateSchema() 版本递进迁移(原先加字段全靠零值兜底)
- scripts/release.sh —— 不可变 tag <版本>-<日期>-<commit> + IfNotPresent,
根治 latest 并发构建互相覆盖导致的静默回滚(本项目已实际踩过)
- PVC 保持 30Gi(按长期增长预留,云盘单价低,不做在线扩容)
=== P2 易用性 ===
- 一键自检 GET /system/self-check + 仪表盘自检卡:真实探活 Loki /ready、
集群凭据解密、磁盘空间、主机状态、近 24h 失败操作
- 审计日志页 /audit:统计卡 + 用户名/模块/结果筛选 + 变更前后 diff 弹窗
- 菜单收敛 13 → 7(页内 tab 切换,URL 全部不变):
资源接入 = 集群 + 主机 + 存储
日志服务 = Loki 实例 + 采集器
日志 = 查询 + 分析
系统管理 = 用户 + 审计 + 设置
新增 GroupTabs 组件 + meta.group/groupKey,侧栏按分组去重并正确高亮
测试:go vet 通过、go test 全绿、tsc 通过
153 lines
6.5 KiB
TypeScript
153 lines
6.5 KiB
TypeScript
// 路由配置 — 全站唯一路由声明处。
|
||
// 页面组件一律 React.lazy 懒加载;菜单(menuItems)与页面标题(findRouteMeta)
|
||
// 均由本文件派生,新增页面只需在这里加一条配置。
|
||
import { lazy } from 'react';
|
||
import {
|
||
LayoutDashboard,
|
||
Server,
|
||
Monitor,
|
||
HardDrive,
|
||
Rocket,
|
||
RadioTower,
|
||
Settings,
|
||
UsersRound,
|
||
Wand2,
|
||
ScrollText,
|
||
BellRing,
|
||
BarChart3,
|
||
History,
|
||
Boxes,
|
||
} from 'lucide-react';
|
||
import type { AppRoute, MenuItem, RouteMeta } from '@/types/router';
|
||
|
||
const Login = lazy(() => import('@/pages/login'));
|
||
const Dashboard = lazy(() => import('@/pages/dashboard'));
|
||
const Clusters = lazy(() => import('@/pages/clusters'));
|
||
const Hosts = lazy(() => import('@/pages/hosts'));
|
||
const Storage = lazy(() => import('@/pages/storage'));
|
||
const Deployments = lazy(() => import('@/pages/deployments'));
|
||
const Collectors = lazy(() => import('@/pages/collectors'));
|
||
const SettingsPage = lazy(() => import('@/pages/settings'));
|
||
const NotFound = lazy(() => import('@/pages/not-found'));
|
||
const WizardPage = lazy(() => import('@/pages/wizard'));
|
||
const LogsPage = lazy(() => import('@/pages/logs'));
|
||
const AlertsPage = lazy(() => import('@/pages/alerts'));
|
||
const UsersPage = lazy(() => import('@/pages/users'));
|
||
const AnalysisPage = lazy(() => import('@/pages/analysis'));
|
||
const AuditPage = lazy(() => import('@/pages/audit'));
|
||
|
||
export const routes: AppRoute[] = [
|
||
{ path: '/login', component: Login, auth: false, meta: { title: 'menu.login', hideInMenu: true } },
|
||
{ path: '/', component: Dashboard, auth: true, meta: { title: 'menu.dashboard', icon: LayoutDashboard, order: 1, adminOnly: true } },
|
||
{ path: '/wizard', component: WizardPage, auth: true, meta: { title: 'menu.wizard', icon: Wand2, order: 2, adminOnly: true } },
|
||
// ── 资源接入(集群 / 主机 / 存储):装完基本不动的配置,合并为一个入口 ──
|
||
{
|
||
path: '/clusters', component: Clusters, auth: true,
|
||
meta: {
|
||
title: 'menu.clusters', icon: Server, order: 3, adminOnly: true,
|
||
group: { key: 'resources', label: 'menu.groups.resources', icon: Boxes, order: 3 },
|
||
},
|
||
},
|
||
{ path: '/hosts', component: Hosts, auth: true, meta: { title: 'menu.hosts', icon: Monitor, order: 4, adminOnly: true, groupKey: 'resources' } },
|
||
{ path: '/storage', component: Storage, auth: true, meta: { title: 'menu.storage', icon: HardDrive, order: 5, adminOnly: true, groupKey: 'resources' } },
|
||
|
||
// ── 日志服务(Loki 实例 / 采集器):部署与采集是同一件事的两半 ──
|
||
{
|
||
path: '/deployments', component: Deployments, auth: true,
|
||
meta: {
|
||
title: 'menu.deployments', icon: Rocket, order: 4, adminOnly: true,
|
||
group: { key: 'services', label: 'menu.groups.services', icon: Rocket, order: 4 },
|
||
},
|
||
},
|
||
{ path: '/collectors', component: Collectors, auth: true, meta: { title: 'menu.collectors', icon: RadioTower, order: 7, adminOnly: true, groupKey: 'services' } },
|
||
|
||
// ── 日志(查询 / 分析):同一个数据源的不同视角 ──
|
||
{
|
||
path: '/logs', component: LogsPage, auth: true,
|
||
meta: {
|
||
title: 'menu.logs', icon: ScrollText, order: 5,
|
||
group: { key: 'logs', label: 'menu.groups.logs', icon: ScrollText, order: 5 },
|
||
},
|
||
},
|
||
{ path: '/analysis', component: AnalysisPage, auth: true, meta: { title: 'menu.analysis', icon: BarChart3, order: 9, adminOnly: true, groupKey: 'logs' } },
|
||
|
||
// ── 告警:独立入口(要盯的,不能被折叠) ──
|
||
{ path: '/alerts', component: AlertsPage, auth: true, meta: { title: 'menu.alerts', icon: BellRing, order: 6 } },
|
||
|
||
// ── 系统管理(用户 / 审计 / 设置):低频管理动作 ──
|
||
{
|
||
path: '/users', component: UsersPage, auth: true,
|
||
meta: {
|
||
title: 'menu.users', icon: UsersRound, order: 7, adminOnly: true,
|
||
group: { key: 'system', label: 'menu.groups.system', icon: Settings, order: 7 },
|
||
},
|
||
},
|
||
{ path: '/audit', component: AuditPage, auth: true, meta: { title: 'menu.audit', icon: History, order: 13, adminOnly: true, groupKey: 'system' } },
|
||
{ path: '/settings', component: SettingsPage, auth: true, meta: { title: 'menu.settings', icon: Settings, order: 12, adminOnly: true, groupKey: 'system' } },
|
||
{ path: '*', component: NotFound, auth: false, meta: { title: 'menu.notFound', hideInMenu: true } },
|
||
];
|
||
|
||
/**
|
||
* 侧栏菜单:登录后可见的路由,按 order 排序。
|
||
*
|
||
* 分组规则(解决"菜单太多"的观感问题):
|
||
* - 带 meta.group 的路由是一个分组的**代表入口**,菜单显示分组的 label/icon
|
||
* - 带 meta.groupKey 的路由说明它属于某个已有分组 → 不进菜单,只在页内 tab 出现
|
||
* 这样 13 个路由收敛成 7 个菜单项,且 URL 全部保持不变(书签不会失效)。
|
||
*/
|
||
export const menuItems: MenuItem[] = (() => {
|
||
const grouped = new Set<string>();
|
||
const items: MenuItem[] = [];
|
||
|
||
const sorted = routes
|
||
.filter((route) => route.auth && !route.meta.hideInMenu)
|
||
.sort((a, b) => {
|
||
const orderA = a.meta.group?.order ?? a.meta.order ?? 0;
|
||
const orderB = b.meta.group?.order ?? b.meta.order ?? 0;
|
||
return orderA - orderB;
|
||
});
|
||
|
||
for (const route of sorted) {
|
||
const { group, groupKey } = route.meta;
|
||
// 属于某个分组的非代表路由:不占菜单位
|
||
if (groupKey && !group) continue;
|
||
if (group) {
|
||
if (grouped.has(group.key)) continue;
|
||
grouped.add(group.key);
|
||
items.push({
|
||
path: route.path,
|
||
icon: group.icon ?? route.meta.icon,
|
||
label: group.label,
|
||
adminOnly: route.meta.adminOnly,
|
||
groupKey: group.key,
|
||
});
|
||
continue;
|
||
}
|
||
items.push({
|
||
path: route.path,
|
||
icon: route.meta.icon,
|
||
label: route.meta.title,
|
||
adminOnly: route.meta.adminOnly,
|
||
});
|
||
}
|
||
return items;
|
||
})();
|
||
|
||
/** 取某个分组下的所有页面(用于页内 tab 切换),按 order 排序 */
|
||
export function findGroupRoutes(groupKey: string): AppRoute[] {
|
||
return routes
|
||
.filter((route) => route.meta.groupKey === groupKey || route.meta.group?.key === groupKey)
|
||
.sort((a, b) => (a.meta.order ?? 0) - (b.meta.order ?? 0));
|
||
}
|
||
|
||
/** 当前路径所属的分组标识(不在分组内返回 undefined) */
|
||
export function findGroupKey(pathname: string): string | undefined {
|
||
const meta = findRouteMeta(pathname);
|
||
return meta?.group?.key ?? meta?.groupKey;
|
||
}
|
||
|
||
/** 按路径查 meta(通配/未匹配返回 undefined,调用方自行兜底) */
|
||
export function findRouteMeta(pathname: string): RouteMeta | undefined {
|
||
return routes.find((route) => route.path === pathname)?.meta;
|
||
}
|