feat: 运维标杆化打磨 P0/P1/P2 —— 审计日志、真实指标、备份恢复、菜单收敛

审计发现 5 处硬伤(/metrics 空壳、零备份、prod 双副本数据分裂、零审计、资源偏紧),
本轮全部落地。

=== P0-1 真实 /metrics(原来是空壳)===
原实现只返回一行注释 `# LokiStack Metrics`,注释却写着"可被 Prometheus 抓取",
平台挂了/DB 写不进去/告警引擎卡死都只能靠用户反馈。
现接入 prometheus/client_golang,四层指标:
  HTTP    : lokistack_http_requests_total / _duration_seconds / _in_flight_requests
  存储    : lokistack_db_size_bytes / _db_save_duration_seconds / _db_save_total / _db_records
  业务    : lokistack_deployments{status,mode} / lokistack_collectors{status,type,run_mode}
  后台任务: lokistack_alert_evaluations_total / _duration_seconds / lokistack_loki_queries_total
  + go_* / process_*(GoCollector、ProcessCollector)

关键点:path 标签用 c.FullPath() 路由模板,不用原始 URL —— 否则 117 个 Deployment
会把 Prometheus 基数炸掉。指标定义下沉 pkg/metrics 叶子包,避开
services ↔ repository 循环依赖。

=== P0-2 备份与恢复(原来 0 处代码)===
平台全部状态在 /app/data/opsxloki.json(51KB)+ kubeconfigs/ + known_hosts/,
DB 丢了 = 失去"这些资源是谁建的"的映射,从此无法管理/卸载/升级在跑的 Loki。

  CLI : lokistack backup --output <path> / restore <pkg> / self-check / version
  API : GET /system/backup(下载)/ POST /system/restore(校验暂存)/
        POST /system/restore/apply(确认应用)/ GET /system/backups
  K8s : kubernetes/pdb-backup.yaml —— 每日 03:30 CronJob,保留最近 7 份

安全设计:
  - manifest.json 记录 ENCRYPTION_KEY 的 sha256 前 16 位**指纹**,用于恢复时
    校验"这把钥匙能不能解开这份备份",不泄露密钥本身
  - 恢复分两步(校验暂存 → 确认应用),应用前自动把当前 DB 备份为
    opsxloki.json.before-restore.<时间戳>
  - 包内不含明文密钥(kubeconfig 本就是加密态)

=== P0-3 prod compose 双副本 → 单副本 ===
lokistack-1 / lokistack-2 各挂独立数据卷 + nginx 轮询 = 数据分裂:
页面刷新两次可能看到不同结果,写操作只落一个副本。不是高可用。
已改单实例(nginx 仅做反代),文件顶部与 nginx.conf 都写明原因。

=== P0-4 操作审计日志(借鉴 /usr/local/src/aiops 的字段口径)===
aiops 是每个 controller 各写 recordXxxAudit();本平台有 60+ 写接口,
逐个埋点必然漏,漏掉的那个恰好最危险(删集群、改存储凭证)。
改为 **gin 中间件统一记录** + handler 通过 SetAudit 补充业务语义:
  models/audit.go     : 字段对齐 aiops,另加 Method/Path/Before/After
  services/audit.go   : 写入 + 90 天/2 万条自动裁剪 + 脱敏 + diff 生成
  middleware/audit.go : 自动识别 module/action/resource,跳过只读接口
  controllers/audit.go: 列表/详情/统计(仅 admin)
敏感字段(password/secretKey/privateKey/token)写入前统一替换为 ***。

=== P1 稳定性 ===
- 平台资源 1C/1Gi → 4C/4Gi 限(500m/512Mi 请求):JSON DB 全量读写 +
  每分钟告警评估 + 批量发布并发,1Gi 会 OOM
- terminationGracePeriodSeconds: 30 —— 收到 SIGTERM 要等 DB 完整落盘,
  否则 kubelet SIGKILL 会把 JSON 写一半(全库丢失)
- liveness timeoutSeconds 10s(默认 1s 写盘时会被误杀)
- PDB minAvailable: 0 —— 让 kubectl drain 正确等待重建,不卡维护
- DB schemaVersion + migrateSchema() 版本递进迁移(原先加字段全靠零值兜底)
- scripts/release.sh —— 不可变 tag <版本>-<日期>-<commit> + IfNotPresent,
  根治 latest 并发构建互相覆盖导致的静默回滚(本项目已实际踩过)
- PVC 保持 30Gi(按长期增长预留,云盘单价低,不做在线扩容)

=== P2 易用性 ===
- 一键自检 GET /system/self-check + 仪表盘自检卡:真实探活 Loki /ready、
  集群凭据解密、磁盘空间、主机状态、近 24h 失败操作
- 审计日志页 /audit:统计卡 + 用户名/模块/结果筛选 + 变更前后 diff 弹窗
- 菜单收敛 13 → 7(页内 tab 切换,URL 全部不变):
    资源接入 = 集群 + 主机 + 存储
    日志服务 = Loki 实例 + 采集器
    日志     = 查询 + 分析
    系统管理 = 用户 + 审计 + 设置
  新增 GroupTabs 组件 + meta.group/groupKey,侧栏按分组去重并正确高亮

测试:go vet 通过、go test 全绿、tsc 通过
This commit is contained in:
opsx committed 2026-10-28 21:27:01 +00:00
1 parent f6af12a731
commit b49677f6a6
37 files changed
+2983 -147

No files matched your search

+389
View File
@@ -0,0 +1,389 @@
import { memo, useCallback, useEffect, useState } from 'react'
import { useTranslation } from 'react-i18next'
import { History, ShieldAlert, Search, RefreshCw, Eye } from 'lucide-react'
import '@/i18n/domains'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
import { PageContainer } from '@/components/PageContainer'
import { LoadingSpinner } from '@/components/LoadingSpinner'
import { EmptyState } from '@/components/EmptyState'
import { ErrorState } from '@/components/ErrorState'
import { PaginationBar } from '@/components/PaginationBar'
import { systemApi, type AuditLog, type AuditStats } from '@/api/system'
import { toast } from 'sonner'
const PAGE_SIZE = 20
/** 模块 → 中文名 + 配色。未知模块回退到原样显示。 */
const MODULE_META: Record<string, { label: string; className: string }> = {
deployments: { label: '部署', className: 'bg-info/10 text-info' },
collectors: { label: '采集器', className: 'bg-warning/10 text-warning' },
storage: { label: '存储', className: 'bg-success/10 text-success' },
clusters: { label: '集群', className: 'bg-primary/10 text-primary' },
hosts: { label: '主机', className: 'bg-primary/10 text-primary' },
users: { label: '用户', className: 'bg-destructive/10 text-destructive' },
logs: { label: '日志', className: 'bg-muted text-muted-foreground' },
alerts: { label: '告警', className: 'bg-warning/10 text-warning' },
auth: { label: '登录', className: 'bg-muted text-muted-foreground' },
system: { label: '系统', className: 'bg-destructive/10 text-destructive' },
}
const ACTION_META: Record<string, string> = {
create: '新增',
update: '修改',
delete: '删除',
deploy: '部署',
stop: '停止',
restart: '重启',
install: '安装',
uninstall: '卸载',
test: '测试',
validate: '校验',
'reset-password': '重置密码',
login: '登录',
'change-password': '改密码',
}
function ModuleBadge({ module }: { module: string }) {
const meta = MODULE_META[module] ?? { label: module || '-', className: 'bg-muted text-muted-foreground' }
return (
<span className={`inline-flex items-center px-2 py-0.5 rounded-full text-xs font-medium ${meta.className}`}>
{meta.label}
</span>
)
}
function StatusBadge({ status }: { status: number }) {
const ok = status >= 200 && status < 400
return (
<span
className={`inline-flex items-center px-2 py-0.5 rounded-full text-xs font-medium ${
ok ? 'bg-success/10 text-success' : 'bg-destructive/10 text-destructive'
}`}
>
{status}
</span>
)
}
/** 详情弹窗:展示 Before/After 变更 diff 与错误原文 */
const DetailDialog = memo(function DetailDialog({
entry,
onClose,
}: {
entry: AuditLog | null
onClose: () => void
}) {
if (!entry) return null
const pretty = (s?: string) => {
if (!s) return ''
try {
return JSON.stringify(JSON.parse(s), null, 2)
} catch {
return s
}
}
return (
<div
className="fixed inset-0 z-50 flex items-center justify-center bg-black/50 p-4"
onClick={onClose}
role="presentation"
>
<div
className="w-full max-w-3xl max-h-[80vh] overflow-auto rounded-lg border border-border bg-background p-5 shadow-xl"
onClick={e => e.stopPropagation()}
role="dialog"
aria-modal="true"
>
<div className="flex items-start justify-between gap-4">
<div>
<h3 className="text-base font-semibold">
{ACTION_META[entry.action] ?? entry.action} · {entry.resource || entry.module}
{entry.resourceId ? ` #${entry.resourceId}` : ''}
</h3>
<p className="mt-1 text-xs text-muted-foreground">
{entry.username || '-'}({entry.role || '-'}) · {entry.ip} ·{' '}
{new Date(entry.createdAt).toLocaleString()} · {entry.durationMs}ms
</p>
</div>
<Button variant="outline" size="sm" onClick={onClose}>
关闭
</Button>
</div>
{entry.detail && (
<div className="mt-4">
<div className="text-xs font-medium text-muted-foreground mb-1">说明</div>
<p className="text-sm">{entry.detail}</p>
</div>
)}
{(entry.before || entry.after) && (
<div className="mt-4 grid gap-3 sm:grid-cols-2">
<div>
<div className="text-xs font-medium text-muted-foreground mb-1">变更前</div>
<pre className="max-h-60 overflow-auto rounded-md border border-border bg-muted/30 p-2 text-xs">
{pretty(entry.before) || '(无)'}
</pre>
</div>
<div>
<div className="text-xs font-medium text-muted-foreground mb-1">变更后</div>
<pre className="max-h-60 overflow-auto rounded-md border border-border bg-muted/30 p-2 text-xs">
{pretty(entry.after) || '(无)'}
</pre>
</div>
</div>
)}
{entry.error && (
<div className="mt-4">
<div className="text-xs font-medium text-destructive mb-1">失败原因</div>
<pre className="max-h-40 overflow-auto rounded-md border border-destructive/30 bg-destructive/5 p-2 text-xs text-destructive">
{entry.error}
</pre>
</div>
)}
<div className="mt-4 text-xs text-muted-foreground">
{entry.method} {entry.path}
{entry.userAgent ? ` · ${entry.userAgent}` : ''}
</div>
</div>
</div>
)
})
/**
* 审计日志页(admin 专用)
*
* 定位:回答「谁在什么时候改了什么」。平台的全部写操作(部署/采集/存储/集群/
* 主机/用户/策略/告警)都由后端中间件自动记录,handler 会补充业务语义与变更 diff。
* 数据保留 90 天 / 最多 2 万条。
*/
export default function AuditPage() {
const { t } = useTranslation('audit')
const [list, setList] = useState<AuditLog[]>([])
const [total, setTotal] = useState(0)
const [page, setPage] = useState(1)
const [stats, setStats] = useState<AuditStats | null>(null)
const [loading, setLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const [detail, setDetail] = useState<AuditLog | null>(null)
// 过滤条件
const [username, setUsername] = useState('')
const [module, setModule] = useState('')
const [statusFilter, setStatusFilter] = useState(0)
const load = useCallback(async () => {
setLoading(true)
setError(null)
try {
const [res, st] = await Promise.all([
systemApi.auditList({ page, pageSize: PAGE_SIZE, username, module, status: statusFilter }),
systemApi.auditStats(),
])
setList(Array.isArray(res.list) ? res.list : [])
setTotal(res.total ?? 0)
setStats(st ?? null)
} catch (e) {
setError(e instanceof Error ? e.message : t('loadFailed'))
} finally {
setLoading(false)
}
}, [page, username, module, statusFilter, t])
useEffect(() => {
void load()
}, [load])
// 过滤条件变化时回到第一页,避免停在空白页
useEffect(() => {
setPage(1)
}, [username, module, statusFilter])
const cards = [
{ label: t('stat.today'), value: stats?.todayCount ?? 0, icon: History },
{ label: t('stat.week'), value: stats?.weekCount ?? 0, icon: History },
{
label: t('stat.failed'),
value: stats?.failCount ?? 0,
icon: ShieldAlert,
danger: (stats?.failCount ?? 0) > 0,
},
{ label: t('stat.total'), value: stats?.totalCount ?? 0, icon: History },
]
return (
<PageContainer
title={t('title')}
description={t('description')}
action={
<Button variant="outline" size="sm" onClick={() => void load()} disabled={loading}>
<RefreshCw className={`w-4 h-4 mr-1.5 ${loading ? 'animate-spin' : ''}`} />
{t('refresh')}
</Button>
}
>
{/* 统计卡片 */}
<div className="grid gap-3 sm:grid-cols-2 lg:grid-cols-4">
{cards.map(c => {
const Icon = c.icon
return (
<Card key={c.label}>
<CardContent className="flex items-center gap-3 p-4">
<div
className={`flex h-9 w-9 items-center justify-center rounded-md ${
c.danger ? 'bg-destructive/10 text-destructive' : 'bg-primary/10 text-primary'
}`}
>
<Icon className="h-4 w-4" />
</div>
<div>
<div className="text-xs text-muted-foreground">{c.label}</div>
<div className={`text-lg font-semibold ${c.danger ? 'text-destructive' : ''}`}>
{c.value.toLocaleString()}
</div>
</div>
</CardContent>
</Card>
)
})}
</div>
{/* 过滤条 */}
<Card>
<CardHeader className="pb-3">
<CardTitle className="text-sm font-medium">{t('filter')}</CardTitle>
</CardHeader>
<CardContent className="flex flex-wrap items-center gap-2">
<div className="relative">
<Search className="pointer-events-none absolute left-2 top-1/2 h-4 w-4 -translate-y-1/2 text-muted-foreground" />
<Input
className="w-48 pl-8"
placeholder={t('filterUsername')}
value={username}
onChange={e => setUsername(e.target.value)}
/>
</div>
<select
className="h-9 rounded-md border border-border bg-background px-2 text-sm"
value={module}
onChange={e => setModule(e.target.value)}
>
<option value="">{t('filterAllModules')}</option>
{Object.entries(MODULE_META).map(([key, meta]) => (
<option key={key} value={key}>
{meta.label}
</option>
))}
</select>
<select
className="h-9 rounded-md border border-border bg-background px-2 text-sm"
value={statusFilter}
onChange={e => setStatusFilter(Number(e.target.value))}
>
<option value={0}>{t('filterAllStatus')}</option>
<option value={1}>{t('filterSuccess')}</option>
<option value={2}>{t('filterFailed')}</option>
</select>
<span className="ml-auto text-xs text-muted-foreground">
{t('total', { count: total })}
</span>
</CardContent>
</Card>
{/* 列表 */}
<Card>
<CardContent className="p-0">
{loading ? (
<div className="flex justify-center py-16">
<LoadingSpinner />
</div>
) : error ? (
<ErrorState message={error} onRetry={() => void load()} />
) : list.length === 0 ? (
<EmptyState icon={History} title={t('emptyTitle')} description={t('emptyDesc')} />
) : (
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead className="border-b border-border bg-muted/30 text-xs text-muted-foreground">
<tr>
<th className="px-3 py-2 text-left font-medium whitespace-nowrap">{t('col.time')}</th>
<th className="px-3 py-2 text-left font-medium">{t('col.user')}</th>
<th className="px-3 py-2 text-left font-medium">{t('col.module')}</th>
<th className="px-3 py-2 text-left font-medium">{t('col.action')}</th>
<th className="px-3 py-2 text-left font-medium">{t('col.target')}</th>
<th className="px-3 py-2 text-left font-medium">{t('col.status')}</th>
<th className="px-3 py-2 text-left font-medium whitespace-nowrap">{t('col.ip')}</th>
<th className="px-3 py-2" />
</tr>
</thead>
<tbody>
{list.map(item => (
<tr
key={item.id}
className="border-b border-border/50 last:border-0 hover:bg-muted/20"
>
<td className="px-3 py-2 whitespace-nowrap text-xs text-muted-foreground">
{new Date(item.createdAt).toLocaleString()}
</td>
<td className="px-3 py-2 whitespace-nowrap">{item.username || '-'}</td>
<td className="px-3 py-2">
<ModuleBadge module={item.module} />
</td>
<td className="px-3 py-2 whitespace-nowrap">
{ACTION_META[item.action] ?? item.action}
</td>
<td className="px-3 py-2">
<div className="max-w-[260px] truncate" title={item.detail || item.resource}>
{item.detail || item.resource || '-'}
{item.resourceId ? ` #${item.resourceId}` : ''}
</div>
</td>
<td className="px-3 py-2">
<StatusBadge status={item.status} />
</td>
<td className="px-3 py-2 whitespace-nowrap text-xs text-muted-foreground">
{item.ip}
</td>
<td className="px-3 py-2 text-right">
<Button
variant="ghost"
size="sm"
onClick={async () => {
// 列表接口已带 Before/After,先直接展示;
// 若被截断再取详情(详情接口返回完整内容)
try {
const full = await systemApi.auditGet(item.id)
setDetail(full)
} catch {
setDetail(item)
toast.warning(t('detailTruncated'))
}
}}
>
<Eye className="w-4 h-4" />
</Button>
</td>
</tr>
))}
</tbody>
</table>
</div>
)}
{!loading && !error && list.length > 0 && (
<div className="border-t border-border px-3 py-2">
<PaginationBar page={page} pageSize={PAGE_SIZE} total={total} onPageChange={setPage} />
</div>
)}
</CardContent>
</Card>
<DetailDialog entry={detail} onClose={() => setDetail(null)} />
</PageContainer>
)
}
@@ -0,0 +1,120 @@
import { memo, useCallback, useEffect, useState } from 'react'
import { useTranslation } from 'react-i18next'
import { Activity, AlertTriangle, CheckCircle2, RefreshCw, XCircle } from 'lucide-react'
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
import { Button } from '@/components/ui/button'
import { systemApi, type SelfCheckResult } from '@/api/system'
/** 单项状态 → 图标与配色 */
function StatusIcon({ status }: { status: 'ok' | 'warn' | 'error' }) {
if (status === 'ok') return <CheckCircle2 className="h-4 w-4 text-success" />
if (status === 'warn') return <AlertTriangle className="h-4 w-4 text-warning" />
return <XCircle className="h-4 w-4 text-destructive" />
}
/**
* 一键自检 — 仪表盘顶部。
*
* 为什么放在仪表盘:新人接手/故障时第一眼看这里,不必在
* 「集群/主机/存储/部署/采集器」五个页面之间来回跳着判断"到底哪坏了"。
* 后端 /system/self-check 会真正探活(Loki /ready、集群凭证解密、磁盘空间),
* 不是只读配置推断。
*/
export const SelfCheckCard = memo(function SelfCheckCard() {
const { t } = useTranslation('dashboard')
const [data, setData] = useState<SelfCheckResult | null>(null)
const [loading, setLoading] = useState(true)
const [failed, setFailed] = useState(false)
const load = useCallback(async () => {
setLoading(true)
setFailed(false)
try {
setData(await systemApi.selfCheck())
} catch {
// 自检失败本身不代表平台坏了(可能是权限或接口异常),
// 这里静默降级,不打断仪表盘其它内容
setFailed(true)
} finally {
setLoading(false)
}
}, [])
useEffect(() => {
void load()
}, [load])
if (loading && !data) {
return (
<Card>
<CardContent className="flex items-center gap-2 p-4 text-sm text-muted-foreground">
<RefreshCw className="h-4 w-4 animate-spin" />
{t('selfCheck.loading')}
</CardContent>
</Card>
)
}
if (failed || !data) return null
const problems = data.items.filter(i => i.status !== 'ok')
const overallText =
data.overall === 'ok'
? t('selfCheck.allGood')
: data.overall === 'warn'
? t('selfCheck.needsAttention', { count: problems.length })
: t('selfCheck.hasError', { count: problems.length })
const borderClass =
data.overall === 'ok'
? 'border-success/30'
: data.overall === 'warn'
? 'border-warning/40'
: 'border-destructive/40'
return (
<Card className={borderClass}>
<CardHeader className="pb-2">
<CardTitle className="flex items-center gap-2 text-sm font-medium">
<Activity className="h-4 w-4" />
{t('selfCheck.title')}
<span
className={`ml-1 text-xs font-normal ${
data.overall === 'ok'
? 'text-success'
: data.overall === 'warn'
? 'text-warning'
: 'text-destructive'
}`}
>
{overallText}
</span>
<span className="ml-auto text-xs font-normal text-muted-foreground">
{new Date(data.checkedAt).toLocaleTimeString()}
</span>
<Button variant="ghost" size="sm" onClick={() => void load()} disabled={loading}>
<RefreshCw className={`h-3.5 w-3.5 ${loading ? 'animate-spin' : ''}`} />
</Button>
</CardTitle>
</CardHeader>
<CardContent className="flex flex-wrap gap-x-6 gap-y-2 pt-0">
{data.items.map(item => (
<div key={item.name} className="flex items-start gap-1.5 text-xs" title={item.detail || ''}>
<StatusIcon status={item.status} />
<span className="text-muted-foreground">{item.name}</span>
<span
className={
item.status === 'ok'
? 'text-foreground'
: item.status === 'warn'
? 'text-warning'
: 'text-destructive'
}
>
{item.message}
</span>
</div>
))}
</CardContent>
</Card>
)
})
+4
View File
@@ -12,6 +12,7 @@ import { ErrorState } from '@/components/ErrorState'
import { useDashboardStats } from './hooks/useDashboardStats'
import { StatCard } from '@/components/StatCard'
import { RecentDeploymentsCard } from './components/RecentDeploymentsCard'
import { SelfCheckCard } from './components/SelfCheckCard'
import { ResourceListCard } from './components/ResourceListCard'
import type { DashboardStats } from '@/types'
@@ -122,6 +123,9 @@ const DashboardPage = memo(function DashboardPage() {
</Card>
)}
{/* 一键自检:把「哪里坏了」的判断收敛到仪表盘首屏 */}
<SelfCheckCard />
{/* Stats Cards */}
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
{statCards}