fix(logs,analysis,alerts): 数据源不可达、分析页超时、告警规则混入基础设施项

=== 1. 主机模式数据源地址 0.0.0.0 → 主机真实 IP(查询页"永远加载中"的根因)===
现象:切换数据源后命名空间/App 下拉不动,一直显示"加载中…",控制台报
    Get "http://0.0.0.0:3100/loki/api/v1/labels": dial tcp 0.0.0.0:3100: connect: connection refused

根因:主机模式部署的 endpoint 记的是**容器监听地址** 0.0.0.0:3100,
而平台进程不在被采集主机上,0.0.0.0 只在本机有意义。
EnsureLogSourceForDeployment 把它原样写成了数据源 URL。

修法:
  - 新增 reachableEndpoint():把 0.0.0.0 / [::] 换成关联主机的真实 IP
  - schemaVersion v3→v4 迁移 fixUnreachableLogSourceURLs(),修正已有脏数据
  - 实测:http://0.0.0.0:3100 → http://172.19.16.8:3100,该源 label 查询由 502 变 200

=== 2. 前端:区分"加载中"与"加载失败" ===
原来用 `loading={!options.length}` 表示加载中 —— 数据源不可达时列表永远为空,
于是界面永远转圈,把真实错误完全藏起来了(这次的 0.0.0.0 就是这样被掩盖的)。
改为独立的 namespaceLoading / namespaceError 两个状态:
  - 切换数据源先清空旧 options(否则显示上一个源的命名空间,看着像"没反应")
  - 失败显示"加载失败"并在旁边给「重试」按钮,不用刷新整页

=== 3. 日志分析页:主图改走索引接口(原来 5 条 count_over_time 全超时)===
实测 astralx(约 370GB/天)15 分钟窗口:
  count_over_time   → exec 29s + 排队 95s,超过 30s 护栏 → 502
  index/volume_range → 0s
改为两阶段渲染:
  - 总量趋势改用 /logs/histogram(内部 index/volume_range),**0s 出图**
  - 错误/警告/app 维度仍需读日志内容,串行执行,先渲染主图再更新
  - heavyQueryTimeout 30s → 90s(实测 error 维度查询 14s 可完成)

=== 4. 告警规则按日志平台定位收敛 ===
  - seed 里删掉「Kubernetes 节点 NotReady」「容器 OOMKilled」:
    这类属于 kube-state-metrics / Prometheus 的职责,用日志字符串匹配不可靠
  - 迁移里清掉线上已存在的同类规则(3 条)
  - 修 AlertRule / NotifyChannel 重复 ID(同 logSources 一样漏在 sequences 重建外)

=== 5. 告警页 UI ===
  - 规则列表由"每条一张卡"改为紧凑表格(一屏看全,阈值可对齐比较)
  - 通知通道(飞书/Lark/Webhook)从「系统设置」搬到告警页折叠区:
    规则和"发到哪"是一件事的两半,分两页配容易漏配导致告警发不出去

=== 6. 权限收敛 ===
  - /alerts 加 adminOnly:普通用户按查询授权只看「日志查询」,不再看到告警菜单
  - 系统设置从侧栏移到右上角齿轮(低频平台配置,不占侧栏一位)

=== 7. 迁移设计教训(写进代码注释)===
v2 时把"告警去重 + 清基础设施规则"塞进 case 1,但线上库已到 v2,
那段代码永远不执行。**结构变更只要逻辑变了、目标库已达该版本,就必须再抬一版**,
不能改历史迁移分支。

=== 实测(v4-1028-2210 / 979bdb4a48e8)===
  v4 迁移:数据源 URL 修正 http://0.0.0.0:3100 → http://172.19.16.8:3100
  source 1 label:200,0s(astralx / log)
  source 2 label:200,0s(不再 connection refused)
  分析页 histogram astralx 15m:0s
  分析页 metrics  error  astralx 15m:14s
  告警规则:10 条(已无基础设施类)、ID 唯一
  另清理 172.19.16.8 上 13 个残留容器,回收 6.1GB
This commit is contained in:
opsx committed 2026-10-28 22:11:48 +00:00
1 parent 1a4c75e28c
commit 95dcd9408c
20 files changed
+941 -87

No files matched your search

@@ -0,0 +1,246 @@
import { memo, useCallback, useEffect, useState } from 'react'
import { Link } from 'react-router'
import { useTranslation } from 'react-i18next'
import { BellRing, Plus, Pencil, Trash2, Send } from 'lucide-react'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import { Switch } from '@/components/ui/switch'
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
import {
Dialog, DialogContent, DialogHeader, DialogTitle,
} from '@/components/ui/dialog'
import { PasswordInput } from '@/components/PasswordInput'
import { ConfirmDialog } from '@/components/ConfirmDialog'
import { alertsApi, type NotifyChannel } from '@/api/alerts'
import { toast } from 'sonner'
interface ChannelForm {
name: string
webhook: string
signingSecret: string
enabled: boolean
}
/**
* 通知通道(飞书/Lark)— 多 webhook 对应不同告警群。
* webhook 加密存储(留空=保留旧值),每通道可独立测试/禁用/删除(有关联规则时禁删)。
*/
export const NotifySection = memo(function NotifySection() {
const { t } = useTranslation('settings')
const [channels, setChannels] = useState<NotifyChannel[]>([])
const [open, setOpen] = useState(false)
const [editing, setEditing] = useState<NotifyChannel | null>(null)
const [form, setForm] = useState<ChannelForm>({ name: '', webhook: '', signingSecret: '', enabled: true })
const [saving, setSaving] = useState(false)
const [testing, setTesting] = useState(false)
const [deleteTarget, setDeleteTarget] = useState<NotifyChannel | null>(null)
const [deleting, setDeleting] = useState(false)
const refresh = useCallback(async () => {
try {
const list = await alertsApi.listChannels()
setChannels(Array.isArray(list) ? list : [])
} catch (err) {
toast.error(err instanceof Error ? err.message : t('notify.errLoad'))
}
}, [t])
useEffect(() => {
void refresh()
}, [refresh])
const openCreate = () => {
setEditing(null)
setForm({ name: '', webhook: '', signingSecret: '', enabled: true })
setOpen(true)
}
const openEdit = (ch: NotifyChannel) => {
setEditing(ch)
setForm({ name: ch.name, webhook: '', signingSecret: '', enabled: ch.enabled })
setOpen(true)
}
const handleSave = async () => {
if (!form.name.trim()) {
toast.error(t('notify.errNameRequired'))
return
}
if (!editing && !form.webhook) {
toast.error(t('notify.errWebhookRequired'))
return
}
setSaving(true)
try {
if (editing) {
await alertsApi.updateChannel(editing.id, {
name: form.name,
webhook: form.webhook || undefined,
signingSecret: form.signingSecret || undefined,
enabled: form.enabled,
})
} else {
await alertsApi.saveChannel({
name: form.name,
webhook: form.webhook,
signingSecret: form.signingSecret || undefined,
enabled: form.enabled,
})
}
setOpen(false)
void refresh()
} catch (err) {
toast.error(err instanceof Error ? err.message : t('notify.errSave'))
} finally {
setSaving(false)
}
}
const handleTest = async (ch: NotifyChannel) => {
setTesting(true)
try {
await alertsApi.testChannel({ id: ch.id })
toast.success(t('notify.testSent', { name: ch.name }))
} catch (err) {
toast.error(err instanceof Error ? err.message : t('notify.errSend'))
} finally {
setTesting(false)
}
}
const handleToggle = async (ch: NotifyChannel, enabled: boolean) => {
try {
await alertsApi.updateChannel(ch.id, { name: ch.name, enabled })
void refresh()
} catch (err) {
toast.error(err instanceof Error ? err.message : t('notify.errToggle'))
}
}
const handleDelete = async () => {
if (!deleteTarget) return
setDeleting(true)
try {
await alertsApi.deleteChannel(deleteTarget.id)
setDeleteTarget(null)
void refresh()
} catch (err) {
toast.error(err instanceof Error ? err.message : t('notify.errDelete'))
} finally {
setDeleting(false)
}
}
return (
<Card className="border-border">
<CardHeader className="pb-3">
<CardTitle className="text-base flex items-center justify-between">
<span className="flex items-center gap-2">
<BellRing className="w-4 h-4" />
{t('notify.cardTitle')}
</span>
<Button size="sm" variant="outline" onClick={openCreate}>
<Plus className="w-4 h-4 mr-1" />
{t('notify.create')}
</Button>
</CardTitle>
</CardHeader>
<CardContent className="space-y-3">
{channels.length === 0 ? (
<p className="text-sm text-muted-foreground py-2 text-center">
{t('notify.emptyBefore')}
<Link to="/alerts" className="text-primary mx-1">{t('notify.emptyLink')}</Link>
{t('notify.emptyAfter')}
</p>
) : (
channels.map((ch) => (
<div key={ch.id} className="flex items-center gap-3 rounded-lg border bg-card px-3 py-2.5">
<Switch checked={ch.enabled} onCheckedChange={(v) => void handleToggle(ch, v)} />
<div className="min-w-0 flex-1">
<p className="font-medium text-sm">{ch.name}</p>
<p className="text-xs text-muted-foreground">
{ch.hasWebhook ? t('notify.webhookConfigured') : t('notify.webhookMissing')}
{ch.hasSecret ? t('notify.secretConfigured') : ''}
</p>
</div>
<div className="flex items-center gap-1.5">
<Button variant="outline" size="sm" aria-label={t('notify.testAria', { name: ch.name })} disabled={testing} onClick={() => void handleTest(ch)}>
<Send className="w-3.5 h-3.5" />
</Button>
<Button variant="outline" size="sm" aria-label={t('notify.editAria', { name: ch.name })} onClick={() => openEdit(ch)}>
<Pencil className="w-3.5 h-3.5" />
</Button>
<Button
variant="outline" size="sm" aria-label={t('notify.deleteAria', { name: ch.name })}
className="text-destructive hover:bg-destructive/10"
onClick={() => setDeleteTarget(ch)}
>
<Trash2 className="w-3.5 h-3.5" />
</Button>
</div>
</div>
))
)}
</CardContent>
<Dialog open={open} onOpenChange={setOpen}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>
{editing ? t('notify.editTitle', { name: editing.name }) : t('notify.createTitle')}
</DialogTitle>
</DialogHeader>
<div className="space-y-4">
<div className="space-y-2">
<Label>{t('notify.nameLabel')}</Label>
<Input
placeholder={t('notify.namePlaceholder')}
value={form.name}
onChange={(e) => setForm((f) => ({ ...f, name: e.target.value }))}
/>
</div>
<div className="space-y-2">
<Label>
{t('notify.webhookLabel')}
{editing ? t('notify.webhookKeepHint') : ' *'}
</Label>
<PasswordInput
placeholder={editing?.hasWebhook ? t('notify.webhookSavedPlaceholder') : 'https://open.feishu.cn/open-apis/bot/v2/hook/xxx'}
value={form.webhook}
onChange={(e) => setForm((f) => ({ ...f, webhook: e.target.value }))}
/>
</div>
<div className="space-y-2">
<Label>{t('notify.signingLabel')}</Label>
<PasswordInput
placeholder={editing?.hasSecret ? t('notify.signingSavedPlaceholder') : t('notify.signingPlaceholder')}
value={form.signingSecret}
onChange={(e) => setForm((f) => ({ ...f, signingSecret: e.target.value }))}
/>
</div>
<div className="flex items-center gap-2">
<Switch checked={form.enabled} onCheckedChange={(v) => setForm((f) => ({ ...f, enabled: v }))} />
<Label>{t('notify.enable')}</Label>
</div>
<div className="flex justify-end gap-2">
<Button variant="outline" onClick={() => setOpen(false)}>{t('actions.cancel')}</Button>
<Button onClick={handleSave} disabled={saving}>
{saving ? t('actions.saving') : editing ? t('notify.saveChanges') : t('notify.createChannel')}
</Button>
</div>
</div>
</DialogContent>
</Dialog>
<ConfirmDialog
open={deleteTarget !== null}
onOpenChange={(o) => { if (!o) setDeleteTarget(null) }}
name={deleteTarget?.name}
title={t('notify.deleteTitle')}
loading={deleting}
onConfirm={handleDelete}
/>
</Card>
)
})
+83 -55
View File
@@ -1,11 +1,11 @@
import { memo, useCallback, useEffect, useState } from 'react'
import { Link } from 'react-router'
import { BellRing, Plus, Play, Pencil, Trash2 } from 'lucide-react'
import { Button } from '@/components/ui/button'
import { Checkbox } from '@/components/ui/checkbox'
import { Input } from '@/components/ui/input'
import { Switch } from '@/components/ui/switch'
import { Card, CardContent } from '@/components/ui/card'
import { NotifySection } from './components/NotifySection'
import {
Dialog, DialogContent, DialogHeader, DialogTitle,
} from '@/components/ui/dialog'
@@ -216,55 +216,72 @@ const AlertsPage = memo(function AlertsPage() {
description={t('emptyDesc')}
/>
) : (
<div className="space-y-3">
{rules.map((rule) => {
const src = sources.find((s) => s.id === rule.sourceId)
return (
<Card key={rule.id} className="border-border">
<CardContent className="p-4 flex items-center gap-4 flex-wrap">
<Switch checked={rule.enabled} onCheckedChange={(v) => void toggleRule(rule, v)} />
<div className="min-w-0 flex-1">
<p className="font-medium">{rule.name}</p>
<p className="text-xs text-muted-foreground truncate font-mono mt-0.5">{rule.query}</p>
<p className="text-xs text-muted-foreground mt-0.5">
{src?.name ?? t('sourceFallback', { id: rule.sourceId })} →{' '}
{(rule.channelIds || [])
.map((id) => channels.find((c) => c.id === id)?.name || `#${id}`)
.join(t('channelSep')) || t('noChannels')}{' '}
· {t('ruleMeta', {
window: rule.windowMin,
threshold: rule.threshold,
cooldown: rule.cooldownMin,
})}
{rule.lastFiredAt && !rule.lastFiredAt.startsWith('0001') && ` · ${t('lastFired', {
time: new Date(rule.lastFiredAt).toLocaleString(),
})}`}
</p>
</div>
<div className="flex items-center gap-2">
<Button
variant="outline"
size="sm"
aria-label={t('editAria', { name: rule.name })}
onClick={() => openEdit(rule)}
>
<Pencil className="w-3.5 h-3.5" />
</Button>
<Button
variant="outline"
size="sm"
aria-label={t('deleteAria', { name: rule.name })}
className="text-destructive hover:bg-destructive/10"
onClick={() => setDeleteTarget(rule)}
>
<Trash2 className="w-3.5 h-3.5" />
</Button>
</div>
</CardContent>
</Card>
)
})}
</div>
/* 紧凑表格:原来每条规则一张 Card(p-4 + 三行文字),10 条规则要滚很久。
规则本质是同构的行数据,用表格能在一屏内看全,也便于对齐比较阈值。 */
<Card className="border-border">
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead className="border-b border-border bg-muted/30 text-xs text-muted-foreground">
<tr>
<th className="w-10 px-3 py-1.5 text-left font-medium">{t('col.enabled')}</th>
<th className="px-3 py-1.5 text-left font-medium">{t('col.name')}</th>
<th className="px-3 py-1.5 text-left font-medium">{t('col.condition')}</th>
<th className="px-3 py-1.5 text-left font-medium">{t('col.notify')}</th>
<th className="w-16 px-3 py-1.5" />
</tr>
</thead>
<tbody>
{rules.map((rule) => {
const src = sources.find((s) => s.id === rule.sourceId)
const chan = (rule.channelIds || [])
.map((id) => channels.find((c) => c.id === id)?.name || `#${id}`)
.join(t('channelSep')) || t('noChannels')
return (
<tr key={rule.id} className="border-b border-border/50 last:border-0 hover:bg-muted/20">
<td className="px-3 py-1.5">
<Switch checked={rule.enabled} onCheckedChange={(v) => void toggleRule(rule, v)} />
</td>
<td className="max-w-[13rem] px-3 py-1.5">
<div className="truncate font-medium" title={rule.name}>{rule.name}</div>
<div className="truncate text-[11px] text-muted-foreground" title={rule.query}>
{src?.name ?? t('sourceFallback', { id: rule.sourceId })}
</div>
</td>
<td className="px-3 py-1.5">
<div className="truncate font-mono text-[11px] text-muted-foreground" title={rule.query}>
{rule.query}
</div>
<div className="text-[11px] text-muted-foreground">
{t('ruleMeta', {
window: rule.windowMin,
threshold: rule.threshold,
cooldown: rule.cooldownMin,
})}
{rule.lastFiredAt && !rule.lastFiredAt.startsWith('0001') && ` · ${t('lastFired', {
time: new Date(rule.lastFiredAt).toLocaleString(),
})}`}
</div>
</td>
<td className="max-w-[10rem] px-3 py-1.5">
<div className="truncate text-xs" title={chan}>{chan}</div>
</td>
<td className="px-3 py-1.5">
<div className="flex items-center justify-end gap-1">
<Button variant="ghost" size="sm" className="h-7 w-7 p-0" aria-label={t('editAria', { name: rule.name })} onClick={() => openEdit(rule)}>
<Pencil className="w-3.5 h-3.5" />
</Button>
<Button variant="ghost" size="sm" className="h-7 w-7 p-0 text-destructive hover:bg-destructive/10" aria-label={t('deleteAria', { name: rule.name })} onClick={() => setDeleteTarget(rule)}>
<Trash2 className="w-3.5 h-3.5" />
</Button>
</div>
</td>
</tr>
)
})}
</tbody>
</table>
</div>
</Card>
)}
{/* 发送历史 */}
@@ -301,11 +318,22 @@ const AlertsPage = memo(function AlertsPage() {
</Card>
)}
<p className="text-xs text-muted-foreground">
{t('notifyHintPre')}
<Link to="/settings" className="text-primary mx-1">{t('common:menu.settings')}</Link>
{t('notifyHintPost')}
</p>
{/* 通知通道配置:从「系统设置」搬到这里。
告警规则和"告警发到哪"是同一件事的两半,分两个页面配很容易漏配导致告警发不出去。
默认折叠,不影响规则列表的紧凑度。 */}
<details className="rounded-lg border border-border" open={channels.length === 0}>
<summary className="cursor-pointer select-none px-3 py-2 text-sm font-medium">
{t('notifySectionTitle')}
<span className="ml-2 text-xs font-normal text-muted-foreground">
{channels.length > 0
? t('notifySectionConfigured', { count: channels.length })
: t('notifySectionEmpty')}
</span>
</summary>
<div className="border-t border-border p-3">
<NotifySection />
</div>
</details>
{/* 规则弹窗 */}
<Dialog open={open} onOpenChange={setOpen}>