fix(logs,analysis,alerts): 数据源不可达、分析页超时、告警规则混入基础设施项
=== 1. 主机模式数据源地址 0.0.0.0 → 主机真实 IP(查询页"永远加载中"的根因)===
现象:切换数据源后命名空间/App 下拉不动,一直显示"加载中…",控制台报
Get "http://0.0.0.0:3100/loki/api/v1/labels": dial tcp 0.0.0.0:3100: connect: connection refused
根因:主机模式部署的 endpoint 记的是**容器监听地址** 0.0.0.0:3100,
而平台进程不在被采集主机上,0.0.0.0 只在本机有意义。
EnsureLogSourceForDeployment 把它原样写成了数据源 URL。
修法:
- 新增 reachableEndpoint():把 0.0.0.0 / [::] 换成关联主机的真实 IP
- schemaVersion v3→v4 迁移 fixUnreachableLogSourceURLs(),修正已有脏数据
- 实测:http://0.0.0.0:3100 → http://172.19.16.8:3100,该源 label 查询由 502 变 200
=== 2. 前端:区分"加载中"与"加载失败" ===
原来用 `loading={!options.length}` 表示加载中 —— 数据源不可达时列表永远为空,
于是界面永远转圈,把真实错误完全藏起来了(这次的 0.0.0.0 就是这样被掩盖的)。
改为独立的 namespaceLoading / namespaceError 两个状态:
- 切换数据源先清空旧 options(否则显示上一个源的命名空间,看着像"没反应")
- 失败显示"加载失败"并在旁边给「重试」按钮,不用刷新整页
=== 3. 日志分析页:主图改走索引接口(原来 5 条 count_over_time 全超时)===
实测 astralx(约 370GB/天)15 分钟窗口:
count_over_time → exec 29s + 排队 95s,超过 30s 护栏 → 502
index/volume_range → 0s
改为两阶段渲染:
- 总量趋势改用 /logs/histogram(内部 index/volume_range),**0s 出图**
- 错误/警告/app 维度仍需读日志内容,串行执行,先渲染主图再更新
- heavyQueryTimeout 30s → 90s(实测 error 维度查询 14s 可完成)
=== 4. 告警规则按日志平台定位收敛 ===
- seed 里删掉「Kubernetes 节点 NotReady」「容器 OOMKilled」:
这类属于 kube-state-metrics / Prometheus 的职责,用日志字符串匹配不可靠
- 迁移里清掉线上已存在的同类规则(3 条)
- 修 AlertRule / NotifyChannel 重复 ID(同 logSources 一样漏在 sequences 重建外)
=== 5. 告警页 UI ===
- 规则列表由"每条一张卡"改为紧凑表格(一屏看全,阈值可对齐比较)
- 通知通道(飞书/Lark/Webhook)从「系统设置」搬到告警页折叠区:
规则和"发到哪"是一件事的两半,分两页配容易漏配导致告警发不出去
=== 6. 权限收敛 ===
- /alerts 加 adminOnly:普通用户按查询授权只看「日志查询」,不再看到告警菜单
- 系统设置从侧栏移到右上角齿轮(低频平台配置,不占侧栏一位)
=== 7. 迁移设计教训(写进代码注释)===
v2 时把"告警去重 + 清基础设施规则"塞进 case 1,但线上库已到 v2,
那段代码永远不执行。**结构变更只要逻辑变了、目标库已达该版本,就必须再抬一版**,
不能改历史迁移分支。
=== 实测(v4-1028-2210 / 979bdb4a48e8)===
v4 迁移:数据源 URL 修正 http://0.0.0.0:3100 → http://172.19.16.8:3100
source 1 label:200,0s(astralx / log)
source 2 label:200,0s(不再 connection refused)
分析页 histogram astralx 15m:0s
分析页 metrics error astralx 15m:14s
告警规则:10 条(已无基础设施类)、ID 唯一
另清理 172.19.16.8 上 13 个残留容器,回收 6.1GB
This commit is contained in:
1 parent
1a4c75e28c
commit
95dcd9408c
20 files changed
+941
-87
No files matched your search
@@ -0,0 +1,246 @@
|
||||
import { memo, useCallback, useEffect, useState } from 'react'
|
||||
import { Link } from 'react-router'
|
||||
import { useTranslation } from 'react-i18next'
|
||||
import { BellRing, Plus, Pencil, Trash2, Send } from 'lucide-react'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Input } from '@/components/ui/input'
|
||||
import { Label } from '@/components/ui/label'
|
||||
import { Switch } from '@/components/ui/switch'
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
import {
|
||||
Dialog, DialogContent, DialogHeader, DialogTitle,
|
||||
} from '@/components/ui/dialog'
|
||||
import { PasswordInput } from '@/components/PasswordInput'
|
||||
import { ConfirmDialog } from '@/components/ConfirmDialog'
|
||||
import { alertsApi, type NotifyChannel } from '@/api/alerts'
|
||||
import { toast } from 'sonner'
|
||||
|
||||
interface ChannelForm {
|
||||
name: string
|
||||
webhook: string
|
||||
signingSecret: string
|
||||
enabled: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* 通知通道(飞书/Lark)— 多 webhook 对应不同告警群。
|
||||
* webhook 加密存储(留空=保留旧值),每通道可独立测试/禁用/删除(有关联规则时禁删)。
|
||||
*/
|
||||
export const NotifySection = memo(function NotifySection() {
|
||||
const { t } = useTranslation('settings')
|
||||
const [channels, setChannels] = useState<NotifyChannel[]>([])
|
||||
const [open, setOpen] = useState(false)
|
||||
const [editing, setEditing] = useState<NotifyChannel | null>(null)
|
||||
const [form, setForm] = useState<ChannelForm>({ name: '', webhook: '', signingSecret: '', enabled: true })
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [testing, setTesting] = useState(false)
|
||||
const [deleteTarget, setDeleteTarget] = useState<NotifyChannel | null>(null)
|
||||
const [deleting, setDeleting] = useState(false)
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
try {
|
||||
const list = await alertsApi.listChannels()
|
||||
setChannels(Array.isArray(list) ? list : [])
|
||||
} catch (err) {
|
||||
toast.error(err instanceof Error ? err.message : t('notify.errLoad'))
|
||||
}
|
||||
}, [t])
|
||||
|
||||
useEffect(() => {
|
||||
void refresh()
|
||||
}, [refresh])
|
||||
|
||||
const openCreate = () => {
|
||||
setEditing(null)
|
||||
setForm({ name: '', webhook: '', signingSecret: '', enabled: true })
|
||||
setOpen(true)
|
||||
}
|
||||
|
||||
const openEdit = (ch: NotifyChannel) => {
|
||||
setEditing(ch)
|
||||
setForm({ name: ch.name, webhook: '', signingSecret: '', enabled: ch.enabled })
|
||||
setOpen(true)
|
||||
}
|
||||
|
||||
const handleSave = async () => {
|
||||
if (!form.name.trim()) {
|
||||
toast.error(t('notify.errNameRequired'))
|
||||
return
|
||||
}
|
||||
if (!editing && !form.webhook) {
|
||||
toast.error(t('notify.errWebhookRequired'))
|
||||
return
|
||||
}
|
||||
setSaving(true)
|
||||
try {
|
||||
if (editing) {
|
||||
await alertsApi.updateChannel(editing.id, {
|
||||
name: form.name,
|
||||
webhook: form.webhook || undefined,
|
||||
signingSecret: form.signingSecret || undefined,
|
||||
enabled: form.enabled,
|
||||
})
|
||||
} else {
|
||||
await alertsApi.saveChannel({
|
||||
name: form.name,
|
||||
webhook: form.webhook,
|
||||
signingSecret: form.signingSecret || undefined,
|
||||
enabled: form.enabled,
|
||||
})
|
||||
}
|
||||
setOpen(false)
|
||||
void refresh()
|
||||
} catch (err) {
|
||||
toast.error(err instanceof Error ? err.message : t('notify.errSave'))
|
||||
} finally {
|
||||
setSaving(false)
|
||||
}
|
||||
}
|
||||
|
||||
const handleTest = async (ch: NotifyChannel) => {
|
||||
setTesting(true)
|
||||
try {
|
||||
await alertsApi.testChannel({ id: ch.id })
|
||||
toast.success(t('notify.testSent', { name: ch.name }))
|
||||
} catch (err) {
|
||||
toast.error(err instanceof Error ? err.message : t('notify.errSend'))
|
||||
} finally {
|
||||
setTesting(false)
|
||||
}
|
||||
}
|
||||
|
||||
const handleToggle = async (ch: NotifyChannel, enabled: boolean) => {
|
||||
try {
|
||||
await alertsApi.updateChannel(ch.id, { name: ch.name, enabled })
|
||||
void refresh()
|
||||
} catch (err) {
|
||||
toast.error(err instanceof Error ? err.message : t('notify.errToggle'))
|
||||
}
|
||||
}
|
||||
|
||||
const handleDelete = async () => {
|
||||
if (!deleteTarget) return
|
||||
setDeleting(true)
|
||||
try {
|
||||
await alertsApi.deleteChannel(deleteTarget.id)
|
||||
setDeleteTarget(null)
|
||||
void refresh()
|
||||
} catch (err) {
|
||||
toast.error(err instanceof Error ? err.message : t('notify.errDelete'))
|
||||
} finally {
|
||||
setDeleting(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Card className="border-border">
|
||||
<CardHeader className="pb-3">
|
||||
<CardTitle className="text-base flex items-center justify-between">
|
||||
<span className="flex items-center gap-2">
|
||||
<BellRing className="w-4 h-4" />
|
||||
{t('notify.cardTitle')}
|
||||
</span>
|
||||
<Button size="sm" variant="outline" onClick={openCreate}>
|
||||
<Plus className="w-4 h-4 mr-1" />
|
||||
{t('notify.create')}
|
||||
</Button>
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-3">
|
||||
{channels.length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground py-2 text-center">
|
||||
{t('notify.emptyBefore')}
|
||||
<Link to="/alerts" className="text-primary mx-1">{t('notify.emptyLink')}</Link>
|
||||
{t('notify.emptyAfter')}
|
||||
</p>
|
||||
) : (
|
||||
channels.map((ch) => (
|
||||
<div key={ch.id} className="flex items-center gap-3 rounded-lg border bg-card px-3 py-2.5">
|
||||
<Switch checked={ch.enabled} onCheckedChange={(v) => void handleToggle(ch, v)} />
|
||||
<div className="min-w-0 flex-1">
|
||||
<p className="font-medium text-sm">{ch.name}</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{ch.hasWebhook ? t('notify.webhookConfigured') : t('notify.webhookMissing')}
|
||||
{ch.hasSecret ? t('notify.secretConfigured') : ''}
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-1.5">
|
||||
<Button variant="outline" size="sm" aria-label={t('notify.testAria', { name: ch.name })} disabled={testing} onClick={() => void handleTest(ch)}>
|
||||
<Send className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" aria-label={t('notify.editAria', { name: ch.name })} onClick={() => openEdit(ch)}>
|
||||
<Pencil className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline" size="sm" aria-label={t('notify.deleteAria', { name: ch.name })}
|
||||
className="text-destructive hover:bg-destructive/10"
|
||||
onClick={() => setDeleteTarget(ch)}
|
||||
>
|
||||
<Trash2 className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
))
|
||||
)}
|
||||
</CardContent>
|
||||
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogContent className="max-w-md">
|
||||
<DialogHeader>
|
||||
<DialogTitle>
|
||||
{editing ? t('notify.editTitle', { name: editing.name }) : t('notify.createTitle')}
|
||||
</DialogTitle>
|
||||
</DialogHeader>
|
||||
<div className="space-y-4">
|
||||
<div className="space-y-2">
|
||||
<Label>{t('notify.nameLabel')}</Label>
|
||||
<Input
|
||||
placeholder={t('notify.namePlaceholder')}
|
||||
value={form.name}
|
||||
onChange={(e) => setForm((f) => ({ ...f, name: e.target.value }))}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label>
|
||||
{t('notify.webhookLabel')}
|
||||
{editing ? t('notify.webhookKeepHint') : ' *'}
|
||||
</Label>
|
||||
<PasswordInput
|
||||
placeholder={editing?.hasWebhook ? t('notify.webhookSavedPlaceholder') : 'https://open.feishu.cn/open-apis/bot/v2/hook/xxx'}
|
||||
value={form.webhook}
|
||||
onChange={(e) => setForm((f) => ({ ...f, webhook: e.target.value }))}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label>{t('notify.signingLabel')}</Label>
|
||||
<PasswordInput
|
||||
placeholder={editing?.hasSecret ? t('notify.signingSavedPlaceholder') : t('notify.signingPlaceholder')}
|
||||
value={form.signingSecret}
|
||||
onChange={(e) => setForm((f) => ({ ...f, signingSecret: e.target.value }))}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<Switch checked={form.enabled} onCheckedChange={(v) => setForm((f) => ({ ...f, enabled: v }))} />
|
||||
<Label>{t('notify.enable')}</Label>
|
||||
</div>
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button variant="outline" onClick={() => setOpen(false)}>{t('actions.cancel')}</Button>
|
||||
<Button onClick={handleSave} disabled={saving}>
|
||||
{saving ? t('actions.saving') : editing ? t('notify.saveChanges') : t('notify.createChannel')}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
|
||||
<ConfirmDialog
|
||||
open={deleteTarget !== null}
|
||||
onOpenChange={(o) => { if (!o) setDeleteTarget(null) }}
|
||||
name={deleteTarget?.name}
|
||||
title={t('notify.deleteTitle')}
|
||||
loading={deleting}
|
||||
onConfirm={handleDelete}
|
||||
/>
|
||||
</Card>
|
||||
)
|
||||
})
|
||||
@@ -1,11 +1,11 @@
|
||||
import { memo, useCallback, useEffect, useState } from 'react'
|
||||
import { Link } from 'react-router'
|
||||
import { BellRing, Plus, Play, Pencil, Trash2 } from 'lucide-react'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Checkbox } from '@/components/ui/checkbox'
|
||||
import { Input } from '@/components/ui/input'
|
||||
import { Switch } from '@/components/ui/switch'
|
||||
import { Card, CardContent } from '@/components/ui/card'
|
||||
import { NotifySection } from './components/NotifySection'
|
||||
import {
|
||||
Dialog, DialogContent, DialogHeader, DialogTitle,
|
||||
} from '@/components/ui/dialog'
|
||||
@@ -216,55 +216,72 @@ const AlertsPage = memo(function AlertsPage() {
|
||||
description={t('emptyDesc')}
|
||||
/>
|
||||
) : (
|
||||
<div className="space-y-3">
|
||||
{rules.map((rule) => {
|
||||
const src = sources.find((s) => s.id === rule.sourceId)
|
||||
return (
|
||||
<Card key={rule.id} className="border-border">
|
||||
<CardContent className="p-4 flex items-center gap-4 flex-wrap">
|
||||
<Switch checked={rule.enabled} onCheckedChange={(v) => void toggleRule(rule, v)} />
|
||||
<div className="min-w-0 flex-1">
|
||||
<p className="font-medium">{rule.name}</p>
|
||||
<p className="text-xs text-muted-foreground truncate font-mono mt-0.5">{rule.query}</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">
|
||||
{src?.name ?? t('sourceFallback', { id: rule.sourceId })} →{' '}
|
||||
{(rule.channelIds || [])
|
||||
.map((id) => channels.find((c) => c.id === id)?.name || `#${id}`)
|
||||
.join(t('channelSep')) || t('noChannels')}{' '}
|
||||
· {t('ruleMeta', {
|
||||
window: rule.windowMin,
|
||||
threshold: rule.threshold,
|
||||
cooldown: rule.cooldownMin,
|
||||
})}
|
||||
{rule.lastFiredAt && !rule.lastFiredAt.startsWith('0001') && ` · ${t('lastFired', {
|
||||
time: new Date(rule.lastFiredAt).toLocaleString(),
|
||||
})}`}
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
aria-label={t('editAria', { name: rule.name })}
|
||||
onClick={() => openEdit(rule)}
|
||||
>
|
||||
<Pencil className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
aria-label={t('deleteAria', { name: rule.name })}
|
||||
className="text-destructive hover:bg-destructive/10"
|
||||
onClick={() => setDeleteTarget(rule)}
|
||||
>
|
||||
<Trash2 className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
/* 紧凑表格:原来每条规则一张 Card(p-4 + 三行文字),10 条规则要滚很久。
|
||||
规则本质是同构的行数据,用表格能在一屏内看全,也便于对齐比较阈值。 */
|
||||
<Card className="border-border">
|
||||
<div className="overflow-x-auto">
|
||||
<table className="w-full text-sm">
|
||||
<thead className="border-b border-border bg-muted/30 text-xs text-muted-foreground">
|
||||
<tr>
|
||||
<th className="w-10 px-3 py-1.5 text-left font-medium">{t('col.enabled')}</th>
|
||||
<th className="px-3 py-1.5 text-left font-medium">{t('col.name')}</th>
|
||||
<th className="px-3 py-1.5 text-left font-medium">{t('col.condition')}</th>
|
||||
<th className="px-3 py-1.5 text-left font-medium">{t('col.notify')}</th>
|
||||
<th className="w-16 px-3 py-1.5" />
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{rules.map((rule) => {
|
||||
const src = sources.find((s) => s.id === rule.sourceId)
|
||||
const chan = (rule.channelIds || [])
|
||||
.map((id) => channels.find((c) => c.id === id)?.name || `#${id}`)
|
||||
.join(t('channelSep')) || t('noChannels')
|
||||
return (
|
||||
<tr key={rule.id} className="border-b border-border/50 last:border-0 hover:bg-muted/20">
|
||||
<td className="px-3 py-1.5">
|
||||
<Switch checked={rule.enabled} onCheckedChange={(v) => void toggleRule(rule, v)} />
|
||||
</td>
|
||||
<td className="max-w-[13rem] px-3 py-1.5">
|
||||
<div className="truncate font-medium" title={rule.name}>{rule.name}</div>
|
||||
<div className="truncate text-[11px] text-muted-foreground" title={rule.query}>
|
||||
{src?.name ?? t('sourceFallback', { id: rule.sourceId })}
|
||||
</div>
|
||||
</td>
|
||||
<td className="px-3 py-1.5">
|
||||
<div className="truncate font-mono text-[11px] text-muted-foreground" title={rule.query}>
|
||||
{rule.query}
|
||||
</div>
|
||||
<div className="text-[11px] text-muted-foreground">
|
||||
{t('ruleMeta', {
|
||||
window: rule.windowMin,
|
||||
threshold: rule.threshold,
|
||||
cooldown: rule.cooldownMin,
|
||||
})}
|
||||
{rule.lastFiredAt && !rule.lastFiredAt.startsWith('0001') && ` · ${t('lastFired', {
|
||||
time: new Date(rule.lastFiredAt).toLocaleString(),
|
||||
})}`}
|
||||
</div>
|
||||
</td>
|
||||
<td className="max-w-[10rem] px-3 py-1.5">
|
||||
<div className="truncate text-xs" title={chan}>{chan}</div>
|
||||
</td>
|
||||
<td className="px-3 py-1.5">
|
||||
<div className="flex items-center justify-end gap-1">
|
||||
<Button variant="ghost" size="sm" className="h-7 w-7 p-0" aria-label={t('editAria', { name: rule.name })} onClick={() => openEdit(rule)}>
|
||||
<Pencil className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" className="h-7 w-7 p-0 text-destructive hover:bg-destructive/10" aria-label={t('deleteAria', { name: rule.name })} onClick={() => setDeleteTarget(rule)}>
|
||||
<Trash2 className="w-3.5 h-3.5" />
|
||||
</Button>
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
)
|
||||
})}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* 发送历史 */}
|
||||
@@ -301,11 +318,22 @@ const AlertsPage = memo(function AlertsPage() {
|
||||
</Card>
|
||||
)}
|
||||
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{t('notifyHintPre')}
|
||||
<Link to="/settings" className="text-primary mx-1">{t('common:menu.settings')}</Link>
|
||||
{t('notifyHintPost')}
|
||||
</p>
|
||||
{/* 通知通道配置:从「系统设置」搬到这里。
|
||||
告警规则和"告警发到哪"是同一件事的两半,分两个页面配很容易漏配导致告警发不出去。
|
||||
默认折叠,不影响规则列表的紧凑度。 */}
|
||||
<details className="rounded-lg border border-border" open={channels.length === 0}>
|
||||
<summary className="cursor-pointer select-none px-3 py-2 text-sm font-medium">
|
||||
{t('notifySectionTitle')}
|
||||
<span className="ml-2 text-xs font-normal text-muted-foreground">
|
||||
{channels.length > 0
|
||||
? t('notifySectionConfigured', { count: channels.length })
|
||||
: t('notifySectionEmpty')}
|
||||
</span>
|
||||
</summary>
|
||||
<div className="border-t border-border p-3">
|
||||
<NotifySection />
|
||||
</div>
|
||||
</details>
|
||||
|
||||
{/* 规则弹窗 */}
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
|
||||
Reference in new issue
Block a user