Commit Graph
219 Commits
Author SHA1 Message Date
opsx 0427b87412 fix(instance): access_mode 值 read/readonly 不一致导致「读」实例被过滤
现象:instance-groups 里给 MySQL 实例授予「只读查询」权限后,
     /queries 页面仍然选不到该实例,admin 也无法访问。

根因(枚举值前后端不一致):
  前端 CreateInstanceForm「读」按钮写入 access_mode = "read"
  后端 AccessibleInstances 只匹配 ["readwrite", "readonly"]
  → "read" 永远不命中,选了「读」的实例(MySQL)被过滤掉;
    Redis 实例是 "readwrite" 所以恰好能查,掩盖了该 bug。

修复:后端 readonly 过滤增加 "read",兼容两种历史写法:
  []string{"readwrite", "readonly", "read"}
  写过滤仍为 ["readwrite", "write"]("read" 实例正确地不能写)

验证(admin):
  GET /instances/accessible?access_type=readonly → 5 个(含 MySQL id=5, access_mode=read)
  GET /instances/accessible?access_type=write    → 4 个(MySQL 正确排除)

镜像:lghost/opsxsql:latest + v0.1 = sha256:6b1a7121da71
2026-10-26 18:30:47 +00:00
opsx f19a84f37e fix(redis): 集群拓扑解析错误,跨机集群节点被强制指向种子机
【现象】Redis 集群实例查 keys 报 500:
  Redis cluster connection failed: dial tcp 172.19.16.16:6005: connect: connection refused

【根因】handler/redis.go 解析 CLUSTER SLOTS:
  nodeAddr := slot.Nodes[0].Addr                    // Redis 上报的真实地址 172.19.16.30:6005
  if idx := strings.LastIndex(nodeAddr, ":"); idx >= 0 {
      nodeAddr = instance.Host + nodeAddr[idx:]     // 强制改写成 种子机主机 + 原端口
  }
  该逻辑假设「集群所有节点在同一台机器、只是端口不同」。
  对跨机集群(16/30/59 三台)会把所有节点地址破坏成
  172.19.16.16:6002 / :6003 / :6005 —— 这些端口在 16 上根本不存在 → refused。

【修复】
  仅当 Redis 上报空 IP / 127.0.0.1 / 0.0.0.0 时才用配置 host 兜底(NAT/端口映射场景),
  其余情况一律保留 Redis 上报的真实主机地址。

【附带数据修复(非代码)】
  实例 5 host: pre-mysql.astx.club → 172.19.16.2
  原因:pre-mysql.astx.club 公网 DNS 解析到 43.135.53.180(另一环境 MySQL),
       认证失败 → 前端数据库下拉为空、无法选择数据库。

【验证】
  GET /api/v1/redis/4/keys?db=0&pattern=*  → 200
    {"cluster":true,"keys":[{"key":"24H_rate_limit_10001_...","type":"string","ttl":42901000},
                            {"key":"CEX:FOLLOW:FOLLOWERS:TRADFI:97127836","type":"hash"},...]}
  GET /api/v1/redis/4/dbs                 → 200 {"cluster":true,"dbs":[0]}
  GET /api/v1/instances/5/databases       → 200 返回 25 个库
    ["cex_activity","cex_admin","cex_agent",...,"cex_user","cex_xxl_job"]

镜像 sha256:165992ef79d5
2026-10-26 18:01:44 +00:00
opsx 6b81dbbd4a feat(ldap): LDAP 改为只读接入,彻底移除账号验证与批量导入写入功能
按需求(sql.astx.club/settings):

1. 移除「测试账号验证」按钮
   - LdapConfigForm.tsx 删按钮 + onOpenAccountTest prop
   - 不再渲染 LdapAccountTestDialog

2. 移除批量导入的 LDAP 写操作(BatchImportModal.tsx)
   - TABS 去掉「导入到 LDAP」「从 LDAP 导入」
   - 删除对应两个 JSX 代码块(tab === 'to-ldap' / 'from-ldap')
   - 删除「创建 OU」子弹窗
   - 只保留「本地导入」(CSV 建本地账号,不触碰 LDAP 目录)

3. 移除 LDAP 账号管理面板(含危险写操作)
   - 不再渲染 LdapManagePanel(新建 OU / 添加用户 / 删除用户)
   - 删除 LdapCreateOuDialog / LdapAddUserDialog / ConfirmDialog 引用
   - LdapConfigPage.tsx 421 → 340 行;BatchImportModal.tsx 557 → 329 行

4. 保留(只读用途)
   - LDAP 配置表单(服务器 / 绑定 / 搜索 / 属性映射)
   - 「测试连接」按钮(验证配置可用,不写 LDAP 目录)

设计原则:LDAP 仅用于域账号登录认证(只读 bind + search),
         目录维护请到域控侧操作。

验证(前端 bundle grep,全部归零):
  测试账号验证 0 | 导入到 LDAP 0 | 从 LDAP 导入 0
  新建 OU 0 | 创建 OU 0 | 危险操作 0 | 此操作不可逆 0
保留:
  测试连接 6 | 本地导入 1

镜像 sha256:3970db141d23
/health → {"backend":"go","status":"ok","version":"0.4.0"}
http://sql.astx.club/settings → HTTP 200
2026-10-26 17:49:34 +00:00
opsx e8355e5d5f feat(ldap): LDAP 改为只读接入,移除账号验证测试与批量导入写操作
按需求调整 settings 页 LDAP 区域:

1. 移除「测试账号验证」按钮
   - LdapConfigForm.tsx 删除按钮 + onOpenAccountTest prop
   - LdapConfigPage.tsx 不再渲染 LdapAccountTestDialog

2. 移除批量导入的 LDAP 写操作
   - BatchImportModal.tsx 的 TABS 去掉「导入到 LDAP」「从 LDAP 导入」
   - 只保留「本地导入」(CSV 建本地账号,不触碰 LDAP 目录)

3. 移除 LDAP 账号管理面板(危险写操作)
   - 不再渲染 LdapManagePanel(新建 OU / 添加用户 / 删除用户)
   - 不再渲染 LdapCreateOuDialog / LdapAddUserDialog
   - 面板含「此操作不可逆」删除功能,与只读接入原则冲突

4. 保留(只读用途)
   - LDAP 配置表单(服务器/绑定/搜索/属性映射)
   - 「测试连接」按钮(验证配置是否可用,不写 LDAP)

设计原则:本系统 LDAP 仅用于域账号登录认证(只读 bind + search),
         目录维护请在域控侧操作。

验证(前端 bundle 实测):
  测试账号验证: 0    导入到 LDAP: 0    从 LDAP 导入: 0
  新建 OU: 0        危险操作: 0       此操作不可逆: 0
  测试连接: 6(保留)  本地导入: 1(保留)
  /health → {"backend":"go","status":"ok","version":"0.4.0"}
  http://sql.astx.club/settings → HTTP 200
2026-10-26 17:45:46 +00:00
opsx d584b64813 feat(security): 默认账号随机初始密码(不再硬编码 admin123)
参考 lokistack 的实现:

1. 首次启动生成 16 位随机密码(大小写+数字,排除 0/O/1/l/I 易混淆字符)
   - 4 个默认账号共用:admin / dba / auditor / developer
   - 写入数据目录 initial-admin-password.txt(0600)
   - 打印到启动日志(含账号列表、密码、文件路径、docker/kubectl 获取命令)

2. 支持 ADMIN_INIT_PASSWORD 环境变量覆盖(自动化部署 / 需要固定密码时)

3. 生成失败直接 log.Fatalf 退出,不降级到公开默认口令

4. 批量导入用户 BatchImportUsers:空密码不再默认 admin123,
   改为随机 14 位(新增 genPassword),保证每个导入用户密码不可预测

5. 幂等:已存在的账号不重建、不改密码;只有缺失的账号才创建

6. imagePullPolicy: IfNotPresent → Always
   原因:latest tag 配 IfNotPresent 会导致节点一直复用缓存的旧镜像,
        新 push 的镜像不生效(本次实测踩到:Pod imageID 仍是旧 digest)

7. 文档
   - README 新增「首次部署获取默认账号密码」章节(含 4 账号角色表 + 3 种获取方式)
   - docker-compose.yml 加 ADMIN_INIT_PASSWORD 环境变量
   - kubernetes/deployment.yaml 加 env 注释

验证(清空 DB 后全新 seed):
  镜像 sha256:eaf0affd8d93
  /app/data/initial-admin-password.txt  199B  权限 0600
  内容含 "适用账号: [admin dba auditor developer]" + 随机密码
  启动日志完整打印获取方式
  登录实测:admin / dba / developer 用新密码均返回 access_token;
           admin123 登录失败(已失效)
  二进制 strings:admin123 命中 0
2026-10-26 17:37:21 +00:00
opsx 396df57c9c build+deploy: 编译加 -trimpath,镜像改 lghost/opsxsql,K8s 部署到 sql 命名空间
1. -trimpath(抹除源码绝对路径)
   - Dockerfile: go build 加 -trimpath
   - run.sh: 两处 go build 加 -trimpath
   效果:二进制内 /app/backend/ 绝对路径命中 66 → 0(module 相对路径 sql-platform/ 仍保留,
        那是 Go 的正常行为,-trimpath 只去绝对前缀)

2. 镜像
   - kubernetes/deployment.yaml: opsxsql:latest → lghost/opsxsql:latest
   - kubernetes/install.sh: 默认 IMAGE 改 lghost/opsxsql:latest
   - 已构建并推送:lghost/opsxsql:latest (digest 70562a408cae, 25.2MB)

3. K8s 部署验证(namespace: sql)
   - Deployment opsxsql 1/1 Running,0 重启
   - Pod: opsxsql-695444d6b9-chrzp (node 172.19.0.161)
   - Service opsxsql.sql.svc.cluster.local:80
   - PVC opsxsql-data Bound 10Gi (cbs)
   - Secret opsxsql-secrets 自动生成(SECRET_KEY / JWT_SECRET)
   - /health → {"backend":"go","status":"ok","version":"0.4.0"}
   - 首页正常,seed 完成(26 条审核规则 + LDAP 默认配置)

部署命令:bash kubernetes/install.sh
2026-10-26 17:22:29 +00:00
Logan.Li dbe9e4ab86 docs(k8s): document cross-cloud install.sh capabilities
- env var table (IMAGE/NS/STORAGE_CLASS/PVC_SIZE/LOCAL_STORAGE/NODE_NAME),
  cloud detection + storage class chain + zero-dependency local PV
  fallback, examples/ layout in file table
2026-10-26 22:10:08 +08:00
Logan.Li 738498fea3 feat(k8s): align install.sh with latest opsx-loki version
- cloud auto-detection (ACK/TKE/EKS/CCE/GKE/AKS/K3s/self-hosted) via
  node providerID, used for deployment banner and failure diagnostics
- storage class resolution chain: explicit STORAGE_CLASS > default
  annotation > sole SC; self-hosted clusters without any SC now get a
  zero-dependency static local PV (hostPath DirectoryOrCreate +
  nodeAffinity pin + WaitForFirstConsumer + Retain), LOCAL_STORAGE
  auto/off/force tri-state
- PVC rendered with resolved storageClassName + PVC_SIZE; image
  rendered into deployment via IMAGE env
- rollout failure diagnostics per waiting-reason: ImagePullBackOff
  lists per-cloud registry suggestions, CreateContainerConfigError
  points to secret, PVC Pending suggests per-cloud SC
- colored info/ok/warn/die helpers, /dev/urandom key fallback,
  completion banner with pod/node/ip, port-forward and backup hints
- kubernetes/examples/: alibaba ALB + tencent qcloud (CLB health
  check annotations, NodePort backend) ingress templates, secret
  example moved here (opsx-loki layout)
- README updated with env-var table and local-storage notes
2026-10-26 22:09:33 +08:00
Logan.Li 0b16488029 feat(deploy): single-image docker-compose and kubernetes install.sh
- multi-stage root Dockerfile (opsx-loki pattern): frontend dist +
  go binary + gh-ost in one alpine image, non-root user, HEALTHCHECK,
  entrypoint warning for missing SECRET_KEY; gin now serves the SPA
  from DIST_DIR (asset/db-logos/favicon passthrough, /api returns JSON
  404, SPA fallback to index.html) - one origin, no CORS surface
- docker-compose.yml rewritten: single service, named data volume,
  no-new-privileges + cap_drop ALL, /health healthcheck, resource
  limits, log rotation; SECRET_KEY required (openssl rand -hex 32),
  JWT_SECRET optional (auto-generated and persisted in prod)
- kubernetes/: install.sh one-command deploy (idempotent secret
  reuse, --nodeport/--ingress flags, rollout wait, backup tips) plus
  namespace/secret/pvc/deployment(single replica + Recreate, probes,
  resource limits, non-root)/service/nodeport/ingress manifests and
  README (sqlite single-writer constraints, snapshot backup)
- docs: README quick-start now has 3 deployment paths; ARCHITECTURE
  ch.14 expanded

single-image runtime verified locally (SPA/assets/api/login coexist
on :9002); full image build left to server (docker hub unreachable
from this machine)
2026-10-26 20:33:26 +08:00
Logan.Li 13fb3aceb3 feat(ldap): ad/openldap dual compat and config gating
- multi-round user filters (config filter first, then
  objectClass=inetOrgPerson, then AD sAMAccountName/userPrincipalName
  fallback) applied to login search, user listing and batch import;
  previously pure-AD directories found no users
- attribute fallback chains (uid/sAMAccountName/userPrincipalName,
  cn/displayName, telephoneNumber/mobile) via ldapEntryAttr
- import errors explain likely cause instead of empty result
- AES-encrypted admin password kept (aiops stores plaintext)
- frontend gating per aiops pattern: LDAP account management panel
  hidden with hint until a config is actually saved (was gated on
  unsaved form values); batch-import modal hides to/from-LDAP tabs
  when unconfigured, local CSV always available
2026-10-26 20:08:46 +08:00
Logan.Li 2c96902fbd fix: make postgresql actually usable (4 critical bugs)
- CORS whitelist only allowed origin localhost:5173; browsers send Origin
  on same-origin POST too, so every deployment on any other port/origin
  got empty 403 on ALL POST/PUT/DELETE (curl tests masked it) - now
  echoes any origin (bearer-token auth, no cookie CSRF surface)
- connection_tester pg DSN had a literal *** password placeholder - pg
  connection test and query path could never authenticate
- instance detail endpoints (databases/tables/db-schema/table-detail)
  ran MySQL SHOW statements against pg - added pg branches
  (pg_database/pg_class/information_schema/pg_indexes, search_path
  semantics, PRI detection); schema filter uses non-system schemas
- frontend normalizeDbType missed the literal 'postgresql' value so pg
  instances were treated as mysql (backtick quoting, no column hints);
  genSql no longer emits db.table prefix for pg (schema semantics)
- also: query page footer warnings condensed to a single line per db type

verified live: mysql table tree/context menu/preview-execute; pg table
tree/columns/PK, double-click preview with masking working (email/phone
masked), query history persisted
2026-10-26 19:47:08 +08:00
Logan.Li c8155e2d33 fix(ticket): implicit executor for default-flow tickets
- list is_executor and ExecuteTicket both required an execute-step
  approval record; tickets created without a workflow (default flow)
  had none, so the execute button never appeared and the API returned
  403 for everyone - admin/dba are now implicit executors (approved
  status only, consistent in both list enrichment and execution check)
- status labels: pending/reviewing/approved were all shown as 审核中;
  now 待初审/审批中/待执行 so the ready-to-execute state is visible
- verified end-to-end on test backend: create -> quick review dialog
  with comment -> impact-summary execute confirm -> success, detail
  page stage steps reach 完成 with execute result visible
2026-10-26 19:14:35 +08:00
Logan.Li a4090123ba feat(ticket): stage steps and approval flow conveniences
- TicketStageSteps: status-driven stepper (opsx-sec pattern, active
  derived from ticket status incl. rejected/failed/cancelled/rollback
  terminal states) at top of ticket detail page
- QuickReviewDialog: inline approve/reject with optional comment for
  list rows and detail dialog (reject prompts reason, notes terminal
  semantics); replaces fixed-comment direct calls and alert() usage
- execute now requires impact-summary confirm (instance + database +
  statement count) in both list and detail dialog
- DetailTimeline action keys aligned with backend values (created/
  pending_review/approve/reject/executed/execution_failed) - icons and
  colors were silently falling back before
- create page: removed dead scheduled_at field (backend never accepted
  it), added rollback SQL collection (backend-supported), real priority
  select, approval-flow preview showing bound workflow steps and
  approvers once instance selected
- empty tickets state now embeds create CTA button
- RedisDetailPage split: key detail panel extracted to feature
  component (996 -> 658 lines after formatter reflow; panel 377 lines)
2026-10-26 19:06:09 +08:00
Logan.Li f6d636fa44 feat(query): sql workspace conveniences per opsxdb patterns
- generic ContextMenu component (viewport flip, esc/outside close)
- SchemaSidebar table right-click menu: preview data, generate
  SELECT/INSERT/UPDATE/DELETE templates with pk-aware WHERE, copy table
  name; double-click table runs preview SELECT; column right-click
  copy/insert; dialect-aware quoting; preview SELECT carries no LIMIT
- SqlEditor selection-first execution (Mod-Enter runs selection else
  statement at cursor via quote/comment-aware splitter); pagination
  replays last executed statement
- query history persisted server-side: drawer fetches query-histories
  with keyword/mine/favorites filters, star toggle (new PUT favorite +
  mine filter endpoints), insert and insert-and-run
- custom templates in localStorage alongside built-ins
- unit tests: genSql + statement splitter (28 tests green)
2026-10-26 18:53:02 +08:00
Logan.Li ff539b47e0 feat(redis): shared key browser with grouped tree + cursor pagination
New src/features/redis/keyBrowser/ shared by RedisDetailPage keys tab and
SchemaSidebar redis branch (both previously flat one-shot lists):
- keyTree: ':'-separated folder tree with recursive key counts, numeric
  sort, exact-hit pinned at root, local key removal (no full refetch),
  type color dots, readable TTL (45s/5m30s/2d5h)
- useRedisKeyBrowser: cursor state machine + request-sequence guard +
  Map merge on load-more; setter-synced args (Enter right after typing
  no longer races stale render), auto-search on db/instance change
- RedisKeyBrowser: enter-to-scan search bar (SCAN has cost - no debounce,
  opsxdb-style) with clear button, effective type filter, load-more
  button with cursor hint, '已全部加载' end state
- useRedisDbs: real db list from /redis/:id/dbs (cluster -> DB0 banner)
- RedisDetailPage: hash row edits now use field-level HSET/HDEL endpoint
  (preserves other fields + TTL); delete/add update tree locally; CLI
  output renders arrays/objects/errors properly
- SchemaSidebar: redis branch swaps to shared browser (compact mode,
  inline detail preview + insert-command preserved), auto-switches to
  Keys tab for redis instances
- vite dev proxy target overridable via API_PROXY_TARGET (local testing)

Browser-verified: 200/page with load-more (200->400->600), exact hit
badge, keyword auto-expand, key detail panel, queries sidebar
2026-10-26 18:39:38 +08:00
Logan.Li 6604fdf48e feat(backend): redis key scan service - cursor pagination + cluster + CLI fix
Modeled on opsxdb's redis scan design:
- opaque base64 scan cursor {pattern,db,type,nodes,index,cursor}; server
  validates search conditions match, rejects stale cursors (400)
- soft page cap 200 keys (whole scan batches returned, no key loss),
  max 20 SCAN calls per page; next_cursor until all shards exhausted
- pattern rule: no glob chars -> *kw* contains match; non-glob input
  also probed with TYPE for exact hit (returned as exact, slot-routed
  so cluster/NAT safe)
- pipeline batch TYPE/TTL replaces per-key serial N+1; type filter now
  effective server-side
- cluster: per-master direct SCAN merged across shards (MasterAddrs on
  wrapper), DB0-only, MOVED errors hint at switching deploy mode;
  SlowLog/ClientList aggregate across masters (were cluster-broken)
- GET /redis/:id/dbs (0-15, cluster -> [0])
- POST /redis/:id/keys/:key/field: hash row-level HSET/HDEL (no whole-
  key rewrite)
- RedisExecuteCommand fixed (was always-403): accepts {db, command}
  full-string, quote-aware tokenizer, whitelist on first word, expanded
  read-only set
- RedisUpdateKey preserves previous TTL unless explicitly set/cleared
  (was silently dropping TTL); string type skips the DEL

Verified live against local redis: 200/page no overlap across pages,
cursor 400 on condition change, exact hit, type filter, hash field
edit round-trip, CLI DBSIZE/HGET ok, SET rejected
2026-10-26 18:27:11 +08:00
Logan.Li 6945c53879 fix(frontend): eliminate boot race between role restore and route guard
Root cause (pre-existing): user state initialized with default role
'developer' while JWT role restoration happened via render-phase setState
~1 render later; first guard pass evaluated tickets/workflow against the
developer writable-datasource gate and bounced admins to dashboard on
direct load.

Fix: initialize user state lazily from the JWT in one pass
(restoreUserFromToken) - first render already carries the correct role.
Verified in browser: direct loads of /tickets and /workflow stay put;
URL query state survives refresh; expired session redirects to
/login?redirect=<path> and login returns to the original page.
2026-10-26 18:06:22 +08:00
Logan.Li d867c276d4 chore(frontend): engineering baseline per guide ch.8
- toolchain: vitest+jsdom (15 tests green: permission pure functions
  exhaustive, storage migration), prettier (printWidth 120 to fit the
  dense existing style + 700-line page budget), commitlint conventional
- npm scripts: typecheck / format / test / lint:fix; build fingerprint
  __APP_VERSION__ (version+git hash+build time) shown in Settings > about
- git hooks via core.hooksPath=frontend/.husky: pre-commit lint-staged
  (prettier + eslint --fix best-effort), commit-msg commitlint
- permission pure functions extracted to lib/permissions.ts (single
  source consumed by useAuth/useLogin); AuditLogsPage filters on URL
- backend: resp package unit tests; health version constant aligned
  with frontend package.json
2026-10-26 17:58:59 +08:00
Logan.Li ac498c7ba2 refactor(frontend): split 7 oversized pages into feature co-location
Per guide ch.7 page split pattern (skeleton orchestrates, private
components live in features/<domain>/):
- InstancePage 1716->328, UsersPage 1309->240, TicketsPage 1076->195,
  TicketDetailPage 911->287, QueriesPage 883->479, LdapConfigPage 860->393,
  WorkflowPage 852->194
- new feature modules: datasource/ rbac/ ticket/ query/ system/ldap/
  governance/workflow/ (components + hooks + types + utils)
- stable keys replace index keys in dynamic lists; polling intervals
  and toast call sites preserved 1:1 (verified by agents)
- filter/pagination state moved to URL query via new useQueryState hook
  (Instance/Users/Tickets/AuditLogs pages): refresh/share/back-button
  restore state; composite updates applied atomically
- permission pure functions extracted to lib/permissions.ts (used by
  useAuth + useLogin); page-size audit: 0 pages over 700-line budget
2026-10-26 17:54:15 +08:00
Logan.Li ac563d1c5e refactor(frontend): unify theme tokens to single --dg-* source
- remove shadcn HSL dual-track vars (523->416->~300 lines index.css);
  tailwind semantic colors (border/background/card/primary/destructive/
  popover/muted/accent) now bridge directly to var(--dg-*) tokens
- drop dead darkMode:['class'] config and inert dark: variants in kept
  ui components; theme switching is data-theme based (4 themes intact)
- first-frame inline theme script in index.html (reads namespaced key
  with legacy fallback, resolves system, sets color-scheme) - no flash
- consolidate duplicated db-type brand colors into lib/constants.ts
  single source (dbTypeColorMap + dbTypeBrandColorMap, with pg/mongodb
  aliases); instance health stroke now uses theme tokens
- verified via browser: dark #0d1117 / light #f5f6f8 both resolve,
  tokens cascade correctly
2026-10-26 17:38:27 +08:00
Logan.Li a213d08e31 chore(frontend): purge dead code per guide red line
- delete 4 dead pages (~2300 lines): Home, ConfigPlatformPage,
  DataDictionaryPage, NotificationsPage (kept the live NotificationPage)
- delete 52 unused shadcn ui components (keep button/card/dialog in use;
  re-generate via shadcn CLI when needed)
- delete App.css, SkeletonCard, empty shared/ and features engine dirs
- remove dead theme blocks (midnight-purple / orange-vitality) from index.css
- uninstall ~40 dead deps (echarts, sonner, next-themes, zod,
  react-hook-form, unused radix packages, cmdk, vaul, ...)
- fix DEVELOPMENT.md drift: mock layer section rewritten to real API layer,
  permission docs now point to menu.ts single source
2026-10-26 17:23:24 +08:00
Logan.Li 16f1f7d65f feat(backend): unified response helpers, refresh token, security hardening
- new resp package (OK/Page/Fail/BadRequest/.../InternalErr): unified error
  envelope {detail, code?}, sanitized 500s (full error logged server-side,
  generic message to client - stops DB error leakage), pagination envelope
  {items,total,page,page_size}; migrate 200+ inline c.JSON error sites
- refresh token: 7d rotation tokens (SHA-256 hashed at rest), public
  POST /auth/refresh with single-use rotation, POST /auth/logout revokes
  all user refresh tokens (endpoint previously missing - frontend 404)
- security: gh-ost command line password masked in task status payload;
  Instance.MarshalJSON guarantees encrypted passwords never leave the API
  (removes reliance on per-handler manual masking); production (DEBUG=false)
  with default JWT secret now auto-generates and persists a random secret
- safe.Go recover wrapper for background goroutines (instance probe,
  gh-ost async)
- delete dead service layer (zero callers)
- fix pre-existing go vet unkeyed bson.E literals
2026-10-26 17:21:00 +08:00
Logan.Li d5b8b8116c fix(frontend): harden auth request layer per web guide
- fix double-unwrap bug in useLogin prefs sync (theme sync silently broken)
- request layer: 401 single-flight refresh queue + debounced single redirect
  with return-url, cancel-aware silent passthrough, skipErrorHandler opt-out,
  typed blob download (replace bare fetch export in QueriesPage)
- useFetchList: AbortController + version guard + error state (three states)
- ErrorBoundary with chunk-load auto recovery (reload once)
- permission single source: derive route guard from menu constants, drop
  hand-maintained routePermissionMap; TicketDetailPage uses unified
  hasActionPermission instead of inline role string comparison
- namespaced localStorage keys (opsxsql.*) with read-old-write-new migration
- fix pre-existing tsc errors (InstancePage/WorkflowPage/QueriesPage/
  PlatformConfigPage/RedisDetailPage) - typecheck now passes clean
2026-10-26 17:12:29 +08:00
hermes-$(hostname) 0fcdbc5f6c update 2026-07-08 08:53:48 +00:00
Logan.Li f5cf3f4440 fix: prevent query execute event state leak 2026-07-07 17:52:44 +08:00
Logan.Li 617c15dbfe feat: harden query read-only workflow 2026-07-07 17:20:44 +08:00
Logan.Li 58bbc394e9 feat: enhance redis and production IA 2026-07-07 10:39:33 +08:00
Logan 644981eef4 feat: improve datasource query audit workflows 2026-07-06 23:01:04 +08:00
logan 74748e4914 fix: 审批流程三个bug修复
1. 点通过不再重复创建审批记录(后端无NULL记录时返回400而非自动创建)
2. 非提交人不再显示撤销按钮(列表页增加applicant_id校验)
3. 已审批工单不再显示通过/拒绝按钮(is_approver只计算pending记录)
4. 前端审批失败时弹出错误提示而非静默忽略
2026-06-03 17:17:12 +08:00
logan 3a6dd99633 feat: 审核规则驱动SQL检查 + db_type过滤 + Redis只读拦截 + 配置文档
1. 审核规则改为从audit_rules表动态匹配(去硬编码)
   - CreateTicket提交时查audit_rules WHERE db_type=? AND is_enabled=true
   - 正则匹配SQL,value=reject拦截,warn/review放行
   - UPDATE/DELETE无WHERE: Go RE2不支持(?!...)负向前瞻
     → 用value=where_check特殊标记+strings.Contains WHERE检测
   - 修复UPDATE/DELETE无WHERE正则(Go兼容)

2. ListAuditRules按db_type过滤
   - ghost.go: 添加c.Query('db_type') WHERE过滤
   - MySQL不显示KEYS规则,Redis不显示DROP规则

3. Redis只读命令拦截(工单提交)
   - 新增Redis只读命令白名单: GET/HGET/MGET/HGETALL/KEYS/SCAN等28个
   - 只读命令走工单→直接拦截,引导用户用查询功能

4. 补充audit_rules规则
   - Redis新增7条: DEBUG/SHUTDOWN/SCRIPT/SAVE/BGSAVE/BGREWRITEAOF/SLAVEOF/CLUSTER
   - MySQL新增1条: DROP SCHEMA
   - 总计34条规则(MySQL12/Redis11/PG7/MongoDB4)

5. docs/CONFIGURATION.md 配置系统文档
   - 三套配置体系: system_settings/audit_rules/ldap_configs
   - 每项配置: key/默认值/作用/代码引用位置
   - 未引用配置标注: executor_batch_size/executor_backup/query_max_rows_examined(待实现)
   - LDAP冗余配置: system_settings中7项ldap_*无代码引用(LDAP用独立表)

验证: 5/5审批流通过, 36/36安全测试通过, 27/27审核规则匹配正确
2026-06-03 03:17:22 +08:00
logan 8fbcb8647d feat: SQL安全防护体系 — 危险SQL黑名单 + 事务回滚 + LIMIT修复
🔴 P0 安全漏洞修复:
- MySQL: 黑名单拦截 DROP DATABASE/DROP SCHEMA
- MySQL: 拦截无WHERE的UPDATE/DELETE(全表操作)
- MySQL: 多语句中含危险SQL也会被检测
- Redis: 拦截 FLUSHALL/FLUSHDB/CONFIG/DEBUG/SHUTDOWN/SCRIPT/SAVE/BGSAVE/KEYS/CLUSTER/REPLICAOF

🔄 事务支持:
- MySQL多语句执行包裹在事务中
- 任一SQL失败自动回滚全部已执行语句
- 验证通过: INSERT成功+UNIQUE冲突=全部回滚

🔧 Bug修复:
- query.go: LIMIT只追加SELECT语句(SHOW/CREATE等不再追加LIMIT报错)
- SHOW VARIABLES/STATUS 恢复正常

📊 测试验证:
- 29/29 安全拦截测试全部通过
- MySQL 8项危险操作全部拦截
- Redis 11项危险命令全部拦截
- 10项正常SQL全部放行
- 事务回滚验证通过
- 类型错误/参数错误正确报告
2026-06-03 02:49:07 +08:00
logan 8ceb249274 fix: MySQL DDL+DML同批EXPLAIN预检bug + Redis集群MOVED重定向 + 清晰错误提示
MySQL修复:
- 批次含DDL时跳过整个EXPLAIN预检(避免CREATE TABLE未执行时INSERT的EXPLAIN失败)
- 纯DML批次仍正常做EXPLAIN预检

Redis集群修复:
- 单节点客户端遇到MOVED时自动映射外部IP+端口重试
- 重试失败给出清晰提示(Docker端口未暴露)
- 新增 isClusterInstance/parseMovedTarget 辅助函数

测试验证通过:
- MySQL 5条DDL+DML ✅ (建表+INSERT+UPDATE+加索引)
- MySQL 纯DML+EXPLAIN预检 ✅
- MySQL dba(logan)执行 ✅
- Redis单机 SET写入 ✅
- Redis集群 ⚠️ Docker内部端口未暴露(基础设施限制)
2026-06-03 02:40:15 +08:00
logan 2984f11181 refactor: tests目录精简重构 + 脏数据清理脚本
- 删除16个旧Python测试文件(970行→400行)
- 重写conftest.py适配真实种子用户(admin/test123/test234/testtest/logan)
- 5个精简测试文件覆盖核心API:
  - test_auth.py: 登录/鉴权/种子用户验证 (6 tests)
  - test_instances.py: 实例列表/详情/连接测试 (6 tests)
  - test_tickets.py: 工单CRUD/只读SQL拦截/auditor权限 (7 tests)
  - test_workflow.py: E2E完整审批流/拒绝/撤销 (3 tests)
  - test_permissions.py: 角色权限/执行权限/workflow权限 (6 tests)
- 新增 cleanup_test_data.py 清理测试脏数据(支持--dry-run)
- 全量29测试3.27s通过
2026-06-03 02:24:40 +08:00
logan bec2cafec2 feat: 执行结果结构化-每条SQL记录status/affected_rows/duration_ms + 去掉工单详情优先级字段
后端:
- 新增 SqlExecItem/SqlExecResult 结构体
- MySQL/Redis/gh-ost 三条执行路径全部改为 JSON 结构化结果
- 每条 SQL 记录完整语句、状态、影响行数、耗时(ms)、错误信息
- 总耗时/开始结束时间一并记录

前端:
- ExecuteResult 接口匹配后端 SqlExecResult
- 表格展示每条 SQL 执行结果(序号/SQL/状态/影响行数/耗时/错误)
- 汇总卡片: 状态/总耗时/SQL数量
- 去掉工单详情中的优先级和变更原因字段
- 保留 plain text fallback 兼容旧数据

E2E验证: INSERT+UPDATE+ALTER 三语句全部成功, 结构化结果正确
2026-06-03 02:15:37 +08:00
logan 1364c737ef fix: 工单列表权限过滤-developer/auditor也能看到待审工单
- developer之前只能看applicant_id=自己的工单,导致审批人看不到待审工单
- 改为admin看全部,其他角色(包括developer/dba/auditor)都能看到自己提交+自己作为审批人的工单
- E2E全流程验证通过: testtest提交→test123初审→test234复审→admin执行→success
2026-06-03 02:06:20 +08:00
logan bd8100bfe0 fix: 撤销工单支持approved状态 + DDL跳过EXPLAIN预检 + 审批菜单仅admin/dba + 审批状态流转修复 + 权限组实例绑定修复 2026-06-03 01:53:21 +08:00
logan b1a3d74661 feat: 角色权限体系改造 — developer可审批, auditor纯只读
角色权限矩阵:
- admin: 全部权限
- dba: 数据源管理+工单全流程+审批+执行
- developer: 查询+提交工单+审批(不能执行)
- auditor: 只读(审计日志+审核规则+审批流程查看)

后端改动:
- ticket.go: 修复GetCurrentUser返回值顺序bug(userID,username,role)
- ticket.go: CreateTicket拦截auditor提交工单
- ticket.go: ApproveTicket允许admin/dba/developer
- ticket.go: ExecuteTicket仅允许admin/dba
- router.go: query执行限制admin/dba/developer
- router.go: workflow写操作限制admin/dba
- router.go: audit-rules写操作限制admin/dba

前端改动:
- menu.ts: tickets去掉auditor, workflow加developer+auditor, audit-logs加auditor
- useAuth.tsx: routePermissionMap和actionPermissionMap同步更新
- TicketDetailPage: canApprove加developer角色

DB迁移: test123/test234从auditor→developer
2026-06-03 01:19:24 +08:00
logan 97fd4ff069 fix: 数据变更工单拦截只读SQL — 前端提交/检测+后端创建三层拦截SELECT/SHOW/DESC/EXPLAIN 2026-06-03 00:48:09 +08:00
logan 70da8a8a8b fix: 工单执行按钮仅在审核通过(approved)状态显示,审核中不再显示执行按钮 2026-06-03 00:38:12 +08:00
logan bbf9522710 fix: 消息推送添加渠道placeholder改为数据变更消息机器人 2026-06-01 21:33:33 +08:00
logan 7564d92c69 fix: NotificationPage 添加渠道弹框 X 图标未 import 导致报错 2026-06-01 21:11:36 +08:00
logan 132b2db2d2 fix: 平台配置页面修复 - 隐藏LDAP重复分组、敏感字段列表textarea多行展示、分组排序优化
- 过滤ldap分组(有独立LDAP页面不重复展示)
- 敏感字段列表从单行input改为textarea每行一个字段名(key修正data_masking_sensitive_fields)
- 添加GROUP_ORDER固定展示顺序:查询控制→安全策略→数据脱敏→日志清理→执行器→Ghost/OSC
- 所有分组输入框均有默认值,cleanup组retention天数输入框正常显示
2026-06-01 21:07:44 +08:00
logan 9c457eb599 chore: Dockerfile gh-ost 版本更新 v1.1.9(macOS本地验证通过)
- gh-ost v1.1.9 真实执行验证通过(macOS arm64 + 远程MySQL)
- ALTER TABLE ADD COLUMN → gh-ost OK (6行表,~8秒完成)
- ALTER TABLE ADD INDEX → gh-ost OK
- ghost_enabled 默认 false,本地开发: ~/bin/gh-ost
2026-06-01 01:55:29 +08:00
logan b2b689f791 feat: 工单执行集成gh-ost通道 + Dockerfile安装gh-ost + 密码解密修复
- ExecuteTicket: 检测 ghost_enabled + ALTER TABLE → 走 gh-ost 同步执行
- parseAlterTable: 解析 ALTER TABLE 提取表名和 alter 子句(支持反引号/db.table)
- buildGhOstTask: 修复密码传递(加密→解密后传给 gh-ost 命令行)
- Dockerfile: 运行阶段下载 gh-ost v1.1.6 Linux amd64 二进制
- ghost_enabled=false 默认关闭,工单流程正常走普通通道
- 端到端验证: ALTER TABLE+ghost_enabled=true → gh-ost; INSERT/关闭 → 普通
2026-06-01 01:47:08 +08:00
logan bc2092a980 feat: 平台配置全面生效 — 配置策略接入实际逻辑
- query组: ExecuteQuery读取query_limit_max/query_timeout/query_dangerous_pattern
  - 查询超时通过context.WithTimeout生效,db.QueryContext/QueryRowContext
  - 危险模式检测拦截无WHERE的DELETE/UPDATE/DROP/TRUNCATE
  - LIMIT上限从settings读取,不再硬编码1000

- security组: Login读取login_max_attempts/login_lockout_minutes
  - 新增User字段: failed_login_attempts + locked_until
  - 登录失败达到阈值自动锁定,成功时清除计数
  - JWT Token过期时间读取session_timeout_hours设置

- executor组: ExecuteTicket读取executor_timeout/executor_dry_run
  - 执行超时通过context.WithTimeout控制
  - 开启dry_run时先EXPLAIN预检查SQL语法
  - db.ExecContext替代db.Exec

- cleanup组: SettingsCleanupPreview/Run从占位改为真实实现
  - 预览统计4类数据(工单/查询历史/慢日志/审计日志)待清理数量
  - 执行清理按retention_days配置删除过期数据

- ghost组: 13项gh-ost配置加入seedDefaultSettings
  - enabled/path/mode/osc_priority/env/cloud/trigger/chunk/load/lag等
  - 已被ghost.go中getGhostConfig等函数读取

- 前端: PlatformConfigPage新增Ghost/OSC分组显示
2026-06-01 01:00:23 +08:00
logan 8af868b3ca fix: 数据源编辑密码保留 + Redis ACL 认证支持
后端:
- UpdateInstance: 空密码和'******'都不更新密码字段,防止编辑后丢失密码
- testRedis/getRedisClient/getRedisMetrics: 增加 Username 参数支持 Redis 6.0+ ACL 认证
- username 为 'default' 或空时走传统密码认证模式

前端:
- EditInstanceDialog: Redis 用户名字段提示'6.0+ ACL 可选',密码 placeholder 改为'不修改请留空'
- CreateInstanceForm: Redis 类型也显示用户名输入框(可选),创建/测试连接时 Redis username 为空则不传
- Redis 提示文案从'无需用户名'改为'6.0+ ACL 支持用户名+密码认证'
2026-05-31 23:55:18 +08:00
logan bfb64d4dce fix: WorkflowEditor 下拉过滤禁用用户 — getEligibleUsers 增加 is_active 条件
- 审核阶段: 只显示 is_active=true 且角色为 admin/dba/auditor 的用户
- 执行阶段: 只显示 is_active=true 且角色为 admin/dba 的用户
- 禁用用户不再出现在审批流程编辑器下拉列表中
2026-05-31 23:35:58 +08:00
logan 1f9d373c08 fix: 审批流程编辑器按步骤类型过滤可选用户
- 审核阶段: 只显示 admin/superadmin/dba/auditor 角色用户
- 执行阶段: 只显示 admin/superadmin/dba 角色用户
- 全选按钮也基于过滤后的用户列表
- UserDropdown 改用 fixed 定位避免 overflow 裁剪
2026-05-31 23:22:08 +08:00
logan aaa7666d66 fix: 统一所有弹框组件 padding + 主题风格
- ModalOverlay: 内容容器加 p-6 + border + shadow-xl
- alert-dialog: 统一主题变量 bg-[var(--dg-bg-secondary)] + rounded-xl
- sheet: SheetContent 加 p-6,移除 Header/Footer 重复 p-4
- drawer: DrawerContent 加 p-6,移除 Header/Footer 重复 p-4
- 所有弹框统一: p-6 padding + dg-border + shadow-xl + z-[101]
2026-05-31 23:08:08 +08:00
logan 363753653e fix: DialogContent 添加默认 padding p-6,修复确认弹框布局压缩问题
- dialog.tsx: DialogContent 基础样式加 p-6 (24px padding)
- ConfirmDialog.tsx: 按钮区域间距 pt-2 → pt-4
- 修复所有使用 Radix Dialog 的弹框内容紧贴边缘的问题
2026-05-31 23:05:51 +08:00