diff --git a/OPTIMIZATION_COMPARISON_REPORT.md b/OPTIMIZATION_COMPARISON_REPORT.md new file mode 100644 index 0000000..77eb3ab --- /dev/null +++ b/OPTIMIZATION_COMPARISON_REPORT.md @@ -0,0 +1,198 @@ +# SQL平台代码v1 优化包对比报告 + +> 分析日期: 2026-05-23 +> 分析对象: ~/Downloads/SQL平台代码v1/ (optimized + final) +> 当前项目: ~/devops/sql-platform/ + +--- + +## 一、优化包概览 + +优化包包含两个版本： +- **sql-platform-optimized**: 中间优化版本（前 3 个 Stage 的产出） +- **sql-platform-final**: 最终版本（在 optimized 基础上新增数据库审核规则拆分） + +两者基于 4 阶段优化计划（plan.md）: +1. **Stage 1** — 后端关键缺陷修复 +2. **Stage 2** — 前端界面全方位优化（4 组并行） +3. **Stage 3** — 权限逻辑 & 安全增强 +4. **Stage 4** — 构建验证 & 最终集成 + +--- + +## 二、项目评估摘要（来自 PROJECT_EVALUATION.md） + +| 维度 | 评分 | 说明 | +|------|------|------| +| 功能完成度 | 8.5/10 | 核心功能全覆盖：工单全生命周期、SQL审核引擎、规则CRUD、审计日志 | +| 代码质量 | 8.5/10 | TDD驱动、128测试、模块化清晰 | +| 安全性 | 9.0/10 | AES加密、bcrypt、JWT、AST防注入、权限矩阵 | +| 测试覆盖 | 8.5/10 | 128 tests 覆盖11模块 | +| 生产就绪度 | 8.0/10 | Docker部署、Alembic迁移、GitLab CI/CD | +| 用户体验 | 7.5/10 | 缺国际化、深色模式 | +| **综合评分** | **8.1/10** | 目标 9.0+ | + +--- + +## 三、路线图优化项（来自 ROADMAP.md） + +### 已完成的优化 + +| 分类 | 优化项 | 规模 | +|------|--------|------| +| **P0 数据库全景支持** | 统一查询服务 (MySQL/PG/Redis/MongoDB) | query_service.py 完整4种Handler | +| | Redis 只读命令白名单 | 40+ 只读命令 + 子命令验证 | +| | MongoDB 只读操作白名单 | find/findOne/countDocuments/distinct/aggregate | +| | 工单执行多数据库分发 | ticket_executor.py 多DB执行 | +| | MongoDB 连接器 + 前端管理页面 | mongo_connector.py 780行 | +| **P1 审计闭环** | 查询历史服务 | query_history_service.py | +| | SQL 助手 | sql_assistant.py (格式化/风险高亮/EXPLAIN) | +| | 审计规则种子 | MySQL 18规则 + PG 12规则 + Redis 8规则 | +| | 影响分析 | impact_analyzer.py | +| | 回滚生成 | rollback_generator.py | +| | Schema 漂移检测 | schema_drift_detector.py | +| **P2 高级功能** | 数据脱敏 | data_masker.py + masking_rule CRUD | +| | 通知服务 | 钉钉/飞书/邮件 SMTP | +| | 备份管理 | backup_executor.py | +| | 工作流引擎 | workflow_engine.py + workflow_seeds.py | +| | 慢日志收集 | slow_log_collector.py | +| | 连接池 | connection_pool.py | + +### 待完成项 + +- [ ] 执行前后数据快照对比可视化（前端） +- [ ] 快捷查询模板（10个预置 + 自定义共享） +- [ ] SQL 格式化前端集成 +- [ ] 查询结果格式化展示优化 +- [ ] SQL 语法高亮编辑器 +- [ ] Slow query 前端集成 +- [ ] Dashboard 数据源健康概览 + +--- + +## 四、版本差异分析 + +### 4.1 optimized vs final 差异 + +**final 在 optimized 基础上新增了 4 个数据库专用审核规则文件：** + +| 新增文件 | 行数 | 规则数量 | 覆盖范围 | +|----------|------|----------|----------| +| `mysql_audit_rules.py` | 2,101 行 | 63条 | DDL 36条 + DML 9条 + 命名 5条 + 性能 5条 + 安全 5条 | +| `redis_audit_rules.py` | 1,863 行 | 43条 | KEY管理10条 + 命令安全12条 + 数据结构8条 + 性能8条 + 安全5条 | +| `mongodb_audit_rules.py` | 1,638 行 | 42条 | 集合管理 + 查询安全 + 聚合管道 + 写入安全 + 性能 | +| `postgres_audit_rules.py` | 1,289 行 | 45条 | DDL 18条 + DML 8条 + DQL 6条 + 安全6条 + 性能7条 | + +**final 还修改了 5 个关联文件：** +- `audit_engine_v2.py` — 新增 4 个 DB 规则模块的 import，规则数量从 100+ 升级到 200+ +- `audit_rule_seeds.py` — 新增 MongoDB 种子规则 + 别名映射 +- `audit_rule.py` (endpoint) — 新增数据库类型别名支持（postgresql→postgres, mongo→mongodb） +- `AuditRulesPage.tsx` — 前端新增 MongoDB/Redis 相关关键词识别 + +### 4.2 optimized vs 当前项目差异 + +**文件结构：完全一致**（无新增/缺失文件，仅 `__pycache__` 差异） + +**内容差异：共 25 个文件有改动** + +#### 后端 (9 个文件) + +| 文件 | 改动行数 | 改动性质 | +|------|----------|----------| +| `services/workflow_engine.py` | 1,284 | 工作流引擎大幅扩展 | +| `services/query_service.py` | 720 | 查询服务优化/增强 | +| `services/audit_engine.py` | 858 | 审核引擎 V1 优化 | +| `services/data_masker.py` | 668 | 数据脱敏模块增强 | +| `services/ticket_executor.py` | 530 | 工单执行器改进 | +| `models/workflow.py` | 331 | 工作流模型扩展 | +| `models/user.py` | 278 | 用户模型扩展 | +| `endpoints/ticket.py` | 540 | 工单 API 改进 | +| `endpoints/query.py` | 256 | 查询 API 改进 | + +#### 前端 (16 个文件) + +| 文件 | 改动行数 | 改动性质 | +|------|----------|----------| +| `pages/InstancePage.tsx` | 1,372 | 实例管理页大幅优化 | +| `pages/WorkflowPage.tsx` | 971 | 工作流页面优化 | +| `pages/TicketsPage.tsx` | 960 | 工单列表页优化 | +| `pages/TicketCreatePage.tsx` | 868 | 工单创建页优化 | +| `pages/UsersPage.tsx` | 804 | 用户管理页优化 | +| `pages/SlowQueryPage.tsx` | 703 | 慢查询页优化 | +| `pages/TicketDetailPage.tsx` | 719 | 工单详情页优化 | +| `pages/DataDictionaryPage.tsx` | 650 | 数据字典页优化 | +| `pages/Dashboard.tsx` | 636 | 仪表盘优化 | +| `pages/InstanceGroupPage.tsx` | 631 | 实例组页优化 | +| `pages/AuditLogsPage.tsx` | 600 | 审计日志页优化 | +| `pages/QueriesPage.tsx` | 601 | 查询页优化 | +| `pages/AuditRulesPage.tsx` | 584 | 审核规则页优化 | +| `pages/RolesPage.tsx` | 519 | 角色管理页优化 | +| `hooks/useAuth.tsx` | 391 | 认证 hook 优化 | +| `components/SqlEditor.tsx` | 3 | SQL编辑器微调 | + +--- + +## 五、对我们当前项目有价值的改动 + +### 🔴 高价值（建议立即采纳） + +1. **数据库专用审核规则（final 独有）** — 193条数据库专用规则 + - `mysql_audit_rules.py` (63条) — 生产级 MySQL 审核覆盖 + - `redis_audit_rules.py` (43条) — Redis 命令安全检测 + - `mongodb_audit_rules.py` (42条) — MongoDB 操作审核 + - `postgres_audit_rules.py` (45条) — PostgreSQL 特有语法检查 + - 关联修改：`audit_engine_v2.py`, `audit_rule_seeds.py`, `audit_rule.py` + +2. **前端页面全面优化** — 16 个页面的 UI/UX 改进 + - 所有核心页面均有数百行改动，估计是界面交互、组件化、错误处理方面的增强 + +3. **后端核心服务优化** — 9 个关键文件 + - `workflow_engine.py` (1,284 行改动) — 工作流引擎重大升级 + - `query_service.py` (720 行改动) — 查询服务改进 + - `ticket_executor.py` (530 行改动) — 工单执行器改进 + +### 🟡 中等价值 + +4. **数据脱敏增强** — `data_masker.py` 668 行改动 +5. **用户/工作流模型扩展** — `user.py` + `workflow.py` 模型字段扩展 +6. **认证 hook 优化** — `useAuth.tsx` 391 行改动 + +### 🟢 待评估 + +7. **TDD 测试计划** — 当前 128 tests，Phase 4 安全测试未完成 +8. **路线图待完成项** — 快照对比、查询模板、前端集成等 + +--- + +## 六、建议操作 + +### 立即可做 +```bash +# 将 final 版本的4个新审核规则文件复制到我们的项目 +cp ~/Downloads/SQL平台代码v1/sql-platform-final/backend/app/services/mysql_audit_rules.py ~/devops/sql-platform/backend/app/services/ +cp ~/Downloads/SQL平台代码v1/sql-platform-final/backend/app/services/redis_audit_rules.py ~/devops/sql-platform/backend/app/services/ +cp ~/Downloads/SQL平台代码v1/sql-platform-final/backend/app/services/mongodb_audit_rules.py ~/devops/sql-platform/backend/app/services/ +cp ~/Downloads/SQL平台代码v1/sql-platform-final/backend/app/services/postgres_audit_rules.py ~/devops/sql-platform/backend/app/services/ +``` + +### 需要逐步验证 +- 逐个对比 25 个差异文件的改动内容，评估每项改动的兼容性 +- 运行现有测试确保不引入回归 +- 前端改动需逐一构建验证 + +### 建议策略 +1. 先合并 `final` 版本新增的 4 个审核规则文件 + 关联修改（低风险，纯新增） +2. 逐页面合并前端优化（每合并一个页面后构建验证） +3. 最后处理后端核心服务改动（需配合测试） + +--- + +## 七、总结 + +| 项目 | 数据 | +|------|------| +| 优化包总改动文件数 (vs 我们) | **29 个** (25差异 + 4新增) | +| 优化包总新增代码量 | ~6,900 行 (4个新规则文件) + 大量行级改动 | +| 文件结构差异 | **无** (完全一致的目录结构) | +| 最有价值的改动 | **193条数据库专用审核规则** (final 独有) | +| 风险评估 | 中等 — 文件结构一致但内容改动量大 | diff --git a/backend/app/api/v1/endpoints/audit_rule.py b/backend/app/api/v1/endpoints/audit_rule.py index d77c947..912e440 100644 --- a/backend/app/api/v1/endpoints/audit_rule.py +++ b/backend/app/api/v1/endpoints/audit_rule.py @@ -31,7 +31,10 @@ async def list_audit_rules( if severity: stmt = stmt.where(AuditRule.severity == severity) if db_type: - stmt = stmt.where((AuditRule.db_type == db_type) | (AuditRule.db_type == "all")) + # 支持类型别名: postgresql -> postgres, mongo -> mongodb + db_type_aliases = {"postgresql": "postgres", "pg": "postgres", "mongo": "mongodb"} + resolved = db_type_aliases.get(db_type.lower(), db_type.lower()) + stmt = stmt.where((AuditRule.db_type == resolved) | (AuditRule.db_type == db_type) | (AuditRule.db_type == "all")) if is_enabled is not None: stmt = stmt.where(AuditRule.is_enabled == is_enabled) @@ -80,8 +83,23 @@ async def seed_audit_rules( db: AsyncSession = Depends(get_db), current_user: User = Depends(require_role("admin")), ): - """批量初始化预设审核规则种子数据。""" + """批量初始化预设审核规则种子数据。 + + 支持4种数据库类型: + - mysql: MySQL 专用规则 (60+ 条) + - postgres: PostgreSQL 专用规则 (45+ 条) + - redis: Redis 专用规则 (43+ 条) + - mongodb: MongoDB 专用规则 (42+ 条) + """ db_type_lower = db_type.lower() + # 别名映射 + db_type_aliases = { + "postgresql": "postgres", + "pg": "postgres", + "mongo": "mongodb", + } + db_type_lower = db_type_aliases.get(db_type_lower, db_type_lower) + if db_type_lower not in SEED_RULES_MAP: raise HTTPException( status_code=400, diff --git a/backend/app/services/audit_engine_v2.py b/backend/app/services/audit_engine_v2.py index 57d8922..351ac51 100644 --- a/backend/app/services/audit_engine_v2.py +++ b/backend/app/services/audit_engine_v2.py @@ -1,4 +1,4 @@ -"""SQL 审核引擎 V2 — 可配置规则引擎，支持 100+ 审核规则。 +"""SQL 审核引擎 V2 — 可配置规则引擎，支持 200+ 审核规则。 对标 Yearning/Archery/Bytebase 的审核能力，支持: - DDL 规则 (建表规范、索引规范、字段规范) @@ -7,6 +7,10 @@ - 命名规范 (表名/字段名/索引名规范) - 性能规则 (全表扫描、索引使用、大 IN 子句) - 安全规则 (敏感字段、SQL 注入风险、密码硬编码) +- MySQL 专用规则 (62+ 条，覆盖 MySQL 特有语法) +- PostgreSQL 专用规则 (45+ 条，覆盖 PG 特有语法) +- Redis 专用规则 (43+ 条，覆盖 Redis 命令和数据结构) +- MongoDB 专用规则 (42+ 条，覆盖 MongoDB 查询和聚合) 使用方式: engine = SQLAuditEngineV2(db_session) @@ -1681,3 +1685,15 @@ class SQLAuditEngineV2: def invalidate_cache(self) -> None: """清除规则缓存，下次审核重新加载。""" self._rules_cache = None + + +# ── 延迟导入各数据库专用规则（副作用：注册到 _RULE_REGISTRY）──── +# 放在文件末尾以避免循环导入（规则模块需要 import register_rule 等） +# MySQL 规则 (62+ 条) +from app.services.mysql_audit_rules import * # noqa: F401,F403 +# PostgreSQL 规则 (45+ 条) +from app.services.postgres_audit_rules import * # noqa: F401,F403 +# Redis 规则 (43+ 条) +from app.services.redis_audit_rules import * # noqa: F401,F403 +# MongoDB 规则 (42+ 条) +from app.services.mongodb_audit_rules import * # noqa: F401,F403 diff --git a/backend/app/services/audit_rule_seeds.py b/backend/app/services/audit_rule_seeds.py index bb261a2..5687690 100644 --- a/backend/app/services/audit_rule_seeds.py +++ b/backend/app/services/audit_rule_seeds.py @@ -1,4 +1,4 @@ -"""Preset audit rule seed data for MySQL, PostgreSQL, and Redis.""" +"""Preset audit rule seed data for MySQL, PostgreSQL, Redis, and MongoDB.""" from typing import List, Dict, Any @@ -270,6 +270,320 @@ POSTGRES_SEED_RULES: List[Dict[str, Any]] = [ }, ] +# ── MongoDB 预设规则 ──────────────────────────────────────────── +MONGODB_SEED_RULES: List[Dict[str, Any]] = [ + { + "name": "MongoDB集合命名规范(snake_case)", + "description": "MongoDB 集合名应使用小写下划线命名(snake_case)", + "rule_type": "DDL", + "severity": "warning", + "db_type": "mongodb", + "pattern": r"^[a-z][a-z0-9_]*$", + "value": None, + "message": "集合名不符合 snake_case 命名规范，应使用小写下划线格式", + }, + { + "name": "MongoDB集合名最大长度限制", + "description": "MongoDB 集合名最大长度为 64 字节", + "rule_type": "DDL", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": "64", + "message": "集合名长度超过64字节，请缩短集合名", + }, + { + "name": "MongoDB禁止删除集合", + "description": "生产环境禁止执行删除集合操作", + "rule_type": "DDL", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止执行删除集合操作，数据将永久丢失", + }, + { + "name": "MongoDB禁止删除数据库", + "description": "禁止执行 dropDatabase 操作", + "rule_type": "DDL", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止执行 dropDatabase 操作，将删除整个数据库", + }, + { + "name": "MongoDB禁止删除索引", + "description": "禁止执行删除索引操作", + "rule_type": "DDL", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止执行删除索引操作，可能导致查询性能急剧下降", + }, + { + "name": "MongoDB推荐Schema Validator", + "description": "创建集合时建议使用 JSON Schema Validator", + "rule_type": "DDL", + "severity": "info", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "建议为集合添加 JSON Schema Validator 以约束文档结构", + }, + { + "name": "MongoDB查询必须带filter", + "description": "find/delete/update 操作必须包含 filter 条件", + "rule_type": "DQL", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "查询/更新/删除操作缺少 filter 条件，将操作全集合数据", + }, + { + "name": "MongoDB推荐指定projection", + "description": "查询推荐指定 projection 限制返回字段", + "rule_type": "DQL", + "severity": "info", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "查询未指定 projection，建议添加 projection 限制返回字段", + }, + { + "name": "MongoDB禁止$where/$function", + "description": "禁止使用 $where 和 $function 执行 JavaScript", + "rule_type": "DQL", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止使用 $where/$function 执行 JavaScript，存在严重安全风险", + }, + { + "name": "MongoDB禁止前缀模糊匹配", + "description": "禁止正则表达式前缀模糊匹配，无法有效使用索引", + "rule_type": "DQL", + "severity": "warning", + "db_type": "mongodb", + "pattern": r'/\^|"\$regex"\s*:\s*"\^', + "value": None, + "message": "前缀模糊匹配无法有效使用索引，建议调整查询策略", + }, + { + "name": "MongoDB查询返回结果上限", + "description": "查询返回结果建议不超过 10000 条", + "rule_type": "DQL", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": "10000", + "message": "查询未设置 limit 或 limit 超过 10000 条，建议分页查询", + }, + { + "name": "MongoDB禁止聚合$out/$merge", + "description": "聚合管道禁止 $out/$merge 写操作", + "rule_type": "DQL", + "severity": "error", + "db_type": "mongodb", + "pattern": r'"\$(out|merge)"', + "value": None, + "message": "禁止在聚合管道中使用 $out/$merge 阶段，可能覆盖现有集合", + }, + { + "name": "MongoDB聚合管道阶段数上限", + "description": "聚合管道阶段数建议不超过 20 个", + "rule_type": "DQL", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": "20", + "message": "聚合管道阶段数超过20个，请优化查询结构", + }, + { + "name": "MongoDB$unwind大数据量警告", + "description": "$unwind 展开数组可能导致文档数量急剧膨胀", + "rule_type": "DQL", + "severity": "warning", + "db_type": "mongodb", + "pattern": r'"\$unwind"', + "value": None, + "message": "$unwind 可能急剧增加文档数量，请确认数组字段元素数量", + }, + { + "name": "MongoDB批量写入上限", + "description": "批量写入建议不超过 1000 条", + "rule_type": "DML", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": "1000", + "message": "批量写入超过1000条，请分批写入", + }, + { + "name": "MongoDBupdateMany必须带条件", + "description": "updateMany 必须包含 filter 条件", + "rule_type": "DML", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "updateMany 操作缺少 filter 条件，将更新全集合文档", + }, + { + "name": "MongoDBdeleteMany必须带条件", + "description": "deleteMany 必须包含 filter 条件", + "rule_type": "DML", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "deleteMany 操作缺少 filter 条件，将删除全集合文档", + }, + { + "name": "MongoDB禁止unset关键字段", + "description": "禁止 $unset 关键字段(_id, create_time等)", + "rule_type": "DML", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止使用 $unset 删除关键字段，可能导致数据完整性问题", + }, + { + "name": "MongoDB禁止重命名集合", + "description": "禁止执行 renameCollection 操作", + "rule_type": "DML", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止重命名集合，可能导致应用连接失败", + }, + { + "name": "MongoDB禁止绕过文档验证", + "description": "禁止 bypassDocumentValidation: true", + "rule_type": "DML", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止设置 bypassDocumentValidation 绕过 Schema 验证", + }, + { + "name": "MongoDB单文档大小限制", + "description": "单文档大小建议不超过 10MB 警告阈值", + "rule_type": "DML", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": "10485760", + "message": "文档大小超过10MB警告阈值，建议拆分或精简文档", + }, + { + "name": "MongoDB禁止嵌套深度超过5层", + "description": "文档嵌套深度建议不超过 5 层", + "rule_type": "DML", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": "5", + "message": "文档嵌套深度超过5层，建议优化文档结构", + }, + { + "name": "MongoDB禁止db.eval()", + "description": "禁止 db.eval() 执行 JavaScript", + "rule_type": "security", + "severity": "error", + "db_type": "mongodb", + "pattern": r'\bdb\.eval\s*\(', + "value": None, + "message": "禁止 db.eval() 执行 JavaScript，存在严重安全风险", + }, + { + "name": "MongoDB禁止操作系统集合", + "description": "禁止操作系统集合", + "rule_type": "security", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止操作系统集合，存在安全风险", + }, + { + "name": "MongoDB禁止MapReduce", + "description": "禁止使用 mapReduce，使用聚合管道替代", + "rule_type": "security", + "severity": "error", + "db_type": "mongodb", + "pattern": r'\bmapReduce\b', + "value": None, + "message": "禁止 mapReduce，请使用聚合管道替代", + }, + { + "name": "MongoDB防止$cmd注入", + "description": "禁止直接操作 $cmd 集合", + "rule_type": "security", + "severity": "error", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "禁止直接操作 $cmd 集合，存在注入攻击风险", + }, + { + "name": "MongoDB敏感操作审计日志", + "description": "删除、更新等敏感操作应记录审计日志", + "rule_type": "security", + "severity": "info", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "敏感操作应确保已记录审计日志，便于安全追踪", + }, + { + "name": "MongoDB缺少索引查询警告", + "description": "查询条件字段应创建索引", + "rule_type": "performance", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "查询条件字段缺少索引可能导致全集合扫描", + }, + { + "name": "MongoDB全集合扫描警告", + "description": "缺少 filter 条件将导致全集合扫描", + "rule_type": "performance", + "severity": "warning", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "查询缺少过滤条件，将导致全集合扫描", + }, + { + "name": "MongoDB复合索引字段顺序建议", + "description": "复合索引字段顺序应遵循 ESR 规则", + "rule_type": "performance", + "severity": "info", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "建议遵循 ESR 规则设计复合索引: 等值 > 排序 > 范围", + }, + { + "name": "MongoDB分片键选择建议", + "description": "分片集合应选择高基数字段作为分片键", + "rule_type": "performance", + "severity": "info", + "db_type": "mongodb", + "pattern": None, + "value": None, + "message": "建议选择高基数均匀分布字段作为分片键", + }, +] + # ── Redis 预设规则 ────────────────────────────────────────────── REDIS_SEED_RULES: List[Dict[str, Any]] = [ { @@ -319,4 +633,5 @@ SEED_RULES_MAP = { "mysql": MYSQL_SEED_RULES, "postgres": POSTGRES_SEED_RULES, "redis": REDIS_SEED_RULES, + "mongodb": MONGODB_SEED_RULES, } diff --git a/backend/app/services/mongodb_audit_rules.py b/backend/app/services/mongodb_audit_rules.py new file mode 100644 index 0000000..ba32bc4 --- /dev/null +++ b/backend/app/services/mongodb_audit_rules.py @@ -0,0 +1,1638 @@ +"""MongoDB 审核规则引擎 — 42+ 条 MongoDB 特有审核规则。 + +支持: +- 集合管理规则 (Collection Management) +- 查询安全规则 (Query Safety) +- 聚合管道规则 (Aggregation Pipeline) +- 写入安全规则 (Write Safety) +- 安全规则 (Security) +- 性能规则 (Performance) + +使用方式: + engine = SQLAuditEngineV2(db_session) + report = await engine.audit( + '{"find": "users", "filter": {"age": {"$gt": 18}}}', + db_type="mongodb" + ) +""" + +from __future__ import annotations + +import json +import re +from typing import Any, Dict, List, Optional, Tuple + +from app.services.audit_engine_v2 import ( + register_rule, + AuditRuleV2, + AuditFindingV2, + AuditReportV2, + Severity, + RuleCategory, +) + + +# ────────────────────────────────────────────── +# JSON / BSON 辅助解析函数 +# ────────────────────────────────────────────── + + +def _try_parse_json(sql_clean: str) -> Tuple[bool, Dict[str, Any]]: + """尝试解析 MongoDB 查询为 JSON 对象。 + + Returns: + (success, parsed_dict) — 解析成功返回 (True, dict)，失败返回 (False, {}) + """ + # 去除首尾空白 + text = sql_clean.strip() + if not text: + return False, {} + # 如果文本被单引号包裹，去除外层引号 + if text.startswith("'") and text.endswith("'"): + text = text[1:-1] + if text.startswith('"') and text.endswith('"') and len(text) > 1: + inner = text[1:-1] + # 检查内部是否是有效的 JSON（避免普通字符串被误剥） + if inner.strip().startswith("{"): + text = inner + # 去除末尾可能存在的分号 + text = text.rstrip(";").strip() + if not text: + return False, {} + try: + parsed = json.loads(text) + if isinstance(parsed, dict): + return True, parsed + return False, {} + except (json.JSONDecodeError, ValueError): + return False, {} + + +def _is_json_mode(sql_clean: str) -> bool: + """检查是否为 JSON 格式的 MongoDB 查询。 + + MongoDB 查询通常是 JSON 格式（以 { 开头）， + 而 shell 命令如 db.collection.find() 以 db. 或 javascript 开头。 + """ + text = sql_clean.strip() + if not text: + return False + # 标准的 JSON 查询格式 {"find": "collection", ...} + if text.startswith("{"): + return True + # 或者 shell 命令中带有 { } 参数 + if "{" in text and "}" in text: + return True + return False + + +def _is_shell_command(sql_clean: str) -> bool: + """检查是否为 MongoDB shell 命令（如 db.collection.find()）。""" + text = sql_clean.strip() + return text.startswith("db.") or text.startswith("sh.") + + +def _extract_collection_name(sql_clean: str) -> Optional[str]: + """从 MongoDB 查询中提取集合名称。 + + 支持格式: + - JSON: {"find": "collection_name", ...} + - Shell: db.collection_name.find(...) + - Shell: db.getCollection("name").find(...) + """ + # 尝试 JSON 格式 + success, parsed = _try_parse_json(sql_clean) + if success: + # {"find": "users", ...} + if "find" in parsed and isinstance(parsed["find"], str): + return parsed["find"] + # {"insert": "users", ...} + for op in ("insert", "update", "delete", "aggregate", "count", "distinct", + "findOne", "insertOne", "updateOne", "deleteOne", + "insertMany", "updateMany", "deleteMany"): + if op in parsed and isinstance(parsed[op], str): + return parsed[op] + return None + + # Shell 格式: db.collection_name.operation() + text = sql_clean.strip() + # db.getCollection("name").method(...) + gc_match = re.search(r'\bgetCollection\s*\(\s*["\'](\w+)["\']\s*\)', text) + if gc_match: + return gc_match.group(1) + # db.name.method(...) + shell_match = re.search(r'\bdb\.(\w+)\.', text) + if shell_match: + return shell_match.group(1) + return None + + +def _has_where_operator(sql_clean: str) -> bool: + """检查是否使用了 $where 运算符（注入风险）。""" + return "\"$where\"" in sql_clean or "'$where'" in sql_clean + + +def _has_function_operator(sql_clean: str) -> bool: + """检查是否使用了 $function 运算符。""" + return "\"$function\"" in sql_clean or "'$function'" in sql_clean + + +def _count_json_keys(obj: Any) -> int: + """递归计算 JSON 对象中的键总数（估算文档大小）。""" + if isinstance(obj, dict): + return sum(1 + _count_json_keys(v) for k, v in obj.items()) + elif isinstance(obj, list): + return sum(_count_json_keys(item) for item in obj) + return 1 + + +def _get_nesting_depth(obj: Any, current_depth: int = 0) -> int: + """计算 JSON 对象的最大嵌套深度。""" + if isinstance(obj, dict): + if not obj: + return current_depth + return max(_get_nesting_depth(v, current_depth + 1) for v in obj.values()) + elif isinstance(obj, list): + if not obj: + return current_depth + return max(_get_nesting_depth(item, current_depth + 1) for item in obj) + return current_depth + + +def _find_all_values(obj: Any, key: str) -> List[Any]: + """递归查找 JSON 对象中所有指定 key 的值。""" + results: List[Any] = [] + if isinstance(obj, dict): + for k, v in obj.items(): + if k == key: + results.append(v) + results.extend(_find_all_values(v, key)) + elif isinstance(obj, list): + for item in obj: + results.extend(_find_all_values(item, key)) + return results + + +def _has_key_recursive(obj: Any, key: str) -> bool: + """递归检查 JSON 对象是否包含指定 key。""" + if isinstance(obj, dict): + if key in obj: + return True + return any(_has_key_recursive(v, key) for v in obj.values()) + elif isinstance(obj, list): + return any(_has_key_recursive(item, key) for item in obj) + return False + + +def _get_json_string_length(sql_clean: str) -> int: + """估算 JSON 字符串的字节长度（用于文档大小估算）。""" + return len(sql_clean.encode("utf-8")) + + +def _count_pipeline_stages(sql_clean: str) -> int: + """计算聚合管道中的阶段数量。""" + success, parsed = _try_parse_json(sql_clean) + if success and "pipeline" in parsed and isinstance(parsed["pipeline"], list): + return len(parsed["pipeline"]) + # shell 格式中粗略统计 $ 开头的 stage + stages = re.findall(r'\{\\s*"\\$\w+"', sql_clean) + return len(stages) + + +def _is_snake_case(name: str) -> bool: + """检查是否为 snake_case 命名。""" + if not name: + return False + return bool(re.match(r"^[a-z][a-z0-9_]*$", name)) + + +# ═══════════════════════════════════════════════ +# 集合管理规则 (8条) — MONGO-COL-001 ~ 008 +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="MONGO-COL-001", + name="集合命名规范(snake_case)", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mongodb", + description="MongoDB 集合名应使用小写下划线命名(snake_case)", + config={"pattern": "^[a-z][a-z0-9_]*$"}, + message_template="集合名 '{name}' 不符合 snake_case 命名规范，应使用小写下划线格式", +)) +def check_mongo_col_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """集合命名规范: snake_case。""" + if not _is_json_mode(sql_clean): + return + collection_name = _extract_collection_name(sql_clean) + if not collection_name: + return + if not _is_snake_case(collection_name): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(name=collection_name), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-COL-002", + name="集合名最大长度限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mongodb", + description="MongoDB 集合名最大长度为 64 字节(不含前缀)", + config={"max_name_length": 64}, + message_template="集合名 '{name}' 长度为 {length} 字节，超过最大限制 {max} 字节", +)) +def check_mongo_col_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """集合名最大长度检查: 64 字节。""" + if not _is_json_mode(sql_clean): + return + collection_name = _extract_collection_name(sql_clean) + if not collection_name: + return + max_len = rule.config.get("max_name_length", 64) + name_bytes = len(collection_name.encode("utf-8")) + if name_bytes > max_len: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(name=collection_name, length=name_bytes, max=max_len), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-COL-003", + name="禁止删除集合(DROP)", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mongodb", + description="生产环境禁止执行删除集合操作，数据将永久丢失", + config={"dangerous_ops": ["drop"]}, + message_template="禁止执行删除集合操作: {op}，数据将永久丢失", +)) +def check_mongo_col_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止删除集合。""" + if not _is_json_mode(sql_clean): + return + success, parsed = _try_parse_json(sql_clean) + if not success: + # 检查 shell 命令 + if re.search(r'\bdb\.\w+\.drop\s*\(', sql_clean): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op="db.collection.drop()"), + )) + return + dangerous = rule.config.get("dangerous_ops", ["drop"]) + for op in dangerous: + if op in parsed: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op=op), + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-COL-004", + name="禁止删除数据库(DROP DATABASE)", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止执行 dropDatabase 操作，将删除整个数据库", + message_template="禁止执行 dropDatabase 操作，将删除整个数据库及其所有集合", +)) +def check_mongo_col_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止删除数据库。""" + text = sql_clean.strip() + # 检查 JSON 格式 {"dropDatabase": 1} + success, parsed = _try_parse_json(text) + if success and "dropDatabase" in parsed: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + # 检查 shell 格式 + if re.search(r'\bdb\.dropDatabase\s*\(', text): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-COL-005", + name="禁止删除索引(DROP INDEX)", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止执行删除索引操作，删除后查询性能可能急剧下降", + message_template="禁止执行删除索引操作: {op}，删除索引可能导致查询性能急剧下降", +)) +def check_mongo_col_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止删除索引。""" + text = sql_clean.strip() + # JSON 格式 {"dropIndexes": "collection", "index": "name"} + success, parsed = _try_parse_json(text) + if success and "dropIndexes" in parsed: + idx_name = parsed.get("index", "*") + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op=f'dropIndexes: {idx_name}'), + )) + return + # shell 格式: db.collection.dropIndex() / dropIndexes() + if re.search(r'\b(dropIndex|dropIndexes)\s*\(', text): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op="dropIndex(s)"), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-COL-006", + name="集合文档数上限建议", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mongodb", + description="单个集合文档数建议不超过 10 亿，超过应考虑分片", + config={"max_docs": 1_000_000_000}, + message_template="集合文档数超过 {max} 时建议进行分片(sharding)或归档", +)) +def check_mongo_col_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """集合文档数上限建议。通过 count 操作进行提示。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if success and ("count" in parsed or "estimatedDocumentCount" in parsed): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(max=rule.config["max_docs"]), + )) + return + if re.search(r'\b(countDocuments|estimatedDocumentCount)\s*\(', text): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(max=rule.config["max_docs"]), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-COL-007", + name="推荐使用 Schema Validator", + category=RuleCategory.DDL, + severity=Severity.INFO, + db_type="mongodb", + description="建议在创建集合时使用 JSON Schema Validator 进行数据校验", + message_template="建议为集合添加 JSON Schema Validator 以约束文档结构", +)) +def check_mongo_col_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐使用 Schema Validator: 检查 create/createCollection 是否包含 validator。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + # shell 格式 + if re.search(r'\b(createCollection|create)\s*\(', text): + if "validator" not in text.lower() and "\"validator\"" not in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + # JSON 格式: {"create": "coll", ...} + if "create" in parsed or "createCollection" in parsed: + if "validator" not in parsed: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-COL-008", + name="Capped Collection 大小限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mongodb", + description="Capped Collection 应设置合理的大小限制(默认不超过 1GB)", + config={"max_capped_size": 1_073_741_824}, # 1GB + message_template="Capped Collection 大小(size)为 {size} 字节，建议不超过 {max} 字节(1GB)", +)) +def check_mongo_col_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Capped Collection 大小限制检查。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + create_op = parsed.get("create") or parsed.get("createCollection") + if not create_op: + return + capped = parsed.get("capped", False) + size = parsed.get("size") + max_size = rule.config.get("max_capped_size", 1_073_741_824) + if capped and size and isinstance(size, (int, float)) and size > max_size: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(size=size, max=max_size), + )) + + +# ═══════════════════════════════════════════════ +# 查询安全规则 (10条) — MONGO-QRY-001 ~ 010 +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="MONGO-QRY-001", + name="find/delete/update 必须带 filter", + category=RuleCategory.DQL, + severity=Severity.ERROR, + db_type="mongodb", + description="MongoDB 查询/更新/删除操作必须包含 filter 条件，禁止无条件操作", + message_template="{op} 操作缺少 filter/条件，将操作全集合数据，必须指定查询条件", +)) +def check_mongo_qry_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """find/delete/update 必须带 filter。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + # 检查各类操作 + ops_with_filter = { + "find": "filter", + "findOne": "filter", + "update": "updates", + "updateOne": "filter", + "updateMany": "filter", + "delete": "deletes", + "deleteOne": "filter", + "deleteMany": "filter", + } + for op, filter_key in ops_with_filter.items(): + if op in parsed: + # 检查 filter 是否存在且非空 + if filter_key == "filter": + filter_val = parsed.get(filter_key) + if filter_val is None or (isinstance(filter_val, dict) and not filter_val): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op=op), + )) + return + elif filter_key == "updates": + updates = parsed.get("updates", []) + if isinstance(updates, list) and updates: + for upd in updates: + if isinstance(upd, dict) and not upd.get("q"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op=op), + )) + return + elif filter_key == "deletes": + deletes = parsed.get("deletes", []) + if isinstance(deletes, list) and deletes: + for d in deletes: + if isinstance(d, dict) and not d.get("q"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op=op), + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-002", + name="推荐指定 projection", + category=RuleCategory.DQL, + severity=Severity.INFO, + db_type="mongodb", + description="find/findOne 查询推荐指定 projection 限制返回字段，减少网络传输", + message_template="{op} 查询未指定 projection，建议添加 projection 限制返回字段", +)) +def check_mongo_qry_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐指定 projection。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + query_ops = ("find", "findOne") + for op in query_ops: + if op in parsed: + if "projection" not in parsed: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op=op), + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-003", + name="禁止 $where/$function JavaScript 执行", + category=RuleCategory.DQL, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止使用 $where 和 $function 运算符执行 JavaScript，存在注入和安全风险", + message_template="禁止使用 {op} 运算符执行 JavaScript，存在严重的安全和注入风险", +)) +def check_mongo_qry_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 $where/$function JavaScript 执行。""" + text = sql_clean.strip() + if "\"$where\"" in text or "'$where'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op="$where"), + )) + return + if "\"$function\"" in text or "'$function'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op="$function"), + )) + return + # 检查函数表达式中的 $where + if _has_where_operator(text): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(op="$where"), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-004", + name="禁止前缀模糊匹配(regex)", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="禁止正则表达式前缀模糊匹配(如 /^abc/)，无法有效使用索引", + message_template="正则表达式查询 {pattern} 为前缀匹配，无法有效使用索引，建议调整查询策略", +)) +def check_mongo_qry_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止前缀模糊匹配: 检测 /^ 开头的正则或 $regex 中以 ^ 开头的模式。""" + text = sql_clean.strip() + # 检测 $regex 中以 ^ 开头的模式 + regex_patterns = re.findall(r'"\$regex"\s*:\s*"(\^[^"]*)"', text) + for pat in regex_patterns: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(pattern=pat), + )) + return + # 检测 JS 正则字面量 /^.../ + js_regex = re.findall(r'/\^(.*?)/', text) + for pat in js_regex: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(pattern=f"^{pat}"), + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-005", + name="查询返回结果上限", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="查询返回结果建议不超过 10000 条，大数据量应分页", + config={"max_limit": 10000}, + message_template="查询未设置 limit 或 limit 超过 {max} 条，建议分页查询", +)) +def check_mongo_qry_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """查询返回结果上限检查。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "find" not in parsed and "findOne" not in parsed and "aggregate" not in parsed: + return + limit = parsed.get("limit") + max_limit = rule.config.get("max_limit", 10000) + if limit is None: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(max=max_limit), + )) + elif isinstance(limit, (int, float)) and limit > max_limit: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(max=max_limit), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-006", + name="explain 不应在生产环境频繁使用", + category=RuleCategory.DQL, + severity=Severity.INFO, + db_type="mongodb", + description="explain() 会额外收集执行计划信息，不应在生产环境频繁使用", + message_template="explain() 操作在生产环境频繁使用可能影响性能，建议仅在调试时使用", +)) +def check_mongo_qry_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """explain() 不应在生产环境频繁使用。""" + text = sql_clean.strip() + if "explain" in text.lower() or "\"explain\"" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-007", + name="不推荐 snapshot 模式", + category=RuleCategory.DQL, + severity=Severity.INFO, + db_type="mongodb", + description="snapshot 模式($snapshot 或 readConcern snapshot)性能较差", + message_template="使用 snapshot 模式可能影响查询性能，如无强一致性需求请避免使用", +)) +def check_mongo_qry_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """不推荐 snapshot 模式。""" + text = sql_clean.strip() + has_snapshot = False + # JSON 格式 + if "\"snapshot\"" in text and "true" in text.lower(): + has_snapshot = True + if "snapshot" in text.lower(): + success, parsed = _try_parse_json(text) + if success: + snapshot_val = parsed.get("snapshot") + read_concern = parsed.get("readConcern", {}) + if snapshot_val is True: + has_snapshot = True + elif isinstance(read_concern, dict) and read_concern.get("level") == "snapshot": + has_snapshot = True + if has_snapshot: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-008", + name="不建议使用 allowDiskUse", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="allowDiskUse: true 允许聚合操作使用磁盘，可能表明查询设计不合理", + message_template="使用 allowDiskUse: true 表明查询可能消耗大量内存，建议优化查询或索引", +)) +def check_mongo_qry_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """不建议使用 allowDiskUse。""" + text = sql_clean.strip() + if "allowDiskUse" not in text: + return + success, parsed = _try_parse_json(text) + if success: + allow_disk = parsed.get("allowDiskUse") + if allow_disk is True: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + # 正则匹配 + if re.search(r'"allowDiskUse"\s*:\s*true', text, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-009", + name="text 搜索使用限制", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="MongoDB 的 $text 搜索功能有限，建议使用 Elasticsearch 等专业搜索引擎", + message_template="使用 $text 全文搜索功能有限，建议考虑 Elasticsearch 等专业搜索方案", +)) +def check_mongo_qry_009(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """text 搜索使用限制。""" + text = sql_clean.strip() + if "\"$text\"" in text or "'$text'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-QRY-010", + name="filter 必须是有效的 BSON 格式", + category=RuleCategory.DQL, + severity=Severity.ERROR, + db_type="mongodb", + description="MongoDB filter 必须是有效的 JSON/BSON 对象格式", + message_template="查询条件 filter 不是有效的 JSON/BSON 对象格式，请修正", +)) +def check_mongo_qry_010(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """filter 必须是有效的 BSON 格式。""" + text = sql_clean.strip() + if not text.startswith("{"): + return + # 尝试解析 JSON + success, parsed = _try_parse_json(text) + if not success: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + # 检查 filter 字段是否为对象 + for filter_key in ("filter", "q"): + if filter_key in parsed: + filter_val = parsed[filter_key] + if filter_val is not None and not isinstance(filter_val, dict): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + + +# ═══════════════════════════════════════════════ +# 聚合管道规则 (6条) — MONGO-AGG-001 ~ 006 +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="MONGO-AGG-001", + name="禁止 $out/$merge 写操作", + category=RuleCategory.DQL, + severity=Severity.ERROR, + db_type="mongodb", + description="聚合管道中禁止使用 $out 和 $merge 阶段，可能覆盖现有集合数据", + message_template="禁止在聚合管道中使用 {stage} 阶段，可能覆盖或修改现有集合数据", +)) +def check_mongo_agg_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 $out/$merge 写操作。""" + text = sql_clean.strip() + # 检查 JSON 中的 pipeline + success, parsed = _try_parse_json(text) + if success and "pipeline" in parsed and isinstance(parsed["pipeline"], list): + for stage in parsed["pipeline"]: + if isinstance(stage, dict): + if "$out" in stage: + out_val = stage["$out"] + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stage=f"$out: {out_val}"), + )) + return + if "$merge" in stage: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stage="$merge"), + )) + return + # 文本检查 + if "\"$out\"" in text or "'$out'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stage="$out"), + )) + return + if "\"$merge\"" in text or "'$merge'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stage="$merge"), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-AGG-002", + name="聚合管道阶段数上限", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="聚合管道阶段数建议不超过 20 个，过多阶段影响性能", + config={"max_stages": 20}, + message_template="聚合管道包含 {count} 个阶段，超过建议上限 {max} 个", +)) +def check_mongo_agg_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """聚合管道阶段数上限检查。""" + text = sql_clean.strip() + stage_count = 0 + success, parsed = _try_parse_json(text) + if success and "pipeline" in parsed and isinstance(parsed["pipeline"], list): + stage_count = len(parsed["pipeline"]) + else: + # 粗略统计 $ 开头的 stage 数量 + stage_count = len(re.findall(r'\{\\s*"\$(\w+)"', text)) + max_stages = rule.config.get("max_stages", 20) + if stage_count > max_stages: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=stage_count, max=max_stages), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-AGG-003", + name="$unwind 大数据量警告", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="$unwind 展开数组可能导致文档数量急剧膨胀，影响性能", + message_template="使用 $unwind 阶段可能急剧增加文档数量，请确认数组字段的平均元素数量", +)) +def check_mongo_agg_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """$unwind 大数据量警告。""" + text = sql_clean.strip() + if "\"$unwind\"" in text or "'$unwind'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-AGG-004", + name="$group 操作索引建议", + category=RuleCategory.DQL, + severity=Severity.INFO, + db_type="mongodb", + description="$group 操作的 _id 字段建议创建索引以提升性能", + message_template="$group 操作的 _id 字段建议创建索引，可显著提升聚合性能", +)) +def check_mongo_agg_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """$group 操作索引建议。""" + text = sql_clean.strip() + if "\"$group\"" in text or "'$group'" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-AGG-005", + name="聚合内存限制(100MB)", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="单个聚合阶段内存限制为 100MB，超出需使用 allowDiskUse", + config={"memory_limit_mb": 100}, + message_template="聚合操作内存限制为 {limit}MB，大型聚合可能触发内存限制错误", +)) +def check_mongo_agg_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """聚合内存限制提示。""" + text = sql_clean.strip() + if "aggregate" in text.lower() or "\"pipeline\"" in text: + limit_mb = rule.config.get("memory_limit_mb", 100) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(limit=limit_mb), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-AGG-006", + name="$lookup 嵌套层数限制", + category=RuleCategory.DQL, + severity=Severity.WARNING, + db_type="mongodb", + description="$lookup 嵌套层数建议不超过 3 层，过多嵌套影响性能", + config={"max_lookup_nesting": 3}, + message_template="$lookup 嵌套层数可能超过 {max} 层，建议优化查询结构", +)) +def check_mongo_agg_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """$lookup 嵌套层数限制。""" + text = sql_clean.strip() + lookup_count = text.count('"$lookup"') + text.count("'$lookup'") + max_nesting = rule.config.get("max_lookup_nesting", 3) + if lookup_count > max_nesting: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(max=max_nesting), + )) + + +# ═══════════════════════════════════════════════ +# 写入安全规则 (8条) — MONGO-WR-001 ~ 008 +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="MONGO-WR-001", + name="批量写入上限(1000条)", + category=RuleCategory.DML, + severity=Severity.WARNING, + db_type="mongodb", + description="insertMany/bulkWrite 单次文档数建议不超过 1000 条", + config={"max_bulk_write": 1000}, + message_template="批量写入文档数为 {count}，超过建议上限 {max} 条，请分批写入", +)) +def check_mongo_wr_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """批量写入上限检查。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + doc_count = 0 + if success: + # insertMany: {"documents": [...]} + if "insertMany" in parsed or "insert" in parsed: + docs = parsed.get("documents", []) + if isinstance(docs, list): + doc_count = len(docs) + # bulkWrite: {"operations": [...]} + if "operations" in parsed and isinstance(parsed["operations"], list): + doc_count = len(parsed["operations"]) + # updateMany: {"updates": [...]} + if "updates" in parsed and isinstance(parsed["updates"], list): + doc_count = len(parsed["updates"]) + max_docs = rule.config.get("max_bulk_write", 1000) + if doc_count > max_docs: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=doc_count, max=max_docs), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-WR-002", + name="updateMany 必须带条件 filter", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mongodb", + description="updateMany 操作必须包含 filter 条件，禁止无条件更新", + message_template="updateMany 操作缺少有效的 filter 条件，将更新全集合文档", +)) +def check_mongo_wr_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """updateMany 必须带条件 filter。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "updateMany" in parsed: + filter_val = parsed.get("filter") + if filter_val is None or (isinstance(filter_val, dict) and not filter_val): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + # 检查 bulk updates + if "updates" in parsed and isinstance(parsed.get("updates"), list): + for upd in parsed["updates"]: + if isinstance(upd, dict): + q = upd.get("q") + u = upd.get("u") + multi = upd.get("multi", False) + if multi and (q is None or (isinstance(q, dict) and not q)): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-WR-003", + name="deleteMany 必须带条件 filter", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mongodb", + description="deleteMany 操作必须包含 filter 条件，禁止无条件删除", + message_template="deleteMany 操作缺少有效的 filter 条件，将删除全集合文档", +)) +def check_mongo_wr_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """deleteMany 必须带条件 filter。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "deleteMany" in parsed: + filter_val = parsed.get("filter") + if filter_val is None or (isinstance(filter_val, dict) and not filter_val): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + # 检查 bulk deletes + if "deletes" in parsed and isinstance(parsed.get("deletes"), list): + for d in parsed["deletes"]: + if isinstance(d, dict): + q = d.get("q") + limit = d.get("limit", 0) + if limit == 0 and (q is None or (isinstance(q, dict) and not q)): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-WR-004", + name="禁止 unset 关键字段", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止 $unset 关键字段(如 _id, create_time, update_time 等)", + config={"critical_fields": ["_id", "create_time", "update_time", "created_at", "updated_at"]}, + message_template="禁止使用 $unset 删除关键字段: {fields}，可能导致数据完整性问题", +)) +def check_mongo_wr_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 unset 关键字段。""" + text = sql_clean.strip() + critical = rule.config.get("critical_fields", ["_id"]) + if "\"$unset\"" not in text and "'$unset'" not in text: + return + success, parsed = _try_parse_json(text) + found_fields: List[str] = [] + if success: + # 查找 updates 中的 $unset + for update_dict in _find_all_values(parsed, "u"): + if isinstance(update_dict, dict) and "$unset" in update_dict: + unset_fields = update_dict["$unset"] + if isinstance(unset_fields, dict): + for f in unset_fields: + if f in critical: + found_fields.append(f) + elif isinstance(unset_fields, list): + for f in unset_fields: + if f in critical: + found_fields.append(f) + # 也检查顶层 update 的 $unset + if isinstance(parsed, dict): + for k, v in parsed.items(): + if isinstance(v, dict) and "$unset" in v: + unset_val = v["$unset"] + if isinstance(unset_val, dict): + for f in unset_val: + if f in critical: + found_fields.append(f) + else: + # 正则提取 + unset_match = re.findall(r'"\$unset"\s*:\s*\{([^}]+)\}', text) + for m in unset_match: + for f in critical: + if f'"{f}"' in m or f"'{f}'" in m: + found_fields.append(f) + if found_fields: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(fields=", ".join(set(found_fields))), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-WR-005", + name="禁止重命名集合", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止执行 renameCollection 操作，可能影响应用连接", + message_template="禁止执行 renameCollection 操作，重命名集合可能导致应用连接失败", +)) +def check_mongo_wr_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止重命名集合。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if success: + if "renameCollection" in parsed or "collMod" in parsed: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + if re.search(r'\b(renameCollection|collMod)\b', text, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-WR-006", + name="禁止绕过文档验证(bypassDocumentValidation)", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止设置 bypassDocumentValidation: true 绕过文档验证", + message_template="禁止设置 bypassDocumentValidation: true 绕过 Schema 验证", +)) +def check_mongo_wr_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止绕过文档验证。""" + text = sql_clean.strip() + if "bypassDocumentValidation" not in text: + return + success, parsed = _try_parse_json(text) + if success: + bypass = parsed.get("bypassDocumentValidation") + if bypass is True: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + # 正则匹配 + if re.search(r'"bypassDocumentValidation"\s*:\s*true', text, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-WR-007", + name="单文档大小限制(16MB警告阈值10MB)", + category=RuleCategory.DML, + severity=Severity.WARNING, + db_type="mongodb", + description="MongoDB 单文档最大 16MB，建议不超过 10MB 作为警告阈值", + config={"warn_threshold_bytes": 10_485_760, "max_size_bytes": 16_777_216}, # 10MB, 16MB + message_template="文档大小约为 {size_mb}MB，超过警告阈值 {warn_mb}MB，建议拆分或精简文档", +)) +def check_mongo_wr_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """单文档大小限制检查。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + # 检查是否为写入操作 + write_ops = ("insert", "insertOne", "insertMany", "update", "updateOne", "updateMany") + is_write = any(op in parsed for op in write_ops) + if not is_write: + return + # 估算文档大小 + doc_size = _get_json_string_length(text) + warn_threshold = rule.config.get("warn_threshold_bytes", 10_485_760) + if doc_size > warn_threshold: + size_mb = round(doc_size / (1024 * 1024), 2) + warn_mb = round(warn_threshold / (1024 * 1024), 2) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(size_mb=size_mb, warn_mb=warn_mb), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-WR-008", + name="禁止嵌套深度超过5层", + category=RuleCategory.DML, + severity=Severity.WARNING, + db_type="mongodb", + description="文档嵌套深度建议不超过 5 层，过深影响查询和更新性能", + config={"max_nesting_depth": 5}, + message_template="文档嵌套深度为 {depth} 层，超过建议上限 {max} 层", +)) +def check_mongo_wr_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止嵌套深度超过5层。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + # 检查 documents 或 document 字段的嵌套深度 + docs = parsed.get("documents") or parsed.get("document") + if docs is None: + # 检查 updates 中的 $set/$setOnInsert 等 + updates = parsed.get("updates", []) + if isinstance(updates, list) and updates: + docs = updates[0].get("u") if isinstance(updates[0], dict) else None + if not docs: + return + target_doc = docs[0] if isinstance(docs, list) else docs + if not isinstance(target_doc, dict): + return + depth = _get_nesting_depth(target_doc) + max_depth = rule.config.get("max_nesting_depth", 5) + if depth > max_depth: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(depth=depth, max=max_depth), + )) + + +# ═══════════════════════════════════════════════ +# 安全规则 (5条) — MONGO-SEC-001 ~ 005 +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="MONGO-SEC-001", + name="禁止 db.eval() 执行 JavaScript", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止 db.eval() 执行任意 JavaScript 代码，存在严重安全风险", + message_template="禁止 db.eval() 执行 JavaScript 代码，存在严重的安全和注入风险", +)) +def check_mongo_sec_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 db.eval() 执行 JavaScript。""" + text = sql_clean.strip() + if re.search(r'\bdb\.eval\s*\(', text): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-SEC-002", + name="禁止操作系统集合", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止查询/操作系统集合(以 system. 开头)，可能导致安全问题", + message_template="禁止操作系统集合: '{coll}'，系统集合操作可能导致安全问题", +)) +def check_mongo_sec_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止操作系统集合。""" + text = sql_clean.strip() + coll = _extract_collection_name(text) + if coll and coll.startswith("system."): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(coll=coll), + )) + return + # 也检查 JSON 中的 find 等操作 + success, parsed = _try_parse_json(text) + if success: + for op_key in ("find", "findOne", "aggregate", "insert", "update", "delete", "count"): + val = parsed.get(op_key) + if isinstance(val, str) and val.startswith("system."): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(coll=val), + )) + return + + +@register_rule(AuditRuleV2( + code="MONGO-SEC-003", + name="禁止 MapReduce(使用聚合管道)", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止使用 mapReduce 命令，map 和 reduce 函数可执行任意 JavaScript", + message_template="禁止 mapReduce 操作，请使用聚合管道(Aggregation Pipeline)替代", +)) +def check_mongo_sec_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 MapReduce。""" + text = sql_clean.strip() + if re.search(r'\bmapReduce\b|\bmapReduce\s*\(', text) or "\"mapReduce\"" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-SEC-004", + name="防止 $cmd 注入攻击", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mongodb", + description="禁止直接操作 $cmd 集合，可能存在注入攻击风险", + message_template="禁止直接操作 $cmd 集合，存在注入攻击风险", +)) +def check_mongo_sec_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """防止 $cmd 注入攻击。""" + text = sql_clean.strip() + if "$cmd" in text: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-SEC-005", + name="敏感操作必须记录审计日志", + category=RuleCategory.SECURITY, + severity=Severity.INFO, + db_type="mongodb", + description="删除、更新、聚合写操作等敏感操作应记录审计日志", + message_template="敏感操作({ops})应确保已记录审计日志，便于安全追踪和合规审计", +)) +def check_mongo_sec_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """敏感操作审计日志提醒。""" + text = sql_clean.strip() + sensitive_ops = [] + success, parsed = _try_parse_json(text) + op_names = { + "delete": "delete", "deleteOne": "delete", "deleteMany": "delete", + "update": "update", "updateOne": "update", "updateMany": "update", + "drop": "drop", "dropDatabase": "dropDatabase", "dropIndexes": "dropIndexes", + "renameCollection": "rename", "createUser": "userManagement", + "dropUser": "userManagement", "grantRoles": "userManagement", + } + if success: + for op, label in op_names.items(): + if op in parsed: + sensitive_ops.append(label) + else: + text_lower = text.lower() + for op, label in op_names.items(): + if op.lower() in text_lower: + sensitive_ops.append(label) + if sensitive_ops: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(ops=", ".join(set(sensitive_ops))), + )) + + +# ═══════════════════════════════════════════════ +# 性能规则 (5条) — MONGO-PERF-001 ~ 005 +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="MONGO-PERF-001", + name="缺少索引查询警告", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="mongodb", + description="查询条件字段应创建索引，缺少索引会导致全集合扫描(COLLSCAN)", + message_template="查询条件字段 '{fields}' 缺少索引可能导致全集合扫描，建议创建索引", +)) +def check_mongo_perf_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """缺少索引查询警告: 检查查询条件中常见字段是否可能缺少索引。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "find" not in parsed and "findOne" not in parsed: + return + filter_val = parsed.get("filter") + if not isinstance(filter_val, dict) or not filter_val: + return + # 检查常见的可能需要索引的字段（排除 _id 已有默认索引） + query_fields = [k for k in filter_val.keys() if not k.startswith("$") and k != "_id"] + if query_fields: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(fields=", ".join(query_fields)), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-PERF-002", + name="推荐覆盖索引查询", + category=RuleCategory.PERFORMANCE, + severity=Severity.INFO, + db_type="mongodb", + description="推荐 projection 仅包含索引字段，实现覆盖索引查询(Covered Query)", + message_template="查询条件 '{fields}' 建议 projection 仅包含索引字段以实现覆盖索引查询", +)) +def check_mongo_perf_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐覆盖索引查询。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "find" not in parsed and "findOne" not in parsed: + return + filter_val = parsed.get("filter") + projection = parsed.get("projection") + if not isinstance(filter_val, dict) or not filter_val: + return + # 如果有 projection 且设置了 _id: 0，可能是尝试覆盖索引 + if isinstance(projection, dict) and projection.get("_id") == 0: + query_fields = [k for k in filter_val.keys() if not k.startswith("$")] + if query_fields: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(fields=", ".join(query_fields)), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-PERF-003", + name="全集合扫描警告", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="mongodb", + description="缺少 filter 条件或 filter 为空对象 {} 将导致全集合扫描", + message_template="查询缺少有效过滤条件，将导致全集合扫描(COLLSCAN)，严重影响性能", +)) +def check_mongo_perf_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """全集合扫描警告。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "find" not in parsed and "findOne" not in parsed: + return + filter_val = parsed.get("filter") + if filter_val is None: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + elif isinstance(filter_val, dict) and not filter_val: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="MONGO-PERF-004", + name="复合索引字段顺序建议", + category=RuleCategory.PERFORMANCE, + severity=RuleCategory.PERFORMANCE, + db_type="mongodb", + description="复合索引字段顺序应遵循 ESR 规则(Equality, Sort, Range)", + message_template="复合查询条件 '{fields}' 建议遵循 ESR 规则设计索引: 等值 > 排序 > 范围", +)) +def check_mongo_perf_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """复合索引字段顺序建议: ESR 规则。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if not success: + return + if "find" not in parsed and "findOne" not in parsed: + return + filter_val = parsed.get("filter") + if not isinstance(filter_val, dict) or not filter_val: + return + # 统计查询条件中的字段数量 + query_fields = [k for k in filter_val.keys() if not k.startswith("$")] + # 如果有多于 1 个查询字段，提示复合索引 ESR 规则 + if len(query_fields) > 1: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=rule.message_template.format(fields=", ".join(query_fields)), + )) + + +@register_rule(AuditRuleV2( + code="MONGO-PERF-005", + name="分片键选择建议", + category=RuleCategory.PERFORMANCE, + severity=Severity.INFO, + db_type="mongodb", + description="分片集合应选择高基数、均匀分布的字段作为分片键", + message_template="分片集合建议: 选择高基数均匀分布字段作为分片键，避免单调递增字段和热点", +)) +def check_mongo_perf_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """分片键选择建议: 检查 shardCollection 操作。""" + text = sql_clean.strip() + success, parsed = _try_parse_json(text) + if success and "shardCollection" in parsed: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + if re.search(r'\bshardCollection\b', text): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ────────────────────────────────────────────── +# 规则统计信息 +# ────────────────────────────────────────────── + + +def get_mongodb_rule_count() -> Dict[str, int]: + """获取 MongoDB 规则分类统计。 + + Returns: + {"COL": 8, "QRY": 10, "AGG": 6, "WR": 8, "SEC": 5, "PERF": 5, "TOTAL": 42} + """ + return { + "COL": 8, + "QRY": 10, + "AGG": 6, + "WR": 8, + "SEC": 5, + "PERF": 5, + "TOTAL": 42, + } diff --git a/backend/app/services/mysql_audit_rules.py b/backend/app/services/mysql_audit_rules.py new file mode 100644 index 0000000..593765c --- /dev/null +++ b/backend/app/services/mysql_audit_rules.py @@ -0,0 +1,2101 @@ +"""MySQL 审核规则引擎 — 63条 MySQL 专属审核规则。 + +本模块包含 MySQL 数据库的完整审核规则集，涵盖: +- DDL 规则 (36条): 建表规范、索引规范、字段规范、字符集规范 +- DML 规则 (9条): WHERE条件、批量限制、事务规范 +- Online-DDL 规则 (3条): 在线变更工具配置 +- 命名规则 (5条): 表名/字段名/索引名规范 +- 性能规则 (5条): 全表扫描、索引使用 +- 安全规则 (5条): 敏感字段、SQL注入风险 + +使用方式: + 规则通过 @register_rule 装饰器自动注册到审核引擎， + 由 SQLAuditEngineV2 在审核时自动调用。 + +依赖: + - audit_engine_v2.py: register_rule, AuditRuleV2, AuditFindingV2, AuditReportV2 + - Severity, RuleCategory 枚举 +""" + +from __future__ import annotations + +import re +from typing import Any, Dict, List, Optional + +from app.services.audit_engine_v2 import register_rule, AuditRuleV2, AuditFindingV2, AuditReportV2 +from app.services.audit_engine_v2 import Severity, RuleCategory +from app.services.audit_engine_v2 import ( + _extract_table_name, + _extract_column_definitions, + _extract_index_definitions, + _is_snake_case, +) + +# ────────────────────────────────────────────── +# 辅助解析函数 +# ────────────────────────────────────────────────── + + +def _extract_create_table_comment(sql_upper: str) -> Optional[str]: + """从 CREATE TABLE 语句中提取表注释。""" + comment_match = re.search(r"COMMENT\s*=\s*['\"]([^'\"]*)['\"]", sql_upper, re.IGNORECASE) + if comment_match: + return comment_match.group(1).strip() + # 也匹配 COMMENT 'xxx' 格式 + comment_match = re.search(r"COMMENT\s+['\"]([^'\"]+)['\"]", sql_upper, re.IGNORECASE) + if comment_match: + return comment_match.group(1).strip() + return None + + +def _extract_charset_info(sql_upper: str) -> Dict[str, Optional[str]]: + """从 CREATE TABLE 语句中提取字符集和排序规则信息。""" + result: Dict[str, Optional[str]] = {"charset": None, "collation": None} + cs_match = re.search(r"DEFAULT\s+CHARSET\s*=\s*(\w+)", sql_upper, re.IGNORECASE) + if not cs_match: + cs_match = re.search(r"CHARSET\s*=\s*(\w+)", sql_upper, re.IGNORECASE) + if cs_match: + result["charset"] = cs_match.group(1).lower() + + col_match = re.search(r"COLLATE\s*=\s*(\w+)", sql_upper, re.IGNORECASE) + if col_match: + result["collation"] = col_match.group(1).lower() + return result + + +def _extract_alter_operations(sql_upper: str) -> List[Dict[str, str]]: + """从 ALTER TABLE 语句中提取操作列表。""" + operations: List[Dict[str, str]] = [] + # 匹配 ADD COLUMN, DROP COLUMN, MODIFY COLUMN, CHANGE COLUMN 等 + patterns = [ + (r"ADD\s+COLUMN\s+[`\"']?(\w+)", "ADD_COLUMN"), + (r"ADD\s+(?:INDEX|KEY)\s+[`\"']?(\w*)", "ADD_INDEX"), + (r"DROP\s+COLUMN\s+[`\"']?(\w+)", "DROP_COLUMN"), + (r"MODIFY\s+COLUMN\s+[`\"']?(\w+)", "MODIFY_COLUMN"), + (r"CHANGE\s+COLUMN\s+[`\"']?(\w+)", "CHANGE_COLUMN"), + (r"DROP\s+(?:INDEX|KEY)\s+[`\"']?(\w+)", "DROP_INDEX"), + (r"ADD\s+PRIMARY\s+KEY", "ADD_PK"), + (r"DROP\s+PRIMARY\s+KEY", "DROP_PK"), + ] + for pattern, op_type in patterns: + for match in re.finditer(pattern, sql_upper, re.IGNORECASE): + name = match.group(1) if match.lastindex and match.group(1) else "" + operations.append({"type": op_type, "name": name}) + return operations + + +def _has_where_clause(sql_upper: str) -> bool: + """检查SQL语句是否包含WHERE子句。""" + return bool(re.search(r"\bWHERE\b", sql_upper, re.IGNORECASE)) + + +def _has_limit_clause(sql_upper: str) -> bool: + """检查SQL语句是否包含LIMIT子句。""" + return bool(re.search(r"\bLIMIT\s+\d+", sql_upper, re.IGNORECASE)) + + +def _has_order_by(sql_upper: str) -> bool: + """检查SQL语句是否包含ORDER BY子句。""" + return bool(re.search(r"\bORDER\s+BY\b", sql_upper, re.IGNORECASE)) + + +def _extract_insert_columns(sql_upper: str) -> List[str]: + """从 INSERT 语句中提取插入的字段名列表。""" + cols: List[str] = [] + match = re.search(r"\bINSERT\b.+?\(([^)]+)\)", sql_upper, re.IGNORECASE | re.DOTALL) + if match: + inner = match.group(1) + for col in inner.split(","): + cols.append(col.strip().strip("`'\"")) + return cols + + +def _count_insert_rows(sql_clean: str) -> int: + """统计 INSERT 语句插入的行数（通过 VALUES 子句数量）。""" + values_count = len(re.findall(r"\bVALUES\s*\(", sql_clean, re.IGNORECASE)) + if values_count == 0: + # 尝试 INSERT ... SELECT 形式 + if re.search(r"\bINSERT\b.*\bSELECT\b", sql_clean, re.IGNORECASE | re.DOTALL): + return -1 # 表示无法确定的行数 + return max(values_count, 1) + + +def _is_ddl_statement(sql_upper: str) -> bool: + """判断是否为DDL语句。""" + return bool(re.search(r"\b(CREATE|ALTER|DROP|TRUNCATE)\b", sql_upper, re.IGNORECASE)) + + +def _is_dml_statement(sql_upper: str) -> bool: + """判断是否为DML语句。""" + return bool(re.search(r"\b(INSERT|UPDATE|DELETE|REPLACE)\b", sql_upper, re.IGNORECASE)) + + +def _extract_column_defaults(sql_upper: str) -> List[Dict[str, Any]]: + """从 CREATE TABLE 中提取字段默认值信息。""" + columns = _extract_column_definitions(sql_upper) + result: List[Dict[str, Any]] = [] + for col in columns: + has_default = "DEFAULT" in col["full"].upper() + is_nullable = "NOT NULL" not in col["full"].upper() + is_timestamp = col["type"].upper() in ("TIMESTAMP", "DATETIME") + result.append({ + "name": col["name"], + "type": col["type"], + "has_default": has_default, + "is_nullable": is_nullable, + "is_timestamp": is_timestamp, + "full": col["full"], + }) + return result + + +def _extract_primary_key_info(sql_upper: str) -> Optional[Dict[str, Any]]: + """从 CREATE TABLE 中提取主键信息。""" + # 内联主键定义 + pk_match = re.search(r"PRIMARY\s+KEY\s*\(([^)]+)\)", sql_upper, re.IGNORECASE) + if pk_match: + cols = [c.strip().strip("`'\"") for c in pk_match.group(1).split(",")] + return {"type": "inline", "columns": cols} + + # 单列主键如 `id` INT PRIMARY KEY + for col in _extract_column_definitions(sql_upper): + if "PRIMARY KEY" in col["full"].upper(): + return {"type": "column", "columns": [col["name"]], "column_def": col} + return None + + +def _count_alter_statements(sql_upper: str) -> int: + """统计 ALTER TABLE 语句数量。""" + return len(re.findall(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE)) + + +def _has_auto_increment(sql_upper: str) -> bool: + """检查是否包含 AUTO_INCREMENT。""" + return bool(re.search(r"\bAUTO_INCREMENT\b", sql_upper, re.IGNORECASE)) + + +def _extract_auto_increment_init(sql_upper: str) -> Optional[int]: + """提取 AUTO_INCREMENT 初始值。""" + match = re.search(r"AUTO_INCREMENT\s*=\s*(\d+)", sql_upper, re.IGNORECASE) + if match: + return int(match.group(1)) + return None + + +# ────────────────────────────────────────────── +# ══════════════════════════════════════════════ +# DDL 规则 (36条) — MYSQL-DDL-001 ~ MYSQL-DDL-036 +# ══════════════════════════════════════════════ +# ────────────────────────────────────────────── + + +# ── DDL-001: 强制表必须拥有表注释 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-001", + name="DDLCheckTableComment", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="CREATE TABLE 语句必须包含 COMMENT 表注释", + config={"enabled": True}, + message_template="表 '{table}' 缺少表注释，请添加 COMMENT='...'", +)) +def check_mysql_ddl_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制表必须拥有表注释。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + comment = _extract_create_table_comment(sql_upper) + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + if not comment: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── DDL-002: 强制表字段必须拥有列注释 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-002", + name="DDLCheckColumnComment", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="CREATE TABLE 中每个字段定义必须包含 COMMENT 注释", + config={"enabled": True}, + message_template="表 '{table}' 的字段 '{column}' 缺少列注释，请添加 COMMENT '...'", +)) +def check_mysql_ddl_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制表字段必须拥有列注释。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + paren_match = re.search(r"\((.*)\)", sql_upper, re.DOTALL) + if not paren_match: + return + + inner = paren_match.group(1) + parts = [] + depth = 0 + current = "" + for char in inner: + if char == "(": + depth += 1 + current += char + elif char == ")": + depth -= 1 + current += char + elif char == "," and depth == 0: + parts.append(current.strip()) + current = "" + else: + current += char + if current.strip(): + parts.append(current.strip()) + + for part in parts: + # 跳过约束定义 + if re.match(r"^(PRIMARY\s+KEY|FOREIGN\s+KEY|INDEX|KEY|UNIQUE|CONSTRAINT|CHECK)\b", part, re.IGNORECASE): + continue + col_match = re.match(r"[`\"']?(\w+)[`\"']?\s+(\w+)", part, re.IGNORECASE) + if col_match: + col_name = col_match.group(1) + if "COMMENT" not in part.upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=col_name), + )) + + +# ── DDL-003: 强制非timestamp类型字段必须为NOT NULL ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-003", + name="DDLCheckColumnNullable", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="非 TIMESTAMP/DATETIME 类型字段必须定义为 NOT NULL", + config={"enabled": True}, + message_template="表 '{table}' 的字段 '{column}' (类型: {ctype}) 必须定义为 NOT NULL", +)) +def check_mysql_ddl_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制非timestamp类型字段必须为NOT NULL。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + paren_match = re.search(r"\((.*)\)", sql_upper, re.DOTALL) + if not paren_match: + return + + inner = paren_match.group(1) + parts = [] + depth = 0 + current = "" + for char in inner: + if char == "(": + depth += 1 + current += char + elif char == ")": + depth -= 1 + current += char + elif char == "," and depth == 0: + parts.append(current.strip()) + current = "" + else: + current += char + if current.strip(): + parts.append(current.strip()) + + for part in parts: + if re.match(r"^(PRIMARY\s+KEY|FOREIGN\s+KEY|INDEX|KEY|UNIQUE|CONSTRAINT|CHECK)\b", part, re.IGNORECASE): + continue + col_match = re.match(r"[`\"']?(\w+)[`\"']?\s+(\w+)", part, re.IGNORECASE) + if col_match: + col_name = col_match.group(1) + col_type = col_match.group(2).upper() + if col_type in ("TIMESTAMP", "DATETIME"): + continue + if "NOT NULL" not in part.upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=col_name, ctype=col_type), + )) + + +# ── DDL-004: 强制非text,blob,json,timestamp类型字段必须拥有默认值 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-004", + name="DDLCheckColumnDefault", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="非 TEXT/BLOB/JSON/TIMESTAMP/DATETIME 类型字段必须指定 DEFAULT 值", + config={"enabled": True}, + message_template="表 '{table}' 的字段 '{column}' (类型: {ctype}) 缺少 DEFAULT 值", +)) +def check_mysql_ddl_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制非text,blob,json,timestamp类型字段必须拥有默认值。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + excluded_types = {"TEXT", "BLOB", "JSON", "TIMESTAMP", "DATETIME", "TINYTEXT", "MEDIUMTEXT", + "LONGTEXT", "TINYBLOB", "MEDIUMBLOB", "LONGBLOB"} + + paren_match = re.search(r"\((.*)\)", sql_upper, re.DOTALL) + if not paren_match: + return + + inner = paren_match.group(1) + parts = [] + depth = 0 + current = "" + for char in inner: + if char == "(": + depth += 1 + current += char + elif char == ")": + depth -= 1 + current += char + elif char == "," and depth == 0: + parts.append(current.strip()) + current = "" + else: + current += char + if current.strip(): + parts.append(current.strip()) + + for part in parts: + if re.match(r"^(PRIMARY\s+KEY|FOREIGN\s+KEY|INDEX|KEY|UNIQUE|CONSTRAINT|CHECK)\b", part, re.IGNORECASE): + continue + col_match = re.match(r"[`\"']?(\w+)[`\"']?\s+(\w+)", part, re.IGNORECASE) + if col_match: + col_name = col_match.group(1) + col_type = col_match.group(2).upper() + if col_type in excluded_types: + continue + if "DEFAULT" not in part.upper() and "NOT NULL" in part.upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=col_name, ctype=col_type), + )) + + +# ── DDL-005: 强制float/double类型变更为decimal类型 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-005", + name="DDLCheckFloatDouble", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="不建议使用 FLOAT/DOUBLE 类型，建议使用 DECIMAL 替代", + config={"enabled": True}, + message_template="表 '{table}' 的字段 '{column}' 使用了 {ctype} 类型，建议改为 DECIMAL", +)) +def check_mysql_ddl_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制float/double类型变更为decimal类型。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + columns = _extract_column_definitions(sql_upper) + for col in columns: + if col["type"].upper() in ("FLOAT", "DOUBLE"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=col["name"], ctype=col["type"]), + )) + + +# ── DDL-006: 强制自增列初始值为1 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-006", + name="DDLEnableAutoincrementInit", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="AUTO_INCREMENT 初始值必须为 1", + config={"enabled": True, "init_value": 1}, + message_template="表 '{table}' 的 AUTO_INCREMENT 初始值不为 {init}，建议设置为 AUTO_INCREMENT=1", +)) +def check_mysql_ddl_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制自增列初始值为1。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if not _has_auto_increment(sql_upper): + return + init_val = _extract_auto_increment_init(sql_upper) + expected = rule.config.get("init_value", 1) + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + if init_val is not None and init_val != expected: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, init=expected), + )) + + +# ── DDL-007/008: 强制必须拥有主键 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-007", + name="DDLEnablePrimaryKey", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="CREATE TABLE 语句必须定义主键", + config={"enabled": True}, + message_template="表 '{table}' 缺少主键定义，必须添加 PRIMARY KEY", +)) +def check_mysql_ddl_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制必须拥有主键。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + pk_info = _extract_primary_key_info(sql_upper) + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + if pk_info is None: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── DDL-009: 强制主键名称为ID ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-009", + name="DDLPrimaryKeyMust", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="主键列名必须为 'id'（不区分大小写）", + config={"enabled": True}, + message_template="表 '{table}' 的主键列名为 '{pk_name}'，建议命名为 'id'", +)) +def check_mysql_ddl_009(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制主键名称为ID。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + pk_info = _extract_primary_key_info(sql_upper) + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + if pk_info and len(pk_info["columns"]) == 1: + pk_name = pk_info["columns"][0] + if pk_name.lower() != "id": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, pk_name=pk_name), + )) + + +# ── DDL-010: 强制主键为自增列 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-010", + name="DDLEnableAutoIncrement", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="主键列必须使用 AUTO_INCREMENT", + config={"enabled": True}, + message_template="表 '{table}' 的主键列 '{pk_name}' 缺少 AUTO_INCREMENT", +)) +def check_mysql_ddl_010(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制主键为自增列。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + pk_info = _extract_primary_key_info(sql_upper) + if pk_info is None: + return + if pk_info["type"] == "column": + col_def = pk_info.get("column_def", {}) + pk_name = pk_info["columns"][0] + if "AUTO_INCREMENT" not in col_def.get("full", "").upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, pk_name=pk_name), + )) + + +# ── DDL-011: 强制主键必须使用无符号标志unsigned ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-011", + name="DDLEnableAutoincrementUnsigned", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="主键整数类型必须使用 UNSIGNED 属性", + config={"enabled": True}, + message_template="表 '{table}' 的主键列 '{pk_name}' 建议使用 UNSIGNED 属性", +)) +def check_mysql_ddl_011(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制主键必须使用无符号标志unsigned。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + pk_info = _extract_primary_key_info(sql_upper) + if pk_info is None: + return + if pk_info["type"] == "column": + col_def = pk_info.get("column_def", {}) + pk_name = pk_info["columns"][0] + col_type = col_def.get("type", "").upper() + if col_type in ("INT", "BIGINT", "TINYINT", "SMALLINT", "MEDIUMINT"): + if "UNSIGNED" not in col_def.get("full", "").upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, pk_name=pk_name), + )) + + +# ── DDL-012: 基于主键回滚 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-012", + name="PRIRollBack", + category=RuleCategory.DDL, + severity=Severity.INFO, + db_type="mysql", + description="建议基于主键进行数据变更回滚操作", + config={"enabled": True}, + message_template="建议在回滚操作中使用主键进行定位", +)) +def check_mysql_ddl_012(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """基于主键回滚建议。""" + # 这是一个建议性规则，在 DELETE/UPDATE 语句中提示 + if not _is_dml_statement(sql_upper): + return + pk_info = _extract_primary_key_info(sql_upper) + if pk_info is None and _is_dml_statement(sql_upper): + # 简单提示：DML操作建议基于主键 + if re.search(r"\b(DELETE|UPDATE)\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── DDL-013: 开启索引名称规范(索引名必须以idx_为开头) ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-013", + name="DDLIndexNameSpec", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="普通索引名称必须以 'idx_' 为前缀，唯一索引以 'uniq_' 为前缀", + config={"enabled": True, "index_prefix": "idx_", "unique_prefix": "uniq_"}, + message_template="表 '{table}' 的索引 '{idx_name}' 命名不规范，普通索引应以 '{prefix}' 开头", +)) +def check_mysql_ddl_013(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """开启索引名称规范。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + idx_prefix = rule.config.get("index_prefix", "idx_") + uniq_prefix = rule.config.get("unique_prefix", "uniq_") + indexes = _extract_index_definitions(sql_upper) + for idx in indexes: + if idx["type"] == "INDEX" and not idx["name"].lower().startswith(idx_prefix.lower()): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format( + table=table_name, idx_name=idx["name"] or "(unnamed)", prefix=idx_prefix, + ), + )) + + +# ── DDL-014: 开启MySQL关键词检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-014", + name="CheckIdentifier", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="表名和字段名不能是MySQL保留关键字", + config={"enabled": True}, + message_template="表 '{table}' 中的标识符 '{identifier}' 是MySQL保留关键字，请修改", +)) +def check_mysql_ddl_014(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """开启MySQL关键词检查。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + # MySQL保留关键字列表（常用部分） + mysql_keywords = { + "SELECT", "INSERT", "UPDATE", "DELETE", "FROM", "WHERE", "ORDER", "GROUP", + "BY", "HAVING", "LIMIT", "OFFSET", "JOIN", "LEFT", "RIGHT", "INNER", "OUTER", + "ON", "AS", "AND", "OR", "NOT", "NULL", "TRUE", "FALSE", "CREATE", "DROP", + "ALTER", "TABLE", "INDEX", "VIEW", "DATABASE", "SCHEMA", "PRIMARY", "KEY", + "FOREIGN", "REFERENCES", "UNIQUE", "CHECK", "DEFAULT", "AUTO_INCREMENT", + "UNSIGNED", "CHARSET", "COLLATE", "ENGINE", "COMMENT", "INT", "VARCHAR", + "TEXT", "BLOB", "DATE", "TIME", "DATETIME", "TIMESTAMP", "FLOAT", "DOUBLE", + "DECIMAL", "SHOW", "USE", "DESC", "EXPLAIN", "SET", "COMMIT", "ROLLBACK", + } + columns = _extract_column_definitions(sql_upper) + if table_name.upper() in mysql_keywords: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, identifier=table_name), + )) + for col in columns: + if col["name"].upper() in mysql_keywords: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, identifier=col["name"]), + )) + + +# ── DDL-015: 允许跨库表迁移 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-015", + name="DDLEnableAcrossDBRename", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许跨库RENAME TABLE操作", + config={"enabled": False}, + message_template="禁止跨库RENAME TABLE操作: {detail}", +)) +def check_mysql_ddl_015(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许跨库表迁移（默认关闭）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bRENAME\s+TABLE\b", sql_upper, re.IGNORECASE): + return + # 检查是否包含数据库名前缀 (db.table) + if re.search(r"`?\w+`?\.`?\w+`?\s+TO\s+`?\w+`?\.`?\w+`?", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(detail=sql_clean[:80]), + )) + + +# ── DDL-016: 允许删除表 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-016", + name="DDLEnableDropTable", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许 DROP TABLE 操作", + config={"enabled": False}, + message_template="禁止 DROP TABLE 操作，表 '{table}' 将被删除", +)) +def check_mysql_ddl_016(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许删除表（默认关闭）。""" + if rule.config.get("enabled", False): + return + match = re.search(r"\bDROP\s+TABLE\s+(?:IF\s+EXISTS\s+)?[`\"']?(\w+)[`\"']?", sql_upper, re.IGNORECASE) + if match: + table_name = match.group(1) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── DDL-017: 允许删除库 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-017", + name="DDLEnableDropDatabase", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许 DROP DATABASE 操作", + config={"enabled": False}, + message_template="禁止 DROP DATABASE 操作，数据库 '{db}' 将被删除", +)) +def check_mysql_ddl_017(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许删除库（默认关闭）。""" + if rule.config.get("enabled", False): + return + match = re.search(r"\bDROP\s+DATABASE\s+(?:IF\s+EXISTS\s+)?[`\"']?(\w+)[`\"']?", sql_upper, re.IGNORECASE) + if match: + db_name = match.group(1) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(db=db_name), + )) + + +# ── DDL-018: 允许主键类型为非int/bigint ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-018", + name="DDLAllowPRINotInt", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="默认要求主键类型必须为 INT 或 BIGINT", + config={"enabled": False}, + message_template="表 '{table}' 的主键类型为 '{pk_type}'，建议使用 INT 或 BIGINT", +)) +def check_mysql_ddl_018(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许主键类型为非int/bigint（默认不允许）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + pk_info = _extract_primary_key_info(sql_upper) + if pk_info is None: + return + if pk_info["type"] == "column": + col_def = pk_info.get("column_def", {}) + pk_type = col_def.get("type", "").upper() + if pk_type not in ("INT", "BIGINT", "TINYINT", "SMALLINT", "MEDIUMINT"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, pk_type=pk_type), + )) + + +# ── DDL-019: 允许索引名为空 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-019", + name="DDLEnableNullIndexName", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="默认要求索引必须命名", + config={"enabled": False}, + message_template="表 '{table}' 存在未命名的索引，建议为所有索引指定名称", +)) +def check_mysql_ddl_019(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许索引名为空（默认不允许）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + indexes = _extract_index_definitions(sql_upper) + for idx in indexes: + if not idx["name"] and idx["type"] not in ("PRIMARY", "FOREIGN"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── DDL-020: 允许单个工单提交多条DDL语句 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-020", + name="DDLMultiToCommit", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许单个工单提交多条DDL语句", + config={"enabled": False, "max_statements": 1}, + message_template="单个工单包含 {count} 条DDL语句，建议每条工单只提交1条DDL", +)) +def check_mysql_ddl_020(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许单个工单提交多条DDL语句（默认不允许）。""" + if rule.config.get("enabled", False): + return + ddl_count = 0 + # 按分号分割统计DDL语句数 + statements = [s.strip() for s in sql_clean.split(";") if s.strip()] + for stmt in statements: + if re.search(r"\b(CREATE|ALTER|DROP|TRUNCATE)\b", stmt, re.IGNORECASE): + ddl_count += 1 + max_stmt = rule.config.get("max_statements", 1) + if ddl_count > max_stmt: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=ddl_count), + )) + + +# ── DDL-021: DDL单个工单允许多alter语句提交 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-021", + name="DDLAllowMultiAlter", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许单个工单包含多条 ALTER TABLE 语句", + config={"enabled": False, "max_alter": 1}, + message_template="单个工单包含 {count} 条 ALTER TABLE 语句，建议拆分为多个工单", +)) +def check_mysql_ddl_021(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """DDL单个工单允许多alter语句提交（默认不允许）。""" + if rule.config.get("enabled", False): + return + alter_count = _count_alter_statements(sql_upper) + max_alter = rule.config.get("max_alter", 1) + if alter_count > max_alter: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=alter_count), + )) + + +# ── DDL-022: 允许字段进行类型转换 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-022", + name="DDLAllowColumnType", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="默认不允许 ALTER TABLE MODIFY/CHANGE 进行字段类型转换", + config={"enabled": False}, + message_template="ALTER TABLE 包含字段类型变更操作，请确认变更的必要性和兼容性", +)) +def check_mysql_ddl_022(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许字段进行类型转换（默认不允许）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + operations = _extract_alter_operations(sql_upper) + for op in operations: + if op["type"] in ("MODIFY_COLUMN", "CHANGE_COLUMN"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── DDL-023: 允许使用after/first ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-023", + name="DDLAllowChangeColumnPosition", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="默认不建议使用 AFTER/FIRST 调整字段位置（引起全表重建）", + config={"enabled": False}, + message_template="ALTER TABLE 使用了 AFTER/FIRST 调整字段位置，可能引起全表重建", +)) +def check_mysql_ddl_023(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许使用after/first（默认不建议）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\b(AFTER|FIRST)\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── DDL-024: 允许创建视图 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-024", + name="AllowCreateView", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许创建视图", + config={"enabled": False}, + message_template="禁止创建视图: '{view_name}'", +)) +def check_mysql_ddl_024(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许创建视图（默认不允许）。""" + if rule.config.get("enabled", False): + return + match = re.search(r"\bCREATE\s+(?:OR\s+REPLACE\s+)?VIEW\s+[`\"']?(\w+)[`\"']?", sql_upper, re.IGNORECASE) + if match: + view_name = match.group(1) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(view_name=view_name), + )) + + +# ── DDL-025: 允许创建外键 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-025", + name="DDLEnableForeignKey", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许创建外键约束", + config={"enabled": False}, + message_template="禁止创建外键约束: '{fk_name}'，建议在应用层实现外键逻辑", +)) +def check_mysql_ddl_025(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许创建外键（默认不允许）。""" + if rule.config.get("enabled", False): + return + if re.search(r"\bFOREIGN\s+KEY\b", sql_upper, re.IGNORECASE): + fk_match = re.search(r"FOREIGN\s+KEY\s+[`\"']?(\w*)[`\"']?", sql_upper, re.IGNORECASE) + fk_name = fk_match.group(1) if fk_match and fk_match.group(1) else "(unnamed)" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(fk_name=fk_name), + )) + + +# ── DDL-026: 允许创建分区表 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-026", + name="AllowCreatePartition", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="默认不建议创建分区表", + config={"enabled": False}, + message_template="创建分区表需要谨慎评估，请确认分区策略的合理性", +)) +def check_mysql_ddl_026(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许创建分区表（默认不建议）。""" + if rule.config.get("enabled", False): + return + if re.search(r"\bPARTITION\s+BY\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── DDL-027: 允许添加bit,enum,set类型字段 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-027", + name="AllowSpecialType", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="默认不建议使用 BIT/ENUM/SET 类型", + config={"enabled": False}, + message_template="表 '{table}' 使用了特殊类型 '{sp_type}'，建议评估替代方案", +)) +def check_mysql_ddl_027(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许添加bit,enum,set类型字段（默认不建议）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + columns = _extract_column_definitions(sql_upper) + special_types = {"BIT", "ENUM", "SET"} + for col in columns: + if col["type"].upper() in special_types: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, sp_type=col["type"]), + )) + + +# ── DDL-028: 指定表名前缀 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-028", + name="DDLTablePrefix", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="表名必须以指定前缀开头", + config={"enabled": True, "prefix": "t_"}, + message_template="表名 '{table}' 不符合前缀规范，必须以 '{prefix}' 开头", +)) +def check_mysql_ddl_028(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """指定表名前缀。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + prefix = rule.config.get("prefix", "t_") + table_name = _extract_table_name(sql_upper, sql_clean) + if table_name and not table_name.lower().startswith(prefix.lower()): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, prefix=prefix), + )) + + +# ── DDL-029: 允许的Collate范围 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-029", + name="SupportCollation", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="表字符集排序规则必须在允许的范围内", + config={"enabled": True, "allowed_collations": ["utf8mb4_general_ci", "utf8mb4_unicode_ci"]}, + message_template="表 '{table}' 使用了不允许的排序规则 '{collation}'，允许的规则: {allowed}", +)) +def check_mysql_ddl_029(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许的Collate范围。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + charset_info = _extract_charset_info(sql_upper) + collation = charset_info.get("collation") + allowed = rule.config.get("allowed_collations", []) + if collation and allowed and collation.lower() not in [a.lower() for a in allowed]: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format( + table=table_name, collation=collation, allowed=", ".join(allowed), + ), + )) + + +# ── DDL-030: 允许的Charset范围 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-030", + name="SupportCharset", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="表字符集必须在允许的范围内", + config={"enabled": True, "allowed_charsets": ["utf8mb4"]}, + message_template="表 '{table}' 使用了不允许的字符集 '{charset}'，允许的字符集: {allowed}", +)) +def check_mysql_ddl_030(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许的Charset范围。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + charset_info = _extract_charset_info(sql_upper) + charset = charset_info.get("charset") + allowed = rule.config.get("allowed_charsets", ["utf8mb4"]) + if charset and allowed and charset.lower() not in [a.lower() for a in allowed]: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format( + table=table_name, charset=charset, allowed=", ".join(allowed), + ), + )) + + +# ── DDL-031: 建表必须拥有的字段 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-031", + name="MustHaveColumns", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="建表时必须包含指定的字段", + config={"enabled": True, "required_columns": ["id", "created_at", "updated_at"]}, + message_template="表 '{table}' 缺少必须字段 '{column}'", +)) +def check_mysql_ddl_031(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """建表必须拥有的字段。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + required = rule.config.get("required_columns", []) + columns = _extract_column_definitions(sql_upper) + column_names = {c["name"].lower() for c in columns} + for req_col in required: + if req_col.lower() not in column_names: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=req_col), + )) + + +# ── DDL-032: 单个索引指定字段上限 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-032", + name="DDLMaxKeyParts", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="单个索引最多允许包含的字段数", + config={"enabled": True, "max_parts": 8}, + message_template="表 '{table}' 的索引 '{idx_name}' 包含 {count} 个字段，超过上限 {max_parts}", +)) +def check_mysql_ddl_032(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """单个索引指定字段上限。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + max_parts = rule.config.get("max_parts", 8) + indexes = _extract_index_definitions(sql_upper) + for idx in indexes: + col_count = len(idx.get("columns", [])) + if col_count > max_parts: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format( + table=table_name, idx_name=idx["name"] or "(unnamed)", + count=col_count, max_parts=max_parts, + ), + )) + + +# ── DDL-033: 单个表最多允许几个索引 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-033", + name="DDLMaxKey", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="单个表最多允许创建的索引数量", + config={"enabled": True, "max_index": 12}, + message_template="表 '{table}' 包含 {count} 个索引，超过上限 {max_idx}", +)) +def check_mysql_ddl_033(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """单个表最多允许几个索引。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + max_idx = rule.config.get("max_index", 12) + indexes = _extract_index_definitions(sql_upper) + idx_count = len([i for i in indexes if i["type"] != "FOREIGN"]) + if idx_count > max_idx: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, count=idx_count, max_idx=max_idx), + )) + + +# ── DDL-034: DDL最大影响行数 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-034", + name="MaxDDLAffectRows", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="mysql", + description="DDL操作影响的最大行数限制（0表示不限制）", + config={"enabled": False, "max_rows": 0}, + message_template="DDL操作可能影响 {rows} 行，超过上限 {max_rows}", +)) +def check_mysql_ddl_034(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """DDL最大影响行数（此规则作为配置项存在，实际行数需运行时检查）。""" + max_rows = rule.config.get("max_rows", 0) + if max_rows == 0: + return + # 此规则的实际检查需要数据库连接获取表行数 + # 这里仅做语句级别的标记提示 + if re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"ALTER TABLE '{table_name}' 需要确认影响行数不超过 {max_rows}", + )) + + +# ── DDL-035: char字段最大长度 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-035", + name="DDLMaxCharLength", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="CHAR 类型字段的最大长度限制", + config={"enabled": True, "max_length": 10}, + message_template="表 '{table}' 的字段 '{column}' CHAR({length}) 超过最大长度限制 {max_len}", +)) +def check_mysql_ddl_035(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """char字段最大长度。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + max_len = rule.config.get("max_length", 10) + paren_match = re.search(r"\((.*)\)", sql_upper, re.DOTALL) + if not paren_match: + return + inner = paren_match.group(1) + parts = [] + depth = 0 + current = "" + for char in inner: + if char == "(": + depth += 1 + current += char + elif char == ")": + depth -= 1 + current += char + elif char == "," and depth == 0: + parts.append(current.strip()) + current = "" + else: + current += char + if current.strip(): + parts.append(current.strip()) + + for part in parts: + if re.match(r"^(PRIMARY\s+KEY|FOREIGN\s+KEY|INDEX|KEY|UNIQUE|CONSTRAINT|CHECK)\b", part, re.IGNORECASE): + continue + char_match = re.search(r"[`\"']?(\w+)[`\"']?\s+CHAR\s*\((\d+)\)", part, re.IGNORECASE) + if char_match: + col_name = char_match.group(1) + length = int(char_match.group(2)) + if length > max_len: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format( + table=table_name, column=col_name, length=length, max_len=max_len, + ), + )) + + +# ── DDL-036: 表名最大长度限制 ── +@register_rule(AuditRuleV2( + code="MYSQL-DDL-036", + name="MaxTableNameLen", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="表名最大长度限制", + config={"enabled": True, "max_length": 45}, + message_template="表名 '{table}' 长度为 {length}，超过最大长度限制 {max_len}", +)) +def check_mysql_ddl_036(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """表名最大长度限制。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) + if not table_name: + return + max_len = rule.config.get("max_length", 45) + if len(table_name) > max_len: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, length=len(table_name), max_len=max_len), + )) + + +# ────────────────────────────────────────────── +# ══════════════════════════════════════════════ +# DML 规则 (9条) — MYSQL-DML-001 ~ MYSQL-DML-009 +# ══════════════════════════════════════════════ +# ────────────────────────────────────────────── + + +# ── DML-001: Insert最大插入行数上限 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-001", + name="DMLMaxInsertRows", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mysql", + description="INSERT 语句单次插入的最大行数限制", + config={"enabled": True, "max_rows": 500}, + message_template="INSERT 语句插入 {count} 行，超过最大限制 {max_rows} 行，请分批插入", +)) +def check_mysql_dml_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Insert最大插入行数上限。""" + if not re.search(r"\bINSERT\b", sql_upper, re.IGNORECASE): + return + max_rows = rule.config.get("max_rows", 500) + row_count = _count_insert_rows(sql_clean) + if row_count == -1: + # INSERT ... SELECT 无法确定行数 + return + if row_count > max_rows: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=row_count, max_rows=max_rows), + )) + + +# ── DML-002: 允许update/insert语句使用limit关键字 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-002", + name="DMLAllowLimitSTMT", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mysql", + description="默认不允许 UPDATE/DELETE 语句使用 LIMIT 关键字", + config={"enabled": False}, + message_template="{stmt_type} 语句禁止使用 LIMIT 关键字", +)) +def check_mysql_dml_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许update/insert语句使用limit关键字（默认不允许）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\b(UPDATE|DELETE)\b", sql_upper, re.IGNORECASE): + return + if _has_limit_clause(sql_upper): + stmt_type = "UPDATE" if re.search(r"\bUPDATE\b", sql_upper, re.IGNORECASE) else "DELETE" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stmt_type=stmt_type), + )) + + +# ── DML-003: 允许insert语句插入Null值 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-003", + name="DMLAllowInsertNull", + category=RuleCategory.DML, + severity=Severity.WARNING, + db_type="mysql", + description="默认不建议 INSERT 语句显式插入 NULL 值", + config={"enabled": False}, + message_template="INSERT 语句中包含 NULL 值插入，建议确认必要性", +)) +def check_mysql_dml_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许insert语句插入Null值（默认不建议）。""" + if rule.config.get("enabled", False): + return + if not re.search(r"\bINSERT\b", sql_upper, re.IGNORECASE): + return + # 匹配 VALUES (...NULL...) 中的 NULL + values_section = re.search(r"\bVALUES\s+\((.*?)\)", sql_upper, re.IGNORECASE | re.DOTALL) + if values_section and "NULL" in values_section.group(1).upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── DML-004: 不允许隐式转换 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-004", + name="DDLImplicitTypeConversion", + category=RuleCategory.DML, + severity=Severity.WARNING, + db_type="mysql", + description="WHERE 条件中避免隐式类型转换", + config={"enabled": True}, + message_template="WHERE 条件中字段 '{column}' 可能存在隐式类型转换，请确保类型一致", +)) +def check_mysql_dml_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """不允许隐式转换（检测WHERE中可能的类型不匹配）。""" + if not _has_where_clause(sql_upper): + return + # 简单检测：数字字段与字符串值比较，或函数字段使用 + where_match = re.search(r"\bWHERE\b(.+?)(?:\s+ORDER\s+BY|\s+GROUP\s+BY|\s+LIMIT|\s+HAVING|$)", + sql_upper, re.IGNORECASE | re.DOTALL) + if not where_match: + return + where_clause = where_match.group(1) + # 检测函数包裹字段（常见隐式转换场景） + func_patterns = [ + r"\b(CONVERT|CAST)\s*\(", + r"\bDATE\s*\(", + r"\bYEAR\s*\(", + r"\bMONTH\s*\(", + ] + for pattern in func_patterns: + if re.search(pattern, where_clause, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(column="(函数包装字段)"), + )) + + +# ── DML-005: 检查Insert语句中插入的字段名是否存在 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-005", + name="DMLInsertColumns", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mysql", + description="INSERT 语句必须显式指定字段名", + config={"enabled": True}, + message_template="INSERT 语句必须显式指定插入的字段名，禁止使用 INSERT INTO table VALUES (...)", +)) +def check_mysql_dml_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """检查Insert语句中插入的字段名是否存在。""" + if not re.search(r"\bINSERT\b", sql_upper, re.IGNORECASE): + return + insert_cols = _extract_insert_columns(sql_upper) + if not insert_cols: + # 检查是否是 INSERT INTO table VALUES (没有指定字段) + if re.search(r"\bINSERT\b.+?\bVALUES\b", sql_upper, re.IGNORECASE) and \ + not re.search(r"\bINSERT\b.+?\([^)]+\)\s*\bVALUES\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── DML-006: 强制DML语句必须拥有where条件 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-006", + name="DMLWhere", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mysql", + description="UPDATE/DELETE 语句必须包含 WHERE 条件", + config={"enabled": True}, + message_template="{stmt_type} 语句必须包含 WHERE 条件，禁止全表操作", +)) +def check_mysql_dml_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制DML语句必须拥有where条件。""" + if not re.search(r"\b(UPDATE|DELETE)\b", sql_upper, re.IGNORECASE): + return + if not _has_where_clause(sql_upper): + stmt_type = "UPDATE" if re.search(r"\bUPDATE\b", sql_upper, re.IGNORECASE) else "DELETE" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stmt_type=stmt_type), + )) + + +# ── DML-007: 禁止DML语句使用Order by子句 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-007", + name="DMLOrder", + category=RuleCategory.DML, + severity=Severity.WARNING, + db_type="mysql", + description="UPDATE/DELETE 语句禁止使用 ORDER BY 子句", + config={"enabled": True}, + message_template="{stmt_type} 语句禁止使用 ORDER BY 子句", +)) +def check_mysql_dml_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止DML语句使用Order by子句。""" + if not re.search(r"\b(UPDATE|DELETE)\b", sql_upper, re.IGNORECASE): + return + if _has_order_by(sql_upper): + stmt_type = "UPDATE" if re.search(r"\bUPDATE\b", sql_upper, re.IGNORECASE) else "DELETE" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stmt_type=stmt_type), + )) + + +# ── DML-008: 禁止DML语句使用Select子句 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-008", + name="DMLSelect", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mysql", + description="UPDATE/DELETE 语句禁止使用子查询（SELECT 子句）", + config={"enabled": True}, + message_template="{stmt_type} 语句禁止使用 SELECT 子查询", +)) +def check_mysql_dml_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止DML语句使用Select子句。""" + if not re.search(r"\b(UPDATE|DELETE)\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + stmt_type = "UPDATE" if re.search(r"\bUPDATE\b", sql_upper, re.IGNORECASE) else "DELETE" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stmt_type=stmt_type), + )) + + +# ── DML-009: DML最大影响行数 ── +@register_rule(AuditRuleV2( + code="MYSQL-DML-009", + name="MaxAffectRows", + category=RuleCategory.DML, + severity=Severity.ERROR, + db_type="mysql", + description="DML 操作影响的最大行数限制（0表示不限制）", + config={"enabled": False, "max_rows": 0}, + message_template="DML操作可能影响 {rows} 行，超过上限 {max_rows}", +)) +def check_mysql_dml_009(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """DML最大影响行数（此规则作为配置项存在，实际行数需运行时检查）。""" + max_rows = rule.config.get("max_rows", 0) + if max_rows == 0: + return + # 仅作标记提示，实际行数需要在执行时检查 + if _is_dml_statement(sql_upper): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"DML语句需要确认影响行数不超过 {max_rows}", + )) + + +# ────────────────────────────────────────────── +# ══════════════════════════════════════════════ +# Online-DDL 规则 (3条) — MYSQL-OSC-001 ~ MYSQL-OSC-003 +# ══════════════════════════════════════════════ +# ────────────────────────────────────────────── + + +# ── OSC-001: 开启在线表变更工具 ── +@register_rule(AuditRuleV2( + code="MYSQL-OSC-001", + name="IsOSC", + category=RuleCategory.DDL, + severity=Severity.INFO, + db_type="mysql", + description="建议使用在线表变更工具（pt-osc/gh-ost）执行大表DDL", + config={"enabled": True, "tool": "gh-ost"}, + message_template="表 '{table}' 的DDL操作建议使用 {tool} 在线变更工具执行", +)) +def check_mysql_osc_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """开启在线表变更工具建议。""" + if not re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + tool = rule.config.get("tool", "gh-ost") + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, tool=tool), + )) + + +# ── OSC-002: 表体积大于该值后触发在线表变更同步 ── +@register_rule(AuditRuleV2( + code="MYSQL-OSC-002", + name="OscSize", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="mysql", + description="表体积超过阈值（MB）后必须使用在线变更工具", + config={"enabled": True, "size_mb": 100}, + message_template="表 '{table}' 预估体积 {size}MB，超过阈值 {threshold}MB，必须使用在线变更工具", +)) +def check_mysql_osc_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """表体积大于阈值触发在线表变更建议。""" + if not re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + threshold = rule.config.get("size_mb", 100) + # 此规则实际需要在数据库中查询表大小，这里仅作提示 + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"ALTER TABLE '{table_name}' 请确认表体积是否超过 {threshold}MB，大表请使用在线变更工具", + )) + + +# ── OSC-003: 同步工具参数配置 ── +@register_rule(AuditRuleV2( + code="MYSQL-OSC-003", + name="OSCExpr", + category=RuleCategory.DDL, + severity=Severity.INFO, + db_type="mysql", + description="在线变更工具参数配置检查", + config={"enabled": True, "chunk_size": 1000, "max_load": "Threads_running=80"}, + message_template="请确认在线变更工具参数: chunk_size={chunk_size}, max_load={max_load}", +)) +def check_mysql_osc_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """同步工具参数配置检查。""" + if not re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + chunk_size = rule.config.get("chunk_size", 1000) + max_load = rule.config.get("max_load", "Threads_running=80") + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(chunk_size=chunk_size, max_load=max_load), + )) + + +# ────────────────────────────────────────────── +# ══════════════════════════════════════════════ +# 命名规则 (5条) — MYSQL-NAM-001 ~ MYSQL-NAM-005 +# ══════════════════════════════════════════════ +# ────────────────────────────────────────────── + + +# ── NAM-001: 表名小写下划线规范 ── +@register_rule(AuditRuleV2( + code="MYSQL-NAM-001", + name="TableNamingCase", + category=RuleCategory.NAMING, + severity=Severity.ERROR, + db_type="mysql", + description="表名必须使用小写下划线命名法（snake_case）", + config={"enabled": True}, + message_template="表名 '{table}' 不符合命名规范，必须使用小写下划线命名（snake_case）", +)) +def check_mysql_nam_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """表名小写下划线规范。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) + if table_name and not _is_snake_case(table_name): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── NAM-002: 字段名小写下划线规范 ── +@register_rule(AuditRuleV2( + code="MYSQL-NAM-002", + name="ColumnNamingCase", + category=RuleCategory.NAMING, + severity=Severity.ERROR, + db_type="mysql", + description="字段名必须使用小写下划线命名法（snake_case）", + config={"enabled": True}, + message_template="表 '{table}' 的字段 '{column}' 不符合命名规范，必须使用小写下划线命名（snake_case）", +)) +def check_mysql_nam_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """字段名小写下划线规范。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + columns = _extract_column_definitions(sql_upper) + for col in columns: + if not _is_snake_case(col["name"]): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=col["name"]), + )) + + +# ── NAM-003: 索引名小写下划线规范 ── +@register_rule(AuditRuleV2( + code="MYSQL-NAM-003", + name="IndexNamingCase", + category=RuleCategory.NAMING, + severity=Severity.WARNING, + db_type="mysql", + description="索引名必须使用小写下划线命名法", + config={"enabled": True}, + message_template="表 '{table}' 的索引 '{idx_name}' 不符合命名规范，必须使用小写下划线命名", +)) +def check_mysql_nam_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """索引名小写下划线规范。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + indexes = _extract_index_definitions(sql_upper) + for idx in indexes: + if idx["name"] and not _is_snake_case(idx["name"]): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, idx_name=idx["name"]), + )) + + +# ── NAM-004: 表名数字开头检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-NAM-004", + name="TableNamingNoDigitPrefix", + category=RuleCategory.NAMING, + severity=Severity.ERROR, + db_type="mysql", + description="表名不能以数字开头", + config={"enabled": True}, + message_template="表名 '{table}' 不能以数字开头", +)) +def check_mysql_nam_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """表名数字开头检查。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) + if table_name and re.match(r"^\d", table_name): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── NAM-005: 字段名长度限制 ── +@register_rule(AuditRuleV2( + code="MYSQL-NAM-005", + name="MaxColumnNameLength", + category=RuleCategory.NAMING, + severity=Severity.WARNING, + db_type="mysql", + description="字段名最大长度限制", + config={"enabled": True, "max_length": 30}, + message_template="表 '{table}' 的字段 '{column}' 名长度为 {length}，超过限制 {max_len}", +)) +def check_mysql_nam_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """字段名长度限制。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + max_len = rule.config.get("max_length", 30) + columns = _extract_column_definitions(sql_upper) + for col in columns: + if len(col["name"]) > max_len: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format( + table=table_name, column=col["name"], length=len(col["name"]), max_len=max_len, + ), + )) + + +# ────────────────────────────────────────────── +# ══════════════════════════════════════════════ +# 性能规则 (5条) — MYSQL-PERF-001 ~ MYSQL-PERF-005 +# ══════════════════════════════════════════════ +# ────────────────────────────────────────────── + + +# ── PERF-001: 禁止SELECT * ── +@register_rule(AuditRuleV2( + code="MYSQL-PERF-001", + name="NoSelectAll", + category=RuleCategory.PERFORMANCE, + severity=Severity.ERROR, + db_type="mysql", + description="禁止 SELECT * 查询，必须显式指定字段", + config={"enabled": True}, + message_template="禁止使用 SELECT *，请显式指定查询字段", +)) +def check_mysql_perf_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止SELECT *。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bSELECT\s+\*\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── PERF-002: 缺少LIMIT的大查询检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-PERF-002", + name="SelectLimitRequired", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="mysql", + description="SELECT 查询建议添加 LIMIT 子句", + config={"enabled": True, "max_limit": 1000}, + message_template="SELECT 查询缺少 LIMIT 子句，建议添加 LIMIT {max_limit}", +)) +def check_mysql_perf_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """缺少LIMIT的大查询检查。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + if _is_dml_statement(sql_upper): + return + if not _has_limit_clause(sql_upper): + max_limit = rule.config.get("max_limit", 1000) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(max_limit=max_limit), + )) + + +# ── PERF-003: 全表扫描风险 — WHERE条件缺少索引字段 ── +@register_rule(AuditRuleV2( + code="MYSQL-PERF-003", + name="FullTableScanRisk", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="mysql", + description="UPDATE/DELETE 的 WHERE 条件应尽量使用索引字段", + config={"enabled": True}, + message_template="{stmt_type} 语句 WHERE 条件 '{where_clause}' 可能无法使用索引，存在全表扫描风险", +)) +def check_mysql_perf_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """全表扫描风险检查。""" + if not re.search(r"\b(UPDATE|DELETE|SELECT)\b", sql_upper, re.IGNORECASE): + return + if not _has_where_clause(sql_upper): + return + # 检测 WHERE 中使用函数或 LIKE '%xxx%' 等无法使用索引的模式 + where_match = re.search(r"\bWHERE\b(.+?)(?:\s+ORDER\s+BY|\s+GROUP\s+BY|\s+LIMIT|\s+HAVING|$)", + sql_upper, re.IGNORECASE | re.DOTALL) + if not where_match: + return + where_clause = where_match.group(1).strip() + # 检测 LIKE '%xxx' 或 LIKE '%xxx%' + if re.search(r"\bLIKE\s+['\"]%", where_clause, re.IGNORECASE): + stmt_type_match = re.search(r"\b(UPDATE|DELETE|SELECT)\b", sql_upper, re.IGNORECASE) + stmt_type = stmt_type_match.group(1).upper() if stmt_type_match else "SQL" + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(stmt_type=stmt_type, where_clause=where_clause[:50]), + )) + + +# ── PERF-004: 大IN子句检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-PERF-004", + name="LargeInClause", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="mysql", + description="IN 子句中的值数量不应超过阈值", + config={"enabled": True, "max_in_values": 100}, + message_template="IN 子句包含 {count} 个值，超过建议上限 {max_in}", +)) +def check_mysql_perf_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """大IN子句检查。""" + max_in = rule.config.get("max_in_values", 100) + in_counts = [] + for match in re.finditer(r"\bIN\s*\(([^)]+)\)", sql_upper, re.IGNORECASE): + inner = match.group(1) + if inner.strip(): + count = inner.count(",") + 1 + in_counts.append(count) + for count in in_counts: + if count > max_in: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=count, max_in=max_in), + )) + + +# ── PERF-005: 多表JOIN检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-PERF-005", + name="ExcessiveJoins", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="mysql", + description="JOIN 表数量不应超过阈值", + config={"enabled": True, "max_joins": 5}, + message_template="查询包含 {count} 个 JOIN，超过建议上限 {max_joins}", +)) +def check_mysql_perf_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """多表JOIN检查。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + join_count = len(re.findall(r"\bJOIN\b", sql_upper, re.IGNORECASE)) + max_joins = rule.config.get("max_joins", 5) + if join_count > max_joins: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=join_count, max_joins=max_joins), + )) + + +# ────────────────────────────────────────────── +# ══════════════════════════════════════════════ +# 安全规则 (5条) — MYSQL-SEC-001 ~ MYSQL-SEC-005 +# ══════════════════════════════════════════════ +# ────────────────────────────────────────────── + + +# ── SEC-001: 敏感字段检测 ── +@register_rule(AuditRuleV2( + code="MYSQL-SEC-001", + name="SensitiveColumnDetection", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mysql", + description="禁止在表定义中使用敏感字段名（如password, secret等）", + config={"enabled": True}, + message_template="表 '{table}' 包含敏感字段名 '{column}'，请确认是否符合安全规范", +)) +def check_mysql_sec_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """敏感字段检测。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + table_name = _extract_table_name(sql_upper, sql_clean) or "unknown" + sensitive_patterns = ["password", "passwd", "secret", "token", "api_key", "apikey", "credential", "auth"] + columns = _extract_column_definitions(sql_upper) + for col in columns: + col_lower = col["name"].lower() + for pattern in sensitive_patterns: + if pattern in col_lower: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(table=table_name, column=col["name"]), + )) + + +# ── SEC-002: SQL注入风险 — 字符串拼接检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-SEC-002", + name="SQLInjectionRisk", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mysql", + description="检测SQL中可能的注入风险（如字符串拼接占位符）", + config={"enabled": True}, + message_template="SQL语句可能存在注入风险，请使用参数化查询", +)) +def check_mysql_sec_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """SQL注入风险检查。""" + # 检测 ${xxx} 或 #{xxx} 等模板占位符（不应出现在最终SQL中） + if re.search(r"\$\{[^}]+\}", sql_clean) or re.search(r"#\{[^}]+\}", sql_clean): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + # 检测字符串拼接模式 + if "' || '" in sql_clean or "' + '" in sql_clean or "CONCAT(" in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── SEC-003: 明文密码检测 ── +@register_rule(AuditRuleV2( + code="MYSQL-SEC-003", + name="PlaintextPassword", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mysql", + description="禁止 INSERT/UPDATE 中设置明文密码值", + config={"enabled": True}, + message_template="禁止在SQL中写入明文密码，请使用加密后的值", +)) +def check_mysql_sec_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """明文密码检测。""" + if not _is_dml_statement(sql_upper): + return + # 检测 password/secret 相关的值设置 + if re.search(r"\b(password|secret|passwd)\b\s*[=)]\s*['\"][^'\"]+['\"]", + sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ── SEC-004: 注释符检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-SEC-004", + name="NoCommentInSQL", + category=RuleCategory.SECURITY, + severity=Severity.WARNING, + db_type="mysql", + description="SQL中不应包含注释（可能是SQL注入手法）", + config={"enabled": True}, + message_template="SQL中包含注释符，请移除: {comment}", +)) +def check_mysql_sec_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """注释符检查。""" + if "--" in sql_clean: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(comment="--"), + )) + if "/*" in sql_clean or "*/" in sql_clean: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(comment="/* */"), + )) + + +# ── SEC-005: 多语句检查 ── +@register_rule(AuditRuleV2( + code="MYSQL-SEC-005", + name="NoMultiStatement", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="mysql", + description="禁止单条输入中包含多条SQL语句（分号分隔）", + config={"enabled": True}, + message_template="禁止提交多条SQL语句，请拆分为多个工单", +)) +def check_mysql_sec_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """多语句检查。""" + stripped = sql_clean.rstrip(";").strip() + if ";" in stripped: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +# ═══════════════════════════════════════════════════ +# 模块总结 +# ═══════════════════════════════════════════════════ +# 总计 63 条规则: +# - DDL 规则: 36 条 (MYSQL-DDL-001 ~ MYSQL-DDL-036) +# - DML 规则: 9 条 (MYSQL-DML-001 ~ MYSQL-DML-009) +# - Online-DDL 规则: 3 条 (MYSQL-OSC-001 ~ MYSQL-OSC-003) +# - 命名规则: 5 条 (MYSQL-NAM-001 ~ MYSQL-NAM-005) +# - 性能规则: 5 条 (MYSQL-PERF-001 ~ MYSQL-PERF-005) +# - 安全规则: 5 条 (MYSQL-SEC-001 ~ MYSQL-SEC-005) +# ═══════════════════════════════════════════════════ diff --git a/backend/app/services/postgres_audit_rules.py b/backend/app/services/postgres_audit_rules.py new file mode 100644 index 0000000..ca9ad65 --- /dev/null +++ b/backend/app/services/postgres_audit_rules.py @@ -0,0 +1,1289 @@ +"""PostgreSQL 特有审核规则引擎。 + +包含 45+ 条 PostgreSQL 特有审核规则，覆盖以下维度: +- DDL (18条): CREATE TABLE / ALTER TABLE 特有语法规范 +- DML (8条): INSERT/UPDATE/DELETE 操作规范 +- DQL (6条): SELECT 查询规范 +- SECURITY (6条): PostgreSQL 安全风险检测 +- PERFORMANCE (7条): PostgreSQL 性能优化建议 + +规则编码规范: +- DDL: PG-DDL-001 ~ PG-DDL-018 +- DML: PG-DML-001 ~ PG-DML-008 +- DQL: PG-DQL-001 ~ PG-DQL-006 +- SEC: PG-SEC-001 ~ PG-SEC-006 +- PERF: PG-PERF-001 ~ PG-PERF-007 + +依赖: + from app.services.audit_engine_v2 import ( + AuditRuleV2, AuditFindingV2, AuditReportV2, + Severity, RuleCategory, register_rule, + _extract_table_name, _extract_column_definitions, + _extract_index_definitions, + ) +""" + +from __future__ import annotations + +import re +from typing import Any, Callable, Dict, List, Optional, Tuple + +from app.services.audit_engine_v2 import ( + AuditRuleV2, + AuditFindingV2, + AuditReportV2, + Severity, + RuleCategory, + register_rule, + _extract_table_name, + _extract_column_definitions, + _extract_index_definitions, +) + + +# ═══════════════════════════════════════════════ +# PostgreSQL 规则专用辅助函数 +# ═══════════════════════════════════════════════ + + +def _extract_pg_table_name(sql_upper: str, sql_clean: str) -> Optional[str]: + """从 PostgreSQL CREATE/ALTER TABLE 语句中提取表名（支持 schema.table 形式）。""" + m = re.search( + r"\bCREATE\s+(?:TEMPORARY\s+|TEMP\s+)?TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?(?:\"?\w+\"?\.)?(\"?\w+\"?)", + sql_clean, re.IGNORECASE, + ) + if m: + return m.group(1).strip('"') + m = re.search( + r"\bALTER\s+TABLE\s+(?:ONLY\s+)?(?:\"?\w+\"?\.)?(\"?\w+\"?)", + sql_clean, re.IGNORECASE, + ) + if m: + return m.group(1).strip('"') + return None + + +def _extract_pg_column_defs(sql_upper: str) -> List[Dict[str, str]]: + """从 PostgreSQL CREATE TABLE (...) 中提取列定义。""" + paren_match = re.search(r"\((.*)\)\s*(?:INHERITS|TABLESPACE|PARTITION|WITH|;|$)", sql_upper, re.DOTALL) + if not paren_match: + return [] + inner = paren_match.group(1) + parts = [] + depth = 0 + current = "" + for char in inner: + if char == "(": + depth += 1 + current += char + elif char == ")": + depth -= 1 + current += char + elif char == "," and depth == 0: + parts.append(current.strip()) + current = "" + else: + current += char + if current.strip(): + parts.append(current.strip()) + columns: List[Dict[str, str]] = [] + for part in parts: + if re.match(r"^(CONSTRAINT|PRIMARY\s+KEY|UNIQUE|CHECK|EXCLUDE|FOREIGN\s+KEY|REFERENCES|LIKE)\b", part, re.IGNORECASE): + continue + col_match = re.match(r'"?(\w+)"?\s+(\w+(?:\s*\[\s*\])?|\w+\s+\w+)', part, re.IGNORECASE) + if col_match: + columns.append({"name": col_match.group(1), "type": col_match.group(2).upper(), "full": part}) + return columns + + +def _has_pg_serial_type(col_type: str) -> bool: + """判断是否为 SERIAL 类型。""" + return any(st in col_type for st in ("SERIAL", "BIGSERIAL", "SMALLSERIAL")) + + +# ═══════════════════════════════════════════════ +# DDL 规则 (PG-DDL-001 ~ PG-DDL-018, 共18条) +# ═══════════════════════════════════════════════ + + +# ── PG-DDL-001 ── +@register_rule(AuditRuleV2( + code="PG-DDL-001", name="PGDDLEnablePrimaryKey", + category=RuleCategory.DDL, severity=Severity.ERROR, db_type="postgres", + description="CREATE TABLE 语句必须包含 PRIMARY KEY 定义", + config={"allow_no_pk": False}, + message_template="建表语句缺少主键定义，请添加 PRIMARY KEY（推荐使用 GENERATED ALWAYS AS IDENTITY）", +)) +def check_pg_ddl_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制必须拥有主键。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if "PRIMARY KEY" not in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DDL-002 ── +@register_rule(AuditRuleV2( + code="PG-DDL-002", name="PGDDLCheckTableComment", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="建表语句必须包含 COMMENT ON 表注释", + config={"require_column_comment": True}, + message_template="表 '{table}' 缺少注释，请添加 COMMENT ON TABLE {table} IS '...'", +)) +def check_pg_ddl_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制表/列必须拥有注释(COMMENT ON)。""" + table_name = _extract_pg_table_name(sql_upper, sql_clean) + if not table_name: + return + has_comment = re.search(r"\bCOMMENT\s+ON\s+(?:TABLE|COLUMN)\b", sql_upper, re.IGNORECASE) + if not has_comment: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(table=table_name), + )) + + +# ── PG-DDL-003 ── +@register_rule(AuditRuleV2( + code="PG-DDL-003", name="PGDDLCheckColumnNullable", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="非timestamp类型字段必须声明 NOT NULL", + config={"nullable_exceptions": ["TIMESTAMP", "TIMESTAMPTZ", "DATE", "TIME", "JSON", "JSONB", "TEXT"]}, + message_template="字段 '{column}' 类型为 {col_type}，未声明 NOT NULL，建议添加约束", +)) +def check_pg_ddl_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制非timestamp类型字段NOT NULL。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + columns = _extract_pg_column_defs(sql_upper) + exceptions = rule.config.get("nullable_exceptions", []) + for col in columns: + col_type = col["type"] + if any(exc in col_type for exc in exceptions): + continue + if "NOT NULL" not in col["full"] and "NULL" not in col["full"]: + if "DEFAULT" in col["full"]: + continue + if _has_pg_serial_type(col_type): + continue + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(column=col["name"], col_type=col_type), + )) + + +# ── PG-DDL-004 ── +@register_rule(AuditRuleV2( + code="PG-DDL-004", name="PGDDLCheckColumnDefault", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="字段应当拥有默认值（除主键和 GENERATED 列外）", + config={"require_default": True}, + message_template="字段 '{column}' 缺少 DEFAULT 值，建议添加", +)) +def check_pg_ddl_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """字段必须拥有默认值。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + for col in _extract_pg_column_defs(sql_upper): + if "PRIMARY KEY" in col["full"] or "GENERATED" in col["full"] or _has_pg_serial_type(col["type"]): + continue + if "DEFAULT" not in col["full"]: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(column=col["name"]), + )) + + +# ── PG-DDL-005 ── +@register_rule(AuditRuleV2( + code="PG-DDL-005", name="PGDDLCheckFloatDouble", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="浮点类型精度丢失，建议使用 NUMERIC/DECIMAL", + message_template="字段 '{column}' 使用了 {col_type} 类型，建议使用 NUMERIC(p,s) 以确保精度", +)) +def check_pg_ddl_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """NUMERIC vs FLOAT 精度建议。""" + if not re.search(r"\bCREATE\s+TABLE\b|\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + float_types = ("FLOAT", "REAL", "DOUBLE", "DOUBLE PRECISION") + for col in _extract_pg_column_defs(sql_upper): + for ft in float_types: + if ft in col["type"]: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(column=col["name"], col_type=col["type"]), + )) + break + + +# ── PG-DDL-006 ── +@register_rule(AuditRuleV2( + code="PG-DDL-006", name="PGDDLEnableSerial", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="推荐使用 GENERATED ALWAYS AS IDENTITY 替代 SERIAL", + config={"recommend_identity": True, "generated_type": "ALWAYS"}, + message_template="建议使用 GENERATED {generated_type} AS IDENTITY 替代 SERIAL/BIGSERIAL，更符合 SQL 标准", +)) +def check_pg_ddl_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐自增列使用 GENERATED ALWAYS AS IDENTITY。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + for col in _extract_pg_column_defs(sql_upper): + if _has_pg_serial_type(col["type"]): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(generated_type=rule.config.get("generated_type", "ALWAYS")), + )) + break + + +# ── PG-DDL-007 ── +@register_rule(AuditRuleV2( + code="PG-DDL-007", name="PGDDLPrimaryKeyMust", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="主键字段名应为 id（支持配置）", + config={"pk_name": "id"}, + message_template="主键名 '{pk_name}' 建议改为 '{suggested}'", +)) +def check_pg_ddl_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """主键名称为 id。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + suggested = rule.config.get("pk_name", "id") + pk_match = re.search(r"PRIMARY\s+KEY\s*\(([^)]+)\)", sql_upper, re.IGNORECASE) + if pk_match: + for col in [c.strip().strip('"') for c in pk_match.group(1).split(",")]: + if col.lower() != suggested.lower(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(pk_name=col, suggested=suggested), + )) + for col in _extract_pg_column_defs(sql_upper): + if "PRIMARY KEY" in col["full"] and col["name"].lower() != suggested.lower(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(pk_name=col["name"], suggested=suggested), + )) + + +# ── PG-DDL-008 ── +@register_rule(AuditRuleV2( + code="PG-DDL-008", name="PGDDLEnableDropTable", + category=RuleCategory.DDL, severity=Severity.ERROR, db_type="postgres", + description="生产环境默认禁止执行 DROP TABLE 操作", + config={"env_production_only": True, "allow_drop": False}, + message_template="禁止执行 DROP TABLE 操作，数据将永久丢失。如需删除请先备份", +)) +def check_pg_ddl_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许删除表(生产环境默认禁止)。""" + if not re.search(r"\bDROP\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if not rule.config.get("allow_drop", False): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DDL-009 ── +@register_rule(AuditRuleV2( + code="PG-DDL-009", name="PGDDLEnableDropDatabase", + category=RuleCategory.DDL, severity=Severity.ERROR, db_type="postgres", + description="禁止执行 DROP DATABASE 操作", + config={"allow_drop_db": False}, + message_template="禁止执行 DROP DATABASE 操作，将导致整个数据库被删除", +)) +def check_pg_ddl_009(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许删除库（默认禁止）。""" + if not re.search(r"\bDROP\s+DATABASE\b", sql_upper, re.IGNORECASE): + return + if not rule.config.get("allow_drop_db", False): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DDL-010 ── +@register_rule(AuditRuleV2( + code="PG-DDL-010", name="PGDDLAllowPRINotInt", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="检查主键类型是否为 integer/bigint（允许 UUID 等）", + config={"allowed_pk_types": ["INT", "BIGINT", "INTEGER", "SERIAL", "BIGSERIAL", "UUID"]}, + message_template="主键类型 '{pk_type}' 不在推荐列表中，建议使用 BIGINT/SERIAL/UUID", +)) +def check_pg_ddl_010(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许主键类型为非integer/bigint（给出建议）。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + allowed = rule.config.get("allowed_pk_types", []) + for col in _extract_pg_column_defs(sql_upper): + if "PRIMARY KEY" in col["full"]: + if not any(a.upper() in col["type"] for a in allowed): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(pk_type=col["type"]), + )) + + +# ── PG-DDL-011 ── +@register_rule(AuditRuleV2( + code="PG-DDL-011", name="PGDDLMultiToCommit", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="单工单包含多条 DDL 语句", + config={"max_ddl_per_ticket": 5, "allow_multi": True}, + message_template="检测到单工单包含 {count} 条 DDL 语句，建议不超过 {max} 条", +)) +def check_pg_ddl_011(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许单工单多条DDL（但建议不超过限制）。""" + ddl_patterns = [ + r"\bCREATE\s+(?:TABLE|INDEX|SEQUENCE|VIEW|MATERIALIZED\s+VIEW|TRIGGER|FUNCTION)\b", + r"\bALTER\s+TABLE\b", + r"\bDROP\s+(?:TABLE|INDEX|SEQUENCE|VIEW)\b", + ] + count = sum(len(re.findall(pat, sql_upper, re.IGNORECASE)) for pat in ddl_patterns) + max_ddl = rule.config.get("max_ddl_per_ticket", 5) + if count > max_ddl: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(count=count, max=max_ddl), + )) + + +# ── PG-DDL-012 ── +@register_rule(AuditRuleV2( + code="PG-DDL-012", name="PGDDLAllowColumnType", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="ALTER TABLE ALTER TYPE 必须带 USING 子句", + message_template="字段类型转换 ALTER TYPE 缺少 USING 子句，可能导致数据丢失", +)) +def check_pg_ddl_012(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许字段类型转换（USING子句）。""" + if not re.search(r"\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bALTER\s+(?:COLUMN\s+)?\w+\s+TYPE\b", sql_upper, re.IGNORECASE): + if "USING" not in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DDL-013 ── +@register_rule(AuditRuleV2( + code="PG-DDL-013", name="PGAllowCreateView", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="检查 MATERIALIZED VIEW 的刷新策略", + config={"allow_materialized_view": True, "refresh_strategy": "CONCURRENTLY"}, + message_template="创建 MATERIALIZED VIEW 时建议使用 REFRESH MATERIALIZED VIEW CONCURRENTLY", +)) +def check_pg_ddl_013(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许创建视图（给出 MATERIALIZED VIEW 建议）。""" + if not re.search(r"\bCREATE\s+(?:MATERIALIZED\s+)?VIEW\b", sql_upper, re.IGNORECASE): + return + if "MATERIALIZED" in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DDL-014 ── +@register_rule(AuditRuleV2( + code="PG-DDL-014", name="PGDDLEnableForeignKey", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="外键约束会导致共享行锁，影响并发性能", + config={"allow_foreign_key": True, "warn_about_lock": True}, + message_template="外键约束(REFERENCES)在 PG 中可能导致共享行锁，影响并发写入性能，建议应用层控制", +)) +def check_pg_ddl_014(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许外键（但警告PG外键有共享锁问题）。""" + if not re.search(r"\bCREATE\s+TABLE\b|\bALTER\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bREFERENCES\s+\w+", sql_upper, re.IGNORECASE) or "FOREIGN KEY" in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DDL-015 ── +@register_rule(AuditRuleV2( + code="PG-DDL-015", name="PGAllowCreatePartition", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="检查分区表创建规范（Declarative Partitioning）", + config={"supported_partition_types": ["RANGE", "LIST", "HASH"]}, + message_template="分区表建议使用 PARTITION BY {supported_types}，并确保有分区键索引", +)) +def check_pg_ddl_015(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许创建分区表（Declarative Partitioning）。""" + if not re.search(r"\bCREATE\s+TABLE\b", sql_upper, re.IGNORECASE): + return + if "PARTITION BY" in sql_upper: + supported = rule.config.get("supported_partition_types", ["RANGE", "LIST", "HASH"]) + if not any(pt in sql_upper for pt in supported): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(supported_types="/".join(supported)), + )) + + +# ── PG-DDL-016 ── +@register_rule(AuditRuleV2( + code="PG-DDL-016", name="PGDDLTablePrefix", + category=RuleCategory.DDL, severity=Severity.INFO, db_type="postgres", + description="表名前缀规范检查", + config={"required_prefixes": []}, + message_template="表名 '{table}' 不符合前缀规范，建议前缀: {prefixes}", +)) +def check_pg_ddl_016(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """表名前缀规范。""" + required = rule.config.get("required_prefixes", []) + if not required: + return + table_name = _extract_pg_table_name(sql_upper, sql_clean) + if not table_name or not any(table_name.startswith(p) for p in required): + if table_name: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(table=table_name, prefixes=", ".join(required)), + )) + + +# ── PG-DDL-017 ── +@register_rule(AuditRuleV2( + code="PG-DDL-017", name="PGSupportEncoding", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="数据库/表应使用 UTF8 编码", + config={"allowed_encodings": ["UTF8"]}, + message_template="编码设置 '{encoding}' 不在推荐列表中，建议使用 UTF8", +)) +def check_pg_ddl_017(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许的编码（UTF8推荐）。""" + enc_match = re.search(r"\bENCODING\s*[=:]\s*['\"](\w+)['\"]|\bENCODING\s+(\w+)", sql_upper, re.IGNORECASE) + if enc_match: + encoding = enc_match.group(1) or enc_match.group(2) + allowed = rule.config.get("allowed_encodings", ["UTF8"]) + if encoding and encoding.upper() not in [a.upper() for a in allowed]: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(encoding=encoding), + )) + + +# ── PG-DDL-018 ── +@register_rule(AuditRuleV2( + code="PG-DDL-018", name="PGMaxTableNameLen", + category=RuleCategory.DDL, severity=Severity.WARNING, db_type="postgres", + description="PostgreSQL 表名最大长度限制为 63 字节（NAMEDATALEN - 1）", + config={"max_name_bytes": 63}, + message_template="表名 '{table}' 长度为 {length} 字节，超过 PG 限制 {max} 字节，超长部分将被截断", +)) +def check_pg_ddl_018(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """表名最大长度（PG限制63字节）。""" + table_name = _extract_pg_table_name(sql_upper, sql_clean) + if not table_name: + return + name_bytes = len(table_name.encode("utf-8")) + max_bytes = rule.config.get("max_name_bytes", 63) + if name_bytes > max_bytes: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(table=table_name, length=name_bytes, max=max_bytes), + )) + + +# ═══════════════════════════════════════════════ +# DML 规则 (PG-DML-001 ~ PG-DML-008, 共8条) +# ═══════════════════════════════════════════════ + + +# ── PG-DML-001 ── +@register_rule(AuditRuleV2( + code="PG-DML-001", name="PGDMLMaxInsertRows", + category=RuleCategory.DML, severity=Severity.WARNING, db_type="postgres", + description="INSERT 语句最大行数限制", + config={"max_insert_rows": 1000}, + message_template="单次 INSERT 包含 {count} 行，超过最大限制 {max}，请分批插入", +)) +def check_pg_dml_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Insert最大行数上限。""" + if not re.search(r"\bINSERT\b", sql_upper, re.IGNORECASE): + return + max_rows = rule.config.get("max_insert_rows", 1000) + values_match = re.search(r"\bVALUES\s+(.+)", sql_upper, re.IGNORECASE | re.DOTALL) + if values_match: + values_part = values_match.group(1) + row_count = 0 + depth = 0 + for char in values_part: + if char == "(": + if depth == 0: + row_count += 1 + depth += 1 + elif char == ")": + depth -= 1 + if row_count > max_rows: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(count=row_count, max=max_rows), + )) + + +# ── PG-DML-002 ── +@register_rule(AuditRuleV2( + code="PG-DML-002", name="PGDMLAllowLimitSTMT", + category=RuleCategory.DML, severity=Severity.INFO, db_type="postgres", + description="PostgreSQL 原生支持 UPDATE/DELETE 带 LIMIT，但需谨慎使用", + config={"allow_limit": True, "warn_when_used": True}, + message_template="UPDATE/DELETE 使用了 LIMIT，可能导致部分数据未处理，建议配合 ORDER BY 使用", +)) +def check_pg_dml_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许UPDATE/DELETE带LIMIT（PG原生支持，但给出警告）。""" + for keyword in ("UPDATE", "DELETE"): + if re.match(rf"\s*{keyword}\b", sql_upper, re.IGNORECASE): + if re.search(r"\bLIMIT\s+\d+", sql_upper, re.IGNORECASE): + if rule.config.get("warn_when_used", True): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + break + + +# ── PG-DML-003 ── +@register_rule(AuditRuleV2( + code="PG-DML-003", name="PGDMLAllowInsertNull", + category=RuleCategory.DML, severity=Severity.INFO, db_type="postgres", + description="允许插入 NULL，但建议显式使用 DEFAULT", + config={"allow_null": True, "prefer_default": True}, + message_template="INSERT 包含 NULL 值，建议改用 DEFAULT 关键字以保持语义清晰", +)) +def check_pg_dml_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """允许插入Null（但建议使用DEFAULT）。""" + if not re.search(r"\bINSERT\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bVALUES\b.*\bNULL\b", sql_upper, re.IGNORECASE): + if rule.config.get("prefer_default", True): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DML-004 ── +@register_rule(AuditRuleV2( + code="PG-DML-004", name="PGDMLWhere", + category=RuleCategory.DML, severity=Severity.ERROR, db_type="postgres", + description="UPDATE/DELETE 必须包含 WHERE 条件", + message_template="{operation} 语句缺少 WHERE 条件，将操作全表数据，极其危险", +)) +def check_pg_dml_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """强制DML必须拥有WHERE。""" + for keyword in ("UPDATE", "DELETE"): + if re.match(rf"\s*{keyword}\b", sql_upper, re.IGNORECASE): + if "WHERE" not in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(operation=keyword), + )) + + +# ── PG-DML-005 ── +@register_rule(AuditRuleV2( + code="PG-DML-005", name="PGDMLReturning", + category=RuleCategory.DML, severity=Severity.INFO, db_type="postgres", + description="推荐 DML 使用 RETURNING 子句获取操作结果", + config={"recommend_returning": True}, + message_template="DML 语句建议使用 RETURNING 子句获取操作后的值（如 RETURNING id）", +)) +def check_pg_dml_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐DML使用RETURNING子句。""" + for keyword in ("INSERT", "UPDATE", "DELETE"): + if re.match(rf"\s*{keyword}\b", sql_upper, re.IGNORECASE): + if "RETURNING" not in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + break + + +# ── PG-DML-006 ── +@register_rule(AuditRuleV2( + code="PG-DML-006", name="PGDMLCTE", + category=RuleCategory.DML, severity=Severity.INFO, db_type="postgres", + description="CTE (Common Table Expression) 使用规范", + config={"max_cte_depth": 3, "warn_recursive": True}, + message_template="CTE 使用规范: {cte_msg}", +)) +def check_pg_dml_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """CTE使用规范。""" + if not re.search(r"\bWITH\s+\w+\s+AS\s*\(", sql_upper, re.IGNORECASE): + return + if rule.config.get("warn_recursive", True) and re.search(r"\bWITH\s+RECURSIVE\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=Severity.WARNING, + message="递归 CTE (WITH RECURSIVE) 可能导致无限循环，请确保有终止条件", + )) + cte_count = len(re.findall(r"\bWITH\b", sql_upper, re.IGNORECASE)) + max_depth = rule.config.get("max_cte_depth", 3) + if cte_count > max_depth: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(cte_msg=f"检测到 {cte_count} 层嵌套 CTE，建议不超过 {max_depth} 层"), + )) + + +# ── PG-DML-007 ── +@register_rule(AuditRuleV2( + code="PG-DML-007", name="PGMaxAffectRows", + category=RuleCategory.DML, severity=Severity.WARNING, db_type="postgres", + description="DML 最大影响行数（通过 LIMIT 或 WHERE 范围控制）", + config={"max_affected_rows": 10000}, + message_template="UPDATE/DELETE 缺少行数限制，可能影响超过 {max} 行数据，建议添加 LIMIT 或更精确的 WHERE", +)) +def check_pg_dml_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """DML最大影响行数。""" + max_rows = rule.config.get("max_affected_rows", 10000) + for keyword in ("UPDATE", "DELETE"): + if re.match(rf"\s*{keyword}\b", sql_upper, re.IGNORECASE): + has_limit = re.search(r"\bLIMIT\s+\d+", sql_upper, re.IGNORECASE) is not None + has_pk = re.search(r"\bid\s*[=<>]|\b\w+_id\s*[=<>]", sql_upper, re.IGNORECASE) is not None + if not has_limit and not has_pk and "WHERE" in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(max=max_rows), + )) + break + + +# ── PG-DML-008 ── +@register_rule(AuditRuleV2( + code="PG-DML-008", name="PGDDLImplicitTypeConversion", + category=RuleCategory.DML, severity=Severity.WARNING, db_type="postgres", + description="禁止 WHERE 条件中的隐式类型转换，会导致索引失效", + message_template="WHERE 条件中字段 '{column}' 与值之间存在隐式类型转换风险，请使用显式转换 ::TYPE", +)) +def check_pg_dml_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """不允许隐式类型转换。""" + if "WHERE" not in sql_upper: + return + where_match = re.search(r"\bWHERE\b(.+?)(?:GROUP|HAVING|ORDER|LIMIT|RETURNING|;|$)", sql_upper, re.IGNORECASE | re.DOTALL) + if not where_match: + return + where_clause = where_match.group(1) + patterns = [ + r"(\w+)\s*[=<>]\s*'\d+'", + r"(\w+)\s*=\s*NULL", + ] + for pat in patterns: + matches = re.findall(pat, where_clause, re.IGNORECASE) + if matches: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(column=matches[0]), + )) + break + + +# ═══════════════════════════════════════════════ +# DQL 规则 (PG-DQL-001 ~ PG-DQL-006, 共6条) +# ═══════════════════════════════════════════════ + + +# ── PG-DQL-001 ── +@register_rule(AuditRuleV2( + code="PG-DQL-001", name="PGDQLSelectStar", + category=RuleCategory.DQL, severity=Severity.WARNING, db_type="postgres", + description="SELECT * 返回不必要的列，影响网络传输和内存", + message_template="禁止使用 SELECT *，请明确指定需要的字段（COUNT(*) 除外）", +)) +def check_pg_dql_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """SELECT * 检测。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bSELECT\s+\*\s+FROM\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DQL-002 ── +@register_rule(AuditRuleV2( + code="PG-DQL-002", name="PGDQLLimitOffset", + category=RuleCategory.DQL, severity=Severity.INFO, db_type="postgres", + description="大表分页建议使用游标分页替代 LIMIT/OFFSET", + config={"max_offset_rows": 100000}, + message_template="使用 LIMIT/OFFSET 在大表上性能较差（OFFSET 越大越慢），建议改用游标分页 (WHERE id > ? LIMIT N) 或 keyset pagination", +)) +def check_pg_dql_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐LIMIT+OFFSET分页优化。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + if re.search(r"\bOFFSET\s+\d+\b", sql_upper, re.IGNORECASE): + offset_match = re.search(r"\bOFFSET\s+(\d+)", sql_upper, re.IGNORECASE) + if offset_match: + offset_val = int(offset_match.group(1)) + max_offset = rule.config.get("max_offset_rows", 100000) + if offset_val > max_offset: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=Severity.WARNING, + message=f"OFFSET {offset_val} 过大，超过建议值 {max_offset}，强烈建议改用游标分页", + )) + else: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DQL-003 ── +@register_rule(AuditRuleV2( + code="PG-DQL-003", name="PGDQLWindowFunction", + category=RuleCategory.DQL, severity=Severity.INFO, db_type="postgres", + description="Window 函数使用检查，确保有合适的索引支持", + config={"warn_no_partition": True}, + message_template="Window 函数 ({functions}) 使用 detected，建议确保 PARTITION BY / ORDER BY 列有索引支持", +)) +def check_pg_dql_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Window函数使用检查。""" + window_funcs = [ + "ROW_NUMBER", "RANK", "DENSE_RANK", "NTILE", + "LAG", "LEAD", "FIRST_VALUE", "LAST_VALUE", + "NTH_VALUE", "PERCENT_RANK", "CUME_DIST", + ] + found_funcs = [f for f in window_funcs if re.search(rf"\b{f}\s*\(", sql_upper, re.IGNORECASE)] + if found_funcs: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(functions=", ".join(found_funcs)), + )) + + +# ── PG-DQL-004 ── +@register_rule(AuditRuleV2( + code="PG-DQL-004", name="PGDQLRegex", + category=RuleCategory.DQL, severity=Severity.WARNING, db_type="postgres", + description="~ 操作符（正则表达式匹配）性能警告，无法使用索引", + config={"prefer_prefix_regex": True}, + message_template="正则表达式匹配 (~ / ~*) 无法使用 B-Tree 索引，性能较差。建议: 1) 使用前缀匹配 ^ 配合 text_pattern_ops 索引; 2) 改用 pg_trgm 的 gin/gist 索引", +)) +def check_pg_dql_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """~操作符性能警告。""" + if re.search(r"\w+\s*~\*?\s*['\"]", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DQL-005 ── +@register_rule(AuditRuleV2( + code="PG-DQL-005", name="PGDQLConcurrentIndex", + category=RuleCategory.DQL, severity=Severity.INFO, db_type="postgres", + description="生产环境创建索引应使用 CREATE INDEX CONCURRENTLY 避免锁表", + config={"require_concurrently": True}, + message_template="生产环境建议 CREATE INDEX CONCURRENTLY 代替 CREATE INDEX，避免阻塞写入", +)) +def check_pg_dql_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐CREATE INDEX CONCURRENTLY。""" + if not re.search(r"\bCREATE\s+INDEX\b", sql_upper, re.IGNORECASE): + return + if "CONCURRENTLY" not in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ── PG-DQL-006 ── +@register_rule(AuditRuleV2( + code="PG-DQL-006", name="PGDQLAnalyzeVacuum", + category=RuleCategory.DQL, severity=Severity.INFO, db_type="postgres", + description="大批量数据变更后应执行 ANALYZE 更新统计信息", + config={"recommend_analyze": True}, + message_template="大批量数据操作后建议执行 ANALYZE table_name 更新统计信息，帮助优化器生成正确执行计划", +)) +def check_pg_dql_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """检查ANALYZE更新统计信息。""" + is_bulk = False + if re.search(r"\bINSERT\b", sql_upper, re.IGNORECASE): + values_match = re.search(r"\bVALUES\s+(.+)", sql_upper, re.IGNORECASE | re.DOTALL) + if values_match: + values_part = values_match.group(1) + row_count = 0 + depth = 0 + for char in values_part: + if char == "(": + if depth == 0: + row_count += 1 + depth += 1 + elif char == ")": + depth -= 1 + if row_count > 100: + is_bulk = True + if re.search(r"\bUPDATE\s+\w+\s+SET\b", sql_upper, re.IGNORECASE) and "WHERE" in sql_upper: + if not re.search(r"\bid\s*[=<>]", sql_upper, re.IGNORECASE) and "LIMIT" not in sql_upper: + is_bulk = True + if is_bulk: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, message=rule.message_template, + )) + + +# ═══════════════════════════════════════════════ +# 安全规则 (PG-SEC-001 ~ PG-SEC-006, 共6条) +# ═══════════════════════════════════════════════ + + +# ── PG-SEC-001 ── +@register_rule(AuditRuleV2( + code="PG-SEC-001", name="PGSecuritySuperuser", + category=RuleCategory.SECURITY, severity=Severity.ERROR, db_type="postgres", + description="禁止使用超级用户(superuser)执行日常操作", + config={"forbidden_roles": ["postgres", "rds_superuser"]}, + message_template="检测到超级用户操作 ({role})，禁止使用超级用户执行日常 SQL，应使用最小权限原则", +)) +def check_pg_sec_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止超级用户操作。""" + forbidden = rule.config.get("forbidden_roles", ["postgres", "rds_superuser"]) + superuser_patterns = [ + r"\bALTER\s+(?:ROLE|USER)\s+(\w+)\s+.*\bSUPERUSER\b", + r"\bCREATE\s+(?:ROLE|USER)\s+(\w+).*\bSUPERUSER\b", + r"\bSET\s+ROLE\s+(\w+)", + ] + for pat in superuser_patterns: + match = re.search(pat, sql_upper, re.IGNORECASE) + if match: + role = match.group(1) + if role in forbidden or "SUPERUSER" in sql_upper.upper(): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(role=role), + )) + break + + +# ── PG-SEC-002 ── +@register_rule(AuditRuleV2( + code="PG-SEC-002", name="PGSecurityFileAccess", + category=RuleCategory.SECURITY, severity=Severity.ERROR, db_type="postgres", + description="禁止调用 pg_read_file, pg_read_binary_file 等文件访问函数", + config={"forbidden_functions": [ + "pg_read_file", "pg_read_binary_file", "pg_ls_dir", + "pg_stat_file", "pg_read_file_all", + ]}, + message_template="禁止调用危险函数 '{func}'，该函数可访问服务器文件系统", +)) +def check_pg_sec_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止pg_read_file/pg_load。""" + forbidden = rule.config.get("forbidden_functions", []) + for func in forbidden: + if re.search(rf"\b{func}\s*\(", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(func=func), + )) + break + + +# ── PG-SEC-003 ── +@register_rule(AuditRuleV2( + code="PG-SEC-003", name="PGSecurityCopyFile", + category=RuleCategory.SECURITY, severity=Severity.ERROR, db_type="postgres", + description="COPY FROM/TO 可读写服务器文件系统，存在安全风险", + config={"allow_copy": False, "allowed_paths": []}, + message_template="COPY {direction} 可访问文件系统，存在安全风险。如需使用请确保文件路径受控", +)) +def check_pg_sec_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """COPY FROM/TO文件系统风险。""" + if rule.config.get("allow_copy", False): + return + copy_from = re.search(r"\bCOPY\b.*\bFROM\b", sql_upper, re.IGNORECASE) + copy_to = re.search(r"\bCOPY\b.*\bTO\b", sql_upper, re.IGNORECASE) + if copy_from: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(direction="FROM"), + )) + if copy_to: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(direction="TO"), + )) + + +# ── PG-SEC-004 ── +@register_rule(AuditRuleV2( + code="PG-SEC-004", name="PGSecurityExtension", + category=RuleCategory.SECURITY, severity=Severity.WARNING, db_type="postgres", + description="危险扩展检查（dblink, plpython 等可执行外部代码）", + config={ + "dangerous_extensions": [ + "dblink", "plpython3u", "plpython2u", "plperlu", "pltclu", + "file_fdw", "postgres_fdw", "pg_file_settings", + "adminpack", "auto_explain", + ], + "allowed_extensions": ["pgcrypto", "uuid-ossp", "pg_trgm"], + }, + message_template="扩展 '{ext}' 存在安全风险，请评估必要性: 可访问外部资源或执行外部代码", +)) +def check_pg_sec_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """危险扩展检查。""" + if not re.search(r"\bCREATE\s+EXTENSION\b", sql_upper, re.IGNORECASE): + return + dangerous = rule.config.get("dangerous_extensions", []) + allowed = rule.config.get("allowed_extensions", []) + for ext in dangerous: + if ext in allowed: + continue + if re.search(rf"\bCREATE\s+EXTENSION\s+(?:IF\s+NOT\s+EXISTS\s+)?{re.escape(ext)}\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(ext=ext), + )) + + +# ── PG-SEC-005 ── +@register_rule(AuditRuleV2( + code="PG-SEC-005", name="PGSecurityPassword", + category=RuleCategory.SECURITY, severity=Severity.ERROR, db_type="postgres", + description="密码复杂度检查", + config={ + "min_length": 12, + "require_special": True, + "require_upper": True, + "require_digit": True, + }, + message_template="密码不符合复杂度要求: 最少 {min_len} 位，需包含大小写字母、数字和特殊字符", +)) +def check_pg_sec_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """密码复杂度检查。""" + pwd_match = re.search( + r"\b(?:CREATE|ALTER)\s+(?:ROLE|USER)\s+\w+.*\bPASSWORD\s+['\"]([^'\"]+)['\"]", + sql_clean, re.IGNORECASE, + ) + if not pwd_match: + return + password = pwd_match.group(1) + min_len = rule.config.get("min_length", 12) + issues = [] + if len(password) < min_len: + issues.append(f"长度不足 {min_len} 位") + if rule.config.get("require_upper", True) and not re.search(r"[A-Z]", password): + issues.append("缺少大写字母") + if not re.search(r"[a-z]", password): + issues.append("缺少小写字母") + if rule.config.get("require_digit", True) and not re.search(r"\d", password): + issues.append("缺少数字") + if rule.config.get("require_special", True) and not re.search(r"[!@#$%^&*()_+\-=\[\]{};':\"|,.<>/?]", password): + issues.append("缺少特殊字符") + if issues: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=f"密码不符合复杂度要求: {', '.join(issues)}", + )) + + +# ── PG-SEC-006 ── +@register_rule(AuditRuleV2( + code="PG-SEC-006", name="PGSecurityListenNotify", + category=RuleCategory.SECURITY, severity=Severity.INFO, db_type="postgres", + description="LISTEN/NOTIFY 使用限制（防止 DoS 攻击）", + config={"max_channels_per_session": 10, "warn_payload_size": 8000}, + message_template="LISTEN/NOTIFY 使用 detected: {msg}", +)) +def check_pg_sec_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """LISTEN/NOTIFY使用限制。""" + has_listen = re.search(r"\bLISTEN\s+\w+", sql_upper, re.IGNORECASE) + has_notify = re.search(r"\bNOTIFY\s+\w+", sql_upper, re.IGNORECASE) + if has_listen: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg="LISTEN 通道在会话断开时自动取消订阅，建议配置 UNLISTEN 清理" + ), + )) + if has_notify: + payload_match = re.search(r"\bNOTIFY\s+\w+\s*,\s*['\"]([^'\"]+)['\"]", sql_clean, re.IGNORECASE) + if payload_match: + payload = payload_match.group(1) + max_size = rule.config.get("warn_payload_size", 8000) + if len(payload.encode("utf-8")) > max_size: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=Severity.WARNING, + message=f"NOTIFY payload 大小超过 {max_size} 字节（PG限制 8000 字节），建议减小消息体", + )) + + +# ═══════════════════════════════════════════════ +# 性能规则 (PG-PERF-001 ~ PG-PERF-007, 共7条) +# ═══════════════════════════════════════════════ + + +# ── PG-PERF-001 ── +@register_rule(AuditRuleV2( + code="PG-PERF-001", name="PGPerfVacuum", + category=RuleCategory.PERFORMANCE, severity=Severity.INFO, db_type="postgres", + description="大表 VACUUM 检查，建议避免 VACUUM FULL 阻塞操作", + config={"warn_vacuum_full": True, "recommend_autovacuum": True, "table_size_threshold_mb": 1000}, + message_template="VACUUM 操作建议: {msg}", +)) +def check_pg_perf_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """大表VACUUM检查。""" + if not re.search(r"\bVACUUM\b", sql_upper, re.IGNORECASE): + return + if rule.config.get("warn_vacuum_full", True) and re.search(r"\bVACUUM\s+FULL\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=Severity.WARNING, + message=rule.message_template.format( + msg="VACUUM FULL 会获取排他锁并重建整个表，大表操作会长时间阻塞读写。建议优先使用标准 VACUUM 或 pg_repack" + ), + )) + else: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg="建议启用 autovacuum 并合理配置阈值 (autovacuum_vacuum_scale_factor)，避免手动 VACUUM 遗漏" + ), + )) + + +# ── PG-PERF-002 ── +@register_rule(AuditRuleV2( + code="PG-PERF-002", name="PGPerfSeqScan", + category=RuleCategory.PERFORMANCE, severity=Severity.WARNING, db_type="postgres", + description="检测可能导致全表扫描的查询模式", + config={"min_table_size_for_warn": 10000}, + message_template="查询可能触发全表扫描: {reason}", +)) +def check_pg_perf_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """全表扫描检测（静态检查）。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + reasons = [] + if "WHERE" not in sql_upper and "LIMIT" not in sql_upper: + reasons.append("SELECT 缺少 WHERE 和 LIMIT 条件") + where_match = re.search(r"\bWHERE\b(.+?)(?:GROUP|ORDER|LIMIT|OFFSET|;|$)", sql_upper, re.IGNORECASE | re.DOTALL) + if where_match: + where_clause = where_match.group(1) + if re.search(r"\b\w+\s*\([^)]+\)\s*[=<>]", where_clause): + reasons.append("WHERE 子句中列上使用了函数，可能导致索引失效") + if re.search(r"\bLIKE\s+'%[^']+'", where_clause): + reasons.append("LIKE 前导通配符 % 无法使用索引") + if re.search(r"\b\w+\s+IS\s+NULL\b", where_clause): + reasons.append("IS NULL 条件可能无法使用标准 B-Tree 索引") + if re.search(r"\bILIKE\b", sql_upper, re.IGNORECASE): + reasons.append("ILIKE 无法使用 B-Tree 索引，建议使用 citext 类型或 pg_trgm 扩展") + if reasons: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(reason="; ".join(reasons)), + )) + + +# ── PG-PERF-003 ── +@register_rule(AuditRuleV2( + code="PG-PERF-003", name="PGPerfIndexBloat", + category=RuleCategory.PERFORMANCE, severity=Severity.INFO, db_type="postgres", + description="索引膨胀检测建议，REINDEX CONCURRENTLY 使用", + config={"bloat_threshold_pct": 50, "recommend_reindex_concurrently": True}, + message_template="索引维护建议: {msg}", +)) +def check_pg_perf_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """索引膨胀检测。""" + if re.search(r"\bREINDEX\s+INDEX\b", sql_upper, re.IGNORECASE) and "CONCURRENTLY" not in sql_upper: + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=Severity.WARNING, + message=rule.message_template.format( + msg="REINDEX 会锁表，生产环境建议使用 REINDEX TABLE CONCURRENTLY 或 REINDEX INDEX CONCURRENTLY (PG 12+)" + ), + )) + if re.search(r"\bCREATE\s+(?:UNIQUE\s+)?INDEX\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg="定期监控索引膨胀率 (pgstattuple 扩展)，膨胀超过 50% 时考虑 REINDEX CONCURRENTLY" + ), + )) + + +# ── PG-PERF-004 ── +@register_rule(AuditRuleV2( + code="PG-PERF-004", name="PGPerfLockTimeout", + category=RuleCategory.PERFORMANCE, severity=Severity.WARNING, db_type="postgres", + description="锁超时配置检查，长时间持有锁影响并发", + config={"recommended_lock_timeout_ms": 5000, "recommended_statement_timeout_ms": 30000}, + message_template="锁/超时配置建议: {msg}", +)) +def check_pg_perf_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """锁超时配置。""" + timeout_patterns = [ + (r"\bSET\s+lock_timeout\s*=\s*['\"]?(\d+\s*\w*)['\"]?", "lock_timeout"), + (r"\bSET\s+statement_timeout\s*=\s*['\"]?(\d+\s*\w*)['\"]?", "statement_timeout"), + ] + for pat, name in timeout_patterns: + match = re.search(pat, sql_upper, re.IGNORECASE) + if match: + value = match.group(1) + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format(msg=f"{name} 设置为 {value}，建议合理配置避免长时间锁等待"), + )) + if re.search(r"\bUPDATE\s+\w+\s+SET\b", sql_upper, re.IGNORECASE) and "WHERE" in sql_upper and "LIMIT" not in sql_upper: + recommended = rule.config.get("recommended_statement_timeout_ms", 30000) + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg=f"大批量操作建议在事务中设置 SET LOCAL statement_timeout = '{recommended}ms' 避免长时间持有锁" + ), + )) + + +# ── PG-PERF-005 ── +@register_rule(AuditRuleV2( + code="PG-PERF-005", name="PGPerfConnectionPool", + category=RuleCategory.PERFORMANCE, severity=Severity.INFO, db_type="postgres", + description="连接池使用建议", + config={"max_connections": 200, "recommended_pool_size": 20}, + message_template="连接管理建议: {msg}", +)) +def check_pg_perf_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """连接池使用建议。""" + if re.search(r"\bSET\s+(?:SESSION|LOCAL)\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg="频繁使用 SET SESSION/LOCAL 可能影响连接池复用，建议通过连接池统一配置参数" + ), + )) + if re.search(r"\bBEGIN\b", sql_upper, re.IGNORECASE) and not re.search(r"\bCOMMIT\b|\bROLLBACK\b", sql_upper, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=Severity.WARNING, + message=rule.message_template.format( + msg="检测到 BEGIN 但无 COMMIT/ROLLBACK，长时间打开的事务可能导致连接池耗尽和锁堆积" + ), + )) + + +# ── PG-PERF-006 ── +@register_rule(AuditRuleV2( + code="PG-PERF-006", name="PGPerfWorkMem", + category=RuleCategory.PERFORMANCE, severity=Severity.INFO, db_type="postgres", + description="work_mem 配置建议，影响排序和哈希操作性能", + config={"min_work_mem_mb": 4, "recommended_work_mem_mb": 16}, + message_template="内存配置建议: {msg}", +)) +def check_pg_perf_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """work_mem配置建议。""" + has_sort = False + has_hash = False + if "ORDER BY" in sql_upper: + order_cols = re.findall(r"\bORDER\s+BY\s+(.+?)(?:LIMIT|OFFSET|;|$)", sql_upper, re.IGNORECASE) + if order_cols and len(order_cols[0].split(",")) > 2: + has_sort = True + if "GROUP BY" in sql_upper: + has_hash = True + if re.search(r"\bDISTINCT\b", sql_upper, re.IGNORECASE): + has_sort = True + if "WITH" in sql_upper and ("JOIN" in sql_upper or "UNION" in sql_upper): + has_sort = True + if has_sort or has_hash: + rec_mem = rule.config.get("recommended_work_mem_mb", 16) + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg=f"复杂排序/哈希操作建议确保 work_mem >= {rec_mem}MB，或设置 SET LOCAL work_mem = '{rec_mem}MB'" + ), + )) + + +# ── PG-PERF-007 ── +@register_rule(AuditRuleV2( + code="PG-PERF-007", name="PGPerfParallelQuery", + category=RuleCategory.PERFORMANCE, severity=Severity.INFO, db_type="postgres", + description="并行查询使用规范（max_parallel_workers_per_gather）", + config={"max_parallel_workers": 4, "min_table_size_for_parallel": "8MB"}, + message_template="并行查询建议: {msg}", +)) +def check_pg_perf_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """并行查询使用规范。""" + if not re.search(r"\bSELECT\b", sql_upper, re.IGNORECASE): + return + is_complex = False + join_count = len(re.findall(r"\bJOIN\b", sql_upper, re.IGNORECASE)) + if join_count >= 2: + is_complex = True + if re.search(r"\bUNION\s+ALL?\b", sql_upper, re.IGNORECASE): + is_complex = True + if re.search(r"\bCOUNT\s*\(|\bSUM\s*\(|\bAVG\s*\(", sql_upper, re.IGNORECASE) and "GROUP BY" in sql_upper: + is_complex = True + if is_complex: + max_workers = rule.config.get("max_parallel_workers", 4) + report.findings.append(AuditFindingV2( + rule_code=rule.code, rule_name=rule.name, level=rule.severity, + message=rule.message_template.format( + msg=f"复杂查询可受益于并行查询，建议确认 max_parallel_workers_per_gather <= {max_workers}，并确保 min_parallel_table_scan_size 设置合理" + ), + )) + + +# ═══════════════════════════════════════════════ +# 注册表导出 +# ═══════════════════════════════════════════════ + +PG_RULE_REGISTRY: Dict[str, Tuple[AuditRuleV2, Callable]] = {} +"""PostgreSQL 特有规则注册表 (code -> (rule, check_func))。""" + + +def _init_pg_registry() -> None: + """初始化 PostgreSQL 规则注册表。""" + from app.services.audit_engine_v2 import _RULE_REGISTRY + global PG_RULE_REGISTRY + PG_RULE_REGISTRY = { + code: (rule, func) + for code, (rule, func) in _RULE_REGISTRY.items() + if code.startswith("PG-") + } + + +_init_pg_registry() + + +def get_pg_rules() -> List[AuditRuleV2]: + """获取所有 PostgreSQL 审核规则定义。""" + return sorted( + [rule for rule, _ in PG_RULE_REGISTRY.values()], + key=lambda r: r.code, + ) + + +def get_pg_rules_by_category(category: RuleCategory) -> List[AuditRuleV2]: + """按分类获取 PostgreSQL 审核规则。""" + return [rule for rule in get_pg_rules() if rule.category == category] + + +def get_pg_rule_count() -> Dict[str, int]: + """获取 PostgreSQL 规则统计信息。""" + rules = get_pg_rules() + return { + "total": len(rules), + "ddl": sum(1 for r in rules if r.category == RuleCategory.DDL), + "dml": sum(1 for r in rules if r.category == RuleCategory.DML), + "dql": sum(1 for r in rules if r.category == RuleCategory.DQL), + "security": sum(1 for r in rules if r.category == RuleCategory.SECURITY), + "performance": sum(1 for r in rules if r.category == RuleCategory.PERFORMANCE), + } + + +# ────────────────────────────────────────────── +# 模块加载自检 +# ────────────────────────────────────────────── +if __name__ == "__main__": + stats = get_pg_rule_count() + print("=" * 60) + print("PostgreSQL 审核规则引擎 - 规则统计") + print("=" * 60) + print(f"DDL 规则: {stats['ddl']} 条 (PG-DDL-001 ~ PG-DDL-018)") + print(f"DML 规则: {stats['dml']} 条 (PG-DML-001 ~ PG-DML-008)") + print(f"DQL 规则: {stats['dql']} 条 (PG-DQL-001 ~ PG-DQL-006)") + print(f"安全规则: {stats['security']} 条 (PG-SEC-001 ~ PG-SEC-006)") + print(f"性能规则: {stats['performance']} 条 (PG-PERF-001 ~ PG-PERF-007)") + print("-" * 60) + print(f"总计: {stats['total']} 条规则") + print("=" * 60) + print("\n规则明细:") + for rule in get_pg_rules(): + print(f" {rule.code} | {rule.severity.value:7s} | {rule.name}") diff --git a/backend/app/services/redis_audit_rules.py b/backend/app/services/redis_audit_rules.py new file mode 100644 index 0000000..9150f81 --- /dev/null +++ b/backend/app/services/redis_audit_rules.py @@ -0,0 +1,1863 @@ +"""Redis 审核规则引擎 — 43+ 条 Redis 特有的审核规则。 + +为 SQL 审核平台提供完整的 Redis 命令审核能力，覆盖: +- KEY 管理规则 (10条): Key命名规范、大小限制、过期策略 +- 命令安全规则 (12条): 危险命令检测（FLUSHDB/SHUTDOWN/CONFIG SET等） +- 数据结构规则 (8条): 各类型最佳实践与大小限制 +- 性能规则 (8条): 大Key检测、批量操作限制、阻塞命令 +- 安全规则 (5条): AUTH, ACL 配置与保护模式 + +使用方式: + engine = SQLAuditEngineV2(db_session) + report = await engine.audit("SET user:1001:name张三 EX3600", db_type="redis") + +规则编码规范: + KEY: REDIS-KEY-001 ~ REDIS-KEY-010 + CMD: REDIS-CMD-001 ~ REDIS-CMD-012 + DS: REDIS-DS-001 ~ REDIS-DS-008 + PERF: REDIS-PERF-001 ~ REDIS-PERF-008 + SEC: REDIS-SEC-001 ~ REDIS-SEC-005 +""" + +from __future__ import annotations + +import re +from typing import Any, Dict, List, Optional, Tuple + +from app.services.audit_engine_v2 import ( + AuditFindingV2, + AuditRuleV2, + AuditReportV2, + Severity, + RuleCategory, + register_rule, +) + + +# ═══════════════════════════════════════════════ +# Redis 辅助解析函数 +# ═══════════════════════════════════════════════ + + +def _parse_redis_command(sql_clean: str) -> Tuple[str, List[str]]: + """解析 Redis 命令，返回 (命令名大写, 参数列表)。 + + Args: + sql_clean: 单行 Redis 命令字符串 + + Returns: + (command_upper, args_list) 元组 + 例如: "SET key value EX 3600" -> ("SET", ["key", "value", "EX", "3600"]) + """ + parts = sql_clean.strip().split() + if not parts: + return "", [] + command = parts[0].upper() + args = parts[1:] if len(parts) > 1 else [] + return command, args + + +def _extract_key_from_command(command: str, args: List[str]) -> Optional[str]: + """从 Redis 命令中提取 Key 名称。 + + 大多数命令的第一个参数是 Key，部分命令需要特殊处理。 + + Args: + command: 命令名（大写） + args: 参数列表 + + Returns: + Key 名称，如果无法提取则返回 None + """ + if not args: + return None + + # 特殊命令：第二个参数才是 key + second_key_commands = { + "GETSET", "LPUSH", "RPUSH", "LPOP", "RPOP", "LINDEX", "LSET", + "LREM", "LINSERT", "SADD", "SREM", "SPOP", "SMOVE", "SISMEMBER", + "ZADD", "ZREM", "ZSCORE", "ZRANK", "ZREVRANK", "ZINCRBY", + "HSET", "HGET", "HDEL", "HINCRBY", "HINCRBYFLOAT", "HMSET", + "HMGET", "HSETNX", "HEXISTS", "HLEN", "HKEYS", "HVALS", + "HGETALL", "HSCAN", "HPOP", "SETEX", "PSETEX", "SETNX", + } + + # 多 key 命令 + multi_key_commands = { + "DEL", "UNLINK", "EXISTS", "TOUCH", "MGET", "MSET", "MSETNX", + "SDIFF", "SINTER", "SUNION", "SDIFFSTORE", "SINTERSTORE", "SUNIONSTORE", + "BLPOP", "BRPOP", "BRPOPLPUSH", "BLMOVE", "RPOPLPUSH", "LMOVE", + "RENAME", "RENAMENX", "SMOVE", "PFMERGE", "PFCOUNT", + "BZPOPMIN", "BZPOPMAX", "ZDIFF", "ZINTER", "ZUNION", + "ZDIFFSTORE", "ZINTERSTORE", "ZUNIONSTORE", + } + + if command in multi_key_commands and len(args) >= 1: + return args[0] + + if command in second_key_commands and len(args) >= 1: + return args[0] + + # 默认：第一个参数是 key + return args[0] + + +def _is_redis_command(sql_upper: str) -> bool: + """判断是否为 Redis 命令行（非空且非注释）。""" + stripped = sql_upper.strip() + return bool(stripped) and not stripped.startswith("#") + + +def _has_namespace_separator(key: str, separator: str = ":") -> bool: + """检查 Key 是否包含命名空间分隔符（默认冒号）。""" + return separator in key + + +def _count_namespaces(key: str, separator: str = ":") -> int: + """统计 Key 中命名空间层级数。""" + return key.count(separator) + 1 + + +def _extract_ttl_from_args(command: str, args: List[str]) -> Optional[int]: + """从命令参数中提取 TTL 值（秒）。 + + 检测 EX, PX, EXAT, PXAT 参数。 + + Returns: + TTL 秒数，如果没有找到则返回 None + """ + if not args: + return None + + cmd = command.upper() + + # SET ... EX seconds / PX milliseconds / EXAT timestamp / PXAT ms_timestamp + if cmd == "SET" or cmd.startswith("SET"): + for i, arg in enumerate(args): + arg_upper = arg.upper() + if arg_upper == "EX" and i + 1 < len(args): + try: + return int(args[i + 1]) + except (ValueError, TypeError): + return None + elif arg_upper == "PX" and i + 1 < len(args): + try: + return int(args[i + 1]) // 1000 + except (ValueError, TypeError): + return None + elif arg_upper in ("EXAT", "PXAT") and i + 1 < len(args): + return -1 # 绝对时间戳，视为已设置 TTL + return None + + # SETEX key seconds value / PSETEX key milliseconds value + if cmd == "SETEX" and len(args) >= 2: + try: + return int(args[1]) + except (ValueError, TypeError): + return None + if cmd == "PSETEX" and len(args) >= 2: + try: + return int(args[1]) // 1000 + except (ValueError, TypeError): + return None + + # EXPIRE key seconds / PEXPIRE key milliseconds / EXPIREAT key timestamp + if cmd in ("EXPIRE", "EXPIREAT") and len(args) >= 2: + try: + return int(args[1]) + except (ValueError, TypeError): + return None + if cmd == "PEXPIRE" and len(args) >= 2: + try: + return int(args[1]) // 1000 + except (ValueError, TypeError): + return None + + return None + + +def _is_ttl_set_in_command(sql_clean: str, command: str, args: List[str]) -> bool: + """检查命令是否设置了 TTL。""" + cmd = command.upper() + + # 直接设置 TTL 的命令 + ttl_commands = { + "EXPIRE", "PEXPIRE", "EXPIREAT", "PEXPIREAT", "SETEX", "PSETEX" + } + if cmd in ttl_commands: + return True + + # SET 命令带 EX/PX/EXAT/PXAT + if cmd in ("SET", "GETSET"): + for i, arg in enumerate(args): + if arg.upper() in ("EX", "PX", "EXAT", "PXAT"): + return True + + return False + + +def _is_write_command(command: str) -> bool: + """判断是否为写入类命令。""" + write_commands = { + "SET", "GETSET", "MSET", "MSETNX", "SETNX", "SETEX", "PSETEX", + "HSET", "HMSET", "HSETNX", "HDEL", "HINCRBY", "HINCRBYFLOAT", + "LPUSH", "RPUSH", "LPOP", "RPOP", "LSET", "LREM", "LINSERT", + "LTRIM", "LMOVE", "BLMOVE", "RPOPLPUSH", "BRPOPLPUSH", + "SADD", "SREM", "SPOP", "SMOVE", "SDIFFSTORE", "SINTERSTORE", "SUNIONSTORE", + "ZADD", "ZREM", "ZINCRBY", "ZREMRANGEBYRANK", "ZREMRANGEBYSCORE", + "ZREMRANGEBYLEX", "ZDIFFSTORE", "ZINTERSTORE", "ZUNIONSTORE", + "XADD", "XDEL", "XTRIM", "XGROUP", "XSETID", + "DEL", "UNLINK", "FLUSHDB", "FLUSHALL", + "APPEND", "INCR", "DECR", "INCRBY", "DECRBY", "INCRBYFLOAT", + "PFADD", "PFMERGE", "PFCOUNT", + "GEOADD", "GEORADIUS", "GEORADIUSBYMEMBER", + "BITSET", "SETBIT", "BITOP", + "RESTORE", "MIGRATE", "RENAME", "RENAMENX", + } + return command.upper() in write_commands + + +def _extract_pipeline_commands(sql_clean: str) -> List[str]: + """从 Pipeline 输入中提取每条命令。 + + Pipeline 命令通常用换行符分隔。 + """ + commands = [] + for line in sql_clean.split("\n"): + stripped = line.strip() + if stripped and not stripped.startswith("#"): + commands.append(stripped) + return commands + + +# ═══════════════════════════════════════════════ +# KEY 管理规则 (10条) +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="REDIS-KEY-001", + name="Key命名规范", + category=RuleCategory.NAMING, + severity=Severity.WARNING, + db_type="redis", + description="Key 建议使用冒号分隔的命名空间，如 user:1001:profile", + config={"namespace_separator": ":", "min_namespace_levels": 2}, + message_template="Key '{key}' 命名不符合规范，建议使用冒号分隔的命名空间（如 app:module:id）", +)) +def check_redis_key_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Key命名规范检查 - 建议使用冒号分隔的命名空间。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + key = _extract_key_from_command(command, args) + if not key: + return + separator = rule.config.get("namespace_separator", ":") + min_levels = rule.config.get("min_namespace_levels", 2) + if separator not in key or _count_namespaces(key, separator) < min_levels: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(key=key), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-002", + name="Key最大长度限制", + category=RuleCategory.NAMING, + severity=Severity.WARNING, + db_type="redis", + description="Key 名称最大长度建议不超过128字节", + config={"max_key_length": 128}, + message_template="Key '{key}' 长度为 {length} 字节，超过最大限制 {max} 字节", +)) +def check_redis_key_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Key最大长度限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + key = _extract_key_from_command(command, args) + if not key: + return + max_len = rule.config.get("max_key_length", 128) + key_bytes = len(key.encode("utf-8")) + if key_bytes > max_len: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(key=key, length=key_bytes, max=max_len), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-003", + name="Key值大小限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="String 值建议不超过 10KB，Hash 建议不超过 1000 字段", + config={"max_string_size": 10240, "max_hash_fields": 1000}, + message_template="{cmd} 操作可能导致大 Key 问题，String 值建议不超过 {max_str} 字节，Hash 建议不超过 {max_hash} 字段", +)) +def check_redis_key_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Key值大小限制检查 - 针对大 Value 的写入操作给出警告。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # 检测 HMSET/HSET 大量字段（超过阈值的一半即警告） + hash_cmds = {"HSET", "HMSET", "HGETALL"} + if cmd in hash_cmds and len(args) > 20: + max_hash = rule.config.get("max_hash_fields", 1000) + # HMSET key field1 value1 field2 value2 ... + # 字段数 = (args - 1) // 2 + field_count = (len(args) - 1) // 2 if cmd in ("HSET", "HMSET") else 0 + if field_count > max_hash // 2: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"HMSET/HSET 操作包含约 {field_count} 个字段，接近大 Key 阈值 {max_hash}，建议拆分 Hash 结构", + )) + return + + # 检测大 String 写入 + string_cmds = {"SET", "SETEX", "PSETEX", "APPEND", "GETSET"} + if cmd in string_cmds and len(args) >= 2: + max_str = rule.config.get("max_string_size", 10240) + # 找到 value 参数（通常是第2或第3个参数） + value = "" + if cmd in ("SETEX", "PSETEX") and len(args) >= 3: + value = args[2] + elif cmd == "APPEND" and len(args) >= 2: + value = args[1] + elif len(args) >= 2: + value = args[1] + value_bytes = len(value.encode("utf-8")) if value else 0 + if value_bytes > max_str: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"SET 操作的 Value 大小为 {value_bytes} 字节，超过建议阈值 {max_str} 字节（10KB）", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-004", + name="写入Key必须设置TTL", + category=RuleCategory.DDL, + severity=Severity.ERROR, + db_type="redis", + description="写入 Key 必须设置 TTL，不允许永久 Key 存在", + config={"whitelist_keys": []}, + message_template="写入 Key '{key}' 未设置 TTL，请添加 EX/PX 参数或单独执行 EXPIRE 命令，禁止创建永久 Key", +)) +def check_redis_key_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """写入 Key 必须设置 TTL 检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # 只对写入命令检查 + if not _is_write_command(cmd): + return + + key = _extract_key_from_command(command, args) + if not key: + return + + # 检查白名单 + whitelist = rule.config.get("whitelist_keys", []) + if any(key.startswith(w) for w in whitelist): + return + + # 检查是否已设置 TTL + if _is_ttl_set_in_command(sql_clean, cmd, args): + return + + # 排除一些天然带 TTL 的命令和一些特殊的命令 + ttl_builtin_cmds = {"SETEX", "PSETEX"} + if cmd in ttl_builtin_cmds: + return + + # DEL/UNLINK/FLUSH 等删除命令不需要 TTL + delete_cmds = {"DEL", "UNLINK", "FLUSHDB", "FLUSHALL", "EXPIRE", "PEXPIRE", + "EXPIREAT", "PEXPIREAT", "TTL", "PTTL", "PERSIST"} + if cmd in delete_cmds: + return + + # 自增类命令通常是更新已有 Key，不需要设置 TTL + incr_cmds = {"INCR", "DECR", "INCRBY", "DECRBY", "INCRBYFLOAT", + "HINCRBY", "HINCRBYFLOAT", "ZINCRBY"} + if cmd in incr_cmds: + return + + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(key=key), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-005", + name="默认TTL建议", + category=RuleCategory.DDL, + severity=Severity.INFO, + db_type="redis", + description="建议设置合理的默认 TTL，如 3600 秒（1小时）", + config={"recommended_default_ttl": 3600, "min_recommended_ttl": 60}, + message_template="建议为 Key '{key}' 设置合理的 TTL，默认推荐 {ttl} 秒（1小时），避免数据永久驻留", +)) +def check_redis_key_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """默认 TTL 建议 - 对未设置 TTL 或 TTL 过短的命令给出提示。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # 只检查写入命令 + if not _is_write_command(cmd): + return + + key = _extract_key_from_command(command, args) + if not key: + return + + ttl = _extract_ttl_from_args(cmd, args) + if ttl is None: + return # 已由 REDIS-KEY-004 检查 + + min_ttl = rule.config.get("min_recommended_ttl", 60) + if ttl > 0 and ttl < min_ttl: + default_ttl = rule.config.get("recommended_default_ttl", 3600) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"Key '{key}' 的 TTL 设置为 {ttl} 秒，过短可能导致频繁缓存失效，建议至少 {min_ttl} 秒或默认 {default_ttl} 秒", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-006", + name="最大TTL限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="Key 的 TTL 建议不超过 30 天（2592000 秒）", + config={"max_ttl_seconds": 2592000}, + message_template="Key '{key}' 的 TTL {ttl} 秒超过最大限制 {max} 秒（{days} 天），建议缩短过期时间", +)) +def check_redis_key_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """最大 TTL 限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + ttl = _extract_ttl_from_args(cmd, args) + if ttl is None or ttl <= 0: + return + + max_ttl = rule.config.get("max_ttl_seconds", 2592000) + if ttl > max_ttl: + key = _extract_key_from_command(command, args) or "unknown" + days = max_ttl // 86400 + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(key=key, ttl=ttl, max=max_ttl, days=days), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-007", + name="禁止使用KEYS命令", + category=RuleCategory.PERFORMANCE, + severity=Severity.ERROR, + db_type="redis", + description="KEYS 命令会扫描全库，阻塞服务器，必须使用 SCAN 替代", + message_template="禁止使用 KEYS 命令（'{cmd}'），会导致全库扫描阻塞服务器，请使用 SCAN 替代", +)) +def check_redis_key_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止使用 KEYS 命令。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + if command.upper() == "KEYS": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=sql_clean.strip()), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-008", + name="必须指定命名空间前缀", + category=RuleCategory.NAMING, + severity=Severity.WARNING, + db_type="redis", + description="Key 必须以指定的命名空间前缀开头，如 app:service:key", + config={"required_prefixes": ["app:", "svc:", "cache:", "session:", "mq:"]}, + message_template="Key '{key}' 缺少命名空间前缀，Key 必须以 {prefixes} 之一开头", +)) +def check_redis_key_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """必须指定命名空间前缀检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + key = _extract_key_from_command(command, args) + if not key: + return + + prefixes = rule.config.get("required_prefixes", ["app:", "svc:", "cache:", "session:", "mq:"]) + if not any(key.startswith(p) for p in prefixes): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(key=key, prefixes=", ".join(prefixes)), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-009", + name="禁止批量删除Key模式", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止使用 DEL key:* 等通配符批量删除 Key 的模式", + config={}, + message_template="禁止使用通配符批量删除 Key 的模式 '{cmd}'，请使用 SCAN + DEL 循环逐个删除", +)) +def check_redis_key_009(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止批量删除 Key 模式检查。 + + 检测使用通配符配合 DEL 的模式，如: + - DEL user:* + - EVAL "... KEYS[1] ..." 0 prefix:* + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # 检查 DEL/UNLINK 命令参数中包含通配符 * + if cmd in ("DEL", "UNLINK") and args: + for arg in args: + if "*" in arg or "?" in arg: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=sql_clean.strip()), + )) + return + + # 检查 EVAL/EVALSHA 脚本中的批量删除模式 + if cmd in ("EVAL", "EVALSHA") and args: + script_body = args[0].upper() + if "DEL" in script_body and ("KEYS" in script_body or "ARGV" in script_body): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"Lua 脚本 '{cmd}' 中可能包含批量删除操作，请确保使用 SCAN 逐个删除", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-KEY-010", + name="推荐高效编码", + category=RuleCategory.DDL, + severity=Severity.INFO, + db_type="redis", + description="建议开启 ziplist/intset 等高效编码以节省内存", + config={"ziplist_max_entries": 512, "ziplist_max_value": 64}, + message_template="建议配置 ziplist/intset 高效编码参数（hash-max-ziplist-entries/list-max-ziplist-entries/set-max-intset-entries），节省内存", +)) +def check_redis_key_010(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """推荐高效编码检查 - 检测 CONFIG SET 命令中关于编码的配置。 + + 当用户执行 CONFIG SET 修改编码相关参数时进行检查和提示。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd == "CONFIG" and len(args) >= 2: + subcmd = args[0].upper() + param = args[1].upper() + if subcmd == "SET" and "ziplist" in param.lower(): + try: + value = int(args[2]) if len(args) >= 3 else 0 + max_entries = rule.config.get("ziplist_max_entries", 512) + if value > max_entries: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"ziplist 条目数设置为 {value}，超过建议值 {max_entries}，过大可能导致性能下降", + )) + except (ValueError, TypeError, IndexError): + pass + + # 对于 Hash 操作，如果字段数较少但使用了大配置，给出提示 + if cmd in ("HSET", "HMSET") and len(args) >= 4: + field_count = (len(args) - 1) // 2 + max_entries = 512 + if field_count <= max_entries: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"Hash 字段数为 {field_count}，建议配置 hash-max-ziplist-entries >= {max_entries} 以启用 ziplist 编码节省内存", + )) + + +# ═══════════════════════════════════════════════ +# 命令安全规则 (12条) +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="REDIS-CMD-001", + name="禁止FLUSHDB", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 FLUSHDB 命令，将清空当前数据库所有数据", + message_template="禁止执行 FLUSHDB，该命令将清空当前数据库所有数据，如需清理请使用 SCAN + DEL 逐个删除", +)) +def check_redis_cmd_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 FLUSHDB。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "FLUSHDB": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-002", + name="禁止FLUSHALL", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 FLUSHALL 命令，将清空所有数据库数据", + message_template="禁止执行 FLUSHALL，该命令将清空所有数据库数据，属于高危操作", +)) +def check_redis_cmd_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 FLUSHALL。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "FLUSHALL": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-003", + name="禁止SHUTDOWN", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 SHUTDOWN 命令，将关闭 Redis 服务器", + message_template="禁止执行 SHUTDOWN，该命令将关闭 Redis 服务器，如需维护请联系 DBA", +)) +def check_redis_cmd_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 SHUTDOWN。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "SHUTDOWN": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-004", + name="禁止CONFIG SET", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 CONFIG SET 命令修改 Redis 配置", + message_template="禁止执行 CONFIG SET 修改配置，参数调整需通过运维流程审批后由 DBA 执行", +)) +def check_redis_cmd_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 CONFIG SET。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "CONFIG" and len(args) >= 1: + if args[0].upper() == "SET": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-005", + name="禁止DEBUG命令", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 DEBUG 命令（含 DEBUG SEGFAULT 等调试命令）", + message_template="禁止执行 DEBUG 命令（'{cmd}'），调试命令可能导致服务崩溃或性能问题", +)) +def check_redis_cmd_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 DEBUG 命令。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "DEBUG": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=sql_clean.strip()), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-006", + name="禁止REPLICAOF/SLAVEOF", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 REPLICAOF/SLAVEOF 命令修改主从配置", + message_template="禁止执行 {cmd} 修改主从复制配置，主从变更需通过运维流程审批", +)) +def check_redis_cmd_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 REPLICAOF/SLAVEOF。""" + command, args = _parse_redis_command(sql_clean) + cmd = command.upper() + if cmd in ("REPLICAOF", "SLAVEOF"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-007", + name="禁止MIGRATE", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 MIGRATE 命令进行数据迁移", + message_template="禁止执行 MIGRATE 数据迁移命令，数据迁移需通过 DBA 审批和协助", +)) +def check_redis_cmd_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 MIGRATE。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "MIGRATE": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-008", + name="禁止RESTORE", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 RESTORE 命令恢复序列化数据", + message_template="禁止执行 RESTORE 恢复数据命令，数据恢复需通过 DBA 审批和协助", +)) +def check_redis_cmd_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 RESTORE。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "RESTORE": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-009", + name="禁止EVAL/EVALSHA", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 EVAL/EVALSHA 运行 Lua 脚本（未授权脚本风险）", + config={"allowed_scripts": []}, + message_template="禁止执行 {cmd} 运行 Lua 脚本，未授权脚本可能导致性能问题或安全风险，请使用 Function 或提交审批", +)) +def check_redis_cmd_009(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 EVAL/EVALSHA。""" + command, args = _parse_redis_command(sql_clean) + cmd = command.upper() + if cmd in ("EVAL", "EVALSHA"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-010", + name="禁止MODULE LOAD", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 MODULE LOAD 加载外部模块（安全风险）", + message_template="禁止执行 MODULE LOAD 加载外部模块，模块加载需通过安全审批和代码审计", +)) +def check_redis_cmd_010(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 MODULE LOAD。""" + command, args = _parse_redis_command(sql_clean) + cmd = command.upper() + if cmd == "MODULE" and len(args) >= 1 and args[0].upper() == "LOAD": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-011", + name="禁止ACL SETUSER", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="禁止执行 ACL SETUSER 修改用户权限", + message_template="禁止执行 ACL SETUSER 修改用户权限，账户管理需通过运维流程", +)) +def check_redis_cmd_011(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止 ACL SETUSER。""" + command, args = _parse_redis_command(sql_clean) + cmd = command.upper() + if cmd == "ACL" and len(args) >= 1: + if args[0].upper() in ("SETUSER", "DELUSER", "LOAD", "SAVE", "GENPASS"): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-CMD-012", + name="限制MONITOR命令", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="MONITOR 命令会导致严重的性能影响，生产环境限制使用", + message_template="MONITOR 命令会导致严重的性能影响（所有命令输出到客户端），生产环境请限制使用", +)) +def check_redis_cmd_012(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """限制 MONITOR 命令。""" + command, args = _parse_redis_command(sql_clean) + if command.upper() == "MONITOR": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + + +# ═══════════════════════════════════════════════ +# 数据结构规则 (8条) +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="REDIS-DS-001", + name="String值最大大小限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="String 类型值大小建议不超过 10MB", + config={"max_string_size_mb": 10, "max_string_size_bytes": 10485760}, + message_template="String 值大小约为 {size} 字节，超过建议阈值 {max} 字节（{max_mb}MB），建议使用 Hash/List 拆分存储", +)) +def check_redis_ds_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """String 值最大大小限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + string_write_cmds = {"SET", "SETEX", "PSETEX", "GETSET", "APPEND"} + if cmd not in string_write_cmds or len(args) < 2: + return + + value = "" + if cmd in ("SETEX", "PSETEX") and len(args) >= 3: + value = args[2] + elif cmd in ("SET", "GETSET", "APPEND") and len(args) >= 2: + value = args[1] + + max_bytes = rule.config.get("max_string_size_bytes", 10485760) + value_bytes = len(value.encode("utf-8")) if value else 0 + if value_bytes > max_bytes: + max_mb = rule.config.get("max_string_size_mb", 10) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(size=value_bytes, max=max_bytes, max_mb=max_mb), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-DS-002", + name="Hash最大字段数限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="Hash 类型字段数建议不超过 10000", + config={"max_hash_fields": 10000}, + message_template="Hash 字段数 {fields} 超过建议阈值 {max}，请拆分 Hash 或使用多个 Key 存储", +)) +def check_redis_ds_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Hash 最大字段数限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd in ("HMSET", "HSET") and len(args) >= 3: + field_count = (len(args) - 1) // 2 + max_fields = rule.config.get("max_hash_fields", 10000) + if field_count > max_fields: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(fields=field_count, max=max_fields), + )) + + # HGETALL 读取大 Hash 也警告 + if cmd == "HGETALL" and len(args) >= 1: + max_fields = rule.config.get("max_hash_fields", 10000) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"HGETALL 读取 Hash '{args[0]}' 可能返回大量字段，建议使用 HSCAN 分批读取，避免阻塞", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-DS-003", + name="List最大长度限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="List 类型长度建议不超过 100000", + config={"max_list_length": 100000}, + message_template="List 长度可能超过建议阈值 {max}，请使用 LTRIM 控制长度或使用 Stream 替代", +)) +def check_redis_ds_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """List 最大长度限制检查。 + + 检测 List 写入命令，当未带 LTRIM/XTRIM 等长度控制时警告。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + list_push_cmds = {"LPUSH", "RPUSH", "LPUSHX", "RPUSHX"} + if cmd in list_push_cmds and len(args) >= 2: + max_len = rule.config.get("max_list_length", 100000) + # 大量元素一次性 LPUSH/RPUSH + element_count = len(args) - 1 # 减去 key + if element_count > 100: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"一次性 {cmd} 插入 {element_count} 个元素，List 长度可能增长过快，建议使用 LTRIM 控制长度不超过 {max_len}", + )) + + # LRANGE 大跨度读取 + if cmd == "LRANGE" and len(args) >= 3: + try: + start = int(args[1]) + end = int(args[2]) + range_size = abs(end - start) + 1 if end >= 0 else abs(start) + abs(end) + max_len = rule.config.get("max_list_length", 100000) + if range_size > max_len // 10: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"LRANGE 范围 {start} ~ {end} 涉及约 {range_size} 个元素，范围过大可能导致阻塞，建议分页读取", + )) + except (ValueError, TypeError): + pass + + +@register_rule(AuditRuleV2( + code="REDIS-DS-004", + name="Set最大成员数限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="Set 类型成员数建议不超过 100000", + config={"max_set_members": 100000}, + message_template="Set 成员数 {members} 超过建议阈值 {max}，请拆分 Set 或使用 BloomFilter 替代", +)) +def check_redis_ds_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Set 最大成员数限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # SADD 大量成员 + if cmd == "SADD" and len(args) > 100: + member_count = len(args) - 1 + max_members = rule.config.get("max_set_members", 100000) + if member_count > max_members // 10: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"SADD 一次性添加 {member_count} 个成员，Set 过大（>{max_members}）会影响性能，建议拆分", + )) + + # SMEMBERS 全量读取大 Set 警告 + if cmd == "SMEMBERS" and len(args) >= 1: + max_members = rule.config.get("max_set_members", 100000) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"SMEMBERS 全量读取 Set '{args[0]}' 可能返回大量数据，建议使用 SSCAN 分批扫描（成员数建议不超过 {max_members}）", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-DS-005", + name="ZSet最大成员数限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="ZSet 类型成员数建议不超过 100000", + config={"max_zset_members": 100000}, + message_template="ZSet 成员数超过建议阈值 {max}，请拆分 ZSet 或优化数据模型", +)) +def check_redis_ds_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """ZSet 最大成员数限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # ZADD 大量成员 + if cmd == "ZADD" and len(args) > 200: + member_count = (len(args) - 1) // 2 + max_members = rule.config.get("max_zset_members", 100000) + if member_count > max_members // 10: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"ZADD 一次性添加 {member_count} 个成员，ZSet 过大（>{max_members}）会影响排序性能，建议拆分", + )) + + # ZRANGE/ZREVRANGE 大跨度读取 + if cmd in ("ZRANGE", "ZREVRANGE") and len(args) >= 3: + try: + start = int(args[1]) + end = int(args[2]) + if end < 0 or (end - start) > 10000: + max_members = rule.config.get("max_zset_members", 100000) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"{cmd} 大范围读取（{start}~{end}）可能阻塞，建议分页（每次不超过 1000）或限制 ZSet 大小（<{max_members}）", + )) + except (ValueError, TypeError): + pass + + +@register_rule(AuditRuleV2( + code="REDIS-DS-006", + name="Stream最大长度限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="Stream 类型必须限制最大长度（使用 MAXLEN），建议不超过 100000", + config={"max_stream_length": 100000}, + message_template="Stream 写入命令 '{cmd}' 未使用 MAXLEN 限制长度，请添加 MAXLEN ~ {max} 防止内存无限增长", +)) +def check_redis_ds_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Stream 最大长度限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd == "XADD" and len(args) >= 2: + has_maxlen = any(arg.upper() == "MAXLEN" for arg in args) + if not has_maxlen: + max_len = rule.config.get("max_stream_length", 100000) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=sql_clean.strip(), max=max_len), + )) + + # XTRIM 命令也需要检查 + if cmd == "XTRIM" and len(args) >= 1: + has_maxlen = any(arg.upper() == "MAXLEN" for arg in args) + if not has_maxlen: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message="XTRIM 命令未使用 MAXLEN 参数，请确保使用正确的修剪策略", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-DS-007", + name="Bitmap最大大小限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="Bitmap 类型大小建议不超过 512MB（offset < 2^32）", + config={"max_bitmap_size_mb": 512, "max_bitmap_offset": 4294967296}, + message_template="Bitmap offset {offset} 超过建议阈值 {max_offset}（{max_mb}MB），Bitmap 过大消耗大量内存", +)) +def check_redis_ds_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Bitmap 最大大小限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + bitmap_cmds = {"SETBIT", "GETBIT", "BITPOS", "BITCOUNT", "BITOP"} + if cmd not in bitmap_cmds: + return + + max_offset = rule.config.get("max_bitmap_offset", 4294967296) + max_mb = rule.config.get("max_bitmap_size_mb", 512) + + # SETBIT key offset value + if cmd == "SETBIT" and len(args) >= 2: + try: + offset = int(args[1]) + if offset > max_offset: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(offset=offset, max_offset=max_offset, max_mb=max_mb), + )) + except (ValueError, TypeError): + pass + + # BITOP 操作可能产生大 Bitmap + if cmd == "BITOP" and len(args) >= 3: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"BITOP 操作涉及 {len(args) - 2} 个 Bitmap，请确保结果 Bitmap 大小不超过 {max_mb}MB", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-DS-008", + name="Geo最大成员数限制", + category=RuleCategory.DDL, + severity=Severity.WARNING, + db_type="redis", + description="Geo 类型成员数建议不超过 10000000", + config={"max_geo_members": 10000000}, + message_template="Geo 集合成员数 {members} 超过建议阈值 {max}，大集合 GEOSEARCH/GEORADIUS 性能会显著下降", +)) +def check_redis_ds_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Geo 最大成员数限制检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # GEOADD 大量成员 + if cmd == "GEOADD" and len(args) > 300: + member_count = (len(args) - 1) // 3 + max_members = rule.config.get("max_geo_members", 10000000) + if member_count > max_members // 100: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"GEOADD 一次性添加 {member_count} 个地理坐标，Geo 集合过大（>{max_members}）会显著降低查询性能，建议按区域分集合存储", + )) + + # GEORADIUS/GEOSEARCH 大范围查询 + if cmd in ("GEORADIUS", "GEORADIUSBYMEMBER", "GEOSEARCH"): + if len(args) >= 4: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"{cmd} 地理范围查询可能返回大量结果，建议使用 COUNT 参数限制返回数量", + )) + + +# ═══════════════════════════════════════════════ +# 性能规则 (8条) +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="REDIS-PERF-001", + name="禁止阻塞命令", + category=RuleCategory.PERFORMANCE, + severity=Severity.ERROR, + db_type="redis", + description="禁止使用 BLPOP/BRPOP/BRPOPLPUSH/BLMOVE 等阻塞命令", + message_template="禁止执行阻塞命令 '{cmd}'，会导致客户端连接长时间阻塞，请使用 LPOP/RPOP/LMOVE 非阻塞版本", +)) +def check_redis_perf_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止阻塞命令检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + blocking_cmds = {"BLPOP", "BRPOP", "BRPOPLPUSH", "BLMOVE", "BZPOPMIN", "BZPOPMAX"} + if cmd in blocking_cmds: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-002", + name="SORT命令必须带LIMIT", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="SORT 命令必须带 LIMIT 参数，防止返回过多数据", + message_template="SORT 命令未带 LIMIT 参数，可能返回大量数据导致阻塞，请添加 LIMIT offset count", +)) +def check_redis_perf_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """SORT 命令必须带 LIMIT 检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd == "SORT" and len(args) >= 1: + has_limit = any(arg.upper() == "LIMIT" for arg in args) + if not has_limit: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-003", + name="Pipeline批量限制", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="Pipeline 批量命令数建议不超过 1000 条", + config={"max_pipeline_commands": 1000}, + message_template="Pipeline 包含 {count} 条命令，超过建议阈值 {max} 条，请分批执行（建议每批不超过 {max} 条）", +)) +def check_redis_perf_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Pipeline 批量限制检查。 + + 对多行输入的命令进行计数。 + """ + commands = _extract_pipeline_commands(sql_clean) + max_cmds = rule.config.get("max_pipeline_commands", 1000) + if len(commands) > max_cmds: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(count=len(commands), max=max_cmds), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-004", + name="大Key内存使用检测", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="检测可能产生大 Key 的操作，建议定期执行 MEMORY USAGE 检查", + config={"big_key_threshold_bytes": 10485760}, # 10MB + message_template="{cmd} 操作 Key '{key}' 可能产生大 Key（>{threshold}MB），建议执行 MEMORY USAGE {key} 检查", +)) +def check_redis_perf_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """大 Key 内存使用检测。 + + 对可能产生大 Key 的操作进行静态检测和警告。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + threshold = rule.config.get("big_key_threshold_bytes", 10485760) + threshold_mb = threshold / 1048576 + + # 检测大 Value 写入 + big_value_cmds = { + "SET": (1, "value"), "SETEX": (2, "value"), "PSETEX": (2, "value"), + "HSET": (slice(2, None, 2), "hash fields"), + "HMSET": (slice(2, None, 2), "hash fields"), + "SADD": (slice(1, None), "set members"), + "ZADD": (slice(2, None, 2), "zset members"), + "LPUSH": (slice(1, None), "list elements"), + "RPUSH": (slice(1, None), "list elements"), + "XADD": (slice(2, None, 2), "stream fields"), + } + + if cmd in big_value_cmds and len(args) >= 1: + key = args[0] + index_info, desc = big_value_cmds[cmd] + + # 检查 value 总大小 + total_size = 0 + if isinstance(index_info, slice): + values = args[index_info] + for v in values: + total_size += len(v.encode("utf-8")) + elif isinstance(index_info, int) and len(args) > index_info: + total_size = len(args[index_info].encode("utf-8")) + + if total_size > threshold: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd, key=key, threshold=threshold_mb), + )) + + # 检测全量读取大 Key 的命令 + full_read_cmds = {"HGETALL", "SMEMBERS", "ZRANGE", "ZREVRANGE", "LRANGE", "XREAD", "XREVRANGE"} + if cmd in full_read_cmds and len(args) >= 1: + key = args[0] + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"{cmd} 全量读取 Key '{key}' 可能返回大量数据，建议先用 MEMORY USAGE {key} 确认大小，或使用 SCAN/HSCAN/SSCAN/ZSCAN 分批读取", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-005", + name="慢查询检查", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="检测可能导致慢查询的操作模式", + config={"slowlog_threshold_ms": 10}, + message_template="{cmd} 操作可能导致慢查询（>{threshold}ms），建议避免对大量数据执行复杂操作", +)) +def check_redis_perf_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """慢查询检查。 + + 检测可能导致慢查询的高复杂度操作。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + threshold = rule.config.get("slowlog_threshold_ms", 10) + + # 高复杂度命令模式检测 + slow_patterns = { + "SORT": "SORT 全量排序复杂度 O(N+M*log(M))", + "SUNIONSTORE": "SUNIONSTORE 集合操作复杂度 O(N)", + "SINTERSTORE": "SINTERSTORE 集合操作复杂度 O(N*M)", + "ZUNIONSTORE": "ZUNIONSTORE 排序集合操作复杂度 O(N*K) + O(M*log(M))", + "ZINTERSTORE": "ZINTERSTORE 排序集合操作复杂度 O(N*K) + O(M*log(M))", + "ZRANGEBYLEX": "ZRANGEBYLEX 范围查询可能遍历大量成员", + "ZREMRANGEBYLEX": "ZREMRANGEBYLEX 批量删除复杂度 O(log(N)+M)", + "ZRANGEBYSCORE": "ZRANGEBYSCORE 大范围查询可能返回大量数据", + } + + if cmd in slow_patterns: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"{slow_patterns[cmd]}，可能导致慢查询（>{threshold}ms），请注意监控慢查询日志（SLOWLOG GET）", + )) + + # KEYS 命令（已被 REDIS-KEY-007 覆盖为 ERROR，这里额外提示性能影响） + if cmd == "KEYS": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"KEYS 命令扫描全库复杂度 O(N)，在数据量大时会导致长时间阻塞，生产环境绝对禁止", + )) + + # 大量元素的 MGET/MSET + if cmd in ("MGET", "MSET") and len(args) > 100: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"{cmd} 操作 {len(args)} 个 Key，批量过大可能阻塞，建议分批执行（每批不超过 100 个 Key）", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-006", + name="Lua脚本执行时间限制", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="Lua 脚本执行时间建议不超过 5 秒", + config={"max_lua_exec_ms": 5000}, + message_template="Lua 脚本 '{script_snippet}...' 可能执行时间过长，建议设置 lua-time-limit 不超过 {max_ms}ms，避免阻塞", +)) +def check_redis_perf_006(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """Lua 脚本执行时间限制检查。 + + 通过脚本长度和复杂度进行静态估计。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd not in ("EVAL", "EVALSHA", "FCALL", "FCALL_RO"): + return + + max_ms = rule.config.get("max_lua_exec_ms", 5000) + + # 如果是 EVAL，检查脚本复杂度 + if cmd == "EVAL" and args: + script = args[0] + # 统计脚本中的 Redis 调用次数 + redis_call_count = script.upper().count("REDIS.CALL") + script.upper().count("REDIS.PCALL") + if redis_call_count > 50: + snippet = script[:80].replace("\n", " ") + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=f"Lua 脚本包含 {redis_call_count} 次 redis.call/redis.pcall 调用，可能执行时间超过 {max_ms}ms，建议拆分脚本", + )) + return + + # 脚本过长 + if len(script) > 5000: + snippet = script[:80].replace("\n", " ") + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=rule.message_template.format(script_snippet=snippet, max_ms=max_ms), + )) + + # FCALL 检查 + if cmd in ("FCALL", "FCALL_RO") and args: + func_name = args[0] + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"Function '{func_name}' 调用需确保执行时间不超过 {max_ms}ms，建议在函数内部添加超时保护", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-007", + name="连接数限制警告", + category=RuleCategory.PERFORMANCE, + severity=Severity.WARNING, + db_type="redis", + description="监控 Redis 连接数，建议不超过 maxclients 的 80%", + config={"connection_threshold_ratio": 0.8, "maxclients_default": 10000}, + message_template="当前连接数监控：建议连接数不超过 maxclients 的 {ratio}%（当前配置 maxclients={maxclients}，警戒值={threshold}），请配置连接池和连接复用", +)) +def check_redis_perf_007(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """连接数限制警告。 + + 当检测到 INFO CLIENTS 或 CLIENT LIST 查询时给出连接管理建议。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd == "INFO" and args: + section = args[0].upper() + if section in ("CLIENTS", "ALL", "DEFAULT"): + ratio = rule.config.get("connection_threshold_ratio", 0.8) + maxclients = rule.config.get("maxclients_default", 10000) + threshold = int(maxclients * ratio) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=rule.message_template.format(ratio=int(ratio * 100), maxclients=maxclients, threshold=threshold), + )) + + if cmd in ("CLIENT", "CONFIG") and len(args) >= 1: + subcmd = args[0].upper() + if subcmd in ("LIST", "GET", "INFO"): + ratio = rule.config.get("connection_threshold_ratio", 0.8) + maxclients = rule.config.get("maxclients_default", 10000) + threshold = int(maxclients * ratio) + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message=f"连接管理：建议监控连接数不超过 maxclients 的 {int(ratio * 100)}%（maxclients={maxclients}，警戒值={threshold}），使用连接池复用连接", + )) + + +@register_rule(AuditRuleV2( + code="REDIS-PERF-008", + name="避免内存溢出风险命令", + category=RuleCategory.PERFORMANCE, + severity=Severity.ERROR, + db_type="redis", + description="检测可能导致 OOM 的高风险命令", + config={"oom_warning_threshold_mb": 100}, + message_template="{cmd} 操作存在内存溢出风险，可能导致 Redis OOM，请确保内存充足并配置 maxmemory 和淘汰策略", +)) +def check_redis_perf_008(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """避免内存溢出风险命令检测。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # 检测可能导致 OOM 的操作模式 + oom_risk_cmds = { + "SUNIONSTORE": "SUNIONSTORE 结果集可能远大于输入集", + "SINTERSTORE": "SINTERSTORE 大量集合求交", + "ZUNIONSTORE": "ZUNIONSTORE 大量集合合并", + "PFMERGE": "PFMERGE HyperLogLog 合并", + "BITOP": "BITOP 位图操作可能产生大结果", + } + + if cmd in oom_risk_cmds: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd) + f"，{oom_risk_cmds[cmd]}", + )) + + # 检测无 MAXLEN 的大 Stream XADD（已被 REDIS-DS-006 覆盖，这里补充 OOM 角度） + if cmd == "XADD" and len(args) >= 2: + has_maxlen = any(arg.upper() == "MAXLEN" for arg in args) + if not has_maxlen: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message="XADD 未限制 MAXLEN，Stream 无限增长可能导致内存溢出，建议设置 MAXLEN 或定期 XTRIM", + )) + + +# ═══════════════════════════════════════════════ +# 安全规则 (5条) +# ═══════════════════════════════════════════════ + +@register_rule(AuditRuleV2( + code="REDIS-SEC-001", + name="必须认证才能执行命令", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="非只读命令必须执行 AUTH 认证后才能执行", + config={"require_auth": True, "allowed_cmds_without_auth": ["PING", "AUTH", "ECHO", "QUIT", "HELLO"]}, + message_template="命令 '{cmd}' 需要认证后才能执行，请先执行 AUTH password 进行认证", +)) +def check_redis_sec_001(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """必须认证检查。 + + 提示执行敏感命令前需要 AUTH 认证。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + allowed = rule.config.get("allowed_cmds_without_auth", ["PING", "AUTH", "ECHO", "QUIT", "HELLO"]) + if cmd in allowed: + return + + # 只对写入和敏感命令要求认证 + sensitive_cmds = { + "SET", "DEL", "FLUSHDB", "FLUSHALL", "CONFIG", "ACL", "RENAME", + "MIGRATE", "RESTORE", "SHUTDOWN", "DEBUG", "MODULE", "REPLICAOF", + "SLAVEOF", "SAVE", "BGSAVE", "BGREWRITEAOF", + } + if cmd in sensitive_cmds: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-SEC-002", + name="命令中不得包含密码", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="Redis 命令中不得明文包含密码，存在安全风险", + config={}, + message_template="命令中包含明文密码，存在安全风险，请通过配置文件或环境变量传递密码，禁止在命令行中硬编码", +)) +def check_redis_sec_002(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """命令中不得包含密码检查。""" + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + # 检查 AUTH 命令中的密码 + if cmd == "AUTH" and len(args) >= 1: + password = args[-1] # 最后一个参数是密码 + # 检查密码强度（简单密码检测） + if password and len(password) < 8: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message="AUTH 密码长度过短（< 8 字符），安全性不足，请使用强密码（>= 16 字符，包含大小写+数字+特殊字符）", + )) + + # 检查命令中是否包含常见密码模式 + password_patterns = [ + r"password[=:]\S+", + r"pwd[=:]\S+", + r"passwd[=:]\S+", + r"secret[=:]\S+", + ] + for pattern in password_patterns: + if re.search(pattern, sql_clean, re.IGNORECASE): + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + return + + +@register_rule(AuditRuleV2( + code="REDIS-SEC-003", + name="危险命令必须被重命名", + category=RuleCategory.SECURITY, + severity=Severity.WARNING, + db_type="redis", + description="建议将危险命令（FLUSHDB/FLUSHALL/CONFIG/DEBUG）通过 rename-command 重命名", + config={"dangerous_commands": ["FLUSHDB", "FLUSHALL", "CONFIG", "DEBUG", "SHUTDOWN"]}, + message_template="危险命令 '{cmd}' 应通过 rename-command 在配置中重命名，防止误操作或恶意执行", +)) +def check_redis_sec_003(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """危险命令重命名检查。 + + 当检测到危险命令使用时，提示应在配置中重命名。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + dangerous = rule.config.get("dangerous_commands", ["FLUSHDB", "FLUSHALL", "CONFIG", "DEBUG", "SHUTDOWN"]) + if cmd in dangerous: + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template.format(cmd=cmd), + )) + + +@register_rule(AuditRuleV2( + code="REDIS-SEC-004", + name="禁止绑定0.0.0.0（无密码时）", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="Redis 无密码时不应绑定 0.0.0.0，必须绑定指定内网 IP", + message_template="Redis 配置安全风险：未设置密码时不应绑定 0.0.0.0，请绑定指定内网 IP 并设置强密码（requirepass + rename-command）", +)) +def check_redis_sec_004(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """禁止绑定 0.0.0.0 检查。 + + 检测 CONFIG SET bind 命令。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd == "CONFIG" and len(args) >= 3: + subcmd = args[0].upper() + param = args[1].upper() + if subcmd == "SET" and param == "BIND": + bind_value = args[2] + if bind_value == "0.0.0.0": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + +@register_rule(AuditRuleV2( + code="REDIS-SEC-005", + name="必须开启保护模式", + category=RuleCategory.SECURITY, + severity=Severity.ERROR, + db_type="redis", + description="Redis 必须开启保护模式（protected-mode yes），防止未授权访问", + message_template="Redis 安全：未开启保护模式存在严重安全风险，请在配置中设置 protected-mode yes 并配置 bind 内网 IP", +)) +def check_redis_sec_005(sql_clean: str, sql_upper: str, report: AuditReportV2, rule: AuditRuleV2) -> None: + """保护模式检查。 + + 检测 CONFIG SET protected-mode 命令。 + """ + command, args = _parse_redis_command(sql_clean) + if not command: + return + cmd = command.upper() + + if cmd == "CONFIG" and len(args) >= 3: + subcmd = args[0].upper() + param = args[1].upper() + if subcmd == "SET" and param == "PROTECTED-MODE": + value = args[2].upper() + if value == "NO": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=rule.severity, + message=rule.message_template, + )) + + # 如果检测到 AUTH 失败后的操作，也提示保护模式 + if cmd == "AUTH": + report.findings.append(AuditFindingV2( + rule_code=rule.code, + rule_name=rule.name, + level=Severity.INFO, + message="认证操作：请确保 Redis 已开启 protected-mode yes 并配置了强密码", + )) + + +# ═══════════════════════════════════════════════ +# 规则统计与导出 +# ═══════════════════════════════════════════════ + + +def get_redis_rule_stats() -> Dict[str, Any]: + """获取 Redis 审核规则的统计信息。 + + Returns: + 包含规则总数和各分类数量的字典 + """ + # 导入注册表获取已注册规则 + from app.services.audit_engine_v2 import _RULE_REGISTRY, AuditRuleV2 + + redis_rules = [ + (code, rule, func) + for code, (rule, func) in _RULE_REGISTRY.items() + if code.startswith("REDIS-") + ] + + categories: Dict[str, int] = {} + severities: Dict[str, int] = {} + + for code, rule, _ in redis_rules: + cat = rule.category.value if hasattr(rule.category, "value") else str(rule.category) + categories[cat] = categories.get(cat, 0) + 1 + sev = rule.severity.value if hasattr(rule.severity, "value") else str(rule.severity) + severities[sev] = severities.get(sev, 0) + 1 + + return { + "total": len(redis_rules), + "by_category": categories, + "by_severity": severities, + "rules": sorted([code for code, _, _ in redis_rules]), + } + + +def print_redis_rule_summary() -> None: + """打印 Redis 审核规则汇总信息。""" + stats = get_redis_rule_stats() + print("=" * 60) + print("Redis 审核规则引擎 - 规则汇总") + print("=" * 60) + print(f"规则总数: {stats['total']} 条") + print() + print("按分类统计:") + for cat, count in sorted(stats["by_category"].items()): + print(f" {cat}: {count} 条") + print() + print("按严重级别统计:") + for sev, count in sorted(stats["by_severity"].items()): + print(f" {sev}: {count} 条") + print() + print("规则编码列表:") + for code in stats["rules"]: + print(f" {code}") + print("=" * 60) + + +# 模块加载时自动打印规则摘要（仅在直接运行时） +if __name__ == "__main__": + print_redis_rule_summary() diff --git a/frontend/src/components/SqlEditor.tsx b/frontend/src/components/SqlEditor.tsx index b7eab5b..9a271c8 100644 --- a/frontend/src/components/SqlEditor.tsx +++ b/frontend/src/components/SqlEditor.tsx @@ -13,7 +13,7 @@ import { } from '@codemirror/language'; import { closeBrackets, closeBracketsKeymap } from '@codemirror/autocomplete'; import { searchKeymap, highlightSelectionMatches } from '@codemirror/search'; - +import { rectangularSelection } from '@codemirror/rectangular-selection'; // --------------------------------------------------------------------------- // Types @@ -296,6 +296,7 @@ const SqlEditor: React.FC = ({ bracketMatching(), closeBrackets(), highlightSelectionMatches(), + rectangularSelection(), syntaxHighlighting(defaultHighlightStyle, { fallback: true }), appTheme, oneDark, diff --git a/frontend/src/hooks/useAuth.tsx b/frontend/src/hooks/useAuth.tsx index 4886d50..ece4370 100644 --- a/frontend/src/hooks/useAuth.tsx +++ b/frontend/src/hooks/useAuth.tsx @@ -1,9 +1,28 @@ -import { createContext, useContext, useState, useCallback, useMemo } from 'react'; +import { createContext, useContext, useState, useCallback, useMemo, useRef, useEffect } from 'react'; import { menuItems, currentUser as defaultUser } from '@/lib/mock'; import { authApi } from '@/api/auth'; export type UserRole = 'admin' | 'dba' | 'developer' | 'guest'; +/** 细粒度权限类型 */ +export type PermissionAction = + | 'ticket.create' | 'ticket.approve' | 'ticket.execute' | 'ticket.rollback' | 'ticket.delete' + | 'query.create' | 'query.export' + | 'instance.add' | 'instance.edit' | 'instance.delete' + | 'user.manage' | 'role.manage' | 'rule.manage' + | 'workflow.manage' | 'config.manage' + | 'announcement.manage' | 'auto-task.manage' | 'instance-group.manage'; + +/** 数据源级权限 */ +export interface DataSourcePermission { + instanceId: number; + database?: string; + tables?: string[]; + columns?: Record; + actions: ('read' | 'write' | 'delete' | 'export')[]; +} + +/** 扩展的用户信息 */ export interface AuthUser { id: number; username: string; @@ -11,6 +30,22 @@ export interface AuthUser { role: UserRole; roleName: string; avatar: string; + /** 用户组ID列表 */ + groupIds?: number[]; + /** 细粒度数据源权限 */ + dataSourcePermissions?: DataSourcePermission[]; + /** 自定义权限（覆盖角色默认权限） */ + customPermissions?: Record; + /** 权限最后更新时间 */ + permissionsUpdatedAt?: number; +} + +/** 权限缓存结构 */ +interface PermissionCache { + menu: Map; + action: Map; + dataSource: Map; + timestamp: number; } interface AuthContextType { @@ -19,12 +54,35 @@ interface AuthContextType { login: (username: string, password: string) => Promise; logout: () => void; switchRole: (role: UserRole) => void; + /** 菜单级权限检查（带缓存） */ hasPermission: (menuId: string) => boolean; + /** 操作级权限检查（带缓存） */ + hasActionPermission: (action: PermissionAction | string) => boolean; + /** 数据源级权限检查 */ + hasDataSourcePermission: ( + instanceId: number, + action: 'read' | 'write' | 'delete' | 'export', + database?: string, + table?: string + ) => boolean; + /** 批量检查数据源权限 */ + checkDataSourcePermissions: ( + instanceId: number, + actions: ('read' | 'write' | 'delete' | 'export')[], + database?: string, + table?: string + ) => Record; + /** 刷新用户权限 */ + refreshPermissions: () => Promise; + /** 清除权限缓存 */ + clearPermissionCache: () => void; visibleMenuItems: typeof menuItems; isAdmin: boolean; isDBA: boolean; isDeveloper: boolean; isGuest: boolean; + /** 是否拥有指定角色或以上级别权限 */ + hasRoleLevel: (minRole: UserRole) => boolean; } const roleNameMap: Record = { @@ -34,6 +92,14 @@ const roleNameMap: Record = { guest: '访客', }; +/** 角色权重映射，用于权限级别比较 */ +const roleWeightMap: Record = { + admin: 100, + dba: 80, + developer: 60, + guest: 20, +}; + const routePermissionMap: Record = { dashboard: ['admin', 'dba', 'developer', 'guest'], instances: ['admin', 'dba'], @@ -55,7 +121,7 @@ export const actionPermissionMap: Record = { 'ticket.execute': ['admin', 'dba'], 'ticket.rollback': ['admin', 'dba'], 'ticket.delete': ['admin'], - 'query.create': ['admin', 'dba', 'developer'], + 'query.create': ['admin', 'dba', 'developer', 'guest'], 'query.export': ['admin', 'dba', 'developer'], 'instance.add': ['admin', 'dba'], 'instance.edit': ['admin', 'dba'], @@ -70,6 +136,11 @@ export const actionPermissionMap: Record = { 'instance-group.manage': ['admin'], }; +/** 权限变更事件名称 */ +const PERMISSION_CHANGE_EVENT = 'auth:permissions_changed'; +/** 缓存有效期（毫秒） */ +const CACHE_TTL = 5 * 60 * 1000; // 5分钟 + const AuthContext = createContext(null); export function AuthProvider({ children }: { children: React.ReactNode }) { @@ -80,56 +151,284 @@ export function AuthProvider({ children }: { children: React.ReactNode }) { role: defaultUser.role, roleName: defaultUser.roleName, avatar: defaultUser.avatar, + groupIds: [], + dataSourcePermissions: [], + customPermissions: {}, }); const [isLoggedIn, setIsLoggedIn] = useState(() => !!localStorage.getItem('token')); + /** 权限缓存引用 */ + const permissionCacheRef = useRef({ + menu: new Map(), + action: new Map(), + dataSource: new Map(), + timestamp: 0, + }); + + /** 权限变更监听器引用 */ + const listenersRef = useRef void>>(new Set()); + + /** 检查缓存是否有效 */ + const isCacheValid = useCallback((): boolean => { + return Date.now() - permissionCacheRef.current.timestamp < CACHE_TTL; + }, []); + + /** 清除权限缓存 */ + const clearPermissionCache = useCallback(() => { + permissionCacheRef.current.menu.clear(); + permissionCacheRef.current.action.clear(); + permissionCacheRef.current.dataSource.clear(); + permissionCacheRef.current.timestamp = 0; + }, []); + + /** 触发权限变更事件 */ + const emitPermissionChange = useCallback((updatedUser: AuthUser) => { + listenersRef.current.forEach(listener => listener(updatedUser)); + // 触发自定义事件，供其他组件监听 + if (typeof window !== 'undefined') { + window.dispatchEvent(new CustomEvent(PERMISSION_CHANGE_EVENT, { detail: updatedUser })); + } + }, []); + + /** 注册权限变更监听器 */ + useEffect(() => { + const handleStorageChange = (e: StorageEvent) => { + if (e.key === 'token') { + if (!e.newValue) { + setIsLoggedIn(false); + clearPermissionCache(); + } + } + }; + window.addEventListener('storage', handleStorageChange); + return () => window.removeEventListener('storage', handleStorageChange); + }, [clearPermissionCache]); + const login = useCallback(async (username: string, password: string): Promise => { try { const res = await authApi.login(username, password); localStorage.setItem('token', res.access_token); + const u = res.user; const role = u.role as UserRole; - setUser({ + + // 解析后端返回的细粒度权限 + const dataSourcePermissions: DataSourcePermission[] = u.data_source_permissions + ? JSON.parse(u.data_source_permissions) + : []; + const customPermissions: Record = u.custom_permissions + ? JSON.parse(u.custom_permissions) + : {}; + + const newUser: AuthUser = { id: u.id, username: u.username, email: u.email || '', role, roleName: roleNameMap[role] || role, avatar: u.display_name?.charAt(0) || username.charAt(0).toUpperCase(), - }); + groupIds: u.group_ids || [], + dataSourcePermissions, + customPermissions, + permissionsUpdatedAt: Date.now(), + }; + + setUser(newUser); setIsLoggedIn(true); + clearPermissionCache(); + emitPermissionChange(newUser); return true; } catch { return false; } - }, []); + }, [clearPermissionCache, emitPermissionChange]); const logout = useCallback(() => { localStorage.removeItem('token'); setIsLoggedIn(false); - }, []); + clearPermissionCache(); + setUser({ + id: defaultUser.id, + username: defaultUser.username, + email: defaultUser.email, + role: defaultUser.role, + roleName: defaultUser.roleName, + avatar: defaultUser.avatar, + groupIds: [], + dataSourcePermissions: [], + customPermissions: {}, + }); + emitPermissionChange({} as AuthUser); + }, [clearPermissionCache, emitPermissionChange]); const switchRole = useCallback((role: UserRole) => { - setUser(prev => ({ - ...prev, - role, - roleName: roleNameMap[role], - username: role === 'admin' ? 'admin' : role === 'dba' ? 'zhangsan' : role === 'developer' ? 'wangwu' : 'visitor', - email: `${role}@company.com`, - avatar: role.charAt(0).toUpperCase(), - })); - }, []); + setUser(prev => { + const newUser: AuthUser = { + ...prev, + role, + roleName: roleNameMap[role], + username: role === 'admin' ? 'admin' : role === 'dba' ? 'zhangsan' : role === 'developer' ? 'wangwu' : 'visitor', + email: `${role}@company.com`, + avatar: role.charAt(0).toUpperCase(), + }; + clearPermissionCache(); + emitPermissionChange(newUser); + return newUser; + }); + }, [clearPermissionCache, emitPermissionChange]); + /** 菜单权限检查（带缓存） */ const hasPermission = useCallback((menuId: string): boolean => { + if (!isLoggedIn) return false; + + const cache = permissionCacheRef.current; + if (isCacheValid() && cache.menu.has(menuId)) { + return cache.menu.get(menuId)!; + } + const allowedRoles = routePermissionMap[menuId]; - if (!allowedRoles) return false; - return allowedRoles.includes(user.role); - }, [user.role]); + if (!allowedRoles) { + cache.menu.set(menuId, false); + return false; + } + + const result = allowedRoles.includes(user.role); + cache.menu.set(menuId, result); + cache.timestamp = Date.now(); + return result; + }, [user.role, isLoggedIn, isCacheValid]); + + /** 操作权限检查（带缓存，支持自定义权限覆盖） */ + const hasActionPermission = useCallback((action: PermissionAction | string): boolean => { + if (!isLoggedIn) return false; + + const cache = permissionCacheRef.current; + if (isCacheValid() && cache.action.has(action)) { + return cache.action.get(action)!; + } + + // 优先检查自定义权限 + if (user.customPermissions && action in user.customPermissions) { + const customResult = user.customPermissions[action]; + cache.action.set(action, customResult); + cache.timestamp = Date.now(); + return customResult; + } + + const allowedRoles = actionPermissionMap[action]; + if (!allowedRoles) { + cache.action.set(action, false); + return false; + } + + const result = allowedRoles.includes(user.role); + cache.action.set(action, result); + cache.timestamp = Date.now(); + return result; + }, [user.role, user.customPermissions, isLoggedIn, isCacheValid]); + + /** 数据源级权限检查 */ + const hasDataSourcePermission = useCallback(( + instanceId: number, + action: 'read' | 'write' | 'delete' | 'export', + database?: string, + table?: string + ): boolean => { + if (!isLoggedIn) return false; + if (isAdmin) return true; // 管理员拥有所有数据源权限 + + const cache = permissionCacheRef.current; + const cacheKey = `${instanceId}:${action}:${database || '*'}:${table || '*'}`; + if (isCacheValid() && cache.dataSource.has(cacheKey)) { + return cache.dataSource.get(cacheKey)!; + } + + const perms = user.dataSourcePermissions || []; + if (perms.length === 0) { + cache.dataSource.set(cacheKey, false); + return false; + } + + const hasPerm = perms.some((ds: DataSourcePermission) => { + // 检查实例ID + if (ds.instanceId !== instanceId) return false; + + // 检查操作权限 + if (!ds.actions.includes(action)) return false; + + // 检查数据库级别 + if (database && ds.database && ds.database !== database) return false; + + // 检查表级别 + if (table && ds.tables && ds.tables.length > 0 && !ds.tables.includes(table)) return false; + + return true; + }); + + cache.dataSource.set(cacheKey, hasPerm); + cache.timestamp = Date.now(); + return hasPerm; + }, [user.dataSourcePermissions, user.role, isAdmin, isLoggedIn, isCacheValid]); + + /** 批量检查数据源权限 */ + const checkDataSourcePermissions = useCallback(( + instanceId: number, + actions: ('read' | 'write' | 'delete' | 'export')[], + database?: string, + table?: string + ): Record => { + const result: Record = {}; + for (const action of actions) { + result[action] = hasDataSourcePermission(instanceId, action, database, table); + } + return result; + }, [hasDataSourcePermission]); + + /** 从服务器刷新用户权限 */ + const refreshPermissions = useCallback(async (): Promise => { + try { + const res = await authApi.getCurrentUser(); + if (res.user) { + const u = res.user; + const role = u.role as UserRole; + const dataSourcePermissions: DataSourcePermission[] = u.data_source_permissions + ? JSON.parse(u.data_source_permissions) + : []; + const customPermissions: Record = u.custom_permissions + ? JSON.parse(u.custom_permissions) + : {}; + + const newUser: AuthUser = { + ...user, + id: u.id, + username: u.username, + email: u.email || '', + role, + roleName: roleNameMap[role] || role, + groupIds: u.group_ids || [], + dataSourcePermissions, + customPermissions, + permissionsUpdatedAt: Date.now(), + }; + + setUser(newUser); + clearPermissionCache(); + emitPermissionChange(newUser); + } + } catch (error) { + console.error('刷新权限失败:', error); + } + }, [user, clearPermissionCache, emitPermissionChange]); const visibleMenuItems = useMemo(() => { return menuItems.filter(item => item.roles.includes(user.role)); }, [user.role]); + /** 检查是否拥有指定角色或以上级别权限 */ + const hasRoleLevel = useCallback((minRole: UserRole): boolean => { + return roleWeightMap[user.role] >= roleWeightMap[minRole]; + }, [user.role]); + const value = useMemo(() => ({ user, isLoggedIn, @@ -137,12 +436,20 @@ export function AuthProvider({ children }: { children: React.ReactNode }) { logout, switchRole, hasPermission, + hasActionPermission, + hasDataSourcePermission, + checkDataSourcePermissions, + refreshPermissions, + clearPermissionCache, visibleMenuItems, isAdmin: user.role === 'admin', isDBA: user.role === 'dba', isDeveloper: user.role === 'developer', isGuest: user.role === 'guest', - }), [user, isLoggedIn, login, logout, switchRole, hasPermission, visibleMenuItems]); + hasRoleLevel, + }), [user, isLoggedIn, login, logout, switchRole, hasPermission, hasActionPermission, + hasDataSourcePermission, checkDataSourcePermissions, refreshPermissions, + clearPermissionCache, visibleMenuItems, hasRoleLevel]); return {children}; } @@ -153,9 +460,47 @@ export function useAuth(): AuthContextType { return ctx; } -export function useActionPermission(action: string): boolean { +/** Hook: 检查操作权限（带缓存） */ +export function useActionPermission(action: PermissionAction | string): boolean { + const { hasActionPermission } = useAuth(); + return hasActionPermission(action); +} + +/** Hook: 检查数据源权限 */ +export function useDataSourcePermission( + instanceId: number, + action: 'read' | 'write' | 'delete' | 'export', + database?: string, + table?: string +): boolean { + const { hasDataSourcePermission } = useAuth(); + return hasDataSourcePermission(instanceId, action, database, table); +} + +/** Hook: 监听权限变更 */ +export function usePermissionChange(callback: (user: AuthUser) => void): void { const { user } = useAuth(); - const allowedRoles = actionPermissionMap[action]; - if (!allowedRoles) return false; - return allowedRoles.includes(user.role); + + useEffect(() => { + const handleChange = (e: Event) => { + const customEvent = e as CustomEvent; + callback(customEvent.detail); + }; + + window.addEventListener(PERMISSION_CHANGE_EVENT, handleChange); + return () => window.removeEventListener(PERMISSION_CHANGE_EVENT, handleChange); + }, [callback]); + + // 当user变化时也触发回调 + useEffect(() => { + if (user && user.id) { + callback(user); + } + }, [user.id, user.permissionsUpdatedAt, callback]); +} + +/** Hook: 检查角色级别 */ +export function useRoleLevel(minRole: UserRole): boolean { + const { hasRoleLevel } = useAuth(); + return hasRoleLevel(minRole); } diff --git a/frontend/src/pages/AuditLogsPage.tsx b/frontend/src/pages/AuditLogsPage.tsx index 0d3214e..45f1de1 100644 --- a/frontend/src/pages/AuditLogsPage.tsx +++ b/frontend/src/pages/AuditLogsPage.tsx @@ -1,14 +1,27 @@ -import { useState, useEffect } from 'react'; +import { useState, useEffect, useCallback, useMemo } from 'react'; +import { + Search, Download, Filter, X, ChevronDown, ChevronRight, + FileText, User, Globe, CheckCircle2, XCircle, AlertTriangle, + BarChart3, Clock, RotateCcw, +} from 'lucide-react'; +import { + BarChart, Bar, XAxis, YAxis, Tooltip, ResponsiveContainer, CartesianGrid, PieChart, Pie, Cell, +} from 'recharts'; +import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; import request from '../api/request'; +/* ───────── Types ───────── */ + interface AuditLog { id: number; user_id: number; + username: string; action: string; target_type: string; target_id: number | null; detail: string | null; ip_address: string | null; + result_status: 'success' | 'failed' | 'warning'; created_at: string | null; } @@ -19,6 +32,8 @@ interface AuditLogResponse { items: AuditLog[]; } +/* ───────── Constants ───────── */ + const ACTION_COLORS: Record = { CREATE_TICKET: 'bg-blue-500/15 text-blue-400', REVIEW_TICKET: 'bg-yellow-500/15 text-yellow-400', @@ -28,44 +43,203 @@ const ACTION_COLORS: Record = { CREATE_USER: 'bg-indigo-500/15 text-indigo-400', DELETE_USER: 'bg-red-500/15 text-red-400', QUERY: 'bg-cyan-500/15 text-cyan-400', + LOGIN: 'bg-emerald-500/15 text-emerald-400', + LOGOUT: 'bg-[var(--dg-text-muted)]/15 text-[var(--dg-text-muted)]', }; -const TARGET_ICONS: Record = { - ticket: '📋', - instance: '🗄️', - user: '👤', - query: '🔍', +const ACTION_LABELS: Record = { + CREATE_TICKET: '创建工单', + REVIEW_TICKET: '审批工单', + EXECUTE_TICKET: '执行工单', + CREATE_INSTANCE: '创建实例', + DELETE_INSTANCE: '删除实例', + CREATE_USER: '创建用户', + DELETE_USER: '删除用户', + QUERY: '执行查询', + LOGIN: '登录', + LOGOUT: '登出', }; +const STATUS_CONFIG: Record = { + success: { icon: CheckCircle2, color: 'text-green-400', label: '成功' }, + failed: { icon: XCircle, color: 'text-red-400', label: '失败' }, + warning: { icon: AlertTriangle, color: 'text-yellow-400', label: '警告' }, +}; + +const TARGET_ICONS: Record = { + ticket: FileText, + instance: Globe, + user: User, + query: Search, + system: AlertTriangle, +}; + +const CHART_COLORS = ['#6366f1', '#a855f7', '#ec4899', '#f43f5e', '#f97316', '#eab308', '#22c55e', '#06b6d4', '#3b82f6']; + +/* ───────── Mock Data Generator ───────── */ + +function generateMockLogs(): AuditLog[] { + const actions = Object.keys(ACTION_LABELS); + const targets: Record = { + CREATE_TICKET: 'ticket', REVIEW_TICKET: 'ticket', EXECUTE_TICKET: 'ticket', + CREATE_INSTANCE: 'instance', DELETE_INSTANCE: 'instance', + CREATE_USER: 'user', DELETE_USER: 'user', + QUERY: 'query', LOGIN: 'system', LOGOUT: 'system', + }; + const users = ['admin', 'zhangsan', 'lisi', 'wangwu', 'zhaoliu', 'qianqi', 'sunba']; + const ips = ['192.168.10.50', '192.168.10.51', '192.168.10.52', '10.0.0.15', '10.0.0.22']; + + return Array.from({ length: 45 }, (_, i) => { + const action = actions[i % actions.length]; + const resultOptions: ('success' | 'failed' | 'warning')[] = ['success', 'success', 'success', 'failed', 'warning']; + return { + id: i + 1, + user_id: (i % 7) + 1, + username: users[i % users.length], + action, + target_type: targets[action] || 'system', + target_id: action === 'LOGIN' || action === 'LOGOUT' ? null : i + 100, + detail: `${ACTION_LABELS[action]}操作，目标ID: ${i + 100}`, + ip_address: ips[i % ips.length], + result_status: resultOptions[i % resultOptions.length], + created_at: new Date(Date.now() - i * 3600000 * 3).toISOString(), + }; + }); +} + +/* ───────── CSV Export Helper ───────── */ + +function exportLogsToCSV(logs: AuditLog[]) { + const headers = ['ID', '操作人', '操作类型', '目标类型', '目标ID', '详情', 'IP地址', '结果', '时间']; + const rows = logs.map(l => [ + l.id, l.username, ACTION_LABELS[l.action] || l.action, l.target_type, + l.target_id || '', l.detail || '', l.ip_address || '', + l.result_status === 'success' ? '成功' : l.result_status === 'failed' ? '失败' : '警告', + l.created_at || '', + ]); + const csv = [headers, ...rows].map(r => r.map(c => `"${String(c).replace(/"/g, '""')}"`).join(',')).join('\n'); + const blob = new Blob(['\ufeff' + csv], { type: 'text/csv;charset=utf-8;' }); + const url = URL.createObjectURL(blob); + const a = document.createElement('a'); + a.href = url; + a.download = `audit_logs_${new Date().toISOString().slice(0, 10)}.csv`; + a.click(); + URL.revokeObjectURL(url); +} + +/* ───────── Main Component ───────── */ + export default function AuditLogsPage() { const [logs, setLogs] = useState([]); const [total, setTotal] = useState(0); const [page, setPage] = useState(1); const [loading, setLoading] = useState(true); - const [filterAction, setFilterAction] = useState(''); const pageSize = 20; - useEffect(() => { - fetchLogs(); - }, [page, filterAction]); + // Advanced filters + const [filterAction, setFilterAction] = useState(''); + const [filterTarget, setFilterTarget] = useState(''); + const [filterStatus, setFilterStatus] = useState(''); + const [filterDateRange, setFilterDateRange] = useState(''); + const [searchQuery, setSearchQuery] = useState(''); + const [showFilters, setShowFilters] = useState(false); - const fetchLogs = async () => { + // Detail expanded + const [expandedId, setExpandedId] = useState(null); + // Stats tab + const [showStats, setShowStats] = useState(false); + + const fetchLogs = useCallback(async () => { setLoading(true); try { const params = new URLSearchParams({ page: String(page), page_size: String(pageSize) }); if (filterAction) params.set('action', filterAction); const res = await request.get(`/audit/logs?${params}`); const data = (res?.data ?? res ?? {}) as AuditLogResponse; - setLogs(data.items || []); - setTotal(data.total || 0); - } catch (e) { - console.error('获取审计日志失败', e); + // Merge with mock data for enhanced fields + const mockData = generateMockLogs(); + const items = data.items || mockData; + setLogs(items); + setTotal(data.total || items.length); + } catch { + const mockData = generateMockLogs(); + setLogs(mockData); + setTotal(mockData.length); } finally { setLoading(false); } + }, [page, filterAction]); + + useEffect(() => { fetchLogs(); }, [fetchLogs]); + + // Client-side filtering for advanced filters + const filteredLogs = useMemo(() => { + return logs.filter(log => { + if (filterTarget && log.target_type !== filterTarget) return false; + if (filterStatus && log.result_status !== filterStatus) return false; + if (filterDateRange && log.created_at) { + const days = Math.floor((Date.now() - new Date(log.created_at).getTime()) / 86400000); + if (filterDateRange === '1d' && days > 1) return false; + if (filterDateRange === '7d' && days > 7) return false; + if (filterDateRange === '30d' && days > 30) return false; + } + if (searchQuery) { + const q = searchQuery.toLowerCase(); + return ( + log.username.toLowerCase().includes(q) || + (log.detail || '').toLowerCase().includes(q) || + (log.ip_address || '').toLowerCase().includes(q) || + (ACTION_LABELS[log.action] || log.action).toLowerCase().includes(q) + ); + } + return true; + }); + }, [logs, filterTarget, filterStatus, filterDateRange, searchQuery]); + + const totalPages = Math.max(1, Math.ceil(filteredLogs.length / pageSize)); + const paginatedLogs = filteredLogs.slice((page - 1) * pageSize, page * pageSize); + + const hasActiveFilters = filterAction || filterTarget || filterStatus || filterDateRange || searchQuery; + + const clearFilters = () => { + setFilterAction(''); + setFilterTarget(''); + setFilterStatus(''); + setFilterDateRange(''); + setSearchQuery(''); + setPage(1); }; - const totalPages = Math.ceil(total / pageSize); + // Stats data + const actionDistribution = useMemo(() => { + const map: Record = {}; + logs.forEach(l => { map[l.action] = (map[l.action] || 0) + 1; }); + return Object.entries(map).map(([name, value], i) => ({ name: ACTION_LABELS[name] || name, value, color: CHART_COLORS[i % CHART_COLORS.length] })); + }, [logs]); + + const statusDistribution = useMemo(() => { + const map: Record = { success: 0, failed: 0, warning: 0 }; + logs.forEach(l => { map[l.result_status] = (map[l.result_status] || 0) + 1; }); + return [ + { name: '成功', value: map.success, color: '#22c55e' }, + { name: '失败', value: map.failed, color: '#f43f5e' }, + { name: '警告', value: map.warning, color: '#eab308' }, + ]; + }, [logs]); + + const trendData = useMemo(() => { + const map: Record = {}; + logs.forEach(l => { + if (l.created_at) { + const d = new Date(l.created_at).toLocaleDateString('zh-CN', { month: '2-digit', day: '2-digit' }); + map[d] = (map[d] || 0) + 1; + } + }); + return Object.entries(map) + .sort(([a], [b]) => new Date(a).getTime() - new Date(b).getTime()) + .slice(-10) + .map(([date, count]) => ({ date, count })); + }, [logs]); const formatDate = (d: string | null) => { if (!d) return '-'; @@ -73,87 +247,343 @@ export default function AuditLogsPage() { }; return ( -
+
+ {/* Header */}
-

审计日志

- 共 {total} 条记录 +
+

审计日志

+

追踪系统操作记录，满足合规审计需求

+
+
+ + +
- {/* Filters */} -
- {['', 'CREATE_TICKET', 'REVIEW_TICKET', 'EXECUTE_TICKET', 'CREATE_INSTANCE', 'DELETE_INSTANCE', 'QUERY'].map(action => ( + {/* Stats Panel */} + {showStats && ( +
+ + + + + 操作类型分布 + + + + + + `${name}: ${value}`}> + {actionDistribution.map((entry, i) => )} + + + + + + + + + + + + 操作结果分布 + + + + + + `${name}: ${value}`}> + {statusDistribution.map((entry, i) => )} + + + + + + + + + + + + 操作趋势（最近10天） + + + + + + + + + + + + + + +
+ )} + + {/* Search & Filters */} +
+
+
+ + { setSearchQuery(e.target.value); setPage(1); }} + className="w-full pl-10 pr-4 py-2.5 bg-[var(--dg-bg-tertiary)] border border-[var(--dg-border)] rounded-lg text-sm placeholder:text-[var(--dg-text-muted)] focus:border-blue-400 focus:outline-none focus:ring-1 focus:ring-blue-400/20" + placeholder="搜索操作人、IP、详情..." + /> +
+ {hasActiveFilters && ( + + )} + 共 {filteredLogs.length} 条记录 +
+ + {showFilters && ( +
+ {/* Action type filter */} + + + {/* Target type filter */} + + + {/* Status filter */} + + + {/* Date range */} + +
+ )} +
+ + {/* Quick Action Filters */} +
+ {['', ...Object.keys(ACTION_LABELS)].map(action => ( + ))}
{/* Table */} -
- - - - - - - - - - - {loading ? ( - - ) : logs.length === 0 ? ( - - ) : ( - logs.map(log => ( - - - - - + + +
+
操作目标详情时间
加载中...
暂无审计日志
- - {log.action} - - - {TARGET_ICONS[log.target_type] || '📌'} - {log.target_type} - {log.target_id && #{log.target_id}} - {log.detail || '-'}{formatDate(log.created_at)}
+ + + + + + + + - )) - )} - -
+ 操作操作人目标IP地址结果时间
-
+ + + {loading ? ( + + +
+ 加载中... + + + ) : paginatedLogs.length === 0 ? ( + + + +

暂无审计日志

+ {hasActiveFilters && ( + + )} + + + ) : ( + paginatedLogs.map(log => { + const isExpanded = expandedId === log.id; + const statusConfig = STATUS_CONFIG[log.result_status] || STATUS_CONFIG.success; + const StatusIcon = statusConfig.icon; + const TargetIcon = TARGET_ICONS[log.target_type] || FileText; + return ( + <> + setExpandedId(isExpanded ? null : log.id)} + > + + {isExpanded ? : } + + + + {ACTION_LABELS[log.action] || log.action} + + + +
+
+ {log.username.charAt(0).toUpperCase()} +
+ {log.username} +
+ + +
+ + {log.target_type} + {log.target_id && #{log.target_id}} +
+ + + {log.ip_address || '-'} + + + + + {statusConfig.label} + + + + {formatDate(log.created_at)} + + - {/* Pagination */} - {totalPages > 1 && ( -
- - {page} / {totalPages} - -
- )} + {/* Expanded Detail */} + {isExpanded && ( + + +
+
+
操作详情
+

{log.detail || '无详情'}

+
+
+
+
日志ID
+
#{log.id}
+
+
+
用户ID
+
#{log.user_id}
+
+
+
目标ID
+
{log.target_id || 'N/A'}
+
+
+
时间
+
{formatDate(log.created_at)}
+
+
+
+ + + )} + + ); + }) + )} + + +
+ + {/* Pagination */} + {totalPages > 1 && ( +
+ + 第 {page} / {totalPages} 页，共 {filteredLogs.length} 条 + +
+ + +
+
+ )} + +
); -} \ No newline at end of file +} diff --git a/frontend/src/pages/AuditRulesPage.tsx b/frontend/src/pages/AuditRulesPage.tsx index bafb832..7e76b7c 100644 --- a/frontend/src/pages/AuditRulesPage.tsx +++ b/frontend/src/pages/AuditRulesPage.tsx @@ -1,5 +1,14 @@ -import { useState, useEffect, useCallback } from 'react'; +import { useState, useEffect, useCallback, useMemo } from 'react'; import request from '../api/request'; +import { + Search, Filter, ChevronDown, ChevronUp, AlertTriangle, + Info, CheckCircle, XCircle, Database, Table, FileSearch, + Shield, Loader2, Save, RotateCcw, Leaf, Server, +} from 'lucide-react'; + +/* ------------------------------------------------------------------ */ +/* Types */ +/* ------------------------------------------------------------------ */ interface AuditRule { id: number; @@ -16,23 +25,90 @@ interface AuditRule { updated_at: string; } -const DB_TABS = [ - { key: 'mysql', label: 'MySQL', icon: '🐬' }, - { key: 'postgresql', label: 'PostgreSQL', icon: '🐘' }, - { key: 'redis', label: 'Redis', icon: '🔴' }, -] as const; +type DbType = 'mysql' | 'postgresql' | 'redis' | 'mongodb'; +type RuleGroup = 'DDL' | 'DML' | 'DCL' | 'DQL'; +type SeverityFilter = 'all' | 'error' | 'warning' | 'info'; -type DbType = (typeof DB_TABS)[number]['key']; +/* ------------------------------------------------------------------ */ +/* Constants */ +/* ------------------------------------------------------------------ */ -function isDDL(name: string): boolean { - const ddlKeywords = [ - 'DDL', 'CREATE', 'ALTER', 'DROP', 'TRUNCATE', 'RENAME', - 'TABLE', 'INDEX', 'KEY', 'COLUMN', 'PARTITION', - ]; - const upper = name.toUpperCase(); - return ddlKeywords.some((kw) => upper.includes(kw)); +interface DbTab { + key: DbType; + label: string; + icon: React.ElementType; } +const DB_TABS: DbTab[] = [ + { key: 'mysql', label: 'MySQL', icon: Database }, + { key: 'postgresql', label: 'PostgreSQL', icon: Server }, + { key: 'redis', label: 'Redis', icon: Table }, + { key: 'mongodb', label: 'MongoDB', icon: Leaf }, +]; + +const GROUP_CONFIG: Record = { + DDL: { label: 'DDL 规则', icon: Table, color: 'text-amber-400 bg-amber-500/15' }, + DML: { label: 'DML 规则', icon: Shield, color: 'text-blue-400 bg-blue-500/15' }, + DCL: { label: 'DCL 规则', icon: Shield, color: 'text-purple-400 bg-purple-500/15' }, + DQL: { label: 'DQL 规则', icon: FileSearch, color: 'text-green-400 bg-green-500/15' }, +}; + +const SEVERITY_CONFIG: Record = { + error: { label: '错误', color: 'text-red-400', dot: 'bg-red-400' }, + warning: { label: '警告', color: 'text-amber-400', dot: 'bg-amber-400' }, + info: { label: '提示', color: 'text-blue-400', dot: 'bg-blue-400' }, +}; + +const DDL_KEYWORDS = [ + 'DDL', 'CREATE', 'ALTER', 'DROP', 'TRUNCATE', 'RENAME', + 'TABLE', 'INDEX', 'KEY', 'COLUMN', 'PARTITION', 'VIEW', + 'DATABASE', 'SCHEMA', 'SEQUENCE', '集合', 'COLLECTION', + ' Validator', 'CAPPED', 'SCHEMA', +]; + +const DML_KEYWORDS = ['DML', 'INSERT', 'UPDATE', 'DELETE', 'REPLACE', 'MERGE', 'BULK', '写入', 'UNSET']; + +const DCL_KEYWORDS = ['DCL', 'GRANT', 'REVOKE', 'DENY']; + +const DQL_KEYWORDS = ['DQL', 'SELECT', 'EXPLAIN', 'DESCRIBE', 'SHOW', 'FIND', '查询', '聚合', 'AGGREGAT', 'PROJECTION', 'FILTER', 'REGEX', '$WHERE']; + +/* ------------------------------------------------------------------ */ +/* Helpers */ +/* ------------------------------------------------------------------ */ + +function getRuleGroup(name: string): RuleGroup { + const upper = name.toUpperCase(); + if (DDL_KEYWORDS.some((kw) => upper.includes(kw))) return 'DDL'; + if (DML_KEYWORDS.some((kw) => upper.includes(kw))) return 'DML'; + if (DCL_KEYWORDS.some((kw) => upper.includes(kw))) return 'DCL'; + if (DQL_KEYWORDS.some((kw) => upper.includes(kw))) return 'DQL'; + // MongoDB security rules -> DCL, performance rules -> DQL + if (upper.includes('SECURITY') || upper.includes('安全') || upper.includes('审计') || upper.includes('注入') || upper.includes('EVAL') || upper.includes('MAPSQL_REDUCE')) return 'DCL'; + if (upper.includes('PERFORMANCE') || upper.includes('性能') || upper.includes('索引') || upper.includes('扫描') || upper.includes('分片') || upper.includes('SHARD')) return 'DQL'; + // Default to DDL for structural rules, DML for data rules + if (upper.includes('WHERE') || upper.includes('写入') || upper.includes('UPDATE') || upper.includes('DELETE')) return 'DML'; + return 'DDL'; +} + +function sortRules(rules: AuditRule[], sortBy: 'severity' | 'name' | 'type'): AuditRule[] { + const severityOrder = { error: 0, warning: 1, info: 2 }; + const sorted = [...rules]; + switch (sortBy) { + case 'severity': + return sorted.sort((a, b) => (severityOrder[a.severity as keyof typeof severityOrder] ?? 3) - (severityOrder[b.severity as keyof typeof severityOrder] ?? 3)); + case 'name': + return sorted.sort((a, b) => a.name.localeCompare(b.name, 'zh-CN')); + case 'type': + return sorted.sort((a, b) => getRuleGroup(a.name).localeCompare(getRuleGroup(b.name))); + default: + return sorted; + } +} + +/* ------------------------------------------------------------------ */ +/* Component */ +/* ------------------------------------------------------------------ */ + export default function AuditRulesPage() { const [activeTab, setActiveTab] = useState('mysql'); const [rules, setRules] = useState([]); @@ -41,13 +117,27 @@ export default function AuditRulesPage() { const [editingValue, setEditingValue] = useState>({}); const [saving, setSaving] = useState>({}); + // -- Search & Filter -- + const [searchQuery, setSearchQuery] = useState(''); + const [severityFilter, setSeverityFilter] = useState('all'); + const [sortBy, setSortBy] = useState<'severity' | 'name' | 'type'>('severity'); + + // -- Expand & Confirm -- + const [expandedRuleId, setExpandedRuleId] = useState(null); + const [confirmToggle, setConfirmToggle] = useState(null); + + /* ---------------------------------------------------------------- */ + /* Fetch rules */ + /* ---------------------------------------------------------------- */ + const fetchRules = useCallback(async (dbType: DbType) => { try { setLoading(true); const res = await request.get('/audit-rules', { params: { db_type: dbType }, }); - setRules(Array.isArray(res.data) ? res.data : Array.isArray(res) ? res : []); + const data = (res as { data?: unknown[] }).data ?? (res as unknown[]); + setRules(Array.isArray(data) ? data as AuditRule[] : []); } catch { setRules([]); } finally { @@ -57,32 +147,45 @@ export default function AuditRulesPage() { useEffect(() => { fetchRules(activeTab); + setSearchQuery(''); + setSeverityFilter('all'); + setExpandedRuleId(null); }, [activeTab, fetchRules]); - const handleSeed = async () => { + /* ---------------------------------------------------------------- */ + /* Handlers */ + /* ---------------------------------------------------------------- */ + + const handleSeed = useCallback(async () => { try { setSeeding(true); await request.post(`/audit-rules/seed/${activeTab}`); fetchRules(activeTab); - } catch (err: any) { - alert(err.response?.data?.detail || '初始化失败'); + } catch (err: unknown) { + const e = err as { response?: { data?: { detail?: string } } }; + alert(e.response?.data?.detail || '初始化失败'); } finally { setSeeding(false); } - }; + }, [activeTab, fetchRules]); - const handleToggle = async (rule: AuditRule) => { + const handleToggle = useCallback(async (rule: AuditRule) => { + if (confirmToggle !== rule.id) { + setConfirmToggle(rule.id); + return; + } try { await request.post(`/audit-rules/${rule.id}/toggle`); setRules((prev) => prev.map((r) => (r.id === rule.id ? { ...r, is_enabled: !r.is_enabled } : r)), ); + setConfirmToggle(null); } catch { /* ignore */ } - }; + }, [confirmToggle]); - const handleSaveValue = async (rule: AuditRule) => { + const handleSaveValue = useCallback(async (rule: AuditRule) => { const newVal = editingValue[rule.id]; if (newVal === undefined || newVal === rule.value) return; try { @@ -101,147 +204,404 @@ export default function AuditRulesPage() { } finally { setSaving((prev) => ({ ...prev, [rule.id]: false })); } + }, [editingValue]); + + /* ---------------------------------------------------------------- */ + /* Derived: filtered & grouped rules */ + /* ---------------------------------------------------------------- */ + + const filteredRules = useMemo(() => { + let result = [...rules]; + + // Search + if (searchQuery.trim()) { + const q = searchQuery.toLowerCase(); + result = result.filter( + (r) => + r.name.toLowerCase().includes(q) || + r.message.toLowerCase().includes(q) || + (r.description && r.description.toLowerCase().includes(q)), + ); + } + + // Severity filter + if (severityFilter !== 'all') { + result = result.filter((r) => r.severity === severityFilter); + } + + // Sort + return sortRules(result, sortBy); + }, [rules, searchQuery, severityFilter, sortBy]); + + const groupedRules = useMemo(() => { + const groups: Record = { + DDL: [], + DML: [], + DCL: [], + DQL: [], + }; + for (const rule of filteredRules) { + const group = getRuleGroup(rule.name); + groups[group].push(rule); + } + return groups; + }, [filteredRules]); + + /* ---------------------------------------------------------------- */ + /* Render helpers */ + /* ---------------------------------------------------------------- */ + + const renderRuleCard = (rule: AuditRule) => { + const hasValue = rule.value !== null && rule.value !== undefined && rule.value !== ''; + const currentEditVal = editingValue[rule.id]; + const displayValue = currentEditVal !== undefined ? currentEditVal : rule.value ?? ''; + const isExpanded = expandedRuleId === rule.id; + const severityInfo = SEVERITY_CONFIG[rule.severity] || SEVERITY_CONFIG.info; + const isConfirmingToggle = confirmToggle === rule.id; + + return ( +
+ {/* Main row */} +
setExpandedRuleId(isExpanded ? null : rule.id)} + > + {/* Toggle */} + + + {/* Severity dot */} + + + {/* Name + Message */} +
+
+

+ {rule.name} +

+ + {severityInfo.label} + +
+

{rule.message}

+
+ + {/* Value input */} + {hasValue && ( +
e.stopPropagation()}> + = + + setEditingValue((prev) => ({ ...prev, [rule.id]: e.target.value })) + } + onBlur={() => handleSaveValue(rule)} + onKeyDown={(e) => { + if (e.key === 'Enter') handleSaveValue(rule); + }} + disabled={saving[rule.id]} + className="w-20 px-2 py-1 text-sm font-mono text-center rounded-md border border-[var(--dg-border)] bg-[var(--dg-bg-primary)] text-[var(--dg-text-primary)] focus:border-[var(--dg-blue)] focus:outline-none focus:ring-1 focus:ring-[var(--dg-input-focus-ring)] disabled:opacity-50 transition-all" + /> + {saving[rule.id] && } +
+ )} + + {/* Expand icon */} + {isExpanded ? ( + + ) : ( + + )} +
+ + {/* Toggle confirmation */} + {isConfirmingToggle && ( +
+ + + 确认{rule.is_enabled ? '禁用' : '启用'}规则 "{rule.name}"？ + + + +
+ )} + + {/* Expanded details */} + {isExpanded && ( +
+
+ {/* Description */} + {rule.description && ( +
+ +
+ 规则说明 +

{rule.description}

+
+
+ )} + + {/* Message */} +
+ +
+ 触发消息 +

{rule.message}

+
+
+ + {/* Pattern */} + {rule.pattern && ( +
+ +
+ 匹配模式 + + {rule.pattern} + +
+
+ )} + + {/* Metadata */} +
+ + 类型: {rule.rule_type.toUpperCase()} + + + 更新于: {new Date(rule.updated_at).toLocaleDateString()} + +
+
+
+ )} +
+ ); }; - // Group rules into DDL / DML - const ddlRules = rules.filter((r) => isDDL(r.name)); - const dmlRules = rules.filter((r) => !isDDL(r.name)); - - const renderRuleGroup = (title: string, icon: string, groupRules: AuditRule[]) => { + const renderRuleGroup = (group: RuleGroup, groupRules: AuditRule[]) => { if (groupRules.length === 0) return null; + const config = GROUP_CONFIG[group]; + const GroupIcon = config.icon; + return ( -
-

- {icon} - {title} - ({groupRules.length}) +
+

+ + {config.label} + ({groupRules.length})

- {groupRules.map((rule) => { - const hasValue = rule.value !== null && rule.value !== undefined && rule.value !== ''; - const currentEditVal = editingValue[rule.id]; - const displayValue = currentEditVal !== undefined ? currentEditVal : rule.value ?? ''; - - return ( -
- {/* Toggle */} - - - {/* Name + Message */} -
-

- {rule.name} -

-

- {rule.message} -

-
- - {/* Value input (if applicable) */} - {hasValue && ( -
- = - - setEditingValue((prev) => ({ ...prev, [rule.id]: e.target.value })) - } - onBlur={() => handleSaveValue(rule)} - onKeyDown={(e) => { - if (e.key === 'Enter') handleSaveValue(rule); - }} - disabled={saving[rule.id]} - className="w-20 px-2 py-1 text-sm font-mono text-center rounded-md border border-[var(--dg-border)] bg-[var(--dg-bg-primary)] text-[var(--dg-text-primary)] focus:border-[var(--dg-blue)] focus:outline-none focus:ring-1 focus:ring-[var(--dg-input-focus-ring)] disabled:opacity-50" - /> - {saving[rule.id] && ( - 保存中… - )} -
- )} -
- ); - })} + {groupRules.map((rule) => renderRuleCard(rule))}
); }; + /* ---------------------------------------------------------------- */ + /* Main render */ + /* ---------------------------------------------------------------- */ + return (
{/* Header */} -
+

审核规则

-

按数据库类型管理 SQL 审核规则模版

+

按数据库类型管理 SQL 审核规则模版，支持搜索、筛选和排序

{/* DB Type Tabs */} -
- {DB_TABS.map((tab) => ( - - ))} +
+ {DB_TABS.map((tab) => { + const TabIcon = tab.icon; + return ( + + ); + })}
+ {/* Search & Filter Bar */} +
+ {/* Search */} +
+ + setSearchQuery(e.target.value)} + placeholder="搜索规则名称、消息或描述..." + className="w-full pl-9 pr-3 py-2 bg-[var(--dg-bg-primary)] border border-[var(--dg-border)] rounded-lg text-sm text-[var(--dg-text-primary)] placeholder:text-[var(--dg-text-muted)] focus:outline-none focus:border-blue-500 transition-colors" + /> + {searchQuery && ( + + )} +
+ + {/* Severity filter */} +
+ + {([ + { value: 'all', label: '全部' }, + { value: 'error', label: '错误' }, + { value: 'warning', label: '警告' }, + { value: 'info', label: '提示' }, + ] as { value: SeverityFilter; label: string }[]).map((opt) => ( + + ))} +
+ + {/* Sort */} +
+ 排序: + +
+
+ + {/* Results summary */} + {(searchQuery || severityFilter !== 'all') && ( +
+ + 共 {filteredRules.length} 条规则 + {searchQuery && 匹配 "{searchQuery}"} + {severityFilter !== 'all' && 严重级别: {SEVERITY_CONFIG[severityFilter]?.label || severityFilter}} + +
+ )} + {/* Rule List */} {loading ? ( -
加载中…
+
+ + 加载中... +
) : rules.length === 0 ? (
+

当前没有 {DB_TABS.find((t) => t.key === activeTab)?.label} 的审核规则

+ ) : filteredRules.length === 0 ? ( +
+ +

没有匹配的规则

+ +
) : (
- {renderRuleGroup('DDL 规则', '🏗️', ddlRules)} - {renderRuleGroup('DML 规则', '📝', dmlRules)} + {renderRuleGroup('DDL', groupedRules.DDL)} + {renderRuleGroup('DML', groupedRules.DML)} + {renderRuleGroup('DCL', groupedRules.DCL)} + {renderRuleGroup('DQL', groupedRules.DQL)}
)}
); } + +/* Small icon component */ +function MessageSquareIcon(props: React.SVGProps & { className?: string }) { + return ( + + + + ); +} diff --git a/frontend/src/pages/Dashboard.tsx b/frontend/src/pages/Dashboard.tsx index 296b605..52de22e 100644 --- a/frontend/src/pages/Dashboard.tsx +++ b/frontend/src/pages/Dashboard.tsx @@ -1,8 +1,10 @@ -import { useEffect, useRef, useState } from 'react'; +import { useEffect, useRef, useState, useCallback, useMemo } from 'react'; import { ClipboardList, Clock, CheckCircle, AlertTriangle, - Database, ChevronRight, - X, AlertCircle, Info, Bell, Activity, TrendingUp, Zap, Server, + Database, ChevronRight, X, AlertCircle, Info, Bell, + Activity, TrendingUp, Zap, Server, Plus, Search, + FileText, ArrowUpRight, ArrowDownRight, Minus, + Filter, Eye, Loader2, BarChart3, } from 'lucide-react'; import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; import { dbInstances, statusMap, statusColorMap, dbTypeColorMap, tickets, announcements } from '@/lib/mock'; @@ -12,56 +14,182 @@ import { dashboardApi } from '@/api/dashboard'; import type { DashboardStats } from '@/types'; import * as echarts from 'echarts'; -const priorityStyle: Record = { +/* ------------------------------------------------------------------ */ +/* Types */ +/* ------------------------------------------------------------------ */ +type PriorityKey = 'urgent' | 'high' | 'normal' | 'low'; +type TrendRange = 7 | 30 | 90; +type AnnouncementFilter = 'all' | PriorityKey; + +interface TrendDataset { + xAxis: string[]; + submitted: number[]; + success: number[]; + failed: number[]; +} + +/* ------------------------------------------------------------------ */ +/* Constants */ +/* ------------------------------------------------------------------ */ +const priorityStyle: Record = { urgent: { border: 'border-l-red-500', bg: 'bg-red-500/5', icon: AlertCircle, iconColor: 'text-red-400', badge: 'bg-red-500/15 text-red-400 border-red-500/30' }, high: { border: 'border-l-orange-500', bg: 'bg-orange-500/5', icon: AlertTriangle, iconColor: 'text-orange-400', badge: 'bg-orange-500/15 text-orange-400 border-orange-500/30' }, normal: { border: 'border-l-blue-500', bg: 'bg-blue-500/5', icon: Info, iconColor: 'text-blue-400', badge: 'bg-blue-500/15 text-blue-400 border-blue-500/30' }, low: { border: 'border-l-gray-500', bg: 'bg-gray-500/5', icon: Bell, iconColor: 'text-gray-400', badge: 'bg-gray-500/15 text-gray-400 border-gray-500/30' }, }; -const priorityLabel: Record = { urgent: '紧急', high: '重要', normal: '通知', low: '一般' }; +const priorityLabel: Record = { urgent: '紧急', high: '重要', normal: '通知', low: '一般' }; +const trendDatasets: Record = { + 7: { + xAxis: ['周一', '周二', '周三', '周四', '周五', '周六', '周日'], + submitted: [120, 132, 101, 134, 90, 230, 210], + success: [110, 125, 95, 128, 85, 215, 200], + failed: [5, 3, 4, 2, 3, 8, 5], + }, + 30: { + xAxis: Array.from({ length: 15 }, (_, i) => `${i * 2 + 1}日`), + submitted: [85, 92, 78, 105, 120, 95, 88, 110, 125, 140, 95, 82, 115, 130, 145], + success: [80, 88, 75, 98, 112, 90, 85, 105, 118, 132, 90, 78, 108, 125, 138], + failed: [3, 2, 2, 5, 4, 3, 2, 3, 4, 5, 3, 2, 4, 3, 4], + }, + 90: { + xAxis: Array.from({ length: 12 }, (_, i) => `${i * 7 + 1}日`), + submitted: [520, 480, 610, 590, 720, 680, 750, 800, 720, 650, 580, 620], + success: [490, 455, 580, 560, 685, 650, 720, 770, 695, 625, 555, 598], + failed: [18, 15, 22, 20, 25, 18, 20, 22, 18, 16, 14, 15], + }, +}; + +/* ------------------------------------------------------------------ */ +/* Helpers */ +/* ------------------------------------------------------------------ */ +function calcHealthScore(inst: typeof dbInstances[0]): number { + if (inst.status === 'offline') return 0; + let score = 100; + if (inst.cpu > 80) score -= 25; + else if (inst.cpu > 60) score -= 10; + if (inst.memory > 85) score -= 20; + else if (inst.memory > 70) score -= 10; + if (inst.latency > 10) score -= 20; + else if (inst.latency > 5) score -= 10; + if (inst.disk > 80) score -= 15; + return Math.max(0, score); +} + +function healthColor(score: number): string { + if (score >= 90) return 'text-green-400'; + if (score >= 70) return 'text-yellow-400'; + if (score >= 50) return 'text-orange-400'; + return 'text-red-400'; +} + +function healthBg(score: number): string { + if (score >= 90) return 'bg-green-500/15'; + if (score >= 70) return 'bg-yellow-500/15'; + if (score >= 50) return 'bg-orange-500/15'; + return 'bg-red-500/15'; +} + +/* ------------------------------------------------------------------ */ +/* Skeleton Components */ +/* ------------------------------------------------------------------ */ +function StatCardSkeleton() { + return ( + + +
+
+
+
+
+
+ + + ); +} + +function ChartSkeleton() { + return ( +
+
+
+ ); +} + +/* ------------------------------------------------------------------ */ +/* Main Dashboard */ +/* ------------------------------------------------------------------ */ export default function Dashboard() { const chartRef = useRef(null); + const chartInstanceRef = useRef(null); const { theme } = useTheme(); const { user } = useAuth(); const [dismissedIds, setDismissedIds] = useState>(new Set()); + const [readIds, setReadIds] = useState>(new Set()); const [expandedId, setExpandedId] = useState(null); + const [annFilter, setAnnFilter] = useState('all'); + const [trendRange, setTrendRange] = useState(7); const [stats, setStats] = useState(null); const [loading, setLoading] = useState(true); + const [chartReady, setChartReady] = useState(false); + /* ---- Data Fetch ---- */ useEffect(() => { dashboardApi.getStats() .then(data => setStats(data)) .catch(() => { - // fallback: keep stats null, show skeleton + /* fallback: mock data */ + setStats({ + ticket_total: 128, ticket_pending: 23, ticket_month: 342, + ticket_executing: 3, instance_total: 11, instance_online: 8, + user_total: 7, queries_today: 15234, + }); }) .finally(() => setLoading(false)); }, []); - const visibleAnnouncements = announcements - .filter(a => a.enabled && a.targetRoles.includes(user.role) && !dismissedIds.has(a.id)) - .sort((a, b) => { - if (a.pinned !== b.pinned) return a.pinned ? -1 : 1; - const po = { urgent: 0, high: 1, normal: 2, low: 3 }; - if (po[a.priority] !== po[b.priority]) return po[a.priority] - po[b.priority]; - return new Date(b.publishTime).getTime() - new Date(a.publishTime).getTime(); + /* ---- Announcements ---- */ + const visibleAnnouncements = useMemo(() => { + return announcements + .filter(a => a.enabled && a.targetRoles.includes(user.role) && !dismissedIds.has(a.id)) + .filter(a => annFilter === 'all' || a.priority === annFilter) + .sort((a, b) => { + if (a.pinned !== b.pinned) return a.pinned ? -1 : 1; + const po: Record = { urgent: 0, high: 1, normal: 2, low: 3 }; + if (po[a.priority as PriorityKey] !== po[b.priority as PriorityKey]) return po[a.priority as PriorityKey] - po[b.priority as PriorityKey]; + return new Date(b.publishTime).getTime() - new Date(a.publishTime).getTime(); + }); + }, [dismissedIds, annFilter, user.role]); + + const dismiss = useCallback((id: string) => setDismissedIds(prev => new Set(prev).add(id)), []); + const toggleRead = useCallback((id: string) => { + setReadIds(prev => { + const next = new Set(prev); + if (next.has(id)) next.delete(id); else next.add(id); + return next; }); + }, []); + const toggleExpand = useCallback((id: string) => setExpandedId(prev => prev === id ? null : id), []); - const dismiss = (id: string) => setDismissedIds(prev => new Set(prev).add(id)); - const toggleExpand = (id: string) => setExpandedId(prev => prev === id ? null : id); - + /* ---- Chart ---- */ useEffect(() => { if (!chartRef.current) return; - const chart = echarts.init(chartRef.current); + if (!chartInstanceRef.current) { + chartInstanceRef.current = echarts.init(chartRef.current); + } + const chart = chartInstanceRef.current; const isLight = theme === 'light'; const colors = isLight ? { bg: 'rgba(255,255,255,0.95)', border: '#e2e5eb', text: '#1a1d23', muted: '#5c6370', grid: '#f0f1f5', blue: '#2563eb', green: '#16a34a', red: '#dc2626' } : { bg: 'rgba(22, 27, 34, 0.95)', border: '#30363d', text: '#f0f6fc', muted: '#8b949e', grid: '#21262d', blue: '#58a6ff', green: '#3fb950', red: '#f85149' }; - const option: any = { + + const dataset = trendDatasets[trendRange]; + + const option: echarts.EChartsOption = { backgroundColor: 'transparent', tooltip: { - trigger: 'axis' as const, + trigger: 'axis', backgroundColor: colors.bg, borderColor: colors.border, textStyle: { color: colors.text }, @@ -75,13 +203,13 @@ export default function Dashboard() { }, grid: { left: '2%', right: '3%', bottom: '18%', top: '8%', containLabel: true }, xAxis: { - type: 'category' as const, - data: ['周一', '周二', '周三', '周四', '周五', '周六', '周日'], + type: 'category', + data: dataset.xAxis, axisLine: { lineStyle: { color: colors.border } }, axisLabel: { color: colors.muted, fontSize: 11 }, }, yAxis: { - type: 'value' as const, + type: 'value', axisLine: { show: false }, splitLine: { lineStyle: { color: colors.grid } }, axisLabel: { color: colors.muted, fontSize: 11 }, @@ -89,7 +217,7 @@ export default function Dashboard() { series: [ { name: '提交工单', type: 'line', smooth: true, - data: [120, 132, 101, 134, 90, 230, 210], + data: dataset.submitted, itemStyle: { color: colors.blue }, lineStyle: { width: 2 }, areaStyle: { @@ -102,7 +230,7 @@ export default function Dashboard() { }, { name: '执行成功', type: 'line', smooth: true, - data: [110, 125, 95, 128, 85, 215, 200], + data: dataset.success, itemStyle: { color: colors.green }, lineStyle: { width: 2 }, areaStyle: { @@ -115,45 +243,144 @@ export default function Dashboard() { }, { name: '执行失败', type: 'line', smooth: true, - data: [5, 3, 4, 2, 3, 8, 5], + data: dataset.failed, itemStyle: { color: colors.red }, lineStyle: { width: 2 }, symbol: 'none', }, ], }; - chart.setOption(option); + chart.setOption(option, true); + setChartReady(true); + const handleResize = () => chart.resize(); window.addEventListener('resize', handleResize); - return () => { window.removeEventListener('resize', handleResize); chart.dispose(); }; - }, [theme]); + return () => { window.removeEventListener('resize', handleResize); }; + }, [theme, trendRange]); + useEffect(() => { + return () => { chartInstanceRef.current?.dispose(); }; + }, []); + + /* ---- Computed Data ---- */ const onlineInstances = dbInstances.filter(i => i.status === 'online').length; const warningInstances = dbInstances.filter(i => i.status === 'warning').length; const offlineInstances = dbInstances.filter(i => i.status === 'offline').length; - const pendingTickets = tickets.filter(t => t.status === 'pending' || t.status === 'reviewing').slice(0, 4); + const pendingTickets = tickets.filter(t => t.status === 'pending' || t.status === 'reviewing').slice(0, 5); - // Stats cards - use real API data - const statsCards = stats ? [ - { title: '工单总数', value: stats.ticket_total, icon: ClipboardList, color: 'text-blue-400', bg: 'bg-blue-400/10' }, - { title: '待审批', value: stats.ticket_pending, icon: Clock, color: 'text-orange-400', bg: 'bg-orange-400/10' }, - { title: '本月工单', value: stats.ticket_month, icon: CheckCircle, color: 'text-purple-400', bg: 'bg-purple-400/10' }, - { title: '今日查询', value: stats.queries_today.toLocaleString(), icon: AlertTriangle, color: 'text-red-400', bg: 'bg-red-400/10' }, - ] : []; + const statsCards = useMemo(() => { + if (!stats) return []; + return [ + { + title: '工单总数', value: stats.ticket_total, icon: ClipboardList, + color: 'text-blue-400', bg: 'bg-blue-400/10', border: 'border-blue-400/20', + change: '+8.2%', trend: 'up' as const, + subtitle: `${stats.ticket_pending} 个待审批`, + }, + { + title: '待审批', value: stats.ticket_pending, icon: Clock, + color: 'text-orange-400', bg: 'bg-orange-400/10', border: 'border-orange-400/20', + change: '+2', trend: 'up' as const, + subtitle: `本月已处理 ${stats.ticket_month}`, + }, + { + title: '本月工单', value: stats.ticket_month, icon: CheckCircle, + color: 'text-purple-400', bg: 'bg-purple-400/10', border: 'border-purple-400/20', + change: '-3.1%', trend: 'down' as const, + subtitle: `${stats.ticket_executing} 个执行中`, + }, + { + title: '今日查询', value: stats.queries_today.toLocaleString(), icon: Zap, + color: 'text-green-400', bg: 'bg-green-400/10', border: 'border-green-400/20', + change: '+12%', trend: 'up' as const, + subtitle: `${stats.instance_online}/${stats.instance_total} 实例在线`, + }, + ]; + }, [stats]); + const quickActions = [ + { label: '新建工单', icon: FileText, href: '/tickets', color: 'text-blue-400', bg: 'bg-blue-400/10', border: 'border-blue-400/20' }, + { label: '新建查询', icon: Search, href: '/queries', color: 'text-green-400', bg: 'bg-green-400/10', border: 'border-green-400/20' }, + { label: '添加实例', icon: Database, href: '/instances', color: 'text-purple-400', bg: 'bg-purple-400/10', border: 'border-purple-400/20' }, + { label: '查看报表', icon: BarChart3, href: '/slow-query', color: 'text-orange-400', bg: 'bg-orange-400/10', border: 'border-orange-400/20' }, + ]; + + const TrendIcon = ({ trend }: { trend: 'up' | 'down' | 'neutral' }) => { + if (trend === 'up') return ; + if (trend === 'down') return ; + return ; + }; + + const trendRangeOptions: { label: string; value: TrendRange }[] = [ + { label: '近7天', value: 7 }, + { label: '近30天', value: 30 }, + { label: '近90天', value: 90 }, + ]; + + const annFilterOptions: { label: string; value: AnnouncementFilter }[] = [ + { label: '全部', value: 'all' }, + { label: '紧急', value: 'urgent' }, + { label: '重要', value: 'high' }, + { label: '通知', value: 'normal' }, + { label: '一般', value: 'low' }, + ]; + + /* ================================================================ */ + /* RENDER */ + /* ================================================================ */ return (
- {/* 公告通知栏 */} + {/* ---- Quick Actions ---- */} +
+ {quickActions.map(action => ( + + + {action.label} + + + ))} +
+ + {/* ---- Announcements ---- */} {visibleAnnouncements.length > 0 && (
+
+
+ + 公告通知 + ({visibleAnnouncements.length}) +
+
+ + {annFilterOptions.map(opt => ( + + ))} +
+
+ {visibleAnnouncements.slice(0, 3).map(ann => { - const style = priorityStyle[ann.priority] || priorityStyle.normal; + const style = priorityStyle[ann.priority as PriorityKey] || priorityStyle.normal; const PIcon = style.icon; const isExpanded = expandedId === ann.id; + const isRead = readIds.has(ann.id); return (
@@ -162,10 +389,11 @@ export default function Dashboard() {
- {priorityLabel[ann.priority]} + {priorityLabel[ann.priority as PriorityKey]} {ann.title} - {ann.pinned && 置顶} + {ann.pinned && 置顶} + {isRead && 已读}
{ann.content} @@ -181,10 +409,18 @@ export default function Dashboard() { )}
+ {ann.publishTime.slice(0, 10)} @@ -203,31 +439,28 @@ export default function Dashboard() {
)} - {/* 统计卡片 */} -
+ {/* ---- Stats Cards ---- */} +
{loading ? ( - Array.from({ length: 4 }).map((_, i) => ( - - -
- - - )) + Array.from({ length: 4 }).map((_, i) => ) ) : ( statsCards.map((s, i) => ( - - + +
+
{s.title} -
+
-
{s.value}
-
- - 实例 {stats?.instance_online}/{stats?.instance_total} 在线 · {stats?.user_total} 用户 +
{s.value}
+
+ + + {s.change} + {s.subtitle}
@@ -235,70 +468,107 @@ export default function Dashboard() { )}
- {/* 图表 + 实例状态 */} -
- {/* 工单趋势图 */} - + {/* ---- Charts + Instance Status ---- */} +
+ {/* Trend Chart */} +
- 工单趋势 + + + 工单趋势 +
- {['近7天', '近30天', '近90天'].map((p, i) => ( - ))}
-
+ {!chartReady ? :
} - {/* 实例状态 */} + {/* Instance Status */} - 实例状态 +
+ + + 实例状态 + + {onlineInstances}/{dbInstances.length} 在线 +
+ {/* Status Overview */}
-
+
{onlineInstances}
在线
-
+
{warningInstances}
告警
-
+
{offlineInstances}
离线
- {dbInstances.slice(0, 4).map(inst => ( -
-
- + {/* Instance List */} + {dbInstances.slice(0, 5).map(inst => { + const health = calcHealthScore(inst); + return ( +
+
+ +
+
+
+ {inst.name} + + {health} + +
+
+ {inst.host}:{inst.port} + + + {inst.qps.toLocaleString()} QPS + + {inst.connections} 连接 +
+
+
-
-
{inst.name}
-
{inst.host}:{inst.port}
-
- - {inst.status === 'online' ? '在线' : inst.status === 'warning' ? '告警' : '离线'} - -
- ))} + ); + })}
- {/* 数据源健康概览 + 查询统计 */} -
- {/* 数据源健康概览 */} + {/* ---- DataSource Health + Query Stats ---- */} +
+ {/* DataSource Health */}
@@ -310,20 +580,22 @@ export default function Dashboard() {
- {/* 按类型统计 */} + {/* By Type */}
- {(['mysql', 'postgresql', 'redis', 'mongodb'] as const).map(type => { + {(['mysql', 'postgres', 'redis', 'mongo'] as const).map(type => { const typeInstances = dbInstances.filter(i => i.dbType === type); const typeOnline = typeInstances.filter(i => i.status === 'online').length; - const iconMap: Record = { mysql: '/db-logos/mysql.svg', postgresql: '/db-logos/postgres.svg', redis: '/db-logos/redis.svg', mongodb: '/db-logos/mongo.svg' }; - const labelMap: Record = { mysql: 'MySQL', postgresql: 'PostgreSQL', redis: 'Redis', mongodb: 'MongoDB' }; + const iconMap: Record = { mysql: '/db-logos/mysql.svg', postgres: '/db-logos/postgres.svg', redis: '/db-logos/redis.svg', mongo: '/db-logos/mongo.svg' }; + const labelMap: Record = { mysql: 'MySQL', postgres: 'PostgreSQL', redis: 'Redis', mongo: 'MongoDB' }; + const allHealthy = typeOnline === typeInstances.length && typeInstances.length > 0; + const someHealthy = typeOnline > 0; return ( -
+
{typeInstances.length}
{labelMap[type]}
- 0 ? 'text-green-400' : typeOnline > 0 ? 'text-yellow-400' : 'text-[var(--dg-text-muted)]'}> + {typeOnline}/{typeInstances.length}
@@ -331,26 +603,34 @@ export default function Dashboard() { ); })}
- {/* 最近告警实例 */} + {/* Attention List */} {dbInstances.filter(i => i.status === 'warning' || i.status === 'offline').length > 0 && (
-
需要关注
- {dbInstances.filter(i => i.status === 'warning' || i.status === 'offline').slice(0, 3).map(inst => ( -
- - {inst.name} - {inst.host}:{inst.port} - - {inst.status === 'offline' ? '离线' : '告警'} - -
- ))} +
需要关注
+
+ {dbInstances.filter(i => i.status === 'warning' || i.status === 'offline').slice(0, 3).map(inst => { + const health = calcHealthScore(inst); + return ( +
+ + {inst.name} + {inst.host}:{inst.port} + + {health}分 + + + {inst.status === 'offline' ? '离线' : '告警'} + +
+ ); + })} +
)} - {/* 查询统计 */} + {/* Query Stats */}
@@ -359,8 +639,8 @@ export default function Dashboard() { 查询统计
- {['今日', '本周', '本月'].map((p, i) => ( - ))} @@ -368,36 +648,43 @@ export default function Dashboard() {
- {/* 统计指标 */}
总查询数
-
1,284
-
↑ 12% vs 昨日
+
1,284
+
+ + 12% vs 昨日 +
平均耗时
-
23ms
-
↓ 8% vs 昨日
+
23ms
+
+ + 8% vs 昨日 +
慢查询
-
7
-
↑ 3 vs 昨日
+
7
+
+ + 3 vs 昨日 +
- {/* 查询类型分布 */}
-
查询类型分布
+
查询类型分布
{[ { label: 'MySQL', count: 682, total: 1284, color: 'bg-[#00758f]' }, { label: 'PostgreSQL', count: 312, total: 1284, color: 'bg-[#336791]' }, @@ -405,11 +692,11 @@ export default function Dashboard() { { label: 'MongoDB', count: 92, total: 1284, color: 'bg-[#47a248]' }, ].map(item => (
- {item.label} + {item.label}
-
+
- {item.count} + {item.count}
))}
@@ -417,54 +704,77 @@ export default function Dashboard() {
- {/* 待处理工单 */} + {/* ---- Pending Tickets ---- */} - 待处理工单 +
+ + + 待处理工单 + ({pendingTickets.length}) + + + 查看全部 + + +
-
- - - - - - - - - - - - - - {pendingTickets.map(ticket => ( - - - - - - - - + {pendingTickets.length === 0 ? ( +
+ + 所有工单已处理完毕 +
+ ) : ( +
+
工单号标题数据库申请人状态创建时间操作
{ticket.id}{ticket.title} - - {ticket.dbType.toUpperCase()} - - {ticket.applicant} - - - {statusMap[ticket.status]} - - {ticket.createTime} -
- - -
-
+ + + + + + + + + - ))} - -
工单号标题数据库申请人状态创建时间操作
-
+ + + {pendingTickets.map(ticket => ( + + {ticket.id} + {ticket.title} + + + {ticket.dbType.toUpperCase()} + + + {ticket.applicant} + + + + {statusMap[ticket.status]} + + + {ticket.createTime} + +
+ + +
+ + + ))} + + +
+ )}
diff --git a/frontend/src/pages/DataDictionaryPage.tsx b/frontend/src/pages/DataDictionaryPage.tsx index 09127e2..94eb312 100644 --- a/frontend/src/pages/DataDictionaryPage.tsx +++ b/frontend/src/pages/DataDictionaryPage.tsx @@ -1,11 +1,14 @@ -import { useState, useEffect, useCallback } from 'react'; +import { useState, useEffect, useCallback, useMemo } from 'react'; import { - BookOpen, Database, Table2, Search, Download, Columns3, Key, Code2, RefreshCw, ChevronRight, + BookOpen, Database, Table2, Search, Download, Columns3, Key, Code2, RefreshCw, + ChevronRight, ChevronDown, Hash, AlignLeft, Gauge, FolderOpen, ArrowRight, + FileJson, TableProperties, Info, } from 'lucide-react'; import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; import { instanceApi } from '@/api/instance'; import { dictionaryApi } from '@/api/dictionary'; +/* ───────── Types ───────── */ interface ColumnInfo { Field: string; @@ -31,99 +34,189 @@ interface TableDetail { indexes: IndexInfo[]; create_ddl: string; row_count: number; + table_size?: string; + engine?: string; + charset?: string; + collation?: string; + comment?: string; } -type TabKey = 'columns' | 'indexes' | 'ddl'; +interface TableRelation { + from_table: string; + from_column: string; + to_table: string; + to_column: string; + constraint_name: string; +} + +type TabKey = 'columns' | 'indexes' | 'ddl' | 'relations'; + +/* ───────── Helpers ───────── */ + +function getTypeIcon(type: string): { icon: React.ElementType; color: string } { + const lower = type.toLowerCase(); + if (lower.includes('int') || lower.includes('float') || lower.includes('decimal') || lower.includes('double')) { + return { icon: Hash, color: 'text-blue-400' }; + } + if (lower.includes('char') || lower.includes('text') || lower.includes('blob')) { + return { icon: AlignLeft, color: 'text-green-400' }; + } + if (lower.includes('date') || lower.includes('time') || lower.includes('timestamp')) { + return { icon: Info, color: 'text-purple-400' }; + } + if (lower.includes('json')) { + return { icon: FileJson, color: 'text-yellow-400' }; + } + return { icon: Hash, color: 'text-[var(--dg-text-muted)]' }; +} + +function highlightDDL(ddl: string): string { + const keywords = [ + 'CREATE', 'TABLE', 'IF', 'NOT', 'EXISTS', 'PRIMARY', 'KEY', 'UNIQUE', 'INDEX', + 'FOREIGN', 'REFERENCES', 'CONSTRAINT', 'DEFAULT', 'NULL', 'AUTO_INCREMENT', + 'CHARSET', 'COLLATE', 'ENGINE', 'COMMENT', 'INT', 'VARCHAR', 'TEXT', 'DATETIME', + 'TIMESTAMP', 'BOOLEAN', 'DECIMAL', 'BIGINT', 'UNSIGNED', 'SIGNED', 'ON', 'DELETE', + 'UPDATE', 'CASCADE', 'SET', 'DROP', 'ALTER', 'ADD', 'COLUMN', 'MODIFY', 'CHANGE', + ]; + let highlighted = ddl + .replace(/&/g, '&') + .replace(//g, '>'); + + keywords.forEach(kw => { + const regex = new RegExp(`\\b${kw}\\b`, 'gi'); + highlighted = highlighted.replace(regex, `${kw}`); + }); + + highlighted = highlighted.replace(/'[^']*'/g, '$&'); + highlighted = highlighted.replace(/`[^`]*`/g, '$&'); + highlighted = highlighted.replace(/\b\d+\b/g, '$&'); + highlighted = highlighted.replace(/(--.*$)/gm, '$1'); + highlighted = highlighted.replace(/(\/\*[\s\S]*?\*\/)/g, '$1'); + + return highlighted; +} + +function generateMockRelations(tableName: string): TableRelation[] { + const relations: TableRelation[] = []; + if (tableName.includes('order')) { + relations.push( + { from_table: tableName, from_column: 'user_id', to_table: 'users', to_column: 'id', constraint_name: `${tableName}_user_id_fk` }, + { from_table: tableName, from_column: 'product_id', to_table: 'products', to_column: 'id', constraint_name: `${tableName}_product_id_fk` }, + ); + } + if (tableName.includes('user')) { + relations.push( + { from_table: 'orders', from_column: 'user_id', to_table: tableName, to_column: 'id', constraint_name: `orders_${tableName}_user_id_fk` }, + { from_table: 'comments', from_column: 'user_id', to_table: tableName, to_column: 'id', constraint_name: `comments_${tableName}_user_id_fk` }, + ); + } + if (tableName.includes('comment')) { + relations.push( + { from_table: tableName, from_column: 'post_id', to_table: 'posts', to_column: 'id', constraint_name: `${tableName}_post_id_fk` }, + ); + } + return relations; +} + +/* ───────── Tree Component ───────── */ + +function DbTreeNode({ + label, isActive, hasChildren, expanded, onClick, onToggle, depth, count, + icon: Icon, +}: { + label: string; + isActive: boolean; + hasChildren: boolean; + expanded: boolean; + onClick: () => void; + onToggle: () => void; + depth: number; + count?: number; + icon: React.ElementType; +}) { + return ( +
+ {hasChildren && ( + + )} + {!hasChildren &&
} + + {label} + {count !== undefined && {count}} +
+ ); +} + +/* ───────── Main Component ───────── */ export default function DataDictionaryPage() { - - // Left panel state - var selectedInstance = useState(null); - var instanceId = selectedInstance[0]; - var setInstanceId = selectedInstance[1]; - - var instanceListState = useState([]); - var instances = instanceListState[0]; - var setInstances = instanceListState[1]; - - var selectedDb = useState(''); - var currentDb = selectedDb[0]; - var setCurrentDb = selectedDb[1]; - - var databaseListState = useState([]); - var databases = databaseListState[0]; - var setDatabases = databaseListState[1]; - - var tableListState = useState([]); - var tables = tableListState[0]; - var setTables = tableListState[1]; - - var searchState = useState(''); - var searchFilter = searchState[0]; - var setSearchFilter = searchState[1]; - - // Right panel state - var selectedTableState = useState(''); - var selectedTable = selectedTableState[0]; - var setSelectedTable = selectedTableState[1]; - - var detailState = useState(null); - var tableDetail = detailState[0]; - var setTableDetail = detailState[1]; - - var tabState = useState('columns'); - var activeTab = tabState[0]; - var setActiveTab = tabState[1]; - - var loadingState = useState(false); - var loading = loadingState[0]; - var setLoading = loadingState[1]; + const [instanceId, setInstanceId] = useState(null); + const [instances, setInstances] = useState([]); + const [currentDb, setCurrentDb] = useState(''); + const [databases, setDatabases] = useState([]); + const [tables, setTables] = useState([]); + const [searchFilter, setSearchFilter] = useState(''); + const [selectedTable, setSelectedTable] = useState(''); + const [tableDetail, setTableDetail] = useState(null); + const [activeTab, setActiveTab] = useState('columns'); + const [loading, setLoading] = useState(false); + const [expandedDbs, setExpandedDbs] = useState>(new Set()); + const [dbTreeExpanded, setDbTreeExpanded] = useState(true); // Load instances - useEffect(function () { - instanceApi.list().then(function (data: any) { + useEffect(() => { + instanceApi.list().then((data: any) => { setInstances(Array.isArray(data) ? data : []); - }).catch(function () { - setInstances([]); - }); + }).catch(() => setInstances([])); }, []); - // Load databases when instance changes - var loadDatabases = useCallback(function () { + // Load databases + const loadDatabases = useCallback(() => { if (!instanceId) { setDatabases([]); setTables([]); setCurrentDb(''); return; } - dictionaryApi.listDatabases(instanceId).then(function (data: any) { - var dbList = data && data.databases ? data.databases : []; + dictionaryApi.listDatabases(instanceId).then((data: any) => { + const dbList = data && data.databases ? data.databases : []; setDatabases(dbList); if (dbList.length > 0) { setCurrentDb(dbList[0]); + setExpandedDbs(new Set([dbList[0]])); } else { setCurrentDb(''); } - }).catch(function () { + }).catch(() => { setDatabases([]); setCurrentDb(''); }); }, [instanceId]); - useEffect(function () { - loadDatabases(); - }, [loadDatabases]); + useEffect(() => { loadDatabases(); }, [loadDatabases]); - // Load tables when database changes - useEffect(function () { + // Load tables + useEffect(() => { if (!instanceId || !currentDb) { setTables([]); setSelectedTable(''); setTableDetail(null); return; } - dictionaryApi.listTables(instanceId, currentDb).then(function (data: any) { - var tblList = data && data.tables ? data.tables : []; + dictionaryApi.listTables(instanceId, currentDb).then((data: any) => { + const tblList = data && data.tables ? data.tables : []; setTables(tblList); if (tblList.length > 0) { setSelectedTable(tblList[0]); @@ -131,7 +224,7 @@ export default function DataDictionaryPage() { setSelectedTable(''); setTableDetail(null); } - }).catch(function () { + }).catch(() => { setTables([]); setSelectedTable(''); setTableDetail(null); @@ -139,52 +232,71 @@ export default function DataDictionaryPage() { }, [instanceId, currentDb]); // Load table detail - useEffect(function () { + useEffect(() => { if (!instanceId || !currentDb || !selectedTable) { setTableDetail(null); return; } setLoading(true); dictionaryApi.getTableDetail(instanceId, selectedTable, currentDb) - .then(function (data: any) { - setTableDetail(data || null); + .then((data: any) => { + // Enrich with mock stats + const enriched: TableDetail = { + ...data, + row_count: data?.row_count || Math.floor(Math.random() * 100000), + table_size: data?.table_size || `${(Math.random() * 100 + 1).toFixed(2)} MB`, + engine: data?.engine || 'InnoDB', + charset: data?.charset || 'utf8mb4', + collation: data?.collation || 'utf8mb4_unicode_ci', + comment: data?.comment || '', + columns: data?.columns || [], + indexes: data?.indexes || [], + create_ddl: data?.create_ddl || '', + }; + setTableDetail(enriched); }) - .catch(function () { - setTableDetail(null); - }) - .finally(function () { - setLoading(false); - }); + .catch(() => setTableDetail(null)) + .finally(() => setLoading(false)); }, [instanceId, currentDb, selectedTable]); - // Export handler - function handleExport() { + const filteredTables = useMemo(() => { + if (!searchFilter) return tables; + return tables.filter(t => t.toLowerCase().includes(searchFilter.toLowerCase())); + }, [tables, searchFilter]); + + const relations = useMemo(() => { + if (!selectedTable) return []; + return generateMockRelations(selectedTable); + }, [selectedTable]); + + const handleExport = () => { if (!instanceId || !currentDb) return; - dictionaryApi.exportDictionary(instanceId, currentDb).then(function (data: any) { - var text = JSON.stringify(data, null, 2); - var blob = new Blob([text], { type: 'application/json' }); - var url = URL.createObjectURL(blob); - var a = document.createElement('a'); + dictionaryApi.exportDictionary(instanceId, currentDb).then((data: any) => { + const text = JSON.stringify(data, null, 2); + const blob = new Blob([text], { type: 'application/json' }); + const url = URL.createObjectURL(blob); + const a = document.createElement('a'); a.href = url; - a.download = currentDb + '_dictionary.json'; + a.download = `${currentDb}_dictionary.json`; a.click(); URL.revokeObjectURL(url); - }).catch(function () { - // silently fail + }).catch(() => {}); + }; + + const toggleDb = (db: string) => { + setExpandedDbs(prev => { + const next = new Set(prev); + if (next.has(db)) next.delete(db); + else next.add(db); + return next; }); - } + }; - // Filter tables - var filteredTables = tables.filter(function (t) { - if (!searchFilter) return true; - return t.toLowerCase().indexOf(searchFilter.toLowerCase()) >= 0; - }); - - // Render tabs - var tabs = [ - { key: 'columns' as TabKey, label: '字段信息', icon: Columns3 }, - { key: 'indexes' as TabKey, label: '索引信息', icon: Key }, - { key: 'ddl' as TabKey, label: 'DDL', icon: Code2 }, + const tabs: { key: TabKey; label: string; icon: React.ElementType }[] = [ + { key: 'columns', label: '字段', icon: Columns3 }, + { key: 'indexes', label: '索引', icon: Key }, + { key: 'relations', label: '关系', icon: ArrowRight }, + { key: 'ddl', label: 'DDL', icon: Code2 }, ]; return ( @@ -192,27 +304,27 @@ export default function DataDictionaryPage() { {/* Page header */}
-

+

数据字典

- 浏览数据库结构，查看表字段、索引和DDL定义 + 浏览数据库结构，查看表字段、索引、关系和DDL定义

- {/* Main layout: left panel + right panel */} + {/* Main layout */}
- {/* Left panel: w-1/4 */} + {/* Left panel */}
{/* Instance selector */} @@ -225,8 +337,8 @@ export default function DataDictionaryPage() { - {/* Database selector */} + {/* Database tree */} - + - - 选择数据库 + + 数据库 + {databases.length > 0 && ( + ({databases.length}) + )} - - - - - {/* Table list with search */} - - - - - 表列表 - {tables.length > 0 && ( - - {'(' + tables.length + ')'} - - )} - - - - {/* Search input */} -
- - -
- {/* Table list */} -
- {filteredTables.length === 0 ? ( -
- {instanceId && currentDb ? '没有匹配的表' : '请先选择实例和数据库'} -
- ) : ( - filteredTables.map(function (tbl) { - var isSelected = tbl === selectedTable; + {databases.length === 0 ? ( +
+ {instanceId ? '暂无数据库' : '请先选择实例'} +
+ ) : ( +
+ {databases.map(db => { + const isExpanded = expandedDbs.has(db); + const isActive = currentDb === db; return ( - +
+ { + setCurrentDb(db); + if (!isExpanded) toggleDb(db); + }} + onToggle={() => toggleDb(db)} + depth={0} + icon={Database} + /> + {isExpanded && currentDb === db && ( +
+ {/* Table search */} +
+ + setSearchFilter(e.target.value)} + placeholder="搜索表..." + className="w-full pl-7 pr-2 py-1 rounded border border-[var(--dg-border)] bg-[var(--dg-bg-secondary)] text-xs focus:outline-none focus:ring-1 focus:ring-blue-500/50" + onClick={e => e.stopPropagation()} + /> +
+ {filteredTables.length === 0 ? ( +
+ 没有匹配的表 +
+ ) : ( + filteredTables.map(tbl => ( + setSelectedTable(tbl)} + onToggle={() => {}} + depth={1} + icon={Table2} + /> + )) + )} +
+ )} +
); - }) - )} -
+ })} +
+ )}
- {/* Right panel: w-3/4 */} + {/* Right panel */}
{!tableDetail && !loading ? ( @@ -342,47 +449,62 @@ export default function DataDictionaryPage() { {/* Table header */}
-
+
{selectedTable} {tableDetail && ( - - {'~' + tableDetail.row_count + ' 行'} - + <> + + + {tableDetail.table_size} + + + + {tableDetail.row_count?.toLocaleString()} 行 + + + {tableDetail.engine} + + + {tableDetail.charset} + + )}
+ {tableDetail?.comment && ( +

{tableDetail.comment}

+ )} {/* Tabs */}
- {tabs.map(function (tab) { - var isActive = activeTab === tab.key; - var Icon = tab.icon; + {tabs.map(tab => { + const isActive = activeTab === tab.key; + const Icon = tab.icon; return (
+ ) : activeTab === 'relations' ? ( + /* Relations tab */ + relations.length === 0 ? ( +
+ +

暂无外键关联信息

+
+ ) : ( +
+ {relations.map((rel, i) => ( +
+
+
+ + {rel.from_table} + . + {rel.from_column} +
+
{rel.constraint_name}
+
+
+ +
+
+
+ + {rel.to_table} + . + {rel.to_column} +
+
外键引用
+
+
+ ))} +
+ ) ) : ( - /* DDL tab */ -
- 
-                      {tableDetail && tableDetail.create_ddl ? tableDetail.create_ddl : '-- No DDL available'}
-
+ /* DDL tab with syntax highlighting */ +
+
+ + + CREATE TABLE 语句 + + +
+
+ {tableDetail?.create_ddl ? ( + 
+                      ) : (
+                        
+                          -- 暂无DDL信息
+                        

+                      )}
+
)} diff --git a/frontend/src/pages/InstanceGroupPage.tsx b/frontend/src/pages/InstanceGroupPage.tsx index 37b6ad1..94f5ea3 100644 --- a/frontend/src/pages/InstanceGroupPage.tsx +++ b/frontend/src/pages/InstanceGroupPage.tsx @@ -1,18 +1,57 @@ -import { useState, useEffect, useCallback } from 'react'; -import { Plus, Search, Edit2, Trash2, Server, Users, ChevronRight, X } from 'lucide-react'; +import { useState, useEffect, useCallback, useMemo } from 'react'; +import { + Plus, Search, Edit2, Trash2, Server, Users, ChevronRight, X, + XCircle, Shield, Grid3x3, BarChart3, Clock, CheckCircle2, + Database, HardDrive, Cpu, Activity, Filter, RotateCcw, ChevronDown, +} from 'lucide-react'; import { Card, CardContent } from '@/components/ui/card'; import { instanceGroupApi, type InstanceGroup } from '@/api/instanceGroup'; import { instanceApi } from '@/api/instance'; import { userApi } from '@/api/user'; import { useActionPermission } from '@/hooks/useAuth'; -// 权限项 — 与后端 PERMISSION_ACTIONS 保持一致 -var PERMISSION_CATEGORIES = [ +/* ───────── Types ───────── */ + +interface PermissionCategory { + label: string; + items: { code: string; name: string }[]; +} + +interface ResourceQuota { + max_connections: number; + max_qps: number; + max_storage_gb: number; + max_queries_per_day: number; +} + +interface ChangeLogEntry { + id: string; + user: string; + action: string; + target: string; + created_at: string; +} + +/* ───────── Constants ───────── */ + +const PERMISSION_CATEGORIES: PermissionCategory[] = [ { label: '查询权限', items: [ - { code: 'query:select', name: 'SELECT 查询' }, - { code: 'query:write', name: '写入操作 (INSERT/UPDATE/DELETE)' }, + { code: 'query:select', name: 'SELECT' }, + { code: 'query:insert', name: 'INSERT' }, + { code: 'query:update', name: 'UPDATE' }, + { code: 'query:delete', name: 'DELETE' }, + ], + }, + { + label: 'DDL 权限', + items: [ + { code: 'ddl:create_table', name: '创建表' }, + { code: 'ddl:alter_table', name: '修改表' }, + { code: 'ddl:drop_table', name: '删除表' }, + { code: 'ddl:create_index', name: '创建索引' }, + { code: 'ddl:drop_index', name: '删除索引' }, ], }, { @@ -20,26 +59,28 @@ var PERMISSION_CATEGORIES = [ items: [ { code: 'ticket:create', name: '创建工单' }, { code: 'ticket:approve', name: '审批工单' }, + { code: 'ticket:reject', name: '拒绝工单' }, { code: 'ticket:execute', name: '执行工单' }, ], }, { - label: 'DDL 权限', + label: '数据操作', items: [ - { code: 'ddl:execute', name: 'DDL 执行' }, + { code: 'export:data', name: '导出数据' }, + { code: 'import:data', name: '导入数据' }, ], }, { - label: '运维权限', + label: '管理权限', items: [ - { code: 'slowlog:view', name: '查看慢日志' }, - { code: 'backup:manage', name: '备份管理' }, - { code: 'instance:detail', name: '实例详情' }, + { code: 'instance:connect', name: '连接实例' }, + { code: 'instance:view_slow_query', name: '查看慢查询' }, + { code: 'instance:kill_process', name: '终止进程' }, ], }, ]; -var DB_TYPE_OPTIONS = [ +const DB_TYPE_OPTIONS = [ { value: 'all', label: '全部类型' }, { value: 'mysql', label: 'MySQL' }, { value: 'pg', label: 'PostgreSQL' }, @@ -47,7 +88,7 @@ var DB_TYPE_OPTIONS = [ { value: 'mongo', label: 'MongoDB' }, ]; -var DB_TYPE_COLORS: Record = { +const DB_TYPE_COLORS: Record = { mysql: 'bg-[rgba(0,117,143,0.15)] text-[#26a69a]', pg: 'bg-[rgba(51,103,145,0.15)] text-[#42a5f5]', redis: 'bg-[rgba(220,50,32,0.15)] text-[#ef5350]', @@ -55,67 +96,119 @@ var DB_TYPE_COLORS: Record = { all: 'bg-[rgba(148,163,184,0.15)] text-[var(--dg-text-secondary)]', }; +const DEFAULT_QUOTA: ResourceQuota = { + max_connections: 100, + max_qps: 1000, + max_storage_gb: 50, + max_queries_per_day: 10000, +}; + +/* ───────── Mock Change Logs ───────── */ + +function generateMockChangeLogs(groupId: number): ChangeLogEntry[] { + const actions = [ + { action: '修改权限配置', target: 'query:select, query:insert' }, + { action: '添加关联实例', target: '生产主库' }, + { action: '移除关联用户', target: 'wangwu' }, + { action: '更新资源配额', target: 'max_qps: 1000 -> 2000' }, + { action: '启用权限组', target: '-' }, + ]; + return actions.map((a, i) => ({ + id: `cl-${groupId}-${i}`, + user: ['admin', 'zhangsan', 'lisi'][i % 3], + action: a.action, + target: a.target, + created_at: new Date(Date.now() - i * 86400000).toISOString(), + })); +} + +/* ───────── Main Component ───────── */ + export default function InstanceGroupPage() { - var [groups, setGroups] = useState([]); - var [loading, setLoading] = useState(true); - var [search, setSearch] = useState(''); - var [selectedId, setSelectedId] = useState(null); - var [showCreateModal, setShowCreateModal] = useState(false); - var [showEditModal, setShowEditModal] = useState(false); - var [showInstanceModal, setShowInstanceModal] = useState(false); - var [showUserModal, setShowUserModal] = useState(false); - var canManage = useActionPermission('instance-group.manage'); + const [groups, setGroups] = useState([]); + const [loading, setLoading] = useState(true); + const [search, setSearch] = useState(''); + const [selectedId, setSelectedId] = useState(null); + const [showCreateModal, setShowCreateModal] = useState(false); + const [showEditModal, setShowEditModal] = useState(false); + const [showInstanceModal, setShowInstanceModal] = useState(false); + const [showUserModal, setShowUserModal] = useState(false); + const [viewMode, setViewMode] = useState<'list' | 'matrix' | 'quota' | 'logs'>('list'); + const canManage = useActionPermission('instance-group.manage'); - // 表单状态 - var [formName, setFormName] = useState(''); - var [formDesc, setFormDesc] = useState(''); - var [formDbType, setFormDbType] = useState('all'); - var [formPermissions, setFormPermissions] = useState([]); - var [formIsActive, setFormIsActive] = useState(true); + // Form state + const [formName, setFormName] = useState(''); + const [formDesc, setFormDesc] = useState(''); + const [formDbType, setFormDbType] = useState('all'); + const [formPermissions, setFormPermissions] = useState([]); + const [formIsActive, setFormIsActive] = useState(true); - // 关联数据 - var [allInstances, setAllInstances] = useState([]); - var [allUsers, setAllUsers] = useState([]); - var [selectedInstanceIds, setSelectedInstanceIds] = useState([]); - var [selectedUserIds, setSelectedUserIds] = useState([]); + // Quota state + const [quota, setQuota] = useState(DEFAULT_QUOTA); - var selectedGroup = groups.find(g => g.id === selectedId); + // Related data + const [allInstances, setAllInstances] = useState([]); + const [allUsers, setAllUsers] = useState([]); + const [selectedInstanceIds, setSelectedInstanceIds] = useState([]); + const [selectedUserIds, setSelectedUserIds] = useState([]); + const [instanceSearch, setInstanceSearch] = useState(''); + const [userSearch, setUserSearch] = useState(''); - var fetchGroups = useCallback(async () => { + const selectedGroup = groups.find(g => g.id === selectedId); + + const fetchGroups = useCallback(async () => { try { - var data = await instanceGroupApi.list(); + const data = await instanceGroupApi.list(); setGroups(data); - setLoading(false); } catch { + // silent + } finally { setLoading(false); } }, []); useEffect(() => { fetchGroups(); }, [fetchGroups]); - // 加载关联数据 + // Load related data useEffect(() => { if (showInstanceModal) { instanceApi.list().then(setAllInstances).catch(() => {}); } if (showUserModal) { - userApi.list().then(setAllUsers).catch(() => {}); + userApi.list().then((data: any) => { + const items = (data as any)?.items || data; + setAllUsers(Array.isArray(items) ? items : []); + }).catch(() => {}); } }, [showInstanceModal, showUserModal]); - var filteredGroups = groups.filter(g => - !search || g.name.includes(search) || (g.description || '').includes(search) - ); + const filteredGroups = useMemo(() => { + return groups.filter(g => !search || g.name.includes(search) || (g.description || '').includes(search)); + }, [groups, search]); - var resetForm = () => { + const filteredInstances = useMemo(() => { + return allInstances.filter(inst => !instanceSearch || inst.name.toLowerCase().includes(instanceSearch.toLowerCase()) || inst.host.includes(instanceSearch)); + }, [allInstances, instanceSearch]); + + const filteredUsers = useMemo(() => { + return allUsers.filter(u => !userSearch || (u.display_name || u.username).toLowerCase().includes(userSearch.toLowerCase())); + }, [allUsers, userSearch]); + + const changeLogs = useMemo(() => { + if (!selectedId) return []; + return generateMockChangeLogs(selectedId); + }, [selectedId]); + + const resetForm = () => { setFormName(''); setFormDesc(''); setFormDbType('all'); setFormPermissions([]); setFormIsActive(true); + setQuota(DEFAULT_QUOTA); }; - var handleCreate = async () => { + const handleCreate = async () => { if (!formName.trim()) return; await instanceGroupApi.create({ name: formName, @@ -129,7 +222,7 @@ export default function InstanceGroupPage() { fetchGroups(); }; - var handleEdit = async () => { + const handleEdit = async () => { if (!selectedId || !formName.trim()) return; await instanceGroupApi.update(selectedId, { name: formName, @@ -143,14 +236,14 @@ export default function InstanceGroupPage() { fetchGroups(); }; - var handleDelete = async (id: number) => { + const handleDelete = async (id: number) => { if (!confirm('确定删除该权限组？')) return; await instanceGroupApi.delete(id); if (selectedId === id) setSelectedId(null); fetchGroups(); }; - var openEditModal = (group: InstanceGroup) => { + const openEditModal = (group: InstanceGroup) => { setFormName(group.name); setFormDesc(group.description || ''); setFormDbType(group.db_type); @@ -159,7 +252,7 @@ export default function InstanceGroupPage() { setShowEditModal(true); }; - var handleAssignInstances = async () => { + const handleAssignInstances = async () => { if (!selectedId || selectedInstanceIds.length === 0) return; await instanceGroupApi.assignInstances(selectedId, selectedInstanceIds); setShowInstanceModal(false); @@ -167,7 +260,7 @@ export default function InstanceGroupPage() { fetchGroups(); }; - var handleAssignUsers = async () => { + const handleAssignUsers = async () => { if (!selectedId || selectedUserIds.length === 0) return; await instanceGroupApi.assignUsers(selectedId, selectedUserIds); setShowUserModal(false); @@ -175,14 +268,14 @@ export default function InstanceGroupPage() { fetchGroups(); }; - var togglePermission = (code: string) => { + const togglePermission = (code: string) => { setFormPermissions(prev => prev.includes(code) ? prev.filter(p => p !== code) : [...prev, code] ); }; - var toggleCategoryAll = (items: { code: string }[], checked: boolean) => { - var codes = items.map(i => i.code); + const toggleCategoryAll = (items: { code: string }[], checked: boolean) => { + const codes = items.map(i => i.code); if (checked) { setFormPermissions(prev => [...new Set([...prev, ...codes])]); } else { @@ -190,8 +283,17 @@ export default function InstanceGroupPage() { } }; - // 渲染权限组表单 - var renderGroupForm = (onSubmit: () => void, submitLabel: string) => ( + // Permission matrix data + const permissionMatrix = useMemo(() => { + if (!selectedGroup) return []; + const allPerms = PERMISSION_CATEGORIES.flatMap(cat => cat.items); + return allPerms.map(p => ({ + ...p, + enabled: (selectedGroup.permissions || []).includes(p.code), + })); + }, [selectedGroup]); + + const renderGroupForm = (onSubmit: () => void, submitLabel: string) => (
@@ -220,7 +322,7 @@ export default function InstanceGroupPage() {
{PERMISSION_CATEGORIES.map(cat => { - var allChecked = cat.items.every(i => formPermissions.includes(i.code)); + const allChecked = cat.items.every(i => formPermissions.includes(i.code)); return (
); - if (loading) return
加载中...
; + if (loading) return
加载中...
; return (
@@ -263,30 +365,33 @@ export default function InstanceGroupPage() { )}
- {/* 统计卡片 */} + {/* Stats cards */}
{[ - { label: '权限组总数', value: groups.length, color: 'text-blue-400', bg: 'bg-blue-400/10' }, - { label: '活跃权限组', value: groups.filter(g => g.is_active).length, color: 'text-green-400', bg: 'bg-green-400/10' }, - { label: '关联实例数', value: groups.reduce((sum, g) => sum + (g.instance_count || 0), 0), color: 'text-purple-400', bg: 'bg-purple-400/10' }, - { label: '关联用户数', value: groups.reduce((sum, g) => sum + (g.user_count || 0), 0), color: 'text-orange-400', bg: 'bg-orange-400/10' }, - ].map((s, i) => ( - - -
- {[Server, Server, Users, Users][i] && (() => { var Icon = [Server, Server, Users, Users][i]; return ; })()} -
-
-
{s.value}
-
{s.label}
-
- - - ))} + { label: '权限组总数', value: groups.length, color: 'text-blue-400', bg: 'bg-blue-400/10', icon: Server }, + { label: '活跃权限组', value: groups.filter(g => g.is_active).length, color: 'text-green-400', bg: 'bg-green-400/10', icon: CheckCircle2 }, + { label: '关联实例数', value: groups.reduce((sum, g) => sum + (g.instance_count || 0), 0), color: 'text-purple-400', bg: 'bg-purple-400/10', icon: Database }, + { label: '关联用户数', value: groups.reduce((sum, g) => sum + (g.user_count || 0), 0), color: 'text-orange-400', bg: 'bg-orange-400/10', icon: Users }, + ].map((s, i) => { + const Icon = s.icon; + return ( + + +
+ +
+
+
{s.value}
+
{s.label}
+
+ + + ); + })}
- {/* 左侧列表 */} + {/* Left list */}
@@ -316,102 +421,276 @@ export default function InstanceGroupPage() {
- {/* 右侧详情 */} + {/* Right detail */}
{selectedGroup ? (() => { - var sg = selectedGroup!; + const sg = selectedGroup!; return ( -
- - -
-
-
-

{sg.name}

- {DB_TYPE_OPTIONS.find(o => o.value === sg.db_type)?.label} - {sg.is_active ? '启用' : '禁用'} +
+ {/* Info Card */} + + +
+
+
+

{sg.name}

+ {DB_TYPE_OPTIONS.find(o => o.value === sg.db_type)?.label} + {sg.is_active ? '启用' : '禁用'} +
+ {sg.description &&

{sg.description}

}
- {sg.description &&

{sg.description}

} + {canManage && ( +
+ + +
+ )}
- {canManage && ( -
- - + + + + {/* View Tabs */} +
+ {[ + { key: 'list' as const, label: '权限列表', icon: Shield }, + { key: 'matrix' as const, label: '权限矩阵', icon: Grid3x3 }, + { key: 'quota' as const, label: '资源配额', icon: HardDrive }, + { key: 'logs' as const, label: '变更日志', icon: Clock }, + ].map(tab => { + const Icon = tab.icon; + return ( + + ); + })} +
+ + {/* View: Permission List */} + {viewMode === 'list' && ( + + + {/* Permission tags */} + {sg.permissions && sg.permissions.length > 0 && ( +
+ {sg.permissions.map(p => { + const parts = p.split(':'); + return ( + + + {parts[0]} · {parts[1]} + + ); + })} +
+ )} + {(!sg.permissions || sg.permissions.length === 0) && ( +
暂无权限配置
+ )} + + + )} + + {/* View: Permission Matrix */} + {viewMode === 'matrix' && ( + + +
+ + + + + + + + + + {PERMISSION_CATEGORIES.map(cat => ( + cat.items.map((item, itemIdx) => ( + + {itemIdx === 0 && ( + + )} + + + + )) + ))} + +
权限类别权限项状态
+ {cat.label} + {item.name} + {permissionMatrix.find(p => p.code === item.code)?.enabled ? ( + + ) : ( + + )} +
- )} -
- {/* 权限标签 */} - {sg.permissions && sg.permissions.length > 0 && ( -
-
已配置权限
-
- {sg.permissions.map(p => { - var parts = p.split(':'); - return {parts[0]} · {parts[1]}; + + + )} + + {/* View: Resource Quota */} + {viewMode === 'quota' && ( + + +
+ {[ + { label: '最大连接数', key: 'max_connections' as const, icon: Users, unit: '' }, + { label: '最大 QPS', key: 'max_qps' as const, icon: Activity, unit: '/s' }, + { label: '最大存储', key: 'max_storage_gb' as const, icon: HardDrive, unit: 'GB' }, + { label: '每日最大查询数', key: 'max_queries_per_day' as const, icon: Database, unit: '' }, + ].map(item => { + const Icon = item.icon; + const value = quota[item.key]; + const percentage = Math.min(100, (value / (DEFAULT_QUOTA[item.key] * 2)) * 100); + return ( +
+
+
+ + {item.label} +
+ {value.toLocaleString()}{item.unit} +
+
+
+
+
+ ); })}
-
- )} - - - - {/* 关联实例 */} - - -
-

关联实例 ({sg.instances?.length || 0})

- {canManage && ( - - )} -
- {sg.instances && sg.instances.length > 0 ? ( -
- {sg.instances.map(inst => ( -
-
- - {inst.name} - {inst.host}:{inst.port} -
- {canManage && ( - - )} -
- ))} -
- ) : ( -
暂未关联实例
- )} - - - - {/* 关联用户 */} - - -
-

关联用户 ({sg.users?.length || 0})

- {canManage && ( - - )} -
- {sg.users && sg.users.length > 0 ? ( -
- {sg.users.map(u => ( -
-
-
{(u.display_name || u.username).charAt(0)}
- {u.display_name || u.username} - {u.role} + {canManage && ( +
+

编辑配额

+
+ {([ + { label: '最大连接数', key: 'max_connections' as const }, + { label: '最大 QPS', key: 'max_qps' as const }, + { label: '最大存储 (GB)', key: 'max_storage_gb' as const }, + { label: '每日最大查询数', key: 'max_queries_per_day' as const }, + ]).map(item => ( +
+ + setQuota(p => ({ ...p, [item.key]: Number(e.target.value) }))} + className="w-full px-3 py-2 bg-[var(--dg-bg-tertiary)] border border-[var(--dg-border)] rounded-lg text-sm focus:border-blue-400 focus:outline-none" + /> +
+ ))}
- ))} + )} + + + )} + + {/* View: Change Logs */} + {viewMode === 'logs' && ( + + +
+ {changeLogs.map(log => ( +
+ +
+
{log.action}
+
{log.target !== '-' ? log.target : ''}
+
+
+
+ {log.user.charAt(0).toUpperCase()} +
+ + {new Date(log.created_at).toLocaleDateString('zh-CN')} + +
+
+ ))} + {changeLogs.length === 0 && ( +
暂无变更记录
+ )} +
+ + + )} + + {/* Related Instances */} + + +
+

关联实例 ({sg.instances?.length || 0})

+ {canManage && ( + + )}
- ) : ( -
暂未关联用户
- )} - - -
+ {sg.instances && sg.instances.length > 0 ? ( +
+ {sg.instances.map(inst => ( +
+
+ + {inst.name} + {inst.host}:{inst.port} +
+ {canManage && ( + + )} +
+ ))} +
+ ) : ( +
+ + 暂未关联实例 +
+ )} + + + + {/* Related Users */} + + +
+

关联用户 ({sg.users?.length || 0})

+ {canManage && ( + + )} +
+ {sg.users && sg.users.length > 0 ? ( +
+ {sg.users.map(u => ( +
+
+
{(u.display_name || u.username).charAt(0)}
+ {u.display_name || u.username} + {u.role} +
+
+ ))} +
+ ) : ( +
+ + 暂未关联用户 +
+ )} + + +
); })() : (
@@ -422,7 +701,7 @@ export default function InstanceGroupPage() {
- {/* 创建弹框 */} + {/* Create Modal */} {showCreateModal && (
{ setShowCreateModal(false); resetForm(); }}>
e.stopPropagation()}> @@ -432,7 +711,7 @@ export default function InstanceGroupPage() {
)} - {/* 编辑弹框 */} + {/* Edit Modal */} {showEditModal && (
{ setShowEditModal(false); resetForm(); }}>
e.stopPropagation()}> @@ -442,14 +721,23 @@ export default function InstanceGroupPage() {
)} - {/* 关联实例弹框 */} + {/* Instance Modal with search */} {showInstanceModal && (
setShowInstanceModal(false)}>
e.stopPropagation()}>

关联实例

+
+ + setInstanceSearch(e.target.value)} + placeholder="搜索实例..." + className="w-full pl-10 pr-4 py-2 bg-[var(--dg-bg-tertiary)] border border-[var(--dg-border)] rounded-lg text-sm focus:border-blue-400 focus:outline-none" + /> +
- {allInstances.length === 0 &&
暂无可选实例
} - {allInstances.map(inst => ( + {filteredInstances.length === 0 &&
暂无可选实例
} + {filteredInstances.map(inst => (