feat(redis): 大 key 分页 + 流式导出(含票据鉴权)+ 查询行数上限默认 5000
=== 一、问题:大 key 会把内存打爆,且查询结果没有导出 ===
RedisGetKeyDetail 原来对每种类型都是全量读取:
string → GET list → LRANGE 0 -1 set → SMEMBERS
zset → ZRANGE 0 -1 hash → HGETALL
然后整包序列化返回。像 PERP:market:* 这类盘口/kline key,
一个 key 几十万条就会:Go 侧整值入内存 → JSON 序列化翻倍 → 浏览器渲染卡死。
=== 二、后端:键详情分页(统一令牌,兼容四种类型)===
GET /redis/:id/keys/:key?db=&count=&token=
count 默认 200,硬上限 2000
token 语义按类型区分,前端无感知:
list / zset → 十进制 offset(LRANGE / ZRANGE 区间)
hash / set → Redis SCAN 游标(HSCAN / SSCAN)
string → 忽略
返回 total / count / nextToken / truncated / previewCap
string 特殊处理:不能分页,改为预览上限 256KB
- 用 STRLEN 拿真实大小(O(1))
- 超过上限时用 GETRANGE 只取前 256KB,truncated=true
- 完整内容只能靠导出
新增 wrapper 方法:HScan / ZScan / HLen / ZCard / StrLen / GetRange
(HGETALL、SMEMBERS、ZRANGE 0 -1 一律不再用于详情读取)
=== 三、后端:流式导出(两段式票据,避免令牌进 URL 日志)===
① POST /redis/:id/keys/:key/export-ticket (走 Bearer 鉴权)
→ HMAC-SHA256(JWTSecret, instance|db|key|exp) 生成票据,2 分钟过期,
绑定具体 instance/db/key,不能跨 key 复用
② GET /redis/:id/keys/:key/export?db=&format=csv|jsonl&ticket=
→ 注册在鉴权组【之外】,靠票据校验
为什么这么做:浏览器原生下载(<a download>/window.open)带不了
Authorization 头;若前端用 axios 取 blob,整份导出会进浏览器内存,
与「大 key 不爆内存」的目标相悖。
导出实现全程流式(服务端内存 = 单批 + 64KB 缓冲):
string → GETRANGE 256KB 分块
list → LRANGE 每批 1000
zset → ZRANGE WITHSCORES 每批 1000
hash → HSCAN 游标每批 1000
set → SSCAN 游标每批 1000
单次上限 200 万行;CSV 带 UTF-8 BOM(Excel 直接打开不乱码);
列顺:hash(field,value) / list(index,value) / set(member) / zset(rank,member,score)
=== 四、前端:详情面板加分页条 + 4 个导出按钮 ===
RedisKeyDetailPanel:
- 「共 N 条 · 第 X 页 · 每页 [100/200/500/1000/2000]」+ 上一页/下一页
- SCAN 游标不可逆,上一页用令牌栈回放
- string 超预览上限时显示黄色提示「仅预览前 256KB,共 N 字节」
- 导出四档:当前页 CSV / 当前页 JSON(纯前端,瞬时)
导出全部 CSV / 导出全部 JSONL(服务端流式,边读边写)
顺带修掉头注释里「已知限制:大值不分页」的过时说明。
=== 五、查询行数上限:默认 1000 → 5000,并修掉硬编码 ===
用户要求:「默认设置里面有查询最大上限吧 比如 mysql pg 查询最大Limit限制 5000」
发现两个问题:
1. seed 默认值只有 1000(main.go query_limit_max)
2. ExecuteQuery 里还有一段硬编码:
if input.Limit <= 0 { input.Limit = maxLimit; if input.Limit > 1000 { input.Limit = 1000 } }
→ 管理员把设置改成 5000 也白搭,前端不传 limit 时永远被压回 1000
修复:
- seed 默认值 1000 → 5000(Value + DefaultValue)
- getSettingInt("query_limit_max", 1000) → 5000
- 删掉硬编码 1000 封顶,唯一口径就是设置值
- 线上已有记录的 value 已通过 PUT /settings 改成 5000(seed 只在缺失时插入,不会自动更新旧值)
对 MySQL / PostgreSQL / Redis / MongoDB 统一生效(ExecuteQuery 按 db_type 分发,
limit 在入口处统一 clamp)。
=== 六、实测结果(线上)===
分页:
zset PERP:market:kline:symbol_kline_12h_AAVEUSDT_PERP_MARK
total=83,count=5 → 第1页 5 条 nextToken=5 → 第2页 5 条 nextToken=10,数据不同 ✅
hash ST:market:ticker:symbol_ticker
total=27,count=3 → 第1页 3 字段 nextToken=24 → 第2页 3 字段(完全不同)✅
string PERP:market:depth:symbol_depth_BTCUSDT_PERP
STRLEN=14192 → 导出 14192 字节,完全一致 ✅(实时 key 两次采样有差异属正常)
导出:
zset CSV 200 / 30970 字节 / 84 行(header+83)✅
zset JSONL 200 / 84 行 ✅
string CSV 200 / 分块 GETRANGE 拼接完整 ✅
无票据 → 403 ✅ 伪造票据 → 403 ✅
查询上限:
limit=99999 → 5000 行 ✅ limit=0(默认)→ 5000 行 ✅
SQL 导出 CSV(原已存在 ExportQueryCSV,支持 mysql/pg/redis/mongo)200 / 101 行 ✅
镜像:lghost/opsxsql:latest + v0.1 = sha256:68fc996681c6
This commit is contained in:
1 parent
48c8fddbe9
commit
7dfbc0b680
7 files changed
+854
-22
No files matched your search
@@ -123,7 +123,7 @@ func main() {
|
|||||||
func seedDefaultSettings(db *gorm.DB) {
|
func seedDefaultSettings(db *gorm.DB) {
|
||||||
defaults := []model.SystemSetting{
|
defaults := []model.SystemSetting{
|
||||||
// ── 查询控制 ──
|
// ── 查询控制 ──
|
||||||
{Key: "query_limit_max", Value: "1000", ValueType: "number", Group: "query", Label: "LIMIT 上限", Desc: strPtr("SELECT 查询返回的最大行数,超过自动截断"), DefaultValue: strPtr("1000"), SortOrder: 1},
|
{Key: "query_limit_max", Value: "5000", ValueType: "number", Group: "query", Label: "LIMIT 上限", Desc: strPtr("SELECT 查询返回的最大行数(默认 5000),超过自动截断;对 MySQL / PostgreSQL / Redis / MongoDB 统一生效"), DefaultValue: strPtr("5000"), SortOrder: 1},
|
||||||
{Key: "query_timeout", Value: "30", ValueType: "number", Group: "query", Label: "查询超时(秒)", Desc: strPtr("单次查询最长执行时间,超时自动终止"), DefaultValue: strPtr("30"), SortOrder: 2},
|
{Key: "query_timeout", Value: "30", ValueType: "number", Group: "query", Label: "查询超时(秒)", Desc: strPtr("单次查询最长执行时间,超时自动终止"), DefaultValue: strPtr("30"), SortOrder: 2},
|
||||||
{Key: "query_auto_limit", Value: "true", ValueType: "boolean", Group: "query", Label: "自动加 LIMIT", Desc: strPtr("用户未写 LIMIT 时自动追加,防止全表扫描"), DefaultValue: strPtr("true"), SortOrder: 3},
|
{Key: "query_auto_limit", Value: "true", ValueType: "boolean", Group: "query", Label: "自动加 LIMIT", Desc: strPtr("用户未写 LIMIT 时自动追加,防止全表扫描"), DefaultValue: strPtr("true"), SortOrder: 3},
|
||||||
{Key: "query_max_rows_examined", Value: "1000000", ValueType: "number", Group: "query", Label: "最大扫描行数", Desc: strPtr("单次查询允许扫描的最大行数"), DefaultValue: strPtr("1000000"), SortOrder: 4},
|
{Key: "query_max_rows_examined", Value: "1000000", ValueType: "number", Group: "query", Label: "最大扫描行数", Desc: strPtr("单次查询允许扫描的最大行数"), DefaultValue: strPtr("1000000"), SortOrder: 4},
|
||||||
|
|||||||
@@ -662,13 +662,16 @@ func ExecuteQuery(c *gin.Context) {
|
|||||||
resp.BadRequest(c, "instance_id and sql are required")
|
resp.BadRequest(c, "instance_id and sql are required")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
// Read query limits from platform settings
|
// 查询行数上限统一由系统设置 query_limit_max 决定(默认 5000)。
|
||||||
maxLimit := getSettingInt("query_limit_max", 1000)
|
// 注意:历史上这里还有一段 `if input.Limit > 1000 { input.Limit = 1000 }` 的硬编码,
|
||||||
|
// 导致管理员把上限调到 5000 也不生效(前端不传 limit 时永远被压回 1000)。
|
||||||
|
// 现已移除,唯一口径就是设置值。
|
||||||
|
maxLimit := getSettingInt("query_limit_max", 5000)
|
||||||
|
if maxLimit <= 0 {
|
||||||
|
maxLimit = 5000
|
||||||
|
}
|
||||||
if input.Limit <= 0 {
|
if input.Limit <= 0 {
|
||||||
input.Limit = maxLimit
|
input.Limit = maxLimit
|
||||||
if input.Limit > 1000 {
|
|
||||||
input.Limit = 1000
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if input.Limit > maxLimit {
|
if input.Limit > maxLimit {
|
||||||
input.Limit = maxLimit
|
input.Limit = maxLimit
|
||||||
|
|||||||
@@ -277,6 +277,52 @@ func (w *redisClientWrapper) SScan(ctx context.Context, key string, cursor uint6
|
|||||||
return w.Client.SScan(ctx, key, cursor, match, count)
|
return w.Client.SScan(ctx, key, cursor, match, count)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// HScan 分批遍历 hash —— 大 key 不能用 HGETALL 整体读,会打爆内存。
|
||||||
|
func (w *redisClientWrapper) HScan(ctx context.Context, key string, cursor uint64, match string, count int64) *redis.ScanCmd {
|
||||||
|
if w.isCluster && w.ClusterClient != nil {
|
||||||
|
return w.ClusterClient.HScan(ctx, key, cursor, match, count)
|
||||||
|
}
|
||||||
|
return w.Client.HScan(ctx, key, cursor, match, count)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ZScan 分批遍历 zset 成员(不含 score,仅在需要成员集合时使用)。
|
||||||
|
func (w *redisClientWrapper) ZScan(ctx context.Context, key string, cursor uint64, match string, count int64) *redis.ScanCmd {
|
||||||
|
if w.isCluster && w.ClusterClient != nil {
|
||||||
|
return w.ClusterClient.ZScan(ctx, key, cursor, match, count)
|
||||||
|
}
|
||||||
|
return w.Client.ZScan(ctx, key, cursor, match, count)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 以下几个 count/长度类命令用于分页时给出 total,都是 O(1),不会因为 key 大而变慢。
|
||||||
|
func (w *redisClientWrapper) HLen(ctx context.Context, key string) *redis.IntCmd {
|
||||||
|
if w.isCluster && w.ClusterClient != nil {
|
||||||
|
return w.ClusterClient.HLen(ctx, key)
|
||||||
|
}
|
||||||
|
return w.Client.HLen(ctx, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *redisClientWrapper) ZCard(ctx context.Context, key string) *redis.IntCmd {
|
||||||
|
if w.isCluster && w.ClusterClient != nil {
|
||||||
|
return w.ClusterClient.ZCard(ctx, key)
|
||||||
|
}
|
||||||
|
return w.Client.ZCard(ctx, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *redisClientWrapper) StrLen(ctx context.Context, key string) *redis.IntCmd {
|
||||||
|
if w.isCluster && w.ClusterClient != nil {
|
||||||
|
return w.ClusterClient.StrLen(ctx, key)
|
||||||
|
}
|
||||||
|
return w.Client.StrLen(ctx, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetRange 按字节区间读取字符串,用于超大 string 的分页预览与流式导出。
|
||||||
|
func (w *redisClientWrapper) GetRange(ctx context.Context, key string, start, end int64) *redis.StringCmd {
|
||||||
|
if w.isCluster && w.ClusterClient != nil {
|
||||||
|
return w.ClusterClient.GetRange(ctx, key, start, end)
|
||||||
|
}
|
||||||
|
return w.Client.GetRange(ctx, key, start, end)
|
||||||
|
}
|
||||||
|
|
||||||
func writeRedisKeyValue(ctx context.Context, client *redisClientWrapper, key string, keyType string, value interface{}) error {
|
func writeRedisKeyValue(ctx context.Context, client *redisClientWrapper, key string, keyType string, value interface{}) error {
|
||||||
switch keyType {
|
switch keyType {
|
||||||
case "string":
|
case "string":
|
||||||
@@ -556,6 +602,24 @@ func RedisDbsize(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// GET /api/v1/redis/:id/keys/:key
|
// GET /api/v1/redis/:id/keys/:key
|
||||||
|
// Redis 键详情分页参数:默认每页 200 条,硬上限 2000 条。
|
||||||
|
// 大 key(如 PERP:market:depth:* 盘口快照)必须分页,否则 HGETALL / LRANGE 0 -1
|
||||||
|
// 会把整个值读进内存再整包序列化,极易把 Pod 内存打爆、前端也会卡死。
|
||||||
|
const (
|
||||||
|
redisKeyPageDefault = 200
|
||||||
|
redisKeyPageMax = 2000
|
||||||
|
// string 类型无法「分页」,改为给一个预览上限,超出时标记 truncated 并给出总字节数。
|
||||||
|
redisStringPreviewBytes = 256 << 10
|
||||||
|
)
|
||||||
|
|
||||||
|
// GET /api/v1/redis/:id/keys/:key?db=0&count=200&token=<分页令牌>
|
||||||
|
//
|
||||||
|
// token 语义由类型决定:
|
||||||
|
// - list / zset → 十进制 offset(偏移量)
|
||||||
|
// - hash / set → Redis SCAN 游标
|
||||||
|
// - string → 忽略
|
||||||
|
//
|
||||||
|
// 返回 token(下一页令牌,空串=没有更多)、total、truncated。
|
||||||
func RedisGetKeyDetail(c *gin.Context) {
|
func RedisGetKeyDetail(c *gin.Context) {
|
||||||
id := c.Param("id")
|
id := c.Param("id")
|
||||||
key := c.Param("key")
|
key := c.Param("key")
|
||||||
@@ -565,6 +629,15 @@ func RedisGetKeyDetail(c *gin.Context) {
|
|||||||
db, _ = strconv.Atoi(dbParam)
|
db, _ = strconv.Atoi(dbParam)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
count, _ := strconv.Atoi(c.DefaultQuery("count", strconv.Itoa(redisKeyPageDefault)))
|
||||||
|
if count <= 0 {
|
||||||
|
count = redisKeyPageDefault
|
||||||
|
}
|
||||||
|
if count > redisKeyPageMax {
|
||||||
|
count = redisKeyPageMax
|
||||||
|
}
|
||||||
|
pageToken := c.Query("token")
|
||||||
|
|
||||||
var instance model.Instance
|
var instance model.Instance
|
||||||
if err := model.DB.First(&instance, id).Error; err != nil {
|
if err := model.DB.First(&instance, id).Error; err != nil {
|
||||||
resp.NotFound(c, "Instance not found")
|
resp.NotFound(c, "Instance not found")
|
||||||
@@ -587,26 +660,117 @@ func RedisGetKeyDetail(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var value interface{}
|
var value interface{}
|
||||||
|
var total int64 = -1
|
||||||
|
nextToken := ""
|
||||||
|
truncated := false
|
||||||
|
|
||||||
|
offset := int64(0)
|
||||||
|
if pageToken != "" {
|
||||||
|
if n, err := strconv.ParseInt(pageToken, 10, 64); err == nil && n > 0 {
|
||||||
|
offset = n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
scanCursor := uint64(0)
|
||||||
|
if pageToken != "" {
|
||||||
|
if n, err := strconv.ParseUint(pageToken, 10, 64); err == nil {
|
||||||
|
scanCursor = n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
switch keyType {
|
switch keyType {
|
||||||
case "string":
|
case "string":
|
||||||
value, _ = client.Get(ctx, key).Result()
|
size, _ := client.StrLen(ctx, key).Result()
|
||||||
|
total = size
|
||||||
|
// 用 GETRANGE 只取预览区间,避免超大字符串整串读入
|
||||||
|
if size > redisStringPreviewBytes {
|
||||||
|
value, _ = client.GetRange(ctx, key, 0, redisStringPreviewBytes-1).Result()
|
||||||
|
truncated = true
|
||||||
|
} else {
|
||||||
|
value, _ = client.Get(ctx, key).Result()
|
||||||
|
}
|
||||||
|
|
||||||
case "list":
|
case "list":
|
||||||
value, _ = client.LRange(ctx, key, 0, -1).Result()
|
total, _ = client.LLen(ctx, key).Result()
|
||||||
|
items, err := client.LRange(ctx, key, offset, offset+int64(count)-1).Result()
|
||||||
|
if err != nil {
|
||||||
|
resp.InternalErr(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if items == nil {
|
||||||
|
items = []string{}
|
||||||
|
}
|
||||||
|
value = items
|
||||||
|
if offset+int64(len(items)) < total {
|
||||||
|
nextToken = strconv.FormatInt(offset+int64(len(items)), 10)
|
||||||
|
truncated = true
|
||||||
|
}
|
||||||
|
|
||||||
case "set":
|
case "set":
|
||||||
value, _ = client.SMembers(ctx, key).Result()
|
total, _ = client.SCard(ctx, key).Result()
|
||||||
|
members, next, err := client.SScan(ctx, key, scanCursor, "", int64(count)).Result()
|
||||||
|
if err != nil {
|
||||||
|
resp.InternalErr(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if members == nil {
|
||||||
|
members = []string{}
|
||||||
|
}
|
||||||
|
value = members
|
||||||
|
if next != 0 {
|
||||||
|
nextToken = strconv.FormatUint(next, 10)
|
||||||
|
truncated = true
|
||||||
|
}
|
||||||
|
|
||||||
case "zset":
|
case "zset":
|
||||||
value, _ = client.ZRangeWithScores(ctx, key, 0, -1).Result()
|
total, _ = client.ZCard(ctx, key).Result()
|
||||||
|
items, err := client.ZRangeWithScores(ctx, key, offset, offset+int64(count)-1).Result()
|
||||||
|
if err != nil {
|
||||||
|
resp.InternalErr(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if items == nil {
|
||||||
|
items = []redis.Z{}
|
||||||
|
}
|
||||||
|
value = items
|
||||||
|
if offset+int64(len(items)) < total {
|
||||||
|
nextToken = strconv.FormatInt(offset+int64(len(items)), 10)
|
||||||
|
truncated = true
|
||||||
|
}
|
||||||
|
|
||||||
case "hash":
|
case "hash":
|
||||||
value, _ = client.HGetAll(ctx, key).Result()
|
total, _ = client.HLen(ctx, key).Result()
|
||||||
|
flat, next, err := client.HScan(ctx, key, scanCursor, "", int64(count)).Result()
|
||||||
|
if err != nil {
|
||||||
|
resp.InternalErr(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// HScan 返回扁平 [field, value, field, value, ...],这里还原成 map,
|
||||||
|
// 与旧的 HGetAll 返回结构保持一致,前端无需改动取值方式。
|
||||||
|
m := make(map[string]string, len(flat)/2)
|
||||||
|
for i := 0; i+1 < len(flat); i += 2 {
|
||||||
|
m[flat[i]] = flat[i+1]
|
||||||
|
}
|
||||||
|
value = m
|
||||||
|
if next != 0 {
|
||||||
|
nextToken = strconv.FormatUint(next, 10)
|
||||||
|
truncated = true
|
||||||
|
}
|
||||||
|
|
||||||
default:
|
default:
|
||||||
value = nil
|
value = nil
|
||||||
}
|
}
|
||||||
|
|
||||||
c.JSON(http.StatusOK, gin.H{
|
c.JSON(http.StatusOK, gin.H{
|
||||||
"key": key,
|
"key": key,
|
||||||
"type": keyType,
|
"type": keyType,
|
||||||
"value": value,
|
"value": value,
|
||||||
"ttl": ttlMs,
|
"ttl": ttlMs,
|
||||||
|
"total": total,
|
||||||
|
"count": count,
|
||||||
|
"nextToken": nextToken,
|
||||||
|
"truncated": truncated,
|
||||||
|
"pageToken": pageToken,
|
||||||
|
"previewCap": redisStringPreviewBytes,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,384 @@
|
|||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"context"
|
||||||
|
"crypto/hmac"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/csv"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
|
||||||
|
"sql-platform/config"
|
||||||
|
"sql-platform/internal/handler/resp"
|
||||||
|
"sql-platform/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Redis 大 Key 导出的两段式设计:
|
||||||
|
//
|
||||||
|
// ① POST /redis/:id/keys/:key/export-ticket (走 Bearer 鉴权)→ 一次性短时效票据
|
||||||
|
// ② GET /redis/:id/keys/:key/export?ticket= (不走 Bearer,凭证在票据里)
|
||||||
|
//
|
||||||
|
// 为什么不给导出接口直接挂 Bearer:浏览器原生下载(<a download>/window.open)带不了
|
||||||
|
// Authorization 头;若前端用 axios 取 blob,则整份导出物会进浏览器内存,与「大 key
|
||||||
|
// 不爆内存」的目标相悖。改为短时效、绑定具体 key 的票据后,浏览器可以边收边写磁盘。
|
||||||
|
|
||||||
|
const redisExportTicketTTL = 2 * time.Minute
|
||||||
|
|
||||||
|
const (
|
||||||
|
redisExportMaxRows = 2_000_000
|
||||||
|
redisExportBatchRows = 1000
|
||||||
|
redisExportChunkBytes = 256 << 10 // 超大 string 按 256KB 分块 GETRANGE
|
||||||
|
)
|
||||||
|
|
||||||
|
func redisExportSignature(instanceID, db, key, exp string) string {
|
||||||
|
mac := hmac.New(sha256.New, []byte(config.C.JWTSecret))
|
||||||
|
mac.Write([]byte(strings.Join([]string{"redis-export", instanceID, db, key, exp}, "\x00")))
|
||||||
|
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildRedisExportTicket(instanceID, db, key string) (string, int64) {
|
||||||
|
exp := strconv.FormatInt(time.Now().Add(redisExportTicketTTL).Unix(), 10)
|
||||||
|
return exp + "." + redisExportSignature(instanceID, db, key, exp), int64(redisExportTicketTTL.Seconds())
|
||||||
|
}
|
||||||
|
|
||||||
|
func verifyRedisExportTicket(ticket, instanceID, db, key string) error {
|
||||||
|
parts := strings.SplitN(ticket, ".", 2)
|
||||||
|
if len(parts) != 2 {
|
||||||
|
return fmt.Errorf("导出票据格式不正确,请重新点击导出")
|
||||||
|
}
|
||||||
|
expUnix, err := strconv.ParseInt(parts[0], 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("导出票据格式不正确,请重新点击导出")
|
||||||
|
}
|
||||||
|
if time.Now().Unix() > expUnix {
|
||||||
|
return fmt.Errorf("导出链接已过期(有效期 %d 秒),请重新点击导出", int(redisExportTicketTTL.Seconds()))
|
||||||
|
}
|
||||||
|
want := redisExportSignature(instanceID, db, key, parts[0])
|
||||||
|
if !hmac.Equal([]byte(want), []byte(parts[1])) {
|
||||||
|
return fmt.Errorf("导出票据无效")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// POST /api/v1/redis/:id/keys/:key/export-ticket
|
||||||
|
func RedisCreateExportTicket(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
key := c.Param("key")
|
||||||
|
db := c.DefaultQuery("db", "0")
|
||||||
|
if key == "" {
|
||||||
|
resp.BadRequest(c, "缺少 key")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ticket, ttl := buildRedisExportTicket(id, db, key)
|
||||||
|
resp.OK(c, gin.H{"ticket": ticket, "url": c.Request.URL.Path, "expiresIn": ttl})
|
||||||
|
}
|
||||||
|
|
||||||
|
// GET /api/v1/redis/:id/keys/:key/export?db=0&format=csv|jsonl&ticket=xxx
|
||||||
|
//
|
||||||
|
// 全程流式:按批从 Redis 取数 → 立即写响应 → flush。
|
||||||
|
// 服务端内存只占用「一批 + 64KB 缓冲」,与 key 总大小无关。
|
||||||
|
func RedisExportKeyStream(c *gin.Context) {
|
||||||
|
id := c.Param("id")
|
||||||
|
key := c.Param("key")
|
||||||
|
dbParam := c.DefaultQuery("db", "0")
|
||||||
|
format := strings.ToLower(c.DefaultQuery("format", "csv"))
|
||||||
|
if format == "json" {
|
||||||
|
format = "jsonl"
|
||||||
|
}
|
||||||
|
if format != "csv" && format != "jsonl" {
|
||||||
|
resp.BadRequest(c, "format 只支持 csv / jsonl")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := verifyRedisExportTicket(c.Query("ticket"), id, dbParam, key); err != nil {
|
||||||
|
resp.Forbidden(c, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
db, _ := strconv.Atoi(dbParam)
|
||||||
|
var instance model.Instance
|
||||||
|
if err := model.DB.First(&instance, id).Error; err != nil {
|
||||||
|
resp.NotFound(c, "Instance not found")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
client, err := getRedisClient(&instance, db)
|
||||||
|
if err != nil {
|
||||||
|
resp.InternalErr(c, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer client.Close()
|
||||||
|
|
||||||
|
ctx := c.Request.Context()
|
||||||
|
keyType, _ := client.Type(ctx, key).Result()
|
||||||
|
if keyType == "none" {
|
||||||
|
resp.NotFound(c, "Key 不存在")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
filename := sanitizeExportFilename(key) + exportExt(format)
|
||||||
|
c.Header("Content-Type", exportContentType(format))
|
||||||
|
c.Header("Content-Disposition", fmt.Sprintf(`attachment; filename="%s"`, filename))
|
||||||
|
c.Header("X-Content-Type-Options", "nosniff")
|
||||||
|
c.Header("Cache-Control", "no-store")
|
||||||
|
c.Status(http.StatusOK)
|
||||||
|
|
||||||
|
w := bufio.NewWriterSize(c.Writer, 64<<10)
|
||||||
|
flusher, _ := c.Writer.(http.Flusher)
|
||||||
|
flush := func() {
|
||||||
|
_ = w.Flush()
|
||||||
|
if flusher != nil {
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ew := &redisExportWriter{w: w, format: format}
|
||||||
|
ew.begin()
|
||||||
|
if err := streamRedisKey(ctx, client, key, keyType, ew, flush); err != nil {
|
||||||
|
// 响应头已发出,无法改状态码;把错误写进文件尾部,方便使用者与排障。
|
||||||
|
ew.errorRow("导出中断: " + err.Error())
|
||||||
|
}
|
||||||
|
ew.end()
|
||||||
|
flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
// redisExportWriter 统一的 CSV / JSONL 输出器
|
||||||
|
type redisExportWriter struct {
|
||||||
|
w *bufio.Writer
|
||||||
|
format string
|
||||||
|
cw *csv.Writer
|
||||||
|
rowCount int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) begin() {
|
||||||
|
if e.format == "csv" {
|
||||||
|
e.cw = csv.NewWriter(e.w)
|
||||||
|
// Excel 打开 UTF-8 CSV 的正确方式:写 BOM
|
||||||
|
e.w.Write([]byte{0xEF, 0xBB, 0xBF})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) header(cols ...string) {
|
||||||
|
if e.format == "csv" {
|
||||||
|
_ = e.cw.Write(cols)
|
||||||
|
e.cw.Flush()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
e.writeJSON(map[string]any{"_type": "header", "columns": cols})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) row(cols ...string) {
|
||||||
|
e.rowCount++
|
||||||
|
if e.format == "csv" {
|
||||||
|
_ = e.cw.Write(cols)
|
||||||
|
if e.rowCount%redisExportBatchRows == 0 {
|
||||||
|
e.cw.Flush()
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
obj := make(map[string]any, 2)
|
||||||
|
obj["value"] = cols[len(cols)-1]
|
||||||
|
e.writeJSON(obj)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) rowKV(k, v string, extra ...string) {
|
||||||
|
e.rowCount++
|
||||||
|
if e.format == "csv" {
|
||||||
|
_ = e.cw.Write(append([]string{k, v}, extra...))
|
||||||
|
if e.rowCount%redisExportBatchRows == 0 {
|
||||||
|
e.cw.Flush()
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
obj := map[string]any{"key": k, "value": v}
|
||||||
|
if len(extra) > 0 {
|
||||||
|
obj["score"] = extra[0]
|
||||||
|
}
|
||||||
|
e.writeJSON(obj)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) writeJSON(v any) {
|
||||||
|
b, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = e.w.Write(b)
|
||||||
|
_ = e.w.WriteByte('\n')
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) errorRow(msg string) {
|
||||||
|
if e.format == "csv" {
|
||||||
|
_ = e.cw.Write([]string{"__error__", msg})
|
||||||
|
e.cw.Flush()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
e.writeJSON(map[string]any{"_error": msg})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *redisExportWriter) end() {
|
||||||
|
if e.format == "csv" && e.cw != nil {
|
||||||
|
e.cw.Flush()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// streamRedisKey 分批读取并写出,任何时刻只有一批数据在内存里。
|
||||||
|
func streamRedisKey(
|
||||||
|
ctx context.Context,
|
||||||
|
client *redisClientWrapper,
|
||||||
|
key, keyType string,
|
||||||
|
ew *redisExportWriter,
|
||||||
|
flush func(),
|
||||||
|
) error {
|
||||||
|
rows := 0
|
||||||
|
budget := func() bool { return rows < redisExportMaxRows }
|
||||||
|
|
||||||
|
switch keyType {
|
||||||
|
case "string":
|
||||||
|
ew.header("offset", "content")
|
||||||
|
total, _ := client.StrLen(ctx, key).Result()
|
||||||
|
var offset int64
|
||||||
|
for offset < total && budget() {
|
||||||
|
chunk, err := client.GetRange(ctx, key, offset, offset+redisExportChunkBytes-1).Result()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if chunk == "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
ew.rowKV(strconv.FormatInt(offset, 10), chunk)
|
||||||
|
rows++
|
||||||
|
offset += int64(len(chunk))
|
||||||
|
flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
case "list":
|
||||||
|
ew.header("index", "value")
|
||||||
|
total, _ := client.LLen(ctx, key).Result()
|
||||||
|
offset := int64(0)
|
||||||
|
for offset < total && budget() {
|
||||||
|
items, err := client.LRange(ctx, key, offset, offset+redisExportBatchRows-1).Result()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(items) == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
for i, v := range items {
|
||||||
|
ew.rowKV(strconv.FormatInt(offset+int64(i), 10), v)
|
||||||
|
rows++
|
||||||
|
}
|
||||||
|
offset += int64(len(items))
|
||||||
|
flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
case "set":
|
||||||
|
ew.header("member")
|
||||||
|
var cursor uint64
|
||||||
|
for budget() {
|
||||||
|
members, next, err := client.SScan(ctx, key, cursor, "", redisExportBatchRows).Result()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, m := range members {
|
||||||
|
ew.row(m)
|
||||||
|
rows++
|
||||||
|
}
|
||||||
|
flush()
|
||||||
|
if next == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
cursor = next
|
||||||
|
}
|
||||||
|
|
||||||
|
case "zset":
|
||||||
|
ew.header("rank", "member", "score")
|
||||||
|
total, _ := client.ZCard(ctx, key).Result()
|
||||||
|
offset := int64(0)
|
||||||
|
for offset < total && budget() {
|
||||||
|
items, err := client.ZRangeWithScores(ctx, key, offset, offset+redisExportBatchRows-1).Result()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(items) == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
for i, z := range items {
|
||||||
|
ew.rowKV(strconv.FormatInt(offset+int64(i), 10),
|
||||||
|
fmt.Sprint(z.Member), strconv.FormatFloat(z.Score, 'f', -1, 64))
|
||||||
|
rows++
|
||||||
|
}
|
||||||
|
offset += int64(len(items))
|
||||||
|
flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
case "hash":
|
||||||
|
ew.header("field", "value")
|
||||||
|
var cursor uint64
|
||||||
|
for budget() {
|
||||||
|
flat, next, err := client.HScan(ctx, key, cursor, "", redisExportBatchRows).Result()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for i := 0; i+1 < len(flat); i += 2 {
|
||||||
|
ew.rowKV(flat[i], flat[i+1])
|
||||||
|
rows++
|
||||||
|
}
|
||||||
|
flush()
|
||||||
|
if next == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
cursor = next
|
||||||
|
}
|
||||||
|
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("不支持导出的类型: %s", keyType)
|
||||||
|
}
|
||||||
|
|
||||||
|
if rows >= redisExportMaxRows {
|
||||||
|
ew.errorRow(fmt.Sprintf("已达单次导出上限 %d 行,后续数据未包含", redisExportMaxRows))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func exportExt(format string) string {
|
||||||
|
if format == "jsonl" {
|
||||||
|
return ".jsonl"
|
||||||
|
}
|
||||||
|
return ".csv"
|
||||||
|
}
|
||||||
|
|
||||||
|
func exportContentType(format string) string {
|
||||||
|
if format == "jsonl" {
|
||||||
|
return "application/x-ndjson; charset=utf-8"
|
||||||
|
}
|
||||||
|
return "text/csv; charset=utf-8"
|
||||||
|
}
|
||||||
|
|
||||||
|
// sanitizeExportFilename 把 key 转成安全文件名(去掉路径分隔符等)
|
||||||
|
func sanitizeExportFilename(key string) string {
|
||||||
|
var b strings.Builder
|
||||||
|
for _, r := range key {
|
||||||
|
switch {
|
||||||
|
case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9':
|
||||||
|
b.WriteRune(r)
|
||||||
|
case r == ':' || r == '-' || r == '_' || r == '.':
|
||||||
|
b.WriteRune(r)
|
||||||
|
default:
|
||||||
|
b.WriteRune('_')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s := b.String()
|
||||||
|
if s == "" {
|
||||||
|
s = "redis-key"
|
||||||
|
}
|
||||||
|
if len(s) > 120 {
|
||||||
|
s = s[:120]
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
@@ -15,6 +15,11 @@ func Setup(r *gin.Engine) {
|
|||||||
api.POST("/auth/refresh", handler.RefreshToken)
|
api.POST("/auth/refresh", handler.RefreshToken)
|
||||||
api.GET("/public/login-style", handler.GetPublicLoginStyle)
|
api.GET("/public/login-style", handler.GetPublicLoginStyle)
|
||||||
|
|
||||||
|
// Redis 大 Key 导出下载(浏览器原生下载带不了 Authorization 头,
|
||||||
|
// 改由一次性短时效票据鉴权,票据绑定 instance/db/key 且 2 分钟过期)。
|
||||||
|
// 必须注册在鉴权组之外,否则 <a download> 会被 401。
|
||||||
|
api.GET("/redis/:id/keys/:key/export", handler.RedisExportKeyStream)
|
||||||
|
|
||||||
// Authenticated routes
|
// Authenticated routes
|
||||||
auth := api.Group("")
|
auth := api.Group("")
|
||||||
auth.Use(middleware.AuthMiddleware())
|
auth.Use(middleware.AuthMiddleware())
|
||||||
@@ -97,6 +102,8 @@ func Setup(r *gin.Engine) {
|
|||||||
auth.GET("/redis/:id/keys", handler.RedisScanKeysPaged)
|
auth.GET("/redis/:id/keys", handler.RedisScanKeysPaged)
|
||||||
auth.POST("/redis/:id/keys/:key/field", handler.RedisUpdateKeyField)
|
auth.POST("/redis/:id/keys/:key/field", handler.RedisUpdateKeyField)
|
||||||
auth.GET("/redis/:id/keys/:key", handler.RedisGetKeyDetail)
|
auth.GET("/redis/:id/keys/:key", handler.RedisGetKeyDetail)
|
||||||
|
// 大 key 导出:先取一次性票据,再由浏览器原生下载 /export?ticket=...
|
||||||
|
auth.POST("/redis/:id/keys/:key/export-ticket", handler.RedisCreateExportTicket)
|
||||||
auth.GET("/redis/:id/slowlog", handler.RedisSlowlog)
|
auth.GET("/redis/:id/slowlog", handler.RedisSlowlog)
|
||||||
auth.GET("/redis/:id/clients", handler.RedisClients)
|
auth.GET("/redis/:id/clients", handler.RedisClients)
|
||||||
auth.POST("/redis/:id/command", handler.RedisExecuteCommand)
|
auth.POST("/redis/:id/command", handler.RedisExecuteCommand)
|
||||||
|
|||||||
@@ -7,6 +7,26 @@ export interface RedisKeySummary {
|
|||||||
ttl: number;
|
ttl: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** 键详情分页响应(大 key 必须分页,避免整值读入内存) */
|
||||||
|
export interface RedisKeyDetailPage {
|
||||||
|
key: string;
|
||||||
|
type: string;
|
||||||
|
ttl: number;
|
||||||
|
value: any;
|
||||||
|
/** 元素总数(string 为字节数);-1 表示未知 */
|
||||||
|
total: number;
|
||||||
|
/** 本页条数 */
|
||||||
|
count: number;
|
||||||
|
/** 下一页令牌:hash/set 是 SCAN 游标,list/zset 是 offset;空串=没有更多 */
|
||||||
|
nextToken: string;
|
||||||
|
/** 是否还有更多数据(或 string 被预览截断) */
|
||||||
|
truncated: boolean;
|
||||||
|
previewCap?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 导出格式 */
|
||||||
|
export type RedisExportFormat = 'csv' | 'jsonl';
|
||||||
|
|
||||||
/** 游标分页扫描响应 */
|
/** 游标分页扫描响应 */
|
||||||
export interface RedisScanPage {
|
export interface RedisScanPage {
|
||||||
keys: RedisKeySummary[];
|
keys: RedisKeySummary[];
|
||||||
@@ -32,9 +52,32 @@ export const redisApi = {
|
|||||||
scanKeys: (instanceId: number, params: { db?: number; pattern?: string; type?: string; cursor?: string }) =>
|
scanKeys: (instanceId: number, params: { db?: number; pattern?: string; type?: string; cursor?: string }) =>
|
||||||
request.get<any, RedisScanPage>('/redis/' + instanceId + '/keys', { params }),
|
request.get<any, RedisScanPage>('/redis/' + instanceId + '/keys', { params }),
|
||||||
|
|
||||||
// 键详情
|
// 键详情(分页:hash/set 走 SCAN 游标,list/zset 走 offset,string 给预览上限)
|
||||||
getKeyDetail: (instanceId: number, key: string, db?: number) =>
|
getKeyDetail: (instanceId: number, key: string, db?: number, opts?: { token?: string; count?: number }) =>
|
||||||
request.get<any, any>('/redis/' + instanceId + '/keys/' + encodeURIComponent(key), { params: { db: db ?? 0 } }),
|
request.get<any, RedisKeyDetailPage>('/redis/' + instanceId + '/keys/' + encodeURIComponent(key), {
|
||||||
|
params: { db: db ?? 0, count: opts?.count, token: opts?.token || undefined },
|
||||||
|
}),
|
||||||
|
|
||||||
|
// 大 key 导出:申请一次性短时效票据(浏览器原生下载带不了 Authorization 头)
|
||||||
|
createExportTicket: (instanceId: number, key: string, db?: number) =>
|
||||||
|
request.post<any, { ticket: string; expiresIn: number }>(
|
||||||
|
'/redis/' + instanceId + '/keys/' + encodeURIComponent(key) + '/export-ticket',
|
||||||
|
null,
|
||||||
|
{ params: { db: db ?? 0 } },
|
||||||
|
),
|
||||||
|
|
||||||
|
// 拼接导出下载地址(带票据,可直接给 <a download> / location.href)
|
||||||
|
buildExportUrl: (instanceId: number, key: string, db: number, format: RedisExportFormat, ticket: string) =>
|
||||||
|
'/api/v1/redis/' +
|
||||||
|
instanceId +
|
||||||
|
'/keys/' +
|
||||||
|
encodeURIComponent(key) +
|
||||||
|
'/export?db=' +
|
||||||
|
(db ?? 0) +
|
||||||
|
'&format=' +
|
||||||
|
format +
|
||||||
|
'&ticket=' +
|
||||||
|
encodeURIComponent(ticket),
|
||||||
|
|
||||||
// 慢日志
|
// 慢日志
|
||||||
getSlowlog: (instanceId: number, count?: number) =>
|
getSlowlog: (instanceId: number, count?: number) =>
|
||||||
|
|||||||
@@ -1,9 +1,11 @@
|
|||||||
/**
|
/**
|
||||||
* Redis 键详情面板:结构化/格式化/编辑三模式查看,TTL 编辑、保存、删除、Hash 行级编辑。
|
* Redis 键详情面板:结构化/格式化/编辑三模式查看,TTL 编辑、保存、删除、Hash 行级编辑。
|
||||||
* 必需 props:instanceId/db/onKeyDeleted/onRefreshBrowser;已知限制:大值不分页,纯展示截断。
|
* 必需 props:instanceId/db/onKeyDeleted/onRefreshBrowser。
|
||||||
|
* 大 key 已分页(hash/set 走 SCAN 游标,list/zset 走 offset,string 走预览上限),
|
||||||
|
* 并提供「导出当前页」与「导出全部(服务端流式)」两种导出。
|
||||||
*/
|
*/
|
||||||
import { forwardRef, useImperativeHandle, useState } from 'react';
|
import { forwardRef, useImperativeHandle, useState } from 'react';
|
||||||
import { Copy, Save, Trash2, Table2, Code2, FileText, Search } from 'lucide-react';
|
import { Copy, Save, Trash2, Table2, Code2, FileText, Search, Download, ChevronLeft, ChevronRight, Loader2 } from 'lucide-react';
|
||||||
import { redisApi } from '@/api/redis';
|
import { redisApi } from '@/api/redis';
|
||||||
import { toast } from '@/hooks/useToast';
|
import { toast } from '@/hooks/useToast';
|
||||||
import { ConfirmDialog } from '@/components/ConfirmDialog';
|
import { ConfirmDialog } from '@/components/ConfirmDialog';
|
||||||
@@ -27,6 +29,14 @@ interface RedisKeyDetail {
|
|||||||
type: string;
|
type: string;
|
||||||
ttl: number;
|
ttl: number;
|
||||||
value: any;
|
value: any;
|
||||||
|
/** 元素总数(string 为字节数);-1 表示未知 */
|
||||||
|
total?: number;
|
||||||
|
/** 下一页令牌(空串=没有更多) */
|
||||||
|
nextToken?: string;
|
||||||
|
/** 是否还有更多数据 / string 是否被预览截断 */
|
||||||
|
truncated?: boolean;
|
||||||
|
/** string 预览上限(字节) */
|
||||||
|
previewCap?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface RedisKeyDetailPanelHandle {
|
export interface RedisKeyDetailPanelHandle {
|
||||||
@@ -65,10 +75,21 @@ const RedisKeyDetailPanel = forwardRef<RedisKeyDetailPanelHandle, Props>(functio
|
|||||||
});
|
});
|
||||||
const [deleteConfirm, setDeleteConfirm] = useState<{ msg: string; onConfirm: () => void } | null>(null);
|
const [deleteConfirm, setDeleteConfirm] = useState<{ msg: string; onConfirm: () => void } | null>(null);
|
||||||
|
|
||||||
const loadKeyDetail = (keyName: string) => {
|
// ── 大 key 分页 ──
|
||||||
|
/** 每页条数(服务端硬上限 2000) */
|
||||||
|
const [pageSize, setPageSize] = useState(200);
|
||||||
|
/** 当前页令牌('' = 第一页);hash/set 是 SCAN 游标,list/zset 是 offset */
|
||||||
|
const [pageToken, setPageToken] = useState('');
|
||||||
|
/** 已翻过的令牌,用于「上一页」(SCAN 游标不可逆,只能回放) */
|
||||||
|
const [tokenStack, setTokenStack] = useState<string[]>([]);
|
||||||
|
/** 导出进行中(避免重复点击) */
|
||||||
|
const [exporting, setExporting] = useState(false);
|
||||||
|
|
||||||
|
const loadKeyDetail = (keyName: string, token = '', keepStack = false) => {
|
||||||
setDetailLoading(true);
|
setDetailLoading(true);
|
||||||
|
if (!keepStack) setTokenStack([]);
|
||||||
redisApi
|
redisApi
|
||||||
.getKeyDetail(instanceId, keyName, db)
|
.getKeyDetail(instanceId, keyName, db, { token, count: pageSize })
|
||||||
.then((res: any) => {
|
.then((res: any) => {
|
||||||
const detail = res as RedisKeyDetail;
|
const detail = res as RedisKeyDetail;
|
||||||
setSelectedKey(detail);
|
setSelectedKey(detail);
|
||||||
@@ -78,6 +99,7 @@ const RedisKeyDetailPanel = forwardRef<RedisKeyDetailPanelHandle, Props>(functio
|
|||||||
setStringFormat(detail.type === 'string' ? 'Json' : 'String');
|
setStringFormat(detail.type === 'string' ? 'Json' : 'String');
|
||||||
setInspectValue(null);
|
setInspectValue(null);
|
||||||
setValueSearch('');
|
setValueSearch('');
|
||||||
|
setPageToken(token);
|
||||||
})
|
})
|
||||||
.catch(() => {
|
.catch(() => {
|
||||||
toast('error', '加载 Redis Key 详情失败');
|
toast('error', '加载 Redis Key 详情失败');
|
||||||
@@ -205,6 +227,124 @@ const RedisKeyDetailPanel = forwardRef<RedisKeyDetailPanelHandle, Props>(functio
|
|||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/** 当前打开的 key 名 */
|
||||||
|
const currentKeyName = () => selectedKey?.key || selectedKey?.name || '';
|
||||||
|
|
||||||
|
/** 下一页:把当前令牌压栈,再用 nextToken 拉取 */
|
||||||
|
const goNextPage = () => {
|
||||||
|
const next = selectedKey?.nextToken || '';
|
||||||
|
if (!next) return;
|
||||||
|
setTokenStack((stack) => [...stack, pageToken]);
|
||||||
|
loadKeyDetail(currentKeyName(), next, true);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** 上一页:从栈里弹出上一个令牌(SCAN 游标不可逆,只能回放) */
|
||||||
|
const goPrevPage = () => {
|
||||||
|
if (tokenStack.length === 0) return;
|
||||||
|
const prev = tokenStack[tokenStack.length - 1];
|
||||||
|
setTokenStack((stack) => stack.slice(0, -1));
|
||||||
|
loadKeyDetail(currentKeyName(), prev, true);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** 改每页条数:回到第一页重新取 */
|
||||||
|
const changePageSize = (size: number) => {
|
||||||
|
setPageSize(size);
|
||||||
|
setTokenStack([]);
|
||||||
|
const keyName = currentKeyName();
|
||||||
|
if (!keyName) return;
|
||||||
|
setDetailLoading(true);
|
||||||
|
redisApi
|
||||||
|
.getKeyDetail(instanceId, keyName, db, { token: '', count: size })
|
||||||
|
.then((res: any) => {
|
||||||
|
setSelectedKey(res as RedisKeyDetail);
|
||||||
|
setEditValue(stringifyValue(res.type, res.value));
|
||||||
|
setPageToken('');
|
||||||
|
})
|
||||||
|
.catch(() => toast('error', '加载 Redis Key 详情失败'))
|
||||||
|
.finally(() => setDetailLoading(false));
|
||||||
|
};
|
||||||
|
|
||||||
|
/** 把当前页数据转成 CSV / JSON 文本(纯前端,不再请求后端) */
|
||||||
|
const buildPageText = (format: 'csv' | 'json') => {
|
||||||
|
if (!selectedKey) return '';
|
||||||
|
const type = selectedKey.type;
|
||||||
|
const val: any = selectedKey.value;
|
||||||
|
const csvCell = (v: any) => {
|
||||||
|
const s = v === null || v === undefined ? '' : String(v);
|
||||||
|
return /[",\n\r]/.test(s) ? '"' + s.replace(/"/g, '""') + '"' : s;
|
||||||
|
};
|
||||||
|
const rows: string[][] = [];
|
||||||
|
if (type === 'hash') {
|
||||||
|
rows.push(['field', 'value']);
|
||||||
|
Object.entries(val || {}).forEach(([f, v]) => rows.push([f, String(v ?? '')]));
|
||||||
|
} else if (type === 'list') {
|
||||||
|
rows.push(['index', 'value']);
|
||||||
|
(val || []).forEach((v: any, i: number) => rows.push([String(i), String(v ?? '')]));
|
||||||
|
} else if (type === 'set') {
|
||||||
|
rows.push(['member']);
|
||||||
|
(val || []).forEach((v: any) => rows.push([String(v ?? '')]));
|
||||||
|
} else if (type === 'zset') {
|
||||||
|
rows.push(['member', 'score']);
|
||||||
|
(val || []).forEach((z: any) => rows.push([String(z.Member ?? z.member ?? ''), String(z.Score ?? z.score ?? '')]));
|
||||||
|
} else {
|
||||||
|
rows.push(['content']);
|
||||||
|
rows.push([String(val ?? '')]);
|
||||||
|
}
|
||||||
|
if (format === 'csv') return rows.map((r) => r.map(csvCell).join(',')).join('\n');
|
||||||
|
return JSON.stringify(type === 'string' ? val : val, null, 2);
|
||||||
|
};
|
||||||
|
|
||||||
|
const downloadBlob = (text: string, filename: string, mime: string) => {
|
||||||
|
const blob = new Blob(['\ufeff' + text], { type: mime });
|
||||||
|
const url = URL.createObjectURL(blob);
|
||||||
|
const a = document.createElement('a');
|
||||||
|
a.href = url;
|
||||||
|
a.download = filename;
|
||||||
|
document.body.appendChild(a);
|
||||||
|
a.click();
|
||||||
|
document.body.removeChild(a);
|
||||||
|
setTimeout(() => URL.revokeObjectURL(url), 1000);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** 导出当前页(纯前端,适合几万行以内,立即完成) */
|
||||||
|
const exportCurrentPage = (format: 'csv' | 'json') => {
|
||||||
|
if (!selectedKey) return;
|
||||||
|
const keyName = currentKeyName();
|
||||||
|
const text = buildPageText(format);
|
||||||
|
const safe = keyName.replace(/[^\w.-]+/g, '_').slice(0, 80) || 'redis-key';
|
||||||
|
downloadBlob(
|
||||||
|
text,
|
||||||
|
`${safe}_page${tokenStack.length + 1}.${format === 'csv' ? 'csv' : 'json'}`,
|
||||||
|
format === 'csv' ? 'text/csv;charset=utf-8' : 'application/json;charset=utf-8',
|
||||||
|
);
|
||||||
|
toast('success', '当前页已导出(如需全量请用「导出全部」)');
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 导出全部(服务端流式):
|
||||||
|
* 先把一次性票据拿到手,再交给浏览器原生下载,避免整份数据进浏览器内存。
|
||||||
|
*/
|
||||||
|
const exportAll = (format: 'csv' | 'jsonl') => {
|
||||||
|
if (!selectedKey) return;
|
||||||
|
const keyName = currentKeyName();
|
||||||
|
setExporting(true);
|
||||||
|
redisApi
|
||||||
|
.createExportTicket(instanceId, keyName, db)
|
||||||
|
.then((res) => {
|
||||||
|
if (!res?.ticket) throw new Error('未取到导出票据');
|
||||||
|
const url = redisApi.buildExportUrl(instanceId, keyName, db, format, res.ticket);
|
||||||
|
const a = document.createElement('a');
|
||||||
|
a.href = url;
|
||||||
|
a.download = '';
|
||||||
|
document.body.appendChild(a);
|
||||||
|
a.click();
|
||||||
|
document.body.removeChild(a);
|
||||||
|
toast('success', '已开始下载(大 key 会边读边写,请耐心等待)');
|
||||||
|
})
|
||||||
|
.catch((err) => toast('error', err instanceof Error ? err.message : '导出失败'))
|
||||||
|
.finally(() => setExporting(false));
|
||||||
|
};
|
||||||
|
|
||||||
const renderFormattedValue = () => {
|
const renderFormattedValue = () => {
|
||||||
if (!selectedKey) return '';
|
if (!selectedKey) return '';
|
||||||
if (inspectValue) {
|
if (inspectValue) {
|
||||||
@@ -334,6 +474,97 @@ const RedisKeyDetailPanel = forwardRef<RedisKeyDetailPanelHandle, Props>(functio
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{/* ── 分页 + 导出条(大 key 必看) ── */}
|
||||||
|
<div className="flex flex-wrap items-center justify-between gap-3 border-b border-[var(--dg-border)] bg-[var(--dg-bg-secondary)] px-4 py-2 text-xs">
|
||||||
|
<div className="flex flex-wrap items-center gap-3">
|
||||||
|
<span className="text-[var(--dg-text-muted)]">
|
||||||
|
共 <b className="text-[var(--dg-text-primary)]">
|
||||||
|
{typeof selectedKey.total === 'number' && selectedKey.total >= 0
|
||||||
|
? selectedKey.total.toLocaleString()
|
||||||
|
: '?'}
|
||||||
|
</b>{' '}
|
||||||
|
{selectedKey.type === 'string' ? '字节' : '条'}
|
||||||
|
<span className="mx-1">·</span>
|
||||||
|
当前第 <b className="text-[var(--dg-text-primary)]">{tokenStack.length + 1}</b> 页,每页
|
||||||
|
</span>
|
||||||
|
<select
|
||||||
|
value={pageSize}
|
||||||
|
onChange={(e) => changePageSize(Number(e.target.value))}
|
||||||
|
disabled={selectedKey.type === 'string' || detailLoading}
|
||||||
|
className="h-7 rounded-md border border-[var(--dg-border)] bg-[var(--dg-bg-primary)] px-2 disabled:opacity-50"
|
||||||
|
>
|
||||||
|
{[100, 200, 500, 1000, 2000].map((n) => (
|
||||||
|
<option key={n} value={n}>
|
||||||
|
{n}
|
||||||
|
</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
<div className="flex items-center gap-1">
|
||||||
|
<button
|
||||||
|
onClick={goPrevPage}
|
||||||
|
disabled={tokenStack.length === 0 || detailLoading}
|
||||||
|
className="inline-flex h-7 items-center gap-1 rounded-md border border-[var(--dg-border)] px-2 text-[var(--dg-text-secondary)] hover:bg-[var(--dg-bg-hover)] disabled:opacity-40"
|
||||||
|
title="上一页"
|
||||||
|
>
|
||||||
|
<ChevronLeft className="h-3.5 w-3.5" />
|
||||||
|
上一页
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
onClick={goNextPage}
|
||||||
|
disabled={!selectedKey.nextToken || detailLoading}
|
||||||
|
className="inline-flex h-7 items-center gap-1 rounded-md border border-[var(--dg-border)] px-2 text-[var(--dg-text-secondary)] hover:bg-[var(--dg-bg-hover)] disabled:opacity-40"
|
||||||
|
title="下一页"
|
||||||
|
>
|
||||||
|
下一页
|
||||||
|
<ChevronRight className="h-3.5 w-3.5" />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
{selectedKey.type === 'string' && selectedKey.truncated && (
|
||||||
|
<span className="rounded bg-amber-500/10 px-2 py-1 text-amber-400">
|
||||||
|
超大字符串:仅预览前 {Math.round((selectedKey.previewCap || 262144) / 1024)} KB(共{' '}
|
||||||
|
{selectedKey.total?.toLocaleString() ?? '?'} 字节),完整内容请「导出全部」
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="flex items-center gap-2">
|
||||||
|
<span className="text-[var(--dg-text-muted)]">导出</span>
|
||||||
|
<button
|
||||||
|
onClick={() => exportCurrentPage('csv')}
|
||||||
|
className="inline-flex h-7 items-center gap-1 rounded-md border border-[var(--dg-border)] px-2 text-[var(--dg-text-secondary)] hover:bg-[var(--dg-bg-hover)]"
|
||||||
|
title="仅导出当前页(纯前端生成,立即完成)"
|
||||||
|
>
|
||||||
|
<Download className="h-3.5 w-3.5" />
|
||||||
|
当前页 CSV
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
onClick={() => exportCurrentPage('json')}
|
||||||
|
className="inline-flex h-7 items-center gap-1 rounded-md border border-[var(--dg-border)] px-2 text-[var(--dg-text-secondary)] hover:bg-[var(--dg-bg-hover)]"
|
||||||
|
title="仅导出当前页(JSON)"
|
||||||
|
>
|
||||||
|
<Download className="h-3.5 w-3.5" />
|
||||||
|
当前页 JSON
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
onClick={() => exportAll('csv')}
|
||||||
|
disabled={exporting}
|
||||||
|
className="inline-flex h-7 items-center gap-1 rounded-md bg-[var(--dg-blue)] px-2.5 font-medium text-white hover:opacity-90 disabled:opacity-50"
|
||||||
|
title="服务端流式导出全部数据,边读边写不占内存"
|
||||||
|
>
|
||||||
|
{exporting ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <Download className="h-3.5 w-3.5" />}
|
||||||
|
导出全部 CSV
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
onClick={() => exportAll('jsonl')}
|
||||||
|
disabled={exporting}
|
||||||
|
className="inline-flex h-7 items-center gap-1 rounded-md border border-[var(--dg-border)] px-2 text-[var(--dg-text-secondary)] hover:bg-[var(--dg-bg-hover)] disabled:opacity-50"
|
||||||
|
title="服务端流式导出全部数据(JSON Lines)"
|
||||||
|
>
|
||||||
|
导出全部 JSONL
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div className="min-h-0 flex-1 overflow-auto p-4 scrollbar-thin">
|
<div className="min-h-0 flex-1 overflow-auto p-4 scrollbar-thin">
|
||||||
{detailLoading ? (
|
{detailLoading ? (
|
||||||
<div className="h-72 animate-pulse rounded-lg bg-[var(--dg-bg-tertiary)]" />
|
<div className="h-72 animate-pulse rounded-lg bg-[var(--dg-bg-tertiary)]" />
|
||||||
|
|||||||
Reference in new issue
Block a user