测试服务器收到 CRITICAL「增量空转:PAC 拉到 0 行,但 DW 有新数据」。排查 sync_logs: 07-25 16:15 UTC failed fetched=0 err="fatal: unexpected end of file" 07-26 00:15 UTC success fetched=239959 tx=103652 ← 下一轮全额补回 即:一次瞬时 ClickHouse 连接断开导致该轮失败,游标未推进,下一轮把积压全额补回 (tx=10万+,远高于平时 ~1万),**无数据丢失**。告警是误报,两处 bug 叠加: ## ① 探针在失败轮上也跑(主因) 空转告警条件只看 `tx+dup=0`,没看 status。失败轮 tx=dup=0 → 走进探针 → 把瞬时网络失败误诊成「游标格式失效/空转」。失败轮本就经日报失败计数 + error_message 暴露,不该再被探针误标。 修:加 `status===SUCCESS` 守卫,只在**成功空轮**上探。 ## ② 探针反查不带业务过滤(放大误报) 探针反查只拼 `cursor > val`,而真实增量拉取还叠加业务过滤(结算正表 `is_refund=0 AND settlement_status=1`、退费表 `is_refund=1 OR settlement_status=4`、 结算方式 `settlement_status='1'`)。于是"游标之后、但会被业务条件过滤掉"的退费行 被探针数进去 → 真实拉取正确地不拉,探针却告警。 告警数据自证:settlement 正/退表各 5788 完全相等,正是这批全为退费单的铁证。 修:探针复用 extractBusinessFilters,与真实拉取同一套 WHERE。 ## 验证 - 388 单测通过(新增 6 例:各表业务过滤保留/剔除、括号 OR 不被 AND 拆开) - 失败原因「unexpected end of file」是瞬时网络错误,已自愈,无需数据补摄 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
| Name |
Last commit
|
Last update |
|---|---|---|
| .claude | Loading commit data... | |
| .design-sync | Loading commit data... | |
| apps | Loading commit data... | |
| clickhouse/config.d | Loading commit data... | |
| deploy | Loading commit data... | |
| docs | Loading commit data... | |
| packages | Loading commit data... | |
| scripts | Loading commit data... | |
| .gitignore | Loading commit data... | |
| .gitlab-ci.yml | Loading commit data... | |
| .npmrc | Loading commit data... | |
| .prettierrc | Loading commit data... | |
| README.md | Loading commit data... | |
| docker-compose.expose.yml | Loading commit data... | |
| docker-compose.managed.yml | Loading commit data... | |
| docker-compose.prod.yml | Loading commit data... | |
| docker-compose.yml | Loading commit data... | |
| eslint.config.mjs | Loading commit data... | |
| liu.cjs | Loading commit data... | |
| package.json | Loading commit data... | |
| pnpm-lock.yaml | Loading commit data... | |
| pnpm-workspace.yaml | Loading commit data... | |
| tsconfig.base.json | Loading commit data... | |
| turbo.json | Loading commit data... |