deployment-data-ingest.md 更新到当前实现: - §2.2 加 sync 资源调优 env(COHORT_BATCH_SIZE/CONCURRENCY/WRITE_BATCH_SIZE) + Docker Compose --env-file 必带 + connection_limit 说明 - §2.4 加 docker compose prod 部署(--env-file + pac-migrate/pac-service 两镜像都要 rebuild) - §3 重写数据摄入 SOP: · 3.1 cold-import → 统一 pnpm sync 首跑(cohort 分批 + 并行,~1.5-2h vs 旧 9-12h) · 3.2 增量走同一 importDirectory(并发锁 + run_start cursor) · 3.3 幂等 DB 双 UNIQUE(含补的 source_event partial unique)+ cohort 宿主无关 · 3.4 新增资源调优表(cohort/concurrency/write-batch/query 并行) - §7.3 cursor 倒退加 --full;§7.4 新增 stale running 锁清理 SOP - §8 已知边界更新(去掉过时的 LIMIT 100;加 COPY/看门狗/并行锁) - 一句话归纳重写 反映:cold-import/sync-incremental 合并;cohort batch(PR2); 统一 mode(PR3);bulk write(PR4);query 并行(PR5a);cohort 并行(PR5b); source_event partial unique 补建;宿主无关 cohort 配置。
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| .obsidian | Loading commit data... | |
| algorithm | Loading commit data... | |
| scenarios | Loading commit data... | |
| architecture-v2.md | Loading commit data... | |
| architecture.md | Loading commit data... | |
| db-design-v2.md | Loading commit data... | |
| db-review-confirm.md | Loading commit data... | |
| db-review.md | Loading commit data... | |
| db-suggest-after-review.md | Loading commit data... | |
| demo-page-prompt.md | Loading commit data... | |
| deployment-data-ingest.md | Loading commit data... | |
| design-philosophy.md | Loading commit data... | |
| design-system.md | Loading commit data... | |
| design-v2.md | Loading commit data... | |
| design.md | Loading commit data... | |
| dw-data-source-issues.md | Loading commit data... | |
| host-data-dictionary-example.yaml | Loading commit data... | |
| host-data-request-checklist-v2.md | Loading commit data... | |
| host-data-request-checklist.md | Loading commit data... | |
| host-integration-clickhouse-view.md | Loading commit data... | |
| host-integration-v2.md | Loading commit data... | |
| host-integration.md | Loading commit data... | |
| onboarding-runbook.md | Loading commit data... | |
| pac-for-committee.md | Loading commit data... | |
| pac-goal-alignment.md | Loading commit data... | |
| prompt.md | Loading commit data... | |
| three-layer-model.md | Loading commit data... | |
| treatment-chain-followup-recall.md | Loading commit data... | |
| w1-report.md | Loading commit data... | |
| w2-report.md | Loading commit data... | |
| w3-report.md | Loading commit data... |