2026-08-04 15:46:09 +08:00
|
|
|
|
# -*- coding: utf-8 -*-
|
|
|
|
|
|
"""iAOP-Core · 数据总线 + 时序库(Data Bus)—— 模板化封装。
|
|
|
|
|
|
|
|
|
|
|
|
对应 PRD 5.2「② 数据总线 + 时序库」与 Issue #4(EPIC):
|
|
|
|
|
|
复用 Kafka + TDengine + PostgreSQL + MinIO,全部改为**按模板配置**
|
|
|
|
|
|
topic / 时序库表 / 关系表 / 对象桶,换行业只改模板资产(YAML + 点位字典),
|
|
|
|
|
|
内核代码零改动;批量写入缓冲 + 幂等去重,保障「数据不丢不重」。
|
2026-08-04 16:58:06 +08:00
|
|
|
|
|
|
|
|
|
|
模块:
|
|
|
|
|
|
- templating 模板命名推导(Kafka topic/分区、TDengine 表、PG schema、MinIO 桶);
|
2026-08-05 00:35:19 +08:00
|
|
|
|
- kafka_naming Kafka topic 命名/分区模板化组件(配置外置 + 点位维度覆盖 + 生产路由);
|
2026-08-04 16:58:06 +08:00
|
|
|
|
- tdengine_schema 时序超级表/子表 DDL + 批量 INSERT SQL 生成;
|
|
|
|
|
|
- postgres_schema 关系库 schema(模板/模型/用户/权限)+ 授权语句;
|
|
|
|
|
|
- batch_writer 批量写入缓冲(StoreSink 抽象 / MemorySink / TdengineSink),
|
|
|
|
|
|
批量聚合 + 幂等去重 + 失败重试(不丢不重)。
|
|
|
|
|
|
|
|
|
|
|
|
测试:`python -m unittest discover -s tests -v`(在 core/data-bus 目录下执行)。
|
2026-08-04 15:46:09 +08:00
|
|
|
|
"""
|
2026-08-04 16:58:06 +08:00
|
|
|
|
__version__ = "0.2.0"
|
|
|
|
|
|
|
|
|
|
|
|
from .batch_writer import (
|
|
|
|
|
|
DEDUP_KEYS,
|
|
|
|
|
|
BatchWriter,
|
|
|
|
|
|
MemorySink,
|
|
|
|
|
|
StoreSink,
|
|
|
|
|
|
TdengineSink,
|
|
|
|
|
|
)
|
2026-08-05 00:35:19 +08:00
|
|
|
|
from .kafka_naming import KafkaTopicNaming
|
2026-08-04 16:58:06 +08:00
|
|
|
|
from .postgres_schema import generate_grant_ddl, generate_schema_ddl
|
|
|
|
|
|
from .tdengine_schema import (
|
|
|
|
|
|
PointSpec,
|
|
|
|
|
|
build_batch_insert,
|
2026-08-04 18:14:37 +08:00
|
|
|
|
generate_schema,
|
2026-08-04 16:58:06 +08:00
|
|
|
|
generate_subtable_ddls,
|
|
|
|
|
|
generate_supertable_ddl,
|
2026-08-04 18:14:37 +08:00
|
|
|
|
specs_from_point_dict_rows,
|
2026-08-04 16:58:06 +08:00
|
|
|
|
)
|
|
|
|
|
|
from .templating import TemplateNaming, sanitize, sanitize_sql
|
|
|
|
|
|
|
|
|
|
|
|
__all__ = [
|
|
|
|
|
|
"TemplateNaming",
|
|
|
|
|
|
"sanitize",
|
|
|
|
|
|
"sanitize_sql",
|
2026-08-05 00:35:19 +08:00
|
|
|
|
"KafkaTopicNaming",
|
2026-08-04 16:58:06 +08:00
|
|
|
|
"PointSpec",
|
|
|
|
|
|
"generate_supertable_ddl",
|
|
|
|
|
|
"generate_subtable_ddls",
|
2026-08-04 18:14:37 +08:00
|
|
|
|
"generate_schema",
|
|
|
|
|
|
"specs_from_point_dict_rows",
|
2026-08-04 16:58:06 +08:00
|
|
|
|
"build_batch_insert",
|
|
|
|
|
|
"generate_schema_ddl",
|
|
|
|
|
|
"generate_grant_ddl",
|
|
|
|
|
|
"StoreSink",
|
|
|
|
|
|
"MemorySink",
|
|
|
|
|
|
"TdengineSink",
|
|
|
|
|
|
"BatchWriter",
|
|
|
|
|
|
"DEDUP_KEYS",
|
|
|
|
|
|
]
|