feat: 完成 issue #8 ⑥ K8s/Helm 部署底座 + 昇腾适配层
This commit is contained in:
@@ -0,0 +1,51 @@
|
||||
# iAOP-Core · 推理后端抽象层(Inference Backend)
|
||||
|
||||
对应 PRD 5.6「⑥ 部署底座」与 EPIC #8:
|
||||
NVIDIA GPU(5090,Triton/ONNX)与华为昇腾 NPU(ACL/CANN)实现**同一**
|
||||
`InferenceBackend` 接口,业务代码仅依赖接口、不感知硬件;
|
||||
**切换后端 = 改适配层配置**(`backend: gpu|npu`),业务代码零改动。
|
||||
|
||||
## 模块结构
|
||||
|
||||
```
|
||||
core/inference-backend/
|
||||
├── __init__.py 包入口(导出接口/请求/结果/工厂)
|
||||
├── base.py InferenceBackend 统一抽象接口(load_model/infer/health/unload)
|
||||
├── gpu_backend.py NVIDIA 5090 后端(Triton/ONNX,OpenAI 兼容)
|
||||
├── npu_backend.py 华为昇腾 NPU 后端(ACL/CANN,MindIE OpenAI 兼容)
|
||||
├── factory.py 可插拔工厂(backend: gpu|npu -> 实现类)
|
||||
├── config/
|
||||
│ └── backends.template.yaml 模板后端适配层配置资产(ti-cl4 示例)
|
||||
└── tests/
|
||||
├── _bootstrap.py 测试引导(目录含连字符,挂载包名 inference_backend)
|
||||
└── test_inference_backend.py 接口/工厂/行为单元测试
|
||||
```
|
||||
|
||||
## 统一接口(PRD 5.6:loadModel / infer / health / unload)
|
||||
|
||||
业务代码只依赖 `InferenceBackend` 接口,不 import 任何具体后端:
|
||||
|
||||
```python
|
||||
from inference_backend.base import InferRequest
|
||||
from inference_backend.factory import build_backend, load_backend_config
|
||||
|
||||
backend = build_backend(load_backend_config("config/backends.template.yaml"))
|
||||
|
||||
backend.load_model()
|
||||
health = backend.health() # 健康巡检
|
||||
result = backend.infer(InferRequest(prompt="请解释炉温报警")) # result.text
|
||||
backend.unload()
|
||||
```
|
||||
|
||||
## 切换后端(仅改配置)
|
||||
|
||||
`config/backends.template.yaml` 中 `backend: gpu` 改为 `backend: npu` 即切换到
|
||||
昇腾适配层;同一份业务代码、同一调用面,硬件差异被适配层隔离。
|
||||
新增硬件:在 `factory.py` 的 `BACKEND_REGISTRY` 注册实现类即可。
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
cd core/inference-backend/tests
|
||||
python -m unittest discover -s . -p "test_*.py"
|
||||
```
|
||||
Reference in New Issue
Block a user