25 Commits

Author SHA1 Message Date
2d9cb35590 feat(ocr,det,slide): 重构项目结构
- 优化 规范化模型目录
- 重构 Ocr,Detector,Slide 拆分规范化
2026-07-09 19:26:58 +08:00
0cf3d5fefb feat(ocr,det,slide): 重构配置解析流程,移除非必要的生命周期方法
- 优化 规范化模型目录
- 重构 Ocr,Detector配置解析流程
2026-07-08 15:48:56 +08:00
31271e80db refactor(slide,det): 优化项目结构,移除不必要的逻辑
- 优化 项目结构,移除不必要的逻辑
2026-07-07 09:55:00 +08:00
7f1ce04f50 refactor(slide,det): 重构目标检测引擎并统一图像输入类型为 DynamicImage以及滑块匹配与比较引擎为 Rust 实现
- 统一 `predict` 和 `get_bbox` 接口参数为 `&DynamicImage`,消除多步处理时的重复图像解码开销。
- 引入轻量级 `DetectionResult` 结构体和固定大小数组 `[f32; 6]` 替代旧的嵌套 `Vec`,彻底消除后处理中的内存碎片。
- 优化 `preproc` 预处理逻辑,使用连续内存切片批量操作替代原有的逐像素迭代遍历。
- 移除多余的 `multiclass_nms_class_agnostic` 转发层,合并并精简 NMS 聚合函数。
- 优化 `calculate_center` 几何中心点计算函数,提高泛型语义并复用于两种匹配模式
- 在执行核心算法前增加尺寸与通道边界守卫(Guard Clauses),提升库的防防御性编程能力与崩溃安全性
- 移除多余的错误二次包装(map_err),改由 Rust 原生 Result 错误传播机制直接向上层抛出
2026-07-03 17:51:28 +08:00
22cc9709ad refactor(predict): 重构预测流水线并优化模型元数据与输出架构
- 优化 `predict` 核心方法:移除冗余日志与深层嵌套,将流程重塑为线性流水线。
- 重构 `compute_f32_full_probability`:解耦逻辑与外部状态,消除并发隐患与生命周期冲突。
- 增强 `ModelMetadata`:引入动态归一化配置并支持 Serde 序列化,解决特定模型漏字问题。
- 升级 `OcrOutput`:
  - 增加 `Unsupported` 变体以支持非致命异常的优雅降级。
  - 实现 `into_text(self)` 方法与 `Display` 特征(应用双重截断保护,防止日志刷屏)。

BREAKING CHANGE: `predict` 返回值由 `anyhow::Result<String>` 改为 `anyhow::Result<OcrOutput>`,将后处理和控制权移交上层。
2026-07-02 20:24:52 +08:00
b352fc344f refactor(ocr): 优化 preprocess_image 逻辑实现,修复部分BUG
- 优化 `OcrBuilder` 重名名为 `OcrPredictor`
 - 优化 `OcrPredictor` 的 `preprocess_image` 支持多种图像管道。
 - 修复 `OcrPredictor` 引发的并发BUG。
2026-07-01 20:22:17 +08:00
48c2cbedb0 refactor(ocr): 优化并精简 color_filter 架构设计
- 重构 `OcrBuilder`,将图像矩阵过滤与像素比对等执行层逻辑彻底剥离解耦。
 - 优化 `OcrBuilder` 的 `color_filter` 链式调用,将其改造为无心智负担的单次覆盖(Overwrite)逻辑。
 - 扩展 `ColorFilter` 特征,新增 `collect_to_vec` 方法,实现底层规则的高内聚收集、精准内存开辟与原地去重排序。
2026-06-29 17:19:01 +08:00
2f86694c54 refactor(ocr): 优化 color_filter.rs
- 重构 `OcrBuilder` 移除is_pixel_matched,filter_image。
 - 优化 `OcrBuilder` 的color_filter方法(部分逻辑转移给merge_to_vec) 。
 - 新增 `ColorFilter` 特征增加merge_to_vec方法。
2026-06-25 20:25:49 +08:00
62d5e7a0ca refactor(ocr): 优化 HSV 颜色过滤架构,实现快捷预设免检与大一统 Custom 变体
- 重构 `ColorPreset` 枚举,新增 `Custom(Vec<HsvRange>)` 变体。
 - 优化 `ColorFilter` 特征兼容多路组合宏。
 - 新增 `validate_self` 特征多态方法,实现责任分离:库担保的快捷预设 0 运行时开销免检放行,仅对 `Custom` 动态数据进行严格自检。
 - 优化 `OcrBuilder::color_filter` 接收 `&dyn ColorFilter` 特征对象,完美兼容原有声明式宏与链式调用熔断机制。
 - 借鉴 `reqwest` 的延迟错误处理模式,完善 `OcrBuilder` 的链式调用熔断(毒化)状态机。
2026-06-18 17:40:29 +08:00
189f2bd697 refactor(ocr): 拆分字符集限制枚举并引入声明式多路组合子宏
- 将原本臃肿的 CharsetRestrict 拆分为 CharRestrict(内容过滤)和 IdRestrict(索引过滤),实现职责解耦
 - 引入 any_of! 声明式宏实现无 Box、无堆内存分配的栈上多路组合,规避孤儿规则
 - 完善 estimated_capacity 容量预估函数,实现真正的 O(1) 精准内存开辟
2026-06-16 09:37:15 +08:00
b7146831f7 refactor: 优化 OcrBuilder 新增通过索引范围控制有效字符集
- 修改 `charset_restrict`类型修改为Option<Vec<usize>> 并重构同名方法数据处理逻辑
- 优化 `ctc_decode_to_string` 内部复用策略计算,通过 `Option` 结构实现无限制请求的全量免检短路加速
- 新增 `CharsetRestrict`枚举新增变体`TopN(usize)` 实现通过索引范围控制有效字符集
2026-06-13 17:13:00 +08:00
0c96fbedbf refactor: 重构 OcrBuilder 过滤机制,实现零克隆、高性能的局部白名单
- 将有状态的有效索引缓存 `valid_indices` 从全局 `Charset` 剥离至局部 `OcrBuilder`
- 解码函数 `ctc_decode_to_string` 内部复用策略计算,通过 `has_any_match` 布尔开关实现全量免检短路加速
- 优化内存分配,根据 `CharsetRestrict` 策略动态精准计算 `HashSet` 初始容量,规避大词表空置浪费
- 增强鲁棒性,在策略与字库完全无交集时,自动触发智能降级,一键恢复全量识别并保持接口自洽
2026-06-09 18:12:02 +08:00
15ce068025 feat: 字符集限制枚举优化与核心解码器对接
- 新增 model_metadata.rs
- 优化 charset.rs
- 其他优化
2026-06-05 17:30:10 +08:00
cb786a7a1a refactor: 重构 ocr.rs 实现丝滑参数设置
- 重构 Ocr
- 新增 OcrTask
2026-05-29 19:13:45 +08:00
0923d92150 feat: 优化 slide.rs 2026-05-11 22:54:05 +08:00
0df9022411 feat: 优化 项目目录结构 2026-05-10 20:52:42 +08:00
a51147c888 refactor: 优化 slide_model.rs
- 新增 cv2.rs 模拟 opencv
2026-05-09 17:52:34 +08:00
e8b365dced feat: 优化 image_io.rs 模块
- 新增 base64_to_image等工具函数。
2026-05-08 22:35:17 +08:00
f0db625bd1 refactor: 完成图像加载模块重构,对齐 ddddocr Python 原版 IO 逻辑 2026-05-08 17:59:42 +08:00
21bd1c93bf feat: 完成 Rust 滑块匹配算法,修复透明留白导致的坐标偏移
- 实现灰度与边缘两种匹配模式
- 对齐 OpenCV NCC 算法逻辑
- 优化图像灰度化与 Alpha 通道转换
- 提升坐标计算精度至像素级
2026-05-08 16:03:33 +08:00
1a329ca273 refactor: 优化Det算法
- 优化 demo_postprocess,nms算法
- 新增 Slide 滑块识别
- 更新 Cargo.toml 依赖项
2026-05-07 18:00:39 +08:00
8fcfa2096e refactor: 移除 OpenCV 依赖并实现纯 Rust 图像处理流水线
- 替换 opencv 为 image 库以简化交叉编译
- 修正 nms 逻辑中的 ArrayView 借用问题
- 增加 save_debug_image 方法用于可视化检测框
- 更新 Cargo.toml 依赖项
2026-05-06 17:37:38 +08:00
cfeb68ad04 feat: 重构模型初始化逻辑
- 重构 DdddOcr。
- 新增 DdddOcrBuilder。
- 其他优化
2026-05-05 22:18:12 +08:00
1c366b7165 feat: 重构 CTC 解码逻辑
- 重构 ctc_decode 为关联函数并优化内存分配。
- 增加 单元测试和集成测试
2026-05-01 21:54:33 +08:00
642fed5d9f feat: 实现 DdddOcr 核心推理流水线与图像预处理
- 封装 `preprocess_image` 方法,实现 PNG 透明背景修复、灰度化、比例缩放及 NCHW 张量转换。
- 提取 `inference` 逻辑,支持通过 tract-onnx 执行模型推理。
- 实现 `extract_indices` 解析输出张量,支持 I64 索引直接读取与 F32 概率矩阵的 Argmax 处理。
- 完善 `decode_ctc` 解码算法,支持标准 CTC 贪婪搜索与字符集映射。
- 重构 `classification` 主入口,将预处理、推理、解析、解码逻辑解耦,提升代码可维护性。
2026-04-30 17:54:08 +08:00
36 changed files with 2765 additions and 65 deletions

8
.idea/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,8 @@
# 默认忽略的文件
/shelf/
/workspace.xml
# 基于编辑器的 HTTP 客户端请求
/httpRequests/
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml

View File

@@ -5,6 +5,16 @@ edition = "2024"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
[dependencies] [dependencies]
tract-onnx = { version = "0.21.1" } tract-onnx = { version = "0.21.10" }
anyhow = "1.0.102" anyhow = "1.0.102"
image = "0.25.10" image = "0.25.10"
base64 = "0.22.1"
imageproc = { version = "0.26.2", default-features = true }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
ndarray="0.16.1"
[features]
default = []
embed-models = [] # 这是一个留给有特殊需求、且自己下载了模型放入 models/ 目录的人的后门

View File

@@ -2,8 +2,43 @@
带带弟弟 OCR (ddddocr) 的 Rust 移植版。高性能、低占用,支持多种验证码识别与检测。 带带弟弟 OCR (ddddocr) 的 Rust 移植版。高性能、低占用,支持多种验证码识别与检测。
🧩 滑块识别算法核心知识点总结
本项目实现了两种核心匹配模式,其底层逻辑与 OpenCV 的对齐情况如下:
1. 匹配模式对比 (Match Modes)
|**模式**|**算法原理**|**适用场景**|**备注**|
|---|---|---|---|
|**边缘模式** (Edge-based)|基于 **Canny 边缘检测** 提取轮廓后再进行匹配。|**推荐方案**
。适用于绝大多数拼图滑块。|天然免疫拼图周边的透明/黑色留白干扰,坐标最精准。|
|**简单模式** (Simple/Gray)|直接基于 **灰度像素值** 进行归一化互相关计算。|适用于无明显边缘、靠颜色差异识别的场景。|对背景和透明边框敏感,可能存在重心偏移。|
2. 数学公式差异 (NCC vs. CCOEFF)
在简单模式下,本项目采用的是 归一化互相关 (NCC),对应 OpenCV 中的 TM_CCORR_NORMED。
逻辑对齐Rust 的 match_template 结果与 Python cv2.TM_CCORR_NORMED 完全一致。
关于偏移若拼图原始图片Target四周包含大量的透明留白
CCORR (本项目):会将留白视为图像的一部分,计算出的是整张图片框的中心。
CCOEFF (OpenCV 默认):会自动进行“均值中心化”,在一定程度上能削弱留白的影响。
最佳实践:若发现坐标有固定位移,建议优先切换至 边缘模式,或对滑块图进行 Bounding Box 裁剪 后再匹配。
3. 图像预处理一致性
为确保识别精度,本项目在 Rust 中完美复刻了 Python OpenCV 的预处理链路:
- **灰度化权重**:采用 OpenCV 标准感光公式 $0.299R + 0.587G + 0.114B$。
- **Alpha 处理**:在将 PNG 转为 RGB 时,自动将透明区域填充为黑色,确保与 PIL (Python Imaging Library) 行为一致。
- **坐标定义**:所有返回坐标均为匹配区域的 **几何中心点** $(x + w/2, y + h/2)$。
💡 开发者建议:
如果识别结果在 $X$ 轴上有大约 $10px$ 左右的固定误差,通常是因为滑块原图自带了透明边距(留白)。此时请确保
simple_target=false。该模式会通过 Canny 边缘检测 提取轮廓特征,能自动锁定拼图实体并忽略背景留白的像素干扰。
鸣谢 (Credits) 鸣谢 (Credits)
- 本项目是 [ddddocr](https://github.com/sml2h3/ddddocr) 的 Rust 移植版本,原作者为 sml2h3。衷心感谢原作者对 OCR 社区做出的杰出贡献。 - 本项目是 [ddddocr](https://github.com/sml2h3/ddddocr) 的 Rust 移植版本,原作者为 sml2h3。衷心感谢原作者对 OCR 社区做出的杰出贡献。

BIN
code3.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.6 KiB

5
examples/simple_usage.rs Normal file
View File

@@ -0,0 +1,5 @@
fn main() {
// let ocr = ddddocr_rs::DdddOcrBuilder::new().build().unwrap();
// let img = image::open("samples/code3.png").unwrap();
// println!("Result: {}", ocr.classification(&img).unwrap());
}

BIN
samples/det1.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

BIN
samples/det2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 95 KiB

BIN
samples/det3.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

BIN
samples/hua.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 KiB

BIN
samples/huatu.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

BIN
samples/ken.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.3 KiB

BIN
samples/kenyuan.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.0 KiB

3
src/algo/mod.rs Normal file
View File

@@ -0,0 +1,3 @@
mod slide;
pub use slide::{SlideResult, Slider};

268
src/algo/slide.rs Normal file
View File

@@ -0,0 +1,268 @@
use crate::utils::cv_ops;
use crate::utils::cv_ops::{abs_diff, min_max_loc, ndarray_to_luma8, rgb_to_gray};
use crate::utils::image_io::image_to_ndarray;
use anyhow::{Result, anyhow};
use image::DynamicImage;
use image::Luma;
use imageproc::contrast::{ThresholdType, threshold};
use imageproc::distance_transform::Norm;
use imageproc::edges::canny;
use imageproc::morphology::{close, open};
use imageproc::region_labelling::{Connectivity, connected_components};
use imageproc::template_matching::{MatchTemplateMethod, match_template};
use std::fmt;
use tract_onnx::prelude::tract_ndarray::{ArrayView2, ArrayView3};
#[derive(Debug)]
pub struct SlideResult {
pub target: [i32; 2],
pub target_x: i32,
pub target_y: i32,
pub confidence: f64,
}
impl fmt::Display for SlideResult {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
writeln!(f, "滑块匹配测试结果:")?;
writeln!(f, "检测坐标: [x: {}, y: {}]", self.target_x, self.target_y)?;
// 注意:这里保留 4 位小数,如果想让外部控制,也可以直接写 {:.4}
write!(f, "置信度: {:.4}", self.confidence)?;
Ok(())
}
}
pub struct Slider;
impl Slider {
pub fn new() -> Result<Self, anyhow::Error> {
Ok(Self)
}
/// 对应 Python: slide_match 滑块匹配接口
pub fn slide_match(
&self,
target_image: &DynamicImage,
background_image: &DynamicImage,
simple_target: bool,
) -> Result<SlideResult> {
let target_array = image_to_ndarray(target_image);
let background_array = image_to_ndarray(background_image);
self.perform_slide_match(target_array.view(), background_array.view(), simple_target)
}
/// 对应 Python: slide_comparison 差异比较接口
/// 用于比较带坑位的图片与原始背景图,定位差异点
pub fn slide_comparison(
&self,
target_image: &DynamicImage,
background_image: &DynamicImage,
) -> Result<SlideResult> {
// 1. 转换为 ndarray (HWC RGB)
let target_array = image_to_ndarray(target_image);
let background_array = image_to_ndarray(background_image);
// 2. 执行比较逻辑 (对应 _perform_slide_comparison)
self.perform_slide_comparison(target_array.view(), background_array.view())
}
/// 对应 Python: _perform_slide_comparison
pub fn perform_slide_comparison(
&self,
target: ArrayView3<u8>,
background: ArrayView3<u8>,
) -> Result<SlideResult> {
// 1. 计算差异数组 (复用 cv2::absdiff)
let (th, tw, tc) = target.dim();
let (bh, bw, bc) = background.dim();
// 1. 比较模式下的严格尺寸校验
if th != bh || tw != bw || tc != bc {
return Err(anyhow!(
"比较模式要求两张图分辨率与通道数完全一致Target: [{}x{}x{}], Background: [{}x{}x{}]",
tw,
th,
tc,
bw,
bh,
bc
));
}
if th == 0 || tw == 0 {
return Err(anyhow!("输入图像尺寸不能为0"));
}
let diff_array = abs_diff(&target, &background);
// 2. 转换为灰度数组 (复用你的 cv2.cvtColor)
let gray_array = rgb_to_gray(diff_array.view());
// 3. 转为 ImageBuffer 以使用 imageproc 的高级功能
let gray_buffer = ndarray_to_luma8(gray_array.view());
// 2. 二值化 (对应 cv2.threshold(..., 30, 255, cv2.THRESH_BINARY))
let binary = threshold(&gray_buffer, 30, ThresholdType::Binary);
// 3. 形态学操作去噪 (对应 cv2.morphologyEx)
// 闭运算 (Close): 先膨胀后腐蚀,用于填补缺口内的细小黑色空洞
// 开运算 (Open): 先腐蚀后膨胀,用于消除背景中的白色噪点点
let norm = Norm::LInf; // 对应 3x3 的矩形内核
let radius = 1u8; // 1 表示 3x3 的范围2 表示 5x5 的范围
let closed = close(&binary, norm, radius);
let cleaned = open(&closed, norm, radius);
// connected_components 会给每个独立的白色区域打上不同的标签 (ID)
let background_label = Luma([0u8]);
let labelled = connected_components(&cleaned, Connectivity::Eight, background_label);
// // 统计每个标签出现的频率(即面积)
// 4. 寻找最大连通区域 (对应 findContours + max area)
if let Some(max_label) = cv_ops::find_contours_and_max(&labelled) {
// 5. 计算最大区域的边界框 (对应 cv2.boundingRect)
let (x, y, w, h) = cv_ops::bounding_rect(&labelled, max_label);
// 6. 计算中心点 (调用之前封装的 calculate_center)
let (center_x, center_y) = cv_ops::calculate_center((x, y), w as usize, h as usize);
Ok(SlideResult {
target: [center_x, center_y],
target_x: center_x,
target_y: center_y,
confidence: 1.0, // Comparison 模式下通常认为找到即为 1.0
})
} else {
Ok(SlideResult {
target: [0, 0],
target_x: 0,
target_y: 0,
confidence: 0.0,
})
}
}
/// 对应 Python: _perform_slide_match
// 在 SlideEngine 中修改此入口进行测试
fn perform_slide_match(
&self,
target: ArrayView3<u8>,
background: ArrayView3<u8>,
simple_target: bool, // 增加这个参数
) -> Result<SlideResult> {
let (th, tw, tc) = target.dim();
let (bh, bw, bc) = background.dim();
// 1. 严格的鲁棒性校验(防止底层的 imageproc 算子崩溃)
if th == 0 || tw == 0 || bh == 0 || bw == 0 {
return Err(anyhow!("输入图像的宽度或高度不能为0"));
}
if th > bh || tw > bw {
return Err(anyhow!(
"尺寸不匹配:滑块模板(target)尺寸 [{}x{}] 不能大于背景图(background) [{}x{}]",
tw,
th,
bw,
bh
));
}
if tc != bc {
return Err(anyhow!(
"目标图与背景图的通道数不一致 (target: {}, bg: {})",
tc,
bc
));
}
// 1. 统一灰度化
let target_gray = rgb_to_gray(target);
let background_gray = rgb_to_gray(background);
if simple_target {
// 2a. 简单模式:直接在灰度图上匹配
self.simple_template_match(target_gray.view(), background_gray.view())
} else {
// 2b. 复杂模式:先提取边缘,再匹配
self.edge_based_match(target_gray.view(), background_gray.view())
}
}
/// 对应 Python: _simple_template_match
/// 使用 SAD (Sum of Absolute Differences) 算法
/// 核心模板匹配SAD + 有效像素过滤
fn simple_template_match(
&self,
target: ArrayView2<u8>,
background: ArrayView2<u8>,
) -> Result<SlideResult> {
// 1. 将 ndarray 转换为 imageproc 需要的 ImageBuffer (无拷贝或轻量转换)
// 转换逻辑 (假设你已经有方法转回 ImageBuffer)
let t_buf = ndarray_to_luma8(target);
let b_buf = ndarray_to_luma8(background);
// t_buf.save("debug_rust_target.png").unwrap();
// 2. 调用 imageproc 的 NCC 算法 (等价于 cv2.TM_CCOEFF_NORMED)
// 模板匹配 (完全对齐 cv2.matchTemplate(..., cv2.TM_CCOEFF_NORMED))
let result = match_template(
&b_buf,
&t_buf,
MatchTemplateMethod::CrossCorrelationNormalized,
);
// save_rust_result(&result, "debug_rust_target2.png");
// 3. 寻找最大值 (等价于 cv2.minMaxLoc)
let (max_val, max_loc) = min_max_loc(&result);
// 4. 计算中心点 (与 Python 逻辑完全一致)
let (th, tw) = target.dim();
let (center_x, center_y) = cv_ops::calculate_center(max_loc, tw as usize, th as usize);
// println!("Rust Target Width (tw): {}", tw);
// println!("Rust Best Max Loc X: {}", max_loc.0);
// println!("Rust Final Center X: {}", center_x);
Ok(SlideResult {
target: [center_x, center_y],
target_x: center_x,
target_y: center_y,
confidence: max_val as f64,
})
}
/// 对应 Python: _edge_based_match
/// 基于边缘检测的滑块匹配 (对齐 Python _edge_based_match)
pub fn edge_based_match(
&self,
target: ArrayView2<u8>,
background: ArrayView2<u8>,
) -> Result<SlideResult> {
// 1. 将 ndarray 转换为 ImageBuffer
// 注意Canny 和 match_template 需要 ImageBuffer 格式
let t_buf = ndarray_to_luma8(target);
let b_buf = ndarray_to_luma8(background);
// 2. 边缘检测 (完全对齐 cv2.Canny(50, 150))
// 这步会生成黑底白线的二值化边缘图
let target_edges = canny(&t_buf, 50.0, 150.0);
let background_edges = canny(&b_buf, 50.0, 150.0);
// target_edges.save("debug_target_edges.png").ok();
// background_edges.save("debug_bg_edges.png").ok();
// 3. 模板匹配 (完全对齐 cv2.matchTemplate(..., cv2.TM_CCOEFF_NORMED))
// 在边缘图上计算归一化互相关系数
let result = match_template(
&background_edges,
&target_edges,
MatchTemplateMethod::CrossCorrelationNormalized,
);
// 4. 找到最佳匹配位置 (对齐 cv2.minMaxLoc)
let (max_val, max_loc) = min_max_loc(&result);
// 5. 计算中心位置 (对齐 Python 逻辑)
// target_w, target_h 来自输入数组的维度
let (th, tw) = target.dim();
let (center_x, center_y) = cv_ops::calculate_center(max_loc, tw as usize, th as usize);
// 打印调试信息,方便与 Python 对比
// println!("Edge Match: max_val: {}, max_loc: {:?}", max_val, max_loc);
println!("-Rust Target Width (tw): {}", tw);
println!("-Rust Best Max Loc X: {}", max_loc.0);
println!("-Rust Final Center X: {}", center_x);
Ok(SlideResult {
target: [center_x, center_y],
target_x: center_x,
target_y: center_y,
confidence: max_val as f64,
})
}
}

18
src/error.rs Normal file
View File

@@ -0,0 +1,18 @@
pub(crate) const MODEL_DOWNLOAD_HELP: &str = "\
================================================================================
[ddddocr-rust] 错误:未找到默认的模型文件!
--------------------------------------------------------------------------------
由于打包体积限制,本库未内置 ONNX 模型。请按照以下步骤操作:
1. 前往官方 GitHub 下载对应的模型权重:
- OCR 模型: https://github.com/sml2h3/ddddocr/raw/master/ddddocr/common_sml2h3_f32.onnx
- DET 模型: https://github.com/sml2h3/ddddocr/raw/master/ddddocr/common_det.onnx
2. 配置加载方式(二选一):
A. 【推荐】设置环境变量指向您下载的文件:
Linux/macOS: export DDDD_OCR_MODEL=\"/path/to/common_sml2h3_f32.onnx\"
Windows (CMD): set DDDD_OCR_MODEL=C:\\path\\to\\common_sml2h3_f32.onnx
Windows (PowerShell): $env:DDDD_OCR_MODEL=\"C:\\path\\to\\common_sml2h3_f32.onnx\"
B. 或者直接将模型文件重命名并放置在您运行程序的“当前工作目录”或“可执行文件同级目录”下。
================================================================================";

9
src/lib.rs Normal file
View File

@@ -0,0 +1,9 @@
mod algo;
mod error;
pub mod models;
pub mod utils;
pub use crate::algo::{SlideResult, Slider};
pub use crate::models::det::{DetBuilder, DetSession, DetectionResult, Detector};
pub use crate::models::ocr::{Ocr, OcrBuilder, OcrResult, OcrSession};
pub use models::ocr::metadata::ModelMetadata;

View File

@@ -1,64 +0,0 @@
mod charset;
use anyhow::Result;
use charset::CHARSET_BETA;
use image::{imageops::FilterType, open};
use tract_onnx::prelude::*;
// 编译时读取字典文件
fn main() -> Result<()> {
// 1. 加载并优化模型 (假设模型文件在根目录)
let model = onnx()
.model_for_path("model/common.onnx")? // 这里替换成你提取的 ddddocr 模型名
.into_optimized()?
.into_runnable()?;
// 2. 加载并处理图片 (需要缩放到模型要求的尺寸,例如 64x30)
let img = open("samples/code3.png")?;
let h = 64u32;
let w = (img.width() as f32 * (h as f32 / img.height() as f32)) as u32;
// 1. 缩放并转灰度
let resized = img.resize_exact(w, h, FilterType::Lanczos3).to_luma8();
let array =
tract_ndarray::Array4::from_shape_fn((1, 1, h as usize, w as usize), |(_, _, y, x)| {
let pixel = resized.get_pixel(x as u32, y as u32)[0] as f32;
(pixel / 255.0 - 0.5) / 0.5
});
let tensor = Tensor::from(array);
// 4. 运行推理
let result = model.run(tvec!(tensor.into()))?;
// 注意:这里需要根据 ddddocr 的要求将图片转为 Tensor
// 简化逻辑:
// let tensor: Tensor = tract_ndarray::Array4::<f32>::zeros((1, 1, 30, 64)).into();
// 3. 运行推理
// let result = model.run(tvec!(tensor.into()))?;
println!("模型输出原始数据: {:?}", result);
let output = result[0].to_array_view::<i64>()?;
let indices: Vec<i64> = output.iter().cloned().collect();
// 2. 将视图转为切片并调用函数
let code = decode_ctc(&indices);
println!("indices模型输出原始数据: {:?}", indices);
println!("最终识别出的验证码是: {}", code);
Ok(())
}
fn decode_ctc(indices: &[i64]) -> String {
let mut res = String::new();
let mut last_idx: i64 = -1;
for &idx in indices {
// idx == 0 通常是 CTC 的 blank 占位符
if idx != 0 && idx != last_idx {
if let Some(&char_str) = CHARSET_BETA.get(idx as usize) {
res.push_str(char_str);
}
}
last_idx = idx;
}
res
}

25
src/models/det/builder.rs Normal file
View File

@@ -0,0 +1,25 @@
use crate::models::det::executor::Detector;
use crate::models::det::session::DetSession;
pub struct DetBuilder {
use_gpu: bool,
device_id: u8,
}
impl DetBuilder {
fn use_gpu(mut self) -> Self {
self.use_gpu = true;
self
}
fn device_id(mut self, device_id: u8) -> Self {
self.device_id = device_id;
self
}
fn build(self, session: &DetSession) -> Detector<'_> {
Detector {
session,
use_gpu: self.use_gpu,
device_id: self.device_id,
}
}
}

300
src/models/det/executor.rs Normal file
View File

@@ -0,0 +1,300 @@
use anyhow::{Context, Result};
use image::{imageops::FilterType, DynamicImage, GenericImageView};
use std::fmt;
use tract_onnx::prelude::tract_ndarray::{prelude::*, s, Array2, Array3, Array4, Axis};
use tract_onnx::prelude::{Tensor};
use crate::models::det::session::DetSession;
#[derive(Debug, Clone, Copy)]
pub struct DetectionResult {
pub x1: i32,
pub y1: i32,
pub x2: i32,
pub y2: i32,
pub score: f32,
pub class_id: u32,
}
impl fmt::Display for DetectionResult {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
// 结构体只管自己这一行怎么显示,不用管外部的索引 [i]
write!(
f,
"x1={}, y1={}, x2={}, y2={}, 分数={:.4}, 类别ID={}",
self.x1, self.y1, self.x2, self.y2, self.score, self.class_id
)
}
}
#[derive(Debug)]
pub struct Detector<'a> {
pub(crate) session: &'a DetSession,
#[allow(dead_code)]
pub(crate) use_gpu: bool,
#[allow(dead_code)]
pub(crate) device_id: u8,
}
impl<'a> Detector<'a> {
pub fn new(session: &'a DetSession) -> Self {
Detector {
session,
use_gpu: false,
device_id: 0,
}
}
pub fn predict(&self, image: &DynamicImage) -> Result<Vec<DetectionResult>> {
// Rust 中通常在调用层处理文件/PIL转换这里直接进入核心逻辑
self.get_bbox(image)
}
/// 2. preproc: 纯 Rust 实现 (替代 OpenCV)
fn preproc(&self, image: &DynamicImage, input_size: (u32, u32)) -> Result<(Tensor, f32)> {
let (target_h, target_w) = input_size;
let (img_w, img_h) = image.dimensions();
// 计算缩放比例 (Letterbox)
let r = (target_h as f32 / img_h as f32).min(target_w as f32 / img_w as f32);
let new_h = (img_h as f32 * r) as u32;
let new_w = (img_w as f32 * r) as u32;
// Resize 图像
let resized = image.resize_exact(new_w, new_h, FilterType::Triangle);
// 2. 关键:将 DynamicImage 显式转换为 RgbImage (Rgb<u8>)
let resized_rgb = resized.to_rgb8();
// 创建 114 灰度填充的背景
let mut base_img =
image::ImageBuffer::from_pixel(target_w, target_h, image::Rgb([114u8, 114, 114]));
// 将 resize 后的图像覆盖到左上角 (类似于原始代码中的 padded_img[:h, :w])
image::imageops::overlay(&mut base_img, &resized_rgb, 0, 0);
// 优化:直接获取底层的扁平 raw buffer比 enumerate_pixels() 快得多
let raw_samples = base_img.as_flat_samples();
let slice = raw_samples.as_slice();
// 构造 NCHW Tensor
let mut array = Array4::<f32>::zeros((1, 3, target_h as usize, target_w as usize));
// 用连续的 stride 步长进行写入,提高 CPU 缓存利用率
for y in 0..target_h as usize {
for x in 0..target_w as usize {
let idx = (y * target_w as usize + x) * 3;
// BGR 赋值
array[[0, 0, y, x]] = slice[idx + 2] as f32; // B
array[[0, 1, y, x]] = slice[idx + 1] as f32; // G
array[[0, 2, y, x]] = slice[idx] as f32; // R
}
}
Ok((array.into(), r))
}
/// 3. demo_postprocess (逻辑与 Python 一致)
fn demo_postprocess(&self, mut outputs: Array3<f32>, img_size: (i32, i32)) -> Array3<f32> {
let strides = [8, 16, 32];
// 遍历每一个 Batch支持动态 Batch 推理)
for mut batch in outputs.axis_iter_mut(Axis(0)) {
let mut offset = 0;
for &stride in &strides {
// 计算当前特征图的尺寸
let h = img_size.0 / stride;
let w = img_size.1 / stride;
let f_stride = stride as f32;
for y in 0..h {
for x in 0..w {
// 计算当前格子在 25200 个锚点中的线性索引
let idx = offset + (y * w + x) as usize;
// 1. 还原中心点坐标 (cx, cy)
// 公式: (output + grid_offset) * stride
batch[[idx, 0]] = (batch[[idx, 0]] + x as f32) * f_stride;
batch[[idx, 1]] = (batch[[idx, 1]] + y as f32) * f_stride;
// 2. 还原宽高 (w, h)
// 公式: exp(output) * stride
batch[[idx, 2]] = batch[[idx, 2]].exp() * f_stride;
batch[[idx, 3]] = batch[[idx, 3]].exp() * f_stride;
}
}
// 移动到下一个步长的起始位置
offset += (h * w) as usize;
}
}
outputs
}
/// 4. nms
fn nms(&self, boxes: &Array2<f32>, scores: &Array1<f32>, nms_thr: f32) -> Vec<usize> {
let mut keep = Vec::new();
let x1 = boxes.column(0);
let y1 = boxes.column(1);
let x2 = boxes.column(2);
let y2 = boxes.column(3);
// 在每一项前加上 &,并确保括号内的计算顺序
// 注意ndarray 的 View 运算需要 &view1 - &view2
let areas = (&x2 - &x1 + 1.0) * (&y2 - &y1 + 1.0);
// 初始排序索引
let mut v: Vec<usize> = (0..scores.len()).collect();
v.sort_unstable_by(|&i, &j| {
scores[j]
.partial_cmp(&scores[i])
.unwrap_or(std::cmp::Ordering::Equal)
});
// 我们不使用 v.remove(0),而是直接通过索引池操作
let mut active_indices = v;
while !active_indices.is_empty() {
// 取出当前池子中得分最高的框(即第一个元素)
let i = active_indices[0];
keep.push(i);
// 如果池子里只剩一个了,直接结束
if active_indices.len() == 1 {
break;
}
// 5. 核心逻辑:使用 retain 一次性过滤掉:
// (a) 当前框自己 (idx == i)
// (b) 与当前框重叠度过高的框 (iou > nms_thr)
active_indices.retain(|&idx| {
// 如果是当前正在处理的框,不保留(因为它已经进入 keep 了)
if idx == i {
return false;
}
// 计算 IoU
let xx1 = x1[i].max(x1[idx]);
let yy1 = y1[i].max(y1[idx]);
let xx2 = x2[i].min(x2[idx]);
let yy2 = y2[i].min(y2[idx]);
let w = (xx2 - xx1 + 1.0).max(0.0);
let h = (yy2 - yy1 + 1.0).max(0.0);
let inter = w * h;
let iou = inter / (areas[i] + areas[idx] - inter);
// 只保留 IoU 小于阈值的框
iou <= nms_thr
});
}
keep
}
/// 5. multiclass_nms
//multiclass_nms_class_agnostic
pub fn multiclass_nms(
&self,
boxes: &Array2<f32>, // [25200, 4] -> xyxy 格式
scores: &Array2<f32>, // [25200, 80] -> 已经乘以 objectness 的得分
nms_thr: f32,
score_thr: f32,
) -> Vec<[f32; 6]> {
let mut candidates = Vec::new();
// 1. 筛选高分框 (单次遍历完成 Argmax 和 Threshold 过滤)
for i in 0..scores.nrows() {
let row = scores.row(i);
// 找到当前行(即当前锚点)得分最高的类别
let mut max_score = 0.0;
let mut cls_id = 0;
for (j, &s) in row.iter().enumerate() {
if s > max_score {
max_score = s;
cls_id = j;
}
}
// 仅保留超过阈值的候选框
if max_score > score_thr {
// 暂时存储索引和元数据,避免频繁创建大数组
candidates.push((i, max_score, cls_id));
}
}
if candidates.is_empty() {
return vec![];
}
// 2. 准备 NMS 输入
// 构造 NMS 需要的子集数组
let mut b_subset = Array2::<f32>::zeros((candidates.len(), 4));
let mut s_subset = Array1::<f32>::zeros(candidates.len());
for (new_idx, &(orig_idx, score, _)) in candidates.iter().enumerate() {
b_subset.row_mut(new_idx).assign(&boxes.row(orig_idx));
s_subset[new_idx] = score;
}
// 3. 执行 NMS (返回保留下来的子集索引)
let keep = self.nms(&b_subset, &s_subset, nms_thr);
// 4. 组装最终结果 [x1, y1, x2, y2, score, class_id]
keep.into_iter()
.map(|k_idx| {
let (orig_idx, score, cls_id) = candidates[k_idx];
let b = boxes.row(orig_idx);
[b[0], b[1], b[2], b[3], score, cls_id as f32]
})
.collect()
}
/// 6. get_bbox (完全解耦 OpenCV)
pub fn get_bbox(&self, dynamic_img: &DynamicImage) -> Result<Vec<DetectionResult>> {
// 使用 utils crate 解码
// let dynamic_img = image::load_from_memory(image_bytes).context("Failed to decode utils")?;
let (orig_w, orig_h) = dynamic_img.dimensions();
let (input_tensor, ratio) = self.preproc(dynamic_img, (416, 416))?;
// tract 推理
// let outputs = self.session.session.run(tvec!(input_tensor.into()))?;
let outputs = self.session.inference(input_tensor)?;
// let output_array = outputs[0]
let output_array = outputs
.to_array_view::<f32>()?
.to_owned()
.into_dimensionality::<Ix3>()?;
let predictions = self.demo_postprocess(output_array, (416, 416));
let pred = predictions.slice(s![0, .., ..]);
let boxes = pred.slice(s![.., 0..4]);
let obj_conf = pred.slice(s![.., 4..5]);
let cls_conf = pred.slice(s![.., 5..]);
let obj_broadcast = obj_conf
.broadcast(cls_conf.dim())
.context("ndarray broadcasting failed for scores calculation")?;
let scores = &obj_broadcast * &cls_conf;
// let scores = &pred.slice(s![.., 4..5]) * &pred.slice(s![.., 5..]);
let mut boxes_xyxy = Array2::<f32>::zeros(boxes.raw_dim());
for i in 0..boxes.nrows() {
boxes_xyxy[[i, 0]] = (boxes[[i, 0]] - boxes[[i, 2]] / 2.0) / ratio;
boxes_xyxy[[i, 1]] = (boxes[[i, 1]] - boxes[[i, 3]] / 2.0) / ratio;
boxes_xyxy[[i, 2]] = (boxes[[i, 0]] + boxes[[i, 2]] / 2.0) / ratio;
boxes_xyxy[[i, 3]] = (boxes[[i, 1]] + boxes[[i, 3]] / 2.0) / ratio;
}
let detections = self.multiclass_nms(&boxes_xyxy, &scores, 0.45, 0.1);
let final_results = detections
.into_iter()
.map(|d| DetectionResult {
x1: (d[0] as i32).max(0).min(orig_w as i32),
y1: (d[1] as i32).max(0).min(orig_h as i32),
x2: (d[2] as i32).max(0).min(orig_w as i32),
y2: (d[3] as i32).max(0).min(orig_h as i32),
score: d[4],
class_id: d[5] as u32,
})
.collect();
Ok(final_results)
}
}

7
src/models/det/mod.rs Normal file
View File

@@ -0,0 +1,7 @@
mod builder;
mod executor;
mod session;
pub use builder::DetBuilder;
pub use executor::{DetectionResult, Detector};
pub use session::DetSession;

43
src/models/det/session.rs Normal file
View File

@@ -0,0 +1,43 @@
use crate::models::loader::{ModelLoader, ModelSession, ModelType};
use anyhow::{Context, Result};
use std::path::Path;
use tract_onnx::prelude::{tvec, Graph, IntoTensor, RunnableModel, Tensor, TypedFact, TypedOp};
#[derive(Debug)]
pub struct DetSession {
pub(crate) session: RunnableModel<TypedFact, Box<dyn TypedOp>, Graph<TypedFact, Box<dyn TypedOp>>>,
}
impl ModelSession for DetSession {
fn get_model_type(&self) -> ModelType {
todo!()
}
fn desc(&self) -> String {
"Detection Model 加载成功".to_string()
}
}
impl DetSession {
pub fn new<P>(model_path: P) -> Result<Self, anyhow::Error>
where
P: AsRef<Path>,
{
let session = ModelLoader::model_for_path(&model_path)?.session;
Ok(Self { session })
}
pub fn model_from_bytes(model_bytes: &[u8]) -> Result<Self, anyhow::Error> {
let session = ModelLoader::model_from_bytes(model_bytes)?.session;
Ok(Self { session })
}
pub fn inference(&self, tensor: Tensor) -> anyhow::Result<Tensor> {
// tract 的 run 会返回一个 Vec<TValue>,我们通常只需要第一个输出
// let result = self.ocr.run(tvec!(tensor.into()))?;
let mut result = self
.session
.run(tvec!(tensor.into()))
.context("执行模型推理失败")?;
println!("模型输出原始数据: {:?}", result);
Ok(result.swap_remove(0).into_tensor())
}
}

48
src/models/loader.rs Normal file
View File

@@ -0,0 +1,48 @@
use anyhow::Context;
use std::io::Cursor;
use tract_onnx::onnx;
use tract_onnx::prelude::*;
/// OCR 模型:包含路径和字符集
pub enum ModelType {
Ocr,
Det,
Custom,
}
// 定义统一的 trait
pub trait ModelSession {
fn get_model_type(&self) -> ModelType;
fn desc(&self) -> String;
}
pub struct ModelLoader {
pub session: RunnableModel<TypedFact, Box<dyn TypedOp>, Graph<TypedFact, Box<dyn TypedOp>>>,
}
impl ModelLoader {
pub fn model_for_path<P>(model_path: P) -> anyhow::Result<Self>
where
P: AsRef<std::path::Path>,
{
let session = onnx()
.model_for_path(model_path)
.with_context(|| "加载 ONNX 模型失败,请检查路径是否正确")?
.into_optimized()?
.into_runnable()?;
Ok(Self { session })
}
/// 策略 B从内存字节流加载模型配合 include_bytes! 使用)
pub fn model_from_bytes(model_bytes: &[u8]) -> anyhow::Result<Self> {
// 使用 std::io::Cursor 将 &[u8] 包装为可读的流(实现 std::io::Read
let mut cursor = Cursor::new(model_bytes);
let session = onnx()
.model_for_read(&mut cursor)
.with_context(|| "从内存字节流解析 ONNX 模型失败")?
.into_optimized()?
.into_runnable()?;
Ok(Self { session })
}
}

3
src/models/mod.rs Normal file
View File

@@ -0,0 +1,3 @@
pub mod loader;
pub mod ocr;
pub mod det;

73
src/models/ocr/builder.rs Normal file
View File

@@ -0,0 +1,73 @@
use crate::models::ocr::executor::Ocr;
use crate::models::ocr::session::OcrSession;
use crate::models::ocr::color_filter::ColorFilter;
use crate::models::ocr::token_filter::TokenFilter;
pub struct OcrBuilder {
/// 是否修复PNG格式问题
png_fix: bool,
/// 是否返回概率信息
probability: bool,
/// 颜色过滤:保留的颜色列表
color_filter: Option<Box<dyn ColorFilter + Send + Sync>>,
/// 字符集范围
charset_restrict: Option<Box<dyn TokenFilter + Send + Sync>>,
}
impl OcrBuilder {
// 初始化任务,设置默认参数
pub fn new() -> Self {
Self {
png_fix: false, // 默认值
probability: false,
color_filter: None,
charset_restrict: None,
}
}
pub fn png_fix(mut self, value: bool) -> Self {
self.png_fix = value;
self
}
pub fn probability(mut self, value: bool) -> Self {
self.probability = value;
self
}
pub fn color_filter<T>(mut self, filter: T) -> Self
where
T: ColorFilter + Send + Sync + 'static,
{
self.color_filter = Some(Box::new(filter));
self
}
pub fn charset_restrict<T>(mut self, restrict: T) -> Self
where
T: TokenFilter + Send + Sync + 'static,
{
self.charset_restrict = Some(Box::new(restrict));
self
}
pub fn build(self, session: &OcrSession) -> Ocr<'_> {
// 1. 原地解析颜色过滤器
let final_color_ranges = match &self.color_filter {
Some(filter) => filter.collect_to_vec(),
None => Ok(None),
};
// 2. 原地解析字符集过滤
let tokens = &session.model_metadata.charset.tokens;
let final_charset_indices = match &self.charset_restrict {
Some(restrict) => restrict.apply_to_charset(tokens),
None => None,
};
// Ocr::new(session, self)
Ocr {
session,
png_fix: self.png_fix, // 原地解构出来
probability: self.probability,
final_color_ranges,
final_charset_indices,
}
}
}

View File

@@ -0,0 +1,287 @@
use crate::utils::cv_ops::rgb_to_opencv_hsv;
use anyhow::anyhow;
use image::{DynamicImage, ImageBuffer, Rgb};
use std::str::FromStr;
/// 核心区间判定辅助函数
#[inline(always)]
fn is_pixel_matched(ranges: &[HsvRange], h: u8, s: u8, v: u8) -> bool {
ranges.iter().any(|range| {
h >= range.lower.0
&& h <= range.upper.0
&& s >= range.lower.1
&& s <= range.upper.1
&& v >= range.lower.2
&& v <= range.upper.2
})
}
pub fn apply_to_image(
image: &DynamicImage,
hsv_ranges: &[HsvRange],
) -> anyhow::Result<DynamicImage> {
// 1. 统一转换为连续内存的 RGB8 缓冲区 (对应 Python 的 Image 到 RGB/BGR 数组转换)
let rgb_img = image.to_rgb8();
let (width, height) = rgb_img.dimensions();
let mut raw_pixels = rgb_img.into_raw();
// 2. 密集计算核心:原地流式迭代修改
// 每次取出 3 个 u8 字节,分别代表 [R, G, B],无多余掩膜矩阵内存分配
for chunk in raw_pixels.chunks_exact_mut(3) {
let r = chunk[0];
let g = chunk[1];
let b = chunk[2];
// 像素级转换为 OpenCV 标准的 HSV
let (h, s, v) = rgb_to_opencv_hsv(r, g, b);
// 模拟 Python 的多范围 mask bitwise_or 并在 mask == 0 处刷白
// 如果该像素没有命中任何一个配置的颜色区间,立刻原地刷白 [255, 255, 255]
if !is_pixel_matched(hsv_ranges, h, s, v) {
chunk[0] = 255;
chunk[1] = 255;
chunk[2] = 255;
}
}
// 3. 将扁平字节数组重新打包回 DynamicImage 容器
let filtered_buffer = ImageBuffer::<Rgb<u8>, Vec<u8>>::from_raw(width, height, raw_pixels)
.ok_or_else(|| anyhow!("图像缓冲重新组装失败,维度与数据大小不匹配"))?;
Ok(DynamicImage::ImageRgb8(filtered_buffer))
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub struct HsvRange {
pub lower: (u8, u8, u8), // (H, S, V)
pub upper: (u8, u8, u8), // (H, S, V)
}
impl HsvRange {
pub const fn new(lower: (u8, u8, u8), upper: (u8, u8, u8)) -> Self {
Self { lower, upper }
}
}
impl HsvRange {
/// 验证当前 HSV 范围是否合法
/// 对应 Python 逻辑H 在 0-180S/V 在 0-255且下界 <= 上界
pub fn validate(&self) -> Result<(), String> {
// 1. 校验 H 通道边界 (OpenCV 中 H 范围是 0-180)
if self.lower.0 > 180 || self.upper.0 > 180 {
return Err("H通道值必须在 0-180 范围内".to_string());
}
// 2. 校验下界不能大于上界
if self.lower.0 > self.upper.0 || self.lower.1 > self.upper.1 || self.lower.2 > self.upper.2
{
return Err("HSV范围下界不能大于上界".to_string());
}
Ok(())
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ColorPreset {
Red,
Blue,
Green,
Yellow,
Orange,
Purple,
Cyan,
Black,
White,
Gray,
Custom(Vec<HsvRange>),
}
impl ColorPreset {
/// 纯裸数据定义,没有任何结构体包装,干净利落
/// 返回值:(范围数量, 范围数组)
/// 完美的零成本抽象:利用常量提升将数据直接打入只读数据段 (.rodata)
pub fn matches(&self) -> &[HsvRange] {
match self {
ColorPreset::Red => &[
HsvRange {
lower: (0, 50, 50),
upper: (10, 255, 255),
},
HsvRange {
lower: (170, 50, 50),
upper: (180, 255, 255),
},
],
ColorPreset::Blue => &[HsvRange {
lower: (100, 50, 50),
upper: (130, 255, 255),
}],
ColorPreset::Green => &[HsvRange {
lower: (40, 50, 50),
upper: (80, 255, 255),
}],
ColorPreset::Yellow => &[HsvRange {
lower: (20, 50, 50),
upper: (40, 255, 255),
}],
ColorPreset::Orange => &[HsvRange {
lower: (10, 50, 50),
upper: (20, 255, 255),
}],
ColorPreset::Purple => &[HsvRange {
lower: (130, 50, 50),
upper: (170, 255, 255),
}],
ColorPreset::Cyan => &[HsvRange {
lower: (80, 50, 50),
upper: (100, 255, 255),
}],
ColorPreset::Black => &[HsvRange {
lower: (0, 0, 0),
upper: (180, 255, 50),
}],
ColorPreset::White => &[HsvRange {
lower: (0, 0, 200),
upper: (180, 30, 255),
}],
ColorPreset::Gray => &[HsvRange {
lower: (0, 0, 50),
upper: (180, 30, 200),
}],
ColorPreset::Custom(ranges) => ranges,
}
}
/// 校验逻辑:在这里实现完美的“责任分离”
pub fn validate(&self) -> Result<(), String> {
match self {
// 1. 快捷变体完全绕过根本不校验0 运行时开销放行!
ColorPreset::Custom(ranges) => {
// 2. 只有 Custom 变体需要接受严格的参数政审
for r in ranges {
r.validate()?;
}
Ok(())
}
_ => Ok(()),
}
}
}
impl FromStr for ColorPreset {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.to_lowercase().as_str() {
"red" => Ok(ColorPreset::Red),
"blue" => Ok(ColorPreset::Blue),
"green" => Ok(ColorPreset::Green),
"yellow" => Ok(ColorPreset::Yellow),
"orange" => Ok(ColorPreset::Orange),
"purple" => Ok(ColorPreset::Purple),
"cyan" => Ok(ColorPreset::Cyan),
"black" => Ok(ColorPreset::Black),
"white" => Ok(ColorPreset::White),
"gray" => Ok(ColorPreset::Gray),
_ => Err(format!("不支持的颜色预设: {}", s)),
}
}
}
// =====================================================================
// 3. 颜色约束特征Trait与组合子设计模式
// =====================================================================
pub struct PixelCtx {
pub hsv: (u8, u8, u8),
}
/// 统一的颜色约束接口
pub trait ColorFilter {
/// 将自身的有效约束平铺追加到统一的目标容器中
fn append_ranges(&self, target: &mut Vec<HsvRange>);
/// 预估范围数量,借助原生内置的 len() 实现 O(1) 完美控容
fn estimated_count(&self) -> usize;
/// 将自身的有效约束平铺追加到统一目标容器中
/// 验证当前过滤器是否合法默认直接放行Ok(())
fn validate_self(&self) -> Result<(), String> {
Ok(())
}
/// 【新扩展的架构方法】将自身安全的合并到已有的普通容器中,并完成去重和排序
/// 完美的责任分离Builder 不再需要关心怎么分配内存、怎么排序去重
fn collect_to_vec(&self) -> Result<Option<Vec<HsvRange>>, String> {
// 1. 触发自检
self.validate_self()?;
let total_capacity = self.estimated_count();
if total_capacity == 0 {
return Ok(None);
}
// 2. 永远一击必中分配精准内存,不需要再考虑追加和扩容!
let mut v = Vec::with_capacity(total_capacity.max(16));
// 2. 倒入数据
self.append_ranges(&mut v);
// 3. 原地完成排序与去重
v.sort_unstable();
v.dedup();
Ok(Some(v))
}
}
impl ColorFilter for ColorPreset {
fn append_ranges(&self, target: &mut Vec<HsvRange>) {
// 直接利用我们第一步写好的 matches() 拿到切片,整块高速拷贝倒入目标容器
target.extend_from_slice(self.matches());
}
fn estimated_count(&self) -> usize {
// 直接获取切片长度
self.matches().len()
}
fn validate_self(&self) -> Result<(), String> {
// 直接调用我们在第一步中为 ColorPreset 实现的精细化分流校验
// 快捷变体在这里会直接返回 Ok(()), 只有 Custom 才会去真正校验
self.validate()
}
}
/// 多路颜色“或”逻辑组合子(并集网络)
pub struct MultiOrColorRestrict<'a> {
pub filters: Vec<&'a dyn ColorFilter>,
}
impl<'a> ColorFilter for MultiOrColorRestrict<'a> {
fn append_ranges(&self, target: &mut Vec<HsvRange>) {
// 管道递延:依次指挥内部每一个子过滤器把数据倒进目标容器
for f in &self.filters {
f.append_ranges(target);
}
}
fn estimated_count(&self) -> usize {
// 数量累加:$O(1)$ 地把所有子过滤器的预估容量加起来
self.filters.iter().map(|f| f.estimated_count()).sum()
}
fn validate_self(&self) -> Result<(), String> {
// 递归政审:只要其中一个子过滤器校验失败(比如某个 Custom 变体非法),立刻熔断
for f in &self.filters {
f.validate_self()?;
}
Ok(())
}
}
// =====================================================================
// 4. 声明式宏:一语定乾坤
// =====================================================================
#[macro_export]
macro_rules! color_any_of {
($only:expr) => {
&$only as &dyn $crate::ColorFilter
};
($($filter:expr),+ $(,)?) => {
&$crate::MultiOrColorRestrict {
filters: vec![ $( &$filter as &dyn $crate::ColorFilter ),+ ]
}
};
}

478
src/models/ocr/executor.rs Normal file
View File

@@ -0,0 +1,478 @@
use crate::models::ocr::metadata::Resize;
use crate::models::ocr::session::OcrSession;
use crate::models::ocr::color_filter::{HsvRange, apply_to_image};
use crate::utils::image_io::png_rgba_white_preprocess;
use crate::utils::image_processor::{convert_to_grayscale, resize_image};
use anyhow::Result;
use image::DynamicImage;
use serde::Serialize;
use std::borrow::Cow;
use std::fmt;
use tract_onnx::prelude::tract_ndarray::{ArrayView2, Ix2, s};
use tract_onnx::prelude::{DatumType, Tensor, tract_ndarray};
#[derive(Debug, Clone, Serialize)]
pub enum OcrResult {
/// 纯文本分支(对应 probability = false
Text(String),
/// 包含全量概率的分支(对应 probability = true
Probability {
text: String,
/// 满额概率矩阵 [Steps, Classes]
probabilities: Vec<Vec<f32>>,
/// 全局平均置信度
confidence: f64,
},
/// 不支持的模型或未知输出
Unsupported { message: String },
}
impl OcrResult {
/// 消费自身,直接提取最终文本
pub fn into_text(self) -> String {
match self {
OcrResult::Text(text) => text,
OcrResult::Probability { text, .. } => text,
OcrResult::Unsupported { message } => {
// 作为库,这里可以返回空,或者直接携带错误信息,取决于你的设计
format!("Error: {}", message)
}
}
}
}
impl fmt::Display for OcrResult {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
OcrResult::Text(text) => {
// 纯文本分支,直接输出文本内容
write!(f, "{}", text)
}
OcrResult::Probability {
text,
probabilities,
confidence,
} => {
// 概率分支,友好地展示文本以及百分比形式的置信度
// 1. 基本信息
write!(f, "{} (置信度: {:.2}%)", text, confidence * 100.0)?;
// 2. 概率矩阵流式安全打印
write!(f, " [概率矩阵预览: ")?;
let max_steps_to_show = 10;
let take_steps = probabilities.iter().take(max_steps_to_show);
for (i, step_probs) in take_steps.enumerate() {
if i > 0 {
write!(f, ", ")?;
}
// 为了防止单行内部数据过长,单行也做一下截断保护(比如每行最多显示前 3 个概率)
let max_classes_to_show = 3;
write!(f, "[")?;
for (j, prob) in step_probs.iter().take(max_classes_to_show).enumerate() {
if j > 0 {
write!(f, ", ")?;
}
write!(f, "{:.4}", prob)?;
}
if step_probs.len() > max_classes_to_show {
write!(f, ", ..")?;
}
write!(f, "]")?;
}
// 如果总 Step 数量超过 10末尾追加 .. 表示截断
if probabilities.len() > max_steps_to_show {
write!(f, ", ..")?;
}
write!(f, "]")
}
OcrResult::Unsupported { message } => {
// 错误分支,直观输出异常原因
write!(f, "未识别成功: {}", message)
}
}
}
}
pub struct Ocr<'a> {
pub(crate) session: &'a OcrSession,
pub(crate) png_fix: bool,
pub(crate) probability: bool,
/// 颜色过滤:保留的颜色列表
pub(crate) final_color_ranges: Result<Option<Vec<HsvRange>>, String>,
/// 字符集范围
pub(crate) final_charset_indices: Option<Vec<usize>>,
}
impl<'a> Ocr<'a> {
// 初始化任务,设置默认参数
pub fn new(session: &'a OcrSession) -> Self {
Ocr {
session,
png_fix: false, // 默认值
probability: false,
final_color_ranges: Ok(None),
final_charset_indices: None,
}
}
}
impl<'a> Ocr<'a> {
pub fn predict(&self, image: &DynamicImage) -> anyhow::Result<OcrResult> {
println!("当前颜色过滤器状态: {:?}", self.final_color_ranges);
// =====================================================================
// 管道节点 1: 颜色过滤流水线
// 使用 Cow (Copy-On-Write) 智能指针。
// 如果未开启过滤img_cow 内部只是持有原图的【只读借用】,发生【零内存分配】!
// =====================================================================
let img_cow = match &self.final_color_ranges {
Err(err_msg) => {
return Err(anyhow::anyhow!(
"颜色过滤器初始化失败,全链路短路: {}",
err_msg
));
}
Ok(None) => {
// 核心优化点:直接借用原图,不发生任何克隆
Cow::Borrowed(image)
}
Ok(Some(ranges)) => {
// 只有真正需要过滤时,才在内部提取像素并生成清洗后的 Owned 新图
let filtered_img = apply_to_image(image, ranges)?;
Cow::Owned(filtered_img)
}
};
let tensor = self.preprocess_image(&img_cow)?;
let raw_tensor = self.session.inference(tensor)?;
// 3. 后处理分流:直接返回 OcrResult
let ocr_output = match raw_tensor.datum_type() {
DatumType::I64 => self.process_i64_tensor(raw_tensor)?,
DatumType::F32 => self.process_f32_tensor(raw_tensor)?,
_ => OcrResult::Unsupported {
message: format!("不支持的模型输出数据类型: {:?}", raw_tensor.datum_type()),
},
};
// let raw_indices = self.ocr.extract_indices_from_tensor(&raw_tensor)?;
// // 步骤 2: 将索引切片 `&[i64]` 传给解码器进行 CTC 去重和字符映射
// let final_text = self.ctc_decode_to_string(&raw_indices);
Ok(ocr_output)
}
/// 对应 Python 的 _preprocess_image
/// 负责:透明背景修复 -> 灰度化 -> 按比例 Resize -> 归一化 -> 4维张量转换
fn preprocess_image(&self, img: &DynamicImage) -> anyhow::Result<Tensor> {
// 1. 获取模型元数据配置
let meta = &self.session.model_metadata;
let norm = &meta.normalization; // 获取归一化器
// A. 修复 PNG 透明背景 (内部逻辑你之前已实现)
let current_img = if self.png_fix && img.color().has_alpha() {
// 只有满足条件才去触发分配,生成新图
Cow::Owned(png_rgba_white_preprocess(img))
} else {
// 正常情况下,仅仅是再次安全借用,无开销
Cow::Borrowed(img)
};
// 3. 管道节点 2: 根据 Resize 策略计算目标宽高并进行缩放
let (target_w, target_h) = match meta.resize {
Resize::Fixed(w, h) => (w, h),
Resize::DynamicWidth(h) => {
// 高度固定宽度根据原始比例动态计算W_target = W_orig * (H_target / H_orig)
let w =
(current_img.width() as f32 * (h as f32 / current_img.height() as f32)) as u32;
(w, h)
}
Resize::Square(size) => {
// 单字识别模型,直接缩放为正方形
(size, size)
}
};
// 执行缩放
let resized_img = resize_image(&current_img, target_w, target_h);
// 4. 管道节点 3: 颜色通道转换(单通道灰度 vs 三通道 RGB与 4D 张量填充
let tensor = match meta.channel {
// --- 情况 A: 单通道(灰度图),对应 Python 的 len(shape) == 2 展开 ---
1 => {
let gray_img = convert_to_grayscale(&resized_img);
let array = tract_ndarray::Array4::from_shape_fn(
(1, 1, target_h as usize, target_w as usize),
|(_, _, y, x)| {
let pixel = gray_img.get_pixel(x as u32, y as u32)[0] as f32;
// pixel / 255.0 // 严格对齐 Python 归一化 [0.0, 1.0]
// (pixel / 255.0 - 0.5) / 0.5
norm.normalize(pixel)
},
);
Tensor::from(array)
}
// --- 情况 B: 三通道RGB对应 Python 的 transpose(2, 0, 1) 的 CHW 布局 ---
3 => {
let rgb_img = resized_img.to_rgb8();
let array = tract_ndarray::Array4::from_shape_fn(
(1, 3, target_h as usize, target_w as usize),
|(_, c, y, x)| {
let pixel = rgb_img.get_pixel(x as u32, y as u32)[c] as f32;
// pixel / 255.0 // 严格对齐 Python 归一化 [0.0, 1.0]
// (pixel / 255.0 - 0.5) / 0.5
norm.normalize(pixel)
},
);
Tensor::from(array)
}
_ => return Err(anyhow::anyhow!("不支持的通道数配置: {}", meta.channel)),
};
Ok(tensor)
// let h = 64u32;
// let w = (current_img.width() as f32 * (h as f32 / current_img.height() as f32)) as u32;
// let gray_img = convert_to_grayscale(&current_img);
// let resized = resize_image(&gray_img, w, h);
// // resized.save("debug_preprocessed.png").unwrap();
// // 1. 预处理:转灰度 -> Resize -> 归一化
// // let resized = img.resize_exact(w, h, FilterType::Lanczos3).to_luma8();
//
// // 使用 tract_ndarray 构造,避免版本冲突
// let array =
// tract_ndarray::Array4::from_shape_fn((1, 1, h as usize, w as usize), |(_, _, y, x)| {
// let pixel = resized.get_pixel(x as u32, y as u32)[0] as f32;
// (pixel / 255.0 - 0.5) / 0.5
// });
//
// let tensor = Tensor::from(array);
//
// Ok(tensor)
}
}
impl<'a> Ocr<'a> {
fn is_valid_indices(&self, idx: usize) -> bool {
if idx >= self.session.model_metadata.charset.size() {
return false;
}
match &self.final_charset_indices {
Some(v) => v.binary_search(&idx).is_ok(),
None => true,
}
}
/// 【按需延迟打印】:当用户真的需要“知道当前有哪些限制字符”时,一秒反查并打印
/// 这里的 &str 完美借用了自 tokens依然是彻底的零拷贝
pub fn valid_tokens(&self) -> Vec<&str> {
let charset = &self.session.model_metadata.charset;
let tokens = &charset.tokens;
match &self.final_charset_indices {
Some(indices) => indices
.iter()
.filter_map(|&idx| tokens.get(idx).map(|cow| cow.as_ref()))
.collect(),
// 如果是 None现场映射出全量 Token 视图给外部
None => tokens.iter().map(|cow| cow.as_ref()).collect(),
}
}
pub fn valid_size(&self) -> usize {
match &self.final_charset_indices {
Some(indices) => indices.len(),
None => self.session.model_metadata.charset.tokens.len(),
}
}
/// 变体 B 核心处理器:单次遍历 2D 视图,融合计算 Softmax、Argmax、置信度并输出概率大包
fn compute_f32_full_probability(
&self,
matrix_view: ArrayView2<f32>,
) -> (Vec<Vec<f32>>, f32, Vec<i64>) {
let steps = matrix_view.nrows();
let classes = matrix_view.ncols();
// 1. 预分配满额概率矩阵内存
let mut prob_matrix = tract_ndarray::Array2::<f32>::zeros((steps, classes));
let mut predicted_indices = Vec::with_capacity(steps);
let mut confidence_sum = 0.0f32;
// 2. 融合单次遍历
for (step_idx, row) in matrix_view.outer_iter().enumerate() {
// 寻找当前 Step 的最大值和最大值索引 (Argmax)
let (row_max_idx, max_logit) = row
.iter()
.enumerate()
.max_by(|(_, a), (_, b)| a.total_cmp(b))
.map(|(idx, &val)| (idx, val))
.unwrap_or((0, 0.0));
predicted_indices.push(row_max_idx as i64);
// 计算单行 exp 溢出防范和
let mut exp_sum = 0.0f32;
for &val in row.iter() {
exp_sum += (val - max_logit).exp();
}
// 归一化 Softmax 顺序写入
for (class_idx, &val) in row.iter().enumerate() {
prob_matrix[[step_idx, class_idx]] = (val - max_logit).exp() / exp_sum;
}
// 当前 Step 最大概率在线累加
confidence_sum += 1.0f32 / exp_sum;
}
// 3. 统计全局平均置信度
let confidence = if steps > 0 {
confidence_sum / steps as f32
} else {
1.0
};
// 4. 将矩阵转化为标准安全序列化格式 [Steps, Classes]
let probabilities_list: Vec<Vec<f32>> =
prob_matrix.outer_iter().map(|row| row.to_vec()).collect();
(probabilities_list, confidence, predicted_indices)
}
/// 变体 A 专属提取器:直接从 I64 Tensor 零拷贝提取 CTC 文本与初始概率包
fn process_i64_tensor(&self, raw_tensor: Tensor) -> anyhow::Result<OcrResult> {
// 1. 拿到底层的动态维度只读视图
let view = raw_tensor.to_array_view::<i64>()?;
// 2. 索要底层连续的只读切片引用
let slice = view
.as_slice()
.ok_or_else(|| anyhow::anyhow!("I64 模型输出内存不连续,无法执行零拷贝解码"))?;
// 3. 直接喂给 CTC 解码器(无任何物理克隆开销)
let final_text = self.ctc_decode_to_string(slice);
// 4. 组装返回
if self.probability {
Ok(OcrResult::Probability {
text: final_text,
probabilities: vec![], // I64 模型物理上丢失了全量 Logits 分值网,降级处理
confidence: 1.0, // 判定即百分之百置信
})
} else {
Ok(OcrResult::Text(final_text))
}
}
/// 变体二F32的总体管线负责降维并分流文本和概率
fn process_f32_tensor(&self, raw_tensor: Tensor) -> anyhow::Result<OcrResult> {
let shape = raw_tensor.shape();
println!("模型输出shape数据: {:?}", shape);
let view = raw_tensor.to_array_view::<f32>()?;
// 1. 极其纯粹的、无拷贝的多维 Shape 压扁清洗
let (steps, classes, data_dyn_view) = match shape.len() {
3 => {
if shape[1] == 1 {
// 形状: [Steps, 1, Classes] -> 你的原有逻辑
(shape[0], shape[2], view.into_dyn())
} else if shape[0] == 1 {
// 形状: [1, Steps, Classes] -> 另一种常见导出格式
(shape[1], shape[2], view.into_dyn())
} else {
// 默认取第一个 batch: [Batch, Steps, Classes]
// 使用 slice 对应 Python 的 output[0, :, :]
let sliced = view.slice(s![0, .., ..]);
(shape[1], shape[2], sliced.into_dyn())
}
}
// 形状: [Steps, Classes] -> 已经剥离了 Batch 维度
2 => (shape[0], shape[1], view.into_dyn()),
// 形状: [Classes] -> 单字符输出(对应 Python 的 ndim == 0 保护逻辑)
// 我们把它虚构成一个 [1, Classes] 的 2D 矩阵来复用后面的 argmax 逻辑
1 => (1, shape[0], view.into_dyn()),
_ => return Err(anyhow::anyhow!("不支持的输出维度: {:?}", shape)),
};
let matrix_cow = data_dyn_view
.to_shape(Ix2(steps, classes))
.map_err(|e| anyhow::anyhow!("转换为2D静态矩阵失败: {:?}", e))?;
let matrix_view: ArrayView2<f32> = matrix_cow.view();
// 2. 根据业务参数明确分流
if self.probability {
// 走向 B1调用刚刚拆分出来的“全量概率计算器”
let (probabilities_list, confidence, predicted_indices) =
self.compute_f32_full_probability(matrix_view);
// 5. 执行 CTC 解码
let final_text = self.ctc_decode_to_string(&predicted_indices);
Ok(OcrResult::Probability {
text: final_text,
probabilities: probabilities_list,
confidence: confidence as f64,
})
} else {
// 走向 B2极速免 Softmax 提取纯文本(代码保持原地提取,简单短小不需要再拆)
let predicted_indices: Vec<i64> = matrix_view
.outer_iter()
.map(|row| {
row.iter()
.enumerate()
.max_by(|(_, a), (_, b)| a.total_cmp(b))
.map(|(idx, _)| idx as i64)
.unwrap_or(0)
})
.collect();
let final_text = self.ctc_decode_to_string(&predicted_indices);
Ok(OcrResult::Text(final_text))
}
}
/// 获取有效字符索引列表 (用于外部验证或过滤)
fn ctc_decode_to_string(&self, predicted_indices: &[i64]) -> String {
println!("indices模型输出原始数据: {:?}", predicted_indices);
let charset = &self.session.model_metadata.charset;
let tokens = &charset.tokens;
// let valid_indices = &charset.valid_indices;
// 对应 _ctc_decode_indices 的逻辑:去重、去 blank (0)
let mut res = String::new();
let mut prev_idx: i64 = -1;
for &idx in predicted_indices {
// 1. CTC 去重:如果是连续重复的,直接跳过
if idx == prev_idx {
continue;
}
// 【关键核心】只要不是连续重复,立刻更新 prev_idx 状态,绝对不能被后续的过滤短路!
prev_idx = idx;
// 2. CTC 过滤 Blank (0)
if idx == 0 {
continue;
}
// 3. 类型安全转换
let u_idx = match usize::try_from(idx) {
Ok(u) => u,
Err(_) => continue,
};
// 史诗级加速点:如果是 None说明没限制根本不进入分支直接放行
// 只有当有具体限制Some才去跑 4-5 次 CPU 寄存器级别的二分查找
if let Some(ref indices) = self.final_charset_indices {
if indices.binary_search(&u_idx).is_err() {
continue;
}
}
// 5. 字符映射
if let Some(char_str) = tokens.get(u_idx) {
res.push_str(char_str);
} else {
eprintln!("警告: 预测索引 {} 超出字符集范围", u_idx);
}
}
res
}
}

200
src/models/ocr/metadata.rs Normal file
View File

@@ -0,0 +1,200 @@
use anyhow::{anyhow, Result};
use serde::Deserialize;
use std::borrow::Cow;
use std::collections::HashMap;
// ==========================================
// 3. 字符集核心结构体 (重命名为 Charset)
// ==========================================
#[derive(Debug, Clone)]
pub struct Charset {
// 使用 Cow 统一静态切片和动态读取的 Vec<String>,内部实现真正的零拷贝
pub tokens: Vec<Cow<'static, str>>,
// 反向查找表,保证字符转索引为 O(1)
pub char_to_idx: HashMap<Cow<'static, str>, usize>,
// 当前处于激活状态的有效索引缓存 (用于 CTC 解码前的过滤加速)
// pub valid_indices: HashSet<usize>,
}
impl Charset {
// 内部底层统一收拢构造
pub fn new(tokens: Vec<Cow<'static, str>>) -> Self {
let mut char_to_idx = HashMap::with_capacity(tokens.len());
for (idx, token) in tokens.iter().enumerate() {
char_to_idx.entry(token.clone()).or_insert(idx);
// 如果字符集有重复,保留第一个遇到的索引 (符合 Python .index 逻辑)
// char_to_idx.entry(token.to_string()).or_insert(idx);
}
Self {
tokens,
char_to_idx,
}
}
// --- 业务策略方法 ---
/// 将字符转为索引,不存在返回 -1 (保持与原 Python 库行为一致)
pub fn char_to_index(&self, char_str: &str) -> i32 {
if let Some(&idx) = self.char_to_idx.get(char_str) {
idx as i32
} else {
-1
}
}
/// 将索引转为字符引用,零拷贝。若越界返回 None
pub fn index_to_char_ref(&self, index: usize) -> Option<&str> {
self.tokens.get(index).map(|cow| cow.as_ref())
}
pub fn is_valid_char(&self, char_str: &str) -> bool {
self.char_to_idx.get(char_str).is_some()
}
pub fn size(&self) -> usize {
self.tokens.len()
}
}
// ==========================================
// 4. 标准 Display 接口实现 (对应 __str__)
// ==========================================
impl std::fmt::Display for Charset {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "Charset [Total Size: {}", self.size(),)
}
}
// =====================================================================
// 1. 辅助定义的枚举与结构体
// =====================================================================
#[derive(Debug, Clone, Copy, Deserialize)]
#[serde(rename_all = "snake_case")] // 支持 json 中写 "zero_to_one" 或 "minus_one_to_one"
pub enum Normalization {
/// 映射到 [0.0, 1.0] -> pixel / 255.0
ZeroToOne,
/// 映射到 [-1.0, 1.0] -> (pixel / 255.0 - 0.5) / 0.5
MinusOneToOne,
}
impl Normalization {
/// 统一归一化计算逻辑
#[inline(always)]
pub fn normalize(&self, pixel: f32) -> f32 {
match self {
Normalization::ZeroToOne => pixel / 255.0,
Normalization::MinusOneToOne => (pixel / 255.0 - 0.5) / 0.5,
}
}
}
/// 图像缩放策略枚举
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Resize {
/// 固定宽高,例如 (64, 64)
Fixed(u32, u32),
/// 高度固定,宽度根据原始比例动态计算(对应 Python 的 [-1, H]
DynamicWidth(u32),
/// 单字识别的正方形切图(对应 Python 的 word 为 True 且 [-1, H]
Square(u32),
}
/// 仅用于反序列化 JSON 的中间临时结构体DTO
#[derive(Deserialize)]
struct ModelMetadataDto {
charset: Vec<String>,
word: bool,
#[serde(alias = "image")]
resize: Vec<i32>,
channel: u8,
/// 新增:允许在配置文件中指定归一化策略。
/// 使用 serde(default) 可以在不配置时提供一个默认值(比如默认 ZeroToOne
#[serde(default = "default_normalization")]
normalization: Normalization,
}
fn default_normalization() -> Normalization {
Normalization::ZeroToOne
}
#[derive(Debug, Clone)]
pub struct ModelMetadata {
/// 字符集管理器
pub charset: Charset,
/// 是否为单字识别模型
pub word: bool,
/// 预处理的缩放策略
pub resize: Resize,
/// 图像通道数 (1 或 3)
pub channel: u8,
/// 新增:传递给核心业务使用的归一化配置
pub normalization: Normalization,
}
impl ModelMetadata {
// --- 优雅的工厂模式构造器 ---
/// 通用的静态切片转换构造器
pub fn from_static_slice(
slice: &[&'static str],
word: bool,
resize: Resize,
channel: u8,
normalization: Normalization,
) -> Self {
let tokens: Vec<Cow<'static, str>> = slice.iter().map(|&s| Cow::Borrowed(s)).collect();
Self {
charset: Charset::new(tokens),
word,
resize,
channel,
normalization,
}
}
pub fn from_json_str(json_str: &str) -> Result<Self> {
let dto: ModelMetadataDto = serde_json::from_str(json_str)
.map_err(|e| anyhow!("JSON 反序列化失败,请检查字段是否完整: {}", e))?;
// 1. 将 DTO 的字符串数组转化为强类型的 Charset
let tokens: Vec<Cow<'static, str>> =
dto.charset.into_iter().map(|s| Cow::Owned(s)).collect();
let charset = Charset::new(tokens);
// 2. 解析 resize 策略(重现 Python 的复杂条件判断)
if dto.resize.len() != 2 {
return Err(anyhow!(
"'resize (or image)' 字段必须是包含两个元素的数组,例如 [-1, 64]"
));
}
let r0 = dto.resize[0];
let r1 = dto.resize[1];
let resize = if r0 == -1 {
if dto.word {
// 如果 word 为 true且包含 -1Python 里是 resize 为 (r1, r1) 的正方形
Resize::Square(r1 as u32)
} else {
// 如果 word 为 false且包含 -1Python 里是高度固定为 r1宽度按原图比例缩放
Resize::DynamicWidth(r1 as u32)
}
} else {
// 正常的固定宽高
Resize::Fixed(r0 as u32, r1 as u32)
};
Ok(Self {
charset,
word: dto.word,
resize,
channel: dto.channel,
normalization: dto.normalization,
})
}
/// 机制 2从内存字节流加载极大地方便 include_bytes! 或网络下载)
pub fn from_json_bytes(bytes: &[u8]) -> Result<Self> {
let json_str = std::str::from_utf8(bytes)
.map_err(|e| anyhow!("JSON 字节流不是合法的 UTF-8 编码: {}", e))?;
Self::from_json_str(json_str)
}
}

10
src/models/ocr/mod.rs Normal file
View File

@@ -0,0 +1,10 @@
mod builder;
mod executor;
mod session;
pub mod metadata;
pub mod color_filter;
mod token_filter;
pub use builder::OcrBuilder;
pub use executor::{Ocr, OcrResult};
pub use session::OcrSession;

53
src/models/ocr/session.rs Normal file
View File

@@ -0,0 +1,53 @@
use crate::models::ocr::metadata::ModelMetadata;
use crate::models::loader::{ModelLoader, ModelSession, ModelType};
use anyhow::Context;
use anyhow::Result;
use std::path::Path;
use tract_onnx::prelude::{tvec, Graph, IntoTensor, RunnableModel, Tensor, TypedFact, TypedOp};
pub struct OcrSession {
pub session: RunnableModel<TypedFact, Box<dyn TypedOp>, Graph<TypedFact, Box<dyn TypedOp>>>,
pub model_metadata: ModelMetadata,
}
impl ModelSession for OcrSession {
fn get_model_type(&self) -> ModelType {
todo!("使用thiserror作为错误处理的库,thiserror 专门用于开发库Library");
}
fn desc(&self) -> String {
"Ocr Model 加载成功".to_string()
}
}
impl OcrSession {
pub fn new<P>(model_path: P, model_metadata: ModelMetadata) -> Result<Self, anyhow::Error>
where
P: AsRef<Path>,
{
let session = ModelLoader::model_for_path(model_path)?.session;
Ok(Self {
session,
model_metadata,
})
}
pub fn model_from_bytes(
model_bytes: &[u8],
model_metadata: ModelMetadata,
) -> Result<Self, anyhow::Error> {
let session = ModelLoader::model_from_bytes(model_bytes)?.session;
Ok(Self {
session,
model_metadata,
})
}
/// 对应 Python 的 _inference
pub fn inference(&self, tensor: Tensor) -> anyhow::Result<Tensor> {
// tract 的 run 会返回一个 Vec<TValue>,我们通常只需要第一个输出
// let result = self.ocr.run(tvec!(tensor.into()))?;
let mut result = self
.session
.run(tvec!(tensor.into()))
.context("执行模型推理失败")?;
println!("模型输出原始数据: {:?}", result);
Ok(result.swap_remove(0).into_tensor())
}
}

View File

@@ -0,0 +1,146 @@
use std::borrow::Cow;
/// 字符集范围限制枚举
pub struct ValidationCtx<'a> {
pub text: &'a str, // 当前 Token 的文本内容
pub token_id: usize, // 当前 Token 的 ID 索引
}
/// 统一的约束接口
pub trait TokenFilter {
fn matches(&self, ctx: &ValidationCtx) -> bool;
/// 预估容量提示,帮助精准开辟 Vec 内存
fn estimated_capacity(&self) -> usize {
128
}
/// 【新引入的架构级核心方法】
/// 统一接管全量字符集的密集遍历、CTC Blank放行、去重、排序及空交集退化兜底
fn apply_to_charset(&self, tokens: &[Cow<str>]) -> Option<Vec<usize>> {
let mut has_any_match = false;
let estimated_capacity = self.estimated_capacity();
// 1. 精准开辟内存,完美利用容量提示,避免动态乱涨
let mut temp_indices = Vec::with_capacity(estimated_capacity.max(16));
// 2. 高性能原地单次流式迭代
for (idx, token) in tokens.iter().enumerate() {
let token_str = token.as_ref();
// 规则 A: CTC Blank 空字符串或 0 号索引无条件放行
if token_str.is_empty() || idx == 0 {
temp_indices.push(idx);
continue; // 关键:直接跳过,防止后续 matches 匹配成功导致重复 push 产生 Bug
}
// 规则 B: 组装无拷贝上下文
let ctx = ValidationCtx {
text: token_str,
token_id: idx,
};
// 规则 C: 路由到各自具体实现的特异性匹配中(如 Digit 判定、TopN 判定、组合子判定等)
if self.matches(&ctx) {
temp_indices.push(idx);
has_any_match = true;
}
}
// 3. 终极防御:如果整个模型字符集除了 Blank一个都没对上直接退化为 None全量识别
if !has_any_match {
println!("警告:当前限制策略与模型字符集完全没有交集!已自动恢复全量识别。");
None
} else {
// 4. 排序并去重,为 Ocr 引擎后续进行极其高频的『二分查找』筑起绝对安全的底层保障
temp_indices.sort_unstable();
temp_indices.dedup();
Some(temp_indices)
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CharRestrict {
Digit,
Lowercase,
Uppercase,
CustomList(Vec<String>),
}
impl TokenFilter for CharRestrict {
fn matches(&self, ctx: &ValidationCtx) -> bool {
match self {
Self::Digit => ctx.text.len() == 1 && ctx.text.as_bytes()[0].is_ascii_digit(),
Self::Lowercase => ctx.text.len() == 1 && ctx.text.as_bytes()[0].is_ascii_lowercase(),
Self::Uppercase => ctx.text.len() == 1 && ctx.text.as_bytes()[0].is_ascii_uppercase(),
Self::CustomList(vec) => vec.iter().any(|t| t == ctx.text),
}
}
fn estimated_capacity(&self) -> usize {
match self {
Self::Digit => 16,
Self::Lowercase | Self::Uppercase => 32,
Self::CustomList(vec) => vec.len() + 1,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum IdRestrict {
TopN(usize),
IdRange(std::ops::Range<usize>),
IdList(Vec<usize>),
}
impl TokenFilter for IdRestrict {
fn matches(&self, ctx: &ValidationCtx) -> bool {
match self {
Self::TopN(n) => ctx.token_id < *n,
Self::IdRange(range) => range.contains(&ctx.token_id),
Self::IdList(vec) => vec.contains(&ctx.token_id),
}
}
fn estimated_capacity(&self) -> usize {
match self {
Self::TopN(n) => *n + 1,
// 2. IdRange标准标准库 Range 的长度
// 注意:因为范围可能是 1000..2000,它的 len() 返回的是 usize
Self::IdRange(range) => range.len() + 1,
// 3. IdListVec 里的元素个数
Self::IdList(vec) => vec.len() + 1,
}
}
}
/// 多路“或”逻辑组合子(支持 N 个规则无缝并集)
pub struct MultiOrRestrict<'a> {
pub filters: Vec<&'a dyn TokenFilter>,
}
impl<'a> TokenFilter for MultiOrRestrict<'a> {
fn matches(&self, ctx: &ValidationCtx) -> bool {
// 核心高阶函数:只要有一个过滤器命中,该 Token 即可放行
self.filters.iter().any(|f| f.matches(ctx))
}
fn estimated_capacity(&self) -> usize {
// 将所有过滤器的预估容量累加,作为最终容量参考
self.filters.iter().map(|f| f.estimated_capacity()).sum()
}
}
// =====================================================================
// 声明式宏:替代 `+` 运算符,解决组合扩展痛苦
// =====================================================================
#[macro_export]
macro_rules! any_of {
// 场景 A如果用户只传了一个规则免去构建 Vec 的开销,直接返回其引用
($only:expr) => {
&$only as &dyn $crate::TokenFilter
};
// 场景 B如果用户传入了多个规则自动织成一张静态组合网
($($filter:expr),+ $(,)?) => {
&$crate::MultiOrRestrict {
filters: vec![ $( &$filter as &dyn $crate::TokenFilter ),+ ]
}
};
}

161
src/utils/cv_ops.rs Normal file
View File

@@ -0,0 +1,161 @@
use image::{ImageBuffer, Luma};
use std::cmp::{max, min};
use tract_onnx::prelude::tract_ndarray::{Array2, Array3, ArrayView2, ArrayView3, azip};
/// 1. 计算两个数组的绝对差值 (对应 cv2.absdiff)
pub fn abs_diff(a: &ArrayView3<u8>, b: &ArrayView3<u8>) -> Array3<u8> {
// 利用 ndarray 的 map_collect生成差值的绝对值数组
// 或者直接使用 zip_mut_with 处理以减少内存分配
let mut diff = Array3::zeros(a.dim());
azip!((res in &mut diff, &va in a, &vb in b) {
*res = (va as i16 - vb as i16).abs() as u8;
});
diff
}
/// RGB 到灰度转换
pub fn rgb_to_gray(rgb: ArrayView3<u8>) -> Array2<u8> {
let (h, w, _) = rgb.dim();
Array2::from_shape_fn((h, w), |(y, x)| {
let r = rgb[[y, x, 0]] as f32;
let g = rgb[[y, x, 1]] as f32;
let b = rgb[[y, x, 2]] as f32;
// 完全忽略 a只按权重计算
(0.299 * r + 0.587 * g + 0.114 * b) as u8
})
}
/// 寻找匹配结果图中的最大值及其坐标 (模拟 cv2.minMaxLoc 的一部分)
pub fn min_max_loc(result_map: &ImageBuffer<Luma<f32>, Vec<f32>>) -> (f32, (u32, u32)) {
// 4. 找到最佳匹配位置 (对齐 cv2.minMaxLoc)
let mut max_val: f32 = -1.0;
let mut max_loc = (0, 0);
// 遍历匹配得分图
for (x, y, score) in result_map.enumerate_pixels() {
let s = score.0[0];
// 可以在此处加入你之前验证过的起始位过滤
// if x < 15 { continue; }
if s > max_val {
max_val = s;
max_loc = (x, y);
}
}
(max_val, max_loc)
}
/// 1. 模拟 findContours 并获取最大面积区域的 Label
/// 返回 Option<u32>,如果找不到任何区域则返回 None
pub fn find_contours_and_max(labelled: &ImageBuffer<Luma<u32>, Vec<u32>>) -> Option<u32> {
// 统计每个标签出现的频率(即面积)
let mut max_label = 0;
let mut max_area = 0;
let mut areas = std::collections::HashMap::new();
for pixel in labelled.pixels() {
let label = pixel.0[0];
if label == 0 {
continue;
} // 跳过背景
let count = areas.entry(label).or_insert(0);
*count += 1;
if *count > max_area {
max_area = *count;
max_label = label;
}
}
if max_label == 0 {
None
} else {
Some(max_label)
}
}
pub fn bounding_rect(
labelled: &ImageBuffer<Luma<u32>, Vec<u32>>,
max_label: u32,
) -> (u32, u32, u32, u32) {
// 5. 计算最大区域的边界框 (对应 cv2.boundingRect)
let mut min_x = labelled.width();
let mut max_x = 0;
let mut min_y = labelled.height();
let mut max_y = 0;
for (x, y, pixel) in labelled.enumerate_pixels() {
if pixel.0[0] == max_label {
min_x = min(min_x, x);
max_x = max(max_x, x);
min_y = min(min_y, y);
max_y = max(max_y, y);
}
}
let w = max_x - min_x;
let h = max_y - min_y;
(min_x, min_y, w, h)
}
pub fn calculate_center(top_left: (u32, u32), width: usize, height: usize) -> (i32, i32) {
let center_x = top_left.0 as i32 + (width as i32 / 2);
let center_y = top_left.1 as i32 + (height as i32 / 2);
(center_x, center_y)
}
pub fn ndarray_to_luma8(array: ArrayView2<u8>) -> ImageBuffer<Luma<u8>, Vec<u8>> {
let (height, width) = array.dim();
let mut buffer = ImageBuffer::new(width as u32, height as u32);
for y in 0..height {
for x in 0..width {
buffer.put_pixel(x as u32, y as u32, Luma([array[[y, x]]]));
}
}
buffer
}
// =====================================================================
// 5. 核心高性能图像转换算法 (纯 Rust 编写)
// =====================================================================
#[inline(always)]
pub fn rgb_to_opencv_hsv(r: u8, g: u8, b: u8) -> (u8, u8, u8) {
// 1. 规避高昂的除法,直接转为 f32 进行比对
let r_f = r as f32;
let g_f = g as f32;
let b_f = b as f32;
let max = r_f.max(g_f).max(b_f);
let min = r_f.min(g_f).min(b_f);
let delta = max - min;
// 2. 计算 H (色调) - 移除负数取余陷阱,改用平铺分支
let mut h = if delta == 0.0 {
0.0
} else if max == r_f {
let mut diff = (g_f - b_f) / delta;
if diff < 0.0 {
diff += 6.0; // 规避 Rust f32 % 负数的行为
}
60.0 * diff
} else if max == g_f {
60.0 * (((b_f - r_f) / delta) + 2.0)
} else {
60.0 * (((r_f - g_f) / delta) + 4.0)
};
// OpenCV 的 H 量化H / 2
// 注意OpenCV 底层使用截断还是四舍五入与特定版本有关,
// 标准的 cvtColor 内部实现通常是: h * (180.0 / 360.0) -> h * 0.5
// 这里使用强转(截断)若单测对齐发现差1可改为 (h * 0.5 + 0.5) 或 round()
let h_opencv = (h * 0.5) as u8;
// 3. 计算 S (饱和度)
// OpenCV 公式: S = max == 0 ? 0 : 255 * delta / max
let s_opencv = if max == 0.0 {
0
} else {
((255.0 * delta) / max) as u8
};
// 4. 计算 V (明度)
let v_opencv = max as u8;
(h_opencv, s_opencv, v_opencv)
}

264
src/utils/image_io.rs Normal file
View File

@@ -0,0 +1,264 @@
use anyhow::{Context, Result, anyhow, bail};
use base64::{Engine as _, engine::general_purpose};
use image::{DynamicImage, GenericImageView, ImageBuffer, ImageFormat, Luma, Rgb, RgbImage, Rgba};
use std::fs;
use std::path::{Path, PathBuf};
use tract_onnx::prelude::tract_ndarray::{Array3, ArrayD, ArrayViewD};
#[derive(Debug)]
pub enum ColorMode {
RGB,
RGBA,
L,
}
/// 定义支持的输入类型枚举
pub enum ImageInput {
Bytes(Vec<u8>),
Array(ArrayD<u8>), // 对应 numpy 数组
Path(PathBuf),
Base64(String),
DynamicImage(DynamicImage),
}
/// 模拟 Python 的 load_image_from_input
#[allow(dead_code)]
pub fn load_image_from_input(img_input: ImageInput) -> Result<DynamicImage> {
match img_input {
// 2. 处理字节流 (Bytes)
ImageInput::Bytes(bytes) => {
image::load_from_memory(&bytes).context("Failed to load utils from bytes")
}
// 1. 已经是 DynamicImage
ImageInput::DynamicImage(i) => Ok(i),
// 5. 处理 ndarray (Numpy-like)
// 假设输入是 HWC 格式的 Array3<u8>
ImageInput::Array(a) => numpy_to_pil_image(a.view()),
// 4. 处理 Base64 字符串
ImageInput::Base64(b) => base64_to_image(&b),
// 3. 处理文件路径 (Path)
ImageInput::Path(p) => image::open(p).context("Failed to open utils from path"),
}
}
fn base64_to_image(b64_str: &str) -> Result<DynamicImage> {
// 过滤掉可能存在的 base64 前缀,例如 "data:utils/png;base64,"
let clean_b64 = if let Some(pos) = b64_str.find(",") {
&b64_str[pos + 1..]
} else {
&b64_str
};
let bytes = general_purpose::STANDARD
.decode(clean_b64.trim())
.map_err(|e| anyhow!("Base64 decode error: {}", e))?;
image::load_from_memory(&bytes).context("Failed to load utils from decoded base64")
}
/// 读取图片文件并转换为 base64 编码字符串
/// 对应 Python 版 get_img_base64
pub fn get_img_base64<P: AsRef<Path>>(image_path: P) -> Result<String> {
// 1. 读取文件原始字节流
// 使用 AsRef<Path> 泛型可以让函数同时支持 String, &str, PathBuf 等类型
let image_data = fs::read(&image_path)
.with_context(|| format!("Failed to read utils file: {:?}", image_path.as_ref()))?;
// 2. 进行 Base64 编码
// 使用 STANDARD 引擎对齐 Python 的 base64.b64encode
let b64_string = general_purpose::STANDARD.encode(image_data);
Ok(b64_string)
}
/// 封装数组转图像的逻辑,对齐 Python 版 _numpy_to_pil_image
fn numpy_to_pil_image(array: ArrayViewD<u8>) -> Result<DynamicImage> {
let shape = array.shape();
let dim = shape.len();
// 1. 确保数据在内存中是连续的 (C order / Standard Layout)
// 如果 arr 是经过切片或转置的,这一步会进行必要的内存拷贝
let standard = array.as_standard_layout();
let (raw_data, _offset) = standard.to_owned().into_raw_vec_and_offset();
match dim {
// 对应 Python: len(array.shape) == 2 (灰度图 H, W)
2 => {
let (h, w) = (shape[0], shape[1]);
ImageBuffer::<Luma<u8>, _>::from_raw(w as u32, h as u32, raw_data)
.map(DynamicImage::ImageLuma8)
.ok_or_else(|| anyhow!("Failed to create Luma utils from 2D array"))
}
// 对应 Python: len(array.shape) == 3 (H, W, C)
3 => {
let (h, w, c) = (shape[0], shape[1], shape[2]);
match c {
// 对应 Python: array.shape[2] == 1 (单通道 H, W, 1)
1 => ImageBuffer::<Luma<u8>, _>::from_raw(w as u32, h as u32, raw_data)
.map(DynamicImage::ImageLuma8),
// 对应 Python: array.shape[2] == 3 (RGB H, W, 3)
3 => ImageBuffer::<Rgb<u8>, _>::from_raw(w as u32, h as u32, raw_data)
.map(DynamicImage::ImageRgb8),
// 对应 Python: array.shape[2] == 4 (RGBA H, W, 4)
4 => ImageBuffer::<Rgba<u8>, _>::from_raw(w as u32, h as u32, raw_data)
.map(DynamicImage::ImageRgba8),
_ => {
return Err(anyhow!("不支持的通道数: {}", c));
}
}
.ok_or_else(|| anyhow!("转换彩色图失败"))
}
_ => Err(anyhow!("不支持的数组维度: {},仅支持 2D 或 3D", dim)),
}
}
/// 对应 Python 的 png_rgba_black_preprocess
/// 将带有透明通道的图片转换为白色背景的 RGB 图片
pub fn png_rgba_white_preprocess(img: &DynamicImage) -> DynamicImage {
// 1. 检查是否包含透明通道,如果没有,直接克隆并返回
if !img.color().has_alpha() {
return DynamicImage::ImageRgb8(img.to_rgb8());
}
let (width, height) = img.dimensions();
// 2. 创建一个新的 RGB 图像缓冲,默认填充为白色 (255, 255, 255)
let mut background = ImageBuffer::from_pixel(width, height, Rgb([255u8, 255u8, 255u8]));
// 3. 获取原图的 RGBA 视图
let rgba_img = img.to_rgba8();
// 4. 遍历像素并手动进行 Alpha 混合
// 对应 Python 的 utils.paste(img, ..., mask=img)
// 使用 enumerate_pixels_mut 同时获取坐标和背景像素的可变引用,减少查找开销
for (x, y, bg_pixel) in background.enumerate_pixels_mut() {
// 安全性说明x, y 源自 background 尺寸,与 rgba_img 一致get_pixel 是安全的
let src_pixel = rgba_img.get_pixel(x, y);
let alpha_u8 = src_pixel[3];
match alpha_u8 {
// 情况 A完全不透明直接覆盖背景色
255 => {
bg_pixel.0 = [src_pixel[0], src_pixel[1], src_pixel[2]];
}
// 情况 B完全透明保持背景色白色无需操作
0 => {
continue;
}
// 情况 C半透明进行 Alpha 混合计算
_ => {
let alpha = alpha_u8 as f32 / 255.0;
let inv_alpha = 1.0 - alpha;
bg_pixel[0] = (src_pixel[0] as f32 * alpha + 255.0 * inv_alpha).round() as u8;
bg_pixel[1] = (src_pixel[1] as f32 * alpha + 255.0 * inv_alpha).round() as u8;
bg_pixel[2] = (src_pixel[2] as f32 * alpha + 255.0 * inv_alpha).round() as u8;
}
}
}
DynamicImage::ImageRgb8(background)
}
pub fn image_to_numpy(image: &DynamicImage, mode: ColorMode) -> Result<Array3<u8>> {
// 1. 模式转换 (对应 utils.convert(target_mode)),此函数在时保留看后续优化是否需要替代image_to_ndarray
// Rust utils 库通过 to_rgb8, to_luma8 等方法实现转换
let (width, height) = image.dimensions();
let (channels, raw) = match mode {
ColorMode::RGB => (3, image.to_rgb8().into_raw()),
ColorMode::L => (1, image.to_luma8().into_raw()),
ColorMode::RGBA => (4, image.to_rgba8().into_raw()),
};
Array3::from_shape_vec((height as usize, width as usize, channels), raw)
.map_err(|e| anyhow!("Failed to build ndarray: {}", e))
}
pub fn numpy_to_image(array: ArrayViewD<u8>, mode: ColorMode) -> Result<DynamicImage> {
let shape = array.shape();
// 1. 基础维度检查 (必须是 H, W, C 三维数组)
if shape.len() != 3 {
bail!("Expected a 3D array (H, W, C), but got {}D", shape.len());
}
let height = shape[0] as u32;
let width = shape[1] as u32;
let channels = shape[2];
// 2. 检查通道数是否与模式匹配
let expected_channels = match mode {
ColorMode::L => 1,
ColorMode::RGB => 3,
ColorMode::RGBA => 4,
};
if channels != expected_channels {
bail!(
"Mode {:?} expects {} channels, but array has {}",
mode,
expected_channels,
channels
);
}
// 确保数据连续性 (C-order)
let standard = array.as_standard_layout();
let (raw_data, _) = standard.to_owned().into_raw_vec_and_offset();
match mode {
ColorMode::L => ImageBuffer::<Luma<u8>, _>::from_raw(width, height, raw_data)
.map(DynamicImage::ImageLuma8),
ColorMode::RGB => ImageBuffer::<Rgb<u8>, _>::from_raw(width, height, raw_data)
.map(DynamicImage::ImageRgb8),
ColorMode::RGBA => ImageBuffer::<Rgba<u8>, _>::from_raw(width, height, raw_data)
.map(DynamicImage::ImageRgba8),
}
.ok_or_else(|| anyhow!("Failed to construct ImageBuffer. Buffer size might be incorrect."))
}
pub fn image_to_ndarray(img: &DynamicImage) -> Array3<u8> {
let (width, height) = img.dimensions();
// 1. 强制转为 RGB8 (丢弃 Alpha 通道,与 Python 的 target_mode='RGB' 对齐)
let rgb_img = img.to_rgb8();
// 2. 获取原始像素数据
let raw_data = rgb_img.into_raw();
// 3. 构造数组 (通道数改为 3)
Array3::from_shape_vec((height as usize, width as usize, 3), raw_data)
.expect("Failed to construct ndarray from utils") // 建议显式报错,而不是返回全黑图
}
#[allow(dead_code)]
fn save_rust_result(result: &ImageBuffer<Luma<f32>, Vec<f32>>, filename: &str) {
let (width, height) = result.dimensions();
// 1. 寻找最值进行归一化
let mut max_val = f32::MIN;
let mut min_val = f32::MAX;
for p in result.pixels() {
if p.0[0] > max_val {
max_val = p.0[0];
}
if p.0[0] < min_val {
min_val = p.0[0];
}
}
// 2. 创建 8 位灰度图
let mut out_buf = ImageBuffer::new(width, height);
for y in 0..height {
for x in 0..width {
let val = result.get_pixel(x, y).0[0];
let normalized = if max_val > min_val {
((val - min_val) / (max_val - min_val) * 255.0) as u8
} else {
0u8
};
out_buf.put_pixel(x, y, Luma([normalized]));
}
}
// 3. 保存
DynamicImage::ImageLuma8(out_buf).save(filename).unwrap();
println!("Rust 结果热力图已保存至: {}", filename);
}

View File

@@ -0,0 +1,40 @@
use image::{DynamicImage, GrayImage, imageops::FilterType, Rgb, ImageBuffer};
use anyhow::{anyhow, Result};
use crate::models::ocr::color_filter::HsvRange;
use crate::utils::cv_ops::rgb_to_opencv_hsv;
/// 对应 Python 的 convert_to_grayscale
/// 将图像转换为灰度图 (L模式)
pub fn convert_to_grayscale(image: &DynamicImage) -> GrayImage {
// Rust utils 库的 to_luma8 会根据标准的亮度公式进行转换
image.to_luma8()
}
/// 对应 Python 的 resize_image
/// 调整图像尺寸。当前版本仅实现 keep_aspect_ratio=false
pub fn resize_image(
image: &DynamicImage,
target_width: u32,
target_height: u32,
// resample 参数我们直接使用 FilterTypeLanczos3 是最接近 Python LANCZOS 的
) -> DynamicImage {
// image::imageops::resize 的最高层封装
// FilterType::Lanczos3 与 Python Pillow 的 Image.LANCZOS 算法完全对齐,缩放质量最高
image.resize_exact(target_width, target_height, FilterType::Lanczos3)
}
// pub fn resize_image(
// image: &GrayImage,
// target_width: u32,
// target_height: u32,
// // resample 参数我们直接使用 FilterTypeLanczos3 是最接近 Python LANCZOS 的
// ) -> GrayImage {
// // 使用 resize 算法进行精确缩放
// image::imageops::resize(
// image,
// target_width,
// target_height,
// FilterType::Lanczos3
// )
// }

3
src/utils/mod.rs Normal file
View File

@@ -0,0 +1,3 @@
pub mod image_io;
pub mod image_processor;
pub mod cv_ops;

View File

@@ -1,3 +1,10 @@
use std::borrow::Cow;
use std::fs::File;
use std::path::Path;
use anyhow::anyhow;
use ddddocr_rs::models::ocr::metadata::Charset;
use ddddocr_rs::models::ocr::metadata::{Normalization, Resize};
pub const CHARSET_BETA: &[&str] = &[ pub const CHARSET_BETA: &[&str] = &[
"", "", "", "", "", "", "", "", "", "", "", "", "", "6", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "6", "", "",
"", "", "", "", "", "", "", "", "", "", "", "鴿", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "鴿", "", "", "", "",
@@ -514,3 +521,80 @@ pub const CHARSET_BETA: &[&str] = &[
"", "", "", "", "婿", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "婿", "", "", "", "", "", "", "", "", "", "", "",
"", "", "", "",
]; ];
pub const CHARSET_OLD: &[&str] = &["", "", "", "", ""];
// pub fn from_builtin_old() -> Self {
// Self::from_static_slice(
// CHARSET_OLD,
// false,
// Resize::DynamicWidth(64),
// 1,
// Normalization::ZeroToOne,
// )
// }
//
// /// 从预设的 Beta 版字符集创建
// pub fn from_builtin_beta() -> Self {
// Self::from_static_slice(
// CHARSET_BETA,
// false,
// Resize::DynamicWidth(64),
// 1,
// Normalization::MinusOneToOne,
// )
// }
// /// 从外部外部 JSON 文件动态加载字符集(在后续优化中移除)
// pub fn from_json_file<P: AsRef<Path>>(path: P) -> anyhow::Result<Self> {
// let path = path.as_ref();
// if !path.exists() {
// return Err(anyhow!("模型元数据配置文件不存在: {:?}", path));
// }
//
// let mut file = File::open(path)?;
// let mut content = String::new();
// file.read_to_string(&mut content)?;
//
// let dto: ModelMetadataDto = serde_json::from_str(&content)
// .map_err(|e| anyhow!("JSON 反序列化失败,请检查字段是否完整: {}", e))?;
//
// // 1. 将 DTO 的字符串数组转化为强类型的 Charset
// let tokens: Vec<Cow<'static, str>> =
// dto.charset.into_iter().map(|s| Cow::Owned(s)).collect();
// let charset = Charset::new(tokens);
//
// // 2. 解析 resize 策略(重现 Python 的复杂条件判断)
// if dto.resize.len() != 2 {
// return Err(anyhow!(
// "'resize (or image)' 字段必须是包含两个元素的数组,例如 [-1, 64]"
// ));
// }
// let r0 = dto.resize[0];
// let r1 = dto.resize[1];
//
// let resize = if r0 == -1 {
// if dto.word {
// // 如果 word 为 true且包含 -1Python 里是 resize 为 (r1, r1) 的正方形
// Resize::Square(r1 as u32)
// } else {
// // 如果 word 为 false且包含 -1Python 里是高度固定为 r1宽度按原图比例缩放
// Resize::DynamicWidth(r1 as u32)
// }
// } else {
// // 正常的固定宽高
// Resize::Fixed(r0 as u32, r1 as u32)
// };
//
// Ok(Self {
// charset,
// word: dto.word,
// resize,
// channel: dto.channel,
// normalization: dto.normalization,
// })
// }

183
tests/ocr_test.rs Normal file
View File

@@ -0,0 +1,183 @@
use ddddocr_rs::models::det::DetectionResult;
use ddddocr_rs::{DetBuilder, DetSession, Detector, ModelMetadata, Ocr, OcrSession, Slider}; // 假设你的包名是这个
use image::{DynamicImage, Rgb};
use std::fs;
use std::path::Path;
mod char_slice;
use char_slice::CHARSET_BETA;
use ddddocr_rs::models::ocr::metadata::{Normalization, Resize};
fn load_image<P: AsRef<Path>>(path: P) -> anyhow::Result<image::DynamicImage> {
// 1. 先将泛型转为具体的 &Path 引用
let path_ref = path.as_ref();
// 2. 调用 open 时传入引用utils::open 支持 AsRef<Path>
image::open(path_ref).map_err(|e| {
// 3. 此时 path_ref 依然有效,可以安全地在闭包中使用
anyhow::anyhow!("无法加载图片 {:?}: {}", path_ref, e)
})
}
/// 将检测结果绘制在图像上并保存
fn save_debug_image(
dynamic_img: &DynamicImage, // 【优化点 1】直接传入解码好的引用拒绝重复解码
bboxes: &[DetectionResult], // 【修改点 1】类型改为自定义结构体切片
output_path: &str,
) -> anyhow::Result<()> {
// 删除了原本的 let dynamic_img = image::load_from_memory(image_bytes)?;
let mut img = dynamic_img.to_rgb8();
let (width, height) = img.dimensions();
let red = Rgb([255u8, 0, 0]);
for bbox in bboxes {
// 【修改点 2】将原来的索引 bbox[0].. 改为结构体字段访问 .x1, .y1 ..
let x1 = bbox.x1.max(0).min(width as i32 - 1) as u32;
let y1 = bbox.y1.max(0).min(height as i32 - 1) as u32;
let x2 = bbox.x2.max(0).min(width as i32 - 1) as u32;
let y2 = bbox.y2.max(0).min(height as i32 - 1) as u32;
// 绘制横向线条
for x in x1..=x2 {
img.put_pixel(x, y1, red);
img.put_pixel(x, y2, red);
if y1 + 1 < height {
img.put_pixel(x, y1 + 1, red);
}
if y2.saturating_sub(1) > 0 {
img.put_pixel(x, y2 - 1, red);
}
}
// 绘制纵向线条
for y in y1..=y2 {
img.put_pixel(x1, y, red);
img.put_pixel(x2, y, red);
if x1 + 1 < width {
img.put_pixel(x1 + 1, y, red);
}
if x2.saturating_sub(1) > 0 {
img.put_pixel(x2 - 1, y, red);
}
}
}
img.save(output_path)?;
Ok(())
}
#[test]
fn test_full_classification() {
// 1. 初始化模型
let ocr = OcrSession::new(
"D:\\CNWei\\CNW\\Rust\\ddddocr-rs\\models\\common_sml2h3_f32.onnx",
ModelMetadata::from_static_slice(
CHARSET_BETA,
false,
Resize::DynamicWidth(64),
1,
Normalization::MinusOneToOne,
),
)
.expect("模型加载失败");
// 2. 加载测试图片
let img = image::open("samples/code2.png").expect("测试图片不存在");
// 3. 执行识别
let result = Ocr::new(&ocr)
.predict(&img)
.expect("识别过程出错")
.into_text();
println!("识别结果: {}", result);
assert!(!result.is_empty());
}
#[test]
fn test_det_load() -> anyhow::Result<()> {
let det = DetSession::new("D:\\CNWei\\CNW\\Rust\\ddddocr-rs\\models\\common_det.onnx")?;
let image_path = "samples/det1.png";
let image_bytes =
fs::read(image_path).map_err(|e| anyhow::anyhow!("无法读取图片 {}: {}", image_path, e))?;
println!("图片读取成功,字节大小: {}", image_bytes.len());
// 【修改点 1】将字节流解码为统一的 DynamicImage
let img = image::load_from_memory(&image_bytes)
.map_err(|e| anyhow::anyhow!("图片解码失败: {}", e))?;
// 【修改点 2】传入统一的 &DynamicImage 引用
let bboxes = Detector::new(&det).predict(&img)?;
// println!("{:?}", det);
println!("检测到的目标数量: {}", bboxes.len());
if bboxes.is_empty() {
println!("未检测到任何目标。");
} else {
// 如果 save_debug_image 报错,记得去把它的入参类型和内部访问也改为 DetectionResult
save_debug_image(&img, &bboxes, "samples/result.jpg")?;
for (i, bbox) in bboxes.iter().enumerate() {
// 【修改点 3】将原来的 bbox[0].. 索引访问改为结构体字段访问
println!("目标 [{}]: {}", i, bbox);
}
}
Ok(())
}
#[test]
fn test_real_slide_match() {
let engine = Slider::new().unwrap();
// 1. 加载你准备好的测试图
// 假设图片放在项目根目录下的 assets 文件夹
let target_img = load_image("samples/hua.png").expect("请确保 samples/hua.png 存在");
let bg_img = load_image("samples/huatu.png").expect("请确保 samples/huatu.png 存在");
// 2. 执行匹配
// 如果是那种带有明显阴影边缘的复杂滑块,建议 simple_target 传 false
let start = std::time::Instant::now();
let result = engine
.slide_match(&target_img, &bg_img, false)
.expect("Slide match 执行失败");
let duration = start.elapsed();
// 3. 打印结果
println!("-------------------------------------------");
println!("{}", result);
println!("耗时: {:?}", duration);
println!("-------------------------------------------");
// 验证基本逻辑:坐标不应为 0 (除非匹配失败)
assert_eq!(result.target_x, 237);
assert_eq!(result.target_y, 77);
assert!(result.confidence > 0.0);
}
#[test]
fn test_real_slide_comparison() {
let engine = Slider::new().unwrap();
// 1. 加载你准备好的测试图
// 假设图片放在项目根目录下的 assets 文件夹
let target_img = load_image("samples/ken.jpg").expect("请确保 samples/ken.jpg 存在");
let bg_img = load_image("samples/kenyuan.jpg").expect("请确保 samples/kenyuan.jpg 存在");
// 2. 执行匹配
// 如果是那种带有明显阴影边缘的复杂滑块,建议 simple_target 传 false
let start = std::time::Instant::now();
let result = engine
.slide_comparison(&target_img, &bg_img)
.expect("Slide match 执行失败");
let duration = start.elapsed();
// 3. 打印结果
println!("-------------------------------------------");
println!("滑块匹配测试结果:");
println!("检测坐标: [x: {}, y: {}]", result.target_x, result.target_y);
println!("置信度: {:.4}", result.confidence);
println!("耗时: {:?}", duration);
println!("-------------------------------------------");
// 验证基本逻辑:坐标不应为 0 (除非匹配失败)
assert_eq!(result.target_x, 171);
assert_eq!(result.target_y, 90);
assert!(result.confidence > 0.0);
}