Is Low NPU Utilization During WITHBOX AI Album Recognition a Performance Issue?

Is Low NPU Utilization During WITHBOX AI Album Recognition a Performance Issue?

1) Low NPU utilization is normal behavior and not a performance issue. The NPU chip used in WITHBOX features a multi-core architecture. The system displays the overall average utilization across multiple NPU cores, and the actual load may be significantly higher than the surface-level data suggests. The specific reasons are as follows:

1) ① The system displays "total utilization," not single-core occupancy. The NPU built into WITHBOX devices contains multiple parallel computing cores, and the system displays resource usage as a global average.

2) ② AI image recognition is a multi-stage process; the NPU is only active in certain stages. The AI album recognition task does not rely on the NPU throughout the entire pipeline; it is a multi-stage collaborative process that includes:

1) Data stage — primary resource: CPU; loads user photos into memory.

2) Image preprocessing — primary resource: CPU; operations such as scaling, cropping, and format standardization.

3) Model inference — primary resource: NPU; initiates neural networks for face recognition, object classification, etc.

4) Result storage — primary resource: CPU + storage system; writes recognition results, tags, and the index database.

3) ③ Low utilization reflects the high efficiency of the NPU.

The NPU is a low-power, high-efficiency computing unit specifically designed for deep learning inference. Its characteristics include:

• High-throughput processing of model tasks;

• Immediate resource release upon task completion;

• Collaborative operation with the CPU to enhance overall system concurrency performance.

If the NPU utilization appears low but recognition tasks complete within a reasonable timeframe and accuracy remains normal, this indicates that the system is operating well and resource allocation is appropriate.

Only the "model inference" stage uses the NPU for deep learning inference, after which resources are released immediately. Therefore, the NPU does not run continuously, and its utilization exhibits a pattern of "short-duration peaks followed by rapid decline."

It is recommended not to use NPU utilization as the sole evaluation metric; instead, focus on task completion speed and recognition accuracy.