Post

Google Launches LiteRT Tensor API for Building, Customizing, and Optimizing ML Models for On-Device

Google’s LiteRT Tensor API lets C++ programs construct model graphs directly and target CPU, GPU and NPU backends. It supports loading weights from checkpoints, including safetensors, and zero-copy interoperability across processing stages.

Google reports a SAM 2 demonstration running the same C++ code across Metal and OpenCL, with a vendor-reported Mac benchmark about 1.5× MLX for the stated workload. Full-graph quantization and broader dynamic-shape support remain planned. An Arm AI director cited by InfoQ recommends starting with model conversion and using the API when graph-level control or accelerator-resident processing is needed.