11.5.20-ICT-Song-Han

• Video details

  Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude less  than mobile phones, not to mention GPUs. I will introduce key technologies for neural network optimization on IoT devices, including model compression (pruning, quantization), neural architecture search, and compiler/runtime optimizations. Based on that, we propose MCUNet, a framework that jointly designs the efficient neural architecture (TinyNAS) and the lightweight inference engine (TinyEngine). MCUNet automatically designs perfectly matched neural architecture and the inference library on MCU. MCUNet enables ImageNet-scale inference on microcontrollers that has only 1MB of FLASH and 320KB SRAM. It achieves significant speedup compared to existing MCU libraries: TF-Lite Micro, CMSIS-NN, and MicroTVM. Our study suggests that the era of tiny machine learning on IoT devices has arrived.

Locked Interactive transcript