On-device deep neural network (DNN) inference is often desirable for user experience and privacy. Existing solutions have fully utilized resources to minimize inference latency. However, they result in severe energy inefficiency by completing DNN inference much earlier than the required service interval. It poses a new challenge of how to make DNN inferences in a punctual and energy-efficient manner. To tackle this challenge, we propose a new resource allocation strategy for DNN processing, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">punctual laziness</i> that disperses its workload as efficiently as possible over time within its strict delay constraint. This strategy is particularly beneficial for neural workloads since a DNN comprises a set of popular operators whose latency and energy consumption are predictable. Through this understanding, we propose NeuroBalancer, an operator-aware core and memory frequency scaling framework that balances those frequencies as efficiently as possible while making timely inferences. We implement and evaluate NeuroBalancer on off-the-shelf Android devices with various state-of-the-art DNN models. Our results show that NeuroBalancer successfully meets a given inference latency requirements while saving energy consumption up to 43.9% and 21.1% compared to the Android's default governor and up to 42.1% and 18.6% compared to SysScale, the state-of-the-art mobile governor on CPU and GPU, respectively.

In this work, we explore the possibility of a new delivery method for lossless data, namely compressive transmission. It aims at minimizing the transmission data volume at runtime by exploiting the tailored information shared between the sender and the receiver. There are two approaches to leverage shared information for compression: 1) using a DNN-based codec as a proxy for shared information and 2) applying redundancy elimination using deduplication. However, these approaches have not been studied in depth to utilize the trade-off between the compression rate and the amount of shared information. Compared to these approaches, compressive transmission is unique as it fully leverages the abundance of information available on both sides, which is chosen and placed purposely. To bring the concept to reality, we propose nCTX, a neural network-powered Compressive Transmission System that adaptively exploits a generative model and matching blocks. nCTX extracts the optimal semantic data from the input data, exploiting shared information to closely imitate the original and compensate it with the offset (i.e., difference). Extensive evaluations in mobile platforms confirm that nCTX reduces the transmission volume significantly by 25.8% and 23.3% compared to FLIF and RC, the state-of-the-art image codecs, respectively, in comparable or shorter computation times.