On-device deep neural network (DNN) inference is often desirable for user experience and privacy. Existing solutions have fully utilized resources to minimize inference latency. However, they result in severe energy inefficiency by completing DNN inference much earlier than the required service interval. It poses a new challenge of how to make DNN inferences in a punctual and energy-efficient manner. To tackle this challenge, we propose a new resource allocation strategy for DNN processing, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">punctual laziness</i> that disperses its workload as efficiently as possible over time within its strict delay constraint. This strategy is particularly beneficial for neural workloads since a DNN comprises a set of popular operators whose latency and energy consumption are predictable. Through this understanding, we propose NeuroBalancer, an operator-aware core and memory frequency scaling framework that balances those frequencies as efficiently as possible while making timely inferences. We implement and evaluate NeuroBalancer on off-the-shelf Android devices with various state-of-the-art DNN models. Our results show that NeuroBalancer successfully meets a given inference latency requirements while saving energy consumption up to 43.9% and 21.1% compared to the Android's default governor and up to 42.1% and 18.6% compared to SysScale, the state-of-the-art mobile governor on CPU and GPU, respectively.

In this work, we explore the possibility of a new delivery method for lossless data, namely compressive transmission. It aims at minimizing the transmission data volume at runtime by exploiting the tailored information shared between the sender and the receiver. There are two approaches to leverage shared information for compression: 1) using a DNN-based codec as a proxy for shared information and 2) applying redundancy elimination using deduplication. However, these approaches have not been studied in depth to utilize the trade-off between the compression rate and the amount of shared information. Compared to these approaches, compressive transmission is unique as it fully leverages the abundance of information available on both sides, which is chosen and placed purposely. To bring the concept to reality, we propose nCTX, a neural network-powered Compressive Transmission System that adaptively exploits a generative model and matching blocks. nCTX extracts the optimal semantic data from the input data, exploiting shared information to closely imitate the original and compensate it with the offset (i.e., difference). Extensive evaluations in mobile platforms confirm that nCTX reduces the transmission volume significantly by 25.8% and 23.3% compared to FLIF and RC, the state-of-the-art image codecs, respectively, in comparable or shorter computation times.

On-device deep neural network (DNN) inference is often desirable for user experience and privacy. Existing solutions have fully utilized resources to minimize inference latency. However, they result in severe energy inefficiency by completing DNN inference much earlier than the required service interval. It poses a new challenge of how to make DNN inferences in a punctual and energy-efficient manner. To tackle this challenge, we propose a new resource allocation strategy for DNN processing, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">punctual laziness</i> that disperses its workload as efficiently as possible over time within its strict delay constraint. This strategy is particularly beneficial for neural workloads since a DNN comprises a set of popular operators whose latency and energy consumption are predictable. Through this understanding, we propose NeuroBalancer, an operator-aware core and memory frequency scaling framework that balances those frequencies as efficiently as possible while making timely inferences. We implement and evaluate NeuroBalancer on off-the-shelf Android devices with various state-of-the-art DNN models. Our results show that NeuroBalancer successfully meets a given inference latency requirements while saving energy consumption up to 43.9% and 21.1% compared to the Android's default governor and up to 42.1% and 18.6% compared to SysScale, the state-of-the-art mobile governor on CPU and GPU, respectively.

In this work, we explore the possibility of a new delivery method for lossless data, namely compressive transmission. It aims at minimizing the transmission data volume at runtime by exploiting the tailored information shared between the sender and the receiver. There are two approaches to leverage shared information for compression: 1) using a DNN-based codec as a proxy for shared information and 2) applying redundancy elimination using deduplication. However, these approaches have not been studied in depth to utilize the trade-off between the compression rate and the amount of shared information. Compared to these approaches, compressive transmission is unique as it fully leverages the abundance of information available on both sides, which is chosen and placed purposely. To bring the concept to reality, we propose nCTX, a neural network-powered Compressive Transmission System that adaptively exploits a generative model and matching blocks. nCTX extracts the optimal semantic data from the input data, exploiting shared information to closely imitate the original and compensate it with the offset (i.e., difference). Extensive evaluations in mobile platforms confirm that nCTX reduces the transmission volume significantly by 25.8% and 23.3% compared to FLIF and RC, the state-of-the-art image codecs, respectively, in comparable or shorter computation times.

In this paper, we conduct a measurement study on operational 5G networks deployed across different frequency bands (mmWave and sub-6GHz) and server locations (mobile edge and Internet cloud). Specifically, we assess 5G performance in both uplink and downlink across multiple operators’ networks. We then carry out extensive comparisons of transport-layer protocols using ten different algorithms in full-fledged 5G networks, including an edge computing environment. Finally, we evaluate representative mobile applications over the 5G network with and without edge servers. Our comprehensive measurements provide several insights that affect the experience of 5G users: (i) With a 5G edge server, existing TCP congestion control algorithms can achieve throughput up to 1.8Gbps with only a single flow. (ii) The maximum TCP receive buffer size, which is set by off-the-shelf 5G phones, can limit the throughput performance of 5G networks, which is not observed in 4G LTE-A networks. (iii) Despite significant latency gains in download-centric applications, the 5G edge service provides limited benefits to CPU-intensive tasks or those that use significant uplink bandwidth. To our knowledge, this is the first measurement-driven understanding of 5G edge computing “in the wild,” which can provide an answer to how edge computing would perform in real 5G networks.

The growing adoption of deep neural network (DNN) inference in mobile applications underscores the need for edge-assisted inference to meet the latency requirements of diverse DNN tasks, given the constrained device capabilities. However, widespread deployment remains hindered by unreliable latency due to the shared nature of cellular networks, where concurrent workloads contend across uplink, computation, and downlink stages. Although several approaches have been proposed, they primarily target specific tasks (e.g., video analytics) and lack support for diverse DNN tasks due to their uplink-centric designs. This paper presents CORA, a system that serves latency-critical DNN inference requests through end-to-end coordination between the RAN and the edge server. CORA dynamically adjusts the latency budgets across stages based on each DNN task's characteristics, balancing resource demands for the radio and compute domains to mitigate contention at each stage. It then aligns the resource schedulers with these per-stage budgets, thereby enabling end-to-end coordination without requiring modifications to end hosts. We prototype and evaluate CORA on an over-the-air testbed with diverse DNN tasks. CORA serves 3.2× more requests within the latency target and reduces the 95th percentile latency by 2.1× compared to baselines.

The Internet's current security landscape remains fundamentally compromised by inherent protocol vulnerabilities. Existing security solutions have been narrowly focused, addressing isolated protocols or specific attack vectors without developing a comprehensive security framework. Our proposed certificate-centric security oracle represents a paradigm shift, offering universal security services that enable protocols to robustly attest their identities and authentically validate their communications. The security oracle operates in a cross-layer fashion so that any protocol (in any layer) that wishes to use such services can be extended individually with minor efforts (say, adding a few fields to carry a certificate or a signature). We also present a few case studies illustrating how existing protocols can be extended to use the services of the security oracle. Prototype-based experiments are carried out for ICMP and DHCP to demonstrate the practical feasibility of the proposed framework.

The end-to-end latency requirement in mobile applications is not limited to the time taken from the server to the user device. The perceived latency by the users also includes the time taken for the data to traverse from the mobile operating system's kernel to the application layer. The kernels of modern mobile devices are designed to reduce the network packet processing delay by enqueuing incoming data in socket buffer and quickly dequeueing socket buffer for immediate data delivery to the application layer. However, this approach results in unnecessary CPU load on the mobile device due to the frequent execution of socket buffer dequeueing. We propose ADQ, which utilizes Application Data Unit (ADU) information—the smallest data unit interpretable by the application—in the mobile kernel to conduct minimal socket buffer dequeueing, aiming to reduce unnecessary CPU load without degrading delay performance. Our evaluation in a real-time video streaming scenario shows that ADQ can maintain a minimal level of latency using a restricted amount of CPU resources. Furthermore, in the presence of high CPU loads from competing tasks, we demonstrate that ADQ can improve delay performance compared to the default mobile kernel.

Offloading neural network inferences from resource-constrained mobile devices to an edge server over wireless networks is becoming more crucial as the neural networks get heavier. To this end, recent studies have tried to make this offloading process more efficient. However, the most fundamental question on extracting and offloading the minimal amount of necessary information that does not degrade the inference accuracy has remained unanswered. We call such an ideal offloading semantic offloading and propose N-epitomizer, a new offloading framework that enables semantic offloading, thus achieving more reliable and timely inferences in highly-fluctuated or even low-bandwidth wireless networks. To realize N-epitomizer, we design an autoencoder-based scalable encoder trained to extract the most informative data and scale its output size to meet the latency and accuracy requirements of inferences over a network. We also accelerate N-epitomizer by exploiting light-weight knowledge distillation for the encoder design and decoder slimming for the decoder design, reducing its overall computation time significantly. Moreover, we extend our N-epitomizer to support multiple DNNs by extracting and offloading the union of the essential information required for each DNN. Our evaluation shows that N-epitomizer achieves exceptionally high compression for images without compromising inference accuracy, which is 21 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> , 77 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> , and 192 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> higher than JPEG compression, and 20 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> , 55 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> , and 86 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> higher than the state-of-the-art DNN-aware image compression GRACE for semantic segmentation, depth estimation, and classification, respectively. Our results show N-epitomizer’s strong potential as the first semantic offloading system to guarantee end-to-end latency even under highly varying cellular networks.

In cellular networks, edge intelligence is often enabled by MultiAccess Edge Computing (MEC), which aims to bring AI services closer to end users. Although MEC reduces latency by placing computation near the network edge, it remains external to the Radio Access Network (RAN) and its native execution environment, thereby introducing additional transport and buffering delays. The emerging AI-on-RAN paradigm proposes to overcome these limitations but remains largely conceptual, lacking practical implementation and feasibility validation. In this paper, we present AoRA, the first framework realizing the AI-on-RAN vision by dynamically utilizing available computational headroom in GPU- and NPU-accelerated RAN platforms to deliver AI services directly from within the base stations. AoRA leverages containerized AI workloads in the 5G RAN stack to enable inlined and opportunistic AI service provisioning without degrading core telecom operations. The framework is fully compliant with O-RAN interfaces and can operate seamlessly alongside existing edge computing infrastructures. Evaluations show that AoRA reduces transport latency by over 30% compared to MEC and 70% compared to cloud-based setups.

With the advent of large language models (LLMs), numerous Post-Training Quantization (PTQ) strategies have been proposed to alleviate deployment barriers created by their enormous parameter counts. Quantization achieves compression by limiting the number of representable points in the data. Therefore, the key to achieving efficient quantization is selecting the optimal combination of representation points, or codes, for the given data. Existing PTQ solutions adopt two major approaches to this problem: Round-To-Nearest (RTN)-based methods and codebook-based methods. RTN-based methods map LLM weights onto uniformly distributed integer grids, failing to account for the Gaussian-like weight distribution of LLM weights. Codebook-based methods mitigate this issue by constructing distribution-aware codebooks; however, they suffer from random and strided memory access patterns, resulting in degraded inference speed that is exacerbated by the limited size of GPU L1 cache. To overcome these limitations, we propose a novel LLM quantization method, SBVR (Summation of BitVector Representation), that enables Gaussian-like code representation in a hardware-friendly manner for fast inference. SBVR maps weight values to non-uniform representation points whose distribution follows the actual distribution of LLM weights, enabling more accurate compression. Additionally, we design a custom CUDA kernel that allows matrix-vector multiplication directly in the SBVR format without decompression, thereby enabling high-performance execution of SBVR-compressed models. Our evaluations of SBVR on various models demonstrate state-of-the-art perplexity and accuracy benchmark performance while delivering a 2.21x- 3.04x end-to-end token-generation speedup over naive FP16 models in the 4-bit quantization regime.