ABSTRACT Knowledge distillation is a widely adopted model compression method aimed at narrowing the performance gap between a high‐capacity teacher network and a lightweight student network. However, in the context of sensor fusion‐based 3D object detection, existing distillation methods predominantly emphasize accuracy enhancement through the introduction of multiple loss functions, which often leads to overly complex training procedures. To address this limitation, we propose a sensor fusion‐based feature distillation framework tailored for camera and radar modalities. Our proposed method utilizes an autoencoder to facilitate efficient knowledge transfer from the teacher to the student model. Additionally, we introduce image‐context and radar‐context knowledge distillation strategies to capture and transfer modality‐specific features effectively. We demonstrate the effectiveness of the proposed method on the nuScenes dataset using a ResNet‐based architecture.

The dominant backbones of neural networks for scene parsing consist of multiple stages, where feature maps in different stages often contain varying levels of spatial and semantic information. High-level features convey more semantics and fewer spatial details, while low-level features possess fewer semantics and more spatial details. Consequently, there are semantic-spatial gaps among features at different levels, particularly in human parsing tasks. Many existing approaches directly upsample multi-stage features and aggregate them through addition or concatenation, without addressing the semantic-spatial gaps present among these features. This inevitably leads to spatial misalignment, semantic mismatch, and ultimately misclassification in parsing, especially for human parsing that demands more semantic information and more fine details of feature maps for the reason of intricate textures, diverse clothing styles, and heavy scale variability across different human parts. In this paper, we effectively alleviate the long-standing challenge of addressing semantic-spatial gaps between features from different stages by innovatively utilizing the subtraction and addition operations to recognize the semantic and spatial differences and compensate for them. Based on these principles, we propose the Channel and Spatial Enhancement Network (CSENet) for parsing, offering a straightforward and intuitive solution for addressing semantic-spatial gaps via injecting high-semantic information to lower-stage features and vice versa, introducing fine details to higher-stage features. Extensive experiments on three dense prediction tasks have demonstrated the efficacy of our method. Specifically, our method achieves the best performance on the LIP and CIHP datasets and we also verify the generality of our method on the ADE20K dataset. • We propose the CSENet which effectively addresses the challenge of semantic and spatial gaps between feature maps from different stages in human parsing. By utilizing the operations of subtraction and addition to calculate and compensate the feature differences, CSENet reduces the semantic gaps and successfully introduce high-semantic information to low-level feature and fine details to high-level feature to benefit recognizing large objects and inconspicuous parts, especially in the context of human parsing. • We introduce CEM and SEM as the main components of CSENet. CEM employs average pooling, subtraction and addition to calculate and compensate semantic differences, while SEM utilizes similar operations to compute and compensate the spatial differences. These modules enhance the discriminative ability of feature representations, improving the recognition of fine details, inner patterns, and accurate spatial locations of human parts. • Our CSENet is shown to be effective and efficient in improving the performance of existing backbones. Our modules are general and can be easily integrated into existing architectures, enabling the effective assembly of feature maps from deep to shallow layers. Experimental results demonstrate the efficacy of our CSENet. Our method achieves SOTA performances on LIP and CIHP datasets without using pose information or the hierarchy structure of the class in the scene. We also validate the generality of our method via using the transformer as the backbone on the scene parsing dataset ADE20K.

Abstract In computer vision, human parsing is challenging due to its demand for accurate human region location and semantic partitioning. This dense prediction task needs powerful computation and high‐precision models. To enable real‐time parsing on resource‐limited devices, the authors introduced a lightweight model using ResNet18 as a core network . The authors simplified the pyramid module, improving context clarity and reducing complexity. The authors integrated a spatial attention fusion strategy to counter precision loss in the light‐weighting process. Traditional models, despite their segmentation precision, are limited by their computational complexity and extensive parameters. The authors implemented knowledge distillation (KD) techniques to enhance the authors’ lightweight network's accuracy. Traditional methods can fail to learn useful knowledge with significant network differences. Hence, the authors used a novel distillation approach based on inter‐class and intra‐class relations in prediction outcomes, noticeably improving parsing accuracy. The authors’ experiments on the Look into Person (LIP) dataset show that their lightweight model significantly reduces parameters while maintaining parsing precision and enhancing inference speed.

Test-time adaptation aims to adapt to realistic environments in an online manner by learning during test time. Entropy minimization has emerged as a principal strategy for test-time adaptation due to its efficiency and adaptability. Nevertheless, it remains underexplored in continual test-time adaptation, where stability is more important. We observe that the entropy minimization method often suffers from model collapse, where the model converges to predicting a single class for all images due to a trivial solution. We propose ranked entropy minimization to mitigate the stability problem of the entropy minimization method and extend its applicability to continuous scenarios. Our approach explicitly structures the prediction difficulty through a progressive masking strategy. Specifically, it gradually aligns the model's probability distributions across different levels of prediction difficulty while preserving the rank order of entropy. The proposed method is extensively evaluated across various benchmarks, demonstrating its effectiveness through empirical results. Our code is available at https://github.com/pilsHan/rem

Test-time adaptation paradigm provides flexibility towards domain shifts by performing immediate adaptation on unlabeled target data from the source model. Vision-Language Models (VLMs) leverage their generalization capabilities for diverse downstream tasks, and test-time prompt tuning has emerged as a prominent solution for adapting VLMs. In this work, we explore contrastive VLMs and identify the modality gap caused by a single dominant feature dimension across modalities. We observe that the dominant dimensions in both text and image modalities exhibit high predictive sensitivity, and that constraining their influence can improve calibration error. Building on this insight, we propose dimensional entropy maximization that regularizes the distribution of textual features toward uniformity to mitigate the dependency of dominant dimensions. Our method alleviates the degradation of calibration performance in test-time prompt tuning, offering a simple yet effective solution to enhance the reliability of VLMs in real-world deployment scenarios.

Groups at MIT and NYU have collected a dataset of millions of tiny colour images from the web.It is, in principle, an excellent dataset for unsupervised training of deep generative models, but previous researchers who have tried this have found it dicult to learn a good set of lters from the images.We show how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex.Using a novel parallelization algorithm to distribute the work among multiple machines connected on a network, we show how training such a model can be done in reasonable time.A second problematic aspect of the tiny images dataset is that there are no reliable class labels which makes it hard to use for object recognition experiments.We created two sets of reliable labels.The CIFAR-10 set has 6000 examples of each of 10 classes and the CIFAR-100 set has 600 examples of each of 100 non-overlapping classes.Using these labels, we show that object recognition is signicantly improved by pre-training a layer of features on a large set of unlabeled tiny images.

ABSTRACT Knowledge distillation is a widely adopted model compression method aimed at narrowing the performance gap between a high‐capacity teacher network and a lightweight student network. However, in the context of sensor fusion‐based 3D object detection, existing distillation methods predominantly emphasize accuracy enhancement through the introduction of multiple loss functions, which often leads to overly complex training procedures. To address this limitation, we propose a sensor fusion‐based feature distillation framework tailored for camera and radar modalities. Our proposed method utilizes an autoencoder to facilitate efficient knowledge transfer from the teacher to the student model. Additionally, we introduce image‐context and radar‐context knowledge distillation strategies to capture and transfer modality‐specific features effectively. We demonstrate the effectiveness of the proposed method on the nuScenes dataset using a ResNet‐based architecture.

The dominant backbones of neural networks for scene parsing consist of multiple stages, where feature maps in different stages often contain varying levels of spatial and semantic information. High-level features convey more semantics and fewer spatial details, while low-level features possess fewer semantics and more spatial details. Consequently, there are semantic-spatial gaps among features at different levels, particularly in human parsing tasks. Many existing approaches directly upsample multi-stage features and aggregate them through addition or concatenation, without addressing the semantic-spatial gaps present among these features. This inevitably leads to spatial misalignment, semantic mismatch, and ultimately misclassification in parsing, especially for human parsing that demands more semantic information and more fine details of feature maps for the reason of intricate textures, diverse clothing styles, and heavy scale variability across different human parts. In this paper, we effectively alleviate the long-standing challenge of addressing semantic-spatial gaps between features from different stages by innovatively utilizing the subtraction and addition operations to recognize the semantic and spatial differences and compensate for them. Based on these principles, we propose the Channel and Spatial Enhancement Network (CSENet) for parsing, offering a straightforward and intuitive solution for addressing semantic-spatial gaps via injecting high-semantic information to lower-stage features and vice versa, introducing fine details to higher-stage features. Extensive experiments on three dense prediction tasks have demonstrated the efficacy of our method. Specifically, our method achieves the best performance on the LIP and CIHP datasets and we also verify the generality of our method on the ADE20K dataset. • We propose the CSENet which effectively addresses the challenge of semantic and spatial gaps between feature maps from different stages in human parsing. By utilizing the operations of subtraction and addition to calculate and compensate the feature differences, CSENet reduces the semantic gaps and successfully introduce high-semantic information to low-level feature and fine details to high-level feature to benefit recognizing large objects and inconspicuous parts, especially in the context of human parsing. • We introduce CEM and SEM as the main components of CSENet. CEM employs average pooling, subtraction and addition to calculate and compensate semantic differences, while SEM utilizes similar operations to compute and compensate the spatial differences. These modules enhance the discriminative ability of feature representations, improving the recognition of fine details, inner patterns, and accurate spatial locations of human parts. • Our CSENet is shown to be effective and efficient in improving the performance of existing backbones. Our modules are general and can be easily integrated into existing architectures, enabling the effective assembly of feature maps from deep to shallow layers. Experimental results demonstrate the efficacy of our CSENet. Our method achieves SOTA performances on LIP and CIHP datasets without using pose information or the hierarchy structure of the class in the scene. We also validate the generality of our method via using the transformer as the backbone on the scene parsing dataset ADE20K.