Interactive visual navigation tasks, which involve following instructions to reach and interact with specific targets, are challenging not only because successful experiences are very rare but also because complex visual inputs require a substantial number of samples. Previous methods for these tasks often rely on intricately designed dense rewards or the use of expensive expert data for imitation learning. To tackle these challenges, we propose a novel approach, Visual Hindsight Self-Imitation Learning (VHS), which enables re-labeling in vision-based and partially observable environments through Prototypical Goal (PG) embedding. We introduce the PG embeddings, which are derived from experienced goal observations, as opposed to handling instructions as word embeddings. This embedding technique allows the agent to visually reinterpret its unsuccessful attempts, enabling vision-based goal re-labeling and self-imitation from enhanced successful experiences. Experimental results show that VHS outperforms existing techniques in interactive visual navigation tasks, confirming its superior performance, sample efficiency, and generalization.

Interactive visual navigation tasks, which involve following instructions to reach and interact with specific targets, are challenging not only because successful experiences are very rare but also because complex visual inputs require a substantial number of samples. Previous methods for these tasks often rely on intricately designed dense rewards or the use of expensive expert data for imitation learning. To tackle these challenges, we propose a novel approach, Visual Hindsight Self-Imitation Learning (VHS), which enables re-labeling in vision-based and partially observable environments through Prototypical Goal (PG) embedding. We introduce the PG embeddings, which are derived from experienced goal observations, as opposed to handling instructions as word embeddings. This embedding technique allows the agent to visually reinterpret its unsuccessful attempts, enabling vision-based goal re-labeling and self-imitation from enhanced successful experiences. Experimental results show that VHS outperforms existing techniques in interactive visual navigation tasks, confirming its superior performance, sample efficiency, and generalization.

Teaching robots desired skills in real-world environments remains challenging, especially for non-experts.A key bottleneck is that collecting robotic data often requires expertise or specialized hardware, limiting accessibility and scalability.We posit that natural language offers an intuitive and accessible interface for robot learning.To this end, we study two aspects:(1) enabling non-experts to collect robotic data through natural language supervision (e.g., "move the arm to the right") and ( 2) training robot policies directly from this supervision.Specifically, we introduce a data collection framework that collects robot demonstrations based on natural language supervision and further augments these demonstrations.We then present CLIP-RT, a new vision-language-action (VLA) model that learns language-conditioned visuomotor policies from this supervision.CLIP-RT adapts the pretrained CLIP model and learns to predict language-based motion primitives via contrastive imitation learning.We train CLIP-RT on the Open X-Embodiment dataset and finetune it on in-domain data collected by our framework.In realworld evaluations, CLIP-RT demonstrates strong capabilities in learning novel manipulation skills, outperforming OpenVLA (7B parameters) by 24% in average success rates, while using 7x fewer parameters (1B).We further assess CLIP-RT's capabilities in few-shot generalization and collaborative scenarios involving large pretrained models or humans.In simulated environments, CLIP-RT also yields strong performance, achieving a 92.8% average success rate on the LIBERO benchmark with an inference throughput of 163 Hz.

Concept bottleneck models (CBMs) are inherently interpretable models that make predictions based on human-understandable visual cues, referred to as concepts. As obtaining dense concept annotations with human labeling is demanding and costly, recent approaches utilize foundation models to determine the concepts existing in the images. However, such label-free CBMs often fail to localize concepts in relevant regions, attending to visually unrelated regions when predicting concept presence. To this end, we propose a framework, coined Locality-aware Concept Bottleneck Model (LCBM), which utilizes rich information from foundation models and adopts prototype learning to ensure accurate spatial localization of the concepts. Specifically, we assign one prototype to each concept, promoted to represent a prototypical image feature of that concept. These prototypes are learned by encouraging them to encode similar local regions, leveraging foundation models to assure the relevance of each prototype to its associated concept. Then we use the prototypes to facilitate the learning process of identifying the proper local region from which each concept should be predicted. Experimental results demonstrate that LCBM effectively identifies present concepts in the images and exhibits improved localization while maintaining comparable classification performance.

Reinforcement learning (RL) agents often face challenges in balancing exploration and exploitation, particularly in environments where sparse or dense rewards bias learning. Biological systems, such as human toddlers, naturally navigate this balance by transitioning from free exploration with sparse rewards to goal-directed behavior guided by increasingly dense rewards. Inspired by this natural progression, we investigate the Toddler-Inspired Reward Transition in goal-oriented RL tasks. Our study focuses on transitioning from sparse to potential-based dense (S2D) rewards while preserving optimal strategies. Through experiments on dynamic robotic arm manipulation and egocentric 3D navigation tasks, we demonstrate that effective S2D reward transitions significantly enhance learning performance and sample efficiency. Additionally, using a Cross-Density Visualizer, we show that S2D transitions smooth the policy loss landscape, resulting in wider minima that improve generalization in RL models. In addition, we reinterpret Tolman's maze experiments, underscoring the critical role of early free exploratory learning in the context of S2D rewards.

본 논문은 최신 마인크래프트 버전(1.21)을 기반으로 한 새로운 강화학습 환경 CraftGround를 소개한다. CraftGround는 유연한 실험 설정을 제공하며, 복잡한 3D 환경에서의 강화학습을 지원한다. 시각적 데이터, 소리 신호, 생물 군계 정보, 게임 내 통계와 같은 다양한 관측 정보를 제공하여 에이전트의 성능을 다각도로 평가할 수 있다. 본 연구는 나무 채취, 적대적 몬스터 회피, 낚시와 같은 여러 태스크에서 VPT, PPO, RecurrentPPO, DQN 에이전트를 평가하였다. VPT는 사전 학습 덕분에 높은 성능과 효율성을 보였으며, PPO 및 RecurrentPPO와 같은 온라인 학습 알고리즘은 환경 변화에 적응하며 시간이 지남에 따라 성능이 향상되었다. 이 결과는 CraftGround가 동적 3D 시뮬레이션에서 적응적 에이전트 행동 연구를 촉진할 가능성을 보여준다.

As the field of robotics advances, Embodied Instruction Following (EIF) has emerged as a key challenge in artificial intelligence. EIF tasks require agents to interpret and execute natural language instructions by predicting and completing sequences of subgoals within a physical environment. Traditional evaluation metrics—such as Success Rate (SR), Goal Condition (GC), Path Length Weighted SR (PLWSR), and Path Length Weighted GC (PLWGC)—primarily focus on task success following low-level control actions. However, these metrics inadequately assess the accuracy of high-level planning, which is critical for overall task performance. Existing methods for evaluating high-level planning often rely on comparing predicted plans to a single human-annotated ground truth trajectory, implicitly assuming the existence of only one correct solution. In practice, many instructions allow for multiple valid trajectories that can achieve the same goal. To address these limitations, we propose Relaxed HLP, a novel metric designed to evaluate high-level planning more flexibly by accounting for alternative valid plans. Relaxed HLP introduces three key considerations: temporal agnosticism, spatial agnosticism, and interchangeable actions, thus enabling a more comprehensive assessment of high-level plan accuracy. We validate the effectiveness of Relaxed HLP through human evaluations, demonstrating that it aligns more closely with human judgment compared to traditional ground truth-based metrics. Our results underscore the robustness of Relaxed HLP in capturing diverse, semantically equivalent plans, offering a more accurate assessment of high-level planning in EIF tasks.

Action recognition models have achieved promising results in understanding instructional videos. However, they often rely on dominant, dataset-specific action sequences rather than true video comprehension, a problem that we define as ordinal bias. To address this issue, we propose two effective video manipulation methods: Action Masking, which masks frames of frequently co-occurring actions, and Sequence Shuffling, which randomizes the order of action segments. Through comprehensive experiments, we demonstrate that current models exhibit significant performance drops when confronted with nonstandard action sequences, underscoring their vulnerability to ordinal bias. Our findings emphasize the importance of rethinking evaluation strategies and developing models capable of generalizing beyond fixed action patterns in diverse instructional videos.