Mobile robots operating in outdoor environments face the challenge of navigating various terrains with different degrees of difficulty. Therefore, traversability estimation is crucial for safe and efficient robot navigation. Current approaches utilize a robot's driving experience to learn traversability in a self-supervised fashion. However, providing sufficient and diverse experience to the robot is difficult in many practical applications. In this paper, we propose a self-supervised traversability learning method that adapts to challenging terrains with limited prior experience. One key aspect is to enable prioritized learning of scarce yet high-risk terrains by using a risk-sensitive approach. To this end, we train a neural network through a risk-aware instance weighting scheme. Another key aspect is to leverage traversability pseudo-labels on the basis of a self-training scheme. The proposed confidence-regularized self-training generates high-quality pseudo-labels, thereby achieving reliable data augmentation for unexperienced terrains. The effectiveness of the proposed method is verified in extensive real-world experiments, ranging from structured urban environments to complex rugged terrains.