This paper proposes a novel reinforcement learning (RL) approach for internet network management (NM) that facilitates the RL agent can handle dynamic preference scenario. Traditional RL-based NM methods typically use fixed preferences to optimize multiple objectives like quality of service (QoS) and computing resource usage. However, in real-world scenarios, preferences of the multiple objectives can be changed dynamically due to factors such as network overloads or server failures. We present a method that allows RL agents to adapt to dynamic preferences during testing. Our experiments show that the proposed method significantly improves generalizability across various network preferences compared to previous RL methods, offering a more efficient and flexible solution for NM.