Highly autonomous driving technology is expected to improve driving safety and convenience, and collision avoidance technology is essential for fully autonomous driving. Planning a collision-free trajectory that includes velocity and path is one of the most challenging objectives. Optimization-based trajectory planners have been proposed in many previous studies because they offer a high degree of freedom and can handle various situations. However, most previous trajectory planners used nonlinear programming due to the nonlinearity or non-convexity of the optimization problem. These methods come with a high computational load. The trajectory planner requires the real-time ability to cope with dynamically changing environments. This article focuses on the trajectory planning of autonomous vehicles through quadratic programming (QP), which requires a low computational load. To achieve this, we introduce the longitudinal-lateral decomposition method. In addition, collision-free constraints are expressed as linear constraints through proposed ingenious dual functions. The proposed weak duality optimization problem has a QP form and optimized trajectory and obstacle avoidance timing through only one QP problem. This study verified that the proposed trajectory planner could plan smooth collision-free maneuvers for several driving situations by simulations.