This paper proposes a first-order optimization framework for nonlinear optimal control problems, efficiently handling complex dynamics via projection onto a lifted, approximately linear constraint manifold constructed using a physics-informed deep Koopman operator. By circumventing repeated convex programming and avoiding penalty-based refinements, the algorithm mitigates sensitivity to hyperparameters and reduces reliance on domain-specific knowledge and manual modeling. A physics-informed loss function preserves physical consistency when mapping back to the original space, enabling fast convergence to near-optimal solutions. Experiments demonstrate improved computational efficiency and stability over established sequential programming approaches.