Depletion of S6K1 signaling in the early stage of human brain development drives the formation of retinal cells distinct from cortical neurons. Our findings demonstrate that S6K1 signaling fine-tunes neuronal and retinal lineage specification during brain development.