Abstract In magnetic resonance imaging (MRI), variations in scan parameters and scanner specifications can result in differences in image appearance. To minimize these differences, harmonization in MRI has been suggested as a crucial image processing technique. In this study, we developed an MR physics-based harmonization framework, Physics-Constrained Deep Neural Network for Multisite and multiscanner Harmonization (PhyCHarm). PhyCHarm includes two deep neural networks: (1) the Quantitative Maps Generator to generate T 1 - and M 0 -maps and (2) the Harmonization Network. We used an open dataset consisting of 3T MP2RAGE images from 50 healthy individuals for the Quantitative Maps Generator and a traveling dataset consisting of 3T T 1 w images from 9 healthy individuals for the Harmonization Network. PhyCHarm was evaluated using the structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and normalized-root-mean square error (NRMSE) for the Quantitative Maps Generator, and using SSIM, PSNR, and volumetric analysis for the Harmonization network, respectively. PhyCHarm demonstrated increased SSIM and PSNR, the highest Dice score in the FSL FAST segmentation results for gray and white matter compared to U-Net, Pix2Pix, and CALAMITI. PhyCHarm showed a greater reduction in volume differences after harmonization for gray and white matter than U-Net, Pix2Pix, or CALAMITI. As an initial step toward developing advanced harmonization techniques, we investigated the applicability of physics-based constraints within a supervised training strategy. The proposed physics constraints could be integrated with unsupervised methods, paving the way for more sophisticated harmonization qualities.