Photoacoustic imaging (PAI) has emerged as a molecular-selective imaging technology based on optical absorption contrast. Dichroism-sensitive photoacoustic (DS-PA) imaging has been reported, where the absorption coefficient has a vector characteristic, featuring dimensions of contrast in polarization and wavelength. Herein, we present a DS-PA microscopy (DS-PAM) system that implements optical anisotropy contrast and molecular selectivity. Moreover, we propose mathematical solutions to fully derive dichroic properties. A wavelength for the PAI of collagenous tissue was used, and the proposed algorithms were validated using linear dichroic materials. We successfully mapped dichroic information in fibrous tissue imaging based on the degree of anisotropy and axis orientation, and also deduced mechanical assessment from the tissue arrangement. The proposed DS-PAM system and algorithms have great potential in various diagnostic fields using polarimetry, such as musculoskeletal and cardiovascular systems.