Smart speakers become a versatile and convenient device to provide digital contents in our daily life. However, given the challenges of privacy management in smart speakers, it is required to examine how users cope with privacy threats of the devices. In this study, we examine how the privacy-enhancing attributes of smart speakers influence users’ preferences when choosing devices. The preference structure was generated from a conjoint approach, dealing with four privacy-enhancing features (parasite function, data storage method, security notice, and speaker recognition function) and two typical factors affecting consumer decision making (brand and price). Data collected from a survey of 516 consumers identified that, aside from the attribute of price, parasite function was ranked as the most preferred attribute. We also compared the findings across three user clusters. By examining concrete and explicit attributes regarding privacy-enhancing features, the present study provides novel insights into users’ understanding of privacy.