In "Dragonfly in Amber" by Diana Gabaldon, the character expresses a disdain for the superficial aspects of society that often accompany the wearing of nice clothing. They prioritize authentic communication and personal freedom over adhering to societal expectations and the pressures of high-class life.
The sentiment suggests that the stress and constraints of gossip, social schemes, and trivial etiquette are not worth the appeal of dressing well. The speaker ultimately values personal expression and simplicity over the complexities of social standing and expectations.