An argument from the ethicist
Wool is a complicated fabric because shearing can be a humane, cruelty-free process when done the right way. Whereas wild sheep naturally shed their thick wooly coats in spring, domesticated sheep have been bred to have an unnaturally thick coat that never stops growing. So, for those sheep, shearing is critical: it allows them to better regulate their body temperature, it prevents parasites and infections, and it can give them greater mobility by preventing overgrowth. (Though, of course, if we didn't breed the sheep to have wool that never stops growing in the first place, the need to shear wouldn't exist).
But the reality of the wool industry is more troubling than a simple trim. Because profit is the primary motivator, shearers are typically paid by the volume of wool they produce — not by the number of hours they work. The result is a process that emphasizes speed, very often at the expense of the animal. Mulesing, the practice of skinning the flesh off of a sheep's hind quarters (without anesthesia) to combat flies is a prime example — but it goes beyond sanctioned practices, general violence is commonplace as well. Multiple undercover investigations by PETA, (most notably those done in 2014 and in 2018), surfaced video footage of widespread animal abuse, including kicking, stomping on, and punching the animals, inflicting blunt forced trauma, and causing gaping wounds to skin, ears and genitalia.  This is not to say that all shearers treat their animals this way — undoubtedly many are thoughtful caregivers. But commoditizing an animal unavoidably shifts a person's lens from 'I'm handling a life' to 'I'm handling a product and a paycheck' — a perspective hard-coated by the animal's eventual fate of being slaughtered for meat at the end of his or her product-bearing tenure (as is usually the case with sheep). This objectifying mentality is the underlying problem with all animal industries, and the reason violence surfaces in practices like shearing that should, in theory, be harmless.
An argument from the environmentalist
"An expansion of pastureland for goats used in the cashmere industry is the primary cause of the decline in Central Asian wild animal populations. Many of these animals native to China’s Tibetan plateau, Mongolia and India are already endangered."
The expansion of pastureland poses substantial environmental concerns, including deforestation and degradation of native species' habitats. A 2013 study titled, “Globalization of the Cashmere Market and the Decline of Large Mammals in Central Asia,”  published in the scientific journal, Conservation Biology, revealed that an expansion of pastureland for goats used in the cashmere industry is the primary cause of the decline in Central Asian wild animal populations. Many of these animals native to China’s Tibetan plateau, Mongolia and India are already endangered. 
Compounding those problems, as with all livestock industries, the wool industry leaves behind a major carbon footprint. In fact, according to a 2017 study on the relative environmental impact of a wide variety of materials used in the fashion industry, wool was one of the very worst (ranked fifth after leather, silk, cotton, and bast fiber) because of sheep's heavy greenhouse gas emissions. 
ConclusionPerhaps there was a time when using animals to create textiles was the only and best option. But we're past that — well, we're not, but we should be. We have incredible synthetic alternatives and the looming commercialization of lab-grown biofabrics. With the incumbent industry utilizing millions of acres of land around the world, producing immeasurable toxic waste and carbon emissions, and causing suffering to billions of living creatures, doesn't the alternative sound more appealing?
 PETA,"PETA's Seventh Wool Expose: Sheep Still Hit, Kicked, Cut, Thrown," 2018.
 Conservation Biology, "Globalization of the Cashmere Market and the Decline of Large Mammals in Central Asia," July 2013.
 One Green Planet, "Central Asian Animals are Becoming Victims of Fashion," 2013.
 Global Fashion Agenda and Boston Consulting Group, "2017 Pulse of the Fashion Industry."