Buying more carefully only tweaks the consumer end of a system whose cheap clothes rest on low-paid, mostly female factory labour. Voluntary individual action is too small a response to change it, and is unlikely to reduce poverty widely.
When a flood of cheap second-hand clothing arrives, people stop buying locally made garments. Factory workers lose their jobs, and the textile industries that poorer countries built after gaining independence are pushed out. Banning the trade alone would not bring those industries back.
A single garment can travel in a circle. Cheap cotton is grown in a poor region, sewn into clothes in low-wage factories, sold in rich countries, and then shipped back as used goods to the region where the cotton was grown. Each step lets someone earn a return, and the unevenness between places is built into how the system works.
Second-hand clothing traders buy tightly wrapped bales whose contents they mostly cannot see. Whether they earn a living depends on luck: getting a bundle with styles that happen to sell in their local market.
Clothes given away in rich countries are not handed to people in need. They are sold in bulk to traders, then to street sellers, and then to shoppers in poorer countries. Charities sit inside the same profit-seeking chain as fashion retailers.