hmm... Well first I would probably go through and donate anything that doesn't fit you. Start there.
it will help you get a handle on what you have to work with.
From there, you're right - it really depends! What's your climate, how often do you do laundry, what are your daily activities, etc.
I always have more shirts than pants/skirts. I like to have 2-3 pairs of jeans that fit.They to put together outfits out of the same pieces to maximize your wardrobe without having too many clothes- in other words, things you can mix and match for more options.
Sorry if that's not super specific, lol. ITU about hanging on to too many clothes, but it sure feels good when you don't have to dig through the things you don't wear to get to the good stuff.