First, if it doesn't fit, get rid of it.
Second, if you don't like it, why keep it?
Third, if it's an issue of bf'ing etc then by all means box it up and stop wasting space in your closet when you can't enjoy it
Fourth, what do you do? What is your life about? Keep a few clothes that go outside that (nice dress if you occasionally go out but not often, suit just in case, etc)