That's why you don't tell anyone in your family that you are going to do anything with the joint account. If they aren't with you every minute of every day, they aren't going to know what you're doing, and thus can't tell your ex what's up. True, you can't do anything about how they are, but you CAN do something about how you are, and as an adult who apparently isn't under guardianship and apparently not locked up anywhere, I don't see how they can stop you from going about your daily business just by being "controlling". As an adult, it's up to you to break free of their control and get on with your own life. I don't know how old you are, but if you were in a relationship with your ex for 16 years then clearly you are not a child. They can only control you as much as you allow yourself to be controlled.
Even if you don't own a car, it seems like there should be some way for you to get to the bank outside of relying on your friend for a ride. How do you get anywhere else? You mentioned your brother yelling at you in a restaurant so you must have had some way to get to the restaurant. I don't drive but I manage to get everywhere I need to go by using public transportation or walking.