Yes he can. If you take the kids out of state, he can file for custody and return of the kids in your current state since the current state will have jurisdiction until you meet the residency requirements.
your best bet is to find a way to leave NOW and not take the kids out of state. If I were you, call legal aid and see about filing for custody.
Why did you have 4 kids with a man you are not married to? Is he the father of all 4 children?