Sure, and you could be arrested for custodial interference. And because you crossed a state line, that could bring in Federal Charges.
When they catch you, and they will, your rights can be terminated. At the very least, you will lose any custody you have, and will prob. only have limmitted visitation.(That's if you're really lucky).
Of course, if you are talking about a short trip for half a day, you would be okay, as long as no order is in place to prevent it. But, if the other parent knows, and you cross that line, you have been warned what COULD happen.