The fact that you do in fact have the children the majority of the time now does, in my opinion open you up to go back to court. He may have primary custody on paper - but he doesn't have them 2-6 hours more than you do any longer. The situation has changed, so I think you could in fact go...
What a jerk - how dare he treat a little boy like that! The poor little guy probably was upset enough with himself, the last thing he needed was threats that he is going to be taken away from his mother!
Don't you dare let him off the hook for child support! You have the doctor's visit to...