The court ordered you to pay the money. There is no SOL on court orders. If you don't pay the ex will have to drag you back in to court. After 4 years it doesn't seem that it will happen, and even if it does a change of circumstances could be in your favor.
I suggest you keep quiet and off the radar if possible.