Dad's MOM is not using his visitation time.... Dad's WIFE is, the stepmom.
OP, the actual advice was correct: If you want only the father to pick up the child, then you'll have to go back to court.
Your ex is wrong that his wife is a legal guardian simply because she is named in the order...