Are you and the father living in the same home? If so, what is it that he is supposed to do and NOT doing?
The short answer is, maybe. Depending on the reason for their intervention they might be able to get a court order to remove the children if you and dad fail to follow the parenting plan...