You need to give more info of what is going on.
A social worker does not restrict you from seeing your child- the court does that.
Did the court issue an order preventing visitation? Why was the order issued?
Is it a temporary order pending further court action?