In 2016 my ex and I had a visitation agreement drawn up when I moved out of state for shared custody and visitation.
Your ex-what? Ex-spouse? Ex-boy/girlfriend? Ex-something else? If this is anything other than an ex-spouse, was the father's paternity ever established? If so, how was it established? Note that "his name is on the birth certificate" does not answer that question. Was this "visitation agreement" adopted by a court as part of a divorce decree or custody/visitation order?
In 2019, the father tells me that our agreement wasnt legal and he wont be allowing her to come back with me, he is also refusing to let me see her.
I assume "the father" is the same person you referred to in the prior sentence as "my ex." Did he offer you any explanation for his opinion that your "agreement wasn[']t legal"? If so, what explanation did he give?
I called the courthouse to see of anything had been filed to make sure. Court house said that there was a petition filed but there wasnt a court order filed. What does that mean?
Umm...first of all, what courthouse did you call, and why did you choose that particular courthouse to the exclusion of all others? Second, to whom did you speak at the courthouse (not a name, but a title -- e.g., court clerk)? Third, I assume it means exactly what it says, although we obviously have no way of knowing what sort of petition that might be.
Everyone has legal rights.
If our agreement isnt legal, can I legally take her back?
Since you're apparently the mother, if you can acquire possession of your child without disturbing the peace, you can do so legally as long as no court order says otherwise.
Please any advice would be greatly appreciated!
It sounds to me like, when you and your ex separated, you disregarded all appropriate legalities as it related to your child. You should now consult with a local family law attorney and seek to obtain appropriate orders regarding custody, visitation and support. You should also answer the questions I asked above.
That means that he has as much right to your child as you do.
I disagree. Since we don't know if this is an ex-husband or something else (my guess is that it's something else), and since there appears to be no existing custody/visitation order, and since we don't even know if the father's paternity was established, it is entirely possible that the OP's ex has NO enforceable legal rights regarding the child.