According to the court, he is the father. Whether you agree or not, this country affords rights to the parents that it does not afford to other relatives. Whether at some time in the past he agreed to allow someone else to fulfill that role or not, that person apparently did not follow through, which means this guy is still the legal parent with all the same rights as though he had been there 24/7 from day 1. In this country we don't just look at who is the "best" person to raise a child. No matter how good a parent someone is, there is always someone out there who could be better or give them more. If he at some point signed something to allow another person to adopt him, he has been the established legal father for some time. The signature of some random dude who is not the father wouldn't mean squat. Your best bet is to do everything in your power to make this easier and less stressful for the child.