If you feel that your neighbor fully knows that the fence is NOT on the true line, you may have no other recourse but to consult an attorney. [I am not an lawyer, but I am going through an adverse possession case with my neighbors and have done extensive research on it] I do believe you...
PS................Aireal photos CAN be off by a few feet (mine shows my shed on the neighbors and the other neighbors house a good bit on mine) so dont rely on it solely.
John...........It sounds to me that with a face to face meeting with your neighbor you two might be able to come to terms that you both can live with. At least he is willing to help with the costs, and no, you dont want to lose land but you can also understand his point of landscaping and...
John....you might want to find all four corners of your property so you know what you have. The Tax office can help with an aerial of the lot and the surrounding lots. You can also check the Deed office and look at the deed.....sometimes a survey is attached. You can try finding the markers...
Send them a letter like the privious post said and do it quick! How long the fence been there?? After so many years the land inside the fence can be theirs. It's called adverse possession. I don't know the statue in CA but here in Alabama it is 10 years. DO NOT let time go by on this!!