He had the right to trim the tree.
You aren't required to acquiesce to his demands.
If he wants to be reimbursed, he'll have to sue you.
If he wins, you still aren't required to pay.
He'll have to collect his judgment.
But, as long as Yiu aren't sued, and a court doesn't order you to pay; you don't have to pay.
Tell him to sue you.
Or, choose to pay him.