There is no where near enough information in your post.
How old are the children? Is the wife still competent to carry on her affairs or is she incapacitate? Who are you in relation to this? Is she still married to the "husband with mental instability?" Is the mental instability medically verifiable or is it a lay person (the wife's) opinion? Is the husband also the legal father? Has there been any abuse of the children? Has family services been involved? Who does she want to have custody? Are they blood relatives?
I don't want a book but I need a lot more than what you said to give advice. If the wife simply dislikes the husband and feels he is mentally unstable, she is going to have a hard time keeping the husband from taking the children when she dies. There are steps she can take now, but much of what will be done will happen after her death by those who have standing to sue for custody.