If the agent named in a durable power of attorney is taking advantage of the principal and the family would like to terminate power of attorney, they will have to apply as a guardian or conservator. This is accomplished by filing a petition with the local court, which should be handled by an attorney.