Complex sentence: By the 1880s, Hamilton had become well known as an alienist, which is an older term for a psychiatrist or psychologist, and was an expert in what is now known as forensic psychology whereby he would evaluate defendants to determine their competency to stand trial.
Split into simpler sentences: By the 1880s, Hamilton had become well known as an "alienist," a term that was then used to describe a psychiatrist or psychologist. He was an early expert in what is now known as forensic psychology, including evaluating defendants to determine their competency to stand trial.