Frederick Dent Grant Frederick Dent Grant ( May 30 , 1850 – April 12 , 1912 ) was a soldier and United States minister to Austria - Hungary . Grant was the first son of General of the Army and President of the United States Ulysses S . Grant and Julia Grant . He was named after his uncle , Frederick Tracy Dent . The Grant family came from a line of Scottish and Irish heritage . Early life . His father was in the United States Army when Frederick was born in St . Louis , Missouri . The family moved as the senior Grant was assigned to posts in Michigan and New York . Frederick spent his early childhood at his paternal grandparent ' s house while his father was stationed on the West Coast . After his father ' s resignation from the army , the family lived in St . Louis and in Galena , Illinois . He attended public school in Galena until the outbreak of the American Civil War in 1861 . Grant ' s father organized a volunteer regiment and was made colonel . Frederick accompanied his father when the regiment was sent to northern Missouri , but he was sent home when it arrived . He then rejoined his father off and on at several campaigns during the war . Early military career . Grant was appointed to West Point in 1866 and graduated in 1871 . He was assigned to the 4th U . S . Cavalry Regiment . He took a leave of absence to work with the Union Pacific Railroad as a civil engineer . Late in 1871 , he was aide - de - camp to General William Tecumseh Sherman in Europe . He rejoined the 4th Cavalry in Texas in 1872 . In 1873 , he was assigned to the staff of General Philip Sheridan and promoted to lieutenant colonel . He was on the Yellowstone Expedition and was with George Armstrong Custer during the Black Hills expedition . In 1874 , Grant married Ida Marie Honoré ( 1854 – 1930 ) , the daughter of Henry Hamilton Honoré , who made his fortune in Chicago real estate . They were married in Chicago and had two children : Julia Dent Grant ( born 1876 ) and Ulysses III ( born 1881 ) . ( Note : Ulysses IV was the son of Ulysses S . ( Buck ) Grant , Jr . ) The birth of his first child , Julia Dent Grant , in essence saved his life . Grant received leave to travel from Custer ' s unit in the Black Hills of South Dakota to Washington , D . C . for her birth . Had he remained with Custer ' s unit , he would have been in the Battle of the Little Bighorn ( June 24 – 25 , 1876 ) in which Custer and five companies of the 7th Cavalry Regiment of the United States Army were killed . In 1877 , he took a leave of absence to accompany his father on a trip around the world . In 1878 , Grant was in the Bannock War and was in the fight against Victorio in New Mexico . West Point controversy . On June 1 , 1870 , the first African American cadet , James Webster Smith , from South Carolina , was admitted into the United States Military Academy . Smith was sponsored by Senator Adelbert Ames and nominated by Representative Solomon L . Hoge . Smith was hand picked for his outstanding character and scholarly ability by David Clark , a northern philanthropist . While at West Point , Smith was forced to endure immense racism , violence , and shunning by other West Point attendees . Among Smith ' s harassers included Frederick Dent Grant , a fellow cadet . The instigators , including Fred , were intent on driving Smith from the Academy . Smith wrote to Clark about the racial hazing , whereupon Clark went to the White House to talk with President Grant . Fred was also at the meeting between Clark and the President . Clark advocated that Grant stop the hazing . Grant said , " Don ' t take him away ; the battle may as well be now as anytime . " The young Fred was noted to say in front of his father , " the time had not come to send colored boys to West Point . " When Clark disagreed , Fred said , " Well , no " damned nigger " will ever graduate from West Point . " Smith was later discharged after failing an unconventional " private " examination test by Professor Peter S . Mitchie . While Fred Grant denied being a leader of the cadets who hazed Smith for being an African American , there is evidence to suggest he actively participated . Civilian career . He resigned from the army in 1881 , and assisted his father in preparing the latter ' s memoirs . During this time , he was in business in New York City . In 1887 , he ran on the Republican ticket for Secretary of State of New York , but was defeated by the Democratic incumbent Frederick Cook . In 1889 , President Benjamin Harrison appointed him Minister to Austria - Hungary . After Grover Cleveland became president , Grant was allowed to continue in his post . Grant resigned in 1893 . Grant became a commissioner of police in New York City in 1894 , an office he held until 1898 . He served alongside future President Theodore Roosevelt . He was a hereditary member of the Military Order of the Loyal Legion of the United States and the Society of Colonial Wars . Later military career . When the Spanish - American War started in 1898 , Grant was colonel of the 14th New York Volunteers and was promoted to brigadier general of volunteers . He served in Puerto Rico . In 1899 , Grant was sent to the Philippines for service in the Philippine - American War , where he remained until 1902 . In 1901 , he was made a brigadier general in the Regular Army . When he returned to the United States , he held various commands and was promoted to major general in 1906 . At the time of his death , he was the commander for the Eastern Division which included the Department of the East and the Department of the Gulf . He died of cancer , the same disease that had claimed his father , at Fort Jay on Governors Island in New York City on April 12 , 1912 , and was buried in West Point Cemetery .