James Madison James Madison , Jr . ( March 16 , 1751 ( O . S . March 5 ) – June 28 , 1836 ) was an American statesman , political theorist and the fourth President of the United States ( 1809 – 1817 ) . He is hailed as the " Father of the Constitution " for being instrumental in the drafting of the United States Constitution and as the key champion and author of the United States Bill of Rights . He served as a politician much of his adult life . After the constitution had been drafted , Madison became one of the leaders in the movement to ratify it . His collaboration with Alexander Hamilton and John Jay produced the " Federalist Papers " ( 1788 ) . Circulated only in New York at the time , they would later be considered among the most important polemics in support of the Constitution . He was also a delegate to the Virginia constitutional ratifying convention , and was instrumental to the successful ratification effort in Virginia . Like most of his contemporaries , Madison changed his political views during his life . During the drafting and ratification of the constitution , he favored a strong national government , though later he grew to favor stronger state governments , before settling between the two extremes late in his life . In 1789 , Madison became a leader in the new House of Representatives , drafting many basic laws . He is notable for drafting the first ten amendments to the Constitution , and thus is known as the " Father of the Bill of Rights " . Madison worked closely with President George Washington to organize the new federal government . Breaking with Hamilton and what became the Federalist Party in 1791 , Madison and Thomas Jefferson organized what they called the Republican Party ( later called by historians the Democratic - Republican Party ) As Jefferson ' s Secretary of State ( 1801 – 1809 ) , Madison supervised the Louisiana Purchase , which doubled the nation ' s size . After his election to the presidency , he presided over renewed prosperity for several years . As president ( 1809 – 17 ) , after the failure of diplomatic protests and a trade embargo against Great Britain , he led the nation into the War of 1812 . He was responding to British encroachments on American honor and rights ; in addition , he wanted to end the influence of the British among their Indian allies , whose resistance blocked United States settlement in the Midwest around the Great Lakes . Madison found the war to be an administrative nightmare , as the United States had neither a strong army nor financial system ; as a result , he afterward supported a stronger national government and a strong military , as well as the national bank , which he had long opposed . Like other Virginia statesmen in the slave society , he was a slaveholder who inherited his plantation known as Montpelier , and owned hundreds of slaves during his lifetime to cultivate tobacco and other crops . Madison supported the three - fifths compromise that allowed three - fifths of the enumerated population of slaves to be counted for representation . Early life and education . James Madison , Jr . was born at Belle Grove Plantation near Port Conway , Virginia on March 16 , 1751 , ( March 5 , 1751 , Old Style , Julian calendar ) , where his mother had returned to her parents ' home to give birth . He grew up as the oldest of twelve children . Nelly and James Sr . had seven more boys and four girls . Three of James Jr . ' s brothers died as infants , including one who was stillborn . In the summer of 1775 , his sister Elizabeth ( age 7 ) and his brother Reuben ( age 3 ) died in a dysentery epidemic that swept through Orange County because of contaminated water . His father , James Madison , Sr . ] ] ( 1723 – 1801 ) , was a tobacco planter who grew up on a plantation , then called Mount Pleasant , in , a 26 year old widow , at Harewood , in what is now < a href = " Jefferson % 20County % 2C % 20West % 20Virginia . % 26lt % 3Bref % 20name % 3D % 26quot % 3Bamadison % 26quot % 3B / % 26gt % 3B % 20Madison % 20had % 20no % 20children % 20but % 20did % 20adopt % 20Todd % 27s % 20one % 20surviving % 20son % 2C % 20 % 5B % 5BJohn % 20Payne % 20Todd % 5D % 5D % 20 % 28known % 20as % 20Payne % 29 % 2C % 20after % 20the % 20marriage . % 0A % 0ADolley % 20Payne % 20was % 20born % 20May % 2020 % 2C % 201768 % 2C % 20at % 20the % 20New % 20Garden % 20Quaker % 20settlement % 20in % 20North % 20Carolina % 2C % 20where % 20her % 20parents % 2C % 20John % 20Payne % 20and % 20Mary % 20Coles % 20Payne % 2C % 20lived % 20briefly . % 20Dolley % 27s % 20sister % 2C % 20 % 5B % 5BLucy % 20Washington " > Lucy Payne ] ] , had recently married [ [ George Steptoe Washington ] ] , a nephew of President Washington . As a member of Congress , Madison had doubtless met the widow Todd at social functions in Philadelphia , then the nation ' s capital . She had been living there with her late husband . In May 1794 , Madison asked their mutual friend [ [ Aaron Burr ] ] to arrange a meeting . By August , she had accepted his proposal of marriage . For marrying Madison , a non - Quaker , she was expelled from the Society of Friends . They were known to have a happy marriage . Dolley Madison put her social gifts to use when the couple lived in Washington , beginning when he was Secretary of State . With the White House still under construction , she advised as to its furnishings and sometimes served as First Lady for ceremonial functions for President Thomas Jefferson , a widower and their friend . When her husband was president , she created the role of First Lady , using her social talents to advance his program . She is credited with adding to his popularity in office . Madison ' s father died in 1801 and at age 50 , Madison inherited the large plantation of Montpelier and other holdings , and his father ' s 108 slaves . He had begun to act as a steward of his father ' s properties by 1780 , but this completed his takeover . United States Secretary of State 1801 – 1809 . When [ [ Thomas Jefferson ] ] was inaugurated as president in 1801 , he named Madison to be his secretary of state . At the start of his term , Madison was a party to the United States Supreme Court case " [ [ Marbury v . Madison ] ] " ( 1803 ) , in which the doctrine of [ [ judicial review ] ] was asserted by the high Court , much to the annoyance of the Jeffersonians who did not want a powerful federalist judiciary . The main challenge to the Jefferson Administration was maintaining neutrality during the [ [ Napoleonic Wars ] ] . Throughout Jefferson ' s presidency , much of Europe was at war , at first between France and Austria . After the [ [ Battle of Austerlitz ] ] in 1805 , where France decisively defeated the Austrian [ [ Hapsburg ] ] s , the conflict transformed into a grinding war between France and Britain . Shortly before Jefferson ' s election , [ [ Napoleon ] ] had seized power from the hapless French Directory , which had recently mismanaged France ' s finances in unsuccessful wars and had lost control of Saint - Domingue ( Haiti ) after a slave rebellion . Beginning in 1802 , Napoleon sent more than 20 , 000 troops to try to restore slavery on the island , as its colonial sugar cane plantations had been the chief revenue producer for France in the New World . The warfare went badly and the troops were further decimated by yellow fever . Napoleon gave up on thoughts of restoring the empire and sold the Louisiana territory to Madison and Jefferson in 1803 . Later that year , the 7 , 000 surviving French troops were withdrawn from the island , and in 1804 Haiti declared its independence as the second republic in the western hemisphere . [ [ File : louisiana purchase treaty . jpg | thumb | The original treaty of the Louisiana Purchase ] ] Many contemporaries and later historians , such as [ [ Ron Chernow ] ] , noted that Madison and President Jefferson ignored their " strict construction " view of the Constitution to take advantage of the purchase opportunity . Jefferson would have preferred to have a constitutional amendment authorizing the purchase , but did not have time nor was he required to do so . The Senate quickly ratified the treaty that completed the purchase . The House , with equal alacrity , passed enabling legislation . With the wars raging in Europe , Madison tried to maintain American neutrality , and insisted on the legal rights of the U . S . as a neutral under international law . Neither London nor Paris showed much respect , however , and the situation deteriorated during Jefferson ' s second term . After Napoleon achieved victory at Austerlitz over his enemies in continental Europe , he became more aggressive and tried to starve Britain into submission with an embargo that was economically ruinous to both sides . Madison and Jefferson had also decided on an embargo to punish Britain and France , which forbade American trade with any foreign nation . The embargo failed in the United States just as it did in France , and caused massive hardships up and down the seaboard , which depended on foreign trade . The Federalists made a comeback in the Northeast by attacking the embargo , which was allowed to expire just as Jefferson was leaving office . Election of 1808 . With Jefferson ' s second term winding down , and his decision to retire widely known , Madison was the party choice for president in 1808 . He was opposed by Rep . [ [ John Randolph of Roanoke | John Randolph ] ] , who had broken earlier with Jefferson and Madison . The Republican Party Congressional caucus chose the candidate and easily selected Madison over James Monroe . As the Federalist party by this time had largely collapsed outside New England , Madison easily defeated Federalist [ [ Charles Cotesworth Pinckney ] ] . Presidency 1809 – 1817 . [ [ File : James Madison Portrait2 . jpg | thumb | upright | James Madison engraving by [ [ David Edwin ] ] from between 1809 and 1817 ] ] Upon his Inauguration in 1809 , Madison immediately had difficulty in his appointment selection of Sec . [ [ Albert Gallatin ] ] as Secretary of State . Under opposition from Sen . [ [ William B . Giles ] ] , Madison chose not to fight Congress for the nomination but kept Sec . Gallatin , a carry over from the Jefferson Administration , in the Treasury . The talented Swiss born Gallatin was Madison ' s primary advisor , confident , and policy planner . Madison appointed [ [ Robert Smith ( Cabinet member ) | Robert Smith ] ] for Secretary of State , Jefferson ' s former Secretary of Navy . For his Secretary of Navy , Madison appointed Paul Hamilton . Madison ' s Cabinet , that included men of [ [ : wikt : mediocre | mediocre ] ] talent , was chosen in terms of national interest and political harmony . When Madison assumed office in 1809 , the federal government had a surplus of $ 9 , 500 , 000 and by 1810 the national debt continued to be reduced and taxes had been cut . Bank of United States . Madison sought to continue Jefferson ' s agenda , in particular the dismantling of the system left behind by the federalists under Washington and Adams . One of the most pressing issues Madison confronted was the [ [ first Bank of the United States ] ] . Its twenty - year charter was scheduled to expire in 1811 , and while Madison ' s treasury secretary said the bank was a necessity , Congress failed to re - authorize it . As the absence of a national bank made war with Britain very difficult to finance , in 1814 Congress passed a bill chartering a second national bank . Madison vetoed it . In 1816 , Congress passed another bill to charter a second national bank ; Madison signed the act , having learned the bank was needed from the war with Britain . Prelude to war . By 1809 the Federalist party was no longer competitive outside a few strongholds . Some former members ( such as [ [ John Quincy Adams ] ] , Madison ' s ambassador to Russia ) had joined Madison ' s Republican party . Though one party appeared to dominate , it had begun to split into rival factions , which would later form the basis of the modern party system . In particular , with hostilities against Britain appearing increasingly likely , factions in favor of and against war with Britain formed in Congress . The predominant faction , the " War Hawks , " were led by [ [ House Speaker ] ] Henry Clay . When war finally did break out , the war effort was led by the War Hawks in Congress under Clay at least as much as it was by Madison ; this accorded with the president ' s preference for checks and balances . [ [ Napoleon ] ] had won a decisive victory at the [ [ Battle of Austerlitz ] ] in 1805 , and as a consequence Europe remained mostly at peace for the next few years . Congress repealed Jefferson ' s [ [ Embargo Act | embargo ] ] shortly before Madison became president . America ' s new [ [ Non - Intercourse Act ( 1809 ) | " nonintercourse " policy ] ] was to trade with all countries including France and Britain if restrictions on shipping were removed . Madison ' s diplomatic efforts in April 1809 , although initially promising , to get the British to withdraw the [ [ Orders in Council ( 1807 ) | Orders in Council ] ] were rejected by British Foreign Secretary [ [ George Canning ] ] . By August 1809 , diplomatic relations with Britain deteriorated as minister [ [ David Erskine , 2nd Baron Erskine | David Erskine ] ] was withdrawn and replaced by " hatchet man " [ [ Francis James Jackson ] ] ; Madison however , resisted calls for war . In his " Political Observations " from April 20 , 1795 Madison had stated : After Jackson accused Madison of duplicity with Erskine , Madison had Jackson barred from the State Department and sent packing to Boston . Madison during his first state of the Union address in November 1809 , asked Congress for advice and alternatives concerning British - American trade crisis and to prepare for war . By Spring 1810 , Madison was specifically asking Congress for more appropriations to increase the Army and Navy in preparation for war with Britain . Together with the effects of European peace , the United States economy began to recover early in Madison ' s presidency . By the time Madison was standing for reelection , the [ [ Peninsular War ] ] in Spain had spread , while at the same time Napoleon [ [ Napoleon ' s invasion of Russia | invaded Russia ] ] , and the entire European continent was once again embroiled in war . War of 1812 . [ [ File : Battle erie . jpg | 300px | thumb | right | Commodore [ [ Oliver Hazard Perry ] ] defeats British Navy at the [ [ Battle of Lake Erie ] ] in 1813 . < br > Powell " 1873 " ] ] The United States entered the [ [ War of 1812 ] ] , which in many respects was a theater of the broader [ [ Napoleonic Wars ] ] . Napoleon began his [ [ Continental System ] ] , intended to force other European countries to join his embargo of Britain . Although he was initially successful in starving out Britain , Portugal refused to capitulate , leading to the Peninsular War throughout Spain . This loosened Spain ' s grip on its South American colonies . Great Britain became the only major power in the Atlantic , and as it increased naval pressure against Napoleon , it inadvertently did the same against American ships . British tactics quickly caused widespread American anger . Britain used its navy to prevent American ships from trading with France . The United States , which was a neutral nation , considered this act to be against international law . Britain also armed Indian tribes in the Northwest Territory and encouraged them to attack settlers , even though Britain had ceded this territory to the United States by treaties in 1783 and 1794 . The Royal Navy boarded American ships on the high seas and [ [ impressment | impressed ] ] its seamen , as it needed more sailors than it could recruit . The United States looked upon this as no less an affront to American sovereignty than if the British had invaded American soil . Americans called for a " second war of independence " to restore honor and stature to the new nation . An angry public elected a " war hawk " Congress , led by Henry Clay and [ [ John C . Calhoun ] ] . Madison asked Congress for a declaration of war , which was passed along sectional and party lines , with intense opposition from the Federalists and the Northeast , where the economy had suffered during Jefferson ' s trade embargo . Hurriedly Madison called on Congress to put the country " into an armor and an attitude demanded by the crisis , " specifically recommending enlarging the army , preparing the militia , finishing the military academy , stockpiling munitions , and expanding the navy . Madison faced formidable obstacles — a divided cabinet , a factious party , a recalcitrant Congress , obstructionist governors , and incompetent generals , together with militia who refused to fight outside their states . Most serious was lack of unified popular support . There were serious threats of disunion from New England , which engaged in extensive smuggling with Canada and refused to provide financial support or soldiers . The problems were worse due to Jefferson ' s and Madison ' s dismantling of the system built by Hamilton and the Federalists . They had reduced the military , closed the Bank of the U . S . , and narrowed the tax system . They distrusted standing armies and banks , and the dismantling of the federalist taxation system meant they could not finance the quick hiring of mercenaries . By the time the war began , Madison ' s military force consisted mostly of poorly trained militia members . [ [ File : USS Constitution vs Guerriere . jpg | left | thumb | 264px | The defeats [ [ HMS Guerriere ( 1806 ) | HMS " Guerriere " ] ] , a significant event during the war . ] ] The senior command at the War Department and in the field proved incompetent or cowardly — the general at Detroit surrendered to a smaller British force without firing a shot . Gallatin at the Treasury discovered the war was almost impossible to fund , since the national bank had been closed and major financiers in the Northeast refused to help . Madison believed the U . S . could easily seize Canada and thus cut off food supplies to the West Indies , making for a good bargaining chip at the peace talks . But the US invasion efforts all failed . Madison had believed the state militias would rally to the flag and invade Canada , but the governors in the Northeast failed to cooperate . Their militias either sat out the war or refused to leave their respective states for action . The British armed American Indians in the Northwest , most notably several tribes allied with the Shawnee chief , [ [ Tecumseh ] ] . But , after losing control of Lake Erie at the naval Battle of Lake Erie in 1813 , the British were forced to retreat . General [ [ William Henry Harrison ] ] caught up with them at the [ [ Battle of the Thames ] ] , where he destroyed the British and Indian armies , killed Tecumseh , and permanently destroyed Indian power in the Great Lakes region . The British raided Washington in 1814 , as Madison headed a dispirited militia . Dolley Madison rescued White House valuables and documents shortly before the British burned the White House , the Capitol and other public buildings . By 1814 , [ [ Andrew Jackson ] ] and [ [ William Henry Harrison ] ] had destroyed the main Indian threats in the South and West , respectively . As part of the war effort , an American naval shipyard was built up at Sackets Harbor , New York , where thousands of men produced twelve warships and had another nearly ready by the end of the war . American frigates and other vessels , such as the , [ [ USS United States ( 1797 ) | USS " United States " ] ] , [ [ USS Chesapeake ( 1799 ) | USS " Chesapeake " ] ] , [ [ USS Hornet ( 1805 brig ) | USS " Hornet " ] ] , , and , won some significant naval battles on the Great Lakes . In a famous three - hour battle with the [ [ HMS Java ( 1811 ) | HMS " Java " ] ] , the USS " Constitution " earned her nickname , " Old Ironsides . " The U . S . fleet on Lake Erie went up against a superior British force there and destroyed or captured the entire British Fleet on the lake . Commander Oliver Hazard Perry reported his victory with the simple statement , " We have met the enemy , and they are ours . " America had built up one of the largest merchant fleets in the world , though it had been partially dismantled under Jefferson and Madison . Madison authorized many of these ships to become privateers in the war . Armed , they captured 1 , 800 British ships . [ [ File : US Capitol 1814c . jpg | 220px | thumb | 254px | The unfinished United States Capitol was set ablaze by the British on August 24 , 1814 . ] ] The courageous , successful defense of [ [ Ft . McHenry ] ] , which guarded the seaway to Baltimore , against one of the most intense naval bombardments in history ( over 24 hours ) , led [ [ Francis Scott Key ] ] to write the poem that was set to music as the U . S . national anthem , " [ [ The Star Spangled Banner ] ] . " In New Orleans , Gen . Andrew Jackson put together a force including regular Army troops , militia , frontiersmen , Creoles , Native American allies and [ [ Jean Lafitte | Jean Lafitte ' s ] ] pirates . The [ [ Battle of New Orleans ] ] took place two weeks after peace treaty was drafted ( but before it was ratified , so the war was not over ) . The Americans smashed the British invasion army in the greatest victory of the war . The [ [ Treaty of Ghent ] ] ended the war in February 1815 , with no territorial gains on either side . The Americans felt that their national honor had been restored in what has been called " the Second War of American Independence . " On March 3 , 1815 , the U . S . Congress authorized deployment of naval power against Algiers , and two squadrons were assembled and readied for war ; the [ [ Second Barbary War ] ] would mark the beginning of the end for piracy in that region . To most Americans , the quick succession of events at the end of the war ( the burning of the capital , the Battle of New Orleans , and the Treaty of Ghent ) appeared as though American valor at New Orleans had forced the British to surrender after almost winning . This view , while inaccurate , strongly contributed to the post - war euphoria that persisted for a decade . It also helps explain the significance of the war , even if it was strategically inconclusive . Napoleon was defeated for the last time at the [ [ Battle of Waterloo ] ] near the end of Madison ' s presidency , and as the Napoleonic Wars ended , so did the War of 1812 . Madison ' s final years began an unprecedented period of peace and prosperity , which was called the [ [ Era of Good Feelings ] ] . Madison ' s reputation as President improved and Americans finally believed the United States had established itself as a world power . Postwar economy and internal improvements . With peace finally established , Americans believed they had secured a solid independence from Britain . The Federalist Party , which had called for secession over the war at the [ [ Hartford Convention ] ] , dissolved and disappeared from politics . With Europe finally at peace , the [ [ Era of Good Feelings ] ] described the prosperity and relatively equable political environment . Some political contention continued , for instance , in 1816 , two - thirds of the incumbents in Congress were defeated for re - election after having voted to increase their salary . Madison approved a Hamiltonian national bank , an effective taxation system based on tariffs , a standing professional military , and the internal improvements championed by Henry Clay under his [ [ American System ( economic plan ) | American System ] ] . However , in his last act before leaving office , he vetoed the [ [ Bonus Bill of 1817 ] ] , which would have financed more internal improvements , including roads , bridges , and canals : Madison rejected the view of Congress that the General Welfare provision of the [ [ Taxing and Spending Clause ] ] justified the bill , stating : Madison urged a variety of measures that he felt were " best executed under the national authority , " including federal support for roads and canals that would " bind more closely together the various parts of our extended confederacy . " Wilkinson affair . [ [ File : James Wilkinson . jpg | thumb | 150px | right | James Wilkinson ] ] [ [ James Wilkinson ] ] was a controversial U . S . military commander and appointed governor of the Louisiana Territory by Thomas Jefferson in 1805 . Wilkinson had earlier been implicated in [ [ Aaron Burr ] ] ' s conspiracy to form a new nation in the West and taking Spanish gold , however , he was exonerated in 1808 . Jefferson chose to retain Wilkinson , a Republican , for political expedience . After Madison assumed the Presidency in 1809 , he placed Wilkinson in charge of [ [ St . Bernard Parish , Louisiana | Terre aux Boeufs ] ] on the Louisiana coast to protect the U . S . from invasion . Wilkinson proved to be an incompetent general ; many soldiers complained that he was ineffectual : their tents were defective , and they became sick by malaria , dysentery , and scurvy ; dozens died daily . Wilkinson made excuses and refused to move inland from the mosquito - infested coastline . A two - year congressional investigation into the Wilkinson matter proved to be inconclusive , and Madison had to decide whether to keep or sack him . Like Jefferson , Madison chose to retain Wilkinson for political reasons , as Wilkinson had influence as a Pennsylvania Republican . By retaining Wilkinson , both Jefferson and Madison supported military leaders in both the Army and Navy for political reasons rather than competence . Historian Robert Allen Rutland stated the Wilkinson affair left " scars on the War Department " and " left Madison surrounded by senior military incompetents . . . " at the beginning of the [ [ War of 1812 ] ] . After Wilkinson ' s two battle defeats by the British , Madison relieved the officer from active military service . Indian policy . [ [ File : Benjamin Hawkins and the Creek Indians . jpg | 220px | thumb | right | Creek men being taught how to use a plow by [ [ Benjamin Hawkins ] ] in 1805 . Madison believed learning European - style agriculture would help the Creek adopt the values of British - American civilization . ] ] Upon assuming office on March 4 , 1809 James Madison , in his first [ [ Inaugural Address ] ] to the nation , stated that the federal government ' s duty was to convert the American Indians by the " participation of the improvements of which the human mind and manners are susceptible in a civilized state " . Like Jefferson , Madison had a paternalistic attitude toward American Indians , encouraging the men to give up hunting and become farmers . Although there are scant details , Madison often met with Southeastern and Western Indians who included the [ [ Muscogee people | Creek ] ] and [ [ Osage Nation | Osage ] ] . As pioneers and settlers moved West into large tracts of Cherokee , Choctaw , [ [ Muscogee people | Creek ] ] , and Chickasaw territory , Madison ordered the US Army to protect Native lands from intrusion by settlers , to the chagrin of his military commander [ [ Andrew Jackson ] ] . He wanted the President to ignore Indian pleas to stop the invasion of their lands and resisted carrying out the president ' s order . In the Northwest Territory after the [ [ Battle of Tippecanoe ] ] in 1811 , Indians were pushed off their tribal lands and replaced entirely by white settlers . By 1815 , with a population of 400 , 000 European - American settlers in Ohio , Indian rights to their lands had effectively become [ [ null and void ] ] . Judicial appointments . Supreme Court . Madison appointed the following Justices to the Supreme Court of the United States : Other courts . Madison appointed eleven other federal judges , two to the [ [ United States Circuit Court of the District of Columbia ] ] , and nine to the various United States district courts . One of those judges was appointed twice , to different seats on the same court . Later life . [ [ File : JamesMadison . jpg | thumb | 224px | " Portrait of James Madison " c . 1821 , < br > by [ [ Gilbert Stuart ] ] ] ] When Madison left office in 1817 , he retired to Montpelier ( James Madison ) | Montpelier < / a > , his tobacco plantation in [ [ Orange County , Virginia , not far from Jefferson ' s [ [ Monticello ] ] . He was 65 years old . Dolley , who thought they would finally have a chance to travel to Paris , was 49 . As with both Washington and Jefferson , Madison left the presidency a poorer man than when he entered , due to the steady financial collapse of his plantation , aided by the continued low price of tobacco and his stepson ' s mismanagement . Insight into Madison is provided by the first " White House memoir , " " A Colored Man ' s Reminiscences of James Madison " ( 1865 ) , told by his former slave [ [ Paul Jennings ( slave ) | Paul Jennings ] ] , who served the president from the age of 10 as a footman , and later as a valet for the rest of Madison ' s life . After Madison ' s death , Jennings was purchased in 1845 from Dolley Madison by arrangement with the senator [ [ Daniel Webster ] ] , who enabled him to work off the cost and gain his freedom . Jennings published his short account in 1865 . He had the highest respect for Madison and said he never struck a slave , nor permitted an overseer to do so . Jennings said that if a slave misbehaved , Madison would meet with the person privately to try to talk about the behavior . Some historians speculate that Madison ' s mounting debt was one of the chief reasons why he refused to allow his notes on the Constitutional Convention , or its official records which he possessed , to be published in his lifetime . " He knew the value of his notes , and wanted them to bring money to his estate for Dolley ' s use as his plantation failed — he was hoping for one hundred thousand dollars from the sale of his papers , of which the notes were the gem . " Madison ' s financial troubles weighed on him , and deteriorating mental and physical health would haunt him . In his later years , Madison became extremely concerned about his historic legacy . He took to modifying letters and other documents in his possession : changing days and dates , adding and deleting words and sentences , and shifting characters . By the time he had reached his late seventies , this " straightening out " had become almost an obsession . As an example , he edited a letter written to Jefferson criticizing Lafayette : Madison not only inked out original passages , but imitated Jefferson ' s handwriting as well in making changes . [ [ File : James madison - Age82 - Edit1 . jpg | thumb | left | 224px | Portrait of Madison , age 82 , < br > c . 1833 ] ] In 1826 , after the death of Jefferson , Madison was appointed as the second Rector ( " President " ) of the University of Virginia . He retained the position as college chancellor for ten years until his death in 1836 . In 1829 , at the age of 78 , Madison was chosen as a representative to the constitutional convention in Richmond for the revising of the Virginia state constitution . It was his last appearance as a legislator and constitutional drafter . The issue of greatest importance at this convention was [ [ apportionment ] ] . The western districts of Virginia complained that they were underrepresented because the state constitution apportioned voting districts by county , not population . The growing population in the Piedmont and western parts of the state were not reflected in their representation in the legislature . Western reformers also wanted to extend suffrage to all white men , in place of the historic property requirement . Madison tried to effect a compromise , but to no avail . Eventually , suffrage rights were extended to renters as well as landowners , but the eastern planters refused to adopt population apportionment . Madison was disappointed at the failure of Virginians to resolve the issue more equitably . Madison was very concerned about the continuing issue of slavery in Virginia and the South . He believed that transportation of free American blacks to Africa offered a solution , as promoted by the [ [ American Colonization Society ] ] ( ACS ) . He told [ [ Gilbert du Motier , marquis de Lafayette | Lafayette ] ] at the time of the convention that colonization would create a " rapid erasure of the blot on our Republican character . " The British sociologist , [ [ Harriet Martineau ] ] , visited with Madison during her tour of the United States in 1834 . She characterized his faith in colonization as the solution to slavery as " bizarre and incongruous . " Madison may have sold or donated his [ [ gristmill ] ] in support of the ACS . The historian [ [ Drew R . McCoy ] ] believes that " The Convention of 1829 , we might say , pushed Madison steadily to the brink of self - delusion , if not despair . The dilemma of slavery undid him . " Like most African Americans of the time , Madison ' s slaves wanted to remain in the U . S . where they had been born and believed their work earned them citizenship ; they resisted " repatriation " . [ [ File : James Madison gravestone IMG 4299 . JPG | thumb | 224px | Madison ' s tombstone , Montpelier ] ] Through failing health , Madison wrote several memoranda on political subjects , including an essay against the appointment of [ [ chaplain ] ] s for Congress and the armed forces . He felt it would produce religious exclusion but not political harmony . Between 1834 and 1835 , Madison sold 25 % of his slaves to make up for financial losses on his plantation . Madison lived until 1836 , increasingly ignored by the new leaders of the American polity . He died at Montpelier on June 28 , as the last of the Founding Fathers . He was buried in the Madison Family Cemetery at Montpelier . In 1842 , Dolley Madison sold the Montpelier mansion , and in 1844 sold the extensive plantation lands to Henry W . Moncure . She leased half of the remaining slaves to Moncure . The other half were inherited by her , her son [ [ John Payne Todd ] ] , and James Madison , Jr . , a nephew . Between 1845 and 1849 Todd sold numerous slaves ; by 1851 he retained only 15 at his residence . By 1850 , the Montpelier plantation was a " ghost of its former self " . In 1851 , Montpelier was owned by Thomas Thorton , an Englishman ; he held 40 slaves . Legacy . The historian [ [ Garry Wills ] ] wrote : [ [ George F . Will ] ] once wrote that if we truly believed that the pen is mightier than the sword , our nation ' s capital would have been called " Madison , D . C . " , instead of Washington , D . C . Madison ' s writings are studied for the debate over human rights among different classes of citizens in the 21st century . Madison appears to have anticipated the danger of a strong majority imposing its will on a weaker minority by popular vote . Madison in " [ [ The Federalist Papers ] ] , " [ [ The Federalist ( Dawson ) / 50 | No . 51 ] ] , wrote : [ [ Madison Square | Madison Cottage ] ] in New York City was named in his honor shortly after his death . It later became [ [ Madison Square ] ] , the center of numerous landmarks . In 1986 , Congress created the [ [ James Madison Memorial Fellowship Foundation ] ] as part of the bicentennial celebration of the Constitution . The Foundation offers $ 24 , 000 graduate level fellowships to secondary teachers to undertake a master ’ s degree which emphasizes the study of the Constitution . jamesmadison . gov External links . [ [ Category : James Madison | James Madison ] ] [ [ Category : 1751 births ] ] [ [ Category : 1836 deaths ] ] [ [ Category : 18th - century American Episcopalians ] ] [ [ Category : 19th - century American Episcopalians ] ] [ [ Category : American deists ] ] [ [ Category : American Episcopalians ] ] [ [ Category : American people of English descent ] ] [ [ Category : American people of the War of 1812 ] ] [ [ Category : American planters ] ] [ [ Category : American political philosophers ] ] [ [ Category : Burials in Virginia ] ] [ [ Category : Continental Congressmen from Virginia ] ] [ [ Category : Democratic - Republican Party Presidents of the United States ] ] [ [ Category : Federalist Papers ] ] [ [ Category : History of the United States ( 1789 – 1849 ) ] ] [ [ Category : Jefferson administration cabinet members ] ] [ [ Category : Madison family ] ] [ [ Category : Members of the American Philosophical Society ] ] [ [ Category : Members of the United States House of Representatives from Virginia ] ] [ [ Category : Members of the Virginia House of Delegates ] ] Category : People from King George County , Virginia Category : People from Orange County , Virginia [ [ Category : People of the American Enlightenment ] ] [ [ Category : People of Virginia in the American Revolution ] ] [ [ Category : Presidents of the United States ] ] [ [ Category : Princeton University alumni ] ] [ [ Category : Signers of the United States Constitution ] ] [ [ Category : United States presidential candidates , 1808 ] ] [ [ Category : United States presidential candidates , 1812 ] ] [ [ Category : United States Secretaries of State ] ] [ [ Category : University of Virginia people ] ] [ [ Category : Virginia colonial people ] ] [ [ Category : Virginia Democratic - Republicans ] ] [ [ Category : Zachary Taylor family ] ]