In architecture, a folly is a building usually constructed strictly for aesthetic pleasure. Originally, buildings were made to provide  shelter or to house people. Follies are just decoration; they no longer have a building's original function.
They were first constructed to put accents into parks and estates. "Folly" is used in the sense of fun or light-heartedness, not in the sense of something ill-advised.

Characteristics

In general, follies have the following properties:

They are buildings, or parts of buildings. This makes them clearly different from other garden ornaments such as sculpture.
They have no purpose other than as an ornament. Often they have some of the appearance of a building constructed for a particular purpose, but this appearance is a sham.
They are purpose-built. Follies are deliberately built as ornaments.
They are often eccentric in design or construction. This is not strictly necessary; however, it is common for these structures to call attention to themselves through unusual details or form.
There is often an element of fakery in their construction. Perhaps the best example of this is the make-believe ruin: a folly which pretends to be the remains of an old building but which was in fact constructed in that state.

In England, these structures are also called "eye-catchers". This points to their basically decorative nature.

Related types
Follies fall within the general realm of fanciful and impractical architecture, and whether a particular structure is a folly is sometimes a matter of opinion. However, there are several types which are related but which can be distinguished from follies.

Fantasy and novelty buildings are essentially the converse of follies. Follies often look like real, usable buildings, but never are; novelty buildings are usable, but have fantastic shapes. The many American shops and water towers in the shapes of commonplace items, for example, are not properly follies.
Eccentric structures may resemble follies, but the mere presence of eccentricity is not proof that a building is a folly. Many mansions and castles are quite eccentric, but being purpose-built to be used as residences, they are not properly follies.
Some structures are popularly referred to as "follies" because they failed to fulfill their intended use. Their design and construction may be foolish, but in the architectural sense, they are not follies.
Visionary art structures frequently blur the line between artwork and folly, if only because it is rather often hard to tell what intent the artist had. The word "folly" carries the connotation that there is something frivolous about the builder's intent, and it is hard to say whether a structure like the Watts Towers was constructed "seriously". Some works (such as the massive complex by Ferdinand Cheval) are considered as follies because they are in the form of useful buildings, but are plainly constructions of extreme and intentional impracticality.
Amusement parks, fairgrounds, and expositions often have fantastical buildings and structures. Some of these are follies, and some are not; the distinction, again, comes in their usage. Shops, restaurants, and other amusements are often housed in strikingly odd and eccentric structures, but these are not follies. On the other hand, fake structures which serve no other purpose than decoration are also common, and these are follies.

History
Follies began as decorative accents on the great estates of the late 16th and early 17th centuries but they flourished especially in the two centuries which followed. Many estates were blessed with picturesque ruins of monastic houses and (in Italy) Roman villas; others, lacking such buildings, constructed their own sham versions of these romantic structures. Such structures were often dubbed "[name of architect or builder]'s Folly", after the single individual who commissioned or designed the project. However, very few follies are completely without a practical purpose. Apart from their decorative aspect, many originally had a use which was lost later, such as hunting towers. Follies are misunderstood structures, according to The Folly Fellowship, a charity that exists to celebrate the history and splendour of these often neglected buildings.

Follies are often found in parks or large grounds of houses and stately homes. Some were deliberately built to look partially ruined. They were especially popular from the end of the 16th century to the 18th century. Theme parks and world's fairs have often contained "follies", although such structures do serve a purpose of attracting people to those parks and fairs.

Famine Follies
The Irish Potato Famine of 1845-49 led to the building of several follies. The society of the day held that laissez faire, not a welfare state, was the appropriate form of civil management. The concept of a welfare state was a century away, and at that time reward without labour, even to those in need, was seen as misguided. However, to hire the needy for work on useful projects would deprive existing workers of their jobs. Thus, construction projects termed "famine follies" came to be built. These include: roads in the middle of nowhere, between two seemingly random points; screen and estate walls; piers in the middle of bogs; etc.

Examples
Follies can be found worldwide, but there seem to be many in Great Britain.

France
 Désert de Retz, folly garden in Chambourcy near Paris, France (18th century)
 Parc de la Villette in Paris has a number of modern follies by architect Bernard Tschumi.
 Ferdinand Cheval in Châteauneuf-de-Galaure, built what he called an Ideal Palace, seen as an example of naive architecture.

Malaysia
 Kellie's Castle, Perak

Hungary 
Bory Castle at Székesfehérvár
Taródi Castle at Sopron
Vajdahunyad vára in the City Park of Budapest

India
 Overbury's Folly, Thalassery, Kerala

Ireland
 Casino at Marino
 Conolly's Folly

Italy
 The Bomarzo Gardens

Romania
 Iulia Hasdeu Castle

Russia
 Ruined towers in Peterhof, Tsarskoe Selo, Gatchina, and Tsaritsino
 Creaking Pagoda and Chinese Village in Tsarskoe Selo
 Dutch Admiralty in Tsarskoe Selo

Ukraine
 Swallow's Nest near Yalta

United Kingdom
 Ashton Memorial, Lancaster, England
 Beckford's Tower, Somerset, England
 Broadway Tower, The Cotswolds, England
 Bettisons Folly, Hornsea, England
 Black Castle Public House, Bristol, England
 The Cage at Lyme Park, Cheshire, England
 The Castle at Roundhay Park, Leeds, England
 Clavell Tower, Dorset, England
 Clytha Castle Monmouthshire
 The Caldwell Tower, Lugton, Renfrewshire, Scotland.
 Dunmore Pineapple, Falkirk, Scotland
 Faringdon Folly, Faringdon, Oxfordshire
 Flounder's Folly, Shropshire, England
 The Folly Tower at Pontypool, Wales
 Fonthill Abbey, Wiltshire, England
 Fort Belvedere, Surrey, England
 Freston Tower, near Ipswich, Suffolk
 Gothic Tower at Goldney Hall, Bristol
 The Great Pagoda at the Royal Botanic Gardens, Kew, London¨
 Gwrych Castle, one of Europe's largest follies, Abergele, North Wales
 Hawkstone Park, follies and gardens in Shropshire, England
 Hume Castle. Berwickshire, Scotland
 King Alfred's Tower, Stourhead, Wiltshire, England
 McCaig's Tower, Oban, Scotland
 Mow Cop Castle, Cheshire, England
 National Monument, Edinburgh, Scotland
 Old John, Bradgate Park, Leicestershire, England
 Penshaw Monument, Penshaw, Sunderland, England
 Perrott's Folly, Birmingham, England
 Pope's Grotto, Twickenham, south west London, England.
 Portmeirion, Wales
 Rushton Triangular Lodge, Northamptonshire (16th century)
 Severndroog Castle, Shooter's Hill, south-east London
 Stowe School has several follies in the grounds
 Sway Tower, New Forest, England
 Tattingstone Wonder, near Ipswich, Suffolk
 The Temple near Castle Semple Loch, Renfrewshire, Scotland.
 Wentworth Follies, Wentworth, South Yorkshire
 Williamson's tunnels, probably the largest underground folly in the world, Liverpool, England

United States
 Belvedere Castle, New York City
 Lawson Tower, Scituate, Massachusetts
 Lucy the Elephant, Margate City, New Jersey
 Bishop Castle, outside of Pueblo, Colorado
 Körner's Folly, Kernersville, North Carolina

Other websites
 The Folly Fellowship- An organization which celebrates architectural follies
Follies in the English Landscape 
Follies and Monuments - A comprehensive catalogue of Follies within the UK
 Images of follies on Odd-stuff! 
European Follies - Book to be published 2007

References

Bibliography
Barton, Stuart Monumental Follies Lyle Publications, 1972
Folly Fellowship, The Follies Magazine, published quarterly
Folly Fellowship, The Follies Journal, published annually
Folly Fellowship, The Foll-e, an electronic bulletin published monthly and available free to all
Hatt, E. M. Follies National Benzole, London 1963
Headley, Gwyn & Meulenkamp, Wim, Follies Grottoes & Garden Buildings, Aurum Press, London 1999
Headley, Gwyn & Meulenkamp, Wim, Follies — A Guide to Rogue Architecture, Jonathan Cape, London 1990
Headley, Gwyn & Meulenkamp, Wim, Follies — A National Trust Guide, Jonathan Cape, London 1986
Headley, Gwyn Architectural Follies in America, John Wiley & Sons, New York 1996
Howley, James The Follies and Garden Buildings of Ireland Yale University Press, New Haven & London, 1993
Jackson, Hazelle Shellhouses and Grottoes, Shire Books, England, 2001
Jones, Barbara Follies & Grottoes, Constable, London 1953 & 1974
Meulenkamp, Wim Follies — Bizarre Bouwwerken in Nederland en België, Arbeiderpers, Amsterdam, 1995

Buildings and structures