Lord Coombe is considered to be the best-dressed man in London. He is also a man whose public reputation, despite his formidable intellect and observant eye, is one of unmitigated wickedness. During one of his social forays, he meets a selfish young woman named 'Feather' with the face of an angel. Fascinated by her, he slowly drifts into her circle. When her husband dies unexpectedly, leaving her alone and desolate in London, he ends up taking her under his wing.
Feather has a daughter named Robin, of whom she takes little notice. She treats Robin with shocking neglect and once Coombe takes over responsibility for the household's finances, Feather readily abandons poor Robin to the less-than-kindly ministrations of her nurse. In fact, Robin doesn't even know Feather is her mother for her first six years, calling her 'The Lady Downstairs'. Robin also hates Coombe, having heard a conversation that blamed him for the loss of her first friend. This was a little boy named Donal who was in fact Coombe's heir. Donal's mother disapproves both of Coombe and Feather and when she discovers that her son has been playing with Robin, she whisks him away, leaving Robin heartbroken. However, Coombe does not return this dislike and in fact makes a point of serving as her guardian, albeit without Robin's knowledge. As Robin grows, he builds her a set of rooms, engages a loving nurse for her, and eventually secures a reputable governess for her. While her mother continues to behave with the selfish freedom of a wanton child.
As Robin grows, she becomes a lovely and intelligent though innocent, girl. Feather's circle includes some unsavory characters, one of whom, a German nobleman, tries to make Robin into his plaything. This caricature of Imperial German stereotypes uses Robin's desire to support herself to trap her in a house of ill repute. His plan fails mainly through the actions of Coombe, but the after-effects leave Robin crushed.
One of Coombe's few true confidants is a dowager Duchess - a woman of both great intellect and great understanding who has recently lost her long-time lady companion. After Robin's experiences with the German, Coombe suggests Robin as a suitable replacement. The Duchess is the one person who knows the secret of Coombe's determination to watch over Robin and why he is willing to tolerate the activities of her mother. This secret is finally communicated to the reader as well during one of Coombe's talks with the Duchess. The Duchess does indeed take in Robin and befriends her. Robin is introduced to the Duchess' children and their friends and finally sponsors a small dance for Robin. At the dance, Robin meets Donal again as Coombe and the Duchess learn that an Austrian Archduke has just been assassinated in Serbia.

Question: As a child, why does Robin refer to Feather as The Woman Downstairs?
Answer: Robin does not know Feather is her mother as a young child, due to Feather's negligence. 
Alternative answer: She was nothing more to him