This memoir tells of a precocious boy who loved the sights and sounds of his childhood but felt reticent about full participation in life. The note is sounded from the first chapters, as James recounts the limitations—and rewards—of the child he was:
"For there was the very pattern and measure of all he was to demand: just to be somewhere—almost anywhere would do—and somehow receive an impression or an accession, feel a relation or a vibration. He was to go without many things, ever so many—as all people do in whom contemplation takes so much the place of action; but everywhere...he was to enjoy more than anything the so far from showy practice of wondering and dawdling and gaping: he was really, I think, much to profit by it."
James clearly suffered from a sense of his inferiority in the "showy", active parts of life. His older brother William always seemed superior in ability; his classmates scorned his hopelessness at math and science; even at a party he felt too embarrassed to join in the dancing.
But he still observed and fantasized about all his family and his surroundings had to offer. When a cousin of his was told "don't make a scene," he suddenly realized that scenes could be made by telling a story or inventing a play. He went often to the theater and fell forever under its spell. His family met William Thackeray and Charles Dickens on their American tours, and James even remembered Thackeray mock-scolding his sister Alice for her crinoline dress: "Crinoline? I was suspecting it! So young and so depraved!"
In fact, for all his surface diffidence, James harbored almost Napoleonic dreams of glory as an artist. Near the close of A Small Boy and Others he tells how, in much later years, he dreamed of routing a monstrous attacker and chasing him away through the halls of the Louvre, which he had seen as a child. The small boy, as hesitant and uncertain as he might seem, would triumph over his fears and develop his superb aptitude for narrative fiction.

Question: What subjects were not easy for James? 
Answer: Math and science
Alternative answer: math and science