James was the eldest son of Henry James, an idiosyncratic and voluble man whose philosophical interests attracted him to the theology of Emanuel Swedenborg. One of William's brothers was the novelist Henry James. The elder Henry James held an "antipathy to all ecclesiasticisms which he expressed with abounding scorn and irony throughout all his later years." Both his physical and his spiritual life were marked by restlessness and wanderings, largely in Europe, that affected the training of his children at school and their education at home. Building upon the works of Swedenborg, which had been proffered as a revelation from God for a new age of truth and reason in religion, the elder James had constructed a system of his own that seems to have served him as a vision of spiritual life. This philosophy provided the permanent intellectual atmosphere of William's home life, to some degree compensating for the undisciplined irregularity of his schooling, which ranged from New York to Boulogne, France, and to Geneva and back. The habits acquired in dealing with his father's views at dinner and at tea carried over into the extraordinarily sympathetic yet critical manner that William displayed in dealing with anybody's views on any occasion.
When James was 18 years of age, he tried his hand at studying art, under the tutelage of William M. Hunt, an American painter of religious subjects. But he soon tired of it and the following year entered the Lawrence Scientific School of Harvard University. From courses in chemistry, anatomy, and similar subjects there, he went to the study of medicine in the Harvard Medical School, but he interrupted this study in order to accompany the eminent naturalist Louis Agassiz, in the capacity of assistant, on an expedition to the Amazon. There James's health failed, and his duties irked him. He returned to the medical school for a term and then during 1867-68 went to Germany for courses with the physicist and physiologist Hermann von Helmholtz, who formulated the law of the conservation of energy; with Rudolf Virchow, a pathologist; with Claude Bernard, the foremost experimentalist of 19th-century medicine; and with others. At the same time he read widely in the psychology and philosophy then current, especially the writings of Charles Renouvier, a Kantian idealist and relativist.
The acquaintance with Renouvier was a focal point in James's personal and intellectual history. He seems from adolescence to have been a delicate boy, always ailing, and at this period of his stay in Germany he suffered a breakdown, with thoughts of suicide. When he returned home in November 1868, after 18 months in Germany, he was still ill. Though he took the degree of M.D. at the Harvard Medical School in June 1869, he was unable to begin practice. Between that date and 1872 he lived in a state of semi-invalidism in his father's house, doing nothing but reading and writing an occasional review. Early in this period he experienced a sort of phobic panic, which persisted until the end of April 1870.