He was born in Brooklyn, New York, to William P. Chambers (1827–1911), a notable corporate and bankruptcy lawyer, and Caroline Smith Boughton (1842-1913). His parents met when Caroline was twelve years old and William P. was interning with her father, Joseph Boughton, a prominent corporate lawyer. Eventually the two formed the law firm of Chambers and Boughton which continued to prosper even after Joseph's death in 1861. Robert's great-grandfather, William Chambers (birth unknown), a lieutenant in the British Royal Navy, was married to Amelia Saunders,(1765-1822), the great grand daughter of Tobias Saunders, of Westerly, Rhode Island. The couple moved from Westerly, to Greenfield, Massachusetts and then to Galway, New York, where their son, also William Chambers, (1798-1874) was born. The second William graduated from Union College at the age of 18, and then went to a college in Boston, where he studied to be a doctor. Upon graduating he and his wife, Eliza P. Allen, (1793-1880) a direct descendant of Roger Williams, the founder of Providence, Rhode Island were among the first settlers of Broadalbin, New York. His brother was architect Walter Boughton Chambers.