The son of Reverend William Barroll the 4th, James Barroll was a prominent Baltimore citizen who married Mary Ann Crockett in 1812. Their second daughter Mary Ann Barroll (1817-1844) married Lewis Washington, the great-grand-nephew of George Washington in 1836. Barroll invested in civic enterprises such as the Phoneix Fire Insurance Company and the Savings Bank of Baltimore, and he owned Bowley’s Wharf, where he erected a warehouse and acted as a commission merchant. Barroll also served as an officer during the War of 1812, fighting with the First Baltimore Hussars in the Battle of North Point in 1814.