Born in New York City, Swanstrom was educated at Fordham University and at St. John's Seminary in Brooklyn, and was ordained a priest in 1928. In 1933, he earned a diploma from the New York School of Social Work, and completed his Ph.D. at Fordham in 1938. Beginning in 1933, he served assistant director of the Brooklyn Catholic Charities. In 1943, Swanstrom became director of War Relief services, an initiative of the U.S. Catholic Bishops to assist war refugees and POWs. Following World War II, Swanstrom was instrumental in the passage of a statement by the Bishops that called on Americans to aid displaced persons of whatever national background, giving them "the help which their very human rights demand." By 1955, the War Relief initiative was transformed into a permanent organization, Catholic Relief Services, with Swanstrom serving as its head. In 1960, he was appointed Auxiliary Bishop of New York. He continued to serve as executive director of CRS until 1973, participating especially in providing relief supplies to South Vietnam. He retired from his episcopal duties in 1978.