Coleman attended a Virginia military school before studying law and serving in the army. While attending the University of Texas, Coleman became attracted to acting, and headed to New York, where he studied at the Neighborhood Playhouse. After stage experience and TV work, Coleman made his movie debut in 1965's The Slender Thread.
Minus his trademarked mustache for the most part in the mid-1960s, Coleman specialized in secondary character roles. He began to branch into comedy during his supporting stint as obstetrician Leon Bessemer on the Marlo Thomas sitcom That Girl, but his most memorable role would come in 1980 as the nasty, chauvinistic boss in 9 to 5. He would go on to appear in other films, like On Golden Pond , The Beverly Hillbillies , You've Got Mail , and Moonlight Mile, but the actor found more success in television, appearing on a few cult hits that were tragically cancelled, like Drexell's Class and Madman of the People, as well as The Guardian, Courting Alex, Heartland, and Boardwalk Empire.