Bryant was born in Memphis on April 4, 1942; his father, Edward, sang with a gospel group called the Four Stars of Harmony, and Bryant himself began singing in church at age five. In 1955, he and four brothers formed their own gospel group, the Five Bryant Brothers, which soon switched to doo wop under the name the Quails.
Not long after, Bryant joined several high school friends in another doo wop group, the Four Canes, who were named after their manager, local DJ D. Cane Cole. When the group split with Cole, Bryant changed their name to the Four Kings, and they caught the attention of trumpeter/bandleader Willie Mitchell, who began performing with them around the Memphis club scene. Through Mitchell, Bryant got his original composition "Is There Someone Else on Your Mind" recorded by the 5 Royales and made his first solo recording in 1964 with a cover of Chris Kenner's "I Like It Like That."
Over the next five years, Bryant recorded several duets with Marion Brittnam and numerous cover songs, the latter of which constituted his first full-length LP, 1969's Precious Soul (issued on Hi). However, his behind-the-scenes career soon became more lucrative, specifically when Mitchell assigned him to work with new Hi signee Ann Peebles. No longer the label's focal point, Bryant resisted at first, but quickly discovered a creative and personal chemistry with Peebles that blossomed into a relationship in 1972 and marriage two years later.
In the meantime, Bryant co-wrote a number of Peebles' signature songs, including "I Can't Stand the Rain" and "99 Pounds"; his success led to a staff songwriting position, where he penned material for other Hi stars like Al Green, Syl Johnson, O.V. Wright and Otis Clay.