She was born Victoria Regina Spivey in Houston, Texas, the daughter of Grant and Addie (Smith) Spivey. Her father was a part-time musician and a flagman for the railroad; her mother was a nurse. Her sisters were Addie "Sweet Peas" Spivey, also a singer and musician, who recorded for several major record labels between 1929 and 1937; and Elton Island Spivey, who also followed a professional singing career.

Victoria Spivey's first professional experience was in a family string band led by her father in Houston. She also played on her own at local parties and, in 1918, was hired to accompany films at the Lincoln Theater in Dallas. As a teenager, she worked in local bars, nightclubs, and buffet flats, mostly alone, but occasionally with singer-guitarists like Blind Lemon Jefferson. In 1926, she moved to St. Louis, Missouri, where she was signed by Okeh Records. Her first recording, "Black Snake Blues", did well, and her association with the record label continued. She made numerous Okeh sides in New York until 1929, then switched to the RCA Victor label. Between 1931 and 1937, more recordings followed on the Vocalion and Decca labels, and, working out of New York, she maintained an active performance schedule. Spivey's recorded accompanists included King Oliver, Louis Armstrong, Lonnie Johnson, and Red Allen. She recorded many of her own songs, which dwelt on disease and crime. images.

Depression did not put an end to Spivey's musical career, but she had found a new outlet for her talent in the year of the crash, when film director King Vidor cast her to play "Missy Rose" in his first sound film, Hallelujah!. Through the 1930s and 1940s, Spivey continued to work in musical films and stage shows, often with her husband, vaudeville dancer Billy Adams, including the Hellzapoppin' Revue.

In 1951 Spivey retired from show business to play the pipe organ and lead a church choir, but she returned to secular music in 1961, when she was reunited with an old singing partner, Lonnie Johnson, to appear on four tracks on his Prestige Bluesville album, Idle Hours. The folk music revival of the 1960s gave her further opportunities to make at least a semblance of a comeback. She recorded again for Prestige Bluesville, sharing an album Songs We Taught Your Mother with fellow veterans Alberta Hunter and Lucille Hegamin and began making personal appearances at festivals and clubs. In 1962 she and jazz historian Len Kunstadt, launched Spivey Records, a low-budget label dedicated to blues and related music. They recorded prolifically such performers as Sippie Wallace, Lucille Hegamin, Otis Rush, Otis Spann, Buddy Tate and Hannah Sylvester, as well as newer artists including Luther Johnson, Brenda Bell, and Larry Johnson.

In March 1962, Bob Dylan contributed harmonica and back-up vocals, accompanying Victoria Spivey and Big Joe Williams on a recording for Spivey Records. The recordings were released on Three Kings And The Queen (Spivey LP 1004) and Kings And The Queen Volume Two (Spivey LP 1014). (Dylan was listed under his own name on the record covers.) In 1964 Spivey made her only recording with an all-white band: the Connecticut based Easy Riders Jazz Band, led by trombonist Big Bill Bissonnette. It was released first on an LP and later re-released on compact disc.

The Spivey Records archives were digitally re-mastered for the re-launch of the label in 2007.

Spivey married four times; her husbands included Ruben Floyd and Billy Adams.

Victoria Spivey died on 3 October 1976, at the age of 69, from an internal hemorrhage.