Gaines was born in Paducah, Kentucky to Lester and Olivia Bolen Gaines. As a young teenager, he helped his family by working in a garage while in high school. He attended local Lincoln High School where he excelled academically, played basketball, and was All-State football player. He graduated as class salutatorian in 1941.
After high school, he decided to enroll at Morgan State College (now Morgan State University), a historically black college in Baltimore, Maryland. While at the school, he was given the name “Big House,” because of his height and weight. Gaines played as a lineman for the Bears football team, was a member of the basketball team, and participated in track. Gaines was an All-CIAA selection as a lineman in football all four seasons and twice elected an All-American. When it came to basketball, he said he was “a very average basketball player.”
He graduated from graduated from Morgan State in 1945 with a Bachelor of Science in Chemistry. He intended to go on into dental school, however his college football coach, Edward P. Hurt, suggested that he temporarily go to what was then known as Winston-Salem Teachers College. At the time, the small southern college had one coach for all sports, Brutus Wilson, who was also a Morgan State graduate; Hurt suggested that Gaines would make a good assistant coach. Gaines agreed and went to Winston-Salem.
In 1946, Wilson left for Shaw University, leaving Gaines as the head coach for football and basketball, athletic director, trainer, and ticket manager. He also taught. He served as football coach for three years (1946–49), and in 1948 was named Central Intercollegiate Athletic Association (CIAA) “Football Coach of the Year” after leading his team to an 8-1 season. He dropped coaching football to focus on basketball in 1949.
He was inducted into the Basketball Hall of Fame in 1982. He is the only African-American inducted in the Basketball Hall of Fame as a coach. After winning the national title in 1967, he was named the NCAA Division II College Coach of the Year. Clarence Gaines died on April 18, 2005.