While he was a freshman at the University of Dayton in 1960, it was revealed that he was a friend of Jack Molinas, a former Columbia All-American who was banned from the NBA for betting on games and point shaving. While it was never proven (or even alleged) that Brown participated in point shaving on any level, he did receive improper benefits, and the sanctions against him were levied. This placed Brown in a basketball wasteland. It would all change when the American Basketball Association formed, as that league had no such sanction, and at age 25, he was the first player signed by the Indiana Pacers.
Brown was back in the game he loved, and with a chip on his shoulder, he would be an All-Star in his rookie season. The Small Forward would become a fixture on the Pacers team that won the 1970, 1972, and 1973 ABA Championship. He would be a First Team All-ABA Selection in 1971 and would have an 18.0 Points per Game Average with three seasons that eclipsed the 20 Point mark.After his career was over, he would be named to the All-ABA Team, again, an incredible accomplishment considering he wasn't allowed to play on a professional level in the United States for years. Brown would be inducted into the Naismith Basketball Hall of Fame in 2013.