Groban's second album
Closer, also produced by Foster, was released on November 11, 2003. Groban said that he believed that this second album was a better reflection of him and that his audience would be able to get a better idea of him personally from listening to it. "What most people know about me, they know through my music. This time, I've tried to open that door as wide as possible. These songs are a giant step closer to who I really am and what my music is all about. Hence the title."

Two months after Closer was released, it rose on the Billboard charts from number 11 to number one. His cover of Brian Kennedy's "You Raise Me Up" became very popular on the adult contemporary charts. Groban also performed the song "Remember" (with Tanja Tzarovska) on the Troy soundtrack, "Believe" on the soundtrack to the 2004 animated film The Polar Express, and a cover of Linkin Park's "My December".
During the summer of 2004, Groban returned to Interlochen, where he gave a performance to local residents and campers, also speaking about his experiences as a young performer. On November 30, 2004, his second live DVD, Live At The Greek, was released. It also ran as a Great Performances special on PBS. Also in 2004 , Groban performed "Remember When It Rained," backed by a full orchestra, at the American Music Awards where he was nominated for Favorite Male Artist in the pop category. Groban and his recordings were nominated for more than a dozen awards in 2004, including the American Music Award, World Music Award, Academy Award, and a Grammy.
Other appearances have included The Oprah Winfrey Show, The Ellen Degeneres Show, Jay Leno, Larry King Live, The Rosie O'Donnell Show, 20/20, The Today Show, Macy's Thanksgiving Day Parade, Super Bowl XXXVIII, and the Rockefeller Tree Lighting.
During the first week of September 2006, Groban's latest single entitled "You Are Loved (Don't Give Up)", was exclusively released to AOL's First Listen. It was taken from his third studio album Awake, which was officially released on November 7, 2006. Josh Groban performed "You Are Loved (Don't Give Up)" as well as two other tracks from "Awake" at his recording session for Live From Abbey Road at Abbey Road Studios on 26 October, 2006. His "Awake" world tour visits 71 cities between February and August 2007, and travelled further to Australia in September and October of 2007. He did a duet with
Barbra Streisand ("All I Know of Love") and also in 2007 a duet with
Mireille Mathieu ("Over the Rainbow"). Groban has expressed an interest in performing on Broadway one day.
In June 2007, Josh spent some weeks in London recording a Christmas album with the London Philharmonic and the African Children's Choir, which he discussed on the DVD from "The Making of Noel". It was released on October 9, 2007 and is titled "Noel"