Modi's Hindu nationalist party won just three of the 70 seats in the Delhi assembly in elections held over the weekend, dealing a setback to his efforts to consolidate power and push through much-needed economic reforms.
Former Delhi chief minister Kejriwal's Aam Aadmi (Common Man) Party won 67 seats, according to official results released hours after Modi conceded defeat and promised the city's new leader his government's "complete support".
Analysts said the result was a blow for a prime minister who has enjoyed an extended honeymoon with voters since his landslide general election victory.
Kejriwal's win marks a stunning comeback for the anti-graft champion and self-styled anarchist, who resigned following a chaotic 49-day spell in charge of the Delhi state government a year ago.
"Thanks for the unprecedented victory," Kejriwal, a former tax official-turned politician, told hundreds of cheering supporters outside AAP headquarters in Delhi.
"But it's very scary, the kind of support the people of Delhi have extended and the mandate we have been given, I appeal to the AAP workers and leaders not to be arrogant," Kejriwal said, as supporters showered him in petals.
After apologising this month for leaving voters without an elected government for a year, Kejriwal was the star of the election campaign, outshining a former policewoman who was the BJP's pick for chief minister.
Rattled by Kejriwal's popularity, Modi headlined several major rallies, portraying his rival as a "back-stabber" who betrayed voters last time round by quitting so early.
But Kejriwal's pledges to tackle entrenched corruption and lower utility bills won over legions of working-class voters willing to give him a second chance. He will be sworn in as chief minister on Saturday.
Congress, which has dominated politics since India's independence, suffered another mauling after being thumped at the general election, winning no seats.