A further 141 people were injured when the bomber rammed a car packed with explosives into two diplomatic vehicles entering the embassy in the centre of the Afghan capital, Reuters reported.
The car exploded near a row of metal turnstiles outside the embassy where dozens of Afghans line up each morning to apply for visas.
It was the deadliest attack in Kabul since the fall of the Taliban in 2001. The death toll was the highest for an attack in Afghanistan since a suicide bomber killed more than 100 people at a dog-fighting competition in Kandahar province in February.
"The interior ministry believes this attack was carried out in coordination and consultation with an active intelligence service in the region," the ministry said in a statement. Afghanistan has previously accused Pakistani agents of being behind a number of attacks on its soil.
The Afghan president, Hamid Karzai, last month threatened to send troops across the Pakistani border to attack militants there if Pakistan did not take action.
Pakistan's intelligence agency has been accused of involvement in terrorist attacks in India in the past, although relations between the two countries have improved recently.
Pakistan supported the Taliban when they ruled Afghanistan and only withdrew backing for the militant group in the face of US pressure following the September 11 attacks.
It denies Afghan accusations that it is still secretly backing the Taliban and strongly condemned today's bombing.
A number of nearby shops were destroyed or damaged in the blast and smouldering ruins covered the street.
"Several shopkeepers have died. I have seen shopkeepers under the rubble," said Ghulam Dastagir, a shopkeeper who was wounded in the blast.
Abdullah Fahim, spokesman for the Afghan ministry of public health, said the explosion killed at least 28 people and wounded 141. The ministry collected information from the scene and several Kabul hospitals. A senior police official later told Reuters 41 people had died.
Shortly after the attack, a woman ran out of a Kabul hospital screaming, crying and hitting her face with both of her hands.
Her two children, a girl named Lima and a boy named Mirwais, had been killed.
The Afghan foreign minister, Rangeen Dadfar Spanta, visited the embassy shortly after the attack, ministry spokesman Sultan Ahmed Baheen said.
"India and Afghanistan have a deep relationship between each other. Such attacks of the enemy will not harm our relations," Spanta told the embassy staff, according to Baheen.
The Indian ambassador and his deputy were not inside the embassy at the time of the blast, he said.
In Delhi, foreign minister Pranab Mukherjee said four Indians, including the country's military attache, were killed in the attack.
"The government of India strongly condemns this cowardly terrorists' attack on its diplomatic mission in Afghanistan," the ministry of external affairs said in a statement.
"Such acts of terror will not deter us from fulfilling our commitments to the government and people of Afghanistan."
Indian offices and projects have been frequent targets for militants in the insurgency that followed the 2001 ousting of the Taliban.
But Kabul had been largely spared from the bomb attacks that have afflicted Afghanistan in recent months as the Taliban have regrouped and grown in strength.
In September 2006, a suicide bomber near the gates of the interior ministry, close to the Indian embassy, killed 12 people and wounded 42 others. After that blast, additional guards and barriers were posted on the street.
Today's attack came 24 hours after a US air strike allegedly killed 27 members of a wedding party in eastern Afghanistan.
The US military said it was investigating the incident and another on Friday on the Nuristan-Kunar border, also in eastern Afghanistan, in which missiles fired from a helicopter killed or wounded 20 people.
But the US military said it did not believe anyone other than combatants were killed.
"Whenever we do an air strike, the first thing they're going to cry is 'air strike killed civilians', when the missile actually struck militant extremists we were targeting in the first place," First Lieutenant Nathan Perry said. "At this time, we don't believe we've harmed anyone except for the combatants."
More than 2,100 people - mostly militants - have been killed in insurgency-related violence in Afghanistan this year. More than 8,000 people died in attacks last year, according to the UN, the most since the 2001 US-led invasion.
In June, militants in Afghanistan killed more US and Nato troops than those in Iraq, in a fresh spate of rebel attacks.