Met Police officers were called by the London Ambulance Service to the scene at the Abbey estate near St John’s Wood just after 7.30pm on Friday.
The victim, who has not been named, was found injured and given first aid by paramedics at the scene.
He was then taken to hospital but died later that evening.
Parts of the estate, near London’s famous Abbey Road, were still cordoned off at nightfall on Saturday amid a heavy police presence as “scared” neighbours watched on.
Officers with torches appeared to be searching for something, with some using a ladder to peer over walls and into nearby gardens.
No arrests have been made “at this early stage”, Scotland Yard said.
The family of the victim have been informed and a post-mortem examination will take place in due course.
Felicia Adarkwa, 65, who lives on the estate, told the PA news agency: “I heard a noise, there were arguments. I was going to pop the window but something told me not to.
“There was heavy noise, there was shouting, boys and girls. I didn’t watch them. They were arguing.
“Really sad. It’s horrible.”
Sheila Gallagher, 69, said she was watching television in her flat when she noticed police were on the estate.
“I was watching the soaps and I didn’t hear a sound and then I came out and saw all the police around, that was after 9pm,” she said.
“I looked out of my kitchen window and saw police. It’s a bit scary.”
Another resident, Abdul, 50, said: “Generally it’s not a good area but the last few months it has been quite nice.
“Of course it’s scary. I’ve got young children, you straight away think about your own children.”
Detective Chief Inspector Neil Rawlinson, who is leading the investigation, said: “We would like to speak to anyone who was in the area of Abbey estate on Friday night between 7pm and 8pm, who may have seen or heard something they believe could be connected to this fatal stabbing.
“Another family have been left grieving the loss of a loved one due to knife crime.
“My team of detectives are working around the clock to identify and arrest the person responsible.”
There were 104 homicides in the capital between January 1 and December 7 this year, the force said.
Anyone with information is asked to call the Met on 101.
