The announcement of the vote's result came just before 0100 (2000 GMT) after multiple adjournments in the lower house caused by members of Khan's party, who said there was a foreign conspiracy to oust the cricket star-turned-politician.
Opposition parties were able to secure 174 votes in the 342-member House in support of the no-confidence motion, presiding Speaker Ayaz Sadiq said, making it a majority vote. Just a few legislators of Khan's ruling party were present for the vote.
Two sources said the voting came after the country's powerful army chief General Qamar Javed Bajwa met Khan, as criticism mounted over the delay in the parliamentary process.
Opposition leader Shehbaz Sharif is the front-runner to lead the nuclear-armed nation of 220 million, where the military has ruled for half its history.
Shehbaz, 70, the younger brother of three-time prime minister Nawaz Sharif, has a reputation as an effective administrator.
Khan, 69, surged to power in 2018 with the military's support, but recently lost his parliamentary majority when allies quit his coalition government. There were also signs he had lost the support of the military, analysts said.
Opposition parties say he has failed to revive an economy battered by COVID-19 or fulfil promises to make Pakistan a corruption-free, prosperous nation respected on the world stage.
His ouster extends Pakistan's unwanted record for political instability: No prime minister has completed their full term since independence in 1947, although Khan is the first to be removed through a no-confidence vote.
A handful of women suppers of Khan protested outside parliament while supporters of opposition parties celebrated victory.
People on the streets of Karachi and Lahore had divided opinions.
Parliament will meet on Monday (April 11) to elect a new prime minister.
Opposition leader Sharif said Khan's ouster was the chance for a new beginning.
“A new dawn has started. A new day is coming. Allah has answered the prayers of millions of Pakistani mothers, sisters, daughters, elders and youth", Sharif, 70, said in parliament.
Elections are not due until August 2023. However, the opposition has said it wants early elections, but only after it delivered a political defeat to Khan and passes legislation it says is required to ensure the next polls are free and fair.
Khan earlier accused the United States of backing moves to oust him because he had visited Moscow for talks with President Vladimir Putin just after Russia launched its invasion of Ukraine on Feb. 24. Washington rejected the charge.
Sharif, who is affiliated with the Pakistan Muslim League, has been in ousted Prime Minister Nawaz Sharif's shadow during their three decades in politics, forging a reputation as a workaholic administrator obsessed with infrastructure mega-projects in the vast Punjab province, the family's powerbase that is home to more than half of Pakistan's 190 million people.
His hands-on style as chief minister of Punjab -- peppering officials with WhatsApp messages in the dead of night -- has won him, admirers, in the provincial capital Lahore, a Mughul-era city spruced up with slick highways and manicured boulevards.
Sharif's governing style is in sharp contrast to Nawaz's hands-off approach, but the two brothers espouse a similar pro-business ideology. Their biggest difference may be in their relations with the military, which plays an outsized role in Pakistani politics and currently controls key areas of policy such as relations with India and the United States.