A gunman shot and killed a soldier at a war memorial in Ottawa, before police engaged in a gun battle inside the parliament building.
Police confirmed a gunman was shot dead, but Canada's capital remained on alert through much of the day.
PM Stephen Harper said the country "would not be intimidated" by violence.
The shooting came hours after Canada raised its terror threat level. On Monday another soldier was killed in a hit-and-run attack by a Muslim convert.
The country earlier this month announced plans to join the US-led campaign of air strikes against Islamic State (IS) militants in Iraq.
There is no confirmation any of this week's attacks are linked to IS or the new military campaign.
But Mr Harper, speaking late on Wednesday, said the attack would strengthen Canada's resolve in the fight against extremism alongside its allies.
"They will have no safe haven," he said.
Mr Harper was addressing MPs at the time of the shooting but was safely evacuated.
Citing unnamed Canadian officials, US and Canadian news agencies identified the dead gunman as Michael Zehaf-Bibeau.
On Wednesday morning, two soldiers guarding the memorial came under fire from a man carrying a rifle.
One soldier, Cpl Nathan Cirillo, died of his injuries. Three other people were treated in hospital and released by evening.
Minutes later, dozens of shots were fired inside the parliament building, Canadian MP Marc Garneau told the BBC.
Canadian parliamentary waiter Alain Merizier described seeing a dark car stop outside parliament's centre block and a driver with "a long gun" get out and run inside the entrance of the building, pursued by a parliamentary officer.
He said: "I was astonished more than frightened. You don't have time to be afraid."
MP John McKay described the moment the gunman attacked parliament: "There was a pop, pop, pop sound so the guards ushered us to the back of the building.
"How the gunman was able to walk down the hall of honour inside parliament with a rifle will become an area of investigation."