Nearly 85 percent of the refugees have been living with relatives or in hired accommodation while around 280,000 remain in makeshift camps.
Contingents of police were escorting the refugee convoys to the battle-hit areas, from where the military, backed by attack helicopters, would provide security cover.
'Enemies of Pakistan will be looking for chances to hamper the return, but we have put in place a comprehensive security plan along the entire route,' Colonel Nadeem Ahmed said.
The military also deployed vehicle scanners to search buses and trucks moving towards Swat.
Prime Minister Yousuf Raza Gilani announced Thursday the plan for the phased return of displaced people to areas cleared of militants.
The two-month offensive was widely supported by political parties and the public, besides earning praise from Islamabad's Western allies.
More than 1,600 Taliban fighters have been killed in the operation, according to the military, which also claims to have recaptured most parts of Swat and its nearby districts. However, isolated clashes are continuing between troops and the Taliban remnants.