Three Elgin families were evacuated from their homes on Christmas Eve after part of a nearby derelict building crashed to the ground.
Strong winds are thought to have caused a large section of the old Fleming’s Sawmill to collapse at 11pm.
Linkwood Road was left covered with several tonnes of bricks and roofing materials.
The emergency services sealed off the road and used heat-seeking equipment to ensure there were no casualties under the rubble.
Two of the families who had to leave their homes were understood to have gone to stay with friends or relations. It is believed they were allowed back into their homes last night.
Moray Council was involved in finding emergency accommodation for the other family.
A police spokesman said: “A substantial section of the building had fallen on to the road and the collapse is thought to be due to the strong winds experienced in the area.”
A structural engineer arrived at the site shortly after the building collapsed to assess the damage.
A 200-yard section of Linkwood Road was cordoned off and the debris was cleared away.
The road remained closed for most of yesterday, with police stationed at barriers at both ends.
Pedestrians were allowed to walk along the road in the morning but were told to stay on the opposite pavement in case of a further collapse.
But at 11am, the road was sealed off as work began to make the building safe.
A digger was used to pull down sections of the sawmill without causing damage to the houses next door.
The building is opposite Elgin Auction Mart and has been derelict for many years.
One neighbour said he had not heard the building collapse, but added that the winds on Christmas Eve were the worst to hit the area for some time.
The sawmill and its grounds were sold by a local businessman to Glasgow-based developer Elphinstone several years ago.
The company planned to build a 70,000sq ft retail development on the site.
Under the £20millionproposals, the sawmill would have been demolished and replaced by shop units and parking for 310 cars.
But the plan was rejected in 2009, with councillors expressing fears that another out-of-town development would take trade away from the centre of Elgin.
The development firm went into administration the following year.