He said the armed men wanted to use force to free southern activists detained at the police headquarters in Zunjubar.
The governor said the separatists also set a police vehicle ablaze.
Police officials said gunmen from the crowd began spraying gunfire at the policemen, who returned the fire.
Authorities imposed a curfew in the city as the clashes continued during the afternoon, witnesses said.
Violent anti-government protests have engulfed cities in Yemen's southern provinces in recent months, leaving dozens of casualties among protesters and security forces amid claims by the southerners that the central government exercises discriminatory policies against them.