North Korean fishing boats often breach the inter-Korean maritime frontier or go adrift towards the South.
However, South Korean authorities towed the ship that crossed the border on Saturday because a white towel was tied to its mast in a potential sign that those on board might want to defect, officials said.
The North Koreans on the vessel said during questioning they had gone off course by mistake and wanted to return home.
South Korea notified Pyongyang that the crew members had willingly left a South Korean port for the North and would be returned later on Monday, the ministry in charge of inter-Korean affairs said in a statement.
On Sunday, North Korea released a Russian fishing boat and 15 Russian and two South Korean crew members who were held after being accused of violating entry regulations.