Some sources have placed the number at 66 while another source placed it at 70.
A Police source, Harun-ur Rashid, said that rescuers managed to save some of the passengers, adding that the search is still on for more survivors.
More than half of the bodies retrieved so far from the river were women and children, Harun said.
Police have seized the trawler and arrested the captain and his two crew, he said.
“The ferry was recovered … and there were 23 bodies inside,” Harun said.
Harun said that there might be more bodies trapped inside the ferry noting that the rescuers from the Bangladesh Inland Water Transport Authority and Red Cross were still searching the river.
The ferry was heading to Paturia from Daulatdia in Rajbari, located on the opposite bank of the river, about 135 km (85 miles) west of the capital, Dhaka.
Shajahan Khan, the shipping minister, told reporters at the scene that an investigation had been launched.
The exact number of people aboard the boat is not known, as Bangladeshi ferries do not usually have formal passenger lists.