According to an eyewitness, the fire broke out overnight in Lawanson area of the Nigerian commercial city.
The couple, grandmother, a little baby and a girl were killed by the fire.
The only survivor was rescued alive and is currently receiving treatment at an undisclosed hospital.
Due to the extent of the injuries, the survivor has not been able to communicate how the fire probably started.
As things stand now, the only thing we are certain of is the death toll which stands at five.
The location of the incident is 23, Michael Ogun Street, Lawanson, Lagos.
Confirming the incident, the spokesman for the National Emergency Management Agency (NEMA), Ibrahim Farinloye, said that the rescued girl was about 6 years old.
Mr Farinloye said that the police were making efforts to trace the immediate family to inform them of what had happened.
Within 24 hours, 12 fire incidents were recorded in Lagos State, a situation that the NEMA official said was human related.
“Averagely, in Lagos, we record not less than 10 fire incidents on a daily basis but in the last 24 hours the casualties have been very, very high.
“Mostly, the cause are human factor – carelessness, lack of safety consciousness, lack of proper handling of fire related materials which go out of hand, especially at odd times, exposing children and other vulnerable persons,” Mr Farinloye explained.
He urged Nigerians to ensure that anything that might lead to danger was properly taken care of.
“Take extreme caution when cooking, turn off all electrical appliances before going to bed, ensure that mobile phones are not left plugged and close to the pillow or foam around where you are sleeping or close to anything that is inflammable,” he advised.