The families were staying at the Talbot Hotel in LeominsterAsylum seekers moved from a hotel in Herefordshire to pods in Warwickshire say they are too small for families and have heating and water issues.
Occupants of The Talbot Hotel in Leominster, an asylum hotel run by Serco, have been dispersed to new sites after government contracts ended.
But two families who moved to Dunchurch in Warwickshire on Friday said they are cold and their children are scared.
The Home Office said it took welfare issues "very seriously".
Warwickshire County Council said the Home Office was responsible for the hotels along with Serco, and it had escalated the concerns to Serco and to the hotel manager who has advised they will investigate.
The pods are self-contained and are open plan with a kitchenette and separate bathroom.
Although the authority understood they have heating, running water and a kitchen area, it added they were "small and not suitable for families in our view".
Speaking anonymously, one asylum seeker said the two pods they were allocated - one each for the children and adults - were not suitable.
He said the pods are close to tall, thick trees which scared the children, and when the wind blows it was loud and frightened them.
"We are not sleeping the last two nights and we are in the room with my children because they are very scared," he said.
"They don't go outside because they are scared and [because] of the darkness."
He said one pod was not warm because the heater was not working and they initially did not have any water, although that has now been fixed by Serco.
The heating issue has been looked at but not yet fixed, he added.
He said he also applied for three school places in Warwickshire but was told there were no spaces.
"We need to protect the children and we want to move [from] here urgently otherwise my children, mentally and physically, they are very bad in this situation," he said.
He added that the bathroom was "very cold" and his children could not use it.
He said facilities and the local environment were good at the hotel in Leominster, which was among a number across the UK to be closed to asylum seekers by the government.
The Talbot housed up to 75 people before the Home Office terminated its contract, meaning it will cease to be used as contingency accommodation for asylum seekers by the end of February.
In a statement, the Home Office said: "The Home Office continues to provide safe accommodation for asylum seekers who would otherwise be destitute, as we work to end the use of hotels which are costing UK taxpayers £8.2million a day.
"We take the welfare of those in our care very seriously and have rigorous safeguarding processes in place.
"Any concerns raised are swiftly addressed through our work with the accommodation provider, and the Migrant Help 24/7 support line is also available to help migrants resolve any accommodation issues quickly."