There is an ongoing court case where a number of foreign domestic workers are arguing that they, having fulfilled the residency requirement, should have the right to apply for permanent residence in Hong Kong. Some politicians have claimed that, should the court rule in favour of the foreign workers, up to 500000 foreign domestic workers (and family members) would migrate to Hong Kong. They proclaimed that the incoming "wave" of people would take away jobs, especially minimum wage jobs, from the locals, lengthen wait times for public housing, overrun the education and healthcare system, and abuse the welfare system (by applying for welfare payments).
While the number sounds staggering, it appears that the people supplying the figure assume that every foreign domestic worker who is eligible to apply for permanent residence would do so should the courts rule in favour of them. However, having observed the way of life in Hong Kong for seven years at the minimum, eligible foreign domestic workers would realize that there are many reasons why taking up permanent residence in Hong Kong does not necessarily mean a better life for them.
If a foreign domestic worker successfully apply for permanent residence, one would be free to seek outside work opportunities. Most of them, from what I have observed, have very limited Chinese writing and reading skills. This would severely limit their competitiveness in the job market. Should they choose to seek outside work, they would be required to find room and board of their own. With Hong Kong's sky-high rents, their new residence would almost certainly be more cramped and less comfortable compared to their residence at present and they would have to pay for water, gas, and electricity. Domestic workers generally eats what their employers eat, as food costs is paid by the employer. When they seek outside work, their meal choices would be more limited, due to stricter monetary constraints. This means that, in spite of (likely) higher pay they would probably have less money to wire to their relatives in their homeland and have a harder life should they choose to search for outside work.
Most of the foreign domestic workers in Hong Kong originated from the Philippines, Thailand, and Indonesia, countries where citizens actually get to elect their (political) leaders. It would be, to me at least, quite difficult to convince someone to renounce citizenship to a state where one get to vote for the state's president/prime minister to take up permanent residence in a place where the locals have little, if any, say in choosing its head of government. Perhaps this is why the local government and the pro-Beijing parties/politicians, who obstruct democratization of the Hong Kong political system in every way possible, tend to be more vocal in stating the negative consequences of the court siding with the foreign domestic workers, worrying that the foreign domestic workers would support the parties and people demanding faster democratization, further reducing the pro-Beijing camp's vote share.
I am sure foreign domestic workers prefer living their family over being alone in a foreign land. Thus many locals are worried that foreign domestic workers would help their (immediate) family apply for local residence. The locals main worry is that the families would then apply for welfare payments and public housing. The fact that no family can apply for general welfare and public housing if fewer than half the family members are permanent residents of Hong Kong means that any foreign domestic worker with children would be ineligible for such benefits (unless the children were born in Hong Kong). The spouses of childless domestic workers would likely think very hard before choosing to move to a foreign land, where the living conditions may not necessarily be better than those back home.
Even if a foreign domestic worker is able to find one's own work and housing after successfully applying for permanent residence and has no interest in politics, one can find it difficult to integrate oneself into society. One may find oneself shunned by one's new neighbours and ends up congregating only with people with similar backgrounds. One's children, should they be allowed to come to Hong Kong, might find themselves isolated or targeted by local students, especially those whose parents have strong sentiments against giving foreign domestic workers the right to apply for permanent residence, at school. These can lead to the new residents forming a ghetto, interacting only with people with the same origins.
Foreign domestic workers should be allowed to apply for permanent residence in Hong Kong if they are indeed eligible. However, with the above, and the strong sentiments of many in town, in mind, it is highly doubtful that more than a handful of them will do so should the court rules in their favour.