Quoting directly from the Gylfaginning (verse 15), with emphasis,
- Then said Ganglere: Where is the chief or most holy place of the gods? Har answered: That is by the ash Ygdrasil. There the gods meet in council every day. Said Ganglere: What is said about this place? Answered Jafnhar: This ash is the best and greatest of all trees; its branches spread over all the world, and reach up above heaven. Three roots sustain the tree and stand wide apart; one root is with the asas and another with the frost-giants, where Ginungagap formerly was; the third reaches into Niflheim; under it is Hvergelmer, where Nidhug gnaws the root from below. But under the second root, which extends to the frost-giants, is the well of Mimer, wherein knowledge and wisdom are concealed. The owner of the well hight Mimer. He is full of wisdom, for he drinks from the well with the Gjallar-horn. Alfather once came there and asked for a drink from the well, but he did not get it before he left one of his eyes as a pledge.
That indicates that the frost giants live "where Giningagap formerly was".
From verse 43,
When the builder saw that his work was not going to be completed, he resumed his giant form. When the asas thus became sure that it was really a mountain-giant that had come among them, they did not heed their oaths, but called on Thor. He came straightaway, swung his hammer, Mjolner, and paid the workman his wages,.---not with the sun and moon, but rather by preventing him from dwelling in Jotunheim; and this was easily done with the first blow of the hammer, which broke his skull into small pieces and sent him down to Niflhel. But Loke had run such a race with Svadilfare that he some time after bore a foal. It was gray, and had eight feet, and this is the best horse among gods and men.
For at least one mountain-giant, Niflhel became home.
In another article, Wikipedia says,
A certain class of jötnar are the fire jötnar (Múspellsmegir, "sons of Muspell", or eldjötnar), said to reside in Muspelheim, the world of heat and fire, ruled by the fire jötunn Surtr ("the black one"). The main role of the fire jötnar in Norse mythology is to wreak the final destruction of the world by setting fire to it at the end of Ragnarök, when the jötnar of Jotunheim and the forces of Hel shall launch an attack on the gods, and kill all but a few of them.
This makes sense, as Muspelheim is the counterpart of Niflheim (fire vs. ice).
That said, Jötunheimr and Útgarðar are also homes of giants (with the former being more important). Curiously enough, though, the giant mentioned who was thrown to Niflheim was originally from Jötunheimr. Note that Jötunheimr and Útgarðar together comprise one of the Nine Worlds; Niflheim and Muspelheim are two other of the worlds. This makes it hard to determine which are older. This says that Ymer (Ymir) was formed from the meeting of Niflheim and Muspelheim, indicating that those are the first homes of the giants (because Ymer was the first being).