Does the idea of ice giants stem from glaciers? They are described as both evil and good and I thought that goes back to how glaciers can grow bigger and take your land, but also can become smaller and offer you land, because not only they give new space but the melted water also makes it so plants and so on can grow, like Ymir and his blood, kinda. But I'm not sure if that is a plausible theory, especially since I'm not familiar with the time frames of glacial periods. For this to work, I guess, the earliest Indogermanics would have had a lot of contacts with glaciers.
The last Ice age glaciers were gone at least by 5000 BCE. This is about 500 years before Proto-Indo-European is usually dated, which is about as far we can trace any shred of mythological knowledge, but which AFAIK does not include any Frost giants. Thus, no connection there.
Next step is the speakers of Proto-German. They are thought to have lived in southern Scandinavia and northern Germany, from about 500 BCE. They could most likely have come into contact with glaciers; there are today several in Norway, even in the southern part of the country. Proto-Germanic had a word for glacier, so they must have known of them.
But did these speakers of Proto-German have any concept of Frost Giants? That, we can not know. The first reliable knowledge of Norse mythology comes from Tacitus Germania, which mentions a few gods identifiable with deities in the Norse pantheon. Archeological finds with images that can be clearly identified with them are younger. However, this does not really tell us anything about giants: they could be older or younger than the Aesir and Vanir.
We can reconstruct several Proto-Germanic words for "giant", such as for jotun, which comes from "eater". The word for "Frost Giant" was "hrimþurs", where "hrim" meant "frost", and "þurs" is a word of more obscure origin but which means giant or monster. Again, there is no reason to connect it to glaciers.
The first sure mention of these Frost giants I've found is in Snorri's Edda from the 11th century CE. There, they are not in any way connected to glaciers. They are, however, mentioned to be evil. In general, though, most of the Norse stories we have show how giants and gods fight with each other, not with humans (at least not until Ragnarök).
To summarise: there is no direct reason, based on the myths or etymology, to suppose any connection between frost giants and glaciers. Glaciers, in general, does not reveal especially fertile lands - it is still far too cold next to them for the Norse to have had any use of the land for anything except maybe grazing. They also retreat rather slowly. However, there was (and still is) an ongoing process by which new, fertile land was revealed: the Plost-glacial rebound. This is especially noticeable in the areas around Stockholm and Uppsala in Sweden. However, I know of no myth that accounts for it.
The date of the Ice age, as well as the archeological finds of pcitures of Norse gods, I found in Sveriges Historia, 13000 f Kr -600 e Kr. The dating of PIE in Ola Wikander, Ett träd med vida grenar. The etymologies as per links in the text, as well as Svenska Akademiens Ordbok.
I found a good answer in Brazilian Wikipedia and it matches with other sources, here is a piece:
"The first living being that was formed in the primeval chaos, called Ginungagap, was a giant of monumental size, called Ymir. When she fell asleep for the first time, a daughter and a giant son grew from their armpits, while their two feet copulated, being born of them a monster with six heads. Supposedly, these three beings gave rise to the race of the hrímþursar (giants of the rhymes or icy giants), that populated Niflheim, the world of the mist, the cold and the ice."
Some information about Loess; relevant to glaciers and fertility:
Loess is a clastic, predominantly silt-sized sediment that is formed by the accumulation of wind-blown dust. Ten percent of the Earth's land area is covered by loess or similar deposits.
Loess tends to develop into very rich soils. Under appropriate climatic conditions, it is some of the most agriculturally productive terrain in the world.
Periglacial (glacial) loess is derived from the floodplains of glacial braided rivers that carried large volumes of glacial meltwater and sediments from the annual melting of continental icesheets and mountain icecaps during the spring and summer. During the autumn and winter, when melting of the icesheets and icecaps ceased, the flow of meltwater down these rivers either ceased or was greatly reduced. As a consequence, large parts of the formerly submerged and unvegetated floodplains of these braided rivers dried out and were exposed to the wind. Because these floodplains consist of sediment containing a high content of glacially ground flour-like silt and clay, they were highly susceptible to winnowing of their silts and clays by the wind. Once entrained by the wind, particles were then deposited downwind. The loess deposits found along both sides of the Mississippi River Alluvial Valley are a classic example of periglacial loess.
During the Quaternary, loess and loess-like sediments were formed in periglacial environments on mid-continental shield areas in Europe and Siberia, on the margins of high mountain ranges like in Tajikistan and on semi-arid margins of some lowland deserts like in China.
Non-glacial loess can originate from deserts, dune fields, playa lakes, and volcanic ash.