I love this story! According to good ol' Wiki, which is sourced, this story comes from an Icelandic rímur cycle. It is also cited in the Gylfaginning.
Your story and the rest of it are from the third and fourth rímas, emphasis mine:
As Thor and his companions approach Útgarða-Loki's fotress they face a strong locked fortress gate (v. 3-6). Unable to open it, they manage to slip through the grating between the bars (v. 7). They go into the hall and Thor greets the king (v. 8-11). Útgarða-Loki welcomes them and asks them where they come from, and whether they excel at any game or sport (v. 11-14).
Loki offers to compete in eating and an opponent, Logi, is assigned to him (v. 15-16). They both eat meat ferociously, but Logi also swallows the knives and the bones and attempts to bite Loki (v. 17-20). Loki is deemed the loser and Útgarða-Loki asks for a better contestant (v. 21). Þjálfi offers to compete in running and suffers a humiliating defeat against Hugi (v. 22-27).
Thor now wants to compete in drinking, and Útgarða-Loki has a horn carried into the hall (v. 28-30). Útgarða-Loki says that the custom is to empty the horn in one go, but Thor drinks from it three times with little success (v. 31-36). Útgarða-Loki then asks Thor to lift his cat from the ground, but despite great effort, Thor can only get one of the cat's feet off the ground (v. 37-41). Útgarða-Loki asks him to get back into his seat and tells him that he has been greatly humiliated (v. 41-44).
Thor now offers to wrestle anyone who will challenge him (v. 2). Útgarða-Loki says he has a mother who once knew how to wrestle, though she is not in good health anymore (v. 3). The woman arrives, large and old, and starts wrestling with Thor (v. 4-5). Thor cannot resist her and falls to his knee (v. 5-8). The woman goes away, but Thor says he wants to go home to Ásgarðr (v. 9).
Útgarða-Loki says he will lead him away from the fortress and then explains to him that he has deceived him (v. 10-11). A cryptic verse explains that the bag of provisions was sealed with trickery (v. 12). When Thor threw his hammer at Skrímnir, revealed to be the same person as Útgarða-Loki, the blow was blocked by a mountain or it would have killed him (v. 13). Loki lost the eating contest to fire (v. 14). The drinking horn had its end out in the sea and Thor's drinking from it has resulted in the tides (v. 15-16). The cat which Thor tried to lift was actually Miðgarðsormr [the serpent which encircles the world — LI] and the woman he wrestled was Elli, old age (v. 17-18). Útgarða-Loki curses the Æsir and disappears (v. 19-20). The last verse of the cycle gives it the name Lokrur (v. 21).