The term 'Celtic' means mythology from the British Isles, so are the gods from Irish, Welsh, and Scottish mythology all considered part of the same pantheon? For example, did people who believe in the Irish Celtic Goddess Morrigan also believe in the Arawn, the Welsh Celtic God of the Underworld? Or do the Welsh, Irish, and Scottish believe in different pantheons of gods altogether and we simply lump them all in the Celtic category?
Celtic refers to any of the peoples who spoke Celtic languages, and this includes France, parts of Germany and Austria, and northern Italy. The peoples of Ireland and Britain are called Insular Celts, because they live on islands off the European coastline.
And no, they did not believe in the same gods, although you can often find similar types of gods and goddesses in different places. For example, you can find the young warrior god Lugh in Ireland, Lugus in Gaul/France, and Llew in Wales. They aren't exactly the same god, but they share characteristics.