I'll start out by noting that Amazons come from Greek mythology but the character known as Wonder Woman is a modern invention, inspired by, but not present in the Classical canon.
The first clue is her name, Diana. Diana was the Roman name for the Greek goddess Artemis, famous for her archery skills. Key point: Diana is not an Ancient Greek name.
Wikipedia has a very comprehensive list of actual names of Amazon warriors from mythology. You'll see names like Deianeira and Dioxippe, but not Diana.
Although Wonder Woman is a modern invention, the legends of the Amazons do derive from actual Greek mythology. The most famous Amazons are surely Penthesilea and her sister, Hippolyta, who took Theseus as her husband. (As a Wonder Woman fan, you will know that her mother is named for this actual, mythological figure.)
Themiscyra does feature in the ancient Greek histories as a legendary town of the Amazons as early as the 5th century BCE. (The Greek histories are entwined with the mythological canon in that the mythical origins of places are often discussed.)
Themiscyra is not associated with Atlantis in the greater Classical canon, although there is a similarity in that Themiscyra's location is a subject of some dispute.
Regarding Atlantis, it also dates from the Classical period, although it is not a location of mythological stories per se, but first appears in Plato as part of an allegory on national hubris.
There is no evidence that Atlantis was anything other than an invention of Plato, although many have speculated to the contrary. For instance, the island of Thera is often put forth as an actual location. Thera was famously subject to a massive volcanic eruption and subsequent submersion of most of the island under the sea. It's also worth noting that Minoan civilization was a pre-cursor of Greek civilization--an advanced culture based on an island dominating the region--so the idea of an Atlantis may also be related to the impact of Minoan hegemony in the ancient Greek cultural consciousness.
In fact, the famous mythological Amazon Hippolyta is linked to Theseus, and Theseus is linked to Minoan Crete through the story of the Minotaur. The tribute of
the Athenian youths to be devoured by the Minotaur is an example of the former domination of the Minoans over the Greek tribes.
So, no, Atlantis is not a location that figures in the formal canon of Classical Mythology, but has subsequently been widely mythologized.
You can find an additional discussion on the location and nature of Atlantis here.