The following is a list of deities regarding whom I have found no occurrences of stories about them being unfaithful to their spouses.
(It might be a noteworthy consideration that most of them are female deities.)
The sea-god Nereus and his wife the Oceanid Doris
The sea-goddess Eurybia, wife of the Titan Creius
The Titaness Tethys, wife of her own brother Oceanus
The Titaness Mnemosyne, Goddess of Memory, who, in Hesiod's Theogony, is Zeus's fifth wife
Erebus, the primaeval personification of darkness, who was married to the night-goddess Nyx
Hemera, the primordial personification of daytime, who was married to Aether
Uranus, the original sky-god, who was married to his own mother Gaia, the Earth
Sidenote 1: The poetess Sappho says that Eros was the son of Uranus and Aphrodite, but Theoi.com interprets this as meaning that Aphrodite herself was born pregnant, with Eros in her womb, she herself having formed out of the severed genitalia of Uranus mingling with the salt-foam of the sea. Besides, the more common parentages of Eros do not involve Uranus.
The river-god Hydaspes, in India, who was married to the nymph Astris
The Arcadian river-god Ladon, who was married to Stymphalis
The Sicilian river-god Anapus, who, in Ovid's Metamorphoses, is married to the nymph Cyane
A few Oceanids: the aforementioned sea-goddess Doris; Electra (wife of the sea-god Thaumas); Meliboea (wife of King Pelasgus of Pelasgia, which later became Arcadia); and the youngest Oceanid Eidyia (wife of King Aeetes of Colchis).
Iris, wife of Zephyrus
Hebe, who married Heracles [Hercules] after he became a god
The Nereid Thetis, wife of the Myrmidon king Peleus
The Nereid Iphianassa, wife of King Enydmion of Elis (though there are different versions of who this king's wife was).
Rhode, wife of Helios
Benthesicyme, who married a king of Aethiopia [Ethiopia]
Nephele, wife of the Boeotian king Athamas
Sidenote 2: Regarding Eros: in Book 32 of his Dionysiaca, Nonnus quotes Hera as devising an elaborate fiction by which to deceive Zeus. In it she says that Eros has become infatuated with the Oceanid Rhodope so much so that he has even abandoned his duties as the god of love.
Since Nonnus does not mention Psyche, who, besides that, appears to be a purely allegorical character invented by Apuleius, it is unclear whether Eros is at this point is to be understood as being married. The point might be moot, however, since the business with Rhodope could all be part of Hera's lie.
Sidenote 3: Algaia (or Aglaea) shouldn't be on your infidelity examples list, by the way. She was legitimately married to Hephaestus after he had divorced Aphrodite.