There are several species of common wasps native to Connecticut. Several of them are described here. The paper wasps (the ones pictured below), and another which is shaped the same but is colored like a yellow jacket (called the Mediterranean Wasp - often confused with true yellow jackets). The Mediterranean wasp is an introduced species, that is slightly more agressive for territory, so it is slowly pushing out our native dark brown wasp. The Mud Dauber wasp is an dark iridescent bluish color. Wasps are very beneficial because they feed on caterpillars such as gypsy moth, and cabbage worm caterpillars. The mud dauber wasp feeds almost exclusively on spiders, so if you don't like spiders they are good to have around. They will not sting! Wasps in general are not particularly aggressive, but can be a problem if their nest is near a doorway, in a swing set for kids, etc.
There are dozens of uncommon wasp species including digger wasps, parasitic wasps, cicada killers, etc., all  of which are basically harmless and will not sting. I have included information about cicada killer wasps because they are very large, and look terrifying as they fly over the surface of the ground above the burrows they dig, in sandy soil, and although they are capable of stinging, they are essentially harmless.