The Blue-Winged Wasp can sting, and this is more likely done by females. Attempts to handle them will elicit that defensive reaction, but they are otherwise considered not aggressive. It would be best to just admire them as they work and allow them to protect your flowers from a very destructive garden insect, the Japanese Beetle.
Both insects have established large ranges in North America. Japanese Beetles are notorious for chewing through the petals and leaves of a variety of flowers, most notably roses and hibiscus. They are rapid breeders and difficult to eliminate from a flower bed. The Blue-Winged Wasp is a natural predator of this beetle (as well as other beetle species) because its larvae eat the beetle grubs. A female wasp will dig up a beetle grub and sting it to paralyze it. She then digs a burrow around the grub and lays a fertilized egg into the hole with the grub. Once finished, she sets out to repeat the process on another beetle grub. Once each wasp larva hatches, it eats the paralyzed, yet still living grub, starting with non-essential organs first.
Adults are most active in the summer. By day, they visit flowers and drink nectar. At night, they rest on plants and may socialize with other Blue-Winged Wasps. Males tend to curl up around a twig or blade or grass in order to rest.