Ecological dilemma

I cherish all gods creatures..well almost, spiders are gently removed in cupped hands to the outdoors, tired bees are offered wet sugar and passing hedgehogs will find a dollop of cat food at the door,.. But wasps I really struggle with, do I let this beautiful structure grow and make access to the shed impossible for the next six months or do I beat 7 shades of shit out of it and hope they find another happy place?