Peony Pests

My favourite flower is the peony. Last year I had several varieties in pots and they were thriving. In the autumn I planted the tubers into the ground. I had hoped that they would all come back with even more blooms this spring. Sadly only two have survived, I don't know why.

Every year the ants seem to love the flower heads as they prepare to bloom. I don't know if they are doing any harm, I don't think so, do they?