Garlic chives - you win!

I give up. I've battled and battled the garlic chives that keep showing up everywhere in my herb garden ever since I allowed them to go to seed before discovering it. I went so far as to pull out the plants and pull up the heavy clay chimney flues that make up the base of the herb bed so I can start again. I took out all the soil, then pulled each plant up by its root. They still kept coming, so I pulled some more. I think I might have eradicated them in a couple of pots, but they have completely taken over the space between the pots and the edging next to the brick walkway.

I am at a loss how to contain them without using a weedkiller, which I won't do around my herb bed. I bought some of that organic "weed killer" with cloves - and the chives just laughed as they licked their leaves murmuring, "Yummy!" Does anyone have any suggestions on killing them without toxins? And yes, that lone plant in the pot is a wayward Mexican petunia, another invasive plant that even pushes up through my brick and concrete walkway. My recommendation: don't plant either.