I finally found out what this is, sticking out of the potato above. There’s a better picture of it in this post about potato worms. Apparently this is a classic example of what happens, when you convert your lawn into a vegetable garden. They’re called wireworms and the problem is not limited to potatoes in particular. These worms are larvae that will eventually grow into click beetles. These beetles like to live in the lawn, and normally when in the larvae stage they will eat the roots of grass. When you remove the grass you take away their food and they start looking for other things to eat, for instance potatoes, or carrots or other root crops.
It takes about 4 to 5 years for the larvae to grow into a beetle, so be prepared to turn the soil frequently when the weather is dry and collect them, when you spot them. And choose plants that develop above ground, like beans, peas, squash etc.