Pre-heat the oven to 180°C.
Heat the olive oil in a large, ovenproof frying pan and add the potato slices. Reduce the heat to low and cover, letting the potatoes cook for 10 minutes until just soft, then remove from the heat.
Whisk together the eggs and milk with the seasoning, chopped black olives and most of the caraway seeds. Pour the egg mixture on top of the potato slices, then sprinkle the remaining caraway seeds on top. Bake for 12-15 minutes until the egg is set and the top is golden brown.
Remove and allow to cool a little before turning out onto a chopping board and cutting into square portions.
Arrange on a serving plate and serve with the green olives skewered on cocktail sticks as a garnish.