Mix flour, rosemary, salt, and pepper together; add cold butter and blend until mixture resembles coarse crumbs. Drizzle cold water into mixture and mix until dough comes together. Transfer dough to a bowl and refrigerate, 1 hour to 2 days.
Heat oven to 400° F. Pierce each sweet potato several times with the tines of a fork. Place the sweet potatoes on a greased baking sheet. Bake until tender, about 45 minutes. Drain and run cold water over sweet potatoes; carefully remove peels.
Stir milk and olive oil together in a bowl.
Beat sweet potato and softened butter together in a bowl using an electric mixer until smooth. Stir milk-olive oil mixture, eggs, Parmesan cheese, and garlic into sweet potato mixture until smooth; season with salt and pepper.
Preheat oven to 350 degrees F (175 degrees C).
Roll dough on a floured surface using a rolling pin. Roll dough around rolling pin and press in a pie dish, trimming edges. Pour sweet potato filling into crust.
Bake in the preheated oven until a knife inserted in the middle of the filling comes out clean, 55 to 60 minutes. Cool pie about 5 minutes before serving.