In a stand mixer fitted with the whisk, beat the butter until smooth then add the sugar and continue to beat until the mixture is light and airy. Add the eggs one at the time, beating well after each addition, and then whisk in vanilla.
Mix together the flour, butter, baking powder, and salt in a medium bowl. Add dry ingredients to the butter mixture and whisk on low until fully incorporated.
Remove the dough and separate it into three equal dough balls. Shape one section of the dough into a 4 x 4 inch square, wrap with plastic wrap, and refrigerate. Add one of the other sections of dough back to the mixer. Add in the blue coloring and mix until the dough is the shade of blue you want. Wash the mixer and whisk and repeat with the dough and red food coloring. Shape the blue and red layers into squares and wrap like the first one.
Refrigerate the three dough squares for 30 minutes to an hour.
Remove the dough from the refrigerator. On a piece of parchment paper sprinkled with flour, place the white slab and sprinkle flour over the top of that slab. Place another piece of parchment paper on top of the slab and roll out the “white” piece of dough as uniformly as possible into a rectangle 1/8-inch thick. Place to the side and repeat with the blue and red slab.
Peel the top layer of parchment paper off the white dough. Peel the top layer off the blue dough and flip the dough side down on top of the white dough. Remove the parchment from the then top of the blue dough. Remove the top parchment from the red dough and flip it onto the blue dough. Do not remove the second parchment paper from the top of the red dough. Lightly roll the 3 layers together until the dough is about a 1/8- to 1/4-inch-thick rectangle.
Starting at the shorter end of the rectangle, roll the dough as tightly as possible into a log, using the bottom parchment paper to help you roll it tightly. Wrap the log of dough in the parchment and chill in the refrigerator for 3 hours or overnight.
Preheat the oven to 350°F. Line a baking sheet with parchment paper. Remove the dough from the refrigerator, set on the counter, and use a sharp knife to slice the roll into 14-inch slices. Place the sliced dough 2 inches apart on the cookie sheet.
Bake cookies for 10 minutes or until golden brown. Transfer the baked cookies from the pan to a cooling rack and allow them to cool to room temperature. Serve with milk.