Preheat oven to 350°F (175°C). Crush crackers finely using a food processor or by placing them in a zip-top bag and crushing with a rolling pin.
In a medium bowl, mix crushed crackers with sugar and melted butter until mixture resembles wet sand.
Press the mixture evenly into bottom and up sides of a 9-inch pie dish, smoothing with the bottom of a measuring cup.
Bake crust for 10 minutes, then remove and let cool slightly.
In a bowl, whisk together sweetened condensed milk and egg yolks until smooth.
Add lemon juice and lemon zest to the mixture and whisk until fully combined and slightly thickened.
Pour filling into the baked crust and smooth the top.
Bake pie for 16 to 18 minutes until center is just set but slightly wobbly.
Let pie cool at room temperature for 30 minutes, then refrigerate at least 3 hours or overnight until fully chilled and set.
For the optional topping, whip heavy cream with powdered sugar until soft peaks form.
Spread or pipe whipped cream over chilled pie and garnish with lemon zest or candied lemon peel if desired.