Preheat the oven to 350 degrees.
Place the graham crackers in a large plastic bag and seal. Using a rolling pin (or an empty bottle of wine) roll over the crackers until they are smooth as sand. You can also pulse them in a food processor.
Mix together in a large bowl with the brown sugar and melted butter. Make sure the entire mixture is combined.
Press the crust mixture into an empty pie pan. To really seal the crust, use the back of a measuring cup and press down the crumbs to pack it into a crust.
Bake in the oven for 10 minutes. Remove the crust and let it cool slightly.
While the crust is cooling, whisk together the separated egg yolks and the lime zest.
Pour in the sweet condesned milk, then the key lime juice. Whisk until combined.
Once you're finished preparing the filling, the crust will be ready. Pour in the key lime filling and place it back in the oven.
Bake for 17 minutes. The middle will be slightly jiggly when you take it out, do not worry. It will settle.
Let it cool completely on a cooling rack (about 30 minutes).
Place in the fridge and let it sit for at least 1 hour before eating. Serve with dollops of heavy whipped cream.