Heat the whole milk and sugar in a medium saucepan over low heat, until just bubbly. Place yolks in small mixing bowl and whisk until yolks are pale and fluffy. Temper the eggs by gently whisking a small amount of the warm cream into the yolks. Pour the tempered yolks into the saucepan, whisking gently. Cook over medium-low heat until thick, stirring constantly, about 5 minutes. Remove from heat.
Gradually pour the heavy cream into the custard mix and stir to combine. Whisk in lime juice and vanilla. Cover with lid or plastic wrap and refrigerate ice cream mix until well chilled.
Freeze the mixture according to your ice cream maker’s instructions. As it freezes, add in the chopped up graham cracker sticks. Transfer the ice cream to a freezer-safe container and allow it to harden for several hours or overnight.