Biometric recognition of newborn babies is an opportunity for the realization of several useful applications, such as improved security against swapping and abduction, accurate census, and effective drug delivery. This paper explores the possibility of using face recognition toward an affordable and friendly biometric modality for newborns. The paper proposes an autoencoder-based feature representation followed by problem specific distance metric learning via one-shot similarity with one class-online support vector machine.