Authors - V. Machaca, J. Grados, K. Lazarte, R. Escobedo, C. Lopez Abstract - The interaction between peptides and the Major Histocompatibility Complex (MHC) is a critical factor in the immune response against various threats. In this work, we fine-tuned protein language models like TAPE, ProtBert-BFD, ESM2(t6), ESM2(t12), ESM2(t30), and ESM2(t33) by adding a BiLSTM block in cascade for the task of peptide-MHC class-I binding prediction. Additionally, we addressed the vanishing gradient problem by employing LoRA, distillation, hyperparameter guidelines, and a layer freezing methodology. After experimentation, we found that TAPE and a distilled version of ESM2(t33) achieved the best results outperforming state-of-the-art tools such as NetMHCpan4.1, MHC urry2.0, Anthem, ACME, and MixMHCpred2.2 in terms of AUC, accuracy, recall, F1 score, and MCC.