We worked with different transformer architectures, using EHR data as input to predict the next patient’s visit. Our work consisted in pre-training with a large dataset and fine-tuning with the domestic violence task. Having a pre-trained model, we were capable of fine-tuning with less amounts of data for new tasks (that can include respiratory problems, heart diseases or else). This scalability at a minimum cost is one of the main benefits of the approach we selected.
Additionally, we worked with attention layers visualizations to add explainability to our predictions. This means that we not only predict why a patient could return but also analyze what in the patient’s history helps us come to that conclusion.