Multilevel Self-Attention Model and its Use on Medical Risk Prediction

• Xianlong Zeng

• Ohio University USA  |  School of Electrical Engineering and Computer Science

Various deep learning models have been developed for different healthcare predictive tasks using Electronic Health Records.In most of the models, medical codes are often aggregated into visit representation without considering their heterogeneity, e.g., the same diagnosis might imply different healthcare concerns with different procedures or medications.To address these limitations, Multilevel Self-Attention Model (MSAM) is developed that can capture the underlying relationships between medical codes and between medical visits.In this task, MSAM is evaluated on two predictive tasks, future disease prediction and future medical cost prediction, with two large real-world datasets, MIMIC-3 and PFK. This multilevel self-attention model (MSAM) utilizes the self-attention mechanism at both medical code-level and visit-level, respectively utilizing the self-attention units, time embedding and the auxiliary task, MSAM is able to capture the underlying relationships among medical claims, handle the irregularity time gap between medical visits and stabilized the prediction result.

Input variables : Sequence of medical visits, visit represented by a set of medical codes
Output Variables : Diagnosis codes in the next visit for MIMIC-3 datset, diagnosis codes and medical cost in the next year for PFK dataset

Visit Model : github.com

Additional links : psb.stanford.edu