Reinforcement Learning Approach to Sedation and Delirium Management in the Intensive Care Unit

Niloufar Eghbali Zarch; Tuke Alhanai; Mohammad Mahdi Ghassemi

Abstract

Common treatments in Intensive Care Units are distressing and involve prolonged sedation. Maintaining adequate sedation levels is challenging and is prone to errors such as incorrect dosing, omission or delay of sedatives, and administering the wrong sedative. In this study, we applied a reinforcement learning (RL) approach to retrospective data, developing a sedation management agent. The agent's goal is to maintain an adequate level of sedation while also keeping the Mean Arterial Pressure (MAP) as an indicator of the vital sign of the patients in the safe therapeutic range. While most of the prior work has focused on a specific medication - propofol - which has no intrinsic analgesic effect and must be co-administered with an opioid or other analgesics for ICU patients, in this work, we build a recommender system to find the optimal concurrent dosage regimen for three commonly used sedatives(propofol and midazolam) and opioid(fentanyl). To mitigate the potential risk of delirium and the adverse effects of over sedation, we integrated a delirium control variable into the reward function. The results indicate that our approach successfully recommends sedative dosages by improving the maintenance of the patients' target sedative level by 29\% compared to the clinicians' policy.

Keywords: sedation management; reinforcement learning; optimal medication dosing; medical decision making

Links
[Full text PDF][Bibtex]