CRNN-Based Modeling of Mental Fatigue from Multimodal Signals: A Systematic Review

Abstract:-Mental fatigue—characterized by reduced vigilance, slowed reaction time, and cognitive inefficiency—poses safety and productivity risks in transportation, healthcare, and industrial settings. Recent work leverages convolutional–recurrent neural networks (CRNNs), typically coupling convolutional layers (for spatial/spectral feature extraction) with recurrent layers (LSTM/GRU for temporal dynamics), to model mental fatigue from multimodal signals such as EEG, EOG, ECG/PPG, fNIRS, eye tracking (PERCLOS), facial video, and behavioral/interaction traces. This systematic review synthesizes CRNN-based methods for mental fatigue estimation from 2013–2025, covering sensing modalities, input encodings (e.g., spectrograms, wavelets, topographical maps), fusion strategies (early/late/hybrid attention), learning paradigms (supervised, transfer, self-supervised), evaluation protocols (within-subject vs. cross-subject), and deployment aspects (real-time/edge). We summarize representative datasets (SEED-VIG, DROZY, NTHUDDD, FatigueSet, and others), identify typical performance regimes, and highlight open problems in label quality, domain shift, personalization, interpretability, and ethics. We conclude with a set of practical recommendations and a forward-looking agenda that integrates self-supervised pretraining, lightweight architectures, and robust human-in-the-loop evaluation

---