EFFICIENT MULTI-MODAL TRANSFORMER HYPER-PARAMETERS OPTIMIZATION FOR STRESS DETECTION

Abstract

Transformers demonstrate great potential for physiological signal analysis. However, their use of multi-class stress classifications is limited, especially in terms of deployment on constrained resource platforms. In this work, we present Efficient-HusFormer, a novel transformer-based architecture developed with hyper-parameter optimization (HPO) for multi-class stress classification using the WESAD and CogLoad dataset. The main contributions of this work are: (1) the design of a structured search space and local optimization strategy based on a priority assumption, targeting effective hyperparameter optimization of the number of layers (L), heads (H), dimension (d_m), and feed-forward network dimension (FFN); (2) a comprehensive ablation study evaluating the impact of architectural decisions across combinations of pairwise, triplet, and four-module configurations; (3) consistent performance improvements over the original HusFormer, with the best configuration achieving an accuracy of 88.41% and F1-score of 0.8815, corresponding to absolute gains of 9.73 percentage points in accuracy and 8.64 points in F1-score. The best-performing configuration is achieved with the {L+d_m} modality combination on WESAD dataset, using single layer, 3 attention heads, a model dimension of 18, and FFN dimension of 120, resulting in a compact model with only ~30k parameters and 575MB of memory. These results imply HPO is an important part of developed transformer-based solutions for physiological computing. The full implementation of Efficient-HusFormer is publicly available on GitHub to support reproducibility and further research in physiological computing.