Abstract We introduce a three-wave, in-the-wild multimodal dataset for affect sensing that integrates smartphone sensing, wearable sensing, and dense experience-sampling-method (ESM) labels collected annually from 2020 to 2022. The dataset supports moment-level affect modeling through a shared dimensional label core across all waves, with additional affective descriptors available in the third wave (D-3). We describe the resource in terms of study design, temporal density of in-situ labels, and sensing and label coverage across waves. To support evaluation within this resource, we define an initial three-setting benchmark spanning temporal prediction from within-user history, within-wave cross-user generalization, and cross-wave generalization in which each wave is treated as a separate dataset. Our benchmark results show that the strongest method family depends on the evaluation setting: supervised baselines perform best in the temporal setting, unsupervised domain adaptation is strongest overall in the within-wave cross-user setting, and domain generalization shows the strongest overall cross-wave performance, although its margin over strong baselines is modest. These findings indicate that robust mobile affective computing is constrained not only by label availability but also by substantial participant-level variability and realistic cross-wave differences inherent in longitudinal in-situ deployments.

Abstract Mobile sensing systems for mental health leverage ubiquitous behavioral data to monitor states like stress and anxiety, enabling just-in-time interventions and context-aware care. However, the real-world scalability of these systems is severely limited by poor cross-user generalization—a challenge often attributed to interpersonal differences but rarely quantified systematically. We present CrossShift, a structured empirical framework for decomposing and measuring user-level covariate, label, conditional, and concept shifts in mobile mental-health sensing. CrossShift unifies shift testing, shift quantification, model evaluation, and shift-to-generalization attribution within a common analytical framework tailored to this domain. We evaluate our framework across eight diverse datasets, comprising three newly collected and five public mobile sensing repositories. Our analysis reveals that phone usage, sleep and social-interaction features are the most consistent sources of covariate shift across users. Crucially, we find that concept shift—the variation in the mapping from behavior to mental state—is the primary driver of performance degradation. Our findings suggest that scalable mobile mental health sensing requires moving beyond one-size-fits-all models toward shift-aware system design. We further discuss how CrossShift can support out-of-distribution user detection and selective personalization in real-world deployments.