Stress detection from social media text

2026Benchmark case studyPyTorchHugging FaceBERT · RoBERTa · DistilBERT

Before I could benchmark the models, I had to benchmark the dataset.

01The dataset, inspected before trusted

This case study uses the public benchmark from “Stress Detection from Social Media Articles: New Dataset Benchmark and Analytical Study” (IEEE WCCI 2022, IJCNN track), which pairs a Reddit corpus with a Twitter corpus that the authors filtered into “advertisement” and “non-advertisement” splits.

Reading samples before training turned out to be the most valuable hour of the project: the “non-advertisement” Twitter split was still full of ads. Training a stress classifier on promotional tweets teaches it to detect marketing, not stress, so I dropped the Twitter data entirely and ran the study on the Reddit subset, where the labels held up.

02The benchmark

On the Reddit data I fine-tuned and compared three transformers: BERT, RoBERTa, and DistilBERT, head-to-head under the same training setup. RoBERTa won at 97.2% accuracy and 0.997 ROC-AUC, with BERT close behind and DistilBERT trading a little accuracy for a lot of speed.

Training and validation loss curves and validation accuracy for BERT, RoBERTa, and DistilBERT over three epochs — Three epochs of fine-tuning: RoBERTa pulls ahead on validation accuracy while validation loss creeps upward for all three models — the early-overfitting signal that capped training.

Test-set confusion matrices: BERT 96.5%, RoBERTa 97.2%, DistilBERT 95.9% accuracy — Test-set confusion matrices. RoBERTa: 97.2% accuracy, 0.997 ROC-AUC. BERT: 96.5%. DistilBERT: 95.9% at roughly half the inference cost.

Read the full report (PDF) ↗

03The part nobody benchmarks: cost

The interesting decision isn’t “which model is most accurate” — it’s whether the last 1% of accuracy is worth 2× the inference cost. DistilBERT runs roughly twice as fast as RoBERTa; for a screening tool processing high message volume, that can decide the deployment. I measured per-class F1, confusion matrices, and inference time side by side so the trade-off is a documented decision, not a default.

The takeaway: a model report that only shows the best number is marketing. A model report that shows what the number costs is engineering.

04What came next

This study was the warm-up for my master’s thesis, which asks a harder question: detecting workplace stress in Indonesian text, where the linguistic signals differ from the English data most published models are trained on. The thesis (a hybrid of transformers and psycholinguistic features, with SHAP interpretability) is in progress and unpublished — the numbers on this page are from the benchmark case study above, not from the thesis.