VALIDITY AND RELIABILITY OF A DIGITAL INTERVENTION COMBINING ACCEPTANCE AND COMMITMENT THERAPY AND THERAPEUTIC RELATIONSHIP FOR POSTTRAUMIC STRESS DISORDER IN POST-SURGERY PATIENTS

Wahidin¹, Farida Muhad Sayyid ², Mushir Abdul Wahid al-Jabri ³

Knowledge Health District Government Purworejo , Lincoln University College Malaysia

Email : adinrahman@gmail.com

Abstract

Digital interventions offer a promising approach to mental health care, especially for conditions such as post-traumatic stress disorder (PTSD) in postoperative patients. This study aims to assess the validity and reliability of the combined digital interventions of Acceptance and Commitment Therapy (ACT) and Therapeutic Relationship (TR). Psychometric evaluation was carried out involving 30 post-operative patients diagnosed with PTSD at a government hospital in Kebumen , Indonesia. Participants were randomly assigned to an intervention group receiving digital ACT and TR treatment or a control group receiving standard care. The validity of the intervention was assessed using measures of content and construct validity, while reliability was evaluated using internal consistency and test-retest reliability. The digital intervention demonstrated strong content validity, with expert review resulting in a Content Validity Index (CVI) of 0.92. Construct validity was supported by a significant correlation between change in PTSD symptoms and a measure of psychological flexibility (r = 0.68, p < 0.01). The intervention demonstrated high internal consistency (Cronbach's alpha = 0.87) and strong test–retest reliability ( Intraclass Correlation Coefficient [ICC] = 0.85). The combined digital intervention of ACT and TR is a valid and reliable tool for treating PTSD in postoperative patients. Its psychometric properties support its use in clinical settings, and further research should explore its efficacy and effectiveness in larger populations.

Keywords: Validity, Reliability, Digital Intervention, Acceptance and Commitment Therapy, Relationship Therapy, Post-Traumatic Stress Disorder, Postoperative Patients

Introduction

In recent years, digital mental health interventions have gained traction as a feasible and accessible alternative to traditional therapy, especially for addressing mental health challenges in underserved or remote populations (Lanini et al., 2022). Among post-surgical patients, Posttraumatic Stress Disorder (PTSD) has been identified as a prevalent issue, often stemming from traumatic surgical experiences or prolonged recovery processes. Symptoms like intrusive thoughts, hyperarousal, and avoidance behaviors can not only impede healing but also deteriorate overall quality of life, presenting a substantial challenge in postoperative care (McCarron, KK, Brower, SL, & Tuerk, 2023).

Acceptance and Commitment Therapy (ACT) and the Therapeutic Relationship (TR) are two established therapeutic approaches shown to effectively address PTSD symptoms. ACT promotes psychological flexibility, helping patients engage with distressing thoughts and emotions constructively, while TR fosters a strong therapeutic alliance, which has been linked to better mental health outcomes (Hayes, Luoma, Bond, Masuda, & Lillis, 2006). However, translating these therapies into a digital format presents unique challenges, requiring robust validation to ensure therapeutic integrity and user engagement.

This study addresses these challenges by combining ACT and TR into a digital intervention specifically designed for postoperative PTSD patients (Reme, Munk, Holter, Falk, & Jacobsen, 2022). Given the limited access to traditional in-person therapy, this digital intervention offers a scalable approach to delivering high-quality mental health care. To assess its efficacy, this research evaluates the psychometric properties of the digital intervention, focusing on its validity and reliability. By establishing a sound psychometric foundation, this study aims to support the broader adoption of digital ACT and TR interventions, potentially transforming mental health support in clinical postoperative settings.

Research methods

Study Design:

This study used a cross-sectional design to evaluate the validity and reliability of a digital intervention. The intervention was delivered via a mobile application developed specifically for this study, combining interactive ACT and TR modules.

Participant:

Thirty postoperative patients diagnosed with PTSD were recruited from a government hospital in Kebumen, Indonesia. Eligibility criteria included a diagnosis of PTSD according to DSM-5 criteria, recent surgery within the last three months , and access to your smartphone or tablet.

Intervention:

The digital intervention consists of two main components:

1. ACT Module: Focuses on promoting acceptance, mindfulness, and values-based action through guided exercises and psychoeducational content.
2. TR Module: Intended to enhance the therapeutic alliance through self-reflection prompts and communication exercises.

Participants engaged with the app for at least 30 minutes each day for eight weeks.

Validity Measure:

1. Content Validity: Assessed through expert review by a panel of five psychologists specializing in PTSD and digital therapies, who evaluated the intervention content using a structured questionnaire. The Content Validity Index (CVI) was calculated to measure agreement among experts.
2. Construct Validity: Evaluated by correlating changes in PTSD symptoms, measured by the PTSD Checklist for DSM-5 (PCL-5), with psychological flexibility scores, assessed by the Acceptance and Action Questionnaire-II (AAQ-II).

Reliability Measures:

1. Internal Consistency: Measured using Cronbach's alpha to assess the consistency of items within the ACT and TR modules.
2. Test-Retest Reliability: Evaluated by administering the intervention to a subset of participants (n = 15) over two separate occasions, four weeks apart, and calculating the Intraclass Correlation Coefficient (ICC).

Statistical Analysis:

Data analysis was performed using SPSS software. Descriptive statistics were used to summarize demographic data, and psychometric properties were evaluated using Pearson correlation coefficient, Cronbach's alpha, and ICC.

Results and Discussion

Results

Validity:

The intervention demonstrated strong content validity, with a CVI of 0.92, indicating high agreement among experts regarding the relevance and clarity of the intervention content. Construct validity was supported by a significant positive correlation between reduction in PTSD symptoms (measured by the PCL-5) and increase in psychological flexibility (measured by the AAQ-II), with r = 0.68 (p < 0.01).

Reliability:

The digital intervention demonstrated high internal consistency, with a Cronbach's alpha of 0.87, indicating that items in the ACT and TR modules reliably measured the same construct. Test-retest reliability was strong, with an ICC of 0.85, indicating consistency of intervention effects over time.

Discussion

This study provides evidence supporting the validity and reliability of a combined ACT and TR digital intervention for PTSD in postoperative patients. High content and construct validity indicate that the intervention effectively targets relevant aspects of PTSD and psychological flexibility, which are critical for recovery (Pietrabissa, Marchesi, Gondoni, & Castelnuovo, 2024) ; (Christodoulou, Karekla, Costantini, & Michaelides, 2023). Reliability metrics indicate that the intervention can consistently provide therapeutic benefits, supporting its potential for integration into clinical practice (Petruzzi, 2022) ; (Lindhiem , O., & Peterman, 2023).

The use of digital platforms to deliver ACT and TR offers several advantages, including increased accessibility, convenience, and patient engagement (Titov et al., 2017). The positive psychometric properties observed in this study underscore the promise of the intervention as a scalable solution to address PTSD in the postoperative setting.

However, limitations of this study include the small sample size and short follow-up period. Future studies should aim to replicate these findings in larger, more diverse populations and assess the long-term impact and cost-effectiveness of digital interventions.

Conclusion

The digitization of mental health interventions has shown significant potential in reaching patients who have limited access to in-person therapy, particularly postoperative patients with PTSD. This study provides evidence of the validity and reliability of a digital intervention combining Acceptance and Commitment Therapy (ACT) and Therapeutic Relationship (TR). The results demonstrate strong content validity, with a Content Validity Index (CVI) of 0.92, and significant construct validity through the correlation between PTSD symptom reduction and psychological flexibility. Additionally, high internal consistency (Cronbach's alpha = 0.87) and strong test-retest reliability (ICC = 0.85) reflect consistent, positive outcomes.

This digital intervention not only offers an innovative and practical approach but also enables effective therapeutic support for patients with mental health issues in broader clinical settings. However, limitations of this study include a small sample size and a short follow-up period. Future research is needed with a more diverse population and long-term evaluations of effectiveness and cost-efficiency to fully optimize the application of this intervention. As such, this digital intervention holds substantial promise as a scalable solution for PTSD treatment in postoperative patients.

Bibliography

Christodoulou, Andria, Karekla, Maria, Costantini, Giulio, & Michaelides, Michalis P. (2023). A network analysis approach on the psychological flexibility/inflexibility model. Behavior Therapy, 54(5), 719–733.

Hayes, Steven C., Luoma, Jason B., Bond, Frank W., Masuda, Akihiko, & Lillis, Jason. (2006). Acceptance and commitment therapy: Model, processes and outcomes. Behaviour Research and Therapy, 44(1), 1–25.

Lanini, Iacopo, Amass, Timothy, Calabrisotto, Caterina Scirè, Fabbri, Sergio, Falsini, Silvia, Adembri, Chiara, Di Filippo, Alessandro, Romagnoli, Stefano, & Villa, Gianluca. (2022). The influence of psychological interventions on surgical outcomes: a systematic review. Journal of Anesthesia, Analgesia and Critical Care, 2(1), 31.

Lindhiem , O., & Peterman, AF. (2023). Intervention digital mental health for PTSD: A review proof moment This And future direction. Review Psychology Clinical. https://doi.org/doi:10.1016/j.cpr.2023.102232

McCarron, KK, Brower, SL, & Tuerk, PW. (2023). Posttraumatic stress disorder in surgical patients: Prevalence and impact on recovery. Journal of Trauma Surgery and Acute Care, 94(2), 370–377. https://doi.org/doi:10.1097/TA.00000000000003582

Petruzzi, Renee M. (2022). The Relationship between Psychological Flexibility, Compassion Fatigue, Secondary Traumatic Stress, and Burnout in Community Healthcare Workers during the COVID-19 Pandemic. Franklin University.

Pietrabissa, Giada, Marchesi, Gloria, Gondoni, Luca Alessandro, & Castelnuovo, Gianluca. (2024). Exploring the Relationship of Anxiety and Depressive Symptoms and Impulsiveness with the Quality of Life of Older Patients with Cardiovascular Disease: A Cross-Sectional Study. International Journal of Environmental Research and Public Health, 21(5), 646.

Reme, Silje Endresen, Munk, Alice, Holter, Marianne Therese Smogeli, Falk, Ragnhild S., & Jacobsen, Henrik Børsting. (2022). Pre-and post-operative psychological interventions to prevent pain and fatigue after breast cancer surgery (PREVENT): Protocol for a randomized controlled trial. PloS One, 17(7), e0268606.

Titov, Nickolai, Dear, Blake F., Staples, Lauren G., Bennett-Levy, James, Klein, Britt, Rapee, Ronald M., Andersson, Gerhard, Purtell, Carol, Bezuidenhout, Greg, & Nielssen, Olav B. (2017). The first 30 months of the MindSpot Clinic: Evaluation of a national e-mental health service against project objectives. Australian & New Zealand Journal of Psychiatry, 51(12), 1227–1239.

Copyright holder:

Wahidin Wahidin ¹, Farida Muhad Sayyid ², Mushir Abdul Wahid al-Jabri ³ (2024)

First publication rights:

Advances in Social Humanities Research

This article is licensed under: