ACT for Pain
I have had many failures replicating the Hayes et al (1999) pain tolerance study. Over the years I have answered criticisms of reviewers in my attempt to have negative results published. As a result I have begun to video record my interventions for analysis by anyone who is interested. I now take a whole host of adherence measures and subjective reports. I have even examined acceptance over the long run in the lab. But still no effects on pain tolerance for ACT-derived protocols over placebos or alternative treatments (relaxation and education or supression). often no effects for acceptance-based interventions at all.
I would love to offer practical suggestions but as you will see from my list I have five years of research here - so I am trying! I am all out of suggestions. I also think that if the procedure has to be contrived so much that effects are only measureable on such measures as the AAQ (which measures what ACT teaches the client - ipso facto we will see changes in scores) or if the research requires a clincian as experimenter or highly elaborate and exhaustive subjective ratings and statistical techniques - its not a very powerful effect! So I hesitate to contrive procedures much mroe complex than what I have got. (The details of which I hope we will discuss as a community in reposne to these postings).
So here is my list of failures....
Failure 1
Examining the effectiveness of acceptance and control – based interventions on pain tolerance.
This study compared the effectiveness of an acceptance-based and control-based intervention on pain tolerance using a cold pressor task, and is a part-replication and extension of the Hayes et al., (1999) study. Twenty college students were exposed to the cold pressor task before, immediately after, and 20 minutes subsequent, to an 8 minute acceptance-based or control-based therapeutic intervention, including the use of physical and abstract metaphors. Half of the participants were also assigned to a high demand characteristic condition in which the experimenter purposely placed subtle social pressure on them to please the experimenter. The results showed that the most significant factor influencing performance on the cold pressor task was the effect of placing social pressure on participants, with no significant overall effect for Acceptance or Control interventions.
Failure 2
A Systematic Analysis of the Role of Demand Characteristics in an Acceptance Based Approach to Pain Tolerance.
This study compared the role of demand characteristics in an acceptance-based approach to pain tolerance and both the long and short-term effects of the acceptance-based versus the no therapy interventions. Forty participants were exposed to a cold pressor task before and immediately after a short intervention. Twenty-eight participants also completed a follow up task three months later. Half of the participants receiving each intervention were also subject to high levels of demand characteristics. In this high demand condition the experimenter placed subtle social pressure on the participants to perform well on the second cold pressor task. The findings showed that participants in the acceptance condition improved more, but not significantly more, than those in the no therapy condition. Participants in the high demand condition performed significantly better than those in the low demand condition. Interaction effects for therapy x demand were also found between the experimental groups.
Failure 3
Comparing the Effectiveness of Acceptance and Control Strategies for Pain Tolerance with a Sub-clinical Population
This study used an experienced psycho synthesis therapist and cognitive behavior therapy postgraduate as an experimenter who had studied ACT and taken the full ACT weekend workshop. This study was a part replication and extension of the Hayes, Bissett, Korn, Zettle, Rosenfarb, Cooper and Grunt (1999) study. Four sub-clinical volunteers (two smokers, one drinker and a tantrum thrower) were exposed to a cold pressor task before, immediately after, and several weeks subsequent to a 90 minute acceptance-based therapeutic intervention. Baseline rates of idiosyncratic problem behaviours were also recorded prior to, and for several weeks subsequent to, the initial intervention. The acceptance-based intervention was then administered weekly for up to 25 weeks by a qualified cognitive-behavior therapist to asses its impact on pain tolerance and target problem behaviour rates in the longer term. The acceptance-based intervention showed weak effects on pain tolerance during all phases of the study and no discernible effects on problem behaviors were observed (in fact they got worse!).
Failure 4
Examining the effectiveness of an acceptance and relaxation-based intervention on pain tolerance
This study attempted to compare the effectiveness of an acceptance and relaxation-based intervention on pain tolerance to a cold pressor task and is a partial replication and extension of the Hayes et al., (1999) study. Forty college students were exposed to a cold pressor task before and immediately after an eight minute acceptance-based and relaxation-based intervention. Half of the participants in each group were also assigned to a high demand condition, in which subtle social pressure was placed on the participants to please the experimenter and do well in the task. The results confirmed that the most significant factor influencing performance on the cold pressor task was placing social pressure on participants to do well. There was no significant overall effect for either the acceptance or relaxation-based intervention although both produced mild improvements in pain tolerance.
Failure 5
The effectiveness of an acceptance and control-based interventions on pain tolerance at two different levels of pain in a cold pressor task.
This study employed the now standard procedure of using 40 subjects – half get an acceptance protocol and half get a control-based protocol following a baseline cold pressor task and before a post-intervention cold pressor task. Half of each group get cold pressors at 0 degrees centigrade – the remainder at 3 degrees. Acceptance has a mild effect – not significant – and does not interact with temperature on an ANOVA. This dashed our hopes that maybe acceptance was more useful for intense pain over mild pain – no lab data to support that idea yet.
Failure 6
So maybe it’s the subject’s fault! Assessing the effectiveness of acceptance and control based interventions with anxious and non-anxious subjects.
We recruited 20 high and 20 low trait anxiety subjects by screening with the STAI. We defined high and low as one SD above and below the mean score as outlined in the standardized distribution scores. Half of each got a control intervention and half got acceptance. Anxious subjects did not benefit more than non anxious from either the acceptance or the control intervention on a cold pressor task. Overall no effect for acceptance. No effects were found on any subjective reports.
- Printer-friendly version
- Login or register to post comments
User login
Upcoming Community Events
- EABCT Helsinki, Finland - Steve Hayes(Event)(3 days)
- ACT Practical Skills Development Sessions with Mary Sawyer - Sydney(Event)(3 days)
- ACT Core Skills and Competencies Workshop: Utilizing Acceptance and Mindfulness in Your Practice - Portland, OR(Event)(6 days)
- Ciclo Formativo sull'ACT: Workshop introduttivo Teorico ed Esperienziale all'Acceptance and Commitment Therapy (due giornate)(Event)(11 days)
- Caring for PTSD in Primary Care(Event)(12 days)
- Openings for ACT Supervisor Consultation Group - Robyn Walser, Ph.D., September 19, 2:00 EASTERN(Event)(12 days)
- Two-day Introductory Experiential ACT Workshop with Russ Harris in Boulder, Colorado September 28, 29 2008(Event)(21 days)
- ACT Workshop, Las Vegas - Steve Hayes(Event)(25 days)
- Introductiecursus ACT in Blooming te Bergen (NH)(Event)(26 days)
- ACT 3-day workshop w/Kelly Wilson; Bath, UK(Event)(26 days)
- Getting Behavioral about ACT: An Online Crash-Course in Contemporary Behavior Analysis(Event)(29 days)
- ACT in Italy: Kelly Wilson due giornate di Workshop a Milano(Event)(30 days)
- Kelly Wilson 3 days in France (Aix-Les-Bains)(Event)(32 days)
- Introductory ACT workshop(Event)(33 days)
- ACT Core Skills and Competencies Workshop: Values-Directed Behavior Change - Portland, OR(Event)(34 days)
- Using Mindfulness & Acceptance to Create a Powerful Therapeutic Relationship - Tempe, AZ, Steve Hayes(Event)(47 days)
- Portland, Maine: Introduction to ACT by Joel Guarna & Elizabeth Kubik(Event)(47 days)
- L'émergence de la 3e vague en thérapie cognitive et comportementale:Implications théoriques et pratiques- F. Dionne PhD-Montreal(Event)(55 days)
- Acceptance & Mindfulness in Clinical Practice, Steve Hayes - Vancouver(Event)(57 days)
- ACT Core Skills and Competencies Workshop: Case Conceptualization in ACT with Application to Difficult Clients - Portland, OR(Event)(62 days)
- ABCT - Orlando(Event)(67 days)
- ACT introductiecursus Amersfoort/Utrecht - Netherlands(Event)(75 days)
- Acceptance & Mindfulness in Clinical Practice, Steve Hayes - Oregon(Event)(89 days)
- Brief Therapy Conference - San Diego(Event)(95 days)
- Vervolgcursus Acceptance and Commitment Therapy in Blooming, Bergen (NH)(Event)(145 days)
- Introductiecursus Acceptance and Commitment Therapy - Bergen, Holland(Event)(173 days)
- Introductiecursus Acceptance and Commitment Therapy - Bergen, Holland(Event)(201 days)
- ACT introductiecursus Utrecht - Netherlands(Event)(264 days)
Comments
Other Ideas
A couple of other things that might be going on:
We have learned some things about how to get acceptance effects over the last few years. I've been warning lots of folks lately that it is dangerous to walk folks through an acceptance rationale only -- there is not enough in the culture to know what to do and sometimes people do strange things under that label. Some really short acceptance interventions have worked (I think Barlow's lab has another one in BRAT this month for example), but we have ourselves done studies with acceptance instructions and when we check on the manipulation we find they have lead to more suppression! ("Oh. OK. Acceptance. I guess I just have to put up with it. Yeah. OK. I'll just not think about it" etc). I recall that Dermot found the same thing until he started requiring iterative summaries of what the subjects thought the instructions meant.
So, first, what happens in the check on the manipulation? What do they think you mean? What are they actually doing?
Second, the Japanese study I mention in my other post showed that you need specific defusion / acceptance exercises, not just a rationale alone. Are you doing that and making sure that this is balanced across conditions?
Third, we now know that the exercises can't just be general ... You can start there but they then have to be targetted toward the specific problem you ar dealing with. We found that in the defusion study in BRAT in 2004 (Masuda, A., Hayes, S. C., Sackett, C. F., & Twohig, M. P. (2004). Cognitive defusion and self-relevant negative thoughts: Examining the impact of a ninety year old technique. Behaviour Research and Therapy, 42, 477-485.) So are you targeting the intervention and doing so equally across conditions?
I have a sense there are other issues like this ... but you get the idea. As I think about what you've written in light of issues like this, about the only way good way to deal adequately with your request for input is to post your manuscripts and protocols. Reviews would be nice, but they are not really necessary and I understand if that seems too private. The summaries you've written just do not have the detail needed to say anything helpful.
If you don't want to post them to this site, you could just email them to the listserve. Either way they should get some good discussion and probably some good ideas
- S
Steven C. Hayes, University of Nevada
The results of my digging in
I have begun to dig in and would like to address the substantive issues I see so far.
Analogs are hard to do. You need to refine your methods over time to get them well controlled and to target the processes of interest. I've been an academic for 30 years. I'd say looking at all labs and all studies I've ever known early preparations have about a 30% chance of being workable. That may be generous. I'm not talking ACT or RFT. I'm talking anything.
Once you get a preparation well controlled you can use it to ask good questions. But unfortunately there are a thousand ways to get it wrong.
On another page under "ACT/research" called "How to do ACT Micro-Studies and Analogs" that contains a list of features to attend to on experimental analogs of ACT processes. These are worth reviewing.
Turning now to the list of failures that have been posted.
Let me just start by saying again that the pain finding does appear to be replicable, both in the lab and the clinic. Lance McCracken at Bath has a new effectiveness study out showing large effects with people who have been in chronic pain for more than 10 years and he has another coming with some of the most truly chronic and severe pain patients he could find. It’s careful work and Lance is a pretty cautious person. His CPAQ and values work is solid empirically. JoAnne Dahl, Rikard Wicksell and many others are getting supportive clinical results too.
The same is true in the lab. Collaborative efforts of Maynooth, Galway, and Almeria have lead to automated and extremely well controlled studies with all the bells and whistles of tight experimental science. Part of what is exciting to me about it is that as the controls get better and the methods get tighter the results seem to get clearer.
Learning how to refine and control methods take time. Some of the protocols now being used were developed from painstaking work conducted by Carmen Luciano and her students in Spain. Students from Yvonne and Dermot's lab presented "failed" analog studies at ABA last year in order to share with others how they worked through the process of refining experimental procedures. Similar work was also presented at the second Summer Institute in Philadelphia, and there was an entire workshop devoted to this at a UK ACT training institute last year. In any case, it appears that the methodological flaws in the early failed studies coming from these other labs have now been worked out and the results are a lot clearer. Some of files from the Spanish-Irish collaborative efforts will hopefully be up on the website soon (they are very large). People around the world will be able to continue to refine our knowledge.
There are several more very large lab studies coming from the Irish and Spanish labs. A pain study using a radiant heat pad with 108 subjects and effects for acceptance against distraction and placebo.
A large defusion study (136 subjects) defusion instructions from exercises and comparing distraction and placebo. Clear effects for defusion exercises.
A two-experiment study (automated/taped/real spider etc) using a graduated spider approach task that found that acceptance/defusion produces significant improvement in approach distraction and placebo do not.
A study under review right now showing neurobiological evidence for experiential avoidance.
I’ve read one of these and it was very well done methodologically. I can't wait to read the others! The authors have a proven track record of independent research studies that have been proven to be of high quality. In any case, given what is out and what is coming, replicability as such does not appear to be a problem in labs across the world.
So why the six sequential failures from a single lab?
None have yet published so we are pretty limited in what we can say. I hope they will soon be posted (hopefully the protocols / studies / and reviews) so that those who are interested can try to understand what is going on. Until that happens or they appear in print, very little detailed analysis is possible.
John Forsyth kindly sent me the first failed study (he is a co-author on it), so I can speak to that one a bit
This study is a 2 by 2 (acceptance versus experiential control; high and low demand). The total N is 20 -- an N of 5 per cell. The protocol is eight minutes long. That is fine, but please do recall that the Hayes et al protocol was 90 minutes long so the language of a “failure to replicate” needs to be applied with considerable care here. The failures listed in the post here mention only my study as a failure to replicate, but I'm not sure why. In this first failure at least we are dealing in this study with a new protocol linked to the concept of acceptance, if there is a failure to replicate it is a failure to replicate not just Hayes et al 1999 but all of the ACT / RFT studies from labs around the world that have shown such effects (counting those published and those completed that would be anywhere between five and fifteen studies, depending on how liberal you are in drawing the line).
This first replication study is designed to look at demand and it’s interaction with acceptance and control. That is a good idea. Indeed we know from the Hayes et al 1999 study itself that demand will have a huge effect. It would be wise to disentangle it from acceptance and control.
We were mindful of this in the Hayes et al study, which has been published and can be downloaded here. We addressed the issue of demand by using high demand instructions in preassessment so we could weed out subject who would respond readily to demand. Subjects were actively encouraged to keep their hand in the water has long as possible. In the Hayes et al. 1999 study 58% of the subjects (44 of 76 subjects screened) were eliminated as a result since they could already do the task (300 seconds of ice water immersion) without any intervention if you pushed them hard in baseline. Only the remaining 32 subjects who still could not do the task were put into the study proper.
In the first replication study subjects were told (quoting from the article): “They were to let their hand hang loosely without making contact with the bottom and were instructed to “remove it when it is no longer comfortable.””
This creates several problems. Subjects are explicitly told by the experimenter to link hand removal to their discomfort. This seems unwise since it sets up an incongruity with the later acceptance intervention. I wonder what the subjects think when the experimenter is both saying “discomfort is the issue” in the preassessment and then that it is not the acceptance intervention. Further, in the “low demand” condition especially, why would acceptance lead to better performance given such instructions? Perhaps subjects would just accept their feelings of embarrassment and remove their hand at the very first sign of discomfort (who goes around freely experiencing pain for no purpose?)
I am not sure why this key feature of the Hayes et al study was changed if it was truly meant to be a replication. The result is that when demand instructions are later applied (subjects in the high demand condition are told “it is important that you do your best for me” while the experimenter sits close and looks at the subject) we can anticipate that a large percentage of the subjects will now hit the ceiling on the task.
That in itself is not so much a difficult as that no power is left to detect effects for the acceptance versus control comparison or its interaction with demand. Let me explain the problem. Apparently none of the subjects in the study described as a replication went to the maximum in preassessment. If we assume this would hold for as many subjects as were screened in Hayes et al and now compare this to the results obtained in the preassessment in the original study, we can calculate the effect size for high demand in this analogue task by comparing the rates of 0/76 with 44/76 dropping out. Applying Yates correction (due to the 0 value) the chi square yields a Cohen’s d of 3.84. That is enormous. If the same rate of response applied to the replication study there were only 8 subjects spread across four conditions who would have been there if the Hayes et al procedure had been followed. The power of the study to detect anything but demand is almost non-existent.
In fact, the actual demand effect in this replication study was less than that shown in the original study, suggesting that the deamdn in the so-called “high demand” condition was in fact a much lower demand than the level controlled for in the original Hayes et al study. Furthermore, note that the very first finding in the Hayes et al study was the primary finding in the first “failure to replicate” study. We did not make much of it -- demand effects are ubiquitous and are more things to control than to get excited about. But in this aspect, it was not a failure to replicate.
I have already explained why acceptance plus low demand might even result in poor tolerance. And although not significant, the replication study actually saw a trend toward that effect. But what about high demand?
Here you might expect that acceptance would aid in greater tolerance since you are asking subject to “do your best for me.” The request is still a bit odd since “your best” is apparently not specified and the original instructions asked subjects to remove their hands when discomfort set in. In the Hayes et al study subjects were asked to keep their hand in the water as long as possible.
Despite all of this, what was actually found was something other than what readers of the list might have guessed given the overall summary posted on this list and especially given the power considerations I have just described.
I will just quote from the manuscript and let it speak for itself:
“Under high demand, the post-intervention difference between the ACT over the control-based group was significantly greater than the difference between groups observed under low demand (Mdiff = +98.2; SE = 46.25; t[18] = 2.12, p < .05; η2 = .22; CI = .15 to 196.25). A similar trend also was observed at follow-up (Mdiff = +109.20; SE = 57.74; t[18] = 1.81, p < .07; η2 = .18; CI = -13.19 to 231.59).”
Furthermore, the study found something else that readers of the list might not have guessed given the characterization posted to the list. Again I will simply quote:
“Using this approach, participants in the acceptance group showed greater reductions in pain intensity at post-intervention (Mdiff = +11.0) and follow-up (Mdiff = 0.0) relative to baseline when compared to those in the control-based intervention group (Mdiff = -1.5 and Mdiff = +9.0, respectively) as supported by the significant Intervention x Time interaction, F(1, 16) = 6.73, p < .02, η2 = .30 (moderate effect). A similar Intervention x Time interaction was observed for ratings of pain sensation, F(1, 16) = 6.84, p < .019, η2 = .30 (moderate effect). This interaction was due, in large part, to the acceptance group showing greater reductions in pain sensation from post-intervention (Mdiff = -2.5) to follow-up (Mdiff = -18.0) relative to the control-based intervention group that showed a relatively stable pattern across both time periods (Mdiff = +4.5 and Mdiff = +7.0, respectively).”
Given all of these facts, am having a very hard time fitting the actual study and its findings with what has been posted, at least in its tone and import. The description is not literally incorrect. It says "The results showed that the most significant factor influencing performance on the cold pressor task was the effect of placing social pressure on participants, with no significant overall effect for Acceptance or Control interventions." But given all we know, it seems odd to mention only the main effect when a full accounting shows that this first “failed” study in fact replicated the Hayes et al demand effect AND the effect of acceptance. when you have interactions you simply cannot interprete only the main effects.
I don’t have the other studies, so we will have to wait to dig in more. But the way they are described seems similar. If the author used a low demand, emotionally-linked preassessment procedure, then you will get a big demand effect later. We already showed that inthe first study. You could get one so huge it will tend to wash out most other effects unless you have a very well crafted and well-powered study. Most of these studies are at least not well powered if you know that you are looking for additional effects beyond a variable that can produce a Cohen's d above 3. Why does this matter? Because you should not be interpreting null findings with studies that are inadequately powered. The null is entirely predictable.
One of these studies worries me based on what is in posts on the ACT listserve: the clinical study with 4 people (failure number 3). In a post to the ACT listserve about these failed attempts, this is what was said by the author:
“I have even examined acceptance over the long run in the lab and conducted one study in which real clients were given full ACT treatments weekly for up to five months by an experienced therapist with a good knowledge of ACT as a postgraduate student. We replicated the Hayes et al (1999) protocol as closely as possible (we used the detailed intervention outlined in Korn's Ph.d -not the brief version reported in the Psychological Record) with a small number of subjects but failed to find effects for ACT on cold pressor task performance taken weekly and with problem behaviours charted daily for the full period of the study.”
Another post, similar to the one to this website, said:
“This study was a part replication and extension of the Hayes, Bissett, Korn, Zettle, Rosenfarb, Cooper and Grunt (1999) study. Four sub-clinical volunteers (two smokers, one drinker and a tantrum thrower) were exposed to a cold pressor task before, immediately after, and several weeks subsequent to a 90 minute acceptance-based therapeutic intervention. Baseline rates of idiosyncratic problem behaviours were also recorded prior to, and for several weeks subsequent to, the initial intervention. The acceptance-based intervention was then administered weekly for up to 25 weeks by a qualified cognitive-behavior therapist to asses its impact on pain tolerance and target problem behaviour rates in the longer term. The acceptance-based intervention showed weak effects on pain tolerance during all phases of the study and no discernible effects on problem behaviors were observed (in fact they got worse!).”
Here is why I’m worried.
The Hayes et al protocol was not meant for use with real clients. The very title of the Hayes et al study describes it as a study not on intervention but on intervention rationales. Zamir Korn’s protocol is also not meant for clinical use: it is an analog. Clinical ACT manuals and books were available when this study was conducted. I don't understand why real clients with real problems would be exposed to such extremely restricted protocols and not to a clinical protocol.
I ordered Zamir’s dissertation and downloaded it (it costs $30 to do so for those who are interested). I can find nothing that says that this study, run 15 years after the Hayes et al study was actually conducted, used the Hayes et al protocol. Zamir was a group leader on the 1981 study that published in 1999, but I has since emailed him and he says that the protocol was new, and furthermore that he included his own ideas that go beyond ACT (specifically he views himself as an Organicist, not a Contextualist). Here is what he said:
"I wrote my diss protocol from scratch based on what I recalled from the '81 study and ideas I developed clinically over the years using core ACT principles. As I reflect back on it now, my adaptation of metaphors and principles have evolved, and I'm not exactly sure what might go beyond ACT boundaries." So, clearly, the Korn protocol is not the Hayes et al. protocol.
There is another, bigger issue that makes me worry that Zamir’s protocol was used with real clients.
Zamir Korn’s dissertation was completed in 1997, apparently well before this clinical study was begun or completed. Without spending the $30 you can go look at the abstract in Dissertation Abstracts International (it is free). You will find that the acceptance protocol in Korn’s dissertation did not differ from the attention placebo (although it was better than the CBT comparison).
The ACT Manual was sold by Context Press continuously since 1993. It stopped the moment the Hayes, Strosahl, and Wilson book becmae available in 1999. So for 13 years ACT clinical manuals have been publicly available. Zamir send me a copy of the email in which we sent his protocol to Dr. Roche. It was dated January 17, 2000.
I just don't understand why a failed analog protocol was used with real clients for 6 months.
Putting that clinical worry aside, on the science side, the phrase “failure to replicate” does not seem to apply: It sounds as though it replicated the earlier results with Zamir's protocol.
To return to the science side: There are multiple clinics and lab around the world who have shown how to use ACT to help with such problems.
For example, it is mentioned for example that half of the subjects were smokers and that this was targeted for change but that the behaviors got worse. There are now three completed randomized trials on ACT for smoking and another underway, so in controlled studies involving several hundred subjects we know what is likely to occur. One study is published comparing ACT to the nicotine patch (Gifford et al, 2004 in Behavior Therapy); another much larger is now within weeks of submission with very similar results comparing ACT to Zyban (also Liz Gifford as first author); a third has been completed in Carmen’s lab (Mónica Hernández-López is the senior author) showing really super results comparing ACT to CBT. All three of these have 1 year follow ups with objective CO measures. All show ACT beating the other conditions … and with nice mediational results in at least the two from my lab. You can see the data for these three studies in the “state of the ACT data” power point slides on this website. Rick Brown at Brown University has another study coming with ACT for distress intolerant smokers who have never been able to quit for a day in their lives – the pilot was presented recently with very nice data.
ACT for smoking is now known in large, controlled studies with several hundred subjects to be helpful as compared not to placebos but to some of the best available pharmacological and psychosocial methods out there. It has replicated in multiple labs with very long and objectively monitored follow ups.
The clinical failure reported with four people using a protocol from an analog study needs to be seen in that light.
We can dig in to the other studies when and if the protocols, manuscripts, and (ideally) the reviews are posted. So far Study 1 an underpowered study that (remarkably given its low power) basically replicates previous findings. Study 3 looks like questionable choice of protocols that has been made moot by subsequent controlled research. Some of the other studies seem to have the problems of Study 1, especially in the failure to follow the high demand preassessment of Hayes et al., 1999 and the resulting effect this has on the ability to detect anything else except demand effects, but we will know for sure when the studies and protocols are posted.
I've heard rumors that the control protocols mention pain repeatedly but that the ACT ones do not for example. If that were true, we might guess that this could be responsible for the difficulty in replicating the results.
I wish I could say we have learned a lot from these failures, but I don't think we have so far.
The best way to learn from failures is to find the boundary conditions in which an effect occurs and does not occur. If you can do that in a single study it is especially helpful. Dr. Roche is in the same department as a team that is regularly finding the effect in entirely automated protocols. The best thing to do would be to get these failures going head to head with successful ones and tease out the specific features that produce the differences. That could be really important.
- S
Steven C. Hayes, University of Nevada
Acceptance/control pain studies from Almería (Carmen Luciano)
Acceptance/Control Pain studies
As most of us I am convinced that we need to do more work, perhaps more creative work, to enhance the connection of clinical ACT methods and RFT and many of us have been in the last years, and actually are, working in this track.
As promised last week, I am incorporating part of the work of Acceptance-control and pain. Among our research areas concerning RFT, one is the RFT-ACT projects, in collaboration with Dermot and Yvonne which have been focused in the last years on making experimental
analogues to isolate the verbal processes involved in clinical methods for defusion, acceptance and the work in values. Experiments have been run and presented in past ABAs, Linköping,in Philadelphia this year.
What follows a very brief summary of some studies concerning our research on evaluating ACCEPTANCE and CONTROL strategies.
Our first series of studies to analyze the relationship between RFT and ACT was concerned with evaluating the change in functions given to pain in acceptance and in control protocols.This series began early in 2002 and the preliminary data were presented in London. These data
have been all published in Gutiérrez, Luciano, Rodríguez and Fink (2004). It is available in Behavior Therapy (vol. 35, pp. 767-783.
Very briefly, our main concern was to advance the study by Hayes et al. (1999) by incorporating a series of controls in the protocols to make the ACT and the Cognitive-control protocols as similar as possible, except in the critical aspects. In the Gutiérrez et al,
study, one of our concerns was to give AN OVERALL motivational context which is not explicitly established in most studies on pain coping. This means giving an explicit valuable goal (i.e.,learning more
about how to cope with chronic pain to keep in doing what they need to do, even when they have to do a boring work and so) that was connected to participants’ performance in the pain procedure, by focusing on the usefulness of the participation in the experiment and how the
information here gained might benefit people suffering from pain.
This motivational context allowed studying the impact of ACT and Cognitive-control-based strategies in a conflict situation where cost/benefit functions of avoidance/approach behaviors to aversive stimulation were actualized: that is, subjects took part in a nonsense-syllables-matching task that involved successive exposures to increasingly painful shocks. At times, throughout the task, participants were asked to choose to continue with the task and be shocked or stop the task and avoid being shocked. Each choice had specific costs and benefits in addition to the overall value-oriented context of being in the task. Participants performed the task twice, both before and after receiving the assigned experimental ACT or CONT protocols(attached).
Very briefly: 40 subjects ran test 1 under a general value condition that was identical for the all subjects (see above). Then, 20 were to an ACT protocol and the rest to a Cognitive-control protocol which were equal in rhetoric components, duration, opportunities to practice the strategies, number of instructions, number of connections between participant`s pain-related thoughts in the first pain-task and the components of the protocols. Consequently, both protocols highlighted that it was important to keep on task, BUT the protocols provided different strategies to pursue the task (so, this is an example of a study done that equalizes both conditions in terms of pain-task overall-meaning while at the same time equalizing as many details as possible in both protocols so that only the strategy to cope with the pain task was different. If the time is taken to read carefully the paper, you will find the necessary differences in the procedures)
ACT participants showed significantly higher tolerance to pain and this effect was more relevant when pain was highly distressing while both, the distraction and acceptance strategies, appear to be equally effective when pain is mildly distressing at least in this analogue, where a valued context is involved. Accordingly, significant lower believability of experienced pain was obtained in test 2 in the ACT condition compared to the Cognitive-Control condition (being believability measured by the number of subjects who stopped the pain
task after an evaluation of “very much pain rating”) This study permitted us to discuss the process involved in changing the function of pain as a barrier, that is, changing from avoidance to accepting and more importantly, analysing what could be the contextual change of the pain experience involved in the protocols and in the overall value-oriented task which might provide such a change.
Second and third studies (Marisa Páez thesis –defended last May) followed this first study. These studies were conducted ending 2003 and were presented in ABA and Granada.
The second study (almost finished to send to review) aimed to control in test 1 for the overall-value oriented context designed to make continuing the task valuable which was involved in the Gutiérrez et al. study for all the participants in the first test, and secondly, to
systematically replicate the ACT and CONT protocols in Gutiérrez`s study. 20 subjects participated for a voluntary pain-task. After completing the statement of informed consent for participating in a pain investigation to study painful process, they were told to go to determining voltaje level and then to calibrating the apparatus to the pain-task. After Test 1, 10 subjects went to an ACT protocol and 10 to a CONT protocol. Experimental protocols involved two different valued contexts (related, respectively, to acceptance strategies or cognitive control strategies) plus a metaphor and an exercise to practice the respectively coping strategy in the pain-task similar to the used in Gutierrez et al.Specifically, the acceptance-based context established a coordinated relation between pain and important actions (i.e., several questions concerning how is that people whose life is very hard because of pain, persist and remain working even with very severe discomfort), while the cognitive control-based one establishes an opposed relation (questions concerning how is that people suffering of pain and even they have important things to do, they do not do).
Additionally, a metaphor and a similar exercise to that provided in the Gutiérrez study were implemented coherent to each condition.
Results: compared to the first test in Gutièrrez study, in the Paez`s study, the subjects chose to be shocked less times. In the second test, after implementing the acceptance-based protocol
or the control-based protocol, tolerance increased in both conditions, HOWEVER, both conditions differed significantly in the believability of pain (measured directly by continuing with the pain- task after rating a shock as “very much painful”). Results obtained confirmed Gutiérrez et al.’s (2004) findings and permit us to discuss the differences between previous studies and these two. Especially we were interested in knowing, on the one hand, the impact of the valued context given to the task, and on the other, we need a control conditions without protocols but exposure to the task without an specific valued context. These were the reasons for the third experiment (also in Paez thesis)
This new experiment aimed to know the mere exposure to the task, and to know the effect of two different valued-contexts for the pain task but without any defusion method. There were three conditions from the very beginning (10 subjects each): Condition EXP-control: exposure to the task “for calibrating and so. (as test 1 in second study).
Condition ACT: the impact of an overall-valued context that was also focused on personal examples in which the subject recognized himself as doing something even when having distress or sadness; and finally connecting all this to the pain-task (a coordinated relation between having distress, sadness..,and acting valued).Condition CONT: idem than (b) except than personal examples were asked in which the
subject recognized himself as not doing something because of distress or sadness or pain. (an opposed relation between having pqin and doing valued action.Consequently, each condition involved a different valued context in the first test without any metaphor or exercise to cope with pain.Among the results in test 1 were:
ACT: 7 out 10 Ss achieved the maximum number of shocks permitted (15 successive exposures to increasingly painful shocks).
COG: 1 out 10 did
EXP control: 2 out 10 did
Later, the subjects who did not achieve the maximum shocks, received the corresponding treatment to the initial condition:
ACT: 3 subjects received the ACT protocol (metaphor and exercise coherent to ACT strategy as in first Paez study)
COG: 9 received the COG protocol (metaphor and exercise coherent to control the pain thoughts as in first Paez study),
EXP control: 8 received no protocol but a new test again for continue calibrating the apparatus):
Results obtained in the second test confirmed findings from the two previous studies and EXP. control no change.
More studies have been done in the Irish labs including more controls (see the McMullan, Barnes-Holmes… attached the other day by Steve). Other more have been done and are now being done with stimulation other than electric shocks and will be presented in London.
Most of the unpublished material is going soon to revision and the protocols will be translated and sent to the contextual.web . In the meantime, we are attaching the protocols for the study published (Gutiérrez et al.) and the rest are available in Spanish language.
----------
It will be a pleasure to discuss all these series of studies and we will have a good opportunity soon in London because many of us will be going. We will try to present a symposium with all this pain series and will be relevant to have the chances to discuss and to see the
differences with other protocols that are not generating what we have repeatedly found with ours.
Replication is the only way and for that we need to know whether we are doing the same or whether we are talking about procedures that share something but do not have the common essential components.
We hope that this will help in comparing ACT and Control strategies
Carmen
Dr.Carmen Luciano
Catedrática de Psicología
Departamento Personalidad, Evaluación y Tratamiento Psicológicos
Universidad de Almería
04120 Almería, España
tf.34-950-015260
fax. 34-950-015471
Gutierrez, O., Luciano, M.C., Rodríguez. M., & Fink. B. (2004). Comparison between an Acceptance-based and a Cognitive-Control-Based Protocol for coping with pain. Behavior Therapy, 35, 767-783.
PROTOCOLS FROM GUTIÉRREZ ET AL. (2004) STUDY
A) INSTRUCTIONS GIVEN TO ALL SUBJECTS BEFORE GOING TO TEST 1 IN THE PAIN TASK.
“ This one is an study about pain and the strategies for coping pain. First, we would like to thank you for your participation because it is indispensable for our research. This one intends to help people who suffer with highly disabled pain, so your collaboration in this preliminary study is very important to do further studies where patients who suffer natural pain will participate. For this reason, the experimental procedures that we are using will be unpleasant because the only way to find out what strategies are useful for people suffering is to do a research where participants have an unpleasant time. Are you willing?... The study involves receiving an electric shock at different times. Every security measures have been adopted. The study lasts approximately two hours and it is important if you are willing to participate. Are you agreed?...”
(If participant agreed, he should sign a informed consent form for research -available by requesting from the authors- where experimenter declared her commitment to adhere to ethical guidelines in their dealings with participants and the collection and handling of data and the participant was informed about: a) the general nature of the study; b) his/her right to withdraw from the study at any time; c) no risks associated with participation; d) confidentiality to treat all data. Then, participants were given specific written instructions about the pain task that included information about the matching-to-sample-trials, b) the choices that participants could make when the red asterisk was presented, c) the duration and frequency of electric shocks, d) tokens and rewards that the participants would receive for continued participation. When participants read the instructions, the experimenter said…)
“ You must remember that we want to find out what people may do when they suffer pain so it is very important you maintain yourself in the task all time that you can. Perhaps, the task is monotonous, boring and not a bit creative but its purpose is just to reproduce the conditions of people who suffer non-continuous pain and they must work or do other things that they do not like. We seek to study the coping strategies that may be helpful for these people who live under those conditions. Do you know some people who suffer with pain and live under these conditions? ... Have you ever gone through a similar situation?... Besides, you should remember that the more tokens you accumulate, better rewards you can get. Please, come with me to the rewards room. (In the rewards room) As you can see, there are three categories of rewards depending of its “price” in tokens (A, B, C). Now, you cannot know how many tokens you need to choose one reward. The only thing that for the moment I can say you is that a reward in Category A need more tokens than a reward in Category B and than Category C. When you finish all the procedure, come to this room and you can exchange the accumulated tokens”
Participants answered some questions regarding the understanding of the pain task instructions given so far. If everything had been understood properly, the participant started the pain task (TEST 1)
INSTRUCTIONS GIVEN TO ALL SUBJECTS BEFORE GOING TO TEST 2 IN THE PAIN TASK
“ ... Now, I am teaching you some strategies for coping with pain-related thoughts and feelings so you can use them when you perform the second Pain Task in some minutes, ok? Our minds sometimes tell us things that are helpful. For example, if I want to go to the town hall, my mind tell me that I must turn to the right, then to the left... and I finally reach my purpose: to arrive to the town hall. But, our minds sometimes tell us things that move ourselves away the important things for us. For example, imagine a person who is about to do an important conference but s/he starts to think: “I am not able, I do not remember anything, I am very anxious, I will mistake, etc. and s/he finally decided not to do the conference. See it with more examples. Perhaps, you have had different pain-related thoughts and feelings while you performed the first Pain Task (It is too painful, this task has not any sense, it is boring, I am afraid for the shocks, etc.) and you have decided not to receive more shocks and not to continue with the task. The thing is that something different can be done with our thoughts and feelings… Just now, we are going to practice in that direction …”
SPECIFIC PROTOCOL FOR THE ACT-BASED CONDITION BEFORE GOING TO TEST 2
- “I can’t get up” exercise (based on cards exercise in Hayes, Strosahl & Wilson, 1999, p. 162)
Tell me a thought that showed up when you decided to quit the Pain-Task and thus to avoid receiving any more shocks... (the thought is written on a piece of paper in capital letters, and the paper is then left aside on the table; this is done twice more, with another two thoughts; after that, the participant is given a folded piece of paper with a sentence written on it, though they can’t read it). Imagine that this (folded paper) were a thought you had. Now, with that thought in your hand, please get up and walk around the room... Now you can sit down. Please read whatever is written on that piece of paper (on the folded paper the participant could read ‘I can’t get up’). Then, is it possible to continue with the Pain-Task though you think that...? (they take the first paper written by the participant and read the thought written on it). Is it possible for you to think (they take the second paper-thought of the participant and read its content) and to keep yourself performing the Pain Task? ...Is it possible to have the thought of... (they take the third paper-thought and read it) and continue performing the task while having that thought...?
The thing is you can keep performing the Pain-Task and thus get more tokens just by noticing the thoughts that show up and all the distress they carry with them... Whatever thoughts show up, no matter how much distress you feel, you can keep performing the Pain-Task... you can act in a certain way even though you have thoughts you don’t like...
- The “dirty swamp” metaphor (Hayes, Strosahl, et al., 1999, pp. 247-248)
‘Imagine that the only way to reach something that’s important for you is to go across a swamp full of dirt, rubbish, leftovers, that smells so badly, that really stinks... to go across the swamp and arrive at the other shore. What kind of thoughts do you think that are going to appear in such a situation? ...It’s likely that thoughts like “I can’t stand this”, “This s unbearable”, “I can’t do anything so unpleasant and disgusting”, “It’s not worth the effort, it’s nonsense”...The best way you could possibly cross the swamp would be to notice all those thoughts and the distress they carry with them and let them be, to notice them and make room for them while you keep crossing the swamp... It’s about being open to all the thoughts that may show up and the distress associated to them, about carrying them with you while you keep doing what’s important in that moment: crossing the swamp and reaching the other shore... Notice all the thoughts that show up while you perform the Pain-Task and carry them with you, because you can have whatever thoughts and act differently to what you think or feel...’
SPECIFIC PROTOCOL FOR THE COGNITIVE-CONTROL-BASED CONDITION BEFORE GOING TO TEST 2.
- Distraction exercise.
It has been proved that trying to distract oneself and to think of other things is an effective way of dealing with our negative feelings and thoughts (e.g. thoughts related to pain, distress...). Right now, what you are going to do is to learn a strategy in order to eliminate the thoughts related to pain and distress that appear while you are performing the Pain-Task, so that you can keep performing it for longer, and thus obtain more tokens. For instance, whenever these thoughts appear, you can try to focus in a pleasant scene that you lived in your past, and so continue with the Pain-Task.
Now I want you to think of that scene of your life. I will help you to imagine it and to place yourself there so that later, while performing the Pain-Task, you can focus on it easily. Please, close your eyes and raise your finger when you’ve got that situation. Now, notice where you were in that situation... Notice what you were doing... Focus on how well you were feeling then, notice the clothes you were wearing... (This exercise took place for 3 minutes, from the moment when the subject raised their finger; after those 3 minutes...)
Now you can open your eyes. What you have to do when you perform the Pain-Task is to distract yourself with either this scene or with another pleasant scene, in order to avoid thinking of pain, and thus to continue in the task for longer and to obtain more tokens... Let’s practice this exercise... Tell me what you were thinking of when you decided to stop with the Pain-Task and thus to avoid receiving any more shocks. Please, evoke that thought and try to distract yourself with the scene we’ve been practicing or with another pleasant scene... Has it been difficult? ...Tell me another thought you had while doing the task... Try to evoke it and then try again to distract yourself with a pleasant scene... Finally, tell me another thought of that sort and try to distract yourself with the pleasant scene... Did you have any problems to do that? ...The thing is that you can continue with the Pain-Task if you try to eliminate your negative thoughts and the distress they carry with them, and you can distract yourself with different thoughts and pleasant scenes.
- The “dirty swamp” metaphor.
‘Imagine that the only way to reach something that’s important for you is to go across a swamp full of dirt, rubbish, leftovers, that smells so badly, that really stinks... to go across the swamp and arrive at the other shore. What kind of thoughts do you think that are going to appear in such a situation? ...It’s likely that thoughts like “I can’t stand this”, “This s unbearable”, “I can’t do anything so unpleasant and disgusting”, “It’s not worth the effort, it’s nonsense”...The best way you could possibly cross the swamp would be to try to think of more pleasant things, to imagine, for instance, that you are in a lovely landscape, and meanwhile to keep crossing the swamp... It’s about removing distress and unpleasant thoughts, and thinking of more positive things, so that you can do what you have to: cross the swamp and reach the other side... Whilst you’re performing the Pain-Task, try to remove pain-related thoughts that show up and think of more pleasant things, positive things, because those thoughts will help you to keep performing the task...
As shown, both experimental protocols were equal in: (a) rhetoric components (both involved initial examples, one metaphor and one exercise), (b) duration (near 20 minutes), (c) number of instructions concerning the acceptance vs. control-based strategies, (d) number of opportunities to practice the strategies, (e) number of connections between participant’s pain-related thoughts in the first Pain-Task and the components of the protocols and (f) the number of instructions given to encourage continuation in the second pain task for as long as possible.
MESSAGES DELIVERED DURING PERFORMANCE (THROUGH THE HEADPHONES) IN THE SECOND PAIN-TASK (POST-TEST).
* Messages delivered to the participants in the ACT condition.
- You can keep performing the task regardless of whatever thoughts you have while doing it.
- Whatever thoughts show up, notice them and let them stay.
- Remember that you can make room for your thoughts and act differently to what they tell you.
- Notice all the thoughts that show up and carry them with you.
- Remember that having certain thoughts doesn’t imply acting in accordance with them.
* Messages delivered to the participants in the COGNITIVE-CONTROL condition.
- You can keep performing the task by distracting yourself and thinking of positive things.
- Try to focus in positive thoughts or images.
- Remember that if you think of pleasant and positive things you’ll be able to act in the direction you want.
- Try to distract yourself with pleasant thoughts or images.
- Focus in a pleasant scene of your life and you’ll be able to keep performing the task.
Time to Dig In
Bryan
So far as I know these are the only failures to replicate in the ACT / RFT literature, so they could be important. I was only aware of one or two of these attempts but I've heard some buzz about them.
ACT failures, and failures to replicate studies raise slightly different issues so I would like to write a longish reply.
For novices reading this (Bryan would know all of this)
There are at least two publications that have replicated the study Bryan mentions [[Hayes, S.C., Bissett, R., Korn, Z., Zettle, R. D., Rosenfarb, I., Cooper, L., & Grundt, A. (1999). The impact of acceptance versus control rationales on pain tolerance. The Psychological Record, 49, 33-47|Hayes et al., 1999] fairly directly.
These are:
Gutiérrez, O., Luciano, C., Rodríguez, M., & Fink, B. C. (2004). Comparison between an acceptance-based and a cognitive-control-based protocol for coping with pain. Behavior Therapy, 35, 767-784.
Takahashi, M., Muto, T., Tada, M., & Sugiyama, M. (2002). Acceptance rationale and increasing pain tolerance: Acceptance-based and FEAR-based practice. Japanese Journal of Behavior Therapy, 28, 35-46.
Dermot Barnes-Holmes (in Bryan's own department) has also reported replicated that study using an extremely well controlled automated procedure, but I have not yet seen the manuscript, I've only seen it presented. This approach is really important because anyone can get the Visual Basic program (Dermot sent me a CD and I think you can download it from his website at the National University of Ireland, Maynooth). It even has video clips of a "therapist" (Yvonne Barnes-Holmes) doing the intervention so there is no worry that the interventions differ person to person, etc etc. I have heard rumors that others have used the computer program and that it replicates, but I have not seen any of these yet. (Dermot, might you weigh in?)
It is worth noting that the published replications come from Spain and Japan, so they come from different labs in different countries. So far I have not met the Japanese researchers. Professor Muto is writing an ACT book in Japanese, which is nearing completion.
There are also now clinical studies showing the same effect, which seems important since these basic studies undergird that work, and in some ways well-controlled clinical trials are the even more important to the universe. The two that are out are:
Dahl, J., Wilson, K. G., & Nilsson, A. (2004). Acceptance and Commitment Therapy and the treatment of persons at risk for long-term disability resulting from stress and pain symptoms: A preliminary randomized trial. Behavior Therapy, 35, 785-802.
McCracken, L. M, Vowles, K. E., & Eccleston, C. (2005). Acceptance-based treatment for persons with complex, long-standing chronic pain: A preliminary analysis of treatment outcome in comparison to a waiting phase. Behaviour Research and Therapy, 43, 1335-1346.
Those come from Sweden and the United Kingdom, so again this is not inside a single lab.
All of these studies are readily available
I am aware of similar studies coming out of other labs such as Rikard Wicksell's in Sweden.
And of course there is the list of replications and extensions with tolerance of others kinds of distress, such as Levitt, J. T., Brown, T. A., Orsillo, S. M., & Barlow, D. H. (2004). The effects of acceptance versus suppression of emotion on subjective and psychophysiological response to carbon dioxide challenge in patients with panic disorder. Behavior Therapy, 35, 747-766 or Eifert, G. H. & Heffner, M. (2003). The effects of acceptance versus control contexts on avoidance of panic-related symptoms. Journal of Behavior Therapy and Experimental Psychiatry, 34, 293-312, among several others.
Thus, it appears that we have multiple published studies from around the world and the list you have posted seemingly in conflict. So far I gather none have been accepted for publication. That part makes it very hard for others to directly compare them.
What to do?
IMHO it helps to think through the purpose of replication. It is not to see whether if you do the same thing you get the same effect. If you do the same thing, at least probablistically you have to get the same effect in a determined world. The purpose of replication is to see if doing what is stated in the studies are enough to get you to do the same thing.
But the details matter hugely in making that determination.
Perhaps it is time to dig in.
I don't know these studies in your lab, so I'm emphatically NOT saying this applies to them (you would have to tell us if reviewers raised such issues, etc), but just in the abstract we would agree if the statistics aren't right, or the manual has a problem (etc etc) then that is a very different matter than if the methods cannot be repeated and yeild results. That is one reason I was so happy to hear that Dermot had replicated it with a computerized preparation ... you can repeat it perfectly.
It would be awesome if we could take your protocols and put them into that same program, since then we might be able to systematically produce the effect AND (ideally) eliminate the effect. That could really tell us something.
Would it be possible to post the submitted manuscripts, manuals, and reviews on this site? If something systematic can be detected, this might help us all.
It would be a shame just to let all your effort be lost. Something explains what is happening in these several others labs as compared to what is happening in your studies. My study aside, some of these other folks are pretty good researchers, so it has to be something different. When it is understood it might even be important.
If it is just left like this, your post could be of value. At the very least it says "Be careful. Details matter." But it seems possible that it could mean more. Unfortunately, I just don't think any of us can armchair it and know, not without either more work or all of the materials in front of us.
Thanks for putting this forward.
Steven C. Hayes, University of Nevada