There are a wide range of ways facilitators decide if their work is effective. However, it is not easy for facilitators to describe the in-the-moment criteria by which they navigate a session. Even so, we believe that even if a facilitator cannot specify how they decide what to do, they have a duty to consider the question: How do you know when what you are doing is (or is not) working?
There are two strands to our interest in this topic. One is ethical , the other is practical.
Let’s start with the ethical aspect. Earlier this year James had a dialogue with Steve Andreas about Andy Austin’s Metaphors of Movement process. (See the full discussion at: Comments on ‘It Certainly Ain’t Clean’) [Link available soon]
Steve noted that:
Andy often responds in metaphor, introducing alternatives, something that clean language would consider ‘dirty,’ offering content not provided by the client.
I agree this is a fundamental difference between the two processes. And anyone using Andy’s (or any other content-introducing) method needs to answer the following questions:
- How do you know what kind of content is and is not appropriate to introduce?
- How do you know when it is inappropriate to offer content?
I strongly believe that trainers of these approaches need to make explicit the calibration required by the facilitator to notice in real time, what is and what isn’t working for a client.
I recently saw Frank Farrelly work and he not only failed to be explicit about how he calibrated whether what he introduced was valuable, he deliberately sidestepped the question when asked. Given that I witnessed a client of Frank’s suffer a severe negative reaction both during the session and, they reported, for several weeks after, I consider Frank’s failure to answer this question unethical.
I am not suggesting what Andy is doing is as impositional as Provocative Therapy, but I am saying that when any method relies on introducing content, the facilitator should have ways of knowing when that is inappropriate for a particular client, since any method that is influential enough to get beneficial results must also have the capacity to produce harmful results.
More recently Steve Andreas was referring to someone using Nick Kemp’s Provocative Coaching when he said “And like all such skills, it’s essential to be sensitive to the client’s response to using it; if it’s not working, try something else.”
I agree and it begs the question how would the provocative therapist/coach know that? I asked Frank Farrelly this and he just ran a provocative pattern on the question – leaving me none the wiser. And therein lies the danger. ‘Resistance’ to the therapist or coach’s approach can always be reframed as ‘the process working’ or as a signal to ‘play’ harder. That puts the client in a bind. When the client has volunteered to be a demo subject in front of a room full of people it’s really hard for them stop a session by “leaving the field” as Bateson put it. That puts them in a double bind. So how do experienced provocative therapists or coaches know when their approach is not working?
[Both Steve Andreas and Nick Kemp replied, unfotunately the ongoing discussion is no longer available at: realpeoplepress.com/blog/provocative-parenting]
For an example of what can go disastrously wrong when a ‘facilitator’ ignores warning signs, see Christopher Goodwin’s article, ‘At the temple of James Arthur Ray’, coincidently published in the Guardian Saturday magazine on the same day as the Developing Group: guardian.co.uk/world/2011/jul/08/james-arthur-ray-sweat-lodge-arizona.
In addition to the ethical question there’s a practical angle. Each school of therapy values certain kinds of evidence that their process is working. Many years ago we gave a demonstration to a conference of Hypo-Analysts. After the demo a leading proponent of the method sidled up to us and asked: How did you get her to cry? Having the client cry wasn’t our aim, and in and of itself it meant little to us. We thought the demonstration had gone well and the client had got a lot out of it – but not because she cried. However we presumed crying was one of that Hypno-Analyst’s top criteria.
A different example took place recently. James was a guest presenter at an international certification training for coaches. He gave a demonstration of Symbolic Modelling and the audience were stunned that he stopped after 10 minutes. The interaction with the group went like this:
|Observer:||Why did you stop?|
|James:||Because we were done.|
|Observer:||How did you know?|
|James:||Let’s ask the client [turns to client]|
|Client:||I was finished. I didn’t need to go on. I’d got more than I expected.|
We guess that the group and James were using different methods to calibrate whether the client had got what they wanted from the session. Here is an unedited video of the demonstration:
NOTE: An double-annotated transcript of the session can be downloaded and is analysed below.
This response of groups to a demonstration of Symbolic Modelling is not an isolated incident. We have demonstrated to all kinds of groups in a variety of countries and we have often noticed a pattern. During the demo we often pause to ask the group what they are attending to. Rarely does anyone suggest something that is high on our list. And if they do, we interpret it in different ways. Why is this? We believe many of the differences stem from how each facilitator calibrates whether what they are doing is working, or not.
This paper is not about the difference between Symbolic Modelling and other approaches. We are just as interested in the way different facilitators calibrate in Symbolic Modelling. Below is a transcript of the start of a session using Clean Language. As you read it you can wonder what on earth the facilitator was calibrating for. The client started by saying: “I want to integrate diversity without dispersing myself and diversify myself without losing focus. I want to find my balance.” The session continued:
|F:||And what would you like to have happen now?|
|C:||I believe I’ve just answered that.|
|F:||And what kind of believe is that?|
|C:||You asked me the question and I had answered it.|
|F:||What happened just before believe?|
|C:||I don’t know because you asked me a question to which I had answered. Your question doesn’t belong to me. [Long silence]|
|F:||And when it doesn’t belong to you, that’s like what?|
|C:||It just doesn’t belong to me anymore.|
|F:||And when it doesn’t belong to you anymore, where is it?|
|C:||The act just doesn’t belong to me anymore. It’s over there on the mat.|
|F:||And what’s the relationship between over there on the mat and your outcome?|
|C:||I don’t know. It’s after the act of being asked the question perhaps.|
|F:||And when balance, where is perhaps?|
|C:||If you say perhaps, it’s hypothetical and I can’t answer it. I know when I do feel balance it’s here [points to chest].|
|F:||And is there anything else about that perhaps?|
|C:||Balance is here, the act is over there [points to the mat].|
|F:||And is there anything else?|
|C:||I don’t think so.|
|F:||And I don’t think so, and balance is here, and what is the relationship between I don’t know and balance?|
|C:||I don’t know. A right reply, I guess.|
|F:||And when a right reply, what kind of right reply?|
|C:||Everything is adjusted.|
|F:||And when a reply is adjusted, that’s like what?|
|C:||I just don’t know.|
|F:||And that’s ‘I don’t know’ like what?|
|C:||An honest reply.|
|F:||And what’s the relationship between an honest reply and balance?|
|C:||Something posed in relation to a demand for an answer.|
|F:||And what happens next after an honest reply?|
|C:||The act is passed. None of this belongs to me anymore.|
|F:||And when it doesn’t belong to you, what kind of doesn’t belong to you is that?|
|C:||[Sighing and rolling their eyes] I don’t know because it doesn’t belong to me.|
Penny intervened at this point. Later the facilitator asked Penny: “What do you do with such a resistant client? How can you push through their resistance and let them know their role, and that the facilitator is in charge?”
The facilitator calibrated the client as “resistant”. Penny on the other hand calibrated the client as “exasperated” which is why she intervened.
The point of the transcript is to show how using Clean Language does not immunise the facilitator from missing signs that what they are doing is not working for the client. It also demonstrates how a facilitator can go on and on despite obvious not-working signals. We added this transcript to ensure that the debate around this subject is not about any particular modality or technique. All are subject to being misused if the facilitator is not calibrating the client’s reactions, from the client’s perspective.
Of course it is fine for us to pose such questions, but can we answer them for ourselves?
Definitions of calibrate
Over many years we have observed thousands of coaches, therapists and other kinds of facilitators working, and we are convinced that most facilitators pay attention to different things than we do.
Our dictionary gives three definitions for ‘calibrate’:
- To mark (a gauge or instrument) with a standard scale of readings.
- To correlate the readings of an instrument with those of a standard in order to check the instrument’s accuracy.
- To adjust (experimental results) to take external factors into account or to allow comparison with other data.
A classic example of definition #2 is someone from the Office of Standards checking the weighing scales in a butcher’s shop. They bring along standard weights to see if the scales are working accurately. i.e. they compare the behaviour of the scales against a criterion. Note that every now and then the standard weights themselves need to be calibrated against another standard. This in turn will need to be calibrated … and so on. At some point someone has to say: This is accurate enough for our purposes.
Apparently the origin of the word calibrate comes from the Arabic for ‘mold,’ which itself may have been based on the Greek meaning a ‘shoemaker’s last’ – the device shaped like a human foot used to fashion a shoe.
In their Encyclopedia of NLP, Robert Dilts and Judith DeLozier say “Calibration is the name given, in NLP, to the process of learning how to read another person’s responses in an ongoing interaction”. This involves correlating external behavioural cues to specific internal cognitive and emotional states. Pictures of Robert are used as an example of calibrating whether he is ‘satisfied’ or ‘dissatisfied’. Used in this way, calibrating an individual’s state is more sophisticated than those popular books that tell you how to read body language based on so-called ‘universal telltale signs’.
Our use of ‘calibrating’ is based on a metaphorical extension of the second dictionary definition above. We use the term in a similar way to its use in NLP although we are less interested in matching behaviour to specific internal states, and more interested in calibrating broad categories of states so we know what to do next. For example our REPROCess model involves calibrating when the client is experiencing a Resource, Explanation, Problem, Remedy, desired Outcome or Change, rather than any specific state within those categories. Calibrating individual states can be useful but only if you have calibrated the broad category accurately first. Of course every facilitator says they can recognise the difference between, say, Problems, Resources and desired Outcomes, and they can, but in real time, during hurly-burly of a session, plenty of first-hand evidence suggests the majority don’t.
We are particularly interested in the calibration that takes place from the very beginning of a session, before any changes have happened. This involves calibrating broad classes of states such as ‘preparing’ or ‘getting ready’ or ‘on the way to’. In the words of Ernest Rossi the client is laying the ground for “a period of creative inner work [and] a living experiential reality [of] re-synthesizing [their] inner world”. (p. 24, Symptom Path to Enlightenment). But how do we know the ground work will bear fruit? We can’t know for sure, but we can play the percentages.
Calibrating whether the behaviour of a client indicates that they are benefiting from the process or not, is a form of ongoing feedback loop to the facilitator [Link available soon]. While it seems like the facilitator is calibrating the client, they are also calibrating their own effectiveness.
Below are a number of aspects which we think have a bearing on calibrating what’s working. They are are more about raising questions than about giving definitive answers. For example:
Faster isn’t necessarily better – every natural system has a ‘too fast’ setting when the speed of change becomes counter-productive. (The same is true of ‘too slow’.)
We can consider these questions from a number of time-frames (from longest to shortest):
At the time the client may not be in a position to assess whether the therapy/coaching was effective and efficient. So how long after do you wait to enquire, and how many times do you sample? We read of a client who changed his assessment three times in the 10 years following therapy. It went from “fantastic”, to “I wish I never gone”, to “I’m glad I did it and there would have been a better way to do it”
The calibration that seems to have less said about it than any of the others is the last one. Before a change has happened, how do we know that what we are doing is likely to be valuable for the client?
Also, at what scale do we calibrate?
|Big chunk||General models, e.g. ‘self-actualised’, ‘authentic’, ‘aligned’, etc.|
|Medium chunk||Common experiential categories, e.g. REPROCess|
|Small chunk||Specific behaviour|
A participant undertaking an NLP Master Practitioner modelling project wanted to know how international-level orienteers managed to stay on course so well. He had thought they created a sophisticated 3D mental map of the landscape. Instead he discovered something much simpler, they checked their map much more frequently than less-accomplished orienteers. That meant when they got off course, which was rare, they could easily retrace their steps to the point of departure and continue in a better direction.
Despite the fact that navigating a client’s mental landscape is more complicated than orientating a physical landscape – it can change without warning and you don’t have a map – we think there is a lesson to be learned from expert orienteers. How often do we consider whether the client is on course? How frequently do we check whether what we are doing is working?
Eugene Gendlin was the first person we know to shine light on this topic. In the 1960s Gendlin engaged in research at the University of Chicago to figure out why some therapy sessions more successful than others. He spent a long time trying to discern consistent patterns in therapists’ behaviour but he kept drawing a blank.
Gendlin made what we consider to be a brilliant methodological shift. He started to study what clients did. He discovered that clients who got the most from therapy experienced a “felt sense” and, if given the space and time found descriptive words that resonated with the felt sense. Usually there followed a “felt shift”, and the person would begin to be able to move beyond their stuck-ness. Gendlin went on develop, Focusing, a simple methodology that facilitates people to engage in these kinds of inner acts.
In outcome-orientated approaches [Link available soon] the client’s desired outcome will form part of the calibration, but in spite of what evidence-based therapies say, it can’t be the whole deal. Why not? Because:
What do we do when the client can’t specify a desired outcome?
How do we know it’s working when the client’s outcome can’t be achieved during the session (which is most of the time)?
A well-specified TOTE (Test-Operate-Test-Exit) will tell us if we have arrived at the destination, it doesn’t tell us if we’re heading in a useful direction. Edward de Bono has said “It may be necessary to go south for a while in order to journey north”, but how long do you travel south before you conclude you’re going in the wrong direction?
The specificity of the client’s evidence for achieving their desired outcome is an important factor. For example, with a straightforward phobia success is usually very clear. When the outcome is something like ‘fulfilling my life’s purpose’, it is not obvious how anyone would know until the client is on their deathbed.
Often simple remedies are easier to calibrate than desired outcomes. It’s not difficult to know if a burst pipe has been fixed. It takes much more skill to know if a water-system is working efficiently and is well-maintained – especially if that is being measured over the life-cycle of the system (c.f. Peter Senge, et. al. A Necessary Revolution).
Top-down or bottom-up approaches [Link available soon] will use qualitatively different ways of calibrating; think of sailing when you can see the destination, compared to sailing blind. To avoid getting locked into top-down criteria Nassim Nicholas Taleb recommends “stochastic tinkering”.
There is a sweet and oft told story:
One day a man was walking along the beach when he noticed a boy picking something up and gently throwing it into the ocean. Approaching the boy, he asked, “What are you doing?” The youth replied, “Throwing starfish back into the ocean. The surf is up and the tide is going out. If I don’t throw them back, they’ll die.
”Son”, the man said, “don’t you realize there are miles and miles of beach and hundreds of starfish? You can’t make a difference.”
After listening politely, the boy bent down, picked up another starfish, and threw it back into the surf. Then, smiling at the man, he said “I made a difference for that one!”
There is no doubt the boy made a difference to the starfish he threw back in the ocean, but was it a valuable difference? And valuable for whom? What natural process was he interfering in? What were the consequences for other species and the rest of the environment? How often has humankind, on a global or individual level, thought it was helping only to realise later it had made matters worse?
In addition, it seems to us that calibrating what’s working is closely related to three topics we have previously covered at the Developing Group: Attending-to-Salience; Vectoring; and Scaling. [Links available soon]
If you are of a mind to undertake a self-exploration, these questions should help:
Of course you can ask these question of yourself too much. The resulting self-doubt can render even the best facilitator ineffective.
Given that there is so little research available on this topic, we realise we have made a large number of assumptions. Below is a list of the ones we are aware of.
Everyone is calibrating against some internal criteria.
Calibrating involves two aspects:
And remember, a single black swan can turn all previous evidence on its head.
Calibrating needs to be dynamic; it has to change depending on:
Calibration is a meta-assessment. Calibrating whether it’s working or not is a level above vectoring. It asks:
And based on that assessment:
It is insufficient to only calibrate the client:
– Being in a good state (They may feel good, temporarily, but that may not be evidence that anything has changed for the better).
– Catharting (“Affect does not equal effect” – as they say at the Findhorn spiritual community)
– Having an insight (“Information does not equal transformation” – Findhorn again)
– Being congruent (Dilemmas/conflicts are sometimes as good as it gets).
– Experiencing a change (Does the change make a valued difference? If not, so what? And who decides what is ‘valued’?)
– Getting what they want (If an ‘infertile’ couple does not have the child they want, was therapy a failure?)
– Praising the work (Eric Berne knew the dangers of being charmed by a client’s praise. He called the TA Game: ‘Gee Professor, you’re wonderful’).
– By their referral of others to you (Great but there can be a huge time lag, and how do you know in the meantime?)
It is also insufficient to only calibrate against:
– Your feelings (You might feel it’s going well or I’m lost/confused – but what’s happening for the client? That’s what counts).
– The ease/flow of the session (You might think if it’s easy it’s going well – but sometimes the opposite is true).
– Having seen something similar before (This client may be the one exception).
No matter how accurate your calibration signals, there will always be exceptions – clients who don’t fit your norms. We once had a demonstration subject who’s signal for ‘I am waiting for the next question’ was to look down in her lap. We completely mis-read the signal and thought that she was continuing to process. It wasn’t until the next day she revealed that she had been sitting waiting for us while we were sitting waiting for her!
Transcript annotated for calibrating
You can download a transcript of the 10-minute video’d demonstration mentioned above at: SyM-Lite-calibrating-transcript-v3.pdf [File to be added soon] It has been annotated in two ways:
An analysis of the transcript revealed that James asked 24 questions in total (excluding the Set Up and Set Down phases and the three repeating back the clients words without asking a question), of which:
3x Like to have happen?
6x Anything else?
4x What kind of?
2x When … What happens to …?
4x Then what happens?
James reflected after the self-modelling:
Modelling the information and calibrating the client are two closely interlinked processes. In this particular case I have put much less in the modelling column than is usual. I think the client played a big part in this (a) because it was a simple landscape with just a few symbols, (b) he gave very concise answers, and (c) he kept close to his original frame. Much of the time it was ‘obvious’ what to ask about next in order to keep the process flowing.
I think this transcript shows that the length and complexity of a session is sometimes down to the client distracting themselves, loosing the frame, talking for the sake of it, not being able to select for salience, and so on. When you add in that the facilitator can do all of these things too, it’s no wonder sessions get convoluted. And while facilitators need to learn to calibrate what is working and what isn’t – so do clients!
What we calibrate against
Below is a tentative first-pass at describing some of the factors we take into account when calibrating whether what we are doing is working or not.
In the notes for the May 2011 Developing Group we introduced ’Three principles of Symbolic Modelling’. The third principle states:
We use the term ‘encouraging conditions’ to emphasize that we can’t make a client change, nor can a technique, and often a client is powerless to change themselves – until the conditions are right for them. Then the system re-synthesises of its own accord, often much to the surprise of the client and facilitator alike.
What are the conditions to be encouraged? And how do we know the client is experiencing them? To assist in answering these questions we have extracted the criteria being calibrated (not the client content being modelled) in the attached transcript:
Calibrations during the 6 SyM phases from attached 10-minute transcript
Set Up (Phase 1)
Identify desired Outcome (Phase 2)
Develop desired Outcome Landscape (Phase 3)
Explore effects of desired Outcome landscape (Phase 4)
Mature Changes (Phase 5)
Set Down (Phase 6)
The first thing to note is that the majority of the criteria are from the perspective that Ken Wilber calls “Zone #2 singular”: an interior (of an “I”) looked at from the outside, i.e. a third-person modelling of a first-person reality, from their perspective. The criteria marked with an * are from different perspectives.
We can summerise the above as calibrating whether the client is (or is not):
The client is at some level always embodying what’s happening. And when that becomes more overt and living-it-now it is often a prelude to a shift in the system (although embodying a memory of a trauma can be counterproductive).
The three criteria above lead to psychoactivity – The client responding unbidden to their own inner symbolic world, especially with curiosity, wonder and surprise.
Other criteria are that the client:
4. Goes with the flow. Although not specifically mentioned in this annotation, there is often a palpable moment when the client lets go of trying to control and
understand what is happening and “takes the hand-brake off” as one client put it. This is not a passive ‘do it to me’ but rather an active participation in the unknown.
5. Is identifying, developing and exploring the effects of their desired outcome landscape.
6. Engages in deeper than usual processing, jumping to a pattern level or making new connections that lead to a different engagement with or perspective on the material.
7. Meta-comments that they are deriving (potential) benefit from what’s happening (their nonverbals are often a kind of meta-comment too).
8. Indicates that a change or shift prompts:
- more changes (a contagion) leading to a difference that makes a valued difference
- a more resourceful response to a previously perceived problem
- an alteration to a pattern that aids the desired outcome happening
- a condition necessary for the desired outcome to occur
- the desired outcome to start happening.
- a reaction which functions to keep things the same. This indicates the client’s system is responding in the here and now and a new aspect is revealing itself – and that can be worked with ‘live’.
As we have hinted, for every criteria we, or anyone defines, there will be contexts when the opposite applies.
Last thoughts …
One of the reasons some people who have lots of experience find the transition to a clean approach challenging is because they have to set aside their usual calibration and learn to calibrate with a new set of filters. We can’t teach them directly how to do that, they have to acquire it for themselves.
We developed our coaching in the moment model in part to give trainee facilitators the chance to experience first-hand being guided by someone who has well-honed calibration skills. The expectation is that the trainee will code the pattern of external
signs and develop a set of internal signals for when the process is
working – and when it’s not.
1 See also our comments on John Grinder’s video about ‘What’s ethical in NLP’
2 p. 137. The section on calibration includes exercises to develop sensory acuity at: nlpuniversitypress.com/html/CaCom.html
3 See James’s blog on Desired Outcomes
4 This is one version of many based on the original Story by Loren Eiseley, see en.wikipedia.org/wiki/The_Star_Thrower
5 See James’ blog, ‘The’ Therapeutic Relationship
6 Since the paper was written James has been involved in discussion about a closely linked topic, modelling criteria: How-DO-you-know
7 Pioneered by Maturana and Varela, this is also known as biological phenomenology, since it attempts to describe the phenomenal inner world of the organism itself. For a very detailed description of Ken Wilber’s “8 native perspectives” see his Excerpt D: The Look of a Feeling: The Importance of Post/Structuralism, kenwilber.com/writings/read_pdf/86.
POSTSCRIPT posted by James Lawley 7 Mar 2013.
Below is a link to a fascinating article about psychotherapists’ attitude to their clients’ outcomes. Here are a couple of headlines:
- 90 percent of psychotherapists not only think they are above average, they think they’re in the top the 25%!
- About 8 percent of adult patients actually deteriorate at the time they leave treatment.
- 15-24 percent of adolescent clients leave treatment worse off than when they started.
- In one study, clinicians case notes were compared with client
outcomes, and clinicians missed 75 percent of people who were getting
- In another study, licensed professionals were no better than trainees
in identifying people who were worse off than when they entered
- Clinicians tend to overrate the “therapeutic alliance” as positive, compared to client self-reports.
- Even when clinician know about these blind spots, they do not improve
their assessment accuracy without independent measurements.
These notes were first presented at The Developing Group, 9 July 2011