To score or not to score

‘Jenny’ visited the Great Ormond Street Hospital MUS clinic for support with her non-epileptic seizures following a road traffic accident. She was experiencing up to 50 seizures a day and as a result was no longer able to attend school or live an independent life. She relied on her parents support to wash and dress her and assist her around the home. During her clinical assessment with the team, Jenny described the accident as something that had ‘just happened to her’, that her life was ‘going well’ with nothing to complain about and her mood overall was good. Jenny’s description of the impact of her seizures on her emotional wellbeing was clearly discrepant from the picture she described of their impact on her daily life.

Questioning questionnaires

The development and use of structured questionnaires in psychology has transformed our discipline, enabling us to go from essentially a qualitative discipline to one that is quantitative, using a scientific approach with predictive validity. We all use measures on a daily basis and their utility in many of the common and complex conditions in mental health – including depression, anxiety and behavioural disorders – is unquestionable. 

Questionnaires can be a very helpful way to gain information about a range of symptoms, physiological, cognitive, behavioural and affective. There are a number of advantages of questionnaires; they are typically short, easy to administer and can collate a large volume of information which can complement a more traditional clinical assessment. Many questionnaire measures are standardised, meaning the respondent’s answers can be compared to a normative population to give an indication as to the severity of difficulties compared to others. The degree to which a standardised clinical instrument is said to be valid is dependent upon many factors. One important factor is the degree of similarity between the respondent and the normative group they are represented by. In addition, questionnaires can help us to monitor outcomes when used as part of a comprehensive clinical assessment.

Drawing on our experiences in working clinically with children with complex and comorbid conditions, we would like to highlight some of the issues that have arisen and the way in which we have overcome these. We highlight four examples of when we have overcome difficulties in using questionnaires and look to discuss how to use measures meaningfully in the face of complexity.

The impact of comorbid neurodevelopmental difficulties

In outpatient mental health settings, there are high-rates of neurodevelopmental and language difficulties which often go unidentified (Cohen et al., 1998). Co-morbidity often impacts on the phenotypic expression and phenomenology of a disorder as well as the response to treatment. For example, rates of tic disorders amongst children with OCD range from 9 per cent to 59 per cent (Stewart et al., 2004) and there are well recognised differences between tic and non-tic related OCD, including different phenomenology, different responses to standardised medication regimes (Ginsberg et al., 2008) and different responses to  psychological therapy (Bennett et al., 2015). For instance, people with tic related OCD tend to report relatively little obsessional content in the context of a relatively high number of compulsions. This differentiated expression of a clinical disorder leads to questions regarding the level of impact comorbidity has on the validity of standardised self-report questionnaires.

This has important implications for the assessment of common mental health conditions, as well as for gauging response to treatment. If questionnaires are limited to one particular focus when a client presents with multiple problems (physical and psychological), questionnaires alone may provide more questions than answers.

The impact of functional somatic symptoms

Functional somatic symptoms are a well-recognised phenomenon (Stone et al., 2006). Some of the children attending a Tier 4 specialist mental health service at Great Ormond Street Hospital (GOSH), experience functional somatic symptoms that have a significant impact on their daily lives. In some cases, mental health symptoms are under-reported, including on self-report questionnaires.

In this clinical setting, clients like Jenny (see above) are the norm rather than the exception. In fact, many children who are found to meet criteria for an anxiety disorder or for depression as a result of an extensive multidisciplinary clinical assessment, score within the normal range or below clinical thresholds on questionnaires assessing anxiety and mood.

One possible cause of this under-reporting of mental-health symptoms may be illness attributions; in other words, an individual’s beliefs about what is causing the symptoms. If they believe the cause to be physical, they may not report those symptoms when completing a questionnaire they know is related to mood or mental health, as they have attributed that symptom to a different cause. Additionally, the stigma surrounding mental ill health may also contribute to this under reporting, as clients may fear being labelled as ‘depressed’ or ‘anxious’.

The impact of alexithymia

Alexithymia was traditionally defined as having difficulties recognizing and describing one’s own emotions. However, people with alexithymia often find it difficult to recognise and discriminate between non-emotional internal states too (e.g. heart rate, hunger, thirst; Brewer et al., 2016). Recent evidence in adult populations suggests that alexithymia is associated with interoceptive difficulties, such as difficulty counting one’s own heart beats (Herbert et al., 2011). Research has consistently shown an association between alexithymia and functional somatic symptoms and with a range of physical health conditions such as diabetes or heart disease. Difficulties in recognising emotions and perceiving bodily sensations may make it difficult for people to accurately recognise physical and psychological symptoms, and therefore, endorse them on self-report questionnaires.

Alexithymia might also mean that a client’s scores on standardised measures worsen during psychological treatment, as their ability to recognise, experience and express emotions increases. Within the Tier 4 CAMHS at GOSH, questionnaires given to children with MUS – together with difficulties with emotional expression at the beginning and end of treatment – most often result in elevated scores, even though their physical symptoms and quality of life has dramatically improved. With better insight into their internal world, children may have greater awareness of their thoughts and feelings that would then be amenable to further psychological intervention.

The impact of comorbid physical health conditions

Symptoms of mental health problems may overlap with symptoms of physical health problems. One specific example is fatigue, which can arise as a result of, say, cancer and its treatment, but is also a symptom of depression. Questionnaire items focusing on depressive symptomology include items related to tiredness and lack of energy, which may lead individuals with cancer to score more highly on measures of depression due to symptom conflation.

Additionally, individual items on questionnaires may not be sufficiently specific or selective enough to enable the respondent to indicate explanations for their particular symptoms. The Beck Anxiety Inventory (BAI), for instance, asks the respondent to rate how much they were bothered by feeling shaky/unsteady during the past month; for a person with Parkinson’s disease, they may endorse a higher score on this item, even in the absence of anxiety (Salazar et al., 2017).

Even if the symptoms reported actually reflects a client’s current symptom profile, relying on population norms based on samples that are most likely to be different to those of our clients in both mental health and physical health settings can mean that we run the risk of drawing an erroneous conclusion about severity. This highlights an urgent need to thoroughly validate commonly used measures within specific mental and/or physical health populations so that we know what cut-offs to apply and how best to use this additional information in this context.

However, as is often the case, multimorbidity means particular individuals may represent multiple different patient groups or populations requiring different validated clinical thresholds. This may lead to the risk of developing smaller and more fragmented normative scores that do not necessarily provide the best representation of the client group as a whole. Inflated scores are not necessarily a problem when working clinically with clients, providing they are merely used as a screening tool and are interpreted in light of other information about the presenting picture as a whole.

How can we best use questionnaires in clinical practice?

Questionnaires form just one part of an overall assessment and formulation of a child’s difficulties. It is important when formulating a case that questionnaires (addressing one aspect of psychopathology) are integrated into the overall conceptualisation of the child’s difficulties. Thus, the potential difficulties we have highlighted should not mean that we abandon questionnaire measures all together; rather, that the results are interpreted in light of other information gained during assessment.

Most importantly, we are taught as psychologists to be cautious about use of questionnaires, and best practice is to use these as adjuncts to a thorough clinical assessment and formulation. Care needs to be taken when using questionnaires as screeners in education and physical health care settings, administered by professionals who do not have the same mental health training or awareness of the limitations to these questionnaires. Additionally, we need to be aware of the limitations of questionnaires being used to help individuals screen themselves for particular difficulties in the context of help-seeking online.

We have highlighted a number of reasons why, although questionnaires can be a valuable source of information, they should not be exclusively relied on. Therefore an important contribution psychologists can make to better overall clinical care is to support and educate teams that are new to questionnaire use in their practice; to better highlight limitations and support individuals in building a formulation that incorporates wider biopsychosocial difficulties.  

Authors: Loades, M.E1,2., Taylor, C3., Stark., D3., McFarlane, F3.,Heyman, I3.,  & Hadji-Michael, M3.

Affiliations: 1 Department of Psychology, University of Bath. [email protected]

2 Bristol Medical School, University of Bristol

3 Department of Child and Adolescent Mental Health, Great Ormond Street Hospital

References 

Brewer, R., Cook, R., & Bird, G. (2016). Alexithymia: a general deficit of interoception. Royal Society of Open Science, 3(10), 150664. DOI: 10.1098/rsos.150664

Cohen, N.J., Menna, R., Vallance, D.D., Barwick, M.A., Im, N., & Horodezky, N.B. (1998). Language, social cognitive processing, and behavioral characteristics of psychiatrically disturbed children with previously identified and unsuspected language impairments. The journal of Child Psychology and Psychiatry and Allied Disciplines, 39(6), 853-864.

Herbert, B.M., Herbert, C., & Pollatos, O. (2011). On the relationship between interoceptive awareness and alexithymia: is interoceptive awareness related to emotional awareness? Journal of Personality, 79(5), 1149-1175.

Salazar, R.D., Le, A.M., Neargarder, S., & Cronin-Golomb, A. (2017). The impact of motor symptoms on self-reported anxiety in Parkinson’s disease. Parkinsonism & Related Disorders, 38, 26-30.

Stone, J., Smyth, R., Carson, A., & Warlow, (2006). La belle indifférence in conversion symptoms and hysteria: systematic review. The British Journal of Psychiatry, 188(3), 204-209.