Deafness and the art of psychometric testing

Bob is a builder. He has been working for 40 years, and his tools have become extra limbs rather than inanimate aids. He has whole areas of cortex dedicated to plastering. Last month he returned to his van to discover the doors prised open and his tools gone. Mike is a psychologist. He has been working for 40 years, and his manner, tone of voice, choice of words, receptivity and empathy are finely honed to the needs of his clients. Last month he began seeing a client with no speech, her first language being British Sign Language (BSL).

Bob quickly contacts his insurance company and within the week has a complete new set of tools. Mike quickly contacts a sign language interpreter agency and arranges for interpreter support to be available for sessions with his deaf client. Relieved to be able to conduct his business as usual, Bob sets about organising jobs and arrives on site. Immediately he realises that these tools are just not the same. Relieved to be able to conduct his business as usual, Mike meets with his client and immediately comes to the same conclusion.

Hearing psychologists faced with a client with whom they cannot communicate have been described as entering a ‘Shock-Withdrawal-Paralysis’ (Schlesinger & Meadow, 1972) in which fears of incompetence regarding deaf people (shorthand here for ‘prelingually profoundly deaf people who use sign language’) lead to a diminishing ability to use existing clinical skills. Obviously this is disabling. In addition, the relief felt when this paralysis is apparently resolved can be so tangible that the extent to which it truly is resolved can be grossly overestimated. Just as a hearing aid will amplify but not correct distortion, so booking an interpreter is a necessary but not sufficient step towards business as usual. Though difficult to admit, it is likely that when de-skilled in this fashion we reach for tools which appear to rely less heavily on our ability to properly connect with our clients – tools which have a validity that appears inherent, tools which have a heavy mass of ‘evidence’ to support them.

There is a small but admirable literature on working with deaf people in psychological contexts (Austen & Crocker, 2004; Hindley & Kitson, 2000); this article is concerned specifically with the art of psychometry when applied to deaf people.

Tests to be faced

There are very few psychometric instruments available that are appropriate for use with prelingually profoundly deaf people. When testing such people, there is a great deal to be borne in mind. Perhaps the most important point to consider is the validity and reliability of the tests used; as you would normally, but also as a result of any changes made to the formal administration in order to communicate the task to the client. Asking four main questions can help (Zieziula, 1982):

Verbal test items, or performance items?

In this context, ‘verbal items’ require reading or writing skills, untaught (indirectly acquired) knowledge, or items dependent upon other abilities based upon English (such as understanding of proverbs); ‘performance’ items do not. Verbal items are inappropriate for deaf people, particularly prelingually deafened. Such people usually have difficulty with English syntax and vocabulary independent of cognitive function, as a result of English being a purely visual (unspoken, and therefore considerably less frequently encountered) language for deaf people. The experiences of hearing people with written and spoken English mutually reinforce their abilities in both, whereas BSL and written English have completely distinct vocabularies and grammars. In addition, deaf school-leavers’ median reading age has been estimated to be at the nine-year level irrespective of intellectual function (Conrad, 1979).

Verbal instructions?

This is important even if the tasks themselves are considered to be performance tasks. For those measures that offer little or no flexibility in the way in which instructions are presented, departure from the formal administration makes comparison with the normative sample misleading in ways that cannot be predicted or allowed for. There is no such thing as a perfect or ideal translation or interpretation – it always presents a departure from the original statement. Therefore it is important to identify the extent and nature of these departures (for more on this see below).

Do items discriminate against (or for) those with auditory impairment?

Items that discriminate against deaf people are more common, but both for and against present problems of comparison with a normative sample that did not include deaf people. An example would be a scale containing the (observational) item ‘Makes telephone calls’. Although one might suppose an observational scale protects against many of the concerns presented here, clearly this item would underestimate the individual’s functioning. Perhaps less obviously, but equally critical, an attempt to address the cultural validity of this item by converting it to ‘Makes minicom (textphone) calls’ still underestimates the ‘true’ level of functioning since a deaf person’s experience of using a textphone differs entirely from a hearing person’s with a telephone (in terms of when one is first seen used, how often they are seen on TV, and so on).

Conversely, users of three-dimensional visual languages, with pronoun forms that rely upon memory of the locations of subjects and objects in the signing-space, might reasonably be expected to be advantaged when using block-tapping tasks (in which blocks in a disorganised array between assessor and subject are tapped in sequences the subject must reproduce – estimating the ability to hold visuospatial sequences in working memory). However, this advantage disappears when the person sits alongside the examiner instead of opposite, again showing the unpredictable nature of the effects of deafness on psychometric testing.

Are deaf people included in the normative sample?

Apart from a very select handful of tests the answer to this question is always no. Norms for deaf people are rarely provided. There are good arguments for and against comparing this deaf person with either deaf or ‘hearing’ norms, and the answer depends of course upon the reason for testing (for example vocational selection may require comparison against norms applicable to the population of applicants for a post, whereas assessment to determine the presence or degree of learning disability would likely require comparison with a reference group of deaf people).

However there is only ever an argument for ‘hearing’ norms if that normative sample contained deaf people in equal proportion to the population of concern, and if the results from that subsample are demonstrably as reliable as those of the hearing subjects. Such norms are rarely if ever established, not least because of difficulties defining and sourcing a homogeneous reference group, and prelingually profoundly deaf people are more likely to be formally excluded from the normative study. The interpretation of results from people (deaf) who do not mirror individuals upon whom the test was designed (hearing) must therefore be explicitly cautious.

Using an interpreter

The four points above are relevant whether or not the assessor is communicating directly with the client in BSL, but of course in the majority of instances this will not be the case. When conducting a psychometric assessment with a BSL interpreter, further issues are raised.

How reliable is your communication support?

The ability of the interpreter(s) must be stated in the report and considered when interpreting the results. Even if the interpreter is NVQ Level IV accredited and a registered sign language interpreter, which ought really to be the only acceptable standard for psychometry, the reliability of the interpretation will decline significantly after 35 minutes (Brasel, 1976). Two interpreters should be used and regular breaks should be introduced (which can itself conflict with a formal test administration). Interpretation will always add to the error in the estimated level of function, and it is impossible to say for each instance whether it contributes to an over- or underestimation. The extent of this error is, in part, a function of the competence of the interpreter.

How else does communication support influence the assessment?

Even if the communication support could be 100 per cent perfect, it remains important that each item of each test be discussed in advance with the interpreters. The assessor must be fully aware of the nature of the signed administration in order that the psychometric equivalence of the standard and the signed administration of that item can be judged. For example, the Similarities subtest of the WAIS-III contains the question, in English, ‘In what way are a coat and a suit alike?’ It is fairly easy to translate this into a BSL equivalent with little debate. However the nature of the most common signs for coat and suit make it self-evident that both are articles of clothing – a response that receives maximum points. Conversely, the signs commonly indicating a coat and a suit ‘rhyme’ in terms of both being bimanual and symmetrical, sharing the same location in space, employing very similar movements, and handshapes differing only in terms of thumb position. It could be argued that these phonological similarities are unreasonably misleading or that they imply alternative false answers (the location of both signs is also commonly used for emotions) in a way that the spoken English items do not.

Recommendations

These concerns all contribute unknown amounts of error to the estimated level of whatever psychological construct is under scrutiny (such as intellectual function). In addition the ways in which these concerns may be addressed (by departing from standard administration in a variety of ways) also contribute error to the estimate. The magnitude of this error and the overall direction of it are both unknowns and lower the reliability of the obtained results. It may sometimes be possible to evaluate the direction of each error component from each shortcoming and adaptation in order to more confidently interpret test results. For example, if most shortcomings and departures tend to disadvantage the candidate, then the result may more usefully be understood to reflect a minimum level of functioning rather than an estimated actual level. However, such interpretations will necessarily be conjectural.
Generally, psychologists should be advised to:
l    assess the deaf person with support from qualified BSL interpreters;
l    discuss each item of each test in advance with the interpreters, and afterwards so that any instances of note may be raised and accommodated in the interpretation of the results;
l    consider which, if any, tests to use in the light of the points above regarding test items, test administration, and reference groups;
l    interpret results with extreme caution in the light of the above points as well as further issues pertaining to communication support;
l    make each of these shortcomings explicit in the report, so that future readers will not jump to erroneous conclusions; and
l    endeavour, when adding error by departures from standard administration, to add error that tends
in one particular direction.

Ultimately it is important to remember that psychometry does not provide the comfort blanket psychologists may seek when working with certain client groups, in this example deaf people. In addition, what may at first glance seem to be straightforward technical choices turn, on reflection, into relatively significant concerns. The overriding approach to be taken is one of uncertainty, doubt and feeling comfortable with ignorance. Well-founded understandings are based on knowing what we do not know and keeping what we do know in the context of that ignorance.

- Jim Cromwell is Principal Clinical Psychologist for National Deaf Services, 146a Bedford Hill, Balham, London.
E-mail: [email protected].

Weblinks

Jim Cromwell’s ‘Peripheral Brain’: brain.jimcromwell.com
British Sign Language fingerspelling: www.jimcromwell.com/BSL/spell.htm
National Deaf Services: www.swlstg-tr.nhs.uk/content/nds

Discuss and debate

This article is reminding us to be rigorous. Is this just routine cross-cultural psychology?
The deaf community requires culturally appropriate measures. How might such within-culture tools be developed when so few native deaf people are in a position to develop them?
Culture? Deafness is a disability. Isn’t it?
Have your say on these or other issues this article raises. Send letters to [email protected] or post on our forum at www.thepsychologist.org.uk. 

References

Austen, S. & Crocker, S. (2004). Deafness in mind. Working psychologically with deaf people across the lifespan. London: Whurr.
Brasel, B.B. (1976). The effects of fatigue on the competence of interpreters for the deaf. In H.J. Murphy (Ed.) Selected readings in the integration of deaf students at C.S.U.N. Centre on Deafness series (No.1). Northridge, CA: California State University.
Conrad, R. (1979). The deaf school child. London: Harper & Row.
Hindley, P. & Kitson, N. (2000). Mental health and deafness. London: Whurr.
Schlesinger, H.S. & Meadow, K.P. (1972). Sound and sign: Childhood deafness and mental health. Berkeley, CA: University of California Press.
Zieziula, F.R. (Ed.) (1982). Assessment of hearing-impaired people. A guide for selecting psychological, educational, and vocational tests. Washington DC: Gallaudet College Press.