Illness - a fact or an experience?
- Two types of information
- A factual question also requires interpretation and assessment
- "It just didn't occur to me"
- "It is not any real illness"
- Factual information – is it really available?
Koko dokumentti sivutettuna
Kaisa-Mari Okkonen, the writer works as Senior Statistician at the Social Statistics department of Statistics Finland.
Health is an experiential state, which, when studied with survey methods, requires sensitivity to the effects of the measurement methods and the context of asking. Pre-testing related to questionnaire and question design helps to understand the nature of knowledge and to make interpretations of this multidimensional phenomenon.
DESIGNING OF A DATA COLLECTION FORM is a complex where the objectives of measurement and the practices of interview interaction must be reconciled. This is often not easy, because from the viewpoint of measurement, good and precise question formulations do not necessarily work well in an interview situation, and on the other hand, question design only on the terms of interaction weakens the quality and comparability of the responses obtained.
In planning Statistics Finland's surveys this tension is managed by means of questionnaire standards. The aim of the questionnaire standards is to make the data collection situation as controlled as possible as regards its measurement objectives in order to retain the validity of measurement (e.g. Houtkoop-Steenstra 2000). They also help to make the work of an interviewer easier by giving enough information about the purpose of questions and the means used to receive answers. Questionnaire standards are thus also interview standards.
In Statistics Finland's standards for interview questionnaires the questions are divided into factual questions and opinion, attitude and knowledge questions (so-called M questions). Factual questions are thought to be of factual nature, while the latter contain both subjective views and assessments and questions measuring the person's knowledge level. Presenting these questions in interview situations is guided by various principles.
The borderline between fact and subjective information is not, however, always clear. This observation was made when in autumn 2010 we were pre-testing at Statistics Finland's Cognitive Laboratory the ad hoc question module implemented in connection with the Labour Force Survey of Eurostat, the Statistical Office of the European Communities concerning health and capacity to function. The module examines the respondent's long-term illnesses, activity limitations and their effects on the capacity to function and work.
Two types of information
The practical difference in interview situations between factual questions and so-called M questions separated in Statistics Finland's questionnaire standards is that various ways of probing can be used more freely in presenting factual questions, while in M questions it is defined more strictly beforehand what kind of probing can be used. (Figure 1.) Questionnaire standards are subject to agreement, and statistical agencies and research institutes in different countries use different variations on the degrees of interview standardisation and interviewer autonomy (e.g. Viterna—Maynard 2002).
Figure 1. Instructions on presenting question forms and probing.
Source: Design standards of interview questionnaires 2001. Statistics Finland.
The definition of question type is connected to the content and has to be made by the researcher designing the questionnaire. The question type cannot be inferred from the outward appearance of the question: externally the same question can be directed, depending on the objective, to collection of factual information or to testing of the respondent's knowledge level, for instance (Design standards of interview questionnaires 2001). Defining the type of question thus depends on the aim of the question, and it is indicated in the questionnaire by marking M questions with red letter M. The other questions are by default factual questions.
A factual question also requires interpretation and assessment
The module connected to the Labour Force Survey measuring health and work capacity inquires about the respondent's longstanding or chronic illnesses and other health problems. Eurostat has defined longstanding illnesses as follows: They refer to such illnesses or health problems which have lasted or are likely to last for at least 6 months or are episodic or otherwise recurrent. Health problems need not be diagnosed by a doctor, and mentioned should also be such illnesses and health problems that do not bother or whose symptoms are kept under control with medication.
Eurostat recommended asking about longstanding illnesses by going through a list of different illnesses with the respondent. In connection with a corresponding survey previously made in 2002, it was found that a simple yes/no question about whether the respondent had longstanding illnesses, does not produce comparable results between countries and reduces actual reporting of illnesses. A list of illnesses was thought to produce more exact and comparable results.
In questionnaire design the intention was to make the inquiry as short and simple as possible. The aim was to avoid using long and burdensome lists of illnesses for those respondents who do not report having any longstanding illnesses. In addition, it appeared from the previous survey that in Finland illnesses were rather over-reported than under-reported: then one third of Finns of working age said they had some longstanding illness (Figure 2). This is clearly more than in the other EU countries, which led us to consider whether the results told more about cultural differences in responding than about the generality of longstanding illnesses in different countries.
Figure 2. Share of working-age people (aged 16 to 64) having reported longstanding illnesses in 2002. Per cent.
Source: Results from the Labour Force Survey ad hoc module for the year 2002. Eurostat.
Before the decision about the eventual mode of asking it was necessary to know in what way responding changes when the mode of asking alters. Even if the question searches for information based on facts, the interviewees go through a set of different cognitive processes when answering that have an effect on responding. These are: 1) understanding and interpreting the question, 2) recall and assessment, 3) formulation of the response, and 4) actual responding (Ahola el al. 2002).
For this reason it was decided to use a comparative array, where the same interviewee was asked about illnesses in different ways and it was then compared how the responding changes. First, a so-called open question was asked (see Table 1, mode of asking 1), where first a yes/no answer was requested and after a yes answer, it was specified what kinds of illnesses or activity difficulties the respondent had. The intention was thus to exclude those respondents who do not have illnesses, and thus lighten the interview burden among those who are completely healthy.
When the whole questionnaire was tested for other questions as well, we returned to longstanding illnesses by means of the list of illnesses recommended by Eurostat (Table 1, mode of asking 2). The mode of asking was thought to influence particularly recalling of illnesses, but the interpretations of the question may also vary when the respondent gains more information about what type of information is being sought. In addition to this, different formulations were tested in the open questions, but the number of persons taking part in the pre-testing was so low that the material was not sufficient for studying differences in formulations.
Table 1. Tested modes of asking.
| Mode of asking 1: open question | Mode of asking 2: structured list |
| Next I will ask about your longstanding health problems. I mean by longstanding such illnesses or health problems that have lasted or will last for at least six months or are recurrent. It need not be diagnosed by a doctor. Please also consider health problems controlled with medication. Do you have any chronic illness or other longstanding health problem? 1) Yes, one 2) Yes, several 3) No illnesses. |
Do you have any of the following longstanding health problems: 01. Musculoskeletal disorder related to arms or hands? 02. Musculoskeletal disorder related to legs or feet? 03. Neck, shoulder or back disorder? 04. Heart or blood-vascular disease? 05. Diabetes? 06. Respiratory disease? 07. Cancer? 08. Internal organ or digestive system disease (liver, kidneys, pancreas, bowels)? 09. Migraine or other severe headache? 10. Skin disease? 11. Learning difficulty, such as dyslexia? 12. Epilepsy? 13. Depression? 14. Chronic anxiety? 15. Other mental problem? 16. Mental disability 17. Some progressive disease (e.g. MS, HIV, Alzheimer's disease, Parkinson's disease)? 18. Some other longstanding illness or health problem (thyroid problems)? |
| If response 1 or 2: Which illness or health problem do you have? |
"It just didn't occur to me"
Regardless of the formulation, the open question and list of illnesses produced in pre-testing a different number and different kinds of illnesses and health problems. Ten of the 13 persons interviewed said in connection with the first question that they had illnesses. When asked about it again with the help of the list of illnesses, seven reported several illnesses and partly different ones than in the open question. The five remaining interviewees started to consider as concerns the listed illnesses whether they should say yes or no. Listing of illnesses thus made respondents think more accurately about their own state of health, and for this reason this produced different responses.
In addition, the list helped to elicit better such health problems that the respondent does not consider illnesses despite reading of the definition of longstanding illnesses. These are especially work-related musculoskeletal system problems, neck, shoulder and back disorders, illnesses that might be seen as "everyday", and such health problems that are not acute at the time of responding.
Interviewer (asks mode 2 question): "Migraine?"
Respondent F2: "Yes. I'm a really typical migraine case. It's not every day, but it can be every week. It did not occur to me earlier as it is such an everyday bother."
If the respondent's illness does not appear directly from the list of illnesses, it may be left unreported. For this reason the list must be as exhaustive as possible to work in the desired way. However, the list acts as an aid to remembering, which is important particularly as concerns general but undiagnosed and minor health problems.
An advantage of the open question is that it makes it possible for the respondent to tell about the illness personally. Its shortcoming is that even if the interviewer reads out the definition of a longstanding illness, respondents easily mention also such serious illnesses that have had a strong effect on the person's personality and identity, but that have been treated and no longer bother them. For instance, treated breast cancer gained prominence with the open question but not with the direct list of illnesses.
Interviewer (asks mode 1 question): "Do you have some chronic illness or other longstanding health problem: yes, one; yes, several or no illnesses?"
Respondent F3: "I have to say several. One of them is heart decease and it is constantly under control. And then I have breast cancer, which is newer."
(Later in the interview)
Interviewer (asks mode 2 question): "Cancer?"
Respondent F3: "No."
Interviewer: "So here cancer is not... is it over?"
Respondent F3: "I feel so because I have to take medication every day because of my heart but not because of cancer. Breast cancer can break out again, though. Therefore, it is still at the back of my head. I am still under constant observation. As women have mammography, it will come out there."
The list question, when made with care, also seems to support respondents' recalling and reduce the experienced response burden, and lower reporting of already treated illnesses that are meaningful to the respondent. An open question encourages respondents to talk more about their experiences of illnesses than a structured question mode.
"It is not any real illness"
The question concerning longstanding illnesses aims for fact-like information, but it was found in the pre-testing that despite reading exact definitions, responding requires respondents' own interpretation of the nature of health problems. The interpretation is made especially in relation to the respondent's idea of longstanding and of illness and the context of the question, i.e. the Labour Force Survey.
In the open question respondents interpreted a longstanding illness to mean illnesses that are hampering everyday life, diagnosed and needing medical care. Lack of "feeling of illness" filtered out such health problems bothering the respondents that did not correspond to their image of "actual" health problems. An exact definition of a longstanding illness does not necessarily affect responding if the respondents do not consider their problems longstanding health problems. In addition, the context of asking, that is, the Labour Force Survey, had a distinct effect on the interpretation of illness. Many respondents filtered out their health problems based on that they did not affect how they managed in their job.
Interviewer (asks mode 2 question): "Respiratory disease?"
Respondent F1: "I have a kind of allergic habit, but I do not think it is an illness because it does not prevent me from doing anything. It did not occur to me when I thought that a longstanding illness is one that is diagnosed. Allergy tests have never shown me to have any allergy. My nose is just blocked. It has been examined and medication has been prescribed for it. I think it is rather a bother than an illness. It is not a disease but it is a nuisance."
* * *
Interviewer (asks a list of illnesses question): "Musculoskeletal disorder related to arms or hands?"
Respondent M2: "Yes, my hands feel numb but I try to take care of it at a masseur. I have gone through all vertebra scans but it has not been diagnosed. I do not think it is an illness as it does not prevent me from working."
Factual information – is it really available?
The borderline between fact and subjective view is not clear especially in questions concerning illnesses. It is evident that listing of illnesses produces a different number and different kinds of answers than a simple question relying on the respondent's interpretation and recalling. The researcher may also try to control the fact content of the responses gained by definitions that partly exclude responses based on personal experiences and definitions of illnesses differing from one another. For this reason, in the final questionnaire it was decided to use the question mode where illnesses are listed one by one.
Illness is in the end an experience whose reporting cannot be controlled all the way. Symptoms people have may fulfil the criteria for illnesses set in the definition but ultimately the respondent's own experience of the nature of illness and the harm its causes influence responding as does the idea of what illness as a concept means. The context of asking - the Labour Force Survey - also guides the respondent's interpretation of illnesses, even if in the question formulation reporting of illnesses is not restricted to the viewpoint of work capacity.
The multidimensionality of the information gained does not remove the basis from good questionnaire and question design, which is still important even if the world being measured is not always divided clearly into different types of information. However, it is important to understand the real nature of assumed factual information. Only then it can be known what was really measured and what can be interpreted from comparisons between countries.
References:
Ahola, A. – Godenhjelm, P. – Lehtinen, M. (2002): The art of asking. Role of the Survey Laboratory in developing questionnaire surveys. Reviews 2/2002. Statistics Finland, Helsinki. (In Finnish)
Houtkoop-Steenstra, Hanneke 2000. Interaction and the Standardized Survey Interview. Cambridge University Press, Cambridge.
Okkonen, K.-M. 2010. TYTI AHM2011: Pre-testing of the work capacity and employment questionnaire. Questionnaire testing series 1/10 Unpublished. Statistics Finland, Helsinki. (In Finnish)
Design standards of interview questionnaires. 2001. Internal working instructions. Statistics Finland, Helsinki. (In Finnish)
Viterna, J. S. – Maynard, D. W. 2002. How Uniform Is Standardization? Variation Within and Across Survey Research Centers Regarding Protocols for Interviewing. In: Standardization and Tacit Knowledge. Ed. H. Houtkoop-Steenstra et al.
Last updated 26.9.2011