The agreement between workers and within workers in regard to occupational exposure to mercury in dental practice assessed from a questionnaire and an interview
© Svendsen and Hilt; licensee BioMed Central Ltd. 2011
Received: 25 October 2010
Accepted: 23 March 2011
Published: 23 March 2011
The correct assessment and classification of exposure is essential in epidemiology. The validity of exposure data obtained by the use of questionnaires is, however, seldom evaluated. When conducting a study on the possible health effects from mercury exposure in dental practice, we compared answers on exposure from a job-specific questionnaire with answers to the same questions given at an interview 6 to 18 months later.
We examined the concordance between workers by comparing answers to the questionnaire given by persons working in the same clinics during the same time spans and the agreement within workers by comparing answers to the same questions from a questionnaire and from an interview. Other aims were to see if there was a difference in the answers to the questionnaire across job titles and to study the impact of missing information on the response rate in a detailed questionnaire.
There was a marked difference between the pairs of employees working in the same clinic regarding the start and termination years for the different preparation methods, and this was partly independent of their occupation. Kappa values for using different preparation methods in the questionnaire and at the interview varied between 0.41(moderate) to 0.88(very good).
The results of this study indicate that a mailed questionnaire will cause misclassification of exposure. The observed occurrence of false positive exposure classifications from the questionnaire compared to the interview was higher than for false negative. This is important and may result in serious bias if the prevalence of exposure is low. Due to missing information, detailed questionnaires may also be inefficient if the goal is to construct exposure measures from combinations of several answers in the questionnaire.
Exposure assessment based on questionnaires can introduce different biases involving both differential and nondifferential misclassification. Differential misclassification of the exposure occurs when the likelihood of being classified as exposed is dependent on the outcome, while nondifferential misclassification is a question of sensitivity and specificity of the tool used. In epidemiological studies, differential misclassification may bias an observed relative risk in both directions, while nondifferential misclassification most often biases the relative risk towards the null effect [1, 2]. Detailed job-specific questionnaires are considered to differentiate fairly well between high and low exposed workers . It has, however, also been shown that self-reported exposure histories obtained by mailed questionnaires tend to underreport occupational exposure [4, 5]. Earlier studies have also shown that while both the sensitivity and specificity of self-reported questionnaires may be satisfactory, missing information, rather than erroneous reporting of exposure, is often the most significant source of misclassification. An extensive review has been published on the literature on the validity and reliability of common case-control exposure assessment methods, including self reported exposure.
When working on a retrospective study on the possible late effects from previous exposure to mercury in dental personnel, we developed a model for calculating a relative exposure score based on the answers from a detailed job-specific questionnaire. This score was intended to represent a value for the individual cumulative exposure to metallic mercury from previous work with dental amalgam. In the questionnaire, the respondents were asked in what years they performed the different kinds of amalgam preparations and how many treatments they performed each week. They were also asked how the amalgam was handled, if there were spills, and details about the work premises. The questionnaire as such and how the relative exposure score was calculated have been described in detail elsewhere . Between 6 and 18 months after having responded to the questionnaire, a selection of the study population was interviewed on the same topics.
Do respondents answer the same about previous exposures on two different occasions and with different methods, one by means of a mailed questionnaire and one by means of an interview?
Do respondents who had worked in the same clinic at the same time answer the same regarding exposures?
Are there differences in agreement between the answers across professional groups (between two dental nurses who have worked in the same clinic and a dental nurse and a dentist)?
Does an attempt to make an exposure score from several detailed questions in a questionnaire reduce the number of valid respondents compared to a simple yes-no question on exposure?
The questionnaire study was conducted during the spring of 2006 in subjects who had worked in the dental health services in three counties in the central part of Norway. From both private and public dental clinics we obtained lists of 2,247 previous and current dental personnel, born between 1913 and 1985, who were candidates for inclusion in the study. All of them received a mailed questionnaire to be returned in a prepaid envelope. The subgroups studied in this investigation were derived from the 1,193 respondents from this primary survey. The questionnaire part of the study is described in a former publication .
Questionnaire for the primary study group
Questions used in the questionnaire and for the interview with an occupational physician.
Questions used from the questionnaire
Questions in the interview guide
Have you used copper amalgam? yes- no
Have you ever worked with copper amalgam?
In that case, to what extent and for how many years?
In what time period did you do this:
What year did you start using copper amalgam?
In what year did all your use of copper amalgam end*
Have you worked with amalgam that was manually mixed in a mortar? yes-no
Have you ever manually weighted mercury and alloy and mixed it in a mortar?
In that case, when and for how many years?
From year...to year...
Have you mixed amalgam in a Dentomat? Yes-no
Did you use a Dentomat (semi-automatic device) to prepare amalgam?
In that case, when and for how many years?
In what time period did you do this:
From year... to year....*.
Was the Dentomat adjusted to produce soft or firm amalgam? Soft.... Firm...*.
How often was the Dentomat filled?*
More often than once a week.*
Once a week.*
Once every second week.*
More seldom than once every second week.*
Identification of the subjects who had worked at the same clinics during the same time - the between worker study
To find pairs of participants from the primary study group who had worked at the same clinic at the same time, we focused on responders from smaller communities. The selection was based on the current postal code of the responders' residence. For this work we chose to concentrate on smaller cities and community centers in order to increase the possibility that persons still lived in the city/community where they had worked. By the use of this restriction we also wanted to increase the probability that two or more participants residing in such a small city/community had in fact worked in the same clinic, as smaller communities in Norway tend to have only one dental clinic, while bigger cities usually have several public clinics and smaller, private clinics. As the name of the clinic was written in a free text field in the questionnaire, the individual responses were then examined one by one in order to create a list of clinics with pairs of employees working in the same clinic. Finally, we cropped the list to include only those pairs who had worked at least between 1980 and 1990 because this was a decade when the two most important preparation methods regarding exposure to mercury (using copper amalgam and using a Dentomat) were still in use, and when many dental clinics changed from these methods to more modern ones (e.g. pre-produced capsules). Most of the persons who had worked during the years 1980 to 1990 also had worked in the same clinic several years before and/or after this time period.
This procedure gave us 48 pairs from 23 clinics with 2 persons or more who had worked there at least in the years between 1980 to1990. The answers that were compared within these pairs regarded details on how, when, and how often they used the two preparation methods for amalgam; the copper amalgam and the Dentomat. The question about soft or firm amalgam had only two outcomes, while the question on how often the Dentomat was filled had four possible answers; i) more often than once a week, ii) once a week, iii) once every second week, iv) and more seldom than every second week. The term "partly agree" refers to answers next to each other on the scale i-iv and the term "partly disagree" refers to answers two positions from each other on the scale i-iv. All these questions are related to conditions in the clinic and not to individual work practice. As all pairs had not answered all the relevant questions, the number of comparisons was limited. For this part of the study it was difficult to cover the whole time period for the different preparation methods. As most of the clinics stopped using manually mixing of amalgam in the mortar and started using copper amalgam before 1980, these questions were excluded in the between worker analysis. The questions used in the between worker study are given in Table 1, and marked with an asterisk. We ascertained from the data that the person had started to work in the specific clinic before the year when they reported having started to use a specific method. In the same way; we ascertained that the person was still working in the same clinic after the year they reported having ended using the specific method. Thus, the methods used in the same clinic at the same time, and the year for the start and end of the use of a method should have been the same for the persons who worked there.
Interview by an occupational health physician - the within worker study
From the 822 female respondents in the original questionnaire study, we intended to invite 100 subjects under the age of 70 for further neuropsychological investigations. Our capacity to do extensive investigations of each person was the main factor that limited the number of participants to be included. In the end we had 95 subjects who showed up for the investigations. Among them, four had to be excluded, three because of other CNS diseases (two with brain tumours and one with sequelae after a cerebral stroke), and one because she was over 70 years of age. As a consequence, we ended up with 91 participants who were both interviewed and underwent neuropsychological investigation.
This interview with an occupational health physician took place between 6 and 18 months after they had responded to the questionnaire. During the interview the occupational physician inquired about the occupational history of the participants and also asked the same questions as in the original questionnaire in regard to the use of the different preparation methods for amalgam i) heated copper amalgam, ii) manually weighting and mixing in a mortar and iii) a Dentomat. The respondents were also asked how long they had used the 3 preparation methods. The questions in the interview guide are given in Table 1.
The study was approved by the ethical committee for medical research in Central Norway, and we had a licence for personal registrations from the Norwegian Social Science Data Services. The conduct of the study was deemed to be in accord with the Helsinki declaration on medical research ethics.
Data were registered and analyzed with the data program Statistical Package for Social Science version 14.0 (SPSS) (SPSS Inc., Chicago, IL, USA). The concordance between participants who had worked in the same clinic regarding the years the participants started and ended the use of copper amalgam and Dentomat was analyzed by calculating the mean difference from the absolute values of the difference between the pairs, standard deviation of the mean difference, and the 10, 50 and 90 percentile of the difference between the pairs of dental personnel in the same clinic, between those pairs that consisted of two dental nurses or two dentists and the pairs that consisted of one dental nurse or one dentist . The within worker agreement on the given number of years using the different treatment methods from the questionnaire and the interview respectively was also calculated in the same way. Cohen's kappa statistic was used as a measure of the within worker agreement between the questionnaire and the interview for the question as to whether they had ever used the three different treatment methods.
The between worker study
For the between worker study, 68 persons who had worked in dental health care between 1980 and 1990 were available. They constituted 48 pairs. Among these persons, there were 43 female dental nurses 6 female dentists and 19 male dentists. The mean age at the time of the questionnaire study for the females was 60.6 (SD 8.3) and for the men 63.2 (SD 5.9). The year of first employment in dental care ranged from 1955 to 1980.
Concordance between workers in regard to information about time.
Comparisons (number of pairs)
Mean Difference years
Std of the mean difference
The year all use of copper amalgam ended
Both dental nurses (13)
One dentist and one dental nurse ( 6)
The year the use of the Dentomat started
Both dental nurses (10)
One dentist and one dental nurse (13)
The year the use of the Dentomat ended.
Both dental nurses (12)
One dentist and one dental nurse (12)
Both dentists (2)
Percent of concordance between pairs that worked in the same clinic in the same years and during 1980 to 1990.
Comparisons (number of pairs)
Percent of pairs that
Partly agree 1
Partly Disagree 2
Did the Dentomat produce soft or firm amalgam?
both dental nurses (16)
one dentist and one dental nurse (14)
both dentists (1)
How often was the Dentomat filled?
both dental nurses (15)
one dentist and one dental nurse (15)
The within worker study
Kappa statistics for the same answers in the questionnaire and the interview.
Yes-no have ever used copper amalgam
Yes-no have ever used a mortar
Yes-no have ever used a Dentomat
Agreement within workers for the number of years they had used the three treatment methods, heated copper amalgam, a mortar and a Dentomat.
number of comparisons
Std of the mean difference
Number of years with copper amalgam
Number of years with use of a mortar
Number of years with use of a Dentomat
When we analyze the answers at the interview for the persons with missing information regarding the use of the different preparation methods in the questionnaire, we see that two out of seven who had missing information on the use of copper amalgam from the questionnaire answered at the interview that they had used this method. For the use of a mortar, the answer was positive for two out of six who had missing information in the questionnaire and for the use of a Dentomat the answer was positive for three out of six. This means that between 29 to 50% of the missing answers in the questionnaire may be "yes".
When looking at the answers to the questionnaires in the primary study group on the use of the three amalgam preparation methods it turned out that only 71.8% of those who had answered "yes" to the question which asked if they had used the copper amalgam method had completed the information on the topic in regard to both how many years they had used the method and how many patients they had treated each week with this method. The similar value for using a mortar was 61.8% and for using a Dentomat it was 52.6%. All in all, only 47.9% of the dental personnel from the primary study group had given all the answers in the questionnaire that were needed in order to calculate the relative exposure score. If we had only used the yes or no answers on the questionnaire for the 4 preparation methods (copper amalgam, mixing manually in mortar, make own capsules or use a Dentomat), the response rate from the questionnaire would have been 81.9%.
In our primary study, an effort was invested in an extensive job-specific questionnaire sent to the dental personnel in order to obtain as accurate information on the cumulative exposure to mercury as possible. However, such a detailed questionnaire turned out to be rather inefficient, as less than 50% of the respondents in the primary study group had answered all the questions that were necessary for the calculation of the exposure score. When studying the validity of the answers from the questionnaire compared to the answers in an interview regarded as the "truth", it became apparent that the agreement, the sensitivity, and the specificity varied for the different questions. In particular, the specificity of the questionnaire compared to the interview was somewhat low. But, using the interview as a "gold standard" may not be correct. We supposed however, that the answers given during the interview with the occupational physician would be more correct than the answers given in a mailed questionnaire. For the interview, the physicians had a guide with the questions that should be asked, but it is assumable that they explained the question and asked for more details when necessary. Both the physicians who did the interviews were engaged in finding the "true" exposure of the participants. We thus assume that the interview information was more like an expert assessment which has been used as a gold standard in other studies [6, 11].
In the period before this study was started, there had been some media attention on the possible health effects in dental nurses due to mercury exposure. Some recall bias with the possibilities of exaggeration of exposure among the dental nurses could be anticipated, while among the dentists, who had been the employers of the dental nurses, a recall bias in the other direction was possible. In this context we saw the possibility that the pairs of one dental nurse and one dentist would differ more in their judgement on the degree of mercury exposure than the pairs of two dental nurses. This was, however, not the case in this study. The greatest differences were in fact between pairs of two nurses at the same clinic. Thus, despite the media attention there does not seem to be any substantial differences in describing the methods used between dentists and dental nurses. There was, however, a marked difference between the pairs that had worked in the same clinic regarding the start and termination years for the different treatment methods, independent of the occupation. The mean difference between workers was, however, not different from the within worker mean difference on the one question that was possible to compare: the number of years with the use of a Dentomat. There was a marked difference in agreement between the dental nurses and the dentist in regard to the questions as to whether the amalgam was soft or firm. As it was the dental nurses in the clinics who filled the Dentomat and who prepared the amalgam for use before treatment, this was not very surprising. The dentists, however, were the ones who used the amalgam and were responsible for the conditions in the clinic. The agreement between two nurses in the same clinic was much better; 70% of the pairs of dental nurses agreed or partly agreed on these questions. In this between worker study, we chose for practical reasons to concentrate on responders from smaller cities and community centers. As we neither think that dental personnel from smaller cities or communities will have answered differently compared to dental personnel in bigger cities, nor that the preparation methods in use were systematically different between smaller and bigger communities we feel confident that the restrictions made did not lead to any decisive bias.
Regarding the agreement within workers, the question if the participants had ever used copper amalgam had a very good agreement based on the kappa statistics. In regard to the other methods, the agreement was moderate. The sensitivity of the questionnaire information, taking the answer at the interview to be true, was quite high for all preparation methods, while the specificity was somewhat lower. In our study, there were more false positive than false negative answers for all the three preparation methods when regarding the interview answers to be true. This is not quite in accordance with earlier studies that have reported the specificity to be higher than the sensitivity [11, 12] and that mailed questionnaires tend to under-report exposure . This is a particular concern in studies with a low prevalence of exposure where it is critical to avoid false positive exposure classification . For instance, the specificity was 0.60 for our question that asked if the participants had ever mixed amalgam manually in a mortar. As the prevalence of this particular exposure was also low, this is a source of misclassification that could lead to serious bias. The specificity for the question about the use of the Dentomat was also low, but for this question, the prevalence of exposure was high and thus the concern less pronounced. We chose to interpret that the low specificity for ever having mixed amalgam manually in a mortar may be explained by some misunderstanding and unclear wording in the questionnaire. The method, "mixing amalgam manually in a mortar", was meant from our side to be the method used in the fifties and the sixties where metallic mercury and metal powder were weighed separately and merged together manually in the mortar. We are afraid that some of the respondents may have misinterpreted the question in the questionnaire and have regarded it as relating to the small mortar used to handle the amalgam subsequent to the mixing in a Dentomat or a capsule. In the interview, the question for this preparation method was much more specific. This shows the necessity that the wording of questions is clear and without possibilities for misunderstanding. This is particularly the case for exposures that are less common in the study group. Missing information on details regarding particular questions was a problem in this study because many single questions from the questionnaire were used together to calculate a relative exposure score. This was difficult because one or more single questions were not answered by most of the respondents. The results showed that the more detailed the questions were, the greater proportion of the respondents gave incomplete answers.
In this study where we had the opportunity to compare a limited number of answers from a questionnaire with interview data, we are quite aware of the limitation both in regard to the number of questions and the number of respondents. Still, we find the results worth reporting, and we also hope that they contribute a little to sought-after knowledge about the possibilities and limitations in retrospective exposure assessment using a questionnaire.
The results of this study indicate that our mailed questionnaire caused misclassification of exposure. In this study the differences in describing the methods used was independent of occupational status but the prevalence of assumed false positive exposure classification was higher than false negative. This may result in serious bias if the prevalence of exposure is low. The necessity to be precise and correct when formulating the specific questions must be emphasized. Due to missing information detailed questionnaires may be inefficient if the goal is to construct exposure measures from combinations of many answers in the questionnaire.
- Kauppinen T: Assessment of exposure in occupational epidemiology. Scand J Work Environ Health 1994, 20(special issue):19–29.PubMedGoogle Scholar
- Flegal K, Brownie C, Haas JD: The effects of exposure misclassification on estimates of relative risk. Am J Epidemiol 1986, 123(4):736–751.PubMedGoogle Scholar
- Tielemans E, Heederik D, Burdof A, Vermeulen R, Vermeulen H, Kromhout H, Hartog K: Assessment of occupational exposures in a general population: comparison of different methods. Occup Environ Med 1999, 56: 45–51. 10.1136/oem.56.3.145View ArticleGoogle Scholar
- Holmes E, Garschick E: The reproducibility of the self-report of occupational exposure to asbestos and dust. J Occup Med 1991, 33(2):134–138.PubMedGoogle Scholar
- Hilt B, Langård S, Andersen A, Rosenberg J: Asbestos exposure, smoking habits, and cancer incidence among production and maintenance workers in an electrochemical plant. Am J Ind Med 1985, 8(6):565–77. 10.1002/ajim.4700080608View ArticlePubMedGoogle Scholar
- Ahlborg GA: Validity of exposure data obtained by questionnaire. Two examples from occupational reproductive studies. Scand J Work Environ Health 1990, 16: 284–288.View ArticlePubMedGoogle Scholar
- Teschke K, Olshan AF, Daniels JL, De Roos AJ, Parks CG, Schulz M, Vaughan TL, Kromhout H: Occupational exposure assessment in case-control studies: opportunities for improvement. Occup Environ Med 2002, 59: 575–594. 10.1136/oem.59.9.575PubMed CentralView ArticlePubMedGoogle Scholar
- Svendsen K, Syversen T, Melø I, Hilt B: Historical exposure to mercury in Norwegian dental personnel. Scand J Work Environ Health 2010, 36(3):231–41.View ArticlePubMedGoogle Scholar
- Hilt B, Svendsen K, Syversen T, Aas O, Qvenild T, Sletvold H, Melø I: Occurrence of cognitive symptoms in dental assistants with previous occupational exposure to metallic mercury. NeuroToxicology 2009, 30: 1202–6. 10.1016/j.neuro.2009.04.012View ArticlePubMedGoogle Scholar
- Altman DG: Practical Statistics for Medical Research. Chapman & Hall, London UK; 1991.Google Scholar
- Fritchi L, Siemiatycki J, Richardson L: Self-assessed versus expert-assessed occupational exposures. Am J Epidemiol 1996, 144: 521–527.View ArticleGoogle Scholar
- Hobson AJ, Sterling DA, Emo B, Evanoff BA, Sterling CS, Good L, Seixas N, Checkoway H, Racette BA: Validity and reliability of an occupational exposure questionnaire for parkinsonism in welders. J Occup Environ Hyg 2009, 6(6):324–31. 10.1080/15459620902836856PubMed CentralView ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.