This Article Abstract Full Text (PDF) All Versions of this Article: 1055-9965.EPI-07-2622v1 17/4/785    most recent Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Beebe, T. J. Articles by Rockwood, T. H. Search for Related Content PubMed PubMed Citation Articles by Beebe, T. J. Articles by Rockwood, T. H. Related Collections Early Detection Early Detection: Biomarkers: Discovery, Methodology, Validation

The Effects of Survey Mode and Asking About Future Intentions on Self-Reports of Colorectal Cancer Screening

¹ Department of Health Sciences Research, Survey Research Center, Mayo Clinic College of Medicine, Rochester, Minnesota and ² Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, Minnesota

Requests for reprints: Timothy J. Beebe, Associate Professor of Health Services Research, Mayo Clinic College of Medicine, 200 First Street Southwest, Rochester, MN 55905. Phone: 507-538-4606; Fax: 507-284-1180. E-mail: beebe.timothy{at}mayo.edu

	Abstract

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 
Colorectal cancer (CRC) screening rates are often ascertained via self-reports but can be subject to overreporting bias. Asking about intention to get screened before asking about past screening may minimize overreporting of cancer screening. In a statewide survey conducted from July through October of 2005, we embedded an experiment that tested the effect of question ordering (asking about future intention to get screened before or after asking about past screening; "future first" and "future second," respectively), crossed with survey mode (mail versus telephone), on CRC screening rates. Weighted analysis focused on 752 respondents who were ages 50 years or older. We found (a) that asking about future intentions to get screened before asking about past screening (future first) statistically significantly lowers reports of past CRC screening [70.9% future second versus 58.0% future first; odds ratio (OR), 1.83; 95% confidence interval (95% CI), 1.08-3.13]; (b) that there was no main effect of survey mode; and (c) that the effect of the ordering of the future intentions item varies by survey mode. In the mailed survey, the odds of reporting past CRC screening were almost thrice greater in the future second condition compared with the future first condition (72.4% versus 49.0%, respectively; OR, 2.74; 95% CI, 1.22-6.17). In the telephone condition, the odds of reporting were only 28% higher in the future second (69.5%) condition than in the future first condition (63.9%; OR, 1.28; 95% CI, 0.64-2.57). The results suggest that asking about future intentions to get screened before the actual behavior elicits lower, and arguably more truthful reports of CRC screening but mainly in mailed surveys. (Cancer Epidemiol Biomarkers Prev 2008;17(4):785–90)

	Introduction

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 
Population-based colorectal cancer (CRC) screening levels are often ascertained via self-report, as this approach represents a cost-effective alternative to more effort-intensive methods such as medical record review (1). However, self-reports of cancer screening—including certain types of CRC screening tests such as fecal occult blood test and sigmoidoscopy—have been found to be prone to overreporting bias (2-4), probably due to the desire of some to cast themselves in a socially desirable light coupled with the fact that cancer screening is a socially desirable health promotion behavior. Recent reviews of the literature on the accuracy of self-reported health care utilization in general (5) and of cancer-screening behavior in particular (3) have implicated questionnaire design features such as item wording and question ordering, as well as mode of data collection (e.g., mail or telephone), as factors affecting report veracity. The focus of our study was to test the unique and interactive effects of these two factors on self-reported CRC screening behavior.

There is ample evidence that survey respondents offer socially desirable responses to questions asking about socially proscribed behaviors (e.g., illicit drug use) more often in a telephone survey context than when asked to complete similar questions in a self-administered, mailed survey (6-9). However, very little information exists on how telephone and mail-survey respondents differ in their reporting of socially desirable behaviors such as cancer screening. The information that exists suggests a certain level of telephone and mail-survey mode equivalence in cancer screening reporting accuracy (10, 11), but the sheer paucity of evidence renders judgment on the issue far from final. Moreover, it is not clear whether the forces driving overreports of socially desirable behaviors are different from those prompting the underreporting of socially undesirable behaviors (12). As such, further investigation of the effect of telephone versus mail data collection modes seems warranted.

Many questionnaire design features and their effects on self-reports of cancer screening could also be investigated. We chose to focus on how screening items are structured as suggested by Bhandari and Wagner (5) and extend the recent work in this area by Johnson and colleagues (12). These latter authors found evidence suggesting that asking about future intentions to get screened for cancer before the actual question about past screening—Pap smears, mammograms, and clinical-gynecologic examinations in this case—increased the accuracy of self-reports of past screening when compared with medical records. The authors posited that by asking respondents if they plan on engaging in a future socially desirable activity before the actual past behavior, they will be under less social pressure to overreport their past practice of that behavior. Because several of their findings did not reach statistical significance in multivariate models, the authors characterized their results as suggestive and call for further methodologic work in this area.

In addition, Vernon and colleagues (3) have observed few methodologic studies of questionnaire design features in the cancer-screening literature. It is in this context that we set out to test the effect of asking about future intention to get screened for CRC (either before or after the past CRC screening behavior question) crossed with survey mode (mail versus telephone). We are aware of no other study that has tested the effect of these two factors simultaneously.

	Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 
Study Population
The population for this survey included noninstitutionalized Minnesota residents who were 18 y and older. The sampling frame used a list-appended random digit dial sample purchased from Genesys Marketing Systems Group, whereby postal addresses were appended to the random digit dial telephone numbers if they were found in listed directories. The overall sample was stratified by region (Twin Cities —Minneapolis/St. Paul— metropolitan, non-Twin Cities metropolitan, and rural) and residents of Olmsted County, and those ages 50 y were intentionally oversampled through the use of geographic and age information on telephone numbers available from Genesys Marketing Systems Group. All data were weighted so that the sample reflected the sex, age, geographic, and racial/ethnic distribution of adults in the state.

Data Collection
From July 15 through October 25 2005, the University of Minnesota Center for Survey Research in Public Health conducted a mixed-mode mail or telephone survey of 4,210 adults as part of their Omnibus Survey series. The Center for Survey Research in Public Health Omnibus Survey is fielded quarterly and allows research institutions, government agencies, and nonprofit organizations to each ask a few questions as part of a larger survey. The questionnaire contained 40 items measuring such health-related issues as general health, health insurance coverage, use of online health information, testing for colon cancer, and oral health. It also measured demographics and attitudes toward tattoos, drug use, and surveys. The current investigation focuses only on the items relating to colon cancer and screening and the data come from an experiment embedded in the larger survey that was conducted for other purposes.

Telephone numbers with an appended address were randomly assigned to receive either a mailed survey or a telephone survey. Participants without appended addresses (i.e., households with unlisted telephone numbers) were assigned to a telephone survey sample. A total of 1,492 and 2,718 participants were assigned to the mail and telephone modes, respectively; 70% of those assigned to the telephone mode had an appended address. Once participants were randomly assigned to data collection mode, they were then assigned to one of two question wording conditions within mode where we altered the position of the future intention to get screened for CRC (either before or after the CRC screening item; hereafter called "future first" and "future second," respectively). The actual wording for the items in each condition is provided in the Appendix.

A multiple-contact data-collection protocol was deployed for both mail and telephone conditions. For those in the mail-survey mode, the initial mailing consisted of a cover letter, survey, and a business reply envelope. A reminder postcard was mailed 10 d after the initial mailing. A third mailing that included a cover letter, another questionnaire, and another business reply envelope was sent to survey nonrespondents 2 wk after the mailing of the postcard reminder. A total of 9 participants in the mail mode were found to be ineligible due to being age <18 y. The response proportion was calculated as the number of completions divided by the number of eligible participants using the response rate calculation formula set forth by the American Association for Public Opinion Research.³ A total of 741 mailed surveys were received, for a response proportion of 50% (741 of 1,483). For participants randomly or necessarily assigned to the telephone sampling frame, the survey instrument used Computer-Assisted Telephone Interview. Calls to attempt an interview were made by trained Center for Survey Research in Public Health interviewers at all times of the day (morning, afternoon, and evening) and days of the week (weekdays and weekends). Telephone numbers were attempted up to 20 times per case and messages left on answering machines to increase participation.

A total of 893 participants were found to be ineligible, mainly due to having nonworking telephone lines, and 895 telephone interviews were completed, for an American Association for Public Opinion Research response rate calculation formula response proportion of 49% (895 of 1,825). The institutional review boards at both the University of Minnesota and Mayo Clinic approved the consent and study procedures.

We focus here on the 759 respondents ages 50 y or more because extant guidelines recommend routine CRC screening in this population subgroup (13); seven participants did not respond to the CRC screening item. A total of 752 participants were available for analysis with 172, 215, 167, and 198 in the mail/future first, telephone/future first, mail/future second, and telephone/future second conditions, respectively.

Statistical Analysis
The analyses were framed by three working hypotheses. First, we hypothesized that lower, and arguably more truthful, reporting of past CRC screening would be observed in the condition where the screening behavior item follows the future intention item (future first). Second, we hypothesized that CRC screening rates would be lower in the mailed version of the questionnaire than in the telephone version. Third, we hypothesized that CRC screening rates would be influenced by an interaction between question ordering and mode of data collection. Specifically, we hypothesized that the effect of asking the future intentions item before past screening would be greatest in the telephone condition where question order is believed to be most salient. There is evidence in the survey research methods literature demonstrating that order effects are more prominent in surveys administered via interview (e.g., in-person and telephone) than in self-administered surveys, largely due to the serial administration of items in the former (14-16).

To compare the respondents in the mail and telephone conditions, we analyzed their sociodemographic characteristics by mode. This was done to assess selection into mode after random assignment—there is some evidence of variation in mode preferences and that different populations may respond to different modes (17-19)—and to identify possible confounders that might have needed to be controlled for in the primary analyses. To assess differences across modes, we used weighted linear regression with mode as the independent variable to compare age (the dependent variable); the Rao-Scott ² test (weighted) to compare sex, race, educational attainment, employment status, marital status, and health status.

Logistic regression analysis was used to determine whether or not CRC screening rates varied by question ordering and/or mode of data collection (both adjusting and not adjusting for covariates). In the adjusted analyses, we included, not only the demographic variables that were found to vary across mail and telephone conditions but also a variable that identified whether a telephone number was found in a listed directory (hereafter called telephone mode listed status) because only those with appended addresses could be assigned to the mail condition. Doing so was important because white-pages frames exclude those with unlisted telephone numbers. Guterbock and colleagues (20) found that those with unlisted numbers were more likely to be African American and younger in age; a finding confirmed by Lepkowski and colleagues (21). However, Lepkowski et al. (21) also found that listed status was not related to any of their substantive economic measures, and Smith and colleagues (22) found that listed status was not related to estimates of HIV-related risk behavior. Nonetheless, we include this variable in the analysis, although use of a listed frame may not alter the substantive findings of our study.

All P values are two sided and a P value of 0.05 was regarded as statistically significant. All reported percentages, means, and analyses are weighted and were done using SAS v. 9.1 software (SAS Institute, Inc.) using SAS survey procedures.

	Results

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 
Table 1 provides the sociodemographic characteristics of respondents in each data collection mode. Sex (P = 0.04), education (P = 0.03), and health status (P = 0.01) all varied significantly by data collection mode where mail-survey respondents were less likely to be female, and more likely to be of lower educational status and less healthy than those responding to the telephone survey. As a result, these three variables were entered as covariates—along with telephone mode listed status—into the adjusted analyses referenced below. No statistically significant differences were observed for age, race, employment status, and marital status—these groups were equivalent across data-collection modes.

	Discussion

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

Contrary to expectations, we found no support for our second hypothesis that CRC screening rates would be lower in the mailed version of the questionnaire than in the telephone version, in either unadjusted analyses or those that adjusted for the possible effects of respondent selection into mode. Our findings are consistent with the scant prior research focusing on self-reports of cancer screening (10, 11) but are inconsistent with what might be expected from the literature on mode effects and socially undesirable behavior such as illicit drug use (6-9). It may be that the forces driving overreports of socially desirable behaviors are different from those prompting the underreporting of socially undesirable behaviors as suggested by Johnson et al. (12).

We had also hypothesized that asking about future intentions before past CRC screening would have its greatest effect in the telephone version where items are asked in a linear and sequential manner. However, we found that our item ordering affected only screening self-reports in the mailed version of the survey where the estimated odds of reporting past CRC screening were close to three times greater when asked about future intentions after the screening item. It may be that respondents do proceed through self-administered forms in a linear manner and/or that question ordering is not as salient in telephone surveys vis-à-vis mail as conjectured by past researchers (14-16).

Our findings in this area might also be attributable to an increased definitional clarity brought about by the manner in which respondents were exposed to the information in the mail-mode condition. Examining the wording of questions in the appendix, it is clear that respondents in the mail mode were afforded the luxury of seeing what might be included in the phrasing "...tested for colon cancer..." asked as part of the question series by looking just beneath or above it to see a listing of different testing methods (e.g., fecal occult test, sigmoidoscopy, etc.). Having access to this information—information that was not at all available to telephone respondents until they had already been asked and then answered this question—could have clarified what was considered as being "tested." This supposition would be consistent with the results of cognitive interview studies that CRC test descriptions help clarify what is included under the general rubric of CRC screening (23), as well as research by Baier and colleagues (24) demonstrating that self-reported CRC screening can be quite accurate if the items are carefully phrased and accompanied by test descriptions.

Therefore, it may be that asking about future intentions of a behavior will have an effect on reports of past behavior only in situations of definitional clarity about the target behavior (i.e., the respondent knows exactly what he or she is being asked about) rather than due to a specific survey mode. If this is indeed true, the current results suggest that inclusion of a future intention item in surveys that have specific descriptions of the screening tests, such as the measures developed by Vernon and colleagues (23) or the screening items used in the Health Information National Trends Survey,⁴ increase the accuracy of self-reported colon cancer screening behavior. However, this may be the case only in situations where the tests in question are unfamiliar (newly introduced) to the respondent, or when there are multiple tests being considered.

In considering the above findings, it is important to note some potentially important limitations. First, the reader must be mindful that, whereas past investigators (4, 11, 12) focused on consistency between self-reports and medical records as the primary measure of accuracy, we looked only at the former in the current study. The early research literature in the area of social desirability has typically pointed to the finding that people tend to overreport socially desirable behaviors such as exercise and underreport socially undesirable behaviors such as substance use (25-27). In this context, higher reports of socially desirable behaviors are usually assumed to reflect less honest self-disclosure (28). Nonetheless, dependence on self-reports as a primary measure of accuracy represents a limitation of the current investigation.

Future research should attempt to replicate our results and incorporate comparisons of self-reports to some external "gold standard," as was done by Johnson et al. (12). Second, our study used items that did not include complete descriptions of the screening tests, contrary to the recommendations of some (23, 24). The fact that our experiment was embedded in a larger study conducted for reasons unrelated to the present experiment limited our ability to do so. As mentioned above, the absence of such descriptions in our question asking may have confounded our results somewhat and, thus, limited the inferential value of our findings.

In conclusion, health researchers and policy makers rely on self-reports of screening behavior. This study has shown that the quality of self-reported CRC screening is affected by the structure and order of the screening items and by the interaction of question ordering and mode of data collection, to a certain extent. Specifically, the findings suggest that asking about future intentions to get screened before asking about past CRC screening elicits lower, and arguably more truthful reports of CRC screening but mainly in mailed surveys. The results also underscore the importance of responding to the calls of many in the field to undertake methodologic studies of factors that affect the accuracy of self-reported cancer-screening behavior (3). We encourage others to continue this line of inquiry by incorporating more defensible measures of report accuracy, using agreed-upon measures of CRC screening behavior, and extending the topic to other types of cancer screening and health-related behaviors.

	Appendix A: Future Intentions Before Screening Item ("Future First")

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 
Q5 Are you planning on being tested for colon cancer in the next 12 months?

1. Yes
   
2. No
   
3. I just had one
   

Q6 Have you ever been tested for colon cancer?

1. Yes
   
2. No Go to page 3 question 8
   

Q7 Doctors use several different methods to test for colon cancer. Which of the following tests have you had?

a. A stool blood test also known as a fecal occult blood test

b. A Sigmoidoscopy

c. A Colonoscopy

d. A Barium Enema

e. A CT Colonoscopy or Virtual Colonoscopy

f. Any other test (Specify):_________________________

	Appendix B: Future Intentions After Screening Item ("Future Second")

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 
Q5 Have you ever been tested for colon cancer?

1. Yes
   
2. No Go to question 7
   

Q6 Doctors use several different methods to test for colon cancer. Which of the following tests have you had?

a. A stool blood test also known as a fecal occult blood test

b. A Sigmoidoscopy

c. A Colonoscopy

d. A Barium Enema

e. A CT Colonoscopy or Virtual Colonoscopy

f. Any other test (Specify):_________________________

Q7 Are you planning on being tested for colon cancer in the next 12 months?

1. Yes
   
2. No
   
3. I just had one
   

	Footnotes

 
Grant support: Mayo Clinic Foundation for Education and Research.

³ http://www.aapor.org

⁴ http://hints.cancer.gov/

Received 9/21/07; revised 1/31/08; accepted 2/ 4/08.

	References

Top Abstract Introduction Materials and Methods Results Discussion Appendix A: Future Intentions... Appendix B: Future Intentions... References

 

1. Lazovich D, Casey-Paal A, Korn JE, Slater JS. Medical records as an alternative to self-report for measuring mammography utilization. J Public Health Manag Pract 1999;5:63–7.[Medline]
2. Madlensky L, McLaughlin JR, Goel V. A comparison of self-reported colorectal cancer screening with medical records. Cancer Epidemiol Biomarkers Prev 2003;12:656–9.[Abstract/Free Full Text]
3. Vernon SW, Briss PA, Tiro JA, Warnecke RB. Some methodologic lessons learned from cancer screening research. Cancer 2004;101:1131–45.[CrossRef][Medline]
4. Warnecke RB, Sudman S, Johnson TP, O'Rourke DP, Davis AM, Jobe JB. Cognitive aspects of recalling and reporting health-related events: Papanicolaou smears, clinical breast examinations, and mammograms. Am J Epidemiol 1997;146:982–92.[Abstract/Free Full Text]
5. Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev 2006;63:217–35.[Abstract/Free Full Text]
6. Beebe TJ, McRae JA, Harrison PA, Davern ME, Quinlan KA. Mail surveys result in more reports of substance use. J Clin Epidemiol 2005;58:421–4.[Medline]
7. Fournier L, Kovess V. A comparison of mail and telephone interview strategies for mental health surveys. Can J Psychiatry 1993;38:525–33.[Medline]
8. Gmel G. The effect of mode of data collection and of non-response on reported alcohol consumption: a split sample study in Switzerland. Addiction 2000;95:123–34.[CrossRef][Medline]
9. Hochstim JR. A critical comparison of three strategies of collecting data from households. J Am Stat Assoc 1967;62:976–89.[CrossRef]
10. Zapka JG, Bigelow C, Hurley T, et al. Mammography use among sociodemographically diverse women: the accuracy of self-report. Am J Public Health 1996;86:1016–21.[Abstract/Free Full Text]
11. Vernon SW, Tiro JA, Vojvodic RW, et al. Reliability and validity of a questionnaire to measure colorectal cancer screening behaviors: Does mode of survey administration matter? [under review].
12. Johnson TP, O'Rourke DP, Burris JE, Warnecke RB. An investigation of the effects of social desirability on the validity of self-reports of cancer screening behaviors. Med Care 2005;43:565–73.[CrossRef][Medline]
13. Winawer S, Fletcher R, Rex D, et al. Colorectal cancer screening and surveillance: clinical guidelines and rationale - update based on new evidence. Gastroenterology 2003;124:544–60.[CrossRef][Medline]
14. Dillman D, Sangster R, Tarnai J, Rockwood T. Understanding differences in people's answers to telephone and mail surveys. New Directions for Evaluation: Advances in Survey Research 1996;70:45–62.
15. Schwarz N, Hippler, HJ. Subsequent questions may influence answers to preceding questions in mail surveys. Public Opin Q 1996;59:93–7.
16. Schwarz N, Strack F, Mai HP. Assimilation and contrast effects in part-whole question sequences: a conversational logic analysis. Public Opin Q 1991;55:2–23.
17. Siemiatycki J. A comparison of mail, telephone and home interview strategies for household health surveys. Am J Public Health 1979;9:238–45.
18. Brambilla DJ, McKinlay SM. A comparison of responses to mailed questionnaires and telephone interview in a mixed mode health survey. Am J Epidemiol 1987;126:962–71.[Abstract/Free Full Text]
19. Link MW, Mokdad AH. Alternative modes for health surveillance surveys: an experiment with web, mail, and telephone. Epidemiology 2005;16:701–4.[CrossRef][Medline]
20. Guterbock TM, Hartman DE, Hubbard RA. RDD vs. Listed: An experimental study of coverage error, costs and non-response in a statewide telephone survey. Paper presented at the Annual Meeting of the American Association for Public Opinion Research, Nashville, TN, May 2003.
21. Lepkowski J, Schuman H, Wang R, Curtin R. Listed numbers and do not call registration in a national telephone Survey paper presented at the annual meeting of the American Association For Public Opinion Association, Fontainebleau Resort, Miami Beach, FL, 2006.
22. Smith AMA, De Visser DO, Rissel CE. Random digit dialing and directory-based samples in telephone surveys of HIV risk: a comparison from the Australian Study of Health and Relationships. Ann Epidemiol 2005;15:232–5.[Medline]
23. Vernon SW, Meissner HI, Klabunde CN et al. Measures for ascertaining use of colorectal cancer screening in behavioral, health services, and epidemiologic research. Cancer Epidemiol Biomarkers Prev 2004;13:898–905.[Free Full Text]
24. Baier M, Calonge N, Cutter G, et al. Validity of self-reported colorectal cancer screening behavior. Cancer Epidemiol Biomarkers Prev 2000;9:229–32.[Abstract/Free Full Text]
25. Hyman H. Do they tell the truth? Public Opin Q 1944;8:557–9.[Abstract]
26. Parry H., Crossley H. Validity of response in survey questions. Public Opin Q 1950;14:61–80.[Abstract]
27. Phillips DL, Clancy KJ. Response biases in field studies of mental illness. Am Sociol Rev 1970;35:503–15.[CrossRef][Medline]
28. Bradburn NM, Sudman S, and Associates. Improving interview method and questionnaire design.San Francisco (CA): Jossey-Bass; 1979.

This article has been cited by other articles:

				J. G. Zapka Validation of Colorectal Cancer Screening Behaviors Cancer Epidemiol. Biomarkers Prev., April 1, 2008; 17(4): 745 - 747. [Full Text] [PDF]

This Article Abstract Full Text (PDF) All Versions of this Article: 1055-9965.EPI-07-2622v1 17/4/785    most recent Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Beebe, T. J. Articles by Rockwood, T. H. Search for Related Content PubMed PubMed Citation Articles by Beebe, T. J. Articles by Rockwood, T. H. Related Collections Early Detection Early Detection: Biomarkers: Discovery, Methodology, Validation