Questionnaire Items to Assess Skin Color and Erythemal Sensitivity: Reliability, Validity, and “the Dark Shift”

The link between skin pigmentation and health outcomes related to UV radiation (UVR) exposure is well established, with populations of light skin color at greater risk of skin cancers and those of dark skin color at greater risk of diseases related to vitamin D deficiency (ref. 1; Fig. 1).

For cancer surveillance and prevention activities and defensible epidemiologic extrapolations related to UVR exposure, a valid and reliable assessment of skin color is required. Such information contributes to the development and appropriate targeting of health promotion programs based on evidence about the direction and relative strength of associations between erythemal sensitivity and a range of other potential risk and protective factors included in survey data. These programs are aimed at reducing the population risk of diseases related to solar UVR exposure, where it is either (a) excessive (e.g., skin cancers and eye diseases) or (b) insufficient (e.g., osteoporosis and osteomalacia; ref. 1).

For population-based studies of large, randomly selected samples, often neither skin biopsy nor spectrophotometry is a practical option. In Australia and New Zealand (NZ), two countries with among the highest cutaneous malignant melanoma incidence and mortality rates (2), data on skin color and the propensity to tan have been obtained for the adult populations using the same self-report items in regular, summertime Computer-Assisted Telephone Interview surveys—since 1988 in the state of Victoria, from 2003 in Australian national surveys (3), and since 1994 in NZ (4).

However, published information about the reliability and validity of these and similar measures, largely derived from the Fitzpatrick skin classification system (5), is scarce for all populations, particularly non-Caucasian, and lacking for NZ. The authors of two studies concluded that self-reported burning/tanning histories formed an unreliable means of skin typing when compared with minimal erythemal dose estimates (6, 7). Another study found poor agreement between perceived and measured skin color among Caucasians, with many overestimating their skin pigmentation (8). However, a recent U.S. study concluded that self-reported skin color is a valuable measurement strategy after validation against a spectrophotometer (9).

Given this context, the main objectives of the present research were to assess the (a) validity and (b) test-retest reliability of two self-report measures (skin color and erythemal sensitivity) commonly used in population sun exposure survey questionnaires. More specific information on the validity and reliability of these measures would enhance the value of existing survey data and increase its potential usefulness for epidemiologic studies and guiding health promotion programs. Related project objectives were to report the extent and direction of any biases observed arising from these measures.

Because a key sampling requirement was that as full a range of skin types as possible should be represented, volunteers were recruited from the population of students domiciled in University of Otago residential colleges. This student population was also relatively easy to access and follow up, allowing an efficient process for obtaining a sample of sufficient size. Toward the end of 2007, well in advance of student recruitment scheduled for early 2008, Heads/Masters of the 14 residential colleges were invited to permit us to carry out the study on site.

As an incentive to return for the follow-up assessment, each participant was offered $20 cash, to be paid after completion of their second assessment, thereby also acknowledging their contribution and compensating for any inconvenience. Each participant was allocated a number that was entered into a random draw for an additional $200 prize. All participants, whether or not they took part in a follow-up assessment, were offered feedback in the form of (a) a brief printed personal report about their skin type and (b) current guidelines about recommended UVR exposure under NZ conditions (10).

Sample size calculations were based on the resulting precision of κ statistics assuming two categories. For test-retest reliability, 280 participants would allow a two-category κ to be estimated to within ±0.15 using 95% confidence intervals assuming that κ was 0.6 or higher and where the smaller category contained at least 15% of the respondents, and allowing for up to 30% loss due to invalid responses and loss to follow-up. Cluster effects were assumed to be negligible for this study.

All participants were treated following a standard protocol. Procedures were pretested for practicality and participant acceptability and the final protocol was incorporated into a printed manual for assessors to use during the project. Data collection occurred between April 29 and May 21 2008 during evening sessions using pretrained, senior university students as research assistants. Participants' skin types were assessed using two standard self-report, population questionnaire items on two separate occasions to allow analysis of test-retest reliability. These two occasions were separated by a period of 7 days to minimize any recall effects from the first occasion and the likelihood of any changes in the participants' reaction to sun exposure during the between-test period. Spectrophotometer (SP) readings were taken only during the first assessment. Figure 2 presents a flow diagram of participant movement through study assessments. Ethical approval was obtained (March 10, 2007) through the Department of Preventive & Social Medicine, following the University of Otago protocols.

Reflectance of the skin was measured using a DATACHECK SP, the study criterion measure. The instrument was used to compare the color of each participant's skin with the color of a standard calibration reference. It works by emitting a flash of white light to the surface under test and measuring the spectral properties of the light reflected back from the surface to the instrument. These were converted to individual typography angle (ITA) scores for comparison to the self-report skin color and photosensitivity categories. Low ITA scores indicate dark skin color. The ITA is calculated using the values of L* (the difference along the lightness-darkness axis) and b* (the difference along the yellow-blue axis). It was calculated according to Del Bino (11) as:

Skin classification by ITA values has physiologic relevance to melanin content (11). The six different skin classifications defined by Del Bino according to ITA are presented in Table 1, along with corresponding classifications for our self-report questionnaire. These classifications were used for comparison with individual perceived skin color. Such methods of comparison have been used in previous research (12).

The SP was held against the skin of the upper inner arm of each participant. At this recommended site (13), the natural (i.e., untanned) skin color can most readily be measured because it is a site that is not usually exposed to sunlight (11) and is relatively noninvasive. The surfaces of the instrument that came into contact with the participants' skin were cleaned with alcohol wipes between assessments. Three replicate skin reflectance measurements were done on each subject and then averaged to provide a single ITA value as a quantitative measure of skin color.

Self-reported information was obtained on ethnicity (multiple responses were obtained using the nine high-level NZ Census categories), sex, and age in years. The first questionnaire item on skin photosensitivity was “How would you describe your natural, untanned skin color at the end of winter? (Please circle only one number).” Seven response options were provided, ranging from “very fair” through “fair,” “medium,” “olive,” “dark” and “very dark” to “black.” The second question was “If you went out in the sun without any protection, in summer for 30 minutes during the middle of the day, would you? (Please circle only one number).” Three response options were provided, ranging from “just burn and not tan afterwards,” through “burn first, and then tan afterwards” to “not burn at all, just tan.” These two questions are those used in the Australian National Sun Survey (3) and the NZ Triennial Sun Survey series. The final question sought a dichotomous (yes/no) response: “In the past week (7 days) have you used a sun-bed or spray-on tan lotion? Please circle only one number.” Participants who provided a positive response to this question were excluded from the analyses as there was the potential to bias SP skin readings.

To examine questionnaire item validity, Spearman's correlation coefficients were calculated between each of the two items and objective skin color (as a continuous variable expressed as an ITA value). Spearman's correlation coefficients were compared between subgroups using bootstrapping with 10,000 samples drawn with replacement. To assess the test-retest reliability of the self-report measures, we calculated weighted κ's comparing responses at each of the research sessions. A box and whisker plot was constructed to show the distributions of objective skin color classifications for each of the self-reported skin color groups. To assess possible self-reporting bias, the percentage of participants for whom ITA values fell either above or below the Del Bino cutoff points was calculated for each self-reported skin color group. Stata 10.1 was used for all analyses (Stata; ref. 14).

Once the Heads/Masters of 6 of the 14 residential colleges agreed to permit resident students to participate, the involvement of additional colleges was not sought because a sufficient number of students (n = 289) was recruited. The frequency distributions of these students and SP-derived skin ITA scores are presented according to the questionnaire categories in Table 2. Participants had a median age of 18 years (range, 16-49).

Of the 289 who took part in the first assessment, 9 were excluded for using a sun-bed or spray tan within the past 7 days, leaving data from 280 participants for analysis. However, full data to assess validity were lacking for two participants, leaving 278 for those analyses. At the second assessment, 265 were followed up, representing a 95% follow-up, but for skin color and erythemal sensitivity, full intrarater reliability data were only available for 264 and 263, respectively.

Data collected in the first research session (Table 1) were used to test the validity of the questionnaire items. The distribution of the actual measurements of reflectance, expressed as ITA values, was negatively skewed for the whole group, with a median value of 46.2 (interquartile range, 36.2-53.9).

After exclusion of the 9 participants who had used a sun-bed or spray-on tan in the past 7 days, statistically significant negative correlations were found for self-reported skin color (n = 278, Spearman's ρ = −0.75, P < 0.001) and skin photosensitivity (n = 278, Spearman's ρ = −0.64, P < 0.001) with ITA scores, indicating that these questionnaire items were correlated with an objective measure of skin color.

There was no evidence of any significant difference in the strength of the correlation between perceived and actual skin color for males and females (Spearman's ρ = −0.76 versus −0.75, respectively; bootstrap P = 0.855). There was also no evidence of a difference between the sexes for correlations between self-reported skin photosensitivity and ITA scores (males, −0.57; females, −0.65; bootstrap P = 0.409).

However, there was a stronger correlation between self-reported skin color and ITA value among those of European than those of non-European ethnicity (Spearman's ρ = −0.78 versus −0.59, bootstrap P = 0.007). A similar pattern was noted for self-reported skin photosensitivity, with stronger correlations for those of European ethnicity (Spearman's ρ = −0.63 versus −0.28, bootstrap P = 0.001).

The distribution of ITA values for each self-reported skin color group is presented in Fig. 3. For those who classified themselves as having fair skin, 36% fell into the range for very fair skin and, therefore, overestimated their degree of skin pigmentation. The percentage of participants who overestimated their skin pigmentation increases to 77% for those who rated their skin as being of medium color, with the majority of these falling into the fair category when objectively classified. Participants who self-reported their skin color as being olive were also strongly biased toward overestimation, with 71% falling into lighter categories when objectively classified. Of those who rated themselves as having dark skin, 58% fell into the lighter categories when objectively classified. Although the number of participants self identifying as having either very dark or black skin was low (n = 7), 71% of these had lighter skin color than they perceived.

Data from the 265 participants who attended both research sessions were used in the analysis of intrarater reliability of questionnaire items. The intrarater reliability of self-reported skin color and photosensitivity items was determined by weighted κ. For skin color, there was 95.5% agreement (versus expected agreement of 78.9%) between assessments (n = 264, k = 0.78, P < 0.001). For skin photosensitivity, there was 91.6% agreement (versus expected agreement of 63.0%) between assessments (n = 263, k = 0.77, P < 0.001). Both values are toward the upper end of the range described as either “substantial” (15) or “good” (16).

The first objective of this research was to assess the validity of two self-report measures commonly used in population sun exposure surveys. For the first, perceptions of nontanned skin color at the end of winter, the questionnaire measure correlated highly with the Del Bino SP-derived ITA value categories (11). This finding is consistent with the results reported for similar studies (9), confirming that the measure could be considered an acceptable and appropriate alternative for use in large population surveys where obtaining individual SP readings is unlikely to be a practical option. We found no sex difference in the strength of the correlation, so the measure is similarly appropriate for assessing skin color among both sexes. However, there was a difference between those of European and those of non-European ethnicities, with stronger correlations found between self-reported color and objective color among Europeans than non-Europeans. The numbers of participants in this study drawn from some ethnic groups were insufficient to identify specific ethnicity effects. However, a U.S. study designed to compare self-reported Fitzpatrick skin types with objective colorimeter measures among participants in five specified ethno-racial groups (Pacific Islanders, Black individuals, Asians, Hispanic, and non-Hispanic white individuals) identified substantial associations for each group except Asians (17).

When we further examined our skin color category findings, a consistent pattern emerged of a tendency to overestimate skin pigmentation—a pattern which we describe as “the dark shift.” This finding is consistent with the findings of another study, which used broadly comparable descriptors, where 67.3% of Caucasians self-identifying as “olive” were measured as “fair” and 12.2% “medium,” whereas in the “medium” and “fair” groups, 36.4% overestimated their skin pigmentation (8). In our study, we also found some evidence that this tendency extended to the darker skin types, although the numbers involved were rather small. A contrary, but much weaker, tendency to underestimate skin pigmentation was mainly observed among the self-reported “very fair” group (40%), with only 5% each among the “medium” and “olive” groups and none in the darker groups. These findings raise a number of issues that are of research interest and public health concern.

First, the dark shift phenomenon may be critical in relation to the effective communication of public health messages appropriate for specific skin color groups. If respondents perceive that their natural, nontanned skin color is darker than it actually is, then they may also see appropriate skin cancer risk messages as less relevant to themselves and, consequently, place themselves at greater potential risk of receiving excess UVR exposure. It has been suggested elsewhere that inaccurate perceptions of personal skin color may reduce the ability of skin cancer prevention messages to reach a significant proportion of the target audience (8). Daniel et al. (9) found a minimal correlation between UV photographs, which objectively depict skin damage and self-reported skin damage, suggesting that people also incorrectly estimate the degree of UVR damage to their skin. This pattern of perceptions may help increase our understanding of the factors that influence inadequate sun-protective practices among at-risk groups. We have previously reported an association between perceived risk of developing skin cancer and sun-protective behaviors among a sample of outdoor workers (18). A recently published study found increases in melanoma among the Maori and Pacific populations of NZ (19), and although we know that there is a range of skin types among Maori (20), further research is required into whether the dark shift phenomenon may have contributed to these outcomes.

Inaccurate self-perceptions have potentially important implications for the design and targeting of skin cancer prevention messages and may, in part, help explain the observed, relatively slow, pace of change in some preventive behavior. One way of addressing this phenomenon would be to include a “correction factor” in targeted public health messages, with the aim of correcting awareness of potential risk among those who self-identify with darker skin types. One possible reason for the observed pattern of self-reported skin color misclassification is that the body areas probably most frequently self-observed are the hands and the face when seen in reflection. These areas also happen to be the most frequently exposed and, consequently, are more likely to be darker than other body areas more usually covered by clothing, including the inside upper arm area routinely taken as indicative of natural, nontanned skin color. Females, in particular, have been shown to be more influenced by tanned versus untanned skin color when rating their own degree of pigmentation (21). Other contributory reasons could be social and optimistic bias, the first relating to widely reported positive associations with tanning, including perceived health and attractiveness (22). Such social norms were thought to have influenced the self-reported phototypes of a Maltese population (23). The second has been a commonly reported phenomenon in other areas of investigation and concerns a tendency to underestimate personal risk, sometimes in relation to the risks of others. With respect to the contrary shift, the identification with a lighter skin color, this has fewer implications for skin cancer prevention health promotion practice, given that skin cancer risk may be overestimated. However, being overly conservative with respect to sun exposure could contribute to vitamin D insufficiency or deficiency.

The second self-report measure, the Fitzpatrick derived scale of erythemal sensitivity, correlated less well with objective ITA values than perceived skin color. This correlation may have been weakened by the lack of a more direct correspondence between the three erythemal sensitivity categories and the six ITA categories. Some of the difference in correlations could also be attributable to the fewer categories for skin sensitivity, but the majority of the difference requires further explanation. Nevertheless, both correlations were strong and our aim was to test the relation with an existing scale that had been used in several population surveys. It has been reported elsewhere that individuals under clinical assessment tend to underestimate their skin sensitivity (9). It seems plausible that this is related to the dark shift phenomenon and that the overestimation of skin pigmentation leads to an underestimation of skin photosensitivity. However, further research using other scales would be useful.

With respect to the second objective of this study, our findings indicate that the questionnaire items are reliable measures of skin color and erythemal sensitivity when assessed with a 7-day interval period. However, further research is needed to confirm whether this holds for longer periods, for example, over a summer season, when salient skin (hands and face) is likely to change color.

Limitations of our study include that it used a convenience sample, and that the age range of participants was not representative of the general NZ population, but there is no evidence from other surveys of any significant differences by age band in the distribution of responses to comparable self-report measures in national surveys. However, it is possible that cohort effects in perceptions exist. A further limitation is the relatively small numbers in the darker skin color categories, weakening our ability to identify associations for this group. This study achieved a reasonable balance between male and female participants, which is important given the distributional differences between men and women according to sun-reactive skin types (24, 21).

Although this study did not include an assessment of skin cancer, the ITA values have sound biological relevance, as they correlate well with melanin density (11), which, when assessed at the upper inner arm, is a strong predictor of the risk of skin cancer (25). In addition, the SP is free from observer bias and errors associated with recall.

To permit comparison between perceived and measured skin color, it was necessary to categorize ITA values. We used the Del Bino classifications, but there was not complete agreement between the specific color descriptors used by Del Bino and the categories used in our questionnaire (Table 1), which introduced an element of subjectivity into these analyses. The descriptors used for both the Del Bino and our own skin color categories are arbitrary and, to our knowledge, not derived from research. For respondents, there may be particular associations with words, such as “light” rather than “fair,” “tanned” rather than “olive,” “brown” and “dark” rather than “dark” and “black.” This may influence responses and is worthy of further investigation. However, the item is ordinal and there was a single extra response option available in our measure (“black”).

It is possible that widespread overestimation of skin color (and by implication, resilience) reduces perceptions of skin cancer risk and the uptake of sun-protective practices. Significant proportions of at-risk groups may be ignoring or misperceiving protective messages. Further research is needed to test these ideas with more diverse populations. In addition to the self-report and SP measures, assessments should be made of beliefs and attitudes toward sun safety practices and personal risk of developing skin cancers. Where possible, assessments should also be made of actual sun-related behavior and skin cancer histories.

The findings of this study suggest that although self-report may be a valid measurement strategy when assessing skin type, there is a bias toward overestimation of skin pigmentation and, potentially, UVR resilience. The implications of this for surveillance and prevention and intervention efforts must be considered. Skin cancer prevention messages have typically been aimed at people with fair skin, but may need to be adjusted to include messages for people of medium skin color to overcome the identified bias. The information gained from population surveys may need to take into account the dark shift phenomenon when estimates of skin type distributions are derived from population-based surveys that use questionnaire items.

No potential conflicts of interest were disclosed.

We thank Dr. Jevon Longdell (Physics Department, University of Otago) for expert advice on spectrometry as well as the student participants who gave their time and the research assistants who helped with data collection: Carly Shaw, Tom Graham, Matthew Jessop, Chelsea Hammond, and Shelley Erskine.

Grant Support: A.I. Reeder, V.A. Hammond, and the Cancer Society Social & Behavioural Research Unit received support from the Cancer Society of New Zealand and the University of Otago.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.