Low-income women are at high risk of developing cervical cancer attributable not only to the higher prevalence of risk factors in this population but also to the lack of timely follow-up of abnormal Pap smears. This study evaluates the efficacy of an aggressive follow-up strategy. Women with abnormal Pap smear results after screening in a public hospital emergency department were randomly assigned to follow-up either by a case-managed approach using computerized tracking and universal colposcopy or by traditional care. The main outcome was the proportion of women receiving follow-up in 6 months. A secondary outcome was the proportion of women receiving follow-up by 6 months and diagnostic resolution in 18 months. Of 54 women in the intervention group, 65% kept at least one follow-up appointment in 6 months compared with 41% of the 54 women in the control group (P = 0.012). Half the women in the intervention group versus 19% of women in the control group had follow-up in 6 months and diagnostic resolution in 18 months (P = 0.001). After adjusting for age, initial Pap smear result, and race/ethnicity, the odds of having follow-up in 6 months were four times greater for women in the intervention group (odds ratio = 4.0; 95% confidence interval, 1.6–9.7), and the odds of having both follow-up in 6 months and diagnostic resolution in 18 months were more than six times greater (odds ratio = 6.5; 95% confidence interval, 2.4–17.8). This study demonstrates that an aggressive follow-up strategy significantly improves the rate of both initial follow-up and diagnostic resolution of abnormal Pap smears among low-income women with atypical squamous cells of undetermined significance and atypical glandular cells of undetermined significance when compared with traditional care.
Since the introduction of the Pap smear, a dramatic reduction has been observed in the incidence of and mortality associated with invasive cervical cancer in the United States. This is attributable to the ability of the Pap smear to identify precancerous changes and the availability of a number of effective treatments. However, not all women in the United States have benefited equally from this preventive measure. Whereas deaths attributable to cervical cancer have dropped >70% among Caucasian women, the same is not true for non-Caucasian women. Although disparities in rates have declined since 1990 (1) , the incidence of cervical cancer for African-American women is still 65% higher than for non-Hispanic Caucasians, and mortality rates remain more than double.3 African-American women have the highest age-adjusted mortality rates from cervical cancer (2) .
The higher mortality from cervical cancer in minority women may be attributable to a lack of access to health care (3) and consequently to preventive health care measures, including cancer screening (4 , 5) . Whereas Hispanic (6) , Chinese, and Vietnamese women, as well as women without insurance, are less likely to have had a Pap smear in the last 3 years (7 , 8) , African-American women may actually have higher rates of screening than Caucasian women (4 , 6) . Nevertheless, from 1983 to 1990, only 45% of invasive cervical cancers in African-American women were detected while still localized, compared with 55% in Caucasian women.3 Later stage disease at diagnosis may also result from inadequate follow-up of screening test abnormalities (9, 10, 11) . Barriers to follow-up result in delays in diagnosis and/or treatment among African-American (9 , 12) , Southeast Asian (13) , Hispanic (14) , and low-income women (15 , 16) . Studies have documented that up to 80% of low-income women fail to obtain the necessary follow-up and treatment for their abnormal Pap smear (15 , 17) and that low-income and ethnic minority women have the lowest rates of completion of follow-up (10 , 16 , 18, 19, 20, 21, 22) . Many ethnic minority and low-income women are screened in public hospitals and community clinics. For these women, disruptions in the continuity of care between the site of screening and the site of diagnostic and treatment services may result in women being lost to follow-up (23) .
In an earlier study, we found that significantly more low-income women could receive screening for cervical cancer if the test is performed during urgent care visits to a public hospital ED4 than when women, seen in the ED, are referred for later screening in an ambulatory care clinic site (24) . We found, however, that 30% of women who had abnormal results on ED screening received no follow-up. To address the lack of follow-up, we designed a randomized, controlled trial evaluating the effectiveness of a comprehensive and aggressive follow-up program aimed at increasing the proportion of low-income women who receive follow-up for Pap smear abnormalities found in the ED setting (23) . The intervention consisted of three components: case management, computerized tracking, and universal colposcopy. To our knowledge, this is the first controlled trial to evaluate the efficacy of different follow-up strategies in a public hospital setting.
Materials and Methods
Setting and Patients.
The HGH is a 300-bed, acute-care public teaching hospital located in Oakland, California. As the acute public hospital and trauma center serving northern Alameda County, the HGH provides ambulatory medical services to >85,000 women annually. Of these visits, 35,000 are to the ED. Over 90% of Highland’s patients earn <200% of federal poverty levels, and 92% depend on some form of government assistance for medical care. The majority of the patients served by the ED are non-Caucasian, medically indigent, and have no routine source of health care. Patients presenting to the ED are triaged to either the urgent care clinic or the emergency room; both sites were used for this study.
The target population was women, 18–74 years old, who visited the ED between October 1, 1993 and June 30, 1995, and whose evaluation included a diagnostic pelvic examination and a coincidental Pap smear. Women were eligible for inclusion in the study if that Pap smear result was abnormal: ASCUS, AGUS, LGSIL, or HGSIL (25) . Women were excluded if they: (a) were hospitalized at the time of the ED visit; (b) resided outside of Alameda County; (c) were currently under evaluation for an abnormal Pap smear; or (d) chose to follow up elsewhere. Neither symptoms nor signs of a sexually transmitted disease or pelvic inflammatory disease excluded women from participation in the study.
A member of the study team contacted eligible women by telephone and/or letter to notify them of the abnormal Pap smear result, the need for a follow-up visit, and to recruit their participation in the study. If a woman indicated that she wished to follow up with an outside physician, the physician was contacted, and a copy of the Pap smear result was then forwarded. Of 2,167 women who received a Pap smear in the ED, 147 (7%) had an abnormal result. A total of 108 women, who consented to participate and return to the HGH for follow-up, were randomly assigned to receive their initial follow-up in either the usual care gynecology clinic or in the intervention clinic, designated the CARE Clinic. Each woman, regardless of assignment, was given a specific follow-up appointment over the telephone either in the gynecology clinic or the intervention clinic; a letter that included directions subsequently confirmed this appointment.
Women were assigned a study number in the order in which their Pap smear results were reported to the HGH laboratory. After study number assignment, a corresponding sequentially numbered, sealed envelope containing a predetermined random study arm assignment (intervention or control) was opened. Women in the intervention group were notified by telephone and in writing of an appointment for follow-up in the CARE Clinic; women in the control group were likewise notified of a follow-up appointment in the gynecology clinic, and a consult form stating the reason for the visit was placed in the woman’s medical record, along with a copy of the ED Pap smear result.
The follow-up protocol in the CARE Clinic differed in multiple ways from the usual care in the gynecology clinic. Women assigned to the intervention were followed by a nurse case manager who gave each woman a reminder call before each appointment and called immediately after any missed appointment to reschedule and to stress the importance of following up. Tracking was facilitated by a computerized database that included a recommended interval to the next follow-up appointment. The clinic was staffed by a specialized nurse practitioner who performed colposcopic examinations, obtained biopsies of suspicious cervical lesions, and performed ECC on women whose transformation zone was inadequately visualized.
Six months after the ED Pap smear, the medical records and the laboratory Pap smear database were reviewed to determine whether women in the control group had kept their appointments for follow-up of their abnormal Pap smear. If a woman had not kept any follow-up appointments in the 6-month interval, she was crossed over to the intervention protocol, and the nurse case manager attempted to contact her and schedule a follow-up appointment in the CARE Clinic. This crossover was done for ethical considerations and was termed a “CARE Rescue.”
The main outcome used to assess the efficacy of the intervention was the proportion of women who received initial follow-up in 6 months. The secondary outcome was the proportion of women who received an initial follow-up in 6 months and whose abnormal Pap smear was resolved in 18 months. A woman was determined to have had follow-up in 6 months if, during the 6-month period after her initial abnormal Pap smear in the ED, she kept at least one clinic visit at which either a Pap smear and/or a colposcopic examination was performed. We considered a Pap smear abnormality to be resolved if one of the following conditions existed: (a) a tissue diagnosis of CIN was made; or (b) a woman completed the follow-up protocol and no abnormality was found. For ASCUS and LGSIL to be resolved as negative for dysplasia, the protocol required that either: (a) three serial Pap smears were normal; (b) the colposcopic examination of the cervix was normal; or (c) the colposcopic examination was abnormal, but a biopsy specimen of the abnormality was negative for dysplasia. If the transformation zone was not visualized at colposcopy, ECC was also required for resolution. For resolution of HGSIL, the protocol required expert review in the event of a negative or low-grade biopsy. All women in the CARE group were followed up by colposcopic examination. In the control group, in accordance with institutional guidelines, women with ASCUS were usually followed up by three serial Pap smears at 3- to 6-month intervals, whereas women with AGUS, LGSIL, and HGSIL underwent colposcopic examination. In addition, an ECC was performed on all women with AGUS. Whereas the above resolution criteria were created a priori, refinements in the definition of a negative resolution of ASCUS and LGSIL were developed during the course of this study.
Our first hypothesis was that the proportion of women who received an initial follow-up in 6 months would be greater among women in the intervention group than among women in the control group. The second hypothesis was that the proportion of women whose abnormality was followed up in 6 months and was resolved in 18 months would be greater among women in the intervention group than among women in the control group.
The study was performed in accordance with the ethical principles set forth in the Declaration of Helsinki and with local regulations. The institutional review boards of the Alameda County Medical Center and the Northern California Cancer Center approved the protocol. Informed consent was obtained verbally from all patients at the time of notification of the abnormal Pap result.
Statistical analyses were carried out using SAS version 6.12 (26) . χ2 tests were used to compare the differences between the intervention and control groups with respect to the distributions of patient characteristics and outcomes. Multiple logistic regression models were used to control for possible confounding and to identify significant predictors of the dependent variables: 6-month follow-up and both 6-month follow-up and 18-month diagnostic resolution. The initial full logistic regression model contained these variables: age (continuous), ethnicity (Non-Hispanic Caucasian, African-American, Hispanic, and Asian and other race), language (English and speak other languages only), insurance status (insurance unknown, has insurance, and no insurance), initial Pap smear result (ASCUS, LGSIL, and HGSIL), and intervention status (control and intervention). We used backward elimination to arrive at the final parsimonious models. The significance level for all of the analyses was set at 0.05, and we computed ORs and 95% CIs for the independent variables. The original power calculations are described in our baseline manuscript (23) .
The response rate of 87% was defined as the number of women who agreed to participate (108) divided by the number of eligible cases (124). We were unable to reestablish contact with 16 women (the 13% nonresponders) after their ED visits. Women with abnormal Pap smears were ineligible because of age (5) , death (3) , incarceration (4) , residing outside of treatment area (3) , participation in other studies (1) , and follow-up of prior abnormal Pap smear in progress (7) . Table 1⇓ shows the demographic characteristics of the study participants and the distribution of initial Pap smear results. There were statistically significant differences between the intervention and control groups in insurance status and initial Pap smear result but no differences in age, ethnicity, and language.
Table 2⇓ shows the proportion of follow-up in 6 months and the proportion of follow-up in 6 months and resolution in 18 months by control and intervention groups. The proportion of women who received an initial follow-up in 6 months was higher among the intervention group than among the control group (P = 0.012). Similarly, the proportion of women in the intervention group who received an initial follow-up in 6 months and whose abnormal pap smear was diagnostically resolved in 18 months was higher than those in the control group (P = 0.001).
Because the initial Pap smear result influences follow-up efforts, we performed analysis stratified by initial Pap smear result. Among women with ASCUS or AGUS, women in the intervention group were more likely to receive a follow-up visit in 6 months (62% versus 37%, P = 0.016) and both 6-month follow-up and 18-month diagnostic resolution (47% versus 11%, P = 0.001) than among the control group. However, for women whose initial Pap smear was LGSIL or HGSIL, there were no significant differences in follow-up between the study arms: 63% of controls and 78% of intervention women had a visit in 6 months, with nearly all of these going on to obtain diagnostic resolution in 18 months.
Table 3⇓ shows the result of the final parsimonious logistic regression models of follow-up in 6 months and of follow-up in 6 months and diagnostic resolution in 18 months. The odds of having a follow-up visit in 6 months were four times greater for women in the intervention group (OR = 4.0), after adjusting for age, insurance, and race/ethnicity. Women who were older (OR = 1.1/year) or had no insurance (OR = 2.8) were more likely to have a 6-month follow-up visit. Asian women or women of “other” race/ethnicity were less likely to have a follow-up in 6 months (OR = 0.2).
The odds of having a follow-up visit in 6 months and resolution in 18 months were over six times greater for women in the intervention group (OR = 6.5), after adjusting for age, initial Pap smear result, and race/ethnicity. Women who were older (OR = 1.1/year) or had an initial Pap smear result of LGSIL or HGSIL (OR = 4.2) were more likely to have follow-up in 6 months and diagnostic resolution in 18 months. Asian women and women of “other” race/ethnicity were less likely to have follow-up and diagnostic resolution on time (OR = 0.1).
We found that among intervention women, those who had a follow-up visit in 6 months were three times more likely to be resolved in 18 months than those who did not (77% versus 26%, P = 0.001). However, among control women, those who had no visit in 6 months (and received the CARE Rescue protocol) were as likely to have 18-month resolution (44%) as those who obtained follow-up in 6 months through the usual channels (45%).
Our results demonstrate that in a population of high-risk women who were screened for cervical cancer in a nonprimary care setting, an aggressive follow-up strategy (case management, computerized tracking, and early universal colposcopy) can be effective at increasing both the proportion of women who return for an initial follow-up visit and whose Pap smear abnormality is diagnostically resolved as compared with the usual care follow-up. The odds of having a follow-up visit in 6 months and diagnostic resolution in 18 months were 6.5 times greater for women who received the intervention.
The randomization strategy used in this study was successful in most respects but resulted in significant differences between the control and the intervention groups in two important ways; more women with public insurance were randomly assigned to the intervention than to the control group, whereas more women with HGSIL on their initial Pap smear were assigned to the control than to the intervention group. We do not, however, believe that these discrepancies account for the significance differences in the outcomes of this study. In fact, we would expect that some of these factors would mitigate against the positive outcome. Women who have insurance, public or private, are at greater liberty to follow-up in other settings than a public hospital. Therefore, it is possible that the proportion of women with such insurance following up in the public hospital would be less than among those who lack insurance. Likewise, more women in the control group than in the intervention group had HGSIL on their initial Pap smear. Because we know that the limited resources available for follow-up in this setting are directed to those with the highest degree of abnormality, this discrepancy would be expected to reduce the observed effect.
In evaluating the relative difference in follow-up rates between the control and the intervention groups, it is important to note that the control group actually received an intervention beyond usual care. We attempted to call all women with abnormal Pap smears, regardless of study assignment, and to notify them of their abnormal Pap smear result, set up a convenient follow-up appointment, and briefly counsel them about the significance of their Pap smear result and the importance of keeping the follow-up appointment. In our experience, women frequently receive a follow-up appointment in the mail with no explanation of their abnormal Pap smear result. Consequently, we expect that the follow-up rates would actually have been even lower in the control group outside the context of this study. Thus, the observed difference is probably an underestimate for this reason as well.
We found no difference between the control and the intervention groups in the rate of diagnostic resolution of HGSIL or LGSIL Pap smear abnormalities. This appeared to reflect the more vigorous follow-up protocol for women with HGSIL and LGSIL than for ASCUS, regardless of study group assignment. This study, however, underscores the importance of vigorously following up high-risk, low-income women with ASCUS Pap smear results. Among women followed up for LGSIL and HGSIL, 12 were eventually found to have biopsy-proven dysplasia, compared with 15 women followed up for ASCUS, despite an overall lower follow-up rate for this group. Of the 10 women found to have CINIII/carcinoma in situ at colposcopy, 6 had an initial Pap smear result of ASCUS, whereas 4 women had LGSIL or HGSIL. The higher absolute number reflects the higher incidence of ASCUS results overall but underscores the importance of this high-risk group and the need for thorough evaluation of women with ASCUS, especially when compliance with long-term follow-up cannot be anticipated (27) . Because women screened in the ED lack access to regular screening, it is likely that some with ASCUS, who were lost to follow-up, have progressed to more serious dysplasia. Indeed, one woman with ASCUS, who had not kept any follow-up appointments, did recontact us 2 years later after having been diagnosed with invasive cervical cancer elsewhere. Colposcopy has been recommended for the follow-up of ASCUS in low-income women because of low rates of long-term follow-up required in relying on serial Pap smears (28) .
Although this study reports the results of an intervention designed to improve follow-up of abnormal Pap smears among women whose initial screening exam was performed in an inner city ED, the results may be generalizable to low-income women screened in other settings. Many medically indigent, low-income women lack a primary care physician and, consequently, seek nonurgent care in hospital EDs (29) . Access to screening Pap smears and mammograms frequently occurs in other nontraditional sites where follow-up is not available, such as storefront clinics, mobile vans, and homeless shelters. Women with abnormal screening tests are subsequently referred for follow-up to another medical facility. This transfer of care for women without a primary care provider from one site to another is, in our experience, a common cause of women being lost to follow-up. We expect, therefore, that an intervention such as this would also improve follow-up rates for women screened in a variety of community sites.
Because the diagnostic resolution of an abnormal Pap smear may require multiple visits over months or years, women who start to follow up often fail to complete all of the necessary visits. Programs that screen low-income women need to address this issue. Loss to follow-up effectively rescinds any benefit of screening. Current reimbursement rates for women with public insurance do not cover the cost of the type of case management needed to ensure timely follow-up (23) .
It is important to consider that, although this intervention succeeded at significantly improving follow-up by 6 months and diagnostic resolution of Pap smear abnormalities by 18 months, 50% of women still did not complete the recommended follow-up protocol. In our experience, the medical records of women newly diagnosed with cervical cancer often reveal prior abnormal Pap smears for which there has been incomplete follow-up. Our results indicate that women who are not followed up sufficiently to resolve cytological abnormalities tend to be younger in age and non-English and non-Spanish speaking.
Asian/Pacific Islanders comprise a number of ethnic groups recognized to have increased rates of cervical cancer. Whereas the etiology of these higher rates is not fully understood, adherence to follow-up recommendations may be a contributing factor (13) . Although educational materials are increasingly available in certain Asian languages, we encountered difficulty in offering our intervention to this group because of the lack of any one predominant Asian language in our study population. More than 10 Asian languages are spoken among the women screened in the ED, including Cambodian, Cantonese, Hindi, Korean, Mandarin, Mien, Laotian, Punjabi, Tagalog, and Vietnamese, creating a notable service barrier and delays in services because of limited interpreter availability. The translation of letters and materials into all of these languages was not possible within the scope of this project. Interventions aimed at facilitating and improving health communication with ethnically and linguistically diverse Asian women in health care settings will be important in reducing the rate of invasive cervical cancer and improving the general health of this population.
These results suggest that one rectifiable reason low-income women are lost to follow-up after an abnormal screening Pap smear result is the lack of resources on the part of the health system performing screening procedures. More support could be provided to assist women in receiving follow-up services. Our findings strongly suggest that more widely available case-management services for low-income women with abnormal screening Pap smears could result in lower rates of cancer morbidity and mortality. Additional work is also needed to evaluate the costs and cost effectiveness of various follow-up strategies. Although no formal cost-effectiveness analysis was performed as part of this study, the ongoing personnel requirements for tracking and case management of women with abnormal results were primarily 8–10 h/week of a nurse follow-up coordinator and 4 h/week clerical time. The staff was supported by a PC equipped with a tracking database programmed specifically for this project.
In conclusion, low-income women are at high risk of being lost to follow-up after having an abnormal Pap smear result. This study demonstrates that case management, computerized tracking, and universal colposcopy significantly improves the rate of follow-up and diagnostic resolution of abnormal Pap smears when compared with traditional care among low-income women screened in a nonprimary care setting. Moreover, these findings suggest that an aggressive follow-up strategy could well be expected to reduce morbidity and mortality attributable to cervical cancer in this high-risk population. Insurers of low-income women, notably Medicaid, could promote improved follow-up by reimbursing programs for evidence-based, case-management expenses.
We thank the following individuals and departments at the Alameda County Medical Center’s Highland Hospital whose advice, assistance, and cooperation made this project possible: Drs. Barry Simon, Andrew Levitt, and Andrea Thach and the residents and staff of the Department of Emergency Medicine; Dr. Harold Brazil and the Department of Pathology and Laboratory Medicine; Dr. James Jackson, John Lennox, and the Department of Maternal and Child Health; and the Department of Community Relations. We also thank Donica Hopkins, Bridgette Jackson, May Leung, and the staff of the Northern California Cancer Center for their invaluable and dedicated support.
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
↵1 Supported by National Cancer Institute Grant 5 PO1 CA55112.
↵2 To whom requests for reprints should be addressed, at Alameda County Medical Center, Highland Hospital, Department of Internal Medicine, 1411 East 31st Street, Oakland, CA 94602. Phone: (510) 437-4121; Fax: (510) 437-5106; E-mail:
↵3 Internet address: http://www.seer.cancer.gov/Publications/CSR1973_1997.
↵4 The abbreviations used are: ED, emergency department; CIN, cervical intraepithelial neoplasia; HGH, Highland Hospital Campus of the Alameda County Medical Center; ASCUS, atypical squamous cells of undetermined significance; AGUS, atypical glandular cells of undetermined significance; LGSIL, low-grade squamous intraepithelial lesion; HGSIL, high-grade squamous intraepithelial lesion; CARE, Cancer Awareness Research and Education; ECC, endocervical curettetage; OR, odds ratio; CI, confidence interval.
- Received April 6, 2001.
- Revision received August 1, 2001.
- Accepted August 14, 2001.