|
Estimates Of Health Insurance Coverage: Comparing State Surveys With The Current Population Survey
The Census Bureau produces annual state-level estimates of health insurance coverage using the Current Population Survey (CPS) Annual Social and Economic Supplement. Many states also conduct their own population surveys of health insurance status; in most cases, the state survey estimates of uninsurance are lower than the estimates produced by the CPS. This discrepancy fuels debate about the true count of uninsured Americans and changes in that number over time. This paper compares state survey and CPS estimates of uninsurance, highlights key reasons for these differences, and discusses the policy implications of this persistent discrepancy.
THE CENSUS BUREAU PRODUCES state-level estimates of the distribution of health insurance coverage each year.1 Although these estimates are often cited in both academic and policy circles, analysts are increasingly critical of them, suspecting that they overstate rates of uninsurance and understate Medicaid enrollment. For a variety of reasons, many state analysts prefer to collect and use their own survey data. More than forty states conduct their own population surveys to get better state-level estimates of health insurance coverage.2 State survey estimates of uninsurance are typically lower than the estimates produced by the Census Bureau’s Current Population Survey (CPS), Annual Social and Economic Supplement (ASEC) (sometimes referred to as the March supplement). These discrepancies fuel the debate about the number of uninsured Americans and could threaten the validity and usefulness of survey data to inform policy decisions around access to health care and insurance. Here we compare state survey and CPS-ASEC estimates of uninsurance, analyze the factors with the greatest potential to explain these differences, and discuss the policy implications of this persistent discrepancy.
The CPS is a monthly survey that the Census Bureau conducts for the Bureau of Labor Statistics. Questions about health insurance coverage and income are collected through the ASEC, which was initially added to the CPS in March of each year and was expanded to February through April beginning in 2001.3 The CPS sample is designed to be representative of each state and the District of Columbia.4 It is the most widely used source of estimates of health insurance coverage.5 The CPS-ASEC estimates are used to compare states on changes in the number of uninsured residents and is also one component of the federal formula used to distribute funds for the State Children’s Health Insurance Program (SCHIP). Several states have funded their own household surveys over many years for a variety of reasons, including monitoring the distribution of health insurance coverage (for example, Hawaii, Kentucky, Maine, Minnesota, Oregon, Utah, Vermont, and Wisconsin).6 Additional state survey activity was stimulated by the Health Resources and Services Administration (HRSA) State Planning Grant program, which awarded grants to states for planning and policy development to increase access to affordable health insurance coverage.7 Many states pursued data collection activities to inform this planning process, including fairly large household surveys. An additional inducement to state-specific data collection is the need to monitor SCHIP enrollment and evaluate the program’s impact on rates of coverage among low-income children in states receiving this funding.8 The issue of disparate survey estimates of the uninsured is not unique to state surveys and the CPS-ASEC. The federal government fields at least six surveys that produce different estimates of uninsurance—differences that are largely explained by methodological issues.9 Here we focus on comparisons between state surveys and the only national survey to produce state estimates: the CPS-ASEC.
Exhibit 1  compares state uninsurance estimates derived from state surveys with estimates from the CPS-ASEC in the same year that each state survey was conducted and CPS three-year average uninsurance estimates. The use of three-year averages is intended to increase the precision of uninsurance estimates, which are otherwise more variable from year to year.10 The state surveys provide point-in-time measures of whether a person had health insurance at the time of the survey. When available, the state full-year uninsurance estimates (lack of coverage for the previous twelve months) are provided. This is an important comparison because the CPS-ASEC is designed to measure whether someone was uninsured for the entire preceding calendar year.
As shown, state survey point-in-time estimates are almost uniformly lower than CPS-ASEC estimates, ranging from 5.3 percent lower in Maine to as much as 44 percent lower in Wisconsin. The exception is Oregon, whose point-in-time uninsurance estimate was higher than that of the CPS-ASEC. On average, the state surveys specify a point-in-time rate of uninsurance that is 23.0 percent lower than the one-year and 22.4 percent lower than the three-year CPS-ASEC estimate. One would expect the number of people uninsured at a point in time to be higher than the number who lacked health insurance coverage for an entire year, as is demonstrated among the fourteen state surveys that produce both estimates. This expectation does not hold when comparing the CPS-ASEC full-year uninsurance estimates with state point-in-time estimates of coverage. When comparing available full-year uninsurance rates from state surveys with those of the CPS-ASEC, estimates are even farther apart. State full-year uninsurance estimates are, on average, 47.4 percent lower than the one-year and 46.0 percent lower than the three-year CPS-ASEC rate of uninsurance. In both cases, on average, using the three-year CPS-ASEC estimate of uninsurance slightly narrows the discrepancy and almost eliminates the difference for one state (Maine).
There are several methodological reasons why state surveys produce lower estimates of uninsurance than those produced by the CPS-ASEC. Although state surveys differ from one another in design and administration, in this analysis the primary comparison is between the CPS-ASEC and state surveys generically. The exception is where differences between state surveys are relevant to the discussion of key methodological reasons for variation in estimates. The key reasons for discrepancies are discussed below. Population coverage and sample design. One explanation for lower estimates of health insurance coverage in state surveys is that most are telephone surveys using random-digit dialing (RDD) to sample households with active telephone lines only. By contrast, the CPS uses an area probability design drawing from address frames, with an in-person component that includes households with and without active telephone lines; however, more than 70 percent of the CPS data are collected over the telephone.11 Approximately 2.4 percent of U.S. households did not have phones in 2000, and members of these households were much more likely than those with telephones to be uninsured.12 Relying exclusively on telephone sampling could bias estimates downward. To account for this population coverage error, many states employ a telephone service adjustment. This adjusts households lacking telephone service for a significant period of time over the prior year to represent households without active telephone service, because studies show that the two groups are similar with respect to health insurance coverage and other important characteristics.13 An emerging population coverage issue for state telephone surveys is the growth of cell phone–only households.14 In 2005 it was estimated that 6.7 percent of households had cell phone service only—up from 4.5 percent in 2004.15 The CPS-ASEC includes cell phone–only households in its area probability design, whereas the state RDD samples attempt to purge cell phone numbers. Evidence suggests that people in cell phone–only households are much more likely to lack health insurance than are people with landline telephone service.16 This difference likely leads to biased estimates of coverage among RDD surveys. Unlike the telephone service interruption adjustment for households without telephones, no methodology exists for making a cell phone adjustment. If the cell phone–only population continues to grow, this population coverage error has the potential to cause major problems for telephone-only surveys. Questions measuring health insurance coverage. The design of health insurance questions and their placement in the overall survey may lead to differences in coverage estimates. We discuss relevant design and placement issues below. Time referent and definitions. Respondents in the CPS-ASEC are asked about their insurance status in February, March, or April for the prior calendar year, to match the referent period for income and unemployment questions. This means that respondents must try to recall coverage for a period that began fourteen to sixteen months prior to the interview. Attending to this reference period, combined with the long recall period, might decrease the accuracy of coverage reports.17 By contrast, most state-specific surveys ask about the respondent’s insurance at the time of the survey, which increases the likelihood of accurate reporting.18 Use of state-specific program names. Use of colloquial names should improve the accuracy of respondents’ reports.19 For example, states often refer to their Medicaid programs by state-specific names, such as "Medical Assistance" in Minnesota and "Medi-Cal" in California. Other state-specific coverage programs have propagated, particularly with the implementation of SCHIP, expanding the list of potential program names included in surveys (for example, "adultBasic" and "CHIP" in Pennsylvania and "BadgerCare" in Wisconsin). The Census Bureau works to include and update state-specific name fields. However, the CPS-ASEC list of program names might be less complete than is true for state-specific surveys, which could result in measurement bias. A recent study indicated that inclusion of state-specific names to a CPS-like survey instrument increased the reporting of public program participation in the survey of public program recipients from 59.6 percent to 89.4 percent.20 Question placement and context. The CPS is a labor-force survey with a supplement at the end that includes health insurance questions. By contrast, many of the state-specific surveys focus specifically on health insurance, placing these items close to the beginning. Exceptions are the California, Oregon, and Wisconsin surveys, which are longer omnibus health surveys that include health insurance questions later in the instrument. State surveys have an advantage over the CPS-ASEC, because the health-related content focuses respondents’ attention in a way that might increase the accuracy of reports about coverage irrespective of their placement in the overall survey. Differences in question placement and content could help explain variation in the amount of missing data; we return to this point later. Nonresponse bias. RDD surveys have experienced a major decline in response rates over the past ten years.21 A study by Scott Keeter and colleagues, which was updated in 2004 by the Pew Research Center, showed that surveys with high response rates yielded few significant differences on most estimates as compared to surveys with much lower response rates.22 Both studies found that surveys with lower response rates (such as 36 and 27 percent) can be as representative of the target population as surveys with higher response rates (such as 61 and 51 percent) when the focus is on opinion and attitude polling. However, it is not known whether low response rates lead to biased health insurance estimates. To respond to this question, a recent study examined data from three state telephone surveys measuring health insurance coverage. It compared responses among those who answered within five days of being called for the first time with the responses of those who initially refused to participate and those who took more than five days of calling to complete the survey. Responses to questions of coverage were very similar between these groups after age, sex, race, ethnicity, and geography (the usual post-stratification adjusters) were controlled for. Thus, leaving out those for whom the surveyors went to greater lengths to obtain responses from would not have greatly biased coverage estimates, even though it would have dropped the response rates from 50 percent to 20 percent.23 Note that the studies above compare only groups of people who eventually respond to the surveys, leaving unknown the characteristics and biasing impact of nonrespondents. The CPS has a higher response rate than the state surveys: 85 percent in 2003.24 If the statistical techniques employed to adjust responders to be representative of the entire target population do not adequately reflect the coverage of nonrespondents, then surveys with lower response rates (such as state surveys) would have more bias. There is no evidence to support this hypothesis; however, nonresponse bias remains a plausible explanation for variation in survey estimates. Data processing. Data processing prior to estimation of coverage rates might also account for some of the discrepancy between estimates. Less than 3 percent of cases have missing data on health insurance items in state surveys; therefore, these data are seldom imputed, and estimates are made from complete cases only. By contrast, the CPS-ASEC data are fully imputed and edited with 10–15 percent of the sample missing health insurance data, mostly for those who respond to the monthly CPS survey but refuse to take the ASEC supplement. The statistical method used to impute missing health insurance data might create bias in state estimates of coverage because "state" is not one of the variables in the model. As a result, people in Texas (the state with the highest rate of uninsurance in the 2004 CPS-ASEC) can have their values imputed from people in Minnesota (the state with the lowest rate of uninsurance) and vice versa.25 In the end, the CPS-ASEC might be underestimating the uninsurance rate for states with higher-than-average uninsurance rates, and vice versa. In addition, recent research shows that the Census Bureau’s method for imputing health insurance might lead to an undercount of people with employer-sponsored coverage and an overcount of people who are uninsured.26 The imputation routine restricts who can be assigned dependent coverage. Yet interviewers can assign dependent coverage to "everyone" in the household whether or not a given household member is on the policy or would legitimately qualify for the policy. As a result, if the imputed health insurance coverage data resembled the reported data, roughly 2.5 million more people (or 1 percent of the noninstitutionalized population below age sixty-five) would be assigned coverage—as opposed to being coded as uninsured—and all of the gain would be among estimates of employer-sponsored coverage.27 The CPS-ASEC also allocates children to Medicaid if a primary family member reports Temporary Assistance for Needy Families (TANF) income, regardless of whether Medicaid coverage is reported. This allocation occurs for approximately forty states that still have a link between cash and medical assistance programs. In the 1997 CPS-ASEC, approximately 16 percent of Medicaid coverage was logically imputed by assigning adult TANF recipients and their children to Medicaid coverage (most of whom would have been otherwise categorized as uninsured).28 Additionally, children under age twenty-one are allocated to Medicaid if the householder or their spouse reports Medicaid coverage. Finally, in states that legally require Medicaid coverage of all Supplemental Security Income (SSI) recipients, respondents who report SSI coverage (and their children) are edited to have Medicaid coverage.29 These edits lead to a greater likelihood of respondents’ being assigned Medicaid coverage in the CPS-ASEC. State surveys that do not ask and use information about SSI/TANF might have a lower estimate of Medicaid enrollment and a slightly higher uninsurance rate than surveys that do.
Our analysis demonstrates that fundamental design and administration features of state surveys and the CPS-ASEC contribute to persistent differences in state estimates of uninsurance. With regard to population coverage, sample design and nonresponse bias (for example, households lacking telephones, increases in cell phone–only households, declining response rates), the CPS-ASEC has the advantage over state surveys, because its area probability sample design includes people living in households with and without active telephone lines. States typically lack the resources to do in-person surveys and must rely on RDD telephone surveys. It remains to be seen whether declining response rates (which are inherently lower in RDD than area probability samples) are associated with increased bias in estimates of health insurance coverage. State surveys have the advantage in the area of measurement, because they typically focus on health insurance and access to care and ask about current health insurance coverage, and their creators have greater familiarity with and flexibility to alter their instruments to accommodate changes in the makeup and names of public programs. By contrast, changing the CPS-ASEC is more complex. The primary purpose of the CPS is to collect information about labor-force participation, which constrains the time referent of the health insurance questions in the CPS-ASEC to correspond with calendar-year income and employment questions. As the term "supplement" implies, the health insurance questions are at the end of a long survey, which increases the amount of missing data and the potential for bias as a result of data editing. Evidence is accumulating to indicate that the data processing that occurs in the CPS-ASEC prior to estimating coverage is the cause of some of the discrepancy between estimates of uninsurance.30 Policy uses of different estimates. The issues play out more practically at the state level when two very different estimates of uninsurance are reported in the popular press. The presence of two estimates, one higher and one lower, provides an opportunity to select the estimate with the greater likelihood of advancing a particular position. For example, advocates of the uninsured might prefer to use the higher CPS-ASEC estimates, because it keeps the pressure on policymakers to address this issue. Conversely, policymakers might prefer the lower estimate to advance claims that the number of uninsured residents is lower than previously thought or that state initiatives have reduced the number of uninsured residents. State analysts might have greater confidence in and preference for using their own state data irrespective of the magnitude of the estimate and might be motivated to advocate for their estimate to demonstrate the value of funding state surveys. State surveys are often conducted only periodically, either when funding is available or at regularly scheduled intervals. By contrast, the CPS-ASEC estimates are released annually and are often cited in the press. Regardless of data preferences, state analysts must be prepared to explain the differences between uninsurance estimates in a clear and straightforward manner. Furthermore, state survey and CPS-ASEC estimates can and should be used for different purposes. Contrasting trends in the number and characteristics of the uninsured across states requires use of the CPS-ASEC. If interested in specific information about subpopulations of uninsured people within a given state—such as children, racial/ethnic groups, or people in specific locales—state surveys are preferred because of their larger sample sizes and the ability to alter the sample design to focus on populations of interest.31 Relative merits. Both sources of state-specific estimates of insurance coverage have merit. Although the estimates derived are consistently different, ongoing research is investigating whether they trend the same over time, telling a coherent story about the distribution of insurance coverage and characteristics of the uninsured. Policy analysts have much to gain in drawing on both sources of data to inform decisions. IT IS IMPORTANT TO RECOGNIZE THAT the number of uninsured people will never be pinned down exactly. Different surveys produce different estimates because of the many complex choices involved in collecting health insurance data. Although research should focus on producing better estimates, the number remains just that: an estimate. The bottom line is that there are many uninsured Americans, and the numbers continue to grow. Focusing on the estimates and methods is important but should not distract from the need to address the health care needs of those without health insurance coverage.
Kathleen Call (callx001{at}umn.edu) is an associate professor in the School of Public Health, University of Minnesota, in Minneapolis. Michael Davern is an assistant professor, and Lynn Blewett is an associate professor. Preparation of this manuscript was supported by funds from the U.S. Department of Health and Human Services (HHS) Office of the Assistant Secretary for Planning and Evaluation and the Agency for Healthcare Research and Quality (AHRQ) (Contract no. 290-00-0017), and the Robert Wood Johnson Foundation (Grant no. 038846) to the State Health Access Data Assistance Center. The authors appreciate Rebecca Nyman’s assistance compiling the data and are grateful to the state analysts who provided information about their surveys and verified estimates. They thank Joel Cohen, Steven Cohen, and Steven Hill of AHRQ for their helpful comments on an earlier version of this manuscript.
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Background
Comparison Of Estimates Derived...
