social science Rising: a Tale of Evidence shaping Public Policy

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 1

Social Science Rising: A Tale of Evidence

Shaping Public Policy

Ron Haskins, Christina Paxson, and Jeanne Brooks-Gunn

POLICY BRIEF FALL 2009

A major goal of social science is to inﬂuence public

policy by generating practical knowledge that can

help policy makers make informed decisions. This

is especially true of social scientists who study chil-

dren. Over the past four decades, they have devel-

oped increasingly reliable methods to test whether

programs affect children’s behavior and develop-

ment and if so, whether their effects are long-lasting.

Stripped to its basics, the model that developmen-

tal scientists follow is to identify an important social

problem, design a treatment for the problem (or for

preventing the problem), and test whether the treat-

ment produces the desired outcome. In some cases,

the ﬁndings can be used to calculate the beneﬁts and

costs of large-scale implementation, thereby provid-

ing policy makers with arguably the most direct and

pertinent information they need to make sound deci-

sions about public spending.

Just such a scenario is now playing out in the nation’s

capital. In his budget blueprint released in Febru-

ary 2009, President Barack Obama recommended

spending up to $8 billion over the next ten years on a

Ab o u t t h e A u t h o r s : Ron Haskins is a senior

editor of The Future of Children, senior fel-

low and co-director of the Center on Children

and Families at the Brookings Institution, and a

senior consultant at the Annie E. Casey Founda-

tion. Christina Paxson is a senior editor of The

Future of Children, dean of the Woodrow Wilson

School of Public and International Affairs, and

the Hughes-Rogers Professor of Economics and

Public Affairs at Princeton University. Jeanne

Brooks-Gunn is the Virginia and Leonard Marx

Professor of Child Development and Education

at Teachers College and the College of Physicians

and Surgeons at Columbia University.

For the full report on Preventing Child Mal-

treatment, edited by Christina Paxson and Ron

Haskins, go to www.futureofchildren.org.

Rigorous social science evaluations of home-visiting programs designed to improve

parenting and reduce child maltreatment convinced President Barack Obama’s admin-

istration to initiate a multi-billion-dollar federal program to expand a particular model of

home visiting. Supporters of other models reacted by lobbying Congress and the admin-

istration to fund other program models as well. In the resulting compromise, programs

with the strongest evidence of success would receive the most money, and those with

modest evidence of success would get some but less money. All programs that are

funded would be subject to continuous evaluation using rigorous methods to ensure

continuing good results. At least in this case, policy makers are focused on social sci-

ence evidence and are using it to identify and support the most successful programs.

2 THE FUTURE OF CHILDREN

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 3

nurse home-visiting program aimed at helping poor

mothers learn parenting behaviors that would boost

their children’s development. Tracing the early his-

tory of this proposal as Congress prepares for legisla-

tive action illustrates both the trials and triumphs of

social scientists’ efforts to produce evidence to shape

public policy.

What, Exactly, Are Home-Visiting Programs?

First, some background. Home-visiting programs

come in all shapes and sizes. There are a host of

program models, many with written curriculums,

trained staff, and elaborate ﬁnancing arrangements.

Some programs already serve thousands of children.

Individual programs vary dramatically with respect to

children’s age, risk status of families served, range of

services offered, and intensity of the intervention as

measured by the frequency and duration of the home

visiting. They also vary by who makes the visits, usu-

ally either a trained paraprofessional or a professional

nurse, teacher, or social worker. Nor do all programs

have the same goals. Some aim speciﬁcally to reduce

child maltreatment, whereas others focus on improv-

ing children’s health and developmental outcomes.

What they all share is the view that services deliv-

ered in a family’s home will have a positive impact on

parenting, which in turn can inﬂuence the long-term

development of the child.

Although home-visiting programs have been around

for more than a hundred years, many newer pro-

grams developed since the 1960s use sophisticated

evaluation methods to test their effectiveness. The

best programs with the strongest reputations have

been evaluated using randomized clinical trials

(RCTs), which have recently been recognized by the

National Academies as providing “the highest level of

conﬁdence” in program efﬁcacy or failure. RCTs ran-

domly assign families eligible for a program either to

an experimental group, which receives the treatment,

or to a control group, which does not; evaluators col-

lect information about parents and children in both

groups over many years during and after the treat-

ment. Random assignment ensures that both groups

are initially equivalent, thereby assuring that any dif-

ferences in parenting or child outcomes between the

groups over time are attributable to the treatment.

There is widespread—but not universal—agreement

in the scholarly world that RCTs are the gold stan-

dard of program evaluation. If programs have not

been evaluated by random assignment, according

to the National Academies, “evidence for efﬁcacy or

effectiveness cannot be considered deﬁnitive, even if

based on the next strongest designs.”

Applying Evidence to Public Policy

Not surprisingly, the process of applying evidence,

even gold standard evidence, to choice in the policy

world does not always go smoothly. The ﬁrst bump in

the road is politics. The views of congressional com-

mittee chairmen, national political party leaders, and

the president almost always outweigh evidence. And

powerful interest groups that can shape legislation

and determine its fate are often motivated primar-

ily by their desire to help the people and organiza-

tions they represent, regardless of what the evidence

shows.

A second problem is that the research evidence

based on carefully executed RCTs is seldom as clear

and straightforward as one would like. The results of

RCTs hinge critically on the design and implementa-

tion of the speciﬁc programs being evaluated and on

the characteristics of the groups participating in the

evaluation. Differences in program implementation,

especially the skill and training of those delivering

the program and the demographic characteristics of

program participants, can have major impacts on out-

comes. For these reasons, evaluations of similar pro-

grams—or even of the same programs administered

in different settings—can yield discrepant results,

leading to no end of dispute about whether the pro-

gram “works.” The result can be a battle of conﬂicting

evidence that is bafﬂing and sometimes annoying to

policy makers.

The current home-visiting policy scramble illustrates

both of these difﬁculties. Many researchers and

reviewers have singled out one particular program,

the Nurse-Family Partnership program developed

by David Olds, as being especially effective and well-

documented. First tested by RCT in rural New York

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 3

on a predominantly poor sample of white teen moth-

ers beginning in 1977, the program was later evalu-

ated using RCTs in Memphis and Denver. In both

replications, some characteristics of the original pro-

gram, as well as the types of participating families,

were varied. The results of evaluations of the three

trials, all of which produced signiﬁcant effects on key

parent or child outcomes such as child health and

safety, parenting quality, and child cognition, have

been reported in peer-reviewed journals, a sine qua

non for a program to claim it is supported by scien-

tiﬁc evidence. In 1996, Olds began expanding the

program by working with state ofﬁcials and others

while trying to ensure ﬁdelity to his program model.

By 2008, Nurse-Family Partnership programs had

spread to twenty-ﬁve states. Seldom has an interven-

tion program been so carefully tested and expanded

with such serious attention to getting new sites to

maintain ﬁdelity to the program model.

The success of the Olds program did not go unno-

ticed by senior ofﬁcials in the Obama campaign and

subsequently in the Obama administration. In addi-

tion to a host of other domestic initiatives, President

Obama’s 2010 budget blueprint included funds for a

“Nurse Home Visitation” program. The text accom-

panying the blueprint leaves little doubt about what

the administration had in mind. First, the initiative

uses the word “nurse,” which is not part of the name

of any prominent home-visiting program except the

Olds Nurse-Family Partnership. Second, in describ-

ing the program, the budget text explains that funds

would be given to states to “provide home visits by

trained nurses to ﬁrst-time low-income mothers and

mothers-to-be.” Again, a perfect ﬁt with the Olds

program. The text then continues: “The program

has been rigorously evaluated over time and proven

to have long-term effects” and produces a “return-

on-investment [of] between 3 to 6 dollars per dol-

lar invested.” With these words, the administration

served notice that it supported only programs with

strong evidence of success. Indeed, the most reason-

able interpretation of the wording is that the admin-

istration intends to fund the Olds program.

Obama’s apparent intention to fund only the Olds

home-visiting program startled the worlds of early

childhood education in general and home visiting

in particular, because it meant that other nationally

prominent programs such as Parents as Teachers,

Healthy Families America, the Parent-Child Home

Program, and HIPPYUSA would be left out. The

concerns of these groups were not without merit.

Some of these other programs had, like the Nurse-

Family Partnership, been subject to rigorous evalu-

ation with RCTs. Furthermore, within the scholarly

world, some believed that the Olds program required

further evaluation: there were inconsistencies in the

results from the three evaluations; the programs had

not been subject to evaluation by researchers outside

of Olds’ team; and the program focused on a narrow

group of mothers—notably low-income ﬁrst-time

mothers who agreed, while pregnant, to participate

in a two-year program.

The Lobbying Begins

With the emphasis on “nurse home visiting” in

Obama’s budget blueprint, the debate left the pris-

tine conﬁnes of academic journals and conferences

and leaped into the rough and tumble forum of fed-

eral policy making. In this venue, the home-visiting

programs that felt slighted by the president’s budget

blueprint initiated a lobbying campaign to broaden

the president’s language to include additional home-

visiting programs. Many of the programs not singled

out by the president were part of a long-established

coalition of inﬂuential and effective Washington child

advocacy groups that included the Center for Law

and Social Policy, the Children’s Defense Fund, the

Child Welfare League of America, and others. The

Differences in program

implementation, especially

the skill and training of those

delivering the program and the

demographic characteristics of

program participants, can have

major impacts on outcomes.

4 THE FUTURE OF CHILDREN

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 5

general line taken by these programs and their advo-

cates was that Obama’s emphasis on home visiting

was an important advance for children and families,

but that his proposal to single out one program for

support was ill-advised. All high-quality, evidence-

based programs, they argued, should be eligible for

funding. Not surprisingly, groups favoring the Olds

program started lobbying, too. All this is standard fare

for federal policy making; the only difference is that

those favoring the Olds program and those favoring

broader inclusion would normally be allies on federal

legislation to support children and families.

Two entries in the debate are especially worthy of

note. The Coalition for Evidence-Based Policy,

an inﬂuential Washington lobby for high-quality

program evaluation, declared its support for the

president’s decision to fund research-proven home-

visitation programs such as the Nurse-Family Part-

nership. Run by Washington veteran Jon Baron,

the coalition has assembled an advisory board that

includes several noted scholars and others with an

interest in applying high-quality evidence to policy

choice, including a Nobel laureate (full disclosure:

one of the authors of this brief is a member of the

coalition’s advisory board, though unhappily not

the Nobel laureate). In April, the coalition issued a

well-reasoned brief that emphasized its nonpartisan

nature as an organization focused on promoting the

development of rigorous evidence. Indeed, Baron

and his coalition have almost single-handedly suc-

ceeded in getting many pieces of federal legislation

to designate funds for program evaluation, especially

RCTs. Citing an “authoritative” evidence review

from The Lancet, a respected medical journal, that

found the Olds program to have the “best evidence

for preventing child abuse and neglect,” the coalition

expressed unqualiﬁed support for funding of pro-

grams, such as the Nurse-Family Partnership, that

meet the highest standards of evidence. A six-page

attachment to the brief reviewed evidence from the

three RCTs by which the Nurse-Family Partnership

had shown its strong impacts while pointing to deﬁ-

ciencies in the RCTs by which ﬁve other home-visiting

programs had been evaluated.

Perhaps spurred by the coalition brickbat against the

non-Olds programs, four highly respected scholars,

including Deborah Daro of the University of Chi-

cago, Ken Dodge of Duke, Heather Weiss of Har-

vard, and Ed Zigler, the child development sage from

Yale, issued a call for broadening the funding. Their

soundly argued letter to the president praised his

proposal for investing in home-visitation programs,

but criticized the focus on one program model. The

impressive quartet argued that a single program

model targeted on ﬁrst-time mothers would leave

out too many at-risk parents. They also cautioned

against a sole reliance on evidence generated from

RCTs, which do not provide guidance on how to

scale up a model program to serve national needs.

Finally, they expressed the view that although at-risk

families merit the most intensive services, all fami-

lies should have access to early child development

programs. The world of social science, it appears,

does not speak with one voice, and even the best evi-

dence can lead to multiple—and sometimes directly

opposing—conclusions.

Possibilities for Compromise

By the time Congress approved its budget resolu-

tion in late April, the forces supporting the broader

language appeared to be making headway, because

the budget supported home-visiting programs that

“will produce sizable, sustained improvements in the

health, well-being, or school readiness of children or

their parents” and contained no mention of nurse vis-

iting. Similarly, the Obama language on nurses was

gone from the ﬁnal administration budget released

in early May.

The next and critical step was for congressional com-

mittees to begin writing the new program into law.

Chairman Jim McDermott (D-WA) of the Human

Resources Subcommittee of the House Ways and

Means Committee was the ﬁrst out of the box. In early

June he circulated draft legislation and then held a

hearing on his bill on June 9. Like the budget resolu-

tion, the McDermott draft bill represents a compro-

mise between the contending forces. Speciﬁcally, it

would give priority funding to programs that “adhere

to clear evidence-based models of home visitation

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 5

that have demonstrated signiﬁcant positive effects on

important program-determined child and parenting

outcomes, such as reducing abuse and neglect and

improving child health and development.” Preferred

programs must also have “well-trained and compe-

tent staff” and include training, technical assistance,

and evaluation.

Many home-visiting programs—some of which

are reviewed in an article in the latest issue of The

Future of Children by Kimberly Howard and Jeanne

Brooks-Gunn of Columbia University—–would seem

to qualify under the McDermott language because

they follow a model that has at least some evidence

of success from RCTs, feature trained staff, and have

other characteristics consistent with McDermott’s

language. A subsequent section of the McDermott

draft, however, stipulates that programs “that do not

adhere to a model of home visitation with the stron-

gest evidence of effectiveness” would be eligible for

less generous funding. McDermott makes the word

“strongest” do a lot of work. Apparently the com-

promise intended is to give preferred funding to the

Nurse-Family Partnership because it is the program

with “strongest evidence of effectiveness,” but still to

provide some funding for the other programs ﬁght-

ing to be included. Whether the McDermott lan-

guage achieves the distinction between these two

types of programs is arguable. Should the legislation

be enacted as drafted, the regulations written by the

Department of Health and Human Services (HHS)

could settle the issue, in which case lobbying efforts

will shift from Congress to HHS. Any party not satis-

ﬁed with the regulations could take HHS to court.

In the end, the courts could decide the issue—at

least until Congress redrafts the provision to clarify

its original intent, at which point the entire process

would begin again.

Perhaps the most important sign of the central

role being played by evidence in this debate is the

June 8 blog posting of Peter Orszag, the director of

the federal Ofﬁce of Management and Budget and

President Obama’s closest adviser on budget policy.

Orszag asserts that he and the president are placing

evidence of program success from “rigorous” evalu-

ations at the center of decision making. He states

emphatically that the Obama administration will

evaluate as many programs as possible, cut off fund-

ing for those that are not working, and expand those

that are. In the case of home-visiting programs, he

endorses the two-tier approach of giving more money

to the programs with the strongest evidence of suc-

cess and some but less money to programs that have

“some supportive evidence but not as much.” Orszag

also cites several examples of how the administration

is expanding funds for conducting rigorous program

evaluations and then using the evidence to make

funding decisions.

A Step Forward for Social Science

Legislation is messy. But as this episode unfolds,

there is a lot to like for the social science commu-

nity. Our own view is that the available research,

combined with language like that drafted by McDer-

mott, does not permit a sharp distinction between

programs with the “strongest evidence” and other

programs. As the Howard and Brooks-Gunn article

makes clear, a number of programs show evidence

of beneﬁts, and no single program clearly dominates

the rest. But it must be counted as a victory for social

science that the federal policy process now hinges

importantly on evidence, a clear sign that both the

administration and Congress want to do everything

they can to fund successful programs. It also augurs

well for the research community that the McDermott

bill requires continuing evaluation of programs that

receive the bill’s funding. Indeed, the bill sets aside

$10 million in guaranteed funding, mostly for pro-

gram evaluation. The emphasis on continued evalu-

ation is especially important in light of the ongoing

The world of social science does

not speak with one voice, and

even the best evidence can lead

to multiple—and sometimes

directly opposing—conclusions.

6 THE FUTURE OF CHILDREN

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 7

debates over which programs are most effective and

concerns about whether the effects of programs will

diminish as they are scaled up.

In the scufﬂe over which programs to fund, we hope

that two important issues receive careful scrutiny.

The ﬁrst is whether home-visiting programs should

be made available only to high-risk families or should

instead be extended to low-risk families that are in

little danger of maltreating their children or providing

them with inadequate care or stimulation. In an ideal

world, it may be worthwhile to fund home-visiting

programs that make good parents even better. It may

also be easier to build political support for universal

programs than for narrowly targeted ones. Given

the budget problems facing the U.S. government,

however, we believe it makes most sense to target

home-visiting programs on the high-risk parents who

need them most and for whom the payoffs are likely

to be the highest.

The second issue is whether, even for high-risk fami-

lies, there can be a “one-size-ﬁts-all” home-visiting

program. Programs that work for ﬁrst-time teen

mothers may not be suitable for other groups of high-

risk parents—for example, those with drug addic-

tions or serious mental health issues. Funding and

then evaluating programs that target different groups

of high-risk families will make it possible to build up

new evidence on which programs are effective, and

for whom. The ﬂexibility written into the current leg-

islative draft will facilitate these efforts.

Like other participants in the policy process, research-

ers and their allies sometimes publicly disagree with

each other, even when funding for programs they all

support is on the line. But over the years, the relent-

less call by researchers, journals like The Future

of Children, and respected organizations like the

National Academies and the Coalition for Evidence-

Based Policy have convinced policy makers that evi-

dence of program success should be a requirement

for program funding. Regardless of the outcome, in

the current debate over home visiting, social scien-

tists have taken another step toward the goal of get-

ting policy makers to consider high-quality evidence

when making program funding decisions. That is a

signal achievement for the research community—

and, in the long run, for the improvement of public

programs for children and families.

RON HASKINS, CHRISTINA PAXSON, AND JEANNE BROOKS-GUNN

SOCIAL SCIENCE RISING: A TALE OF EVIDENCE SHAPING PUBLIC POLICY 7

Additional Reading

Richard Behrman, “Home Visiting: Recent Program Evaluations:

Analysis and Recommendations,” The Future of Children 9, no.

1 (1999): 4–26.

Kimberly S. Howard and Jeanne Brooks-Gunn, “The Role

of Home-Visiting Programs in Preventing Child Abuse and

Neglect,” The Future of Children 19, no. 2 (2009): 77–105.

Harriet L. MacMillan and others, “Interventions to Prevent

Child Maltreatment and Associated Impairment,” Lancet 373

(2009): 250–66.

Mary Ellen O’Connell, Thomas Boat, and Kenneth E. Warner,

editors, Preventing Mental, Emotional, and Behavioral Disorders

among Young People: Progress and Possibilities (Washington:

National Academies Press, 2009).

David Olds and others, “Long-Term Effects of Nurse Home

Visitation on Children’s Criminal and Antisocial Behavior:

15-Year Follow-up of a Randomized Controlled Trial,” Journal

of the American Medical Association (JAMA) 280, no. 14 (1998):

1238–44.

Senior Editorial Staff

Editor-in-Chief

Sara McLanahan

Princeton University

Director, Center for Research on Child Wellbeing,

and William S. Tod Professor of Sociology and Public Affairs

Senior Editors

Ron Haskins

Brookings Institution

Senior Fellow and Co-Director,

Center on Children and Families

Christina Paxson

Princeton University

Dean, Woodrow Wilson School of Public and

International Affairs, and Hughes-Rogers Professor

of Economics and Public Affairs

Isabel Sawhill

Brookings Institution

Senior Fellow, Cabot Family Chair, and

Co-Director, Center on Children and Families

Journal Staff

Executive Director

Elisabeth Hirschhorn Donahue

Princeton University

Managing Editor

Brenda Szittya

Princeton University

Program Manager

Kris Emerson

Princeton University

Outreach Director

Lisa Markman-Pithers

Princeton University

Outreach Director

Julie Clover

Brookings Institution

Communications Coordinator

Regina Leidy

Princeton University

Outreach Coordinator

Mary Baugh

Brookings Institution

8 THE FUTURE OF CHILDREN

NONPROFIT ORG.

U.S. POSTAGE

PAID

PERMIT NO. 186

PRINCETON, NJ

CHANGE SERVICE REQUESTED

This policy brief is a companion piece

to Preventing Child Maltreatment,

which can be found at no charge on our

website, www.futureofchildren.org.

Print copies of Preventing Child

Maltreatment also can be purchased

on our website. While visiting the site,

please sign up for our e-newsletter to

be notiﬁed about our next volume,

Transitions to Adulthood, as well as

other projects.

The Future of Children would like to thank the David

and Lucile Packard Foundation for their generous

support. The Future of Children is a collaboration of the

Woodrow Wilson School of Public and International

Affairs at Princeton University and the Brookings

Institution.

The Future of Children

Princeton University

267 Wallace Hall

Princeton, NJ 08544

VOLUME 19 NUMBER 2 FALL 2009

Preventing Child

Maltreatment

A COLLABORATION OF THE WOODROW WILSON SCHOOL OF PUBLIC AND INTERNATIONAL AFFAIRS AT

PRINCETON UNIVERSITY AND THE BROOKINGS INSTITUTION

A COLLABORATION OF THE WOODROW WILSON SCHOOL OF PUBLIC AND INTERNATIONAL AFFAIRS AT

PRINCETON UNIVERSITY AND THE BROOKINGS INSTITUTION

www.futureofchildren.org

The Future of Children

Child Maltreatment

Volume 19 Number 2 Fall 2009

3 Introducing the Issue

19 Progress toward a Prevention Perspective

39 Epidemiological Perspectives on Maltreatment Prevention

67 Creating Community Responsibility for Child Protection:

Possibilities and Challenges

95 Preventing Child Abuse and Neglect with Parent Training:

Evidence and Opportunities

119 The Role of Home-Visiting Programs in Preventing Child Abuse

and Neglect

147 Prevention and Drug Tr eatment

169 The Prevention of Childhood Sexual Abuse

195 Prevention and the Child Protection System