Tag Archives: Education

PISA: The Morning After

Yesterday was PISA Day, an opportunity for concerned educators and citizens to think about the latest round of results from this important international comparative assessment

Not surprisingly, at least to those of us who follow the rhetoric and reality of comparative data, U.S. performance was basically unchanged from three years ago; some other countries (e.g., Poland, Germany) improved; some of the traditional “stars” (e.g., Finland) experienced a decline; and policy makers and commentators were quick to pronounce on the meaning of the results.

PISA is a remarkable program, in terms of the breadth of its coverage (65 education systems, including selected states in the US and Shanghai as separate from China) and the care taken to provide reliable estimates of the math, reading, and scientific literacy of samples of 15-year olds.  We have come a long way since the early days of international comparative assessment, in terms of sampling methods, psychometric quality, and reporting of results. 

Interpretation, though, remains a challenge.  For descriptive purposes, PISA provides a trove of interesting information, which, along with TIMSS, NAEP, and PIAAC (another OECD project), should be studied by anyone who cares about the ongoing pursuit of improved educational opportunity locally, nationally, and globally.  The more complicated task, though, is deriving sound policy inferences from these descriptive data.  There is no clear enough pattern of relationships to infer anything definitive about the relative success of various reforms in the U.S. and elsewhere; about the relationship of test performance to national economic outcomes; or about what exactly we should do next as we struggle to expand access and high quality educational opportunities for our students. 

For example, it seems that Massachusetts again did better than the overall U.S. average and on par with some of the biggest international “winners.”  Florida fared more poorly.  So people who like what Massachusetts has been doing must be pleased, and would be inclined therefore to like what PISA measures.  People who like Florida’s hard-charging accountability reforms are surely disappointed, and some of them must now be skeptical about whether PISA is the right tool to gauge the effects of reform.  It is no small irony that some of the harshest critics of PISA (and testing generally) are willing to use the latest results to vindicate their claims about the success or failure of various reform initiatives. 

Similarly, there is the implicit (and in some cases explicit) attempt to tie PISA scores to our current or future economic stature.  Here, too, the hazards of inferring cause and making predictions from purely correlational and descriptive data are profound.  As I’ve written elsewhere, the U.S. performed near or at the bottom on the First International Mathematics assessment in 1964, and indeed our economic productivity growth declined in the subsequent decade (when many of those high school kids who had taken the test were in the labor market).  But countries that significantly outperformed us on the math test, e.g., Britain and Japan, experienced an even more dramatic productivity growth slowdown, suggesting that those test results could not, alone, explain much about the economy or predict much about the future.  Average annual labor productivity increased by about 4.2% in the U.S., between 1979 and 2011, compared to 2.5% in Germany, which outperforms us on PISA.  Again, it’s not easy to infer simple relations from these data.  Does academic achievement matter to economic outcomes?  Yes, without a doubt.  But so much else matters, and probably more, that to draw quick inferences – and again sound loud alarm bells about our impending economic doom and, worse yet, blame everything on schools and teachers – from results on one assessment of one age group is a recipe for the ultimate erosion of respect for what is otherwise a useful tool for comparative study.  I’ve tried to make this argument elsewhere.

Finally, there is the question about what to do next.  On this, I was inspired by the wise comment of John Jackson, of the Schott Foundation for Public Education, whom I met at a dinner with a number of education policy makers and researchers.  John asked whether it was possible that we in the U.S. had essentially “maxed out” on the impact of reform in terms of its effects on PISA scores, and if so what should guide us as we continue to work on educational improvement.  That’s the right question, and though I believe there is useful information in PISA I also believe that answering it will require considerably more nuance in our understanding of the results.  (To get a sense of the complexity of these issues, see the recent work of Martin Carnoy and Richard Rothstein). 

For me, the most important issue to focus on is not where we stand on average, but rather how to cope with the ravaging effects of growing economic inequality on educational opportunity and the life chances of our youth.  In other words, we need to work on the variance more than the mean, to acknowledge the effects of poverty, and to concentrate on policies and programs that can restore opportunity (my colleagues Richard Murnane and Greg Duncan are co-authors of a book with that title, due out early next year).  A good place to start would be with a sustained program of research and policy that builds on the foundations of work such as Whither Opportunity?.  If we care about the American dream – and really want to reaffirm the nation’s commitment to upward mobility and improved quality of life for all our people – we should not distract ourselves with foolish attempts to use a single assessment as the guide to policy and practice.  PISA poses important questions; the answers aren’t as obvious. 


MJF
December 4, 2013

Advertisements

Leave a comment

Filed under Uncategorized

Reflecting on a New Challenge

I was honored by my colleagues to be inducted as the President of the National Academy of Education at a ceremony last Friday.  My wonderful family was at my side as I accepted this honor – and though I don’t get to tell them often enough, they deserve great thanks for their abiding support.

It will be a great pleasure to work with my GW colleagues, NAEd members and leaders, and other friends in the broad education research and policy community as we continue to advance the cause of improved education and increased opportunity. A challenge, no doubt, but one that I feel privileged to take on.

President Knapp (and Diane Knapp), as well as Provost Lerman (and Lori Lerman), were kind enough to attend the ceremony, as were a number of our senior leadership faculty and staff.  I’m grateful for President Knapp’s most generous introduction and welcome.  GW Trustee Titulola Williams-Davies and former trustee Bob Perry also joined us, and it was again a joy to see their enthusiasm for what’s happening in GSEHD.   And it was a special treat to have Undersecretary of Education Martha Kanter in our company.  Let me add my very special thanks to members of the GSEHD National Council who were present:  Laura Taddeucci Downs, Ericka Miller, Elizabeth Lodal, Kathy Manatt, Dorothy Moore, and Ed Vest continue to grace us with their wisdom and support.  My heartfelt thanks to all.

Following a memorable and moving introduction by my dear friend and mentor, Carl Kaestle (whom many of you will remember of course from his masterful “master class” earlier this year), I offered some reflections about the Academy, the world of education research and policy, and my hopes for the future.  As I contemplate this new role, I am glad to share a slightly edited version of my remarks from Friday’s induction ceremony.


MJF
October 29, 2013

[Abridged remarks from NAEd ceremony on October 25, 2013]
I have been thinking about the name of our organization.  Although I was an English major I am not really a big fan of deconstruction; but it is interesting to think about the words in our name, especially in the current policy environment.

The word “national” evokes, for some people, and maybe for more people today than in previous times, an anxiety about the excesses of federal involvement in those sacred rights of states and localities and individuals to keep doing their own thing.  It’s gotten to the point where one wonders, listening to the shrill rhetoric, if anyone was in class on the day when the principles of individual choice and social well-being were taught.  What can we do to remind ourselves that education is essentially a “public good?”

Continue reading

Leave a comment

Filed under Uncategorized

EdWeek Commentary Condemns Misdirected Blame in Atlanta Cheating Scandal; Garners Mixed Reaction from Readers

Reactions to my recent Education Week commentary on the Atlanta cheating scandal have been interesting (see also this Washington Post blog post by Valerie Strauss that highlights my commentary). They mostly relate to the question I raised, whether the system is to blame or whether individuals, even when faced with strong pressure and incentives for opportunistic behavior, should act morally and legally.  And, of course, several of the commentaries rehearse the somewhat well known criticisms of testing.

As best I can tell though, readers didn’t take up the issue I mentioned regarding the National Assessment of Educational Progress (NAEP) scores, which apparently improved during Beverly Hall’s tenure in Atlanta.  If student performance was really getting better, then there was arguably less reason to engage in the alleged tampering.  We won’t know whether the people accused of the cheating considered any of this, or indeed if most of them knew about the NAEP results in the first place.  Meanwhile, suspicions have been raised about whether there had also been mischief in the NAEP sampling and scoring. For an especially lucid analysis of the NAEP results and how they relate to the Atlanta situation, I refer you to this excellent article by our friend Marshall (Mike) Smith, former Under Secretary of Education and Dean of the Stanford Graduate School of Education.

MJF
April 15, 2013

Leave a comment

Filed under Uncategorized

STEM Education and the Condition of American Science

I have written elsewhere about my doubts concerning the validity of arguments that link US economic performance to achievement test results (see William H. Angoff Lecture Series Report; The Huffington Post; GSEHD Working Paper 2.3).  And in my last posting I had the pleasure of welcoming guest blogger Sharon Lynch, whose innovative work on STEM education is spurring considerable discussion and addressing some of the most important education policy issues we face.

Recently I had the honor and pleasure of serving as guest editor for the current issue of The Bridge, the flagship quarterly of the National Academy of Engineering.  In my introductory note I suggest that there are two compelling and compatible narratives about the importance of attention to STEM (and I’m grateful to my friend and former colleague, Naomi Chudowsky, for helping me understand and articulate these ideas).  The “national need” narrative is based on the hypothesis that the future of American economic competitiveness, especially in a changing global environment, hinges on an increased supply of well-educated scientists and engineers, qualified and ready to work in demanding jobs that require higher order science and math skills.  The second narrative, what might be called the “equity narrative” emphasizes the glaring and persistent inequities in the distribution of educational opportunities and the deficits experienced by underrepresented minorities and women in the STEM fields.

Continue reading

Leave a comment

Filed under Uncategorized

STEM Education: The Pursuit of Excellence and Diversity

Featuring Faculty Guest Blogger Dr. Sharon J. Lynch

Participants in our 10th annual Educational Symposium for Research Innovations (ESRI), or at least those who were lucky enough to wake up early on Saturday morning for the keynote address, had the extra bonus of hearing Dr. Joan Ferrini-Mundy’s superb talk about the National Science Foundation’s commitment to improved STEM (science, technology, engineering, mathematics) education at all levels.   The timing of ESRI and of Ferrini-Mundy’s visit was exquisite, coming just a few weeks after President Obama reaffirmed that the state of our union hinges on continued investments in education to meet changing job and skill needs of the coming century. (By the way, major kudos to Professor Rick Jakeman, the faculty supervisor for ESRI, and to Chris Harriss, Student Chair, for pulling off yet another outstanding conference!)

Today I am happy to welcome my first faculty “guest blogger” to this site, Professor Sharon J. Lynch, who is involved in critically important research on STEM education.   Here Professor Lynch reviews preliminary findings from her NSF-funded project.  I urge you to read on for her thoughtful summary, which is not only substantively rich but also is a model for the ways in which GSEHD strives to connect rigorous research to improved policy and practice.

MJF
February 25, 2013

Continue reading

1 Comment

Filed under Guest Blogger

Pulling Rank

U.S. News and World Report has decided to reclassify GW as “unranked” in light of the University’s disclosure of an error in the way one statistic, percentage of incoming freshmen who graduated in the top 10% of their high school class, had been reported.  This unfortunate move on the part of USNWR is explained by its director of data research here.

I’d like to offer a few thoughts about this story to our students and their families, and to our faculty, staff, national council members, current and future employers of our graduates, alumni, and other friends in the community.

First, we can all be proud of the way President Knapp, Provost Lerman, and Vice Provost Maltzman handled the discovery that data had been reported incorrectly: they came right out and said so, voluntarily, quickly, and with complete transparency.  We should applaud this decision, which included a full independent audit and restructuring of internal oversight procedures, because it makes clear that for GW ethics precedes expediency.  As President Knapp notes in his public statement to the community, “we [disclosed the mistake] without regard to any possible action that U.S. News might take as a result…”

Continue reading

3 Comments

Filed under Uncategorized

Some Random Thoughts Post November 6

Regardless of your ideological or political preferences, this year’s election results were a powerful reminder of what’s special and perhaps unique about the American system.  Just over the bridge from here, “swing-state” Virginians swung toward President Obama and elected Democrat Tim Kaine to be their Senator, while in the Richmond area they overwhelmingly voted for Republican (majority leader) Eric Cantor.  Up the road, our friends in Maryland approved gay marriage, rights for immigrant children – and casino gambling.  Californians, seemingly fed up with years of education cuts, supported Governor Brown’s plan to raise taxes to support needed reinvestment in what was once among the best university systems in the world; and they rejected the proposal to abolish capital punishment.  In Florida, where we still are waiting to see where the state’s swing will land when the final votes are counted, an initiative to overturn the prohibition on public funding of religious organizations failed; as did a proposed amendment that would have banned use of state funds for abortion.  (Maybe I’m out of touch, but I don’t think I would have predicted these outcomes from Floridians!) Voters in DC expressed their concerns for good government by approving draconian measures authorizing the Council to expel members for “gross misconduct,” requiring the mayor or a council member to resign immediately if convicted of a felony, and prohibiting elected officials convicted of felonies from ever holding office again.

Continue reading

Leave a comment

Filed under Uncategorized