Measuring Long-Term Impact of 21st Century Learning

I was traveling with my kids this weekend across two states and the majestic views of the plains and the west. If measured correctly at the end of the drive, or even the end of the year, I am sure that there is no measurable impact on their standardized test achievement. Does that means they should stay home? Or conversly that we shouldn't teach 21st century skills.

The same question can be asked about any field trip, shadowing, or performance students are exposed to. Does that mean we should not do them? If the answer is no, we should continue doing these things as a part of a whole child education What then is our verdict about the quality of most of our educational research?


If we focus on short-term simple effects right after an intervention of any kind, we may well be missing two things: the long-term impact (or lack thereof- what Calfee called the poop-out effect), and exploring the impact of hard to measure meaningful activities.

How would we measure the impact of Day of Code, Read Across America, or Project based Learning?

I would argue that we need new paradigms, new instruments, and a vivid imagination exploring what outcomes of note can be. The relationship between researchers, teachers, students, and community members should change. The goal of research should change and become a cooperative endeavor that requires different structures than we have now. For example, a school can have a resident researcher who teaches and conducts a design experiment that serves school goals, as well as increasing our research knowledge. This is especially true of digital and other 21st century skills one's we know very little about. Still thinking about it...

Five ways to help students find the Sweet Read

Today is Read across America day. This morning I watched my son Oren (11) read right after he got up. He is rereading for the ??th time the Percy Jackson series. It made me think about a book or a series of books that becomes a second home to readers- a Sweet Read. It is about the power of returning to a favorite book where the charaters are like old friends that you have known for many years.

When I was Oren's age I had Asimov's Foundation series and later Lord of the Rings. For Erez (now 20) it was Harry Potter, Asaf (now 18) had David Edding's Belgariad. Even as adults I find that Sarah and I are returning to favorites and redefining what is our literary home.

This is an experiece I would like all students to have- a sweet read- that is yours. Here are a few completely unoriginal ideas to help students find the Sweet Read:

1. Read outloud to them. Reading outloud connects readers and listeners to the books in a way that helps new and struggling readers reimagine what the reading experience should be.
2. Provide choice. I learned the hard way that what you love and find a home in does not necessarily mean that others will as well. We all must figure it out for ourselves.
3. Do not judge. The Box Car Children can as much of a literary home as Boewolf. It is after all about comfort and joy.
4. Share you experiences. Share your passion for books, not so they can read the same books, but so they can feel the emotion and excitement in your voice.
5. Library often. (shouldn't library be a verb?)

Full disclosure: In the last five years or so my Sweet Reads are A Hundred Years of Solitude and Bitchfest.

What should Classrooms look like?

Last week I had the chance to speak at the E N Thompson Forum as a warm up act for the inspiring Milton Chen. I decided to talk about what creativity looks like and can look like in Elementary classrooms. To start off I asked my audience to draw what classrooms should look like. The results (I am attaching examples of work in progress) were very interesting.

What was very clear from the work is that most of the people showing up thought classrooms should look very different than they do now.

The question that occured to me was: if there is a growing concensus that classrooms should be more open and integrated why aren't we trying to do something about it? The example of Pegasus Bay in New Zealand sticks in my mind as an option for new schools but I believe that there is much we can do inside older buildings as well.

I believe that with a bit of ingenuity we can do a lot. It is NOT a replacement for excellent teaching but it provides an environment that encourages teacher and students to learn and teach creatively.

Reflections on China Tour II- When the Stakes are High

Here in the US we often talk about High Stakes tests and their impact for example the work by Berliner and colleagues. In China the conversation about High Stakes was very different.

We had lunch on our last day in Shandong province hosted by one of the lead directors of the school district. After the exchange of gifts and pleasentries we had an inetersting discussion which started with his concern for student well being. He relayed that families are putting too much pressure on students to excel within the system, pressure that may harm some students maybe all. At the same both of us acknowledge the immense impact educational success measured by tests can have on individual lives.

We saw the importance of high stakes testing in almost every conversation with teachers and parents. Our research in China (With Stephanie Wessles and Guo Ji) is looking at the interaction between school and family. We had a chance to see the interaction in our first meeting with parents. Teachers took charge and directed parents who, in turn, complied without question. The parents were professionals from a middle class background but they followed teacher's demands. In the US middle class parents would have responded very differently probably actively resisting what they did not like and asking for a voice in the discussion. Here in China it was different and we were intrigued by it. In conversation some have speculated that this was part of the culture and Confusian ideals. Culture may have had something to do with it, though in private conversations and interviews parents were often critical of teacher's actions and did not think that teachers "knew better". The question that emerged was why parents did not resist what they thought was bad practice?

The answer seems to be linked to High Stakes. In China high stakes are meaningful most often to the individual. Starting very early students take tests that are critical for their advancement into the next level. There is a middle school test, high school etc. Each one of these has potentially dire implications for the student and his/her future path. The High Stakes for students and their family (pressure is intensified by the one child policy) create a need to comply. Parents relayed to us: "I do not always agree with the teacher  but I will not say anything because I fear there will be negative outcomes for my child." In the large classrooms (we saw elementary schools with 40-50 students) teachers cannot attend to all student needs. Each parent is keenly aware of the high stakes and the positive role the teacher can play, thus they do not want to rock the boat fearing that their students will be ignored or underserved.

In this case the impact of high stakes testing is a lost voice for students and parents who should be part of the conversation about education. This is not all one sided. This very same situation helped our efforts to integrate iPads into classroom instruction. Not all parents were in favor and a few worried about it but none resisted it This gave them an opportunity to see the impact on their students. After parents saw the impact they were decidedly positive. This is similar to the model Guskey suggested for teachers.

Generation D- the impulse to re-engineer

I was playing a quiet game of Candy Crush yesterday and my 8 (soon to be 9) year old son Itai came and sat next to me. For me, casual games like Candy Crush are a great way to pass a few minutes and do some problem solving. Itai, however, is generation D (digital) child and reacted to the game in a very different way.       

As I was olaying Itai was making suggestions about moves and figuring out how the game worked. Finally he said: "Wouldn't it be great if you could design your own board and could decide where the jelly and chocolate went?" He continued musing: "you could design your own special candy like a cross between the fish and chocolate". His stream of ideas went on as I was playing and I cannot remember them all, but what I do remember is how easily he has focused on the creation side. 

This of course is not accidental. I have been observing in schools and at home the impact of games like Minecraft and Little Big Planet. For adults they are games, but I argue that for kids they create new ways of thinking. As a result generation D maybe growing up the most creative one yet, a generation that has a creative instinct. A generation that idenftifies a problem and doesn't just want to solve it, they want to re-engineer it. The question for us is how do we design schools that cultivate and support this world view?

First Reflcetions on our China Tour- Common Ground

Photo Op in a First Grade Classroom Linzi, Shandong, China

We came back from our China Tour two weeks ago. It was a whirlwind tour focused on Linzi in Shandong province and Chengdu in Schizuan province. In each we visited elementary schools that are integrating iPads with our support. There were many lessons to learn and I am still thinking through much of what we saw. In addition we visited only three schools and observed instruction in only two. There is no way for me to know how representative this ample is so please read with great care. This, however, are some of my observations:

There are many differences between US and Chinese schools. For example Chinese classrooms were much larger (over 40 students), and the stakes to students future are higher (high stakes in China is much higher stakes for students not teachers. What struck me though were the similarities. When we observed teaching, our Chinese partners and us were often in agreement about high quality instruction and what it should look like. In our last school after three days of work the principal asked to see me privately. She sat opposite me with her four assistant principals (one each for instruction, professional development, organization, and discipline) and with a tense expression asked for my opinion on the instruction we saw. I laid out a step by step analysis of the lessons (I used LessonNote to annotate lessons carefully). At the end of my exposition she was visibly more relaxed. Smiling she asked: "Do you think it is possible to integrate technology into our traditional lessons?" [translation].

Earlier in our visit I thought traditional meant a focus on memorization and recitation, but at this point it has become clear to me that she was referring simply to the existing curriculum. This is the same question/ concern I often encounter in schools. Teachers and administrators interpret our effort in professional development as an addition or even substitution of the existing curriculum, the reality is that we see it first and foremost as part of the curriculum already taught with some extra skills integrated when they are relevant (e.g. digital citizenship). I carefully responded that yes I thought there could be such integration that would benefit students and help instruction as well as 21st century skills. I went back to the SAMR model as a core foundation to move forward and for the first time since we entered the school we were on the same page.


At the heart of the matter was the fact that both sides did not understand how close our positions were. We were seeing the same instruction and evaluating it in similar way but all of us were also hung up on cultural differences not wanting to assume common ground that was actually there.

Losing Faith in Journalism- a response to "Can Students Have too Much Tech?"

My dean directed me toward an opinion article in the prestigious New York Times by Susan Pinker. The title was "can students have too much tech?" Who can resist this title? Of course you can have too much tech- thinks the person reading this on her iPad seating at Starbucks on a staurday morning. Kids these days all they do is play video games and waste their time texting.
A closer read of the article actually disproves the main thesis quite clearly. I expect more from a published author and a psychologist by training! I almost never comment on writing like this. In this case, however, I am mostly because we all expect better from a publication like the New York Times.

I would like to say that I agree with some of the premises in the article namely:
1. It sucks to be poor. Children raised in poverty have lower outcomes on standardized tests.
2. Devices are no magic. It depends what you do with them. Duh.
3. We still need teachers to teach even if we have devices.
4. Putting crappy devices in students hands without support will do very little to improve academic outcomes (sorry Sugata Mitra I am not a believer).

While we definitely need to be careful about technology use and balance in this just like any other facet of our lives a careful of the article and the sources cited bring a totally different picture.

Here is what is inflammatory, cherry picked, and untrue:

The story starts with the Obama initiatives on free and open Internet and providing access. Both policies are crucial for long term success of our educational system and social justice but are also completely unrelated to the evidence cited later. The critique is mostly about tech use at home

(where they gathered some correlational data) but the implication is that the president¹s agenda in this area is wrong.

The main concerns I have about the data presented (you can read the report here):

1. There is no consideration that technology is an area of literacy that is just as important than any other. Without computer/Internet literacy students are behind (if you can't conduct an excellent Internet search for research- how good is your research paper going to be?).

2. They basically point to an interaction between poverty and home access to technology. Does that mean that all kids are better off without access at school or even at home? Is she advocating letting middle class students have access at home but not for African American boys? Really?

3. The data is old and it predates smart phones, high speed internet, and the wide array of educational resources avaialbale and required in education (for example GAFE). Smart phones are now ubiquitous and most students from mid school up have them- including children growing up in poverty. That means that access is already there all that is left to schools and parents and to try and channel the activity to educational benefit as well as social and entertainment.

4. Now for the main source of data. The report is from 2010 the data is from 2000-2005 (what tech did we have then?). The report is by two economists. They actually claim: (1) that students that always had a computer actually improve over time (2) some students do better after they get a computer and finally and 

5. Most importantly their effect sizes are all in single digit % effect size that is for them an effect size of .02 standard deviation is fairly large. In educational research any effect smaller than .40 (that is 20 times higher than that reported). This effects are considered  small and not educationally meaningful.

The clearest part of this is that the author has failed at critical reading and thinking. She does not (want to?) understand that the devil in any report is in the details and in complete reporting including the context of a decade old study. It is legitimate to have concerns, it is also legitimate to question the ways technology can be used. Support for the argument should be based on a reasoned argument, and facts that are relevant to our current context.