AI and Academic Publishing

 Like many others, I have been playing with generative AI for the past few months. I am an author of scientific papers and, even more so a frequent reviewer. I have been elated by the potential of generative AI to bridge the gap between English knowledge and conducting high-quality research, especially for international scholars whose first language is not English. This is an opportunity to level the playing field and allow equal access to academic publishing, which is predominantly conducted in English. Many times I have reviewed articles with good ideas but really hard-to-understand language that required many rounds of review and editing before it was publishable.

On the other hand, generative AI is quite as capable of generating data that isn't there (often referred to as hallucinating). For example, after being asked about my publications Chat GPT 3.5 spit out this list: None of the publications are real! This will require our publication engines to allow us to track every in-text reference with quick access so reviewers can check the veracity of such claims that may be "halucinated".

Even more challenging is generative AI's ability to "hallucinate" research studies. In a manner of a few minutes, I was able to have Chat GPT generate two potential studies about reading instruction (synthetic phonics and reading recovery) with ANOVA designs, including result tables. I even got Chat GPT to design and execute a study about the impact of a Wind Surfing intervention on Math achievement of second graders. For example, examine this paragraph generated after I requested a qualitative study instead:

"As this study focuses on qualitative exploration, the quantitative results will not be the primary focus. However, to provide a broader context for the qualitative findings, basic descriptive statistics of math achievement scores may be reported for both the windsurfing instruction group and the control group. These scores will be collected through pre- and post-intervention math assessments administered to all participants. The quantitative results will be used to complement and contextualize the qualitative findings, providing a broader perspective on students' math achievement in relation to their windsurfing experiences."

I am sure that generative AI will create an increase of papers submitted for publication. To prevent science from being overwhelmed and suspicious we may need to write new rules and accelerate existing trends.

1. Demand researchers pre-register their research.

2. Ask that each paper submitted will include a statement about the use of generative AI and will include the transcripts of their use.

3. Create ethical standards for AI use in scientific publishing AND teach about it in graduate schools.

4. Create reviewing mechanisms that allow easy tracking of citations to the source.

Voice Assistants

Our spaces our filling with voice assistants, Google, Apple, and Amazon all have created an interface that allows users including young learners to interact without needing keyboards or even touch screens. I believe that we need to launch a serious effort to see what are the affordances and unintended consequences of these devices to learning as become increasingly ubiquitous.

While some research has begun, more common are anecdotal observations by parents researchers, and teachers. For example, earlier today in a conversation Ji Guo, a Doctoral Graduate and current colleague brought up the fact that his son was using google to ask questions as he was reading. His observation was that his son was leaning on google to clarify as a "while reading" strategy. Instead of stopping his reading to answer a quick question (e.g. how large is a Dolphin), he could ask and get a quick answer and keep on reading without getting any further distracted by the interaction with a screen. I love this example because it parallels the use of digital dictionaries embedded in digital texts. Both allow the reading to continue quickly and with minimal interruption while allowing the learner to collect further information.

The main danger described by parents is that students start relying exclusively on the assistant to supply information that students have yet to internalize, which is still important. The first example is multiplication. Google, Amazon, and Siri all can give quick answers, but understanding the concepts behind multiplication is a key numeracy skill that all students should acquire. In this case, the assistant can create false learning paths that will undermine the future development of learners. The answer of course is not to resist the use of devices but instead to think about the ways and times they can use it. This is especially true since many years ago we had the same discussion about the use of calculators in classrooms. 

I am excited to look for researchers looking into this new area for exploration!

Clean Your Windows so you can see the fireflies

 I am continuing my journaling journey. This week I started thinking about my learning for the week while cleaning the windows in my living room. One of my sons came in and asked: Don't you have people that do that? Yes, I answered: but I love cleaning the windows occasionally. I get so used to the dirty-ish window that I stop noticing it. I sit in the chair and look outside, accepting the dirty window as part of the view. I literally forget that it is just a distortion that I have some power over and that I can remove. 

As I was cleaning, I realized that it was an interesting metaphor that calls on me as a researcher to stop, slow down, and examine what in my process of looking at the world needs cleaning. Is the distortion I see a result of dirt/noise in my control? This can go to weak beliefs and theories that stop me from seeing clearly. It can be unrelated (yet powerful) emotion or just constant activity that prevents me from realizing what I need to be paying attention to.

This may also be true of the devices and apps we use to see the world, algorithms, scanning, and attentional processes obscure what there is to see. Once in a while, we need to stop and clean our windows making sure that we are doing our best to see what is out there. Making what we are seeing is not just the distortion on our window.

This is my Journal page, I noticed that my processing has many more questions than answers or solutions. It could very well be that many of the questions are the ways I am scaffolding my process, or it could be that this early in the research into Art TEAMS, there are questions with answers pending. Leading to one of the only declarations: I have more questions than answers.

I am still leaning on the firefly metaphor. Systemic change is tough, and most of the time, efforts to innovate and make change are limited to our immediate environment. The light fireflies make is the light of individual change agents. While making the world better for others (very few for a short time), we are also looking for others like us to collaborate with us. 

In many ways, the grant is trying to help new fireflies increase their signal, find their light, and join the other fireflies. Yes, there is a slight chance of systemic change, but even if that does not happen, we change ourselves and the lives of our students. The role of projects and universities is to create communities of fireflies. Places where they are safe, cherished and supported, so they can continue.

\

Wikipedia First: New Rules for Online Research

A recent article in Wired revisited how attitudes toward Wikipedia come face to face with the reality of Wikipedia. It is the world's eighth most visited site, it is free, it is not monetized. Students still recite that you cannot get information from Wikipedia- that it is not a reliable site.
In short, there is a gap between what we say "Wikipedia is unreliable, and the information on it cannot be trusted" and the reality that we all use Wikipedia and often for all the right reasons.

One of the common critiques about Wikipedia is that consensus might not be the best method to determine what is "true." That is a compelling argument and not without merit. It is, however, precisely what researchers do. Send papers to scrutiny "peer review" and research results become "true" when most of the research community thinks they converge, and we reach consensus until new information disrupts it.
In fact, a whole section of the Conceptual map about the Nature of Science (bottom right in the figure). Has to do with the community.

I will not repeat all the support and critiques of Wikipedia - you can read those on Wired in the original. I would like to instead suggest an addendum to how we treat Wikipedia when we teach students about Information Technology.
Let's make it Wikipedia first, never Wikipedia only. We always need to corroborate any information, but Wikipedia, with its basic information and the next set of links, can launch a search that is not guided just by the commercial and parochial interests of the monetized search engines only.
Wikipedia first

Why Silent Reading Rates and Stamina are More Important than Ever!

Freddy Hiebert currently of TextProject, Emily Hayden of Iowa State and I recently published an empirical paper about patterns of silent reading. This is the work we have been pursuing for quite a while. Studies of silent reading in the context of reading development is a crucial element in expanding our understanding of reading in the context of new literacies. Reading in the 21st century is quickly becoming a substantially different task than before- a task that, as educators and researchers, we need to explore. The reading task in the age of new literacies combines much of the old skills (e.g. decoding, word knowledge, fluency)  with newer challenges emerging from distractions, nonlinear texts, and the richness of multimodal texts.

I believe that one of the least understood elements of reading in the 21st century is persistence. This is inherently a different challenge than it was as little as twenty years ago. Modern readers seem to be almost assaulted by distractions and behaviors that challenge their attention to a continuous text. At the same time, comprehension of complex texts can emerge only from sustained close reading. In this, I go back to the notion of motivation as the ability to sustain attention to a specific task- reading.

The results of the study show that many students do not persist in reading even when they comprehend well. At the same time, it is clear that students that do persist in reading with comprehension adjust their reading rate after one text section and then slowly increase their reading rate. These patterns show the benefits of persistence and highlight the challenges of distraction. If students are distracted they will attempt each piece of the text as if new, unable to use the feedback from the previous section to adjust their reading rate to ensure comprehension. It is very likely that such behaviors make reading considerably less efficient and thus increase the odds that readers will not persist reading longer texts with comprehension.

I believe we have two challenges, the first is to research reading in modern contexts and the second to develop a framework for instructing silent reading with persistence and comprehension. From our conclusion:

"...we believe that the most pressing issue within reading instruction at present pertains to instructional tasks and interventions that support silent reading proficiency. One of the few projects on supporting silent reading within classroom settings in the archival literature is that of Reutzel, Fawson, and Smith (2008). This study, conducted with third graders, showed that a treatment of silent reading produced similar results on assessments of ORF as oral reading practice... To date, we have been unable to find a framework for designing tier-one, classroom instruction that begins in the early grades and ensures that students develop strong patterns of silent reading. If we are to prepare students for the tasks of the twenty-first century, such frameworks for instruction are urgently needed." (Hayden, Hiebert, & Trainin, 2019)

Three reasons ed researchers should create digitally

I have a large digital footprint with hundreds of blog posts and video series with over 100,000 views. In addition I put everything I publish online without pay walls the second I am allowed to. I have profiles on Academia.com, LinkedIn, Google Scholar and Researchgate. If you are an educational researcher you are most likely to have less of an online  presence. Many researchers do not have any. Here are three reasons to have a meaningful online presence:

1. You gain readership. Let's face it, most professionals and amateurs start learning about any topic using a simple google search. If you want to find your audience and your audience to find you, you MUST be online where they can find you. Once they find you (through a piece you wrote, a blog post etc.), they can follow up on anything else you published on that or any other topics. They might even register to follow any updates you make. This is a great way to connect and have an impact. Because:
2. Educational research is highly contextualized. As a result it has limited shelf life. That means that you need to reach your audience quickly. In a few decades (or even less) contexts changes enough to render many of our conclusion invalid. If no one consumes (read, watch, listen) to ideas, and results now they may be obsolete by the time people find them. Which brings me to my last point:
3. We need to talk to a wide constituency. It includes students, teachers, administrators, policy makes and the public. Writing for a wide audience is much more effective through digital channels that give everyone free immediate access to research findings and thinking.

The four things I wish tech innovators would do & discuss now

ISTE is now over, the keynotes done. The trumpets have quieted down. I have to be honest, I lately leave conferences with a sense of hunger, not for more but for something else. This sense got a direction when I met with Evi this week. So here are four things I would like to hear and see:

1. Educators who are staying with an innovation for a while. Too many of us are running from one innovation to the next permanent novices in a rush to get to the next big thing. From iPads to flipped classroom to Minecraft to coding to makers space. There is great value in repeating refining and becoming an expert.

2. Patience to train everyone, not just a select few. It is connected to the first section. If you stay with the same good innovation as a school/ district then you can get most teachers to join you. If, however, you keep moving the target it makes it hard for many of the teachers to join.

3. Research- innovation is nice but how do we measure its impact? If we really believe that technology can make a real and lasting difference we must insist that innovation comes with research and evaluation.

4. Race and inequality- many even most of the presentations and innovations ignore the social challenges we face as a nation every day. It is time, that race and inequality were a part of our discussion and action.

I believe that it is time for us to move beyond the excellent work done so far- up to the next level.

Why I love teaching in the summer

This post is an ode to summer teaching. As I do most summers I have a full summer load. Many of the people passing me in the corridors are only too happy to remind me that it is summer and a time to take a break especially given that most of us work on a 9-month contract.

I usually reply that I have two sons going to college next year and other acceptable financial comments. The truth, however, is more complicated. I like teaching in the summer. The intensive time spent with practicing teachers and budding researchers is probably the best professional development I get all year.

Most of the students in the summer are practicing teachers who have, for a change, time to reflect and think long-term. Most of what I teach this summer is linked to research methods and inquiry, as a result graduate students are bringing their own content and questions challenging my thinking and adding connections and ideas.

I read, teach, and prep most days- the result is that my brain feels on fire, in a good way.

Measuring Long-Term Impact of 21st Century Learning

I was traveling with my kids this weekend across two states and the majestic views of the plains and the west. If measured correctly at the end of the drive, or even the end of the year, I am sure that there is no measurable impact on their standardized test achievement. Does that means they should stay home? Or conversly that we shouldn't teach 21st century skills.

The same question can be asked about any field trip, shadowing, or performance students are exposed to. Does that mean we should not do them? If the answer is no, we should continue doing these things as a part of a whole child education What then is our verdict about the quality of most of our educational research?


If we focus on short-term simple effects right after an intervention of any kind, we may well be missing two things: the long-term impact (or lack thereof- what Calfee called the poop-out effect), and exploring the impact of hard to measure meaningful activities.

How would we measure the impact of Day of Code, Read Across America, or Project based Learning?

I would argue that we need new paradigms, new instruments, and a vivid imagination exploring what outcomes of note can be. The relationship between researchers, teachers, students, and community members should change. The goal of research should change and become a cooperative endeavor that requires different structures than we have now. For example, a school can have a resident researcher who teaches and conducts a design experiment that serves school goals, as well as increasing our research knowledge. This is especially true of digital and other 21st century skills one's we know very little about. Still thinking about it...

Bob Calfee- A Mentor

Robert Calfee 1933-2014

Bob died last night. Bob was my mentor, the kind that sticks in your head long after you moved out of state. I remember the first time Bob spoke inside my head. It was 1999 my first AERA in New Orleans. I went to a session about early reading acquisition. Mid presentation by one of the leading researchers in the field I heard Bob's voice and unique cadence "It's articulation stupid".

Bob has taught me to think about variance, his metaphor of variance as a sausage still lives whenever I teach a methods class. Probably more than anything else Bob showed me how you can manage multiple projects and ideas by switching mindset. I remember watching Bob make the switch. Our meeting time was 30 minutes and when the time was up Bob simply moved to the next thing. We were still there in his office finishing the last details but he has already moved on.

I never accounted really for just how much I've learned from Bob, his analytic approach, his passion, his ever present mentorship.
A colleague just wrote me a note saying we should have our mentors forever. My first thought was, we will.

Finally I remember Bob giving me and Sarah money for dinner at the Mission Inn on our anniversary in those day of graduate school poverty. I would say rest in peace, but that too was not Bob's way.
He voice will always be with me.

iPads in China- Excerpts from the Chinese media (loosely translated)

Working in China exposes the cultural differences AND the similarities of concerns. Despite all the concerns and challenges our project just won first prize in a National competition for Technology Integrated classroom. This is a great boost to our work and I am excited to continue.
I think that in the following excerpt from Chinese media in Chengdu you can see what concerns the Chinese public and how my comments are interpreted.

WCC: With the introduction of technology into traditional teaching, whiteboard, book bag, IPAD all applied to the classroom, how do you see the development proceeding? 
　　Dr. Guy Trainin: Today's kids are exposed to smart phones, computers every day. Their parents and teachers are still from the 20th century. Without technology the teacher, the school can not meet the needs of 21st century child's development. So the idea of how we can use technology to help teachers to teach  21st century kids. 
　　WCC: Chinese schools require the exam, how will students do on traditional exams? Do you have parental support? 
　　Dr. Guy Trainin: In our classroom (with Du Yu as teacher) students have mastered more words, electronic production than other classrooms, their overall quality has improved significantly. Support from parents is not difficult to imagine, as long as parents to see the students really active and growing, parents will be supportive. 
　　Today, young parents are more willing to accept new ways of education. If schools do nothing to change the direction, either to promote any new technology or method, students will not be ready to learn and work in the 21st century. Technology integration with our project TechEDGE has been practiced for several years in the United States, transfer to other countries with different national and cultural backgrounds, ideas differences, makes us need to find a new path to our ultimate goal and effect. 
Link to original story.

Testing Teachers: Arts and Technology Integration

This week I was invited to participate in a state panel examining which test Nebraska should use as one of the criteria for certification. Teacher testing has become very popular across the states with encouragement from the office of education. There is very little evidence that such tests are connected in any way to teacher quality. For example in a recent report Angrist and Guryan (2013) say: "The results suggest that state-mandated teacher testing increases teacher wages with no corresponding increase in quality." The tests, however, are apparently here to stay and even Nebraska usually one of the last holdouts on testing has decided to cave in.

Nebraska has chosen to work with ETS and our task at the panels was to review from a selection of tests and make a recommendation about which tests are most appropriate and what should a cutoff score be. One of the more relevant options we considered was the Parxis II with emphasis on pedagogical decision making. As we read through the items (which I cannot disclose) I found that quit a few addressed arts integration through theatre, movement and visual art. It was clear that integration ideas were well integrated (at least into the version of the test I saw). 

Technology was mentioned in two items only. The technologies mentioned were: looms and books on tape... There was nothing that incorporated Internet searches, evaluation of Internet resources, reading on screen, or any of the other skills mentioned in our state standards, the common core standards and professional organizations. Now, I know there is no consensus over what exactly do new teachers need to know, but no technology integration, no reference to digital modes of literacy?

We made sure our concern registered. I worry because tests (even marginally reliable ones) cause some educators to "reverse engineer" their curriculum. We need more about technology integration in our pre-service programs not less. As for the tests, they need to adapt quickly to these changes to stay relevant.

Home Literacy Environment in the Digital Age

Recently I have been working on home literacy environment and came across the Home Literacy Environment Questionnaire (HLEQ) by Griffin and Morrison. The measure was designed in 1997 and addresses paper based print only. In only 15 years the measure has become less and less relevant.
This brought me back to an observation that Berliner, perhaps my favorite educational thinker, made in an Educational Researcher piece. His claim was that some social research is very time dependent and has an "expiration date" [my phrase].

The rapid changes in what it means to be literate and the ways literacy plays out in a media rich digital world have made a large variety of research and practice tools irrelevant. Surveys and interviews that are paper centric in reading and writing miss whole potential worlds of engagement that exist parallel to the print world. In today's world access to magazines newspapers and libraries is paralleled to websites, applications and online newsstands.

It also means that publications cycles for research must be shorter if they explore new tools. These tools need to be comprised of modular pieces that can be removed when they become irrelevant and added to as new technologies become relevant.

Some examples can include: Adding to "How many hours of TV watching does your child do daily?"
"how many hours does your child play video games?" "How many hours does your child spend online?"
In addition to "how many books do you have at home?" We could add- How often do you use e-readers/ tablets to access magazines or books?"

I am working on such an instrument right now and will report some results soon!