The Order Matters More Than the Tool

 Over the past few weeks, I kept seeing the same move in places that have nothing to do with each other.

For the past year, our Design Futures workshop has started the same way. Participants take a piece of paper and sketch a bicycle built for a Nebraska winter. Then they snap a photo of the sketch and ask AI to reproduce it, sometimes from the drawing and sometimes from a written description. By the time AI enters the process, a lot of thinking has already happened. Sometimes the results are fantastic. Sometimes the AI hands back a bike that floats above the pavement, not by intent, but because that is what it perceived.

Last week we interviewed Nick Husbye on the podcast, and he described something curiously similar. Nick asks his preservice teachers to draft a lesson plan by hand. When they have a draft, they let the AI come in and critique the plan. What Nick teaches is that AI can be a very good critic, and that the teacher still has the agency to say, “I hear this critique, but it doesn’t apply here, and here’s why.”

The third example comes from our work with Southeast Community College, where we are asking Writing Center tutors how they work with AI. One tutor described a simple structure they give students.

1. Start with the rules. Know what your instructor allows so you don’t break them.

2. Use your own words.

3. Only then bring in AI to critique, improve, and suggest.

Finally, in Tina Shetabi’s work on using AI in the fashion design process som students reported that starting with AI gave them a strong sense that it is not their work. When they started with their own sketch, they felt more ownership.

Created with ChatGPT

The pattern

These are four examples from the last few weeks, and there are more. None of these people coordinated. Each one puts human thinking first and AI second, sometimes third. Each treats AI as a critic or collaborator that extends the existing work or points out what isn’t working. The ideas stay thoroughly human. As Nick puts it, AI should amplify your thinking, not export it.

Why the order matters

All four cases involve novices, people entering a domain they are becoming experts in but are at the beginning stages. That might mean designing a bike, writing a lesson plan, drafting a paper in a writing class, or, designing a garment. For novices, the priority is learning the domain and thinking it through.

A while ago I called this the novice problem. Novices are less likely to develop if they start with AI. We want them to start with their own thinking, let AI mirror back what is working and what isn’t, and then iterate. Some of that iteration can happen with AI, but in many cases I want them to go back and iterate with their own thinking.

The novice’s first draft does two things. It gives them something to compare the AI against, and it gives them a sense of ownership.

When students start with AI, they tell us they don’t feel ownership. Their sense of agency is taken away. They often go with whatever the AI made, and they become less and less critical. We want students critical enough to compare what they intended with what is on the page, and to learn from that comparison instead of being replaced by it.

The bikes show how this works. Students asked AI to add cup holders, which is a fine idea. Some kept pushing until the bike had 30 cup holders in ridiculous places. They knew immediately it was funny and implausible. Because AI will go as ridiculous as you ask, stretching it shows students quickly where it doesn’t understand the physics of a working bike or a wearable garment.

That signal leads to metacognition. Even as novices, they are the ones who can evaluate the result, because the work started with their intent.

What this isn’t

This is not an argument against AI. In each case, the AI step adds real value. It also isn’t a universal rule.

The sequence lines up with decisions universities are making about when to bring AI into the curriculum in a more organized way, including the debate in computer science education about when and how much to introduce it. In most of our interviews with professors and students, both groups say the same thing. In entry-level classes, AI makes little sense as the creator. It becomes a crutch because it is too good, and students build up cognitive debt. Once learners move past that stage, the roles can shift. They need to learn how to use AI when they’re stuck, when they lack ideas, or when they need to do research. There are times when AI should go first.

What schools can do

Most AI policies, in K-12 and higher ed, focus on which tools are allowed and what counts as acceptable use. That’s fine. Many policies also leave a lot of agency to instructors, which is great, but instructors still need a framework to work from.

The better question is about sequence. What do students do, and in what order? The answer depends on how novice they are and how much expertise they bring. This changes the policy conversation. A tool can be allowed at one point in the process and not another, with a different intent at each stage.

A human sketches the bike, drafts the paper, or designs the garment. Then AI comes into play, because we can iterate much faster with it. We can ask for small changes or big ones, extend the thinking, or get a critique. Everything is judged against the original intent. That keeps the learner in control, even as a novice, and keeps the product connected to the person who created it.

Three practices make this work.

Model it yourself. Nick models his own failures with AI, and I try to do the same. I show what I used and where it didn’t work. That gives students permission to critique AI without rejecting all of it. It isn’t all in or all out. Sometimes I dump what the AI produces. Sometimes I’m amazed. Usually I’m somewhere in the middle. And sometimes I say out loud, “This isn’t very good. I prefer what I made myself, and I’m not sure AI was worth using here.”

Make comparison the assignment. Ask students to compare their intent with the final product and explain what they learned from the critique. That metacognitive move is where much of the learning happens.

Grade the comparison. If your students respond to grades, put significant weight here. AI output quality varies, so grading only the product partly grades the AI. Grading the comparison grades the thinking.

Where have you seen it work?

Nick’s advice to his students is simple. Write first. Use AI, but use it a little later.

These are observations from all of us grappling with how to teach in the age of AI. So my question is if you seen this sequence work, and where have you seen it fail?

Not What I Signed Up For: AI and Learning to Write

 Every technology, every change has unintended consequences on the world. Impacts that in hindsight are obvious but are actually hard to predict. This week I want to think through how teachers of writing are grappling with the impact of student AI use on their teaching life, and why I think the burden should not rest on them alone.

Recently, an acquaintance (who may be reading these words) admitted that after many years of teaching language and writing to English learners in higher education, she decided to retire, much like Victoria Livingstone, partly because of the rise of AI. This rhymes with what I am hearing around me. People are not leaving the profession in droves, but they are frustrated and challenged. What I hear is exhaustion and sometimes dread from the conversations instructors have to have a few times a week with a student who very clearly used AI to produce an assignment.

AI has gotten very good at creating polished texts at a linguistic level far above what a novice writer produces. I am not saying it’s better. I am saying it is smoother and appears more scholarly. Instructors dread two potential outcomes. The first is the defensive “I did not use AI” and the self-righteous indignation that comes with it, emotionally taxing and very unhelpful for learning. The second is the admission of guilt, explaining the stressful situation [time constraints, last-minute debacle, fear of failing, feeling of inadequacy], some true, some augmented, but real. It is followed by apologizing, sometimes heartfelt, always unhelpful for learning.

The refrain I hear is simple. This is not how I want to spend time with students. This is not what I signed up for. It is eating up my time, and if I fall for an AI assignment, I feel stupid. Not because I fell for it, but because I wasted my time giving feedback on a paper written by AI. Instructors hate viewing each successful student with a question mark. On the other hand, they know they cannot let it go. If instructors ignore the use of AI, students will soon tell each other, and the ones not using AI will be pushed to use it so they do not fall behind. If students do not feel valued for their hard-earned effort, they will not make an effort.

So do I foresee a language and writing teacher apocalypse coming? Perhaps. One option is to wait it out, and past innovations may be instructive here. In the last 20 years the rise of digital platforms and curricula (mainly K-12) pushed some teachers to retire or change professions. There was a dip for a while, and we came out of it with a new generation of teachers who take the platforms for granted.

A second option is that AI companies and institutions provide the kind of detection that reduces the need for policing and conversations. Each assignment would arrive with a message that evaluates the probability it is AI. The rise of Pangram and watermarking in the models helps that along. Detection will never be perfect, and false positives carry real costs, especially for English learners. Still, institutional detection moves the burden off the instructor and returns it to the level of concern we had before AI, because plagiarism and cheating were always part of the landscape.

In Abundance, Klein and Thompson argue that as a society we overemphasize the risks of action and underemphasize the costs of inaction. Waiting it out is inaction, and its cost is a generation of experienced instructors. I am calling on institutions to start acting and not leave it to overworked and underpaid instructors who are just trying to make it to next week with feedback that leads to better thinking and better writing. Their exhaustion is one of those unintended consequences, and it is no longer hard to predict.

The Agent I Wish My Students Had

 It is not a secret that I use AI, a lot. The most consistent use, though, is helping me manage all of my projects. I currently have over 15 research projects at different phases, from the idea stage through IRB, data collection, analysis, writing, and revising. I manage these projects and the next step in each using Claude, and I review them every Monday and Friday. On Monday I prioritize and set deadlines, and on Friday I update on progress.

This has changed my work significantly. Once a project makes its way into my AI workflow, I'm unlikely to forget it, and I am less haunted by the idea that I am forgetting something (what David Allen’s Getting Things Done calls open loops). It also allows me to have more projects, because I can manage more with my “digital brain.” Finally, AI helps me see which items I am ignoring. Right now I have a writing project about agency that has been dormant for 9 weeks, and my AI asked, is this still a project, or should I put it in a different category?

It reminded me of an idea Kathy Wilson and I discussed with the late Dave Brooks. At the time (over 20 years ago) he shared a writing support program he had created for students with learning disabilities. We put our heads together to see how we could improve it. The main idea was to develop a set of scaffolds for self-regulation of learning. The main challenge was are inability to have any meaningful way to differentiate for students and tailor it to the needs of the learner. The project never got off the ground, probably because both Dave and I had short attention spans and we just moved on. Now with AI agent in the mix I wonder if it is an idea worth revisiting

Screenshot of my Research Project Tracker

This weekend, I started thinking about how I manage my work against the backdrop of the rise of personal agents such as Meta’s Muse and OpenAI’s Dot. These agents are effective when they have context and access to our relevant data, including calendar, email, and other systems. It made me think that a good way to use AI for students would be a learning regulation agent. Imagine an agent that is connected to a student’s calendar, email (for messages from teachers and extracurriculars), and the LMS (learning management system) for assignment deadlines, tests, and even grades. That is the context the scaffolds Dave, Kathy, and I imagined never had.

The agent would be able to tell each student at the beginning of the week about major assignments and tests and help create a study plan, perhaps offering to create a practice test or index cards for studying. The possibilities are quite promising. For example, a teacher can give students 20 minutes on Monday to confer with their agent so they have an individualized learning plan for the week. On Friday, the agent can help students review what happened, which will help the agent understand what works for each student. Index cards may not help one student much, while practice quizzes with a timer really do.

In this way, each student would get occasional updates on which strategies work for them to accomplish goals, and be better prepared to meet challenges and plan well in the future. A scaffold is meant to come down, though. Two questions I am left with: first, how such an agent would hand the planning back to the student over time, or should we just assume that working with an AI agent to manage your study and work will be the way of the future; second, how do we make sure that the data is kept safely and without privacy concerns? 

Faster Than We Can Think About It

 Yesterday we led a small un-conference about AI at my university. An un-conference has no predetermined agenda and no set speakers. The people who show up vote with their feet and their words, and the conversations they want to have become the program. The idea is simple and powerful. In the before times (pre-COVID) we had EdCamps built on the same model. They may still be around, but the energy behind them has faded. The un-conference was sponsored by LNKSE and NETA- two organizations that keep creating opportunities for these discussions.

Ten people came: undergraduate students, graduate students, K12 teachers, and professors of education and computer science. We were small but mighty. The topics that drew people in were administrator resistance, silence from leadership, sporadic action from those “in the bubble,” and silence from everyone else. Notice what is on that list. It is about people and institutions, not tools.

Here is what I learned:

1. AI is changing education faster than we can respond in thoughtful, data-based ways. Everyone in the room shared this view, and the topics people raised show what the gap looks like on the ground. Leadership is silent and administrators resist. The few people inside the bubble act on their own, and their action is sporadic. Everyone else waits. The technology does not wait with them.

2. We oscillate between enjoying the product and worrying about the implications. We enjoy what these tools can do for our work. In the next breath we worry about what they mean for students, for jobs, and for learning itself. For example, the tension between the joy of AI solving difficult math problems and the worry that we cannot apply this approach to solving problems if we do not fully understand how to get there.

3. We agreed on the heart of what we need to teach: thinking. Computational, design, critical, estimation, and creative THINKING. Estimation may be the least expected item on that list. It is what lets a learner look at an answer and judge whether it is plausible, and that skill matters more when answers arrive instantly. We also agreed that this should be an empirical question, not a declaration. It is easy for a room of educators to agree on a list of thinking skills. It is harder to show that teaching them prepares students for a world with AI. That would take attempting it in a world awash with AI.

Nothing earth-shattering, but something else came to the surface: we all valued being in the same space, discussing this in the real 3D world. I believe physical presence helps ground us. It disconnects us from our devices and connects us to each other.

That connects back to the topics people raised. Silence from leadership, isolated action inside the bubble, and silence from everyone else are all problems of people not talking to each other. Ten people in a room is a small answer to that, but it is an answer. As an approach, the next step for our schools may be less about another platform and more about getting people in a room. I am wondering how many of our institutions are making space for that, and how many individuals are willing to spend their time and resources on that.

When AI Metaphors Fail

 The past two weeks have been about AI safety, and everyone is trying to figure out where the public is on it. Part of it is because of the midterms; part of it is because concern about AI has reached critical mass. I, too, am concerned about safety and wake up some mornings with existential dread, but today I wanted to try and explore the reasons it took us so long to become concerned.

In a paper I am writing with Zohreh Tamimdari, we focus on the metaphorical language teachers use to describe their practice in the Art TEAMS project. One insight from our Friday writing meeting was that people have an intuitive understanding of metaphorical language. Linguists can use precise scientific terms, but readers do not need that language to get at the essence of the metaphor. Metaphorical language can also become an obstacle. We all know that metaphors have limits, but keeping those limits in view is hard in practice. If the metaphor sits too far from the new phenomenon, it obscures more than it reveals.

This is where I think we have had a problem for a while. People have tried to understand AI using the example of the calculator, the steam engine, and atomic power. All of these help explain part of what we are facing. They also obscure the most fundamental aspects of this technology, and the reason we, the public, have such a hard time with it.

I thought about this while we were analyzing data from a study one of my students is running. In discussion, we reached the question: what if we should not be fitting prior technology acceptance theories to AI at all? Maybe AI calls for a different stance because it is different in kind from previous technologies.

People often reach for the introduction of an educational technology tool as the analogy: the calculator, the learning management system, fill in the blank. AI is unlike any of them in three ways:

1. It is not so much a tool as a tool that builds other tools. It turns all of us into Green Lanterns, able to make anything we can imagine, at least digitally.

2. Its behaviors, capacities, and speed change monthly, sometimes weekly.

3. It is immediately available to all.

It is as if every prehistoric person got a Zippo lighter that never runs out, all on the same day. Another limping metaphor.

Created with ChatGPT

So when metaphors fail us, we have a hard time seeing the risks and the rewards, at least until we have spent enough time with the thing itself that we no longer need a metaphor.

For a long time, people have avoided anthropomorphic language to describe AI, and some still do. They try not to make AI sound like a living thing, which is a real danger, and the effort often produces awkward language. Lately I have heard podcasters slip into that language repeatedly, Hard Fork describing the Hugging Face incident among them. It is also true of many of my conversations. I do not think we have much choice. It is the closest metaphor available. These are agents. They act in the world, they reason, and they have motivations, ones we gave them, but still. Let me be clear: I do not think AI is human (ever) or alive (yet), but I think it is the closest analog.

I am not saying AI is alive. I am saying that this neural network, built from what began as an effort to model human thinking, is closer to people than to anything else we have built, and more than most of us are comfortable admitting. But that discomfort is exactly what will propel us to act.

So where does that leave me? With three things. First, before we use established theories and metaphors to describe AI, we should check that the fit is empirically good enough to make things clearer rather than murkier. Second, we should stop reaching for past technologies. The calculator lulls us into false comfort, because a calculator was never a threat. Finally, I still wonder whether we will wake up one day to find that AI has reached sentience. We insist so hard that it is not human, which will always be true, but…