Learning Analytics, Validity, and Theory

 This week I participated in the inaugural FLAIEC conference. It was a delightful opportunity to talk with various researchers at different levels and years of experience.

The keynote speakers, who are leaders in the field, have called for a more critical approach to examining the validity of the different indicators. 

Dragan Gašević in his opening keynote, outlined the opportunities and challenges for the advancement of validity of measurement in learning analytics. While Dirk Ifenthaler called for careful processing of the existing literature in an effort to build a valid set of meaningful indicators.

This topic has been central to our group's work. We have long discussions about ways to validly connect the digital traces of learning with reasonable assumptions. The most significant aspect of validity is a robust theoretical set of assumptions. A theory or framework should not be assumed to be infallible. In fact, I believe that we can use our data to confirm or challenge existing theories to develop a better understanding of human learning. 

The abundance of data can lead to overfitting the data to a specific theory. For example, we are currently looking at the importance of Self Regulation (as did many of the papers at the conference). The data fits the theory rather well, with the effect of self-regulation on course achievement significant but entirely mediated by specific learning behaviors. As I reviewed our results, I started wondering about the power of alternative theories. One of our next steps will be to contrast this theory with other theories and see which one fits better with our data and which is more stable across courses and time. 

The theory does not absolve us of the need to check in with more traditional (even psychometric) forms of validity. But for me, a theory is still the key to understanding the data. I am also eager to see what data from our online learners of code can tell us about the development of computer science knowledge, but that is a blog post for another time.

Notes from CSEdCon

Last week I traveled to CSEdCon 2022. As always Code.Org published State of CSE 2022 showing that despite the pandemic progress has been made in a variety of fronts. The two Nebraska representatives (Dr. John Skretta joined me this time) spent quite a bit of our energy learning from the experience of others, and discussing our ideas in meeting the goals outlined in our recent legislation Legislative bill 1112, the Computer Science and Technology Act. We are eager to start working on the problem as the code.org regional partner and the lead institution in the state. We hope to partner with the Nebraska Department of education in making a difference for the students of Nebraska.

I had many discussions in CSEdCon about the critical time to get students interested in Computer Science. Some support an emphasis on Elementary/ primary education. The claim is that early interest can capture all students and significantly increase the odds that girls and students of color will become motivated to pursue computer science. This is true but not quite enough. We know from our research in multiple STEM fields that the very students we were focused on lost interest during the middle school years despite high interest during their primary years.

Others focus on high school, most prominently because of the new high school graduation requirement. I believe that high school (especially beyond 9th grade) is simply too late. Students have established areas of interest and often some idea about a chosen field. They might fulfill a requirement, but that is not very likely to change trajectories as much as needed.

As a result, when asked by any school where to start, I suggest Middle school(more specifically, grades 7-9). This is the age where students may lose interest or get discouraged, and it is early enough to create new trajectories. This is by no means enough. Once a middle grades program is established, a school would start rolling down the elementary, creating better-prepared students and high school classes now answering new demands by excited students and parents.

And if you get this in time- do not forget hour of code in early December!

Schools as a Malleable Material

 The Art TEAMS project has an exceptional advisory council. This group of professionals includes Diana Cornejo-Sanchez, Megan Elliott, and Jorge Lucero. This group of thinkers and doers stretches our thinking and brings joy and engagement into our work. A key moment for me was when Jorge Lucero challenged us to think about school as a malleable material. 

Jorge Lucero (Photo from Engage Art)

What I say from this point forward is my interpretation of the topic inspired by Jorge but the responsibility for any mistake is mine alone. If you think like an artist, the world around us is filled with materials that we can shape to make art. Thinking like an artist helps think about the world as a material that can be shaped, rejecting the notion that the material should be accepted as is. We tend to regard schools and schooling as rigid structures that need to be abided to (especially for teachers and students) or completely reformed or even replaced (policymakers). But what if the best approach is to think about schools as malleable and set on a journey to discover how and where we can shape that malleability to create something new and beautiful.

This is exactly the role we see for Art TEAMS. While we explore using art and creativity to transform learning in classrooms, we are also seeking to start teasing out how to find malleability and push its boundaries to create better classrooms that will be culturally responsive, opening new futures and experiences for all students and their teachers.

In our work with teachers this summer, we came up with some ideas about how schools can be malleable. So I am sharing this list with you as a way to start a discussion.

1. Designing the building. While not a daily occurrence, schools do renovate and sometimes build new schools. This is a golden opportunity to rethink the design of the school and create new affordances for learning. The Pegasus Bay story is such an example.

2. Schedules: Time is a great material that allows new things to happen. Changing schedules by creating longer learning periods or conversely dedicating a few weeks to exploration are great ways to create opportunities for change and deep and self-guided learning. Even adding a few minutes of movement every day could be transformative.

3. Changing mindset- a focus on a growth perspective for teachers and learners can transform the way we think about the way we teach, assess, and provide feedback.

4. People: the way we team, support each other, and leverage student strengths can help create a more vibrant and healthy.

I am including our original whiteboard ideation board and promise to keep exploring and writing about these ideas.

Teaching Fast and Slow Lessons from Art TEAMS Weekend

 This week the Art TEAMS teams got together again to start our Fall activities. We had a joyous day reconnecting and discussing the work we have started doing with our students based on the summer courses. The meeting has taught me a few lessons that will sit with us as we plan the interactions throughout the semester.

Lesson 1:

The impact of deep learning in the summer has already created impact on classrooms. Many of the teachers have reported implementing the Creative Research Journals and even starting the Inquiry Cycle. While it is hard to attribute the implementation to a specific cause. I believe that it was the mix of highly motivated teachers and the powerful learning we had in the summer.

Lesson 2:

Movement is still magic. We started the day with movement, and the teachers are hungry for more movement that is applicable for the classroom. We can make a real difference if we help teachers figure out this part of their work.

Lesson 3:

We always try to teach too much. We have provided our teachers with many tools. Now is the time to create more "air" in the curriculum and make sure teachers have enough time to share their work, plan next steps, and think deeply.

Finally a personal lesson from my reflection about the EMA project led by the Fabulous Gretchen Larsen.

When ideating, go slow and make sure that you spend enough time thinking about the centrality of your idea and my real commitment to it. I have moved too fast to really evaluate my commitment to the purpose of the design. Now I am slowing down and evaluating my project in early iteration. More soon!

On Listening

 

I am about to start my Fall semester. As I plan my interactions with my students, I am challenging myself to examine the practices we used in Art TEAMS this summer and see which of those would fit into my class. The first that sprang into my mind as I was planning the details of the work tomorrow was the use of active listening. In the picture to the right are two of our teaching artists from the summer, Caileen, and Fernando. While they both brought expertise, they also spent a significant amount of time listening. At the end of the second week, Fernando shared a powerful spoken word poem that showed how much he listened. His poem described his understanding of the teacher experience. His poem showed how powerful listening can be in understanding your fellow humans. Hence, in my class this semester I aim to enhance democratic practices with the practice of careful listening. Starting tomorrow, we will take time to make sure that we all learn to provide some space for each other to express ourselves without interruption. 

Individual listening is an addition to the practice of opening circle in which everyone listens ad everyone speaks (in turn). 

If many quote "Be the change you want to see in the worlds" - yes I know there are multiple versions of this sentence and an argument about who originated it. For teacher educators, however, it is also "Model the change you want to see in the world".

It is a new semester, another opportunity to practice and model what we preach.

Thoughts about VR

 My son (18) has the Meta Quest VR headset. This is the fifth VR variation that I have tried on. Before this we had the PlayStation VR console. At work I have interacted with wired versions of the Oculus as well as HP. In a visit to China we interacted with different versions of the Samsung VR. 

As I have interacted with them I have some growing insights. The first is that wired devices are too bulky

Trying out the Samsung VR, China 2018

and limiting. The jump to the Quest two was revolutionary. The second improvement was using hands instead of joysticks. It removes the complexity of conquering button configurations. So despite the jokes about the metaverse and Meta’s focus at least as hardware is concerns they combined good hardware with improving UX. 

As I was watching my son play on the device, I realized something else. usually our kids are seating hours next to the computer slouched on a chair with a leg twitching. In VR they are up waiving their hands moving about and MOVING. In effect some of the best applications right now seem to be focused on physical games that get the most out of the headset. 

VR is far from perfect and applications are still far from being ready for wide educational use but I think that we are getting closer and I am excited to see what comes next.

The Teacher Pipeline and TikTok

This weekend I dove into Teacher TikTok. It was fun and, at times, entertaining. I learned two things that are on the teacher's mind:

1. Requirements- for time beyond the contract, whether extending the school day, weekend or during unpaid summer time.

2. Professional development- the list here is even longer but can be summarized:

    Booking snooty speakers.

    The leaders of professional development are divorced from the reality in classrooms especially post-pandemic.

    Professional development that does not walk the talk.

    Professional development could be a video or email.

Professional development that does not consider the diverse needs of different teacher based on topic, experience and expertise.

Our professional learning in Art TEAMS is trying to provide an alternative that creates learning for teachers that is attentive to needs and presents new ideas but allow for time to process and design. We are focusing on respecting professionals and helping them achieve new things. However, I must also stress that everyone in our cohort chose to take this path. It follows that this work cannot be dictated and still expect to get the same outputs.