Improving Course Community

When I took CS15, I had no prior experience programming. I remember sitting in the computer lab, lost on how to even start the assignment. I looked around the lab, and found a classmate testing out their fully functional, multi-player game with intricate animations. I felt disheartened and angry, upset with myself and my inability to learn seemingly simple concepts. Learning how to program was a frustrating experience, and watching other students quickly grasp concepts and build complex programs only heightened those emotions. Of course, as I went on in the department, I realized that I was not the only student feeling this way.

As a Head TA, I wanted to make the course more accessible, welcoming, and engaging. I wanted to build an environment where people knew it was ok to feel frustrated, where people felt comfortable experimenting with material, and where people would realize that they weren’t alone. If there was a way to drive a sense of community while still keeping it relevant to course material and assignments, all the better for the course. Retention numbers and student surveys were 2 key metrics of success for these experiments. I approached these (admittedly lofty) goals from two angles- assignments and lectures.

Increase Engagement in Lecture

CS15 lecture is 80 minutes, delivered twice a week to an auditorium of about 300 people. It’s said that the average attention span is ~10 minutes. With technology in the mix, I’m sure that number is even less. Historically, every CS15 lecture begins with a short skit where Andy is delivered his bottle of water, and pauses in the middle for a brief, 1 minute, stretch break. The material is complex, the PowerPoints are long, and most importantly, the lectures are recorded. Attendance of lecture throughout the semester dwindles.

As Lecture Head TA, I wanted to see if we could use clicker questions to help students engage with the material during lecture. A mid-level class, CS0330, had incorporated clickers in their lectures the previous fall, with resounding success. It had quickly become a favorite part of the class for their students. I hoped that we could mimic their workflow, and use clickers as effectively as they did. I was attracted to the idea of using clickers as a way to inspire discussion amongst students, to spur debate over the material, and to help students meet one another. However, the reality is that unless the question is interesting, the students are given enough time, and the motivation to answer clicker questions is clear, clicker questions are often useless. A lot of my TAs had negative experiences with courses that used clickers- CS0330 seemed to be the exception.

There were several questions that needed to be answered before I felt comfortable introducing clicker questions into class. What made an interesting clicker question? How could I incentivize students to answer, without penalizing them if they didn’t? How do I budget time into an already hectic lecture schedule? I spoke at length with the former Head TAs of CS0330, who showed me how they incorporated clickers and why they recommended it.

“I thought clickers were extremely effective in keeping students engaged in lectures. CS33 had a major lecture attendance problem in previous years but we saw very high attendance throughout the semester this year. I thought the clicker quizzes were also an effective way to catch student misconceptions - it was not uncommon for Tom to get several good questions from students after a clicker quiz that the whole class probably benefitted from.”

— FORMER CS33 HEAD TA

My co-Head TAs and I held several discussions over the summer on how to incentivize students to answer clicker questions. We acknowledged that it would have to be some portion of their grade in order for them to properly care. However, we didn’t want to penalize students who couldn’t attend class or stress the students out if they got the answer wrong. My goal was to simply create an environment where students could engage with each other and discuss the current material. Ultimately, we decided to grade each question on a scale of 0-3 (0 = no answer, 2 = answered incorrectly, 3 = answered correctly) and the lowest 6 scores would be dropped from the final grade. The clicker questions were also only worth 5% of their final grade. We felt that this struck the correct balance between encouraging students to come to class and engage with the material, but not penalize them if they chose not to.

As part of my summer work for the course, I worked with two UTAs to develop some sample clicker questions. I felt that clicker questions could take the place of some of the copious examples Andy included in his lectures, and would allow the students to take a moment to synthesize the information they had just learned. There were a few types of clicker questions that we felt worked best with our lecture- syntax questions, flow of control questions, and theoretical design questions. As the course moved on, and students became more comfortable with the basic concepts, we were able to ask fewer of the former and more of the latter.

Increase Engagement With Assignments

One of the biggest complaints in CS15 is how isolating it can feel. CS15 teaches students with a range of experience in programming- some students have been coding since they were 8, and others are completely new to the concept. This, coupled with a strict collaboration policy, unfortunately left students feeling alone and like they were far behind their classmates. As a former student of the class, I could relate and as Head TA, I wanted to restructure the class to address this. I was also acutely aware that these feelings of isolation were stronger amongst underrepresented minorities in the department. I care deeply about creating a welcoming environment, where everyone feels comfortable learning. Computer Science is becoming increasingly important to every career path, and I strongly believe that knowing the basics of programming will be critical in years to come. As such, I wanted to make sure I was doing everything I could to help underrepresented minorities be successful and feel supported in CS15. My co-Head TAs and I collaborated closely with MOSAIC+, a student-led organization within the campus dedicated to supporting underrepresented minorities. We realized that the class is a heavy time commitment, and TA Hours are the sole system of support for many students. This tends to lead to long lines at hours, with many students getting frustrated as they have to wait hours to be seen. MOSAIC+ suggested on finding other avenues to offer support to students. Small group, informal sessions were encouraged, as students would be able to get more face time with TAs and interact with others in a safe space.

However, any changes we made needed to not add more requirements for the students. As stated earlier, the course is a heavy time commitment and adding more requirements would simply be unfair to students and their other responsibilities. My Head TAs and I drew inspiration from the follow up intro class taught in the spring (CS16: Data Structures and Algorithms). CS16 has weekly sections, where students meet in small groups with 2 TAs and talk in depth about the lecture material. Sections are positively reviewed by current and former CS16 students alike- section ends up being like a bonding experience, and many form friendships with others in their sections. My Head TAs and I felt that if we could mimic this environment, the feelings of intimidation and isolation in CS15 would abate. The big question, though, was how do we add sections without putting more work on the students?

The answer came in form of “Design Questions”. Each assignment in CS15 was normally preceded by a Design Questions worksheet. It was meant to help students think about the assignment before they had to implement it, and ideally save them time later on. Students were expected to detail how they planned to design and implement the next assignment. In reality, most students put a half-hearted attempt (including me) into the worksheet, knowing that the “real work” could be done later. It was a nightmare for TAs to grade, as it could be impossible to decipher the student’s thought process. Moreover, there is no “one right answer” when it comes to designing a program. There could be many valid ways to build Tetris (the penultimate assignment in class) and it was hard to assign grades using one common, rubric. Perhaps most critically, there was a delay in feedback- students could complete this worksheet and immediately begin work on the assignment before receiving feedback from the TA. If the student had misunderstood parts of the assignment, they would only be told a few days later and would have to start over.

It made sense to replace the Design Questions worksheets with in-person “Design Discussions”. There would be 5 total, one before each large assignment. Instead of having students work on these assignments alone, they would complete them together, in groups of 10, under the guidance of 2 CS15 TAs. This had several benefits compared to the old system- it allowed students to have more face-to-face time with TAs, strengthened the role of TAs as mentors to introductory students, created a space for students to meet new people in the department, all while promoting CS15’s design centered pedagogy. Students would be able to engage with the new concepts the assignment addressed, review old concepts, and debate various approaches to implementing the assignment. TAs were not stuck deciphering students complicated architecture diagrams and could instead talk directly to students to help them plan out their assignment.

Implementing Design Discussions was a bit of a logistical nightmare (our department head was slightly aghast when we tried to book 40 different rooms, weekly) but with the help of our professor and department head we were able to move ahead. Up until this point, TAs mostly worked with students one on one, or with large groups of students during lab. Design Discussions required TAs to lead a small group discussion and handle all the problems that come with small groups- no one participating, one person dominating the conversation, etc. During our TA Camp training before the semester started, we made sure to hold extensive sessions to guide TAs on how to build a safe and welcoming environment during these Design Discussions.

Results

These two major changes to the course (clickers and Design Discussions) were moderately successful. End of semester data shows that retention numbers did increase from the previous year- from 72% to 77%. Student surveys also show that more students believed CS15 was accessible to beginners (86.8% of survey responders said yes in 2016, compared to 77.9% of survey responders in 2015). 86.1% of survey responders stated that they found Design Discussions either extremely or somewhat helpful, with many students noting that the discussions were crucial in later, more advanced projects. One student noted, “I think design discussions are awesome. They help you to start thinking about the project and how to structure it. It's also a place to talk through difficulties you may be having with concepts or how to do things, which becomes more difficult outside of DD [because] of the (old) no-collaboration policy.” In general, students felt comfortable participating in Design Discussions and valued the “head start” they got on the assignment during the time. Clickers were met with more mixed reviews- most likely due to the fact that clickers were not worth a significant amount to their grade and because lecture tended to run long, and clicker questions were often glided over as a result.

While the data is fairly promising, students in my class still noted several isolating incidents. One student wrote, “I think it's pretty easy at the beginning to feel like you're the only one struggling, and everyone gets it (also in part because I feel that the people who tend to ask questions [in lecture] are those who really understand the material and want clarification on more nuanced portions which is completely fine, but for people who are really sitting through class going ‘I have no clue what's going on’ it does make you feel like everyone else just gets it)”. This feeling might have been exacerbated by clicker questions, the opposite reaction of what I had intended. Several students echoed this response and added that classmates with prior experience in programming tended to show flashier projects, which added to their fears that they were falling behind. While design discussions certainly seemed to help add some sense of community, these incidents were not completely erased (nor did we expect they would be).

Ultimately, the largest complaint in the class was about the strict collaboration policy. Students wanted to bounce ideas off each other or work together on assignments, and felt the collaboration was too harsh and incited fear amongst the students. The collaboration policy has been contentious for all 3 years I TA’d this course. I believe in the strength of collaborating with others- I’ve often done my best work when I can talk out my ideas with my coworkers. However, this needs to be balanced with students learning and understanding these concepts. In the 50 years Andy has taught this course, he has noticed that collaboration can sometimes end up with a “driver-passenger” pattern, where one student does all the work and another does nothing. While we did not focus on loosening the collaboration policy while I was Head TA, I believe that careful restructuring of the course could allow for students to collaborate on assignments without resulting in these unbalanced partnerships. It would likely be the best way to help solve these repeated feelings of isolation.

Design Discussions and clickers showed promise in its first year. The last 3 years have likely seen many improvements to their processes as the class (and the TA staff) has grown. While it certainly didn’t solve all the problems facing the class, Design Discussions and clickers did seem to be steps in the right direction.

< CS15: Intro to Object Oriented Programming