Wednesday, July 9, 2014

Peer evaluations for coding projects on Khan Academy

In our programming courses on Khan Academy, we have two ways that students practice what they've learnt: challenges and projects.

A tutorial showing both challenges and a project

Challenges give you step by step instructions, and help guide you through learning a new concept - like "here's a black and white ice cream cup, color it in with fill commands." 

Projects are more free-form. They give you a goal to complete, and let you loose to be creative and get practice with programming - like "here's an empty plate, draw something to eat on it, using any of the commands you just learned."

Challenges can be checked for validity automatically using structured.js which parses a student's program into an Abstract Syntax Tree (AST). Each challenge has grading code that matches the user's code against known good structures and outputs messages for known bad structures.

But for projects, the student's code varies so much that they can't be graded as easily automatically - and that's a good thing, we love seeing the variety in the programs that pop up. We still wanted some way to judge projects as being done, however, and to give students feedback on what they'd written.

To do this, we've now introduced the concept of project evaluations: rubric-based code reviews that students can learn from and improve their program based on.

What's it like to be an evaluator?

If a member of our community is eligible to be an evaluator, they can visit our project feedback page to see a list of the projects awaiting evaluation.

For each project, there’s a list of objectives that the reviewer should look out for.  Each objective can be marked as "Passed" or "Needs more work".

Performing an evaluation

When there’s a failed objective, the evaluator can leave additional comments, to give the student ideas of ways they can meet that objective.

The evaluator can additionally leave additional comments at the end, which they often use to leave encouraging remarks or pro tips.

What's it like to be a student?

When a student starts on a project, they are given a summary of what to do and the project is marked as started.  

When they decide they're finished working on a project, they can request an evaluation.

What the person doing the project sees

After some time, the evaluation will happen, and if it's successful, the student  will then get a notification that their project is now marked as completed, or needs more work. If it needs more work, the student will make adjustments and re-request evaluation.  Once the project evaluation passes, the student will get points for their project.

How is eligibility determined for evaluators?

Community members who have earned the "Intro to JS mastery badge" are eligible to do peer evaluations. They can earn this badge by completing the "Intro to JS: Drawing & Animation" computer programming course. 

The eligibility may be tweaked in the future depending on the mean wait time of project evaluation requests and the quality of reviews.

How did the launch go?

So far, the launch has gone amazing. There's an initial surge of project evaluations to be caught up with, but we're hoping to keep the average wait time around a day.  We've immediately seen hundreds of project evaluation requests as well as dozens of awesome and supportive evaluations.  The advanced programmers in our community love the opportunity to help new programmers learn: 

They're also forthcoming with their praise for a well done program - both in terms of functionality and readability.

We've also got a ton of great feedback from the top evaluators, and we'll continue iterating on the system (just like the students are iterating on their projects) to help our community keep learning from each other.

Wednesday, June 25, 2014

Lowering the barriers to learning programming

At Velocity 2014, I gave a talk to the attendees, a room full of web developers and backend ops, about why we need to lower the barriers to programming, and ways we can do that.

Since the goal of the talk was to give everyone an action item to lower a barrier in some way, here's a run-down of the ways you can help:

  1. Donate hardware:
    You can donate your own unused laptops to local schools or organizations (like Boys & Girls Club, YMCA, or coding specific ones).
  2. Fund a project:
    You can search on for projects that are funding laptops to teach CS in their classrooms. You can also filter by topic, poverty level, or location, and you can turn your donations into a gift in the name of a friend or family member.
  3. Contribute to online programming environments:
    You can contribute to existing open-source environments like,, or to the tools that power them like JSHint, ACE editor. You could also build curriculum around existing environments, or create an entirely new environment, like for Arduino, Objective-C, or 3D games.
  4. Make CS count in your K-12 state/district:
    You can visit, select your state, and follow the instructions to sign a petition or email a politician. You could also write letters to your local high school, especially if you have a student there, and offer to explain to the administration why it's important. Perhaps you could even volunteer to teach yourself (like via the TEALS program or mentor a teacher.
  5. Volunteer at a local coding club:
    Search on for local coding clubs, and volunteer as a mentor or teacher. If you can't find one, start one (like a CoderDojo chapter). You can use online self-guided curriculum like Khan Academy, so you don't even have to feel comfortable teaching.
  6. Encourage your kids:
    According to recent Google research, encouragement - especially parental - is very important. Encourage your kids, especially daughters, but do not pressure them.
  7. Clear up misconceptions about CS as a career:
    Many students don't know what CS is like as a career and realize that it can have social impact. Share stories of CS in the real world, like from Khan Academy's Meet the Computing Professional, Computing is Everywhere, Made With Code. Share those with kids you know, and encourage teachers to highlight one a week.

If you're interested in the whole talk, you can watch the video recording, see the slides embedded below or see slides here with speaker notes. Think of other ways that industry professionals can help? Let me know in the comments!

Monday, June 23, 2014

CS4HS @ Cal: Meeting CS high school teachers in person!

This past Friday, on June 20th, I was invited and honored to stop by CS4HS @ Cal, a gathering of local high school Computer Science teachers. As you can see from their group photo, they were a lively bunch:

I presented slides giving an overview of the Khan Academy programming curriculum, community, and challenges, plus our future plans:

It was lovely to speak with teachers that were using our curriculum or thinking of using it, and informative to hear about the struggles (and joys!) they face as CS teachers. If you're a CS teacher, you might be interested in reading through the extensive notes they took during the meeting.

We look forward to more opportunities to connect with CS teachers in the future! Thanks again to the inimitable Dan Garcia for inviting us.

Monday, May 12, 2014

New advanced JS course: Learn to program natural simulations!

Last fall, I discovered an online and print book about programming natural simulations using the Java version of ProcessingJS. I started going through the book myself, porting the programs to JavaScript and trying the suggested exercises. It was so much fun that I thought, hey, let's put this on Khan Academy so that all of our students can have this much fun learning!

Fortunately, author Daniel Shiffman CC-licensed the book, so we were allowed to port over the text, and add our interactive coding challenges and projects on top.

We've just released the new course as "Advanced JS: Natural Simulations," and we think that teachers and students alike will find it a great way to practice JavaScript, learn new ProcessingJS functionality, and brush up on basic math and physics concepts along the way. Here's a run-down of the units, to give you a better idea of what's covered:
  • Randomness: Learn about randomness, probability, and normal distributions, and use those techniques to make all sorts of "random walkers."
  • Noise: Learn about Perlin noise, which you can then use to create animated clouds and mountain ranges.
  • Vectors: Learn how to program with vectors and use them to simulate lightsabers, braking cars, and mouse-following motion.
  • Forces: Use forces to make more realistic simulations, modeling gravity, friction, and fluid resistance.
  • Angular movement: Use angles, polar coordinates, and trigonometry to simulate angular movement like batons spinning, boulders rolling, and cars rotating as they turn.
  • Oscillations: See how you can use the sine function to model simple harmonic motion, how to simulate a pendulum, and how to program Hooke's law for spring forces. Create programs that simulate slinkies, waves, and puppets along the way.
  • Particle Systems: Now that you've learned many ways to program the movement of single objects, you'll create particle systems, the standard way that games and movies simulate movement of groups of particles, like for smoke, wind, fire, and water.

Intrigued? Check out the intro video here, and you'll be on your way!

Monday, April 21, 2014

New case study: Going beyond our curriculum with cryptography, technology news, and games

We've added a new case study to our coach resources, highlighting Jenny Oliver's classroom in Thailand. She's a software engineer that suddenly found herself in charge of the CS class at the local high school, so she decided to base her class on our intro curriculum and build on top of it. Together with her husband, also an engineer, they added lessons about cryptography, technology news, and created new exercises and projects, like Rock-Paper-Scissors-Lizard-Spock. Interested? Read more here!.

Monday, April 14, 2014

New teacher tool: class progress viewer!

Want to get a general idea of how your class is progressing through our Intro to JS course? How many challenges have most students gone through? Are they behind or ahead of where you thought? Which students haven't started the later ones? Now, instead of looking at each individual student's progress, you can look at your class progress overall to find answers to all of those questions.

Just select the "Intro to JS" mission for the "Skills Progress" report, and you'll see bars representing the percent of your students that have started or completed bits of the course:

You can click on those bars to find out precisely which students are implied by the bars:

If you only care about their progress on types of content, like challenges or videos/talk-throughs, you can filter down what you see:

If you'd like your progress picker to always show "Intro to JS", you can now select that as your class' default mission, in the "Manage Students" tab:

We hope you enjoy the new tool and look forward to hearing how you use it!

Monday, April 7, 2014

CoderDojo JS workshop for girls: Great success!

CoderDojo is a network of clubs around the world that hold coding workshops for kids, and they teach a wide variety of topics, depending on who can teach what in that city. The local chapter, CoderDojo Silicon Valley, teaches evening workshops in Minecraft, HTML/CSS, Python, and SCRATCH.

We think that what CoderDojo is awesome, and wanted to try using our Khan Academy JS curriculum to teach their audience, so we worked together to put on a workshop for middle school girls, last Saturday afternoon.

Around 40 girls flowed into the room on Saturday (the waitlist was 3x that!) and 10 volunteer mentors scattered around the room, while parents sat by and watched. We started off by talking about what programming is, including playing a fun game where they program each other with stacks of commands (like one of the crowd favorites: "get on all fours", "crawl 10 steps", "say 'moo'").

Then we got into actual programming in JS, working the first talk-through and challenge together. After that, we set them off to go at their own pace through the curriculum, with mentors around to help. Once they got through the drawing and coloring sections, we paired them with another ready student, and they programmed together on "Project: What's for Dinner" for the rest of the time. We ended with a Show & Tell, with each pair showing off their team chant, secret handshake, and of course, their dinner. Here are some of their pair projects:

From "Fire-Breathing Rubber Duckies":
From "Crazy Eaters":
From "Ninja Kitties":
From "Imagine Bosses":

From "Wafflicious":
From "KT Stars":

It was a lot of fun, and a workshop that I hope that both I and others can repeat. To make it easier for others to replicate, I've put together a detailed lesson plan with print outs and a suggested agenda. Let me know if you have any questions or put one on yourself!