Friday, August 30, 2013

Launch: Introducing Programming Challenges

- Sophia Westwood, Khan Academy CS software engineering

Today we're launching programming challenges, a major new part of Khan Academy Computer Science! We have been working on designing and building the challenge platform over the last few months, and we're excited to introduce you to our first iteration. Go ahead and try your hand at a challenge -- good luck!

We envision the challenges as a core component of our curriculum alongside the talkthroughs. While the talkthroughs involve mostly passive learning (similar to watching a video), the challenges emphasize active, participatory learning.

Why challenges?

At the start of the summer, our team discussed where we wanted to take Khan Academy Computer Science over the next few months. We agreed that students would learn best (and have more fun!) by actually coding, and we wanted more students to experience the thrill of creating something neat with code.

But, for a beginner programmer, there is an enormous gap between watching the talkthroughs on drawing or animation, and actually drawing or animating something on your own. We didn’t have anything to bridge the gap between guided, passive lessons and open-ended program creation.

So, we started tossing around the idea of coding challenges. The challenges would provide guidance and help students start coding and creating right away. Rather than just listening to talkthroughs, the students would have to figure things out and apply what what they just learned. And, after completing the challenge, they would have a program that they could continue to extend and customize.

The challenges are in essence bite-size versions of offline programming assignments that include live feedback, hints, and steps. They form a structured environment for students to gain experience with new concepts by actually coding and creating, allowing for exploration while also providing guidance on program correctness.

What is a challenge?

After a student watches a talkthrough, we encourage them to start a challenge to begin coding with the concept they just learned. A challenge is a series of steps that the student must complete to create a program. The steps break down the program, providing guidance to the student on how to approach each part. We check whether the student has completed each step, providing hints when the student goes astray.

Our challenges try to balance guidance and flexibility, centered around StructuredJS, the library we wrote to verify student code and give hints. Expect to hear more over the next few weeks about the design considerations and technical infrastructure behind the challenge framework!

Credit reel

Challenges have been a major feature for the Computer Science team over the last few months, particularly the two summer interns.

Leif Foged took charge of the challenge user interface, tirelessly working towards a more intuitive and beautiful frontend while building out functionality. I worked on the challenge infrastructure, building StructuredJS as the core static analysis driving our challenge verification and hints, as well as the necessary backend for user progress. Pamela Fox contributed creative and fun ideas for challenges, leading to crazy face, ice cream, and cloudy day. Designer Kitt Hirasaki helped us iterate all summer on the display, and after starting only four days prior, new hire Tabitha Yong designed the mock-ups for the interface that led to our final design.

And of course, our team lead John Resig masterfully guided the process, from the initial meetings where we threw out the possibility of challenges, all through the launch today, with countless code reviews and design conversations along the way.

Although we're just getting started, we're excited to see the community response and learn from your feedback. We are also looking forward to extending the curriculum with new challenges and talkthroughs.

Monday, August 26, 2013

Our Design Guidelines for Teaching Programming Talkthroughs

- Sophia Westwood, Khan Academy CS software engineering

As we slowly ramp up content creation for Khan Academy Computer Science, I put together this list of design guidelines as a reference point for teaching our programming talkthroughs. We'd like to use them as a starting point to spark a conversation with the teaching community.

It's our first iteration, and we're under no illusion that we've figured it all out. We know that teaching is complicated and difficult, and we're still trying to live up to our own guidelines given the time and tool limitations. While we have to strike a balance between covering material, developing strong content, and enhancing the platform, we always like to discuss what we could do better.

So, we would love to hear reactions to these design guidelines based your experiences teaching kids programming. How can we better fulfill these guidelines? How can we support more effective learning within the talkthrough format? What are the tips and tricks you've picked up from your teaching or research? Please write a comment, or get in touch with us at

What is a Talkthrough?

A "talkthrough" is how we deliver online programming lessons on Khan Academy. You can check out the current talkthroughs in the Khan Academy Programming tutorial. Most recently, last week I created Text Part One, Text Part Two, and our Intro Sneak Peek, and Pamela created Variable Expressions. The rest were created by Jessica Liu last summer.

As background, a talkthrough tries to mimic the experience of sitting down next to someone and sharing a computer. In essence, the teacher records audio while typing code and drawing in our live execution environment. During playback, the student sees the code changing, and they can pause at any point to experiment with the code. The student can also create a "spinoff" to make changes and save the program as their own.

If you're interested in learning more about the design of the platform, check out John's introductory post. As he discusses, the goal of the platform at this point is not to make software engineers or teach AP Computer Science, but rather "to get people (especially kids) excited about programming."

Design Guidelines for Teaching

Pass the Interesting Test

What would your middle-school cousin think if they saw this example? Do you have a compelling example, or is it contrived? Focus on a program you could actually envision kids wanting to spinoff and customize.

Also, it's great to show off how programming connects to all kinds of different interests, not just "CS things." Self-expression also seems to be a big draw.

Concretely: Penguins and monsters are interesting! Adding numbers for the sake of adding numbers not so much. Winston, a simple face, was a big hit from last summer due to his appearance in Intro to Variables -- just look at the spinoffs to see the excitement and creativity.

Prove what you're saying, live

But how do we know that the ellipse is positioned based on its center? How do we know that the default text color is white? How do we know that this rectangle controls that leg?

Use the live coding environment to demonstrate that what you're saying actually is true. Don't require the listener to take it on faith if possible. Put the ellipse at (0, 0) to show what happens. Set the background to red so that the white text is visible. Comment out that rect and see the leg disappear. As you explain, prove it with a demonstration. It is way easier to watch something relevant happen as you explain than just to stare at a static screen while you talk. The visual also helps the concept feel real.

Concretely: In the middle of the Text Tutorial, I say that the x and y coordinates for text specify the lower-left coordinate. I prove that is true by setting y = height (the text sits on the bottom). Then I try y = 0 (the text disappears) and show how the text starts peeking out from the top as I increase y, asking the listener if this is what they would expect.

Make beginner mistakes

Seeing when things break can help students better grasp the material, especially when it is unexpected. Plus, it can get boring if you are always writing perfect code. So, throw in a few (well-chosen) mistakes which you then clearly explain and fix. It can help challenge the student to figure out what is going wrong on their own, and it better prepares them for writing their own programs.

In particular, rather than mentioning possible errors that you could be making but aren't, go ahead and make those errors. Access your array out of bounds. Don't declare a variable. Your students won't be writing perfect programs, so show them what is easy to mess up, what goes wrong, and how to recover. While doing this, keep an eye on pacing and flow so that the mistakes don't feel jarring.

Concretely: In Intro to Animation, Jessica does a great job making mistakes midway through as she tries to animate the car moving. Rather than just presenting "Here is how you animate," her first attempt just changes x from 10 to 11. Of course, the car goes to 11 but still doesn't animate, and we need to figure out what is wrong. She also "forgets" to set the background, causing the car to smear across the screen.

Connect to previous lessons

Explicitly point back to previous lessons when you use an idea that's already been covered, or to draw a similarity -- "We already know how to draw rectangles and circles. But what if we wanted to draw more complicated shapes?" Not only does this help the student feel "I have learned something!" it gives a strong sense of progress. It can also be a quick refresher.

Concretely: This one is pretty easy -- for example, at the start of Mouse Interaction, Jessica points back at regular variables to point forward to mouseX and mouseY, special mouse variables. Or, toward the end of Text Part Two, I point out how we've already seen animation -- we need a variable, and we also need to be changing the variable to get animation, and we're just doing that with text instead of shapes now.

Build strong mental models

How do you communicating the essence of the topic? Does it give the new programmer a good intuition? When they make sense, simple analogies to real life or quick drawings can build helpful mental models for how things are working. A strong mental model helps the student make predictions about how the concept will behave beyond the examples you show. It creates the feeling, "I understand how this works!"

Concretely: About a third of the way through Intro to Coloring, Jessica explains the fill command with a "color butler" who is in charge of colors. He can only hold one colored marker in his left hand and one bucket of paint in his right hand. Stroke changes the color of the marker, and fill changes the color of the paint. She even includes a drawing. This is exactly the intuition a programmer needs to understand the state-driven coloring (instead of object-specific coloring), and it is presented in a fun and memorable way.

Weave together explanations and code, avoiding repetition

Don't try to explain everything in advance. Provide a brief explanation or motivation, then write some code, and then deepen the explanation, then code some more. It can be hard for a student to understand why something matters until they see it happen. Do your best to spiral back and hit the same concept in different ways at different times, pushing the student's understanding.

Concretely: In Text Part Two, I cover using strings with variables. Rather than explain in advance every detail of using a variable, what it will be good for, and adding together string variables, I begin by just asking, "Could we use variables with strings?" and then refactor the code to show that yes, indeed we can, and it works as expected. Then I address, "This looks exactly the same. What was the point of adding the variable?" I add to the code to show that now, if we're using the variable to write our name three times, we can just change that one variable to write a different name. Then I point out, "Well, we added together variables when they were numbers -- could we add together variables with strings?" and continue on in this fashion, interleaving the explanations and the code demonstrations.

Reduce mental overhead

The more extra concepts involved in your example, the larger the mental overhead for a student seeing something new for the first time. Not only do they have to grasp how this crazy new thing works, but they also need to figure out everything else going on.

So, try to keep your code as simple as possible when you introduce a new idea. An easy test is to show an experienced programmer your example code -- ask if they think that the main idea is clear and obvious just from the code, without even showing them the audio.

Then, after you teach the basics of a concept, go ahead and demonstrate how it interacts with concepts already covered.

Concretely: In the text tutorials, Part One introduces the concept by building up from a single call to text, while Part Two shows how text can interact with variables, animation, and mouse interaction. If I had tried to explain the basics of text starting from the final mouse-following, animated name program in part two, I would have muddied the introduction with a bunch of unnecessary overhead.

Cushion tricky concepts

If this is a tricky concept, reassure the student that it is okay to be a little confused at the start. Try to build up based on what they already know. It can be useful to explain that the concept is not a brand new idea, but rather something they already have seen outside of code -- we're just using a different language now. You might discuss, "What in real life is like an array?" Learning to program can be scary, so try to help out.

Concretely: In Variable Expressions, Pamela uses a real-life example to make variable dependencies more approachable. She talks about how in real life our body parts grow together over time, and how we just want to use code to do something similar with the penguin.

Stay short

Go for bite-sized. We're fighting short attention spans, and research on boredom (Slide 19 from one of Pamela's presentations, for example) suggests that it is easy to start getting bored after 10 minutes.

To a beginner, mistakes like rambling explanations and verbal stumbles can be really confusing and distracting. Go through your talkthrough and replace any long, wordy segments with snappier explanations. On a related note, try not to get bogged down making repetitive changes -- either simplify the code (by far the best option), copy-and-paste the end result once the student gets the point, or tell really really good jokes.

Concretely: All our talkies try to stay under 7 minutes, and ideally shorter. To balance, we'll split up topics into parts -- Intro to Variables, More on Variables, for example. We're also working on some other changes that should help engage users.

Amp up the energy!

Treat your talkthrough as entertainment. Do ridiculous (on-topic) things with code to keep it fun. Be a little silly. Your students are gracing you with their presence! Give them a reason to stay. (This can be hard when recording for the 12th time in a dark room.)

Concretely: The start of Pamela's Variable Expressions has some fantastic moments with penguin hippy-hop and penguin shuffle to show off the dependence on the two position variables. She also makes the penguin head float far above the body, then puts it back "because he's cuter that way." She makes us want to see what the penguin does next, and it is way more engaging than if she had just said, "These variables control the placement of the shapes."

Avoid jargon

Only use a term if it is necessary. If you do use it, be sure to introduce it and explain it -- don't just drop it in. Also try to make your introduction a little fun (decomposition? like rotting plants? ew!). It can be easy to use jargon without realizing it, so keep an ear out.

Concretely: At the start of Text Part Two, I introduce the word "string." The debate went back and forth, but our team decided that it was important to familiarize students with the term. Still, I only introduce it in Part Two, after the student has learned the basics of text. I explain it as a fancy programmer term for something they already know about -- I also throw in that "it's not what your cat plays with" because cats are cute, and because to a beginner, "string" does at first mean a cotton strand.

Directly encourage exploration

Explicitly recommend that your student explore the program and engage with the code. Suggest moments when they should pause the recording and try something on their own, even if it as simple as changing a number or playing with an animation. Give the student a few clear next steps when the talkthrough ends.

In our live coding environment, the student only has to press pause to transition between watching a talkthrough and coding on their own. Even so, we've seen that beginners can be tremendously hesistant to change the code without direct permission or guidance. Special moments in the talkthroughs that suggest exactly what to try can help lower the intimidation factor, encouraging these students to dive in and overcome their fear of breaking things or making a mistake.

Concretely: In the middle of the Sneak Peek Intro, I tell students to pause the program to see how the ellipse follows their mouse around. Or, at the end of Variable Expressions, Pamela recommends that students finish fixing the penguin's feet on their own. During Text Part One, I use my name as an example and encourage the student to switch it out for their name. We throw in these next steps to help the student develop the attitude of jumping into programs and exploring how they work.


By the end of your talkthrough, the student should know:

What is this new concept? How does it fit in with what I already have learned? Why are we even covering it?

When would I ever want to use this new concept? In particular, what can I do now that I couldn't do (or was very tedious to do) before?

What is tricky when using this new concept? What are the common mistakes I'm going to make, and how can I discover and fix them?

By the end of your talkthrough, the student should feel:

That was sort of neat!

I could start using this right away in some of my programs!

I should make a spinoff to customize this!

I understand everything going on in this code, and I feel confident I could explain it to someone else! (ideally)

Note: As a prerequisite, of course, the student needs to get to the end of the talkthrough in the first place -- expect another post soon on the metrics we are using to quantitatively evaluate talkthroughs and attrition!

Feedback from the Teaching Community?

We would love to hear suggestions and feedback from the teaching community based on your experiences engaging kids in programming. Again, we are still far from where we want to be even in fulfilling these guidelines, but we welcome your feedback. We would appreciate your reactions and suggestions from insights into how kids learn coding to tips on their favorite examples.

We want to make this the best platform possible for getting kids creating with code -- it is a work in progress, and we're excited to be ramping up. Feel free to leave a comment with your thoughts, or get in touch at

Wednesday, August 21, 2013

Variables Expressions (or How I Learned to Refactor a Penguin)

We've been working on plugging the cognitive holes in our intro tutorial to programming, and realized we were missing a lesson on variable expressions. So far, we'd only been saving single values into variables, but as you know, it's quite powerful when you can save expressions too.

To demonstrate that, I put together a talk-through about a penguin that starts off with all hard-coded values for its body parts, and thanks to the power of expressions, ends up with values dependent on other values. That makes it easy for us to resize the penguin just by changing one variable - and we're all about scalable penguins in these parts.

Check it out here!

Tuesday, August 20, 2013

Terrific Text: 2 New Talk-Throughs

Thanks to the diligence and creativity of Sophia, we have two new coding talk-throughs explaining how to use strings and the text functions to write text to the canvas, resize it, and animate it. These tutorials are part of the "Basics" series, and come after variables and animation. Check out Part 1 and Part 2.

New Intro to Programming Videos!

We've re-worked our introductory section for the Programming tutorial to give a better introduction to programming generally, the programming environment on Khan Academy, and our programming community.

Check them out here, and let us know what you think.

Friday, August 16, 2013

Teaching Novice Programming: Learning from Alice

At the office today, we got a visit from Stanford professor Stephen Cooper, who's most well known for his work on the Alice project. Alice was developed to teach programming in a novice-friendly way, using a highly visual drag-and-drop editor to create a 3d story or game.

The Alice project was a great experiment in more learnable programming, and much has been published about it since, like in this paper from Cooper. One of the findings was that Alice appealed to a wide audience, across cultures and genders, because story telling appeals to everyone - people just tell different stories and create different worlds! They also found that many students would rely *too much* on trial and error to figure out a solution, and they started emphasizing design and storyboarding more to prevent that.

We showed Stephen what we're currently working on with our tutorials and challenges, and he had some great feedback that we'll incorporate. There's one crucial thing he says that everyone loved in Alice, and that we're missing entirely: Penguins. Apparently, penguins are a universal truth.

Well, that was an easy fix:

...It will take more time to address his other points, but we're happy we got the opportunity to hear them in person. If you know of research in teaching programming that we should check out, let us know in the comments.

Teaching CS? Make Custom Challenges

We love finding out when teachers are using the Khan Academy CS platform in new and creative ways. Bryan Harms decided to come up with his own supplementary programming challenges for his 8th grade math science class.

My favorite is "Draw your own face", because it's just like customizing your Mii on the Wii, but even better because you have full programmatic control. I can only assume that this is what Bryan's own face looks like:

In the challenges, Bryan gives instructions, starter code, and questions to think about, and he provides a separate link to solutions. If you're giving custom challenges in your class, you could also decide to distribute the solutions a few days after the problems, so that students aren't tempted to peek. If you have the time, review the code and let students know how great they did and how they could change this or that to make it even better.

Have you come up with your own programming challenges? Let us know!

The "Tutorial-ization" of our CS Curriculum

When we first introduced Khan Academy Computer Science, our focus was on exploration and discovery, so we didn't follow the same tutorial structure as all the other topics on Khan Academy, with their progressions of tutorials and exercises. We loved seeing the crazy ways that students used the platform, to create everything from hypnotic animations to addictive games.

But we decided it was time to give students the option for a more structured approach to learning programming, and we embarked to "tutorial"-ize the CS curriculum. Thanks to our super summer interns Leif and Sophia, we launched the new interface for CS this week. This is our programming tutorial, what we recommend every learner start with:

Right now, the tutorials are a mix of videos and coding "talk-throughs", where you can watch us code on the left and see the result on the right. But we don't want students just passively watching, we want them practicing what they learn in the videos! So we're building out a whole new challenge framework based on StructuredJS, where we'll be able to provide step-by-step instructions, example code with placeholders, and suggestions. We'll also be adding more talk-throughs and subtopics, like functions, arrays, loops, and objects. Stay tuned!