Wednesday, November 19, 2014

Learn how to make webpages with HTML & CSS on Khan Academy

I often get asked how I got into programming. Well, it all started with me forgetting to buy a gift for my mum. I was in 6th grade, and it was Mother's Day. We lived a few hours walk from the shops in town, and I had to walk by a high-security penitentiary to get to those shops - given the recent surge in prison escapes, I didn't feel like taking the risk.

I'd been browsing the web a lot recently, checking out the fan websites for my favorite boy bands and hanging out in pet-themed chat rooms. I always liked the idea of making gifts instead of buying gifts, so I thought "hey, I'll make my mum a webpage!"

I searched on Yahoo! for instructions (this was even before the days of Google'ing), and made a webpage for my mum with a big cheesy "Happy Mother's Day" GIF at the top (thanks, Microsoft Word Art!). My mum was happy that I'd taken the effort to learn something new; I was happy that I'd discovered how to get away with never buying gifts and I was left with a newfound curiosity for making things on the web.

And that's how I got into programming. I started off in HTML and then got into Perl, Java, JavaScript, PHP—anything I could get my hands that would enable me to share creations on the web.

Now, nearly 2 decades later, the web is ubiquitous and there are more than 1 billion websites on the internet. Every single one of those websites is built with HTML, and probably many of them include CSS for styling and JavaScript for interactivity.

That's why I'm thrilled that we can now teach HTML and CSS on Khan Academy. HTML/CSS is the first step on the path to being a web developer, plus it's also a skill that non-developers can hugely benefit from - like bloggers, marketers, and librarians - because HTML shows up in so many of their jobs, too.

You can try out our HTML/CSS environment here. It's interactive and real-time, just like our ProcessingJS environment, plus includes a color picker, image picker, and number scrubbers.



To help you learn how to make webpages, we've put together an Intro to HTML/CSS course. It's around 8 hours of talk-throughs, coding challenges, projects, and quizzes, and will give you a great basis in the most common HTML tags and a wide range of CSS selectors and properties.



For something more bite-sized, you can start with the Hour of Webpages, which is one of our Hour of Code offerings this year.



Thank you to the team for all their hard work in making this possible - John Resig, Brian Bondy, and Alex Rodrigues. Let's get more people hooked on HTML!

Monday, November 17, 2014

Algorithms on Khan Academy, a collaboration with Dartmouth College professors

What is an algorithm? It’s a sequence of steps that you follow to solve a problem. In everyday life, you might have an algorithm for hanging up your laundry, efficiently going through a shopping list, or finding an empty parking space in a lot. In computer science, an algorithm is a sequence of instructions that a computer program follows. Algorithms form the basis of the most interesting and important programs we use, such as the algorithm that Google uses to calculate driving directions, or the algorithm that Facebook uses to automatically tag you in a photo.

Because algorithms are so important to computer science, they are a core part of a computer science curriculum. The AP CS A class teaches object-oriented programming with algorithms,  every college CS student will have at least one algorithms class and encounter algorithms everywhere, and every software engineer interviewing for a job will review algorithms while they’re prepping for an interview.

Given how important algorithms are, we were elated when Dartmouth professors Thomas Cormen and Devin Balkcom suggested writing an online course on Algorithms, available to anyone for free, forever, on Khan Academy. If you’re a college CS student, you might recognize the name “Cormen” - he’s the “C” in the “CLRS”-authored Algorithms textbook, the most popular algorithms textbook used by college classes. Balkcom is a fellow professor at Dartmouth, and he’s actually rewritten their introductory CS class, so he’s an expert in teaching algorithms to new computer science students.

 We worked over the summer to create an introductory Algorithms class that’s highly interactive. Algorithms can be hard to wrap your head around, so we have both step-by-step diagrams and interactive visualizations to explain each algorithm:

We also want to give you a chance to try coding the algorithms yourself, so we’ve used our JavaScript coding challenge framework to create 20 challenges with unit tests (and you’ll have to write unit tests yourself!):

We’ve also sprinkled in a few quizzes, to make sure you understand concepts like asymptotic and graph notation:

This curriculum covers everything you’d find in an intro course - asymptotic notation, binary search, selection/insertion sort, recursion, merge/quick sort, graph representation, and breadth first search. There’s much more to cover, of course, including going more into how you can design your own algorithms, but we’re so excited about what we have now and how much it could help software engineers in all stages of life that we want to get it in your hands now.

Please dive into the course and let us know what you think - you can leave comments beneath the articles, or email us more detailed feedback at compsci-feedback@khanacademy.org.

Thank you again to Thomas Cormen, Devin Balkcom, and their supporting staff at Dartmouth for making this Algorithms class a possibility.

Saturday, November 8, 2014

BusinessInsider goes inside Khan Academy Computer Programming

BusinessInsider reporter Madeline Stone does a lot of reporting about the tech sector, so she decided to get a taste for it herself by going through an entire Intro to JavaScript course. She wrote up a great article about her experience, including a few areas where we can improve (and we're working on those!). Check out one of her projects:


I also answered a bunch of great questions for BusinessInsider about why I think learning to code is important, and what I love about teaching coding for Khan Academy.

Thank you to BusinessInsider and Madeline for taking the time to experience what so many of our students are experiencing. Let's hope even more folks get code-curious!

Friday, September 19, 2014

NCWIT Award for Aspirations in Computing: Applications open now!

NCWIT is an organization committed to getting more women in computing, across K-12, college, and industry. Every year, they have NCWIT awards that recognize women for interest in computing and give them an additional community to support them and opportunities for growth. At last year's NCWIT summit, I had the chance to meet a bunch of the award winners, and they were all bursting with enthusiasm for computer science.

If you know high school females with an interest in computing (but not necessarily an expertise!), encourage them to apply here.

In addition, NCWIT is now recognizing educators, so you can nominate any teachers, parents, or coaches that are helping the world get more women in computing.

If you'd like to spread the word, check out NCWIT's sharing kit.

More women in computing = more people in computing!

Tuesday, August 19, 2014

Auto-suggest and live documentation on Khan Academy CS

A new feature familiar to many programmers went live on Khan Academy's programming editor: Auto-suggest, parameter information, and live documentation.

I did a small screencast of that feature when it was introduced as a hackathon project, and before it went live, here.

We expect this feature to:
  • Reduce the number of times you need to interrupt your workflow to scroll down to the documentation.
  • Save time when writing programs by allowing you to click, tab, or navigate to an auto-suggested function call without having to type it all.
  • Make it easier to learn the ProcessingJS library by allowing you to get inline help.
There are 3 main parts to this feature and all of them are included as part of the recently open sourced live-editor:

Inline auto-suggest:



Inline auto-suggest


The ACE editor does provide autocomplete, but we had to modify it to disallow some keywords that we ban (like window), and add the ProcessingJS keywords that we add to the environment.

Parameter help:


Shows you the parameters for the current function call.  The current parameter is displayed in bold and blue.

Parameter help detects when you’re specifying function parameters, and will do a lookup for you.

It will show you the function name, and a list of its parameters.   As you type an argument, it will bold and color the current parameter.

Live documentation:


Highlight over the function name, or parameter to get a description of what the function does.




This allows you to hover over the parameter help and get a description of each function name and parameter relating to the current function.

Click on any function name to go to an example program using that function.

Thursday, August 14, 2014

Peer evaluations: one month retrospective

A few weeks ago we launched a new feature called Project Evaluations in the Computer Programming section on Khan Academy.

This feature allows students to request evaluations on the coding projects that they complete.
Another peer on the site would then see their evaluation request, evaluate them based on a rubric, and if they passed, they'd earn points for the project.

This blog post will analyze the data we’ve gathered since launching the feature, and talk about various changes we’ve made to address challenges that came up.

Uh oh, too many requests!

In the first day, we got a big surge of project evaluation requests. We thought this might have been because it was just released, but it turns out that it would be normal traffic.

Overall only 20% of evaluation requests were being satisfied per day in the first few days. That meant that every day, the average wait queue for getting an evaluation was getting longer.  The challenge was clear: How do we increase the amount of evaluations, without losing quality on the evaluations?


First 4 days since launch (Horizontal axis) vs. New evaluation requests (Purple) per day and evaluations (Pink) per day (Vertical axis)


We overcame this challenge relatively quickly, but only by making dozens of small changes.
See below for a list of all of those.

The same chart over a longer period of time shows the effect of these small changes.  You'll notice that the Evaluations (pink) bars start to dominate the requests (Purple)  bars per day.

Days since launch (Horizontal axis) vs. New evaluation requests (Purple) per day and evaluations (Pink) per day (Vertical axis)

Aggregating all of this data together gives us a pretty good picture of where we're at today.

Overall "Evaluated" (Green) vs "Pending evaluation" (Blue)

How did we increase evaluations?

We started off by thinking about what sort of people might evaluate projects, and there are a few key groups:
  • current students: those that are going through the curriculum themselves,
  • alumni: the students that have completed the curriculum (but may not be particularly active),
  • super users: those that already know how to program (via our curriculum or elsewhere) and are already active in the community
  • coaches of current students

When we launched, we only let the curriculum alumni perform evaluations, so all of the evaluations were coming from students that were both very active (enough to notice the new feature) and had completed the curriculum. They're still some of our most prolific reviewers, but we realized we needed to bring in more types of reviewers from across our student base, and that's where we made a lot of changes--

Lowering the bar overall:
  • We made it so that simply completing any coding challenge would qualify you to be a reviewer (which would enable us to bring in more types of reviewers).
  • To offset the lowered bar for performing evaluations, we added moderation tools and flagging to ensure quality is maintained.
Bring in current students as evaluators:
  • We added a next action after a student submits a project and passes a project, which is a link to evaluate another students' project. Those projects are picked based on the highest difficulty level that the student has passed themselves.
Bringing in more alumni reviewers:
  • We sent out notifications letting thousands of people know about the new feature, and that they were eligible to evaluate. 
  • We added a video that every student will watch at the end of the curriculum, showing them how they can be active in the community in various ways.
  • We added a notification that will get sent to every student when they earn the curriculum badge, encouraging them to be active.
Encouraging more super users:
  • We made the page listing evaluations requests easier to find and linked to it from more places.
  • We enabled the reviewers to be more efficient by presenting them immediately with a link to do another evaluation when they're done with the current one, or to skip the current evaluation if it isn't right for them.
  • We added features to make evaluations more useful (such as allowing for formatted code inside evaluations).
  • We added evaluations to profiles so that reviewers can show off the evaluations they've done.
Bringing in coach reviewers:

  • We improved coach integration so coaches can view which of their students' projects are pending.
  • We added a notification for coaches to find out about a students' project evaluation request.


Besides increasing the number of reviewers, we also reduced the number of requests, by adding messaging to prompt the students requesting evaluation to fix their own problems before asking for an evaluation.

How accurate are the evaluations?

We've seen very few cases of bad evaluations (like students making un-constructive negative comments, or failing projects that should pass), and we're generally very pleased with the quality of evaluations.

We've heard reviewers comment that they've learnt from the requestor's code, and seen reviewers share valuable tidbits of knowledge that wouldn't be gained otherwise.

We do have more work in progress for monitoring and understanding the quality of evaluations at a deeper level.

What's the distribution of evaluations across reviewers?


Initially, there was a very small amount of reviewers with a few outliers.

Since the changes we’ve made, we’ve managed to bring in a lot more people trying reviews. Over 4,000 people have performed at least one review.  2,000 have performed 1-2 evaluations, and around 100 have performed over 60.

The top evaluator has performed a staggering 3400 evaluations on their own.


Shows the number of people (vertical axis) who performed that many evaluates (x-axis)

How many projects pass?


Another interesting metric is how often students actually pass the evaluation.  During the first few days, only 54% of evaluations passed. By adding ways for users to catch their own problems before submitting for review, we’ve helped turn this into 66% passing rate. This lead to less failed evaluations, and in general it feels better when your evaluation passes!

It's worth noting that evaluations are an iterative process. When a project is marked as needing more work, the user will typically make changes and request evaluation. So even without our changes, we'd expect to see a growing number of passing evaluations over time.
 

"Passing" evaluations (Green) vs. "Needs more work" evaluations (Blue)



Closely monitoring changes in data


We’ve caught up with evaluations vs. requests but we’ll continue to watch this data closely.

From what we’ve gathered so far, it seems like peer evaluations works well and people are getting good evaluations on their projects.  We'll be monitoring and focusing on quality of evaluations even more in the coming weeks.


Wednesday, July 9, 2014

Peer evaluations for coding projects on Khan Academy

In our programming courses on Khan Academy, we have two ways that students practice what they've learnt: challenges and projects.


A tutorial showing both challenges and a project


Challenges give you step by step instructions, and help guide you through learning a new concept - like "here's a black and white ice cream cup, color it in with fill commands." 

Projects are more free-form. They give you a goal to complete, and let you loose to be creative and get practice with programming - like "here's an empty plate, draw something to eat on it, using any of the commands you just learned."

Challenges can be checked for validity automatically using structured.js which parses a student's program into an Abstract Syntax Tree (AST). Each challenge has grading code that matches the user's code against known good structures and outputs messages for known bad structures.

But for projects, the student's code varies so much that they can't be graded as easily automatically - and that's a good thing, we love seeing the variety in the programs that pop up. We still wanted some way to judge projects as being done, however, and to give students feedback on what they'd written.

To do this, we've now introduced the concept of project evaluations: rubric-based code reviews that students can learn from and improve their program based on.


What's it like to be an evaluator?

If a member of our community is eligible to be an evaluator, they can visit our project feedback page to see a list of the projects awaiting evaluation.

For each project, there’s a list of objectives that the reviewer should look out for.  Each objective can be marked as "Passed" or "Needs more work".


Performing an evaluation

When there’s a failed objective, the evaluator can leave additional comments, to give the student ideas of ways they can meet that objective.

The evaluator can additionally leave additional comments at the end, which they often use to leave encouraging remarks or pro tips.
  


What's it like to be a student?

When a student starts on a project, they are given a summary of what to do and the project is marked as started.  

When they decide they're finished working on a project, they can request an evaluation.

What the person doing the project sees

After some time, the evaluation will happen, and if it's successful, the student  will then get a notification that their project is now marked as completed, or needs more work. If it needs more work, the student will make adjustments and re-request evaluation.  Once the project evaluation passes, the student will get points for their project.
 

How is eligibility determined for evaluators?

Community members who have earned the "Intro to JS mastery badge" are eligible to do peer evaluations. They can earn this badge by completing the "Intro to JS: Drawing & Animation" computer programming course. 

The eligibility may be tweaked in the future depending on the mean wait time of project evaluation requests and the quality of reviews.


How did the launch go?

So far, the launch has gone amazing. There's an initial surge of project evaluations to be caught up with, but we're hoping to keep the average wait time around a day.  We've immediately seen hundreds of project evaluation requests as well as dozens of awesome and supportive evaluations.  The advanced programmers in our community love the opportunity to help new programmers learn: 





They're also forthcoming with their praise for a well done program - both in terms of functionality and readability.






We've also got a ton of great feedback from the top evaluators, and we'll continue iterating on the system (just like the students are iterating on their projects) to help our community keep learning from each other.