Autograding Using Jupyter Notebooks

Autograding Using Jupyter Notebooks

Did you know you could use Jupyter notebooks to create and autograding assignments? The community developed tool nbgrader provides a low level API that can be used to create assignment notebooks that can be distributed to students. After the students submit the completed assignments, they can be graded automatically! Without further ado, to see how this works, let’s create an assignment with a few simple problems along with automatic tests.

We create a notebook, Python 2 in this case, and then enable the “Create Assignment” cell toolbar:

create assignment cell toolbar

Enabling this will allow us to mark cells as several types such as:

We can now start creating the assignment. First, we should include some functions from the nose library for writing tests:

from import assert_equal, assert_raises

Problem: Write a function average() that accepts a list of numbers and returns the average. If an empty list is supplied, return 0 and if the input is not a list, raise a TypeError.

We write our solution in a cell marked as Autograded answer:

def average(numbers):
    if type(numbers) is not list:
        raise TypeError('Input must be a list!')
    if len(numbers) == 0:
        return 0
    sum = 0
    for n in numbers:
        sum += n
    average = sum/float(len(numbers))
    return average

Note the ### BEGIN SOLUTION and ### BEGIN SOLUTION markers in our solution code. These indicate that the solution code is contained between them. When we release the assignment, code between these markers will be removed and replaced by raise NotImplementedError().

We can write tests for this problem in a cell marked as Autograded tests and assign 10 points to it:

assert_equal(average([2, 3, 4]), 3.0)
assert_equal(average([]), 0)
assert_raises(TypeError, average, 2)

Problem: Write a function factorial() that returns the factorial of a number. If the input number is 0, return 0. If the input number is a list or not an integer, raise a TypeError.

Again, we write our solution in a cell marked as Autograded answer:

def factorial(n):
    if type(n) == list:
        raise TypeError('Input cannot be a list!')
    elif type(n) == float:
        raise TypeError('Input cannot be a float!')
    if n == 0:
        return 0
    if n==1:
        return 1
        return n*factorial(n-1)

and tests in a cell marked as Autograded tests and assign 10 points to it:

assert_equal(factorial(1), 1)
assert_equal(factorial(4), 24)
assert_raises(TypeError, factorial, [1, 2, 3])

In addition to the tests already written, we also add some hidden tests in the last cell and assign an additional 10 points to it:

assert_equal(average([2, 2, 4.5, 5, 50]), 12.7)
assert_equal(factorial(0), 0)
assert_raises(TypeError, factorial, 1.5)

To make sure the solutions we have written pass all our tests, we validate the assignment using the “Validate” button in the toolbar at the top:

When we release this assignment to students, here is what the student version looks like:

Note that the solution code has been removed in both functions and the hidden tests (that were in the last cell) have been removed. This last cell which contained the hidden tests will be a read-only cell that students will not be able to modify.

Once the student completes the assignment and submits it, the instructor can simply autograde the assignment as shown here.


  1. pdfs vom iphone auf pc says:
    Your comment is awaiting moderation.

    You revealed it superbly.

  2. Fabien says:
    Your comment is awaiting moderation.

    Perfect ! Wonderful idea / work guys !

  3. […] To learn more about Cloud Jupyter Notebooks, read this blog post! […]

  4. Abhilash P says:

    Awesome work guys..! Love you..!

Leave a Comment

Your email address will not be published. Name and email are required.