Checking Memory Leakage of Student Programs

So far my assignment grading system uses a legacy (and buggy) library written by some former TA to check memory leakage of student programs. Now that it’s time for a major OS upgrade (Ubuntu 14.04), this solution exposed more bugs. Time to retire it and make my own ones.

The Problem

Given some buggy student code, how can you write a program that can tell whether there is memory leaked or not without human interference?

#include <stdio.h>
#include <stdlib.h>

int main(void) {
    char *p = malloc(12);
    *p = 'a';
    //let's add some segfault
    *(p = NULL) = 'b';
    return 0;
}

Note that in the program, the segfault may prevent the automated process from going as planned.

Solution A: Wrappers

Use preprocessor to wrap malloc/calloc/realloc/free and take notes before calling the actual functions:

#define malloc(x) gd_malloc(x, __LINE__, __FILE__)
#define free(p) gd_free(p, __LINE__, __FILE__)

Notes for this approach:

• Must check to make sure that student code includes the header.

Solution B: Valgrind

Use pipe to feed input, and call valgrind. Use regular expression to grab critical information from the report.

cat input.txt | valgrind ./memleak

The valgrind information will be printed to stderr.

A sample memleak report by valgrind:

==1460== HEAP SUMMARY:
==1460== in use at exit: 20 bytes in 1 blocks
==1460== total heap usage: 1 allocs, 0 frees, 20 bytes allocated

A sample memfree report by Valgrind:

==1464== HEAP SUMMARY:
==1464== in use at exit: 0 bytes in 0 blocks 
==1464== total heap usage: 1 allocs, 1 frees, 20 bytes allocated 
==1464== 
==1464== All heap blocks were freed -- no leaks are possible

Known issues:

• Unacceptably slow, particularly because of the resource restriction (CPU and memory use, time limit, etc.) on the grading environment.

• Student could print to stderr and confound grading result.

Solution C: mtrace

mtrace adds handlers to malloc/calloc/realloc/free and logs the events when they get called. One analyzes the log to decide if memory is leaked or not.

Notes:

• Need to inject mtrace header.

• Need a log parser.

Solution D: my own library

libmemchecker is a series of C libraries (or more precisely, hooks) that record the function calls of dynamic memory allocation, and parse the dump logs to analyze if the tested program leaks memory at exit.

They are lightweight and fit better in automated, jailed testing scenario than valgrind, which may use a lot more resources (exceeding resource limit) and run much more slowly (causing execution time-out).

There are three independent libraries in the repository and please read README.md for more details.

GitHub Repository: https://github.com/xybu/libmemchecker

Please note that they support only C source code, and it is unknown how it works on C++ programs.