You've been handed the unpleasant task of performing an audit of code written by your company's least-favorite contractor, Shoddycorp's Cut-Rate Contracting. The program is designed to read in gift card files in a legacy binary file format, and then display them in either a simple text output format, or in JSON format. Unfortunately, Shoddycorp isn't returning your calls, so you'll have to make do with the comments in the file and your own testing.
Justin Cappos (JAC) and Brendan Dolan-Gavitt (BDG) have read through the code already. It's a mess. We've tried to annotate a few places that look suspicious, but there's no doubt that there are many bugs lurking in this code. Make no mistake, this is not an example of C code that you want to imitate.
In order to complete this assignment, you are required to use the git VCS. Before beginning to write your code, you should first install git and clone the repository from GitHub classroom. The git binaries can be installed by your local package manager or at https://git-scm.com/downloads. For a cheat-sheet of git commands, please see https://github.com/nyutandononline/CLI-Cheat-Sheet/blob/master/git-commands.md. Although we will not be checking your commit messages or git log, it is recommended that you write descriptive commit messages so that the evolution of your repository is easy to understand. For a guide to writing good commit messages, please read https://chris.beams.io/posts/git-commit/ and the Linux kernel's advice on writing good commit messages.
After git is installed, you will want to configure your git user to sign your commits. This can be done by following the guide at https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work. You will also need to add your GPG public key to your GitHub profile, and make sure that the email address set in your GitHub account matches the one you specified when generating your keys. You can find more information about about this in GitHub's documentation on commit signature verification. To avoid having to type in your password all the time you, may also want to set up SSH key access to your GitHub account.
After accepting the invitation on GitHub Classroom, you can clone the assignment repository by clicking the green "Code" button, copying the repository URL under "Clone", and then running:
git clone <your_repository_url>
Note that if you have set up an SSH key, you will want to make sure you
copy the SSH URL (which looks like git@github.com:NYUAppSec/...).
The next step is to set up Github Actions to automatically build and test your code when you push a commit. You can find a tutorial on GitHub Actions here.
For now, you should set up GitHub Actions to just run
echo "Hello world"
whenever a new commit is pushed to the repository. To do this, you'll create a
file named .github/workflows/hello.yml. Check that the Action is running
correctly in the GitHub interface.
Important Note: Because GitHub Actions has a classroom-wide limit on the
number of minutes, we have set up our own GitHub Actions runner that is not
subject to those limits. To use it, just change the runs-on line in your
actions YML file to:
runs-on: self-hosted
Read through the giftcardreader.c program (and its accompanying
header, giftcard.h) to get a feel for it. You should also try building
and running it with the included examplefile.gft file, to see what its
normal output is. You may find it helpful to use a debugger like gdb
to step through the program as it executes, to get an understanding of
its normal flow of control.
For this part, your job will be to find some flaws in the program, and then create test cases (i.e., binary gift cards) that expose flaws in the program. You should write:
crash1.gft and crash2.gft, that cause the
program to crash; each crash should have a different root cause.hang.gft, that causes the program to loop
infinitely. (Hint: you may want to examine the "animation" record type
to find a bug that causes the program to loop infinitely.)bugs.txt explaining the bug triggered by each of your
three test cases.To create your own test files, you may want to refer to the gengift.py
and genanim.py programs, which are Python scripts that create gift
card files of different types.
Finally, fix the bugs that are triggered by your test cases, and verify
that the program no longer crashes / hangs on your test cases. To make
sure that these bugs don't come up again as the code evolves, have
Github Actions automatically build and run the program on your test suite.
There are a few ways to do this, but the simplest is to modify the Makefile's
existing test target to run the giftcardreader on your gift card files
and then have GitHub Actions run make test. Note that you do not need
to run your tests on the unfixed version of the code---the tests are intended
to verify that the code is fixed and prevent the bugs from being reintroduced
in later versions.
As discussed in class, an important part of understanding how well your
test suite exercises your program's behaviors is coverage. To start
off, measure the coverage that your program achieves with the test cases
you created in Part 2. To do this, you should build giftcardreader
with the --coverage option to gcc, run your test suite, and then
produce a coverage report using lcov (details on how to do this can be
found in the lecture slides).
You should notice that there are portions of the program that are uncovered (i.e., the code was not executed while processing your test suite). Pick two lines of code from the program that are currently not covered and create test cases that cover them.
An easy and effective way of finding crashes and getting higher coverage
in a program is to fuzz it with a fuzzer like AFL++. Fuzz the program
using AFL++, following the quick-start
instructions. To
make the fuzzing more effective, you should provide AFL
with all of the test files you have created in its input directory. Let
the fuzzer run for at least two hours, and then examine the test cases
(in the queue directory) and crashes/hangs (in the crashes and
hangs directories).
Add the non-crashing test cases to your test suite, and produce a new coverage report. You should see that the tests generated by the fuzzer reach more parts of the gift card program.
Finally, pick two crashes/hangs and fix the bugs in the program that cause them. You should include these test cases in the tests you run in GitHub Actions.
To complete the assignment, commit your updated code, your handwritten tests, the fuzzer-generated tests, and a brief writeup explaining the bugs you found and fixed in this part.
Makefile also includes a target that will build the gift card reader
using ASAN, which you can invoke with make asan.gdb or lldb debuggers;
guides and tutorials can be found online. Your IDE (if you use one) may also
provide a built-in debugger.Despite the fixes you've made, there are almost certainly still many bugs lurking in the program. Although it is possible to get to a secure program by repeatedly finding and fixing bugs (at least when the program is this small), it's a lot of work, and just because a fuzzer stops finding bugs doesn't mean that the program is bug-free!
Realistically, this program is probably unsalvageable in its current state. It would be better in this case to rewrite it from scratch, either in C using a very defensive programming style, or in a safer language like Python or Rust. In the "clean" directory, you can find a cleanly-written version of the program (written in C) that should be relatively bug-free [1]. You'll notice that it's a lot more verbose, and checks for many more errors than the buggy version---writing safe C code is difficult!
[1] Although you are encouraged to try to prove us wrong by finding bugs in it!