[dev] Submit — CS2680 Leaderboard, Assignment 4

CS2680 Leaderboard - Assignment 4

Submit

Name

A label for this submission, shown on your Past submissions page (the leaderboard shows your preferred name, or else your registered name). Up to 64 characters.

Number of tasks (optional)

Grade only the first k of the 26 tasks, e.g. for a quick check. Such a grading is not ranked on the leaderboard and does not count for the baseline, but it counts toward your gradings. Empty, or more than 26, runs all 26.

Submission: one zip holding src/ and engine/

What goes in the zip

submission.zip
├── src/                 your agent (agentic_loop.py, ...)
└── engine/              your server
    ├── engine.json      the engine and the model
    ├── launch.sh        starts the server
    ├── flags.txt        optional
    ├── sglang.patch     optional (or vllm.patch)
    ├── requirements.txt only for "engine": "other"
    └── files/           optional: text files

src/ and engine/ sit at the top of the zip, or both inside one folder. Anything else in the zip is not graded.
engine.json has exactly three keys. "format" is 1. "engine" is sglang, vllm or other. "model" is either a course catalog entry <org>/<name>@<40-hex revision> (the baseline is nvidia/Qwen3.6-35B-A3B-NVFP4@1355db6a052410cfd62085d94b58866fd0f2c3c5) or your model's folder, /shared/courseSharedFolders/176914outer/176914/a4-models/<your cluster username>/<folder>, which is copied when your grading starts.
launch.sh is run with bash in place of serve.sh or your own batch script; #SBATCH lines in it have no effect. It must have Unix line endings. Your server has 45 minutes from the start of the grading job to answer, installation included; the end-to-end time starts when it is ready.
The patch is git diff --no-renames output against the course's reference source, and must match "engine": for SGLang only files under python/sglang/ (not python/sglang/kernels/aot/), for vLLM only files under vllm/. No binary changes, symlinks or submodules.
For another framework ("engine": "other"), requirements.txt lists what to install. It is installed before your server starts, within the 45 minutes, and a failed install counts like a server that does not start.
engine/ holds nothing else (no lock files, wheels or weights: the environment is the course's, and you change the engine's source with a patch). Every file in it is UTF-8 text.
Sizes: the zip at most 50 MB; engine/ at most 2 MB, with launch.sh and requirements.txt at most 64 KB each, flags.txt 16 KB, the patch 1 MB, and files/ at most 32 files and 256 KB.

The upload is checked against these rules, and every problem is listed under the Submit button. A zip that is refused is not queued and does not count. Part 4 of the Assignment 4 page describes the submission.

Drop your zip here, or click to choose one

Any file name, as long as it is a zip archive. Up to 50 MB.

You can have one submission waiting or being graded at a time (cancel it on Past submissions), and 3 gradings a day, midnight to midnight Eastern Time (EST/EDT). A grading counts once its job has a GPU node, and so does one you cancel after that. Waiting for a GPU is waiting, not grading: a submission waits in the queue, then for a GPU node (which can take a while when the course's GPUs are busy), and cancelling it at any point before its node comes up does not count. Neither does a grading that fails the course's engine check or stops for a problem on the course's side.