Static Test Cases

Static test cases live in _expected_outputs.json. When a student submits, AlgoStudio calls their function once per case and compares the return value to the expected output. (For randomised cases, use dynamic tests instead — they take priority over this file when both exist.)

Format

A JSON array of pairs. Each pair is [[inputs], expected_output].

[
  [[arg1, arg2], expected],
  [[arg1, arg2], expected]
]

• inputs — always a list, even for a single argument
• expected_output — whatever the function should return

Example

Problem: add(a, b) returns the sum. Function name set to add (see the function-name contract).

[
  [[1, 2], 3],
  [[5, 5], 10],
  [[-1, 1], 0],
  [[0, 0], 0]
]

The runner calls add(1, 2) and checks the result equals 3, then add(5, 5) for 10, and so on. The same file drives both languages:

# Python — what the student completes
def add(a, b):
    return a + b

// C# — the student's class is always named Solution;
// instance or static methods both work
class Solution
{
    public int add(int a, int b) => a + b;
}

For C#, arguments from the JSON are coerced to the method's parameter types (including arrays), so [[1, 2], 3] cleanly calls add(int, int).

Single-argument functions still wrap the argument in a list: [[5], 120] calls factorial(5).

How results are compared

The default grading is return-value equality — per language:

	Python	C#
Comparison	result == expected	Structural: both sides JSON-serialized, strings compared
Float tolerance	None — 0.30000000000000004 != 0.3	None
List/array order	Matters	Matters
Dicts/objects	== (key order irrelevant)	Serialized form must match
Gotcha	A returned tuple ≠ the JSON list in the file — have students return lists	Strings/numbers must serialize identically

And what does not count:

• stdout never affects grading. In Python it's captured and shown to the student; in C# it's currently not captured at all.
• An exception = a failed case, not a crashed run: the message appears as that case's stderr and the run continues.
• A file that fails to load (syntax error, wrong function name) fails the whole run with a top-level error.

Need float tolerance, order-insensitive comparison, or anything fancier? Write a custom resolver.

Tips

• More cases = better feedback. Students see which passed and which failed. Aim for at least 4–6.
• Cover edge cases. Zeros, negatives, empty inputs, boundary values — bugs hide there.
• Simple cases first, edge cases last — the list renders in order.

Must be valid JSON

A syntax error in _expected_outputs.json stops the problem from running entirely. Use a JSON validator if you're unsure.