Tests
The test suite is run by build.qdx after the compiler is built. It compiles small qdx programs with the new compiler and checks what happens.
bootstrap/qdx run std/*.qdx build.qdxThe last line counts the tests that passed and failed, and the exit status is 0 only if none failed. Each failure is printed with the test’s name and what went wrong.
Layout
| Directory | Listing | A test passes when |
|---|---|---|
tests/accept/ | tests/accept.expect | The program compiles with the standard library, and running it gives the listed exit status and output. |
tests/reject/ | tests/reject.expect | The compiler refuses the program with a message containing the listed text. |
tests/engine/ | tests/engine.expect | The same as accept, compiled with the standard library and the engine. |
A listing has one line per test.
main_result_is_exit_status 42
concat_joins_in_written_order 0 abcdThe first word is the test’s name, which is also its file name without .qdx. In an accept listing the second word is the exit status and the rest of the line, if any, is the exact standard output. In a reject listing everything after the name is the text the error must contain.
A listing must name exactly the programs in its directory. A file without a line, or a line without a file, fails the run.
Two more checks are written directly in build.qdx: one for the compiler’s run command, and one that runs the example program from the guide.
What earns a test
A test exists to protect one rule of the language, the library or the engine. Its name states the rule: subtraction_groups_left_to_right, match_must_cover_every_case.
Before adding one, name the change to the project’s own code that would make it fail. If there is no such change, the test is checking the processor or the kernel and does not belong.
Adding a test
- Write the smallest program that shows the rule, in the right directory, named after the rule.
- Add its line to the listing.
- Run the build.
Output has to fit on one line of the listing. A test whose natural output has several lines replaces the line ends before printing, as the engine tests do with with_byte_replaced.
A test that needs to see a program stop, or to read what it wrote to standard error, runs itself as a second process with run_program and reports on the result.
What a test may leave behind
Compiled tests and their output go in build/tests/. A test that writes a file writes it there, under its own name with an ending, so it cannot collide with another test or with its own executable.