Traditionally, testing Kaitai Struct relied on running specs which do the following:
for every target language
an input binary file,
a library generated by Kaitai Struct Compiler for that target language
we run library’s parsing procedure on an input file, getting ourselves some structure in memory with the parsed contents of input file
finally, we compare the actual results of parsing with expected result: we take certain key members of structure in memory and assert that they should be equal to what we expect, or within certain boundaries.
While the procedure itself is solid, it quickly becomes very labor-intensive to support that, as number of languages that KS supports grows. Adding a new test now requires addition of ~10-12 per-language spec files, and doing that manually is both inefficient and error-prone.
The solution for that comes from Kaitai Struct compiler itself: we already have an expression language that might be used to address structures in memory, so why not use that to generate test specs as well?
Cross-language test specs are described in KST (=Kaitai Struct Test) format, which is also a YAML-based format, similar to KSY.
id: foo_bar data: expr_array.bin asserts: - actual: aint_size expected: 4
idis a name of a test spec, which MUST match exactly the name of a KSY file in formats dir.
datais the name of input binary datafile, which MUST be present in src dir; by convention, we use ".bin" file extension for all these files.
assertsis a sequence of assertions that we’ll use to check that contents of parsed structure in memory match our expectations. Every assertion has two parts:
actualis a KS expression, executed in context of root element of KSY file that we’re tested, used to extract a certain member of structure in memory to check.
expectedis normally some constant that we expect it to match (but it is also a KS expression, so technically it can be non-constant).
Assertions are expected to be fatal (i.e. first failed assertion stops
the test process) and are specifically executed in sequence in
This file must be saved as
foo_bar.kst (i.e. name MUST match the
id and the name of relevant .ksy file) in
When test is expected to fail with an exception, a slightly different KST file is used:
id: valid_fail_contents data: fixed_struct.bin exception: ValidationNotEqualError<bytes>
asserts are present in this case, but there is a top-level
exception element, which lists the expected exception to be raised
during the parsing stage of this test. Available exception names are
Invoking KST translator
To make use of KST specs, they need to be translated into actual target languages (just like KSY gets compiler into target languages), so there is a translator project to do that.
KST translator is heavily based of kaitai-struct-compiler, so one needs to compile and publish .jar files with kaitai-struct-compiler to local repository first:
# assuming we're in root project directory cd compiler sbt publishLocal
after that, one can invoke it using [spec-kst-to-all](https://github.com/kaitai-io/kaitai_struct_tests/blob/master/spec_kst_to_all) script in tests:
For example, to generate specs in all possible target languages right in the all relevant dirs for one KST spec "foo_bar", one can run:
cd ../tests ./spec-kst-to-all -f foo_bar
Developing KST translator