<!--
Copyright (C) Internet Systems Consortium, Inc. ("ISC")

SPDX-License-Identifier: MPL-2.0

This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0.  If a copy of the MPL was not distributed with this
file, you can obtain one at https://mozilla.org/MPL/2.0/.

See the COPYRIGHT file distributed with this work for additional
information regarding copyright ownership.
-->

# BIND9 System Test Cookbook

Task-oriented recipes for the system test framework.  Each recipe is
self-contained and shows working code that can be copied and adapted.
For concepts and reference documentation (build setup, fixtures, runner
internals), see [README.md](README.md).

Prerequisite for all recipes: a built tree and the test network interfaces
configured — see "Running the Tests" in the README.


## Iterate on a single test

The basic loop while developing a test:

```sh
cd bin/tests/system
pytest mytest                                     # whole directory
pytest mytest/tests_mytestmod.py::test_one_case   # one test function
```

When a test fails, its temporary directory is kept and a stable symlink to it
is created (e.g. `mytest-mytestmod` for `mytest/tests_mytestmod.py`).  Look
there for `pytest.log.txt` (the test's own log) and `ns*/named.run` (each
server's debug-level log).  Pass `--noclean` to keep the directory even on
success.

To poke at the servers interactively, pause the test at any point by
inserting a breakpoint, or start pdb right away with `--trace`; to get a pdb
prompt automatically when a test fails, use `--pdb`.  In all cases the
servers keep running while the test is paused, so their state can be
inspected:

```sh
pytest mytest --trace   # pdb at the start of each test
pytest mytest --pdb     # pdb when a test fails
```

Timing-sensitive bugs hide in system tests: before declaring a test done, run
it several times (and ideally once under parallel load, `pytest -n auto`).
A test that flakes is treated as a bug in the test.


## Add a new system test directory

This skeleton sets up one authoritative server and queries it.  Pick a name
that starts with a letter and uses underscores as word separators (hyphens
are not allowed), and create:

```
demo/
├── ns1/
│   ├── named.conf.j2
│   └── example.db
└── tests_demo.py
```

`demo/ns1/named.conf.j2` — the config template; the runner renders it to
`named.conf` at setup time, filling in the assigned ports:

```jinja
options {
	{% include_indented "_common/options.conf.j2" %}
	recursion no;
	dnssec-validation no;
};

{% include "_common/controls.conf.j2" %}

zone "example" {
	type primary;
	file "example.db";
};
```

The `_common/options.conf.j2` include expands to the standard plumbing
(configuring interfaces, source addresses, port and pid file).  Write only the
options your test actually cares about below the include.  When the standard
block doesn't fit, compose the pieces from `_common/options/` instead and put
anything nonstandard inline.

The plumbing templates are built on the `ns` variable: templates inside an
`nsN`/`ansN` subdirectory automatically get one describing that server, so a
config never hardcodes its own address — `@ns.ip@` renders to 10.53.0.1 in
ns1, 10.53.0.2 in ns2, ..., and `@ns.ip6@` to the matching
fd92:7065:b8e:ffff::N address.  Use it wherever a config refers to the
instance's own address; references to *other* servers stay literal.

The `_common/controls.conf.j2` include sets up the rndc control channel, so
the test (and the runner's shutdown sequence) can use `rndc`.  A resolver
instance would also include `_common/root.hint.conf` to get the standard
root hints (ns1 is the root server by convention).  Inside an indented
section such as a view statement, use `{% include_indented "..." %}` instead
of `{% include %}` — it aligns the inserted block with the tag's own
indentation.

`demo/ns1/example.db` — a plain zone file:

```
$TTL 300
example.        IN SOA  ns1.example. hostmaster.example. 1 600 600 1200 600
example.        NS      ns1.example.
ns1.example.    A       10.53.0.1
a.example.      A       10.0.0.1
```

`demo/tests_demo.py` — the test module:

```python
import isctest


def test_a_record(ns1):
    msg = isctest.query.create("a.example.", "A")
    response = isctest.query.udp(msg, ns1.ip)
    isctest.check.noerror(response)
    assert str(response.answer[0][0]) == "10.0.0.1"
```

That's it — run it with `pytest demo`.  The pytest runner discovers new
directories automatically; to also include the test in `make check` runs,
add it to `TESTS` in `Makefile.am`.

Notes:

- The `ns1` fixture is the started server (an
  `isctest.instance.NamedInstance`); `ns1.ip` is 10.53.0.1.  Servers in
  `nsN`/`ansN` subdirectories are started automatically.
- If your test writes files beyond the usual ones (e.g. `dig.out.*`,
  `dsset-*`), declare them as a module-level marker —
  `pytestmark = pytest.mark.extra_artifacts(["dig.out.*"])` — or the
  artifact check at teardown will fail.  Most real test modules carry one.


## Set up zones in bootstrap()

A module-level `bootstrap()` function runs before the config templates are
rendered and before the servers start.  This is where zone files and DNSSEC
keys can be generated; whatever dict it returns becomes template data.

The simplest case is a plain unsigned zone — use this instead of writing
the same boilerplate zone file by hand in every `nsN` directory:

```python
from isctest.template import NS1, zones
from isctest.zone import Zone


def bootstrap():
    zone = Zone("example", NS1)    # signed=False is the default
    zone.configure()               # render the zone file
    return {"zones": zones([zone])}
```

`Zone.configure()` renders the zone file from
`_common/zones/template.db.j2.manual`, a generic zone with SOA, NS, and a
few test records.  To provide your own content, drop a template named after
the zone file — `ns1/zones/<name>.db.j2.manual` — and `configure()` picks
it up automatically instead of the generic one.  (The `.j2.manual` suffix
keeps the file out of the runner's automatic template pass, which renders
every plain `*.j2` at setup time; here `bootstrap()` renders it instead.)

For DNSSEC-signed zones, pass `signed=True` and make sure to configure the
`trust_anchors`:

```python
from isctest.template import NS1, NS2, zones
from isctest.zone import Zone, configure_root


def bootstrap():
    zone = Zone("signed-example", NS2, signed=True)
    zone.configure()           # keys, zone file, dnssec-signzone

    # root zone on ns1, delegating to (and providing DS records for)
    # the zone above
    root = configure_root([zone])

    return {
        "trust_anchors": root.trust_anchors(),
        "zones": zones([root, zone]),
    }
```

With `signed=True`, `configure()` also generates a KSK+ZSK with
`dnssec-keygen` and signs the rendered zone with `dnssec-signzone`.  To
sign a hand-written zone file checked into git (no rendering at all), put
it at `ns2/zones/<name>.db` and drive the steps yourself:

```python
def bootstrap():
    zone = Zone("signed-example", NS2, signed=True)
    zone.add_keys()
    zone.sign("-3 ABCD")       # extra dnssec-signzone args, e.g. NSEC3

    root = configure_root([zone])
    return {
        "trust_anchors": root.trust_anchors(),
        "zones": zones([root, zone]),
    }
```

On the template side, the `_common` includes consume the returned data: the
authoritative server's `named.conf.j2` ends with

```jinja
{% include "_common/controls.conf.j2" %}
{% include "_common/zones.conf.j2" %}
```

(`zones.conf.j2` emits a `zone` block for every zone whose nameserver
matches the rendering server) and the validating resolver's with

```jinja
{% include "_common/controls.conf.j2" %}
{% include "_common/trusted.conf.j2" %}
{% include "_common/root.hint.conf" %}
```

(`trusted.conf.j2` emits `trust-anchors` from the `trust_anchors` data, and
`root.hint.conf` points the resolver at ns1 for the root zone).

The `dnssec_py` directory is the canonical example of this pattern,
including several modules with different zone setups sharing one directory.
Declare the generated files as artifacts:

```python
pytestmark = pytest.mark.extra_artifacts(
    [
        "ns*/dsset-*",
        "ns*/trusted.conf",
        "ns*/zones/*.db",
        "ns*/zones/*.db.signed",
    ]
)
```


## Drive named and watch its logs

The `NamedInstance` fixtures (`ns1` ... `ns11`) control the running servers.
The cardinal rule: never `time.sleep()` waiting for the server — watch the
log for the line that proves the event happened.

Wait for a log line caused by an action — enter the watcher *before*
triggering the action, so the line cannot slip past unobserved:

```python
def test_cache_flush(ns4):
    with ns4.watch_log_from_here() as watcher:
        ns4.rndc("flush")
        watcher.wait_for_line("flushing caches in all views succeeded")
```

`wait_for_line()` accepts a string, a compiled regex, or a list of either
(any match wins) and returns the `re.Match`.  `wait_for_all([...])` waits
until every pattern has appeared, `wait_for_sequence([...])` requires them
in order.  To match lines that may already have been logged (e.g. during
startup), use `watch_log_from_start()` instead.

Reconfigure a server mid-test by re-rendering its config template with new
data, then reloading.  `reconfigure()` and `reload()` send the rndc command
and wait for the corresponding completion line in the log:

```python
def test_with_new_config(ns1, templates):
    templates.render("ns1/named.conf", {"flag": True})
    ns1.reconfigure()
```

Dynamic updates go through `nsupdate`:

```python
import dns.update

def test_update(ns1):
    update = dns.update.UpdateMessage("example.")
    update.add("added.example.", 300, "A", "10.0.0.2")
    ns1.nsupdate(update)
```

Send queries and check the responses with `isctest.query` and
`isctest.check`:

```python
msg = isctest.query.create("a.example.", "A")
response = isctest.query.udp(msg, ns1.ip)     # or .tcp(), .tls()
isctest.check.noerror(response)               # rcode checks: nxdomain,
                                              #  servfail, refused, ...
isctest.check.adflag(response)                # flag checks: rdflag,
                                              #  noraflag, ...
isctest.check.same_answer(response, expected) # compare two responses
```

To wait until a zone transfer has happened, poll the SOA serial:

```python
isctest.query.wait_for_serial(ns2.ip, "example.", expected_serial)
```

For grepping a file the server produced (dumps, statistics), use
`isctest.text`:

```python
from re import compile as Re

ns4.rndc("dumpdb -all")
dump = isctest.text.TextFile("ns4/named_dump.db")
assert dump.grep(Re(r"^a\.example\..*10\.0\.0\.1"))
```