Home | Sitemap | Recent Changes | Login

SPF Logo

Sender Policy Framework

Test Suite/Schema

This is an informal schema, illustrated with examples. The structure is agreed upon. A formal schema is underway, and the council will vote on naming issues if necessary.

The syntax is YAML. The top level object is a "scenario". A file can consist of multiple scenarios separated by '---' on a line by itself. Lexical comments are introduced by '#' and continue to the end of a line. Lexical comments are ignored. There are also comment fields which are part of a scenario, and used for purposes such as automating the annotated RFC. Here are two sample scenarios:

# Sample scenarios from pyspf test suite.
description: >-
  check trailing dot with redirect and exp
    spec:        8.1
    description: Check that trailing dot is accepted for domains
    helo:        msgbas2x.cos.example.com
    mailfrom:    test@example.com
    result:      pass
    spec:        8.1
    description: Check that trailing dot is not removed from explanation
    helo:        msgbas2x.cos.example.com
    mailfrom:    test@exp.example.com
    result:      fail
    explanation: This is a test.
    - SPF:  v=spf1 redirect=a.spf.example.com
    - SPF:  >-
            v=spf1 mx:example.com include:o.spf.example.com -exists:%{s}.S.bl.spf.example.com
            exists:%{s}.S.%{i}.AI.spf.example.com ~all
    - SPF:  v=spf1 ip4: ip4: ip4:
    - A:
    - A:
    - SPF:  v=spf1 redirect=%{d}.d.spf.example.com.
    - MX:   [10, msgbas1x.cos.example.com]
    - SPF:  v=spf1 exp=msg.example.com. -all
    - TXT:  This is a test.
description: test empty MX
    helo:        mail.example.com
    mailfrom:    ''
    result:      neutral
    - MX:   [0, '']
    - SPF:  v=spf1 mx

A scenario has four attributes: zonedata, tests, description, comment. description is a free form multi-line string. tests and zonedata are maps. comment is additional commentary about the test of less importance than the description.


Each key in the zonedata map is a DNS name, and its value is a list of maps, each with a single entry. The key of each single entry map is the DNS record type as text (upper case prefered). The value is a string or list. If the value of an SPF or TXT record is a string, it is promoted to a list of length 1. Note that the value of SPF or TXT is always a list of strings.

DNS errors

When the last entry for a DNS name is the string TIMEOUT, the test driver should simulate a DNS timeout exception for queries that do not match any preceding records. When the last map for a DNS name is the map RCODE: n, the test driver should simulate a DNS error with code n. The test suite currently uses TIMEOUT, but not RCODE.

Records of type SPF get special treatment. If no records of type TXT are given for the same DNS name, then an identical TXT record is also generated for the DNS data. This reflects the recommendation of section 3.1.1 and allows the test suite to be used with implementations that choose any of the three options in section 4.4. In addition, when the value of an SPF name is the string NONE, then that record is not added to the DNS data. As a result, TXT: NONE serves to suppress the auto copy of SPF records to TXT. This allows testing of record selection rules.


Each key in the tests map is the name of a test case. All the test cases in the scenario will use the DNS records provided in zonedata, and *only* those records. Your test driver should load the DNS records, and hook DNS lookups for your SPF library to use those records. If needed, you can load the records into a DNS server under a subdomain you control, for example scenario1._spftest.example.com, and arrange to append that subdomain to all lookups within your SPF library. A key design goal is for DNS data used by the tests to be contained within the test specification.

Each test case has four required attributes, helo, host, mailfrom, result. helo is an arbitrary string, and not necessarily a legal rfc2821 helo name. Remember, spammers and clueless senders put whatever garbage they want in HELO. host is an IP4 address in dotted decimal notation, or an IPV6 address in standard notation. If your implementation does not handle IP6, it should ignore test cases where host is not an IP4 address. Such a library should never get an IP6 connection. Note that even an IP4 only library must handle some IP6 syntax elements, for example dual-cidr-length. mailfrom is an arbitrary string. Again, your library must be able to handle garbage.

result is one of the defined SPF results as a lower case string. Your test driver should call your SPF library with the basic test case data, and verify that your library gets the expected SPF result. In some cases, because of ambiguity in RFC4408, there are multiple results. Multiple results are supplied as a list. The first result in the list is the "preferred" result, and is the result required according the majority interpretation of the RFC. Your test driver should print a warning if the library result is in the list, but is not the preferred result. You can, of course, ignore the warning if you disagree with the majority.

Optional attributes for test cases are spec, description, comment, and explanation. spec is a reference to section and paragraph of RFC 4408. It is a simple string, or a list of strings if there are multiple references. The format of spec references is section/paragraph/sentence, where paragraph and sentence are optional. Paragraphs are numbered using the official IETF text (despite some formatting errors) counting each block separated by blank lines or list of ABNF rules (ignoring page breaks) as a paragraph. For example, section 8.1 has 29 paragraphs, and 8.1/5 reads:

The following macro letters are expanded in term arguments:

If explanation is provided, and your implementation supports the exp modifier, then the test driver should verify that the failure explanation returned by your library matches. If the explanation attribute is the magic string DEFAULT, then the test driver should verify that the default explanation was returned. If the implementation supports setting the default explanation, this is easily accomplished by setting the default explanation to "DEFAULT". The test driver should ignore the explanation attribute for implementations that do not support the exp modifier.

Each test case should provide description as a brief (one or two sentence) description of what is being tested. The comment attribute should be used for an extended explanation.

Sample Driver

The python test driver used for pyspf may be helpful in creating drivers for other SPF libraries.

Edit text of this page | View other revisions
Last edited 2007-03-12 20:48 (UTC) by Stuart Gathman (diff)