Home | Sitemap | Recent Changes | Login

SPF Logo

Sender Policy Framework

Test Suite

Showing revision 23

Releases

The latest test suite release is 2007.01. A bug was fixed that caused TXT only implementations to fail several tests incorrectly due to a TXT record blocking the automatic copy of SPF to TXT records for those tests. In addition, more test cases for exp and redirect handling have been added.

The first test suite release was 2006.11. The test cases are specificied in the YAML syntax using this schema.

You can also try the latest version.

Development

The following describes the procedures and work environment used for developing the official SPFv1 implementation test suite.

Discussion

Discussion about the test suite and its development should happen on the spf-devel mailing list.

Goals

The test suite should...

  • be pure, implementation-independent data that can be used offline without access to the DNS.
  • be available online in the DNS, too.
  • cover all the RFC 2119 keywords (MUST, SHOULD, RECOMMENDED, MAY) in RFC 4408 that are relevant to SPF implementations (some are not); there should be at least one positive and one negative test for each keyword.
  • cover other important aspects (syntax, tables, common mistakes, tricky corner cases, etc.) of the specification that are relevant to SPF implementations.
  • optionally cover non-definitive best practices (e.g. strict vs. lax interpretation) that are relevant to SPF implementations.

Development Roadmap

  1. Gather all the old test data that is available from:
    1. Wayne's test suite
    2. the libspf2 Subversion repository
    3. the spf1-test.mailzone.com. and spftest.org. DNS zones
  2. Create an annotatable version of the SPF specification so tests can be added inline.
  3. Evaluate test cases from the old test data with regard to suitability for testing the relevant parts of the specification, or invent new test cases.
  4. Define a suitable format for a text database of test cases with comments. The official format is YAML (which can be automatically converted to XML if needed) using this schema.
  5. Create a formal schema for the test cases.
  6. Create the text database of test cases.
  7. Create a logo authorized for use only by implementations that pass the test suite.
  8. Create a script to load test case data into an online DNS server suitable for testing implementations that cannot hook DNS lookups.
  9. Write a script to merge test cases into the annotatable specification, either statically or on demand (CGI).
  10. Apply the test suite to existing implementations to find any remaining deviations of the test suite from the specification and to determine the implementations' standards compliance.

Revision Control Repository

The actual test suite is maintained in a Subversion repository:

ComponentRepository URLBrowserAccess
SPFv1 Test Suitehttp://www.openspf.org/svn/project/test-suite/browseeveryone (r), council members (rw)