Read first
Desktop
Running on try
To run the Reftests on try, add the following try: block to the end of the topmost commit message when you push to try.
try: -b do -u reftest
For more information, refer to the TryChooser documentation.
Running with build environment
First, setup a Firefox build environment. In the following examples, $(OBJDIR) is the build's object directory. This is prefixed with obj-, and the exact name depends on your system and OS.
Run all tests
$ make -C $(OBJDIR) reftest
Run a subset of tests
$ TEST_PATH=path/from/sourcedir/reftest.list make -C $(OBJDIR) reftest
Running with built packages
First, download and extract the built test packages. The following commands require the extracted tests path as the working directory.
$ python reftest/runreftest.py --appname=$(APPLICATION) --utility-path=bin --extra-profile-file=bin/plugins reftest/tests/layout/reftests/reftest.list
Substitute $(APPLICATION) with the relative or absolute path to the Firefox binary. For example on Windows this might be firefox/firefox.exe and on Mac OS FirefoxNightly.app/Contents/MacOS/firefox-bin.
Manifests
The final argument is the manifest file, which determines the tests to be run. If you don't want to use the {{ Source("layout/reftests/reftest.list", "default manifest") }}, simply change this argument to point to an alternative. The format of the manifest is covered in Writing Reftests.
Crash diagnosis
If you need to diagnose crashes that occur during the testrun then you will need to include the --symbols-path argument. This is an absolute path to a directory containing breakpad symbols, or the URL of a zip file containing symbols. The tinderbox and try builds include these symbols, and can be found by the file ending with crashreporter-symbols.zip.
Mobile
Running on try
Running the Reftests on try against mobile is the same as for desktop, however you can limit the platform to only run the mobile Reftests. To do this, include the -p argument in the commit message when you push to try.
try: -b do -p android,android-armv6 -u reftest
For more information, refer to the TryChooser documentation.
Running with build environment
It is currently recommended to run mobile reftests on try. For details on how to run reftests agains a local build, see this documentation.
Running with built packages
This is not currently recommended. See documentation for running on try or with a build environment.