Below you will find a list of the various automated testing frameworks in use at Mozilla, along with a brief description and a link to read more. The letters in parentheses are the abbreviations used by TBPL.
Generally, you should pick the lowest-level framework that you can. If you are testing JavaScript but don't need a window, use XPCShell. If you're testing page layout, try to use Reftest. The advantage is that you don't drag in a lot of other components that might have their own problems, so you can hone in quickly on any bugs in what you are specifically testing.
If you have questions, join #automation on irc.mozilla.org or post to the public mailing list.
| AutoPhone | a platform for running automated tests on physical mobile devices. |
| Compiled-Code (B) | written in C++, compiled-code tests can test pretty much anything but are difficult to write properly. |
| Crashtest (C) | If you've found a page that crashes Firefox, add a crashtest to make sure future versions don't experience this crash again. |
| Eideticker | an automated test harness that captures and analyzes browser output via HDMI. |
| Firebug Test Runner | automates the process of running the Firebug tests. |
| JS Reftest | used to run the JavaScript tests in the browser using the Reftest framework. |
| JS Shell (J) | test every piece of the JavaScript engine. |
| Marionette | used to drive the UI and JS/XPCOM layer of remote or local instances. |
| Mochitest: A11y (Moth) | tests accessibility interfaces. |
| Mochitest: Browser Chrome (Moth) | runs in the scope of the browser window, this is a rough UI automation tool testing how the browser interacts with itself and with content. |
| Mochitest: Chrome (Moth) | run with high privileges that can test a lot of the browser's functionality. Tests that verify JavaScript interactions with chrome-level objects should go here. |
| Mochitest: IPC (Moth) | tests plugin APIs, particularly out-of-process plugins. |
| Mochitest: Plain (M) | used to test DOM APIs and other pieces of functionality exposed to web content (i.e. requiring no special permissions). |
| Mochitest: Robocop (Mrc) | designed for testing the native UI of Android devices by sending events to the front end. |
| Mozmill | an extensive framework for browser automation. |
| Peptest | measures responsiveness, how "snappy" Firefox/Thunderbird feels, by issuing alerts when the event loop is stuck for more than 50 ms. |
| Reftest (R) | compares the display of two web pages. |
| Reftest: IPC (Ripc) | identical to Reftest, but running in a separate process. |
| SpeedTests | a framework for executing arbitrary tests, which report results via JavaScript calls, in several browsers. |
| Talos (T) | a framework for performance testing. |
| TPS | a multi-profile test automation framework for Firefox Sync. |
| XPCShell (X) | console JavaScript tests. There is no chrome, no content, no window. XPCShell is useful for testing low-level things, such as XPCOM components. |