pub repository

mirror.p-hatan.com

test

1.25.7

A full featured library for writing and running Dart tests across platforms.

Download archive

Readme

pub package package publisher

test provides a standard way of writing and running tests in Dart.

Using package:test

Writing Tests
Running Tests
Asynchronous Tests
Running Tests With Custom HTML
- Providing a custom HTML template
Configuring Tests
Tagging Tests
Debugging
Browser/VM Hybrid Tests
Support for Other Packages
- build_runner
- term_glyph
Further Reading

Writing Tests

Tests are specified using the top-level test() function. Test asserts can be made using expect from package:matcher

import 'package:test/test.dart';

void main() {
  test('String.split() splits the string on the delimiter', () {
    var string = 'foo,bar,baz';
    expect(string.split(','), equals(['foo', 'bar', 'baz']));
  });

  test('String.trim() removes surrounding whitespace', () {
    var string = '  foo ';
    expect(string.trim(), equals('foo'));
  });
}

Tests can be grouped together using the group() function. Each group's description is added to the beginning of its test's descriptions.

import 'package:test/test.dart';

void main() {
  group('String', () {
    test('.split() splits the string on the delimiter', () {
      var string = 'foo,bar,baz';
      expect(string.split(','), equals(['foo', 'bar', 'baz']));
    });

    test('.trim() removes surrounding whitespace', () {
      var string = '  foo ';
      expect(string.trim(), equals('foo'));
    });
  });

  group('int', () {
    test('.remainder() returns the remainder of division', () {
      expect(11.remainder(3), equals(2));
    });

    test('.toRadixString() returns a hex string', () {
      expect(11.toRadixString(16), equals('b'));
    });
  });
}

You can use the setUp() and tearDown() functions to share code between tests. The setUp() callback will run before every test in a group or test suite, and tearDown() will run after. tearDown() will run even if a test fails, to ensure that it has a chance to clean up after itself.

import 'package:test/test.dart';

void main() {
  late HttpServer server;
  late Uri url;
  setUp(() async {
    server = await HttpServer.bind('localhost', 0);
    url = Uri.parse('http://${server.address.host}:${server.port}');
  });

  tearDown(() async {
    await server.close(force: true);
    server = null;
    url = null;
  });

  // ...
}

Running Tests

A single test file can be run just using dart test path/to/test.dart (as of Dart 2.10 - prior sdk versions must use pub run test instead of dart test).

Single file being run via "dart test"

Many tests can be run at a time using dart test path/to/dir.

Directory being run via "dart test".

It's also possible to run a test on the Dart VM only by invoking it using dart path/to/test.dart, but this doesn't load the full test runner and will be missing some features.

The test runner accepts one or more path arguments. If there are no path arguments the runner defaults to running tests under the test/ directory. When running for test/ or any other directory, the runner will recursively search the directory for files that match the test name pattern *_test.dart. The pattern can be overridden in dart_test.yaml. When a path argument is a file instead of a directory it will be run as a test, regardless of the file name. Arguments which use shell globbing should avoid including non-test files in the path argument. Tests which are run by other test runners may use a different default path, such as integration_test/. Directories other than test/ are ignored by dart test unless passed explicitly as a path to run.

You can select specific tests cases to run by name using dart test -n "test name". The string is interpreted as a regular expression, and only tests whose description (including any group descriptions) match that regular expression will be run. You can also use the -N flag to run tests whose names contain a plain-text string.

By default, tests are run in the Dart VM, but you can run them in the browser as well by passing dart test -p chrome path/to/test.dart. test will take care of starting the browser and loading the tests, and all the results will be reported on the command line just like for VM tests. In fact, you can even run tests on both platforms with a single command: dart test -p "chrome,vm" path/to/test.dart.

By default each platform has a default compiler, but some of them support more than one compiler. You can choose which compiler to use by passing dart test -c source, which would run all VM tests from source instead of compiling them to kernel. This also supports targeting a specific platform using normal platform selectors, like this dart test -c vm:source.

Test Path Queries

Some query parameters are supported on test paths, which allow you to filter the tests that will run within just those paths. These filters are merged with any global options that are passed, and all filters must match for a test to be ran.

name: Works the same as --name (simple contains check).
- This is the only option that supports more than one entry.
full-name: Requires an exact match for the name of the test.
line: Matches any test that originates from this line in the test suite.
col: Matches any test that originates from this column in the test suite.

Example Usage: dart test "path/to/test.dart?line=10&col=2"

Line/Col Matching Semantics

The line and col filters match against the current stack trace taken from the invocation to the test function, and are considered a match if any frame in the trace meets all of the following criteria:

The URI of the frame matches the root test suite uri.
- This means it will not match lines from imported libraries.
If both line and col are passed, both must match the same frame.
The specific line and col to be matched are defined by the tools creating the stack trace. This generally means they are 1 based and not 0 based, but this package is not in control of the exact semantics and they may vary based on platform implementations.

Sharding Tests

Tests can also be sharded with the --total-shards and --shard-index arguments, allowing you to split up your test suites and run them separately. For example, if you wanted to run 3 shards of your test suite, you could run them as follows:

dart test --total-shards 3 --shard-index 0 path/to/test.dart
dart test --total-shards 3 --shard-index 1 path/to/test.dart
dart test --total-shards 3 --shard-index 2 path/to/test.dart

Sharding: This refers to the process of splitting up a large test suite into smaller subsets (called shards) that can be run independently. Sharding is particularly useful for distributed testing, where multiple machines are used to run tests simultaneously. By dividing the test suite into smaller subsets, you can run tests in parallel across multiple machines, which can significantly reduce the overall testing time.

Test concurrency

Test suites run concurrently by default, using half of the host's CPU cores. Use --concurrency to control the number of test suites that runs concurrently, meaning that multiple tests in independent suites or platforms can run at the same time. For example, if you wanted to run the tests on 4 threads, you could run the tests as follows:

dart test --concurrency=4

This can speed up the overall testing process, especially if you have a large number of test suites.

Shuffling Tests

Test order can be shuffled with the --test-randomize-ordering-seed argument. This allows you to shuffle your tests with a specific seed (deterministic) or a random seed for each run. For example, consider the following test runs:

dart test --test-randomize-ordering-seed=12345
dart test --test-randomize-ordering-seed=random

Setting --test-randomize-ordering-seed=0 will have the same effect as not specifying it at all, meaning the test order will remain as-is.

Selecting a Test Reporter

You can adjust the output format of test results using the --reporter=<option> command line flag. The default format is the compact output format - a single line, continuously updated as tests are run. When running on the GitHub Actions CI however (detected via checking the GITHUB_ACTIONS environment variable for true), the default changes to the github output format - a reporter customized for that CI/CD system.

The available options for the --reporter flag are:

compact: a single, continuously updated line
expanded: a separate line for each update
github: a custom reporter for GitHub Actions
json: a machine-readable format; see https://dart.dev/go/test-docs/json_reporter.md

Collecting Code Coverage

To collect code coverage, you can run tests with the --coverage <directory> argument. The directory specified can be an absolute or relative path. If a directory does not exist at the path specified, a directory will be created. If a directory does exist, files may be overwritten with the latest coverage data, if they conflict.

This option will enable code coverage collection on a suite-by-suite basis, and the resulting coverage files will be outputted in the directory specified. The files can then be formatted using the package:coverage format_coverage executable.

Coverage gathering is currently only implemented for tests run on the Dart VM or Chrome.

Here's an example of how to run tests and format the collected coverage to LCOV:

## Run Dart tests and output them at directory `./coverage`:
dart run test --coverage=./coverage

## Activate package `coverage` (if needed):
dart pub global activate coverage

## Format collected coverage to LCOV (only for directory "lib")
dart pub global run coverage:format_coverage --packages=.dart_tool/package_config.json --report-on=lib --lcov -o ./coverage/lcov.info -i ./coverage

## Generate LCOV report:
genhtml -o ./coverage/report ./coverage/lcov.info

## Open the HTML coverage report:
open ./coverage/report/index.html

LCOV is a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case.
The binary genhtml is one of the LCOV tools.
See the LCOV project for more: https://github.com/linux-test-project/lcov
See the Homebrew LCOV formula: https://formulae.brew.sh/formula/lcov

Restricting Tests to Certain Platforms

Some test files only make sense to run on particular platforms. They may use dart:html or dart:io, they might test Windows' particular filesystem behavior, or they might use a feature that's only available in Chrome. The @TestOn annotation makes it easy to declare exactly which platforms a test file should run on. Just put it at the top of your file, before any library or import declarations:

@TestOn('vm')

import 'dart:io';

import 'package:test/test.dart';

void main() {
  // ...
}

The string you pass to @TestOn is what's called a "platform selector", and it specifies exactly which platforms a test can run on. It can be as simple as the name of a platform, or a more complex Dart-like boolean expression involving these platform names.

You can also declare that your entire package only works on certain platforms by adding a test_on field to your package config file.

Platform Selectors

Platform selectors use the boolean selector syntax defined in the boolean_selector package, which is a subset of Dart's expression syntax that only supports boolean operations. The following identifiers are defined:

vm: Whether the test is running on the command-line Dart VM.
chrome: Whether the test is running on Google Chrome.
firefox: Whether the test is running on Mozilla Firefox.
safari: Whether the test is running on Apple Safari.
edge: Whether the test is running on Microsoft Edge browser.
node: Whether the test is running on Node.js.
dart-vm: Whether the test is running on the Dart VM in any context. It's identical to !js.
browser: Whether the test is running in any browser.
js: Whether the test has been compiled to JS. This is identical to !dart-vm.
blink: Whether the test is running in a browser that uses the Blink rendering engine.
windows: Whether the test is running on Windows. This can only be true if either vm or node is true.
mac-os: Whether the test is running on MacOS. This can only be true if either vm or node is true.
linux: Whether the test is running on Linux. This can only be true if either vm or node is true.
android: Whether the test is running on Android. If vm is false, this will be false as well, which means that this won't be true if the test is running on an Android browser.
ios: Whether the test is running on iOS. If vm is false, this will be false as well, which means that this won't be true if the test is running on an iOS browser.
posix: Whether the test is running on a POSIX operating system. This is equivalent to !windows.
dart2js: Whether the test has been compiled with Dart2Js.
dart2wasm: Whether the test has been compiled with Dart2Wasm.
kernel: Whether the test has been compiled to kernel.
source: Whether the test has been run with no compiler (from source).

For example, if you wanted to run a test on every browser but Chrome, you would write @TestOn('browser && !chrome').

Running Tests on Node.js

The test runner also supports compiling tests to JavaScript and running them on Node.js by passing --platform node. Note that Node has access to neither dart:html nor dart:io, so any platform-specific APIs will have to be invoked using the js package. However, it may be useful when testing APIs that are meant to be used by JavaScript code.

The test runner looks for an executable named node (on Mac OS or Linux) or node.exe (on Windows) on your system path. When compiling Node.js tests, it passes -Dnode=true, so tests can determine whether they're running on Node using const bool.fromEnvironment('node'). It also sets --server-mode, which will tell the compiler that dart:html is not available.

If a top-level node_modules directory exists, tests running on Node.js can import modules from it.

Asynchronous Tests

Tests written with async/await will work automatically. The test runner won't consider the test finished until the returned Future completes.

import 'dart:async';

import 'package:test/test.dart';

void main() {
  test('Future.value() returns the value', () async {
    var value = await Future.value(10);
    expect(value, equals(10));
  });
}

Uncaught Async Errors

Any uncaught asynchronous error throws within the zone that a test is running in will cause the test to be considered a failure. This can cause a test which was previously considered complete and passing to change into a failure if the uncaught async error is raised late. If all test cases within the suite have completed this may cause some errors to be missed, or to surface in only some runs.

Avoid uncaught async errors by ensuring that all futures have an error handler before they complete as an error.

Running Tests With Custom HTML

By default, the test runner will generate its own empty HTML file for browser tests. However, tests that need custom HTML can create their own files. These files have three requirements:

They must have the same name as the test, with .dart replaced by .html. You can also provide a configuration path to an HTML file if you want it to be reused across all tests. See Providing a custom HTML template below.
They must contain a link tag with rel="x-dart-test" and an href attribute pointing to the test script.
They must contain <script src="packages/test/dart.js"></script>.

For example, if you had a test called custom_html_test.dart, you might write the following HTML file:

<!doctype html>
<!-- custom_html_test.html -->
<html>
  <head>
    <title>Custom HTML Test</title>
    <link rel="x-dart-test" href="custom_html_test.dart">
    <script src="packages/test/dart.js"></script>
  </head>
  <body>
    // ...
  </body>
</html>

Providing a custom HTML template

If you want to share the same HTML file across all tests, you can provide a custom_html_template_path configuration option to your configuration file. This file should follow the rules above, except that instead of the link tag add exactly one {{testScript}} in the place where you want the template processor to insert it.

You can also optionally use any number of {{testName}} placeholders which will be replaced by the test filename.

The template can't be named like any test file, as that would clash with using the custom HTML mechanics. In such a case, an error will be thrown.

For example:

custom_html_template_path: html_template.html.tpl

<!doctype html>
<!-- html_template.html.tpl -->
<html>
  <head>
    <title>{{testName}} Test</title>
    {{testScript}}
    <script src="packages/test/dart.js"></script>
  </head>
  <body>
    // ...
  </body>
</html>

Configuring Tests

Skipping Tests

If a test, group, or entire suite isn't working yet, and you just want it to stop complaining, you can mark it as "skipped". The test or tests won't be run, and, if you supply a reason why, that reason will be printed. In general, skipping tests indicates that they should run but is temporarily not working. If they're fundamentally incompatible with a platform, @TestOn/testOn should be used instead.

To skip a test suite, put a @Skip annotation at the top of the file:

@Skip('currently failing (see issue 1234)')

import 'package:test/test.dart';

void main() {
  // ...
}

The string you pass should describe why the test is skipped. You don't have to include it, but it's a good idea to document why the test isn't running.

Groups and individual tests can be skipped by passing the skip parameter. This can be either true or a String describing why the test is skipped. For example:

import 'package:test/test.dart';

void main() {
  group('complicated algorithm tests', () {
    // ...
  }, skip: "the algorithm isn't quite right");

  test('error-checking test', () {
    // ...
  }, skip: 'TODO: add error-checking.');
}

Timeouts

By default, tests will time out after 30 seconds of inactivity. The timeout applies to deadlocks or cases where the test stops making progress, it does not ensure that an overall test case or test suite completes within any set time.

Timeouts can be configured on a per-test, -group, or -suite basis. To change the timeout for a test suite, put a @Timeout annotation at the top of the file:

@Timeout(Duration(seconds: 45))

import 'package:test/test.dart';

void main() {
  // ...
}

In addition to setting an absolute timeout, you can set the timeout relative to the default using @Timeout.factor. For example, @Timeout.factor(1.5) will set the timeout to one and a half times as long as the default—45 seconds.

Timeouts can be set for tests and groups using the timeout parameter. This parameter takes a Timeout object just like the annotation. For example:

import 'package:test/test.dart';

void main() {
  group('slow tests', () {
    // ...

    test('even slower test', () {
      // ...
    }, timeout: Timeout.factor(2));
  }, timeout: Timeout(Duration(minutes: 1)));
}

Nested timeouts apply in order from outermost to innermost. That means that "even slower test" will take two minutes to time out, since it multiplies the group's timeout by 2.

Platform-Specific Configuration

Sometimes a test may need to be configured differently for different platforms. Windows might run your code slower than other platforms, or your DOM manipulation might not work right on Safari yet. For these cases, you can use the @OnPlatform annotation and the onPlatform named parameter to test() and group(). For example:

@OnPlatform({
  // Give Windows some extra wiggle-room before timing out.
  'windows': Timeout.factor(2)
})

import 'package:test/test.dart';

void main() {
  test('do a thing', () {
    // ...
  }, onPlatform: {
    'safari': Skip('Safari is currently broken (see #1234)')
  });
}

Both the annotation and the parameter take a map. The map's keys are platform selectors which describe the platforms for which the specialized configuration applies. Its values are instances of some of the same annotation classes that can be used for a suite: Skip and Timeout. A value can also be a list of these values.

If multiple platforms match, the configuration is applied in order from first to last, just as they would in nested groups. This means that for configuration like duration-based timeouts, the last matching value wins.

You can also set up global platform-specific configuration using the package configuration file.

Tagging Tests

Tags are short strings that you can associate with tests, groups, and suites. They don't have any built-in meaning, but they're very useful nonetheless: you can associate your own custom configuration with them, or you can use them to easily filter tests so you only run the ones you need to.

Tags are defined using the @Tags annotation for suites and the tags named parameter to test() and group(). For example:

@Tags(['browser'])

import 'package:test/test.dart';

void main() {
  test('successfully launches Chrome', () {
    // ...
  }, tags: 'chrome');

  test('launches two browsers at once', () {
    // ...
  }, tags: ['chrome', 'firefox']);
}

If the test runner encounters a tag that wasn't declared in the package configuration file, it'll print a warning, so be sure to include all your tags there. You can also use the file to provide default configuration for tags, like giving all browser tests twice as much time before they time out.

Tests can be filtered based on their tags by passing command line flags. The --tags or -t flag will cause the test runner to only run tests with the given tags, and the --exclude-tags or -x flag will cause it to only run tests without the given tags. These flags also support boolean selector syntax. For example, you can pass --tags "(chrome || firefox) && !slow" to select quick Chrome or Firefox tests.

Note that tags must be valid Dart identifiers, although they may also contain hyphens.

Whole-Package Configuration

For configuration that applies across multiple files, or even the entire package, test supports a configuration file called dart_test.yaml. At its simplest, this file can contain the same sort of configuration that can be passed as command-line arguments:

# This package's tests are very slow. Double the default timeout.
timeout: 2x

# This is a browser-only package, so test on chrome by default.
platforms: [chrome]

The configuration file sets new defaults. These defaults can still be overridden by command-line arguments, just like the built-in defaults. In the example above, you could pass --platform firefox to run on Firefox.

A configuration file can do much more than just set global defaults. See the full documentation for more details.

Compiler flags

The test runner does not support general purpose flags to control compilation such as -D defines or flags like --no-sound-null-safety. In most cases it is preferable to avoid writing tests that depend on the fine-grained compiler configuration. For instance to choose between sound and unsound null safety, prefer to choose a language version for each test which has the desired behavior by default - choose a language version below 2.12 to disable sound null safety, and a language version above 2.12 to enable sound null safety. When fine-grained configuration is unavoidable, the approach varies by platform.

Compilation for browser and node tests can be configured by passing arguments to dart compile js with --dart2js-args options.

Fine-grained compilation configuration is not supported for the VM. Any configuration which impacts runtime behavior for the entire VM, such as -D defines (when used for non-const values) and runtime behavior experiments, will influence both the test runner and the isolates spawned to run test suites. Experiments which are breaking may cause incompatibilities with the test runner. These may be specified with a DART_VM_OPTIONS environment variable when running with pub run test, or by passing them to the dart command before the test subcommand when using dart test.

Debugging

Tests can be debugged interactively using platforms' built-in development tools. Tests running on browsers can use those browsers' development consoles to inspect the document, set breakpoints, and step through code. Those running on the Dart VM use Dart DevTools.

The first step when debugging is to pass the --pause-after-load flag to the test runner. This pauses the browser after each test suite has loaded, so that you have time to open the development tools and set breakpoints. For the Dart VM it will print the remote debugger URL.

Once you've set breakpoints, either click the big arrow in the middle of the web page or press Enter in your terminal to start the tests running. When you hit a breakpoint, the runner will open its own debugging console in the terminal that controls how tests are run. You can type "restart" there to re-run your test as many times as you need to figure out what's going on.

Normally, browser tests are run in hidden iframes. However, when debugging, the iframe for the current test suite is expanded to fill the browser window so you can see and interact with any HTML it renders. Note that the Dart animation may still be visible behind the iframe; to hide it, just add a background-color to the page's HTML.

Browser/VM Hybrid Tests

Code that's written for the browser often needs to talk to some kind of server. Maybe you're testing the HTML served by your app, or maybe you're writing a library that communicates over WebSockets. We call tests that run code on both the browser and the VM hybrid tests.

Hybrid tests use one of two functions: spawnHybridCode() and spawnHybridUri(). Both of these spawn Dart VM isolates that can import dart:io and other VM-only libraries. The only difference is where the code from the isolate comes from: spawnHybridCode() takes a chunk of actual Dart code, whereas spawnHybridUri() takes a URL. They both return a StreamChannel that communicates with the hybrid isolate. For example:

// ## test/web_socket_server.dart

// The library loaded by spawnHybridUri() can import any packages that your
// package depends on, including those that only work on the VM.
import 'package:shelf/shelf_io.dart' as io;
import 'package:shelf_web_socket/shelf_web_socket.dart';
import 'package:stream_channel/stream_channel.dart';

// Once the hybrid isolate starts, it will call the special function
// hybridMain() with a StreamChannel that's connected to the channel
// returned spawnHybridCode().
hybridMain(StreamChannel channel) async {
  // Start a WebSocket server that just sends "hello!" to its clients.
  var server = await io.serve(webSocketHandler((webSocket) {
    webSocket.sink.add('hello!');
  }), 'localhost', 0);

  // Send the port number of the WebSocket server to the browser test, so
  // it knows what to connect to.
  channel.sink.add(server.port);
}


// ## test/web_socket_test.dart

@TestOn('browser')

import 'dart:html';

import 'package:test/test.dart';

void main() {
  test('connects to a server-side WebSocket', () async {
    // Each spawnHybrid function returns a StreamChannel that communicates with
    // the hybrid isolate. You can close this channel to kill the isolate.
    var channel = spawnHybridUri('web_socket_server.dart');

    // Get the port for the WebSocket server from the hybrid isolate.
    var port = await channel.stream.first;

    var socket = WebSocket('ws://localhost:$port');
    var message = await socket.onMessage.first;
    expect(message.data, equals('hello!'));
  });
}

A diagram showing a test in a browser communicating with a Dart VM isolate outside the browser.

Note: If you write hybrid tests, be sure to add a dependency on the stream_channel package, since you're using its API!

Support for Other Packages

`build_runner`

If you are using package:build_runner to build your package, then you will need a dependency on build_test in your dev_dependencies, and then you can use the pub run build_runner test command to run tests.

To supply arguments to package:test, you need to separate them from your build args with a -- argument. For example, running all web tests in release mode would look like this pub run build_runner test --release -- -p vm.

`term_glyph`

The term_glyph package provides getters for Unicode glyphs with ASCII alternatives. test ensures that it's configured to produce ASCII when the user is running on Windows, where Unicode isn't supported. This ensures that testing libraries can use Unicode on POSIX operating systems without breaking Windows users.

1.25.7

Enable asserts for dart2wasm tests.

1.25.6

Point API doc links to package:test canonical libraries.
Fix testing with dart2wasm - use dart compile wasm instead of depending on SDK internals
Update min SDK constraint to 3.2.0.

1.25.5

Update the package:web_socket_channel version constraint to allow 3.x.
Update the package:shelf_web_socket version constraint to allow 2.x.

1.25.4

Add @doNotSubmit to more declarations of the solo parameter.

1.25.3

Remove outdated StreamMatcher link from README table of contents.
Document the silent reporter in CLI help output.
Support enabling experiments with the dart2wasm compiler.
Added @doNotSubmit to test(solo: ...) and group(solo: ...). In practice, this means that code that was relying on ignoring deprecation warnings and using solo or group with a skip parameter will now fail if dart analyze --fatal-infos (or similar) is enabled.

1.25.2

Fix a bug running browser tests with paths containing windows directory separator follow by a character which is an invalid Dart string escape sequence.

1.25.1

Fix a bug where in precompiled mode, html files for tests were no longer created.
Support the latest version of package:js.
Document the silent reporter in CLI help output.

1.25.0

Handle paths with leading / when spawning test isolates.
Add support for the dart2wasm compiler in chrome and firefox.
BREAKING: Remove the experimental-chrome-wasm platform, you can now use -p chrome -c dart2wasm instead.
- Note that this has always been advertised as a change that would happen in a future non-breaking release.
BREAKING:Dropped support for --pub-serve which has long not been tested or supported.
- We do not anticipate much if any actual breakage or existing usage of this feature, which is why we are making this change in a non-breaking release.
- If you do require this feature, file an issue and we can look at adding it back.
BREAKING: Fully remove support for Internet Explorer.
Fix running of tests defined under lib/ with relative imports to other libraries in the package.

1.24.9

Update the vm_service constraint to allow version 13.x.

1.24.8

Remove spurious deprecation during autocomplete for setUp and tearDown.

1.24.7

Simplify the initialization of the per-suite message channel within browser tests. See https://github.com/dart-lang/test/issues/2065
Add a timeout to browser test suite loads.
Fix running of browser tests that use deferred loaded libraries.

1.24.6

Fix communication failures between minified test apps and the non-minified host app.
Add support for discontinuing after the first failing test with --fail-fast.

1.24.5

Change compiling <path> to loading <path> message in all cases. Surface the "loading" messages in the situations where previously only the "compiling" message would be used.
Support browser tests where the frame creates the message channel.

1.24.4

Drop support for null unsafe Dart, bump SDK constraint to 3.0.0.
Make some annotation classes final: OnPlatform, Retry, Skip, Tags, TestOn, Timeout.
Fix the root_ fields in the JSON reporter when running a test on Windows with an absolute path.
Add support for SAFARI_EXECUTABLE, FIREFOX_EXECUTABLE and MS_EDGE_EXECUTABLE for custom browser installations.
Allow the latest analyzer (6.x.x).
Add MOZ_AUTOMATION=1 environmental variable to Firefox runner, to make launcher process on Windows wait for browser exit.

1.24.3

Fix compatibility with wasm number semantics.

1.24.2

Copy an existing nonce from a script on the test HTML page to the script created by the test runner host javascript. This only impacts environments testing with custom HTML that includes a nonce.
Support the Microsoft Edge browser (use the edge platform in your test configuration file or -p edge on the command line).

1.24.1

Handle a missing 'compiler' value when running a test compiled against a newer test_api than the runner back end is using. The expectation was that the json protocol is only used across packages compatible with the same major version of the test_api package, but flutter test does not check the version of packages in the pub solve for user test code.

1.24.0

Support the --compiler flag, which can be used to configure which compiler to use.
- To specify a compiler by platform, the argument supports platform selectors through this syntax [<platform>:]<compiler>. For example the command line argument --compiler vm:source would run all vm tests from source instead of compiling to kernel first.
- If no given compiler is compatible for a platform, it will use its default compiler instead.
Add support for running tests as native executables (vm platform only).
- You can run tests this way with --compiler exe.
Support compiler identifiers in platform selectors.
List the supported compilers for each platform in the usage text.
Update all reporters to print the compiler along with the platform name when configured to print the platform. Extend the logic for printing platofrm information to do so if any compilers are explicitly configured.
Deprecate --use-data-isolate-strategy. It is now an alias for -c vm:source which is roughly equivalent. If this is breaking for you please file an issue.

1.23.1

Fix running paths by absolute path (with drive letter) on windows.

1.23.0

Avoid empty expandable groups for tests without extra output in Github reporter.
Add support for CHROME_EXECUTABLE environment variable. This overrides any config file settings.
Support running tests by absolute file uri.

1.22.2

Don't run tearDown until the test body and outstanding work is complete, even if the test has already failed.
Change URL secrets for browser tests to always be alphanumeric characters.

1.22.1

Add documentation for the --ignore-timeouts argument.
Merge command lines args repeating the same test path to run the suite one time with all the test cases across the different arguments.
Fix VM tests which run after some test has changed the working directory. There are still issues with browser tests after changing directory.
Deprecate throwsNullThrownError, use throwsA(isA<TypeError>()) instead. The implementation has been changed to ease migrations.
Deprecate throwsCyclicInitializationError and replace the implementation with Throws(TypeMatcher<Error>()). The specific exception no longer exists and there is no guarantee about what type of error will be thrown.

1.22.0

Fix an issue with the github reporter where tests that fail asynchronously after they've completed would show up as succeeded tests.
Add the experimental-chrome-wasm platform. This is very unstable and will eventually be deleted, to be replaced by a --compiler flag. See https://github.com/dart-lang/test/issues/1776 for more information on future plans.

1.21.7

Support package:matcher version 0.12.13.

1.21.6

Require Dart >= 2.18.0
Fix the coverage usage example in the README.md
Support the latest package:test_api and package:test_core.

1.21.5

Fix printOnFailure output to be associated with the correct test.
Migrate all dom interactions to static interop.

1.21.4

Make the labels for test loading more readable in the compact and expanded reporters, use gray instead of black.
Print a command to re-run the failed test after each failure in the compact reporter.
Fix the package config path used when running pre-compiled vm tests.

1.21.3

Support the latest package:test_api and package:test_core.

1.21.2

Add Target to restrict TestOn annotation to library level.
Update the github reporter to output the platform in the test names when multiple platforms are used.
Fix spawnHybridUri support for package: uris.

1.21.1

Fix a bug loading JS sources with non-utf8 content while parsing coverage information from chrome.

1.21.0

Allow analyzer version 4.x.
Add a github reporter option for use with GitHub Actions.
Make the github test reporter the default when we detect we're running on GitHub Actions.

1.20.2

Drop dart2js-path command line argument.
Allow loading tests under a path with the directory named packages.
Add retry for launching browsers. Reduce timeout back to 30 seconds.

1.20.1

Allow the latest vm_service package.

1.20.0

Update analyzer constraint to >=2.0.0 <4.0.0.
Add an --ignore-timeouts command line flag, which disables all timeouts for all tests. This can be useful when debugging, so tests don't time out during debug sessions.
Create a trusted types policy when available for assigning the script URL for web tests.

1.19.5

Try to get more logging from chrome on windows to diagnose intermittent failures.

1.19.4

Wait for paused VM platform isolates before shutdown.
TestFailure implements Exception for compatibility with only_throw_exceptions.

1.19.3

Remove duplicate logging of suggestion to enable the chain-stack-traces flag, a single log will now appear at the end.

1.19.2

Republish with missing JS file for browser tests.

1.19.1

Fix parsing of file paths into a URI on windows.

1.19.0

Support query parameters name, full-name, line, and col on test paths, which will apply the filters to only those test suites.
- All specified filters must match for a test to run.
- Global filters (ie: --name) are also still respected and must match.
- The line and col will match if any frame from the test trace matches (the test trace is the current stack trace where test is invoked).
Give a better exception when using markTestSkipped outside of a test.

1.18.2

Publish with the host.dart.js file.

1.18.1

Add defaulting for older test backends that don't pass a configuration for the allow_duplicate_test_names parameter to the remote listener.

1.18.0

Add configuration to disallow duplicate test and group names. See the docs for more information.
Remove dependency on pedantic.

1.17.12

Support the latest test_core.
Re-use the cached dill file from previous runs on subsequent runs.

1.17.11

Use the latest package:matcher.
- Change many argument types from dynamic to Object?.
- Fix stringContainsInOrder to account for repetitions and empty strings.
  - Note: This may break some existing tests, as the behavior does change.

1.17.10

Report incomplete tests as errors in the JSON reporter when the run is canceled early.
Update analyzer constraint to >=1.0.0 <3.0.0.

1.17.9

Fix a bug where a tag level configuration would cause test suites with that tag to ignore the --test-randomize-ordering-seed argument.

1.17.8

Update json reporter docs with updated nullability annotations and descriptions.
Add time field to the json reporters allSuites event type so that all event types can be unified.

1.17.7

Support the latest test_core.

1.17.6

Give a better error when printOnFailure is called from outside a test zone.

1.17.5

Support the latest vm_service release (7.0.0).

1.17.4

Fix race condition between compilation of vm tests and the running of isolates.

1.17.3

Forward experiment args from the runner executable to the compiler with the new vm test loading strategy.

1.17.2

Fix a windows issue with the new loading strategy.

1.17.1

Fix an issue where you couldn't have tests compiled in both sound and unsound null safety modes.

1.17.0

Change the default way VM tests are launched and ran to greatly speed up loading performance.
- You can force the old strategy with --use-data-isolate-strategy flag if you run into issues, but please also file a bug.
Disable stack trace chaining by default. It can be re-enabled by explicitly passing the --chain-stack-traces flag.
Remove phantomjs support completely, it was previously broken.
Fix expectAsync function type checks.
Add libraries scaffolding.dart, and expect.dart to allow importing a subset of the normal surface area.

1.16.8

Fix an issue where coverage collection could hang on Chrome.
~~Disable stack trace chaining by default. It can be re-enabled by explicitly passing the --chain-stack-traces flag.~~

1.16.7

Update spawnHybridCode to default to the current packages language version.
Update test_core and test_api deps.

1.16.6

Complete the migration to null safety.

1.16.5

Expand several deps to allow the latest versions.

1.16.4

Update test_core dependency to 0.3.14.

1.16.3

Update web_socket_channel dependency to support latest.

1.16.2

Update test_core dependency to 0.3.13.

1.16.1

Allow the latest analyzer 1.0.0.

1.16.0

Stable null safety release.

1.16.0-nullsafety.19

Use the test_api for stable null safety.

1.16.0-nullsafety.18

Expand upper bound constraints for some null safe migrated packages.

1.16.0-nullsafety.17

Support the latest shelf release (1.x.x).

1.16.0-nullsafety.16

Support the latest vm_service release (6.x.x).

1.16.0-nullsafety.15

Support the latest coverage release (0.15.x).

1.16.0-nullsafety.14

Allow the latest args release (2.x).

1.16.0-nullsafety.13

Allow the latest glob release (2.x).

1.16.0-nullsafety.12

Fix spawnHybridUri on windows.
Fix failures running tests on the node platform.
Allow package:yaml version 3.x.x.

1.16.0-nullsafety.11

Set up a stack trace mapper in precompiled mode if source maps exist. If the stack traces are already mapped then this has no effect, otherwise it will try to map any JS lines it sees.

1.16.0-nullsafety.10

Allow injecting a test channel for browser tests.
Allow package:analyzer version 0.41.x.

1.16.0-nullsafety.9

Fix spawnHybridUri to respect language versioning of the spawned uri.

1.16.0-nullsafety.8

Update SDK constraints to >=2.12.0-0 <3.0.0 based on beta release guidelines.

1.16.0-nullsafety.7

Allow prerelease versions of the 2.12 sdk.

1.16.0-nullsafety.6

Add markTestSkipped API.

1.16.0-nullsafety.5

Allow 2.10 stable and 2.11.0-dev SDKs.
Annotate the classes used as annotations to restrict their usage to library level.
Stop required a SILENT_OBSERVATORY environment variable to run with debugging and the JSON reporter.

1.16.0-nullsafety.4

Depend on the latest test_core.

1.16.0-nullsafety.3

Clean up --help output.

1.16.0-nullsafety.2

Allow version 0.40.x of analyzer.

1.16.0-nullsafety.1

Depend on the latest test_core.

1.16.0-nullsafety

Support running tests with null safety.
- Note that the test runner itself is not fully migrated yet.
Add the Fake class, available through package:test_api/fake.dart. This was previously part of the Mockito package, but with null safety it is useful enough that we decided to make it available through package:test. In a future release it will be made available directly through package:test_api/test_api.dart (and hence through package:test_core/test_core.dart and package:test/test.dart).

1.15.7 (Backport)

Fix spawnHybridUri on windows.

1.15.6 (Backport)

Support package:analyzer version 0.41.x.

1.15.5 (Backport)

Fix spawnHybridUri to respect language versioning of the spawned uri.

1.15.4

Allow analyzer 0.40.x.

1.15.3

Update to matcher version 0.12.9 which improves the mismatch description for deep collection equality matchers and TypeMatcher.

1.15.2

Use the latest test_core which resolves an issue with the latest package:meta.

1.15.1

Avoid a confusing stack trace when there is a problem loading a platform when using the JSON reporter and enabling debugging.
Restore behavior of listening for both IPv6 and IPv4 sockets for the node platform.

1.15.0

Update bootstrapping logic to ensure the bootstrap library has the same language version as the test.
The Node platform will now communicate over only IPv6 if it is available.

1.14.7

Support the latest package:coverage.

1.14.6

Update test_core to 0.3.6.

1.14.5

Add additional information to an exception when we end up with a null RunnerSuite.

1.14.4

Use non-headless Chrome when provided the flag --pause-after-load.

1.14.3

Fix an issue where coverage tests could not run in Chrome headless.
Fix an issue where coverage collection would not work with source maps that contained absolute file URIs.
Fix error messages for incorrect string literals in test annotations.
Update test_core to 0.3.4.

1.14.2

Update test_core to 0.3.3.

1.14.1

Allow the latest shelf_packages_handler.

1.14.0

Drop the package_resolver dependency for the package_config dependency which is lower level.

1.13.0

Enable asserts in code running through spawnHybrid APIs.
Exit with a non-zero code if no tests were ran, whether due to skips or having no tests defined.
Fix the stack trace labels in SDK code for dart2js compiled tests.
Cancel any StreamQueue that is created as a part of a stream matcher once it is done matching.
- This fixes a bug where using a matcher on a custom stream controller and then awaiting the close() method on that controller would hang.
Avoid causing the test runner to hang if there is a timeout during a tearDown callback following a failing test case.

1.12.0

Bump minimum SDK to 2.4.0 for safer usage of for-loop elements.
Deprecate PhantomJS and provide warning when used. Support for PhantomJS will be removed in version 2.0.0.
Support coverage collection for the Chrome platform. See README.md for usage details.

1.11.1

Allow test_api 0.2.13 to work around a bug in the SDK version 2.3.0.

1.11.0

Add file_reporters configuration option and --file-reporter CLI option to allow specifying a separate reporter that writes to a file instead of stdout.

1.10.0

Add customHtmlTemplateFile configuration option to allow sharing an html template between tests
Depend on the latest package:test_core.
Depend on the latest package:test_api.

1.9.4

Extend the timeout for synthetic tests, e.g. tearDownAll.
Depend on the latest package:test_core.
Depend on the latest package:test_api.

1.9.3

Depend on the latest package:test_core.
Support the latest package:analyzer.
Update to latest package:matcher. Improves output for instances of private classes.

1.9.2

Depend on the latest package:test_api and package:test_core.
While using solo tests that are not run will now be reported as skipped.

1.9.1

Depend on latest test_core.

1.9.0

Implement code coverage collection for VM based tests

1.8.0

Expose the previously hidden sharding arguments
- --total-shards specifies how many shards the suite should be split into
- --shard-index specifies which shard should be run

1.7.0

Add a --debug flag for running the VM/Chrome in debug mode.

1.6.11

Depend on the latest test_core and test_api.

1.6.10

Depend on the latest test_core.

1.6.9

Add --disable-dev-shm-usage to the default Chrome flags.

1.6.8

Depend on the latest test_core and test_api.

1.6.7

Allow analyzer version 0.38.x.

1.6.6

Pass --server-mode to dart2js instead of --categories=Server to fix a warning about the flag deprecation.
Drop dependency on pub_semver.
Fix issue with the latest Utf8Decoder and the node platform.

1.6.5

Depend on the latest test_core.
Depend on the latest package:analyzer.

1.6.4

Don't swallow exceptions from callbacks in expectAsync*.
Internal cleanup - fix lints.

1.6.3

Depend on latest package:test_core.
- This fixes an issue where non-completed tests were considered passing.

1.6.2

Avoid dart:isolate imports on code loaded in tests.

1.6.1

Allow stream_channel version 2.0.0.

1.6.0

Allow analyzer version 0.36.x.
Matcher changes:
- Add isA() to create TypeMatcher instances in a more fluent way.
- Add isCastError.
- Potentially breaking bug fix. Ordering matchers no longer treat objects with a partial ordering (such as NaN for double values) as if they had a complete ordering. For instance greaterThan now compares with the > operator rather not < and not =. This could cause tests which relied on this bug to start failing.

1.5.3

Allow analyzer version 0.35.x.

1.5.2

Require Dart SDK >=2.1.0.
Depend on latest test_core and test_api.

1.5.1

Depend on latest test_core and test_api.

1.5.0

Depend on package:test_core for core functionality.

1.4.0

Depend on package:test_api for core functionality.

1.3.4

Allow remote_listener to be closed and sent an event on close.

1.3.3

Add conditional imports so that dart:io is not imported from the main test.dart entrypoint unless it is available.
Fix an issue with dartdevc in precompiled mode and the json reporter.
Fix an issue parsing test metadata annotations without explicit const.

1.3.2

Widen the constraints on the analyzer package.

1.3.1

Handle parsing annotations which omit const on collection literals.
Fix an issue where root_line, root_column, and root_url in the JSON reported may not be populated correctly on Windows.
Removed requirement for the test/pub_serve transformer in --pub-serve mode.

1.3.0

When using --precompiled, the test runner now allows symlinks to reach outside the precompiled directory. This allows more efficient creation of precompiled directories (using symlinks instead of copies).
Updated max sdk range to <3.0.0.

1.2.0

Added support for using precompiled kernel files when running vm tests.
- When using the --precompiled flag we will now first check for a <original-test-path>.vm_test.vm.app.dill file, and if present load that directly in the isolate. Otherwise the <original-test-path>.vm_test.dart file will be used.

1.1.0

Added a new pid field to the StartEvent in the json runner containing the pid of the VM process running the tests.

1.0.0

No change from 0.12.42. We are simply signalling to users that this is a well supported package and is the preferred way to write Dart tests.

0.12.42

Add support for solo test and group. When the argument is true only tests and groups marked as solo will be run. It is still recommended that users instead filter their tests by using the runner argument -n.
Updated exported package:matcher to 0.12.3 which includes these updates:
- Many improvements to TypeMatcher
  - Can now be used directly as const TypeMatcher<MyType>().
  - Added a type parameter to specify the target Type.
    - Made the name constructor parameter optional and marked it deprecated. It's redundant to the type parameter.
  - Migrated all isType matchers to TypeMatcher.
  - Added a having function that allows chained validations of specific features of the target type.
```
/// Validates that the object is a [RangeError] with a message containing
/// the string 'details' and `start` and `end` properties that are `null`.
final _rangeMatcher = isRangeError
   .having((e) => e.message, 'message', contains('details'))
   .having((e) => e.start, 'start', isNull)
   .having((e) => e.end, 'end', isNull);
```
- Deprecated the isInstanceOf class. Use TypeMatcher instead.
- Improved the output of Matcher instances that fail due to type errors.

0.12.41

Add support for debugging VM tests.
Tweak default reporter and color logic again so that they are always enabled on all non-windows platforms.

0.12.40

Added some new optional fields to the json reporter, root_line, root_column, and root_url. These will be present if url is not the same as the suite url, and will represent the location in the original test suite from which the call to test originated.

0.12.39

Change the default reporter and color defaults to be based on stdout.supportsAnsiEscapes instead of based on platform (previously both were disabled on windows).

0.12.38+3

Fix Dart 2 runtime errors around communicating with browsers.

0.12.38+2

Fix more Dart 2 runtime type errors.

0.12.38+1

Fix several Dart 2 runtime type errors.

0.12.38

Give neverCalled a type that works in Dart 2 semantics.
Support package:analyzer 0.32.0.

0.12.37

Removed the transformer, and the pub_serve.dart entrypoint. This is not being treated as a breaking change because the minimum sdk constraint now points to an sdk which does not support pub serve or barback any more anyways.
Drop the dependency on barback.

0.12.36

Expose the test bootstrapping methods, so that build systems can precompile tests without relying on internal apis.

0.12.35

Dropped support for Dart 1. Going forward only Dart 2 will be supported.
- If you experience blocking issues and are still on the Dart 1 sdk, we will consider bug fixes on a per-case basis based on severity and impact.
- Drop support for dartium and content-shell platforms since those are removed from the Dart 2 SDK.
Fixed an issue --precompiled node tests in subdirectories.
Fixed some dart2 issues with node test bootstrapping code so that dartdevc tests can run.
Fixed default custom html handler so it correctly includes the packages/test/dart.js file. This allows you to get proper errors instead of timeouts if there are load exceptions in the browser.
Upgrade to package:matcher 0.12.2

0.12.34

Requires at least Dart 1.24.0.
The --precompiled flag is now supported for the vm platform and the node platform.
On browser platforms the --precompiled flag now serves all sources directly from the precompiled directory, and will never attempt to do its own compilation.

0.12.33

Pass --categories=Server to dart2js when compiling tests for Node.js. This tells it that dart:html is unavailable.
Don't crash when attempting to format stack traces when running via dart path/to/test.dart.

0.12.32+2

Work around an SDK bug that caused timeouts in asynchronous code.

0.12.32+1

Fix a bug that broke content shell on Dart 1.24.

0.12.32

Add an include configuration field which specifies the path to another configuration file whose configuration should be used.
Add a google platform selector variable that's only true on Google's internal infrastructure.

0.12.31

Add a headless configuration option for Chrome.
Re-enable headless mode for Chrome by default.
Don't hang when a Node.js test fails to compile.

0.12.30+4

Stop running Chrome in headless mode temporarily to work around a browser bug.

0.12.30+3

Fix a memory leak when loading browser tests.

0.12.30+2

Avoid loading test suites whose tags are excluded by --excluded-tags.

0.12.30+1

Internal changes.

0.12.30

Platform selectors for operating systems now work for Node.js tests (#742).
fail() is now typed to return Null, so it can be used in the same places as a raw throw.
Run Chrome in headless mode unless debugging is enabled.

0.12.29+1

Fix strong mode runtime cast failures.

0.12.29

Node.js tests can now import modules from a top-level node_modules directory, if one exists.
Raw console.log() calls no longer crash Node.js tests.
When a browser crashes, include its standard output in the error message.

0.12.28+1

Add a pumpEventQueue() function to make it easy to wait until all asynchronous tasks are complete.
Add a neverCalled getter that returns a function that causes the test to fail if it's ever called.

0.12.27+1

Increase the timeout for loading tests to 12 minutes.

0.12.27

When addTearDown() is called within a call to setUpAll(), it runs its callback after all tests instead of running it after the setUpAll() callback.
When running in an interactive terminal, the test runner now prints status lines as wide as the terminal and no wider.

0.12.26+1

Fix lower bound on package stack_trace. Now 1.6.0.
Manually close browser process streams to prevent test hangs.

0.12.26

The spawnHybridUri() function now allows root-relative URLs, which are interpreted as relative to the root of the package.

0.12.25

Add a override_platforms configuration field which allows test platforms' settings (such as browsers' executables) to be overridden by the user.
Add a define_platforms configuration field which makes it possible to define new platforms that use the same logic as existing ones but have different settings.

0.12.24+8

spawnHybridUri() now interprets relative URIs correctly in browser tests.

0.12.24+7

Declare support for async 2.0.0.

0.12.24+6

Small refactoring to make the package compatible with strong-mode compliant Zone API. No user-visible change.

0.12.24+5

Expose a way for tests to forward a loadException to the server.

0.12.24+4

Drain browser process stdout and stdin. This resolves test flakiness, especially in Travis with the Precise image.

0.12.24+3

Extend deserializeTimeout.

0.12.24+2

Only force exit if FORCE_TEST_EXIT is set in the environment.

0.12.24+1

Widen version constraint on analyzer.

0.12.24

Add a node platform for compiling tests to JavaScript and running them on Node.js.

0.12.23+1

Remove unused imports.

0.12.23

Add a fold_stack_frames field for dart_test.yaml. This will allow users to customize which packages' frames are folded.

0.12.22+2

Properly allocate ports when debugging Chrome and Dartium in an IPv6-only environment.

0.12.22+1

Support args 1.0.0.
Run tear-down callbacks in the same error zone as the test function. This makes it possible to safely share Futures and Streams between tests and their tear-downs.

0.12.22

Add a retry option to test() and group() functions, as well as @Retry() annotation for test files and a retry configuration field for dart_test.yaml. A test with reties enabled will be re-run if it fails for a reason other than a TestFailure.
Add a --no-retry runner flag that disables retries of failing tests.
Fix a "concurrent modification during iteration" error when calling addTearDown() from within a tear down.

0.12.21

Add a doesNotComplete matcher that asserts that a Future never completes.
throwsA() and all related matchers will now match functions that return Futures that emit exceptions.
Respect onPlatform for groups.
Only print browser load errors once per browser.
Gracefully time out when attempting to deserialize a test suite.

0.12.20+13

Upgrade to package:matcher 0.12.1

0.12.20+12

Now support v0.30.0 of pkg/analyzer
The test executable now does a "hard exit" when complete to ensure lingering isolates or async code don't block completion. This may affect users trying to use the Dart service protocol or observatory.

0.12.20+11

Refactor bootstrapping to simplify the test/pub_serve transformer.

0.12.20+10

Refactor for internal tools.

0.12.20+9

Introduce new flag --chain-stack-traces to conditionally chain stack traces.

0.12.20+8

Fixed more blockers for compiling with dev_compiler.
Dartfmt the entire repo.
Note: 0.12.20+5-0.12.20+7 were tagged but not officially published.

0.12.20+4

Fixed strong-mode errors and other blockers for compiling with dev_compiler.

0.12.20+3

--pause-after-load no longer deadlocks with recent versions of Chrome.
Fix Dartified stack traces for JS-compiled tests run through pub serve.

0.12.20+2

Print "[E]" after test failures to make them easier to identify visually and via automated search.

0.12.20+1

Tighten the dependency on stream_channel to reflect the APIs being used.
Use a 1024 x 768 iframe for browser tests.

0.12.20

Breaking change: The expect() method no longer returns a Future, since this broke backwards-compatibility in cases where a void function was returning an expect() (such as void foo() => expect(...)). Instead, a new expectLater() function has been added that return a Future that completes when the matcher has finished running.
The verbose parameter to expect() and the formatFailure() function are deprecated.

0.12.19+1

Make sure asynchronous matchers that can fail synchronously, such as throws*() and prints(), can be used with synchronous matcher operators like isNot().

0.12.19

Added the StreamMatcher class, as well as several built-in stream matchers: emits(), emitsError(), emitsDone, mayEmit(), mayEmitMultiple(), emitsAnyOf(), emitsInOrder(), emitsInAnyOrder(), and neverEmits().
expect() now returns a Future for the asynchronous matchers completes, completion(), throws*(), and prints().
Add a printOnFailure() method for providing debugging information that's only printed when a test fails.
Automatically configure the term_glyph package to use ASCII glyphs when the test runner is running on Windows.

Deprecate the throws matcher in favor of throwsA().
Deprecate the Throws class. These matchers should only be constructed via throwsA().

0.12.18+1

Fix the deprecated expectAsync() function. The deprecation caused it to fail to support functions that take arguments.

0.12.18

Add an addTearDown() function, which allows tests to register additional tear-down callbacks as they're running.
Add the spawnHybridUri() and spawnHybridCode() functions, which allow browser tests to run code on the VM.
Fix the new expectAsync functions so that they don't produce analysis errors when passed callbacks with optional arguments.

0.12.17+3

Internal changes only.

0.12.17+2

Fix Dartium debugging on Windows.

0.12.17+1

Fix a bug where tags couldn't be marked as skipped.

0.12.17

Deprecate expectAsync and expectAsyncUntil, since they currently can't be made to work cleanly in strong mode. They are replaced with separate methods for each number of callback arguments:
- expectAsync0, expectAsync1, ... expectAsync6, and
- expectAsyncUntil0, expectAsyncUntil1, ... expectAsyncUntil6.

0.12.16

Allow tools to interact with browser debuggers using the JSON reporter.

0.12.15+12

Fix a race condition that could cause the runner to stall for up to three seconds after completing.

0.12.15+11

Make test iframes visible when debugging.

0.12.15+10

Throw a better error if a group body is asynchronous.

0.12.15+9

Widen version constraint on analyzer.

0.12.15+8

Make test suites with thousands of tests load much faster on the VM (and possibly other platforms).

0.12.15+7

Fix a bug where tags would be dropped when on_platform was defined in a config file.

0.12.15+6

Fix a broken link in the --help documentation.

0.12.15+5

Internal-only change.

0.12.15+4

Widen version constraint on analyzer.

0.12.15+3

Move nestingMiddleware to lib/src/util/path_handler.dart to enable a cleaner separation between test-runner files and test writing files.

0.12.15+2

Support running without a packages/ directory.

0.12.15+1

Declare support for version 1.19 of the Dart SDK.

0.12.15

Add a skip parameter to expect(). Marking a single expect as skipped will cause the test itself to be marked as skipped.
Add a --run-skipped parameter and run_skipped configuration field that cause tests to be run even if they're marked as skipped.

0.12.14+1

Narrow the constraint on yaml.

0.12.14

Add test and group location information to the JSON reporter.

0.12.13+5

Declare support for version 1.18 of the Dart SDK.
Use the latest collection package.

0.12.13+4

Compatibility with an upcoming release of the collection package.

0.12.13+3

Internal changes only.

0.12.13+2

Fix all strong-mode errors and warnings.

0.12.13+1

Declare support for version 1.17 of the Dart SDK.

0.12.13

Add support for a global configuration file. On Windows, this file defaults to %LOCALAPPDATA%\DartTest.yaml. On Unix, it defaults to ~/.dart_test.yaml. It can also be explicitly set using the DART_TEST_CONFIG environment variable. See the configuration documentation for details.
The --name and --plain-name arguments may be passed more than once, and may be passed together. A test must match all name constraints in order to be run.
Add names and plain_names fields to the package configuration file. These allow presets to control which tests are run based on their names.
Add include_tags and exclude_tags fields to the package configuration file. These allow presets to control which tests are run based on their tags.
Add a pause_after_load field to the package configuration file. This allows presets to enable debugging mode.

0.12.12

Add support for test presets. These are defined using the presets field in the package configuration file. They can be selected by passing --preset or -P, or by using the add_presets field in the package configuration file.
Add an on_os field to the package configuration file that allows users to select different configuration for different operating systems.
Add an on_platform field to the package configuration file that allows users to configure all tests differently depending on which platform they run on.
Add an ios platform selector variable. This variable will only be true when the test executable itself is running on iOS, not when it's running browser tests on an iOS browser.

0.12.11+2

Update to shelf_web_socket 0.2.0.

0.12.11+1

Purely internal change.

0.12.11

Add a tags field to the package configuration file that allows users to provide configuration for specific tags.
The --tags and --exclude-tags command-line flags now allow boolean selector syntax. For example, you can now pass --tags "(chrome || firefox) && !slow" to select quick Chrome or Firefox tests.

0.12.10+2

Re-add help output separators.
Tighten the constraint on args.

0.12.10+1

Temporarily remove separators from the help output. Version 0.12.8 was erroneously released without an appropriate args constraint for the features it used; this version will help ensure that users who can't use args 0.13.1 will get a working version of test.

0.12.10

Add support for a package-level configuration file called dart_test.yaml.

0.12.9

Add SuiteEvent to the JSON reporter, which reports data about the suites in which tests are run.
Add AllSuitesEvent to the JSON reporter, which reports the total number of suites that will be run.
Add Group.testCount to the JSON reporter, which reports the total number of tests in each group.

0.12.8

Organize the --help output into sections.
Add a --timeout flag.

0.12.7

Add the ability to re-run tests while debugging. When the browser is paused at a breakpoint, the test runner will open an interactive console on the command line that can be used to restart the test.
Add support for passing any object as a description to test() and group(). These objects will be converted to strings.
Add the ability to tag tests. Tests with specific tags may be run by passing the --tags command-line argument, or excluded by passing the --exclude-tags parameter.

This feature is not yet complete. For now, tags are only intended to be added temporarily to enable use-cases like focusing on a specific test or group. Further development can be followed on the issue tracker.
Wait for a test's tear-down logic to run, even if it times out.

0.12.6+2

Declare compatibility with http_parser 2.0.0.

0.12.6+1

Declare compatibility with http_multi_server 2.0.0.

0.12.6

Add a machine-readable JSON reporter. For details, see the protocol documentation.
Skipped groups now properly print skip messages.

0.12.5+2

Declare compatibility with Dart 1.14 and 1.15.

0.12.5+1

Fixed a deadlock bug when using setUpAll() and tearDownAll().

0.12.5

Add setUpAll() and tearDownAll() methods that run callbacks before and after all tests in a group or suite. Note that these methods are for special cases and should be avoided—they make it very easy to accidentally introduce dependencies between tests. Use setUp() and tearDown() instead if possible.
Allow setUp() and tearDown() to be called multiple times within the same group.
When a tearDown() callback runs after a signal has been caught, it can now schedule out-of-band asynchronous callbacks normally rather than having them throw exceptions.
Don't show package warnings when compiling tests with dart2js. This was accidentally enabled in 0.12.2, but was never intended.

0.12.4+9

If a tearDown() callback throws an error, outer tearDown() callbacks are still executed.

0.12.4+8

Don't compile tests to JavaScript when running via pub serve on Dartium or content shell.

0.12.4+7

Support http_parser 1.0.0.

0.12.4+6

Fix a broken link in the README.

0.12.4+5

Internal changes only.

0.12.4+4

Widen the Dart SDK constraint to include 1.13.0.

0.12.4+3

Make source maps work properly in the browser when not using --pub-serve.

0.12.4+2

Fix a memory leak when running many browser tests where old test suites failed to be unloaded when they were supposed to.

0.12.4+1

Require Dart SDK >= 1.11.0 and shelf >= 0.6.0, allowing test to remove various hacks and workarounds.

0.12.4

Add a --pause-after-load flag that pauses the test runner after each suite is loaded so that breakpoints and other debugging annotations can be added. Currently this is only supported on browsers.
Add a Timeout.none value indicating that a test should never time out.
The dart-vm platform selector variable is now true for Dartium and content shell.
The compact reporter no longer prints status lines that only update the clock if they would get in the way of messages or errors from a test.
The expanded reporter no longer double-prints the descriptions of skipped tests.

0.12.3+9

Widen the constraint on analyzer to include 0.26.0.

0.12.3+8

Fix an uncaught error that could crop up when killing the test runner process at the wrong time.